diff options
| author | MuQun Yang <ymuqun@hdfgroup.org> | 2004-07-20 21:35:41 (GMT) |
|---|---|---|
| committer | MuQun Yang <ymuqun@hdfgroup.org> | 2004-07-20 21:35:41 (GMT) |
| commit | 8933c596946b9ba88fb2ca66a3f46f1f1d209aed (patch) | |
| tree | 84e3d9a39e134160a92b648c0b29b7696c8603ca /src/H5Distore.c | |
| parent | a86779f3f79905ce923d4689c1fb614969370031 (diff) | |
| download | hdf5-8933c596946b9ba88fb2ca66a3f46f1f1d209aed.zip hdf5-8933c596946b9ba88fb2ca66a3f46f1f1d209aed.tar.gz hdf5-8933c596946b9ba88fb2ca66a3f46f1f1d209aed.tar.bz2 | |
[svn-r8906] Purpose:
Adding the first round of patches about supporting collective chunk IO in HDF5
Description:
The current HDF5 library doesn't support collective MPIO with chunk storage. When users set collective option in their data transfer with chunk storage, the library silently converted the option to INDEPENDENT and that caused trememdous performance penalty. Some application like
WRF-parallel HDF5 IO module has to use contiguous storage for this reason. However, chunking storage has its own advantage(supporting compression filters and extensible dataset), so to make collective MPIO possible inside HDF5 with chunking storage is a very important task.
This check-in make collective chunk IO possible for some special cases. The condition is as follows(either case is fine with using collective chunk IO)
1. for each process, the hyperslab selection of the file data space of each dataset is regular and it is fit in one chunk.
2. for each process, the hyperslab selection of the file data space of each dataset is single and the number of chunks for the hyperslab selection should be equal.
Solution:
Lift up the contiguous storage requirement for collective IO.
Use H5D_isstore_get_addr to get the corresponding chunk address. Then the original library routines will take care of getting the correct address to make sure that MPI FILE TYPE is built correctly for collective IO>
Platforms tested:
arabica(sol), copper(AIX), eirene(Linux)
parallel test is checked at copper.
Misc. update:
Diffstat (limited to 'src/H5Distore.c')
| -rw-r--r-- | src/H5Distore.c | 4 |
1 files changed, 1 insertions, 3 deletions
diff --git a/src/H5Distore.c b/src/H5Distore.c index 1720a09..da9776a 100644 --- a/src/H5Distore.c +++ b/src/H5Distore.c @@ -142,8 +142,6 @@ typedef struct H5D_istore_ud1_t { #define H5D_HASH(D,ADDR) H5F_addr_hash(ADDR,(D)->cache.chunk.nslots) /* Private prototypes */ -static haddr_t H5D_istore_get_addr(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout, - const hssize_t offset[], H5D_istore_ud1_t *_udata); static void *H5D_istore_chunk_alloc(size_t size, const H5O_pline_t *pline); static void *H5D_istore_chunk_xfree(void *chk, const H5O_pline_t *pline); static herr_t H5D_istore_shared_free (void *page); @@ -2153,7 +2151,7 @@ done: * *------------------------------------------------------------------------- */ -static haddr_t +haddr_t H5D_istore_get_addr(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout, const hssize_t offset[], H5D_istore_ud1_t *_udata) { |