diff options
| author | MuQun Yang <ymuqun@hdfgroup.org> | 2004-07-20 21:35:41 (GMT) |
|---|---|---|
| committer | MuQun Yang <ymuqun@hdfgroup.org> | 2004-07-20 21:35:41 (GMT) |
| commit | 8933c596946b9ba88fb2ca66a3f46f1f1d209aed (patch) | |
| tree | 84e3d9a39e134160a92b648c0b29b7696c8603ca /src/H5Dio.c | |
| parent | a86779f3f79905ce923d4689c1fb614969370031 (diff) | |
| download | hdf5-8933c596946b9ba88fb2ca66a3f46f1f1d209aed.zip hdf5-8933c596946b9ba88fb2ca66a3f46f1f1d209aed.tar.gz hdf5-8933c596946b9ba88fb2ca66a3f46f1f1d209aed.tar.bz2 | |
[svn-r8906] Purpose:
Adding the first round of patches about supporting collective chunk IO in HDF5
Description:
The current HDF5 library doesn't support collective MPIO with chunk storage. When users set collective option in their data transfer with chunk storage, the library silently converted the option to INDEPENDENT and that caused trememdous performance penalty. Some application like
WRF-parallel HDF5 IO module has to use contiguous storage for this reason. However, chunking storage has its own advantage(supporting compression filters and extensible dataset), so to make collective MPIO possible inside HDF5 with chunking storage is a very important task.
This check-in make collective chunk IO possible for some special cases. The condition is as follows(either case is fine with using collective chunk IO)
1. for each process, the hyperslab selection of the file data space of each dataset is regular and it is fit in one chunk.
2. for each process, the hyperslab selection of the file data space of each dataset is single and the number of chunks for the hyperslab selection should be equal.
Solution:
Lift up the contiguous storage requirement for collective IO.
Use H5D_isstore_get_addr to get the corresponding chunk address. Then the original library routines will take care of getting the correct address to make sure that MPI FILE TYPE is built correctly for collective IO>
Platforms tested:
arabica(sol), copper(AIX), eirene(Linux)
parallel test is checked at copper.
Misc. update:
Diffstat (limited to 'src/H5Dio.c')
| -rw-r--r-- | src/H5Dio.c | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/src/H5Dio.c b/src/H5Dio.c index fb07831..1c00c79 100644 --- a/src/H5Dio.c +++ b/src/H5Dio.c @@ -745,7 +745,7 @@ H5D_read(H5D_t *dataset, hid_t mem_type_id, const H5S_t *mem_space, } /* end switch */ /* Get dataspace functions */ - if (NULL==(sconv=H5S_find(mem_space, file_space, sconv_flags, &use_par_opt_io))) + if (NULL==(sconv=H5S_find(dataset->ent.file,mem_space, file_space, sconv_flags, &use_par_opt_io,&dataset->layout))) HGOTO_ERROR (H5E_DATASET, H5E_UNSUPPORTED, FAIL, "unable to convert from file to memory data space") #ifdef H5_HAVE_PARALLEL @@ -961,7 +961,7 @@ H5D_write(H5D_t *dataset, hid_t mem_type_id, const H5S_t *mem_space, } /* end switch */ /* Get dataspace functions */ - if (NULL==(sconv=H5S_find(mem_space, file_space, sconv_flags, &use_par_opt_io))) + if (NULL==(sconv=H5S_find(dataset->ent.file,mem_space, file_space, sconv_flags, &use_par_opt_io,&dataset->layout))) HGOTO_ERROR (H5E_DATASET, H5E_UNSUPPORTED, FAIL, "unable to convert from memory to file data space") #ifdef H5_HAVE_PARALLEL |