From e02ae065f6a2728d33af673e97405a3b409c9202 Mon Sep 17 00:00:00 2001 From: Quincey Koziol Date: Fri, 17 May 2002 07:54:42 -0500 Subject: [svn-r5430] Purpose: Document Bug fix/Code improvement below: Description: Currently, the chunk data allocation routine invoked to allocate space for the entire dataset is inefficient. It writes out each chunk in the dataset, whether it is already allocated or not. Additionally, this happens not only when it is created, but also anytime it is opened for writing, or the dataset is extended. Worse, there's too much parallel I/O syncronization, which slows things down even more. Solution: Only attempt to write out chunks that don't already exist. Additionally, share the I/O writing between all the nodes, instead of writing everything with process 0. Then, only block with MPI_Barrier if chunks were actually created. --- release_docs/RELEASE.txt | 12 ++++++++---- 1 file changed, 8 insertions(+), 4 deletions(-) diff --git a/release_docs/RELEASE.txt b/release_docs/RELEASE.txt index d712d57..c3a5d6e 100644 --- a/release_docs/RELEASE.txt +++ b/release_docs/RELEASE.txt @@ -164,6 +164,10 @@ Documentation New Features ============ + * Changed method for allocating chunked dataset blocks in parallel to only + allocate blocks that don't already exist, instead of attempting to create + all the blocks all the time. This improves parallel I/O performance for + chunked datasets. QAK - 2002/05/17 * Allowed the call to MPI_File_sync to be avoided when the file is going to immediately be closed, improving performance. QAK - 2002/05/13 * Allow the metadata writes to be shared among all processes, easing the @@ -184,9 +188,12 @@ New Features datatype conversion is unneccessary. QAK - 2002/04/02 * Improved performance of single hyperslab I/O when datatype conversion is unneccessary. QAK - 2002/04/02 + * New API function H5Dset_extent. Modifies the dimensions of a dataset, + allows change to a lower dimension. The unused space in the file is freed. + PVN - 2002/03/31 * Added new "H5Sget_select_type" API function to determine which type of selection is defined for a dataspace ("all", "none", "hyperslab" or - "point"). QAK - 2002/02/7 + "point"). QAK - 2002/02/07 * Added support to read/write portions of chunks directly, if they are uncompressed and too large to cache. This should speed up I/O on chunked datasets for a few more cases. QAK - 2002/01/31 @@ -279,9 +286,6 @@ New Features * A helper script called ``h5cc'', which helps compilation of HDF5 programs, is now distributed with HDF5. See the reference manual for information on how to use this feature. - * New API function H5Dset_extent. Modifies the dimensions of a dataset, - allows change to a lower dimension. The unused space in the file is freed. - PVN - 2002/03/31 Support for new platforms and languages ======================================= -- cgit v0.12