summaryrefslogtreecommitdiffstats
path: root/src/H5Fseq.c
diff options
context:
space:
mode:
authorQuincey Koziol <koziol@hdfgroup.org>2003-01-09 17:20:03 (GMT)
committerQuincey Koziol <koziol@hdfgroup.org>2003-01-09 17:20:03 (GMT)
commit9a433b99a56dc575f1c0b11f95b744de61859dbb (patch)
treed8c766537cb9adc364c902bd45477d97f67a4a9f /src/H5Fseq.c
parent7fd449cb7987772a2881a5ced2ae7ad5231f1fa3 (diff)
downloadhdf5-9a433b99a56dc575f1c0b11f95b744de61859dbb.zip
hdf5-9a433b99a56dc575f1c0b11f95b744de61859dbb.tar.gz
hdf5-9a433b99a56dc575f1c0b11f95b744de61859dbb.tar.bz2
[svn-r6252] Purpose:
Lots of performance improvements & a couple new internal API interfaces. Description: Performance Improvements: - Cached file offset & length sizes in shared file struct, to avoid constantly looking them up in the FCPL. - Generic property improvements: - Added "revision" number to generic property classes to speed up comparisons. - Changed method of storing properties from using a hash-table to the TBBT routines in the library. - Share the propery names between classes and the lists derived from them. - Removed redundant 'def_value' buffer from each property. - Switching code to use a "copy on write" strategy for properties in each list, where the properties in each list are shared with the properties in the class, until a property's value is changed in a list. - Fixed error in layout code which was allocating too many buffers. - Redefined public macros of the form (H5open()/H5check, <variable>) internally to only be (<variable>), avoiding innumerable useless calls to H5open() and H5check_version(). - Reuse already zeroed buffers in H5F_contig_fill instead of constantly re-zeroing them. - Don't write fill values if writing entire dataset. - Use gettimeofday() system call instead of time() system when checking the modification time of a dataset. - Added reference counted string API and use it for tracking the names of objects opening in a file (for the ID->name code). - Removed redundant H5P_get() calls in B-tree routines. - Redefine H5T datatype macros internally to the library, to avoid calling H5check redundantly. - Keep dataspace information for dataset locally instead of reading from disk each time. Added new module to track open objects in a file, to allow this (which will be useful eventually for some FPH5 metadata caching issues). - Remove H5AC_find macro which was inlining metadata cache lookups, and call function instead. - Remove redundant memset() calls from H5G_namei() routine. - Remove redundant checking of object type when locating objects in metadata cache and rely on the address only. - Create default dataset object to use when default dataset creation property list is used to create datasets, bypassing querying for all the property list values. - Use default I/O vector size when performing raw data with the default dataset transfer property list, instead of querying for I/O vector size. - Remove H5P_DEFAULT internally to the library, replacing it with more specific default property list based on the type of property list needed. - Remove redundant memset() calls in object header message (H5O*) routines. - Remove redunant memset() calls in data I/O routines. - Split free-list allocation routines into malloc() and calloc()- like routines, instead of one combined routine. - Remove lots of indirection in H5O*() routines. - Simplify metadata cache entry comparison routine (used when flushing entire cache out). - Only enable metadata cache statistics when H5AC_DEBUG is turned on, instead of always tracking them. - Simplify address comparison macro (H5F_addr_eq). - Remove redundant metadata cache entry protections during dataset creation by protecting the object header once and making all the modifications necessary for the dataset creation before unprotecting it. - Reduce # of "number of element in extent" computations performed by computing and storing the value during dataspace creation. - Simplify checking for group location's file information, when file has not been involving in file-mounting operations. - Use binary encoding for modification time, instead of ASCII. - Hoist H5HL_peek calls (to get information in a local heap) out of loops in many group routine. - Use static variable for iterators of selections, instead of dynamically allocation them each time. - Lookup & insert new entries in one step, avoiding traversing group's B-tree twice. - Fixed memory leak in H5Gget_objname_idx() routine (tangential to performance improvements, but fixed along the way). - Use free-list for reference counted strings. - Don't bother copying object names into cached group entries, since they are re-created when an object is opened. The benchmark I used to measure these results created several thousand small (2K) datasets in a file and wrote out the data for them. This is Elena's "regular.c" benchmark. These changes resulted in approximately ~4.3x speedup of the development branch when compared to the previous code in the development branch and ~1.4x speedup compared to the release branch. Additionally, these changes reduce the total memory used (code and data) by the development branch by ~800KB, bringing the development branch back into the same ballpark as the release branch. I'll send out a more detailed description of the benchmark results as a followup note. New internal API routines: Added "reference counted strings" API for tracking strings that get used by multiple owners without duplicating the strings. Added "ternary search tree" API for text->object mappings. Platforms tested: Tested h5committest {arabica (fortran), eirene (fortran, C++) modi4 (parallel, fortran)} Other platforms/configurations tested? FreeBSD 4.7 (sleipnir) serial & parallel Solaris 2.6 (baldric) serial
Diffstat (limited to 'src/H5Fseq.c')
-rw-r--r--src/H5Fseq.c46
1 files changed, 20 insertions, 26 deletions
diff --git a/src/H5Fseq.c b/src/H5Fseq.c
index 6176003..9592725 100644
--- a/src/H5Fseq.c
+++ b/src/H5Fseq.c
@@ -59,9 +59,9 @@ static int interface_initialize_g = 0;
*/
herr_t
H5F_seq_read(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout,
- H5P_genplist_t *dc_plist,
- const H5S_t *file_space, size_t elmt_size,
- size_t seq_len, hsize_t dset_offset, void *buf/*out*/)
+ H5P_genplist_t *dc_plist, const H5O_efl_t *efl,
+ const H5S_t *file_space, size_t elmt_size,
+ size_t seq_len, hsize_t dset_offset, void *buf/*out*/)
{
herr_t ret_value=SUCCEED; /* Return value */
@@ -70,10 +70,11 @@ H5F_seq_read(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout,
/* Check args */
assert(f);
assert(layout);
+ assert(efl);
assert(buf);
assert(TRUE==H5P_isa_class(dxpl_id,H5P_DATASET_XFER));
- if (H5F_seq_readv(f, dxpl_id, layout, dc_plist, file_space, elmt_size, 1, &seq_len, &dset_offset, buf)<0)
+ if (H5F_seq_readv(f, dxpl_id, layout, dc_plist, efl, file_space, elmt_size, 1, &seq_len, &dset_offset, buf)<0)
HGOTO_ERROR(H5E_IO, H5E_READERROR, FAIL, "vector read failed");
done:
@@ -104,9 +105,9 @@ done:
*/
herr_t
H5F_seq_write(H5F_t *f, hid_t dxpl_id, H5O_layout_t *layout,
- H5P_genplist_t *dc_plist,
- const H5S_t *file_space, size_t elmt_size,
- size_t seq_len, hsize_t dset_offset, const void *buf)
+ H5P_genplist_t *dc_plist, const H5O_efl_t *efl,
+ const H5S_t *file_space, size_t elmt_size,
+ size_t seq_len, hsize_t dset_offset, const void *buf)
{
herr_t ret_value=SUCCEED; /* Return value */
@@ -115,10 +116,11 @@ H5F_seq_write(H5F_t *f, hid_t dxpl_id, H5O_layout_t *layout,
/* Check args */
assert(f);
assert(layout);
+ assert(efl);
assert(buf);
assert(TRUE==H5P_isa_class(dxpl_id,H5P_DATASET_XFER));
- if (H5F_seq_writev(f, dxpl_id, layout, dc_plist, file_space, elmt_size, 1, &seq_len, &dset_offset, buf)<0)
+ if (H5F_seq_writev(f, dxpl_id, layout, dc_plist, efl, file_space, elmt_size, 1, &seq_len, &dset_offset, buf)<0)
HGOTO_ERROR(H5E_IO, H5E_WRITEERROR, FAIL, "vector write failed");
done:
@@ -151,7 +153,7 @@ done:
*/
herr_t
H5F_seq_readv(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout,
- H5P_genplist_t *dc_plist,
+ H5P_genplist_t *dc_plist, const H5O_efl_t *efl,
const H5S_t *file_space, size_t elmt_size,
size_t nseq, size_t seq_len_arr[], hsize_t dset_offset_arr[],
void *_buf/*out*/)
@@ -172,7 +174,6 @@ H5F_seq_readv(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout,
unsigned u; /*counters */
size_t v; /*counters */
int i,j; /*counters */
- struct H5O_efl_t efl; /* External File List info */
#ifdef H5_HAVE_PARALLEL
H5FD_mpio_xfer_t xfer_mode=H5FD_MPIO_INDEPENDENT;
H5P_genplist_t *plist=NULL; /* Property list */
@@ -184,6 +185,7 @@ H5F_seq_readv(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout,
/* Check args */
assert(f);
assert(layout);
+ assert(efl);
assert(real_buf);
/* Make certain we have the correct type of property list */
assert(TRUE==H5P_isa_class(dxpl_id,H5P_DATASET_XFER));
@@ -221,14 +223,10 @@ H5F_seq_readv(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout,
HGOTO_ERROR (H5E_DATASET, H5E_READERROR, FAIL, "collective access on non-contiguous datasets not supported yet");
#endif /* H5_HAVE_PARALLEL */
- /* Get necessary properties from property list */
- if(H5P_get(dc_plist, H5D_CRT_EXT_FILE_LIST_NAME, &efl) < 0)
- HGOTO_ERROR(H5E_DATASET, H5E_CANTGET, FAIL, "can't get EFL value");
-
switch (layout->type) {
case H5D_CONTIGUOUS:
/* Read directly from file if the dataset is in an external file */
- if (efl.nused>0) {
+ if (efl->nused>0) {
/* Iterate through the sequence vectors */
for(v=0; v<nseq; v++) {
#ifdef H5_HAVE_PARALLEL
@@ -260,7 +258,7 @@ H5F_seq_readv(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout,
* file offsets, totally mixing up the data sieve buffer information. -QAK
*/
H5_CHECK_OVERFLOW(dset_offset_arr[v],hsize_t,haddr_t);
- if (H5O_efl_read(f, &efl, (haddr_t)dset_offset_arr[v], seq_len_arr[v], real_buf)<0)
+ if (H5O_efl_read(f, efl, (haddr_t)dset_offset_arr[v], seq_len_arr[v], real_buf)<0)
HGOTO_ERROR(H5E_IO, H5E_READERROR, FAIL, "external data read failed");
/* Increment offset in buffer */
@@ -281,7 +279,7 @@ H5F_seq_readv(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout,
/*
* This method is unable to access external raw data files
*/
- if (efl.nused>0)
+ if (efl->nused>0)
HGOTO_ERROR(H5E_IO, H5E_UNSUPPORTED, FAIL, "chunking and external files are mutually exclusive");
/* Compute the file offset coordinates and hyperslab size */
@@ -542,7 +540,7 @@ done:
*/
herr_t
H5F_seq_writev(H5F_t *f, hid_t dxpl_id, H5O_layout_t *layout,
- H5P_genplist_t *dc_plist,
+ H5P_genplist_t *dc_plist, const H5O_efl_t *efl,
const H5S_t *file_space, size_t elmt_size,
size_t nseq, size_t seq_len_arr[], hsize_t dset_offset_arr[],
const void *_buf)
@@ -563,7 +561,6 @@ H5F_seq_writev(H5F_t *f, hid_t dxpl_id, H5O_layout_t *layout,
unsigned u; /*counters */
size_t v; /*counters */
int i,j; /*counters */
- struct H5O_efl_t efl; /* External File List info */
#ifdef H5_HAVE_PARALLEL
H5FD_mpio_xfer_t xfer_mode=H5FD_MPIO_INDEPENDENT;
H5P_genplist_t *plist=NULL; /* Property list */
@@ -575,6 +572,7 @@ H5F_seq_writev(H5F_t *f, hid_t dxpl_id, H5O_layout_t *layout,
/* Check args */
assert(f);
assert(layout);
+ assert(efl);
assert(real_buf);
/* Make certain we have the correct type of property list */
assert(TRUE==H5P_isa_class(dxpl_id,H5P_DATASET_XFER));
@@ -612,14 +610,10 @@ H5F_seq_writev(H5F_t *f, hid_t dxpl_id, H5O_layout_t *layout,
HGOTO_ERROR (H5E_DATASET, H5E_WRITEERROR, FAIL, "collective access on chunked datasets not supported yet");
#endif /* H5_HAVE_PARALLEL */
- /* Get necessary properties from property list */
- if(H5P_get(dc_plist, H5D_CRT_EXT_FILE_LIST_NAME, &efl) < 0)
- HGOTO_ERROR(H5E_DATASET, H5E_CANTGET, FAIL, "can't get EFL value");
-
switch (layout->type) {
case H5D_CONTIGUOUS:
/* Write directly to file if the dataset is in an external file */
- if (efl.nused>0) {
+ if (efl->nused>0) {
/* Iterate through the sequence vectors */
for(v=0; v<nseq; v++) {
#ifdef H5_HAVE_PARALLEL
@@ -650,7 +644,7 @@ H5F_seq_writev(H5F_t *f, hid_t dxpl_id, H5O_layout_t *layout,
* file offsets, totally mixing up the data sieve buffer information. -QAK
*/
H5_CHECK_OVERFLOW(dset_offset_arr[v],hsize_t,haddr_t);
- if (H5O_efl_write(f, &efl, (haddr_t)dset_offset_arr[v], seq_len_arr[v], real_buf)<0)
+ if (H5O_efl_write(f, efl, (haddr_t)dset_offset_arr[v], seq_len_arr[v], real_buf)<0)
HGOTO_ERROR(H5E_IO, H5E_WRITEERROR, FAIL, "external data write failed");
/* Increment offset in buffer */
@@ -671,7 +665,7 @@ H5F_seq_writev(H5F_t *f, hid_t dxpl_id, H5O_layout_t *layout,
/*
* This method is unable to access external raw data files
*/
- if (efl.nused>0)
+ if (efl->nused>0)
HGOTO_ERROR(H5E_IO, H5E_UNSUPPORTED, FAIL, "chunking and external files are mutually exclusive");
/* Compute the file offset coordinates and hyperslab size */