[svn-r6252] Purpose:

    Lots of performance improvements & a couple new internal API interfaces.

Description:
    Performance Improvements:
        - Cached file offset & length sizes in shared file struct, to avoid
            constantly looking them up in the FCPL.
        - Generic property improvements:
            - Added "revision" number to generic property classes to speed
                up comparisons.
            - Changed method of storing properties from using a hash-table
                to the TBBT routines in the library.
            - Share the propery names between classes and the lists derived
                from them.
            - Removed redundant 'def_value' buffer from each property.
            - Switching code to use a "copy on write" strategy for
                properties in each list, where the properties in each list
                are shared with the properties in the class, until a
                property's value is changed in a list.
        - Fixed error in layout code which was allocating too many buffers.
        - Redefined public macros of the form (H5open()/H5check, <variable>)
            internally to only be (<variable>), avoiding innumerable useless
            calls to H5open() and H5check_version().
        - Reuse already zeroed buffers in H5F_contig_fill instead of
            constantly re-zeroing them.
        - Don't write fill values if writing entire dataset.
        - Use gettimeofday() system call instead of time() system when
            checking the modification time of a dataset.
        - Added reference counted string API and use it for tracking the
            names of objects opening in a file (for the ID->name code).
        - Removed redundant H5P_get() calls in B-tree routines.
        - Redefine H5T datatype macros internally to the library, to avoid
            calling H5check redundantly.
        - Keep dataspace information for dataset locally instead of reading
            from disk each time.  Added new module to track open objects
            in a file, to allow this (which will be useful eventually for
            some FPH5 metadata caching issues).
        - Remove H5AC_find macro which was inlining metadata cache lookups,
            and call function instead.
        - Remove redundant memset() calls from H5G_namei() routine.
        - Remove redundant checking of object type when locating objects
            in metadata cache and rely on the address only.
        - Create default dataset object to use when default dataset creation
            property list is used to create datasets, bypassing querying
            for all the property list values.
        - Use default I/O vector size when performing raw data with the
            default dataset transfer property list, instead of querying for
            I/O vector size.
        - Remove H5P_DEFAULT internally to the library, replacing it with
            more specific default property list based on the type of
            property list needed.
        - Remove redundant memset() calls in object header message (H5O*)
            routines.
        - Remove redunant memset() calls in data I/O routines.
        - Split free-list allocation routines into malloc() and calloc()-
            like routines, instead of one combined routine.
        - Remove lots of indirection in H5O*() routines.
        - Simplify metadata cache entry comparison routine (used when
            flushing entire cache out).
        - Only enable metadata cache statistics when H5AC_DEBUG is turned
            on, instead of always tracking them.
        - Simplify address comparison macro (H5F_addr_eq).
        - Remove redundant metadata cache entry protections during dataset
            creation by protecting the object header once and making all
            the modifications necessary for the dataset creation before
            unprotecting it.
        - Reduce # of "number of element in extent" computations performed
            by computing and storing the value during dataspace creation.
        - Simplify checking for group location's file information, when file
            has not been involving in file-mounting operations.
        - Use binary encoding for modification time, instead of ASCII.
        - Hoist H5HL_peek calls (to get information in a local heap)
            out of loops in many group routine.
        - Use static variable for iterators of selections, instead of
            dynamically allocation them each time.
        - Lookup & insert new entries in one step, avoiding traversing
            group's B-tree twice.
        - Fixed memory leak in H5Gget_objname_idx() routine (tangential to
            performance improvements, but fixed along the way).
        - Use free-list for reference counted strings.
        - Don't bother copying object names into cached group entries,
            since they are re-created when an object is opened.

        The benchmark I used to measure these results created several thousand
        small (2K) datasets in a file and wrote out the data for them.  This is
        Elena's "regular.c" benchmark.

        These changes resulted in approximately ~4.3x speedup of the
        development branch when compared to the previous code in the
        development branch and ~1.4x speedup compared to the release
        branch.

        Additionally, these changes reduce the total memory used (code and
        data) by the development branch by ~800KB, bringing the development
        branch back into the same ballpark as the release branch.

        I'll send out a more detailed description of the benchmark results
        as a followup note.

    New internal API routines:
        Added "reference counted strings" API for tracking strings that get
            used by multiple owners without duplicating the strings.
        Added "ternary search tree" API for text->object mappings.

Platforms tested:
    Tested h5committest {arabica (fortran), eirene (fortran, C++)
	modi4 (parallel, fortran)}
    Other platforms/configurations tested?
        FreeBSD 4.7 (sleipnir) serial & parallel
        Solaris 2.6 (baldric) serial

1 files changed, 20 insertions, 26 deletions

diff --git a/src/H5Dseq.c b/src/H5Dseq.c
index 6176003..9592725 100644
--- a/src/H5Dseq.c
+++ b/src/H5Dseq.c
@@ -59,9 +59,9 @@ static int interface_initialize_g = 0;
  */
 herr_t
 H5F_seq_read(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout,
-        H5P_genplist_t *dc_plist,
-        const H5S_t *file_space, size_t elmt_size,
-        size_t seq_len, hsize_t dset_offset, void *buf/*out*/)
+    H5P_genplist_t *dc_plist, const H5O_efl_t *efl,
+    const H5S_t *file_space, size_t elmt_size,
+    size_t seq_len, hsize_t dset_offset, void *buf/*out*/)
 {
     herr_t      ret_value=SUCCEED;       /* Return value */
 
@@ -70,10 +70,11 @@ H5F_seq_read(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout,
     /* Check args */
     assert(f);
     assert(layout);
+    assert(efl);
     assert(buf);
     assert(TRUE==H5P_isa_class(dxpl_id,H5P_DATASET_XFER));
 
-    if (H5F_seq_readv(f, dxpl_id, layout, dc_plist, file_space, elmt_size, 1, &seq_len, &dset_offset, buf)<0)
+    if (H5F_seq_readv(f, dxpl_id, layout, dc_plist, efl, file_space, elmt_size, 1, &seq_len, &dset_offset, buf)<0)
         HGOTO_ERROR(H5E_IO, H5E_READERROR, FAIL, "vector read failed");
 
 done:
@@ -104,9 +105,9 @@ done:
  */
 herr_t
 H5F_seq_write(H5F_t *f, hid_t dxpl_id, H5O_layout_t *layout,
-        H5P_genplist_t *dc_plist,
-        const H5S_t *file_space, size_t elmt_size,
-        size_t seq_len, hsize_t dset_offset, const void *buf)
+    H5P_genplist_t *dc_plist, const H5O_efl_t *efl,
+    const H5S_t *file_space, size_t elmt_size,
+    size_t seq_len, hsize_t dset_offset, const void *buf)
 {
     herr_t      ret_value=SUCCEED;       /* Return value */
 
@@ -115,10 +116,11 @@ H5F_seq_write(H5F_t *f, hid_t dxpl_id, H5O_layout_t *layout,
     /* Check args */
     assert(f);
     assert(layout);
+    assert(efl);
     assert(buf);
     assert(TRUE==H5P_isa_class(dxpl_id,H5P_DATASET_XFER));
 
-    if (H5F_seq_writev(f, dxpl_id, layout, dc_plist, file_space, elmt_size, 1, &seq_len, &dset_offset, buf)<0)
+    if (H5F_seq_writev(f, dxpl_id, layout, dc_plist, efl, file_space, elmt_size, 1, &seq_len, &dset_offset, buf)<0)
         HGOTO_ERROR(H5E_IO, H5E_WRITEERROR, FAIL, "vector write failed");
 
 done:
@@ -151,7 +153,7 @@ done:
  */
 herr_t
 H5F_seq_readv(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout,
-        H5P_genplist_t *dc_plist,
+        H5P_genplist_t *dc_plist, const H5O_efl_t *efl,
         const H5S_t *file_space, size_t elmt_size,
         size_t nseq, size_t seq_len_arr[], hsize_t dset_offset_arr[],
         void *_buf/*out*/)
@@ -172,7 +174,6 @@ H5F_seq_readv(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout,
     unsigned	u;				/*counters		*/
     size_t      v;                              /*counters              */
     int	i,j;				/*counters		*/
-    struct H5O_efl_t efl;               /* External File List info */
 #ifdef H5_HAVE_PARALLEL
     H5FD_mpio_xfer_t xfer_mode=H5FD_MPIO_INDEPENDENT;
     H5P_genplist_t *plist=NULL;                 /* Property list */
@@ -184,6 +185,7 @@ H5F_seq_readv(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout,
     /* Check args */
     assert(f);
     assert(layout);
+    assert(efl);
     assert(real_buf);
     /* Make certain we have the correct type of property list */
     assert(TRUE==H5P_isa_class(dxpl_id,H5P_DATASET_XFER));
@@ -221,14 +223,10 @@ H5F_seq_readv(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout,
         HGOTO_ERROR (H5E_DATASET, H5E_READERROR, FAIL, "collective access on non-contiguous datasets not supported yet");
 #endif /* H5_HAVE_PARALLEL */
 
-    /* Get necessary properties from property list */
-    if(H5P_get(dc_plist, H5D_CRT_EXT_FILE_LIST_NAME, &efl) < 0)
-        HGOTO_ERROR(H5E_DATASET, H5E_CANTGET, FAIL, "can't get EFL value");
-
     switch (layout->type) {
         case H5D_CONTIGUOUS:
             /* Read directly from file if the dataset is in an external file */
-            if (efl.nused>0) {
+            if (efl->nused>0) {
                 /* Iterate through the sequence vectors */
                 for(v=0; v<nseq; v++) {
 #ifdef H5_HAVE_PARALLEL
@@ -260,7 +258,7 @@ H5F_seq_readv(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout,
                      *  file offsets, totally mixing up the data sieve buffer information. -QAK
                      */
                     H5_CHECK_OVERFLOW(dset_offset_arr[v],hsize_t,haddr_t);
-                    if (H5O_efl_read(f, &efl, (haddr_t)dset_offset_arr[v], seq_len_arr[v], real_buf)<0)
+                    if (H5O_efl_read(f, efl, (haddr_t)dset_offset_arr[v], seq_len_arr[v], real_buf)<0)
                         HGOTO_ERROR(H5E_IO, H5E_READERROR, FAIL, "external data read failed");
 
                     /* Increment offset in buffer */
@@ -281,7 +279,7 @@ H5F_seq_readv(H5F_t *f, hid_t dxpl_id, const H5O_layout_t *layout,
             /*
              * This method is unable to access external raw data files 
              */
-            if (efl.nused>0)
+            if (efl->nused>0)
                 HGOTO_ERROR(H5E_IO, H5E_UNSUPPORTED, FAIL, "chunking and external files are mutually exclusive");
 
             /* Compute the file offset coordinates and hyperslab size */
@@ -542,7 +540,7 @@ done:
  */
 herr_t
 H5F_seq_writev(H5F_t *f, hid_t dxpl_id, H5O_layout_t *layout,
-        H5P_genplist_t *dc_plist,
+        H5P_genplist_t *dc_plist, const H5O_efl_t *efl,
         const H5S_t *file_space, size_t elmt_size,
         size_t nseq, size_t seq_len_arr[], hsize_t dset_offset_arr[],
         const void *_buf)
@@ -563,7 +561,6 @@ H5F_seq_writev(H5F_t *f, hid_t dxpl_id, H5O_layout_t *layout,
     unsigned	u;				/*counters		*/
     size_t      v;                              /*counters              */
     int	i,j;				/*counters		*/
-    struct H5O_efl_t efl;               /* External File List info */
 #ifdef H5_HAVE_PARALLEL
     H5FD_mpio_xfer_t xfer_mode=H5FD_MPIO_INDEPENDENT;
     H5P_genplist_t *plist=NULL;                 /* Property list */
@@ -575,6 +572,7 @@ H5F_seq_writev(H5F_t *f, hid_t dxpl_id, H5O_layout_t *layout,
     /* Check args */
     assert(f);
     assert(layout);
+    assert(efl);
     assert(real_buf);
     /* Make certain we have the correct type of property list */
     assert(TRUE==H5P_isa_class(dxpl_id,H5P_DATASET_XFER));
@@ -612,14 +610,10 @@ H5F_seq_writev(H5F_t *f, hid_t dxpl_id, H5O_layout_t *layout,
         HGOTO_ERROR (H5E_DATASET, H5E_WRITEERROR, FAIL, "collective access on chunked datasets not supported yet");
 #endif /* H5_HAVE_PARALLEL */
 
-    /* Get necessary properties from property list */
-    if(H5P_get(dc_plist, H5D_CRT_EXT_FILE_LIST_NAME, &efl) < 0)
-        HGOTO_ERROR(H5E_DATASET, H5E_CANTGET, FAIL, "can't get EFL value");
-
     switch (layout->type) {
         case H5D_CONTIGUOUS:
             /* Write directly to file if the dataset is in an external file */
-            if (efl.nused>0) {
+            if (efl->nused>0) {
                 /* Iterate through the sequence vectors */
                 for(v=0; v<nseq; v++) {
 #ifdef H5_HAVE_PARALLEL
@@ -650,7 +644,7 @@ H5F_seq_writev(H5F_t *f, hid_t dxpl_id, H5O_layout_t *layout,
                      *  file offsets, totally mixing up the data sieve buffer information. -QAK
                      */
                     H5_CHECK_OVERFLOW(dset_offset_arr[v],hsize_t,haddr_t);
-                    if (H5O_efl_write(f, &efl, (haddr_t)dset_offset_arr[v], seq_len_arr[v], real_buf)<0)
+                    if (H5O_efl_write(f, efl, (haddr_t)dset_offset_arr[v], seq_len_arr[v], real_buf)<0)
                         HGOTO_ERROR(H5E_IO, H5E_WRITEERROR, FAIL, "external data write failed");
 
                     /* Increment offset in buffer */
@@ -671,7 +665,7 @@ H5F_seq_writev(H5F_t *f, hid_t dxpl_id, H5O_layout_t *layout,
             /*
              * This method is unable to access external raw data files 
              */
-            if (efl.nused>0)
+            if (efl->nused>0)
                 HGOTO_ERROR(H5E_IO, H5E_UNSUPPORTED, FAIL, "chunking and external files are mutually exclusive");
 
             /* Compute the file offset coordinates and hyperslab size */