[svn-r6252] Purpose:

    Lots of performance improvements & a couple new internal API interfaces.

Description:
    Performance Improvements:
        - Cached file offset & length sizes in shared file struct, to avoid
            constantly looking them up in the FCPL.
        - Generic property improvements:
            - Added "revision" number to generic property classes to speed
                up comparisons.
            - Changed method of storing properties from using a hash-table
                to the TBBT routines in the library.
            - Share the propery names between classes and the lists derived
                from them.
            - Removed redundant 'def_value' buffer from each property.
            - Switching code to use a "copy on write" strategy for
                properties in each list, where the properties in each list
                are shared with the properties in the class, until a
                property's value is changed in a list.
        - Fixed error in layout code which was allocating too many buffers.
        - Redefined public macros of the form (H5open()/H5check, <variable>)
            internally to only be (<variable>), avoiding innumerable useless
            calls to H5open() and H5check_version().
        - Reuse already zeroed buffers in H5F_contig_fill instead of
            constantly re-zeroing them.
        - Don't write fill values if writing entire dataset.
        - Use gettimeofday() system call instead of time() system when
            checking the modification time of a dataset.
        - Added reference counted string API and use it for tracking the
            names of objects opening in a file (for the ID->name code).
        - Removed redundant H5P_get() calls in B-tree routines.
        - Redefine H5T datatype macros internally to the library, to avoid
            calling H5check redundantly.
        - Keep dataspace information for dataset locally instead of reading
            from disk each time.  Added new module to track open objects
            in a file, to allow this (which will be useful eventually for
            some FPH5 metadata caching issues).
        - Remove H5AC_find macro which was inlining metadata cache lookups,
            and call function instead.
        - Remove redundant memset() calls from H5G_namei() routine.
        - Remove redundant checking of object type when locating objects
            in metadata cache and rely on the address only.
        - Create default dataset object to use when default dataset creation
            property list is used to create datasets, bypassing querying
            for all the property list values.
        - Use default I/O vector size when performing raw data with the
            default dataset transfer property list, instead of querying for
            I/O vector size.
        - Remove H5P_DEFAULT internally to the library, replacing it with
            more specific default property list based on the type of
            property list needed.
        - Remove redundant memset() calls in object header message (H5O*)
            routines.
        - Remove redunant memset() calls in data I/O routines.
        - Split free-list allocation routines into malloc() and calloc()-
            like routines, instead of one combined routine.
        - Remove lots of indirection in H5O*() routines.
        - Simplify metadata cache entry comparison routine (used when
            flushing entire cache out).
        - Only enable metadata cache statistics when H5AC_DEBUG is turned
            on, instead of always tracking them.
        - Simplify address comparison macro (H5F_addr_eq).
        - Remove redundant metadata cache entry protections during dataset
            creation by protecting the object header once and making all
            the modifications necessary for the dataset creation before
            unprotecting it.
        - Reduce # of "number of element in extent" computations performed
            by computing and storing the value during dataspace creation.
        - Simplify checking for group location's file information, when file
            has not been involving in file-mounting operations.
        - Use binary encoding for modification time, instead of ASCII.
        - Hoist H5HL_peek calls (to get information in a local heap)
            out of loops in many group routine.
        - Use static variable for iterators of selections, instead of
            dynamically allocation them each time.
        - Lookup & insert new entries in one step, avoiding traversing
            group's B-tree twice.
        - Fixed memory leak in H5Gget_objname_idx() routine (tangential to
            performance improvements, but fixed along the way).
        - Use free-list for reference counted strings.
        - Don't bother copying object names into cached group entries,
            since they are re-created when an object is opened.

        The benchmark I used to measure these results created several thousand
        small (2K) datasets in a file and wrote out the data for them.  This is
        Elena's "regular.c" benchmark.

        These changes resulted in approximately ~4.3x speedup of the
        development branch when compared to the previous code in the
        development branch and ~1.4x speedup compared to the release
        branch.

        Additionally, these changes reduce the total memory used (code and
        data) by the development branch by ~800KB, bringing the development
        branch back into the same ballpark as the release branch.

        I'll send out a more detailed description of the benchmark results
        as a followup note.

    New internal API routines:
        Added "reference counted strings" API for tracking strings that get
            used by multiple owners without duplicating the strings.
        Added "ternary search tree" API for text->object mappings.

Platforms tested:
    Tested h5committest {arabica (fortran), eirene (fortran, C++)
	modi4 (parallel, fortran)}
    Other platforms/configurations tested?
        FreeBSD 4.7 (sleipnir) serial & parallel
        Solaris 2.6 (baldric) serial

1 files changed, 47 insertions, 26 deletions

diff --git a/src/H5Shyper.c b/src/H5Shyper.c
index 3eec64e..994238f 100644
--- a/src/H5Shyper.c
+++ b/src/H5Shyper.c
@@ -156,10 +156,14 @@ H5S_hyper_iter_init (const H5S_t *space, size_t elmt_size, H5S_sel_iter_t *sel_i
             sel_iter->hyp.iter_rank=cont_dim;
 
             /* Allocate the position & initialize to initial location */
-            sel_iter->hyp.off = H5FL_ARR_ALLOC(hsize_t,cont_dim,0);
-            sel_iter->hyp.diminfo = H5FL_ARR_ALLOC(H5S_hyper_dim_t,cont_dim,0);
-            sel_iter->hyp.size = H5FL_ARR_ALLOC(hsize_t,cont_dim,0);
-            sel_iter->hyp.sel_off = H5FL_ARR_ALLOC(hssize_t,cont_dim,0);
+            sel_iter->hyp.off = H5FL_ARR_MALLOC(hsize_t,cont_dim);
+            assert(sel_iter->hyp.off);
+            sel_iter->hyp.diminfo = H5FL_ARR_MALLOC(H5S_hyper_dim_t,cont_dim);
+            assert(sel_iter->hyp.diminfo);
+            sel_iter->hyp.size = H5FL_ARR_MALLOC(hsize_t,cont_dim);
+            assert(sel_iter->hyp.size);
+            sel_iter->hyp.sel_off = H5FL_ARR_MALLOC(hssize_t,cont_dim);
+            assert(sel_iter->hyp.sel_off);
 
             /* "Flatten" dataspace extent and selection information */
             for(i=space->extent.u.simple.rank-1, acc=1; i>=0; i--) {
@@ -197,10 +201,20 @@ H5S_hyper_iter_init (const H5S_t *space, size_t elmt_size, H5S_sel_iter_t *sel_i
         } /* end if */
         else {
             /* Allocate the position & initialize to initial location */
-            sel_iter->hyp.off = H5FL_ARR_ALLOC(hsize_t,space->extent.u.simple.rank,0);
+            sel_iter->hyp.off = H5FL_ARR_MALLOC(hsize_t,space->extent.u.simple.rank);
+            assert(sel_iter->hyp.off);
             for(u=0; u<space->extent.u.simple.rank; u++)
                 sel_iter->hyp.off[u]=tdiminfo[u].start;
+
+            /* Initialize other regular region information also (for release) */
+            sel_iter->hyp.diminfo = NULL;
+            sel_iter->hyp.size = NULL;
+            sel_iter->hyp.sel_off = NULL;
         } /* end else */
+
+        /* Initialize irregular region information also (for release) */
+        sel_iter->hyp.spans=NULL;
+        sel_iter->hyp.span=NULL;
     } /* end if */
     else {
 /* Initialize the information needed for non-regular hyperslab I/O */
@@ -211,8 +225,10 @@ H5S_hyper_iter_init (const H5S_t *space, size_t elmt_size, H5S_sel_iter_t *sel_i
         H5S_hyper_span_precompute(sel_iter->hyp.spans,elmt_size);
 
         /* Allocate the span tree pointers, span pointers and positions */
-        sel_iter->hyp.span = H5FL_ARR_ALLOC(H5S_hyper_span_t,space->extent.u.simple.rank,0);
-        sel_iter->hyp.off = H5FL_ARR_ALLOC(hsize_t,space->extent.u.simple.rank,0);
+        sel_iter->hyp.span = H5FL_ARR_MALLOC(H5S_hyper_span_t,space->extent.u.simple.rank);
+        assert(sel_iter->hyp.span);
+        sel_iter->hyp.off = H5FL_ARR_MALLOC(hsize_t,space->extent.u.simple.rank);
+        assert(sel_iter->hyp.off);
 
         /* Initialize the starting span_info's and spans */
         spans=sel_iter->hyp.spans;
@@ -230,6 +246,11 @@ H5S_hyper_iter_init (const H5S_t *space, size_t elmt_size, H5S_sel_iter_t *sel_i
             /* Get the pointer to the next level down */
             spans=spans->head->down;
         } /* end for */
+
+        /* Initialize regular region information also (for release) */
+        sel_iter->hyp.diminfo = NULL;
+        sel_iter->hyp.size = NULL;
+        sel_iter->hyp.sel_off = NULL;
     } /* end else */
 
 done:
@@ -499,7 +520,7 @@ H5S_hyper_new_span (hssize_t low, hssize_t high, H5S_hyper_span_info_t *down, H5
     FUNC_ENTER_NOINIT(H5S_hyper_new_span);
 
     /* Allocate a new span node */
-    if((ret_value = H5FL_ALLOC(H5S_hyper_span_t,0))==NULL)
+    if((ret_value = H5FL_MALLOC(H5S_hyper_span_t))==NULL)
         HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "can't allocate hyperslab span");
        
     /* Copy the span's basic information */
@@ -710,7 +731,7 @@ H5S_hyper_copy_span_helper (H5S_hyper_span_info_t *spans)
     } /* end if */
     else {
         /* Allocate a new span_info node */
-        if((ret_value = H5FL_ALLOC(H5S_hyper_span_info_t,0))==NULL)
+        if((ret_value = H5FL_MALLOC(H5S_hyper_span_info_t))==NULL)
             HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "can't allocate hyperslab span");
            
         /* Copy the span_info information */
@@ -1033,7 +1054,7 @@ H5S_hyper_copy (H5S_t *dst, const H5S_t *src)
     /* Check if there is regular hyperslab information to copy */
     if(src->select.sel_info.hslab.diminfo!=NULL) {
         /* Create the per-dimension selection info */
-        if((new_diminfo = H5FL_ARR_ALLOC(H5S_hyper_dim_t,src->extent.u.simple.rank,0))==NULL)
+        if((new_diminfo = H5FL_ARR_MALLOC(H5S_hyper_dim_t,src->extent.u.simple.rank))==NULL)
             HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, FAIL, "can't allocate per-dimension array");
 
         /* Copy the per-dimension selection info */
@@ -1046,7 +1067,7 @@ H5S_hyper_copy (H5S_t *dst, const H5S_t *src)
         dst->select.sel_info.hslab.diminfo = new_diminfo;
 
         /* Create the per-dimension selection info */
-        if((new_diminfo = H5FL_ARR_ALLOC(H5S_hyper_dim_t,src->extent.u.simple.rank,0))==NULL)
+        if((new_diminfo = H5FL_ARR_MALLOC(H5S_hyper_dim_t,src->extent.u.simple.rank))==NULL)
             HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, FAIL, "can't allocate per-dimension array");
 
         /* Copy the per-dimension selection info */
@@ -1676,13 +1697,13 @@ H5S_hyper_deserialize (H5S_t *space, const uint8_t *buf)
     UINT32DECODE(buf,num_elem);  /* decode the number of points */
 
     /* Allocate space for the coordinates */
-    if((start = H5FL_ARR_ALLOC(hsize_t,rank,0))==NULL)
+    if((start = H5FL_ARR_MALLOC(hsize_t,rank))==NULL)
         HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, FAIL, "can't allocate hyperslab information");
-    if((end = H5FL_ARR_ALLOC(hsize_t,rank,0))==NULL)
+    if((end = H5FL_ARR_MALLOC(hsize_t,rank))==NULL)
         HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, FAIL, "can't allocate hyperslab information");
-    if((block = H5FL_ARR_ALLOC(hsize_t,rank,0))==NULL)
+    if((block = H5FL_ARR_MALLOC(hsize_t,rank))==NULL)
         HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, FAIL, "can't allocate hyperslab information");
-    if((count = H5FL_ARR_ALLOC(hsize_t,rank,0))==NULL)
+    if((count = H5FL_ARR_MALLOC(hsize_t,rank))==NULL)
         HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, FAIL, "can't allocate hyperslab information");
     
     /* Set the count for all blocks */
@@ -2603,7 +2624,7 @@ H5S_hyper_append_span (H5S_hyper_span_t **prev_span, H5S_hyper_span_info_t ** sp
         assert(*span_tree==NULL);
 
         /* Allocate a new span_info node */
-        if((*span_tree = H5FL_ALLOC(H5S_hyper_span_info_t,0))==NULL)
+        if((*span_tree = H5FL_MALLOC(H5S_hyper_span_info_t))==NULL)
             HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, FAIL, "can't allocate hyperslab span");
            
         /* Set the span tree's basic information */
@@ -3631,7 +3652,7 @@ H5S_hyper_make_spans (unsigned rank, const hssize_t *start, const hsize_t *strid
         /* Generate all the spans segments for this dimension */
         for(u=0; u<count[i]; u++) {
             /* Allocate a span node */
-            if((span = H5FL_ALLOC(H5S_hyper_span_t,0))==NULL)
+            if((span = H5FL_MALLOC(H5S_hyper_span_t))==NULL)
                 HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "can't allocate hyperslab span");
 
             /* Set the span's basic information */
@@ -3661,7 +3682,7 @@ H5S_hyper_make_spans (unsigned rank, const hssize_t *start, const hsize_t *strid
         } /* end for */
 
         /* Allocate a span info node */
-        if((down = H5FL_ALLOC(H5S_hyper_span_info_t,0))==NULL)
+        if((down = H5FL_MALLOC(H5S_hyper_span_info_t))==NULL)
             HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "can't allocate hyperslab span");
 
         /* Set the reference count */
@@ -3931,7 +3952,7 @@ H5S_select_hyperslab (H5S_t *space, H5S_seloper_t op,
         hssize_t fill=1;
 
         /* Allocate temporary buffer */
-        if ((_stride=H5FL_ARR_ALLOC(hsize_t,space->extent.u.simple.rank,0))==NULL)
+        if ((_stride=H5FL_ARR_MALLOC(hsize_t,space->extent.u.simple.rank))==NULL)
             HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, FAIL, "memory allocation failed for stride buffer");
         H5V_array_fill(_stride,&fill,sizeof(hssize_t),space->extent.u.simple.rank);
         stride = _stride;
@@ -3942,7 +3963,7 @@ H5S_select_hyperslab (H5S_t *space, H5S_seloper_t op,
         hssize_t fill=1;
 
         /* Allocate temporary buffer */
-        if ((_block=H5FL_ARR_ALLOC(hsize_t,space->extent.u.simple.rank,0))==NULL)
+        if ((_block=H5FL_ARR_MALLOC(hsize_t,space->extent.u.simple.rank))==NULL)
             HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, FAIL, "memory allocation failed for stride buffer");
         H5V_array_fill(_block,&fill,sizeof(hssize_t),space->extent.u.simple.rank);
         block = _block;
@@ -4046,7 +4067,7 @@ H5S_select_hyperslab (H5S_t *space, H5S_seloper_t op,
             HGOTO_ERROR(H5E_DATASPACE, H5E_CANTDELETE, FAIL, "can't release hyperslab");
 
         /* Copy all the application per-dimension selection info into the space descriptor */
-        if((diminfo = H5FL_ARR_ALLOC(H5S_hyper_dim_t,space->extent.u.simple.rank,0))==NULL)
+        if((diminfo = H5FL_ARR_MALLOC(H5S_hyper_dim_t,space->extent.u.simple.rank))==NULL)
             HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, FAIL, "can't allocate per-dimension vector");
         for(u=0; u<space->extent.u.simple.rank; u++) {
             diminfo[u].start = start[u];
@@ -4057,7 +4078,7 @@ H5S_select_hyperslab (H5S_t *space, H5S_seloper_t op,
         space->select.sel_info.hslab.app_diminfo = diminfo;
 
         /* Allocate room for the optimized per-dimension selection info */
-        if((diminfo = H5FL_ARR_ALLOC(H5S_hyper_dim_t,space->extent.u.simple.rank,0))==NULL)
+        if((diminfo = H5FL_ARR_MALLOC(H5S_hyper_dim_t,space->extent.u.simple.rank))==NULL)
             HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, FAIL, "can't allocate per-dimension vector");
 
         /* Optimize the hyperslab selection to detect contiguously selected block/stride information */
@@ -4479,7 +4500,7 @@ H5S_select_hyperslab (H5S_t *space, H5S_seloper_t op,
         hssize_t fill=1;
 
         /* Allocate temporary buffer */
-        if ((_stride=H5FL_ARR_ALLOC(hsize_t,space->extent.u.simple.rank,0))==NULL)
+        if ((_stride=H5FL_ARR_MALLOC(hsize_t,space->extent.u.simple.rank))==NULL)
             HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, FAIL, "memory allocation failed for stride buffer");
         H5V_array_fill(_stride,&fill,sizeof(hssize_t),space->extent.u.simple.rank);
         stride = _stride;
@@ -4490,7 +4511,7 @@ H5S_select_hyperslab (H5S_t *space, H5S_seloper_t op,
         hssize_t fill=1;
 
         /* Allocate temporary buffer */
-        if ((_block=H5FL_ARR_ALLOC(hsize_t,space->extent.u.simple.rank,0))==NULL)
+        if ((_block=H5FL_ARR_MALLOC(hsize_t,space->extent.u.simple.rank))==NULL)
             HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, FAIL, "memory allocation failed for stride buffer");
         H5V_array_fill(_block,&fill,sizeof(hssize_t),space->extent.u.simple.rank);
         block = _block;
@@ -4595,7 +4616,7 @@ H5S_select_hyperslab (H5S_t *space, H5S_seloper_t op,
             HGOTO_ERROR(H5E_DATASPACE, H5E_CANTDELETE, FAIL, "can't release hyperslab");
 
         /* Copy all the application per-dimension selection info into the space descriptor */
-        if((diminfo = H5FL_ARR_ALLOC(H5S_hyper_dim_t,space->extent.u.simple.rank,0))==NULL)
+        if((diminfo = H5FL_ARR_MALLOC(H5S_hyper_dim_t,space->extent.u.simple.rank))==NULL)
             HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, FAIL, "can't allocate per-dimension vector");
         for(u=0; u<space->extent.u.simple.rank; u++) {
             diminfo[u].start = start[u];
@@ -4606,7 +4627,7 @@ H5S_select_hyperslab (H5S_t *space, H5S_seloper_t op,
         space->select.sel_info.hslab.app_diminfo = diminfo;
 
         /* Allocate room for the optimized per-dimension selection info */
-        if((diminfo = H5FL_ARR_ALLOC(H5S_hyper_dim_t,space->extent.u.simple.rank,0))==NULL)
+        if((diminfo = H5FL_ARR_MALLOC(H5S_hyper_dim_t,space->extent.u.simple.rank))==NULL)
             HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, FAIL, "can't allocate per-dimension vector");
 
         /* Optimize the hyperslab selection to detect contiguously selected block/stride information */