[svn-r6252] Purpose:

    Lots of performance improvements & a couple new internal API interfaces.

Description:
    Performance Improvements:
        - Cached file offset & length sizes in shared file struct, to avoid
            constantly looking them up in the FCPL.
        - Generic property improvements:
            - Added "revision" number to generic property classes to speed
                up comparisons.
            - Changed method of storing properties from using a hash-table
                to the TBBT routines in the library.
            - Share the propery names between classes and the lists derived
                from them.
            - Removed redundant 'def_value' buffer from each property.
            - Switching code to use a "copy on write" strategy for
                properties in each list, where the properties in each list
                are shared with the properties in the class, until a
                property's value is changed in a list.
        - Fixed error in layout code which was allocating too many buffers.
        - Redefined public macros of the form (H5open()/H5check, <variable>)
            internally to only be (<variable>), avoiding innumerable useless
            calls to H5open() and H5check_version().
        - Reuse already zeroed buffers in H5F_contig_fill instead of
            constantly re-zeroing them.
        - Don't write fill values if writing entire dataset.
        - Use gettimeofday() system call instead of time() system when
            checking the modification time of a dataset.
        - Added reference counted string API and use it for tracking the
            names of objects opening in a file (for the ID->name code).
        - Removed redundant H5P_get() calls in B-tree routines.
        - Redefine H5T datatype macros internally to the library, to avoid
            calling H5check redundantly.
        - Keep dataspace information for dataset locally instead of reading
            from disk each time.  Added new module to track open objects
            in a file, to allow this (which will be useful eventually for
            some FPH5 metadata caching issues).
        - Remove H5AC_find macro which was inlining metadata cache lookups,
            and call function instead.
        - Remove redundant memset() calls from H5G_namei() routine.
        - Remove redundant checking of object type when locating objects
            in metadata cache and rely on the address only.
        - Create default dataset object to use when default dataset creation
            property list is used to create datasets, bypassing querying
            for all the property list values.
        - Use default I/O vector size when performing raw data with the
            default dataset transfer property list, instead of querying for
            I/O vector size.
        - Remove H5P_DEFAULT internally to the library, replacing it with
            more specific default property list based on the type of
            property list needed.
        - Remove redundant memset() calls in object header message (H5O*)
            routines.
        - Remove redunant memset() calls in data I/O routines.
        - Split free-list allocation routines into malloc() and calloc()-
            like routines, instead of one combined routine.
        - Remove lots of indirection in H5O*() routines.
        - Simplify metadata cache entry comparison routine (used when
            flushing entire cache out).
        - Only enable metadata cache statistics when H5AC_DEBUG is turned
            on, instead of always tracking them.
        - Simplify address comparison macro (H5F_addr_eq).
        - Remove redundant metadata cache entry protections during dataset
            creation by protecting the object header once and making all
            the modifications necessary for the dataset creation before
            unprotecting it.
        - Reduce # of "number of element in extent" computations performed
            by computing and storing the value during dataspace creation.
        - Simplify checking for group location's file information, when file
            has not been involving in file-mounting operations.
        - Use binary encoding for modification time, instead of ASCII.
        - Hoist H5HL_peek calls (to get information in a local heap)
            out of loops in many group routine.
        - Use static variable for iterators of selections, instead of
            dynamically allocation them each time.
        - Lookup & insert new entries in one step, avoiding traversing
            group's B-tree twice.
        - Fixed memory leak in H5Gget_objname_idx() routine (tangential to
            performance improvements, but fixed along the way).
        - Use free-list for reference counted strings.
        - Don't bother copying object names into cached group entries,
            since they are re-created when an object is opened.

        The benchmark I used to measure these results created several thousand
        small (2K) datasets in a file and wrote out the data for them.  This is
        Elena's "regular.c" benchmark.

        These changes resulted in approximately ~4.3x speedup of the
        development branch when compared to the previous code in the
        development branch and ~1.4x speedup compared to the release
        branch.

        Additionally, these changes reduce the total memory used (code and
        data) by the development branch by ~800KB, bringing the development
        branch back into the same ballpark as the release branch.

        I'll send out a more detailed description of the benchmark results
        as a followup note.

    New internal API routines:
        Added "reference counted strings" API for tracking strings that get
            used by multiple owners without duplicating the strings.
        Added "ternary search tree" API for text->object mappings.

Platforms tested:
    Tested h5committest {arabica (fortran), eirene (fortran, C++)
	modi4 (parallel, fortran)}
    Other platforms/configurations tested?
        FreeBSD 4.7 (sleipnir) serial & parallel
        Solaris 2.6 (baldric) serial

1 files changed, 112 insertions, 37 deletions

diff --git a/src/H5S.c b/src/H5S.c
index 6424ada..0127056 100644
--- a/src/H5S.c
+++ b/src/H5S.c
@@ -303,8 +303,21 @@ H5S_create(H5S_class_t type)
     FUNC_ENTER_NOAPI(H5S_create, NULL);
 
     /* Create a new data space */
-    if((ret_value = H5FL_ALLOC(H5S_t,1))!=NULL) {
+    if((ret_value = H5FL_CALLOC(H5S_t))!=NULL) {
         ret_value->extent.type = type;
+
+        switch(type) {
+            case H5S_SCALAR:
+                ret_value->extent.nelem = 1;
+                break;
+            case H5S_SIMPLE:
+                ret_value->extent.nelem = 0;
+                break;
+            default:
+                assert("unknown dataspace (extent) type" && 0);
+                break;
+        } /* end switch */
+
         if(H5S_select_all(ret_value,0)<0)
             HGOTO_ERROR (H5E_DATASPACE, H5E_CANTSET, NULL, "unable to set all selection");
     } /* end if */
@@ -647,12 +660,12 @@ H5S_extent_copy(H5S_extent_t *dst, const H5S_extent_t *src)
 
         case H5S_SIMPLE:
             if (src->u.simple.size) {
-                dst->u.simple.size = H5FL_ARR_ALLOC(hsize_t,src->u.simple.rank,0);
+                dst->u.simple.size = H5FL_ARR_MALLOC(hsize_t,src->u.simple.rank);
                 for (u = 0; u < src->u.simple.rank; u++)
                     dst->u.simple.size[u] = src->u.simple.size[u];
             }
             if (src->u.simple.max) {
-                dst->u.simple.max = H5FL_ARR_ALLOC(hsize_t,src->u.simple.rank,0);
+                dst->u.simple.max = H5FL_ARR_MALLOC(hsize_t,src->u.simple.rank);
                 for (u = 0; u < src->u.simple.rank; u++)
                     dst->u.simple.max[u] = src->u.simple.max[u];
             }
@@ -697,7 +710,7 @@ H5S_copy(const H5S_t *src)
 
     FUNC_ENTER_NOAPI(H5S_copy, NULL);
 
-    if (NULL==(dst = H5FL_ALLOC(H5S_t,0)))
+    if (NULL==(dst = H5FL_MALLOC(H5S_t)))
         HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed");
 
     /* Copy the field in the struct */
@@ -740,30 +753,14 @@ hssize_t
 H5S_get_simple_extent_npoints(const H5S_t *ds)
 {
     hssize_t    ret_value;
-    unsigned		u;
 
     FUNC_ENTER_NOAPI(H5S_get_simple_extent_npoints, -1);
 
     /* check args */
     assert(ds);
 
-    switch (ds->extent.type) {
-        case H5S_SCALAR:
-            ret_value = 1;
-            break;
-
-        case H5S_SIMPLE:
-            for (ret_value=1, u=0; u<ds->extent.u.simple.rank; u++)
-                ret_value *= ds->extent.u.simple.size[u];
-            break;
-
-        case H5S_COMPLEX:
-            HGOTO_ERROR(H5E_DATASPACE, H5E_UNSUPPORTED, FAIL, "complex data spaces are not supported yet");
-
-        default:
-            assert("unknown data space class" && 0);
-            HGOTO_ERROR(H5E_DATASPACE, H5E_UNSUPPORTED, FAIL, "internal error (unknown data space class)");
-    }
+    /* Get the number of elements in extent */
+    ret_value = ds->extent.nelem;
 
 done:
     FUNC_LEAVE(ret_value);
@@ -1078,7 +1075,7 @@ done:
  *-------------------------------------------------------------------------
  */
 herr_t
-H5S_modify(H5G_entry_t *ent, const H5S_t *ds)
+H5S_modify(H5G_entry_t *ent, const H5S_t *ds, hbool_t update_time)
 {
     herr_t ret_value=SUCCEED;   /* Return value */
 
@@ -1090,7 +1087,7 @@ H5S_modify(H5G_entry_t *ent, const H5S_t *ds)
     switch (ds->extent.type) {
         case H5S_SCALAR:
         case H5S_SIMPLE:
-            if (H5O_modify(ent, H5O_SDSPACE, 0, 0, &(ds->extent.u.simple))<0)
+            if (H5O_modify(ent, H5O_SDSPACE, 0, 0, update_time, &(ds->extent.u.simple))<0)
                 HGOTO_ERROR(H5E_DATASPACE, H5E_CANTINIT, FAIL, "can't update simple data space message");
             break;
 
@@ -1108,6 +1105,52 @@ done:
 
 
 /*-------------------------------------------------------------------------
+ * Function:	H5S_append
+ *
+ * Purpose:	Updates a data space by adding a message to an object
+ *		header.
+ *
+ * Return:	Non-negative on success/Negative on failure
+ *
+ * Programmer:	Quincey Koziol
+ *		Tuesday, December 31, 2002
+ *
+ * Modifications:
+ *
+ *-------------------------------------------------------------------------
+ */
+herr_t
+H5S_append(H5F_t *f, H5O_t *oh, const H5S_t *ds)
+{
+    herr_t ret_value=SUCCEED;   /* Return value */
+
+    FUNC_ENTER_NOAPI(H5S_append, FAIL);
+
+    assert(f);
+    assert(oh);
+    assert(ds);
+
+    switch (ds->extent.type) {
+        case H5S_SCALAR:
+        case H5S_SIMPLE:
+            if (H5O_append(f, oh, H5O_SDSPACE, 0, &(ds->extent.u.simple))<0)
+                HGOTO_ERROR(H5E_DATASPACE, H5E_CANTINIT, FAIL, "can't update simple data space message");
+            break;
+
+        case H5S_COMPLEX:
+            HGOTO_ERROR(H5E_DATASPACE, H5E_UNSUPPORTED, FAIL, "complex data spaces are not implemented yet");
+
+        default:
+            assert("unknown data space class" && 0);
+            break;
+    }
+
+done:
+    FUNC_LEAVE(ret_value);
+} /* end H5S_append() */
+
+
+/*-------------------------------------------------------------------------
  * Function:	H5S_read
  *
  * Purpose:	Reads the data space from an object header.
@@ -1136,23 +1179,34 @@ H5S_read(H5G_entry_t *ent)
     /* check args */
     assert(ent);
 
-    if (NULL==(ds = H5FL_ALLOC(H5S_t,1)))
+    if (NULL==(ds = H5FL_CALLOC(H5S_t)))
         HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed");
 
     if (H5O_read(ent, H5O_SDSPACE, 0, &(ds->extent.u.simple)) == NULL)
         HGOTO_ERROR(H5E_DATASPACE, H5E_CANTINIT, NULL, "unable to load dataspace info from dataset header");
 
-    if(ds->extent.u.simple.rank != 0)
-         ds->extent.type = H5S_SIMPLE;
-    else
-         ds->extent.type = H5S_SCALAR;
+    if(ds->extent.u.simple.rank != 0) {
+        hsize_t nelem;  /* Number of elements in extent */
+        unsigned u;     /* Local index variable */
+
+        ds->extent.type = H5S_SIMPLE;
+
+        /* Compute the number of elements in the extent */
+        for(u=0, nelem=1; u<ds->extent.u.simple.rank; u++)
+            nelem*=ds->extent.u.simple.size[u];
+        ds->extent.nelem = nelem;
+    } /* end if */
+    else {
+        ds->extent.type = H5S_SCALAR;
+        ds->extent.nelem = 1;
+    } /* end else */
 
     /* Default to entire dataspace being selected */
     if(H5S_select_all(ds,0)<0)
         HGOTO_ERROR (H5E_DATASPACE, H5E_CANTSET, NULL, "unable to set all selection");
 
     /* Allocate space for the offset and set it to zeros */
-    if (NULL==(ds->select.offset = H5FL_ARR_ALLOC(hssize_t,ds->extent.u.simple.rank,1)))
+    if (NULL==(ds->select.offset = H5FL_ARR_CALLOC(hssize_t,ds->extent.u.simple.rank)))
         HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed");
 
     /* Set the value for successful return */
@@ -1407,7 +1461,7 @@ H5S_set_extent_simple (H5S_t *space, unsigned rank, const hsize_t *dims,
         space->select.offset=H5FL_ARR_FREE(hssize_t,space->select.offset);
 
     /* Allocate space for the offset and set it to zeros */
-    if (NULL==(space->select.offset = H5FL_ARR_ALLOC(hssize_t,rank,1)))
+    if (NULL==(space->select.offset = H5FL_ARR_CALLOC(hssize_t,rank)))
         HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, FAIL, "memory allocation failed");
 
     /* shift out of the previous state to a "simple" dataspace */
@@ -1433,18 +1487,27 @@ H5S_set_extent_simple (H5S_t *space, unsigned rank, const hsize_t *dims,
 
     if (rank == 0) {		/* scalar variable */
         space->extent.type = H5S_SCALAR;
+        space->extent.nelem = 1;
         space->extent.u.simple.rank = 0;	/* set to scalar rank */
     } else {
+        hsize_t nelem;  /* Number of elements in extent */
+        unsigned u;     /* Local index variable */
+
         space->extent.type = H5S_SIMPLE;
 
         /* Set the rank and copy the dims */
         space->extent.u.simple.rank = rank;
-        space->extent.u.simple.size = H5FL_ARR_ALLOC(hsize_t,rank,0);
+        space->extent.u.simple.size = H5FL_ARR_MALLOC(hsize_t,rank);
         HDmemcpy(space->extent.u.simple.size, dims, sizeof(hsize_t) * rank);
 
+        /* Compute the number of elements in the extent */
+        for(u=0, nelem=1; u<space->extent.u.simple.rank; u++)
+            nelem*=space->extent.u.simple.size[u];
+        space->extent.nelem = nelem;
+
         /* Copy the maximum dimensions if specified */
         if(max!=NULL) {
-            space->extent.u.simple.max = H5FL_ARR_ALLOC(hsize_t,rank,0);
+            space->extent.u.simple.max = H5FL_ARR_MALLOC(hsize_t,rank);
             HDmemcpy(space->extent.u.simple.max, max, sizeof(hsize_t) * rank);
         } /* end if */
     }
@@ -1629,6 +1692,7 @@ H5S_extend (H5S_t *space, const hsize_t *size)
     assert (space && H5S_SIMPLE==space->extent.type);
     assert (size);
 
+    /* Check through all the dimensions to see if modifying the dataspace is allowed */
     for (u=0; u<space->extent.u.simple.rank; u++) {
         if (space->extent.u.simple.size[u]<size[u]) {
             if (space->extent.u.simple.max &&
@@ -1641,10 +1705,16 @@ H5S_extend (H5S_t *space, const hsize_t *size)
 
     /* Update */
     if (ret_value) {
-        for (u=0; u<space->extent.u.simple.rank; u++) {
+        hsize_t nelem;  /* Number of elements in extent */
+
+        /* Change the dataspace size & re-compute the number of elements in the extent */
+        for (u=0, nelem=1; u<space->extent.u.simple.rank; u++) {
             if (space->extent.u.simple.size[u]<size[u])
                 space->extent.u.simple.size[u] = size[u];
+
+            nelem*=space->extent.u.simple.size[u];
         }
+        space->extent.nelem = nelem;
     }
 
 done:
@@ -1872,7 +1942,7 @@ H5Soffset_simple(hid_t space_id, const hssize_t *offset)
 
     /* Allocate space for new offset */
     if(space->select.offset==NULL) {
-        if (NULL==(space->select.offset = H5FL_ARR_ALLOC(hssize_t,space->extent.u.simple.rank,0)))
+        if (NULL==(space->select.offset = H5FL_ARR_MALLOC(hssize_t,space->extent.u.simple.rank)))
             HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, FAIL, "memory allocation failed");
     }
 
@@ -1922,9 +1992,14 @@ H5S_set_extent( H5S_t *space, const hsize_t *size )
 
     /* Update */
     if (ret_value) {
-        /* Update dimensions with new values */
-        for ( u = 0; u < space->extent.u.simple.rank; u++ )
+        hsize_t nelem;  /* Number of elements in extent */
+
+        /* Change the dataspace size & re-compute the number of elements in the extent */
+        for (u=0, nelem=1; u < space->extent.u.simple.rank; u++ ) {
             space->extent.u.simple.size[u] = size[u];
+            nelem*=space->extent.u.simple.size[u];
+        } /* end for */
+        space->extent.nelem = nelem;
     } /* end if */
 
 done: