[svn-r6252] Purpose:

    Lots of performance improvements & a couple new internal API interfaces.

Description:
    Performance Improvements:
        - Cached file offset & length sizes in shared file struct, to avoid
            constantly looking them up in the FCPL.
        - Generic property improvements:
            - Added "revision" number to generic property classes to speed
                up comparisons.
            - Changed method of storing properties from using a hash-table
                to the TBBT routines in the library.
            - Share the propery names between classes and the lists derived
                from them.
            - Removed redundant 'def_value' buffer from each property.
            - Switching code to use a "copy on write" strategy for
                properties in each list, where the properties in each list
                are shared with the properties in the class, until a
                property's value is changed in a list.
        - Fixed error in layout code which was allocating too many buffers.
        - Redefined public macros of the form (H5open()/H5check, <variable>)
            internally to only be (<variable>), avoiding innumerable useless
            calls to H5open() and H5check_version().
        - Reuse already zeroed buffers in H5F_contig_fill instead of
            constantly re-zeroing them.
        - Don't write fill values if writing entire dataset.
        - Use gettimeofday() system call instead of time() system when
            checking the modification time of a dataset.
        - Added reference counted string API and use it for tracking the
            names of objects opening in a file (for the ID->name code).
        - Removed redundant H5P_get() calls in B-tree routines.
        - Redefine H5T datatype macros internally to the library, to avoid
            calling H5check redundantly.
        - Keep dataspace information for dataset locally instead of reading
            from disk each time.  Added new module to track open objects
            in a file, to allow this (which will be useful eventually for
            some FPH5 metadata caching issues).
        - Remove H5AC_find macro which was inlining metadata cache lookups,
            and call function instead.
        - Remove redundant memset() calls from H5G_namei() routine.
        - Remove redundant checking of object type when locating objects
            in metadata cache and rely on the address only.
        - Create default dataset object to use when default dataset creation
            property list is used to create datasets, bypassing querying
            for all the property list values.
        - Use default I/O vector size when performing raw data with the
            default dataset transfer property list, instead of querying for
            I/O vector size.
        - Remove H5P_DEFAULT internally to the library, replacing it with
            more specific default property list based on the type of
            property list needed.
        - Remove redundant memset() calls in object header message (H5O*)
            routines.
        - Remove redunant memset() calls in data I/O routines.
        - Split free-list allocation routines into malloc() and calloc()-
            like routines, instead of one combined routine.
        - Remove lots of indirection in H5O*() routines.
        - Simplify metadata cache entry comparison routine (used when
            flushing entire cache out).
        - Only enable metadata cache statistics when H5AC_DEBUG is turned
            on, instead of always tracking them.
        - Simplify address comparison macro (H5F_addr_eq).
        - Remove redundant metadata cache entry protections during dataset
            creation by protecting the object header once and making all
            the modifications necessary for the dataset creation before
            unprotecting it.
        - Reduce # of "number of element in extent" computations performed
            by computing and storing the value during dataspace creation.
        - Simplify checking for group location's file information, when file
            has not been involving in file-mounting operations.
        - Use binary encoding for modification time, instead of ASCII.
        - Hoist H5HL_peek calls (to get information in a local heap)
            out of loops in many group routine.
        - Use static variable for iterators of selections, instead of
            dynamically allocation them each time.
        - Lookup & insert new entries in one step, avoiding traversing
            group's B-tree twice.
        - Fixed memory leak in H5Gget_objname_idx() routine (tangential to
            performance improvements, but fixed along the way).
        - Use free-list for reference counted strings.
        - Don't bother copying object names into cached group entries,
            since they are re-created when an object is opened.

        The benchmark I used to measure these results created several thousand
        small (2K) datasets in a file and wrote out the data for them.  This is
        Elena's "regular.c" benchmark.

        These changes resulted in approximately ~4.3x speedup of the
        development branch when compared to the previous code in the
        development branch and ~1.4x speedup compared to the release
        branch.

        Additionally, these changes reduce the total memory used (code and
        data) by the development branch by ~800KB, bringing the development
        branch back into the same ballpark as the release branch.

        I'll send out a more detailed description of the benchmark results
        as a followup note.

    New internal API routines:
        Added "reference counted strings" API for tracking strings that get
            used by multiple owners without duplicating the strings.
        Added "ternary search tree" API for text->object mappings.

Platforms tested:
    Tested h5committest {arabica (fortran), eirene (fortran, C++)
	modi4 (parallel, fortran)}
    Other platforms/configurations tested?
        FreeBSD 4.7 (sleipnir) serial & parallel
        Solaris 2.6 (baldric) serial

1 files changed, 102 insertions, 19 deletions

diff --git a/src/H5Ofill.c b/src/H5Ofill.c
index dd734b9..92d2000 100644
--- a/src/H5Ofill.c
+++ b/src/H5Ofill.c
@@ -10,6 +10,7 @@
  */
 #include "H5private.h"
 #include "H5Eprivate.h"
+#include "H5FLprivate.h"	/*Free Lists	  */
 #include "H5Iprivate.h"
 #include "H5MMprivate.h"
 #include "H5Oprivate.h"
@@ -22,6 +23,7 @@ static herr_t H5O_fill_new_encode(H5F_t *f, uint8_t *p, const void *_mesg);
 static void  *H5O_fill_new_copy(const void *_mesg, void *_dest);
 static size_t H5O_fill_new_size(H5F_t *f, const void *_mesg);
 static herr_t H5O_fill_new_reset(void *_mesg);
+static herr_t H5O_fill_new_free(void *_mesg);
 static herr_t H5O_fill_new_debug(H5F_t *f, const void *_mesg, FILE *stream,
 			     int indent, int fwidth);
 
@@ -30,6 +32,7 @@ static herr_t H5O_fill_encode(H5F_t *f, uint8_t *p, const void *_mesg);
 static void  *H5O_fill_copy(const void *_mesg, void *_dest);
 static size_t H5O_fill_size(H5F_t *f, const void *_mesg);
 static herr_t H5O_fill_reset(void *_mesg);
+static herr_t H5O_fill_free(void *_mesg);
 static herr_t H5O_fill_debug(H5F_t *f, const void *_mesg, FILE *stream,
 			     int indent, int fwidth);
 
@@ -43,7 +46,7 @@ const H5O_class_t H5O_FILL[1] = {{
     H5O_fill_copy,              /*copy the native value                 */
     H5O_fill_size,              /*raw message size                      */
     H5O_fill_reset,             /*free internal memory                  */
-    NULL,                       /* free method   	                */
+    H5O_fill_free,		/* free method			*/
     NULL,                       /*get share method                      */
     NULL,                       /*set share method                      */
     H5O_fill_debug,             /*debug the message                     */
@@ -59,7 +62,7 @@ const H5O_class_t H5O_FILL_NEW[1] = {{
     H5O_fill_new_copy,		/*copy the native value			*/
     H5O_fill_new_size,		/*raw message size			*/
     H5O_fill_new_reset,		/*free internal memory			*/
-    NULL,		        /* free method				*/
+    H5O_fill_new_free,		/* free method			*/
     NULL,			/*get share method			*/
     NULL,			/*set share method			*/
     H5O_fill_new_debug,		/*debug the message			*/
@@ -71,6 +74,12 @@ const H5O_class_t H5O_FILL_NEW[1] = {{
 static int interface_initialize_g = 0;
 #define INTERFACE_INIT NULL
 
+/* Declare a free list to manage the H5O_fill_new_t struct */
+H5FL_DEFINE(H5O_fill_new_t);
+
+/* Declare a free list to manage the H5O_fill_t struct */
+H5FL_DEFINE(H5O_fill_t);
+
 
 /*-------------------------------------------------------------------------
  * Function:	H5O_fill_new_decode
@@ -104,7 +113,7 @@ H5O_fill_new_decode(H5F_t UNUSED *f, const uint8_t *p,
     assert(p);
     assert(!sh);
 
-    if (NULL==(mesg=H5MM_calloc(sizeof(H5O_fill_new_t))))
+    if (NULL==(mesg=H5FL_CALLOC(H5O_fill_new_t)))
 	HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed for fill value message");
  
     /* Version */
@@ -137,7 +146,7 @@ done:
     if (!ret_value && mesg) {
         if(mesg->buf)
             H5MM_xfree(mesg->buf);
-	H5MM_xfree(mesg);
+	H5FL_FREE(H5O_fill_new_t,mesg);
     }
 
     FUNC_LEAVE(ret_value);
@@ -173,7 +182,7 @@ H5O_fill_decode(H5F_t UNUSED *f, const uint8_t *p,
     assert(p);
     assert(!sh);
 
-    if (NULL==(mesg=H5MM_calloc(sizeof(H5O_fill_t))))
+    if (NULL==(mesg=H5FL_CALLOC(H5O_fill_t)))
         HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed for fill value message");
     UINT32DECODE(p, mesg->size);
     if (mesg->size>0) {
@@ -189,7 +198,7 @@ done:
     if (!ret_value && mesg) {
         if(mesg->buf)
             H5MM_xfree(mesg->buf);
-        H5MM_xfree(mesg);
+	H5FL_FREE(H5O_fill_t,mesg);
     }
 
     FUNC_LEAVE(ret_value);
@@ -311,13 +320,16 @@ H5O_fill_new_copy(const void *_mesg, void *_dest)
 
     assert(mesg);
 
-    if (!dest && NULL==(dest=H5MM_calloc(sizeof(H5O_fill_new_t))))
+    if (!dest && NULL==(dest=H5FL_MALLOC(H5O_fill_new_t)))
 	HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed for fill message");
 
     /* Copy data type of fill value */
-    if (mesg->type &&
-            NULL==(dest->type=H5T_copy(mesg->type, H5T_COPY_TRANSIENT)))
-	HGOTO_ERROR(H5E_DATATYPE, H5E_CANTINIT, NULL, "unable to copy fill value data type");
+    if (mesg->type) {
+        if(NULL==(dest->type=H5T_copy(mesg->type, H5T_COPY_TRANSIENT)))
+            HGOTO_ERROR(H5E_DATATYPE, H5E_CANTINIT, NULL, "unable to copy fill value data type");
+    } /* end if */
+    else
+        dest->type=NULL;
 
     /* Copy fill value and its size */
     if (mesg->buf) {
@@ -326,7 +338,11 @@ H5O_fill_new_copy(const void *_mesg, void *_dest)
 	    HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed for fill value");
 	dest->size = mesg->size;
 	HDmemcpy(dest->buf, mesg->buf, (size_t)mesg->size);
-    }
+    } /* end if */
+    else {
+        dest->buf=NULL;
+        dest->size=0;
+    } /* end else */
 
     /* Copy three fill value attributes */
     dest->alloc_time   = mesg->alloc_time;
@@ -343,7 +359,7 @@ done:
 	if (dest->type)
             H5T_close(dest->type);
 	if (!_dest)
-            H5MM_xfree(dest);
+            H5FL_FREE(H5O_fill_new_t,dest);
     }
 
     FUNC_LEAVE(ret_value);
@@ -378,17 +394,24 @@ H5O_fill_copy(const void *_mesg, void *_dest)
 
     assert(mesg);
 
-    if (!dest && NULL==(dest=H5MM_calloc(sizeof(H5O_fill_t))))
+    if (!dest && NULL==(dest=H5FL_CALLOC(H5O_fill_t)))
         HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed for fill message");
-    if (mesg->type &&
-            NULL==(dest->type=H5T_copy(mesg->type, H5T_COPY_TRANSIENT)))
-        HGOTO_ERROR(H5E_DATATYPE, H5E_CANTINIT, NULL, "unable to copy fill value data type");
+
+    if (mesg->type) {
+        if(NULL==(dest->type=H5T_copy(mesg->type, H5T_COPY_TRANSIENT)))
+            HGOTO_ERROR(H5E_DATATYPE, H5E_CANTINIT, NULL, "unable to copy fill value data type");
+    } /* end if */
+    else
+        dest->type=NULL;
+
     if (mesg->buf) {
         if (NULL==(dest->buf=H5MM_malloc(mesg->size)))
             HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed for fill value");
         dest->size = mesg->size;
         HDmemcpy(dest->buf, mesg->buf, mesg->size);
-    }
+    } /* end if */
+    else
+        dest->buf=NULL;
 
     /* Set return value */
     ret_value = dest;
@@ -400,7 +423,7 @@ done:
         if (dest->type)
             H5T_close(dest->type);
         if (!_dest)
-            H5MM_xfree(dest);
+            H5FL_FREE(H5O_fill_t,dest);
     }
 
     FUNC_LEAVE(ret_value);
@@ -567,6 +590,66 @@ done:
 
 
 /*-------------------------------------------------------------------------
+ * Function:	H5O_fill_new_free
+ *
+ * Purpose:	Free's the message
+ *
+ * Return:	Non-negative on success/Negative on failure
+ *
+ * Programmer:	Quincey Koziol
+ *              Thursday, December 5, 2002
+ *
+ * Modifications:
+ *
+ *-------------------------------------------------------------------------
+ */
+static herr_t
+H5O_fill_new_free (void *mesg)
+{
+    herr_t ret_value=SUCCEED;   /* Return value */
+
+    FUNC_ENTER_NOAPI(H5O_fill_new_free, FAIL);
+
+    assert (mesg);
+
+    H5FL_FREE(H5O_fill_new_t,mesg);
+
+done:
+    FUNC_LEAVE (ret_value);
+}
+
+
+/*-------------------------------------------------------------------------
+ * Function:	H5O_fill_free
+ *
+ * Purpose:	Free's the message
+ *
+ * Return:	Non-negative on success/Negative on failure
+ *
+ * Programmer:	Quincey Koziol
+ *              Thursday, December 5, 2002
+ *
+ * Modifications:
+ *
+ *-------------------------------------------------------------------------
+ */
+static herr_t
+H5O_fill_free (void *mesg)
+{
+    herr_t ret_value=SUCCEED;   /* Return value */
+
+    FUNC_ENTER_NOAPI(H5O_fill_free, FAIL);
+
+    assert (mesg);
+
+    H5FL_FREE(H5O_fill_t,mesg);
+
+done:
+    FUNC_LEAVE (ret_value);
+}
+
+
+/*-------------------------------------------------------------------------
  * Function:	H5O_fill_new_debug
  *
  * Purpose:	Prints debugging info for the message.
@@ -721,7 +804,7 @@ H5O_fill_convert(void *_fill, H5T_t *dset_type)
 	HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, FAIL, "memory allocation failed for type conversion");
 
     /* Do the conversion */
-    if (H5T_convert(tpath, src_id, dst_id, (hsize_t)1, 0, 0, buf, bkg, H5P_DEFAULT)<0)
+    if (H5T_convert(tpath, src_id, dst_id, (hsize_t)1, 0, 0, buf, bkg, H5P_DATASET_XFER_DEFAULT)<0)
 	HGOTO_ERROR(H5E_DATATYPE, H5E_CANTINIT, FAIL, "data type conversion failed");
 
     /* Update the fill message */