[svn-r6252] Purpose:

    Lots of performance improvements & a couple new internal API interfaces.

Description:
    Performance Improvements:
        - Cached file offset & length sizes in shared file struct, to avoid
            constantly looking them up in the FCPL.
        - Generic property improvements:
            - Added "revision" number to generic property classes to speed
                up comparisons.
            - Changed method of storing properties from using a hash-table
                to the TBBT routines in the library.
            - Share the propery names between classes and the lists derived
                from them.
            - Removed redundant 'def_value' buffer from each property.
            - Switching code to use a "copy on write" strategy for
                properties in each list, where the properties in each list
                are shared with the properties in the class, until a
                property's value is changed in a list.
        - Fixed error in layout code which was allocating too many buffers.
        - Redefined public macros of the form (H5open()/H5check, <variable>)
            internally to only be (<variable>), avoiding innumerable useless
            calls to H5open() and H5check_version().
        - Reuse already zeroed buffers in H5F_contig_fill instead of
            constantly re-zeroing them.
        - Don't write fill values if writing entire dataset.
        - Use gettimeofday() system call instead of time() system when
            checking the modification time of a dataset.
        - Added reference counted string API and use it for tracking the
            names of objects opening in a file (for the ID->name code).
        - Removed redundant H5P_get() calls in B-tree routines.
        - Redefine H5T datatype macros internally to the library, to avoid
            calling H5check redundantly.
        - Keep dataspace information for dataset locally instead of reading
            from disk each time.  Added new module to track open objects
            in a file, to allow this (which will be useful eventually for
            some FPH5 metadata caching issues).
        - Remove H5AC_find macro which was inlining metadata cache lookups,
            and call function instead.
        - Remove redundant memset() calls from H5G_namei() routine.
        - Remove redundant checking of object type when locating objects
            in metadata cache and rely on the address only.
        - Create default dataset object to use when default dataset creation
            property list is used to create datasets, bypassing querying
            for all the property list values.
        - Use default I/O vector size when performing raw data with the
            default dataset transfer property list, instead of querying for
            I/O vector size.
        - Remove H5P_DEFAULT internally to the library, replacing it with
            more specific default property list based on the type of
            property list needed.
        - Remove redundant memset() calls in object header message (H5O*)
            routines.
        - Remove redunant memset() calls in data I/O routines.
        - Split free-list allocation routines into malloc() and calloc()-
            like routines, instead of one combined routine.
        - Remove lots of indirection in H5O*() routines.
        - Simplify metadata cache entry comparison routine (used when
            flushing entire cache out).
        - Only enable metadata cache statistics when H5AC_DEBUG is turned
            on, instead of always tracking them.
        - Simplify address comparison macro (H5F_addr_eq).
        - Remove redundant metadata cache entry protections during dataset
            creation by protecting the object header once and making all
            the modifications necessary for the dataset creation before
            unprotecting it.
        - Reduce # of "number of element in extent" computations performed
            by computing and storing the value during dataspace creation.
        - Simplify checking for group location's file information, when file
            has not been involving in file-mounting operations.
        - Use binary encoding for modification time, instead of ASCII.
        - Hoist H5HL_peek calls (to get information in a local heap)
            out of loops in many group routine.
        - Use static variable for iterators of selections, instead of
            dynamically allocation them each time.
        - Lookup & insert new entries in one step, avoiding traversing
            group's B-tree twice.
        - Fixed memory leak in H5Gget_objname_idx() routine (tangential to
            performance improvements, but fixed along the way).
        - Use free-list for reference counted strings.
        - Don't bother copying object names into cached group entries,
            since they are re-created when an object is opened.

        The benchmark I used to measure these results created several thousand
        small (2K) datasets in a file and wrote out the data for them.  This is
        Elena's "regular.c" benchmark.

        These changes resulted in approximately ~4.3x speedup of the
        development branch when compared to the previous code in the
        development branch and ~1.4x speedup compared to the release
        branch.

        Additionally, these changes reduce the total memory used (code and
        data) by the development branch by ~800KB, bringing the development
        branch back into the same ballpark as the release branch.

        I'll send out a more detailed description of the benchmark results
        as a followup note.

    New internal API routines:
        Added "reference counted strings" API for tracking strings that get
            used by multiple owners without duplicating the strings.
        Added "ternary search tree" API for text->object mappings.

Platforms tested:
    Tested h5committest {arabica (fortran), eirene (fortran, C++)
	modi4 (parallel, fortran)}
    Other platforms/configurations tested?
        FreeBSD 4.7 (sleipnir) serial & parallel
        Solaris 2.6 (baldric) serial

1 files changed, 62 insertions, 44 deletions

diff --git a/src/H5Dcontig.c b/src/H5Dcontig.c
index fce1ba9..a63a9ce 100644
--- a/src/H5Dcontig.c
+++ b/src/H5Dcontig.c
@@ -36,8 +36,11 @@ static int		interface_initialize_g = 0;
 /* Declare a PQ free list to manage the sieve buffer information */
 H5FL_BLK_DEFINE(sieve_buf);
 
-/* Extern the free list to manage blocks of type conversion data */
-H5FL_BLK_EXTERN(type_conv);
+/* Declare the free list to manage blocks of non-zero fill-value data */
+H5FL_BLK_DEFINE_STATIC(non_zero_fill);
+
+/* Declare the free list to manage blocks of zero fill-value data */
+H5FL_BLK_DEFINE_STATIC(zero_fill);
 
 
 /*-------------------------------------------------------------------------
@@ -56,11 +59,10 @@ H5FL_BLK_EXTERN(type_conv);
  */
 herr_t
 H5F_contig_fill(H5F_t *f, hid_t dxpl_id, struct H5O_layout_t *layout,
-    struct H5P_genplist_t *dc_plist, const struct H5S_t *space,
-    size_t elmt_size)
+    struct H5P_genplist_t *dc_plist, const struct H5O_efl_t *efl,
+    const struct H5S_t *space,
+    const struct H5O_fill_t *fill, size_t elmt_size)
 {
-    H5O_fill_t  fill;           /* Fill value information */
-    H5O_efl_t   efl;            /* External File List info */
     hssize_t    snpoints;       /* Number of points in space (for error checking) */
     size_t      npoints;        /* Number of points in space */
     size_t      ptsperbuf;      /* Maximum # of points which fit in the buffer */
@@ -77,6 +79,7 @@ H5F_contig_fill(H5F_t *f, hid_t dxpl_id, struct H5O_layout_t *layout,
     unsigned    blocks_written=0; /* Flag to indicate that chunk was actually written */
     unsigned    using_mpi=0;    /* Flag to indicate that the file is being accessed with an MPI-capable file driver */
 #endif /* H5_HAVE_PARALLEL */
+    int         non_zero_fill_f=(-1);   /* Indicate that a non-zero fill-value was used */
     herr_t	ret_value=SUCCEED;	/* Return value */
     
     FUNC_ENTER_NOAPI(H5F_contig_fill, FAIL);
@@ -91,12 +94,6 @@ H5F_contig_fill(H5F_t *f, hid_t dxpl_id, struct H5O_layout_t *layout,
     assert(space);
     assert(elmt_size>0);
 
-    /* Get necessary properties from dataset creation property list */
-    if(H5P_get(dc_plist, H5D_CRT_FILL_VALUE_NAME, &fill) < 0)
-        HGOTO_ERROR(H5E_STORAGE, H5E_CANTGET, FAIL, "can't get fill value");
-    if(H5P_get(dc_plist, H5D_CRT_EXT_FILE_LIST_NAME, &efl) < 0)
-        HGOTO_ERROR(H5E_DATASET, H5E_CANTGET, FAIL, "can't retrieve external file list");
-
 #ifdef H5_HAVE_PARALLEL
     /* Retrieve up MPI parameters */
     if(IS_H5FD_MPIO(f)) {
@@ -136,46 +133,62 @@ H5F_contig_fill(H5F_t *f, hid_t dxpl_id, struct H5O_layout_t *layout,
     assert(snpoints>=0);
     H5_ASSIGN_OVERFLOW(npoints,snpoints,hssize_t,size_t);
 
-    /* Don't write default fill-values to external files */
-    if(efl.nused>0 && !fill.buf)
-        HGOTO_DONE(SUCCEED);
-
-    /* If fill value is library default, use the element size */
-    if(!fill.buf)
-        fill.size=elmt_size;
+    /* If fill value is not library default, use it to set the element size */
+    if(fill->buf)
+        elmt_size=fill->size;
 
     /*
      * Fill the entire current extent with the fill value.  We can do
      * this quite efficiently by making sure we copy the fill value
      * in relatively large pieces.
      */
-     ptsperbuf = MAX(1, bufsize/fill.size);
-     bufsize = ptsperbuf*fill.size;
-
-     /* Allocate temporary buffer */
-     if ((buf=H5FL_BLK_ALLOC(type_conv,bufsize,0))==NULL)
-         HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, FAIL, "memory allocation failed for fill buffer");
-
-     /* Fill the buffer with the user's fill value */
-     if(fill.buf)
-        H5V_array_fill(buf, fill.buf, fill.size, ptsperbuf);
-     else /* Fill the buffer with the default fill value */
-        HDmemset(buf,0,bufsize);
+     ptsperbuf = MAX(1, bufsize/elmt_size);
+     bufsize = ptsperbuf*elmt_size;
+
+    /* Fill the buffer with the user's fill value */
+    if(fill->buf) {
+        /* Allocate temporary buffer */
+        if ((buf=H5FL_BLK_MALLOC(non_zero_fill,bufsize))==NULL)
+            HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, FAIL, "memory allocation failed for fill buffer");
+
+        H5V_array_fill(buf, fill->buf, elmt_size, ptsperbuf);
+
+        /* Indicate that a non-zero fill buffer was used */
+        non_zero_fill_f=1;
+    } /* end if */
+    else {      /* Fill the buffer with the default fill value */
+        htri_t buf_avail;
+
+        /* Check if there is an already zeroed out buffer available */
+        buf_avail=H5FL_BLK_AVAIL(zero_fill,bufsize);
+        assert(buf_avail!=FAIL);
+
+        /* Allocate temporary buffer (zeroing it if no buffer is available) */
+        if(!buf_avail)
+            buf=H5FL_BLK_CALLOC(zero_fill,bufsize);
+        else
+            buf=H5FL_BLK_MALLOC(zero_fill,bufsize);
+        if(buf==NULL)
+            HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, FAIL, "memory allocation failed for fill buffer");
+
+        /* Indicate that a zero fill buffer was used */
+        non_zero_fill_f=0;
+    } /* end else */
      
-     /* Start at the beginning of the dataset */
-     addr = 0;
+    /* Start at the beginning of the dataset */
+    addr = 0;
 
-     /* Loop through writing the fill value to the dataset */
-     while (npoints>0) {
-          size = MIN(ptsperbuf, npoints) * fill.size;
+    /* Loop through writing the fill value to the dataset */
+    while (npoints>0) {
+          size = MIN(ptsperbuf, npoints) * elmt_size;
 
 #ifdef H5_HAVE_PARALLEL
             /* Check if this file is accessed with an MPI-capable file driver */
             if(using_mpi) {
                 /* Round-robin write the chunks out from only one process */
                 if(mpi_round==mpi_rank) {
-                    if (H5F_seq_write(f, dxpl_id, layout, dc_plist, space,
-                            fill.size, size, addr, buf)<0)
+                    if (H5F_seq_write(f, dxpl_id, layout, dc_plist, efl, space,
+                            elmt_size, size, addr, buf)<0)
                         HGOTO_ERROR(H5E_DATASET, H5E_CANTINIT, FAIL, "unable to write fill value to dataset");
                 } /* end if */
                 mpi_round=(++mpi_round)%mpi_size;
@@ -185,8 +198,8 @@ H5F_contig_fill(H5F_t *f, hid_t dxpl_id, struct H5O_layout_t *layout,
             } /* end if */
             else {
 #endif /* H5_HAVE_PARALLEL */
-                if (H5F_seq_write(f, dxpl_id, layout, dc_plist, space,
-                        fill.size, size, addr, buf)<0)
+                if (H5F_seq_write(f, dxpl_id, layout, dc_plist, efl, space,
+                        elmt_size, size, addr, buf)<0)
                     HGOTO_ERROR(H5E_DATASET, H5E_CANTINIT, FAIL, "unable to write fill value to dataset");
 #ifdef H5_HAVE_PARALLEL
             } /* end else */
@@ -211,8 +224,13 @@ H5F_contig_fill(H5F_t *f, hid_t dxpl_id, struct H5O_layout_t *layout,
 
 done:
     /* Free the buffer for fill values */
-    if (buf)
-        H5FL_BLK_FREE(type_conv,buf);
+    if (buf) {
+        assert(non_zero_fill_f>=0);
+        if(non_zero_fill_f)
+            H5FL_BLK_FREE(non_zero_fill,buf);
+        else
+            H5FL_BLK_FREE(zero_fill,buf);
+    } /* end if */
 
     FUNC_LEAVE(ret_value);
 }
@@ -661,7 +679,7 @@ H5F_contig_readv(H5F_t *f, hsize_t _max_data, H5FD_mem_t type, haddr_t _addr,
                 } /* end if */
                 else {
                     /* Allocate room for the data sieve buffer */
-                    if (NULL==(f->shared->sieve_buf=H5FL_BLK_ALLOC(sieve_buf,f->shared->sieve_buf_size,0)))
+                    if (NULL==(f->shared->sieve_buf=H5FL_BLK_MALLOC(sieve_buf,f->shared->sieve_buf_size)))
                         HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, FAIL, "memory allocation failed");
 
                     /* Determine the new sieve buffer size & location */
@@ -1128,7 +1146,7 @@ H5F_contig_writev(H5F_t *f, hsize_t _max_data, H5FD_mem_t type, haddr_t _addr,
                 } /* end if */
                 else {
                     /* Allocate room for the data sieve buffer */
-                    if (NULL==(f->shared->sieve_buf=H5FL_BLK_ALLOC(sieve_buf,f->shared->sieve_buf_size,0)))
+                    if (NULL==(f->shared->sieve_buf=H5FL_BLK_MALLOC(sieve_buf,f->shared->sieve_buf_size)))
                         HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, FAIL, "memory allocation failed");
 
                     /* Determine the new sieve buffer size & location */