[svn-r6252] Purpose:

    Lots of performance improvements & a couple new internal API interfaces.

Description:
    Performance Improvements:
        - Cached file offset & length sizes in shared file struct, to avoid
            constantly looking them up in the FCPL.
        - Generic property improvements:
            - Added "revision" number to generic property classes to speed
                up comparisons.
            - Changed method of storing properties from using a hash-table
                to the TBBT routines in the library.
            - Share the propery names between classes and the lists derived
                from them.
            - Removed redundant 'def_value' buffer from each property.
            - Switching code to use a "copy on write" strategy for
                properties in each list, where the properties in each list
                are shared with the properties in the class, until a
                property's value is changed in a list.
        - Fixed error in layout code which was allocating too many buffers.
        - Redefined public macros of the form (H5open()/H5check, <variable>)
            internally to only be (<variable>), avoiding innumerable useless
            calls to H5open() and H5check_version().
        - Reuse already zeroed buffers in H5F_contig_fill instead of
            constantly re-zeroing them.
        - Don't write fill values if writing entire dataset.
        - Use gettimeofday() system call instead of time() system when
            checking the modification time of a dataset.
        - Added reference counted string API and use it for tracking the
            names of objects opening in a file (for the ID->name code).
        - Removed redundant H5P_get() calls in B-tree routines.
        - Redefine H5T datatype macros internally to the library, to avoid
            calling H5check redundantly.
        - Keep dataspace information for dataset locally instead of reading
            from disk each time.  Added new module to track open objects
            in a file, to allow this (which will be useful eventually for
            some FPH5 metadata caching issues).
        - Remove H5AC_find macro which was inlining metadata cache lookups,
            and call function instead.
        - Remove redundant memset() calls from H5G_namei() routine.
        - Remove redundant checking of object type when locating objects
            in metadata cache and rely on the address only.
        - Create default dataset object to use when default dataset creation
            property list is used to create datasets, bypassing querying
            for all the property list values.
        - Use default I/O vector size when performing raw data with the
            default dataset transfer property list, instead of querying for
            I/O vector size.
        - Remove H5P_DEFAULT internally to the library, replacing it with
            more specific default property list based on the type of
            property list needed.
        - Remove redundant memset() calls in object header message (H5O*)
            routines.
        - Remove redunant memset() calls in data I/O routines.
        - Split free-list allocation routines into malloc() and calloc()-
            like routines, instead of one combined routine.
        - Remove lots of indirection in H5O*() routines.
        - Simplify metadata cache entry comparison routine (used when
            flushing entire cache out).
        - Only enable metadata cache statistics when H5AC_DEBUG is turned
            on, instead of always tracking them.
        - Simplify address comparison macro (H5F_addr_eq).
        - Remove redundant metadata cache entry protections during dataset
            creation by protecting the object header once and making all
            the modifications necessary for the dataset creation before
            unprotecting it.
        - Reduce # of "number of element in extent" computations performed
            by computing and storing the value during dataspace creation.
        - Simplify checking for group location's file information, when file
            has not been involving in file-mounting operations.
        - Use binary encoding for modification time, instead of ASCII.
        - Hoist H5HL_peek calls (to get information in a local heap)
            out of loops in many group routine.
        - Use static variable for iterators of selections, instead of
            dynamically allocation them each time.
        - Lookup & insert new entries in one step, avoiding traversing
            group's B-tree twice.
        - Fixed memory leak in H5Gget_objname_idx() routine (tangential to
            performance improvements, but fixed along the way).
        - Use free-list for reference counted strings.
        - Don't bother copying object names into cached group entries,
            since they are re-created when an object is opened.

        The benchmark I used to measure these results created several thousand
        small (2K) datasets in a file and wrote out the data for them.  This is
        Elena's "regular.c" benchmark.

        These changes resulted in approximately ~4.3x speedup of the
        development branch when compared to the previous code in the
        development branch and ~1.4x speedup compared to the release
        branch.

        Additionally, these changes reduce the total memory used (code and
        data) by the development branch by ~800KB, bringing the development
        branch back into the same ballpark as the release branch.

        I'll send out a more detailed description of the benchmark results
        as a followup note.

    New internal API routines:
        Added "reference counted strings" API for tracking strings that get
            used by multiple owners without duplicating the strings.
        Added "ternary search tree" API for text->object mappings.

Platforms tested:
    Tested h5committest {arabica (fortran), eirene (fortran, C++)
	modi4 (parallel, fortran)}
    Other platforms/configurations tested?
        FreeBSD 4.7 (sleipnir) serial & parallel
        Solaris 2.6 (baldric) serial

1 files changed, 390 insertions, 0 deletions

diff --git a/src/H5RS.c b/src/H5RS.c
new file mode 100644
index 0000000..f6f18ac
--- /dev/null
+++ b/src/H5RS.c
@@ -0,0 +1,390 @@
+/****************************************************************************
+ * NCSA HDF								    *
+ * Software Development Group						    *
+ * National Center for Supercomputing Applications			    *
+ * University of Illinois at Urbana-Champaign				    *
+ * 605 E. Springfield, Champaign IL 61820				    *
+ *									    *
+ * For conditions of distribution and use, see the accompanying		    *
+ * hdf/COPYING file.							    *
+ *									    *
+ ****************************************************************************/
+
+/*
+ * Reference counted string algorithms.
+ *
+ * These are used for various internal strings which get copied multiple times.
+ *
+ */
+
+#include "H5Eprivate.h"		/* Error handling		  	*/
+#include "H5FLprivate.h"	/* Free lists                           */
+#include "H5RSprivate.h"        /* Reference-counted strings            */
+
+#define PABLO_MASK	H5RS_mask
+
+/* Interface initialization */
+static int		interface_initialize_g = 0;
+#define INTERFACE_INIT	NULL
+
+/* Declare a free list to manage the H5RS_str_t struct */
+H5FL_DEFINE_STATIC(H5RS_str_t);
+
+/* Declare the PQ free list for the wrapped strings */
+H5FL_BLK_DEFINE(str_buf);
+
+
+/*--------------------------------------------------------------------------
+ NAME
+    H5RS_xstrdup
+ PURPOSE
+    Duplicate the string being reference counted
+ USAGE
+    char *H5RS_xstrdup(s)
+        const char *s;          IN: String to duplicate
+
+ RETURNS
+    Returns a pointer to a new string on success, NULL on failure.
+ DESCRIPTION
+    Duplicate a string buffer being reference counted.  Use this instead of
+    [H5MM_][x]strdup, in order to use the free-list memory routines.
+ GLOBAL VARIABLES
+ COMMENTS, BUGS, ASSUMPTIONS
+ EXAMPLES
+ REVISION LOG
+--------------------------------------------------------------------------*/
+static char *
+H5RS_xstrdup(const char *s)
+{
+    char *ret_value;   /* Return value */
+
+    FUNC_ENTER_NOINIT(H5RS_xstrdup);
+
+    if (s) {
+        ret_value = H5FL_BLK_MALLOC(str_buf,HDstrlen(s) + 1);
+        assert (ret_value);
+        HDstrcpy(ret_value, s);
+    } /* end if */
+    else
+        ret_value=NULL;
+
+    FUNC_LEAVE(ret_value);
+} /* end H5RS_xstrdup() */
+
+
+/*--------------------------------------------------------------------------
+ NAME
+    H5RS_create
+ PURPOSE
+    Create a reference counted string
+ USAGE
+    H5RS_str_t *H5RS_create(s)
+        const char *s;          IN: String to initialize ref-counted string with
+
+ RETURNS
+    Returns a pointer to a new ref-counted string on success, NULL on failure.
+ DESCRIPTION
+    Create a reference counted string.  The string passed in is copied into an
+    internal buffer.
+ GLOBAL VARIABLES
+ COMMENTS, BUGS, ASSUMPTIONS
+ EXAMPLES
+ REVISION LOG
+--------------------------------------------------------------------------*/
+H5RS_str_t *
+H5RS_create(const char *s)
+{
+    H5RS_str_t *ret_value=NULL;   /* Return value */
+
+    FUNC_ENTER_NOAPI(H5RS_create,NULL);
+
+    /* Allocate ref-counted string structure */
+    if((ret_value=H5FL_MALLOC(H5RS_str_t))==NULL)
+        HGOTO_ERROR(H5E_RS,H5E_NOSPACE,NULL,"memory allocation failed");
+
+    /* Set the internal fields */
+    ret_value->s=H5RS_xstrdup(s);
+    ret_value->wrapped=0;
+    ret_value->n=1;
+
+done:
+    FUNC_LEAVE(ret_value);
+} /* end H5RS_create() */
+
+
+/*--------------------------------------------------------------------------
+ NAME
+    H5RS_wrap
+ PURPOSE
+    "Wrap" a reference counted string around an existing string
+ USAGE
+    H5RS_str_t *H5RS_wrap(s)
+        const char *s;          IN: String to wrap ref-counted string around
+
+ RETURNS
+    Returns a pointer to a new ref-counted string on success, NULL on failure.
+ DESCRIPTION
+    Wrap a reference counted string around an existing string, which is not
+    duplicated, unless its reference count gets incremented.
+ GLOBAL VARIABLES
+ COMMENTS, BUGS, ASSUMPTIONS
+ EXAMPLES
+ REVISION LOG
+--------------------------------------------------------------------------*/
+H5RS_str_t *
+H5RS_wrap(char *s)
+{
+    H5RS_str_t *ret_value=NULL;   /* Return value */
+
+    FUNC_ENTER_NOAPI(H5RS_wrap,NULL);
+
+    /* Allocate ref-counted string structure */
+    if((ret_value=H5FL_MALLOC(H5RS_str_t))==NULL)
+        HGOTO_ERROR(H5E_RS,H5E_NOSPACE,NULL,"memory allocation failed");
+
+    /* Set the internal fields */
+    ret_value->s=s;
+    ret_value->wrapped=1;
+    ret_value->n=1;
+
+done:
+    FUNC_LEAVE(ret_value);
+} /* end H5RS_wrap() */
+
+
+/*--------------------------------------------------------------------------
+ NAME
+    H5RS_own
+ PURPOSE
+    Transfer ownership of a regular string to  a reference counted string
+ USAGE
+    H5RS_str_t *H5RS_own(s)
+        const char *s;          IN: String to transfer ownership of
+
+ RETURNS
+    Returns a pointer to a new ref-counted string on success, NULL on failure.
+ DESCRIPTION
+    Transfer ownership of a dynamically allocated string to a reference counted
+    string.  The routine which passed in the string should not attempt to free
+    it, the reference counting string routines will do that when the reference
+    count drops to zero.
+ GLOBAL VARIABLES
+ COMMENTS, BUGS, ASSUMPTIONS
+ EXAMPLES
+ REVISION LOG
+--------------------------------------------------------------------------*/
+H5RS_str_t *
+H5RS_own(char *s)
+{
+    H5RS_str_t *ret_value=NULL;   /* Return value */
+
+    FUNC_ENTER_NOAPI(H5RS_own,NULL);
+
+    /* Allocate ref-counted string structure */
+    if((ret_value=H5FL_MALLOC(H5RS_str_t))==NULL)
+        HGOTO_ERROR(H5E_RS,H5E_NOSPACE,NULL,"memory allocation failed");
+
+    /* Set the internal fields */
+    ret_value->s=s;
+    ret_value->wrapped=0;
+    ret_value->n=1;
+
+done:
+    FUNC_LEAVE(ret_value);
+} /* end H5RS_own() */
+
+
+/*--------------------------------------------------------------------------
+ NAME
+    H5RS_decr
+ PURPOSE
+    Decrement the reference count for a ref-counted string
+ USAGE
+    herr_t H5RS_decr(rs)
+        H5RS_str_t *rs;     IN/OUT: Ref-counted string to decrement count of
+
+ RETURNS
+    Non-negative on success/Negative on failure
+ DESCRIPTION
+    Decrement the reference count for a reference counted string.  If the
+    reference count drops to zero, the reference counted string is deleted.
+ GLOBAL VARIABLES
+ COMMENTS, BUGS, ASSUMPTIONS
+ EXAMPLES
+ REVISION LOG
+--------------------------------------------------------------------------*/
+herr_t
+H5RS_decr(H5RS_str_t *rs)
+{
+    herr_t ret_value=SUCCEED;   /* Return value */
+
+    FUNC_ENTER_NOAPI(H5RS_decr,FAIL);
+
+    /* Sanity check */
+    assert(rs);
+    assert(rs->n > 0);
+
+    /* Decrement reference count for string */
+    if((--rs->n)==0) {
+        if(!rs->wrapped)
+            H5FL_BLK_FREE(str_buf,rs->s);
+        H5FL_FREE(H5RS_str_t,rs);
+    } /* end if */
+
+done:
+    FUNC_LEAVE(ret_value);
+} /* end H5RS_decr() */
+
+
+/*--------------------------------------------------------------------------
+ NAME
+    H5RS_incr
+ PURPOSE
+    Increment the reference count for a ref-counted string
+ USAGE
+    herr_t H5RS_incr(rs)
+        H5RS_str_t *rs;     IN/OUT: Ref-counted string to increment count of
+
+ RETURNS
+    Non-negative on success/Negative on failure
+ DESCRIPTION
+    Increment the reference count for a reference counted string.
+ GLOBAL VARIABLES
+ COMMENTS, BUGS, ASSUMPTIONS
+ EXAMPLES
+ REVISION LOG
+--------------------------------------------------------------------------*/
+herr_t
+H5RS_incr(H5RS_str_t *rs)
+{
+    herr_t ret_value=SUCCEED;   /* Return value */
+
+    FUNC_ENTER_NOAPI(H5RS_incr,FAIL);
+
+    /* Sanity check */
+    assert(rs);
+    assert(rs->n > 0);
+
+    /* If the ref-counted string started life as a wrapper around an existing
+     * string, duplicate the string now, so that the wrapped string can go out
+     * scope appropriately.
+     */
+    if(rs->wrapped) {
+        rs->s=H5RS_xstrdup(rs->s);
+        rs->wrapped=0;
+    } /* end if */
+
+    /* Increment reference count for string */
+    rs->n++;
+
+done:
+    FUNC_LEAVE(ret_value);
+} /* end H5RS_incr() */
+
+
+/*--------------------------------------------------------------------------
+ NAME
+    H5RS_dup
+ PURPOSE
+    "Duplicate" a ref-counted string
+ USAGE
+    H5RS_str_t H5RS_incr(rs)
+        H5RS_str_t *rs;     IN/OUT: Ref-counted string to "duplicate"
+
+ RETURNS
+    Returns a pointer to ref-counted string on success, NULL on failure.
+ DESCRIPTION
+    Increment the reference count for the reference counted string and return
+    a pointer to it.
+ GLOBAL VARIABLES
+ COMMENTS, BUGS, ASSUMPTIONS
+ EXAMPLES
+ REVISION LOG
+--------------------------------------------------------------------------*/
+H5RS_str_t *
+H5RS_dup(H5RS_str_t *ret_value)
+{
+    FUNC_ENTER_NOAPI(H5RS_dup,NULL);
+
+    /* Check for valid reference counted string */
+    if(ret_value!=NULL)
+        /* Increment reference count for string */
+        ret_value->n++;
+
+done:
+    FUNC_LEAVE(ret_value);
+} /* end H5RS_dup() */
+
+
+/*--------------------------------------------------------------------------
+ NAME
+    H5RS_cmp
+ PURPOSE
+    Compare two ref-counted strings
+ USAGE
+    int H5RS_cmp(rs1,rs2)
+        const H5RS_str_t *rs1;  IN: First Ref-counted string to compare
+        const H5RS_str_t *rs2;  IN: Second Ref-counted string to compare
+
+ RETURNS
+    Returns positive, negative or 0 for comparison of two r-strings [same as
+    strcmp()]
+ DESCRIPTION
+    Compare two ref-counted strings and return a value indicating their sort
+    order [same as strcmp()]
+ GLOBAL VARIABLES
+ COMMENTS, BUGS, ASSUMPTIONS
+ EXAMPLES
+ REVISION LOG
+--------------------------------------------------------------------------*/
+int
+H5RS_cmp(const H5RS_str_t *rs1, const H5RS_str_t *rs2)
+{
+    /* Can't return invalid value from this function */
+    FUNC_ENTER_NOINIT(H5RS_cmp);
+
+    /* Sanity check */
+    assert(rs1);
+    assert(rs1->s);
+    assert(rs2);
+    assert(rs2->s);
+
+    FUNC_LEAVE(HDstrcmp(rs1->s,rs2->s));
+} /* end H5RS_cmp() */
+
+
+/*--------------------------------------------------------------------------
+ NAME
+    H5RS_len
+ PURPOSE
+    Compute the length of a ref-counted string
+ USAGE
+    ssize_t H5RS_cmp(rs)
+        const H5RS_str_t *rs;  IN: Ref-counted string to compute length of
+
+ RETURNS
+    Returns non-negative value on success, negative value on failure
+ DESCRIPTION
+    Compute the length of a ref-counted string.  [same as strlen()]
+ GLOBAL VARIABLES
+ COMMENTS, BUGS, ASSUMPTIONS
+ EXAMPLES
+ REVISION LOG
+--------------------------------------------------------------------------*/
+ssize_t
+H5RS_len(const H5RS_str_t *rs)
+{
+    ssize_t ret_value;     /* Return value */
+
+    FUNC_ENTER_NOAPI(H5RS_len,FAIL);
+
+    /* Sanity check */
+    assert(rs);
+    assert(rs->s);
+
+    ret_value=HDstrlen(rs->s);
+
+done:
+    FUNC_LEAVE(ret_value);
+} /* end H5RS_len() */
+