[svn-r6252] Purpose:

    Lots of performance improvements & a couple new internal API interfaces.

Description:
    Performance Improvements:
        - Cached file offset & length sizes in shared file struct, to avoid
            constantly looking them up in the FCPL.
        - Generic property improvements:
            - Added "revision" number to generic property classes to speed
                up comparisons.
            - Changed method of storing properties from using a hash-table
                to the TBBT routines in the library.
            - Share the propery names between classes and the lists derived
                from them.
            - Removed redundant 'def_value' buffer from each property.
            - Switching code to use a "copy on write" strategy for
                properties in each list, where the properties in each list
                are shared with the properties in the class, until a
                property's value is changed in a list.
        - Fixed error in layout code which was allocating too many buffers.
        - Redefined public macros of the form (H5open()/H5check, <variable>)
            internally to only be (<variable>), avoiding innumerable useless
            calls to H5open() and H5check_version().
        - Reuse already zeroed buffers in H5F_contig_fill instead of
            constantly re-zeroing them.
        - Don't write fill values if writing entire dataset.
        - Use gettimeofday() system call instead of time() system when
            checking the modification time of a dataset.
        - Added reference counted string API and use it for tracking the
            names of objects opening in a file (for the ID->name code).
        - Removed redundant H5P_get() calls in B-tree routines.
        - Redefine H5T datatype macros internally to the library, to avoid
            calling H5check redundantly.
        - Keep dataspace information for dataset locally instead of reading
            from disk each time.  Added new module to track open objects
            in a file, to allow this (which will be useful eventually for
            some FPH5 metadata caching issues).
        - Remove H5AC_find macro which was inlining metadata cache lookups,
            and call function instead.
        - Remove redundant memset() calls from H5G_namei() routine.
        - Remove redundant checking of object type when locating objects
            in metadata cache and rely on the address only.
        - Create default dataset object to use when default dataset creation
            property list is used to create datasets, bypassing querying
            for all the property list values.
        - Use default I/O vector size when performing raw data with the
            default dataset transfer property list, instead of querying for
            I/O vector size.
        - Remove H5P_DEFAULT internally to the library, replacing it with
            more specific default property list based on the type of
            property list needed.
        - Remove redundant memset() calls in object header message (H5O*)
            routines.
        - Remove redunant memset() calls in data I/O routines.
        - Split free-list allocation routines into malloc() and calloc()-
            like routines, instead of one combined routine.
        - Remove lots of indirection in H5O*() routines.
        - Simplify metadata cache entry comparison routine (used when
            flushing entire cache out).
        - Only enable metadata cache statistics when H5AC_DEBUG is turned
            on, instead of always tracking them.
        - Simplify address comparison macro (H5F_addr_eq).
        - Remove redundant metadata cache entry protections during dataset
            creation by protecting the object header once and making all
            the modifications necessary for the dataset creation before
            unprotecting it.
        - Reduce # of "number of element in extent" computations performed
            by computing and storing the value during dataspace creation.
        - Simplify checking for group location's file information, when file
            has not been involving in file-mounting operations.
        - Use binary encoding for modification time, instead of ASCII.
        - Hoist H5HL_peek calls (to get information in a local heap)
            out of loops in many group routine.
        - Use static variable for iterators of selections, instead of
            dynamically allocation them each time.
        - Lookup & insert new entries in one step, avoiding traversing
            group's B-tree twice.
        - Fixed memory leak in H5Gget_objname_idx() routine (tangential to
            performance improvements, but fixed along the way).
        - Use free-list for reference counted strings.
        - Don't bother copying object names into cached group entries,
            since they are re-created when an object is opened.

        The benchmark I used to measure these results created several thousand
        small (2K) datasets in a file and wrote out the data for them.  This is
        Elena's "regular.c" benchmark.

        These changes resulted in approximately ~4.3x speedup of the
        development branch when compared to the previous code in the
        development branch and ~1.4x speedup compared to the release
        branch.

        Additionally, these changes reduce the total memory used (code and
        data) by the development branch by ~800KB, bringing the development
        branch back into the same ballpark as the release branch.

        I'll send out a more detailed description of the benchmark results
        as a followup note.

    New internal API routines:
        Added "reference counted strings" API for tracking strings that get
            used by multiple owners without duplicating the strings.
        Added "ternary search tree" API for text->object mappings.

Platforms tested:
    Tested h5committest {arabica (fortran), eirene (fortran, C++)
	modi4 (parallel, fortran)}
    Other platforms/configurations tested?
        FreeBSD 4.7 (sleipnir) serial & parallel
        Solaris 2.6 (baldric) serial

1 files changed, 35 insertions, 11 deletions

diff --git a/src/H5Fprivate.h b/src/H5Fprivate.h
index 42f174c..395b5e2 100644
--- a/src/H5Fprivate.h
+++ b/src/H5Fprivate.h
@@ -152,8 +152,10 @@ typedef struct H5F_t H5F_t;
 				 (X)+(haddr_t)(Z)<(X))
 #define H5F_addr_hash(X,M)	((unsigned)((X)%(M)))
 #define H5F_addr_defined(X)	(X!=HADDR_UNDEF)
+/* The H5F_addr_eq() macro guarantees that Y is not HADDR_UNDEF by making
+ * certain that X is not HADDR_UNDEF and then checking that X equals Y
+ */
 #define H5F_addr_eq(X,Y)	((X)!=HADDR_UNDEF &&			      \
-				 (Y)!=HADDR_UNDEF &&			      \
 				 (X)==(Y))
 #define H5F_addr_ne(X,Y)	(!H5F_addr_eq((X),(Y)))
 #define H5F_addr_lt(X,Y) 	((X)!=HADDR_UNDEF &&			      \
@@ -174,12 +176,22 @@ typedef struct H5F_t H5F_t;
 #define H5F_addr_overlap(O1,L1,O2,L2) ((O1<O2 && (O1+L1)>O2) ||               \
                                  (O1>=O2 && O1<(O2+L2)))
 
+/* If the module using this macro is allowed access to the private variables, access them directly */
+#ifdef H5F_PACKAGE
 /* size of size_t and off_t as they exist on disk */
+#define H5F_SIZEOF_ADDR(F)      ((F)->shared->sizeof_addr)
+#define H5F_SIZEOF_SIZE(F)      ((F)->shared->sizeof_size)
+/* Size of symbol table leafs */
+#define H5F_SYM_LEAF_K(F)       ((F)->shared->sym_leaf_k)
+/* B-tree key value size */
+#define H5F_KVALUE(F,T)         ((F)->shared->btree_k[(T)->id])
+#else /* H5F_PACKAGE */
 #define H5F_SIZEOF_ADDR(F)      (H5F_sizeof_addr(F))
 #define H5F_SIZEOF_SIZE(F)      (H5F_sizeof_size(F))
+#define H5F_SYM_LEAF_K(F)       (H5F_sym_leaf_k(F))
+#define H5F_KVALUE(F,T)         (H5F_Kvalue(F,T))
+#endif /* H5F_PACKAGE */
 
-H5_DLL size_t H5F_sizeof_addr(const H5F_t *f);
-H5_DLL size_t H5F_sizeof_size(const H5F_t *f);
 
 /* Macros to encode/decode offset/length's for storing in the file */
 #ifdef NOT_YET
@@ -344,6 +356,7 @@ struct H5O_pline_t;
 struct H5O_fill_t;
 struct H5P_genplist_t;
 struct H5S_t;
+struct H5B_class_t;
 
 /* Private functions, not part of the publicly documented API */
 H5_DLL herr_t H5F_init(void);
@@ -356,6 +369,17 @@ H5_DLL herr_t H5F_get_obj_ids(H5F_t *f, unsigned types, hid_t *obj_id_list);
 H5_DLL herr_t H5F_get_vfd_handle(H5F_t *file, hid_t fapl, void** file_handle);
 H5_DLL haddr_t H5F_get_base_addr(const H5F_t *f);
 
+/* Functions than check file mounting information */
+H5_DLL htri_t H5F_is_mount(const H5F_t *file);
+H5_DLL htri_t H5F_has_mount(const H5F_t *file);
+
+/* Functions than retrieve values set from the FCPL */
+H5_DLL size_t H5F_sizeof_addr(const H5F_t *f);
+H5_DLL size_t H5F_sizeof_size(const H5F_t *f);
+H5_DLL unsigned H5F_sym_leaf_k(const H5F_t *f);
+H5_DLL int H5F_Kvalue(const H5F_t *f, const struct H5B_class_t *type);
+
+
 /* Functions that operate on array storage */
 H5_DLL herr_t H5F_arr_read (H5F_t *f, hid_t dxpl_id,
                             const struct H5O_layout_t *layout,
@@ -381,24 +405,24 @@ H5_DLL herr_t H5F_block_write(H5F_t *f, H5FD_mem_t type, haddr_t addr,
 /* Functions that operate on byte sequences */
 H5_DLL herr_t H5F_seq_read(H5F_t *f, hid_t dxpl_id,
         const struct H5O_layout_t *layout,
-        struct H5P_genplist_t *dc_plist,
+        struct H5P_genplist_t *dc_plist, const struct H5O_efl_t *efl,
         const struct H5S_t *file_space, size_t elmt_size, size_t seq_len,
         hsize_t file_offset, void *_buf/*out*/);
 H5_DLL herr_t H5F_seq_write (H5F_t *f, hid_t dxpl_id,
         struct H5O_layout_t *layout,
-        struct H5P_genplist_t *dc_plist,
+        struct H5P_genplist_t *dc_plist, const struct H5O_efl_t *efl,
         const struct H5S_t *file_space, size_t elmt_size, size_t seq_len,
         hsize_t file_offset, const void *_buf);
 
 /* Functions that operate on vectors of byte sequences */
 H5_DLL herr_t H5F_seq_readv(H5F_t *f, hid_t dxpl_id,
         const struct H5O_layout_t *layout,
-        struct H5P_genplist_t *dc_plist,
+        struct H5P_genplist_t *dc_plist, const struct H5O_efl_t *efl,
         const struct H5S_t *file_space, size_t elmt_size, size_t nseq,
         size_t seq_len[], hsize_t file_offset[], void *_buf/*out*/);
 H5_DLL herr_t H5F_seq_writev(H5F_t *f, hid_t dxpl_id,
         struct H5O_layout_t *layout,
-        struct H5P_genplist_t *dc_plist,
+        struct H5P_genplist_t *dc_plist, const struct H5O_efl_t *efl,
         const struct H5S_t *file_space, size_t elmt_size, size_t nseq,
         size_t seq_len[], hsize_t file_offset[], const void *_buf);
 
@@ -406,15 +430,15 @@ H5_DLL herr_t H5F_seq_writev(H5F_t *f, hid_t dxpl_id,
 /* Functions that operate on contiguous storage */
 H5_DLL herr_t H5F_contig_fill(H5F_t *f, hid_t dxpl_id,
         struct H5O_layout_t *layout, struct H5P_genplist_t *dc_plist,
-        const struct H5S_t *space, size_t elmt_size);
+        const struct H5O_efl_t *efl, const struct H5S_t *space,
+        const struct H5O_fill_t *fill, size_t elmt_size);
 
 /* Functions that operate on indexed storage */
 H5_DLL herr_t H5F_istore_create(H5F_t *f,
 				 struct H5O_layout_t *layout/*in,out*/);
 H5_DLL herr_t H5F_istore_allocate (H5F_t *f, hid_t dxpl_id,
-                                const struct H5O_layout_t *layout,
-                                const hsize_t *space_dim,
-                                struct H5P_genplist_t *dc_plist);
+    const struct H5O_layout_t *layout, const hsize_t *space_dim,
+    struct H5P_genplist_t *dc_plist, hbool_t full_overwrite);
 H5_DLL hsize_t H5F_istore_allocated(H5F_t *f, unsigned ndims, haddr_t addr);
 H5_DLL herr_t H5F_istore_dump_btree(H5F_t *f, FILE *stream, unsigned ndims,
         haddr_t addr);