summaryrefslogtreecommitdiffstats
path: root/src/H5Ocomp.c
diff options
context:
space:
mode:
authorQuincey Koziol <koziol@hdfgroup.org>2003-01-09 17:20:03 (GMT)
committerQuincey Koziol <koziol@hdfgroup.org>2003-01-09 17:20:03 (GMT)
commit9a433b99a56dc575f1c0b11f95b744de61859dbb (patch)
treed8c766537cb9adc364c902bd45477d97f67a4a9f /src/H5Ocomp.c
parent7fd449cb7987772a2881a5ced2ae7ad5231f1fa3 (diff)
downloadhdf5-9a433b99a56dc575f1c0b11f95b744de61859dbb.zip
hdf5-9a433b99a56dc575f1c0b11f95b744de61859dbb.tar.gz
hdf5-9a433b99a56dc575f1c0b11f95b744de61859dbb.tar.bz2
[svn-r6252] Purpose:
Lots of performance improvements & a couple new internal API interfaces. Description: Performance Improvements: - Cached file offset & length sizes in shared file struct, to avoid constantly looking them up in the FCPL. - Generic property improvements: - Added "revision" number to generic property classes to speed up comparisons. - Changed method of storing properties from using a hash-table to the TBBT routines in the library. - Share the propery names between classes and the lists derived from them. - Removed redundant 'def_value' buffer from each property. - Switching code to use a "copy on write" strategy for properties in each list, where the properties in each list are shared with the properties in the class, until a property's value is changed in a list. - Fixed error in layout code which was allocating too many buffers. - Redefined public macros of the form (H5open()/H5check, <variable>) internally to only be (<variable>), avoiding innumerable useless calls to H5open() and H5check_version(). - Reuse already zeroed buffers in H5F_contig_fill instead of constantly re-zeroing them. - Don't write fill values if writing entire dataset. - Use gettimeofday() system call instead of time() system when checking the modification time of a dataset. - Added reference counted string API and use it for tracking the names of objects opening in a file (for the ID->name code). - Removed redundant H5P_get() calls in B-tree routines. - Redefine H5T datatype macros internally to the library, to avoid calling H5check redundantly. - Keep dataspace information for dataset locally instead of reading from disk each time. Added new module to track open objects in a file, to allow this (which will be useful eventually for some FPH5 metadata caching issues). - Remove H5AC_find macro which was inlining metadata cache lookups, and call function instead. - Remove redundant memset() calls from H5G_namei() routine. - Remove redundant checking of object type when locating objects in metadata cache and rely on the address only. - Create default dataset object to use when default dataset creation property list is used to create datasets, bypassing querying for all the property list values. - Use default I/O vector size when performing raw data with the default dataset transfer property list, instead of querying for I/O vector size. - Remove H5P_DEFAULT internally to the library, replacing it with more specific default property list based on the type of property list needed. - Remove redundant memset() calls in object header message (H5O*) routines. - Remove redunant memset() calls in data I/O routines. - Split free-list allocation routines into malloc() and calloc()- like routines, instead of one combined routine. - Remove lots of indirection in H5O*() routines. - Simplify metadata cache entry comparison routine (used when flushing entire cache out). - Only enable metadata cache statistics when H5AC_DEBUG is turned on, instead of always tracking them. - Simplify address comparison macro (H5F_addr_eq). - Remove redundant metadata cache entry protections during dataset creation by protecting the object header once and making all the modifications necessary for the dataset creation before unprotecting it. - Reduce # of "number of element in extent" computations performed by computing and storing the value during dataspace creation. - Simplify checking for group location's file information, when file has not been involving in file-mounting operations. - Use binary encoding for modification time, instead of ASCII. - Hoist H5HL_peek calls (to get information in a local heap) out of loops in many group routine. - Use static variable for iterators of selections, instead of dynamically allocation them each time. - Lookup & insert new entries in one step, avoiding traversing group's B-tree twice. - Fixed memory leak in H5Gget_objname_idx() routine (tangential to performance improvements, but fixed along the way). - Use free-list for reference counted strings. - Don't bother copying object names into cached group entries, since they are re-created when an object is opened. The benchmark I used to measure these results created several thousand small (2K) datasets in a file and wrote out the data for them. This is Elena's "regular.c" benchmark. These changes resulted in approximately ~4.3x speedup of the development branch when compared to the previous code in the development branch and ~1.4x speedup compared to the release branch. Additionally, these changes reduce the total memory used (code and data) by the development branch by ~800KB, bringing the development branch back into the same ballpark as the release branch. I'll send out a more detailed description of the benchmark results as a followup note. New internal API routines: Added "reference counted strings" API for tracking strings that get used by multiple owners without duplicating the strings. Added "ternary search tree" API for text->object mappings. Platforms tested: Tested h5committest {arabica (fortran), eirene (fortran, C++) modi4 (parallel, fortran)} Other platforms/configurations tested? FreeBSD 4.7 (sleipnir) serial & parallel Solaris 2.6 (baldric) serial
Diffstat (limited to 'src/H5Ocomp.c')
-rw-r--r--src/H5Ocomp.c492
1 files changed, 0 insertions, 492 deletions
diff --git a/src/H5Ocomp.c b/src/H5Ocomp.c
deleted file mode 100644
index 8899c34..0000000
--- a/src/H5Ocomp.c
+++ /dev/null
@@ -1,492 +0,0 @@
-/*
- * Copyright (C) 1998-2001 NCSA
- * All rights reserved.
- *
- * Programmer: Robb Matzke <matzke@llnl.gov>
- * Wednesday, April 15, 1998
- *
- * Purpose: Data filter pipeline message.
- */
-#include "H5private.h"
-#include "H5Eprivate.h"
-#include "H5FLprivate.h" /*Free Lists */
-#include "H5MMprivate.h"
-#include "H5Oprivate.h"
-
-/* Interface initialization */
-#define PABLO_MASK H5O_pline_mask
-static int interface_initialize_g = 0;
-#define INTERFACE_INIT NULL
-
-#define H5O_PLINE_VERSION 1
-
-static herr_t H5O_pline_encode (H5F_t *f, uint8_t *p, const void *mesg);
-static void *H5O_pline_decode (H5F_t *f, const uint8_t *p, H5O_shared_t *sh);
-static void *H5O_pline_copy (const void *_mesg, void *_dest);
-static size_t H5O_pline_size (H5F_t *f, const void *_mesg);
-static herr_t H5O_pline_reset (void *_mesg);
-static herr_t H5O_pline_free (void *_mesg);
-static herr_t H5O_pline_debug (H5F_t *f, const void *_mesg,
- FILE * stream, int indent, int fwidth);
-
-/* This message derives from H5O */
-const H5O_class_t H5O_PLINE[1] = {{
- H5O_PLINE_ID, /* message id number */
- "filter pipeline", /* message name for debugging */
- sizeof(H5O_pline_t), /* native message size */
- H5O_pline_decode, /* decode message */
- H5O_pline_encode, /* encode message */
- H5O_pline_copy, /* copy the native value */
- H5O_pline_size, /* size of raw message */
- H5O_pline_reset, /* reset method */
- H5O_pline_free, /* free method */
- NULL, /* get share method */
- NULL, /* set share method */
- H5O_pline_debug, /* debug the message */
-}};
-
-
-/* Declare a free list to manage the H5O_pline_t struct */
-H5FL_DEFINE(H5O_pline_t);
-
-
-/*-------------------------------------------------------------------------
- * Function: H5O_pline_decode
- *
- * Purpose: Decodes a filter pipeline message.
- *
- * Return: Success: Ptr to the native message.
- *
- * Failure: NULL
- *
- * Programmer: Robb Matzke
- * Wednesday, April 15, 1998
- *
- * Modifications:
- *
- *-------------------------------------------------------------------------
- */
-static void *
-H5O_pline_decode(H5F_t UNUSED *f, const uint8_t *p,
- H5O_shared_t UNUSED *sh)
-{
- H5O_pline_t *pline = NULL;
- void *ret_value;
- unsigned version;
- size_t i, j, n, name_length;
-
- FUNC_ENTER_NOAPI(H5O_pline_decode, NULL);
-
- /* check args */
- assert(p);
-
- /* Decode */
- if (NULL==(pline = H5FL_ALLOC(H5O_pline_t,1)))
- HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed");
- version = *p++;
- if (version!=H5O_PLINE_VERSION)
- HGOTO_ERROR(H5E_PLINE, H5E_CANTLOAD, NULL, "bad version number for filter pipeline message");
- pline->nfilters = *p++;
- if (pline->nfilters>32)
- HGOTO_ERROR(H5E_PLINE, H5E_CANTLOAD, NULL, "filter pipeline message has too many filters");
- p += 6; /*reserved*/
- pline->nalloc = pline->nfilters;
- pline->filter = H5MM_calloc(pline->nalloc*sizeof(pline->filter[0]));
- if (NULL==pline->filter)
- HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed");
- for (i=0; i<pline->nfilters; i++) {
- UINT16DECODE(p, pline->filter[i].id);
- UINT16DECODE(p, name_length);
- if (name_length % 8)
- HGOTO_ERROR(H5E_PLINE, H5E_CANTLOAD, NULL, "filter name length is not a multiple of eight");
- UINT16DECODE(p, pline->filter[i].flags);
- UINT16DECODE(p, pline->filter[i].cd_nelmts);
- if (name_length) {
- /*
- * Get the name, allocating an extra byte for an extra null
- * terminator just in case there isn't one in the file (there
- * should be, but to be safe...)
- */
- pline->filter[i].name = H5MM_malloc(name_length+1);
- HDmemcpy(pline->filter[i].name, p, name_length);
- pline->filter[i].name[name_length] = '\0';
- p += name_length;
- }
- if ((n=pline->filter[i].cd_nelmts)) {
- /*
- * Read the client data values and the padding
- */
- pline->filter[i].cd_values = H5MM_malloc(n*sizeof(unsigned));
- if (NULL==pline->filter[i].cd_values)
- HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed for client data");
- for (j=0; j<pline->filter[i].cd_nelmts; j++)
- UINT32DECODE(p, pline->filter[i].cd_values[j]);
- if (pline->filter[i].cd_nelmts % 2)
- p += 4; /*padding*/
- }
- }
-
- /* Set return value */
- ret_value = pline;
-
-done:
- if (NULL==ret_value && pline) {
- if (pline->filter) {
- for (i=0; i<pline->nfilters; i++) {
- H5MM_xfree(pline->filter[i].name);
- H5MM_xfree(pline->filter[i].cd_values);
- }
- H5MM_xfree(pline->filter);
- }
- H5FL_FREE(H5O_pline_t,pline);
- }
-
- FUNC_LEAVE(ret_value);
-}
-
-
-/*-------------------------------------------------------------------------
- * Function: H5O_pline_encode
- *
- * Purpose: Encodes message MESG into buffer P.
- *
- * Return: Non-negative on success/Negative on failure
- *
- * Programmer: Robb Matzke
- * Wednesday, April 15, 1998
- *
- * Modifications:
- *
- *-------------------------------------------------------------------------
- */
-static herr_t
-H5O_pline_encode (H5F_t UNUSED *f, uint8_t *p/*out*/, const void *mesg)
-{
- const H5O_pline_t *pline = (const H5O_pline_t*)mesg;
- size_t i, j, name_length;
- const char *name=NULL;
- H5Z_class_t *cls=NULL;
- herr_t ret_value=SUCCEED; /* Return value */
-
- FUNC_ENTER_NOAPI(H5O_pline_encode, FAIL);
-
- /* Check args */
- assert (p);
- assert (mesg);
-
- *p++ = H5O_PLINE_VERSION;
- *p++ = (uint8_t)(pline->nfilters);
- *p++ = 0; /*reserved 1*/
- *p++ = 0; /*reserved 2*/
- *p++ = 0; /*reserved 3*/
- *p++ = 0; /*reserved 4*/
- *p++ = 0; /*reserved 5*/
- *p++ = 0; /*reserved 6*/
-
- for (i=0; i<pline->nfilters; i++) {
- /*
- * Get the filter name. If the pipeline message has a name in it then
- * use that one. Otherwise try to look up the filter and get the name
- * as it was registered.
- */
- if (NULL==(name=pline->filter[i].name) &&
- (cls=H5Z_find(pline->filter[i].id)))
- name = cls->name;
- name_length = name ? HDstrlen(name)+1 : 0;
-
- /* Encode the filter */
- UINT16ENCODE(p, pline->filter[i].id);
- UINT16ENCODE(p, H5O_ALIGN(name_length));
- UINT16ENCODE(p, pline->filter[i].flags);
- UINT16ENCODE(p, pline->filter[i].cd_nelmts);
- if (name_length>0) {
- HDmemcpy(p, name, name_length);
- p += name_length;
- while (name_length++ % 8)
- *p++ = 0;
- }
- for (j=0; j<pline->filter[i].cd_nelmts; j++)
- UINT32ENCODE(p, pline->filter[i].cd_values[j]);
- if (pline->filter[i].cd_nelmts % 2)
- UINT32ENCODE(p, 0);
- }
-
-done:
- FUNC_LEAVE (ret_value);
-}
-
-
-/*-------------------------------------------------------------------------
- * Function: H5O_pline_copy
- *
- * Purpose: Copies a filter pipeline message from SRC to DST allocating
- * DST if necessary. If DST is already allocated then we assume
- * that it isn't initialized.
- *
- * Return: Success: Ptr to DST or allocated result.
- *
- * Failure: NULL
- *
- * Programmer: Robb Matzke
- * Wednesday, April 15, 1998
- *
- * Modifications:
- *
- *-------------------------------------------------------------------------
- */
-static void *
-H5O_pline_copy (const void *_src, void *_dst/*out*/)
-{
- const H5O_pline_t *src = (const H5O_pline_t *)_src;
- H5O_pline_t *dst = (H5O_pline_t *)_dst;
- size_t i;
- H5O_pline_t *ret_value = NULL;
-
- FUNC_ENTER_NOAPI(H5O_pline_copy, NULL);
-
- if (!dst && NULL==(dst = H5FL_ALLOC (H5O_pline_t,0)))
- HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed");
-
- *dst = *src;
- dst->nalloc = dst->nfilters;
- if (dst->nalloc>0) {
- dst->filter = H5MM_calloc(dst->nalloc * sizeof(dst->filter[0]));
- if (NULL==dst->filter)
- HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed");
- } else {
- dst->filter = NULL;
- }
-
- for (i=0; i<src->nfilters; i++) {
- dst->filter[i] = src->filter[i];
- if (src->filter[i].name) {
- dst->filter[i].name = H5MM_xstrdup(src->filter[i].name);
- }
- if (src->filter[i].cd_nelmts>0) {
- dst->filter[i].cd_values = H5MM_malloc(src->filter[i].cd_nelmts*
- sizeof(unsigned));
- if (NULL==dst->filter[i].cd_values)
- HGOTO_ERROR (H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed");
- HDmemcpy (dst->filter[i].cd_values, src->filter[i].cd_values,
- src->filter[i].cd_nelmts * sizeof(unsigned));
- }
- }
-
- /* Set return value */
- ret_value = dst;
-
-done:
- if (!ret_value && dst) {
- if (dst->filter) {
- for (i=0; i<dst->nfilters; i++) {
- H5MM_xfree(dst->filter[i].name);
- H5MM_xfree(dst->filter[i].cd_values);
- }
- H5MM_xfree(dst->filter);
- }
- if (!_dst)
- H5FL_FREE(H5O_pline_t,dst);
- }
-
- FUNC_LEAVE (ret_value);
-}
-
-
-/*-------------------------------------------------------------------------
- * Function: H5O_pline_size
- *
- * Purpose: Determines the size of a raw filter pipeline message.
- *
- * Return: Success: Size of message.
- *
- * Failure: zero
- *
- * Programmer: Robb Matzke
- * Wednesday, April 15, 1998
- *
- * Modifications:
- *
- *-------------------------------------------------------------------------
- */
-static size_t
-H5O_pline_size (H5F_t UNUSED *f, const void *mesg)
-{
- const H5O_pline_t *pline = (const H5O_pline_t*)mesg;
- size_t i, size, name_len;
- const char *name = NULL;
- H5Z_class_t *cls = NULL;
- size_t ret_value; /* Return value */
-
- FUNC_ENTER_NOAPI(H5O_pline_size, 0);
-
- /* Message header */
- size = 1 + /*version */
- 1 + /*number of filters */
- 6; /*reserved */
-
- for (i=0; i<pline->nfilters; i++) {
- /* Get the name of the filter, same as done with H5O_pline_encode() */
- if (NULL==(name=pline->filter[i].name) &&
- (cls=H5Z_find(pline->filter[i].id)))
- name = cls->name;
- name_len = name ? HDstrlen(name)+1 : 0;
-
-
- size += 2 + /*filter identification number */
- 2 + /*name length */
- 2 + /*flags */
- 2 + /*number of client data values */
- H5O_ALIGN(name_len); /*length of the filter name */
-
- size += pline->filter[i].cd_nelmts * 4;
- if (pline->filter[i].cd_nelmts % 2)
- size += 4;
- }
-
- /* Set return value */
- ret_value=size;
-
-done:
- FUNC_LEAVE (ret_value);
-}
-
-
-/*-------------------------------------------------------------------------
- * Function: H5O_pline_reset
- *
- * Purpose: Resets a filter pipeline message by clearing all filters.
- * The MESG buffer is not freed.
- *
- * Return: Non-negative on success/Negative on failure
- *
- * Programmer: Robb Matzke
- * Wednesday, April 15, 1998
- *
- * Modifications:
- *
- *-------------------------------------------------------------------------
- */
-static herr_t
-H5O_pline_reset (void *mesg)
-{
- H5O_pline_t *pline = (H5O_pline_t*)mesg;
- size_t i;
- herr_t ret_value=SUCCEED; /* Return value */
-
- FUNC_ENTER_NOAPI(H5O_pline_reset, FAIL);
-
- assert (pline);
-
- for (i=0; i<pline->nfilters; i++) {
- H5MM_xfree(pline->filter[i].name);
- H5MM_xfree(pline->filter[i].cd_values);
- }
- H5MM_xfree(pline->filter);
- HDmemset(pline, 0, sizeof *pline);
-
-done:
- FUNC_LEAVE (ret_value);
-}
-
-
-/*-------------------------------------------------------------------------
- * Function: H5O_pline_free
- *
- * Purpose: Free's the message
- *
- * Return: Non-negative on success/Negative on failure
- *
- * Programmer: Quincey Koziol
- * Saturday, March 11, 2000
- *
- * Modifications:
- *
- *-------------------------------------------------------------------------
- */
-static herr_t
-H5O_pline_free (void *mesg)
-{
- herr_t ret_value=SUCCEED; /* Return value */
-
- FUNC_ENTER_NOAPI(H5O_pline_free, FAIL);
-
- assert (mesg);
-
- H5FL_FREE(H5O_pline_t,mesg);
-
-done:
- FUNC_LEAVE (ret_value);
-}
-
-
-/*-------------------------------------------------------------------------
- * Function: H5O_pline_debug
- *
- * Purpose: Prints debugging information for filter pipeline message MESG
- * on output stream STREAM. Each line is indented INDENT
- * characters and the field name takes up FWIDTH characters.
- *
- * Return: Non-negative on success/Negative on failure
- *
- * Programmer: Robb Matzke
- * Wednesday, April 15, 1998
- *
- * Modifications:
- *
- *-------------------------------------------------------------------------
- */
-static herr_t
-H5O_pline_debug (H5F_t UNUSED *f, const void *mesg, FILE *stream,
- int indent, int fwidth)
-{
- const H5O_pline_t *pline = (const H5O_pline_t *)mesg;
- size_t i, j;
- herr_t ret_value=SUCCEED; /* Return value */
-
- FUNC_ENTER_NOAPI(H5O_pline_debug, FAIL);
-
- /* check args */
- assert(f);
- assert(pline);
- assert(stream);
- assert(indent >= 0);
- assert(fwidth >= 0);
-
- fprintf(stream, "%*s%-*s %lu/%lu\n", indent, "", fwidth,
- "Number of filters:",
- (unsigned long)(pline->nfilters),
- (unsigned long)(pline->nalloc));
-
- for (i=0; i<pline->nfilters; i++) {
- char name[32];
- sprintf(name, "Filter at position %lu", (unsigned long)i);
- fprintf(stream, "%*s%-*s\n", indent, "", fwidth, name);
- fprintf(stream, "%*s%-*s 0x%04x\n", indent+3, "", MAX(0, fwidth-3),
- "Filter identification:",
- (unsigned)(pline->filter[i].id));
- if (pline->filter[i].name) {
- fprintf(stream, "%*s%-*s \"%s\"\n", indent+3, "", MAX(0, fwidth-3),
- "Filter name:",
- pline->filter[i].name);
- } else {
- fprintf(stream, "%*s%-*s NONE\n", indent+3, "", MAX(0, fwidth-3),
- "Filter name:");
- }
- fprintf(stream, "%*s%-*s 0x%04x\n", indent+3, "", MAX(0, fwidth-3),
- "Flags:",
- (unsigned)(pline->filter[i].flags));
- fprintf(stream, "%*s%-*s %lu\n", indent+3, "", MAX(0, fwidth-3),
- "Num CD values:",
- (unsigned long)(pline->filter[i].cd_nelmts));
- for (j=0; j<pline->filter[i].cd_nelmts; j++) {
- char field_name[32];
- sprintf(field_name, "CD value %lu", (unsigned long)j);
- fprintf(stream, "%*s%-*s %lu\n", indent+6, "", MAX(0, fwidth-6),
- field_name,
- (unsigned long)(pline->filter[i].cd_values[j]));
- }
- }
-
-done:
- FUNC_LEAVE(ret_value);
-}