summaryrefslogtreecommitdiffstats
path: root/src/H5Sprivate.h
Commit message (Collapse)AuthorAgeFilesLines
* [svn-r13012] Description:Quincey Koziol2006-12-031-1/+1
| | | | | | | | | | | | | | Refactor object header code to separate process of creating an object header message from the process of writing to an existing one. Start renaming operations that deal with object header messages to have "H5O_msg_" prefix... Tested on: Mac OS X/32 10.4.8 (amazon) FreeBSD/32 4.11 (sleipnir) Linux/32 2.4 (heping) AIX/32 5.? (copper)
* [svn-r13002] Description:Quincey Koziol2006-11-301-0/+3
| | | | | | | | | | | | Fix problem with copying objects when the dataset to copy has an "older" version of the layout message and needs to compute the size of the dataset's storage after the dataspace information is available. Minor random cleanups also... Tested on: FreeBSD/32 4.11 (sleipnir) Too minor/weird to test w/h5committest...
* [svn-r11899] Purpose:Quincey Koziol2006-01-281-1/+4
| | | | | | | | | | | | | | | | Bug fix & new feature Description: Support variable-length datatypes in compact data storage and chunked data storage, along with attributes. Bug fix on the H5T_vlen_set_loc to allow for changing the file on a variable-length datatype on disk. Platforms tested: FreeBSD 4.11 (sleipnir) Linux 2.4 Can't h5committest right now, due to missing cache files.
* [svn-r11712] Purpose:Quincey Koziol2005-11-151-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | New feature Description: Check in baseline for compact group revisions, which radically revises the source code for managing groups and object headers. WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! This initiates the "unstable" phase of the 1.7.x branch, leading up to the 1.8.0 release. Please test this code, but do _NOT_ keep files created with it - the format will change again before the release and you will not be able to read your old files!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! Solution: There's too many changes to really describe them all, but some of them include: - Stop abusing the H5G_entry_t structure and split it into two separate structures for non-symbol table node use within the library: H5O_loc_t for object locations in a file and H5G_name_t to store the path to an opened object. H5G_entry_t is now only used for storing symbol table entries on disk. - Retire H5G_namei() in favor of a more general mechanism for traversing group paths and issuing callbacks on objects located. This gets us out of the business of hacking H5G_namei() for new features, generally. - Revised H5O* routines to take a H5O_loc_t instead of H5G_entry_t - Lots more... Platforms tested: h5committested and maybe another dozen configurations.... :-)
* [svn-r11485] Purpose:Quincey Koziol2005-09-301-3/+3
| | | | | | | | | | | | Code cleanup Description: Clean up & optimize (a little) the recent "hyperslab rebuilding" code w/Kent. Platforms tested: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r11245] Purpose:Quincey Koziol2005-08-131-1/+1
| | | | | | | | | | | | | | | | | | | | Code cleanup Description: Trim trailing whitespace, which is making 'diff'ing the two branches difficult. Solution: Ran this script in each directory: foreach f (*.[ch] *.cpp) sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f end Platforms tested: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r11078] Purpose:Quincey Koziol2005-07-171-1/+1
| | | | | | | | | | | | | | | Bug fix Description: Correct memory leak when a dataset is attempted to be opened, but turns out to be a group or named datatype. Also, clean up code that was leading to the leak and zero out empty group entries to help prevent similar errors in the future. Platforms tested: FreebSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r11014] Purpose:Quincey Koziol2005-07-021-1/+1
| | | | | | | | | | | | Code cleanup Description: Refactor metadata cache to merge "dirtied" flag in with other flags for H5AC_unprotect and H5C_unprotect. Platforms tested: FreeBSD 4.11 (sleipnir) h5committest
* [svn-r10978] Purpose:John Mainzer2005-06-241-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | Interim checkin of code changes moving management of the is_dirty flag into the cache code. Description: Prior to this checkin, management of the is_dirty flag was handled above the level of the metadata cache. This can no longer be allowed, as it introduces a race condition in the proposed fix for a cache coherency bug in PHDF5. Solution: Move management fo the is_dirty flag to the cache code proper. Entries are now marked as dirty via a flag on the unprotect call. Platforms tested: h5committested Misc. update:
* [svn-r10951] Purpose:Quincey Koziol2005-06-181-1/+2
| | | | | | | | | | | | | | | | | Bug fix Description: Hyperslab selections that had a selection offset and were applied to a chunked dataset could get into an infinite loop or core dump if the same selection was used multiple times, with different selection offsets. Solution: "Normalize" the selection with the selection offset, generate the selections for the chunks overlapped and then "denormalize" the selection. Platforms tested: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r9727] Purpose:Quincey Koziol2004-12-291-9/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bug Fix/Code Cleanup/Doc Cleanup/Optimization/Branch Sync :-) Description: Generally speaking, this is the "signed->unsigned" change to selections. However, in the process of merging code back, things got stickier and stickier until I ended up doing a big "sync the two branches up" operation. So... I brought back all the "infrastructure" fixes from the development branch to the release branch (which I think were actually making some improvement in performance) as well as fixed several bugs which had been fixed in one branch, but not the other. I've also tagged the repository before making this checkin with the label "before_signed_unsigned_changes". Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel & fphdf5 FreeBSD 4.10 (sleipnir) w/threadsafe FreeBSD 4.10 (sleipnir) w/backward compatibility Solaris 2.7 (arabica) w/"purify options" Solaris 2.8 (sol) w/FORTRAN & C++ AIX 5.x (copper) w/parallel & FORTRAN IRIX64 6.5 (modi4) w/FORTRAN Linux 2.4 (heping) w/FORTRAN & C++ Misc. update:
* [svn-r9519] Purpose:MuQun Yang2004-11-111-0/+9
| | | | | | | | | | | | | | | | | | | | Adding codes for the general MPI derived datatype in order to better incorporate new fixes of HDF5 library. Description: Note: These codes have not been tested for general use. Don't call these functions in your developments of the HDF5 library. Also these codes are stand-alone codes, they should not affect other library codes. Solution: Platforms tested: Heping(C and Parallel linux 2.4, mpich 1.2.6) Arabica(C,C++,Fortran, Solaris 2.7) Copper(C,c++,Fortran, AIX 5.1, NOTE: c++ FAILED, seems not due to the recent check-in) Misc. update:
* [svn-r9354] Purpose:Quincey Koziol2004-10-011-50/+9
| | | | | | | | | | | | | | | | Bug fix & code cleanup Description: More dataset cleanups to get to a point where we can fix the chunked I/O bug. Also fix a couple of errors in the recent file object resurrection changes which should hopefully address the recent daily test failres (H5T.c) Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Solaris 2.7 (arabica) h5committest
* [svn-r9342] Purpose:Quincey Koziol2004-09-301-12/+12
| | | | | | | | | | | | | | | | Bug fix/code cleanup Description: Clean up raw data I/O code to bundle the I/O parameters (dataset, DXPL ID, etc) into a single struct to pass around through the dataset I/O routines, since they are always passed together, until very near the bottom of the I/O stack. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Solaris 2.7 (arabica) IRIX64 6.5 (modi4) h5committest
* [svn-r8987] Purpose:Quincey Koziol2004-08-021-1/+1
| | | | | | | | | | | | Code cleanup Description: Fix another batch of minor differences between the development and release branches. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8981] Purpose:Quincey Koziol2004-08-021-2/+0
| | | | | | | | | | | | | Code cleanup Description: Various minor tweaks to clean code up and bring it into closer syncronization with the release branch. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel h5committested IRIX64 6.5 (modi4)
* [svn-r8953] James Laird2004-07-271-0/+1
| | | | | | | | | | | | | | | | Purpose: Bug fix Description: When a simple dataspace is created, its extent should be set before using it, or it will silently function as a NULL dataspace. Solution: Added checks on user-supplied dataspaces. Now dataspaces without extents set will throw errors; users must explicitly set a dataspace to be NULL. Platforms tested: sleipnir, windows
* [svn-r8932] Purpose:Quincey Koziol2004-07-221-0/+8
| | | | | | | | | | | | | | | | | | | | Code cleanup Description: Clean up collective chunking code a bit. Also, add '--enable-instrument' configure flag to have a mechanism for determining that optimized operations happened correctly in the library (instead of just the "normal" way) by allowing 'flag' properties to be set outside the library and set when the "right" thing happens. This is mainly for debugging and regression checks, so we make certain we don't break optimized I/O by accident. It's enabled by default when --enable-debug is on (which is on by default in the development branch and off by default in the release branch), but can also be independently controlled with its own configure flag. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel IBM p690 (copper) w/parallel
* [svn-r8921] Purpose: new featureRaymond Lu2004-07-211-0/+2
| | | | | | | | | | | | | | Description: This is the second step of checkin for encoding and decoding objects. H5Tencode and H5Tdecode have been committed in the previous step. H5Sencode and H5Sdecode are checked in this time. Solution: Given object ID, these functions encode and decode object information into and from binary buffer and return new object ID. They take advantage of the existing codes of object header message and encode in the same format. Platforms tested: fuss and h5committest. Misc. update: RELEASE.txt
* [svn-r8906] Purpose:MuQun Yang2004-07-201-2/+2
| | | | | | | | | | | | | | | | | | | | | | | Adding the first round of patches about supporting collective chunk IO in HDF5 Description: The current HDF5 library doesn't support collective MPIO with chunk storage. When users set collective option in their data transfer with chunk storage, the library silently converted the option to INDEPENDENT and that caused trememdous performance penalty. Some application like WRF-parallel HDF5 IO module has to use contiguous storage for this reason. However, chunking storage has its own advantage(supporting compression filters and extensible dataset), so to make collective MPIO possible inside HDF5 with chunking storage is a very important task. This check-in make collective chunk IO possible for some special cases. The condition is as follows(either case is fine with using collective chunk IO) 1. for each process, the hyperslab selection of the file data space of each dataset is regular and it is fit in one chunk. 2. for each process, the hyperslab selection of the file data space of each dataset is single and the number of chunks for the hyperslab selection should be equal. Solution: Lift up the contiguous storage requirement for collective IO. Use H5D_isstore_get_addr to get the corresponding chunk address. Then the original library routines will take care of getting the correct address to make sure that MPI FILE TYPE is built correctly for collective IO> Platforms tested: arabica(sol), copper(AIX), eirene(Linux) parallel test is checked at copper. Misc. update:
* [svn-r8782] Purpose:Quincey Koziol2004-07-011-1/+1
| | | | | | | | | | | Code cleanup Description: Clean up almost all warnings from Windows builds. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8731] Purpose:Quincey Koziol2004-06-231-2/+0
| | | | | | | | | | | | | | | Code cleanup & minor optimization Description: Re-work the way interface initialization routines are specified in the library to avoid the overhead of checking for them in routines where there is no interface initialization routine. This cleans up warnings with gcc 3.4, reduces the library binary size a bit (about 2-3%) and should speedup the library's execution slightly. Platforms tested: FreeBSD 4.10 (sleipnir) w/gcc34 h5committest
* [svn-r8683] Purpose:Quincey Koziol2004-06-141-6/+6
| | | | | | | | | | | | | Code optimization Description: Use 'size_t' instead of 'hsize_t' to track the number of elements in memory buffers, especially for type conversion. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8677] Purpose:Quincey Koziol2004-06-141-1/+1
| | | | | | | | | | | | | | | | | | | Code cleanup & optimization Description: Remove old structures that used a union to store information about the dataspace extent and just store the information directly in the dataspace extent itself. Remove ifdef'd references to permutation ordering in dataspaces. We'll definitely need more than this code if/when we implement this feature. Change allocation of dataspace information from calloc() to malloc(). Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8673] Purpose:Quincey Koziol2004-06-131-50/+33
| | | | | | | | | | | | | | | | | | Code optimization Description: Revised dataspace selections to use a more "object oriented" mechanism to set the function pointers for each selection and selection iterator. This reduces the amount and number of times that dataspace selection info has to be copied. Additionally, change hyperslab selection information to be dynamically allocated instead of an inline struct. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8613] Purpose:Quincey Koziol2004-06-051-26/+28
| | | | | | | | | | | | | | | | | Refactor code Description: Move chunk and contiguous cached raw data from file information to dataset information. This simplifies a number of internal interfaces, aligns the code with it's purpose better and should allow more optimizations to the chunked data I/O performance. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) h5committest Misc. update:
* [svn-r8590] Purpose:Quincey Koziol2004-05-271-0/+2
| | | | | | | | | | | | | | | | | | | | | Code optimization & bug fix Description: When dimension information is being stored in the storage layout message on disk, it is stored as 32-bit quantities, possibly truncating the dimension information, if a dimension is greater than 32-bits in size. Solution: Fix the storage layout message problem by revising file format to not store dimension information, since it is already available in the dataspace. Also revise the storage layout data structures to be more compartmentalized for the information for contiguous, chunked and compact storage. Platforms tested: FreeBSD 4.9 (sleipnir) w/parallel Solaris 2.7 (arabica) h5committest
* [svn-r8560] Purpose:Quincey Koziol2004-05-211-0/+2
| | | | | | | | | | | | Bug fix Description: Add prototypes for H5S_select_serial_size() and H5S_select_serialize() that were causing the library to fail when built with C++ compiler. Platforms tested: FreeBSD 4.9 (sleipnir) w/CC=g++ Only affects C++ builds, so no h5committest
* [svn-r8544] Purpose:Quincey Koziol2004-05-201-0/+52
| | | | | | | | | | | | | | Code optimization Description: Expand the use of macros to inline trivial function pointer lookup and calls to reduce the overall number of functions invoked during normal operation of the library. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8513] Purpose:Quincey Koziol2004-05-131-1/+1
| | | | | | | | | | | | Code optimization Description: Defer creating the span trees for hyperslab selections until they are actually needed (which may be never, in certain circumstances). Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel
* [svn-r8496] Purpose:Quincey Koziol2004-05-081-2/+2
| | | | | | | | | | | | | Code optimization Description: Further reduce the number of copies we make of a hyperslab selection for chunked I/O, especially when we are only going to throw the old selection away for a new one. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel
* [svn-r8462] Purpose:Quincey Koziol2004-05-011-4/+4
| | | | | | | | | | | | | Code optimization Description: Reduce the number of times the number of elements in a selection is computed. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel too minor to require h5committest
* [svn-r8460] Purpose:Quincey Koziol2004-05-011-5/+7
| | | | | | | | | | | | | | | Code optimization Description: Move the element size for the selection into the selection iterator instead of always passing it as a parameter. Also, eleminate another 64-bit multiply for "all" selections. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r8408] Purpose:Quincey Koziol2004-04-221-6/+14
| | | | | | | | | | | | | | Code optimization Description: Instead of dynamicly allocating various arrays for various pieces of information about a selection or selection iterator, just use fixed size array of size H5S_MAX_RANK (as the rest of the library does). Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) h5committest
* [svn-r8335] Purpose:Quincey Koziol2004-04-101-0/+3
| | | | | | | | | | | | | | | Code optimization Description: Change algorithm to directly use coordinates describing a chunk's position in a dataspace instead of creating a dataspace with the chunk's position selected. This reduces the number of copies of dataspaces we need to keep around. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel too minor to require h5committest
* [svn-r8158] Purpose:Quincey Koziol2004-02-061-16/+18
| | | | | | | | | | | | | | | | | Code cleanup/optimization Description: Query property list values once, at the beginning of the I/O routines, instead of querying the property list values multiple (lots!) of times in lower level routines. Solution: Create "property list caches" for internal library queries of the property list values. Platforms tested: IBM p690 (copper) w/parallel & fphdf5 h5committest
* [svn-r7943] Purpose:Quincey Koziol2003-12-131-2/+3
| | | | | | | | | | | | | | | Bug fix. Description: Using a selection offset with hyperslab selections in chunked datasets was getting into an infinite loop and hanging the application. Solution: Apply the selection offset to the hyperslab selection properly. Platforms tested: FreeBSD 4.9 (sleipnir) w & w/o parallel h5committest
* [svn-r7789] Purpose:Quincey Koziol2003-10-291-1/+0
| | | | | | | | | | | | | | | Bug fix & code cleanups Description: Change our use of MPI derived datatypes to not create datatypes with "0-sized" lengths, which causes the LANL Q machine to hang. Also, get rid of "prefer MPI derived datatypes" environment variable since it has no advantage. Platforms tested: FreeBSD 4.9 (sleipnir) w & w/o parallel h5committest
* [svn-r7754] Purpose:Quincey Koziol2003-10-271-1/+1
| | | | | | | | | | | | | Code cleanup Description: Straighten out more goofiness in the MPI code dealing with collective I/O transfers - mostly make certain that a view is set if-and-only-if collective I/O is occurring on raw data (and vice versa for views and independent I/O) Platforms tested: FreeBSD 4.9 (sleipnir) w/parallel & FPHDF5 too minor to repquire h5committest
* [svn-r7259] Purpose:Quincey Koziol2003-07-231-0/+10
| | | | | | | | | | | | | | | Bug fix Description: I/O on chunked datasets with point selections was not working correctly. Solution: Re-wrote some parts of raw data I/O routines that build the selections for each chunk to correctly handle point selections. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest
* [svn-r7223] Purpose:Quincey Koziol2003-07-141-1/+1
| | | | | | | | | | | Code cleanup Description: Clean up more warnings from lint Platforms tested: FreeBSD 4.8 (sleipnir) too minor for h5committest
* [svn-r6957] Purpose:Quincey Koziol2003-06-041-1/+12
| | | | | | | | | | | | | | | | Code cleanup & performance improvements Description: Optimize hyperslabs that are built to detect situations where "regular" hyperslabs can be recovered from span tree descriptions. Also, improve "same shape" routine to correctly work with all the different combinations of selections. Platforms tested: FreeBSD 4.8 (sleipnir) w/C++ FreeBSD 4.8 (sleipnir) w/parallel h5committested
* [svn-r6942] Purpose:Quincey Koziol2003-05-311-2/+11
| | | | | | | | | | | | | | | | | Performance improment Description: Speed up chunked dataset I/O. This breaks down into several areas: - Compute chunk selections in the file by using hyperslab operations instead of iterating over each element in the selection. - If the file and memory selections are the same shape, use the file chunk selections to compute the memory chunk selections. This required several additional dataspace, dataspace selection and hyperslab routines. Platforms tested: h5committestted (although Fortran tests failed for some reason)
* [svn-r6843] Purpose:Quincey Koziol2003-05-091-0/+9
| | | | | | | | | | | | Code cleanup Description: Clean up warnings exposed by compiling on O2K. Also, revert some of Bill and my changes to the H5S_mpi_opt_types_g, etc. and settle them back into their original location. Platforms tested: h5committested.
* [svn-r6837] Purpose:Quincey Koziol2003-05-081-65/+9
| | | | | | | | | | | | | | | Code cleanup. Description: Move many package or internal function prototypes and macro definitions into tighter scope according to their current use. Added more comments where appropriate. Eliminate ancient, unused functions. Added a couple "accessor" functions to get parts of data structures which were moved out of scope. Platforms tested: h5committested
* [svn-r6825] Purpose:Quincey Koziol2003-05-071-19/+59
| | | | | | | | | | | | | | | | | | | | | | | | | New feature/enhancement Description: Chunked datasets are handled poorly in several circumstances involving certain selections and chunks that are too large for the chunk cache and/or chunks with filters, causing the chunk to be read from disk multiple times. Solution: Rearrange raw data I/O infrastructure to handle chunked datasets in a much more friendly way by creating a selection in memory and on disk for each chunk in a chunked dataset and performing all of the I/O on that chunk at one time. There are still some scalability (the current code attempts to create a selection for all the chunks in the dataset, instead of just the chunks that are accessed, requiring portions of the istore.c and fillval.c tests to be commented out) and performance issues, but checking this in will allow the changes to be tested by a much wider audience while I address the remaining issues. Platforms tested: h5committested, FreeBSD 4.8 (sleipnir) serial & parallel, Linux 2.4 (eirene)
* [svn-r6608] Purpose:Quincey Koziol2003-04-091-0/+2
| | | | | | | | | | | | | | | | | | New feature Description: Added new internal API function: H5S_create_simple() for creating simple dataspaces inside the library. Solution: Platforms tested: FreeBSD 4.8 (sleipnir) w/szip Linux 2.4 (sleipnir) w/szip Solaris 2.7 (arabica) w/FORTRAN IRIX64 6.5 (modi4) w/szip, FORTRAN & parallel Misc. update:
* [svn-r6411] Purpose:Quincey Koziol2003-02-171-12/+14
| | | | | | | | | | | | | | | | | | | | | | Code cleanup Description: Clean up miscellaneous warnings which have crept into the code. Fix "_POSIX_C_SOURCE not defined" warning on FreeBSD. Adjust gcc compiler flags to be more concise for production mode. Refactor the H5O code so that there is a stronger boundary between code in the H5O package and code in the library which just calls H5O routines. Platforms tested: Tested h5committest {arabica (fortran), eirene (fortran, C++) modi4 (parallel, fortran)} FreeBSD 4.7 (sleipnir) serial & parallel and gcc 2.95.4 & gcc 3.2.2 Misc. update: Update MANIFEST if you add or remove any file.
* [svn-r6387] Purpose:Quincey Koziol2003-02-101-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | Bug Fix Description: Metadata cache in parallel I/O can cause hangs in applications which perform independent I/O on chunked datasets, because the metadata cache can attempt to flush out dirty metadata from only a single process, instead of collectively from all processes. Solution: Pass a dataset transfer property list down from every API function which could possibly trigger metadata I/O. Then, split the metadata cache into two sets of entries to allow dirty metadata to be set aside when a hash table collision occurs during independent I/O. Platforms tested: Tested h5committest {arabica (fortran), eirene (fortran, C++) modi4 (parallel, fortran)} FreeBSD 4.7 (sleipnir) serial & parallel Misc. update: Updated release_docs/RELEASE
* [svn-r6252] Purpose:Quincey Koziol2003-01-091-17/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Lots of performance improvements & a couple new internal API interfaces. Description: Performance Improvements: - Cached file offset & length sizes in shared file struct, to avoid constantly looking them up in the FCPL. - Generic property improvements: - Added "revision" number to generic property classes to speed up comparisons. - Changed method of storing properties from using a hash-table to the TBBT routines in the library. - Share the propery names between classes and the lists derived from them. - Removed redundant 'def_value' buffer from each property. - Switching code to use a "copy on write" strategy for properties in each list, where the properties in each list are shared with the properties in the class, until a property's value is changed in a list. - Fixed error in layout code which was allocating too many buffers. - Redefined public macros of the form (H5open()/H5check, <variable>) internally to only be (<variable>), avoiding innumerable useless calls to H5open() and H5check_version(). - Reuse already zeroed buffers in H5F_contig_fill instead of constantly re-zeroing them. - Don't write fill values if writing entire dataset. - Use gettimeofday() system call instead of time() system when checking the modification time of a dataset. - Added reference counted string API and use it for tracking the names of objects opening in a file (for the ID->name code). - Removed redundant H5P_get() calls in B-tree routines. - Redefine H5T datatype macros internally to the library, to avoid calling H5check redundantly. - Keep dataspace information for dataset locally instead of reading from disk each time. Added new module to track open objects in a file, to allow this (which will be useful eventually for some FPH5 metadata caching issues). - Remove H5AC_find macro which was inlining metadata cache lookups, and call function instead. - Remove redundant memset() calls from H5G_namei() routine. - Remove redundant checking of object type when locating objects in metadata cache and rely on the address only. - Create default dataset object to use when default dataset creation property list is used to create datasets, bypassing querying for all the property list values. - Use default I/O vector size when performing raw data with the default dataset transfer property list, instead of querying for I/O vector size. - Remove H5P_DEFAULT internally to the library, replacing it with more specific default property list based on the type of property list needed. - Remove redundant memset() calls in object header message (H5O*) routines. - Remove redunant memset() calls in data I/O routines. - Split free-list allocation routines into malloc() and calloc()- like routines, instead of one combined routine. - Remove lots of indirection in H5O*() routines. - Simplify metadata cache entry comparison routine (used when flushing entire cache out). - Only enable metadata cache statistics when H5AC_DEBUG is turned on, instead of always tracking them. - Simplify address comparison macro (H5F_addr_eq). - Remove redundant metadata cache entry protections during dataset creation by protecting the object header once and making all the modifications necessary for the dataset creation before unprotecting it. - Reduce # of "number of element in extent" computations performed by computing and storing the value during dataspace creation. - Simplify checking for group location's file information, when file has not been involving in file-mounting operations. - Use binary encoding for modification time, instead of ASCII. - Hoist H5HL_peek calls (to get information in a local heap) out of loops in many group routine. - Use static variable for iterators of selections, instead of dynamically allocation them each time. - Lookup & insert new entries in one step, avoiding traversing group's B-tree twice. - Fixed memory leak in H5Gget_objname_idx() routine (tangential to performance improvements, but fixed along the way). - Use free-list for reference counted strings. - Don't bother copying object names into cached group entries, since they are re-created when an object is opened. The benchmark I used to measure these results created several thousand small (2K) datasets in a file and wrote out the data for them. This is Elena's "regular.c" benchmark. These changes resulted in approximately ~4.3x speedup of the development branch when compared to the previous code in the development branch and ~1.4x speedup compared to the release branch. Additionally, these changes reduce the total memory used (code and data) by the development branch by ~800KB, bringing the development branch back into the same ballpark as the release branch. I'll send out a more detailed description of the benchmark results as a followup note. New internal API routines: Added "reference counted strings" API for tracking strings that get used by multiple owners without duplicating the strings. Added "ternary search tree" API for text->object mappings. Platforms tested: Tested h5committest {arabica (fortran), eirene (fortran, C++) modi4 (parallel, fortran)} Other platforms/configurations tested? FreeBSD 4.7 (sleipnir) serial & parallel Solaris 2.6 (baldric) serial
generated by cgit v0.12 at 2024-08-22 16:54:09 (GMT)