summaryrefslogtreecommitdiffstats
path: root/src/H5FD.c
Commit message (Collapse)AuthorAgeFilesLines
* [svn-r12570] Description:Quincey Koziol2006-08-131-17/+15
| | | | | | | | | | Refactor file memory allocation code slightly, to make it clearer, along with adding/correcting some comments. Tested on: FreeBSD 4.11 (sleipnir) Linux/64 2.4 (mir) Solaris/64 2.9 (shanti)
* [svn-r12504] Pedro Vicente Nunes2006-07-271-0/+4
| | | | | | | | | | | | bug fix calling h5tools_get_fapl running the mpio driver caused H5FD_pl_copy to fail silently in some conditions. a MPI call was made before MPI_Init. solution: corrected the MPI call to be made after MPI_Init and added error return conditions to H5FD_pl_copy and h5tools_get_fapl tested on copper parallel, mir, shanti
* [svn-r12292] Purpose:Quincey Koziol2006-04-201-53/+0
| | | | | | | | | | | | | Code maintenance Description: Remove flexible parallel code Platforms tested: FreeBSD 4.11 (sleipnir) Linux 2.4 (heping) Solaris 2.9 (shanti) Linux 2.4/64 (mir)
* [svn-r12155] Purpose:Quincey Koziol2006-03-261-58/+63
| | | | | | | | | | | Code optimization Description: Tweak file free space allocation routines to optimize them a bit. Platforms tested: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r11470] Purpose:John Mainzer2005-09-271-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Repair synchronization bug in the metadata cache in PHDF5 Also repair numerous other bugs that surfaced in testing the bug fix. Description: While operations modifying metadata must be collective, we allow independant reads. This allows metadata caches on different processes to adjust to different sizes, and to place the entries on their dirty lists in different orders. Since only process 0 actually writes metadata to disk (all other processes thought they did, but the writes were discarded on the theory that they had to be collective), this made it possible for another process to modify metadata, flush it, and then read it back in in its original form (pre-modification) form. The possibilities for file corruption should be obvious. Solution: Make the policy that only process 0 can write to file explicit, and visible to the metadata caches. Thus only process 0 may flush dirty entries -- all other caches must retain dirty entries until they are informed by process 0 that the entries are clean. Synchronization is handled by counting the bytes of dirty cache entries created, and then synching up between the caches whenever the sum exceeds an (eventually user specified) limit. Dirty metadata creation is consistent across all processes because all operations modifying metadata must be collective. This change uncovered may bugs which are repaired in this checkin. It also required modification of H5HL and H5O to allocate file space on insertion rather than on flush from cache. Platforms tested: H5committest, heping(parallel & serial) Misc. update:
* [svn-r11338] Purpose:Quincey Koziol2005-09-021-1/+3
| | | | | | | | | | | | | Bug fix Description: Check in fix that John found for bug in H5FD_extend() code - adjust beginning as well as size of free block when "extending" another block of space in the file. Platforms tested: FreeBsd 4.11 (sleipnir) Too minor to require h5committest
* [svn-r11245] Purpose:Quincey Koziol2005-08-131-53/+53
| | | | | | | | | | | | | | | | | | | | Code cleanup Description: Trim trailing whitespace, which is making 'diff'ing the two branches difficult. Solution: Ran this script in each directory: foreach f (*.[ch] *.cpp) sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f end Platforms tested: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r10919] Purpose: Take Out An Unused FunctionRaymond Lu2005-06-141-0/+1
| | | | | | | | | | | Description: Data conversion function H5T_conv_i32le_f64le() was unused. Because we have new conversion functions for integers and floating numbers, this old function is taken out. Platforms tested: fuss - simple change. Misc. update:
* [svn-r10628] Purpose:Quincey Koziol2005-04-181-2/+2
| | | | | | | | | | | Code cleanup Description: Clean up various warnings reported by the Windows team. Platforms tested: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r10570] Purpose: MaintenanceElena Pourmal2005-04-071-4/+0
| | | | | | | | | | | Description: Removed support for SRB driver Solution: Removed or modified appropriate files; ran reconfigure to regenerate Makefile.in and configure files. Platforms tested: heping and shanti Misc. update: ran bin/chkmanifest on heping
* [svn-r10505] Purpose:Quincey Koziol2005-03-291-47/+112
| | | | | | | | | | | | New feature Description: Search free list when checking for, or actually performing, extension of an object in the file. Platforms tested: FreeBSD 4.11 (sleipnir) Solaris 2.9 (shanti)
* [svn-r10130] Purpose:Albert Cheng2005-03-031-4/+0
| | | | | | | Retired GASS driver. Platforms tested: H5committested.
* [svn-r9857] Purpose: MaintenanceElena Pourmal2005-01-221-3/+0
| | | | | | | | | | | | | | Description: Removed PABLO from the source Solution: Platforms tested: arabica with 64-bit, copper with parallel, heping with GNU C and C++ and PGI fortran (but I disabled hl, there is some weird problem only on heping: F9XMODFLAG is not propagated to the Makefile files Misc. update:
* [svn-r9727] Purpose:Quincey Koziol2004-12-291-18/+19
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bug Fix/Code Cleanup/Doc Cleanup/Optimization/Branch Sync :-) Description: Generally speaking, this is the "signed->unsigned" change to selections. However, in the process of merging code back, things got stickier and stickier until I ended up doing a big "sync the two branches up" operation. So... I brought back all the "infrastructure" fixes from the development branch to the release branch (which I think were actually making some improvement in performance) as well as fixed several bugs which had been fixed in one branch, but not the other. I've also tagged the repository before making this checkin with the label "before_signed_unsigned_changes". Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel & fphdf5 FreeBSD 4.10 (sleipnir) w/threadsafe FreeBSD 4.10 (sleipnir) w/backward compatibility Solaris 2.7 (arabica) w/"purify options" Solaris 2.8 (sol) w/FORTRAN & C++ AIX 5.x (copper) w/parallel & FORTRAN IRIX64 6.5 (modi4) w/FORTRAN Linux 2.4 (heping) w/FORTRAN & C++ Misc. update:
* [svn-r8981] Purpose:Quincey Koziol2004-08-021-4/+3
| | | | | | | | | | | | | Code cleanup Description: Various minor tweaks to clean code up and bring it into closer syncronization with the release branch. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel h5committested IRIX64 6.5 (modi4)
* [svn-r8946] Purpose:Quincey Koziol2004-07-261-0/+15
| | | | | | | | | | | | | | | | | | Code cleanup, sorta Description: Added ifdef sections for "H5_USING_PURIFY" in various places in the code, which are designed to reduce the spurious "uninitialized memory read" warnings from purify which are actually OK. Note that this macro will have to be turned on by adding it to the CFLAGS for the build - I didn't think it was important enough to add a configure flag for. Also, the changes in H5HG.c optimize the walks through the objects in a heap to only look at the 'used' entries instead of all the 'allocated' entries. Platforms tested: Solaris 2.7 (arabica) w/purify Not tested by h5committest
* [svn-r8877] James Laird2004-07-141-0/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | Purpose: Bug Fix Description: If an HDF5 file grows larger than its address space, it dies and is unable to write any data. This is more likely to happen since users are able to change the number of bytes used to store addresses in the file. Solution: HDF5 now throws an error instead of dying. In addition, it "reserves" address space for the local heap and for object headers (which do not allocate space immediately). This ensures that after the error occurs, there is enough address space left to flush the entire file to disk, so no data is lost. A more complete explanation is at /doc/html/TechNotes/ReservedFileSpace.html Platforms tested: sleipnir, copper (parallel), verbena, arabica, Windows (Visual Studio 7) Solution: Platforms tested: Misc. update:
* [svn-r8731] Purpose:Quincey Koziol2004-06-231-7/+5
| | | | | | | | | | | | | | | Code cleanup & minor optimization Description: Re-work the way interface initialization routines are specified in the library to avoid the overhead of checking for them in routines where there is no interface initialization routine. This cleans up warnings with gcc 3.4, reduces the library binary size a bit (about 2-3%) and should speedup the library's execution slightly. Platforms tested: FreeBSD 4.10 (sleipnir) w/gcc34 h5committest
* [svn-r8707] Changed the way HDF5 handles hid_t's and added API functions to ↵James Laird2004-06-181-3/+3
| | | | allow users to register IDs and ID types at runtime.
* [svn-r8664] Purpose:Quincey Koziol2004-06-121-0/+158
| | | | | | | | | | | | | | Code optimization Description: Allow global heap collections to grow in size (up to a 64K limit) if they are able to. This allows them to grow to a more reasonable size than the 4K minimum size. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8630] Purpose:Quincey Koziol2004-06-091-1/+0
| | | | | | | | | | | | | | | | | Bug fix Description: Correct potential file corruption from mis-setting the metadata accumulator address. (Bugfix contributed by Sean Monner <seanm@quadrus.com>) Solution: When the new metadata overlaps with the end of the metadata accumulator and the accumulator grows in size, don't reset the address of the accumulator, since it hasn't moved. Platforms tested: FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r8391] Purpose:John Mainzer2004-04-191-43/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Checkpoint checkin of FP bug fixes. FP is still quite buggy, but I must go deal with other matters. Description: Fixed two major bugs: 1) H5FPserver.c was clobbering meta data in its care. 2) H5FPserver.c was allocating the same space multiple times, causing both data and meta data corruption. Also made minor fixes, added debugging code, and familiarized myself with the FP code. All development work with FP enabled was done on Eirene. On this platform, FP now passes its test reliably with up to 9 processes. At 10 processes it seg faults every time. I haven't looked into this issue. There are also several known locking bugs which have to be fixed. However, they are of sufficiently low probability that I didn't bother with them on this pass. FP has not been tested with deletions -- this should be done. Also, need to test FP chunked I/O. Solution: 1) Modified cache in H5FPserver.c to merge changes correctly. Found and fixed a bug in H5TB.c in passing. 2) Multiple space allocation was caused by a race condition with set eoa requests. Most of these eoa requests appeared to be superfluous, so I deleted them. Those issued during the superblock read seemed necessary, so I inserted a barrier at the end of the superblock read, to prevent races with allocations. Platforms tested: h5committested
* [svn-r8142] Purpose:Quincey Koziol2004-02-021-1/+10
| | | | | | | | | | | Fix build with C++ compiler Description: Add file driver header files for prototypes. Platforms tested: FreeBSD 4.9 (sleipnir) w/CC=g++ too obscure to require h5committest
* [svn-r8134] Purpose:Quincey Koziol2004-01-311-0/+26
| | | | | | | | | | | | | | | | Code cleanup Description: Add destructor to match constructor fr VFLs when they are shut down by the library. Solution: Added H5FD_*_term() routines to "undo" changes made in H5FD_*_init() routines. Platforms tested: IBM p690 (copper) too minor to require h5committest
* [svn-r8126] Purpose:Quincey Koziol2004-01-311-12/+64
| | | | | | | | | | | | | | | | | Bug fix/optimization Description: Address slowdown in MPI-I/O file metadata operations that was introduced mid-stream. We now _require_ a POSIX compliant parallel file system for the MPI-I/O file driver (as well as for the MPI-POSIX file driver). Also optimized file open operation when the file is being created by reducing the number of collective & syncronizing calls. Additionally, refactor the MPI routines into a common place, eliminating duplicated code. Platforms tested: FreeBSD 4.9 (sleipnir) w/parallel h5committest
* [svn-r7917] Purpose:Quincey Koziol2003-12-061-7/+7
| | | | | | | | | | | | | | | | | | | Code cleanup Description: Clean up compiler warnings, especially the 'FUNC' variable not used which comes out in production mode. Solution: Had to add a new FUNC_ENTER_NOAPI_NOINIT_NOFUNC macro for those non-API functions which don't need the 'FUNC' variable defined. (This will be _so_ much easier when C99 is standard on all our supposed platforms, since it has a __FUNC__ macro... ) Platforms tested: FreeBSD 4.9 (sleipnir) too minor for h5committest (although there were lots of files changed, the change was minor in each one)
* [svn-r7818] Purpose:Quincey Koziol2003-11-061-2/+2
| | | | | | | | | | | Warning cleanup Description: Make a parameter 'const' to clean up a warning during compiles. Platforms tested: FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r7725] Purpose:Quincey Koziol2003-10-241-79/+214
| | | | | | | | | | | | Code cleanup Description: Refactored handlier of VFL drivers in file access and data transfer property lists in order to simplify and unify the code dealing with them. Platforms tested: FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r7559] Purpose:Quincey Koziol2003-10-071-0/+47
| | | | | | | | | | | | | | Add feature Description: Add H5Fget_freespace() routine, to check the amount of free space in a file. This information is only valid until the file is closed currently, however (until we start recording the free space information in the file itself). Platforms tested: FreeBSD 4.9 (sleipnir) h5committest
* [svn-r7489] Purpose:Bill Wendling2003-09-181-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Update Description: A lot of modifications for the FPHDF5 stuff: H5AC.c H5ACprivate.h - Removed AC_find (it's replaced with AC_protect and AC_unprotect). Added flushing if it's an FPHDF5 driver and we're doing an AC_set or AC_unprotect with the dirty flag set. H5B.c - Split up the B_flush function into different functions since the one function was doing serialization which is better left as a separate entity. H5D.c - Removed some FPHDF5 code that was incorrect H5F.c - Split up the F_flush function so that it no longer allocates file space. Created new functions (F_init_superblock, F_read_superblock, and F_write_superblock) for greater modularity and so that the FPHDF5 non-captain processes can read the superblock after the captain process writes it. H5FD.c - Error message correction. H5FDfphdf5.c - Removed MPI barrier call that wasn't needed. H5FPclient.c H5FPserver.c - Modified so that if a process requests data that isn't exactly aligned, we can return it if we have the block that contains the requested address. H5G.c H5Gent.c H5Gnode.c H5HL.c H5HLpkg.h H5HLprivate.h H5Oefl.c - Removed the H5HL_peek function since it was doing a (now unsafe) holding of the information in the cache. Replaced with protect and unprotect calls. H5TB.c - Error fix. The TB_dless function wasn't working properly. H5Gstab.c - Format change. H5err.txt H5Edefin.h H5Einit.h H5Epubgen.h H5Eterm.h - Added new error code. Platforms tested: Modi4 (paralle, Fortran) Sol (Fortran) Linux (C++, Fortran) Copper (Parallel, Fortran) Misc. update:
* [svn-r7386] Purpose:Quincey Koziol2003-08-201-9/+9
| | | | | | | | | | | | | | | Code cleanup & bug fix Description: More de-linting... Also correct a bug that lint exposed which was incrementing the incorrect driver ID of a file driver and also no saving the correct driver ID for the newly opened file. Platforms tested: FreeBSD 4.8 (sleipnir) too minor for h5committest
* [svn-r7384] Purpose:Quincey Koziol2003-08-191-284/+272
| | | | | | | | | | | Code cleanup Description: More linting... Platforms tested: FreeBSD 4.8 (sleipnir) too minor to need h5committest
* [svn-r7372] Purpose:Quincey Koziol2003-08-151-2/+4
| | | | | | | | Added some comments, etc. Platforms tested: FreeBSD 4.8 (sleipnir) too trivial for h5committest
* [svn-r7367] Purpose:Bill Wendling2003-08-151-4/+27
| | | | | | | | | | | | | | | | | | | Update Description: Added an extra flag to the clear functions that triggers a "destroy" of the object being cleared if necessary. This is a fix for the FPHDF5 stuff which had an object sticking around after it was cleared. (In FPHDF5, some processes are in charge of destroying the object, but all processes might actually allocate the object.) Platforms tested: AIX (Copper: Fortran & C++) Linux (Verbena: Fortran & C++) IRIX (Modi4: Parallel & Fortran) (Sol is down) Misc. update:
* [svn-r7317] Purpose:Quincey Koziol2003-08-081-14/+12
| | | | | | | | | | | | | | | | Feature change & code cleanup Description: Changed fileno field in H5FD_t struct from 'unsigned long[2]' to just 'unsigned long'. Changed over code to handle the fileno change. Chased error API changes Platforms tested: h5committested
* [svn-r7052] Purpose:Quincey Koziol2003-06-181-1/+1
| | | | | | | | | | | Code cleanup Description: Correct name of H5FD_term_interface function in FUNC_ENTER macro. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest
* [svn-r6649] Purpose:Quincey Koziol2003-04-141-1/+3
| | | | | | | | | | | | | | | | | | Bug fix. Description: Corrected fencepost error which was incorrectly checking for merging metadata at the front of the metadata accumulator. This now allows the metadata cache to cache I/O operations more efficiently. Platforms tested: FreeBSD 4.8 (sleipnir) w/C++ Linux 2.4 (burrwhite) w/FORTRAN Solaris 2.7 (arabica) w/FORTRAN IRIX64 6.5 (modi4) w/parallel & FORTRAN (h5committest not run due to my ongoing difficulties with C++ on burrwhite).
* [svn-r6632] Purpose:Quincey Koziol2003-04-111-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Bug fix Description: This fixes a bug in the low-level metadata caching code in the library which could possibly lose metadata during file I/O when a lot of objects are inserted into a group. This also fixes a couple of (similar) fencepost bugs in the B-tree deletion code. Solution: For the metadata bug - call the low-level driver's 'write' routine instead of H5FD_write. For the B-tree bug - include the correct number of keys. Platforms tested: FreeBSD 4.8 (sleipnir) w/C++ Linux 2.4 (burrwhite) w/FORTRAN Solaris 2.7 (arabica) w/FORTRAN IRIX64 6.5 (modi4) w/FORTRAN & parallel (h5committest is still not working for me on burrwhite) Misc. update:
* [svn-r6556] Purpose:Bill Wendling2003-04-011-21/+0
| | | | | | | | | | | | | | | | | | | Update & Bug Fix Description: The "free" protocol was missing. Added that to the server side. When doing a "create" of a file (with no other data structures created), the freespace in the file wasn't being reclaimed. Solution: After adding the free protocol, we put the burden of running through the FD_free function on the SAP instead of each client. Platforms tested: Linux Misc. update:
* [svn-r6527] Purpose:Bill Wendling2003-03-271-5/+11
| | | | | | | | | | | | | | | | | | | | | | | Bug Fix & Update Description: FPHDF5 was creating files which didn't have the EOA field in the superblock set correctly. It turns out that the SAP was keeping this information to itself instead of giving it to the client processes. Naughty SAP! Solution: Have the SAP send this information back to the clients so that they can update the superblock as necessary. This now creates a file (with just the root group) that looks correct! Only problem is that there's extra file space being allocated. Also, at program termination, there's an infinite loop... Platforms tested: H5committests (run by hand on burrwhite, arabica, and modi4) Misc. update:
* [svn-r6514] Purpose:Bill Wendling2003-03-201-11/+0
| | | | | | | | | | | | | | | | | | | | | | | | | Bug fix and Update Description: From Quincey's comments on the code I checked in last night: - In H5F_close call, the "private" processes should call the H5F_flush with the "CLEAR_ONLY" flag. - There's no need for a special case for FPHDF5 in the FD_real_alloc function since FPHDF5 doesn't define an alloc function. - The return type of H5Pset_fapl_fphdf5 should be herr_t instead of hid_t. I don't know how it got that way in the first place. - The variable names for MPI types and the structure typedefs should be switched: H5FP_request/H5FP_request_t to H5FP_request_t/H5FP_request and so on. - In the H5FP.c module, I was commiting the H5FP_request MPI datatype but using the wrong offset field... Platforms tested: Linux...will test on others, but these are mostly FPHDF5 changes. Misc. update:
* [svn-r6506] Purpose:Bill Wendling2003-03-191-7/+43
| | | | | | | | | | | | | Update Description: When doing an allocation, check to see, if it's an FPHDF5 driver, if only the captain should be performing the allocation Platforms tested: Linux, Modi4, Sol Misc. update:
* [svn-r6497] Purpose:Quincey Koziol2003-03-191-11/+4
| | | | | | | | | | | | | | | | | | | Finish code cleanup Description: Wrap up the conversion of H5F_flush's multiple boolean flags into a single bitfield of flags by pushing the flags down into the H5AC_flush and H5F_istore_flush routines. Also, changed the flags from H5_FLUSH_<foo> to H5F_FLUSH_<foo> to be more consistent with rest of library. And reverted the changes to H5FDflush and H5FD_flush routines. Platforms tested: FreeBSD 4.7 (sleipnir) Solaris 5.8 (sol) IRIX64 6.5 (modi4) w/parallel Misc. update:
* [svn-r6496] Purpose:Bill Wendling2003-03-191-4/+11
| | | | | | | | | | | | | | | | | | | | | | Refactoring of Flush Logic Description: The Flushing logic passed in multiple flags to indicate what type of flush it was: closing, invalidate, alloc only. This made extending the function of the flush logic to handle other flags annoying. Solution: I changed it to be just one bitmasked flag to indicate what type of flushing to do. I also added the CLEAR_ONLY flag, which will be used in the FPHDF5 stuff. Platforms tested: h5committest doesn't work for me (my environment isn't setup on the other machines I guess). I tested it manually: Linux parallel & C++ Sol Fortran Modi4 parallel & Fortran.
* [svn-r6425] Purpose:Bill Wendling2003-02-211-387/+707
| | | | | | | | | | | | | | | | Update Description: Big change to the H5FD.c module: - Split apart H5FD_alloc and H5FD_free. H5FD_alloc was huge and H5FD_free had a freeing of the freelist part which I needed to call from the SAP. - Added support for FPHDF5. If it's a client, then it sends the allocation or free request to the SAP. The SAP will call the same code, but it'll actually do the allocation/freeing in that case. Platforms tested: Linux & Modi4
* [svn-r6400] Raymond Lu2003-02-121-4/+4
| | | | | | | | | | | Purpose: Change feature Description: Switch to Fletcher32 from Adler32 checksum Platforms tested: arabica, eirene, modi4 Misc. update: MANIFEST and RELEASE.txt updated.
* [svn-r6387] Purpose:Quincey Koziol2003-02-101-54/+78
| | | | | | | | | | | | | | | | | | | | | | | | | | | Bug Fix Description: Metadata cache in parallel I/O can cause hangs in applications which perform independent I/O on chunked datasets, because the metadata cache can attempt to flush out dirty metadata from only a single process, instead of collectively from all processes. Solution: Pass a dataset transfer property list down from every API function which could possibly trigger metadata I/O. Then, split the metadata cache into two sets of entries to allow dirty metadata to be set aside when a hash table collision occurs during independent I/O. Platforms tested: Tested h5committest {arabica (fortran), eirene (fortran, C++) modi4 (parallel, fortran)} FreeBSD 4.7 (sleipnir) serial & parallel Misc. update: Updated release_docs/RELEASE
* [svn-r6296] Purpose:Quincey Koziol2003-01-171-3/+3
| | | | | | | | | | Code cleanup Description: Reduce warnings on Windows Platforms tested: FreeBSD 4.7 (sleipnir)
* [svn-r6266] Purpose:Quincey Koziol2003-01-101-44/+44
| | | | | | | | | | | | Code cleanup/new feature. Description: Split FUNC_LEAVE into API and non-API specific versions. This allows a solution to compiling this branch with C++, as well as reducing the size of the binaries produced. Platforms tested: FreeBSD 4.7 (sleipnir) w/serial, parallel (including MPE) & thread-safe
* [svn-r6252] Purpose:Quincey Koziol2003-01-091-16/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Lots of performance improvements & a couple new internal API interfaces. Description: Performance Improvements: - Cached file offset & length sizes in shared file struct, to avoid constantly looking them up in the FCPL. - Generic property improvements: - Added "revision" number to generic property classes to speed up comparisons. - Changed method of storing properties from using a hash-table to the TBBT routines in the library. - Share the propery names between classes and the lists derived from them. - Removed redundant 'def_value' buffer from each property. - Switching code to use a "copy on write" strategy for properties in each list, where the properties in each list are shared with the properties in the class, until a property's value is changed in a list. - Fixed error in layout code which was allocating too many buffers. - Redefined public macros of the form (H5open()/H5check, <variable>) internally to only be (<variable>), avoiding innumerable useless calls to H5open() and H5check_version(). - Reuse already zeroed buffers in H5F_contig_fill instead of constantly re-zeroing them. - Don't write fill values if writing entire dataset. - Use gettimeofday() system call instead of time() system when checking the modification time of a dataset. - Added reference counted string API and use it for tracking the names of objects opening in a file (for the ID->name code). - Removed redundant H5P_get() calls in B-tree routines. - Redefine H5T datatype macros internally to the library, to avoid calling H5check redundantly. - Keep dataspace information for dataset locally instead of reading from disk each time. Added new module to track open objects in a file, to allow this (which will be useful eventually for some FPH5 metadata caching issues). - Remove H5AC_find macro which was inlining metadata cache lookups, and call function instead. - Remove redundant memset() calls from H5G_namei() routine. - Remove redundant checking of object type when locating objects in metadata cache and rely on the address only. - Create default dataset object to use when default dataset creation property list is used to create datasets, bypassing querying for all the property list values. - Use default I/O vector size when performing raw data with the default dataset transfer property list, instead of querying for I/O vector size. - Remove H5P_DEFAULT internally to the library, replacing it with more specific default property list based on the type of property list needed. - Remove redundant memset() calls in object header message (H5O*) routines. - Remove redunant memset() calls in data I/O routines. - Split free-list allocation routines into malloc() and calloc()- like routines, instead of one combined routine. - Remove lots of indirection in H5O*() routines. - Simplify metadata cache entry comparison routine (used when flushing entire cache out). - Only enable metadata cache statistics when H5AC_DEBUG is turned on, instead of always tracking them. - Simplify address comparison macro (H5F_addr_eq). - Remove redundant metadata cache entry protections during dataset creation by protecting the object header once and making all the modifications necessary for the dataset creation before unprotecting it. - Reduce # of "number of element in extent" computations performed by computing and storing the value during dataspace creation. - Simplify checking for group location's file information, when file has not been involving in file-mounting operations. - Use binary encoding for modification time, instead of ASCII. - Hoist H5HL_peek calls (to get information in a local heap) out of loops in many group routine. - Use static variable for iterators of selections, instead of dynamically allocation them each time. - Lookup & insert new entries in one step, avoiding traversing group's B-tree twice. - Fixed memory leak in H5Gget_objname_idx() routine (tangential to performance improvements, but fixed along the way). - Use free-list for reference counted strings. - Don't bother copying object names into cached group entries, since they are re-created when an object is opened. The benchmark I used to measure these results created several thousand small (2K) datasets in a file and wrote out the data for them. This is Elena's "regular.c" benchmark. These changes resulted in approximately ~4.3x speedup of the development branch when compared to the previous code in the development branch and ~1.4x speedup compared to the release branch. Additionally, these changes reduce the total memory used (code and data) by the development branch by ~800KB, bringing the development branch back into the same ballpark as the release branch. I'll send out a more detailed description of the benchmark results as a followup note. New internal API routines: Added "reference counted strings" API for tracking strings that get used by multiple owners without duplicating the strings. Added "ternary search tree" API for text->object mappings. Platforms tested: Tested h5committest {arabica (fortran), eirene (fortran, C++) modi4 (parallel, fortran)} Other platforms/configurations tested? FreeBSD 4.7 (sleipnir) serial & parallel Solaris 2.6 (baldric) serial
generated by cgit v0.12 at 2024-12-27 11:11:09 (GMT)