summaryrefslogtreecommitdiffstats
path: root/src/H5D.c
Commit message (Collapse)AuthorAgeFilesLines
* [svn-r7453] Purpose:Quincey Koziol2003-09-101-4/+2
| | | | | | | Checkpoint file format spec. Description: Clarifications and cleanups related to file format review.
* [svn-r7384] Purpose:Quincey Koziol2003-08-191-22/+28
| | | | | | | | | | | Code cleanup Description: More linting... Platforms tested: FreeBSD 4.8 (sleipnir) too minor to need h5committest
* [svn-r7373] Purpose:Quincey Koziol2003-08-151-1/+1
| | | | | | | | | | | | Code cleanup Description: Cleanup H5Pclose code and also switch a few internal users of H5Pclose to use H5I_dec_ref instead. Platforms tested: FreeBSD 4.8 (sleipnir) too small for h5committest
* [svn-r7370] Purpose:Quincey Koziol2003-08-151-1/+21
| | | | | | | | | | | | | Code cleanup & commentary Description: Add comments to the recent FPHDF5 additions to this file to describe why the changes shouldn't be in this file and attempt to find the proper home for them. Platforms tested: FreeBSD 4.8 (sleipnir) just comments mostly, no need to run h5committest
* [svn-r7367] Purpose:Bill Wendling2003-08-151-0/+74
| | | | | | | | | | | | | | | | | | | Update Description: Added an extra flag to the clear functions that triggers a "destroy" of the object being cleared if necessary. This is a fix for the FPHDF5 stuff which had an object sticking around after it was cleared. (In FPHDF5, some processes are in charge of destroying the object, but all processes might actually allocate the object.) Platforms tested: AIX (Copper: Fortran & C++) Linux (Verbena: Fortran & C++) IRIX (Modi4: Parallel & Fortran) (Sol is down) Misc. update:
* [svn-r7334] Purpose:Quincey Koziol2003-08-111-6/+6
| | | | | | | | | | | | | | Bug fix Description: Object references were not getting written out correctly to the file, with recent changes to their memory structure. Solution: Convert the object references correctly. Platforms tested: h5committested
* [svn-r7297] Purpose:Quincey Koziol2003-08-081-4/+4
| | | | | | | | | | | | | | | | Code cleanup Description: Changed lines like: H5E_clear(H5E_get_my_stack()); to: H5E_clear(NULL); which performs the same operation. Platforms tested: h5committested
* [svn-r7265] *** empty log message ***Raymond Lu2003-07-261-5/+5
|
* [svn-r7243] Purpose:Quincey Koziol2003-07-211-6/+19
| | | | | | | | | | | | Bug fix Description: Fix bug with combination of fill-values, chunked datasets and variable-length strings. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest
* [svn-r7211] Purpose:Quincey Koziol2003-07-111-433/+483
| | | | | | | | | | | Code cleanup Description: Various code cleanups suggested by lint tool Platforms tested: FreeBSD 4.8 (sleipnir) h5committest
* [svn-r7181] Purpose:Quincey Koziol2003-07-071-14/+0
| | | | | | | | | | | Version update Description: Removed 1.4 compatibility code in the library. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest
* [svn-r7116] Purpose:Quincey Koziol2003-06-271-3/+4
| | | | | | | | | | | | Update code Description: Move "PABLO_MASK" definition above header file inclusion to avoid problems with inline functions in header files. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest
* [svn-r7058] Purpose:Quincey Koziol2003-06-181-22/+25
| | | | | | | | | | | | | | Bug fix and refactored code Description: Return the file space we have allocated already when a dataset creation fails. Also track the change to the H5Fget_obj_<foo> APIs Platforms tested: FreeBSD 4.8 (sleipnir) h5committest
* [svn-r7037] Purpose:Quincey Koziol2003-06-131-1/+4
| | | | | | | | | | | | | | | | | | Bug fix Description: The fill time in a dataset with no fill value information created with an older version of the library was getting set to H5D_FILL_TIME_ALLOC instead of the new default H5D_FILL_TIME_IFSET and was causing H5Dcreate() calls with that dataset creation property list to fail now. Solution: Set the new default in the fill time initialization for missing fill value information. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest
* [svn-r7029] Purpose:Quincey Koziol2003-06-121-20/+52
| | | | | | | | | | | | | New feature/Bug fix Description: Add new fill time value - H5D_FILL_TIME_IFSET which writes the fill value to a dataset if the user has defined one, otherwise not writing the fill value to the dataset. Platforms tested: FreeBSD 4.8 (sleipnir) serial & parallel h5committest
* [svn-r7016] Purpose:Quincey Koziol2003-06-101-17/+30
| | | | | | | | | | | | | | | | Bug fix Description: Chunked datasets with early space allocation and unlimited dimensions were running into problems where the dataset's "layout" message was marked as constant too early, preventing the dataset's dimensions from being extended. Solution: Change logic for marking the layout message constant to wait a bit longer. Platforms tested: FreeBSD 4.8 (sleipnir) w/parallel h5committest
* [svn-r6878] Purpose:Quincey Koziol2003-05-151-3/+3
| | | | | | | | | | | Code cleanup Description: Limit the scope on more function prototypes/macros/typedefs. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest not necessary.
* [svn-r6869] Purpose:Quincey Koziol2003-05-131-1/+23
| | | | | | | | | | | | | | | | | Bug fix. Description: When the same dataset is opened more than once and those references to the object are left dangling, the library would not correctly shut down the dataset API, potentially allowing the file to be truncated. Solution: "force" the dataset IDs to be released when the dataset API is being shut down by the library. Platforms tested: FreeBSD 4.8 (sleipnir) h5committested
* [svn-r6837] Purpose:Quincey Koziol2003-05-081-4/+15
| | | | | | | | | | | | | | | Code cleanup. Description: Move many package or internal function prototypes and macro definitions into tighter scope according to their current use. Added more comments where appropriate. Eliminate ancient, unused functions. Added a couple "accessor" functions to get parts of data structures which were moved out of scope. Platforms tested: h5committested
* [svn-r6825] Purpose:Quincey Koziol2003-05-071-1208/+7
| | | | | | | | | | | | | | | | | | | | | | | | | New feature/enhancement Description: Chunked datasets are handled poorly in several circumstances involving certain selections and chunks that are too large for the chunk cache and/or chunks with filters, causing the chunk to be read from disk multiple times. Solution: Rearrange raw data I/O infrastructure to handle chunked datasets in a much more friendly way by creating a selection in memory and on disk for each chunk in a chunked dataset and performing all of the I/O on that chunk at one time. There are still some scalability (the current code attempts to create a selection for all the chunks in the dataset, instead of just the chunks that are accessed, requiring portions of the istore.c and fillval.c tests to be commented out) and performance issues, but checking this in will allow the changes to be tested by a much wider audience while I address the remaining issues. Platforms tested: h5committested, FreeBSD 4.8 (sleipnir) serial & parallel, Linux 2.4 (eirene)
* [svn-r6738] Purpose:Quincey Koziol2003-04-231-1/+2
| | | | | | | | | | | | Code cleanup Description: Add & improve assertion that was removed in last commit, to check for compact dataset storage. Platforms tested: FreeBSD 4.8 (sleipnir) triple check not necessary.
* [svn-r6729] Raymond Lu2003-04-221-3/+2
| | | | | | | | | | | Purpose: bug fix Description: VL datatype in compact dataset fails because a wrong type size is used. Platforms tested: h5committested Misc. update:
* [svn-r6659] Purpose:Quincey Koziol2003-04-141-3/+4
| | | | | | | | | | | | | | | | New feature, sorta Description: Track changes to various internal APIs Platforms tested: FreeBSD 4.8 (sleipnir) w/C++ Linux 2.4 (burrwhite) w/FORTRAN Solaris 2.7 (arabica) w/FORTRAN IRIX64 6.5 (modi4) w/parallel & FORTRAN (h5committest not run due to my ongoing difficulties with C++ on burrwhite).
* [svn-r6604] Purpose:Quincey Koziol2003-04-091-30/+45
| | | | | | | | | | | | | | | | | | New feature Description: Hooked up internal dataset creation call (H5D_create) to the new "can apply" and "set local" callbacks for filters. Solution: Platforms tested: FreeBSD 4.8 (sleipnir) w/szip Linux 2.4 (sleipnir) w/szip Solaris 2.7 (arabica) w/FORTRAN IRIX64 6.5 (modi4) w/szip, FORTRAN & parallel Misc. update:
* [svn-r6499] Purpose:Bill Wendling2003-03-191-17/+18
| | | | | | | | | | | | | Update Description: Added support for FPHDF5 wherever there was support for MPI/IO and/or MPIPOSIX. Platforms tested: Linux, Modi4, Sol Misc. update:
* [svn-r6411] Purpose:Quincey Koziol2003-02-171-42/+42
| | | | | | | | | | | | | | | | | | | | | | Code cleanup Description: Clean up miscellaneous warnings which have crept into the code. Fix "_POSIX_C_SOURCE not defined" warning on FreeBSD. Adjust gcc compiler flags to be more concise for production mode. Refactor the H5O code so that there is a stronger boundary between code in the H5O package and code in the library which just calls H5O routines. Platforms tested: Tested h5committest {arabica (fortran), eirene (fortran, C++) modi4 (parallel, fortran)} FreeBSD 4.7 (sleipnir) serial & parallel and gcc 2.95.4 & gcc 3.2.2 Misc. update: Update MANIFEST if you add or remove any file.
* [svn-r6388] Purpose:Quincey Koziol2003-02-101-2/+2
| | | | | | | | | | | | | Update feature Description: Relax collective constraint for API functions which only read metadata from a file. Platforms tested: Tested h5committest {arabica (fortran), eirene (fortran, C++) modi4 (parallel, fortran)} FreeBSD 4.7 w/parallel
* [svn-r6387] Purpose:Quincey Koziol2003-02-101-132/+126
| | | | | | | | | | | | | | | | | | | | | | | | | | | Bug Fix Description: Metadata cache in parallel I/O can cause hangs in applications which perform independent I/O on chunked datasets, because the metadata cache can attempt to flush out dirty metadata from only a single process, instead of collectively from all processes. Solution: Pass a dataset transfer property list down from every API function which could possibly trigger metadata I/O. Then, split the metadata cache into two sets of entries to allow dirty metadata to be set aside when a hash table collision occurs during independent I/O. Platforms tested: Tested h5committest {arabica (fortran), eirene (fortran, C++) modi4 (parallel, fortran)} FreeBSD 4.7 (sleipnir) serial & parallel Misc. update: Updated release_docs/RELEASE
* [svn-r6375] Raymond Lu2003-02-041-0/+12
| | | | | | | | | | | Purpose: New feature Description: Added Adler32 checksum as a filter in pipeline Platforms tested: arabica (fortran), eirene (, C++), modi4 (parallel, fortran) Misc. update: Update release_docs/RELEASE.
* [svn-r6308] Purpose:Quincey Koziol2003-01-211-0/+8
| | | | | | | | | | | | | | | | | Bug fix Description: Currently, when the library encounters an object header message that isn't know, it fails to open that object in the file. Solution: Allow the library to skip over the unknown object header message and continue to process the remaining messages, in the hope that the skipped message isn't important later. If it is important, it will be caught at a higher level of the library. Platforms tested: FreeBSD 4.7 (sleipnir)
* [svn-r6296] Purpose:Quincey Koziol2003-01-171-15/+21
| | | | | | | | | | Code cleanup Description: Reduce warnings on Windows Platforms tested: FreeBSD 4.7 (sleipnir)
* [svn-r6266] Purpose:Quincey Koziol2003-01-101-49/+49
| | | | | | | | | | | | Code cleanup/new feature. Description: Split FUNC_LEAVE into API and non-API specific versions. This allows a solution to compiling this branch with C++, as well as reducing the size of the binaries produced. Platforms tested: FreeBSD 4.7 (sleipnir) w/serial, parallel (including MPE) & thread-safe
* [svn-r6255] Purpose:Quincey Koziol2003-01-091-5/+5
| | | | | | | | | | Code cleanup Description: Clean up a few more warnings and update dependencies. Platforms tested: Linux 2.2.18smp (eirene) serial & parallel
* [svn-r6252] Purpose:Quincey Koziol2003-01-091-549/+547
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Lots of performance improvements & a couple new internal API interfaces. Description: Performance Improvements: - Cached file offset & length sizes in shared file struct, to avoid constantly looking them up in the FCPL. - Generic property improvements: - Added "revision" number to generic property classes to speed up comparisons. - Changed method of storing properties from using a hash-table to the TBBT routines in the library. - Share the propery names between classes and the lists derived from them. - Removed redundant 'def_value' buffer from each property. - Switching code to use a "copy on write" strategy for properties in each list, where the properties in each list are shared with the properties in the class, until a property's value is changed in a list. - Fixed error in layout code which was allocating too many buffers. - Redefined public macros of the form (H5open()/H5check, <variable>) internally to only be (<variable>), avoiding innumerable useless calls to H5open() and H5check_version(). - Reuse already zeroed buffers in H5F_contig_fill instead of constantly re-zeroing them. - Don't write fill values if writing entire dataset. - Use gettimeofday() system call instead of time() system when checking the modification time of a dataset. - Added reference counted string API and use it for tracking the names of objects opening in a file (for the ID->name code). - Removed redundant H5P_get() calls in B-tree routines. - Redefine H5T datatype macros internally to the library, to avoid calling H5check redundantly. - Keep dataspace information for dataset locally instead of reading from disk each time. Added new module to track open objects in a file, to allow this (which will be useful eventually for some FPH5 metadata caching issues). - Remove H5AC_find macro which was inlining metadata cache lookups, and call function instead. - Remove redundant memset() calls from H5G_namei() routine. - Remove redundant checking of object type when locating objects in metadata cache and rely on the address only. - Create default dataset object to use when default dataset creation property list is used to create datasets, bypassing querying for all the property list values. - Use default I/O vector size when performing raw data with the default dataset transfer property list, instead of querying for I/O vector size. - Remove H5P_DEFAULT internally to the library, replacing it with more specific default property list based on the type of property list needed. - Remove redundant memset() calls in object header message (H5O*) routines. - Remove redunant memset() calls in data I/O routines. - Split free-list allocation routines into malloc() and calloc()- like routines, instead of one combined routine. - Remove lots of indirection in H5O*() routines. - Simplify metadata cache entry comparison routine (used when flushing entire cache out). - Only enable metadata cache statistics when H5AC_DEBUG is turned on, instead of always tracking them. - Simplify address comparison macro (H5F_addr_eq). - Remove redundant metadata cache entry protections during dataset creation by protecting the object header once and making all the modifications necessary for the dataset creation before unprotecting it. - Reduce # of "number of element in extent" computations performed by computing and storing the value during dataspace creation. - Simplify checking for group location's file information, when file has not been involving in file-mounting operations. - Use binary encoding for modification time, instead of ASCII. - Hoist H5HL_peek calls (to get information in a local heap) out of loops in many group routine. - Use static variable for iterators of selections, instead of dynamically allocation them each time. - Lookup & insert new entries in one step, avoiding traversing group's B-tree twice. - Fixed memory leak in H5Gget_objname_idx() routine (tangential to performance improvements, but fixed along the way). - Use free-list for reference counted strings. - Don't bother copying object names into cached group entries, since they are re-created when an object is opened. The benchmark I used to measure these results created several thousand small (2K) datasets in a file and wrote out the data for them. This is Elena's "regular.c" benchmark. These changes resulted in approximately ~4.3x speedup of the development branch when compared to the previous code in the development branch and ~1.4x speedup compared to the release branch. Additionally, these changes reduce the total memory used (code and data) by the development branch by ~800KB, bringing the development branch back into the same ballpark as the release branch. I'll send out a more detailed description of the benchmark results as a followup note. New internal API routines: Added "reference counted strings" API for tracking strings that get used by multiple owners without duplicating the strings. Added "ternary search tree" API for text->object mappings. Platforms tested: Tested h5committest {arabica (fortran), eirene (fortran, C++) modi4 (parallel, fortran)} Other platforms/configurations tested? FreeBSD 4.7 (sleipnir) serial & parallel Solaris 2.6 (baldric) serial
* [svn-r6145] Raymond Lu2002-12-021-1/+11
| | | | | | | | | | Purpose: New feature to H5Dget_offset Description: If user block is set, H5Dget_offset should be able to return the absolute offset from the beginning of file. Platforms tested: eirene, arabica
* [svn-r6072] Purpose:Bill Wendling2002-11-111-11/+1
| | | | | | | | | | Code Movement Description: Moved definition of the H5D_t data structure into the H5Dprivate.h header file. With the FPHDF5 stuff, we need access to this data structure. Platforms tested: Eirene...change small, so no need for all three tests.
* [svn-r6063] Purpose:Quincey Koziol2002-11-071-5/+7
| | | | | | | | | | | Code cleanup Description: Cleaned up some error values that were incorrect, added some assertions and refactored the integer & floating-point native conversions some. Platforms tested: FreeBSD 4.7 (sleipnir), too small for h5committest
* [svn-r6062] Raymond Lu2002-11-071-0/+82
| | | | | | | | | | | | Purpose: New function. Description: H5Dget_offset returns the offset of a dataset's data relative to the beginning of a file. Platforms tested: arabica(simple function, one test should be enough.) Misc. update: RELEASE.txt
* [svn-r6057] Purpose:Bill Wendling2002-11-051-99/+244
| | | | | | | | | | | | | | | | | | | | | | | New Feature. Support for new FPHDF5 Feature Description: More support for the FPHDF5 feature. This splits the H5D_create function into separate parts which update the metadata cache. It was necessary to split apart the H5O_create function to do a similar thing since it would allocate real space on the hard disk. I'm checking this up now so that I don't get too far away from what the CVS repository has and so that others may look at the code and retch^H^H^H^H^Hmarvel at it. If there are any comments, send them my way. There is one outstanding issue, though. I need to use non-API functions in the FPHDF5 stuff. I'm using some APIs right now and should migrate to using others. At the moment, I'm putting that on the back burner until I'm more along in the implementation. Platforms tested: Arabica (C++) Eirene (Parallel) Modi4 (Parallel)
* [svn-r6030] Purpose:Bill Wendling2002-10-231-403/+11
| | | | | | | | | | Another Oops Description: I accidentally checked in the H5D.c module when I shouldn't have. Solution: Rolled back to 1.281 version Platforms tested: Linux
* [svn-r6029] Purpose:Bill Wendling2002-10-231-11/+403
| | | | | | | | | | | | Oops Description: The H5FPprivate.h file was being #included even if PARALLEL was turned off. Solution: put the #include of the H5FPprivate.h file within the #ifdef block so that it doesn't get included if H5_HAVE_FPHDF5 isn't defined. Platforms tested: Linux...
* [svn-r5995] Purpose:Quincey Koziol2002-10-141-2/+2
| | | | | | | | | | | | | | | Code cleanup/Bug Fix Description: Re-do the ID->name code again, taking into account many more weird and wonderful special cases stumbled across during the last set of test writing. Platforms tested: FreeBSD 4.6 (sleipnir) w and w/o parallel Linux 2.2.x (eirene) w/FORTRAN & C++ Solaris 2.7 (arabica) w/FORTRAN IRIX64 6.5 (modi4) w/FORTRAN & parallel
* [svn-r5959] Raymond Lu2002-10-041-1/+4
| | | | | | | Purpose: Performance improvement for compact dataset. Platforms tested: Linux 2.2(eirene), Solaris 2.7(arabica), IRIX64 6.5(modi4)
* [svn-r5947] Purpose:Quincey Koziol2002-09-251-12/+6
| | | | | | | | | | | | | | | | Code cleanup Description: Clean up ID->name code: - Reformat to better match library coding standard - Changed several algorithms to be more efficient - Integrated into library more smoothly Platforms tested: eirene w/FORTRAN & C++ arabica w/FORTRAN modi4 w/FORTRAN & parallel sleipnir
* [svn-r5929] Pedro Vicente Nunes2002-09-181-0/+7
| | | | | | | | | | | | | | Purpose: bug fix Description: memory leak regarding the ID to name buffer Solution: added a new function H5G_free_ent_name that is called on several places of the library Platforms tested: windows 2000 linux, with cpp solaris, with fortran, cpp irix64, with parallel, fortran
* [svn-r5926] Purpose:Quincey Koziol2002-09-131-38/+38
| | | | | | | | | | | | | API name change Description: Change all "space time" references to "alloc time", including API functions and macro definitions, etc. Platforms tested: FreeBSD 4.6 (sleipnir) w/C++ Solaris 2.7 (arabica) w/FORTRAN IRIX64 6.5 (modi4) w/parallel & FORTRAN
* [svn-r5925] Purpose:Quincey Koziol2002-09-131-2/+5
| | | | | | | | | | | | Code cleanup Description: Eliminate memory leak of compact storage raw data buffer found by purify. Platforms tested: FreeBSD 4.6 (sleipnir) w/C++ Solaris 2.7 (arabica) w/C++ IRIX64 6.5 (modi4) w/parallel & FORTRAN
* [svn-r5910] Purpose:Bill Wendling2002-09-031-5/+5
| | | | | | | | | | Bug Fix Description: Some macros had "NULL" instead of "FAIL" as the return value. Solution: Change from NULL to FAIL. Platforms tested: Eirene (C++), Arabica (Fortran), Modi4 (pp and Fortran).
* [svn-r5904] Pedro Vicente Nunes2002-08-281-2/+7
| | | | | | | | | | | | | | | | | | | Purpose: Added 'ID to name' support Description: There is a new API function H5Iget_name Most of the changes are on H5G.c , regarding the symbol table entry struct H5G_entry_t which has 2 new fields 'name' and 'old_name' A new private function was introduced H5G_ent_copy, that does a deep copy between 2 symbol table entries The test file is getname.c Platforms tested: windows 2000, Linux, Solaris
* [svn-r5894] Purpose:Quincey Koziol2002-08-271-241/+327
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bug fix/Code cleanup/New Feature Description: Correct problems with writing fill-values to external storage and allocate the data storage at the correct times. Also, mostly straighten out the strange code which allocates and fills raw data storage for datasets. Things are still a bit odd in that the fill-values for chunked datasets are written when the space is allocated, instead of in a separate routine, but there are two reasons for this: it's inefficient (especially in parallel) to iterate through all the chunks twice, and (more importantly) the space needed to store compressed chunks isn't known until we've got a buffer of compressed fill-values ready to write to the chunk. Additionally, add in the H5D_SPACE_ALLOC_INCR and H5D_SPACE_ALLOC_DEFAULT setting for the "space time", which incorporate the previous behavior of the space allocation for chunked datasets. The default settings for the different types of dataset storage are now as follows: Contiguous - Late Chunked - Incremental Compact - Early This checkin also incorporates a change to the behavior of external data storage in two ways - fill-values are _never_ written to external storage (under the assumption that writing fill-values is triggered by allocating space in an HDF5 file, and since space is not allocated in the file, the fill-values should not be written) and external data files are now created if they don't exist when data is written to them. The fill-value will probably need to be revisited at some time in the future, this just seemed like the safer course currently. I think I cleaned up some compiler errors also, before getting bogged down in the fixes for the space allocation and fill-values. Platforms tested: FreeBSD 4.6 (sleipnir) w/serial & parallel. Will be testing on IRIX64 6.5 (modi4) in serial & parallel shortly.
generated by cgit v0.12 at 2024-10-16 02:19:30 (GMT)