summaryrefslogtreecommitdiffstats
path: root/src/H5D.c
Commit message (Collapse)AuthorAgeFilesLines
* [svn-r8802] Purpose:Quincey Koziol2004-07-031-36/+89
| | | | | | | | | | | | | | | Code optimization Description: Set up datatype ID for dataset's datatype on disk. This allows us to avoid repeatedly copying the datatype when an ID is needed. Also, clean up a few warnings in various other places. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8753] Nat Furrer2004-06-281-0/+53
| | | | | | | | | | | | | | | | | | | Purpose: Handled SZIP without the encoder present. Description: It is now an error for a user to try to create, extend, or write to a dataset without the encoder present in their SZIP library. Added H5Zget_filter_info to provide users with a way to query HDF5 about the presence (or lack thereof) of the SZIP encoder. Platforms tested: Windows Verbena Arabica Copper Misc. update:
* [svn-r8733] Purpose:Quincey Koziol2004-06-231-3/+3
| | | | | | | | | | | | Code cleanup Description: Reduce compiler warnings on SGI IRIX Platforms tested: FreeBSD 4.10 (sleipnir) SGI IRIX6 (Cheryl's machine) Too minor to require full h5committest
* [svn-r8693] Purpose:Quincey Koziol2004-06-151-4/+4
| | | | | | | | | | | | | Code optimization Description: Avoid making copy of default vlen allocation info when default DXPL is used. Just retarget pointer to point to default info directly. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8687] Purpose:Quincey Koziol2004-06-141-3/+4
| | | | | | | | | | | | | Code optimization Description: Eliminate memcpy() when using default DXPL by pointing at existing default object, instead of copying it. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8674] Purpose:Quincey Koziol2004-06-131-7/+7
| | | | | | | | | | | | | | | | | | Code optimization Description: Revised dataspace selections to use a more "object oriented" mechanism to set the function pointers for each selection and selection iterator. This reduces the amount and number of times that dataspace selection info has to be copied. Additionally, change hyperslab selection information to be dynamically allocated instead of an inline struct. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8667] Purpose:Quincey Koziol2004-06-131-1/+6
| | | | | | | | | | | | | | | | Code optimization Description: Restructure conversion loop of variable-length objects to avoid walking through memory backwards and allocating as many temporary buffers. (This uses the optimized method used in the atomic type conversions). Also bring back another optimization for variable-length datatypes that avoids querying the DXPL so many times. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8619] Purpose:Quincey Koziol2004-06-051-1/+1
| | | | Fix makefiles, dependencies & manifest after source file rename.
* [svn-r8614] Purpose:Quincey Koziol2004-06-051-84/+144
| | | | | | | | | | | | | | | Refactor code Description: Move chunk and contiguous cached raw data from file information to dataset information. This simplifies a number of internal interfaces, aligns the code with it's purpose better and should allow more optimizations to the chunked data I/O performance. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) h5committest
* [svn-r8601] Purpose:Quincey Koziol2004-05-311-406/+456
| | | | | | | | | | | | | | | | Code optimization & cleanup Description: Don't recompute the internal index value for looking up the chunk in the hash table, just use the value already computed from iterating through the chunks. Also, back-port some of the various cleanups to the source code from the development branch, to make diffing the code easier. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel
* [svn-r8592] Purpose:Quincey Koziol2004-05-271-137/+165
| | | | | | | | | | | | | | | | | | | | | Code optimization & bug fix Description: When dimension information is being stored in the storage layout message on disk, it is stored as 32-bit quantities, possibly truncating the dimension information, if a dimension is greater than 32-bits in size. Solution: Fix the storage layout message problem by revising file format to not store dimension information, since it is already available in the dataspace. Also revise the storage layout data structures to be more compartmentalized for the information for contiguous, chunked and compact storage. Platforms tested: FreeBSD 4.9 (sleipnir) w/parallel Solaris 2.7 (arabica) h5committest
* [svn-r8543] Purpose:Quincey Koziol2004-05-201-2/+2
| | | | | | | | | | | | | | Code optimization Description: Expand the use of macros to inline trivial function pointer lookup and calls to reduce the overall number of functions invoked during normal operation of the library. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8497] Purpose:Quincey Koziol2004-05-081-2/+2
| | | | | | | | | | | | | Code optimization Description: Further reduce the number of copies we make of a hyperslab selection for chunked I/O, especially when we are only going to throw the old selection away for a new one. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel
* [svn-r8455] Purpose:Quincey Koziol2004-05-011-1/+5
| | | | | | | | | | | | | Code optimization Description: Eliminate more 64-bit multiplies by remebering the size of contiguous datasets as well as chunked datasets. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r8384] Purpose:Quincey Koziol2004-04-181-21/+21
| | | | | | | | | | | | | Code cleanup Description: Clean up lots of warnings based on those reported from the SGI compilers as well as gcc. Platforms tested: SGI O3900, IRIX64 6.5 (Cheryl's SGI machine) FreeBSD 4.9 (sleipnir) w/ & w/o parallel h5committest
* [svn-r8317] Purpose:Quincey Koziol2004-04-071-12/+93
| | | | | | | | | | | | Code optimization Description: Query the dataset creation and transfer properties less often. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel h5committested
* [svn-r8304] Purpose:Quincey Koziol2004-04-061-4/+6
| | | | | | | | | | | | Code optimization Description: Compute the size of a chunk once, when the layout information is set, instead of each time I/O is performed on the chunk. Platforms tested: h5committest Solaris 2.7 (arabica)
* [svn-r8083] Purpose:Quincey Koziol2004-01-191-2/+6
| | | | | | | | | | | | | Bug fix Description: Tweak address comparison of external file list's heap address a bit to pacify Windows. Platforms tested: FreeBSD 4.9 (sleipnir) Asking Pedro to test on Windows Too minor to require h5committest
* [svn-r8058] Purpose:Quincey Koziol2004-01-131-0/+4
| | | | | | | | | | | Bug fix Description: Plug another small memory leak for fill-values. Platforms tested: FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r8039] Purpose:Quincey Koziol2004-01-081-25/+250
| | | | | | | | | | | | | | | | | | | | Bug fix Description: When two property lists are compared, the H5Pequal routine was just comparing the raw information for the property values. This causes problems when the raw information contains pointers to other information. Solution: Allow a 'compare' callback to be registered for properties, so that a user application get perform the comparison itself, allowing for "deep" compares of the property value. This was exported to the H5Pregister & H5Pinsert routines in the development branch, but not the release branch. Platforms tested: FreeBSD 4.9 (sleipnir) h5committest
* [svn-r8027] Purpose:Quincey Koziol2004-01-061-0/+4
| | | | | | | | | | | | | | | Bug fix Description: Fix small memory leak of fill-value information inside the library. Solution: "reset" the fill-value object header message, so that the internal buffer gets freed. Platforms tested: FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r7918] Purpose:Quincey Koziol2003-12-061-18/+14
| | | | | | | | | | | | | | | | | | | Code cleanup Description: Clean up compiler warnings, especially the 'FUNC' variable not used which comes out in production mode. Solution: Had to add a new FUNC_ENTER_NOAPI_NOINIT_NOFUNC macro for those non-API functions which don't need the 'FUNC' variable defined. (This will be _so_ much easier when C99 is standard on all our supposed platforms, since it has a __FUNC__ macro... ) Platforms tested: FreeBSD 4.9 (sleipnir) too minor for h5committest (although there were lots of files changed, the change was minor in each one)
* [svn-r7837] Purpose:Quincey Koziol2003-11-121-12/+15
| | | | | | | | | | | Bug fix. Description: Skip over situation where malloc(0) might occur. Platforms tested: IBM p690 (copper) too minor to require h5committest
* [svn-r7726] Purpose:Quincey Koziol2003-10-241-21/+29
| | | | | | | | | | | | Code cleanup Description: Refactored handlier of VFL drivers in file access and data transfer property lists in order to simplify and unify the code dealing with them. Platforms tested: FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r7543] Purpose:Quincey Koziol2003-10-051-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bug fixes and code cleanup Description: Lots of changes here: - Fixed bug #691 - when shared datatypes are used in attributes they are incorrectly copied into the attribute instead of referring the the named datatype in the file. This required bumping the version of the attribute message. The new version of the attribute message is only written out when a shared datatype is used in the attribute. [Also, this format change made the size of the attribute smaller.] - Added information to attribute debugging routine so that shared datatypes are displayed correctly with the h5debug tool. - Refactored the H5O* routines to extract code that was common to several routines into subroutines to call. - Added 'link' method for H5O message sub-classes, which increments the link count on shared objects when a message is created which shares them. - Corrected [unreported] bug where the link count was not being decremented on the shared object when a object header message with a reference to that object was deleted from the file. - Reduced size of shared message from 49 bytes (which was incorrect anyway and should have been 48 bytes) to 10 bytes, which required bumping the version of "shared" messages. - Refactored some of the shared datatype routines to allow for easier queries of "committedness" internally to the library and also added routine to easily increment/decrement the reference count of a shared datatype. Platforms tested: FreeBSD 4.9 (sleipnir) h5committest
* [svn-r7242] Purpose:Quincey Koziol2003-07-211-6/+19
| | | | | | | | | | | | Bug fix Description: Fix bug with combination of fill-values, chunked datasets and variable-length strings. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest
* [svn-r7116] Purpose:Quincey Koziol2003-06-271-3/+4
| | | | | | | | | | | | Update code Description: Move "PABLO_MASK" definition above header file inclusion to avoid problems with inline functions in header files. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest
* [svn-r7058] Purpose:Quincey Koziol2003-06-181-22/+25
| | | | | | | | | | | | | | Bug fix and refactored code Description: Return the file space we have allocated already when a dataset creation fails. Also track the change to the H5Fget_obj_<foo> APIs Platforms tested: FreeBSD 4.8 (sleipnir) h5committest
* [svn-r7037] Purpose:Quincey Koziol2003-06-131-1/+4
| | | | | | | | | | | | | | | | | | Bug fix Description: The fill time in a dataset with no fill value information created with an older version of the library was getting set to H5D_FILL_TIME_ALLOC instead of the new default H5D_FILL_TIME_IFSET and was causing H5Dcreate() calls with that dataset creation property list to fail now. Solution: Set the new default in the fill time initialization for missing fill value information. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest
* [svn-r7029] Purpose:Quincey Koziol2003-06-121-20/+52
| | | | | | | | | | | | | New feature/Bug fix Description: Add new fill time value - H5D_FILL_TIME_IFSET which writes the fill value to a dataset if the user has defined one, otherwise not writing the fill value to the dataset. Platforms tested: FreeBSD 4.8 (sleipnir) serial & parallel h5committest
* [svn-r7016] Purpose:Quincey Koziol2003-06-101-17/+30
| | | | | | | | | | | | | | | | Bug fix Description: Chunked datasets with early space allocation and unlimited dimensions were running into problems where the dataset's "layout" message was marked as constant too early, preventing the dataset's dimensions from being extended. Solution: Change logic for marking the layout message constant to wait a bit longer. Platforms tested: FreeBSD 4.8 (sleipnir) w/parallel h5committest
* [svn-r6878] Purpose:Quincey Koziol2003-05-151-3/+3
| | | | | | | | | | | Code cleanup Description: Limit the scope on more function prototypes/macros/typedefs. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest not necessary.
* [svn-r6869] Purpose:Quincey Koziol2003-05-131-1/+23
| | | | | | | | | | | | | | | | | Bug fix. Description: When the same dataset is opened more than once and those references to the object are left dangling, the library would not correctly shut down the dataset API, potentially allowing the file to be truncated. Solution: "force" the dataset IDs to be released when the dataset API is being shut down by the library. Platforms tested: FreeBSD 4.8 (sleipnir) h5committested
* [svn-r6837] Purpose:Quincey Koziol2003-05-081-4/+15
| | | | | | | | | | | | | | | Code cleanup. Description: Move many package or internal function prototypes and macro definitions into tighter scope according to their current use. Added more comments where appropriate. Eliminate ancient, unused functions. Added a couple "accessor" functions to get parts of data structures which were moved out of scope. Platforms tested: h5committested
* [svn-r6825] Purpose:Quincey Koziol2003-05-071-1208/+7
| | | | | | | | | | | | | | | | | | | | | | | | | New feature/enhancement Description: Chunked datasets are handled poorly in several circumstances involving certain selections and chunks that are too large for the chunk cache and/or chunks with filters, causing the chunk to be read from disk multiple times. Solution: Rearrange raw data I/O infrastructure to handle chunked datasets in a much more friendly way by creating a selection in memory and on disk for each chunk in a chunked dataset and performing all of the I/O on that chunk at one time. There are still some scalability (the current code attempts to create a selection for all the chunks in the dataset, instead of just the chunks that are accessed, requiring portions of the istore.c and fillval.c tests to be commented out) and performance issues, but checking this in will allow the changes to be tested by a much wider audience while I address the remaining issues. Platforms tested: h5committested, FreeBSD 4.8 (sleipnir) serial & parallel, Linux 2.4 (eirene)
* [svn-r6738] Purpose:Quincey Koziol2003-04-231-1/+2
| | | | | | | | | | | | Code cleanup Description: Add & improve assertion that was removed in last commit, to check for compact dataset storage. Platforms tested: FreeBSD 4.8 (sleipnir) triple check not necessary.
* [svn-r6729] Raymond Lu2003-04-221-3/+2
| | | | | | | | | | | Purpose: bug fix Description: VL datatype in compact dataset fails because a wrong type size is used. Platforms tested: h5committested Misc. update:
* [svn-r6659] Purpose:Quincey Koziol2003-04-141-3/+4
| | | | | | | | | | | | | | | | New feature, sorta Description: Track changes to various internal APIs Platforms tested: FreeBSD 4.8 (sleipnir) w/C++ Linux 2.4 (burrwhite) w/FORTRAN Solaris 2.7 (arabica) w/FORTRAN IRIX64 6.5 (modi4) w/parallel & FORTRAN (h5committest not run due to my ongoing difficulties with C++ on burrwhite).
* [svn-r6604] Purpose:Quincey Koziol2003-04-091-30/+45
| | | | | | | | | | | | | | | | | | New feature Description: Hooked up internal dataset creation call (H5D_create) to the new "can apply" and "set local" callbacks for filters. Solution: Platforms tested: FreeBSD 4.8 (sleipnir) w/szip Linux 2.4 (sleipnir) w/szip Solaris 2.7 (arabica) w/FORTRAN IRIX64 6.5 (modi4) w/szip, FORTRAN & parallel Misc. update:
* [svn-r6499] Purpose:Bill Wendling2003-03-191-17/+18
| | | | | | | | | | | | | Update Description: Added support for FPHDF5 wherever there was support for MPI/IO and/or MPIPOSIX. Platforms tested: Linux, Modi4, Sol Misc. update:
* [svn-r6411] Purpose:Quincey Koziol2003-02-171-42/+42
| | | | | | | | | | | | | | | | | | | | | | Code cleanup Description: Clean up miscellaneous warnings which have crept into the code. Fix "_POSIX_C_SOURCE not defined" warning on FreeBSD. Adjust gcc compiler flags to be more concise for production mode. Refactor the H5O code so that there is a stronger boundary between code in the H5O package and code in the library which just calls H5O routines. Platforms tested: Tested h5committest {arabica (fortran), eirene (fortran, C++) modi4 (parallel, fortran)} FreeBSD 4.7 (sleipnir) serial & parallel and gcc 2.95.4 & gcc 3.2.2 Misc. update: Update MANIFEST if you add or remove any file.
* [svn-r6388] Purpose:Quincey Koziol2003-02-101-2/+2
| | | | | | | | | | | | | Update feature Description: Relax collective constraint for API functions which only read metadata from a file. Platforms tested: Tested h5committest {arabica (fortran), eirene (fortran, C++) modi4 (parallel, fortran)} FreeBSD 4.7 w/parallel
* [svn-r6387] Purpose:Quincey Koziol2003-02-101-132/+126
| | | | | | | | | | | | | | | | | | | | | | | | | | | Bug Fix Description: Metadata cache in parallel I/O can cause hangs in applications which perform independent I/O on chunked datasets, because the metadata cache can attempt to flush out dirty metadata from only a single process, instead of collectively from all processes. Solution: Pass a dataset transfer property list down from every API function which could possibly trigger metadata I/O. Then, split the metadata cache into two sets of entries to allow dirty metadata to be set aside when a hash table collision occurs during independent I/O. Platforms tested: Tested h5committest {arabica (fortran), eirene (fortran, C++) modi4 (parallel, fortran)} FreeBSD 4.7 (sleipnir) serial & parallel Misc. update: Updated release_docs/RELEASE
* [svn-r6375] Raymond Lu2003-02-041-0/+12
| | | | | | | | | | | Purpose: New feature Description: Added Adler32 checksum as a filter in pipeline Platforms tested: arabica (fortran), eirene (, C++), modi4 (parallel, fortran) Misc. update: Update release_docs/RELEASE.
* [svn-r6308] Purpose:Quincey Koziol2003-01-211-0/+8
| | | | | | | | | | | | | | | | | Bug fix Description: Currently, when the library encounters an object header message that isn't know, it fails to open that object in the file. Solution: Allow the library to skip over the unknown object header message and continue to process the remaining messages, in the hope that the skipped message isn't important later. If it is important, it will be caught at a higher level of the library. Platforms tested: FreeBSD 4.7 (sleipnir)
* [svn-r6296] Purpose:Quincey Koziol2003-01-171-15/+21
| | | | | | | | | | Code cleanup Description: Reduce warnings on Windows Platforms tested: FreeBSD 4.7 (sleipnir)
* [svn-r6266] Purpose:Quincey Koziol2003-01-101-49/+49
| | | | | | | | | | | | Code cleanup/new feature. Description: Split FUNC_LEAVE into API and non-API specific versions. This allows a solution to compiling this branch with C++, as well as reducing the size of the binaries produced. Platforms tested: FreeBSD 4.7 (sleipnir) w/serial, parallel (including MPE) & thread-safe
* [svn-r6255] Purpose:Quincey Koziol2003-01-091-5/+5
| | | | | | | | | | Code cleanup Description: Clean up a few more warnings and update dependencies. Platforms tested: Linux 2.2.18smp (eirene) serial & parallel
* [svn-r6252] Purpose:Quincey Koziol2003-01-091-549/+547
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Lots of performance improvements & a couple new internal API interfaces. Description: Performance Improvements: - Cached file offset & length sizes in shared file struct, to avoid constantly looking them up in the FCPL. - Generic property improvements: - Added "revision" number to generic property classes to speed up comparisons. - Changed method of storing properties from using a hash-table to the TBBT routines in the library. - Share the propery names between classes and the lists derived from them. - Removed redundant 'def_value' buffer from each property. - Switching code to use a "copy on write" strategy for properties in each list, where the properties in each list are shared with the properties in the class, until a property's value is changed in a list. - Fixed error in layout code which was allocating too many buffers. - Redefined public macros of the form (H5open()/H5check, <variable>) internally to only be (<variable>), avoiding innumerable useless calls to H5open() and H5check_version(). - Reuse already zeroed buffers in H5F_contig_fill instead of constantly re-zeroing them. - Don't write fill values if writing entire dataset. - Use gettimeofday() system call instead of time() system when checking the modification time of a dataset. - Added reference counted string API and use it for tracking the names of objects opening in a file (for the ID->name code). - Removed redundant H5P_get() calls in B-tree routines. - Redefine H5T datatype macros internally to the library, to avoid calling H5check redundantly. - Keep dataspace information for dataset locally instead of reading from disk each time. Added new module to track open objects in a file, to allow this (which will be useful eventually for some FPH5 metadata caching issues). - Remove H5AC_find macro which was inlining metadata cache lookups, and call function instead. - Remove redundant memset() calls from H5G_namei() routine. - Remove redundant checking of object type when locating objects in metadata cache and rely on the address only. - Create default dataset object to use when default dataset creation property list is used to create datasets, bypassing querying for all the property list values. - Use default I/O vector size when performing raw data with the default dataset transfer property list, instead of querying for I/O vector size. - Remove H5P_DEFAULT internally to the library, replacing it with more specific default property list based on the type of property list needed. - Remove redundant memset() calls in object header message (H5O*) routines. - Remove redunant memset() calls in data I/O routines. - Split free-list allocation routines into malloc() and calloc()- like routines, instead of one combined routine. - Remove lots of indirection in H5O*() routines. - Simplify metadata cache entry comparison routine (used when flushing entire cache out). - Only enable metadata cache statistics when H5AC_DEBUG is turned on, instead of always tracking them. - Simplify address comparison macro (H5F_addr_eq). - Remove redundant metadata cache entry protections during dataset creation by protecting the object header once and making all the modifications necessary for the dataset creation before unprotecting it. - Reduce # of "number of element in extent" computations performed by computing and storing the value during dataspace creation. - Simplify checking for group location's file information, when file has not been involving in file-mounting operations. - Use binary encoding for modification time, instead of ASCII. - Hoist H5HL_peek calls (to get information in a local heap) out of loops in many group routine. - Use static variable for iterators of selections, instead of dynamically allocation them each time. - Lookup & insert new entries in one step, avoiding traversing group's B-tree twice. - Fixed memory leak in H5Gget_objname_idx() routine (tangential to performance improvements, but fixed along the way). - Use free-list for reference counted strings. - Don't bother copying object names into cached group entries, since they are re-created when an object is opened. The benchmark I used to measure these results created several thousand small (2K) datasets in a file and wrote out the data for them. This is Elena's "regular.c" benchmark. These changes resulted in approximately ~4.3x speedup of the development branch when compared to the previous code in the development branch and ~1.4x speedup compared to the release branch. Additionally, these changes reduce the total memory used (code and data) by the development branch by ~800KB, bringing the development branch back into the same ballpark as the release branch. I'll send out a more detailed description of the benchmark results as a followup note. New internal API routines: Added "reference counted strings" API for tracking strings that get used by multiple owners without duplicating the strings. Added "ternary search tree" API for text->object mappings. Platforms tested: Tested h5committest {arabica (fortran), eirene (fortran, C++) modi4 (parallel, fortran)} Other platforms/configurations tested? FreeBSD 4.7 (sleipnir) serial & parallel Solaris 2.6 (baldric) serial
* [svn-r6145] Raymond Lu2002-12-021-1/+11
| | | | | | | | | | Purpose: New feature to H5Dget_offset Description: If user block is set, H5Dget_offset should be able to return the absolute offset from the beginning of file. Platforms tested: eirene, arabica
generated by cgit v0.12 at 2024-11-08 14:40:38 (GMT)