summaryrefslogtreecommitdiffstats
path: root/src/H5Distore.c
Commit message (Collapse)AuthorAgeFilesLines
* [svn-r10145] Purpose:Quincey Koziol2005-03-041-0/+6
| | | | | | | | | | | | Bug fix Description: Add some "safety net" sort of assertions and sanity checks contributed by user. Platforms tested: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r9857] Purpose: MaintenanceElena Pourmal2005-01-221-3/+0
| | | | | | | | | | | | | | Description: Removed PABLO from the source Solution: Platforms tested: arabica with 64-bit, copper with parallel, heping with GNU C and C++ and PGI fortran (but I disabled hl, there is some weird problem only on heping: F9XMODFLAG is not propagated to the Makefile files Misc. update:
* [svn-r9727] Purpose:Quincey Koziol2004-12-291-245/+246
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bug Fix/Code Cleanup/Doc Cleanup/Optimization/Branch Sync :-) Description: Generally speaking, this is the "signed->unsigned" change to selections. However, in the process of merging code back, things got stickier and stickier until I ended up doing a big "sync the two branches up" operation. So... I brought back all the "infrastructure" fixes from the development branch to the release branch (which I think were actually making some improvement in performance) as well as fixed several bugs which had been fixed in one branch, but not the other. I've also tagged the repository before making this checkin with the label "before_signed_unsigned_changes". Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel & fphdf5 FreeBSD 4.10 (sleipnir) w/threadsafe FreeBSD 4.10 (sleipnir) w/backward compatibility Solaris 2.7 (arabica) w/"purify options" Solaris 2.8 (sol) w/FORTRAN & C++ AIX 5.x (copper) w/parallel & FORTRAN IRIX64 6.5 (modi4) w/FORTRAN Linux 2.4 (heping) w/FORTRAN & C++ Misc. update:
* [svn-r9358] Purpose:Quincey Koziol2004-10-041-15/+13
| | | | | | | | | | | | | | | Bug fix Description: Relax restrictions on parallel I/O to allow compressed, chunked datasets to be read in parallel (collective access will be degraded to independent access, but will retrieve the information still). Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Solaris 2.7 (arabica) IRIX64 6.5 (modi4) h5committest
* [svn-r9342] Purpose:Quincey Koziol2004-09-301-170/+214
| | | | | | | | | | | | | | | | Bug fix/code cleanup Description: Clean up raw data I/O code to bundle the I/O parameters (dataset, DXPL ID, etc) into a single struct to pass around through the dataset I/O routines, since they are always passed together, until very near the bottom of the I/O stack. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Solaris 2.7 (arabica) IRIX64 6.5 (modi4) h5committest
* [svn-r9329] James Laird2004-09-281-112/+115
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Purpose: Feature Description: Datatypes and groups now use H5FO "file object" code that was previously only used by datasets. These objects will hold a file open if the file is closed but they have not yet been closed. If these objects are unlinked then relinked, they will not be destroyed. If they are opened twice (even by two different names), both IDs will "see" changes made to the object using the other ID. When an object is opened using two different names (e.g., if a dataset was opened under one name, then mounted and opened under its new name), calling H5Iget_name() on a given hid_t will return the name used to open that hid_t, not the current name of the object (this is a feature, and a change from the previous behavior of datasets). Solution: Used H5FO code that was already in place for datasets. Broke H5D_t's, H5T_t's, and H5G_t's into a "shared" struct and a private struct. The shared structs (H5D_shared_t, etc.) hold the object's information and are used by all IDs that point to a given object in the file. The private structs are pointed to by the hid_t and contain the object's group entry information (including its name) and a pointer to the shared struct for that object. This changed the naming of structs throughout the library (e.g., datatype->size is now datatype->shared->size). I added an updated H5Tinit.c to windows.zip. Platforms tested: Visual Studio 7, sleipnir, arabica, verbena Misc. update:
* [svn-r9322] Purpose:Quincey Koziol2004-09-271-66/+104
| | | | | | | | | | | | | Bug fix Description: Fix situation where deleting a chunked datasets with B-tree nodes that weren't in the metadata cache would die with a core dump. Platforms tested: FreeBSD 4.10 (sleipnir) Linux 2.4 (heping) Solaris 2.7 (arabica)
* [svn-r9183] Purpose: New featureRaymond Lu2004-09-011-2/+2
| | | | | | | | | | | | Description: Restore 6 old error API functions back to the library to be backward compatible with v1.6. They are H5Epush, H5Eprint, H5Ewalk, H5Eclear, H5Eset_auto, H5Eget_auto. These functions do not have error stack as parameter. Solution: Internally, these functions use default error stack. Platforms tested: h5committest and fuss. Misc. update: RELEASE.txt
* [svn-r9155] Purpose:Quincey Koziol2004-08-251-2/+80
| | | | | | | | | | | | | | | Bug fix (sorta) Description: Change reading of "missing" chunks from datasets with undefined fill values to not overwrite the application buffer with random garbage from memory. Note that this is almost the same, since whatever garbage the application had in those locations will still be there... Platforms tested: FreeBSD 4.10 (sleipnir) IRIX64 6.5 (modi4) h5committested
* [svn-r9101] Purpose:Quincey Koziol2004-08-171-8/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Bug fix Description: 1 - Dataset contiguous storage cache information had a bug where it was possible to try to access invalid cache information if the cache wasn't filled the first time it attempted to loop through the list of offset/length vectors. 2 - Additionally, the contiguous storage cache information was being used in certain circumstances from the chunked dataset I/O code path, which was generally fatal since the chunk storage and contiguous storage information were stored together in a union. Solution: 1 - Avoid special case of first trip through loop over offset/length I/O vectors and always check for the contiguous storage sieve buffer buffer being NULL. 2 - Change the union containing the chunk and contiguous storage cache information into a struct, allowing both to be used at the same time. Platforms tested: FreeBSD 4.10 (sleipnir) h5committested
* [svn-r8969] Purpose:Quincey Koziol2004-07-301-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | Bug fix. Description: Address two problems: - The computation of the scanline in the szip filter was being performed in the "can apply" callback routine instead of the "set local" routine. - The routine which allocated all the chunks for an entire dataset (which is invoked when the allocation time is early or late, rather than incremental) wasn't recording a failed filter in the information for the chunk, causing the library to believe that the chunk had the filter applied when it really hadn't. Solution: - Move the scanline computation to the "set local" callback. - Record the filter mask with each chunk created when allocating them. Platforms tested: FreeBSD 4.10 (sleipnir) w/szip Too obscure to require h5committest
* [svn-r8958] Purpose:Quincey Koziol2004-07-271-1/+3
| | | | | | | | | | | | | Bug fix Description: Always write fill values to chunks when initializing entire B-tree and any filters are defined. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Solaris 2.7 (arabica) Too minor to require h5committest
* [svn-r8946] Purpose:Quincey Koziol2004-07-261-0/+7
| | | | | | | | | | | | | | | | | | Code cleanup, sorta Description: Added ifdef sections for "H5_USING_PURIFY" in various places in the code, which are designed to reduce the spurious "uninitialized memory read" warnings from purify which are actually OK. Note that this macro will have to be turned on by adding it to the CFLAGS for the build - I didn't think it was important enough to add a configure flag for. Also, the changes in H5HG.c optimize the walks through the objects in a heap to only look at the 'used' entries instead of all the 'allocated' entries. Platforms tested: Solaris 2.7 (arabica) w/purify Not tested by h5committest
* [svn-r8906] Purpose:MuQun Yang2004-07-201-3/+1
| | | | | | | | | | | | | | | | | | | | | | | Adding the first round of patches about supporting collective chunk IO in HDF5 Description: The current HDF5 library doesn't support collective MPIO with chunk storage. When users set collective option in their data transfer with chunk storage, the library silently converted the option to INDEPENDENT and that caused trememdous performance penalty. Some application like WRF-parallel HDF5 IO module has to use contiguous storage for this reason. However, chunking storage has its own advantage(supporting compression filters and extensible dataset), so to make collective MPIO possible inside HDF5 with chunking storage is a very important task. This check-in make collective chunk IO possible for some special cases. The condition is as follows(either case is fine with using collective chunk IO) 1. for each process, the hyperslab selection of the file data space of each dataset is regular and it is fit in one chunk. 2. for each process, the hyperslab selection of the file data space of each dataset is single and the number of chunks for the hyperslab selection should be equal. Solution: Lift up the contiguous storage requirement for collective IO. Use H5D_isstore_get_addr to get the corresponding chunk address. Then the original library routines will take care of getting the correct address to make sure that MPI FILE TYPE is built correctly for collective IO> Platforms tested: arabica(sol), copper(AIX), eirene(Linux) parallel test is checked at copper. Misc. update:
* [svn-r8867] Purpose:Quincey Koziol2004-07-131-3/+13
| | | | | | | | | | | | | | | | | | | Bug fix Description: Fix error in chunked dataset I/O where data written out wasn't read correctly from a chunked, extendible dataset after the dataset was extended. Also, fix parallel I/O tests to gather error results from all processes, in order to detect errors that only occur on one process. Solution: Bypass chunk cache for reads as well as writes, if parallel I/O driver is used and file is opened for writing. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8865] Purpose:Quincey Koziol2004-07-131-2/+4
| | | | | | | | | | | | | | | | | Code optimization Description: Re-work the insertion of a new child into an existing node, to exploit some speedups for adding the rightmost child, since this is a very common case when appending records to an unlimited size dataset. Also, hoist the checks for the tree's 'K' value into a field in the shared information about the tree, instead of re-calculating them all the time. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8860] Purpose:Quincey Koziol2004-07-121-12/+41
| | | | | | | | | | | | Code optimization Description: Avoid calling vector comparison routine when operating on 1-D chunks. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8853] Purpose:Quincey Koziol2004-07-101-50/+101
| | | | | | | | | | | | | | | | | | | | Code optimization Description: Refactor B-tree code to extract all common information for a B-tree into a shared structure that is pointed to by all the nodes in tree (instead of being included in each node). Also re-order B-tree node comparison checks for chunked datasets to check for >= the upper node first, since the comparison is a bit "heavy" and this check is more likely to succeed when you are adding records to the dataset. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest (also, recent h5dump commits have broken testing...)
* [svn-r8844] Purpose:Quincey Koziol2004-07-091-4/+40
| | | | | | | | | | | | | | | | | | Bug fix Description: The "shared" raw B-tree node can get freed before all the B-tree nodes had been flushed out to disk and released by the cache. Solution: Implement a simple reference counting wrapper for objects in the library and use it to hold the shared raw B-tree nodes so they aren't freed before all references to them in memory are released. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) IRIX64 6.5 (modei4)
* [svn-r8823] Purpose:Quincey Koziol2004-07-071-1/+57
| | | | | | | | | | | | | | | Code optimization Description: Since the raw B-tree nodes are the same size and only used when reading in or writing out a B-tree node, move raw B-tree node buffer from being per node to a single node that is shared among all B-tree nodes of a particular tree, freeing up a lot of space and eliminating lots of memory copies, etc. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8807] Purpose:Quincey Koziol2004-07-051-42/+43
| | | | | | | | | | | | | Code optimization Description: Don't copy layout information, just point to existing information. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8732] Purpose:Quincey Koziol2004-06-231-0/+4
| | | | | | | | | | | | Code cleanup Description: Reduce compiler warnings on SGI IRIX Platforms tested: FreeBSD 4.10 (sleipnir) SGI IRIX6 (Cheryl's machine) Too minor to require full h5committest
* [svn-r8731] Purpose:Quincey Koziol2004-06-231-33/+16
| | | | | | | | | | | | | | | Code cleanup & minor optimization Description: Re-work the way interface initialization routines are specified in the library to avoid the overhead of checking for them in routines where there is no interface initialization routine. This cleans up warnings with gcc 3.4, reduces the library binary size a bit (about 2-3%) and should speedup the library's execution slightly. Platforms tested: FreeBSD 4.10 (sleipnir) w/gcc34 h5committest
* [svn-r8686] Purpose:Quincey Koziol2004-06-141-10/+14
| | | | | | | | | | | | | Code optimization Description: Eliminate memcpy() when using default DXPL by pointing at existing default object, instead of copying it. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8625] Purpose:Quincey Koziol2004-06-081-0/+10
| | | | | | | | | | | | | | | Code optimization Description: Avoid pushing errors on error stack when an object is not found in a B-tree. Sometimes we are just checking if the object exists before we insert it into the B-tree and the higher levels in the library should be responsible for determining if not finding the object in B-tree is really an error. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel too minor to require h5committest
* [svn-r8613] Purpose:Quincey Koziol2004-06-051-536/+499
| | | | | | | | | | | | | | | | | Refactor code Description: Move chunk and contiguous cached raw data from file information to dataset information. This simplifies a number of internal interfaces, aligns the code with it's purpose better and should allow more optimizations to the chunked data I/O performance. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) h5committest Misc. update:
* [svn-r8600] Purpose:Quincey Koziol2004-05-311-51/+183
| | | | | | | | | | | | | Code optimization Description: Don't recompute the internal index value for looking up the chunk in the hash table, just use the value already computed from iterating through the chunks. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel
* [svn-r8590] Purpose:Quincey Koziol2004-05-271-110/+112
| | | | | | | | | | | | | | | | | | | | | Code optimization & bug fix Description: When dimension information is being stored in the storage layout message on disk, it is stored as 32-bit quantities, possibly truncating the dimension information, if a dimension is greater than 32-bits in size. Solution: Fix the storage layout message problem by revising file format to not store dimension information, since it is already available in the dataspace. Also revise the storage layout data structures to be more compartmentalized for the information for contiguous, chunked and compact storage. Platforms tested: FreeBSD 4.9 (sleipnir) w/parallel Solaris 2.7 (arabica) h5committest
* [svn-r8474] Purpose:Quincey Koziol2004-05-031-35/+36
| | | | | | | | | | | | Code optimization Description: Reduce amount of information allocated/freed for chunk cache. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/paralell) too minor to require h5committest
* [svn-r8472] Purpose:Quincey Koziol2004-05-031-10/+2
| | | | | | | | | | | | | Code optimization Description: Eliminate the B-tree "split_ratios" as a parameter and pull it from the DXPL instead. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel too minor to require h5committest
* [svn-r8464] Purpose:Quincey Koziol2004-05-021-23/+30
| | | | | | | | | | | | Code optimization Description: Eliminate redundant B-tree lookups for chunk information. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel too minor to require h5committest
* [svn-r8446] Purpose:Quincey Koziol2004-05-011-18/+10
| | | | | | | | | | | | | Code optimization Description: Avoid dividing the chunk coordinates at the top levels of the chunk I/O routines, only to multiply them at the bottom of the routines. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r8439] Purpose:Quincey Koziol2004-04-301-59/+60
| | | | | | | | | | | | | Code cleanup Description: Refactor code to eliminate duplication of setup & calls to H5B_find to locate information about a chunk on disk. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r8434] Purpose:Quincey Koziol2004-04-291-17/+89
| | | | | | | | | | | | | Code optimization. Description: Use free-list pool of blocks for allocating chunk buffers, when there are no filters in the pipeline. The avoids calling malloc/free so much. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r8393] Purpose:Quincey Koziol2004-04-191-1/+3
| | | | | | | | | | | | Code optimization Description: Avoid clearing the error stack unless necessary. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r8366] Purpose:Quincey Koziol2004-04-171-4/+3
| | | | | | | | | | | | Code optimization Description: Eliminate memset() calls to clear structures that we completely initialize Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r8303] Purpose:Quincey Koziol2004-04-061-37/+21
| | | | | | | | | | | | Code optimization Description: Compute the size of a chunk once, when the layout information is set, instead of each time I/O is performed on the chunk. Platforms tested: h5committest Solaris 2.7 (arabica)
* [svn-r8287] Purpose:Quincey Koziol2004-03-301-1/+1
| | | | | | | | | | | | | | Bug fix/code cleanup Description: Copy Robb's feature in SSlib that checks that the name of the function used in the FUNC_ENTER macro is actually the name of function. Fixed a bunch of typos & copy-n-pasto's for functions with incorrect names. Platforms tested: FreeBSD 4.9 (sleipnir) w/parallel too minor to require h5committest
* [svn-r8158] Purpose:Quincey Koziol2004-02-061-93/+63
| | | | | | | | | | | | | | | | | Code cleanup/optimization Description: Query property list values once, at the beginning of the I/O routines, instead of querying the property list values multiple (lots!) of times in lower level routines. Solution: Create "property list caches" for internal library queries of the property list values. Platforms tested: IBM p690 (copper) w/parallel & fphdf5 h5committest
* [svn-r8126] Purpose:Quincey Koziol2004-01-311-30/+4
| | | | | | | | | | | | | | | | | Bug fix/optimization Description: Address slowdown in MPI-I/O file metadata operations that was introduced mid-stream. We now _require_ a POSIX compliant parallel file system for the MPI-I/O file driver (as well as for the MPI-POSIX file driver). Also optimized file open operation when the file is being created by reducing the number of collective & syncronizing calls. Additionally, refactor the MPI routines into a common place, eliminating duplicated code. Platforms tested: FreeBSD 4.9 (sleipnir) w/parallel h5committest
* [svn-r8038] Purpose:Quincey Koziol2004-01-081-5/+5
| | | | | | | | | | | | | | | | | | | | Bug fix Description: When two property lists are compared, the H5Pequal routine was just comparing the raw information for the property values. This causes problems when the raw information contains pointers to other information. Solution: Allow a 'compare' callback to be registered for properties, so that a user application get perform the comparison itself, allowing for "deep" compares of the property value. This was exported to the H5Pregister & H5Pinsert routines in the development branch, but not the release branch. Platforms tested: FreeBSD 4.9 (sleipnir) h5committest
* [svn-r7917] Purpose:Quincey Koziol2003-12-061-11/+11
| | | | | | | | | | | | | | | | | | | Code cleanup Description: Clean up compiler warnings, especially the 'FUNC' variable not used which comes out in production mode. Solution: Had to add a new FUNC_ENTER_NOAPI_NOINIT_NOFUNC macro for those non-API functions which don't need the 'FUNC' variable defined. (This will be _so_ much easier when C99 is standard on all our supposed platforms, since it has a __FUNC__ macro... ) Platforms tested: FreeBSD 4.9 (sleipnir) too minor for h5committest (although there were lots of files changed, the change was minor in each one)
* [svn-r7763] Purpose:Quincey Koziol2003-10-281-1/+2
| | | | | | | | | | | | | | | | | Bug fix Description: Code was using an internal DXPL with collective I/O turned on during independent I/O operations to fill dataset on disk. Solution: Switched to internal DXPL with independent I/O Platforms tested: FreeBSD 4.9 (sleipnir) FreeBSD 4.9 (sleipnir) w/parallel Linux 2.4 (verbena) w/fortran & C++ Solaris 2.7 (arabica) w/64-bit extensions enabled
* [svn-r7426] Purpose:Quincey Koziol2003-08-281-1/+1
| | | | | | | | | | | | | | | | | Bug fix Description: When datasets are deleted from a file, they are removed from the sieve buffer, but instead of invalidating only the part of the sieve buffer affected, the sieve buffer code would throw away the entire sieve buffer, potentially including other raw data in the buffer that hadn't been written to disk yet. Solution: Improve the sieve buffer clearing code to handle partial invalidations. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest
* [svn-r7362] Purpose:Quincey Koziol2003-08-141-1/+1
| | | | | | | | | | | | | Bug fix Description: Allow a user block to be "inserted" in front of a file (probably by writing a validly-sized userblock to a new file and then appending another HDF5 file to the new file). Platforms tested: FreeBSD 4.8 (sleipnir) h5committested
* [svn-r7297] Purpose:Quincey Koziol2003-08-081-2/+2
| | | | | | | | | | | | | | | | Code cleanup Description: Changed lines like: H5E_clear(H5E_get_my_stack()); to: H5E_clear(NULL); which performs the same operation. Platforms tested: h5committested
* [svn-r7265] *** empty log message ***Raymond Lu2003-07-261-2/+3
|
* [svn-r7210] Purpose:Quincey Koziol2003-07-111-11/+5
| | | | | | | | | | | | | Refactoring code Description: Refactored "IS_H5FD_MPIO || IS_H5FD_MPIPOSIX || IS_H5FD_FPHDF5" combination of macros in many places into single IS_H5FD_MPI macro, which has the same definition, but should be easier to maintain. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest
* [svn-r7192] Purpose:Quincey Koziol2003-07-091-8/+0
| | | | | | | | | | | Code cleanup Description: Clean up various minor warnings in the library. Platforms tested: FreeBSD 4.8 (sleipnir) too small to need h5committest
* [svn-r7189] Purpose:Quincey Koziol2003-07-091-0/+2
| | | | | | | | | | | Code cleanup Description: Break some of the "debugging" routines into their own module, so they aren't pulled into every executable, which certainly isn't going to use them. Platforms tested: h5committested
generated by cgit v0.12 at 2024-12-12 17:12:32 (GMT)