summaryrefslogtreecommitdiffstats
path: root/src/H5Dio.c
Commit message (Collapse)AuthorAgeFilesLines
* [svn-r8337] Purpose:Quincey Koziol2004-04-101-18/+15
| | | | | | | | | | | | Code optimization Description: Avoid another extraneous dataspace copy. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel too minor to require h5committest
* [svn-r8335] Purpose:Quincey Koziol2004-04-101-19/+7
| | | | | | | | | | | | | | | Code optimization Description: Change algorithm to directly use coordinates describing a chunk's position in a dataspace instead of creating a dataspace with the chunk's position selected. This reduces the number of copies of dataspaces we need to keep around. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel too minor to require h5committest
* [svn-r8328] Purpose:Quincey Koziol2004-04-081-80/+96
| | | | | | | | | | | | | Code optimization Description: Check for only performing I/O on a single chunk and re-use memory dataspace, instead of re-creating the same dataspace & selection the hard way. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel too minor to require h5committest
* [svn-r8326] Purpose:Quincey Koziol2004-04-081-28/+15
| | | | | | | | | | | | Code optimization Description: Eliminate more dataspace copying, etc. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel too minor to require h5committest
* [svn-r8321] Purpose:Quincey Koziol2004-04-081-17/+19
| | | | | | | | | | | | Code optimization Description: Reduce the number of dataspace copies made when performing I/O on chunked datasets. Platforms tested: Solaris 2.7 (arabica) too minor to require h5committest
* [svn-r8316] Purpose:Quincey Koziol2004-04-071-79/+52
| | | | | | | | | | | Code optimization Description: Query the dataset creation and transfer properties less often. Platforms tested: Solaris 2.7 (arabica) h5committested
* [svn-r8276] *** empty log message ***Raymond Lu2004-03-241-3/+11
|
* [svn-r8158] Purpose:Quincey Koziol2004-02-061-157/+307
| | | | | | | | | | | | | | | | | Code cleanup/optimization Description: Query property list values once, at the beginning of the I/O routines, instead of querying the property list values multiple (lots!) of times in lower level routines. Solution: Create "property list caches" for internal library queries of the property list values. Platforms tested: IBM p690 (copper) w/parallel & fphdf5 h5committest
* [svn-r8053] Purpose:Quincey Koziol2004-01-131-13/+18
| | | | | | | | | | | | | | | | | | | | Bug fix Description: Correct two problems with variable-length datatypes in datasets: - When overwriting an entire dataset, writing the fill value to the file would be skipped, causing problems for VL datatypes when objects in the file had been unlinked (and thus the space in the file was not all zeros) - When an application has set a fill-value for a dataset and the dataset's datatype contained a VL datatype, the library was filling space on disk with the memory form of the VL information, instead of the disk form. Platforms tested: FreeBSD 4.9 (sleipnir) h5committest
* [svn-r7943] Purpose:Quincey Koziol2003-12-131-2/+10
| | | | | | | | | | | | | | | Bug fix. Description: Using a selection offset with hyperslab selections in chunked datasets was getting into an infinite loop and hanging the application. Solution: Apply the selection offset to the hyperslab selection properly. Platforms tested: FreeBSD 4.9 (sleipnir) w & w/o parallel h5committest
* [svn-r7917] Purpose:Quincey Koziol2003-12-061-16/+16
| | | | | | | | | | | | | | | | | | | Code cleanup Description: Clean up compiler warnings, especially the 'FUNC' variable not used which comes out in production mode. Solution: Had to add a new FUNC_ENTER_NOAPI_NOINIT_NOFUNC macro for those non-API functions which don't need the 'FUNC' variable defined. (This will be _so_ much easier when C99 is standard on all our supposed platforms, since it has a __FUNC__ macro... ) Platforms tested: FreeBSD 4.9 (sleipnir) too minor for h5committest (although there were lots of files changed, the change was minor in each one)
* [svn-r7842] Purpose:Quincey Koziol2003-11-131-16/+19
| | | | | | | | | | | | | | | | | | | Bug fix Description: Variable length strings and sequences with NULL pointers were not handled by library, causing problems access the data. This also affected fill values for variable-length datatypes. Solution: Address the issues in the library by detecting NULL sequences/strings and avoid trying to convert them. Patched up dumper to display NULL sequences/strings. Platforms tested: FreeBSD 4.9 (sleipnir) h5committest
* [svn-r7759] Purpose:Quincey Koziol2003-10-281-2/+2
| | | | | | | | | | | | | | Fix serial build Description: I accidentally put the "use_par_opt_io" variable in an #ifdef PARALLEL section. Solution: Hoist it out of parallel section Platforms tested: Eyeballed it - very trivial
* [svn-r7754] Purpose:Quincey Koziol2003-10-271-4/+10
| | | | | | | | | | | | | Code cleanup Description: Straighten out more goofiness in the MPI code dealing with collective I/O transfers - mostly make certain that a view is set if-and-only-if collective I/O is occurring on raw data (and vice versa for views and independent I/O) Platforms tested: FreeBSD 4.9 (sleipnir) w/parallel & FPHDF5 too minor to repquire h5committest
* [svn-r7744] Purpose:Quincey Koziol2003-10-271-109/+53
| | | | | | | | | | | | | | Code cleanup Description: More refactoring on setup of MPI-IO parameters: hoisted the code to change the transfer mode from collective to independent up into the H5D_read and H5D_write routines, instead of duplicating it inside each of the chunk/contig read/write routines. Platforms tested: FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r7742] Purpose:Quincey Koziol2003-10-271-77/+44
| | | | | | | | | | | | Code cleanup Description: Clean up setup code for collective I/O transfers to make the logic more obvious. Platforms tested: FreeBSD 4.9 (sleipnir) too minor for h5committest
* [svn-r7551] Purpose:Quincey Koziol2003-10-061-8/+0
| | | | | | | | | | | | | | | Bug/feature fix. Description: Relax restriction on parallel writing to compact datasets to allow partial I/O. Updates to reference manual mentioning the issues involved are delayed until reference manual 'lock' is removed later this week. Platforms tested: FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r7478] Purpose:Quincey Koziol2003-09-161-1/+4
| | | | | | | | | | | | | | | Bug fix Description: H5Dwrite was only checking the "top level" of the datatype to stop parallel I/O on variable-length datatypes. Solution: Make checks "deeper". Platforms tested: FreeBSD 4.9 (sleipnir) h5committest
* [svn-r7384] Purpose:Quincey Koziol2003-08-191-14/+19
| | | | | | | | | | | Code cleanup Description: More linting... Platforms tested: FreeBSD 4.8 (sleipnir) too minor to need h5committest
* [svn-r7383] Purpose:Quincey Koziol2003-08-191-7/+29
| | | | | | | | | | | | | | | | Bug fix Description: I/O on chunked datasets with a scalar dataspace for the memory dataspace was not working correctly. Solution: Translate the scalar dataspace into a n-dimensional (where n is the number of dimensions of the file's dataspace) dataspace of dimensions 1x1x1... Platforms tested: FreeBSD 4.8 (sleipnir) h5committest (modulo modi4 which is not working correctly)
* [svn-r7259] Purpose:Quincey Koziol2003-07-231-363/+345
| | | | | | | | | | | | | | | Bug fix Description: I/O on chunked datasets with point selections was not working correctly. Solution: Re-wrote some parts of raw data I/O routines that build the selections for each chunk to correctly handle point selections. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest
* [svn-r7223] Purpose:Quincey Koziol2003-07-141-318/+397
| | | | | | | | | | | Code cleanup Description: Clean up more warnings from lint Platforms tested: FreeBSD 4.8 (sleipnir) too minor for h5committest
* [svn-r7219] Purpose:Quincey Koziol2003-07-141-16/+16
| | | | | | | | | | | | | | Bug fix Description: H5T_BKG_TEMP was accidentally removed from library code, but is used by application's datatype conversion routines. Solution: Revert removal of H5T_BKG_TEMP. Platforms tested: h5committest
* [svn-r7210] Purpose:Quincey Koziol2003-07-111-20/+9
| | | | | | | | | | | | | Refactoring code Description: Refactored "IS_H5FD_MPIO || IS_H5FD_MPIPOSIX || IS_H5FD_FPHDF5" combination of macros in many places into single IS_H5FD_MPI macro, which has the same definition, but should be easier to maintain. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest
* [svn-r7116] Purpose:Quincey Koziol2003-06-271-3/+4
| | | | | | | | | | | | Update code Description: Move "PABLO_MASK" definition above header file inclusion to avoid problems with inline functions in header files. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest
* [svn-r7018] Purpose:Quincey Koziol2003-06-101-0/+8
| | | | | | | | | | | | | | | | | | | | | Bug fix Description: The dataset's modification time was getting set whenever raw data was written with H5Dwrite. Unfortunately, this is a metadata change (which are required to be performed collectively) and H5Dwrite may be called independently from a parallel program, resulting in metadata cache corruption and/or program hangs. Solution: Don't update the modification time when raw data it written. :-( Platforms tested: FreeBSD 4.8 (sleipnir) w/parallel h5committest Misc. update: Noted in release notes and also sent to Frank for updating the docs.
* [svn-r6975] Purpose:Quincey Koziol2003-06-051-8/+8
| | | | | | | | | | | Bug fix Description: Don't attempt to perform collective I/O on chunked datasets. Platforms tested: FreeBSD 4.8 (sleipnir) w/parallel h5committested
* [svn-r6951] Purpose:Quincey Koziol2003-06-041-6/+4
| | | | | | | | | | | | Code cleanup Description: Clean up various warnings & comment out unused code. Platforms tested: FreeBSD 4.8 (sleipnir) w/C++ FreeBSD 4.8 (sleipnir) w/parallel h5committested
* [svn-r6942] Purpose:Quincey Koziol2003-05-311-139/+511
| | | | | | | | | | | | | | | | | Performance improment Description: Speed up chunked dataset I/O. This breaks down into several areas: - Compute chunk selections in the file by using hyperslab operations instead of iterating over each element in the selection. - If the file and memory selections are the same shape, use the file chunk selections to compute the memory chunk selections. This required several additional dataspace, dataspace selection and hyperslab routines. Platforms tested: h5committestted (although Fortran tests failed for some reason)
* [svn-r6909] Purpose:Quincey Koziol2003-05-211-10/+14
| | | | | | | | | | | | Bug fix Description: Correct typo which didn't show up during my previous testing in production mode. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest not necessary.
* [svn-r6908] Purpose:Quincey Koziol2003-05-211-6/+22
| | | | | | | | | | | Code cleanup Description: Fix some unused parameter warnings. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest not necessary.
* [svn-r6907] Purpose:Quincey Koziol2003-05-201-0/+8
| | | | | | | | | | | | | | Bug fix Description: The chunk dataspace selection information for raw data I/O are being leaked. Solution: Free the chunk information during the cleanup code. Platforms tested: Solaris 2.6 (baldric) w/purify h5committest not needed.
* [svn-r6891] Purpose:Quincey Koziol2003-05-171-299/+639
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bug fix Description: Raw data I/O on chunked datasets would attempt to allocate data structures proportional to the number of chunks in the dataset on disk, instead of just the number of chunks that the I/O operation would interact with, causing operations on datasets with large #'s of chunks to fail (or become very slow), even though the actual I/O operation was very modest. Solution: This is the "scalability fix" for chunked datasets that I've mentioned we need to do, althought it's not the complete fix for the issue. Read on for the details... Only create data structures for the chunks that the I/O operation will actually act on, reducing the amount of information allocated in memory, normally. I say "normally", because this algorithm has the same problems as the original algorithm (worse actually, since the data structure for each chunk is larger now) if _all_ the chunks in a dataset with a lot of chunks are actually involved in the I/O operation. If that is the case, this code will fail in a similar way. To truly fix the problem, we would need to only create data structures for a particular number of chunks, perform the I/O on just those chunks, then release the data structures for those chunks and create data structures for the next set of chunks to access, etc. However, I think this case is pretty rare right now and we should worry about it after the 1.6.0 release. Platforms tested: h5committested
* [svn-r6860] Purpose:Quincey Koziol2003-05-131-8/+20
| | | | | | | | | | | | | Small code cleanup Description: Remove redundant sanity check. Added a note about a potential bug. Platforms tested: FreeBSD 4.8 (sleipnir) h5committest not necessary.
* [svn-r6857] Purpose: bug fixRaymond Lu2003-05-131-2/+18
| | | | | | | | | Description: when a datatype element size is bigger than I/O datatype conversion buffer(1MB), library reports failure. Solution: adjust conversion buffer to be at least equal to one element size Platforms tested: h5committest
* [svn-r6843] Purpose:Quincey Koziol2003-05-091-9/+3
| | | | | | | | | | | | Code cleanup Description: Clean up warnings exposed by compiling on O2K. Also, revert some of Bill and my changes to the H5S_mpi_opt_types_g, etc. and settle them back into their original location. Platforms tested: h5committested.
* [svn-r6841] Purpose:Bill Wendling2003-05-081-0/+5
| | | | | | | | | | | | | | | | Bug Fix Description: A variable wasn't declared in this file. Solution: Put the extern declaration of the variable in this file. It's defined in the H5S.c module. Platforms tested: Modi4 (This affects only parallel platforms) Misc. update:
* [svn-r6836] Purpose:Quincey Koziol2003-05-081-68/+72
| | | | | | | | | | Bug fix Description: Clean up small memory leak. Platforms tested: h5committested.
* [svn-r6829] Purpose:Quincey Koziol2003-05-081-8/+0
| | | | | | | | | | | Code cleanup. Description: Cleaned up an unusued variable and some commented out code. Platforms tested: FreeBSD 4.8 (sleipnir) Triple check not needed.
* [svn-r6825] Purpose:Quincey Koziol2003-05-071-0/+2293
New feature/enhancement Description: Chunked datasets are handled poorly in several circumstances involving certain selections and chunks that are too large for the chunk cache and/or chunks with filters, causing the chunk to be read from disk multiple times. Solution: Rearrange raw data I/O infrastructure to handle chunked datasets in a much more friendly way by creating a selection in memory and on disk for each chunk in a chunked dataset and performing all of the I/O on that chunk at one time. There are still some scalability (the current code attempts to create a selection for all the chunks in the dataset, instead of just the chunks that are accessed, requiring portions of the istore.c and fillval.c tests to be commented out) and performance issues, but checking this in will allow the changes to be tested by a much wider audience while I address the remaining issues. Platforms tested: h5committested, FreeBSD 4.8 (sleipnir) serial & parallel, Linux 2.4 (eirene)
generated by cgit v0.12 at 2024-08-30 11:58:10 (GMT)