summaryrefslogtreecommitdiffstats
path: root/src/H5Dio.c
Commit message (Collapse)AuthorAgeFilesLines
* [svn-r8941] Purpose:Quincey Koziol2004-07-261-2/+6
| | | | | | | | | | | | | | Code optimization Description: Changed calloc() calls to malloc() calls allocating background buffers during dataset writes, since the background buffer information will be read from disk anyway, overwriting any existing values. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8932] Purpose:Quincey Koziol2004-07-221-18/+23
| | | | | | | | | | | | | | | | | | | | Code cleanup Description: Clean up collective chunking code a bit. Also, add '--enable-instrument' configure flag to have a mechanism for determining that optimized operations happened correctly in the library (instead of just the "normal" way) by allowing 'flag' properties to be set outside the library and set when the "right" thing happens. This is mainly for debugging and regression checks, so we make certain we don't break optimized I/O by accident. It's enabled by default when --enable-debug is on (which is on by default in the development branch and off by default in the release branch), but can also be independently controlled with its own configure flag. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel IBM p690 (copper) w/parallel
* [svn-r8923] Purpose:MuQun Yang2004-07-211-0/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To add a special section of codes for collective-chunk IO tests. Description: The current patch of collective chunk IO support in HDF5 can only handle some special("however, can cover many applications") cases, Inside source code, we did a careful checking to make sure other cases would not fall into this category and would not use collective IO. We also would like to test whether those collective conditions were met in our test programs. The current parallel HDF5 handled those collective IO requests in a special way. If the library finds it cannot do collective IO, it will silently change to independent IO. So basically there is no better way to check whether the library is doing what it should do without "hacking" into HDF5 source codes for the purpose of testing. But the "hacking" should not affect library work and should be easily pulled out after we get more general collective IO algorithm to work. With Quincey's suggestion, we used HDF5 property APIs to finish the job. Solution: The approach includes three parts: 1) In the test program, insert a property inside data transfer property list. Set a default value for this property. 2) Inside H5Dio.c, when the library finds that it cannot do collective IO with chunking storage, it will change the default value. 3) Then the test program will recheck the value after H5Dwrite or H5Dread to evaluate whether the current collective IO case is doing the right thing. Note: The test won't stop after it finds that the library is not doing the right thing and probably it will finish normally. The current approach is that the test program just printed out an error message. It should be changed later. Platforms tested:o copper,arabica,eirene Misc. update:
* [svn-r8906] Purpose:MuQun Yang2004-07-201-2/+2
| | | | | | | | | | | | | | | | | | | | | | | Adding the first round of patches about supporting collective chunk IO in HDF5 Description: The current HDF5 library doesn't support collective MPIO with chunk storage. When users set collective option in their data transfer with chunk storage, the library silently converted the option to INDEPENDENT and that caused trememdous performance penalty. Some application like WRF-parallel HDF5 IO module has to use contiguous storage for this reason. However, chunking storage has its own advantage(supporting compression filters and extensible dataset), so to make collective MPIO possible inside HDF5 with chunking storage is a very important task. This check-in make collective chunk IO possible for some special cases. The condition is as follows(either case is fine with using collective chunk IO) 1. for each process, the hyperslab selection of the file data space of each dataset is regular and it is fit in one chunk. 2. for each process, the hyperslab selection of the file data space of each dataset is single and the number of chunks for the hyperslab selection should be equal. Solution: Lift up the contiguous storage requirement for collective IO. Use H5D_isstore_get_addr to get the corresponding chunk address. Then the original library routines will take care of getting the correct address to make sure that MPI FILE TYPE is built correctly for collective IO> Platforms tested: arabica(sol), copper(AIX), eirene(Linux) parallel test is checked at copper. Misc. update:
* [svn-r8901] Purpose:Quincey Koziol2004-07-201-4/+2
| | | | | | | | | | | | Bug fix Description: Allow buffer parameter to H5Dread & H5Dwrite to be NULL if there are no elements to transfer. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8801] Purpose:Quincey Koziol2004-07-031-61/+23
| | | | | | | | | | | | | | | Code optimization Description: Set up datatype ID for dataset's datatype on disk. This allows us to avoid repeatedly copying the datatype when an ID is needed. Also, clean up a few warnings in various other places. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8782] Purpose:Quincey Koziol2004-07-011-8/+8
| | | | | | | | | | | Code cleanup Description: Clean up almost all warnings from Windows builds. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8781] James Laird2004-07-011-0/+23
| | | | | | | | | | | | | | | | Purpose: HDF5 now supports SZIP with no encoder. Description: SZIP can be configured to have both encoder and decoder or just to have the decoder. HDF5 can now query the configuration of any filter, and will throw errors if users try to write using a filter with encoding disabled. Solution: Added H5Zget_filter_info function, changed API for H5Pget_filter and H5P_get_filter_by_id. See SZIP RFC. Platforms tested: Copper (fortran, C++, parallel), Sleipnir (C++), Arabica (fortran, C++), Verbena (fortran, C++) Misc. update:
* [svn-r8731] Purpose:Quincey Koziol2004-06-231-4/+0
| | | | | | | | | | | | | | | Code cleanup & minor optimization Description: Re-work the way interface initialization routines are specified in the library to avoid the overhead of checking for them in routines where there is no interface initialization routine. This cleans up warnings with gcc 3.4, reduces the library binary size a bit (about 2-3%) and should speedup the library's execution slightly. Platforms tested: FreeBSD 4.10 (sleipnir) w/gcc34 h5committest
* [svn-r8694] Purpose:Quincey Koziol2004-06-151-1/+2
| | | | | | | | | | | | Code optimization Description: Avoid memcpy() when setting up new chunk coordinates Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8686] Purpose:Quincey Koziol2004-06-141-15/+20
| | | | | | | | | | | | | Code optimization Description: Eliminate memcpy() when using default DXPL by pointing at existing default object, instead of copying it. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8683] Purpose:Quincey Koziol2004-06-141-17/+13
| | | | | | | | | | | | | Code optimization Description: Use 'size_t' instead of 'hsize_t' to track the number of elements in memory buffers, especially for type conversion. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8673] Purpose:Quincey Koziol2004-06-131-4/+3
| | | | | | | | | | | | | | | | | | Code optimization Description: Revised dataspace selections to use a more "object oriented" mechanism to set the function pointers for each selection and selection iterator. This reduces the amount and number of times that dataspace selection info has to be copied. Additionally, change hyperslab selection information to be dynamically allocated instead of an inline struct. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8651] Purpose:Quincey Koziol2004-06-101-3/+3
| | | | | | | | | | | Code optimization Description: Eliminate redundant memset() when creating chunk map structure. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel
* [svn-r8636] Purpose:Quincey Koziol2004-06-091-37/+65
| | | | | | | | | | | | Code optimization Description: Don't allocate conversion buffer larger than the user's buffer. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel too minor to require h5committest
* [svn-r8613] Purpose:Quincey Koziol2004-06-051-37/+45
| | | | | | | | | | | | | | | | | Refactor code Description: Move chunk and contiguous cached raw data from file information to dataset information. This simplifies a number of internal interfaces, aligns the code with it's purpose better and should allow more optimizations to the chunked data I/O performance. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) h5committest Misc. update:
* [svn-r8600] Purpose:Quincey Koziol2004-05-311-45/+17
| | | | | | | | | | | | | Code optimization Description: Don't recompute the internal index value for looking up the chunk in the hash table, just use the value already computed from iterating through the chunks. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel
* [svn-r8590] Purpose:Quincey Koziol2004-05-271-41/+51
| | | | | | | | | | | | | | | | | | | | | Code optimization & bug fix Description: When dimension information is being stored in the storage layout message on disk, it is stored as 32-bit quantities, possibly truncating the dimension information, if a dimension is greater than 32-bits in size. Solution: Fix the storage layout message problem by revising file format to not store dimension information, since it is already available in the dataspace. Also revise the storage layout data structures to be more compartmentalized for the information for contiguous, chunked and compact storage. Platforms tested: FreeBSD 4.9 (sleipnir) w/parallel Solaris 2.7 (arabica) h5committest
* [svn-r8544] Purpose:Quincey Koziol2004-05-201-43/+43
| | | | | | | | | | | | | | Code optimization Description: Expand the use of macros to inline trivial function pointer lookup and calls to reduce the overall number of functions invoked during normal operation of the library. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8513] Purpose:Quincey Koziol2004-05-131-2/+2
| | | | | | | | | | | | Code optimization Description: Defer creating the span trees for hyperslab selections until they are actually needed (which may be never, in certain circumstances). Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel
* [svn-r8496] Purpose:Quincey Koziol2004-05-081-6/+6
| | | | | | | | | | | | | Code optimization Description: Further reduce the number of copies we make of a hyperslab selection for chunked I/O, especially when we are only going to throw the old selection away for a new one. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel
* [svn-r8488] Purpose:Leon Arber2004-05-061-17/+36
| | | | | | | | | | | | | | | | | | | Bug fix. Description: Fixed handling of trivial data transform expressions (like 5/3 + 3) and some data coversion fixes. Also added more tests to dtransform.c Solution: Added some more checks in the H5Z_xform_reduce_tree function to see if perhaps the transform expression is complicated and is a non-trivial reduction. Added tests for data conversion to dtransform as well as tests for a trivial data transform expression. Platforms tested: h5committest'ed, except used arabica instead of sol and didn't do on copper b/c no logon there. Problem noted with mtime test...doesn't appear to be related to anything having to do with data transforms.
* [svn-r8482] Purpose:Quincey Koziol2004-05-051-47/+20
| | | | | | | | | | | | Code cleanup Description: Refactored data transform code to reduce amount of symbols in the global scope and also cleaned up & simplified the code a bit. Platforms tested: h5committest (minus copper, plus serial modi4) FreeBSD 4.9 (sleipnir) w & w/o parallel
* [svn-r8479] Purpose:Albert Cheng2004-05-031-14/+80
| | | | | | | | | | | | | New Feature Description: Add the data transform function, H5Pset_transform(). Platforms tested: "h5committested". Copper was down. Ran parallel tests in sol instead. Misc. update:
* [svn-r8462] Purpose:Quincey Koziol2004-05-011-69/+48
| | | | | | | | | | | | | Code optimization Description: Reduce the number of times the number of elements in a selection is computed. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel too minor to require h5committest
* [svn-r8460] Purpose:Quincey Koziol2004-05-011-12/+12
| | | | | | | | | | | | | | | Code optimization Description: Move the element size for the selection into the selection iterator instead of always passing it as a parameter. Also, eleminate another 64-bit multiply for "all" selections. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r8446] Purpose:Quincey Koziol2004-05-011-50/+10
| | | | | | | | | | | | | Code optimization Description: Avoid dividing the chunk coordinates at the top levels of the chunk I/O routines, only to multiply them at the bottom of the routines. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r8383] Purpose:Quincey Koziol2004-04-181-4/+4
| | | | | | | | | | | | | Code cleanup Description: Clean up lots of warnings based on those reported from the SGI compilers as well as gcc. Platforms tested: SGI O3900, IRIX64 6.5 (Cheryl's SGI machine) FreeBSD 4.9 (sleipnir) w/ & w/o parallel h5committest
* [svn-r8376] Purpose:Quincey Koziol2004-04-171-8/+1
| | | | | | | | | | | | | | | | | Code cleanup Description: Update null dataspace changes to try to write older version of dataspace information whenever possible. Refactor common code to only one location. Allow I/O operations to succeed on null dataspaces. Platforms tested: FreeBSD 4.9 (sleipnir) h5committest
* [svn-r8364] Purpose:Quincey Koziol2004-04-171-4/+8
| | | | | | | | | | | | Code optimization Description: Remove redundant function call by re-using value we already know. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r8342] Purpose:Quincey Koziol2004-04-101-5/+20
| | | | | | | | | | | | Code optimization Description: Remove another dataspace copy, in certain circumstances. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r8337] Purpose:Quincey Koziol2004-04-101-18/+15
| | | | | | | | | | | | Code optimization Description: Avoid another extraneous dataspace copy. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel too minor to require h5committest
* [svn-r8335] Purpose:Quincey Koziol2004-04-101-19/+7
| | | | | | | | | | | | | | | Code optimization Description: Change algorithm to directly use coordinates describing a chunk's position in a dataspace instead of creating a dataspace with the chunk's position selected. This reduces the number of copies of dataspaces we need to keep around. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel too minor to require h5committest
* [svn-r8328] Purpose:Quincey Koziol2004-04-081-80/+96
| | | | | | | | | | | | | Code optimization Description: Check for only performing I/O on a single chunk and re-use memory dataspace, instead of re-creating the same dataspace & selection the hard way. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel too minor to require h5committest
* [svn-r8326] Purpose:Quincey Koziol2004-04-081-28/+15
| | | | | | | | | | | | Code optimization Description: Eliminate more dataspace copying, etc. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel too minor to require h5committest
* [svn-r8321] Purpose:Quincey Koziol2004-04-081-17/+19
| | | | | | | | | | | | Code optimization Description: Reduce the number of dataspace copies made when performing I/O on chunked datasets. Platforms tested: Solaris 2.7 (arabica) too minor to require h5committest
* [svn-r8316] Purpose:Quincey Koziol2004-04-071-79/+52
| | | | | | | | | | | Code optimization Description: Query the dataset creation and transfer properties less often. Platforms tested: Solaris 2.7 (arabica) h5committested
* [svn-r8276] *** empty log message ***Raymond Lu2004-03-241-3/+11
|
* [svn-r8158] Purpose:Quincey Koziol2004-02-061-157/+307
| | | | | | | | | | | | | | | | | Code cleanup/optimization Description: Query property list values once, at the beginning of the I/O routines, instead of querying the property list values multiple (lots!) of times in lower level routines. Solution: Create "property list caches" for internal library queries of the property list values. Platforms tested: IBM p690 (copper) w/parallel & fphdf5 h5committest
* [svn-r8053] Purpose:Quincey Koziol2004-01-131-13/+18
| | | | | | | | | | | | | | | | | | | | Bug fix Description: Correct two problems with variable-length datatypes in datasets: - When overwriting an entire dataset, writing the fill value to the file would be skipped, causing problems for VL datatypes when objects in the file had been unlinked (and thus the space in the file was not all zeros) - When an application has set a fill-value for a dataset and the dataset's datatype contained a VL datatype, the library was filling space on disk with the memory form of the VL information, instead of the disk form. Platforms tested: FreeBSD 4.9 (sleipnir) h5committest
* [svn-r7943] Purpose:Quincey Koziol2003-12-131-2/+10
| | | | | | | | | | | | | | | Bug fix. Description: Using a selection offset with hyperslab selections in chunked datasets was getting into an infinite loop and hanging the application. Solution: Apply the selection offset to the hyperslab selection properly. Platforms tested: FreeBSD 4.9 (sleipnir) w & w/o parallel h5committest
* [svn-r7917] Purpose:Quincey Koziol2003-12-061-16/+16
| | | | | | | | | | | | | | | | | | | Code cleanup Description: Clean up compiler warnings, especially the 'FUNC' variable not used which comes out in production mode. Solution: Had to add a new FUNC_ENTER_NOAPI_NOINIT_NOFUNC macro for those non-API functions which don't need the 'FUNC' variable defined. (This will be _so_ much easier when C99 is standard on all our supposed platforms, since it has a __FUNC__ macro... ) Platforms tested: FreeBSD 4.9 (sleipnir) too minor for h5committest (although there were lots of files changed, the change was minor in each one)
* [svn-r7842] Purpose:Quincey Koziol2003-11-131-16/+19
| | | | | | | | | | | | | | | | | | | Bug fix Description: Variable length strings and sequences with NULL pointers were not handled by library, causing problems access the data. This also affected fill values for variable-length datatypes. Solution: Address the issues in the library by detecting NULL sequences/strings and avoid trying to convert them. Patched up dumper to display NULL sequences/strings. Platforms tested: FreeBSD 4.9 (sleipnir) h5committest
* [svn-r7759] Purpose:Quincey Koziol2003-10-281-2/+2
| | | | | | | | | | | | | | Fix serial build Description: I accidentally put the "use_par_opt_io" variable in an #ifdef PARALLEL section. Solution: Hoist it out of parallel section Platforms tested: Eyeballed it - very trivial
* [svn-r7754] Purpose:Quincey Koziol2003-10-271-4/+10
| | | | | | | | | | | | | Code cleanup Description: Straighten out more goofiness in the MPI code dealing with collective I/O transfers - mostly make certain that a view is set if-and-only-if collective I/O is occurring on raw data (and vice versa for views and independent I/O) Platforms tested: FreeBSD 4.9 (sleipnir) w/parallel & FPHDF5 too minor to repquire h5committest
* [svn-r7744] Purpose:Quincey Koziol2003-10-271-109/+53
| | | | | | | | | | | | | | Code cleanup Description: More refactoring on setup of MPI-IO parameters: hoisted the code to change the transfer mode from collective to independent up into the H5D_read and H5D_write routines, instead of duplicating it inside each of the chunk/contig read/write routines. Platforms tested: FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r7742] Purpose:Quincey Koziol2003-10-271-77/+44
| | | | | | | | | | | | Code cleanup Description: Clean up setup code for collective I/O transfers to make the logic more obvious. Platforms tested: FreeBSD 4.9 (sleipnir) too minor for h5committest
* [svn-r7551] Purpose:Quincey Koziol2003-10-061-8/+0
| | | | | | | | | | | | | | | Bug/feature fix. Description: Relax restriction on parallel writing to compact datasets to allow partial I/O. Updates to reference manual mentioning the issues involved are delayed until reference manual 'lock' is removed later this week. Platforms tested: FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r7478] Purpose:Quincey Koziol2003-09-161-1/+4
| | | | | | | | | | | | | | | Bug fix Description: H5Dwrite was only checking the "top level" of the datatype to stop parallel I/O on variable-length datatypes. Solution: Make checks "deeper". Platforms tested: FreeBSD 4.9 (sleipnir) h5committest
* [svn-r7384] Purpose:Quincey Koziol2003-08-191-14/+19
| | | | | | | | | | | Code cleanup Description: More linting... Platforms tested: FreeBSD 4.8 (sleipnir) too minor to need h5committest
generated by cgit v0.12 at 2024-11-14 22:50:41 (GMT)