summaryrefslogtreecommitdiffstats
path: root/src/H5Dio.c
Commit message (Collapse)AuthorAgeFilesLines
* [svn-r12844] Description:Quincey Koziol2006-11-021-21/+0
| | | | | | | Fix parallel build failure for property list class initialization refactor. Tested on: AIX (copper) w/parallel
* [svn-r12842] Description:Quincey Koziol2006-11-021-4/+4
| | | | | | | | | | | | | | | | Refactor generic property list initialization code to put property list specific routines in property list modules, instead of scattered to the four winds. Also, introduce property list class initialization objects, to make adding new property list classes in the library easier. Fix daily test failure by using H5Pget_elink_prefix() API routine instead of looking at the "raw" generic property list information. Tested on: Mac OS X/32 10.4.8 (amazon) FreeBSD/32 4.11 (sleipnir) w/threadsafe Linux/32 2.4 (heping) w/C++ & FORTRAN Linux/64 2.4 (mir) w/build-all & 1.6 compat
* [svn-r12700] Alert:Quincey Koziol2006-10-021-240/+237
| | | | | | | | | | | | | | | | | | | | | | | | | | | | File format is not stable, don't keep files produced! Description: First stage of checkins modifying the format of groups to support creation order. Implement "dense" storage for links in groups. Try to clarify some of the symbols for the H5L API. Add the H5Pset_latest_format() flag for FAPLs, to choose to use the newest file format options (including "dense" link storage in groups) Add the H5Pset_track_creation_order() flag for GCPLs, to enable creation order tracking in groups (although no index on creation order yet). Remove --enable-group-revision configure flag, as file format issues are now handled in a backwardly/forwardly compatible way. Clean up lots of compiler warnings and other minor formatting issues. Tested on: FreeBSD/32 4.11 (sleipnir) w/threadsafe Linux/32 2.4 (heping) w/FORTRAN & C++ Linux/64 2.4 (mir) w/enable-v1.6 compa Mac OSX/32 10.4.8 (amazon) AIX 5.3 (copper) w/parallel & FORTRAN
* [svn-r12593] Description:Quincey Koziol2006-08-171-10/+14
| | | | | | | | | Clean up some of the parallel chunk I/O compiler warnings and other minor code rearrangement. Tested on: FreeBSD 4.11 (sleipnir) w/paralllel h5committest
* [svn-r12553] This check-in includes the following part of parallel ↵MuQun Yang2006-08-091-1/+3
| | | | | | | | | | | | | | | optimization codes: 1. Provide another option for users to do independent IO with MPI file setview(collectively) 2. With the request of collective IO from users, using Independent IO with MPI file setview if we find collective IO is not good for the applications for IO per chunk(multi-chunk IO) case. Previously we used pure independent IO and that actually performed small IO(IO each row) for this case. The recent performance study suggested the independent IO with file setview can acheieve significantly better performance than collective IO when not many processes participate in the IO. 3. For applications that explicitly choose to do collective IO per chunk case, the library won't do any optimization(gather/broadcast) operations. The library simply passes the collective IO request to MPI-IO. Tested at copper, kagiso, heping, mir and tungsten(cmpi and mpich) Kagiso is using LAM, t_mpi test was broken even. The cchunk10 test failed at heping and mir. I suspected it was an MPICH problem. Will investigate later. Everything passed at copper. at tungsten: the old cmpi bug(failed at esetw) is still there. Other tests passed. Some sequential fheap tests failed at kagiso.
* [svn-r12474] Quincey Koziol2006-07-151-53/+43
| | | | | | | | Clean up some compiler warnings. Tested on: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r12440] Purpose:Quincey Koziol2006-06-271-10/+10
| | | | | | | | | | | Code cleanup Description: Trim trailing whitespace in Makefile.am and C/C++ source files to make diffing changes easier. Platforms tested: None necessary, whitespace only change
* [svn-r12128] Purpose:Quincey Koziol2006-03-221-2/+2
| | | | | | | | | | | Code cleanup Description: Clean up some compiler warnings (esp. those flagged on Windows builds) Platforms tested: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r11963] Purpose:MuQun Yang2006-02-221-0/+1
| | | | | | | | | | | | | | | | | | | | bug fix for daily test Description: if (fm->select_chunk) is not NULL at sequential mode and not set by the library, it shouldn't be free. Otherwise, it will cause failures in the daily test. Solution: Explicitly set (fm->select_chunk) to NULL for non-parallel case. Platforms tested: copper with parallel heping with both sequential and parallel shanti with sequential Misc. update: Hopefully only mir failed at daily test.
* [svn-r11961] Purpose:MuQun Yang2006-02-211-5/+6
| | | | | | | | | | | | | | | | | | | | Bug fix in collective IO code Description: When select none occurs in one process, fm->select_chunk hasn't been allocated the memory(not allocated until create_file_hyper); however, this needs to be checked; it was caught with cmpi at NCSA linux cluster. Solution: Relocate the place to allocate the memory space for fm->select_chunk. Platforms tested: cmpi at tungsten mpich at heping kind of minor and let daily test catch other errors. Misc. update: This won't fix the failure at mir.
* [svn-r11960] Purpose:MuQun Yang2006-02-211-1/+1
| | | | | | | | | | | | | | code clean up Description: Clean up some warnings in collective chunk IO code Solution: Platforms tested: heping Misc. update:
* [svn-r11959] Purpose:MuQun Yang2006-02-201-18/+12
| | | | | | | | | | | | | | | | | | | | bug fix Description: During the process of the combination of collective chunk IO code and the original IO code, The return value of a function call hasn't been initialized. On 32-bit mode at modi4, that value isn't set to 0 by default. So a fake error value returns. Solution: Re-arrange the code so that the return value will be set properly. Platforms tested: parallel: modi4 - 32bit heping seq: heping Misc. update:
* [svn-r11950] Purpose:MuQun Yang2006-02-161-309/+127
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Enhanced collective chunk IO supports Description: 1. When using collective IO with chunking storage without any tuning, performance may become worse under some circumstances. 2. Current HDF5 handles raw-data IO per chunk. So for many small chunks, many small IOs will be passed into MPI-IO. That may cause bad performance. 3. For one IO per chunk case, sometimes performance with collective is worse than performance with independent. An obvious case is when only one process is doing IO and all other process are not doing IO, the collective IO will only add overheads for communication. We want to avoid this case. Some management inside our library needs to be done. Solution: - Added managements of collective IO supports for chunking storage inside parallel HDF5 1) Implemented One IO with collective mode for all chunks in the application by building one MPI derived datatype accross all chunks. 2) Implemented the decision-making support to do collective IO inside MPI-IO per chunk. 3) Added the decision-making support to do one IO accross all chunks or to do multiple IOs with each IO per chunk. 4) Added the support to handle the case some processes won't do any IOs in collectively. 5) Some MPI-IO package(mpich 1.2.6 or lower, e.g.) cannot handle collective IO correctly for the case when some processes have no contributions to IOs, a special macro is added to change collective IO mode to independent IO mode inside HDF5 library. Platforms tested: Parallel: IBM AIX 5.2(copper) Linux (heping) mpich-1.2.6 SDSC Teragrid mpich-1.2.5 Linux(Tungsten) mpich-1.2.6 Altix(NCSA cobalt) Seq: Linux(heping) Misc. update:
* [svn-r11712] Purpose:Quincey Koziol2005-11-151-42/+49
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | New feature Description: Check in baseline for compact group revisions, which radically revises the source code for managing groups and object headers. WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! This initiates the "unstable" phase of the 1.7.x branch, leading up to the 1.8.0 release. Please test this code, but do _NOT_ keep files created with it - the format will change again before the release and you will not be able to read your old files!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! Solution: There's too many changes to really describe them all, but some of them include: - Stop abusing the H5G_entry_t structure and split it into two separate structures for non-symbol table node use within the library: H5O_loc_t for object locations in a file and H5G_name_t to store the path to an opened object. H5G_entry_t is now only used for storing symbol table entries on disk. - Retire H5G_namei() in favor of a more general mechanism for traversing group paths and issuing callbacks on objects located. This gets us out of the business of hacking H5G_namei() for new features, generally. - Revised H5O* routines to take a H5O_loc_t instead of H5G_entry_t - Lots more... Platforms tested: h5committested and maybe another dozen configurations.... :-)
* [svn-r11593] Purpose:Quincey Koziol2005-10-211-146/+30
| | | | | | | | | | | | Code cleanup Description: Clean up & standardize a bit in preparation for coding standards discussion. Platforms tested: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r11415] Purpose: Internal design change.Raymond Lu2005-09-141-3/+3
| | | | | | | | | | | | | | | | | Description: H5T_register() replaces any existing data conversion path with a new path. Solution: Added a parameter to H5T_register() and H5T_path_find() to signal the library whether the new conversion path is from API function H5Tregister() or from private function like H5T_init_interface(). If it is from H5Tregister(), replace existing path. If it's from H5T_init_interface() and the library is trying to register default hard conversion functions, don't replace existing path because the path is registered by the library. For example, the library registered H5T_conv_int_float() and is trying to register H5T_conv_long_float(). The size of long is equal to int. There's no need to replace the path H5T_conv_int_float(). Platforms tested: h5committest and fuss.
* [svn-r11402] Purpose: MaintenanceElena Pourmal2005-09-131-2/+2
| | | | | | | | | | | | Description: Compiler complained about const type * variables that were used in the functions that require "non const" parameters. Compilation failed. Solution: Use casting to avoid compilation errors. Platforms tested: SX-6, heping and mir with new g95 Misc. update:
* [svn-r11245] Purpose:Quincey Koziol2005-08-131-7/+7
| | | | | | | | | | | | | | | | | | | | Code cleanup Description: Trim trailing whitespace, which is making 'diff'ing the two branches difficult. Solution: Ran this script in each directory: foreach f (*.[ch] *.cpp) sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f end Platforms tested: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r11241] Purpose:Quincey Koziol2005-08-131-334/+113
| | | | | | | | | | | | | Code cleanup Description: Fix logic error in previous checkin and also finish refactoring I/O initialization, including simplifying all the collective & parallel cases into a more unified mechanism. Platforms tested: FreeBSD 4.11 (sleipnir) w/ & w/o parallel Linux 2.4 (mir)
* [svn-r11240] Purpose:Quincey Koziol2005-08-131-66/+82
| | | | | | | | | | | Bug fix Description: Fix logic error for non-parallel file I/O with library built for parallel I/O capability. Platforms tested: Linux 2.4 (mir) w/parallel
* [svn-r11236] Purpose:Quincey Koziol2005-08-131-421/+298
| | | | | | | | | | | | | | Code cleanup & bug fix Description: Refactor recent collective chunk changes to simplify them. Also, compute the number of chunks by checking the number of objects in the chunk skip list instead of getting the boundaries of the selection and counting the chunks that it overlaps, which is faster and computes correct number for certain selections. Platforms tested: FreeBSD 4.11 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r11235] Purpose:Quincey Koziol2005-08-121-464/+377
| | | | | | | | | | | | | Code cleanup Description: Refactor, simplify and cleanup Kent's recent checking for collective chunk I/O. There's a bug that I need to talk to Kent about and some more cleanups still, but this is reasonable for an interim point. Platforms tested: FreeBSD 4.11 (sleipnir) w/parallel Too minor for h5committest
* [svn-r11231] Purpose:MuQun Yang2005-08-111-38/+492
| | | | | | | | | | | | | | | | | | | | | | | bug fix for collective chunk IO, phase 1 Optimization hasn't been done yet, the collective chunk IO bug should be fixed. Description: In chunking storage, memory space and file space will be remapped, So to check whether file space and memory space are regular in order to use optimized MPI derived datatype for collective call one has to check per-chunk wise instead of per hyperslab wise. Even a regular memory space will be stored in span-tree and will be irregular before chunk IO. Solution: 1. Check file space and memory space per chunk wise instead of per hyperslab wise. 2. For collective IO mode, number of chunks covered by hyperslab may be different. Since we are handing per chunk per IO, for the extra chunk IO for some(not all) processors, collective mode will cause program hanged. So for the extra chunk Io mode independent IO has to be used. 3. On some platforms, Complex MPI derived datatype is not working, so we have to use independent IO for collective IO mode if the selection is irregular. However, when the selection is regular, we do want to use collective IO since that will improve performance. Special cares have to be added for this case. Platforms tested: copper(AIX 5.1) Linux(heping mpich 1.2.6), Teragrid machine, Cobalt(altix), modi4 Misc. update:
* [svn-r10951] Purpose:Quincey Koziol2005-06-181-6/+16
| | | | | | | | | | | | | | | | | Bug fix Description: Hyperslab selections that had a selection offset and were applied to a chunked dataset could get into an infinite loop or core dump if the same selection was used multiple times, with different selection offsets. Solution: "Normalize" the selection with the selection offset, generate the selections for the chunks overlapped and then "denormalize" the selection. Platforms tested: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r10705] Purpose:Quincey Koziol2005-04-301-1/+1
| | | | | | | | | | | Code cleanup Description: Clean up a few compiler warnings of various sorts... Platforms tested: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r10574] Purpose:Quincey Koziol2005-04-081-2/+2
| | | | | | | | | | | Code cleanup Description: Clarify error string. Platforms tested: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r10545] Purpose:MuQun Yang2005-04-051-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | Activating collective IO supports for irregular selction inside HDF5 dataset. This support doesn't include to build the final complicated MPI derived datatype support for chunking storage. Description: Support collective chunk IO for both contiguous and chunking storage for irregular selection( using H5S_SELECT_OR for multiple hyperslab selection) Solution: Using MPI derived datatype to realize this feature. Problems still need to be investigated: Big size irregular hyperslab selection test may cause MPI hang or abnormalexit with MPICH family on various platforms. This is really hard to debug since sometimes it can work and sometimes it cannot work. We will continue investigating those cases. This may not be parallel HDF5 bugs since with the recent version of poe at IBM platforms, all tests passed. Platforms tested: 1. Linux heping 2.4.21-27.0.1 with mpich 2. AIX 5.1 copper with mpcc_r 3. Altix cobalt SGI linux 2.4.21-sgi304rp05031014_10149 with icc -lmpi 4. Linux Cluster SDSC TG, intel 8-r2 with mpich 1.2.5 5. NCSA Linux Cluster Tungsten, MPICH-TCP-1.2.5.2, Intel 8.0 under lustre 6. NCSA Linux Cluster Tungsten, MPICH-LAM-INTEL711, sometimes not working 7. NCSA Linux CLuster Tungsten, champion-pro-1.2.0-3, not working for other collective IO tests, but work for irregular selection collective IO test. Misc. update:
* [svn-r10187] Purpose:Quincey Koziol2005-03-111-1/+1
| | | | | | | | | | | | | Code cleanup, mostly Description: Remove remaining TBBT error info Add new error code for block tracker Platforms tested: FreeBSD 4.11 (sleipnir) Solaris 2.9 (shanti)
* [svn-r9959] Purpose: Bug fixRaymond Lu2005-02-081-1/+1
| | | | | | | | | | | | Description: For variable-length string, H5Tget_class returned H5T_STRING as its class. But H5Tdetect_class and H5Tget_member_class considered it as H5T_VLEN. This is fixed to let all these 3 functions treat it as H5T_STRING. Some test cases have been added to dtypes.c Platforms tested: heping - already tested for v1.6 with h5committest Misc. update: RELEASE.txt
* [svn-r9857] Purpose: MaintenanceElena Pourmal2005-01-221-3/+0
| | | | | | | | | | | | | | Description: Removed PABLO from the source Solution: Platforms tested: arabica with 64-bit, copper with parallel, heping with GNU C and C++ and PGI fortran (but I disabled hl, there is some weird problem only on heping: F9XMODFLAG is not propagated to the Makefile files Misc. update:
* [svn-r9762] Purpose:Quincey Koziol2005-01-061-1/+1
| | | | | | | | | | Code cleanup Description: Change pablo mask to conform to the style used by the rest of the library Platforms tested: None needed - very, very minor
* [svn-r9734] Purpose:Quincey Koziol2004-12-301-105/+98
| | | | | | | | | | | | Code cleanup Description: Convert chunk iteration code to use skip lists instead of threaded, balanced binary trees. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel & szip Too minor to require h5committest
* [svn-r9727] Purpose:Quincey Koziol2004-12-291-55/+81
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bug Fix/Code Cleanup/Doc Cleanup/Optimization/Branch Sync :-) Description: Generally speaking, this is the "signed->unsigned" change to selections. However, in the process of merging code back, things got stickier and stickier until I ended up doing a big "sync the two branches up" operation. So... I brought back all the "infrastructure" fixes from the development branch to the release branch (which I think were actually making some improvement in performance) as well as fixed several bugs which had been fixed in one branch, but not the other. I've also tagged the repository before making this checkin with the label "before_signed_unsigned_changes". Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel & fphdf5 FreeBSD 4.10 (sleipnir) w/threadsafe FreeBSD 4.10 (sleipnir) w/backward compatibility Solaris 2.7 (arabica) w/"purify options" Solaris 2.8 (sol) w/FORTRAN & C++ AIX 5.x (copper) w/parallel & FORTRAN IRIX64 6.5 (modi4) w/FORTRAN Linux 2.4 (heping) w/FORTRAN & C++ Misc. update:
* [svn-r9665] Purpose:Quincey Koziol2004-12-141-0/+1
| | | | | | | | | | Fix whitespace Description: Clean up whitespace a bit Platforms tested: None, just eyeballed, very minor
* [svn-r9657] Purpose:MuQun Yang2004-12-131-2/+2
| | | | | | | | | | | | | | | | | | small bug fix Description: When checking whether the current chunk covers the irregular hyperslab, the ending point of the chunk is not updated. This will cause the wrong checking output and fail for some irregular hyperslab selection. Solution: Updating the ending point of the chunk. Platforms tested: Linux 2.4 + parallel (too small to use h5committest) Misc. update:
* [svn-r9574] Purpose:MuQun Yang2004-11-241-9/+10
| | | | | | | | | | | | | | | Adding code for using MPI derived datatype to handle collective IO Description: No testing yet, won't affect the library. Solution: Platforms tested: linux 2.4 + mpich 1.2.6 Aix 5.1 + mpcc_r Misc. update:
* [svn-r9529] Purpose:MuQun Yang2004-11-151-3/+8
| | | | | | | | | | | | | | | | | | | Check in some new fixes for MPI derived datatype routines Description: MPI derived datatype algorithm seems working for a simple case; however, there are still other problems need to be solved. So the code cannot be used for the time being. Check-in only for debugging. It won't affect other part of the library. Solution: Platforms tested: Linux 2.4 (heping, serial and parallel) (Since no new tests were added and changes are mostly restricted to one fuction, no need to test three platforms). Misc. update:
* [svn-r9519] Purpose:MuQun Yang2004-11-111-0/+6
| | | | | | | | | | | | | | | | | | | | Adding codes for the general MPI derived datatype in order to better incorporate new fixes of HDF5 library. Description: Note: These codes have not been tested for general use. Don't call these functions in your developments of the HDF5 library. Also these codes are stand-alone codes, they should not affect other library codes. Solution: Platforms tested: Heping(C and Parallel linux 2.4, mpich 1.2.6) Arabica(C,C++,Fortran, Solaris 2.7) Copper(C,c++,Fortran, AIX 5.1, NOTE: c++ FAILED, seems not due to the recent check-in) Misc. update:
* [svn-r9413] Purpose:Quincey Koziol2004-10-121-2/+2
| | | | | | | | | | | Code cleanup Description: Fix a couple of return values from NULL -> FAIL. Platforms tested: FreeBSD 4.10 (sleipnir) Too minor to require h5committest
* [svn-r9358] Purpose:Quincey Koziol2004-10-041-1/+1
| | | | | | | | | | | | | | | Bug fix Description: Relax restrictions on parallel I/O to allow compressed, chunked datasets to be read in parallel (collective access will be degraded to independent access, but will retrieve the information still). Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Solaris 2.7 (arabica) IRIX64 6.5 (modi4) h5committest
* [svn-r9354] Purpose:Quincey Koziol2004-10-011-137/+342
| | | | | | | | | | | | | | | | Bug fix & code cleanup Description: More dataset cleanups to get to a point where we can fix the chunked I/O bug. Also fix a couple of errors in the recent file object resurrection changes which should hopefully address the recent daily test failres (H5T.c) Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Solaris 2.7 (arabica) h5committest
* [svn-r9342] Purpose:Quincey Koziol2004-09-301-181/+45
| | | | | | | | | | | | | | | | Bug fix/code cleanup Description: Clean up raw data I/O code to bundle the I/O parameters (dataset, DXPL ID, etc) into a single struct to pass around through the dataset I/O routines, since they are always passed together, until very near the bottom of the I/O stack. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Solaris 2.7 (arabica) IRIX64 6.5 (modi4) h5committest
* [svn-r9329] James Laird2004-09-281-74/+74
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Purpose: Feature Description: Datatypes and groups now use H5FO "file object" code that was previously only used by datasets. These objects will hold a file open if the file is closed but they have not yet been closed. If these objects are unlinked then relinked, they will not be destroyed. If they are opened twice (even by two different names), both IDs will "see" changes made to the object using the other ID. When an object is opened using two different names (e.g., if a dataset was opened under one name, then mounted and opened under its new name), calling H5Iget_name() on a given hid_t will return the name used to open that hid_t, not the current name of the object (this is a feature, and a change from the previous behavior of datasets). Solution: Used H5FO code that was already in place for datasets. Broke H5D_t's, H5T_t's, and H5G_t's into a "shared" struct and a private struct. The shared structs (H5D_shared_t, etc.) hold the object's information and are used by all IDs that point to a given object in the file. The private structs are pointed to by the hid_t and contain the object's group entry information (including its name) and a pointer to the shared struct for that object. This changed the naming of structs throughout the library (e.g., datatype->size is now datatype->shared->size). I added an updated H5Tinit.c to windows.zip. Platforms tested: Visual Studio 7, sleipnir, arabica, verbena Misc. update:
* [svn-r9312] Purpose:Quincey Koziol2004-09-231-6/+6
| | | | | | | | | | Bug fix. Description: Allow I/O to occur on 0 element selections. Platforms tested: h5committest
* [svn-r8987] Purpose:Quincey Koziol2004-08-021-3/+2
| | | | | | | | | | | | Code cleanup Description: Fix another batch of minor differences between the development and release branches. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8953] James Laird2004-07-271-0/+16
| | | | | | | | | | | | | | | | Purpose: Bug fix Description: When a simple dataspace is created, its extent should be set before using it, or it will silently function as a NULL dataspace. Solution: Added checks on user-supplied dataspaces. Now dataspaces without extents set will throw errors; users must explicitly set a dataspace to be NULL. Platforms tested: sleipnir, windows
* [svn-r8941] Purpose:Quincey Koziol2004-07-261-2/+6
| | | | | | | | | | | | | | Code optimization Description: Changed calloc() calls to malloc() calls allocating background buffers during dataset writes, since the background buffer information will be read from disk anyway, overwriting any existing values. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8932] Purpose:Quincey Koziol2004-07-221-18/+23
| | | | | | | | | | | | | | | | | | | | Code cleanup Description: Clean up collective chunking code a bit. Also, add '--enable-instrument' configure flag to have a mechanism for determining that optimized operations happened correctly in the library (instead of just the "normal" way) by allowing 'flag' properties to be set outside the library and set when the "right" thing happens. This is mainly for debugging and regression checks, so we make certain we don't break optimized I/O by accident. It's enabled by default when --enable-debug is on (which is on by default in the development branch and off by default in the release branch), but can also be independently controlled with its own configure flag. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel IBM p690 (copper) w/parallel
* [svn-r8923] Purpose:MuQun Yang2004-07-211-0/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | To add a special section of codes for collective-chunk IO tests. Description: The current patch of collective chunk IO support in HDF5 can only handle some special("however, can cover many applications") cases, Inside source code, we did a careful checking to make sure other cases would not fall into this category and would not use collective IO. We also would like to test whether those collective conditions were met in our test programs. The current parallel HDF5 handled those collective IO requests in a special way. If the library finds it cannot do collective IO, it will silently change to independent IO. So basically there is no better way to check whether the library is doing what it should do without "hacking" into HDF5 source codes for the purpose of testing. But the "hacking" should not affect library work and should be easily pulled out after we get more general collective IO algorithm to work. With Quincey's suggestion, we used HDF5 property APIs to finish the job. Solution: The approach includes three parts: 1) In the test program, insert a property inside data transfer property list. Set a default value for this property. 2) Inside H5Dio.c, when the library finds that it cannot do collective IO with chunking storage, it will change the default value. 3) Then the test program will recheck the value after H5Dwrite or H5Dread to evaluate whether the current collective IO case is doing the right thing. Note: The test won't stop after it finds that the library is not doing the right thing and probably it will finish normally. The current approach is that the test program just printed out an error message. It should be changed later. Platforms tested:o copper,arabica,eirene Misc. update:
* [svn-r8906] Purpose:MuQun Yang2004-07-201-2/+2
| | | | | | | | | | | | | | | | | | | | | | | Adding the first round of patches about supporting collective chunk IO in HDF5 Description: The current HDF5 library doesn't support collective MPIO with chunk storage. When users set collective option in their data transfer with chunk storage, the library silently converted the option to INDEPENDENT and that caused trememdous performance penalty. Some application like WRF-parallel HDF5 IO module has to use contiguous storage for this reason. However, chunking storage has its own advantage(supporting compression filters and extensible dataset), so to make collective MPIO possible inside HDF5 with chunking storage is a very important task. This check-in make collective chunk IO possible for some special cases. The condition is as follows(either case is fine with using collective chunk IO) 1. for each process, the hyperslab selection of the file data space of each dataset is regular and it is fit in one chunk. 2. for each process, the hyperslab selection of the file data space of each dataset is single and the number of chunks for the hyperslab selection should be equal. Solution: Lift up the contiguous storage requirement for collective IO. Use H5D_isstore_get_addr to get the corresponding chunk address. Then the original library routines will take care of getting the correct address to make sure that MPI FILE TYPE is built correctly for collective IO> Platforms tested: arabica(sol), copper(AIX), eirene(Linux) parallel test is checked at copper. Misc. update:
generated by cgit v0.12 at 2024-10-30 18:16:19 (GMT)