summaryrefslogtreecommitdiffstats
path: root/src/H5Dprivate.h
Commit message (Collapse)AuthorAgeFilesLines
* [svn-r25490] bring 25468 from trunk. Mohamad Chaarawi2014-07-291-0/+3
| | | | | | | Aligning code with changes made in VOL branch. This will make porting non VOL related changes from trunkto the 1.8 branch much easier. tested h5commitest.
* [svn-r24785] Description:Quincey Koziol2014-03-121-3/+0
| | | | | | | | | | | | | Bring r24769 & r24772 from trunk to 1.8 branch: Check in Mohamad's changes to support collective I/O on point selections, along with some other minor cleanups. Correct some dynamically sized arrays that fail on Windows. Tested on: Mac OSX/64 10.9.2 (amazon) w/parallel (And has been through the daily test regime on the trunk)
* [svn-r23702] Issue 8380 - H5Zunregister caused seg fault. This is a porting ↵Raymond Lu2013-05-141-0/+1
| | | | | | | | | of code from the trunk. It fixes the problem by first iterating through all opened datasets and groups to see if any of them uses the filter. If it finds one, the function fails with a message. Then the function flushes all opened files. Tested with h5committest.
* [svn-r23182] I'm merging the changes for DECTRIS project from the trunk ↵Raymond Lu2013-01-211-0/+3
| | | | | | (r23162). Tested on koala, jam, ostrich.
* [svn-r23011] Purpose:Jonathan Kim2012-11-051-2/+0
| | | | | | | | | | | | HDFFV-8146 - Remove "multi-chunk IO without optimization" sub-feature from MPI I/O optimization for chunked dataset feature Description: The “multi-chunk IO without optimization” feature is removed and made the related xfer property (H5FD_MPIO_CHUNK_MULTI_IO) go directly to “multi-chunk-io” feature. Also update/fix/cleanup testings (chunk collective IO and actual chunk opt mode) accordingly. Merged from HDF5 trunk r22992. Tested: jam-pp (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), fred (mac64-LE), linew (solaris-BE), koala-pp, wallby-pp, cmake (jam-pp), Windows (32-LE cmake)
* [svn-r22780] Purpose:Jonathan Kim2012-09-171-0/+2
| | | | | | | | | | | | | | | | | | | | | HDFFV-8143 Provide a routine(s) for telling the user why the library broke collective data access Description: Added H5Pget_mpio_no_collective_cause() function that retrive reasons why the collective I/O was broken during Read/Write IO access. Reasons to break collective I/O: - SET_INDEPENDENT - DATATYPE_CONVERSION - DATA_TRANSFORMS - MPIPOSIX - NOT_SIMPLE_OR_SCALAR_DATASPACES (NULL Space) - POINT_SELECTIONS - NOT_CONTIGUOUS_OR_CHUNKED_DATASET (Compact or External-Storage) - FILTERS Merged from HDF5 trunk r22735, r22741 , 22743, 22744, 22763. Tested: jam-pp (linux32-LE), koala-pp (linux64-LE), wallaby-pp
* [svn-r22291] Description:Quincey Koziol2012-04-171-6/+0
| | | | | | | | | | Bring r22287 from trunk to 1.8 branch: Clean up more FUNC_ENTER/FUNC_LEAVE macros and move H5D & H5T code toward the final design (as exemplified by the H5EA & H5FA code). Tested on: Mac OSX/64 10.7.3 (amazon) w/debug & parallel
* [svn-r21533] Description:Quincey Koziol2011-10-121-0/+2
| | | | | | | | | Bring r21442 and r21529 from trunk to 1.8 branch: Add new API routines for querying if collective raw data I/O was actually performed collectively. Tested: Linux/32 2.6 (jam) (h5committested on trunk)
* [svn-r21524] Reverts 20920 (HDFFV-7579 fix) (this is 20913/21 from the trunk)Dana Robinson2011-10-111-1/+0
| | | | | | This change reverts the HDFFV-7579 fix while we more carefully consider enum conversions as a part of the h5compare work. Tested on jam, Windows 7 and linew.
* [svn-r20920] Issue 7579 - The overflowing ENUM values are inconsistent. ↵Raymond Lu2011-05-271-0/+1
| | | | | | | | | | When no conversion is involved in reading or writing the data, overflowing values are retained. When conversion happens, the values become -1. The conversion function puts -1 when overflow happens. I added two new dataset transfer property to control whether to fill 0xff in the destination data or convert to the destination data when overflow happens. The two new functions are H5Pset(get)_enum_conv_overflow. I also added test cases in enum.c and dtypes.c. Tested on jam, koala, and heiwa.
* [svn-r17807] Description:Quincey Koziol2009-11-031-3/+2
| | | | | | | | | | | | | | | | | | | | | | | Bring r17623 from trunk to 1.8 branch: Bring "compress group's fractal heap" feature from feature branch. Tested on: FreeBSD/32 6.3 (duty) in debug mode FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/Intel compilers, w/default API=1.6.x, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode Mac OS X/32 10.6.1 (amazon) in debug mode Mac OS X/32 10.6.1 (amazon) w/C++ & FORTRAN, w/threadsafe, in production mode
* [svn-r17600] Description:Quincey Koziol2009-10-061-1/+1
| | | | | | | | | | | Bring r17599 from trunk to 1.8 branch: Minor code cleanups, based on cleanups in the compress_group_fheap branch. Tested on: FreeBSD/32 6.3 (duty) in debug mode (h5committested on trunk)
* [svn-r17404] Description:Quincey Koziol2009-08-211-1/+1
| | | | | | | | | | | | | | | | | | | | | | Bring r17402 & r17403 from trunk to 1.8 branch: r17402: Switch from using a 'flags' parameter to the flush routine(s), to just using a single 'closing' boolean parameter, since that's all we're doing with the flags anyway and this makes things more obvious. r17403: Eliminate 'closing' flag to dataset flush routine, since all cached dataset information has already been flushed when a file is closed and the datasets themselves will be closed (and the dataset caches destroyed). Skip calling more flush routines when the file is closing, in preparation for eventual separation of the 'flush' concept from the 'shutdown the cache' concept. Tested on: FreeBSD/32 6.3 (duty) in debug mode (h5committest performed on trunk)
* [svn-r17282] Description:Quincey Koziol2009-07-311-0/+7
| | | | | | | | | | | Bring r17281 from trunk to 1.8 branch: Further refactoring of the dataset layout information, to separate the storage information from the layout info. Tested on: Mac OS X/32 10.5.7 (amazon) debug & production h5committested on trunk
* [svn-r17277] Description:Quincey Koziol2009-07-301-3/+3
| | | | | | | | Bring r17275 and r17276 from trunk to 1.8 branch. Tested on: Mac OS X/32 10.5.7 (amazon) w/debug & production (too minor to require h5committest)
* [svn-r17147] Description:Quincey Koziol2009-07-021-2/+3
| | | | | | | | | | | Bring r17146 from trunk to 1.8 branch: Bring a bunch of misc. improvements & tweaks from the revise_chunks branch back to the trunk, so that future merges won't be so painful. Tested on: FreeBSd/32 6.3 (duty) (h5committested on trunk)
* [svn-r16356] Description:Quincey Koziol2009-01-271-0/+5
| | | | | | | | | | | Bring r16354 back from trunk: Refactor internal layout information, making it easier to add another type of chunk index. Tested on: FreeBSD/32 (duty) (other configurations tested with original change)
* [svn-r16280] Description:Quincey Koziol2009-01-081-2/+0
| | | | | | | | | | | | | | Bring revision 16278 back from revise_chunks branch: Update layout information in DCPL to unify all information in one underlying property and switch to using H5O_layout_t for storing it, which simplifies things considerably. Also, fix many compiler warnings. Tested on: FreeBSD/32 6.3 (duty) in debug mode (Original patch tested on many machines)
* [svn-r16183] Description:Quincey Koziol2008-12-111-1/+1
| | | | | | | | | | Bring r16182 back from trunk: Rename internal routines, variables, macros, typedefs, etc. for chunked dataset storage from "istore" to some variant of "chunk" or "btree". Tested on: FreeBSD/32 6.3 (duty) in debug mode
* [svn-r16087] Purpose: Port H5Pset_chunk_cache, H5Pget_chunk_cache, and ↵Neil Fortner2008-11-171-1/+6
| | | | | | | | H5Dget_access_plist functions to 1.8 branch Tested: kagiso, jam (trunk version tested with h5committest)
* [svn-r15624] This check-in fixes a bug inside parallel HDF5 testsuite.MuQun Yang2008-09-151-0/+4
| | | | | | When enable-debug is turned on, a special macro block H5_HAVE_INSTRUMENTED_LIBRARY inside HDF5 will be executed to check if some collective chunk IO test cases are being run with the correct settings(one link, multiple chunk etc.). However,when complicated derived datatype in some mpi-io packages are not supported, the library has to switch one link IO with/without the optimization to multiple chunk IO with/without the optimization. The current testsuite doesn't know this and generates a false assertion failure message. This check-in fix this problem by providing a second property to avoid the false faiure message.
* [svn-r15132] Description:Quincey Koziol2008-06-031-2/+3
| | | | | | | | | | | | | | Bring back revision 15131 from trunk: Finish omnibus chunked dataset I/O refactoring, to separate general actions on chunked datasets from actions that are specific to using the v1 B-tree index. Cleaned up a few bugs and added some additional tests also. Tested on: Mac OS X/32 10.5.3 (amazon) Linux/32 2.4 (chicago)
* [svn-r14861] Description:Quincey Koziol2008-04-241-21/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Bring revision 14860 back to 1.8 branch, change log for rev 14860 is: Omnibus raw data I/O revisions, with wide-ranging changes and refactoring, in order to prepare for implementing "fast append" feature. These changes remove the majority of the code duplication for raw data I/O which has crept in over the last ten years and introduces a more object- oriented design for operating on different types of dataset storage. Description: Omnibus raw data I/O revisions, with wide-ranging changes and refactoring, in order to prepare for implementing "fast append" feature. These changes remove the majority of the code duplication for raw data I/O which has crept in over the last ten years and introduces a more object- oriented design for operating on different types of dataset storage. Chunked storage no longer has it's own I/O routines, it is now handled as either contiguous (if chunk is not pulled into the cache) or compact (if the chunk is cached in memory). No bug or feature changes, at least intentionally... :-) Tested on: Mac OS X/32 10.5.2 (amazon) w/production
* [svn-r13253] Updated all C and C++ style source code files with the THG ↵Albert Cheng2007-02-071-2/+3
| | | | | | | | | copyright notice. Tested platform: Kagiso only since it is only a comment block change. If it works in one machine, it should work in all, I hope. Still need to check the parallel build on copper.
* [svn-r13167] Description:Quincey Koziol2007-01-221-4/+2
| | | | | | | | | | | Get rid of two different types of fill value struct (merge H5O_fill_t and H5O_fill_new_t) and clean up & simplify dataset initialization code. (In preparation for shared object header message method call refactoring). Tested on: FreeBSD/32 6.2 (duty) Mac OS X/32 10.4.8 (amazon)
* [svn-r12845] Description:Quincey Koziol2006-11-021-0/+1
| | | | | | | | | | | | | Finish refactoring job on the library's property list class initialization code, so that the library determines the parent class dependencies at run-time, eliminating the need for developers to initialize the classes in a particular order. Also eliminates some more redundant code... Tested on: FreeBSD/32 4.11 (sleipnir) w/threadsafe & debugging turned on Linux/32 2.4 (heping) w/FORTRAN & C++ Linux/64 2.4 (mir) w/build-all & 1.6 compat enabled AIX/32 5.x (copper) w/FORTRAN & parallel
* [svn-r12844] Description:Quincey Koziol2006-11-021-1/+4
| | | | | | | Fix parallel build failure for property list class initialization refactor. Tested on: AIX (copper) w/parallel
* [svn-r12842] Description:Quincey Koziol2006-11-021-150/+47
| | | | | | | | | | | | | | | | Refactor generic property list initialization code to put property list specific routines in property list modules, instead of scattered to the four winds. Also, introduce property list class initialization objects, to make adding new property list classes in the library easier. Fix daily test failure by using H5Pget_elink_prefix() API routine instead of looking at the "raw" generic property list information. Tested on: Mac OS X/32 10.4.8 (amazon) FreeBSD/32 4.11 (sleipnir) w/threadsafe Linux/32 2.4 (heping) w/C++ & FORTRAN Linux/64 2.4 (mir) w/build-all & 1.6 compat
* [svn-r12553] This check-in includes the following part of parallel ↵MuQun Yang2006-08-091-0/+10
| | | | | | | | | | | | | | | optimization codes: 1. Provide another option for users to do independent IO with MPI file setview(collectively) 2. With the request of collective IO from users, using Independent IO with MPI file setview if we find collective IO is not good for the applications for IO per chunk(multi-chunk IO) case. Previously we used pure independent IO and that actually performed small IO(IO each row) for this case. The recent performance study suggested the independent IO with file setview can acheieve significantly better performance than collective IO when not many processes participate in the IO. 3. For applications that explicitly choose to do collective IO per chunk case, the library won't do any optimization(gather/broadcast) operations. The library simply passes the collective IO request to MPI-IO. Tested at copper, kagiso, heping, mir and tungsten(cmpi and mpich) Kagiso is using LAM, t_mpi test was broken even. The cchunk10 test failed at heping and mir. I suspected it was an MPICH problem. Will investigate later. Everything passed at copper. at tungsten: the old cmpi bug(failed at esetw) is still there. Other tests passed. Some sequential fheap tests failed at kagiso.
* [svn-r12400] Purpose:MuQun Yang2006-06-031-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some collective chunk IO macro names are confusing, change them to more meaningful names. Description: H5Pset_dxpl_mpio_chunk_opt will set a flag so that the library can do one linked IO or multi-chunk IO with collective in chunking storage directly. That is, the library won't do analyses to determine this. The flags for the enum type we used before are: H5FD_MPIO_OPT_ONE_IO H5FD_MPIO_OPT_MULTI_IO They are not good names because of the following two reasons: 1. It doesn't reflect chunking storage 2. OPT is kind of redundant and misleading, Solution: We change the names to H5FD_MPIO_CHUNK_ONE_IO H5FD_MPIO_CHUNK_MULTI_IO Platforms tested: Since only macro names are changed, no need to test with h5committest. heping(mpich 1.2.6) Misc. update:
* [svn-r12173] Purpose:MuQun Yang2006-03-291-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | Adding parallel tests for optional collective chunk APIs Description: Three new APIs "H5Pset_dxpl_mpio_chunk_opt_ratio H5Pset_dxpl_mpio_chunk_opt_num H5Pset_dxpl_mpio_chunk_opt" for optional optimization choices from users have been added to the libraries. This check-in adds six tests to verify the funcationality and correctedness of these APIs. These tests need to be verified with 3 or more processors and with MPI-IO driver only. Solution: Using H5Pinsert, H5Pget, H5Pset to verify that the library indeed goes into the branch we hope for. Using H5_HAVE_INSTRUMENT macro to isolate these changes so that it won't affect or be misused by the application. Platforms tested: h5committest(shanti still refused to be connected) Parallel tests on heping somehow are skipped. Manually testing at heping. Have checked 1,2,3,4,5 processes. Misc. update:
* [svn-r12090] Purpose:MuQun Yang2006-03-141-0/+12
| | | | | | | | | | | | | | | | | | | New APIs to add for collective chunk IO Description: Three new APIs H5Pset_dxpl_mpio_chunk_opt_ratio H5Pset_dxpl_mpio_chunk_opt_num H5Pset_dxpl_mpio_chunk_opt for optional optimization choices from users. Solution: Haven't added tests yet, won't affect other parts of the library. Will add tests after urgent investigations of memory leaking problems from NASA Aura team. Platforms tested: heping: both parallel and sequential shanti Misc. update:
* [svn-r11899] Purpose:Quincey Koziol2006-01-281-0/+5
| | | | | | | | | | | | | | | | Bug fix & new feature Description: Support variable-length datatypes in compact data storage and chunked data storage, along with attributes. Bug fix on the H5T_vlen_set_loc to allow for changing the file on a variable-length datatype on disk. Platforms tested: FreeBSD 4.11 (sleipnir) Linux 2.4 Can't h5committest right now, due to missing cache files.
* [svn-r11758] Purpose:Quincey Koziol2005-12-041-1/+0
| | | | | | | | | | | | | New feature Description: Add in a combination of Peter's & my code to support copying variable-length data from one file to another, although currently only supported with contiguous data storage. Platforms tested: FreeBSD 4.11 (sleipnir) h5committest
* [svn-r11712] Purpose:Quincey Koziol2005-11-151-3/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | New feature Description: Check in baseline for compact group revisions, which radically revises the source code for managing groups and object headers. WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! This initiates the "unstable" phase of the 1.7.x branch, leading up to the 1.8.0 release. Please test this code, but do _NOT_ keep files created with it - the format will change again before the release and you will not be able to read your old files!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! Solution: There's too many changes to really describe them all, but some of them include: - Stop abusing the H5G_entry_t structure and split it into two separate structures for non-symbol table node use within the library: H5O_loc_t for object locations in a file and H5G_name_t to store the path to an opened object. H5G_entry_t is now only used for storing symbol table entries on disk. - Retire H5G_namei() in favor of a more general mechanism for traversing group paths and issuing callbacks on objects located. This gets us out of the business of hacking H5G_namei() for new features, generally. - Revised H5O* routines to take a H5O_loc_t instead of H5G_entry_t - Lots more... Platforms tested: h5committested and maybe another dozen configurations.... :-)
* [svn-r11593] Purpose:Quincey Koziol2005-10-211-2/+13
| | | | | | | | | | | | Code cleanup Description: Clean up & standardize a bit in preparation for coding standards discussion. Platforms tested: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r11245] Purpose:Quincey Koziol2005-08-131-6/+6
| | | | | | | | | | | | | | | | | | | | Code cleanup Description: Trim trailing whitespace, which is making 'diff'ing the two branches difficult. Solution: Ran this script in each directory: foreach f (*.[ch] *.cpp) sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f end Platforms tested: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r10736] Purpose:Quincey Koziol2005-05-071-1/+1
| | | | | | | | | | | Code cleanup Description: Clean up some compiler warnings Platforms tested: FreeBSD 4.11 (sleipnir) h5committest
* [svn-r9727] Purpose:Quincey Koziol2004-12-291-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bug Fix/Code Cleanup/Doc Cleanup/Optimization/Branch Sync :-) Description: Generally speaking, this is the "signed->unsigned" change to selections. However, in the process of merging code back, things got stickier and stickier until I ended up doing a big "sync the two branches up" operation. So... I brought back all the "infrastructure" fixes from the development branch to the release branch (which I think were actually making some improvement in performance) as well as fixed several bugs which had been fixed in one branch, but not the other. I've also tagged the repository before making this checkin with the label "before_signed_unsigned_changes". Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel & fphdf5 FreeBSD 4.10 (sleipnir) w/threadsafe FreeBSD 4.10 (sleipnir) w/backward compatibility Solaris 2.7 (arabica) w/"purify options" Solaris 2.8 (sol) w/FORTRAN & C++ AIX 5.x (copper) w/parallel & FORTRAN IRIX64 6.5 (modi4) w/FORTRAN Linux 2.4 (heping) w/FORTRAN & C++ Misc. update:
* [svn-r9563] Purpose:Quincey Koziol2004-11-231-9/+11
| | | | | | | | | | | | | Code optimization Description: Change how default allocation time is handled internally to the library, to avoid some performance issues with property lists. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Solaris 2.7 (arabica) Too minor to require h5committest
* [svn-r9354] Purpose:Quincey Koziol2004-10-011-59/+0
| | | | | | | | | | | | | | | | Bug fix & code cleanup Description: More dataset cleanups to get to a point where we can fix the chunked I/O bug. Also fix a couple of errors in the recent file object resurrection changes which should hopefully address the recent daily test failres (H5T.c) Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Solaris 2.7 (arabica) h5committest
* [svn-r9342] Purpose:Quincey Koziol2004-09-301-32/+31
| | | | | | | | | | | | | | | | Bug fix/code cleanup Description: Clean up raw data I/O code to bundle the I/O parameters (dataset, DXPL ID, etc) into a single struct to pass around through the dataset I/O routines, since they are always passed together, until very near the bottom of the I/O stack. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Solaris 2.7 (arabica) IRIX64 6.5 (modi4) h5committest
* [svn-r9329] James Laird2004-09-281-1/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Purpose: Feature Description: Datatypes and groups now use H5FO "file object" code that was previously only used by datasets. These objects will hold a file open if the file is closed but they have not yet been closed. If these objects are unlinked then relinked, they will not be destroyed. If they are opened twice (even by two different names), both IDs will "see" changes made to the object using the other ID. When an object is opened using two different names (e.g., if a dataset was opened under one name, then mounted and opened under its new name), calling H5Iget_name() on a given hid_t will return the name used to open that hid_t, not the current name of the object (this is a feature, and a change from the previous behavior of datasets). Solution: Used H5FO code that was already in place for datasets. Broke H5D_t's, H5T_t's, and H5G_t's into a "shared" struct and a private struct. The shared structs (H5D_shared_t, etc.) hold the object's information and are used by all IDs that point to a given object in the file. The private structs are pointed to by the hid_t and contain the object's group entry information (including its name) and a pointer to the shared struct for that object. This changed the naming of structs throughout the library (e.g., datatype->size is now datatype->shared->size). I added an updated H5Tinit.c to windows.zip. Platforms tested: Visual Studio 7, sleipnir, arabica, verbena Misc. update:
* [svn-r8932] Purpose:Quincey Koziol2004-07-221-6/+30
| | | | | | | | | | | | | | | | | | | | Code cleanup Description: Clean up collective chunking code a bit. Also, add '--enable-instrument' configure flag to have a mechanism for determining that optimized operations happened correctly in the library (instead of just the "normal" way) by allowing 'flag' properties to be set outside the library and set when the "right" thing happens. This is mainly for debugging and regression checks, so we make certain we don't break optimized I/O by accident. It's enabled by default when --enable-debug is on (which is on by default in the development branch and off by default in the release branch), but can also be independently controlled with its own configure flag. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel IBM p690 (copper) w/parallel
* [svn-r8906] Purpose:MuQun Yang2004-07-201-0/+7
| | | | | | | | | | | | | | | | | | | | | | | Adding the first round of patches about supporting collective chunk IO in HDF5 Description: The current HDF5 library doesn't support collective MPIO with chunk storage. When users set collective option in their data transfer with chunk storage, the library silently converted the option to INDEPENDENT and that caused trememdous performance penalty. Some application like WRF-parallel HDF5 IO module has to use contiguous storage for this reason. However, chunking storage has its own advantage(supporting compression filters and extensible dataset), so to make collective MPIO possible inside HDF5 with chunking storage is a very important task. This check-in make collective chunk IO possible for some special cases. The condition is as follows(either case is fine with using collective chunk IO) 1. for each process, the hyperslab selection of the file data space of each dataset is regular and it is fit in one chunk. 2. for each process, the hyperslab selection of the file data space of each dataset is single and the number of chunks for the hyperslab selection should be equal. Solution: Lift up the contiguous storage requirement for collective IO. Use H5D_isstore_get_addr to get the corresponding chunk address. Then the original library routines will take care of getting the correct address to make sure that MPI FILE TYPE is built correctly for collective IO> Platforms tested: arabica(sol), copper(AIX), eirene(Linux) parallel test is checked at copper. Misc. update:
* [svn-r8686] Purpose:Quincey Koziol2004-06-141-1/+1
| | | | | | | | | | | | | Code optimization Description: Eliminate memcpy() when using default DXPL by pointing at existing default object, instead of copying it. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8613] Purpose:Quincey Koziol2004-06-051-1/+54
| | | | | | | | | | | | | | | | | Refactor code Description: Move chunk and contiguous cached raw data from file information to dataset information. This simplifies a number of internal interfaces, aligns the code with it's purpose better and should allow more optimizations to the chunked data I/O performance. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) h5committest Misc. update:
* [svn-r8600] Purpose:Quincey Koziol2004-05-311-1/+4
| | | | | | | | | | | | | Code optimization Description: Don't recompute the internal index value for looking up the chunk in the hash table, just use the value already computed from iterating through the chunks. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.9 (sleipnir) w/parallel
* [svn-r8590] Purpose:Quincey Koziol2004-05-271-1/+1
| | | | | | | | | | | | | | | | | | | | | Code optimization & bug fix Description: When dimension information is being stored in the storage layout message on disk, it is stored as 32-bit quantities, possibly truncating the dimension information, if a dimension is greater than 32-bits in size. Solution: Fix the storage layout message problem by revising file format to not store dimension information, since it is already available in the dataspace. Also revise the storage layout data structures to be more compartmentalized for the information for contiguous, chunked and compact storage. Platforms tested: FreeBSD 4.9 (sleipnir) w/parallel Solaris 2.7 (arabica) h5committest
* [svn-r8482] Purpose:Quincey Koziol2004-05-051-6/+6
| | | | | | | | | | | | Code cleanup Description: Refactored data transform code to reduce amount of symbols in the global scope and also cleaned up & simplified the code a bit. Platforms tested: h5committest (minus copper, plus serial modi4) FreeBSD 4.9 (sleipnir) w & w/o parallel
generated by cgit v0.12 at 2024-11-20 10:41:26 (GMT)