summaryrefslogtreecommitdiffstats
path: root/src/H5Dmpio.c
Commit message (Collapse)AuthorAgeFilesLines
* Change copyright headers to replace url referring to file to belrknox2017-05-161-6/+4
| | | | removed and replace it with new url for COPYING file.
* [svn-r27165] Merge 27133 & 27158 from trunk:Mohamad Chaarawi2015-06-081-10/+10
| | | | | | | | | - Add a new attribute function characterstic for format: * H5_ATTR_FORMAT(X,Y,Z) __attribute__((format(X, Y, Z))) - Rename UNUSED attribute characterstic to H5_ATTR_UNUSED. - Rename NORETURN attribute characterstic to H5_ATTR_NORETURN tested with h5committest.
* [svn-r27099] Merge of r27045 from trunkDana Robinson2015-05-181-4/+4
| | | | | | | | | Renamed H5_ASSIGN_OVERFLOW() to H5_CHECKED_ASSIGN() and re-ordered the arguments to be in a more logical order. Fixes HDFFV-9322 Tested on: h5committest
* [svn-r26189] Merge revision 26149 to 1.8 branchNeil Fortner2015-02-161-23/+21
| | | | | | | | | Fix potential memory error reported by valgrind in H5D__link_chunk_collective_io. This would occur when performing collective I/O on a dataset with only 1 chunk, and a participating process had nothing to read or write. Tested: jam, koala, ostrich (h5committest); in trunk and 1.8
* [svn-r25996] - merge changes from trunk to eliminate usage of IS_H5FD_MPI Macro.Mohamad Chaarawi2015-01-211-2/+2
| | | | | | - add 2 missing feature flags for VFDs: H5FD_FEAT_HAS_MPI & H5FD_FEAT_ALLOCATE_EARLY and use them as appropriate. tested with h5commitest
* [svn-r25528] bring 25514 and 25516 from trunk:Mohamad Chaarawi2014-08-111-4/+4
| | | | Replace MPI deprecated routines with undeprecated ones.
* [svn-r25198] Description:Quincey Koziol2014-05-181-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Bring r25197 from trunk to 1.8 branch: Bring changes from hdf5_1_8_coverity branch back to trunk: r20878: Issue 76: Check if H5Tget_nmembers(type) fails and simply return(FALSE). Also move printf to after check. r20880: Issue 192: Create ret_val var set to -1. Add out label for failures to jump to, return ret_val at bottom. r20882: Fixes for coverity: 1) bug #1679: remove dead code in test/mf.c 2) bug #1680: remove dead code in tools/lib/h5diff_dset.c r20883: Fix coverity issue 585 Description: Changed variable "c" in processStrData in h5import.c to an int, to match fgetc return value, and removed call to feof, instead checking if c == EOF. Tested on: MacOSX/64 10.9.3 (amazon) w/C++, FORTRAN & parallel (too minor to require h5committest)
* [svn-r24878] Description:Quincey Koziol2014-03-241-24/+25
| | | | | | | | | | | | | | | | | | | | | Bring r24864 from trunk to 1.8 branch: Remove all traces of MPI-POSIX VFD and GPFS detection/code. Remove remaining traces of stream VFD. Remove testpar/t_posix_compliant test (it's not actually verifying anything). Clean up H5D__mpio_opt_possible() further. Moved environment variable that disables MPI collective operations into MPI-IO VFD (instead of it being in src/H5S.c). A few other small code cleanups. Tested on: Mac OSX/64 10.9.2 (amazon) w/parallel & serial (daily tested on trunk)
* [svn-r24785] Description:Quincey Koziol2014-03-121-31/+98
| | | | | | | | | | | | | Bring r24769 & r24772 from trunk to 1.8 branch: Check in Mohamad's changes to support collective I/O on point selections, along with some other minor cleanups. Correct some dynamically sized arrays that fail on Windows. Tested on: Mac OSX/64 10.9.2 (amazon) w/parallel (And has been through the daily test regime on the trunk)
* [svn-r24720] bring r24709 from trunk:Mohamad Chaarawi2014-02-181-1/+1
| | | | | | | rename H5V to H5VM since H5V is needed in the fastforward project for view objects. The addition of view objects in the fastforward project is expected to be brough into the trunk sometimes in the future, which is why we need to make this change.
* [svn-r23719] Description:Quincey Koziol2013-05-291-12/+27
| | | | | | | | | | | | Bring r23713 and r23717 from trunk to 1.8 branch: Clean up warnings, switch library code to use Standard C/POSIX wrapper macros, remove internal calls to API routines, update checkapi and checkposix scripts. Tested on: Mac OSX/64 10.8.3 (amazon) w/C++ & FORTRAN Big-Endian Linux/64 (ostrich)
* [svn-r23011] Purpose:Jonathan Kim2012-11-051-249/+53
| | | | | | | | | | | | HDFFV-8146 - Remove "multi-chunk IO without optimization" sub-feature from MPI I/O optimization for chunked dataset feature Description: The “multi-chunk IO without optimization” feature is removed and made the related xfer property (H5FD_MPIO_CHUNK_MULTI_IO) go directly to “multi-chunk-io” feature. Also update/fix/cleanup testings (chunk collective IO and actual chunk opt mode) accordingly. Merged from HDF5 trunk r22992. Tested: jam-pp (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), fred (mac64-LE), linew (solaris-BE), koala-pp, wallby-pp, cmake (jam-pp), Windows (32-LE cmake)
* [svn-r22780] Purpose:Jonathan Kim2012-09-171-27/+39
| | | | | | | | | | | | | | | | | | | | | HDFFV-8143 Provide a routine(s) for telling the user why the library broke collective data access Description: Added H5Pget_mpio_no_collective_cause() function that retrive reasons why the collective I/O was broken during Read/Write IO access. Reasons to break collective I/O: - SET_INDEPENDENT - DATATYPE_CONVERSION - DATA_TRANSFORMS - MPIPOSIX - NOT_SIMPLE_OR_SCALAR_DATASPACES (NULL Space) - POINT_SELECTIONS - NOT_CONTIGUOUS_OR_CHUNKED_DATASET (Compact or External-Storage) - FILTERS Merged from HDF5 trunk r22735, r22741 , 22743, 22744, 22763. Tested: jam-pp (linux32-LE), koala-pp (linux64-LE), wallaby-pp
* [svn-r22379] make the file and memory MPI datatypes as default properties in theMohamad Chaarawi2012-05-181-8/+1
| | | | | DXPL so that we don't have to insert and remove them everytime we perform collective I/O.
* [svn-r22291] Description:Quincey Koziol2012-04-171-138/+152
| | | | | | | | | | Bring r22287 from trunk to 1.8 branch: Clean up more FUNC_ENTER/FUNC_LEAVE macros and move H5D & H5T code toward the final design (as exemplified by the H5EA & H5FA code). Tested on: Mac OSX/64 10.7.3 (amazon) w/debug & parallel
* [svn-r21925] Description:Quincey Koziol2012-02-101-20/+20
| | | | | | | | | | | | | | Bring r21919-21924 from trunk to 1.8 branch: cleanup FUNC_ENTER macros, etc. Also removed Subversion mergeinfo tags from a few directories and files. These are artifacts from non-root merges using pre-1.6 Subversion clients. (This should fix the problem of unchanged directories looking "changed" during checkins.) Tested on: Mac OSX/64 10.7.3 (amazon) w/debug & production (daily tested on trunk)
* [svn-r21797] Description:Quincey Koziol2011-12-021-33/+30
| | | | | | | Back out r21782 while I figure out what the problem is with the change. Tested on: Daily tests... :-/
* [svn-r21784] Description:Quincey Koziol2011-11-301-30/+33
| | | | | | | | | | | | Bring r21782 from trunk to 1.8 branch: Rearrange checks for reasons why we break collective I/O back to independent I/O into "global" and "local" sections. We should try to minimize the checks in the "local" section... Tested on: Mac OS X/32 10.7.2 (amazon) w/parallel (too minor to require h5committest)
* [svn-r21533] Description:Quincey Koziol2011-10-121-0/+77
| | | | | | | | | Bring r21442 and r21529 from trunk to 1.8 branch: Add new API routines for querying if collective raw data I/O was actually performed collectively. Tested: Linux/32 2.6 (jam) (h5committested on trunk)
* [svn-r21479] Merge revision 21101 from trunk to 1.8 branch:Mike McGreevy2011-10-051-342/+135
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | HDFFV-7639 Purpose: Remove H5_MPI_SPECIAL_COLLECTIVE_IO_WORKS and H5_MPI_COMPLEX_DERIVED_DATATYPE_WORKS #defines from source. Description: Two advanced parallel functionalities, special collective IO and complex derived datatypes, are not supported by older implementations of mpi, and thus our code limits the use of these features with #ifdefs and has checks in configure to set them (or not). Unfortunately, configure can't actually run a parallel check to see if these features are working (nor not) so it resorts to looking in the config files where they are explicity enabled or disabled based on versions of mpi, sytems being built on, or for no documented reason at all (i.e. just set to on or off as some 'default'). Overriding these settings is easy if need be, provided it is known that it needs to be done to get improved performance, and oftentimes it is not. Most new MPI implementations successfully handle the functionality requested when these #defines are set, and many of the "turn these features off" cases in the config files are for old (> 5 years) versions of MPI and retired systems (such as NCSA's tungsten). Therefore, the decision has been made to remove the support for these old versions of MPI and systems that cannot handle these behaviors. The #ifdefs and supporting setup in the config/ files and configure script has been removed, and the code executed when these options were not set removed from the source. In passing, this commit also cleans up some whitespace issues in both t_mpi.c and H5Dmpio.c. Furthermore, in t_mpi.c, the special collective IO test was not getting regularly run due to it being written to work only with four processes (we regularly test with six, previously with three), and thus it failed when actually run due to an out of bounds data buffer assignment. It has been modified to run at any number of processes greater than four, and the memory problem has been fixed so the test passes. Tested: Jam, h5committest, Ember
* [svn-r19093] Description:Quincey Koziol2010-07-191-58/+66
| | | | | | | | | | | | | | | | | | | | Bring r19092 from trunk to 1.8 branch: Bring "shape same" changes from LBL branch to trunk. These changes allow shapes that are the same, but projected into dataspaces with different ranks to be detected correctly, and also contains code to project a dataspace into greater/lesser number of dimensions, so the I/O can proceed in a faster way. These changes also contain several bug fixes and _lots_ of code cleanups to the MPI datatype creation code. Many other misc. code cleanup are included as well... Tested on: FreeBSD/32 6.3 (duty) in debug mode FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode (h5committested on trunk)
* [svn-r19007] Purpose: Improve performance of the the chunk cacheNeil Fortner2010-06-151-10/+12
| | | | | | | | | | Description: Previously, the chunk cache would, when looking for a chunk, alwaylook for the chunk in the b-tree before checking the cache. Reworked the functions that look up chunks to always check the cache first and avoid the b-tree lookup if the chunk is in cache. Tested: jam, linew, amani (h5committest)
* [svn-r18566] Description:Quincey Koziol2010-04-151-6/+5
| | | | | | | | | Bring r18564 & r18565 from trunk to 1.8 branch: clean up compiler warnings and whitespace issues. Tested on: Mac OS X/32 10.6.3 (amazon) (too minor to require h5committest)
* [svn-r17150] Description:Quincey Koziol2009-07-021-10/+10
| | | | | | | | | | | | | | | | | | | | | | | | Bring r17148:17149 from trunk into 1.8 branch: Refactor how chunked dataset information is computed, moving it earlier and avoiding more recomputation. Tested on: FreeBSD/32 6.3 (duty) in debug mode FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode Mac OS X/32 10.5.7 (amazon) in debug mode Mac OS X/32 10.5.7 (amazon) w/C++ & FORTRAN, w/threadsafe, in production mode
* [svn-r16920] Description:Quincey Koziol2009-05-051-1/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Merge these trunk revisions which occurred during the 1.8.3 release code freeze back to the 1.8 branch: From Quincey: 16845 16847 16849 16851 16858 16869 16897 From Ray: 16859 16880 From Allen: 16863 Tested on: FreeBSD/32 6.3 (duty) in debug mode FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode Mac OS X/32 10.5.6 (amazon) in debug mode Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe, in production mode
* [svn-r16566] Description:Quincey Koziol2009-03-111-14/+15
| | | | | | | | Bring r16560 back from trunk. Tested on: Mac OS X/32 10.5.6 (amazon) debug & production (Following up with tests on more platforms)
* [svn-r16536] Description:Quincey Koziol2009-03-021-2/+2
| | | | | | | | | | | Bring v16532 back from trunk to v1.8 branch: Pass the chunk "user data" to H5D_chunk_unlock(), so that chunks with an address already aren't reallocated. Tested on: FreeBSD/32 6.3 (duty) in debug mode (h5committest not needed here, performed on trunk)
* [svn-r16524] Purpose: Fix bugs related to H5Dset_extent and fill valuesNeil Fortner2009-02-271-5/+13
| | | | | | | | | Description: In some situations it was possible for the fill value to not be written to parts of a chunked dataset, particularly when extending and/or shrinking. Prior to the fix for the chunk cache (1015) these bugs would have been exceedingly rare. Tested: jam, smirom, linew (h5committest)
* [svn-r16467] Performance Improvement(bug #1450). When a chunk is bigger ↵Raymond Lu2009-02-111-2/+2
| | | | | | | | | | than the cache size and isn't allocated on disk, the library still loaded it in the cache, which is redundant. I changed it to bypass the cache and added a test in dsets.c. Tested on jam and smirom.
* [svn-r15801] Description:Quincey Koziol2008-10-071-12/+16
| | | | | | | | | | | | | | | | | | | | | | | Bring r15800 back from trunk: - File free space branch changes through r15794 Tested on: Mac OS X/32 10.5.5 (amazon) in debug mode Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe, in production mode FreeBSD/32 6.3 (duty) in debug mode FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
* [svn-r15629] Description:Quincey Koziol2008-09-161-72/+72
| | | | | | | | | | | | | Remove trailing whitespace from C/C++ source files, with the following script: foreach f (*.[ch] *.cpp) sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f end Tested on: Mac OS X/32 10.5.5 (amazon) No need for h5committest, just whitespace changes...
* [svn-r15624] This check-in fixes a bug inside parallel HDF5 testsuite.MuQun Yang2008-09-151-2/+28
| | | | | | When enable-debug is turned on, a special macro block H5_HAVE_INSTRUMENTED_LIBRARY inside HDF5 will be executed to check if some collective chunk IO test cases are being run with the correct settings(one link, multiple chunk etc.). However,when complicated derived datatype in some mpi-io packages are not supported, the library has to switch one link IO with/without the optimization to multiple chunk IO with/without the optimization. The current testsuite doesn't know this and generates a false assertion failure message. This check-in fix this problem by providing a second property to avoid the false faiure message.
* [svn-r15132] Description:Quincey Koziol2008-06-031-30/+16
| | | | | | | | | | | | | | Bring back revision 15131 from trunk: Finish omnibus chunked dataset I/O refactoring, to separate general actions on chunked datasets from actions that are specific to using the v1 B-tree index. Cleaned up a few bugs and added some additional tests also. Tested on: Mac OS X/32 10.5.3 (amazon) Linux/32 2.4 (chicago)
* [svn-r15016] Description:Quincey Koziol2008-05-161-4/+8
| | | | | | | | | | | | | | | | | | | Port revision 15015 back to 1.8 branch: > Detect chunks that are >4GB before dataset gets created and return error > to application. > > Tweak lots of internal variables that hold the chunk size/dimensions to > use an 'uint32_t', instead of a 'size_t', so that the integer size is constant. > > Correct a number of our tests which were creating datasets with chunks > that were >4GB and add some specific tests for >4GB chunk size detection. > > Minor whitespace & other code cleanups. Tested on: Mac OS X/32 10.5.2 (amazon) Forthcoming testing on other platforms...
* [svn-r14861] Description:Quincey Koziol2008-04-241-1341/+1317
| | | | | | | | | | | | | | | | | | | | | | | | | | | | Bring revision 14860 back to 1.8 branch, change log for rev 14860 is: Omnibus raw data I/O revisions, with wide-ranging changes and refactoring, in order to prepare for implementing "fast append" feature. These changes remove the majority of the code duplication for raw data I/O which has crept in over the last ten years and introduces a more object- oriented design for operating on different types of dataset storage. Description: Omnibus raw data I/O revisions, with wide-ranging changes and refactoring, in order to prepare for implementing "fast append" feature. These changes remove the majority of the code duplication for raw data I/O which has crept in over the last ten years and introduces a more object- oriented design for operating on different types of dataset storage. Chunked storage no longer has it's own I/O routines, it is now handled as either contiguous (if chunk is not pulled into the cache) or compact (if the chunk is cached in memory). No bug or feature changes, at least intentionally... :-) Tested on: Mac OS X/32 10.5.2 (amazon) w/production
* [svn-r14326] Description:Quincey Koziol2007-12-061-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | - Keep skip list for tracking chunks with dataset (instead of creating/ destroying it for each I/O operation) and just delete the skip list nodes. - Avoid computations for normalizing selection offset when offset not set. - Avoid updating object modification time twice during dataset creation. - Avoid updating dataset layout message (and object modification time) until dataset is closed. Tested on: FreeBSD/32 6.2 (duty) in debug mode FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Mac OS X/32 10.4.10 (amazon) in debug mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
* [svn-r14250] Description:Quincey Koziol2007-11-101-27/+29
| | | | | | | | | | | | | | | | | | | | | - Changed name of chunk mapping structure from "fm_map" to "H5D_chunk_map_t" - Pushed parallel I/O specific fields into ifdef'ed sections, when building chunk map - Took out most calls to IS_H5FD_MPI macro Tested on: FreeBSD/32 6.2 (duty) in debug mode FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Mac OS X/32 10.4.10 (amazon) in debug mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
* [svn-r14096] There're 3 changes in this checkin as below:Raymond Lu2007-08-201-18/+80
| | | | | | | | | | | | | | | | | | 1. In H5Dwrite and H5Dread, let the data buffer point to a fake address if the application passes in an empty buffer. This is mainly for MPIO programs that some processes may not have any data to write or read but still participate the I/O. This solution solves some MPI's problem like the ChaMPIon on tungsten which doesn't support empty buffer. 2. The ChaMPIon on tungsten doesn't support complex derived MPI data type correctly and collective I/O when some processes don't have any data to write or read correctly. Detect the compiler "cmpicc" in the system-specific config file and set the variables for these two cases to false. The PHDF5 library already has set up a way to switch collective chunked I/O to independent under these two cases. 3. A bug fix - During the work of the optimization for compound data I/O, the case for switching collective chunked I/O to independent I/O was leftout. Fixed it by adding I/O caching to it in H5D_multi_chunk_collective_io in H5Dmpio.c. Tested on tungsten, cobalt, and kagiso for parallel; on linew and smirom for serial.
* [svn-r13997] Moved H5D_istore_lock and H5D_istore_unlock from ↵Raymond Lu2007-07-211-10/+8
| | | | | | | | | H5D_istore_writevv and H5D_istore_readvv to H5D_chunk_write and H5D_chunk_read to avoid frequent lock and unlock and to improve some performance. Tested with h5committest on THG machines.
* [svn-r13730] Remove debug information inside H5Dmpio.c when enable-debug is ↵MuQun Yang2007-05-031-48/+71
| | | | | | | | turned on. More code will be cleaned up tomorrow. Since only debugging information is removed, I tested this only at kagiso(linux mpich). Kent
* [svn-r13253] Updated all C and C++ style source code files with the THG ↵Albert Cheng2007-02-071-2/+3
| | | | | | | | | copyright notice. Tested platform: Kagiso only since it is only a comment block change. If it works in one machine, it should work in all, I hope. Still need to check the parallel build on copper.
* [svn-r12727] 1. Cleaning up the comments of the source code.MuQun Yang2006-10-061-92/+85
| | | | | | | 2. Using H5Dmpio_Debug macro to reflect the debugging information of new parallel IO supports. They are very minor changes. Tested at heping. t_pflush2 failed. Others passed.
* [svn-r12593] Description:Quincey Koziol2006-08-171-339/+161
| | | | | | | | | Clean up some of the parallel chunk I/O compiler warnings and other minor code rearrangement. Tested on: FreeBSD 4.11 (sleipnir) w/paralllel h5committest
* [svn-r12555] 1. Erased a strange character added to this codeMuQun Yang2006-08-091-8/+35
| | | | | | | 2. Used to workaround solution to fix the cchunk10 failure caused by the MPICH bug Machine test: heping, mir
* [svn-r12553] This check-in includes the following part of parallel ↵MuQun Yang2006-08-091-200/+385
| | | | | | | | | | | | | | | optimization codes: 1. Provide another option for users to do independent IO with MPI file setview(collectively) 2. With the request of collective IO from users, using Independent IO with MPI file setview if we find collective IO is not good for the applications for IO per chunk(multi-chunk IO) case. Previously we used pure independent IO and that actually performed small IO(IO each row) for this case. The recent performance study suggested the independent IO with file setview can acheieve significantly better performance than collective IO when not many processes participate in the IO. 3. For applications that explicitly choose to do collective IO per chunk case, the library won't do any optimization(gather/broadcast) operations. The library simply passes the collective IO request to MPI-IO. Tested at copper, kagiso, heping, mir and tungsten(cmpi and mpich) Kagiso is using LAM, t_mpi test was broken even. The cchunk10 test failed at heping and mir. I suspected it was an MPICH problem. Will investigate later. Everything passed at copper. at tungsten: the old cmpi bug(failed at esetw) is still there. Other tests passed. Some sequential fheap tests failed at kagiso.
* [svn-r12440] Purpose:Quincey Koziol2006-06-271-138/+138
| | | | | | | | | | | Code cleanup Description: Trim trailing whitespace in Makefile.am and C/C++ source files to make diffing changes easier. Platforms tested: None necessary, whitespace only change
* [svn-r12400] Purpose:MuQun Yang2006-06-031-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some collective chunk IO macro names are confusing, change them to more meaningful names. Description: H5Pset_dxpl_mpio_chunk_opt will set a flag so that the library can do one linked IO or multi-chunk IO with collective in chunking storage directly. That is, the library won't do analyses to determine this. The flags for the enum type we used before are: H5FD_MPIO_OPT_ONE_IO H5FD_MPIO_OPT_MULTI_IO They are not good names because of the following two reasons: 1. It doesn't reflect chunking storage 2. OPT is kind of redundant and misleading, Solution: We change the names to H5FD_MPIO_CHUNK_ONE_IO H5FD_MPIO_CHUNK_MULTI_IO Platforms tested: Since only macro names are changed, no need to test with h5committest. heping(mpich 1.2.6) Misc. update:
* [svn-r12343] Purpose:MuQun Yang2006-05-111-16/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bug fix for collective chunk IO Description: Several options have been provided inside HDF5 library for obtaining chunk addresses when doing collective chunk IO. One option is to obtain chunk addresses for one process and broadcast to other processes. This option needs all processes to participate. If using link-chunked IO without any optimizations, sometimes this is not true due to the random initialization for one variable at mpich 1.2.7. This is a bug inside the collective chunk IO code. Solution: 1.Initalize all the variables to some safe numbers, 2. Avoid using MPI broadcast to obtain the chunk address if possible until more performance studies have been done. 3. Seems okay to obtain chunk addresses individually for each processor. This option may cover most cases. Platforms tested: h5committest(copper is not usable) NCSA teragrid (mpich 1.2.5) mir 64-bit linux (mpich 1.2.6) Misc. update:
* [svn-r12173] Purpose:MuQun Yang2006-03-291-1/+95
| | | | | | | | | | | | | | | | | | | | | | | | | | | Adding parallel tests for optional collective chunk APIs Description: Three new APIs "H5Pset_dxpl_mpio_chunk_opt_ratio H5Pset_dxpl_mpio_chunk_opt_num H5Pset_dxpl_mpio_chunk_opt" for optional optimization choices from users have been added to the libraries. This check-in adds six tests to verify the funcationality and correctedness of these APIs. These tests need to be verified with 3 or more processors and with MPI-IO driver only. Solution: Using H5Pinsert, H5Pget, H5Pset to verify that the library indeed goes into the branch we hope for. Using H5_HAVE_INSTRUMENT macro to isolate these changes so that it won't affect or be misused by the application. Platforms tested: h5committest(shanti still refused to be connected) Parallel tests on heping somehow are skipped. Manually testing at heping. Have checked 1,2,3,4,5 processes. Misc. update:
* [svn-r12123] Purpose:MuQun Yang2006-03-201-12/+14
| | | | | | | | | | | | | | Add more comments Description: Add more comments to H5Dmpio.c, which describes a little bit more about collective IO management. Solution: Platforms tested: Only test at heping since only comments were added. Misc. update:
generated by cgit v0.12 at 2024-10-12 05:03:13 (GMT)