summaryrefslogtreecommitdiffstats
path: root/src/H5Dmpio.c
Commit message (Collapse)AuthorAgeFilesLines
* [svn-r25273] Description:Quincey Koziol2014-06-131-1/+1
| | | | | | | | | | | Bring in Chao/Neil/my changes to optimize hyperslab selection operations further, along with 3 new public API routines: H5Scombine_hyperslab(), H5Sselect_select() and H5Scombine_select(), along with many minor cleanups to the code and fixing a few compiler warnings. Tested on: Mac OSX/64 10.9.3 w/gcc 4.9.x and parallel w/OpenMPI (h5commttest forthcoming)
* [svn-r25197] Description:Quincey Koziol2014-05-171-2/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | Bring changes from hdf5_1_8_coverity branch back to trunk: r20878: Issue 76: Check if H5Tget_nmembers(type) fails and simply return(FALSE). Also move printf to after check. r20880: Issue 192: Create ret_val var set to -1. Add out label for failures to jump to, return ret_val at bottom. r20882: Fixes for coverity: 1) bug #1679: remove dead code in test/mf.c 2) bug #1680: remove dead code in tools/lib/h5diff_dset.c r20883: Fix coverity issue 585 Description: Changed variable "c" in processStrData in h5import.c to an int, to match fgetc return value, and removed call to feof, instead checking if c == EOF. Tested on: MacOSX/64 10.9.3 (amazon) w/C++, FORTRAN & parallel (too minor to require h5committest)
* [svn-r24864] Description:Quincey Koziol2014-03-211-24/+25
| | | | | | | | | | | | | | | | | | Remove all traces of MPI-POSIX VFD and GPFS detection/code. Remove remaining traces of stream VFD. Remove testpar/t_posix_compliant test (it's not actually verifying anything). Clean up H5D__mpio_opt_possible() further. Moved environment variable that disables MPI collective operations into MPI-IO VFD (instead of it being in src/H5S.c). A few other small code cleanups. Tested on: Mac OSX/64 10.9.2 (amazon) w/parallel & serial
* [svn-r24769] Description:Quincey Koziol2014-03-071-31/+98
| | | | | | | | | Check in Mohamad's changes to support collective I/O on point selections, along with some other minor cleanups. Tested on: Mac OSX/64 10.9.2 (amazon) w/parallel & serial (h5committest forthcoming)
* [svn-r24709] rename H5V to H5VM since H5V is needed in the fastforward ↵Mohamad Chaarawi2014-02-131-1/+1
| | | | | | | | | | | | project for view objects. The addition of view objects in the fastforward project is expected to be brough into the trunk sometimes in the future, which is why we need to make this change. Tested Manually on Jam and Ostrich. Tested with h5commitest - Koala with intel compilers failed, but nothing had to do with those changes. error on Koala: error while loading shared libraries: libirng.so
* [svn-r23713] Description:Quincey Koziol2013-05-211-12/+27
| | | | | | | | | | Clean up warnings, switch library code to use Standard C/POSIX wrapper macros, remove internal calls to API routines, update checkapi and checkposix scripts. Tested on: Mac OSX/64 10.8.3 (amazon) w/C++ & FORTRAN Big-Endian Linux/64 (ostrich)
* [svn-r22992] Purpose:Jonathan Kim2012-10-301-249/+53
| | | | | | | | | | | HDFFV-8146 - Remove "multi-chunk IO without optimization" sub-feature from MPI I/O optimization for chunked dataset feature Description: The “multi-chunk IO without optimization” feature is removed and made the related xfer property (H5FD_MPIO_CHUNK_MULTI_IO) go directly to “multi-chunk-io” feature. Also update/fix/cleanup testings (chunk collective IO and actual chunk opt mode) accordingly. Tested: jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), fred (mac64-LE), Windows (32-LE cmake), cmake (jam)
* [svn-r22735] Purpose:Jonathan Kim2012-09-041-27/+39
| | | | | | | | | | | | | | | | | | | HDFFV-8143 Provide a routine(s) for telling the user why the library broke collective data access Description: Added H5Pget_mpio_no_collective_cause() function that retrive reasons why the collective I/O was broken during Read/Write IO access. Reasons to break collective I/O: - SET_INDEPENDENT - DATATYPE_CONVERSION - DATA_TRANSFORMS - MPIPOSIX - NOT_SIMPLE_OR_SCALAR_DATASPACES (NULL Space) - POINT_SELECTIONS - NOT_CONTIGUOUS_OR_CHUNKED_DATASET (Compact or External-Storage) - FILTERS Tested: jam (linux32-LE), koala (linux64-LE), ostrich (linuxppc64-BE), tejeda (mac32-LE), linew (solaris-BE)
* [svn-r22378] removed the parallel #endif by mistake.. oopsMohamad Chaarawi2012-05-181-2/+0
| | | | | remove an un-needed bool
* [svn-r22377] make the file and memory MPI datatypes as default properties in theMohamad Chaarawi2012-05-181-6/+1
| | | | | | | DXPL so that we don't have to insert and remove them everytime we perform collective I/O. Test with h5committest
* [svn-r22287] Description:Quincey Koziol2012-04-161-138/+152
| | | | | | | | Clean up more FUNC_ENTER/FUNC_LEAVE macros and move H5D & H5T code toward the final design (as exemplified by the H5EA & H5FA code). Tested on: Mac OSX/64 10.7.3 (amazon) w/debug & parallel
* [svn-r21919] Description:Quincey Koziol2012-02-091-20/+20
| | | | | | | | Refactor function name macros and simplify the FUNC_ENTER macros, to clear away the cruft and prepare for further cleanups. Tested on: Mac OSX/64 10.7.3 (amazon) w/debug, production & parallel
* [svn-r21796] Description:Quincey Koziol2011-12-021-33/+30
| | | | | | | Back out r21782 while I figure out what the problem is with the change. Tested on: Daily tests... :-/
* [svn-r21782] Description:Quincey Koziol2011-11-301-30/+33
| | | | | | | | | | Rearrange checks for reasons why we break collective I/O back to independent I/O into "global" and "local" sections. We should try to minimize the checks in the "local" section... Tested on: Mac OS X/32 10.7.2 (amazon) w/parallel (too minor to require h5committest)
* [svn-r21442] Description:Quincey Koziol2011-10-011-0/+77
| | | | | | | | | | | | | | | | | | | | | | Check in "actual I/O mode" feature to trunk. Will merge back to 1.8 branch after it bakes over the weekend. Tested on: FreeBSD/32 8.2 (loyalty) w/gcc4.6, w/C++ & FORTRAN, in debug mode FreeBSD/64 8.2 (freedom) w/gcc4.6, w/C++ & FORTRAN, in debug mode Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (koala) w/Intel compilers, w/default API=1.6.x, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, w/threadsafe, in production mode Linux/PPC 2.6 (heiwa) w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-ia64 2.6 (ember) w/Intel compilers, w/C++ & FORTRAN, in production mode Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode Mac OS X/32 10.6.8 (amazon) in debug mode Mac OS X/32 10.6.8 (amazon) w/C++ & FORTRAN, w/threadsafe, in production mode
* [svn-r21101] HDFFV-7639Mike McGreevy2011-07-141-342/+135
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Purpose: Remove H5_MPI_SPECIAL_COLLECTIVE_IO_WORKS and H5_MPI_COMPLEX_DERIVED_DATATYPE_WORKS #defines from source. Description: Two advanced parallel functionalities, special collective IO and complex derived datatypes, are not supported by older implementations of mpi, and thus our code limits the use of these features with #ifdefs and has checks in configure to set them (or not). Unfortunately, configure can't actually run a parallel check to see if these features are working (nor not) so it resorts to looking in the config files where they are explicity enabled or disabled based on versions of mpi, sytems being built on, or for no documented reason at all (i.e. just set to on or off as some 'default'). Overriding these settings is easy if need be, provided it is known that it needs to be done to get improved performance, and oftentimes it is not. Most new MPI implementations successfully handle the functionality requested when these #defines are set, and many of the "turn these features off" cases in the config files are for old (> 5 years) versions of MPI and retired systems (such as NCSA's tungsten). Therefore, the decision has been made to remove the support for these old versions of MPI and systems that cannot handle these behaviors. The #ifdefs and supporting setup in the config/ files and configure script has been removed, and the code executed when these options were not set removed from the source. In passing, this commit also cleans up some whitespace issues in both t_mpi.c and H5Dmpio.c. Furthermore, in t_mpi.c, the special collective IO test was not getting regularly run due to it being written to work only with four processes (we regularly test with six, previously with three), and thus it failed when actually run due to an out of bounds data buffer assignment. It has been modified to run at any number of processes greater than four, and the memory problem has been fixed so the test passes. Tested: jam, h5committest, ember
* [svn-r19092] Description:Quincey Koziol2010-07-191-58/+66
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bring "shape same" changes from LBL branch to trunk. These changes allow shapes that are the same, but projected into dataspaces with different ranks to be detected correctly, and also contains code to project a dataspace into greater/lesser number of dimensions, so the I/O can proceed in a faster way. These changes also contain several bug fixes and _lots_ of code cleanups to the MPI datatype creation code. Many other misc. code cleanup are included as well... Tested on: FreeBSD/32 6.3 (duty) in debug mode FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode Mac OS X/32 10.6.3 (amazon) in debug mode Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe, in production mode
* [svn-r19006] Purpose: Improve performance of the the chunk cacheNeil Fortner2010-06-151-10/+12
| | | | | | | | | | Description: Previously, the chunk cache would, when looking for a chunk, alwaylook for the chunk in the b-tree before checking the cache. Reworked the functions that look up chunks to always check the cache first and avoid the b-tree lookup if the chunk is in cache. Tested: jam, linew, amani (h5committest)
* [svn-r18565] Description:Quincey Koziol2010-04-151-6/+5
| | | | | | | | Minor whitespace and compiler warning cleanups Tested on: Mac OS X/32 10.6.3 (amazon) w/debug (too minor to require h5committest)
* [svn-r17148] Description:Quincey Koziol2009-07-021-10/+10
| | | | | | | | | | | | | | | | | | | | | | Refactor how chunked dataset information is computed, moving it earlier and avoiding more recomputation. Tested on: FreeBSD/32 6.3 (duty) in debug mode FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode Mac OS X/32 10.5.7 (amazon) in debug mode Mac OS X/32 10.5.7 (amazon) w/C++ & FORTRAN, w/threadsafe, in production mode
* [svn-r16847] Description:Quincey Koziol2009-04-231-1/+4
| | | | | | | | | | | | | Bring r16846 from revise_chunks branch back to trunk: Fix broken (for how long?) H5_ASSIGN_OVERFLOW macro to actually detect overflows during assignments, along with several errors it [now] detected. Cleaned up a fix minor warnings and/or pieces of code also. Tested on: FreeBSD/32 6.3 (duty) in debug mode (h5committest not needed - multi-platform test performed on branch)
* [svn-r16560] Description:Quincey Koziol2009-03-101-14/+15
| | | | | | | | | | Remove another call to H5E_clear_stack() from within the library. Clean up lots of compiler warnings. Tested on: Mac OS X/32 10.5.6 (amazon) (followup on other platforms forthcoming)
* [svn-r16532] Description:Quincey Koziol2009-03-021-2/+2
| | | | | | | | | | Pass the chunk "user data" to H5D_chunk_unlock(), so that chunks with an address already aren't reallocated. Tested on: FreeBSD/32 6.3 (duty) in debug mode FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
* [svn-r16523] Purpose: Fix bugs related to H5Dset_extent and fill valuesNeil Fortner2009-02-271-5/+13
| | | | | | | | | Description: In some situations it was possible for the fill value to not be written to parts of a chunked dataset, particularly when extending and/or shrinking. Prior to the fix for the chunk cache (1015) these bugs would have been exceedingly rare. Tested: jam, smirom, linew (h5committest)
* [svn-r16465] Performance Improvement(bug #1450). When a chunk is bigger ↵Raymond Lu2009-02-111-2/+2
| | | | | | | | | than the cache size and isn't allocated on disk, the library still loaded it in the cache, which is redundant. I changed it to bypass the cache and added a test in dsets.c. Tested on jam and smirom.
* [svn-r15800] Description:Quincey Koziol2008-10-071-14/+18
| | | | | | | | | | | | | | | | | | | | | | | Bring file free space branch changes through r15795 into trunk, which includes a fair bit of code cleanup & rearrangement along with a couple of bug fixes also. Tested on: Mac OS X/32 10.5.5 (amazon) in debug mode Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe, in production mode FreeBSD/32 6.3 (duty) in debug mode FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
* [svn-r15628] Description:Quincey Koziol2008-09-161-70/+70
| | | | | | | | | | | | | Remove trailing whitespace from C/C++ source files, with the following script: foreach f (*.[ch] *.cpp) sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f end Tested on: Mac OS X/32 10.5.5 (amazon) No need for h5committest, just whitespace changes...
* [svn-r15626] This check-in fixes a bug inside parallel HDF5 testsuite.MuQun Yang2008-09-161-2/+28
| | | | | | | | | When enable-debug is turned on, a special macro block H5_HAVE_INSTRUMENTED_LIBRARY inside HDF5 will be executed to check if some collective chunk IO test cases are being run with the correct settings(one link, multiple chunk etc.). However,when complicated derived datatype in some mpi-io packages are not supported, the library has to switch one link IO with/without the optimization to multiple chunk IO with/without the optimization. The current testsuite doesn't know this and generates a false assertion failure message. This check-in fix this problem by providing a second property to avoid the false faiure message. Tested at abe(NCSA linux cluster) and kagiso.
* [svn-r15131] Description:Quincey Koziol2008-06-031-30/+16
| | | | | | | | | | | | | | | | | | | | | | Finish omnibus chunked dataset I/O refactoring, to separate general actions on chunked datasets from actions that are specific to using the v1 B-tree index. Cleaned up a few bugs and added some additional tests also. Tested on: FreeBSD/32 6.2 (duty) in debug mode FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Mac OS X/32 10.5.2 (amazon) in debug mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
* [svn-r15015] Description:Quincey Koziol2008-05-161-4/+8
| | | | | | | | | | | | | | | | | Detect chunks that are >4GB before dataset gets created and return error to application. Tweak lots of internal variables that hold the chunk size/dimensions to use an 'uint32_t', instead of a 'size_t', so that the integer size is constant. Correct a number of our tests which were creating datasets with chunks that were >4GB and add some specific tests for >4GB chunk size detection. Minor whitespace & other code cleanups. Tested on: Mac OS X/32 10.5.2 (amazon) Forthcoming testing on other platforms...
* [svn-r14903] Undoing change committed in r14902.Scot Breitenfeld2008-04-301-1341/+1317
|
* [svn-r14902] Merged fortran_1_8 branch changes r14505:14901 into the trunk. ↵Scot Breitenfeld2008-04-301-1317/+1341
| | | | New fortran wrappers added.
* [svn-r14860] Description:Quincey Koziol2008-04-241-1341/+1317
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Omnibus raw data I/O revisions, with wide-ranging changes and refactoring, in order to prepare for implementing "fast append" feature. These changes remove the majority of the code duplication for raw data I/O which has crept in over the last ten years and introduces a more object- oriented design for operating on different types of dataset storage. Chunked storage no longer has it's own I/O routines, it is now handled as either contiguous (if chunk is not pulled into the cache) or compact (if the chunk is cached in memory). No bug or feature changes, at least intentionally... :-) Tested on: FreeBSD/32 6.2 (duty) in debug mode FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Mac OS X/32 10.5.2 (amazon) in debug mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
* [svn-r14326] Description:Quincey Koziol2007-12-061-9/+9
| | | | | | | | | | | | | | | | | | | | | | | | - Keep skip list for tracking chunks with dataset (instead of creating/ destroying it for each I/O operation) and just delete the skip list nodes. - Avoid computations for normalizing selection offset when offset not set. - Avoid updating object modification time twice during dataset creation. - Avoid updating dataset layout message (and object modification time) until dataset is closed. Tested on: FreeBSD/32 6.2 (duty) in debug mode FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Mac OS X/32 10.4.10 (amazon) in debug mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
* [svn-r14250] Description:Quincey Koziol2007-11-101-27/+29
| | | | | | | | | | | | | | | | | | | | | - Changed name of chunk mapping structure from "fm_map" to "H5D_chunk_map_t" - Pushed parallel I/O specific fields into ifdef'ed sections, when building chunk map - Took out most calls to IS_H5FD_MPI macro Tested on: FreeBSD/32 6.2 (duty) in debug mode FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Mac OS X/32 10.4.10 (amazon) in debug mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
* [svn-r14096] There're 3 changes in this checkin as below:Raymond Lu2007-08-201-18/+80
| | | | | | | | | | | | | | | | | | 1. In H5Dwrite and H5Dread, let the data buffer point to a fake address if the application passes in an empty buffer. This is mainly for MPIO programs that some processes may not have any data to write or read but still participate the I/O. This solution solves some MPI's problem like the ChaMPIon on tungsten which doesn't support empty buffer. 2. The ChaMPIon on tungsten doesn't support complex derived MPI data type correctly and collective I/O when some processes don't have any data to write or read correctly. Detect the compiler "cmpicc" in the system-specific config file and set the variables for these two cases to false. The PHDF5 library already has set up a way to switch collective chunked I/O to independent under these two cases. 3. A bug fix - During the work of the optimization for compound data I/O, the case for switching collective chunked I/O to independent I/O was leftout. Fixed it by adding I/O caching to it in H5D_multi_chunk_collective_io in H5Dmpio.c. Tested on tungsten, cobalt, and kagiso for parallel; on linew and smirom for serial.
* [svn-r13997] Moved H5D_istore_lock and H5D_istore_unlock from ↵Raymond Lu2007-07-211-10/+8
| | | | | | | | | H5D_istore_writevv and H5D_istore_readvv to H5D_chunk_write and H5D_chunk_read to avoid frequent lock and unlock and to improve some performance. Tested with h5committest on THG machines.
* [svn-r13730] Remove debug information inside H5Dmpio.c when enable-debug is ↵MuQun Yang2007-05-031-48/+71
| | | | | | | | turned on. More code will be cleaned up tomorrow. Since only debugging information is removed, I tested this only at kagiso(linux mpich). Kent
* [svn-r13253] Updated all C and C++ style source code files with the THG ↵Albert Cheng2007-02-071-2/+3
| | | | | | | | | copyright notice. Tested platform: Kagiso only since it is only a comment block change. If it works in one machine, it should work in all, I hope. Still need to check the parallel build on copper.
* [svn-r12727] 1. Cleaning up the comments of the source code.MuQun Yang2006-10-061-92/+85
| | | | | | | 2. Using H5Dmpio_Debug macro to reflect the debugging information of new parallel IO supports. They are very minor changes. Tested at heping. t_pflush2 failed. Others passed.
* [svn-r12593] Description:Quincey Koziol2006-08-171-339/+161
| | | | | | | | | Clean up some of the parallel chunk I/O compiler warnings and other minor code rearrangement. Tested on: FreeBSD 4.11 (sleipnir) w/paralllel h5committest
* [svn-r12555] 1. Erased a strange character added to this codeMuQun Yang2006-08-091-8/+35
| | | | | | | 2. Used to workaround solution to fix the cchunk10 failure caused by the MPICH bug Machine test: heping, mir
* [svn-r12553] This check-in includes the following part of parallel ↵MuQun Yang2006-08-091-200/+385
| | | | | | | | | | | | | | | optimization codes: 1. Provide another option for users to do independent IO with MPI file setview(collectively) 2. With the request of collective IO from users, using Independent IO with MPI file setview if we find collective IO is not good for the applications for IO per chunk(multi-chunk IO) case. Previously we used pure independent IO and that actually performed small IO(IO each row) for this case. The recent performance study suggested the independent IO with file setview can acheieve significantly better performance than collective IO when not many processes participate in the IO. 3. For applications that explicitly choose to do collective IO per chunk case, the library won't do any optimization(gather/broadcast) operations. The library simply passes the collective IO request to MPI-IO. Tested at copper, kagiso, heping, mir and tungsten(cmpi and mpich) Kagiso is using LAM, t_mpi test was broken even. The cchunk10 test failed at heping and mir. I suspected it was an MPICH problem. Will investigate later. Everything passed at copper. at tungsten: the old cmpi bug(failed at esetw) is still there. Other tests passed. Some sequential fheap tests failed at kagiso.
* [svn-r12440] Purpose:Quincey Koziol2006-06-271-138/+138
| | | | | | | | | | | Code cleanup Description: Trim trailing whitespace in Makefile.am and C/C++ source files to make diffing changes easier. Platforms tested: None necessary, whitespace only change
* [svn-r12400] Purpose:MuQun Yang2006-06-031-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Some collective chunk IO macro names are confusing, change them to more meaningful names. Description: H5Pset_dxpl_mpio_chunk_opt will set a flag so that the library can do one linked IO or multi-chunk IO with collective in chunking storage directly. That is, the library won't do analyses to determine this. The flags for the enum type we used before are: H5FD_MPIO_OPT_ONE_IO H5FD_MPIO_OPT_MULTI_IO They are not good names because of the following two reasons: 1. It doesn't reflect chunking storage 2. OPT is kind of redundant and misleading, Solution: We change the names to H5FD_MPIO_CHUNK_ONE_IO H5FD_MPIO_CHUNK_MULTI_IO Platforms tested: Since only macro names are changed, no need to test with h5committest. heping(mpich 1.2.6) Misc. update:
* [svn-r12343] Purpose:MuQun Yang2006-05-111-16/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bug fix for collective chunk IO Description: Several options have been provided inside HDF5 library for obtaining chunk addresses when doing collective chunk IO. One option is to obtain chunk addresses for one process and broadcast to other processes. This option needs all processes to participate. If using link-chunked IO without any optimizations, sometimes this is not true due to the random initialization for one variable at mpich 1.2.7. This is a bug inside the collective chunk IO code. Solution: 1.Initalize all the variables to some safe numbers, 2. Avoid using MPI broadcast to obtain the chunk address if possible until more performance studies have been done. 3. Seems okay to obtain chunk addresses individually for each processor. This option may cover most cases. Platforms tested: h5committest(copper is not usable) NCSA teragrid (mpich 1.2.5) mir 64-bit linux (mpich 1.2.6) Misc. update:
* [svn-r12173] Purpose:MuQun Yang2006-03-291-1/+95
| | | | | | | | | | | | | | | | | | | | | | | | | | | Adding parallel tests for optional collective chunk APIs Description: Three new APIs "H5Pset_dxpl_mpio_chunk_opt_ratio H5Pset_dxpl_mpio_chunk_opt_num H5Pset_dxpl_mpio_chunk_opt" for optional optimization choices from users have been added to the libraries. This check-in adds six tests to verify the funcationality and correctedness of these APIs. These tests need to be verified with 3 or more processors and with MPI-IO driver only. Solution: Using H5Pinsert, H5Pget, H5Pset to verify that the library indeed goes into the branch we hope for. Using H5_HAVE_INSTRUMENT macro to isolate these changes so that it won't affect or be misused by the application. Platforms tested: h5committest(shanti still refused to be connected) Parallel tests on heping somehow are skipped. Manually testing at heping. Have checked 1,2,3,4,5 processes. Misc. update:
* [svn-r12123] Purpose:MuQun Yang2006-03-201-12/+14
| | | | | | | | | | | | | | Add more comments Description: Add more comments to H5Dmpio.c, which describes a little bit more about collective IO management. Solution: Platforms tested: Only test at heping since only comments were added. Misc. update:
* [svn-r12117] Purpose:MuQun Yang2006-03-181-38/+25
| | | | | | | | | | | | | | | | | | | | | Enhancing the optimiziation of collective IO per chunk Description: When the user does one of the following two things: 1. to do collective IO per chunk without using our optimization code 2. or the user passes the percent of number of process per chunk to be 0 when choosing to do collective IO per chunk It is not necessary that the library uses MPI-IO collective calls to do any optimization. Solution: Modify the code so that no MPI communication-involved analyses will be done for the above cases. Chunk addresses are obtained globally and IO modes are assigned to collective always. Platforms tested: h5committest Misc. update:
* [svn-r12111] Purpose:MuQun Yang2006-03-171-11/+44
| | | | | | | | | | | | | minor change for collective code Description: Solution: Platforms tested: mir Misc. update:
generated by cgit v0.12 at 2024-11-18 15:37:06 (GMT)