summaryrefslogtreecommitdiffstats
path: root/src/H5Smpio.c
Commit message (Collapse)AuthorAgeFilesLines
* Add configure check for MPI_Mprobe and MPI_Imrecv functionsJordan Henderson2018-07-261-3/+32
| | | | | | | | | | | | | | | Add line to libhdf5settings file for status of Parallel writes to filtered datasets status Surround Parallel Compression code in MPI_VERSION >= 3 checks Add disabled message for Parallel Compression built w/ MPI-2 Modify Parallel Compression tests to only run the parallel filtered read tests when parallel filtered writes are disabled Update big I/O code to handle being built with MPI-2 Add checks to CMakeLists.txt for MPI_Mprobe and MPI_Imrecv
* Fixed HDFFV-10404Binh-Minh Ribler2018-07-131-4/+3
| | | | | | | | | Description: Applied the typo fixes from user's report. The previous pull request couldn't be merged because it was too old, and it was too complicated for me to resolve conflicts. Platform tested: Linux/64 (jelly) - very minor
* Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/hdffv/hdf5 into ↵Quincey Koziol2018-04-291-52/+462
|\ | | | | | | merge_hyperslab_updates
| * Include code fixes and additional modifications pointed out by code reviewersRichard Warren2017-07-101-6/+6
| |
| * Commited changes to the development branch here to allow a pull request to ↵Richard Warren2017-07-051-46/+458
| | | | | | | | be published
| * Merge pull request #426 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:hdf5_1_10 to ↵Larry Knox2017-04-251-6/+4
| | | | | | | | | | | | | | | | | | | | hdf5_1_10 * commit '54957d37f5aa73912763dbb6e308555e863c43f4': Commit copyright header change for src/H5PLpkg.c which was added after running script to make changes. Add new files in release_docs to MANIFEST. Cimmit changes to Makefile.in(s) and H5PL.c that resulted from running autogen.sh. Merge pull request #407 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:hdf5_1_10_1 to hdf5_1_10_1 Change copyright headers to replace url referring to file to be removed and replace it with new url for COPYING file.
* | Checkpoint normalization against incoming hyperslab / selection / dataspaceQuincey Koziol2017-03-281-1/+1
|/ | | | improvements.
* [svn-r27768] Description:Quincey Koziol2015-09-141-1/+1
| | | | | | | | | | | | | | | | | | | | | | Complete revamp of package initialization/shutdown mechanism in the library. Each package now has a single init/term routine. This new way should avoid packages being re-initialized during library shutdown and is also be _much_ more proactive about giving feedback for resource leaks internal to the library. Introduces a new "module" header file for packages in the library (e.g src/H5Fmodule.h) which sets up some necessary package configuration macros for the FUNC_ENTER/LEAVE macros. (The VFL drivers have their own slightly modified version of this header, src/H5FDdrvr_module.h) Also cleaned up a bunch of resources leaks all across the library and tests, along with addressing many warnings, as I encountered them. Tested on: MacOSX/64 10.10.5 (amazon) w/serial & parallel Linux/64 3.10.x (kituo) w/serial & parallel Linux/64 2.6.x (ostrich) w/serial
* [svn-r27045] Renamed H5_ASSIGN_OVERFLOW() to H5_CHECKED_ASSIGN() and re-orderedDana Robinson2015-05-101-3/+3
| | | | | | the arguments to be in a more logical order. Tested on: h5committest
* [svn-r25922] mpi version check for new mpi 3 features should be >= 3 rather ↵Mohamad Chaarawi2014-12-231-1/+1
| | | | than == 3.
* [svn-r25921] - wrap MPI 3 usage with MPI_VERSION compile time check.Mohamad Chaarawi2014-12-231-2/+7
| | | | - build out unbuffered stdout/err for windows
* [svn-r25766] Description:Quincey Koziol2014-11-031-1/+1
| | | | | | | | | | Revert r25273, 25283 & 25439 (the hyperslab improvement changes). They are buggy and it's taking me a long time to correct the problem. I'll check in a revised form of the changes when I've got them straightened out. Tested on: Mac OSX 10.10.0 (amazon) w/gcc 4.9.x, C++, FORTRAN Linux 2.6.x (jam) w/parallel
* [svn-r25516] remove use of MPI_LB and MPI_UB deprecated markers and use ↵Mohamad Chaarawi2014-08-061-29/+16
| | | | resize instead.
* [svn-r25514] Replace MPI deprecated routines with undeprecated ones.Mohamad Chaarawi2014-08-051-13/+14
|
* [svn-r25386] remove double declaration of off and len.Mohamad Chaarawi2014-07-011-2/+0
|
* [svn-r25273] Description:Quincey Koziol2014-06-131-1/+1
| | | | | | | | | | | Bring in Chao/Neil/my changes to optimize hyperslab selection operations further, along with 3 new public API routines: H5Scombine_hyperslab(), H5Sselect_select() and H5Scombine_select(), along with many minor cleanups to the code and fixing a few compiler warnings. Tested on: Mac OSX/64 10.9.3 w/gcc 4.9.x and parallel w/OpenMPI (h5commttest forthcoming)
* [svn-r24769] Description:Quincey Koziol2014-03-071-37/+406
| | | | | | | | | Check in Mohamad's changes to support collective I/O on point selections, along with some other minor cleanups. Tested on: Mac OSX/64 10.9.2 (amazon) w/parallel & serial (h5committest forthcoming)
* [svn-r24709] rename H5V to H5VM since H5V is needed in the fastforward ↵Mohamad Chaarawi2014-02-131-2/+2
| | | | | | | | | | | | project for view objects. The addition of view objects in the fastforward project is expected to be brough into the trunk sometimes in the future, which is why we need to make this change. Tested Manually on Jam and Ostrich. Tested with h5commitest - Koala with intel compilers failed, but nothing had to do with those changes. error on Koala: error while loading shared libraries: libirng.so
* [svn-r22312] Description:Quincey Koziol2012-04-231-0/+1
| | | | | | | | Correct error reported by user (Martin Otte) where we weren't using realloc'ed buffer in MPI datatype code. Tested on: Mac OSX/64 10.7.3 (amazon) w/parallel
* [svn-r21919] Description:Quincey Koziol2012-02-091-7/+7
| | | | | | | | Refactor function name macros and simplify the FUNC_ENTER macros, to clear away the cruft and prepare for further cleanups. Tested on: Mac OSX/64 10.7.3 (amazon) w/debug, production & parallel
* [svn-r19744] Description:Quincey Koziol2010-11-081-38/+46
| | | | | | | Clean up compiler warnings and neaten up code a bit. Tested on: Linxu/64 2.6.32 (ember) w/parallel
* [svn-r19092] Description:Quincey Koziol2010-07-191-362/+265
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bring "shape same" changes from LBL branch to trunk. These changes allow shapes that are the same, but projected into dataspaces with different ranks to be detected correctly, and also contains code to project a dataspace into greater/lesser number of dimensions, so the I/O can proceed in a faster way. These changes also contain several bug fixes and _lots_ of code cleanups to the MPI datatype creation code. Many other misc. code cleanup are included as well... Tested on: FreeBSD/32 6.3 (duty) in debug mode FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (amani) w/Intel compilers, w/default API=1.6.x, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in debug mode Mac OS X/32 10.6.3 (amazon) in debug mode Mac OS X/32 10.6.3 (amazon) w/C++ & FORTRAN, w/threadsafe, in production mode
* [svn-r18565] Description:Quincey Koziol2010-04-151-32/+28
| | | | | | | | Minor whitespace and compiler warning cleanups Tested on: Mac OS X/32 10.6.3 (amazon) w/debug (too minor to require h5committest)
* [svn-r16560] Description:Quincey Koziol2009-03-101-177/+159
| | | | | | | | | | Remove another call to H5E_clear_stack() from within the library. Clean up lots of compiler warnings. Tested on: Mac OS X/32 10.5.6 (amazon) (followup on other platforms forthcoming)
* [svn-r15628] Description:Quincey Koziol2008-09-161-1/+1
| | | | | | | | | | | | | Remove trailing whitespace from C/C++ source files, with the following script: foreach f (*.[ch] *.cpp) sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f end Tested on: Mac OS X/32 10.5.5 (amazon) No need for h5committest, just whitespace changes...
* [svn-r14903] Undoing change committed in r14902.Scot Breitenfeld2008-04-301-7/+7
|
* [svn-r14902] Merged fortran_1_8 branch changes r14505:14901 into the trunk. ↵Scot Breitenfeld2008-04-301-7/+7
| | | | New fortran wrappers added.
* [svn-r14860] Description:Quincey Koziol2008-04-241-7/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Omnibus raw data I/O revisions, with wide-ranging changes and refactoring, in order to prepare for implementing "fast append" feature. These changes remove the majority of the code duplication for raw data I/O which has crept in over the last ten years and introduces a more object- oriented design for operating on different types of dataset storage. Chunked storage no longer has it's own I/O routines, it is now handled as either contiguous (if chunk is not pulled into the cache) or compact (if the chunk is cached in memory). No bug or feature changes, at least intentionally... :-) Tested on: FreeBSD/32 6.2 (duty) in debug mode FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe, in debug mode Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN, in production mode Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN, in production mode Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN, w/szip filter, in production mode Mac OS X/32 10.5.2 (amazon) in debug mode Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
* [svn-r13733] Changed more parallel debugging info. to a more standard way by ↵MuQun Yang2007-05-041-33/+52
| | | | | | using H5DEBUG. Have tested at linux to make sure the change won't cause any compiling errors or testing errors.
* [svn-r13253] Updated all C and C++ style source code files with the THG ↵Albert Cheng2007-02-071-2/+3
| | | | | | | | | copyright notice. Tested platform: Kagiso only since it is only a comment block change. If it works in one machine, it should work in all, I hope. Still need to check the parallel build on copper.
* [svn-r12218] Purpose:Quincey Koziol2006-04-101-4/+0
| | | | | | | | | | | | | Code cleanup Description: Remove remnents of references to unimplemented H5S_COMPLEX dataspace. Platforms tested: FreeBSD 4.11 (sleipnir) Linux 2.4 32-bit (heping) Linux 2.4 64-bit (mir) Solaris 2.9 (shanti)
* [svn-r11321] MuQun Yang2005-08-311-8/+18
| | | | | | | | | | | | | Purpose: Add some comments and error checking codes for collective chunk IO code. Description: Solution: Platforms tested: AIX 5.1(copper), too minor to test at other platforms. Misc. update:
* [svn-r11316] MuQun Yang2005-08-301-2/+2
| | | | | | | | | | | | | | Purpose: bug fix for collective chunk IO Description: When using calloc to allocate an MPI datatype, one should use sizeof(MPI datatype) instead of sizeof(equalivent non-MPI datatype) to assign the value, this causes the problem at 64-bit platforms such IRIX65 and AIX 5.1. Solution: Correct it. Platforms tested: AIX 5.1 and IRIX6.5(at NCAR). Misc. update:
* [svn-r11245] Purpose:Quincey Koziol2005-08-131-1/+1
| | | | | | | | | | | | | | | | | | | | Code cleanup Description: Trim trailing whitespace, which is making 'diff'ing the two branches difficult. Solution: Ran this script in each directory: foreach f (*.[ch] *.cpp) sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f end Platforms tested: FreeBSD 4.11 (sleipnir) Too minor to require h5committest
* [svn-r11235] Purpose:Quincey Koziol2005-08-121-40/+38
| | | | | | | | | | | | | Code cleanup Description: Refactor, simplify and cleanup Kent's recent checking for collective chunk I/O. There's a bug that I need to talk to Kent about and some more cleanups still, but this is reasonable for an interim point. Platforms tested: FreeBSD 4.11 (sleipnir) w/parallel Too minor for h5committest
* [svn-r11231] Purpose:MuQun Yang2005-08-111-185/+101
| | | | | | | | | | | | | | | | | | | | | | | bug fix for collective chunk IO, phase 1 Optimization hasn't been done yet, the collective chunk IO bug should be fixed. Description: In chunking storage, memory space and file space will be remapped, So to check whether file space and memory space are regular in order to use optimized MPI derived datatype for collective call one has to check per-chunk wise instead of per hyperslab wise. Even a regular memory space will be stored in span-tree and will be irregular before chunk IO. Solution: 1. Check file space and memory space per chunk wise instead of per hyperslab wise. 2. For collective IO mode, number of chunks covered by hyperslab may be different. Since we are handing per chunk per IO, for the extra chunk IO for some(not all) processors, collective mode will cause program hanged. So for the extra chunk Io mode independent IO has to be used. 3. On some platforms, Complex MPI derived datatype is not working, so we have to use independent IO for collective IO mode if the selection is irregular. However, when the selection is regular, we do want to use collective IO since that will improve performance. Special cares have to be added for this case. Platforms tested: copper(AIX 5.1) Linux(heping mpich 1.2.6), Teragrid machine, Cobalt(altix), modi4 Misc. update:
* [svn-r10545] Purpose:MuQun Yang2005-04-051-125/+136
| | | | | | | | | | | | | | | | | | | | | | | | Activating collective IO supports for irregular selction inside HDF5 dataset. This support doesn't include to build the final complicated MPI derived datatype support for chunking storage. Description: Support collective chunk IO for both contiguous and chunking storage for irregular selection( using H5S_SELECT_OR for multiple hyperslab selection) Solution: Using MPI derived datatype to realize this feature. Problems still need to be investigated: Big size irregular hyperslab selection test may cause MPI hang or abnormalexit with MPICH family on various platforms. This is really hard to debug since sometimes it can work and sometimes it cannot work. We will continue investigating those cases. This may not be parallel HDF5 bugs since with the recent version of poe at IBM platforms, all tests passed. Platforms tested: 1. Linux heping 2.4.21-27.0.1 with mpich 2. AIX 5.1 copper with mpcc_r 3. Altix cobalt SGI linux 2.4.21-sgi304rp05031014_10149 with icc -lmpi 4. Linux Cluster SDSC TG, intel 8-r2 with mpich 1.2.5 5. NCSA Linux Cluster Tungsten, MPICH-TCP-1.2.5.2, Intel 8.0 under lustre 6. NCSA Linux Cluster Tungsten, MPICH-LAM-INTEL711, sometimes not working 7. NCSA Linux CLuster Tungsten, champion-pro-1.2.0-3, not working for other collective IO tests, but work for irregular selection collective IO test. Misc. update:
* [svn-r9857] Purpose: MaintenanceElena Pourmal2005-01-221-3/+0
| | | | | | | | | | | | | | Description: Removed PABLO from the source Solution: Platforms tested: arabica with 64-bit, copper with parallel, heping with GNU C and C++ and PGI fortran (but I disabled hl, there is some weird problem only on heping: F9XMODFLAG is not propagated to the Makefile files Misc. update:
* [svn-r9727] Purpose:Quincey Koziol2004-12-291-3/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Bug Fix/Code Cleanup/Doc Cleanup/Optimization/Branch Sync :-) Description: Generally speaking, this is the "signed->unsigned" change to selections. However, in the process of merging code back, things got stickier and stickier until I ended up doing a big "sync the two branches up" operation. So... I brought back all the "infrastructure" fixes from the development branch to the release branch (which I think were actually making some improvement in performance) as well as fixed several bugs which had been fixed in one branch, but not the other. I've also tagged the repository before making this checkin with the label "before_signed_unsigned_changes". Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel & fphdf5 FreeBSD 4.10 (sleipnir) w/threadsafe FreeBSD 4.10 (sleipnir) w/backward compatibility Solaris 2.7 (arabica) w/"purify options" Solaris 2.8 (sol) w/FORTRAN & C++ AIX 5.x (copper) w/parallel & FORTRAN IRIX64 6.5 (modi4) w/FORTRAN Linux 2.4 (heping) w/FORTRAN & C++ Misc. update:
* [svn-r9529] Purpose:MuQun Yang2004-11-151-26/+84
| | | | | | | | | | | | | | | | | | | Check in some new fixes for MPI derived datatype routines Description: MPI derived datatype algorithm seems working for a simple case; however, there are still other problems need to be solved. So the code cannot be used for the time being. Check-in only for debugging. It won't affect other part of the library. Solution: Platforms tested: Linux 2.4 (heping, serial and parallel) (Since no new tests were added and changes are mostly restricted to one fuction, no need to test three platforms). Misc. update:
* [svn-r9519] Purpose:MuQun Yang2004-11-111-0/+299
| | | | | | | | | | | | | | | | | | | | Adding codes for the general MPI derived datatype in order to better incorporate new fixes of HDF5 library. Description: Note: These codes have not been tested for general use. Don't call these functions in your developments of the HDF5 library. Also these codes are stand-alone codes, they should not affect other library codes. Solution: Platforms tested: Heping(C and Parallel linux 2.4, mpich 1.2.6) Arabica(C,C++,Fortran, Solaris 2.7) Copper(C,c++,Fortran, AIX 5.1, NOTE: c++ FAILED, seems not due to the recent check-in) Misc. update:
* [svn-r9354] Purpose:Quincey Koziol2004-10-011-173/+0
| | | | | | | | | | | | | | | | Bug fix & code cleanup Description: More dataset cleanups to get to a point where we can fix the chunked I/O bug. Also fix a couple of errors in the recent file object resurrection changes which should hopefully address the recent daily test failres (H5T.c) Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Solaris 2.7 (arabica) h5committest
* [svn-r9342] Purpose:Quincey Koziol2004-09-301-303/+0
| | | | | | | | | | | | | | | | Bug fix/code cleanup Description: Clean up raw data I/O code to bundle the I/O parameters (dataset, DXPL ID, etc) into a single struct to pass around through the dataset I/O routines, since they are always passed together, until very near the bottom of the I/O stack. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Solaris 2.7 (arabica) IRIX64 6.5 (modi4) h5committest
* [svn-r9018] Purpose:Albert Cheng2004-08-051-4/+276
| | | | | | | | | | | | | | | | | | | | | | | | | | Bug fix (Failures when dataset size >= 1 GB, reported by Bill Loewe.) Description: In the IBM AIX system using 32bit mode, if a dataset size was 1GB or larger, when the "end" of the dataset was selected, MPI would complain it could not keep the Upper bound of a datatype within the range of MPI_Aint. This was because the old algorithm would derive the selection with extent of each row first. After all dimensions were processed, it then calculate the start position and just displace the whole MPI derived type. So, the final MPI type was actually the start position plus the whole dataset. Since the start can be as big as the whole dataset, this made the final derived twice as big as 1GB. That would hit the 2GB MPI_Aint range limit in the 32 bit mode. Solution: Use a different algorithm to include the start position in the defining of MPI type for each dimension. When all dimensions are processed, the MPI type represents the selection exactly. Platforms tested: h5committested Misc. update:
* [svn-r8987] Purpose:Quincey Koziol2004-08-021-1/+0
| | | | | | | | | | | | Code cleanup Description: Fix another batch of minor differences between the development and release branches. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8932] Purpose:Quincey Koziol2004-07-221-355/+94
| | | | | | | | | | | | | | | | | | | | Code cleanup Description: Clean up collective chunking code a bit. Also, add '--enable-instrument' configure flag to have a mechanism for determining that optimized operations happened correctly in the library (instead of just the "normal" way) by allowing 'flag' properties to be set outside the library and set when the "right" thing happens. This is mainly for debugging and regression checks, so we make certain we don't break optimized I/O by accident. It's enabled by default when --enable-debug is on (which is on by default in the development branch and off by default in the release branch), but can also be independently controlled with its own configure flag. Platforms tested: FreeBSD 4.10 (sleipnir) w/parallel IBM p690 (copper) w/parallel
* [svn-r8906] Purpose:MuQun Yang2004-07-201-12/+165
| | | | | | | | | | | | | | | | | | | | | | | Adding the first round of patches about supporting collective chunk IO in HDF5 Description: The current HDF5 library doesn't support collective MPIO with chunk storage. When users set collective option in their data transfer with chunk storage, the library silently converted the option to INDEPENDENT and that caused trememdous performance penalty. Some application like WRF-parallel HDF5 IO module has to use contiguous storage for this reason. However, chunking storage has its own advantage(supporting compression filters and extensible dataset), so to make collective MPIO possible inside HDF5 with chunking storage is a very important task. This check-in make collective chunk IO possible for some special cases. The condition is as follows(either case is fine with using collective chunk IO) 1. for each process, the hyperslab selection of the file data space of each dataset is regular and it is fit in one chunk. 2. for each process, the hyperslab selection of the file data space of each dataset is single and the number of chunks for the hyperslab selection should be equal. Solution: Lift up the contiguous storage requirement for collective IO. Use H5D_isstore_get_addr to get the corresponding chunk address. Then the original library routines will take care of getting the correct address to make sure that MPI FILE TYPE is built correctly for collective IO> Platforms tested: arabica(sol), copper(AIX), eirene(Linux) parallel test is checked at copper. Misc. update:
* [svn-r8731] Purpose:Quincey Koziol2004-06-231-9/+6
| | | | | | | | | | | | | | | Code cleanup & minor optimization Description: Re-work the way interface initialization routines are specified in the library to avoid the overhead of checking for them in routines where there is no interface initialization routine. This cleans up warnings with gcc 3.4, reduces the library binary size a bit (about 2-3%) and should speedup the library's execution slightly. Platforms tested: FreeBSD 4.10 (sleipnir) w/gcc34 h5committest
* [svn-r8677] Purpose:Quincey Koziol2004-06-141-3/+3
| | | | | | | | | | | | | | | | | | | Code cleanup & optimization Description: Remove old structures that used a union to store information about the dataspace extent and just store the information directly in the dataspace extent itself. Remove ifdef'd references to permutation ordering in dataspaces. We'll definitely need more than this code if/when we implement this feature. Change allocation of dataspace information from calloc() to malloc(). Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
* [svn-r8673] Purpose:Quincey Koziol2004-06-131-6/+6
| | | | | | | | | | | | | | | | | | Code optimization Description: Revised dataspace selections to use a more "object oriented" mechanism to set the function pointers for each selection and selection iterator. This reduces the amount and number of times that dataspace selection info has to be copied. Additionally, change hyperslab selection information to be dynamically allocated instead of an inline struct. Platforms tested: Solaris 2.7 (arabica) FreeBSD 4.10 (sleipnir) w/parallel Too minor to require h5committest
generated by cgit v0.12 at 2024-10-31 19:21:08 (GMT)