summaryrefslogtreecommitdiffstats
path: root/src/H5FDmpio.c
Commit message (Collapse)AuthorAgeFilesLines
* [svn-r5871] Purpose:Quincey Koziol2002-08-121-7/+7
| | | | | | | | | | | | | | | | Code cleanup Description: Combined H5P_isa_class and H5I_object functionality into a new internal H5P API function: H5P_object_verify, which checks that a property list is the appropriate class and then returns the property list object associated with the property list ID. This reduces the source code by about 200 LOC and trims the library binary some more. Platforms tested: FreeBSD 4.6 (sleipnir)
* [svn-r5867] Purpose:Quincey Koziol2002-08-091-8/+42
| | | | | | | | | | | | | Code cleanup Description: Changed the last HRETURN* statements in the FUNC_ENTER macros into HGOTO* macros, which reduces the size of the library binary in certain configurations by another 10% Platforms tested: FreeBSD 4.6 (sleipnir) serial & parallel, IRIX64 6.5 (modi4) serial & parallel
* [svn-r5842] Purpose:Quincey Koziol2002-08-081-95/+120
| | | | | | | | | | | | | | | | Code cleanup Description: Change most (all?) HRETURN_ERROR macros to HGOTO_ERROR macros, along with HRETURN macros to HGOTO_DONE macros. This unifies the error return path from functions and reduces the size of the library by up to 10% on some platforms. Additionally, I improved a lot of the error cleanup code in many routines. Platforms tested: FreeBSD 4.6 (sleipnir) serial & parallel and IRIX64 6.5 (modi4) serial & parallel.
* [svn-r5814] Purpose:Quincey Koziol2002-07-191-29/+40
| | | | | | | | | | | | | | | | | | Bug Fix Description: It was possible to create corrupted metadata information (either in memory or in the file or both) with a parallel I/O program because of the way metadata writes were being handled for writes out of the metadata cache. Solution: Added a dataset transfer property called "block before metadata write" which is used by the MPI-I/O and MPI-posix drivers to sync up all the processes before attempting a metadata write. This property is currently only for metadata writes from the metadata cache. Platforms tested: IRIX64 6.5 (modi4) w/parallel
* [svn-r5799] Purpose:Quincey Koziol2002-07-151-2/+2
| | | | | | | | | | | | New feature. Description: Added MPI-posix VFL driver. This driver uses MPI to coordinate actions, but performs I/O directly with posix sec(2) I/O functions. This driver should _NOT_ be used if the file accessed is not on a parallel filesystem. Platforms tested: FreeBSD 4.6 (sleipnir) w/parallel & IRIX64 6.5 (modi4) w/parallel
* [svn-r5690] Purpose:Quincey Koziol2002-06-241-29/+42
| | | | | | | | | | | | | | | | | | | | | | | | | | Bug Fix Description: When parallel I/O is used, the MPI-I/O VFL driver uses a "lazy" model to call MPI_File_set_view() in order to reduce the number of calls to this function. However, this is unsafe, because if a collective I/O which uses MPI derived types (and thus uses MPI_File_set_view()) is immediately followed by an independent I/O, the code will attempt to call MPI_File_set_view() in order to switch back to the default view of the file. MPI_File_set_view() is a collective call however, and this causes the application to hang. Solution: Removed "lazy" MPI_File_set_view() code, instead set the file view when it is needed (with MPI derived types) and immediately set the file view back to the default view before leaving the I/O routine. Platforms tested: IRIX64 6.5 (modi4) w/parallel. Also, tested with the latest development and release code for the SAF library, which now works correctly with this change. (Although the release branch of the SAF library seems to have a bug, this 1.4.4 release candidate code gets as far as the version the SAF library is released on top of (1.4.2-patch1, I believe)).
* [svn-r5677] Purpose:Quincey Koziol2002-06-191-1/+1
| | | | | | | | | | | Code improvement Description: Some small code cleanups and took out the code the was turning off the metadata cache for parallel I/O (!) Platforms tested: IRIX64 6.5 (modi4) w/parallel
* [svn-r5674] Purpose:Quincey Koziol2002-06-191-8/+5
| | | | | | | | | | Code cleanup Description: Removed more compiler warnings, etc. Platforms tested: Linux 2.2.x (eirene) w/parallel
* [svn-r5667] Purpose:Quincey Koziol2002-06-191-0/+2
| | | | | | | | | | Code cleanup Description: Turn on more warnings in the IRIX builds and clean them up. Platforms tested: IRIX64 6.5 (modi4) w/parallel
* [svn-r5660] Purpose:Quincey Koziol2002-06-181-39/+37
| | | | | | | | | | | | Code optimization Description: Avoid creating MPI types (and thus requiring a MPI_File_set_view() call) when contiguous selections are used for dataset I/O. This should be a performance improvement for those sorts of selections. Platforms tested: Linux 2.2.x (eirene) w/parallel && IRIX64 6.5 (modi4) w/parallel & FORTRAN
* [svn-r5652] Purpose:Quincey Koziol2002-06-171-28/+89
| | | | | | | | | | | | Code cleanup Description: Use dataset transfer property list to hold information about the MPI types for the current transfer, instead of setting pseudo-global variables in the file's struct. Platforms tested: Linux 2.2.x (eirene) w/parallel & IRIX64 6.5 (modi4) w/parallel & FORTRAN
* [svn-r5650] Purpose:Quincey Koziol2002-06-171-17/+53
| | | | | | | | | | | | Code cleanup Description: Change MPI-I/O code to use the address of the dataset for the displacement, instead of having a separate displacement value. Removed displacement parameter from H5FD_mpio_setup parameters. Platforms tested: Linux 2.2.x (eirene) w/parallel & IRIX64 6.5 (modi4) w/parallel.
* [svn-r5550] Purpose:Quincey Koziol2002-06-071-0/+9
| | | | | | | | | | | | | | | | | Code Improvement Description: Split the metadata accumulator code into two parts: one for allowing writes of the accumulator buffer during reads (when the buffer is dirty and needs to be flushed to disk in order to hold the new metadata being read in) and another for only allowing writes of the buffer during writes. This allows the MPI-I/O VFL driver to use the metadata accumulator (but only during writes) and benefit from the reduced number of metadata I/O operations that it brings. Platforms tested: IRIX64 6.5 (modi4) w/parallel
* [svn-r5536] Purpose:Quincey Koziol2002-06-051-0/+1
| | | | | | | | | | | | | | | | | New feature. Description: Added a "small data" block allocation mechanism to the library, similar to the mechanism used for allocating metadata currently. See the RFC for more details: http://hdf.ncsa.uiuc.edu/RFC/SmallData/SmallData.html This reduces the number of I/O operations which hit the disk for my test program from 19 to 15 (i.e. from 393 to 15, overall). Platforms tested: Solaris 2.7 (arabica) w/FORTRAN and FreeBSD 4.5 (sleipnir) w/C++
* [svn-r5530] Purpose:Bill Wendling2002-06-041-3/+5
| | | | | | | | | | | | | | | Code Cleanup Description: Removed some compiler warnings. Solution: In a few cases, NULL was being returned when a FAIL was supposed to be returned instead. There were some header files which needed to be included in a few of the sources. A couple of if-then statements had assignments in the conditional part. The compiler warned that they should have extra "()"s around them. Made the code check the values instead. Platforms tested: Linux (parallel) Modi4 (parallel)
* [svn-r5471] Purpose:Quincey Koziol2002-05-291-37/+48
| | | | | | | | | | | | | | | | Code cleanup Description: Broke the FUNC_ENTER macro into several macros, with more specialized uses (which followup mail will describe). This was designed to move most/all of the checks which could be done at compile time to that point, instead of needlessly performing them (over & over :-) at run-time. This reduces the library's size (and thus staticly linked binaries) and has a minor speedup effect also. Platforms tested: IRIX64 6.5 (modi4) with parallel & FORTRAN enabled, and additional testing on FreeBSD and Solaris immediately after the checkin.
* [svn-r5467] Purpose:Quincey Koziol2002-05-281-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | Code cleanup. Description: Took Robb's recent ideas for improving the FUNC_ENTER/FUNC_LEAVE macros equivalents in the SAF library and adapted them to our library. I added an additional macro which is equivalent to FUNC_ENTER: FUNC_ENTER_NOINIT - Has the API tracing code, etc. from FUNC_ENTER but none of the library or interface initialization code. This is to be used _only_ for static functions and those which explicitly cannot have the library or interface initialization code enabled (like the API termination routines, etc.). This allowed many more of the functions in the library [but not all yet :-(] to be wrapped with FUNC_ENTER[_NOINIT]/FUNC_LEAVE pairs. It also reduced the size of the library and executables (by cutting out a bunch of code which was never executed), I'll e-mail the exact results when I've finished editing it. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5447] Purpose:Quincey Koziol2002-05-211-34/+40
| | | | | | | | | | | Code cleanup Description: Guard against getting into metadata broadcast in write routine and clean up some error handling. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5440] Purpose:Quincey Koziol2002-05-201-38/+3
| | | | | | | | | | | New feature Description: Add 'closing' parameter to H5FDflush and VFL "flush" functions, per http://hdf.ncsa.uiuc.edu/RFC/VFLFlush/VFLFlush.html Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5429] Purpose:Quincey Koziol2002-05-171-0/+56
| | | | | | | | | | | | | | | | | | | | | Bug fix/Code improvement. Description: Currently, the chunk data allocation routine invoked to allocate space for the entire dataset is inefficient. It writes out each chunk in the dataset, whether it is already allocated or not. Additionally, this happens not only when it is created, but also anytime it is opened for writing, or the dataset is extended. Worse, there's too much parallel I/O syncronization, which slows things down even more. Solution: Only attempt to write out chunks that don't already exist. Additionally, share the I/O writing between all the nodes, instead of writing everything with process 0. Then, only block with MPI_Barrier if chunks were actually created. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5419] Purpose:Quincey Koziol2002-05-141-0/+11
| | | | | | | | | | | Bug fix. Description: Added barrier to flush routine to prevent race condition where file could be truncated. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5418] Purpose:Quincey Koziol2002-05-141-30/+26
| | | | | | | | | | | | | | | | Code cleanup/Performance enhancement Description: The code to extend the file size in H5FD_mpio_flush is getting run even when the file size doesn't change. Also, it's sort of sidestepping MPI-I/O when extending the file, instead of using MPI-I/O features to set the file's size. Solution: Only extend the file's size when the allocated size has increased. Also use MPI_File_set_size() to change the file's size. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5412] Purpose:Quincey Koziol2002-05-141-17/+50
| | | | | | | | | | | | | | | | Bug fix Description: Calling MPI_Get_count needs to be done with the same MPI type as was used for the transfer and we are always using MPI_BYTE, even when a different MPI type was used for the transfer. Solution: Only query MPI_Get_count with MPI_BYTE when we really used MPI_BYTE for the transfer. Wait for later to query MPI_Get_count with other MPI types. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5408] Purpose:Quincey Koziol2002-05-131-2/+41
| | | | | | | | | | | | | | | Performance enhancement Description: Doing an MPI_File_sync() just before a file is closed causing a large performance loss. Solution: Add flag to MPI file driver to avoid performance the MPI_File_sync() when the flag is set before a call to H5F_flush(). Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5403] Purpose:Quincey Koziol2002-05-131-9/+4
| | | | | | | | | | Back out changes. Description: Back out changes to VFL 'flush' API function, pending review. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5393] Purpose:Quincey Koziol2002-05-101-5/+9
| | | | | | | | | | | | | | | | | | | | | | New Feature Description: The VFL flush function is called immediately before a file is closed. This can cause duplicate syncronization actions to occur, if the VFL close function also performs them. Solution: Added 'closing' parameter to VFL 'flush' operation. This allows the VFL flush function to bypass operations that will be duplicated within the VFL close function. Additionally, use the 'closing' parameter to bypass calls to MPI_File_sync() when set. Since MPI_File_close() also syncronizes the file, this avoids the terrible performance hit taken when calling MPI_File_sync() as the file is closing. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5390] Purpose:Quincey Koziol2002-05-101-48/+11
| | | | | | | | | | | | | | | | | | | | | | Code cleanup Description: The parallel I/O file driver is optimized to only write metadata with one process (and broadcast the results to the other processes). This is currently enabled by a separate call to H5FD_mpio_tas_allsame() before each metadata write to the file. This can easily lead to problems where the prelude function call is omitted before the actual write code or, in a threaded environment, lead to race condititions where the value set is reset before being used. Solution: Since we only want to write metadata from one process, key off of the 'type' parameter (which has information about whether the data being written it metadata or raw data) to H5FD_mpio_write() as the method for determining whether to only write from one process or not. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5385] Purpose:Quincey Koziol2002-05-101-4/+21
| | | | | | | | | | | | | | | New Feature Description: Currently, only process 0 writes metadata to disk, leading to a potential performance bottleneck as the other processors wait for it to catch up. Solution: Rotate the metadata responsibilities among all processes, speading out the work. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5381] Purpose:Quincey Koziol2002-05-091-32/+19
| | | | | | | | | | | Small code cleanup/improvement Description: Query the MPI rank only once and store it in the file structure for each process. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5135] Purpose:Albert Cheng2002-04-031-6/+11
| | | | | | | | | | | | | | | | | | | | | | Features. Description: Error stack used to report only hdf5 predefined error messages because it takes only static strings. Runtime defined messages were not pushed to the stack. Added the means and macros to push MPI error strings onto the hdf5 error stack. Added a new minor error class as H5E_MPIERR for this class of messages. H5Epulbic.h, H5E.c: Added H5E_MPIERR and its minor class description. H5Eprivate.h: Added HMPI_XXX macros to push MPI error strings to the stack. H5FDmpio.c: Changed couple places to use the new macros to test the new macros. A more through changes to make use of these new macros will be done later. Platforms tested: eirene (serial, parallel) modi4(parallel)
* [svn-r4732] Bill Wendling2001-12-181-3/+3
| | | | | | | | | | | | | | | Purpose: Bug Fix Description: Committing the changes in the MPI/IO stuff so that parallel HDF5 will work on HP-UX. It seems that on HP-UX, the MPI_Status variable needs to be initialized to 0 for it to work (i.e., in some other MPI calls, if there's garbage in the MPI_Status variable, then it will barf). Solution: Initialized to 0. Platforms tested: HP-UX parallel, Linux.
* [svn-r4696] Raymond Lu2001-12-111-0/+1
| | | | | | | | | | | | | | | | | Purpose: Modify H5Fclose behavior Description: The HDF5 actual file close behaves in several ways in terms of if there are still objects(dataset, group, datatype) opened in file. Solution: Added a new file access property, file close degree. It has four values, H5F_CLOSE_DEFAULT H5F_CLOSE_WEAK H5F_CLOSE_SEMI H5F_CLOSE_STRONG The way a file is closed is decided by these values. Platforms tested: IRIX64 6.5, SunOS 5.6, FreeBSD 4.4
* [svn-r4634] Purpose:Quincey Koziol2001-11-211-2/+2
| | | | | | | | | Bug fix Description: A couple more compile problems from the recent elimination of IDs from internal function calls. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r4620] Purpose:Quincey Koziol2001-11-201-28/+37
| | | | | | | | | Code cleanup Description: Get rid of IDs from internal function calls and some small cleanups from the old-stype => generic property list conversion. Platforms tested: FreeBSD 4.4 (hawkwind)
* [svn-r4589] Purpose:Quincey Koziol2001-11-031-2/+3
| | | | | | | | Code cleanup Description: Clean up various compiler warnings from generic property updates. Platforms tested: FreeBSD 4.4 (hawkwind)
* [svn-r4572] Raymond Lu2001-10-251-1/+1
| | | | | | | Purpose: Followup file access property list changes. Platforms tested: IRIX64, SunOS 5.7, FreeBSD.
* [svn-r4569] Raymond Lu2001-10-241-8/+23
| | | | | | | | | Purpose: Generic Property List Change Description: Changed file access list to the new generic list. Platforms tested: IRIX64, SunOS5.7, FreeBSD
* [svn-r4488] Purpose:Quincey Koziol2001-09-281-4/+4
| | | | | | | | | Fix on Kludge Description: Forgot another chunk of parallel I/O code that needed to change for the generic property list kludge... :-/ Platforms tested: Parallel Linux 2.2.18smp (eirene)
* [svn-r4396] Purpose:Albert Cheng2001-08-201-0/+3
| | | | | | | | | | | bug fix Description: H5FD_mpio_flush() would try to file seek negative if the file->eoa is 0 (e.g., doing mpio on the raw-file of the split file driver). Put in a code to catch this case by returning succeed immediately. Platforms tested: modi4(pp)
* [svn-r4355] Purpose:Quincey Koziol2001-08-141-1/+1
| | | | | | | | | | | | | | | | | | | | | | Code cleanup (sorta) Description: When the first versions of the HDF5 library were designed, I remembered vividly the difficulties of porting code from a 32-bit platform to a 16-bit platform and asked that people use intn & uintn instead of int & unsigned int, respectively. However, in hindsight, this was overkill and unnecessary since we weren't going to be porting the HDF5 library to 16-bit architectures. Currently, the extra uintn & intn typedefs are causing problems for users who'd like to include both the HDF5 and HDF4 header files in one source module (like Kent's h4toh5 library). Solution: Changed the uintn & intn's to unsigned and int's respectively. Platforms tested: FreeBSD 4.4 (hawkwind)
* [svn-r4327] Purpose:Quincey Koziol2001-08-111-5/+13
| | | | | | | | | More code cleanups Description: Wrap up the code cleanups for changing the dataset transfer property lists over to using the generic property list code. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r4181] Purpose:Quincey Koziol2001-07-101-4/+4
| | | | | | | | | Bug Fix, Code Cleanup, Code Optimization, etc. Description: Fold in the hyperslab speedups, clean up compile warnings and change a few things from using 'unsigned' or 'hsize_t' to use 'size_t' instead. Platforms tested: FreeBSD 4.3 (hawkwind), Solaris 2.7 (arabica), Irix64 6.5 (modi4)
* [svn-r3914] Purpose:Bill Wendling2001-05-111-2/+12
| | | | | | | | | | | | | Replace Kludge with Another Kludge Description: The MPI_Get_count function isn't found on most systems. We need to replace this with a configure call which checks and defines the macro H5_HAVE_MPI_GET_COUNT or something. Solution: If-def'ed out the code for everyone (since H5_HAVE_MPI_GET_COUNT is never defined anywhere). Platforms tested: SDSC HP N9000
* [svn-r3781] Purpose:Bill Wendling2001-04-051-9/+9
| | | | | | | | | | | | | | | | | | Update Description: Changed #include <hdf_file.h> construct to #include "hdf_file.h" so that the GNU compiler can more easily pick up the dependencies which it places in the .depend and Dependencies files. Also regenerated the Dependencies to go along with this. Platforms tested: Linux
* [svn-r3256] Purpose:Albert Cheng2001-01-101-7/+6
| | | | | | | | | | Typo fix and comment Description: The FUNC_ENTER of H5FD_mpio_flush had a wrong function name. Corrected it. Also filled in the missing original author name. Added a 'this is collective.' to all collective functions. Platforms tested: It was just editing. Can't go wrong. :-)
* [svn-r3219] ./hdf5/src/H5FDmpio.cRobb Matzke2000-12-291-6/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 2000-12-29 13:26:28 Robb Matzke <matzke@llnl.gov> * H5FD_mpio_write: Avoid MPI_Get_count() when using the LAM implementation of MPI. This code must have other problems anyway because Kim Yates has a similar kludge in place to avoid negative return values from the same function. 2000-12-29 13:25:30 Robb Matzke <matzke@llnl.gov> * H5FD_mpio_read: Avoid MPI_Get_count() when using the LAM implementation of MPI. This code must have other problems anyway because Kim Yates has a similar kludge in place to avoid negative return values from the same function. 2000-12-29 11:28:13 Robb Matzke <matzke@llnl.gov> * H5FD_mpio_flush: Read and rewrite the last allocated byte of the file to insure that EOF>=EOA. 2000-12-29 09:32:49 Robb Matzke <matzke@llnl.gov> * H5FD_mpio_write: Removed unused variable. 2000-12-29 09:32:15 Robb Matzke <matzke@llnl.gov> * H5FD_mpio_write: Fixed a failure return value that should have been FAIL instead of NULL. 2000-12-29 09:31:47 Robb Matzke <matzke@llnl.gov> * H5FD_mpio_read: Removed unused variable. 2000-12-28 16:09:31 Robb Matzke <matzke@llnl.gov> * H5FD_mpio_query: The `flags' argument is advertised to be output only. Therefore I added code to zero its value before the feature bits are assigned.
* [svn-r3120] Purpose:Albert Cheng2000-12-121-1/+0
| | | | | | | | | | | | | Bug fix Description: In H5FD_mpio_open, there was a mem-xfree of the file variable before it ever gets alloc'ed. This is inside a failure condition that rarely happened. But if it happens, it would be freeing a null pointer. Solution: Removed the error code. Platforms tested: Modi4 -64 parallel. (Everything still works. :-)
* [svn-r3010] Purpose:Quincey Koziol2000-11-281-5/+5
| | | | | | | | | | | | | | Bug fix Description: When the v1.2 compatibility code was turned on, internal functions in the library were getting confused. Solution: Separated guts of H5Pget_driver call into an API function (the definition of which depends on the compatibility switch) and an internal function which always behaves like the v1.3/4 function. Replaced API function calls in the library code with the internal function. Platforms tested: FreeBSD 4.2 (hawkwind)
* [svn-r3005] Purpose:Quincey Koziol2000-11-271-2/+2
| | | | | | | | Backward compatibility code Description: Add in code to allow the library to emulate the v1.2 API and behavior. Platforms tested: FreeBSD 4.2 (hawkwind)
* [svn-r2935] Purpose:Albert Cheng2000-11-151-2/+2
| | | | | | | | | | | | | | | | | Bug fix Description: The optimized MPIO code was not working with the TFLOPS machine. It kept reporting "can't convert from size to size_i" errors in H5FD_mpio_write() and H5FD_mpio_read(). Solution: The error was actually in H5S_mpio_spaces_xfer() in H5Smpio.c in which the variable mpi_count was declared as size_t (only 4 bytes big in TFLOPS) when it should be hsize_t (8 bytes big in TFLOPS). Corrected and also changed to use the (hsize_t) to case size_i in H5FD_mpio_write() and H5FD_mpio_read() since that is what it should be. Platforms tested: TFLOPS and modi4 (-64, parallel)
generated by cgit v0.12 at 2024-11-03 14:48:32 (GMT)