summaryrefslogtreecommitdiffstats
path: root/src/H5FDmpio.c
Commit message (Collapse)AuthorAgeFilesLines
* [svn-r5652] Purpose:Quincey Koziol2002-06-171-28/+89
| | | | | | | | | | | | Code cleanup Description: Use dataset transfer property list to hold information about the MPI types for the current transfer, instead of setting pseudo-global variables in the file's struct. Platforms tested: Linux 2.2.x (eirene) w/parallel & IRIX64 6.5 (modi4) w/parallel & FORTRAN
* [svn-r5650] Purpose:Quincey Koziol2002-06-171-17/+53
| | | | | | | | | | | | Code cleanup Description: Change MPI-I/O code to use the address of the dataset for the displacement, instead of having a separate displacement value. Removed displacement parameter from H5FD_mpio_setup parameters. Platforms tested: Linux 2.2.x (eirene) w/parallel & IRIX64 6.5 (modi4) w/parallel.
* [svn-r5550] Purpose:Quincey Koziol2002-06-071-0/+9
| | | | | | | | | | | | | | | | | Code Improvement Description: Split the metadata accumulator code into two parts: one for allowing writes of the accumulator buffer during reads (when the buffer is dirty and needs to be flushed to disk in order to hold the new metadata being read in) and another for only allowing writes of the buffer during writes. This allows the MPI-I/O VFL driver to use the metadata accumulator (but only during writes) and benefit from the reduced number of metadata I/O operations that it brings. Platforms tested: IRIX64 6.5 (modi4) w/parallel
* [svn-r5536] Purpose:Quincey Koziol2002-06-051-0/+1
| | | | | | | | | | | | | | | | | New feature. Description: Added a "small data" block allocation mechanism to the library, similar to the mechanism used for allocating metadata currently. See the RFC for more details: http://hdf.ncsa.uiuc.edu/RFC/SmallData/SmallData.html This reduces the number of I/O operations which hit the disk for my test program from 19 to 15 (i.e. from 393 to 15, overall). Platforms tested: Solaris 2.7 (arabica) w/FORTRAN and FreeBSD 4.5 (sleipnir) w/C++
* [svn-r5530] Purpose:Bill Wendling2002-06-041-3/+5
| | | | | | | | | | | | | | | Code Cleanup Description: Removed some compiler warnings. Solution: In a few cases, NULL was being returned when a FAIL was supposed to be returned instead. There were some header files which needed to be included in a few of the sources. A couple of if-then statements had assignments in the conditional part. The compiler warned that they should have extra "()"s around them. Made the code check the values instead. Platforms tested: Linux (parallel) Modi4 (parallel)
* [svn-r5471] Purpose:Quincey Koziol2002-05-291-37/+48
| | | | | | | | | | | | | | | | Code cleanup Description: Broke the FUNC_ENTER macro into several macros, with more specialized uses (which followup mail will describe). This was designed to move most/all of the checks which could be done at compile time to that point, instead of needlessly performing them (over & over :-) at run-time. This reduces the library's size (and thus staticly linked binaries) and has a minor speedup effect also. Platforms tested: IRIX64 6.5 (modi4) with parallel & FORTRAN enabled, and additional testing on FreeBSD and Solaris immediately after the checkin.
* [svn-r5467] Purpose:Quincey Koziol2002-05-281-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | Code cleanup. Description: Took Robb's recent ideas for improving the FUNC_ENTER/FUNC_LEAVE macros equivalents in the SAF library and adapted them to our library. I added an additional macro which is equivalent to FUNC_ENTER: FUNC_ENTER_NOINIT - Has the API tracing code, etc. from FUNC_ENTER but none of the library or interface initialization code. This is to be used _only_ for static functions and those which explicitly cannot have the library or interface initialization code enabled (like the API termination routines, etc.). This allowed many more of the functions in the library [but not all yet :-(] to be wrapped with FUNC_ENTER[_NOINIT]/FUNC_LEAVE pairs. It also reduced the size of the library and executables (by cutting out a bunch of code which was never executed), I'll e-mail the exact results when I've finished editing it. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5447] Purpose:Quincey Koziol2002-05-211-34/+40
| | | | | | | | | | | Code cleanup Description: Guard against getting into metadata broadcast in write routine and clean up some error handling. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5440] Purpose:Quincey Koziol2002-05-201-38/+3
| | | | | | | | | | | New feature Description: Add 'closing' parameter to H5FDflush and VFL "flush" functions, per http://hdf.ncsa.uiuc.edu/RFC/VFLFlush/VFLFlush.html Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5429] Purpose:Quincey Koziol2002-05-171-0/+56
| | | | | | | | | | | | | | | | | | | | | Bug fix/Code improvement. Description: Currently, the chunk data allocation routine invoked to allocate space for the entire dataset is inefficient. It writes out each chunk in the dataset, whether it is already allocated or not. Additionally, this happens not only when it is created, but also anytime it is opened for writing, or the dataset is extended. Worse, there's too much parallel I/O syncronization, which slows things down even more. Solution: Only attempt to write out chunks that don't already exist. Additionally, share the I/O writing between all the nodes, instead of writing everything with process 0. Then, only block with MPI_Barrier if chunks were actually created. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5419] Purpose:Quincey Koziol2002-05-141-0/+11
| | | | | | | | | | | Bug fix. Description: Added barrier to flush routine to prevent race condition where file could be truncated. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5418] Purpose:Quincey Koziol2002-05-141-30/+26
| | | | | | | | | | | | | | | | Code cleanup/Performance enhancement Description: The code to extend the file size in H5FD_mpio_flush is getting run even when the file size doesn't change. Also, it's sort of sidestepping MPI-I/O when extending the file, instead of using MPI-I/O features to set the file's size. Solution: Only extend the file's size when the allocated size has increased. Also use MPI_File_set_size() to change the file's size. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5412] Purpose:Quincey Koziol2002-05-141-17/+50
| | | | | | | | | | | | | | | | Bug fix Description: Calling MPI_Get_count needs to be done with the same MPI type as was used for the transfer and we are always using MPI_BYTE, even when a different MPI type was used for the transfer. Solution: Only query MPI_Get_count with MPI_BYTE when we really used MPI_BYTE for the transfer. Wait for later to query MPI_Get_count with other MPI types. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5408] Purpose:Quincey Koziol2002-05-131-2/+41
| | | | | | | | | | | | | | | Performance enhancement Description: Doing an MPI_File_sync() just before a file is closed causing a large performance loss. Solution: Add flag to MPI file driver to avoid performance the MPI_File_sync() when the flag is set before a call to H5F_flush(). Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5403] Purpose:Quincey Koziol2002-05-131-9/+4
| | | | | | | | | | Back out changes. Description: Back out changes to VFL 'flush' API function, pending review. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5393] Purpose:Quincey Koziol2002-05-101-5/+9
| | | | | | | | | | | | | | | | | | | | | | New Feature Description: The VFL flush function is called immediately before a file is closed. This can cause duplicate syncronization actions to occur, if the VFL close function also performs them. Solution: Added 'closing' parameter to VFL 'flush' operation. This allows the VFL flush function to bypass operations that will be duplicated within the VFL close function. Additionally, use the 'closing' parameter to bypass calls to MPI_File_sync() when set. Since MPI_File_close() also syncronizes the file, this avoids the terrible performance hit taken when calling MPI_File_sync() as the file is closing. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5390] Purpose:Quincey Koziol2002-05-101-48/+11
| | | | | | | | | | | | | | | | | | | | | | Code cleanup Description: The parallel I/O file driver is optimized to only write metadata with one process (and broadcast the results to the other processes). This is currently enabled by a separate call to H5FD_mpio_tas_allsame() before each metadata write to the file. This can easily lead to problems where the prelude function call is omitted before the actual write code or, in a threaded environment, lead to race condititions where the value set is reset before being used. Solution: Since we only want to write metadata from one process, key off of the 'type' parameter (which has information about whether the data being written it metadata or raw data) to H5FD_mpio_write() as the method for determining whether to only write from one process or not. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5385] Purpose:Quincey Koziol2002-05-101-4/+21
| | | | | | | | | | | | | | | New Feature Description: Currently, only process 0 writes metadata to disk, leading to a potential performance bottleneck as the other processors wait for it to catch up. Solution: Rotate the metadata responsibilities among all processes, speading out the work. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5381] Purpose:Quincey Koziol2002-05-091-32/+19
| | | | | | | | | | | Small code cleanup/improvement Description: Query the MPI rank only once and store it in the file structure for each process. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r5135] Purpose:Albert Cheng2002-04-031-6/+11
| | | | | | | | | | | | | | | | | | | | | | Features. Description: Error stack used to report only hdf5 predefined error messages because it takes only static strings. Runtime defined messages were not pushed to the stack. Added the means and macros to push MPI error strings onto the hdf5 error stack. Added a new minor error class as H5E_MPIERR for this class of messages. H5Epulbic.h, H5E.c: Added H5E_MPIERR and its minor class description. H5Eprivate.h: Added HMPI_XXX macros to push MPI error strings to the stack. H5FDmpio.c: Changed couple places to use the new macros to test the new macros. A more through changes to make use of these new macros will be done later. Platforms tested: eirene (serial, parallel) modi4(parallel)
* [svn-r4732] Bill Wendling2001-12-181-3/+3
| | | | | | | | | | | | | | | Purpose: Bug Fix Description: Committing the changes in the MPI/IO stuff so that parallel HDF5 will work on HP-UX. It seems that on HP-UX, the MPI_Status variable needs to be initialized to 0 for it to work (i.e., in some other MPI calls, if there's garbage in the MPI_Status variable, then it will barf). Solution: Initialized to 0. Platforms tested: HP-UX parallel, Linux.
* [svn-r4696] Raymond Lu2001-12-111-0/+1
| | | | | | | | | | | | | | | | | Purpose: Modify H5Fclose behavior Description: The HDF5 actual file close behaves in several ways in terms of if there are still objects(dataset, group, datatype) opened in file. Solution: Added a new file access property, file close degree. It has four values, H5F_CLOSE_DEFAULT H5F_CLOSE_WEAK H5F_CLOSE_SEMI H5F_CLOSE_STRONG The way a file is closed is decided by these values. Platforms tested: IRIX64 6.5, SunOS 5.6, FreeBSD 4.4
* [svn-r4634] Purpose:Quincey Koziol2001-11-211-2/+2
| | | | | | | | | Bug fix Description: A couple more compile problems from the recent elimination of IDs from internal function calls. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r4620] Purpose:Quincey Koziol2001-11-201-28/+37
| | | | | | | | | Code cleanup Description: Get rid of IDs from internal function calls and some small cleanups from the old-stype => generic property list conversion. Platforms tested: FreeBSD 4.4 (hawkwind)
* [svn-r4589] Purpose:Quincey Koziol2001-11-031-2/+3
| | | | | | | | Code cleanup Description: Clean up various compiler warnings from generic property updates. Platforms tested: FreeBSD 4.4 (hawkwind)
* [svn-r4572] Raymond Lu2001-10-251-1/+1
| | | | | | | Purpose: Followup file access property list changes. Platforms tested: IRIX64, SunOS 5.7, FreeBSD.
* [svn-r4569] Raymond Lu2001-10-241-8/+23
| | | | | | | | | Purpose: Generic Property List Change Description: Changed file access list to the new generic list. Platforms tested: IRIX64, SunOS5.7, FreeBSD
* [svn-r4488] Purpose:Quincey Koziol2001-09-281-4/+4
| | | | | | | | | Fix on Kludge Description: Forgot another chunk of parallel I/O code that needed to change for the generic property list kludge... :-/ Platforms tested: Parallel Linux 2.2.18smp (eirene)
* [svn-r4396] Purpose:Albert Cheng2001-08-201-0/+3
| | | | | | | | | | | bug fix Description: H5FD_mpio_flush() would try to file seek negative if the file->eoa is 0 (e.g., doing mpio on the raw-file of the split file driver). Put in a code to catch this case by returning succeed immediately. Platforms tested: modi4(pp)
* [svn-r4355] Purpose:Quincey Koziol2001-08-141-1/+1
| | | | | | | | | | | | | | | | | | | | | | Code cleanup (sorta) Description: When the first versions of the HDF5 library were designed, I remembered vividly the difficulties of porting code from a 32-bit platform to a 16-bit platform and asked that people use intn & uintn instead of int & unsigned int, respectively. However, in hindsight, this was overkill and unnecessary since we weren't going to be porting the HDF5 library to 16-bit architectures. Currently, the extra uintn & intn typedefs are causing problems for users who'd like to include both the HDF5 and HDF4 header files in one source module (like Kent's h4toh5 library). Solution: Changed the uintn & intn's to unsigned and int's respectively. Platforms tested: FreeBSD 4.4 (hawkwind)
* [svn-r4327] Purpose:Quincey Koziol2001-08-111-5/+13
| | | | | | | | | More code cleanups Description: Wrap up the code cleanups for changing the dataset transfer property lists over to using the generic property list code. Platforms tested: IRIX64 6.5 (modi4)
* [svn-r4181] Purpose:Quincey Koziol2001-07-101-4/+4
| | | | | | | | | Bug Fix, Code Cleanup, Code Optimization, etc. Description: Fold in the hyperslab speedups, clean up compile warnings and change a few things from using 'unsigned' or 'hsize_t' to use 'size_t' instead. Platforms tested: FreeBSD 4.3 (hawkwind), Solaris 2.7 (arabica), Irix64 6.5 (modi4)
* [svn-r3914] Purpose:Bill Wendling2001-05-111-2/+12
| | | | | | | | | | | | | Replace Kludge with Another Kludge Description: The MPI_Get_count function isn't found on most systems. We need to replace this with a configure call which checks and defines the macro H5_HAVE_MPI_GET_COUNT or something. Solution: If-def'ed out the code for everyone (since H5_HAVE_MPI_GET_COUNT is never defined anywhere). Platforms tested: SDSC HP N9000
* [svn-r3781] Purpose:Bill Wendling2001-04-051-9/+9
| | | | | | | | | | | | | | | | | | Update Description: Changed #include <hdf_file.h> construct to #include "hdf_file.h" so that the GNU compiler can more easily pick up the dependencies which it places in the .depend and Dependencies files. Also regenerated the Dependencies to go along with this. Platforms tested: Linux
* [svn-r3256] Purpose:Albert Cheng2001-01-101-7/+6
| | | | | | | | | | Typo fix and comment Description: The FUNC_ENTER of H5FD_mpio_flush had a wrong function name. Corrected it. Also filled in the missing original author name. Added a 'this is collective.' to all collective functions. Platforms tested: It was just editing. Can't go wrong. :-)
* [svn-r3219] ./hdf5/src/H5FDmpio.cRobb Matzke2000-12-291-6/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 2000-12-29 13:26:28 Robb Matzke <matzke@llnl.gov> * H5FD_mpio_write: Avoid MPI_Get_count() when using the LAM implementation of MPI. This code must have other problems anyway because Kim Yates has a similar kludge in place to avoid negative return values from the same function. 2000-12-29 13:25:30 Robb Matzke <matzke@llnl.gov> * H5FD_mpio_read: Avoid MPI_Get_count() when using the LAM implementation of MPI. This code must have other problems anyway because Kim Yates has a similar kludge in place to avoid negative return values from the same function. 2000-12-29 11:28:13 Robb Matzke <matzke@llnl.gov> * H5FD_mpio_flush: Read and rewrite the last allocated byte of the file to insure that EOF>=EOA. 2000-12-29 09:32:49 Robb Matzke <matzke@llnl.gov> * H5FD_mpio_write: Removed unused variable. 2000-12-29 09:32:15 Robb Matzke <matzke@llnl.gov> * H5FD_mpio_write: Fixed a failure return value that should have been FAIL instead of NULL. 2000-12-29 09:31:47 Robb Matzke <matzke@llnl.gov> * H5FD_mpio_read: Removed unused variable. 2000-12-28 16:09:31 Robb Matzke <matzke@llnl.gov> * H5FD_mpio_query: The `flags' argument is advertised to be output only. Therefore I added code to zero its value before the feature bits are assigned.
* [svn-r3120] Purpose:Albert Cheng2000-12-121-1/+0
| | | | | | | | | | | | | Bug fix Description: In H5FD_mpio_open, there was a mem-xfree of the file variable before it ever gets alloc'ed. This is inside a failure condition that rarely happened. But if it happens, it would be freeing a null pointer. Solution: Removed the error code. Platforms tested: Modi4 -64 parallel. (Everything still works. :-)
* [svn-r3010] Purpose:Quincey Koziol2000-11-281-5/+5
| | | | | | | | | | | | | | Bug fix Description: When the v1.2 compatibility code was turned on, internal functions in the library were getting confused. Solution: Separated guts of H5Pget_driver call into an API function (the definition of which depends on the compatibility switch) and an internal function which always behaves like the v1.3/4 function. Replaced API function calls in the library code with the internal function. Platforms tested: FreeBSD 4.2 (hawkwind)
* [svn-r3005] Purpose:Quincey Koziol2000-11-271-2/+2
| | | | | | | | Backward compatibility code Description: Add in code to allow the library to emulate the v1.2 API and behavior. Platforms tested: FreeBSD 4.2 (hawkwind)
* [svn-r2935] Purpose:Albert Cheng2000-11-151-2/+2
| | | | | | | | | | | | | | | | | Bug fix Description: The optimized MPIO code was not working with the TFLOPS machine. It kept reporting "can't convert from size to size_i" errors in H5FD_mpio_write() and H5FD_mpio_read(). Solution: The error was actually in H5S_mpio_spaces_xfer() in H5Smpio.c in which the variable mpi_count was declared as size_t (only 4 bytes big in TFLOPS) when it should be hsize_t (8 bytes big in TFLOPS). Corrected and also changed to use the (hsize_t) to case size_i in H5FD_mpio_write() and H5FD_mpio_read() since that is what it should be. Platforms tested: TFLOPS and modi4 (-64, parallel)
* [svn-r2811] Dan Wells2000-11-081-1/+1
| | | | | | | | | | | | Purpose: Bug fix. Description: The names of some entries did not correspond with the names in the FUNC_ENTER macro. This would cause inaccuracies during Pablo tracing. Solution: The names were changed in FUNC_ENTER to agree with the entry names. Platforms tested: Solaris, Irix, AIX, HP Vclass
* [svn-r2774] H5FD.c:Albert Cheng2000-10-311-4/+32
| | | | | | | | | | | | | | | | | | | | Purpose: Bug fix Description: H5FD_close() wiped clean the public information before closing the FD. This caused problem to the lower level close() which wants to verify it has indeed handed a valid file-driver to close because that information is wiped out prematurely. Solution: Moved the public information part cleaning after it is closed. H5FDmpio.c: Added in a bunch of assertion to make sure the file handle is indeed an MPIO file. Platforms tested: IRIX64 -64 parallel
* [svn-r2722] Purpose:Quincey Koziol2000-10-241-2/+2
| | | | | | | | | | Feature symmetry Description: A while ago I needed to get the 'type' of data being accessed during writes to the VFL driver, so I put in code to get the information down there. Albert asked for the same information during reads, so I've added that in. Tested: FreeBSD 4.1.1 (hawkwind)
* [svn-r2657] Quincey Koziol2000-10-101-1/+1
| | | | | | | | | | | | Purpose: Parallel Bug Fixes Description: Was out of sync with header file re-arrangements I checked in last night. Solution: Fixed to use new header files, etc. Platforms tested: O2K (modi4)
* [svn-r2652] Purpose:Quincey Koziol2000-10-101-0/+1
| | | | | | | | | | | | | | | Maintainance & performance enhancements Description: Re-arranged header files to protect private symbols better. Changed optimized regular hyperslab I/O to compute the offsets more efficiently from previous method of using matrix operations. Added sequential I/O operations at a more abstract level (at the same level as H5F_arr_read/write), to support the optimized hyperslab I/O. Platforms tested: Solaris 2.6 (baldric) & FreeBSD 4.1.1 (hawkwind)
* [svn-r2645] Albert Cheng2000-10-101-17/+25
| | | | | | | | | | | | | | | | Purpose: [is this a bug fix? feature? ...] Description: [describe the bug, or describe the new feature, etc] Solution: [details about the changes, algorithm, etc...] [Please as detail as you can since your own explanation is better than others guessing it from the code.] Platforms tested: [machines you have tested the changed version. This is absolute important. Test it out on at least two or three different platforms such as Big-endian-32bit (SUN/IRIX), little-endian-32(LINUX) and 64-bit (IRIX64/UNICOS/DEC-ALPHA) would be good.]
* [svn-r2498] Added new VFL 'query' code and added new 'type of data' ↵Quincey Koziol2000-08-311-2/+38
| | | | | | parameter to write call.
* [svn-r2057] Purpose:Albert Cheng2000-03-241-0/+1
| | | | | | | | | | | | | | Bug fix for parallel mode. Description: H5FD_mpio_tas_allsame was called for all cases, even when MPIO is not used for access. That corrupted the internal file handle structure. Solution: Define a macro, IS_H5FD_MPIO(f), for testing if f is opened with MPIO access. Will call H5FD_mpio_tas_allsame only if this condition is true. Platform tested: O2K, both -64 and -n32 modes.
* [svn-r1917] Updated the change history of H5FD_mpio_write().Albert Cheng1999-12-201-0/+7
|
* [svn-r1915] Purpose:Albert Cheng1999-12-201-15/+21
| | | | | | | | | | | | | | | | | | | Bug fix. Description: When only p0 will write for allsame metadata, other processes returns with SUCCEED immediately. In some racing conditions, the other processes may start reading the metadata before p0 has finished writing them out. That results in errors. (I verified by putting an intentional delay in the p0 code. Other processes then got read errors always.) Solution: When only-p0-writes is the case, p0 will Bcast the result to other processes. This provides both a synchoronization and more valid logic since the other processes will return FAIL too IF p0 write fails. Also fixed a typo of the H5F_MPIO_DEBUG_KEY name. Also removed the incorrect #define's of FALSE and TRUE. They are already defined in H5private.h.
generated by cgit v0.12 at 2024-11-03 13:59:39 (GMT)