summaryrefslogtreecommitdiffstats
path: root/src/H5FPclient.c
Commit message (Collapse)AuthorAgeFilesLines
* [svn-r8731] Purpose:Quincey Koziol2004-06-231-4/+0
| | | | | | | | | | | | | | | Code cleanup & minor optimization Description: Re-work the way interface initialization routines are specified in the library to avoid the overhead of checking for them in routines where there is no interface initialization routine. This cleans up warnings with gcc 3.4, reduces the library binary size a bit (about 2-3%) and should speedup the library's execution slightly. Platforms tested: FreeBSD 4.10 (sleipnir) w/gcc34 h5committest
* [svn-r8391] Purpose:John Mainzer2004-04-191-3/+170
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Checkpoint checkin of FP bug fixes. FP is still quite buggy, but I must go deal with other matters. Description: Fixed two major bugs: 1) H5FPserver.c was clobbering meta data in its care. 2) H5FPserver.c was allocating the same space multiple times, causing both data and meta data corruption. Also made minor fixes, added debugging code, and familiarized myself with the FP code. All development work with FP enabled was done on Eirene. On this platform, FP now passes its test reliably with up to 9 processes. At 10 processes it seg faults every time. I haven't looked into this issue. There are also several known locking bugs which have to be fixed. However, they are of sufficiently low probability that I didn't bother with them on this pass. FP has not been tested with deletions -- this should be done. Also, need to test FP chunked I/O. Solution: 1) Modified cache in H5FPserver.c to merge changes correctly. Found and fixed a bug in H5TB.c in passing. 2) Multiple space allocation was caused by a race condition with set eoa requests. Most of these eoa requests appeared to be superfluous, so I deleted them. Those issued during the superblock read seemed necessary, so I inserted a barrier at the end of the superblock read, to prevent races with allocations. Platforms tested: h5committested
* [svn-r8126] Purpose:Quincey Koziol2004-01-311-6/+6
| | | | | | | | | | | | | | | | | Bug fix/optimization Description: Address slowdown in MPI-I/O file metadata operations that was introduced mid-stream. We now _require_ a POSIX compliant parallel file system for the MPI-I/O file driver (as well as for the MPI-POSIX file driver). Also optimized file open operation when the file is being created by reducing the number of collective & syncronizing calls. Additionally, refactor the MPI routines into a common place, eliminating duplicated code. Platforms tested: FreeBSD 4.9 (sleipnir) w/parallel h5committest
* [svn-r8038] Purpose:Quincey Koziol2004-01-081-1/+1
| | | | | | | | | | | | | | | | | | | | Bug fix Description: When two property lists are compared, the H5Pequal routine was just comparing the raw information for the property values. This causes problems when the raw information contains pointers to other information. Solution: Allow a 'compare' callback to be registered for properties, so that a user application get perform the comparison itself, allowing for "deep" compares of the property value. This was exported to the H5Pregister & H5Pinsert routines in the development branch, but not the release branch. Platforms tested: FreeBSD 4.9 (sleipnir) h5committest
* [svn-r7993] Purpose:Quincey Koziol2003-12-291-3/+1
| | | | | | | | | | | | Code cleanup, bug fixes Description: Wrap up rest of changes necessary for fixing the "short" MPI-I/O read problem that Robb reported. Platforms tested: FreeBSD 4.9 (sleipnir) too minor to require h5committest
* [svn-r7917] Purpose:Quincey Koziol2003-12-061-2/+2
| | | | | | | | | | | | | | | | | | | Code cleanup Description: Clean up compiler warnings, especially the 'FUNC' variable not used which comes out in production mode. Solution: Had to add a new FUNC_ENTER_NOAPI_NOINIT_NOFUNC macro for those non-API functions which don't need the 'FUNC' variable defined. (This will be _so_ much easier when C99 is standard on all our supposed platforms, since it has a __FUNC__ macro... ) Platforms tested: FreeBSD 4.9 (sleipnir) too minor for h5committest (although there were lots of files changed, the change was minor in each one)
* [svn-r7847] Purpose:Bill Wendling2003-11-131-0/+14
| | | | | | | | | | | | | | | | | | Bug Fix Description: The SAP was sending back replies to the client but the client wasn't picking them up (this was after a dump from the server. Solution: Read the extra replies from the server. Platforms tested: AIX (w/ FPHDF5) Linux (w/ FPHDF5) Solaris (w/ Fortran & C++) Misc. update:
* [svn-r7805] Purpose:Bill Wendling2003-10-311-13/+116
| | | | | | | | | | | | | | | | | | | | Bug Fix Description: The End of Address information needed to be kept by the SAP. Some processes were trying to get the EOA information in collective mode, but the action wasn't collective at the time. Solution: Keep the EOA information for the file on the SAP. Clients query the SAP to get/set it when needed. Platforms tested: Linux (w/ FPHDF5) Copper (w/o FPHDF5) (FPHDF5 specific, so no need for full testing) Misc. update:
* [svn-r7789] Purpose:Quincey Koziol2003-10-291-2/+2
| | | | | | | | | | | | | | | Bug fix & code cleanups Description: Change our use of MPI derived datatypes to not create datatypes with "0-sized" lengths, which causes the LANL Q machine to hang. Also, get rid of "prefer MPI derived datatypes" environment variable since it has no advantage. Platforms tested: FreeBSD 4.9 (sleipnir) w & w/o parallel h5committest
* [svn-r7498] Purpose:Quincey Koziol2003-09-191-1/+1
| | | | | | | | | | | Code cleanup Description: Clean up various warnings and parameter mis-matches, etc. Platforms tested: FreeBSD 4.9 (sleipnir) too minor to need h5committest
* [svn-r7489] Purpose:Bill Wendling2003-09-181-7/+20
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Update Description: A lot of modifications for the FPHDF5 stuff: H5AC.c H5ACprivate.h - Removed AC_find (it's replaced with AC_protect and AC_unprotect). Added flushing if it's an FPHDF5 driver and we're doing an AC_set or AC_unprotect with the dirty flag set. H5B.c - Split up the B_flush function into different functions since the one function was doing serialization which is better left as a separate entity. H5D.c - Removed some FPHDF5 code that was incorrect H5F.c - Split up the F_flush function so that it no longer allocates file space. Created new functions (F_init_superblock, F_read_superblock, and F_write_superblock) for greater modularity and so that the FPHDF5 non-captain processes can read the superblock after the captain process writes it. H5FD.c - Error message correction. H5FDfphdf5.c - Removed MPI barrier call that wasn't needed. H5FPclient.c H5FPserver.c - Modified so that if a process requests data that isn't exactly aligned, we can return it if we have the block that contains the requested address. H5G.c H5Gent.c H5Gnode.c H5HL.c H5HLpkg.h H5HLprivate.h H5Oefl.c - Removed the H5HL_peek function since it was doing a (now unsafe) holding of the information in the cache. Replaced with protect and unprotect calls. H5TB.c - Error fix. The TB_dless function wasn't working properly. H5Gstab.c - Format change. H5err.txt H5Edefin.h H5Einit.h H5Epubgen.h H5Eterm.h - Added new error code. Platforms tested: Modi4 (paralle, Fortran) Sol (Fortran) Linux (C++, Fortran) Copper (Parallel, Fortran) Misc. update:
* [svn-r7460] Purpose:Bill Wendling2003-09-101-2/+2
| | | | | | | | | | | | | | | Update Description: Added extra parameter to the H5AC_protect() function that indicates if the cache being asked for is going to be written to or just read from. Those AC_protect calls that were H5AC_find calls are now read-only. The rest are writes. Platforms tested: Linux (h5committest not needed due to size and small impact) Misc. update:
* [svn-r7373] Purpose:Quincey Koziol2003-08-151-2/+2
| | | | | | | | | | | | Code cleanup Description: Cleanup H5Pclose code and also switch a few internal users of H5Pclose to use H5I_dec_ref instead. Platforms tested: FreeBSD 4.8 (sleipnir) too small for h5committest
* [svn-r7367] Purpose:Bill Wendling2003-08-151-19/+83
| | | | | | | | | | | | | | | | | | | Update Description: Added an extra flag to the clear functions that triggers a "destroy" of the object being cleared if necessary. This is a fix for the FPHDF5 stuff which had an object sticking around after it was cleared. (In FPHDF5, some processes are in charge of destroying the object, but all processes might actually allocate the object.) Platforms tested: AIX (Copper: Fortran & C++) Linux (Verbena: Fortran & C++) IRIX (Modi4: Parallel & Fortran) (Sol is down) Misc. update:
* [svn-r6556] Purpose:Bill Wendling2003-04-011-10/+12
| | | | | | | | | | | | | | | | | | | Update & Bug Fix Description: The "free" protocol was missing. Added that to the server side. When doing a "create" of a file (with no other data structures created), the freespace in the file wasn't being reclaimed. Solution: After adding the free protocol, we put the burden of running through the FD_free function on the SAP instead of each client. Platforms tested: Linux Misc. update:
* [svn-r6527] Purpose:Bill Wendling2003-03-271-1/+3
| | | | | | | | | | | | | | | | | | | | | | | Bug Fix & Update Description: FPHDF5 was creating files which didn't have the EOA field in the superblock set correctly. It turns out that the SAP was keeping this information to itself instead of giving it to the client processes. Naughty SAP! Solution: Have the SAP send this information back to the clients so that they can update the superblock as necessary. This now creates a file (with just the root group) that looks correct! Only problem is that there's extra file space being allocated. Also, at program termination, there's an infinite loop... Platforms tested: H5committests (run by hand on burrwhite, arabica, and modi4) Misc. update:
* [svn-r6514] Purpose:Bill Wendling2003-03-201-38/+38
| | | | | | | | | | | | | | | | | | | | | | | | | Bug fix and Update Description: From Quincey's comments on the code I checked in last night: - In H5F_close call, the "private" processes should call the H5F_flush with the "CLEAR_ONLY" flag. - There's no need for a special case for FPHDF5 in the FD_real_alloc function since FPHDF5 doesn't define an alloc function. - The return type of H5Pset_fapl_fphdf5 should be herr_t instead of hid_t. I don't know how it got that way in the first place. - The variable names for MPI types and the structure typedefs should be switched: H5FP_request/H5FP_request_t to H5FP_request_t/H5FP_request and so on. - In the H5FP.c module, I was commiting the H5FP_request MPI datatype but using the wrong offset field... Platforms tested: Linux...will test on others, but these are mostly FPHDF5 changes. Misc. update:
* [svn-r6504] Purpose:Bill Wendling2003-03-191-41/+37
| | | | | | | | | | | | | | | | | | | | | | | | | Why not? Description: Lots of changes. This is basically a "commit because there are a lot of changes" commit. The server now acts more like a metadata caching server. The dumping mechanism finally works (I think). At least in the test I did it seemed to write things back. There's a new MPI TAG to use (H5FP_TAG_DUMP) when dumping things. Changed a lot of the functions from passing a structure around to just passing a pointer to that structure. I have no clue why I did it the previous way... Um...And more synchronization stuff between the client and server. The protocol between the two was being broke in a couple of places. We're getting closer! Platforms tested: Linux (probably Modi4 too) Misc. update:
* [svn-r6423] Purpose:Bill Wendling2003-02-211-11/+124
| | | | | | | | | Update Description: Added support for the allocation and freeing of space in the file. This information is kept on the Server. Platforms tested: Linux & Modi4
* [svn-r6401] Purpose:Bill Wendling2003-02-121-30/+114
| | | | | | | | | | | | | | | | Update Description: Added a "metadata flush" function which allows the clients to force a dump of the metadata from the SAP. This should be done before closing the file. Modified the information stored on an "open" of the file. The filename is unnecessary, so I got rid of it. More integration with the File Driver code: Passing in a data xfer property list. Platforms tested: Linux
* [svn-r6390] Purpose:Bill Wendling2003-02-101-26/+47
| | | | | | | | | | | | | | | | Update Description: Removed some obsolete fields - such as the AC_subst_t object ID - which isn't needed. Fixed so that there are fewer global variables. There are still some left, however. Modified the client and server code so that it handles read requests better. There were some flaws in how this was done before (it was calling H5FD_read() instead of just returning the status that it couldn't find the metadata in the cache). Platforms tested: Linux
* [svn-r6379] Purpose:Bill Wendling2003-02-061-41/+46
| | | | | | | | | | | | | | Update Description: H5FP.c, H5FPclient.c, H5FPprivate.h, H5FPserver.c: Update. More progression towards the SAP as metadata cache. It only lacks the ability to take care of metadata allocations. H5FDfphdf5.[ch]: Start of a new driver for FPHDF5. Not fully implemented just yet... Platforms tested: Linux
* [svn-r6371] Purpose:Bill Wendling2003-02-031-64/+233
| | | | | | | | | | | | | | Update Description: Changes: - Added support for the server dumping metadata writes to a client. - Some of the code wasn't handling allocated buffers correctly (freeing twice). - Modified server so that it handles metadata only. Platforms tested: Linux
* [svn-r6336] Purpose:Bill Wendling2003-01-271-316/+97
| | | | | | | | | | | | | | | Update Description: This is the first conversion of the FPHDF5 code to be a metadata cache. There's an extra error message. I rewrote the sync/change code to be read metadata/write metadata instead. I still need to hook these changes into the HDF5 code so that it looks at the SAP first before checking the file for metadata. Platforms tested: Linux
* [svn-r6266] Purpose:Quincey Koziol2003-01-101-9/+9
| | | | | | | | | | | | Code cleanup/new feature. Description: Split FUNC_LEAVE into API and non-API specific versions. This allows a solution to compiling this branch with C++, as well as reducing the size of the binaries produced. Platforms tested: FreeBSD 4.7 (sleipnir) w/serial, parallel (including MPE) & thread-safe
* [svn-r6252] Purpose:Quincey Koziol2003-01-091-2/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Lots of performance improvements & a couple new internal API interfaces. Description: Performance Improvements: - Cached file offset & length sizes in shared file struct, to avoid constantly looking them up in the FCPL. - Generic property improvements: - Added "revision" number to generic property classes to speed up comparisons. - Changed method of storing properties from using a hash-table to the TBBT routines in the library. - Share the propery names between classes and the lists derived from them. - Removed redundant 'def_value' buffer from each property. - Switching code to use a "copy on write" strategy for properties in each list, where the properties in each list are shared with the properties in the class, until a property's value is changed in a list. - Fixed error in layout code which was allocating too many buffers. - Redefined public macros of the form (H5open()/H5check, <variable>) internally to only be (<variable>), avoiding innumerable useless calls to H5open() and H5check_version(). - Reuse already zeroed buffers in H5F_contig_fill instead of constantly re-zeroing them. - Don't write fill values if writing entire dataset. - Use gettimeofday() system call instead of time() system when checking the modification time of a dataset. - Added reference counted string API and use it for tracking the names of objects opening in a file (for the ID->name code). - Removed redundant H5P_get() calls in B-tree routines. - Redefine H5T datatype macros internally to the library, to avoid calling H5check redundantly. - Keep dataspace information for dataset locally instead of reading from disk each time. Added new module to track open objects in a file, to allow this (which will be useful eventually for some FPH5 metadata caching issues). - Remove H5AC_find macro which was inlining metadata cache lookups, and call function instead. - Remove redundant memset() calls from H5G_namei() routine. - Remove redundant checking of object type when locating objects in metadata cache and rely on the address only. - Create default dataset object to use when default dataset creation property list is used to create datasets, bypassing querying for all the property list values. - Use default I/O vector size when performing raw data with the default dataset transfer property list, instead of querying for I/O vector size. - Remove H5P_DEFAULT internally to the library, replacing it with more specific default property list based on the type of property list needed. - Remove redundant memset() calls in object header message (H5O*) routines. - Remove redunant memset() calls in data I/O routines. - Split free-list allocation routines into malloc() and calloc()- like routines, instead of one combined routine. - Remove lots of indirection in H5O*() routines. - Simplify metadata cache entry comparison routine (used when flushing entire cache out). - Only enable metadata cache statistics when H5AC_DEBUG is turned on, instead of always tracking them. - Simplify address comparison macro (H5F_addr_eq). - Remove redundant metadata cache entry protections during dataset creation by protecting the object header once and making all the modifications necessary for the dataset creation before unprotecting it. - Reduce # of "number of element in extent" computations performed by computing and storing the value during dataspace creation. - Simplify checking for group location's file information, when file has not been involving in file-mounting operations. - Use binary encoding for modification time, instead of ASCII. - Hoist H5HL_peek calls (to get information in a local heap) out of loops in many group routine. - Use static variable for iterators of selections, instead of dynamically allocation them each time. - Lookup & insert new entries in one step, avoiding traversing group's B-tree twice. - Fixed memory leak in H5Gget_objname_idx() routine (tangential to performance improvements, but fixed along the way). - Use free-list for reference counted strings. - Don't bother copying object names into cached group entries, since they are re-created when an object is opened. The benchmark I used to measure these results created several thousand small (2K) datasets in a file and wrote out the data for them. This is Elena's "regular.c" benchmark. These changes resulted in approximately ~4.3x speedup of the development branch when compared to the previous code in the development branch and ~1.4x speedup compared to the release branch. Additionally, these changes reduce the total memory used (code and data) by the development branch by ~800KB, bringing the development branch back into the same ballpark as the release branch. I'll send out a more detailed description of the benchmark results as a followup note. New internal API routines: Added "reference counted strings" API for tracking strings that get used by multiple owners without duplicating the strings. Added "ternary search tree" API for text->object mappings. Platforms tested: Tested h5committest {arabica (fortran), eirene (fortran, C++) modi4 (parallel, fortran)} Other platforms/configurations tested? FreeBSD 4.7 (sleipnir) serial & parallel Solaris 2.6 (baldric) serial
* [svn-r6220] Purpose:Bill Wendling2002-12-191-75/+209
| | | | | | | | | | Update Description: A few generic changes to the FPHDF5 code. Some error messages reworked a bit. Cleaning up in case of failure improved in some cases. Added another field to the synchronization messages... Platforms tested: Linux
* [svn-r6084] Purpose:Bill Wendling2002-11-121-44/+19
| | | | | | | | | | | FPHDF5 Fixups Description: - Stopped using API calls in the H5FPclient code. It now uses internal library calls. - Removed some FIXME comments because they've been fixed. - Small fix for an enum starting at 37 for no reason :-) Platforms tested: Eirene (only affects FPHDF5, so no need to test 3 platforms just yet).
* [svn-r6077] Purpose:Quincey Koziol2002-11-121-7/+11
| | | | | | | | | | Code cleanup Description: Added some comments and made some minor code cleanups Platforms tested: minor change, only testing on FreeBSD 4.7 (sleipnir) w/parallel
* [svn-r6073] Purpose:Bill Wendling2002-11-121-43/+97
| | | | | | | | | | | Update Description: Added code which performs an update when there's a dataset creation. Commented on some FIXMEs which were in the code. Platforms tested: Eirene PP Arabica Fortran Modi4 PP Fortran
* [svn-r6057] Purpose:Bill Wendling2002-11-051-12/+0
| | | | | | | | | | | | | | | | | | | | | | | New Feature. Support for new FPHDF5 Feature Description: More support for the FPHDF5 feature. This splits the H5D_create function into separate parts which update the metadata cache. It was necessary to split apart the H5O_create function to do a similar thing since it would allocate real space on the hard disk. I'm checking this up now so that I don't get too far away from what the CVS repository has and so that others may look at the code and retch^H^H^H^H^Hmarvel at it. If there are any comments, send them my way. There is one outstanding issue, though. I need to use non-API functions in the FPHDF5 stuff. I'm using some APIs right now and should migrate to using others. At the moment, I'm putting that on the back burner until I'm more along in the implementation. Platforms tested: Arabica (C++) Eirene (Parallel) Modi4 (Parallel)
* [svn-r6040] Purpose:Quincey Koziol2002-10-281-8/+4
| | | | | | | | | | Code cleanup Description: Add more comments and clean up small bits of the FPH5 code. Platforms tested: FreeBSD 4.7 (sleipnir), changes too minor to affect other platforms.
* [svn-r6029] Purpose:Bill Wendling2002-10-231-1/+2
| | | | | | | | | | | | Oops Description: The H5FPprivate.h file was being #included even if PARALLEL was turned off. Solution: put the #include of the H5FPprivate.h file within the #ifdef block so that it doesn't get included if H5_HAVE_FPHDF5 isn't defined. Platforms tested: Linux...
* [svn-r6025] Purpose:Bill Wendling2002-10-231-0/+501
Feature Add Description: New files for the Flexible Parallel HDF5 stuff. H5FP.c - Module housing the APIs to FPHDF5 H5FPclient.c - Module housing the internal client APIs H5FPserver.c - Module housing the internal server APIs H5FPpublic.h - Header for public APIs H5FPprivate.h - Header for private APIs H5Ofphdf5.c - Way of serializing FPHDF5 information to and from the SAP H5Oplist.c - Way of serializing a generic property list. Solution: [details about the changes, algorithm, etc...] [Please as detail as you can since your own explanation is better than others guessing it from the code.] Platforms tested: Tested h5committest {arabica (fortran), eirene (fortran, C++) modi4 (parallel, fortran)}? [If no, why not?] Other platforms/configurations tested? Misc. update: Update MANIFEST if you add or remove any file. Update release_docs/RELEASE for bug fixes, new features, etc. Update applicable document files too.
generated by cgit v0.12 at 2024-11-14 05:21:53 (GMT)