| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup & minor optimization
Description:
Re-work the way interface initialization routines are specified in the
library to avoid the overhead of checking for them in routines where there is
no interface initialization routine. This cleans up warnings with gcc 3.4,
reduces the library binary size a bit (about 2-3%) and should speedup the
library's execution slightly.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/gcc34
h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Checkpoint checkin of FP bug fixes. FP is still quite
buggy, but I must go deal with other matters.
Description:
Fixed two major bugs:
1) H5FPserver.c was clobbering meta data in its care.
2) H5FPserver.c was allocating the same space multiple
times, causing both data and meta data corruption.
Also made minor fixes, added debugging code, and familiarized
myself with the FP code.
All development work with FP enabled was done on Eirene.
On this platform, FP now passes its test reliably with
up to 9 processes. At 10 processes it seg faults every
time. I haven't looked into this issue.
There are also several known locking bugs which have to
be fixed. However, they are of sufficiently low probability
that I didn't bother with them on this pass.
FP has not been tested with deletions -- this should be
done.
Also, need to test FP chunked I/O.
Solution:
1) Modified cache in H5FPserver.c to merge changes correctly.
Found and fixed a bug in H5TB.c in passing.
2) Multiple space allocation was caused by a race condition
with set eoa requests.
Most of these eoa requests appeared to be superfluous, so
I deleted them.
Those issued during the superblock read seemed necessary,
so I inserted a barrier at the end of the superblock read,
to prevent races with allocations.
Platforms tested:
h5committested
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix/optimization
Description:
Address slowdown in MPI-I/O file metadata operations that was introduced
mid-stream. We now _require_ a POSIX compliant parallel file system for the
MPI-I/O file driver (as well as for the MPI-POSIX file driver).
Also optimized file open operation when the file is being created by
reducing the number of collective & syncronizing calls.
Additionally, refactor the MPI routines into a common place, eliminating
duplicated code.
Platforms tested:
FreeBSD 4.9 (sleipnir) w/parallel
h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up compiler warnings, especially the 'FUNC' variable not used which
comes out in production mode.
Solution:
Had to add a new FUNC_ENTER_NOAPI_NOINIT_NOFUNC macro for those non-API
functions which don't need the 'FUNC' variable defined. (This will be _so_
much easier when C99 is standard on all our supposed platforms, since it has a
__FUNC__ macro... )
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor for h5committest (although there were lots of files changed, the
change was minor in each one)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
The SAP was sending back replies to the client but the client wasn't
picking them up (this was after a dump from the server.
Solution:
Read the extra replies from the server.
Platforms tested:
AIX (w/ FPHDF5)
Linux (w/ FPHDF5)
Solaris (w/ Fortran & C++)
Misc. update:
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix and File Addition
Description:
- Fixed when reading from the file with FPHDF5. It wasn't recording
how many bytes it read.
- Added Arithmetic Transformation modules. These haven't been
included into the HDF5 build. I just added them here for future
porposes...
Solution:
Added a call to "MPI_Get_count" to get the number of bytes read.
Platforms tested:
Linux (w/ FPHDF5)
AIX (w/ and w/o FPHDF5)
FPHDF5-specific fixes...No need for H5committest
Misc. update:
|
| |
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Add H5MMprivate.h header, to fix builds.
Platforms tested:
Eyeballed it, too minor to require testing...
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix & code cleanup
Description:
Allowing the library to call malloc with a size of 0 bytes causes problems
for some users, so we check for allocations of 0 bytes and disallow them now.
Cleaned up some code which could call malloc with 0 size.
Changed some code calling HDmalloc directly to call H5MM_malloc(), which
allows us to check for 0 sized allocations.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
The End of Address information needed to be kept by the SAP. Some
processes were trying to get the EOA information in collective mode,
but the action wasn't collective at the time.
Solution:
Keep the EOA information for the file on the SAP. Clients query the
SAP to get/set it when needed.
Platforms tested:
Linux (w/ FPHDF5)
Copper (w/o FPHDF5)
(FPHDF5 specific, so no need for full testing)
Misc. update:
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
The owners and reference counts of locks weren't being handled
correctly.
Solution:
Allocate an array the size of the COMM FPHDF5 is invoked with for
each lock. Then reference count within this array. Keep an extra
"counter" to see how many different processes have this particular
lock. Free the lock when that counter reaches zero.
Platforms tested:
Linux & Copper (FPHDF5 specific).
Misc. update:
|
| |
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up various warnings and parameter mis-matches, etc.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to need h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
FPHDF5 Update
Description:
Added locking/unlocking to the AC_protect/AC_unprotect calls. The SAP
does all of the caching so nothing should be saved on the client
side. So, when the client unlocks the metadata, it can destroy it.
The metadata is always read from the SAP during a lock.
Platforms tested:
Linux (FPHDF5 specific change. No need for h5committest)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update
Description:
A lot of modifications for the FPHDF5 stuff:
H5AC.c
H5ACprivate.h - Removed AC_find (it's replaced with AC_protect
and AC_unprotect). Added flushing if it's an FPHDF5 driver and
we're doing an AC_set or AC_unprotect with the dirty flag set.
H5B.c - Split up the B_flush function into different functions
since the one function was doing serialization which is better
left as a separate entity.
H5D.c - Removed some FPHDF5 code that was incorrect
H5F.c - Split up the F_flush function so that it no longer
allocates file space. Created new functions (F_init_superblock,
F_read_superblock, and F_write_superblock) for greater modularity
and so that the FPHDF5 non-captain processes can read the
superblock after the captain process writes it.
H5FD.c - Error message correction.
H5FDfphdf5.c - Removed MPI barrier call that wasn't needed.
H5FPclient.c
H5FPserver.c - Modified so that if a process requests data that
isn't exactly aligned, we can return it if we have the block that
contains the requested address.
H5G.c
H5Gent.c
H5Gnode.c
H5HL.c
H5HLpkg.h
H5HLprivate.h
H5Oefl.c - Removed the H5HL_peek function since it was doing a
(now unsafe) holding of the information in the cache. Replaced
with protect and unprotect calls.
H5TB.c - Error fix. The TB_dless function wasn't working
properly.
H5Gstab.c - Format change.
H5err.txt
H5Edefin.h
H5Einit.h
H5Epubgen.h
H5Eterm.h - Added new error code.
Platforms tested:
Modi4 (paralle, Fortran)
Sol (Fortran)
Linux (C++, Fortran)
Copper (Parallel, Fortran)
Misc. update:
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
With the new datatypes for OIDs, I didn't write the comparison
function correctly.
Solution:
Changed it from:
return oid1 == oid2
to
return oid1 - oid2
Platforms tested:
Linux (FPHDF5 specific)
Misc. update:
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update
Description:
Added an extra flag to the clear functions that triggers a "destroy"
of the object being cleared if necessary. This is a fix for the
FPHDF5 stuff which had an object sticking around after it was
cleared. (In FPHDF5, some processes are in charge of destroying the
object, but all processes might actually allocate the object.)
Platforms tested:
AIX (Copper: Fortran & C++)
Linux (Verbena: Fortran & C++)
IRIX (Modi4: Parallel & Fortran)
(Sol is down)
Misc. update:
|
| |
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Use HDONE_ERROR instead of HCOMMON_ERROR
Platforms tested:
h5committested
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix...
Description:
The previous bug was in the _free function as well
Solution:
Used the correct structure
Platforms tested:
Modi4
Misc. update:
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
Was using the wrong pointer into a structure...
Solution:
Used the correct pointer to get the file_id...
Platforms tested:
Modi4
Misc. update:
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update & Bug Fix
Description:
The "free" protocol was missing. Added that to the server side.
When doing a "create" of a file (with no other data structures
created), the freespace in the file wasn't being reclaimed.
Solution:
After adding the free protocol, we put the burden of running through
the FD_free function on the SAP instead of each client.
Platforms tested:
Linux
Misc. update:
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix & Update
Description:
FPHDF5 was creating files which didn't have the EOA field in the
superblock set correctly. It turns out that the SAP was keeping this
information to itself instead of giving it to the client processes.
Naughty SAP!
Solution:
Have the SAP send this information back to the clients so that they
can update the superblock as necessary. This now creates a file (with
just the root group) that looks correct! Only problem is that there's
extra file space being allocated. Also, at program termination,
there's an infinite loop...
Platforms tested:
H5committests (run by hand on burrwhite, arabica, and modi4)
Misc. update:
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix and Update
Description:
From Quincey's comments on the code I checked in last night:
- In H5F_close call, the "private" processes should call the
H5F_flush with the "CLEAR_ONLY" flag.
- There's no need for a special case for FPHDF5 in the
FD_real_alloc function since FPHDF5 doesn't define an alloc
function.
- The return type of H5Pset_fapl_fphdf5 should be herr_t instead
of hid_t. I don't know how it got that way in the first place.
- The variable names for MPI types and the structure typedefs
should be switched: H5FP_request/H5FP_request_t to
H5FP_request_t/H5FP_request and so on.
- In the H5FP.c module, I was commiting the H5FP_request MPI
datatype but using the wrong offset field...
Platforms tested:
Linux...will test on others, but these are mostly FPHDF5 changes.
Misc. update:
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Why not?
Description:
Lots of changes. This is basically a "commit because there are a lot
of changes" commit.
The server now acts more like a metadata caching server. The dumping
mechanism finally works (I think). At least in the test I did it
seemed to write things back. There's a new MPI TAG to use
(H5FP_TAG_DUMP) when dumping things. Changed a lot of the functions
from passing a structure around to just passing a pointer to that
structure. I have no clue why I did it the previous way...
Um...And more synchronization stuff between the client and server.
The protocol between the two was being broke in a couple of places.
We're getting closer!
Platforms tested:
Linux (probably Modi4 too)
Misc. update:
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update
Description:
Big change to the H5FD.c module:
- Split apart H5FD_alloc and H5FD_free. H5FD_alloc was huge and
H5FD_free had a freeing of the freelist part which I needed to
call from the SAP.
- Added support for FPHDF5. If it's a client, then it sends the
allocation or free request to the SAP. The SAP will call the
same code, but it'll actually do the allocation/freeing in that
case.
Platforms tested:
Linux & Modi4
|
| |
|
|
|
|
|
|
|
| |
Update
Description:
Added support for the allocation and freeing of space in the file.
This information is kept on the Server.
Platforms tested:
Linux & Modi4
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update
Description:
Added a "metadata flush" function which allows the clients to force a
dump of the metadata from the SAP. This should be done before closing
the file.
Modified the information stored on an "open" of the file. The
filename is unnecessary, so I got rid of it.
More integration with the File Driver code: Passing in a data xfer
property list.
Platforms tested:
Linux
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update
Description:
Removed some obsolete fields - such as the AC_subst_t object ID -
which isn't needed. Fixed so that there are fewer global variables.
There are still some left, however. Modified the client and server
code so that it handles read requests better. There were some flaws
in how this was done before (it was calling H5FD_read() instead of
just returning the status that it couldn't find the metadata in the
cache).
Platforms tested:
Linux
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update
Description:
H5FP.c, H5FPclient.c, H5FPprivate.h, H5FPserver.c:
Update. More progression towards the SAP as metadata cache. It
only lacks the ability to take care of metadata allocations.
H5FDfphdf5.[ch]:
Start of a new driver for FPHDF5. Not fully implemented just
yet...
Platforms tested:
Linux
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update
Description:
Changes:
- Added support for the server dumping metadata writes to a
client.
- Some of the code wasn't handling allocated buffers correctly
(freeing twice).
- Modified server so that it handles metadata only.
Platforms tested:
Linux
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update
Description:
This is the first conversion of the FPHDF5 code to be a metadata
cache. There's an extra error message. I rewrote the sync/change code
to be read metadata/write metadata instead.
I still need to hook these changes into the HDF5 code so that it
looks at the SAP first before checking the file for metadata.
Platforms tested:
Linux
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup/new feature.
Description:
Split FUNC_LEAVE into API and non-API specific versions. This allows a
solution to compiling this branch with C++, as well as reducing the size
of the binaries produced.
Platforms tested:
FreeBSD 4.7 (sleipnir) w/serial, parallel (including MPE) & thread-safe
|
| |
|
|
|
|
|
|
|
|
| |
Update
Description:
A few generic changes to the FPHDF5 code. Some error messages
reworked a bit. Cleaning up in case of failure improved in some
cases. Added another field to the synchronization messages...
Platforms tested:
Linux
|
| |
|
|
|
|
|
|
|
|
|
| |
FPHDF5 Fixups
Description:
- Stopped using API calls in the H5FPclient code. It now uses
internal library calls.
- Removed some FIXME comments because they've been fixed.
- Small fix for an enum starting at 37 for no reason :-)
Platforms tested:
Eirene (only affects FPHDF5, so no need to test 3 platforms just yet).
|
| |
|
|
|
|
|
|
|
|
|
| |
Update
Description:
Added code which performs an update when there's a dataset creation.
Commented on some FIXMEs which were in the code.
Platforms tested:
Eirene PP
Arabica Fortran
Modi4 PP Fortran
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New Feature. Support for new FPHDF5 Feature
Description:
More support for the FPHDF5 feature. This splits the H5D_create
function into separate parts which update the metadata cache. It was
necessary to split apart the H5O_create function to do a similar
thing since it would allocate real space on the hard disk.
I'm checking this up now so that I don't get too far away from what
the CVS repository has and so that others may look at the code and
retch^H^H^H^H^Hmarvel at it. If there are any comments, send them my
way.
There is one outstanding issue, though. I need to use non-API
functions in the FPHDF5 stuff. I'm using some APIs right now and
should migrate to using others. At the moment, I'm putting that on
the back burner until I'm more along in the implementation.
Platforms tested:
Arabica (C++)
Eirene (Parallel)
Modi4 (Parallel)
|
| |
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Add more comments and clean up small bits of the FPH5 code.
Platforms tested:
FreeBSD 4.7 (sleipnir), changes too minor to affect other platforms.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Oops
Description:
The H5FPprivate.h file was being #included even if PARALLEL was
turned off.
Solution:
put the #include of the H5FPprivate.h file within the #ifdef block so
that it doesn't get included if H5_HAVE_FPHDF5 isn't defined.
Platforms tested:
Linux...
|
|
|
Feature Add
Description:
New files for the Flexible Parallel HDF5 stuff.
H5FP.c - Module housing the APIs to FPHDF5
H5FPclient.c - Module housing the internal client APIs
H5FPserver.c - Module housing the internal server APIs
H5FPpublic.h - Header for public APIs
H5FPprivate.h - Header for private APIs
H5Ofphdf5.c - Way of serializing FPHDF5 information to and from the
SAP
H5Oplist.c - Way of serializing a generic property list.
Solution:
[details about the changes, algorithm, etc...]
[Please as detail as you can since your own explanation is
better than others guessing it from the code.]
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}?
[If no, why not?]
Other platforms/configurations tested?
Misc. update:
Update MANIFEST if you add or remove any file.
Update release_docs/RELEASE for bug fixes, new features, etc.
Update applicable document files too.
|
