| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Trim trailing whitespace, which is making 'diff'ing the two branches
difficult.
Solution:
Ran this script in each directory:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New feature
Description:
Initial checkin of B+ Tree code.
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committest
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Refactor metadata cache to merge "dirtied" flag in with other flags for
H5AC_unprotect and H5C_unprotect.
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Interim checkin of code changes moving management of the is_dirty flag
into the cache code.
Description:
Prior to this checkin, management of the is_dirty flag was handled
above the level of the metadata cache. This can no longer be allowed,
as it introduces a race condition in the proposed fix for a cache
coherency bug in PHDF5.
Solution:
Move management fo the is_dirty flag to the cache code proper.
Entries are now marked as dirty via a flag on the unprotect call.
Platforms tested:
h5committested
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Remove C99 types from new metadata cache related API calls
Description:
Windows (and perhaps others) don't like int32_t and int64_t. While
we have dealt with the issue internally, it is more of a problem
in API calls.
Solution:
Convert int32_t to int and int64_t to long int in the
H5AC_cache_config_t structure used by the new metadata cache
related API calls. Added explicit type casts to convert
between internal and external representations.
Platforms tested:
h5committested
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add API calls allowing user control of the metadata cache.
Description:
Prior to this update, the metadata cache was not configurable
from outside the library.
Solution:
Add API calls allowing the user to configure the metadata cache
either at file open time, or for any open file. Also added calls
permitting the user to monitor cache size and hit rate. These
latter facilities are needed for "manual" cache size control
Platforms tested:
h5committested
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
| |
New feature
Description:
Add first iteration of "segmented heap" code, which will be used to store
links in groups in a more flexible way than the previous "local heap" mechanism.
Platforms tested:
FreeBSD 4.11 (sleipnir) w/parallel
Solaris 2.9 (shanti)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New feature
Description:
Add new "block tracker" data structure to library, for tracking blocks of
bytes in a file. Block trackers will be used to keep track of the blocks
belonging to the soon-to-be-implemented "segmented heap" which is designed to
replace the current local & global heaps (starting with the local heap).
Block trackers will also keep track of the free space in the segmented heap
and someday could be used to track the free space in the entire HDF5 file.
They are implemented as a small header of information to cache the state
of the blocks (max & min sizes of blocks tracked, etc.) and the records of
the blocks themselves are stored in a v2 B-tree.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Solaris 2.9 (shanti)
h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New feature
Description:
Expand v2 B-tree code to support splitting the root node when enough
records are inserted and move metadata cache callbacks into their own source
file.
Platforms tested:
FreeBSD 4.11 (sleipnir) w/parallel
Too minor for h5committest
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Add names for new B-tree objects to metadata cache code.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor for h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Description: Removed PABLO from the source
Solution:
Platforms tested: arabica with 64-bit, copper with parallel,
heping with GNU C and C++ and PGI fortran (but
I disabled hl, there is some weird problem only
on heping: F9XMODFLAG is not
propagated to the Makefile files
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1) Provide facilities in cache to allow us to avoid a potential cache
consistency bug in the parallel case.
2) Clean up a off by one sanity checking bug.
3) Turn off execution of long running tests in debug mode.
Description:
1) In the parallel case, all writes to metadata must be collective,
but reads may not be. In pricipal, this allows us to different
contents in different caches. This isn't a problem as long as the
correct data is always on disk, but unless we can force certain
writes immediately, that need not be the case.
2) & 3) should need no further explanation.
Solution:
1) Add code allowing us to mark cache entries, and then force
these entries to be flushed at a later time.
Note that to actually avoid the bug, we will have to modify
existing code to use these new features.
2) & 3) should need no further explanation.
Platforms tested:
heping (serial debug and production)
committest (copper, sol, and heping). test failed on heping in the
c++ portion of the build, but at Quincey's siggestion, I am proceeding
with the checkin.
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix/Code Cleanup/Doc Cleanup/Optimization/Branch Sync :-)
Description:
Generally speaking, this is the "signed->unsigned" change to selections.
However, in the process of merging code back, things got stickier and stickier
until I ended up doing a big "sync the two branches up" operation. So... I
brought back all the "infrastructure" fixes from the development branch to the
release branch (which I think were actually making some improvement in
performance) as well as fixed several bugs which had been fixed in one branch,
but not the other.
I've also tagged the repository before making this checkin with the label
"before_signed_unsigned_changes".
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel & fphdf5
FreeBSD 4.10 (sleipnir) w/threadsafe
FreeBSD 4.10 (sleipnir) w/backward compatibility
Solaris 2.7 (arabica) w/"purify options"
Solaris 2.8 (sol) w/FORTRAN & C++
AIX 5.x (copper) w/parallel & FORTRAN
IRIX64 6.5 (modi4) w/FORTRAN
Linux 2.4 (heping) w/FORTRAN & C++
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Modify the cache code (H5C) to support automatic cache resizing to
adapt to the work load at run time.
Description:
Different applications require different sized caches to maintain
an acceptable hit rate. This set of changes attempts to provide the
ability to adjust to circumstances automatically.
Solution:
Added highly configurable code to allow the user to either set a
fixed cache size, or allow the cache to grow and shrink according to
conditions.
If enabled, cache size increases are triggered when the hit rate
drops below a user specified threshold in a user specified interval.
Cache size reductions (if enabled) are triggered when either the
hit rate exceeds some user specified threshold over a user specified
interval, when the cache contains "enough" entries that haven't been
accessed for a user specified interval, or some mix of the above.
See the header comments on the H5C_auto_size_ctl_t structure in
H5Cprivate.h for further details.
At present, the cache resize configuration options are not
accessible via the user API. Must add this.
Platforms tested:
h5committested, heping (serial), and copper (parallel)
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up minor warnings and align with release branch.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel & FPH5
Solaris 2.7 (arabica) w/production mode
Linux 2.4 (heping) w/C++ and FORTRAN
|
|
|
|
|
|
|
|
|
|
|
|
| |
Description: Restore 6 old error API functions back to the library to be backward
compatible with v1.6. They are H5Epush, H5Eprint, H5Ewalk, H5Eclear, H5Eset_auto,
H5Eget_auto. These functions do not have error stack as parameter.
Solution: Internally, these functions use default error stack.
Platforms tested: h5committest and fuss.
Misc. update: RELEASE.txt
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code optimization
Description:
Set up datatype ID for dataset's datatype on disk. This allows us to avoid
repeatedly copying the datatype when an ID is needed.
Also, clean up a few warnings in various other places.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Fix problems when compiling with C++ compiler.
Also clean up some warnings with gcc 3.4.x
Platforms tested:
FreeBSD 4.10 (sleipnir)
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
performance.
Description:
Replaced the old metadata cache with a cache with a modified LRU
replacement policy. This should improve the hit rate.
Solution:
Since we want to flush cache entries in increasing address order, I
used the threaded binary B-tree code to store the cache entries.
There is a fair bit of overhead here, so we may want to consider
other options.
While the code is designed to allow the support of other replacement
algorithms, at present, only a modified version of LRU is supported.
The modified LRU algorithm requires that a user selectable portion
of the cache entries be clean. The clean entries are evicted first
when writes are not permitted. If the pool of clean entries is used
up, the cache grows beyond its user specified maximum size. The
cache can also exceed its maximum size if the combined size of the
protected (or locked) entries exceeds the maximum size of the cache.
Platforms tested:
eirene (serial, parallel, fp), h5committested
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup & minor optimization
Description:
Re-work the way interface initialization routines are specified in the
library to avoid the overhead of checking for them in routines where there is
no interface initialization routine. This cleans up warnings with gcc 3.4,
reduces the library binary size a bit (about 2-3%) and should speedup the
library's execution slightly.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/gcc34
h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code optimization
Description:
Use default dataset transfer property list for internal metadata I/O
operations when parallel I/O is disabled. This avoids multiple copies of
the properties being made when the information is not needed.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir) w/parallel)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Checkpoint checkin of FP bug fixes. FP is still quite
buggy, but I must go deal with other matters.
Description:
Fixed two major bugs:
1) H5FPserver.c was clobbering meta data in its care.
2) H5FPserver.c was allocating the same space multiple
times, causing both data and meta data corruption.
Also made minor fixes, added debugging code, and familiarized
myself with the FP code.
All development work with FP enabled was done on Eirene.
On this platform, FP now passes its test reliably with
up to 9 processes. At 10 processes it seg faults every
time. I haven't looked into this issue.
There are also several known locking bugs which have to
be fixed. However, they are of sufficiently low probability
that I didn't bother with them on this pass.
FP has not been tested with deletions -- this should be
done.
Also, need to test FP chunked I/O.
Solution:
1) Modified cache in H5FPserver.c to merge changes correctly.
Found and fixed a bug in H5TB.c in passing.
2) Multiple space allocation was caused by a race condition
with set eoa requests.
Most of these eoa requests appeared to be superfluous, so
I deleted them.
Those issued during the superblock read seemed necessary,
so I inserted a barrier at the end of the superblock read,
to prevent races with allocations.
Platforms tested:
h5committested
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code optimization
Description:
Move handling for free list arrays that have no maximum size to separate
set of routines and optimize computations for free list arrays with maximum
size to avoid re-computing sizes all the time.
Platforms tested:
h5committest
Solaris 2.7 (arabica)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
When two property lists are compared, the H5Pequal routine was just
comparing the raw information for the property values. This causes problems
when the raw information contains pointers to other information.
Solution:
Allow a 'compare' callback to be registered for properties, so that a user
application get perform the comparison itself, allowing for "deep" compares of
the property value.
This was exported to the H5Pregister & H5Pinsert routines in the development
branch, but not the release branch.
Platforms tested:
FreeBSD 4.9 (sleipnir)
h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up compiler warnings, especially the 'FUNC' variable not used which
comes out in production mode.
Solution:
Had to add a new FUNC_ENTER_NOAPI_NOINIT_NOFUNC macro for those non-API
functions which don't need the 'FUNC' variable defined. (This will be _so_
much easier when C99 is standard on all our supposed platforms, since it has a
__FUNC__ macro... )
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor for h5committest (although there were lots of files changed, the
change was minor in each one)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Message Correction
Description:
Uncapitalized the beginning of the error messages.
Solution:
Platforms tested:
Linux (Very small change).
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up various warnings and parameter mis-matches, etc.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to need h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Removed Dead Code
Description:
Some of the FPHDF5 code was dead (I thought it'd be useful at one
point, but was wrong).
Solution:
Removed
Platforms tested:
Linux (FPHDF5 specific. No need for h5committest)
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
FPHDF5 Update
Description:
Added locking/unlocking to the AC_protect/AC_unprotect calls. The SAP
does all of the caching so nothing should be saved on the client
side. So, when the client unlocks the metadata, it can destroy it.
The metadata is always read from the SAP during a lock.
Platforms tested:
Linux (FPHDF5 specific change. No need for h5committest)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update
Description:
A lot of modifications for the FPHDF5 stuff:
H5AC.c
H5ACprivate.h - Removed AC_find (it's replaced with AC_protect
and AC_unprotect). Added flushing if it's an FPHDF5 driver and
we're doing an AC_set or AC_unprotect with the dirty flag set.
H5B.c - Split up the B_flush function into different functions
since the one function was doing serialization which is better
left as a separate entity.
H5D.c - Removed some FPHDF5 code that was incorrect
H5F.c - Split up the F_flush function so that it no longer
allocates file space. Created new functions (F_init_superblock,
F_read_superblock, and F_write_superblock) for greater modularity
and so that the FPHDF5 non-captain processes can read the
superblock after the captain process writes it.
H5FD.c - Error message correction.
H5FDfphdf5.c - Removed MPI barrier call that wasn't needed.
H5FPclient.c
H5FPserver.c - Modified so that if a process requests data that
isn't exactly aligned, we can return it if we have the block that
contains the requested address.
H5G.c
H5Gent.c
H5Gnode.c
H5HL.c
H5HLpkg.h
H5HLprivate.h
H5Oefl.c - Removed the H5HL_peek function since it was doing a
(now unsafe) holding of the information in the cache. Replaced
with protect and unprotect calls.
H5TB.c - Error fix. The TB_dless function wasn't working
properly.
H5Gstab.c - Format change.
H5err.txt
H5Edefin.h
H5Einit.h
H5Epubgen.h
H5Eterm.h - Added new error code.
Platforms tested:
Modi4 (paralle, Fortran)
Sol (Fortran)
Linux (C++, Fortran)
Copper (Parallel, Fortran)
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update
Description:
Added extra parameter to the H5AC_protect() function that indicates
if the cache being asked for is going to be written to or just read
from. Those AC_protect calls that were H5AC_find calls are now
read-only. The rest are writes.
Platforms tested:
Linux (h5committest not needed due to size and small impact)
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Removal of H5AC_find()
Description:
The H5AC_find() function is mostly redundant and with the new
Flexible Parallel HDF5 stuff, we need to do locking on metadata
returned from the H5AC_find() anyway. So, all of the locking stuff
will be placed in the H5AC_{un}protect() functions. The H5AC_find()
is no longer needed.
Solution:
Replaced all H5AC_finds with H5AC_protects and H5AC_unprotects.
Platforms tested:
Linux (Fortran & C++)
Solaris (Fortran)
Irix (Parallel & Fortran)
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup/bug fix
Description:
Move metadata cache calls around a bit so they protect the usage of the
metadata better.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too small to need h5committest
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Cleanup H5Pclose code and also switch a few internal users of H5Pclose
to use H5I_dec_ref instead.
Platforms tested:
FreeBSD 4.8 (sleipnir)
too small for h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
The metadata cache 'destroy' callback routines need the file handle in
order for certain callback routines (currently just the H5HG one) to perform
extra cleanups. The recent change to call the 'destroy' callback from the
'clear' callback omitted this parameter.
Solution:
Add the file handle to the metadata cache 'clear' callbacks.
Platforms tested:
FreeBSD 4.8 (sleipnir)
too small to need h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update
Description:
Added an extra flag to the clear functions that triggers a "destroy"
of the object being cleared if necessary. This is a fix for the
FPHDF5 stuff which had an object sticking around after it was
cleared. (In FPHDF5, some processes are in charge of destroying the
object, but all processes might actually allocate the object.)
Platforms tested:
AIX (Copper: Fortran & C++)
Linux (Verbena: Fortran & C++)
IRIX (Modi4: Parallel & Fortran)
(Sol is down)
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Changed lines like:
H5E_clear(H5E_get_my_stack());
to:
H5E_clear(NULL);
which performs the same operation.
Platforms tested:
h5committested
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Refactoring code
Description:
Refactored "IS_H5FD_MPIO || IS_H5FD_MPIPOSIX || IS_H5FD_FPHDF5" combination
of macros in many places into single IS_H5FD_MPI macro, which has the same
definition, but should be easier to maintain.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Further code cleanups suggested by lint tool.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Break some of the "debugging" routines into their own module, so they
aren't pulled into every executable, which certainly isn't going to use them.
Platforms tested:
h5committested
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Limit the scope on more function prototypes/macros/typedefs.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest not necessary.
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up warnings exposed by compiling on O2K. Also, revert some of Bill
and my changes to the H5S_mpi_opt_types_g, etc. and settle them back into their
original location.
Platforms tested:
h5committested.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup, bug fix & new feature.
Description:
Moved some "package-scope" information out of the private header file and
into the code module. (code cleanup).
Fixed a few places where the "destroy but don't flush" code wasn't working
correctly, which mostly affects the FPH5 effort. (bug fix).
Added the ability to "unprotect" a metadata object and delete it at the
same time. (new feature).
Platforms tested:
FreeBSD 4.8 (sleipnir) w/C++
Linux 2.4 (burrwhite) w/FORTRAN
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/parallel & FORTRAN
(h5committest not run due to my ongoing difficulties with C++ on burrwhite).
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Finish code cleanup
Description:
Wrap up the conversion of H5F_flush's multiple boolean flags into a single
bitfield of flags by pushing the flags down into the H5AC_flush and
H5F_istore_flush routines.
Also, changed the flags from H5_FLUSH_<foo> to H5F_FLUSH_<foo> to be more
consistent with rest of library.
And reverted the changes to H5FDflush and H5FD_flush routines.
Platforms tested:
FreeBSD 4.7 (sleipnir)
Solaris 5.8 (sol)
IRIX64 6.5 (modi4) w/parallel
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update
Description:
Final support addition for FPHDF5. Wherever there was an equivalent
MPIO or MPIO/POSIX call/check, I placed an FPHDF5 call/check there as
well, with the appropriate #ifdefs in place.
The hdf5.h is updated with the H5FDfphdf5.h driver #include.
Platforms tested:
Linux & Modi4
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Update dependencies and clean up a few warnings.
Platforms tested:
Linux 2.2 (eirene) w/parallel
|
|
|
|
|
|
|
|
|
|
| |
Bug fix.
Description:
Correct compile errors when configured with --enable-debug=all.
Platforms tested:
FreeBSD 4.7 (sleipnir) serial & parallel
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
Metadata cache in parallel I/O can cause hangs in applications which
perform independent I/O on chunked datasets, because the metadata cache
can attempt to flush out dirty metadata from only a single process, instead
of collectively from all processes.
Solution:
Pass a dataset transfer property list down from every API function which
could possibly trigger metadata I/O.
Then, split the metadata cache into two sets of entries to allow dirty
metadata to be set aside when a hash table collision occurs during
independent I/O.
Platforms tested:
Tested h5committest {arabica (fortran), eirene (fortran, C++)
modi4 (parallel, fortran)}
FreeBSD 4.7 (sleipnir) serial & parallel
Misc. update:
Updated release_docs/RELEASE
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup/new feature.
Description:
Split FUNC_LEAVE into API and non-API specific versions. This allows a
solution to compiling this branch with C++, as well as reducing the size
of the binaries produced.
Platforms tested:
FreeBSD 4.7 (sleipnir) w/serial, parallel (including MPE) & thread-safe
|