| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
| |
Finish renaming cache-related files & symbols on metadata journaling
branch. Metadata cache-related symbols/files with no suffix now have a '1'
suffix and symbols/files with a '2' suffix now have no suffix.
Tested on:
Mac OS X/32 10.6.2 (amazon) w/debug, prod & parallel
(h5committest not required on this branch)
|
| |
|
|
|
|
|
| |
Remove trailing whitespace from source code files.
Tested on:
None - just eyeballed
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Code cleanups for better alignment with trunk, along with style issues,
etc.
Also, get journal test files to work correctly when using a srcdir
build.
Tested on:
Mac OS X/32 10.6.2 (amazon) w/debug, production & parallel
(h5committest not required on this branch)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Review and get code to conform to standard library coding style, also
add 'const' keyword to "set" routines.
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.5.2 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Use metadata journaling callback to allow dataset code to track journal
status changes and flush cached info appropriately.
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.5.2 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Convert local heap cache client to use metadata journaling cache.
Other minor cleanups & simplifications, etc.
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.5.2 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
revised cache.
Note that this conversion is not as efficient as it should be. Specifically,
it does it more memcpy's between the metadata cache's on disk image of the
direct block and the fractal heap's on disk image of the direct block than
is absolutely essential. Eventually, we will want to fix this -- probably
by allowing the metadata cache and the fractal heap direct block to share
a common on disk image of the direct block. However, this will require
extensions to the client / metadata cache interface, and some reworking of
the fractal heap as well.
This checkin also includes Mike M's fix to the Linew specific bug mentioned
in my checkin of 22 Aug 2008.
Tested on Phoenix (serial debug and production),
Kagiso (parallel), and
Linew (serial)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
1) Removed references to H5AC2__CURR_JNL_CONFIG_VER from H5Fget_jnl_config()
and H5Pget_jnl_config(), and also references to
H5AC__CURR_CACHE_CONFIG_VERSION from H5Fget_mdc_config() and
H5Pget_mdc_config().
2) Removed H5Pset_journal() from H5C2journal.c, and modified test
code to use H5F/Pget/set_mdj_config() instead.
3) Implemented support for callbacks on metadata journaling status change
along with the associated registration / deregistration calls and
associated test code.
4) Fixed bug in journaling shutdown exposed by 3 above.
Tested and passed on Phoenix (serial), Linew (serial), and Kagiso (parallel).
However, while I was testing there were a couple of checkins, forcing
an update and second round of testing.
On the second round, tested and passed on Phoenix (serial) and
Kagiso (parallel), but failed on Linew (serial).
As best I can tell, this was caused by Mike M's checkin -- which
broke the smoke checks in cache2_journal on Linew but not Phoenix
or Kagiso. A typical delta in the architype files follows:
linew.hdfgroup.uiuc.edu% diff -ctw cache2_journal_sc00_000.jnl tmp/cache2_journal_sc00_000.jnl
*** cache2_journal_sc00_000.jnl Fri Aug 22 08:28:49 2008
--- tmp/cache2_journal_sc00_000.jnl Fri Aug 22 05:08:41 2008
***************
*** 1,5 ****
! 0 ver_num 1 target_file_name cache_journal_test.h5 creation_date Fri Aug 22 human_readable 1
! E eoa_value 0x0
C comment Begin transaction on transaction 1.0.
1 bgn_trans 1
2 trans_num 1 length 1 base_addr 0x401 body 01
--- 1,5 ----
! 0 ver_num 1 target_file_name cache_journal_test.h5 creation_date Wed Aug 20 human_readable 1
! E eoa_value 0x772a9c01
C comment Begin transaction on transaction 1.0.
1 bgn_trans 1
2 trans_num 1 length 1 base_addr 0x401 body 01
As you can see, it looks like garbage is getting into the first
eoa write on Linew.
I'm checking in anyway, as Quincey needs my changes, and I will not
have time to work on this for several days.
Mike: Let me know if you are tackling this one -- if not, I'll deal with it.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
from the H5AC2_cache_config_t structure and the associated
H5P/H5Fget/set_mdc_config() API calls, and into the newly created
H5AC2_jnl_config_t structure and H5P/H5Fget/set_jnl_config() calls.
Updated test code accordingly.
Updated the trace file test code for journaling.
Also folded in a fix to an assertion bug in H5C2pkh.h
Tested serial on Phoenix and Linew, and parallel (with and without the
trace file enabled) on kagiso.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Convert object header cache client to use the new metadata journaling
cache, which included adding a new client for handling continuation chunks.
Added "real" protect calls around modifying chunks in object headers.
Switched a few more metadata cache library API routines to drop the
file pointer, when it is not needed (pinning/unpinning entries, etc.)
Fixed bug in journaling cache handling of 'image_len' callbacks and
also changed cache to retry deserializing entries when the entry's size is
larger than the speculative size initially tried.
Retrying for 'image_len' callbacks has problems with the 'multi' VFD,
so the h5dump and FORTRAN 'multi' tests are commented out, until the changes to
the 'multi' VFD from the file free space branch are brought back into the
trunk.
Currently, the 'h5recover' tool has a bug which requires it to be run
twice before replaying the journal "sticks". However, this is from an earlier
checkin, since the code in the branch already has this behavior... :-(
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.5.4 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
specifically:
1) Fix for failure to detect journaling in progress on HDF5 files which
were not closed correctly. Also associated test code.
Note that this required addition of code to test for journaling in
progress and enable journaling at the end of H5F_flush(). In passing,
I was able to get rid of the wacky code that queued journaling setup
at cache creation time.
2) Test code for startup and shutdown of journaling on an open file.
3) Updates to start checking journal output against architype files
instead of just generating architypes at test time.
Note that per Quincey's request, I have checked in gziped versions
of the architype files. At some point, we will have to add code
to automatically unzip these files, but for the time being you will
have to go to test/testfiles and "gunzip *.gz". The journal tests
will still pass if you don't, but you will get a warning about
missing test files.
4) Fixed bug in journal entry logging code that allowed a comment to
appear in the journal file before the journal file header. (Mike M,:
Please review my fix to verify that I haven't clobbered anything.)
5) Additional test code.
Note that more test code would be a good idea, but this set of bug
fixes should be enough to get us through the basic demo -- at least
as far as the metadata cache is concerned.
Tested serial on Phoenix, and parallel on Kagiso. Also, tested serial
on Linew just prior to some last minute minor edits.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Added a pointer to the cache that an entry is contained within to the
cache entry structure. This allows us to remove the file pointer from some of
the H5AC2 calls, easing the conversion of some of the cache clients (the free
space section info and fractal heap direct blocks, and probably others).
Removed file pointer from the H5AC2_unpin_entry() call.
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.5.2 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Convert the symbol table node metadata cache client to use the new
journaling cache callbacks.
Also added a 'H5F_t *' parameter to the 'serialize' callback for the
journaling cache, which makes the client's job much easier.
Various minor coding cleanups, etc. also.
Tested on:
FreeBSD/32 6.2 (duty) in debug mode
FreeBSD/64 6.2 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/default API=1.6.x, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Mac OS X/32 10.5.3 (amazon) in debug mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
While this code doesn't break any of the existing tests, it HAS NOT
been tested beyond that.
Also mods needed to integrate the journaling code with Quincey's latest
mods, and to adapt existing test code to slight changes caused by the
addition of journaling.
Finally, fixed an undefined variable bug in the HL code exposed by the
journaling mods.
Tested serial under Linux (Phoenix) and parallel under Linux (Kagiso).
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Switch v1 B-tree nodes from using previous cache to use the new journaling
cache code. This is a major switch for the cache callbacks.
Switched the metadata caching code from having a pointer to the file it's
in to receiving a pointer to the file, when needed. This was necessary in
order to avoid crashing when two files IDs were open on the same underlying
file and one of those files was closed with cache entries using the file
pointers. Also took out the check in the caching code for reading off the
end of the file, which didn't play nicely with the multi-file VFD.
Switching the cache from having a pointer internally to requiring one
externally meant tweaking almost all the test code. :-/
Added correct exit codes to cache2 tests that didn't have them already,
so the 'make check' will stop when they fail.
Use the path to the h5diff in this build in the tools/h5recover testing
script, since we can't guarantee a user has HDF5 already installed.
Assorted minor tweaks to get everything to compile more cleanly and pass
all the tests (on my Mac :-).
Tested on:
Mac OS X (10.5.2) w/C++
(more testing coming up shortly, I just didn't have my "rsync testbed" set
up for this branch when I started making changes to the code)
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
note that both the H5C and H5C2 code have been updated.
Also checked in code to track journaling status in the super block.
Note that this code has not been tested -- but as best I can tell,
it does not break the existing regression tests.
Tested serial (debug and production) on Phoenix. Also tested parallel
on kagiso.
Note that regression test fails on kagiso (but not on phoenix) if
the cache2 serial tests are configured to use the core file driver.
Thus this code is check in with the core file driver optimization
of the cache2 tests disabled. To turn it on, set the USE_CORE_DRIVER
#define to TRUE.
|
|
|
branch -- this commit needed at I forgot to svn add the new files
created in support of metadata journaling.
Again, this version may not compile.
|
