| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
| |
h5committested.
|
|
|
|
| |
h5committested.
|
|
|
|
| |
h5committested.
|
|
|
|
| |
h5committested.
|
|
|
|
| |
h5committested.
|
|
|
|
| |
h5committested.
|
| |
|
|
|
|
|
|
|
|
|
| |
ended the program.
Solution: added exit(0) after usage(). This is not a good fix but will work
for now.
Test: jam.
|
|
|
|
|
|
|
|
| |
The script execute the program by just $program which will not work if the
user does not have "." in the $PATH.
Changed it to "./$program" to avoid this problem.
Tested: koala.
|
|
|
|
|
|
| |
1) H5O_load() in H5Ocache.c: when reading a block that is > spec_read siez, read the whole block in again and possibly decode the header.
2) H5F_accum_write() in H5Faccum.c: for a large write that is >= H5F_ACCUM_MAX_SIZE, flush the metadata in the accumulator first before the write.
Tests are added to test/ohdr.c and test/accum.c. h5committested.
|
| |
|
| |
|
|
|
|
|
|
|
|
| |
Replace H5Fflush in writer with H5Dflush and file close/reopen with
H5Drefresh. Use cases seem to run faster. Keeping the old code so that
later on, I may test the performance between the two.
Tested: koala.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Implemented use case 1.9: Appending n-1 dimensional planes
Adding the option "-y" which allows chunks to be thicker
(more than 1 plane). This supports use case 1.9 which have chunks
of multiple planes but writing is still appending one plane at a time.
Using -y with "use_append_chunk", would provide the use case
of writing a plane to a partial chunk.
Using -y with "use_append_mchunks", would provide the use case
of writing a plane to multiple partial chunks.
Also added the option -n which specifies how many planes to write
instead of the defaults of chunksize number of planes. This allows
the final dataset to be other than a cube.
test/use_append_mchunks.c:
test/use_common.c:
test/use.h:
test/use_append_chunk.c:
I also overhauled code in use_append_mchunks and use_append_chunk:
1. creating a set_parameter() to customize individual use case;
2. combine the create_uc_file(), read_uc_file() and write_uc_file()
of both use cases appending one plane per chunks and one plane per
multiple chunks. Moved these combined create/read/write functions
into use_common.c to make future modification and maintence easier.
test/test_usecases.sh.in:
Added a simple test to demonstarte how to use -y to run use case 1.9.
Tested: h5committest, except cmake, passed. Hand tested in Koala for various options.
|
|
|
|
| |
read-write operation on the system.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Implemented use case 1.8 program--SWRM read/write multiple chunks at a time.
test/use_append_mchunks.c:
The program.
test/Makefile.am:
test/Makefile.in:
Added program use_append_mchunks.
test/test_usecases.sh.in:
Added simple tests for use_append_mchunks.
MANIFEST:
Updated for new file.
Tested: h5committest, except cmake, passed. Hand tested in Koala for various options.
|
|
|
|
| |
Tested: h5committested.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When zlib is not available (use "configure --without-zlib"), some tests
failed to not using compression features.
Solution:
test/dsets.c:
Conditional blocked out use of zlib compression code.
tools/h5ls/testh5ls.sh.in:
Temporary commented out test code that uses datafile that has zlib
compressed data.
Tested: h5committest (cmake does not work but that is ignored for now.) Also,
hand tested in koala using "configure --without-zlib".
|
| |
|
| |
|
|
|
|
| |
Tested: koala.
|
|
|
|
| |
Tested: h5committest.
|
|
|
|
|
|
|
|
|
|
| |
usecase1_7.c -> use_append_chunk;
usecase_common.c -> use_common.c;
usecase.h -> use.h.
MANIFEST updated with new names.
Tested: h5committest.
|
|
|
|
|
|
|
|
|
|
|
| |
1. "-f filename" can create the test file somewhere other than
the current directory or <progname>.h5. This allows running tests in different
filesystems, for example.
2. "-l w|r" can launch only the writer or the reader (default does both).
This allows launching writer (includes the "create file") in one process and
launching the reader somewhere else. (The "-f" helps too.)
Tested: h5committested.
|
|
|
|
| |
Tested: h5committest.
|
|
|
|
| |
h5committtested.
|
|
|
|
| |
h5committested.
|
|
|
|
| |
h5committested.
|
|
|
|
| |
h5committested.
|
|
|
|
| |
h5committested.
|
|
|
|
| |
h5committested.
|
|
|
|
| |
h5committested.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Descriptions:
When removing object header messages, it is possible for object header
continuation messages to move to a different chunk. When this happens, flush
dependencies need to be updated to reflect the new structure. This change adds
code to update the flush dependencies, and a test for this.
Also fixed a bug where the flush dependency no the object header proxy was not
being destroyed when an object header chunk was deleted.
Tested: ummon
|
|
|
|
|
|
|
|
|
|
|
| |
Description:
Added flush dependencies between object header chunks. Also added a new object
header proxy object, which is a child dependency of all object header chunks.
All objects which are child dependencies of an object header should set up a
flush dependency on the object header proxy, so the dependency applies to all
object header chunks.
Tested: ummon; jam (fails ph5diff -v, unrelated), koala, ostrich (h5committest)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
multiple parents
Description:
Reworked how flush dependencies worked internally, allowing multiple flush
dependency parents, and removing the notion of flush dependency heights, instead
keeping track of the number of dirty descendents to determine if parents can be
flushed. Also removed the requirement that cache clients destroy flush
dependencies before eviction (this is now handled by the cache) and removed the
maximum number of passes (the cache should detect infinite loops elsewhere).
Added test cases for this.
Tested: durandal
|
|
|
|
|
|
|
|
|
|
|
| |
Moved the file open/close back inside the loop in the SWMR reader
code. Without the open/close, the cache never refreshes so we
always see the initial amount.
Made a couple of other minor changes to the SWMR code.
Tested on:
jam
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Added comments to files, functions, etc.
- Some readers changed to keep the file open throughout the
life of the program instead of closing it before sleeping.
This is in the hopes that there will be more opportunities for
cache issues without the close.
- Other minor changes.
Tested on:
jam
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Updates the SWMR testing
- All readers/writers now consume and emit their random seeds
for debugging.
- Output uses unbuffered I/O for more readable output.
- The random seeds are now based on a more fine-grained time
measurement to ensure each reader has a different random
seed.
Tested on:
jam
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add flush dependencies to the fixed array code.
Description:
Added basic flush dependency wiring to the fixed array code. Does not
include the test code, which will be added soon.
NOTE: The Makefile.in change in test/ is not due to any changes made
here. It appears to be from a missing bin/reconfigure in a prior
checkin.
Tested:
jam (there is an existing flush/refresh test error)
|
|
|
|
|
|
|
|
|
|
| |
Description:
Patched H5B2 shadowed list implementation to remove nodes from the shadowed list
when they are evicted, and added some missing initializations. Also removed
inadvertent temporary changes to the tests that disabled srandom, added
assertions and fixed formatting.
Tested: ummon
|
|
|
|
|
|
|
|
| |
Merge r22634:2266 from trunk to revise_chunks branch
Tested on:
Mac OSX/64 10.7.4 (amazon) w/debug, gcc-4.7.x, C++, FORTRAN and threadsafe
(h5committest not needed on this branch)
|
|
|
|
|
|
|
|
| |
Bring r22599:22634 from trunk to revise_chunks branch
Tested on:
Mac OSX/64 10.7.4 (amazon) w/gcc 4.7.x, C++ & FORTRAN
(No need for h5committest yet on this branch)
|
|
|
|
|
|
|
|
| |
Bring r22251:22599 from trunk to revise_chunks branch.
Tested on:
Mac OSX/64 10.7.4 (amazon) w/gcc 4.7.1, C++ & FORTRAN
(Not h5committested yet, as this branch doesn't require it)
|
|
|
|
|
| |
Tested on jam. This branch still has h5diff errors. The library tests
all pass, though.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Description:
Adds SWMR support to the v2 b-trees when used as a chunk index. Because each
node keeps track of the total number of records reachable through each of its
children, nodes must be shadowed every time a descendent record is inserted or
removed. This implementation prevents this from happening, however, if the node
has already been shadowed since the last time the v2 b-tree header was flushed.
Also modified SWMR test to include v2 b-trees.
Tested: jam, koala, ostrich (h5committest), durandal
Note: There is a preexisting failure in the ph5diff (-v) test
|
|
|
|
|
|
|
|
| |
working on v2 b-trees,
but not related to v2 b-trees.
Tested: durandal
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r22085:22251 from trunk to revise_chunks branch.
Also tackle some testing issues in test/objcopy.c test and clean up
some warnings.
Tested on:
FreeBSD/32 8.2 (loyalty) w/gcc4.6, w/C++ & FORTRAN, in debug mode
FreeBSD/64 8.2 (freedom) w/gcc4.6, w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/default API=1.8.x,
w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-amd64 2.6 (koala) w/Intel compilers, w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, w/threadsafe, in production mode
Linux/PPC 2.6 (ostrich) w/C++ & FORTRAN, w/threadsafe, in debug mode
Linux/64-ia64 2.6 (ember) w/Intel compilers, w/paralle, C++ & FORTRAN,
in production mode
Mac OS X/32 10.7.3 (amazon) in debug mode
Mac OS X/32 10.7.3 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
Mac OS X/32 10.7.3 (amazon) w/parallel, in debug mode
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Description:
In H5Dearray.c, functions would "swizzle" the chunk offset and "down" number of
chunks in order to more the unlimited dimension to be the first dimension, but
they would not swizzle the chunk dimensions. This could cause two chunks to
have the same index, causing problems. Modified code to swizzle the chunk
dimensions, and added a test.
Note: There is still a problem with h5watch that appears to be unrelated.
Tested: durandal
|
|
|
|
|
|
|
|
|
|
| |
Bring r20557:22085 from trunk to this branch, also fixing some other
issues/failures in the branch simultaneously. The h5repack tests are still
failing, but Neil will be checking into those, so the branch can be fully
functional again.
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Description:
Adds SWMR capability to v1 b-trees, and the chunk index using v1 b-trees. With
this implementation, flush dependencies are always on when in the cache. This
will allow attritbutes to be used for "checkpointing" data when object header
dependencies are fixed - i.e. if a writer writes data before an attribute in
that dataset's object header, then if a reader sees the updated attribute the
written data is guaranteed to be visible, as long as that dataset's b-tree nodes
are evicted from the reader's cache.
Also adds support for compression with SWMR.
Also fixes earray implementation to not free (reuse) the file space for deleted
chunks and outdated versions of compressed chunks when doing SWMR writes. These
should eventually be added to a timeout list.
Adds testing for these cases.
Tested: durandal
|