| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
| |
cache. Fix issues involving datasets being "not allocated" when they
contain cached raw data.
|
|\
| |
| |
| | |
https://bitbucket.hdfgroup.org/scm/~lrknox/hdf5_lrk into hdf5_1_8
|
| |
| |
| |
| |
| |
| |
| | |
Fix for issue #2 reported in HDFFV-10051:
The layout "dirty" flag for a compact dataset is not reset properly after flusing the data at dataset close.
--Reset the "dirty" flag before flushing the message.
Issue #1 reported in HDFFV-10051 is not a problem for 1.8.
|
|/
|
|
| |
removed and replace it with new url for COPYING file.
|
|
|
|
|
|
|
| |
1) Change fprintf to HDfprintf in hl/test/test_dset_opt.c
2) Remove extra line in src/H5Dpkg.h
3) Add one more test to hl/test/test_dset_opt.c
Tested on mayll and osx1010test.
|
|
|
|
|
|
|
| |
1) Integrate the patch
2) Fix bugs when debugging the patch
3) Add test to hl/test/test_dset_opt.c
Tested on moohan, ostrich, platypus, emu, osx1010test, quail, kituo, mayll.
|
|
|
|
|
|
|
|
|
| |
Fixes HDFFV-8740 (external file storage relative path fix).
Tested on: 64-bit Ubuntu 15.10 (Linux 4.2.0 x86_64) gcc 5.2.1
serial autotools w/ C++ & Fortran
parallel autotools (MPICH 3.1.4) w/ Fortran
serial CMake
|
|
|
|
|
| |
Add internal library callback to H5S_select_iterate to avoid having
to pass hid_t objects internally
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r26195 from trunk to 1.8 branch (minus public API changes):
Switch from enum H5D_CHUNK_BTREE to H5D_CHUNK_IDX_BTREE, along with a couple
of minor whitespace touchups.
Tested on:
MacOSX/64 10.10.3 (amazon) w/serial & parallel
(tested on trunk for 2+ weeks)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r24995 and r24998 from the trunk to the 1.8 branch:
r24995:
Improve checks for NULL buffer from user during H5Dread/H5Dwrite calls.
r24998:
Check in Neil's changes for correctly handling case where library is
reinitialized through a deprecated routine.
Tested on:
Mac OSX/64 10.9.2 (amazon) w/serial & paralle
(change has baked on trunk for >7 days)
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r24769 & r24772 from trunk to 1.8 branch:
Check in Mohamad's changes to support collective I/O on point selections,
along with some other minor cleanups.
Correct some dynamically sized arrays that fail on Windows.
Tested on:
Mac OSX/64 10.9.2 (amazon) w/parallel
(And has been through the daily test regime on the trunk)
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r23713 and r23717 from trunk to 1.8 branch:
Clean up warnings, switch library code to use Standard C/POSIX wrapper
macros, remove internal calls to API routines, update checkapi and checkposix
scripts.
Tested on:
Mac OSX/64 10.8.3 (amazon) w/C++ & FORTRAN
Big-Endian Linux/64 (ostrich)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
HDFFV-8143 Provide a routine(s) for telling the user why the library broke collective data access
Description:
Added H5Pget_mpio_no_collective_cause() function that retrive reasons why the collective I/O was broken during Read/Write IO access.
Reasons to break collective I/O:
- SET_INDEPENDENT
- DATATYPE_CONVERSION
- DATA_TRANSFORMS
- MPIPOSIX
- NOT_SIMPLE_OR_SCALAR_DATASPACES (NULL Space)
- POINT_SELECTIONS
- NOT_CONTIGUOUS_OR_CHUNKED_DATASET (Compact or External-Storage)
- FILTERS
Merged from HDF5 trunk r22735, r22741 , 22743, 22744, 22763.
Tested:
jam-pp (linux32-LE), koala-pp (linux64-LE), wallaby-pp
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r22582 from trunk to 1.8 branch:
Refactor ID class registration to be more like how other "class"s in the
library are registered, and clean up compiler warnings.
Tested on:
Mac OSX/64 10.7.4 (amazon) w/debug
(Too minor to require h5comittest)
|
|
|
|
|
|
|
|
|
|
| |
Bring r22287 from trunk to 1.8 branch:
Clean up more FUNC_ENTER/FUNC_LEAVE macros and move H5D & H5T code toward
the final design (as exemplified by the H5EA & H5FA code).
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug & parallel
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
the object's
flush class action to ensure that cached data is flushed so that H5Ocopy will get
the correct data. (HDFFV-7853)
this line, and those below, will be ignored--
_M .
M test/objcopy.c
M src/H5Ocopy.c
M src/H5Goh.c
M src/H5Dint.c
M src/H5Opkg.h
M src/H5Doh.c
M src/H5Toh.c
M src/H5O.c
M src/H5Dpkg.h
M release_docs/RELEASE.txt
_M fortran
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r21919-21924 from trunk to 1.8 branch: cleanup FUNC_ENTER macros,
etc.
Also removed Subversion mergeinfo tags from a few directories and files.
These are artifacts from non-root merges using pre-1.6 Subversion clients. (This
should fix the problem of unchanged directories looking "changed" during
checkins.)
Tested on:
Mac OSX/64 10.7.3 (amazon) w/debug & production
(daily tested on trunk)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r20135 from trunk to 1.8 branch:
Bring r20134 from metadata journaling merging branch to trunk:
Bring changes from metadata journaling branch to "merging" branch:
Unify routine to mark a dataset's metadata as changed.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
(h5committested on trunk)
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r19638 from trunk to 1.8 branch:
Bring some of Neil's changes to help support SWMR access in the v1
B-tree code from the revise_chunks branch back to the trunk.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
(h5committested on trunk)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r19498 from trunk to 1.8 branch:
Optimize the vector-vector memcpy() routine even further, for a total
of ~2x speedup. :-)
Make a generic vector-vector operation routine and convert other
vector-vector read & write routines to use generic routine instead of multiple
copies of the basic algorithm.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
(h5committested on trunk)
|
|
|
|
|
|
|
|
|
|
| |
Description:
Previously, the chunk cache would, when looking for a chunk, alwaylook for the
chunk in the b-tree before checking the cache. Reworked the functions that
look up chunks to always check the cache first and avoid the b-tree lookup if
the chunk is in cache.
Tested: jam, linew, amani (h5committest)
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
with non-default VL
fill values.
Description:
Previously, H5D_chunk_allocate would, if a dataset were filtered and had a non-default VL fill value, allocate a new buffer for every chunk. Changed
H5D_chunk_allocate to reuse the existing buffer if possible. Also other
miscellaneous cleanup.
Tested: jam, linew, amani (h5committest)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Description:
Previously, it was possible for a chunk to be flushed due to chunk operations in
the callback from H5B_iterate in H5D_chunk_prune_by_extent. Because flushing
the chunk can force it to be reallocated if it is filtered, this can change the
contents of the chunk b-tree in the middle of H5B_iterate. Because H5B_iterate
uses a locally cached copy of the b-tree, this causes subsequent operations
to be passed incorrect data. Rewrote H5D_chunk_prune_by_extent to avoid
H5B_iterate entirely.
Also fixed a bug in the dataset expand algorithm that could cause extra chunks
to be created.
Tested: jam, linew, smirom (h5committest)
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r18650 from trunk to 1.8 branch:
Bring back various minor tweaks & cleanups from the revise_chunks
branch.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
(h5committested on trunk)
|
|
|
|
|
|
|
|
|
|
| |
Description:
Modified algorithm for extending a dataset with early allocation so it only
deals with the new chunks. Formerly, it would loop over all chunks, checking
to see if each existed in cache and on disk, causing major performance issues
with large numbers of chunks.
Tested: jam, linew, amani (h5committest)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r17553 from trunk to 1.8 branch:
Bring general fixes/improvements from file_free_space branch back to
trunk.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.8 (amazon) in debug mode
Mac OS X/32 10.5.8 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r17402 & r17403 from trunk to 1.8 branch:
r17402:
Switch from using a 'flags' parameter to the flush routine(s), to just
using a single 'closing' boolean parameter, since that's all we're doing with
the flags anyway and this makes things more obvious.
r17403:
Eliminate 'closing' flag to dataset flush routine, since all cached
dataset information has already been flushed when a file is closed and the
datasets themselves will be closed (and the dataset caches destroyed).
Skip calling more flush routines when the file is closing, in preparation
for eventual separation of the 'flush' concept from the 'shutdown the cache'
concept.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
(h5committest performed on trunk)
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r17284 from trunk to 1.8 branch:
Remove unneccessary dest. layout information from chunked dataset copying
routine.
Tested on:
Mac OS X/32 10.5.7 (amazon) debug & production
h5committested on trunk
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r17281 from trunk to 1.8 branch:
Further refactoring of the dataset layout information, to separate the
storage information from the layout info.
Tested on:
Mac OS X/32 10.5.7 (amazon) debug & production
h5committested on trunk
|
|
|
|
|
|
|
|
| |
Bring r17275 and r17276 from trunk to 1.8 branch.
Tested on:
Mac OS X/32 10.5.7 (amazon) w/debug & production
(too minor to require h5committest)
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r17269 from trunk to 1.8 branch:
Refactor how serialized size of layout message is computed, slightly.
Tested on:
Mac OS X/32 10.5.7 (amazon) debug & production
FreeBSD/32 6.3 (duty) debug
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r17243 from trunk to 1.8 branch:
Refactor how the layout message is created/written/read in the
dataset's object header.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
(change h5committested on trunk)
|
|
|
|
|
|
|
|
|
| |
Bring r17230 back from trunk to 1.8 branch:
Various minor tuneups noticed in working on the revise_chunks branch.
Tested on:
Mac OS X/32 10.5.7 (amazon) debug & production
full testing done on trunk before bringing change back.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r17148:17149 from trunk into 1.8 branch:
Refactor how chunked dataset information is computed, moving it earlier and
avoiding more recomputation.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.7 (amazon) in debug mode
Mac OS X/32 10.5.7 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r17146 from trunk to 1.8 branch:
Bring a bunch of misc. improvements & tweaks from the revise_chunks
branch back to the trunk, so that future merges won't be so painful.
Tested on:
FreeBSd/32 6.3 (duty)
(h5committested on trunk)
|
|
|
|
|
|
|
|
|
|
| |
Bring r16973 back from trunk to 1.8 branch:
Refactor chunk cache entry information to remove some [actually] unused
fields.
Tested on:
<tested on trunk>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Merge these trunk revisions which occurred during the 1.8.3 release code
freeze back to the 1.8 branch:
From Quincey: 16845 16847 16849 16851 16858 16869 16897
From Ray: 16859 16880
From Allen: 16863
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (jam) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in debug mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
Mac OS X/32 10.5.6 (amazon) in debug mode
Mac OS X/32 10.5.6 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
|
|
|
|
|
|
|
|
|
|
|
| |
Bring v16532 back from trunk to v1.8 branch:
Pass the chunk "user data" to H5D_chunk_unlock(), so that chunks with
an address already aren't reallocated.
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
(h5committest not needed here, performed on trunk)
|
|
|
|
|
|
|
|
|
| |
Description:
In some situations it was possible for the fill value to not be written to parts
of a chunked dataset, particularly when extending and/or shrinking. Prior to
the fix for the chunk cache (1015) these bugs would have been exceedingly rare.
Tested: jam, smirom, linew (h5committest)
|
|
|
|
|
|
|
|
|
|
|
| |
Description:
The meaning of the "nbytes" field in H5D_rdcc_t was not clear, and some places
assumed it was the maximum size of the chunk cache, while some assumed it was
the current size of the chunk cache. The end result was that only 1 chunk could
be held in cache at a time. This field has been replaced by "nbytes_max" and
"nbytes_used". Performance of cached I/O should improve greatly.
Tested: jam, smirom (h5committest)
|
|
|
|
|
|
|
|
|
|
| |
than the cache size and isn't allocated
on disk, the library still loaded it in the cache, which is redundant. I changed it to bypass the
cache and added a test in dsets.c.
Tested on jam and smirom.
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r16354 back from trunk:
Refactor internal layout information, making it easier to add another
type of chunk index.
Tested on:
FreeBSD/32 (duty)
(other configurations tested with original change)
|
|
|
|
|
|
|
|
|
|
| |
Bring r16182 back from trunk:
Rename internal routines, variables, macros, typedefs, etc. for chunked
dataset storage from "istore" to some variant of "chunk" or "btree".
Tested on:
FreeBSD/32 6.3 (duty) in debug mode
|
|
|
|
|
|
|
|
| |
H5Dget_access_plist
functions to 1.8 branch
Tested: kagiso, jam (trunk version tested with h5committest)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bring r15800 back from trunk:
- File free space branch changes through r15794
Tested on:
Mac OS X/32 10.5.5 (amazon) in debug mode
Mac OS X/32 10.5.5 (amazon) w/C++ & FORTRAN, w/threadsafe,
in production mode
FreeBSD/32 6.3 (duty) in debug mode
FreeBSD/64 6.3 (liberty) w/C++ & FORTRAN, in debug mode
Linux/32 2.6 (kagiso) w/PGI compilers, w/C++ & FORTRAN, w/threadsafe,
in debug mode
Linux/64-amd64 2.6 (smirom) w/Intel compilers w/default API=1.6.x,
w/C++ & FORTRAN, in production mode
Solaris/32 2.10 (linew) w/deprecated symbols disabled, w/C++ & FORTRAN,
w/szip filter, in production mode
Linux/64-ia64 2.6 (cobalt) w/Intel compilers, w/C++ & FORTRAN,
in production mode
Linux/64-ia64 2.4 (tg-login3) w/parallel, w/FORTRAN, in production mode
Linux/64-amd64 2.6 (abe) w/parallel, w/FORTRAN, in production mode
|
|
|
|
|
|
|
|
|
|
|
| |
Description: The optimized "subset" compound conversion function would
improperly copy the "blank" space at the end of compound types. Modified
H5T_conv_struct_init to detect when the subset type has extra space at the end,
and calculate the size of the data that should be copied into the conversion
buffer for each element. Changes to the functions that implement these
conversions.
Tested: kagiso, linew, smirom (h5committest)
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Remove trailing whitespace from C/C++ source files, with the following
script:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Tested on:
Mac OS X/32 10.5.5 (amazon)
No need for h5committest, just whitespace changes...
|
|
|
|
|
|
|
|
|
| |
Bring r15609 back from trunk:
Omnibus compiler warning cleanup & some formatting happiness.
Tested on:
Mac OS X/32 10.5.4 (amazon)
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bring back revision 15131 from trunk:
Finish omnibus chunked dataset I/O refactoring, to separate general
actions on chunked datasets from actions that are specific to using the v1
B-tree index.
Cleaned up a few bugs and added some additional tests also.
Tested on:
Mac OS X/32 10.5.3 (amazon)
Linux/32 2.4 (chicago)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Port revision 15015 back to 1.8 branch:
> Detect chunks that are >4GB before dataset gets created and return error
> to application.
>
> Tweak lots of internal variables that hold the chunk size/dimensions to
> use an 'uint32_t', instead of a 'size_t', so that the integer size is constant.
>
> Correct a number of our tests which were creating datasets with chunks
> that were >4GB and add some specific tests for >4GB chunk size detection.
>
> Minor whitespace & other code cleanups.
Tested on:
Mac OS X/32 10.5.2 (amazon)
Forthcoming testing on other platforms...
|