| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
| |
Get rid of two different types of fill value struct (merge H5O_fill_t
and H5O_fill_new_t) and clean up & simplify dataset initialization code.
(In preparation for shared object header message method call
refactoring).
Tested on:
FreeBSD/32 6.2 (duty)
Mac OS X/32 10.4.8 (amazon)
|
|
|
|
|
|
|
|
|
| |
checking in
code to copy shared messages. Should be fewer memory leaks and fewer
"uninitialized memory reads."
Tested on kagiso, smirom, and Windows.
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix problem with copying objects when the dataset to copy has an "older"
version of the layout message and needs to compute the size of the dataset's
storage after the dataspace information is available.
Minor random cleanups also...
Tested on:
FreeBSD/32 4.11 (sleipnir)
Too minor/weird to test w/h5committest...
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Finish implementation of H5Literate() [still needs real tests]
Clean up datatype macro warnings a bit more.
Unify iterator callback macros and put up in public header file, with the
iterator directions.
Tested on:
Mac OS X/32 10.4.8 (amazon)
FreeBSD/32 4.11 (sleipnir)
Linux/32 2.4 (heping)
Linux/64 2.4 (mir)
AIX/32 5.? (copper)
|
|
|
|
|
|
|
|
|
| |
Clean up compiler warnings, esp. on 64-bit Linux boxes.
Tested on:
FreeBSD/32 4.11 (sleipnir) w/threadsafe
Linux/32 2.4 (heping) w/C++ & FORTRAN
Linux/64 2.4 (mir) w/build-all & 1.6 compat
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Finish new version of the I/O pipeline message, which is much smaller than
the previous version. This version is used with the "use the latest version
of the format" flag.
Closed several memory leaks/overruns (found with valgrind).
Also, lots of compiler & formatting cleanups.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
|
|
|
|
|
|
|
|
|
| |
Review, revise & checkin in Peter's latest round of object copy changes,
which add basic support for datasets & attributes with reference datatypes.
Tested on:
Linux/32 2.6 (chicago)
Linux/64 2.6 (chicago2)
|
|
|
|
|
|
|
|
|
| |
Clean up some of the parallel chunk I/O compiler warnings and other minor
code rearrangement.
Tested on:
FreeBSD 4.11 (sleipnir) w/paralllel
h5committest
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Trim trailing whitespace in Makefile.am and C/C++ source files to make
diffing changes easier.
Platforms tested:
None necessary, whitespace only change
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code maintenance
Description:
Remove flexible parallel code
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4 (heping)
Solaris 2.9 (shanti)
Linux 2.4/64 (mir)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New/expanded features
Description:
Check in Peter's changed for the object copy code:
- Allow/fix copying datasets using named variable-length datatypes
- Start adding framework for property list to control how object
copying occurs.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up some compiler warnings (esp. those flagged on Windows builds)
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix & new feature
Description:
Support variable-length datatypes in compact data storage and chunked
data storage, along with attributes.
Bug fix on the H5T_vlen_set_loc to allow for changing the file on a
variable-length datatype on disk.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4
Can't h5committest right now, due to missing cache files.
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Fix a bunch of warnings flagged by Windows compilers.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
slight code change for efficiency of iteration of chunk address
Description:
Quincey picked this up because of my inexperience of C. A pair of redundant malloc and free calls will be removed.
Solution:
Platforms tested:
heping(Linux 2.4)
too little to be tested.
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
To support collective chunk IO, obtain the chunk addresses globally from one process.
Description:
Implement a call-back function to iterate B-tree to do the job.
Check in this code for major check-ins later.
Solution:
Platforms tested:
heping(Linux)
Should not be used by any library functions now, so won't affect library and tool tests.
No need to test on 3 platforms.
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New feature
Description:
Check in baseline for compact group revisions, which radically revises the
source code for managing groups and object headers.
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
This initiates the "unstable" phase of the 1.7.x branch, leading up
to the 1.8.0 release. Please test this code, but do _NOT_ keep files created
with it - the format will change again before the release and you will not
be able to read your old files!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!! WARNING!!!!
Solution:
There's too many changes to really describe them all, but some of them
include:
- Stop abusing the H5G_entry_t structure and split it into two separate
structures for non-symbol table node use within the library: H5O_loc_t
for object locations in a file and H5G_name_t to store the path to
an opened object. H5G_entry_t is now only used for storing symbol
table entries on disk.
- Retire H5G_namei() in favor of a more general mechanism for traversing
group paths and issuing callbacks on objects located. This gets us out
of the business of hacking H5G_namei() for new features, generally.
- Revised H5O* routines to take a H5O_loc_t instead of H5G_entry_t
- Lots more...
Platforms tested:
h5committested and maybe another dozen configurations.... :-)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New feature
Description:
Add in baseline "object copy" code from Peter [in the form of a new API
routine: H5Gcopy()]. There's still some work to do (like handling variable-
length datatypes and possibly support for references) and it hasn't been tested
on mounted files yet, but the core functionality is there and working
correctly.
I've also got a set of patches to update the 1.6 branch with tweaks to
keep the branches mostly in sync, but Elena will kill me if I import them
before the 1.6.5 release is out... :-)
Platforms tested:
FreeBSD 4.11 (sleipnir)
h5committested
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up & standardize a bit in preparation for coding standards
discussion.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New debugging feature
Description:
Add some code to track where memory from the internal free list allocator
is allocated within the library. It's not quite ready for "prime time" yet,
but it's close enough to be useful.
This is disabled by default and requires the H5FL_TRACK macro in
src/H5FLprivate.h to be uncommented to activate during debugging.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Linux 2.4
Too minor to require full h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Trim trailing whitespace, which is making 'diff'ing the two branches
difficult.
Solution:
Ran this script in each directory:
foreach f (*.[ch] *.cpp)
sed 's/[[:blank:]]*$//' $f > sed.out && mv sed.out $f
end
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Add some "safety net" sort of assertions and sanity checks contributed by
user.
Platforms tested:
FreeBSD 4.11 (sleipnir)
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Description: Removed PABLO from the source
Solution:
Platforms tested: arabica with 64-bit, copper with parallel,
heping with GNU C and C++ and PGI fortran (but
I disabled hl, there is some weird problem only
on heping: F9XMODFLAG is not
propagated to the Makefile files
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix/Code Cleanup/Doc Cleanup/Optimization/Branch Sync :-)
Description:
Generally speaking, this is the "signed->unsigned" change to selections.
However, in the process of merging code back, things got stickier and stickier
until I ended up doing a big "sync the two branches up" operation. So... I
brought back all the "infrastructure" fixes from the development branch to the
release branch (which I think were actually making some improvement in
performance) as well as fixed several bugs which had been fixed in one branch,
but not the other.
I've also tagged the repository before making this checkin with the label
"before_signed_unsigned_changes".
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel & fphdf5
FreeBSD 4.10 (sleipnir) w/threadsafe
FreeBSD 4.10 (sleipnir) w/backward compatibility
Solaris 2.7 (arabica) w/"purify options"
Solaris 2.8 (sol) w/FORTRAN & C++
AIX 5.x (copper) w/parallel & FORTRAN
IRIX64 6.5 (modi4) w/FORTRAN
Linux 2.4 (heping) w/FORTRAN & C++
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Relax restrictions on parallel I/O to allow compressed, chunked datasets
to be read in parallel (collective access will be degraded to independent
access, but will retrieve the information still).
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Solaris 2.7 (arabica)
IRIX64 6.5 (modi4)
h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix/code cleanup
Description:
Clean up raw data I/O code to bundle the I/O parameters (dataset, DXPL ID,
etc) into a single struct to pass around through the dataset I/O routines,
since they are always passed together, until very near the bottom of the I/O
stack.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Solaris 2.7 (arabica)
IRIX64 6.5 (modi4)
h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
Feature
Description:
Datatypes and groups now use H5FO "file object" code that was previously
only used by datasets. These objects will hold a file open if the file
is closed but they have not yet been closed. If these objects are unlinked
then relinked, they will not be destroyed. If they are opened twice (even
by two different names), both IDs will "see" changes made to the object
using the other ID.
When an object is opened using two different names (e.g., if a dataset was
opened under one name, then mounted and opened under its new name), calling
H5Iget_name() on a given hid_t will return the name used to open that hid_t,
not the current name of the object (this is a feature, and a change from the
previous behavior of datasets).
Solution:
Used H5FO code that was already in place for datasets. Broke H5D_t's, H5T_t's,
and H5G_t's into a "shared" struct and a private struct. The shared structs
(H5D_shared_t, etc.) hold the object's information and are used by all IDs
that point to a given object in the file. The private structs are pointed
to by the hid_t and contain the object's group entry information (including its
name) and a pointer to the shared struct for that object.
This changed the naming of structs throughout the library (e.g., datatype->size
is now datatype->shared->size). I added an updated H5Tinit.c to windows.zip.
Platforms tested:
Visual Studio 7, sleipnir, arabica, verbena
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Fix situation where deleting a chunked datasets with B-tree nodes that
weren't in the metadata cache would die with a core dump.
Platforms tested:
FreeBSD 4.10 (sleipnir)
Linux 2.4 (heping)
Solaris 2.7 (arabica)
|
|
|
|
|
|
|
|
|
|
|
|
| |
Description: Restore 6 old error API functions back to the library to be backward
compatible with v1.6. They are H5Epush, H5Eprint, H5Ewalk, H5Eclear, H5Eset_auto,
H5Eget_auto. These functions do not have error stack as parameter.
Solution: Internally, these functions use default error stack.
Platforms tested: h5committest and fuss.
Misc. update: RELEASE.txt
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix (sorta)
Description:
Change reading of "missing" chunks from datasets with undefined fill
values to not overwrite the application buffer with random garbage from
memory. Note that this is almost the same, since whatever garbage the
application had in those locations will still be there...
Platforms tested:
FreeBSD 4.10 (sleipnir)
IRIX64 6.5 (modi4)
h5committested
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
1 - Dataset contiguous storage cache information had a bug where it was
possible to try to access invalid cache information if the cache wasn't filled
the first time it attempted to loop through the list of offset/length vectors.
2 - Additionally, the contiguous storage cache information was being used
in certain circumstances from the chunked dataset I/O code path, which was
generally fatal since the chunk storage and contiguous storage information
were stored together in a union.
Solution:
1 - Avoid special case of first trip through loop over offset/length
I/O vectors and always check for the contiguous storage sieve buffer buffer
being NULL.
2 - Change the union containing the chunk and contiguous storage cache
information into a struct, allowing both to be used at the same time.
Platforms tested:
FreeBSD 4.10 (sleipnir)
h5committested
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix.
Description:
Address two problems:
- The computation of the scanline in the szip filter was being
performed in the "can apply" callback routine instead of the
"set local" routine.
- The routine which allocated all the chunks for an entire dataset
(which is invoked when the allocation time is early or late,
rather than incremental) wasn't recording a failed filter in
the information for the chunk, causing the library to believe
that the chunk had the filter applied when it really hadn't.
Solution:
- Move the scanline computation to the "set local" callback.
- Record the filter mask with each chunk created when allocating them.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/szip
Too obscure to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Always write fill values to chunks when initializing entire B-tree and
any filters are defined.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Solaris 2.7 (arabica)
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup, sorta
Description:
Added ifdef sections for "H5_USING_PURIFY" in various places in the code,
which are designed to reduce the spurious "uninitialized memory read" warnings
from purify which are actually OK. Note that this macro will have to be
turned on by adding it to the CFLAGS for the build - I didn't think it was
important enough to add a configure flag for.
Also, the changes in H5HG.c optimize the walks through the objects in a
heap to only look at the 'used' entries instead of all the 'allocated' entries.
Platforms tested:
Solaris 2.7 (arabica) w/purify
Not tested by h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Adding the first round of patches about supporting collective chunk IO in HDF5
Description:
The current HDF5 library doesn't support collective MPIO with chunk storage. When users set collective option in their data transfer with chunk storage, the library silently converted the option to INDEPENDENT and that caused trememdous performance penalty. Some application like
WRF-parallel HDF5 IO module has to use contiguous storage for this reason. However, chunking storage has its own advantage(supporting compression filters and extensible dataset), so to make collective MPIO possible inside HDF5 with chunking storage is a very important task.
This check-in make collective chunk IO possible for some special cases. The condition is as follows(either case is fine with using collective chunk IO)
1. for each process, the hyperslab selection of the file data space of each dataset is regular and it is fit in one chunk.
2. for each process, the hyperslab selection of the file data space of each dataset is single and the number of chunks for the hyperslab selection should be equal.
Solution:
Lift up the contiguous storage requirement for collective IO.
Use H5D_isstore_get_addr to get the corresponding chunk address. Then the original library routines will take care of getting the correct address to make sure that MPI FILE TYPE is built correctly for collective IO>
Platforms tested:
arabica(sol), copper(AIX), eirene(Linux)
parallel test is checked at copper.
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Fix error in chunked dataset I/O where data written out wasn't read
correctly from a chunked, extendible dataset after the dataset was extended.
Also, fix parallel I/O tests to gather error results from all processes,
in order to detect errors that only occur on one process.
Solution:
Bypass chunk cache for reads as well as writes, if parallel I/O driver is
used and file is opened for writing.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code optimization
Description:
Re-work the insertion of a new child into an existing node, to exploit
some speedups for adding the rightmost child, since this is a very common case
when appending records to an unlimited size dataset.
Also, hoist the checks for the tree's 'K' value into a field in the shared
information about the tree, instead of re-calculating them all the time.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code optimization
Description:
Avoid calling vector comparison routine when operating on 1-D chunks.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code optimization
Description:
Refactor B-tree code to extract all common information for a B-tree into a
shared structure that is pointed to by all the nodes in tree (instead of being
included in each node).
Also re-order B-tree node comparison checks for chunked datasets to
check for >= the upper node first, since the comparison is a bit "heavy" and
this check is more likely to succeed when you are adding records to the
dataset.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
(also, recent h5dump commits have broken testing...)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
The "shared" raw B-tree node can get freed before all the B-tree nodes
had been flushed out to disk and released by the cache.
Solution:
Implement a simple reference counting wrapper for objects in the library
and use it to hold the shared raw B-tree nodes so they aren't freed before all
references to them in memory are released.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir)
IRIX64 6.5 (modei4)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code optimization
Description:
Since the raw B-tree nodes are the same size and only used when reading in
or writing out a B-tree node, move raw B-tree node buffer from being per node
to a single node that is shared among all B-tree nodes of a particular tree,
freeing up a lot of space and eliminating lots of memory copies, etc.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code optimization
Description:
Don't copy layout information, just point to existing information.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Reduce compiler warnings on SGI IRIX
Platforms tested:
FreeBSD 4.10 (sleipnir)
SGI IRIX6 (Cheryl's machine)
Too minor to require full h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup & minor optimization
Description:
Re-work the way interface initialization routines are specified in the
library to avoid the overhead of checking for them in routines where there is
no interface initialization routine. This cleans up warnings with gcc 3.4,
reduces the library binary size a bit (about 2-3%) and should speedup the
library's execution slightly.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/gcc34
h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code optimization
Description:
Eliminate memcpy() when using default DXPL by pointing at existing
default object, instead of copying it.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir) w/parallel
Too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code optimization
Description:
Avoid pushing errors on error stack when an object is not found in a B-tree.
Sometimes we are just checking if the object exists before we insert it into
the B-tree and the higher levels in the library should be responsible for
determining if not finding the object in B-tree is really an error.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir) w/parallel
too minor to require h5committest
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Refactor code
Description:
Move chunk and contiguous cached raw data from file information to dataset
information. This simplifies a number of internal interfaces, aligns the
code with it's purpose better and should allow more optimizations to the
chunked data I/O performance.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.10 (sleipnir)
h5committest
Misc. update:
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code optimization
Description:
Don't recompute the internal index value for looking up the chunk in the
hash table, just use the value already computed from iterating through the
chunks.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.9 (sleipnir) w/parallel
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code optimization & bug fix
Description:
When dimension information is being stored in the storage layout message
on disk, it is stored as 32-bit quantities, possibly truncating the dimension
information, if a dimension is greater than 32-bits in size.
Solution:
Fix the storage layout message problem by revising file format to not store
dimension information, since it is already available in the dataspace.
Also revise the storage layout data structures to be more compartmentalized
for the information for contiguous, chunked and compact storage.
Platforms tested:
FreeBSD 4.9 (sleipnir) w/parallel
Solaris 2.7 (arabica)
h5committest
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code optimization
Description:
Reduce amount of information allocated/freed for chunk cache.
Platforms tested:
Solaris 2.7 (arabica)
FreeBSD 4.9 (sleipnir) w/paralell)
too minor to require h5committest
|