| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Tweaked internal error handling macros to reduce the size of the library's
object code by about 10-20%.
Also cleaned up some compiler warnings...
Platforms tested:
FreeBSD 4.4 (sleipnir)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
The code in H5Sselect_hyperslab_valid contained an fencepost error and is
allowing selections which overlap the extent by exactly one element in any
dimension to pass as valid instead of flagging the selection as invalid.
This bug only affects hyperslabs which have been OR'ed together, not the
selection from a single H5Sselect_hyperslab.
This fixes bug #550.
Solution:
Changed an '>' to an '>=' and added new regression test to check for error.
Platforms tested:
FreeBSD 4.4 (sleipnir)
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Fix small warning on SGI compilers
Platforms tested:
IRIX64 6.5 (modi4)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Windows is generating hundreds of warnings from some of the practices in
the library. Mostly, they are because size_t is 32-bit and hsize_t is
64-bit on Windows and we were carelessly casting the larger values down to
the smaller ones without checking for overflow.
Also, some other small code cleanups,etc.
Solution:
Re-worked some algorithms to eliminate the casts and also added more
overflow checking for assignments and function parameters which needed
casts.
Kent did most of the work, I just went over his changes and fit them into
the the library code a bit better.
Platforms tested:
FreeBSD 4.4 (hawkwind)
|
|
|
|
|
|
|
|
| |
Fix typo...
Description:
Copy & paste error... :-)
Platforms tested:
FreeBSD 4.4 (hawkwind)
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Get rid of IDs from internal function calls and some small cleanups from
the old-stype => generic property list conversion.
Platforms tested:
FreeBSD 4.4 (hawkwind)
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Fix a bunch of warnings and also make the linux compilers happy with
some casts.
Platforms tested:
Linux 2.2 (eirene)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code speedups, etc.
Description:
Bring in new algorithms and data structures for dealing with hyperslabs.
This speeds up the hyperslab I/O for non-regular hyperslabs by a huge
amount.
Currently, the new API functions are ifdef'ed out, pending discussion
and consensus approval.
Platforms tested:
FreeBSD 4.4 (hawkwind)
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix.
Description:
H5S_select_hyperslab fails to correctly define a hyperslab when the space
to put the hyperslab into is currently a 'none' selection and the hyperslab
operation is the 'or' operation.
Solution:
Detect the situation and internally modify the hyperslab operation to be
a 'set' operation, since that's the next affect desired.
Platforms tested:
FreeBSD 4.4. (hawkwind)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup for better compatibility with C++ compilers
Description:
C++ compilers are choking on our C code, for various reasons:
we used our UNUSED macro incorrectly when referring to pointer types
we used various C++ keywords as variables, etc.
we incremented enum's with the ++ operator.
Solution:
Changed variables, etc.to avoid C++ keywords (new, class, typename, typeid,
template)
Fixed usage of UNUSED macro from this:
char UNUSED *c
to this:
char * UNUSED c
Switched the enums from x++ to x=x+1
Platforms tested:
FreeBSD 4.4 (hawkwind)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup (sorta)
Description:
When the first versions of the HDF5 library were designed, I remembered
vividly the difficulties of porting code from a 32-bit platform to a 16-bit
platform and asked that people use intn & uintn instead of int & unsigned
int, respectively. However, in hindsight, this was overkill and
unnecessary since we weren't going to be porting the HDF5 library to
16-bit architectures.
Currently, the extra uintn & intn typedefs are causing problems for users
who'd like to include both the HDF5 and HDF4 header files in one source
module (like Kent's h4toh5 library).
Solution:
Changed the uintn & intn's to unsigned and int's respectively.
Platforms tested:
FreeBSD 4.4 (hawkwind)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
New Features!
Description:
Start migrating the internal use of property lists in the library from the
older implementation to the new generic property lists.
Currently, only the dataset transfer property lists are migrated to the
new architecture, all the rest of the property list types are still using
the older architecture.
Also, the backward compatibility features are not implemented yet, so
applications which use dataset transfer properties may need to make the
following changes:
H5Pcreate(H5P_DATASET_XFER) -> H5Pcreate_list(H5P_DATASET_XFER_NEW)
and
H5Pclose(<a dataset transfer property list>) -> H5Pclose_list(id)
This still may have some bugs in it, especially with Fortran, but I should
be wrapping up those later today.
Platforms tested:
FreeBSD 4.4 (hawkwind)
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
Using a 'long long' type (i.e. hsize_t) in a switch statement is apparently
not allowed by the HP/UX 10.20 system compiler and causes the compile to
fail.
Solution:
Cast the variable to a size_t for the switch.
Platforms tested:
HP/UX 10.20 (sangamon)
|
|
|
|
|
|
|
|
|
| |
Bug Fix, Code Cleanup, Code Optimization, etc.
Description:
Fold in the hyperslab speedups, clean up compile warnings and change a
few things from using 'unsigned' or 'hsize_t' to use 'size_t' instead.
Platforms tested:
FreeBSD 4.3 (hawkwind), Solaris 2.7 (arabica), Irix64 6.5 (modi4)
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code clean/bug fix
Description:
H5FL (free-list manager) code currently is taking an hsize_t as the size
of a memory block to allocate. On many machines, the size of an hsize_t
is greater than the size of a size_t, potentially leading to incorrect
memory allocations in rare circumstances.
Solution:
Changed hsize_t parameters and variables to size_t.
Platforms tested:
FreeBSD 4.3 (hawkwind)
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix/code improvement.
Description:
'all' selections were (ab)using the array reading code and required that
the internal data transfer buffer size be big enough to hold the an entire
slab of the data, which was confusing and limiting for users.
Solution:
Changed 'all' selections to use sequence reading code instead of array
reading code.
Platforms tested:
FreeBSD 4.3 (hawkwind)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Update
Description:
Changed
#include <hdf_file.h>
construct to
#include "hdf_file.h"
so that the GNU compiler can more easily pick up the dependencies
which it places in the .depend and Dependencies files. Also
regenerated the Dependencies to go along with this.
Platforms tested:
Linux
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fixes
Description:
Fixed a bug in H5Shyper.c where 'contiguous' hyperslabs (i.e. ones which
took up an entire dataset) were not being detected correctly and would
instead be read a part at a time instead of all at once.
Also fixed a bug in the handling of hyperslabs for chunked datasets where
hyperslabs from chunks which weren't aligned on exact dimension bounaries
were not reading/writing data correctly.
Solution:
H5Shyper.c was a single line change from a 'block' size to a 'count' size.
H5Fseq.c changes we much more significant and involved detecting when
non-chunk aligned sequences of data were being written and constructing
hyperslab blocks to pass down to the chunking I/O routine (which only
understand hyperslab I/O requests, not element sequence requests).
This was complicated by the need to align the hyperslabs requested on
dimension boundaries...
Platforms tested:
FreeBSD 4.2. (hawkwind)
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Certain combinations of hyperslabs, especially those that have to be
strip-mined during I/O were causing data to be incorrectly transferred.
Solution:
Changed the code in H5S_get_hyper_regions to be more careful about the
regions of the current dimension that are valid. Sometimes, regions which
had already been iterated through were being re-processed.
Platforms tested:
FreeBSD 4.2 (hawkwind)
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup.
Description:
Fixed _lots_ (I mean _tons_) of warnings spit out by the gcc with the
extra warnings. Including a few show-stoppers for compression on IRIX
machines.
Solution:
Changed lots of variables' types to more sensible and consistent types,
more range-checking, more variable typecasts, etc.
Platforms tested:
FreeBSD 4.2 (hawkwind), IRIX64-64 (modi4)
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Remove some warnings...
Platforms tested:
FreeBSD 4.2 (hawkwind) & IRIX64 6.5 (modi4)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix.
Description:
Cray T3E (and J90?) was failing on partial I/O tests. The bug was in
the optimized hyperslab routines for cases where a hyperslab filled the
temporary datatype conversion buffer and the optimized hyperslab routine
needed to start in the middle of a hyperslab.
Solution:
Corrected code to detect the situation and read in the correct amount of
elements in the hyperslab.
Platforms tested:
Cray T3E (mcurie.nersc.gov)
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug Fix
Description:
Selections in 1-D dataspaces would get into an infinite loop in
H5S_hyper_select_iterate_mem_opt (called by H5Diterate and
H5Dget_vlen_buf_size) due to some incorrect logic.
Solution:
Handled 1-D cases correctly and get out of loop.
Platforms tested:
Eyeballed...
|
|
|
|
|
|
|
|
|
|
| |
Code optimization
Description:
Minor tweaks throughout the optimized regular hyperslab code to increase
speed. This set of improvements increase the benchmark time from taking
~5.46 seconds to ~4.50 seconds, or around a 20% further speedup.
Platforms tested:
Solaris 2.6 (baldric)
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Optimization for parallel I/O
Description:
When contiugous hyperslabs are defined (i.e. with the block=stride), the
library was only aggregating the hyperslabs together for the fastest
changing dimension.
Solution:
Add some extra code to detect when contiguous hyperslabs span more than one
row and output the entire contiguous section at once.
Platforms tested:
FreeBSD 4.1.1 (hawkwind)
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
Parallel Bug Fixes
Description:
Was out of sync with header file re-arrangements I checked in last night.
Solution:
Fixed to use new header files, etc.
Platforms tested:
O2K (modi4)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Maintainance & performance enhancements
Description:
Re-arranged header files to protect private symbols better.
Changed optimized regular hyperslab I/O to compute the offsets more
efficiently from previous method of using matrix operations.
Added sequential I/O operations at a more abstract level (at the same level
as H5F_arr_read/write), to support the optimized hyperslab I/O.
Platforms tested:
Solaris 2.6 (baldric) & FreeBSD 4.1.1 (hawkwind)
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fiX
Description:
H5S_hyper_select_valid would report hyperslab invalid if the one of
the count values is zero. The verifying algorithm did not take into
consideration that block or count can contain zeros to indicate no
element is wanted.
Solution:
Added code to test if block or count is zero. If so, skip the rest
of the validity check.
Platforms tested:
IRIX64 -64.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code Optimization.
Description:
The optimized routines for copying regular hyperslabs in memory have been
using the same matrix routines to copy their hyperslab pieces as the
routines for irregularly shaped hyperslabs. This ends up imposing lots of
extra overhead on the optimized routine, since it basically "knows" all the
matrix information it needs.
Solution:
Keep track of the [small] amount of matrix information necessary to perform
the regular hyperslab copies in the optimized routines themselves instead of
using the matrix routines. This improves the performance for the benchmark
I'm running from ~18 seconds to ~12 seconds and should apply to parallel
I/O situations also.
Platforms tested:
Solaris 2.6 (i.e. baldric)
|
|
|
|
|
|
|
|
|
|
|
|
| |
Small Code Cleanup
Description:
Code to optimize adjacent (i.e. contiguous) hyperslab was ugly and used too
many temporary variables.
Solution:
Computed the optimized hyperslabs slightly differently and got rid of
unnecessary temporary variables.
Platforms tested:
FreeBSD 4.1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix (sorta)
Description:
When the stride and block size of a hyperslab selection are equal, the
blocks that are selected are contiguous in the dataset. Prior to my
hyperslab optimizations, this situation used to be detected and somewhat
optimized to improve performance. I've added more code to optimize for
this situation and integrated it with the new hyperslab optimization that
weren't very efficient for that case as they should have been.
Solution:
Detect contiguous hyperslab selections (i.e. block size in a dimension is
the same as the stride in that dimension) and store the optimized,
contiguous version of that hyperslab. We also store the original, un-
optimized version of the hyperslab to give back to the user if they query
the hyperslab selection they just made.
Platforms tested:
FreeBSD 4.1
|
|
|
|
|
|
| |
Clean up compiler warnings.
Platforms tested:
FreeBSD 4.1
|
|
|
|
|
|
|
|
|
|
|
|
| |
Restore file
Description:
It appears that Robb's checkin earlier today erroneously overwrote this
file with an older version... *grumble*
Solution:
Found another copy of newest version, verified that it is operating
correctly and re-checked it in.
Platforms tested:
FreeBSD 4.1
|
|
|
|
|
|
| |
Clean up small compiler warnings and add missing function prototypes.
Platforms tested:
FreeBSD 4.1
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix Irix pmake bugs
Description:
Build fails on Irix when builddir != srcdir
Solution:
* acconfig.h
* src/H5config.h.in [REGENERATED]
Added definition for HAVE_STREAM
* config/conclude.in
* config/depend1.in
* config/depend2.in
* config/depend3.in
* config/depend4.in
The `Dependencies' file is located in the source
tree. This fixes bugs for Irix pmake when compiling
outside the source tree. Hopefully it still preserves
Albert's changes which allow concurrent compilations
to not stomp on each other's Dependencies files.
* examples/Dependencies [REGENERATED]
* src/Dependencies [REGENERATED]
* test/Dependencies [REGENERATED]
* tools/Dependencies [REGENERATED]
Regenerated for testing purposes.
Platforms:
i686-pc-linux
mips-sgi-irix6.5
sparc-sun-solaris2.6
|
|
|
|
|
|
| |
tests are
passing, but not all of them yet...
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
On my benchmarks,
they are about 4-5 times faster than before. We no longer generate "general"
hyperslab data structures for regular hyperslabs, the general data structures
are only generated when needed for irregular hyperslabs.
Still fixing a couple of nook-and-cranny functions to understand the new
information for the regular hyperslabs, so the tests aren't completely passing,
but I wanted to get this checked in for Elena's benchmarks. I should have
more/all tests passing later today.
|
|
|
|
|
|
|
|
| |
variable instead of
multi-dereferenced pointer chains. This buys us another ~20% improvement in
the hyperslab I/O speed. (From ~30 seconds to ~25 seconds on the h5hypers
benchmark)
|
|
|
|
|
|
|
|
| |
arrays of
hyperslab boundaries after adding them all, instead of maintaining the sorted
order during each addition. This boosts performance for sub-sampled (i.e.
strided) hyperslabs by about a factor of 10! :-)
|
| |
|
| |
|
|
|
|
|
|
| |
realloc()
calls. Speeds user's test code up by ~25 times... :-)
|
|
|
|
|
|
|
| |
several
situations which weren't tested before. Things should be both faster and
easier to understand in this code now.
|
|
|
|
|
|
| |
situations where
hyperslabs were staggered in the dataset.
|
|
|
|
|
|
|
| |
"temporary buffer"
code, since the functionality was superceded. See the followup document for
details on the free-list code.
|
|
|
|
|
|
| |
references.
Data is ok in the file, just mixed up when trying to dereference the region.
|
|
|
|
|
|
|
| |
non-equal
VL types to compare as equal. Added some asserts to make certain nothing
slips through again. Also cleaned up a few warnings from the SGI compiler.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
----------------------
This extensive change is the virtual file layer implementation. I've
ported and tested the sec2, family, and core drivers and only ported
the mpio driver (Albert will test it). So if you need MPIO I would
recommend sticking with the previous version for a while.
You will get a few compile warnings about split and stdio drivers not
being implemented and possibly tracing information not inserted in
some of the drivers. You can safely ignore them but I plan to fix
them.
I'm still working on the split driver because I just realized that it
needs a part of the VFL that isn't written yet.
Documentation is being updated also because there were some minor
changes (mostly just name changes). It should be available on my web
site later this week.
./MANIFEST
./src/Makefile.in
./src/hdf5.h
./src/H5Flow.c [REMOVED]
./src/H5Fstdio.c [REMOVED]
./src/H5Fsec2.c [REMOVED]
./src/H5Fsplit.c [REMOVED]
./src/H5Fmpio.c [REMOVED]
./src/H5Ffamily.c [REMOVED]
./src/H5Fcore.c [REMOVED]
./src/H5MFpublic.h [REMOVED]
./src/H5FD.c [NEW]
./src/H5FDcore.c [NEW]
./src/H5FDcore.h [NEW]
./src/H5FDfamily.c [NEW]
./src/H5FDfamily.h [NEW]
./src/H5FDmpio.c [NEW]
./src/H5FDmpio.h [NEW]
./src/H5FDprivate.h [NEW]
./src/H5FDpublic.h [NEW]
./src/H5FDsec2.c [NEW]
./src/H5FDsec2.h [NEW]
Removed/added files for virtual file layer.
./bin/trace
./src/H5.c
Removed unused public datatypes and added new VFL public
datatypes.
Changed an error message.
./config/BlankForm
./config/dec-flags
./config/gnu-flags
./config/hpux10.20
./config/hpux9.03
./config/irix5.x
./config/irix6.x
./config/solaris2.x
./config/unicosmk
Removed the H5F_OPT_SEEK and H5F_LOW_DFLT constants from the
configuration since they're no longer applicable. The default
file driver is always the sec2 driver and it always optimizes
calls to lseek() or lseek64().
./config/depend.in
C preprocessor errors generated during automatic dependency
building are sent to /dev/null to prevent them from appearing
twice in the make output.
./src/H5AC.c
./src/H5B.c
./src/H5D.c
./src/H5F.c
./src/H5G.c
./src/H5Gent.c
./src/H5Gnode.c
./src/H5HG.c
./src/H5HL.c
./src/H5O.c
./src/H5Oattr.c
./src/H5Odtype.c
./src/H5Oefl.c
./src/H5Oshared.c
./src/H5T.c
./src/H5detect.c
./test/ohdr.c
Changed H5F_ADDR_UNDEF to HADDR_UNDEF to be more consistent
with the `haddr_t' datatype which is now a public type.
./src/H5D.c
./src/H5P.c
./src/H5Ppublic.h
./src/H5Tconv.c
./test/cmpd_dset.c
./test/dsets.c
./test/overhead.c
./test/tselect.c
./test/tvltypes.c
The H5P_DATASET_XFER constant was changed to H5P_DATA_XFER
because the properties apply to all types of I/O operations,
not just datasets.
./src/H5B.c
./src/H5Bprivate.h
./src/H5D.c
./src/H5Dpublic.h
./src/H5F.c
./src/H5Farray.c
./src/H5Fistore.c
./src/H5Fprivate.h
./src/H5Fpublic.h
./src/H5Gnode.c
./src/H5Gpkg.h
./src/H5HG.c
./src/H5HL.c
./src/H5O.c
./src/H5R.c
./src/H5Sall.c
./src/H5Shyper.c
./src/H5Smpio.c
./src/H5Spoint.c
./src/H5Sprivate.h
./test/big.c
./test/h5test.c
./test/istore.c
./testpar/t_dset.c
./testpar/t_file.c
./tools/h5debug.c
./tools/h5ls.c
Modified to work with the virtual file layer by calling H5FD_*
functions instead of H5F_low_* functions and by passing file
access and data transfer properties by object ID instead of
pointer.
Changed H5D_transfer_t to H5FD_mpio_xfer_t since the
COLLECTIVE vs. INDEPENDENT transfer mode is specific to the
MPIO file driver.
Moved MPIO-specific stuff into the MPIO driver.
./src/H5B.c
./src/H5D.c
./src/H5Fprivate.h
The H5F_mpio_* private functions were renamed and placed in
the H5FDmpio driver except those which appeared in H5Smpio.c.
./src/H5E.c
./src/H5Epublic.h
Added major error number H5E_VFL for virtual file layer
related errors.
./src/H5F.c
./src/H5Fprivate.h
Changed the logic that controls whether the boot block is
written. Instead of assuming that the first call to write the
boot block is only to allocate space, I've added a function
argument which makes this explicit.
Changed the way files are compared so that a driver-defined
comparison function can be called. Files which belong to
different drivers are always considered different.
Removed H5F_driver_t since file drivers are now identified by
object ID instead of a special non-user-extendible datatype.
Removed all the hard-coded low-level file properties which
have been replaced by the various file drivers.
./src/H5I.c
./src/H5Iprivate.h
Added the H5I_inc_ref() which was removed a few months ago
since we finally have a use for it.
./src/H5Ipublic.h
Added the H5I_VFL object ID type to identify file drivers in
the virtual file layer.
./src/H5MF.c
./src/H5MFprivate.h
Moved all the allocation/deallocation code into the virtual
file layer which allows file drivers to override much of it.
./src/H5P.c
./src/H5Ppublic.h
Moved file driver-specific code into the various file driver
files.
The H5Pcopy() and H5Pclose() functions make calls into the
virtual file driver to manage the memory for driver-specific
file access and data transfer properties.
./src/H5private.h
./src/H5public.h
The `haddr_t' type is now public.
./test/tfile.c
Added a few more comments.
|
| |
|