| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose: Improvement
Description: HDF5 Library set pixels_per_scanline parameter to the size of the chunk's
fastest changing dimension. As a result, fastest changing dimension
of the chunk could not be bigger than 4K and smaller than pixels_per_block
value and szip compression couldn't be used for many real datasets.
Solution: Reworked algorithm how HDF5 sets pixels_per_scanline value; only chunks
with the total number of elements less than pixels_per_block value are rejected.
There is no restriction on the size of the chunk's fastest changing
dimension anymore.
Platforms tested: verbena, copper, sol
Misc. update:
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Purpose:
HDF5 now supports SZIP with no encoder.
Description:
SZIP can be configured to have both encoder and decoder or just to have the decoder. HDF5 can now query the configuration of any filter, and will throw errors if users try to write using a filter with encoding disabled.
Solution:
Added H5Zget_filter_info function, changed API for H5Pget_filter and H5P_get_filter_by_id. See SZIP RFC.
Platforms tested:
Copper (fortran, C++, parallel), Sleipnir (C++), Arabica (fortran, C++), Verbena (fortran, C++)
Misc. update:
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup & minor optimization
Description:
Re-work the way interface initialization routines are specified in the
library to avoid the overhead of checking for them in routines where there is
no interface initialization routine. This cleans up warnings with gcc 3.4,
reduces the library binary size a bit (about 2-3%) and should speedup the
library's execution slightly.
Platforms tested:
FreeBSD 4.10 (sleipnir) w/gcc34
h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Range check the szip 'pixels per block' against the chunk size of the
dataset when attempting to create a new dataset, since the szip library
requires the PPB to be at least the size of the fastest changing dimension
in the chunk.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor for h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Don't check the number of szip parameters set during the "can apply" and
"set local" callbacks, be safe about setting the parameters instead.
Platforms tested:
FreeBSD 4.9 (sleipnir)
too minor to require h5committest
|
| |
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Move PABLO_MASK above including headers.
Platforms tested:
FreeBSD 4.8 (sleipnir)
too minor for h5committest
|
| |
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Clean up more warnings from lint.
Platforms tested:
FreeBSD 4.8 (sleipnir)
too minor for h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Add in rest of szip "options mask" macros that were missing. Also made
"raw" options mask set by the library, instead of requiring users to always set
it.
Platforms tested:
FreeBSD 4.8 (sleipnir)
Minor tweaks too small fo h5committest
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Bug fix
Description:
Fix three (!) bugs in szip filter:
- we were using bytes per pixel instead of bits per pixel
- we were using the size of the slowest changing dimension instead of
the fastest changing dimension to compute the blocks per scanline
parameter
- we swapped two parameters when setting up szip_options block.
Solution:
Addressed bugs above.
Platforms tested:
FreeBSD 4.8 (sleipnir)
h5committest not needed
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup
Description:
Uncompressed buffers can't get to the szip filter's decompression code now
that they are handled correctly by the chunk's filter mask.
Solution:
Remove handling of uncompressed buffers from szip filter's decompression
code.
Platforms tested:
FreeBSD 4.8 (sleipnir) w/szip
h5committest not necessary & doesn't test szip code.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fix
Description:
This is an analogue to the previous bug-fix for filters not being
applied to data written but being applied when read. The old way was
if the SZlib library couldn't deflate a dataset, then we'd munge
along pretending that it was okay.
Solution:
Trigger it as an error in this situation. The H5Z_pipeline code which
calls this function can clear the error stack if need be.
Platforms tested:
Modi4 (Parallel & Fortran)
Burrwhite (Fortran & C++)
Baldric (Fortran), but make check didn't work because of "libucb.so"
error that I can't fix...)
Misc. update:
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup/new features
Description:
Switch over to a new style for registering filters with the library -
instead passing in an ID, a string and a callback function to H5Zregister,
the client should pass in a single pointer to a H5Z_claass_t struct which
contains the ID, the description string and all the function callbacks as
fields.
Added support for a new "can apply" callback for each filter, which is
called when a dataset is created to check whether the parameters for that
filter apply correctly to the combination of the datatype and the chunk size
(i.e. dataspace) for the dataset.
Added support for a new "set local" callback for each filter, which
is called when a dataset is created (after the "can apply" filter callback)
and sets filter parameters that are specific to that particular dataset.
Switched the filters we ship over to use the new H5Z_class_t struct for
their internal registrations and also added "set local" callbacks to the
szip and shuffle filters and a "can apply" callback to the szip filter.
Lots of other code cleanups, etc. also
Solution:
Platforms tested:
FreeBSD 4.8 (sleipnir) w/szip
Linux 2.4 (sleipnir) w/szip
Solaris 2.7 (arabica) w/FORTRAN
IRIX64 6.5 (modi4) w/szip, FORTRAN & parallel
Misc. update:
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Code cleanup & bug fix
Description:
Cleaned up comments & copyright statement.
Blocks which were unable to be compressed were missing the "length" prefix,
causing the decompression routine to fail.
Solution:
Always add "length" prefix to all blocks, even ones the fail to compress.
Platforms tested:
FreeBSD 4.8 (sleipnir)
IRIX64 6.5 (modi4) w/parallel
Misc. update:
|
|
|
To support szip compression in HDF5
Description:
This is where szip filter function is located.
Solution:
The filter function composes of "encode" and "decode" part, which
is similar as deflate filter.
One critical difference is:
when szip decompresses the data, it needs to know the buffer size to hold
the decompressed data in advance.
Currently it's hard for HDF5 lib to give the buffer size easily. So to avoid
this problem, in each chunk, we add a small header to hold the buffer size of
each chunk. The code will use UINT32 ENCODE and UINT32 DECODE to finish this
part of work.
Platforms tested:
Since there are changes of configure.in and configure,I didn't use h5committest.
I tested with four platforms.
1) Linux 2.4 (eirene)
2) Solaris 2.7(arabica)
3) windows 2000(VS 6.0)
4) SGI IRIX6.5-64(modi4)
For test 1)-3), only basic C tests were done
For modi4 test, I tested 64-bit C,parallel and fortran.
All tests passed, except a warning message from szip library when checksum is used in some order, which doesn't
e any real problems.
Misc. update:
|
