H5Z: Filter and Compression Interface

Filter and Compression API Functions

HDF5 supports a filter pipeline that provides the capability for standard and customized raw data processing during I/O operations. HDF5 is distributed with a small set of standard filters such as compression (gzip and a shuffling algorithm) and error checking (Fletcher32 checksum). For further flexibility, the library includes tools enabling a user application to extend the pipeline through the creation and registration of customized filters.

As mentioned above, one set of filters distributed with HDF5 provides built-in methods for raw data compression. The flexibility of the filter pipeline implementation enables the definition of additional compression methods by a user application. A compression method
 —  is associated with a dataset when the dataset is created,
 —  can be used only with chunked data (ie., datasets stored in the H5D_CHUNKED storage layout), and
 —  is applied independently to each chunk of the dataset.

The HDF5 library does not support compression for contiguous datasets because of the difficulty of implementing random access for partial I/O. Compact dataset compression is not supported because it would not produce significant results.

See The Dataset Interface (H5D) in the HDF5 User's Guide for further information regarding data compression.

The FORTRAN90 Interfaces:
In general, each FORTRAN90 subroutine performs exactly the same task as the corresponding C function. The links below go to the C function descriptions, which serve as general descriptions for both. A button, under Non-C API(s) at the end of the C function description, opens an external browser window displaying the FORTRAN90-specific information. You will probably want to adjust the size and location of this external window so that both browser windows are visible and to facilitate moving easily between them.

h5zunregister_f

h5zfilter_avail_f

Name: H5Zregister

Signature:

herr_t H5Zregister(H5Z_filter_t filter, const char *comment, H5Z_func_t function )

Purpose:

Registers new filter.

Description:

H5Zregister registers a new filter with the HDF5 library.

Making a new filter available to an application is a two-step process. The first step is to define the new filter function as described below. This step can be skipped only when the filter is predifined, as is the case with the Fletcher32 checksum and shuffle filters that are distributed with the HDF5 Library. This call to H5Zregister, registering the filter with the library, is the second step.

The new filter is specified by the filter identifier filter.

comment is used for debugging and may be the null pointer.

function is a user-defined function which performs the action of the filter.

The statistics associated with a filter are not reset by this function; they accumulate over the life of the library.

New filters. Before a filter can be linked into an application with H5Zregister, the filter must be defined as described in the HDF5 Library header file H5Zpublic.h:

typedef size_t (*H5Z_func_t) (unsigned int flags, size_t cd_nelmts, const unsigned int cd_values[], size_t nbytes, size_t *buf_size, void **buf)

The parameters flags, cd_nelmts, and cd_values are the same as for the function The additional flag H5Z_FLAG_REVERSE is set when the filter is called as part of the input pipeline. The parameter *buf points to the input buffer which has a size of *buf_size bytes, nbytes of which are valid data.

The filter should perform the transformation in place if possible and return the number of valid bytes or zero for failure. If the transformation cannot be done in place, then the filter should allocate a new buffer with malloc() and assign it to *buf, assigning the allocated size of that buffer to *buf_size. The old buffer should be freed by calling free().

Parameters:

H5Z_filter_t method

IN: Filter identifier.

const char *comment

IN: String associated with the filter; used for debugging purposes only. May be the null pointer.

H5Z_func_t function

IN: Filter function.

Returns:

Returns a non-negative value if successful; otherwise returns a negative value.

Name: H5Zunregister

Signature:

herr_t H5Zunregister(H5Z_filter_t filter)

Purpose:

Unregisters a filter.

Description:

H5Zunregister unregisters the filter specified in filter.  

After a call to H5Zunregister, the filter specified in filter will no longer be available to the application.

Parameters:

H5Z_filter_t filter

IN: Identifier of the filter to be unregistered.

Returns:

Returns a non-negative value if successful; otherwise returns a negative value.

Non-C API(s):

Name: H5Zfilter_avail

Signature:

herr_t H5Zfilter_avail(H5Z_filter_t filter)

Purpose:

Determines whether a filter is available.

Description:

H5Zfilter_avail determines whether the filter specified in filter is available to the application.

Parameters:

H5Z_filter_t filter

IN: Filter identifier.

Returns:

Returns a non-negative value if successful; otherwise returns a negative value.

Non-C API(s):