Attributes

Creating "Large" HDF5 Attributes

Problem

You would like to use HDF5 attributes the size of whose values exceeds a few dozen kilobytes

Solution

A file format change in the HDF5 1.8.x family of library releases made it possible to have attributes larger than about 64 KiB. Ensure that the lower library version bound for new HDF5 item creation is at least 1.8.x, and create larger attributes as usual.
In the example below, we create an attribute whose value occupies 4 MiB.

Note

This feature is only supported in HDF5 1.8.x+

    {
        __label__ fail_attr, fail_aspace, fail_fapl, fail_file;
        hid_t fapl, file, aspace, attr;
 
        if ((fapl = H5Pcreate(H5P_FILE_ACCESS)) < 0) {
            ret_val = EXIT_FAILURE;
            goto fail_fapl;
        }
#if H5_VERSION_GE(1, 10, 0)
        if (H5Pset_libver_bounds(fapl, H5F_LIBVER_V18, H5F_LIBVER_LATEST) < 0) {
#elif H5_VERSION_GE(1, 8, 0)
        if (H5Pset_libver_bounds(fapl, H5F_LIBVER_LATEST, H5F_LIBVER_LATEST) < 0) {
#else
#error Only HDF5 1.8.x and later supported.
#endif
            ret_val = EXIT_FAILURE;
            goto fail_file;
        }
        if ((file = H5Fcreate("large_attribute.h5", H5F_ACC_TRUNC, H5P_DEFAULT, fapl)) < 0) {
            ret_val = EXIT_FAILURE;
            goto fail_file;
        }
 
        if ((aspace = H5Screate_simple(1, (hsize_t[]){1024 * 1024}, NULL)) < 0) {
            ret_val = EXIT_FAILURE;
            goto fail_aspace;
        }
        if ((attr = H5Acreate(file, "4MiB", H5T_IEEE_F32LE, aspace, H5P_DEFAULT, H5P_DEFAULT)) < 0) {
            ret_val = EXIT_FAILURE;
            goto fail_attr;
        }
 
        H5Aclose(attr);
fail_attr:
        H5Sclose(aspace);
fail_aspace:
        H5Fclose(file);
fail_file:
        H5Pclose(fapl);
fail_fapl:;
    }

Discussion

Large attributes are supported only in HDF5 versions 1.8.x and higher. This has implications for the accessibility of your HDF5 files and is your call.
Since there are no size limitations for large attributes, it might seem tempting to mistake them for dataset stand-ins. This is not the case, for at least two reasons:

1. Attributes decorate HDF5 objects, have their own local namespace, and can't be link targets.
2. Attribute I/O treats the attribute value as atomic, i.e., there is no support for partial I/O. A large attribute will always be read or written in its entirety.

See Also

See Maintaining Compatibility with other HDF5 Library Versions for HDF5 compatibility implications.