/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * Copyright by The HDF Group. * * All rights reserved. * * * * This file is part of HDF5. The full HDF5 copyright notice, including * * terms governing use, modification, and redistribution, is contained in * * the COPYING file, which can be found at the root of the source code * * distribution tree, or in https://www.hdfgroup.org/licenses. * * If you do not have access to either file, you may request a copy from * * help@hdfgroup.org. * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */ /* * Programmer: Quincey Koziol * Saturday, September 12, 2015 * * Purpose: This file contains declarations which define macros for the * H5A package. Including this header means that the source file * is part of the H5A package. */ #ifndef H5Amodule_H #define H5Amodule_H /* Define the proper control macros for the generic FUNC_ENTER/LEAVE and error * reporting macros. */ #define H5A_MODULE #define H5_MY_PKG H5A #define H5_MY_PKG_ERR H5E_ATTR /** \page H5A_UG HDF5 Attributes * * \section sec_attribute HDF5 Attributes * * An HDF5 attribute is a small metadata object describing the nature and/or intended usage of a primary data * object. A primary data object may be a dataset, group, or committed datatype. * * \subsection subsec_attribute_intro Introduction * * Attributes are assumed to be very small as data objects go, so storing them as standard HDF5 datasets would * be quite inefficient. HDF5 attributes are therefore managed through a special attributes interface, * \ref H5A, which is designed to easily attach attributes to primary data objects as small datasets * containing metadata information and to minimize storage requirements. * * Consider, as examples of the simplest case, a set of laboratory readings taken under known temperature and * pressure conditions of 18.0 degrees Celsius and 0.5 atmospheres, respectively. The temperature and pressure * stored as attributes of the dataset could be described as the following name/value pairs: * \li temp=18.0 * \li pressure=0.5 * * While HDF5 attributes are not standard HDF5 datasets, they have much in common: * \li An attribute has a user-defined dataspace and the included metadata has a user-assigned datatype * \li Metadata can be of any valid HDF5 datatype * \li Attributes are addressed by name * * But there are some very important differences: * \li There is no provision for special storage such as compression or chunking * \li There is no partial I/O or sub-setting capability for attribute data * \li Attributes cannot be shared * \li Attributes cannot have attributes * \li Being small, an attribute is stored in the object header of the object it describes and is thus * attached directly to that object * * \subsection subsec_error_H5A Attribute Function Summaries * @see H5A reference manual * * \subsection subsec_attribute_program Programming Model for Attributes * * The figure below shows the UML model for an HDF5 attribute and its associated dataspace and datatype. * * * * *
* \image html UML_Attribute.jpg "The UML model for an HDF5 attribute" *
* * Creating an attribute is similar to creating a dataset. To create an attribute, the application must * specify the object to which the attribute is attached, the datatype and dataspace of the attribute * data, and the attribute creation property list. * * The following steps are required to create and write an HDF5 attribute: * \li Obtain the object identifier for the attribute’s primary data object * \li Define the characteristics of the attribute and specify the attribute creation property list * * \li Create the attribute * \li Write the attribute data (optional) * \li Close the attribute (and datatype, dataspace, and attribute creation property list, if necessary) * \li Close the primary data object (if appropriate) * * The following steps are required to open and read/write an existing attribute. Since HDF5 attributes * allow no partial I/O, you need specify only the attribute and the attribute’s memory datatype to read it: * \li Obtain the object identifier for the attribute’s primary data object * \li Obtain the attribute’s name or index * \li Open the attribute * \li Get attribute dataspace and datatype (optional) * \li Specify the attribute’s memory type * \li Read and/or write the attribute data * \li Close the attribute * \li Close the primary data object (if appropriate) * * * * * * * * * * * *
CreateUpdate
* \snippet{lineno} H5A_examples.c create * * \snippet{lineno} H5A_examples.c update *
ReadDelete
* \snippet{lineno} H5A_examples.c read * * \snippet{lineno} H5A_examples.c delete *
* * \subsection subsec_attribute_work Working with Attributes * * \subsubsection subsubsec_attribute_work_struct The Structure of an Attribute * * An attribute has two parts: name and value(s). * * HDF5 attributes are sometimes discussed as name/value pairs in the form name=value. * * An attribute’s name is a null-terminated ASCII or UTF-8 character string. Each attribute attached to an * object has a unique name. * * The value portion of the attribute contains one or more data elements of the same datatype. * * HDF5 attributes have all the characteristics of HDF5 datasets except that there is no partial I/O * capability. In other words, attributes can be written and read only in full with no sub-setting. * * \subsubsection subsubsec_attribute_work_create Creating, Writing, and Reading Attributes * * If attributes are used in an HDF5 file, these functions will be employed: \ref H5Acreate, \ref H5Awrite, * and \ref H5Aread. \ref H5Acreate and \ref H5Awrite are used together to place the attribute in the file. If * an attribute is to be used and is not currently in memory, \ref H5Aread generally comes into play * usually in concert with one each of the H5Aget_* and H5Aopen_* functions. * * To create an attribute, call H5Acreate: * \code * hid_t H5Acreate (hid_t loc_id, const char *name, * hid_t type_id, hid_t space_id, hid_t create_plist, * hid_t access_plist) * \endcode * loc_id identifies the object (dataset, group, or committed datatype) to which the attribute is to be * attached. name, type_id, space_id, and create_plist convey, respectively, the attribute’s name, datatype, * dataspace, and attribute creation property list. The attribute’s name must be locally unique: it must be * unique within the context of the object to which it is attached. * * \ref H5Acreate creates the attribute in memory. The attribute does not exist in the file until * \ref H5Awrite writes it there. * * To write or read an attribute, call H5Awrite or H5Aread, respectively: * \code * herr_t H5Awrite (hid_t attr_id, hid_t mem_type_id, const void *buf) * herr_t H5Aread (hid_t attr_id, hid_t mem_type_id, void *buf) * \endcode * attr_id identifies the attribute while mem_type_id identifies the in-memory datatype of the attribute data. * * \ref H5Awrite writes the attribute data from the buffer buf to the file. \ref H5Aread reads attribute data * from the file into buf. * * The HDF5 Library converts the metadata between the in-memory datatype, mem_type_id, and the in-file * datatype, defined when the attribute was created, without user intervention. * * \subsubsection subsubsec_attribute_work_access Accessing Attributes by Name or Index * * Attributes can be accessed by name or index value. The use of an index value makes it possible to iterate * through all of the attributes associated with a given object. * * To access an attribute by its name, use the \ref H5Aopen_by_name function. \ref H5Aopen_by_name returns an * attribute identifier that can then be used by any function that must access an attribute such as \ref * H5Aread. Use the function \ref H5Aget_name to determine an attribute’s name. * * To access an attribute by its index value, use the \ref H5Aopen_by_idx function. To determine an attribute * index value when it is not already known, use the H5Oget_info function. \ref H5Aopen_by_idx is generally * used in the course of opening several attributes for later access. Use \ref H5Aiterate if the intent is to * perform the same operation on every attribute attached to an object. * * \subsubsection subsubsec_attribute_work_info Obtaining Information Regarding an Object’s Attributes * * In the course of working with HDF5 attributes, one may need to obtain any of several pieces of information: * \li An attribute name * \li The dataspace of an attribute * \li The datatype of an attribute * \li The number of attributes attached to an object * * To obtain an attribute’s name, call H5Aget_name with an attribute identifier, attr_id: * \code * ssize_t H5Aget_name (hid_t attr_id, size_t buf_size, char *buf) * \endcode * As with other attribute functions, attr_id identifies the attribute; buf_size defines the size of the * buffer; and buf is the buffer to which the attribute’s name will be read. * * If the length of the attribute name, and hence the value required for buf_size, is unknown, a first call * to \ref H5Aget_name will return that size. If the value of buf_size used in that first call is too small, * the name will simply be truncated in buf. A second \ref H5Aget_name call can then be used to retrieve the * name in an appropriately-sized buffer. * * To determine the dataspace or datatype of an attribute, call \ref H5Aget_space or \ref H5Aget_type, * respectively: \code hid_t H5Aget_space (hid_t attr_id) hid_t H5Aget_type (hid_t attr_id) \endcode \ref * H5Aget_space returns the dataspace identifier for the attribute attr_id. \ref H5Aget_type returns the * datatype identifier for the attribute attr_id. * * To determine the number of attributes attached to an object, use the \ref H5Oget_info function. The * function signature is below. \code herr_t H5Oget_info( hid_t object_id, H5O_info_t *object_info ) \endcode * The number of attributes will be returned in the object_info buffer. This is generally the preferred first * step in determining attribute index values. If the call returns N, the attributes attached to the object * object_id have index values of 0 through N-1. * * \subsubsection subsubsec_attribute_work_iterate Iterating across an Object’s Attributes * * It is sometimes useful to be able to perform the identical operation across all of the attributes attached * to an object. At the simplest level, you might just want to open each attribute. At a higher level, you * might wish to perform a rather complex operation on each attribute as you iterate across the set. * * To iterate an operation across the attributes attached to an object, one must make a series of calls to * \ref H5Aiterate * \code * herr_t H5Aiterate (hid_t obj_id, H5_index_t index_type, * H5_iter_order_t order, hsize_t *n, H5A_operator2_t op, * void *op_data) * \endcode * \ref H5Aiterate successively marches across all of the attributes attached to the object specified in * loc_id, performing the operation(s) specified in op_func with the data specified in op_data on each * attribute. * * When \ref H5Aiterate is called, index contains the index of the attribute to be accessed in this call. When * \ref H5Aiterate returns, index will contain the index of the next attribute. If the returned index is the * null pointer, then all attributes have been processed, and the iterative process is complete. * * op_func is a user-defined operation that adheres to the \ref H5A_operator_t prototype. This prototype and * certain requirements imposed on the operator’s behavior are described in the \ref H5Aiterate entry in the * \ref RM. * * op_data is also user-defined to meet the requirements of op_func. Beyond providing a parameter with which * to pass this data, HDF5 provides no tools for its management and imposes no restrictions. * * \subsubsection subsubsec_attribute_work_delete Deleting an Attribute * * Once an attribute has outlived its usefulness or is no longer appropriate, it may become necessary to * delete it. * * To delete an attribute, call \ref H5Adelete * \code * herr_t H5Adelete (hid_t loc_id, const char *name) * \endcode * \ref H5Adelete removes the attribute name from the group, dataset, or committed datatype specified in * loc_id. * * \ref H5Adelete must not be called if there are any open attribute identifiers on the object loc_id. Such a * call can cause the internal attribute indexes to change; future writes to an open attribute would then * produce unintended results. * * \subsubsection subsubsec_attribute_work_close Closing an Attribute * * As is the case with all HDF5 objects, once access to an attribute it is no longer needed, that attribute * must be closed. It is best practice to close it as soon as practicable; it is mandatory that it be closed * prior to the H5close call closing the HDF5 Library. * * To close an attribute, call \ref H5Aclose * \code * herr_t H5Aclose (hid_t attr_id) * \endcode * \ref H5Aclose closes the specified attribute by terminating access to its identifier, attr_id. * * \subsection subsec_attribute_special Special Issues * * Some special issues for attributes are discussed below. * *

Large Numbers of Attributes Stored in Dense Attribute Storage

* * The dense attribute storage scheme was added in version 1.8 so that datasets, groups, and committed * datatypes that have large numbers of attributes could be processed more quickly. * * Attributes start out being stored in an object's header. This is known as compact storage. For more * information, see "Storage Strategies." * * As the number of attributes grows, attribute-related performance slows. To improve performance, dense * attribute storage can be initiated with the H5Pset_attr_phase_change function. See the HDF5 Reference * Manual for more information. * * When dense attribute storage is enabled, a threshold is defined for the number of attributes kept in * compact storage. When the number is exceeded, the library moves all of the attributes into dense storage * at another location. The library handles the movement of attributes and the pointers between the locations * automatically. If some of the attributes are deleted so that the number falls below the threshold, then * the attributes are moved back to compact storage by the library. * * The improvements in performance from using dense attribute storage are the result of holding attributes * in a heap and indexing the heap with a B-tree. * * Note that there are some disadvantages to using dense attribute storage. One is that this is a new feature. * Datasets, groups, and committed datatypes that use dense storage cannot be read by applications built with * earlier versions of the library. Another disadvantage is that attributes in dense storage cannot be * compressed. * *

Large Attributes Stored in Dense Attribute Storage

* * We generally consider the maximum size of an attribute to be 64K bytes. The library has two ways of storing * attributes larger than 64K bytes: in dense attribute storage or in a separate dataset. Using dense * attribute storage is described in this section, and storing in a separate dataset is described in the next * section. * * To use dense attribute storage to store large attributes, set the number of attributes that will be stored * in compact storage to 0 with the H5Pset_attr_phase_change function. This will force all attributes to be * put into dense attribute storage and will avoid the 64KB size limitation for a single attribute in compact * attribute storage. * * The example code below illustrates how to create a large attribute that will be kept in dense storage. * * * * * * *
Create
* \snippet{lineno} H5A_examples.c create *
* *

Large Attributes Stored in a Separate Dataset

* * In addition to dense attribute storage (see above), a large attribute can be stored in a separate dataset. * In the figure below, DatasetA holds an attribute that is too large for the object header in Dataset1. By * putting a pointer to DatasetA as an attribute in Dataset1, the attribute becomes available to those * working with Dataset1. * This way of handling large attributes can be used in situations where backward compatibility is important * and where compression is important. Applications built with versions before 1.8.x can read large * attributes stored in separate datasets. Datasets can be compressed while attributes cannot. * * * * *
* \image html Shared_Attribute.jpg "A large or shared HDF5 attribute and its associated dataset(s)" *
* Note: In the figure above, DatasetA is an attribute of Dataset1 that is too large to store in Dataset1's * header. DatasetA is associated with Dataset1 by means of an object reference pointer attached as an * attribute to Dataset1. The attribute in DatasetA can be shared among multiple datasets by means of * additional object reference pointers attached to additional datasets. * *

Shared Attributes

* * Attributes written and managed through the \ref H5A interface cannot be shared. If shared attributes are * required, they must be handled in the manner described above for large attributes and illustrated in * the figure above. * *

Attribute Names

* * While any ASCII or UTF-8 character may be used in the name given to an attribute, it is usually wise * to avoid the following kinds of characters: * \li Commonly used separators or delimiters such as slash, backslash, colon, and semi-colon (\, /, :, ;) * \li Escape characters * \li Wild cards such as asterisk and question mark (*, ?) * NULL can be used within a name, but HDF5 names are terminated with a NULL: whatever comes after the NULL * will be ignored by HDF5. * * The use of ASCII or UTF-8 characters is determined by the character encoding property. See * #H5Pset_char_encoding in the \ref RM. * *

No Special I/O or Storage

* * HDF5 attributes have all the characteristics of HDF5 datasets except the following: * \li Attributes are written and read only in full: there is no provision for partial I/O or sub-setting * \li No special storage capability is provided for attributes: there is no compression or chunking, and * attributes are not extendable * * Previous Chapter \ref sec_dataspace - Next Chapter \ref sec_error * * \defgroup H5A Attributes (H5A) * * An HDF5 attribute is a small metadata object describing the nature and/or intended usage of a primary data * object. A primary data object may be a dataset, group, or committed datatype. * * @see sec_attribute * */ #endif /* H5Amodule_H */