/* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
* Copyright by The HDF Group. *
* All rights reserved. *
* *
* This file is part of HDF5. The full HDF5 copyright notice, including *
* terms governing use, modification, and redistribution, is contained in *
* the COPYING file, which can be found at the root of the source code *
* distribution tree, or in https://www.hdfgroup.org/licenses. *
* If you do not have access to either file, you may request a copy from *
* help@hdfgroup.org. *
* * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * */
/*
* Programmer: Quincey Koziol
* Saturday, September 12, 2015
*
* Purpose: This file contains declarations which define macros for the
* H5F package. Including this header means that the source file
* is part of the H5F package.
*/
#ifndef H5Fmodule_H
#define H5Fmodule_H
/* Define the proper control macros for the generic FUNC_ENTER/LEAVE and error
* reporting macros.
*/
#define H5F_MODULE
#define H5_MY_PKG H5F
#define H5_MY_PKG_ERR H5E_FILE
#define H5_MY_PKG_INIT YES
/** \page H5F_UG The HDF5 File
*
* \section sec_file The HDF5 File
* \subsection subsec_file_intro Introduction
* The purpose of this chapter is to describe how to work with HDF5 data files.
*
* If HDF5 data is to be written to or read from a file, the file must first be explicitly created or
* opened with the appropriate file driver and access privileges. Once all work with the file is
* complete, the file must be explicitly closed.
*
* This chapter discusses the following:
* \li File access modes
* \li Creating, opening, and closing files
* \li The use of file creation property lists
* \li The use of file access property lists
* \li The use of low-level file drivers
*
* This chapter assumes an understanding of the material presented in the data model chapter. For
* more information, @see @ref sec_data_model.
*
* \subsection subsec_file_access_modes File Access Modes
* There are two issues regarding file access:
*
* Access flags and modes
*
* Access Flag |
* Resulting Access Mode |
*
*
* #H5F_ACC_EXCL |
* If the file already exists, #H5Fcreate fails. If the file does not exist,
* it is created and opened with read-write access. (Default) |
*
*
* #H5F_ACC_TRUNC |
* If the file already exists, the file is opened with read-write access,
* and new data will overwrite any existing data. If the file does not exist,
* it is created and opened with read-write access. |
*
*
* #H5F_ACC_RDONLY |
* An existing file is opened with read-only access. If the file does not
* exist, #H5Fopen fails. (Default) |
*
*
* #H5F_ACC_RDWR |
* An existing file is opened with read-write access. If the file does not
* exist, #H5Fopen fails. |
*
*
*
* By default, #H5Fopen opens a file for read-only access; passing #H5F_ACC_RDWR allows
* read-write access to the file.
*
* By default, #H5Fcreate fails if the file already exists; only passing #H5F_ACC_TRUNC allows
* the truncating of an existing file.
*
* \subsection subsec_file_creation_access File Creation and File Access Properties
* File creation and file access property lists control the more complex aspects of creating and
* accessing files.
*
* File creation property lists control the characteristics of a file such as the size of the userblock,
* a user-definable data block; the size of data address parameters; properties of the B-trees that are
* used to manage the data in the file; and certain HDF5 Library versioning information.
*
* For more information, @see @ref subsubsec_file_property_lists_props.
*
* This section has a more detailed discussion of file creation properties. If you have no special
* requirements for these file characteristics, you can simply specify #H5P_DEFAULT for the default
* file creation property list when a file creation property list is called for.
*
* File access property lists control properties and means of accessing a file such as data alignment
* characteristics, metadata block and cache sizes, data sieve buffer size, garbage collection
* settings, and parallel I/O. Data alignment, metadata block and cache sizes, and data sieve buffer
* size are factors in improving I/O performance.
*
* For more information, @see @ref subsubsec_file_property_lists_access.
*
* This section has a more detailed discussion of file access properties. If you have no special
* requirements for these file access characteristics, you can simply specify #H5P_DEFAULT for the
* default file access property list when a file access property list is called for.
*
*
* General library functions and macros
*
* Function |
* Purpose |
*
*
* #H5check_version |
* Verifies that HDF5 library versions are consistent. |
*
*
* #H5close |
* Flushes all data to disk, closes all open identifiers, and cleans up memory. |
*
*
* #H5dont_atexit |
* Instructs the library not to install the atexit cleanup routine. |
*
*
* #H5garbage_collect |
* Garbage collects on all free-lists of all types. |
*
*
* #H5get_libversion |
* Returns the HDF library release number. |
*
*
* #H5open |
* Initializes the HDF5 library. |
*
*
* #H5set_free_list_limits |
* Sets free-list size limits. |
*
*
* #H5_VERSION_GE |
* Determines whether the version of the library being used is greater than or equal
* to the specified version. |
*
*
* #H5_VERSION_LE |
* Determines whether the version of the library being used is less than or equal
* to the specified version. |
*
*
*
*
* File functions
*
* Function |
* Purpose |
*
*
* #H5Fclear_elink_file_cache |
* Clears the external link open file cache for a file. |
*
*
* #H5Fclose |
* Closes HDF5 file. |
*
*
* #H5Fcreate |
* Creates new HDF5 file. |
*
*
* #H5Fflush |
* Flushes data to HDF5 file on storage medium. |
*
*
* #H5Fget_access_plist |
* Returns a file access property list identifier. |
*
*
* #H5Fget_create_plist |
* Returns a file creation property list identifier. |
*
*
* #H5Fget_file_image |
* Retrieves a copy of the image of an existing, open file. |
*
*
* #H5Fget_filesize |
* Returns the size of an HDF5 file. |
*
*
* #H5Fget_freespace |
* Returns the amount of free space in a file. |
*
*
* #H5Fget_info |
* Returns global information for a file. |
*
*
* #H5Fget_intent |
* Determines the read/write or read-only status of a file. |
*
*
* #H5Fget_mdc_config |
* Obtain current metadata cache configuration for target file. |
*
*
* #H5Fget_mdc_hit_rate |
* Obtain target file’s metadata cache hit rate. |
*
*
* #H5Fget_mdc_size |
* Obtain current metadata cache size data for specified file. |
*
*
* #H5Fget_mpi_atomicity |
* Retrieves the atomicity mode in use. |
*
*
* #H5Fget_name |
* Retrieves the name of the file to which the object belongs. |
*
*
* #H5Fget_obj_count |
* Returns the number of open object identifiers for an open file. |
*
*
* #H5Fget_obj_ids |
* Returns a list of open object identifiers. |
*
*
* #H5Fget_vfd_handle |
* Returns pointer to the file handle from the virtual file driver. |
*
*
* #H5Fis_hdf5 |
* Determines whether a file is in the HDF5 format. |
*
*
* #H5Fmount |
* Mounts a file. |
*
*
* #H5Fopen |
* Opens an existing HDF5 file. |
*
*
* #H5Freopen |
* Returns a new identifier for a previously-opened HDF5 file. |
*
*
* #H5Freset_mdc_hit_rate_stats |
* Reset hit rate statistics counters for the target file. |
*
*
* #H5Fset_mdc_config |
* Use to configure metadata cache of target file. |
*
*
* #H5Fset_mpi_atomicity |
* Use to set the MPI atomicity mode. |
*
*
* #H5Funmount |
* Unmounts a file. |
*
*
*
*
* File creation property list functions
*
* Function |
* Purpose |
*
*
* #H5Pset_userblock/#H5Pget_userblock |
* Sets/retrieves size of userblock. |
*
*
* #H5Pset_sizes/#H5Pget_sizes |
* Sets/retrieves byte size of offsets and lengths used to address objects in HDF5 file. |
*
*
* #H5Pset_sym_k/#H5Pget_sym_k |
* Sets/retrieves size of parameters used to control symbol table nodes. |
*
*
* #H5Pset_istore_k/#H5Pget_istore_k |
* Sets/retrieves size of parameter used to control B-trees for indexing chunked datasets. |
*
*
* #H5Pset_file_image |
* Sets an initial file image in a memory buffer. |
*
*
* #H5Pget_file_image |
* Retrieves a copy of the file image designated as the initial content and structure of a file. |
*
*
* #H5Pset_shared_mesg_nindexes/#H5Pget_shared_mesg_nindexes |
* Sets or retrieves number of shared object header message indexes in file
* creation property list. |
*
*
* #H5Pset_shared_mesg_index |
* Configures the specified shared object header message index. |
*
*
* #H5Pget_shared_mesg_index |
* Retrieves the configuration settings for a shared message index. |
*
*
* #H5Pset_shared_mesg_phase_change/#H5Pget_shared_mesg_phase_change |
* Sets or retrieves shared object header message storage phase change thresholds. |
*
*
* #H5Pget_version |
* |
*
*
*
*
* File access property list functions
*
* Function |
* Purpose |
*
*
* #H5Pset_alignment/#H5Pget_alignment |
* Sets/retrieves alignment properties. |
*
*
* #H5Pset_cache/#H5Pget_cache |
* Sets/retrieves metadata cache and raw data chunk cache parameters. |
*
*
* #H5Pset_elink_file_cache_size/#H5Pget_elink_file_cache_size |
* Sets/retrieves the size of the external link open file cache from the specified
* file access property list. |
*
*
* #H5Pset_gc_references/#H5Pget_gc_references |
* Sets/retrieves garbage collecting references flag. |
*
*
* #H5Pset_family_offset |
* Sets offset property for low-level access to a file in a family of files. |
*
*
* #H5Pget_family_offset |
* Retrieves a data offset from the file access property list. |
*
*
* #H5Pset_meta_block_size/#H5Pget_meta_block_size |
* Sets the minimum metadata blocksize or retrieves the current metadata block size setting. |
*
*
* #H5Pset_mdc_config |
* Set the initial metadata cache configuration in the indicated File Access Property List
* to the supplied value. |
*
*
* #H5Pget_mdc_config |
* Get the current initial metadata cache config-uration from the indicated File Access
* Property List. |
*
*
* #H5Pset_sieve_buf_size/#H5Pget_sieve_buf_size |
* Sets/retrieves maximum size of data sieve buffer. |
*
*
* #H5Pset_libver_bounds |
* Sets bounds on library versions, and indirectly format versions, to be used
* when creating objects. |
*
*
* #H5Pget_libver_bounds |
* Retrieves library version bounds settings that indirectly control the format
* versions used when creating objects. |
*
*
* #H5Pset_small_data_block_size |
* Sets the size of a contiguous block reserved for small data. |
*
*
* #H5Pget_small_data_block_size |
* Retrieves the current small data block size setting. |
*
*
*
*
* File driver functions
*
* Function |
* Purpose |
*
*
* #H5Pset_driver |
* Sets a file driver. |
*
*
* #H5Pget_driver |
* Returns the identifier for the driver used to create a file. |
*
*
* #H5Pget_driver_info |
* Returns a pointer to file driver information. |
*
*
* #H5Pset_fapl_core/#H5Pget_fapl_core |
* Sets the driver for buffered memory files (in RAM) or retrieves information regarding
* the driver. |
*
*
* #H5Pset_fapl_direct/#H5Pget_fapl_direct |
* Sets up use of the direct I/O driver or retrieves the direct I/O driver settings. |
*
*
* #H5Pset_fapl_family/#H5Pget_fapl_family |
* Sets driver for file families, designed for systems that do not support files
* larger than 2 gigabytes, or retrieves information regarding driver. |
*
*
* #H5Pset_fapl_log |
* Sets logging driver. |
*
*
* #H5Pset_fapl_mpio/#H5Pget_fapl_mpio |
* Sets driver for files on parallel file systems (MPI I/O) or retrieves information
* regarding the driver. |
*
*
* H5Pset_fapl_mpiposix/H5Pget_fapl_mpiposix |
* No longer available. |
*
*
* #H5Pset_fapl_multi/#H5Pget_fapl_multi |
* Sets driver for multiple files, separating categories of metadata and raw data,
* or retrieves information regarding driver. |
*
*
* #H5Pset_fapl_sec2 |
* Sets driver for unbuffered permanent files or retrieves information regarding driver. |
*
*
* #H5Pset_fapl_split |
* Sets driver for split files, a limited case of multiple files with one metadata file
* and one raw data file. |
*
*
* #H5Pset_fapl_stdio |
* Sets driver for buffered permanent files. |
*
*
* #H5Pset_fapl_windows |
* Sets the Windows I/O driver. |
*
*
* #H5Pset_multi_type |
* Specifies type of data to be accessed via the MULTI driver enabling more direct access. |
*
*
* #H5Pget_multi_type |
* Retrieves type of data property for MULTI driver. |
*
*
*
* \subsection subsec_file_create Creating or Opening an HDF5 File
* This section describes in more detail how to create and how to open files.
*
* New HDF5 files are created and opened with #H5Fcreate; existing files are opened with
* #H5Fopen. Both functions return an object identifier which must eventually be released by calling
* #H5Fclose.
*
* To create a new file, call #H5Fcreate:
* \code
* hid_t H5Fcreate (const char *name, unsigned flags, hid_t fcpl_id, hid_t fapl_id)
* \endcode
*
* #H5Fcreate creates a new file named name in the current directory. The file is opened with read
* and write access; if the #H5F_ACC_TRUNC flag is set, any pre-existing file of the same name in
* the same directory is truncated. If #H5F_ACC_TRUNC is not set or #H5F_ACC_EXCL is set and
* if a file of the same name exists, #H5Fcreate will fail.
*
* The new file is created with the properties specified in the property lists fcpl_id and fapl_id.
* fcpl is short for file creation property list. fapl is short for file access property list. Specifying
* #H5P_DEFAULT for either the creation or access property list will use the library’s default
* creation or access properties.
*
* If #H5Fcreate successfully creates the file, it returns a file identifier for the new file. This
* identifier will be used by the application any time an object identifier, an OID, for the file is
* required. Once the application has finished working with a file, the identifier should be released
* and the file closed with #H5Fclose.
*
* To open an existing file, call #H5Fopen:
* \code
* hid_t H5Fopen (const char *name, unsigned flags, hid_t fapl_id)
* \endcode
*
* #H5Fopen opens an existing file with read-write access if #H5F_ACC_RDWR is set and read-only
* access if #H5F_ACC_RDONLY is set.
*
* fapl_id is the file access property list identifier. Alternatively, #H5P_DEFAULT indicates that the
* application relies on the default I/O access parameters. Creating and changing access property
* lists is documented further below.
*
* A file can be opened more than once via multiple #H5Fopen calls. Each such call returns a unique
* file identifier and the file can be accessed through any of these file identifiers as long as they
* remain valid. Each of these file identifiers must be released by calling #H5Fclose when it is no
* longer needed.
*
* For more information, @see @ref subsubsec_file_property_lists_access.
* For more information, @see @ref subsec_file_property_lists.
*
* \subsection subsec_file_closes Closing an HDF5 File
* #H5Fclose both closes a file and releases the file identifier returned by #H5Fopen or #H5Fcreate.
* #H5Fclose must be called when an application is done working with a file; while the HDF5
* Library makes every effort to maintain file integrity, failure to call #H5Fclose may result in the
* file being abandoned in an incomplete or corrupted state.
*
* To close a file, call #H5Fclose:
* \code
* herr_t H5Fclose (hid_t file_id)
* \endcode
* This function releases resources associated with an open file. After closing a file, the file
* identifier, file_id, cannot be used again as it will be undefined.
*
* #H5Fclose fulfills three purposes: to ensure that the file is left in an uncorrupted state, to ensure
* that all data has been written to the file, and to release resources. Use #H5Fflush if you wish to
* ensure that all data has been written to the file but it is premature to close it.
*
* Note regarding serial mode behavior: When #H5Fclose is called in serial mode, it closes the file
* and terminates new access to it, but it does not terminate access to objects that remain
* individually open within the file. That is, if #H5Fclose is called for a file but one or more objects
* within the file remain open, those objects will remain accessible until they are individually
* closed. To illustrate, assume that a file, fileA, contains a dataset, data_setA, and that both are
* open when #H5Fclose is called for fileA. data_setA will remain open and accessible, including
* writable, until it is explicitly closed. The file will be automatically and finally closed once all
* objects within it have been closed.
*
* Note regarding parallel mode behavior: Once #H5Fclose has been called in parallel mode, access
* is no longer available to any object within the file.
*
* \subsection subsec_file_property_lists File Property Lists
* Additional information regarding file structure and access are passed to #H5Fcreate and
* #H5Fopen through property list objects. Property lists provide a portable and extensible method of
* modifying file properties via simple API functions. There are two kinds of file-related property
* lists:
* \li File creation property lists
* \li File access property lists
*
* In the following sub-sections, we discuss only one file creation property, userblock size, in detail
* as a model for the user. Other file creation and file access properties are mentioned and defined
* briefly, but the model is not expanded for each; complete syntax, parameter, and usage
* information for every property list function is provided in the \ref H5P
* section of the HDF5 Reference Manual.
*
* For more information, @see @ref sec_plist.
*
* \subsubsection subsubsec_file_property_lists_create Creating a Property List
* If you do not wish to rely on the default file creation and access properties, you must first create
* a property list with #H5Pcreate.
* \code
* hid_t H5Pcreate (hid_t cls_id)
* \endcode
* cls_id is the type of property list being created. In this case, the appropriate values are
* #H5P_FILE_CREATE for a file creation property list and #H5P_FILE_ACCESS for a file access
* property list.
*
* Thus, the following calls create a file creation property list and a file access property list with
* identifiers fcpl_id and fapl_id, respectively:
* \code
* fcpl_id = H5Pcreate (H5P_FILE_CREATE)
* fapl_id = H5Pcreate (H5P_FILE_ACCESS)
* \endcode
*
* Once the property lists have been created, the properties themselves can be modified via the
* functions described in the following sub-sections.
*
* \subsubsection subsubsec_file_property_lists_props File Creation Properties
* File creation property lists control the file metadata, which is maintained in the superblock of the
* file. These properties are used only when a file is first created.
*
*
* Supported file drivers
*
* Driver Name |
* Driver Identifier |
* Description |
* Related API |
*
*
* POSIX |
* #H5FD_SEC2 |
* This driver uses POSIX file-system functions like read and write to perform I/O to a single,
* permanent file on local disk with no system buffering. This driver is POSIX-compliant and is
* the default file driver for all systems. |
* #H5Pset_fapl_sec2 |
*
*
* Direct |
* #H5FD_DIRECT |
* This is the #H5FD_SEC2 driver except data is written to or read from the file
* synchronously without being cached by the system. |
* #H5Pset_fapl_direct |
*
*
* Log |
* #H5FD_LOG |
* This is the #H5FD_SEC2 driver with logging capabilities. |
* #H5Pset_fapl_log |
*
*
* Windows |
* #H5FD_WINDOWS |
* This driver was modified in HDF5-1.8.8 to be a wrapper of the POSIX driver,
* #H5FD_SEC2. This change should not affect user applications. |
* #H5Pset_fapl_windows |
*
*
* STDIO |
* #H5FD_STDIO |
* This driver uses functions from the standard C stdio.h to perform I/O
* to a single, permanent file on local disk with additional system buffering. |
* #H5Pset_fapl_stdio |
*
*
* Memory |
* #H5FD_CORE |
* With this driver, an application can work with a file in memory for faster reads and
* writes. File contents are kept in memory until the file is closed. At closing, the memory
* version of the file can be written back to disk or abandoned. |
* #H5Pset_fapl_core |
*
*
* Family |
* #H5FD_FAMILY |
* With this driver, the HDF5 file’s address space is partitioned into pieces and sent to
* separate storage files using an underlying driver of the user’s choice. This driver is for
* systems that do not support files larger than 2 gigabytes. |
* #H5Pset_fapl_family |
*
*
* Multi |
* #H5FD_MULTI |
* With this driver, data can be stored in multiple files according to the type of the data.
* I/O might work better if data is stored in separate files based on the type of data. The Split
* driver is a special case of this driver. |
* #H5Pset_fapl_multi |
*
*
* Split |
* H5FD_SPLIT |
* This file driver splits a file into two parts. One part stores metadata, and the other part
* stores raw data. This splitting a file into two parts is a limited case of the Multi driver. |
* #H5Pset_fapl_split |
*
*
* Parallel |
* #H5FD_MPIO |
* This is the standard HDF5 file driver for parallel file systems. This driver uses the MPI
* standard for both communication and file I/O. |
* #H5Pset_fapl_mpio |
*
*
* Parallel POSIX |
* H5FD_MPIPOSIX |
* This driver is no longer available |
* |
*
*
* Stream |
* H5FD_STREAM |
* This driver is no longer available. |
* |
*
*
*
* For more information, see the HDF5 Reference Manual entries for the function calls shown in
* the column on the right in the table above.
*
* Note that the low-level file drivers manage alternative file storage layouts. Dataset storage
* layouts (chunking, compression, and external dataset storage) are managed independently of file
* storage layouts.
*
* If an application requires a special-purpose low-level driver, the VFL provides a public API for
* creating one. For more information on how to create a driver,
* @see @ref VFL in the HDF5 Technical Notes.
*
* \subsubsection subsubsec_file_alternate_drivers_id Identifying the Previously‐used File Driver
* When creating a new HDF5 file, no history exists, so the file driver must be specified if it is to be
* other than the default.
*
* When opening existing files, however, the application may need to determine which low-level
* driver was used to create the file. The function #H5Pget_driver is used for this purpose. See the
* example below.
*
*