![]() |
HDF5
1.15.0.fb38a8b
API Reference
|
Data Structures | |
struct | H5FD_subfiling_params_t |
Subfiling parameter structure that is shared between the H5FD_SUBFILING and H5FD_IOC drivers. More... | |
struct | H5FD_subfiling_config_t |
Configuration structure for H5Pset_fapl_subfiling() / H5Pget_fapl_subfiling() More... | |
Macros | |
#define | H5FD_SUBFILING (H5FDperform_init(H5FD_subfiling_init)) |
#define | H5FD_SUBFILING_NAME "subfiling" |
#define | H5FD_SUBFILING_CURR_FAPL_VERSION 1 |
#define | H5FD_SUBFILING_FAPL_MAGIC 0xFED01331 |
#define | H5FD_SUBFILING_DEFAULT_STRIPE_SIZE (32 * 1024 * 1024) |
#define | H5FD_SUBFILING_DEFAULT_STRIPE_COUNT -1 |
#define | H5FD_SUBFILING_FILENAME_TEMPLATE "%s.subfile_%" PRIu64 "_%0*d_of_%d" |
#define | H5FD_SUBFILING_CONFIG_FILENAME_TEMPLATE "%s.subfile_%" PRIu64 ".config" |
#define | H5FD_SUBFILING_STRIPE_SIZE "H5FD_SUBFILING_STRIPE_SIZE" |
#define | H5FD_SUBFILING_IOC_PER_NODE "H5FD_SUBFILING_IOC_PER_NODE" |
#define | H5FD_SUBFILING_IOC_SELECTION_CRITERIA "H5FD_SUBFILING_IOC_SELECTION_CRITERIA" |
#define | H5FD_SUBFILING_SUBFILE_PREFIX "H5FD_SUBFILING_SUBFILE_PREFIX" |
#define | H5FD_SUBFILING_CONFIG_FILE_PREFIX "H5FD_SUBFILING_CONFIG_FILE_PREFIX" |
Enumerations | |
enum | H5FD_subfiling_ioc_select_t { SELECT_IOC_ONE_PER_NODE = 0 , SELECT_IOC_EVERY_NTH_RANK , SELECT_IOC_WITH_CONFIG , SELECT_IOC_TOTAL , ioc_selection_options } |
Functions | |
H5_DLL hid_t | H5FD_subfiling_init (void) |
Internal routine to initialize H5FD_SUBFILING driver. Not meant to be called directly by an HDF5 application. More... | |
H5_DLL herr_t | H5Pset_fapl_subfiling (hid_t fapl_id, const H5FD_subfiling_config_t *vfd_config) |
Modifies the specified File Access Property List to use the H5FD_SUBFILING driver. More... | |
H5_DLL herr_t | H5Pget_fapl_subfiling (hid_t fapl_id, H5FD_subfiling_config_t *config_out) |
Queries a File Access Property List for H5FD_SUBFILING file driver properties. More... | |
#define H5FD_SUBFILING (H5FDperform_init(H5FD_subfiling_init)) |
Macro that returns the identifier for the H5FD_SUBFILING driver. Returns a file driver identifier if successful; otherwise returns H5I_INVALID_HID.
#define H5FD_SUBFILING_CONFIG_FILE_PREFIX "H5FD_SUBFILING_CONFIG_FILE_PREFIX" |
Macro for name of the environment variable that specifies a prefix to apply to the subfiling configuration filename. Useful for cases where the application wants to place the configuration file in a different directory than the default of putting it alongside the generated subfiles. For example, when writing to node-local storage one may wish to place the configuration file on a scratch file system readable by all nodes, while the subfiles are initially written to the node-local storage.
The value set for this environment variable is interpreted as a pathname that must already exist.
NOTE: As this prefix string will be encoded in the driver info message that gets written to the file, there is an upper limit of about ~900 single-byte characters for this string, though possibly less due to other information the driver may encode. Avoid long prefix names where possible.
#define H5FD_SUBFILING_CONFIG_FILENAME_TEMPLATE "%s.subfile_%" PRIu64 ".config" |
The basic template for a H5FD_SUBFILING driver configuration filename. The format specifiers correspond to:
s -> base filename, e.g. "file.h5" PRIu64 -> file inode, e.g. 11273556
yielding a filename such as:
file.h5.subfile_11273556.config
#define H5FD_SUBFILING_CURR_FAPL_VERSION 1 |
The version number of the H5FD_subfiling_config_t configuration structure for the H5FD_SUBFILING driver
#define H5FD_SUBFILING_DEFAULT_STRIPE_COUNT -1 |
Macro for the default Subfiling stripe count value. The default is currently to use one subfile per node.
#define H5FD_SUBFILING_DEFAULT_STRIPE_SIZE (32 * 1024 * 1024) |
The default stripe size (in bytes) for data stripes in subfiles
#define H5FD_SUBFILING_FAPL_MAGIC 0xFED01331 |
Unique number used to distinguish the H5FD_SUBFILING driver from other HDF5 file drivers
#define H5FD_SUBFILING_FILENAME_TEMPLATE "%s.subfile_%" PRIu64 "_%0*d_of_%d" |
The basic template for a subfile filename. The format specifiers correspond to:
s -> base filename, e.g. "file.h5" PRIu64 -> file inode, e.g. 11273556 %0*d -> number (starting at 1) signifying the Nth (out of total number of subfiles) subfile. Zero-padded according to the number of digits in the number of subfiles (calculated by log10(num_subfiles) + 1) d -> number of subfiles
yielding filenames such as:
file.h5.subfile_11273556_01_of_10 file.h5.subfile_11273556_02_of_10 file.h5.subfile_11273556_10_of_10
#define H5FD_SUBFILING_IOC_PER_NODE "H5FD_SUBFILING_IOC_PER_NODE" |
Macro for name of the environment variable that specifies the number of MPI ranks per node to use as I/O concentrators
The value set for this environment variable is interpreted as a long value and must be > 0.
#define H5FD_SUBFILING_IOC_SELECTION_CRITERIA "H5FD_SUBFILING_IOC_SELECTION_CRITERIA" |
Macro for name of the environment variable that provides information for selection MPI ranks as I/O concentrators
The value set for this environment variable is interpreted differently, depending on the IOC selection method chosen.
For SELECT_IOC_ONE_PER_NODE, this value is ignored.
For SELECT_IOC_EVERY_NTH_RANK, this value is interpreted as a long value and must be > 0. The value will correspond to the N
value when selecting every N
-th MPI rank as an I/O concentrator.
For SELECT_IOC_WITH_CONFIG, this value is ignored as that particular IOC selection method is not currently supported.
For SELECT_IOC_TOTAL, this value is interpreted as a long value and must be > 0. The value will correspond to the total number of I/O concentrators to be used.
#define H5FD_SUBFILING_NAME "subfiling" |
The canonical name for the H5FD_SUBFILING driver
#define H5FD_SUBFILING_STRIPE_SIZE "H5FD_SUBFILING_STRIPE_SIZE" |
Macro for name of the environment variable that specifies the size (in bytes) for data stripes in subfiles
The value set for this environment variable is interpreted as a long long value and must be > 0.
#define H5FD_SUBFILING_SUBFILE_PREFIX "H5FD_SUBFILING_SUBFILE_PREFIX" |
Macro for name of the environment variable that specifies a prefix to apply to the filenames generated for subfiles
The value set for this environment variable is interpreted as a pathname.
This enum defines the various constants to allow different allocations of MPI ranks as I/O concentrators.
Enumerator | |
---|---|
SELECT_IOC_ONE_PER_NODE | Default selection method. One MPI rank per node is used as an I/O concentrator. If this selection method is used, the number of I/O concentrators per node can be adjusted with the H5FD_SUBFILING_IOC_PER_NODE environment variable. |
SELECT_IOC_EVERY_NTH_RANK | Starting with MPI rank 0, a stride of 'N' is applied to the MPI rank values to determine the next I/O concentrator. The H5FD_SUBFILING_IOC_SELECTION_CRITERIA environment variable must be set to the value desired for 'N'. |
SELECT_IOC_WITH_CONFIG | Currently unsupported. Use a configuration file to determine the mapping from MPI ranks to I/O concentrators. The H5FD_SUBFILING_IOC_SELECTION_CRITERIA environment variable must be set to the path to the configuration file. |
SELECT_IOC_TOTAL | Specifies that a total of 'N' I/O concentrators should be used. Starting with MPI rank 0, a stride of 'MPI comm size' / 'N' is applied to the MPI rank values to determine the next I/O concentrator. The H5FD_SUBFILING_IOC_SELECTION_CRITERIA environment variable must be set to the value desired for 'N'. |
ioc_selection_options | Unused. Sentinel value |
H5_DLL hid_t H5FD_subfiling_init | ( | void | ) |
Internal routine to initialize H5FD_SUBFILING driver. Not meant to be called directly by an HDF5 application.