diff options
Diffstat (limited to 'doxygen/examples/H5.format.1.1.html')
| -rw-r--r-- | doxygen/examples/H5.format.1.1.html | 6439 |
1 files changed, 6439 insertions, 0 deletions
diff --git a/doxygen/examples/H5.format.1.1.html b/doxygen/examples/H5.format.1.1.html new file mode 100644 index 0000000..ebbbe8e --- /dev/null +++ b/doxygen/examples/H5.format.1.1.html @@ -0,0 +1,6439 @@ +<html> + <head> + <title> + HDF5 File Format Specification Version 1.1 + </title> + +<STYLE TYPE="text/css"> + +P { text-indent: 2em} +P.item { margin-left: 2em; text-indent: -2em} +P.item2 { margin-left: 2em; text-indent: 2em} + +TABLE.format { border:solid; border-collapse:collapse; caption-side:top; text-align:center; width:80%;} +TABLE.format TH { border:ridge; padding:4px; width:25%;} +TABLE.format TD { border:ridge; padding:4px; } +TABLE.format CAPTION { font-weight:bold; font-size:larger;} + +TABLE.note {border:none; text-align:right; width:80%;} + +TABLE.desc { border:solid; border-collapse:collapse; caption-size:top; text-align:left; width:80%;} +TABLE.desc TR { vertical-align:top;} +TABLE.desc TH { border-style:ridge; font-size:larger; padding:4px; text-decoration:underline;} +TABLE.desc TD { border-style:ridge; padding:4px; } +TABLE.desc CAPTION { font-weight:bold; font-size:larger;} + +TABLE.list { border:none; } +TABLE.list TR { vertical-align:top;} +TABLE.list TH { border:none; text-decoration:underline;} +TABLE.list TD { border:none; } + +</STYLE> +</head> + <body> + + <center> + <table border=0 width=90%> + <tr> + <td valign=top> + <ol type=I> + <li><a href="#Intro">Introduction</a> + <li><a href="#FileMetaData">Disk Format Level 0 - File Metadata</a> + <font size=-2> + <ol type=A> + <li><a href="#SuperBlock">Disk Format Level 0A - File Signature and Super Block</a> + <li><a href="#DriverInfo">Disk Format Level 0B - File Driver Info</a> + </ol> + </font> + <li><a href="#FileInfra">Disk Format Level 1 - File Infrastructure</a> + <font size=-2> + <ol type=A> + <li><a href="#Btrees">Disk Format Level 1A - B-link Trees and B-tree Nodes</a> + <li><a href="#SymbolTable">Disk Format Level 1B - Group</a> + <li><a href="#SymbolTableEntry">Disk Format Level 1C - Group Entry</a> + <li><a href="#LocalHeap">Disk Format Level 1D - Local Heaps</a> + <li><a href="#GlobalHeap">Disk Format Level 1E - Global Heap</a> + <li><a href="#FreeSpaceIndex">Disk Format Level 1F - Free-space Index</a> + </ol> + </font> + <li><a href="#DataObject">Disk Format Level 2 - Data Objects</a> + <font size=-2> + <ol type=A> + <li><a href="#ObjectHeader">Disk Format Level 2a - Data Object Headers</a> + <ol type=1> + <li><a href="#NILMessage">Name: NIL</a> <!-- 0x0000 --> + <li><a href="#SimpleDataSpace">Name: Simple Dataspace</a> <!-- 0x0001 --> +<!-- <li><a href="#DataSpaceMessage">Name: Complex Dataspace</a> --> <!-- 0x0002 --> + <li><a href="#ReservedMessage_0002">Name: Reserved - not assigned yet</a> <!-- 0x0002 --> + <li><a href="#DataTypeMessage">Name: Datatype</a> <!-- 0x0003 --> + <li><a href="#OldFillValueMessage">Name: Data Storage - Fill Value (Old)</a> <!-- 0x0004 --> + <li><a href="#FillValueMessage">Name: Data Storage - Fill Value</a> <!-- 0x0005 --> + </ol> + </ol> + </font> + </ol> + </td><td> </td><td valign=top> + <ol type=I start=4> + + <li><a href="#DataObject">Disk Format Level 2 - Data Objects</a> + <font size=-2><i>(Continued)</i> + <ol type=A> + <li><a href="#ObjectHeader">Disk Format Level 2a - Data Object Headers</a><i>(Continued)</i> + <ol type=1 start=6> +<!-- <li><a href="#CompactDataStorageMessage">Name: Data Storage - Compact</a> --> <!-- 0x0006 --> + <li><a href="#ReservedMessage_0006">Name: Reserved - not assigned yet</a> <!-- 0x0006 --> + <li><a href="#ExternalFileListMessage">Name: Data Storage - External Data Files</a> <!-- 0x0007 --> + <li><a href="#LayoutMessage">Name: Data Storage - Layout</a> <!-- 0x0008 --> + <li><a href="#ReservedMessage_0009">Name: Reserved - not assigned yet</a> <!-- 0x0009 --> + <li><a href="#ReservedMessage_000A">Name: Reserved - not assigned yet</a> <!-- 0x000a --> + <li><a href="#FilterMessage">Name: Data Storage - Filter Pipeline</a> <!-- 0x000b --> + <li><a href="#AttributeMessage">Name: Attribute</a> <!-- 0x000c --> + <li><a href="#CommentMessage">Name: Object Comment</a> <!-- 0x000d --> + <li><a href="#OldModifiedMessage">Name: Object Modification Date and Time (Old)</a> <!-- 0x000e --> + <li><a href="#SharedMessage">Name: Shared Object Message</a> <!-- 0x000f --> + <li><a href="#ContinuationMessage">Name: Object Header Continuation</a> <!-- 0x0010 --> + <li><a href="#SymbolTableMessage">Name: Group Message</a> <!-- 0x0011 --> + <li><a href="#ModifiedMessage">Name: Object Modification Date and Time</a> <!-- 0x0012 --> + </ol> + <li><a href="#DataStorage">Disk Format: Level 2b - Data Object Data Storage</a> + </ol> + </font> + <LI><A href="#Appendix">Appendix</A> + </ol> +</td></tr> +</table> +</center> + + <BR> + <HR> + + + <h2>Introduction</h2> + + <table align=right width=100> + <tr><td> </td><td align=center> + <hr> + <img src="FF-IH_FileGroup.gif" alt="HDF5 Groups" hspace=15 vspace=15> + </td><td> </td></tr> + <tr><td> </td><td align=center> + <strong>Figure 1:</strong> Relationships among the HDF5 root group, other groups, and objects + <hr> + </td><td> </td></tr> + + <tr><td> </td><td align=center> + <img src="FF-IH_FileObject.gif" alt="HDF5 Objects" hspace=15 vspace=15> + </td><td> </td></tr> + <tr><td> </td><td align=center> + <strong>Figure 2:</strong> HDF5 objects -- datasets, datatypes, or dataspaces + <hr> + </td><td> </td></tr> + </table> + + + <P>The format of an HDF5 file on disk encompasses several + key ideas of the HDF4 and AIO file formats as well as + addressing some shortcomings therein. The new format is + more self-describing than the HDF4 format and is more + uniformly applied to data objects in the file. + + <P>An HDF5 file appears to the user as a directed graph. + The nodes of this graph are the higher-level HDF5 objects + that are exposed by the HDF5 APIs: + + <ul> + <li>Groups + <li>Datasets + <li>Named datatypes + </ul> + + <P>At the lowest level, as information is actually written to the disk, + an HDF5 file is made up of the following objects: + <ul> + <li>A super block + <li>B-tree nodes (containing either symbol nodes or raw data chunks) + <li>Object headers + <li>A global heap + <li>Local heaps + <li>Free space + </ul> + + <P>The HDF5 library uses these low-level objects to represent the + higher-level objects that are then presented to the user or + to applications through the APIs. + For instance, a group is an object header that contains a message that + points to a local heap and to a B-tree which points to symbol nodes. + A dataset is an object header that contains messages that describe + datatype, space, layout, filters, external files, fill value, etc + with the layout message pointing to either a raw data chunk or to a + B-tree that points to raw data chunks. + + + <h3>This Document</h3> + + <p>This document describes the lower-level data objects; + the higher-level objects and their properties are described + in the <a href="H5.user.html"><cite>HDF5 User's Guide</cite></a>. + + <P>Three levels of information comprise the file format. + Level 0 contains basic information for identifying and + defining information about the file. Level 1 information contains + the information about the pieces of a file shared by many objects + in the file (such as a B-trees and heaps). Level 2 is the rest + of the file and contains all of the data objects, with each object + partitioned into header information, also known as + <em>metadata</em>, and data. + + <p>The sizes of various fields in the following layout tables are + determined by looking at the number of columns the field spans + in the table. There are three exceptions: (1) The size may be + overridden by specifying a size in parentheses, (2) the size of + addresses is determined by the <em>Size of Offsets</em> field + in the super block and is indicated in this document with a + superscripted 'O', and (3) the size of length fields is determined + by the <em>Size of Lengths</em> field in the super block and is + indicated in this document with a superscripted 'L'. + + <P>Values for all fields in this document should be treated as unsigned + integers, unless otherwise noted in the description of a field. + Additionally, all metadata fields are stored in little-endian byte + order. + </P> + + <BR> + <HR> + + <h2><a name="FileMetaData"> + Disk Format: Level 0 - File Metadata</a></h2> + + <H3><A name="SuperBlock"> + Disk Format: Level 0A - File Signature and Super Block</A></H3> + + <P>The super block may begin at certain predefined offsets within + the HDF5 file, allowing a block of unspecified content for + users to place additional information at the beginning (and + end) of the HDF5 file without limiting the HDF5 library's + ability to manage the objects within the file itself. This + feature was designed to accommodate wrapping an HDF5 file in + another file format or adding descriptive information to the + file without requiring the modification of the actual file's + information. The super block is located by searching for the + HDF5 file signature at byte offset 0, byte offset 512 and at + successive locations in the file, each a multiple of two of + the previous location, i.e. 0, 512, 1024, 2048, etc. + + <P>The super block is composed of a file signature, followed by + super block and group version numbers, information + about the sizes of offset and length values used to describe + items within the file, the size of each group page, + and a group entry for the root object in the file. + + <br> + <div align=center> + <table class=format> + <caption> + HDF5 Super Block Layout + </caption> + + <tr> + <th>byte</th> + <th>byte</th> + <th>byte</th> + <th>byte</th> + </tr> + + <tr> + <td colspan=4><br>HDF5 File Signature (8 bytes)<br><br></td> + </tr> + + <tr> + <td>Version # of Super Block</td> + <td>Version # of Global Free-space Storage</td> + <td>Version # of Root Group Symbol Table Entry</td> + <td>Reserved (zero)</td> + </tr> + + <tr> + <td>Version # of Shared Header Message Format</td> + <td>Size of Offsets</td> + <td>Size of Lengths</td> + <td>Reserved (zero)</td> + </tr> + + <tr> + <td colspan=2>Group Leaf Node K</td> + <td colspan=2>Group Internal Node K</td> + </tr> + + <tr> + <td colspan=4>File Consistency Flags</td> + </tr> + + <tr> + <td colspan=2 style="border:dotted;">Indexed Storage Internal Node K<sup>1</sup></td> + <td colspan=2 style="border:dotted;">Reserved (zero)<sup>1</sup></td> + </tr> + + <tr> + <td colspan=4>Base Address<sup>O</sup></td> + </tr> + + <tr> + <td colspan=4>Address of Global Free-space Heap<sup>O</sup></td> + </tr> + + <tr> + <td colspan=4>End of File Address<sup>O</sup></td> + </tr> + + <tr> + <td colspan=4>Driver Information Block Address<sup>O</sup></td> + </tr> + + <tr> + <td colspan=4>Root Group Symbol Table Entry</td> + </tr> + </table> + + <table class=note> + <tr><td> + (Items marked with an 'O' the above table are + <br> + of the size specified in "Size of Offsets.") + </td></tr> + <tr><td> + (Items marked with an '1' the above table are + <br> + new in version 1 of the superblock) + </td></tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>HDF5 File Signature</td> + <td> + <P>This field contains a constant value and can be used to + quickly identify a file as being an HDF5 file. The + constant value is designed to allow easy identification of + an HDF5 file and to allow certain types of data corruption + to be detected. The file signature of an HDF5 file always + contains the following values: + </P> + + <center> + <table border align=center cellpadding=4> + <tr align=center> + <td align=right>Decimal:</td> + <td width="8%">137</td> + <td width="8%">72</td> + <td width="8%">68</td> + <td width="8%">70</td> + <td width="8%">13</td> + <td width="8%">10</td> + <td width="8%">26</td> + <td width="8%">10</td> + </tr> + + <tr align=center> + <td align=right>Hexadecimal:</td> + <td>89</td> + <td>48</td> + <td>44</td> + <td>46</td> + <td>0d</td> + <td>0a</td> + <td>1a</td> + <td>0a</td> + </tr> + + <tr align=center> + <td align=right>ASCII C Notation:</td> + <td>\211</td> + <td>H</td> + <td>D</td> + <td>F</td> + <td>\r</td> + <td>\n</td> + <td>\032</td> + <td>\n</td> + </tr> + </table> + </center> + <br> + + <P>This signature both identifies the file as an HDF5 file + and provides for immediate detection of common + file-transfer problems. The first two bytes distinguish + HDF5 files on systems that expect the first two bytes to + identify the file type uniquely. The first byte is + chosen as a non-ASCII value to reduce the probability + that a text file may be misrecognized as an HDF5 file; + also, it catches bad file transfers that clear bit + 7. Bytes two through four name the format. The CR-LF + sequence catches bad file transfers that alter newline + sequences. The control-Z character stops file display + under MS-DOS. The final line feed checks for the inverse + of the CR-LF translation problem. (This is a direct + descendent of the <A href="http://www.libpng.org/pub/png/spec/PNG-Rationale.html#R.PNG-file-signature">PNG</A> file + signature.) + </P> + + <P><EM>This field is present in version 0+ of the superblock.</EM> + </P> + </td> + </tr> + + <tr> + <td>Version Number of the Super Block</td> + <td> + <P>This value is used to determine the format of the + information in the super block. When the format of the + information in the super block is changed, the version number + is incremented to the next integer and can be used to + determine how the information in the super block is + formatted. + </P> + + <P>Values of 0 and 1 are defined for this field. + </P> + + <P><EM>This field is present in version 0+ of the superblock.</EM> + </P> + </td> + </tr> + + <tr> + <td>Version Number of the File Free-space Information</td> + <td> + <P>This value is used to determine the format of the + information in the File Free-space Information. + </P> + <P>The only value currently valid in this field is '0', which + indicates that the free space index is formatted as described + <A href="#FreeSpaceIndex">below</A>. + </P> + + <P><EM>This field is present in version 0+ of the superblock.</EM> + </P> + </td> + </tr> + + <tr> + <td>Version Number of the Root Group Symbol Table Entry</td> + <td> + <P>This value is used to determine the format of the + information in the Root Group Symbol Table Entry. When the + format of the information in that field is changed, the + version number is incremented to the next integer and can be + used to determine how the information in the field + is formatted. + </P> + <P>The only value currently valid in this field is '0', which + indicates that the root group symbol table entry is formatted as + described <A href="#SymbolTableEntry">below</A>. + </P> + + <P><EM>This field is present in version 0+ of the superblock.</EM> + </P> + </td> + </tr> + + <tr> + <td>Version Number of the Shared Header Message Format</td> + <td> + <P>This value is used to determine the format of the + information in a shared object header message. Since the format + of the shared header messages differs from the other private + header messages, a version number is used to identify changes + in the format. + </P> + <P>The only value currently valid in this field is '0', which + indicates that shared header messages are formatted as + described <A href="#SharedMessage">below</A>. + </P> + + <P><EM>This field is present in version 0+ of the superblock.</EM> + </P> + </td> + </tr> + + <tr> + <td>Size of Offsets</td> + <td> + <P>This value contains the number of bytes used to store + addresses in the file. The values for the addresses of + objects in the file are offsets relative to a base address, + usually the address of the super block signature. This + allows a wrapper to be added after the file is created + without invalidating the internal offset locations. + </P> + + <P><EM>This field is present in version 0+ of the superblock.</EM> + </P> + </td> + </tr> + + <tr> + <td>Size of Lengths</td> + <td> + <P>This value contains the number of bytes used to store + the size of an object. + </P> + + <P><EM>This field is present in version 0+ of the superblock.</EM> + </P> + </td> + </tr> + + <tr> + <td>Group Leaf Node K</td> + <td> + <P>Each leaf node of a group B-tree will have at + least this many entries but not more than twice this + many. If a group has a single leaf node then it + may have fewer entries. + </P> + <P>This value must be greater than zero. + </P> + <P>See the <A href="#Btrees">description</A> of B-trees below. + </P> + + <P><EM>This field is present in version 0+ of the superblock.</EM> + </P> + </td> + </tr> + + <tr> + <td>Group Internal Node K</td> + <td> + <P>Each internal node of a group B-tree will have at + least this many entries but not more than twice this + many. If the group has only one internal + node then it might have fewer entries. + </P> + <P>This value must be greater than zero. + </P> + <P>See the <A href="#Btrees">description</A> of B-trees below. + </P> + + <P><EM>This field is present in version 0+ of the superblock.</EM> + </P> + </td> + </tr> + + <tr> + <td>File Consistency Flags</td> + <td> + <P>This value contains flags to indicate information + about the consistency of the information contained + within the file. Currently, the following bit flags are + defined: + <ul> + <li>Bit 0 set indicates that the file is opened for + write-access. + <li>Bit 1 set indicates that the file has + been verified for consistency and is guaranteed to be + consistent with the format defined in this document. + <li>Bits 2-31 are reserved for future use. + </ul> + Bit 0 should be + set as the first action when a file is opened for write + access and should be cleared only as the final action + when closing a file. Bit 1 should be cleared during + normal access to a file and only set after the file's + consistency is guaranteed by the library or a + consistency utility. + </P> + + <P><EM>This field is present in version 0+ of the superblock.</EM> + </P> + </td> + </tr> + + <tr> + <td>Indexed Storage Internal Node K</td> + <td> + <P>Each internal node of a indexed storage B-tree will have at + least this many entries but not more than twice this + many. If the group has only one internal + node then it might have fewer entries. + </P> + <P>This value must be greater than zero. + </P> + <P>See the <A href="#Btrees">description</A> of B-trees below. + </P> + + <P><EM>This field is present in version 1+ of the superblock.</EM> + </P> + </td> + </tr> + + <tr> + <td>Base Address</td> + <td> + <P>This is the absolute file address of the first byte of + the HDF5 data within the file. The library currently + constrains this value to be the absolute file address + of the super block itself when creating new files; + future versions of the library may provide greater + flexibility. When opening an existing file and this address does + not match the offset of the superblock, the library assumes + that the entire contents of the HDF5 file have been adjusted in + the file and adjusts the base address and end of file address to + reflect their new positions in the file. Unless otherwise noted, + all other file addresses are relative to this base + address. + </P> + + <P><EM>This field is present in version 0+ of the superblock.</EM> + </P> + </td> + </tr> + + <tr> + <td>Address of Global Free-space Index</td> + <td> + <P>Free-space management is not yet defined in the HDF5 + file format and is not handled by the library. + Currently this field always contains the + <A href="#UndefinedAddress">undefined address</A>. + </P> + + <P><EM>This field is present in version 0+ of the superblock.</EM> + </P> + </td> + </tr> + + <tr> + <td>End of File Address</td> + <td> + <P>This is the absolute file address of the first byte past + the end of all HDF5 data. It is used to determine whether a + file has been accidently truncated and as an address where + file data allocation can occur if space from the free list is + not used. + </P> + + <P><EM>This field is present in version 0+ of the superblock.</EM> + </P> + </td> + </tr> + + <tr> + <td>Driver Information Block Address</td> + <td> + <P>This is the relative file address of the file driver + information block which contains driver-specific + information needed to reopen the file. If there is no + driver information block then this entry should be the + <A href="#UndefinedAddress">undefined address</A>. + </P> + + <P><EM>This field is present in version 0+ of the superblock.</EM> + </P> + </td> + </tr> + + <tr> + <td>Root Group Symbol Table Entry</td> + <td> + <P>This is the <A href="#SymbolTableEntry">symbol table entry</A> + of the root group, which serves as the entry point into + the group graph for the file. + </P> + + <P><EM>This field is present in version 0+ of the superblock.</EM> + </P> + </td> + </tr> + </table> + </div> + + <H3><A name="DriverInfo"> + Disk Format: Level 0B - File Driver Info</A></H3> + + <p>The <em>file driver information block</em> is an optional region of the + file which contains information needed by the file driver in + order to reopen a file. The format of the file driver information + block is: + + <br> + <div align=center> + <table class=format> + <caption> + Driver Information Block + </caption> + + <tr> + <th>byte</th> + <th>byte</th> + <th>byte</th> + <th>byte</th> + </tr> + + <tr> + <td>Version</td> + <td colspan=3>Reserved (zero)</td> + </tr> + + <tr> + <td colspan=4>Driver Information Size (4 bytes)</td> + </tr> + + <tr> + <td colspan=4><br>Driver Identification (8 bytes)<br><br></td> + </tr> + + <tr> + <td colspan=4><br><br>Driver Information (<em>n</em> bytes)<br><br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Version</td> + <td> + <P>The version number of the driver information block. The + file format documented here is version zero. + </P> + </td> + </tr> + + <tr> + <td>Driver Information Size</td> + <td> + <P>The size in bytes of the Driver Information part of this + structure. + </P> + </td> + </tr> + + <tr> + <td>Driver Identification</td> + <td> + <P>This is an eight-byte ASCII string without null + termination which identifies the driver and version number + of the Driver Information block. The predefined drivers + supplied with the HDF5 library are identified by the + letters <code>NCSA</code> followed by the first four characters of + the driver name. If the Driver Information block is not + the original version then the last letter(s) of the + identification will be replaced by a version number in + ASCII. + </P> + <P> + For example, the various versions of the <em>multi driver</em> + will be identified by <code>NCSAmult</code>. + (<code>NCSAmult</code> is simply <code>NCSAmulti</code> truncated + to eight characters. Subsequent identifiers will be created by + substituting sequential numerical values for the final character, + starting with zero.) <em>multi driver</em> is the only default driver that + is encoded in this field. + </P> + <P> + Identification for user-defined drivers + is eight-byte long and arbitrary but should be unique and avoid + the four character prefix "NCSA". + </P> + </td> + </tr> + + <tr valign=top> + <td>Driver Information</td> + <td>Driver information is encoded/decoded in a format defined by the + file driver. <em>multi driver</em> is the only default driver that has driver + information stored in this field. Its format is explained in the + following block.</td> + </tr> + </table> + </div> + + <BR> + <P><em>Multi driver</em> has the following format:</P> + + <div align=center> + <table class=format> + <caption> + Multi Driver Message + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td>Member Mapping</td> + <td>Member Mapping</td> + <td>Member Mapping</td> + <td>Member Mapping</td> + </tr> + + <tr> + <td>Member Mapping</td> + <td>Member Mapping</td> + <td>Reserved</td> + <td>Reserved</td> + </tr> + + <tr> + <td colspan=4><br>Address of Member File 1<br><br></td> + </tr> + + <tr> + <td colspan=4><br>End of Address for Member File 1<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Address of Member File 2<br><br></td> + </tr> + + <tr> + <td colspan=4><br>End of Address for Member File 2<br><br></td> + </tr> + + <tr> + <td colspan=4><br>... ...<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Name of Member File 1<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Name of Member File 2<br><br></td> + </tr> + + <tr> + <td colspan=4><br>... ...<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Member Mapping</td> + <td><P><em>Multi driver</em> enables different types of HDF5 data and + metadata to be written to separate files. These files are viewed by the + library as a single virtual HDF5 file with a single file address. + It allows maximal 6 files to be created. + In sequence, these <em>Member Mapping</em> fields are for super block, + B-tree, raw data, global heap, local heap, + and object header. More than one type of data can be written to the + same file.</P> + <P>These <em>Member Mapping</em> fields are integer values from 1 to 6 + indicating how the data can be mapped to or merged with another type of + data. + <table class=list> + <tr> + <th width="30%">Member Mapping</th> + <th align=left>Description</th> + </tr> + <tr> + <td align=center>1</td> + <td>The super block data.</td> + </tr> + <tr> + <td align=center>2</td> + <td>The B-tree data.</td> + </tr> + <tr> + <td align=center>3</td> + <td>The raw data.</td> + </tr> + <tr> + <td align=center>4</td> + <td>The global heap data.</td> + </tr> + <tr> + <td align=center>5</td> + <td>The local heap data.</td> + </tr> + <tr> + <td align=center>6</td> + <td>The object header data.</td> + </tr> + </table></P> + For example, if the third field has the value 3 and all the rest have the + value 1, it means there are two files, one for raw data, one for super block, + B-tree, global heap, local heap, and object header. + </td> + </tr> + + <tr> + <td>Reserved</td> + <td><P>These fields are reserved and should always be zero.</P></td> + </tr> + + <tr> + <td>Address of Member File</td> + <td><P>Specifies the virtual address. A normally eight-byte integer with + the value from <em>0</em> (zero) to maximal value, + at which the member file starts.</P></td> + </tr> + + <tr> + <td>End of Address for Member File</td> + <td><P>The end of allocated address for the member file. A normally eight-byte + integer value.</P></td> + </tr> + + <tr> + <td>Name of Member File</td> + <td><P>The null-terminated name of member file. Its length should be multiples of + 8 bytes. Additional bytes will be padded with <em>NULL</em>s. The default naming + convention is <em>%%s-X.h5</em>, where <em>X</em> is one of the letters + <em>s</em> (for super block), <em>b</em> (for B-tree), <em>r</em> (for raw data), + <em>g</em> (for global heap), <em>l</em> (for local heap), and <em>o</em> (for + object header). The name for the whole HDF5 file will substitute the <em>%s</em> + in the string. + </P> + </td> + </tr> + </table> + </div> + + <BR> + <HR> + + <h2><a name="FileInfra"> + Disk Format: Level 1 - File Infrastructure</a></h2> + <h3><a name="Btrees">Disk Format: Level 1A - B-link Trees and B-tree Nodes</a></h3> + + <p>B-link trees allow flexible storage for objects which tend to grow + in ways that cause the object to be stored discontiguously. B-trees + are described in various algorithms books including "Introduction to + Algorithms" by Thomas H. Cormen, Charles E. Leiserson, and Ronald + L. Rivest. The B-link tree, in which the sibling nodes at a + particular level in the tree are stored in a doubly-linked list, + is described in the "Efficient Locking for Concurrent Operations + on B-trees" paper by Phillip Lehman and S. Bing Yao as published + in the <cite>ACM Transactions on Database Systems</cite>, Vol. 6, + No. 4, December 1981. + + <p>The B-link trees implemented by the file format contain one more + key than the number of children. In other words, each child + pointer out of a B-tree node has a left key and a right key. + The pointers out of internal nodes point to sub-trees while + the pointers out of leaf nodes point to symbol nodes and + raw data chunks. + Aside from that difference, internal nodes and leaf nodes + are identical. + + <br> + <div align=center> + <table class=format> + <caption> + B-tree Nodes + </caption> + + <tr> + <th>byte</th> + <th>byte</th> + <th>byte</th> + <th>byte</th> + + <tr> + <td colspan=4>Signature</td> + + <tr> + <td>Node Type</td> + <td>Node Level</td> + <td colspan=2>Entries Used</td> + + <tr> + <td colspan=4>Address of Left Sibling<sup>O</sup></td> + + <tr> + <td colspan=4>Address of Right Sibling<sup>O</sup></td> + + <tr> + <td colspan=4>Key 0 (variable size)</td> + + <tr> + <td colspan=4>Address of Child 0<sup>O</sup></td> + + <tr> + <td colspan=4>Key 1 (variable size)</td> + + <tr> + <td colspan=4>Address of Child 1<sup>O</sup></td> + + <tr> + <td colspan=4>...</td> + + <tr> + <td colspan=4>Key 2<em>K</em> (variable size)</td> + + <tr> + <td colspan=4>Address of Child 2<em>K</em><sup>O</sup></td> + + <tr> + <td colspan=4>Key 2<em>K</em>+1 (variable size)</td> + </table> + + <table class=note> + <tr><td> + (Items marked with an 'O' the above table are + <br> + of the size specified in "Size of Offsets.") + </td></tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Signature</td> + <td> + <P>The ASCII character string "<code>TREE</code>" is + used to indicate the + beginning of a B-link tree node. This gives file + consistency checking utilities a better chance of + reconstructing a damaged file. + </P> + </td> + </tr> + + <tr> + <td>Node Type</td> + <td> + <P>Each B-link tree points to a particular type of data. + This field indicates the type of data as well as + implying the maximum degree <em>K</em> of the tree and + the size of each Key field. + </P> + + <table class=list> + <tr> + <th width="30%">Node Type</th> + <th align=left>Description</th> + </tr> + <tr> + <td align=center>0</td> + <td>This tree points to group nodes.</td> + </tr> + <tr> + <td align=center>1</td> + <td>This tree points to raw data chunk nodes.</td> + </tr> + </table> + </td> + </tr> + + <tr> + <td>Node Level</td> + <td> + <P>The node level indicates the level at which this node + appears in the tree (leaf nodes are at level zero). Not + only does the level indicate whether child pointers + point to sub-trees or to data, but it can also be used + to help file consistency checking utilities reconstruct + damanged trees. + </P> + </td> + </tr> + + <tr valign=top> + <td>Entries Used</td> + <td> + <P>This determines the number of children to which this + node points. All nodes of a particular type of tree + have the same maximum degree, but most nodes will point + to less than that number of children. The valid child + pointers and keys appear at the beginning of the node + and the unused pointers and keys appear at the end of + the node. The unused pointers and keys have undefined + values. + </P> + </td> + </tr> + + <tr valign=top> + <td>Address of Left Sibling</td> + <td> + <P>This is the relative file address of the left sibling of + the current node. If the current + node is the left-most node at this level then this field + is the <A href="#UndefinedAddress">undefined address</A>. + </P> + </td> + </tr> + + <tr valign=top> + <td>Address of Right Sibling</td> + <td> + <P>This is the relative file address of the right sibling of + the current node. If the current + node is the right-most node at this level then this + field is the <A href="#UndefinedAddress">undefined address</A>. + </P> + </td> + </tr> + + <tr valign=top> + <td>Keys and Child Pointers</td> + <td> + <P>Each tree has 2<em>K</em>+1 keys with 2<em>K</em> + child pointers interleaved between the keys. The number + of keys and child pointers actually containing valid + values is determined by the node's <em>Entries Used</em> field. + If that field is <em>N</em> then the B-link tree contains + <em>N</em> child pointers and <em>N</em>+1 keys. + </P> + </td> + </tr> + + <tr valign=top> + <td>Key</td> + <td> + <P>The format and size of the key values is determined by + the type of data to which this tree points. The keys are + ordered and are boundaries for the contents of the child + pointer; that is, the key values represented by child + <em>N</em> fall between Key <em>N</em> and Key + <em>N</em>+1. Whether the interval is open or closed on + each end is determined by the type of data to which the + tree points. + </P> + + <P> + The format of the key depends on the node type. + For nodes of node type 0 (group nodes), the key is formatted as + follows: + <center> + <table class=list> + <tr> + <td width=30%>A single field of <i>Size of Lengths</i> + bytes:</td> + <td>Indicates the byte offset into the local heap + for the first object name in the subtree which + that key describes. + </td> + </tr> + </table> + </center> + </P> + + <P> + For nodes of node type 1 (chunked raw data nodes), the key is + formatted as follows: + <center> + <table class=list> + <tr> + <td width=30%>Bytes 1-4:</td> + <td>Size of chunk in bytes.</td> + </tr> + <tr> + <td>Bytes 4-8:</td> + <td>Filter mask, a 32-bit bitfield indicating which + filters have been skipped for this chunk. Each filter + has an index number in the pipeline (starting at 0, with + the first filter to apply) and if that filter is skipped, + the bit corresponding to it's index is set.</td> + </tr> + <tr> + <td><em>N</em> 64-bit fields:</td> + <td>A 64-bit index indicating the offset of the + chunk within the dataset where <i>N</i> is the number + of dimensions of the dataset. For example, if + a chunk in a 3-dimensional dataset begins at the + position <code>[5,5,5]</code>, there will be three + such 64-bit indices, each with the value of + <code>5</code>.</td> + </tr> + </table> + </center> + </P> + </td> + </tr> + + <tr valign=top> + <td>Child Pointer</td> + <td> + <P>The tree node contains file addresses of subtrees or + data depending on the node level. Nodes at Level 0 point + to data addresses, either raw data chunk or group nodes. + Nodes at non-zero levels point to other nodes of the + same B-tree. + </P> + <P>For raw data chunk nodes, the child pointer is the address + of a single raw data chunk. For group nodes, the child pointer + points to a <A href="#SymbolTable">symbol table</A>, which contains + information for multiple symbol table entries. + </P> + </td> + </tr> + </table> + </div> + + <p> + Conceptually, each B-tree node looks like this: + <center> + <table> + <tr valign=top align=center> + <td>key[0]</td><td> </td> + <td>child[0]</td><td> </td> + <td>key[1]</td><td> </td> + <td>child[1]</td><td> </td> + <td>key[2]</td><td> </td> + <td>...</td><td> </td> + <td>...</td><td> </td> + <td>key[<i>N</i>-1]</td><td> </td> + <td>child[<i>N</i>-1]</td><td> </td> + <td>key[<i>N</i>]</td> + </tr> + </table> + </center> + <br> + + where child[<i>i</i>] is a pointer to a sub-tree (at a level + above Level 0) or to data (at Level 0). + Each key[<i>i</i>] describes an <i>item</i> stored by the B-tree + (a chunk or an object of a group node). The range of values + represented by child[<i>i</i>] is indicated by key[<i>i</i>] + and key[<i>i</i>+1]. + + + <p>The following question must next be answered: + "Is the value described by key[<i>i</i>] contained in + child[<i>i</i>-1] or in child[<i>i</i>]?" + The answer depends on the type of tree. + In trees for groups (node type 0) the object described by + key[<i>i</i>] is the greatest object contained in + child[<i>i</i>-1] while in chunk trees (node type 1) the + chunk described by key[<i>i</i>] is the least chunk in + child[<i>i</i>]. + + <p>That means that key[0] for group trees is sometimes unused; + it points to offset zero in the heap, which is always the + empty string and compares as "less-than" any valid object name. + + <p>And key[<i>N</i>] for chunk trees is sometimes unused; + it contains a chunk offset which compares as "greater-than" + any other chunk offset and has a chunk byte size of zero + to indicate that it is not actually allocated. + + + <h3><a name="SymbolTable">Disk Format: Level 1B - Group and Symbol Nodes</a></h3> + + <p>A group is an object internal to the file that allows + arbitrary nesting of objects within the file (including other groups). + A group maps a set of names in the group to a set of relative + file addresses where objects with those names are located in + the file. Certain metadata for an object to which the group points + can be cached in the group's symbol table in addition to the + object's header. + + <p>An HDF5 object name space can be stored hierarchically by + partitioning the name into components and storing each + component in a group. The group entry for a + non-ultimate component points to the group containing + the next component. The group entry for the last + component points to the object being named. + + <p>A group is a collection of group nodes pointed + to by a B-link tree. Each group node contains entries + for one or more symbols. If an attempt is made to add a + symbol to an already full group node containing + 2<em>K</em> entries, then the node is split and one node + contains <em>K</em> symbols and the other contains + <em>K</em>+1 symbols. + + <br> + <div align=center> + <table class=format> + <caption> + Group Node (A Leaf of a B-tree) + </caption> + + <tr> + <th>byte</th> + <th>byte</th> + <th>byte</th> + <th>byte</th> + + <tr> + <td colspan=4>Signature</td> + + <tr> + <td>Version Number</td> + <td>Reserved (0)</td> + <td colspan=2>Number of Symbols</td> + + <tr> + <td colspan=4><br><br>Group Entries<br><br><br></td> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Signature</td> + <td> + <P>The ASCII character string "<code>SNOD</code>" is + used to indicate the + beginning of a group node. This gives file + consistency checking utilities a better chance of + reconstructing a damaged file. + </P> + </td> + </tr> + + <tr> + <td>Version Number</td> + <td> + <P>The version number for the group node. This + document describes version 1. (There is no version '0' + of the group node) + </P> + </td> + </tr> + + <tr> + <td>Number of Symbols</td> + <td> + <P>Although all group nodes have the same length, + most contain fewer than the maximum possible number of + symbol entries. This field indicates how many entries + contain valid data. The valid entries are packed at the + beginning of the group node while the remaining + entries contain undefined values. + </P> + </td> + </tr> + + <tr> + <td>Group Entries</td> + <td> + <P>Each symbol has an entry in the group node. + The format of the entry is described below. + There are 2<EM>K</EM> entries in each group node, where + <EM>K</EM> is the "Group Leaf Node K" value from the + <A href="#SuperBlock">super block</A>. + </P> + </td> + </tr> + </table> + </div> + + <h3><a name="SymbolTableEntry"> + Disk Format: Level 1C - Group Entry </a></h3> + + <p>Each group entry in a group node is designed + to allow for very fast browsing of stored objects. + Toward that design goal, the group entries + include space for caching certain constant metadata from the + object header. + + <br> + <div align=center> + <table class=format> + <caption> + Group Entry + </caption> + + <tr> + <th>byte</th> + <th>byte</th> + <th>byte</th> + <th>byte</th> + </tr> + + <tr> + <td colspan=4>Name Offset<sup>O</sup></td> + </tr> + + <tr> + <td colspan=4>Object Header Address<sup>O</sup></td> + </tr> + + <tr> + <td colspan=4>Cache Type</td> + </tr> + + <tr> + <td colspan=4>Reserved</td> + </tr> + + <tr> + <td colspan=4><br><br>Scratch-pad Space (16 bytes)<br><br><br></td> + </tr> + </table> + + <table class=note> + <tr><td> + (Items marked with an 'O' the above table are + <br> + of the size specified in "Size of Offsets.") + </td></tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Name Offset</td> + <td> + <P>This is the byte offset into the group local + heap for the name of the object. The name is null + terminated. + </P> + </td> + </tr> + + <tr> + <td>Object Header Address</td> + <td> + <P>Every object has an object header which serves as a + permanent location for the object's metadata. In addition + to appearing in the object header, some metadata can be + cached in the scratch-pad space. + </P> + </td> + </tr> + + <tr> + <td>Cache Type</td> + <td> + <P>The cache type is determined from the object header. + It also determines the format for the scratch-pad space: + <br> + <table class=list> + <tr align=left> + <th>Type:</th> + <th>Description:</th> + </tr> + <tr> + <td width="10%" align=center>0</td> + <td>No data is cached by the group entry. This + is guaranteed to be the case when an object header + has a link count greater than one. + </td> + </tr> + <tr> + <td align=center>1</td> + <td>Object header metadata is cached in the group + entry. This implies that the group + entry refers to another group. + </td> + </tr> + <tr> + <td align=center>2</td> + <td>The entry is a symbolic link. The first four bytes + of the scratch-pad space are the offset into the local + heap for the link value. The object header address + will be undefined. + </td> + </tr> + <tr> + <td align=center><em>N</em></td> + <td>Other cache values can be defined later and + libraries that do not understand the new values will + still work properly. + </td> + </tr> + </table> + </P> + </td> + </tr> + + <tr> + <td>Reserved</td> + <td> + <P>These four bytes are present so that the scratch-pad + space is aligned on an eight-byte boundary. They are + always set to zero. + </P> + </td> + </tr> + + <tr> + <td>Scratch-pad Space</td> + <td> + <P>This space is used for different purposes, depending + on the value of the Cache Type field. Any metadata + about a dataset object represented in the scratch-pad + space is duplicated in the object header for that + dataset. This metadata can include the datatype + and the size of the dataspace for a dataset whose datatype + is atomic and whose dataspace is fixed and less than + four dimensions. + </P> + <P> + Furthermore, no data is cached in the group + entry scratch-pad space if the object header for + the group entry has a link count greater than + one. + </P> + </td> + </tr> + </table> + </div> + + <h4>Format of the Scratch-pad Space</h4> + + <p>The group entry scratch-pad space is formatted + according to the value in the Cache Type field. + + <p>If the Cache Type field contains the value zero + <code>(0)</code> then no information is + stored in the scratch-pad space. + + <p>If the Cache Type field contains the value one + <code>(1)</code>, then the scratch-pad space + contains cached metadata for another object header + in the following format: + + <br> + <div align=center> + <table class=format> + <caption> + Object Header Scratch-pad Format + </caption> + + <tr> + <th>byte</th> + <th>byte</th> + <th>byte</th> + <th>byte</th> + + <tr> + <td colspan=4>Address of B-tree<sup>O</sup></td> + + <tr> + <td colspan=4>Address of Name Heap<sup>O</sup></td> + </table> + + <table class=note> + <tr><td> + (Items marked with an 'O' the above table are + <br> + of the size specified in "Size of Offsets.") + </td></tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Address of B-tree</td> + <td> + <P>This is the file address for the root of the + group's B-tree. + </P> + </td> + </tr> + + <tr> + <td>Address of Name Heap</td> + <td> + <P>This is the file address for the group's local + heap, in which are stored the group's symbol names. + </P> + </td> + </tr> + </table> + </div> + + + <P>If the Cache Type field contains the value two + <code>(2)</code>, then the scratch-pad space + contains cached metadata for another symbolic link + in the following format: + + <br> + <div align=center> + <table class=format> + <caption> + Symbolic Link Scratch-pad Format + </caption> + + <tr> + <th>byte</th> + <th>byte</th> + <th>byte</th> + <th>byte</th> + </tr> + + <tr> + <td colspan=4>Offset to Link Value</td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Offset to Link Value</td> + <td> + <P>The value of a symbolic link (that is, the name of the + thing to which it points) is stored in the local heap. + This field is the 4-byte offset into the local heap for + the start of the link value, which is null terminated. + </P> + </td> + </tr> + </table> + </div> + + <h3><a name="LocalHeap">Disk Format: Level 1D - Local Heaps</a></h3> + + <P>A heap is a collection of small heap objects. Objects can be + inserted and removed from the heap at any time. + The address of a heap does not change once the heap is created. + References to objects are stored in the group table; + the names of those objects are stored in the local heap. + </P> + + <br> + <div align=center> + <table class=format> + <caption> + Local Heap + </caption> + + <tr> + <th>byte</th> + <th>byte</th> + <th>byte</th> + <th>byte</th> + </tr> + + <tr> + <td colspan=4>Signature</td> + </tr> + + <tr> + <td>Version</td> + <td colspan=3>Reserved (zero)</td> + </tr> + + <tr> + <td colspan=4>Data Segment Size<sup>L</sup></td> + </tr> + + <tr> + <td colspan=4>Offset to Head of Free-list<sup>L</sup></td> + </tr> + + <tr> + <td colspan=4>Address of Data Segment<sup>O</sup></td> + </tr> + </table> + + <table class=note> + <tr><td> + (Items marked with an 'L' the above table are + <br> + of the size specified in "Size of Lengths.") + </td></tr> + <tr><td> + (Items marked with an 'O' the above table are + <br> + of the size specified in "Size of Offsets.") + </td></tr> + </table> + </div> + + <p> + <center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Signature</td> + <td> + <P>The ASCII character string "<code>HEAP</code>" + is used to indicate the + beginning of a heap. This gives file consistency + checking utilities a better chance of reconstructing a + damaged file. + </P> + </td> + </tr> + + <tr> + <td>Version</td> + <td> + <P>Each local heap has its own version number so that new + heaps can be added to old files. This document + describes version zero (0) of the local heap. + </P> + </td> + </tr> + + <tr> + <td>Data Segment Size</td> + <td> + <P>The total amount of disk memory allocated for the heap + data. This may be larger than the amount of space + required by the objects stored in the heap. The extra + unused space in the heap holds a linked list of free blocks. + </P> + </td> + </tr> + + <tr> + <td>Offset to Head of Free-list</td> + <td> + <P>This is the offset within the heap data segment of the + first free block (or the + <A href="#UndefinedAddress">undefined address</A> if there is no + free block). The free block contains "Size of Lengths" bytes that + are the offset of the next free block (or the + value '1' if this is the + last free block) followed by "Size of Lengths" bytes that store + the size of this free block. The size of the free block includes + the space used to store the offset of the next free block and + the of the current block, making the minimum size of a free block + 2 * "Size of Lengths". + </P> + </td> + </tr> + + <tr> + <td>Address of Data Segment</td> + <td> + <P>The data segment originally starts immediately after + the heap header, but if the data segment must grow as a + result of adding more objects, then the data segment may + be relocated, in its entirety, to another part of the + file. + </P> + </td> + </tr> + </table> + </center> + + <p>Objects within the heap should be aligned on an 8-byte boundary. + + <h3><a name="GlobalHeap">Disk Format: Level 1E - Global Heap</a></h3> + + <P>Each HDF5 file has a global heap which stores various types of + information which is typically shared between datasets. The + global heap was designed to satisfy these goals: + + <ol type="A"> + <li>Repeated access to a heap object must be efficient without + resulting in repeated file I/O requests. Since global heap + objects will typically be shared among several datasets, it is + probable that the object will be accessed repeatedly. + <li>Collections of related global heap objects should result in + fewer and larger I/O requests. For instance, a dataset of + object references will have a global heap object for each + reference. Reading the entire set of object references + should result in a few large I/O requests instead of one small + I/O request for each reference. + <li>It should be possible to remove objects from the global heap + and the resulting file hole should be eligible to be reclaimed + for other uses. + </ol> + </P> + + <P>The implementation of the heap makes use of the memory + management already available at the file level and combines that + with a new top-level object called a <em>collection</em> to + achieve Goal B. The global heap is the set of all collections. + Each global heap object belongs to exactly one collection and + each collection contains one or more global heap objects. For + the purposes of disk I/O and caching, a collection is treated as + an atomic object. + </P> + + <P>The HDF5 library creates global heap collections as needed, so there may + be multiple collections throughout the file. The set of all of them is + abstractly called the "global heap", although they don't actually link + to each other, and there is no global place in the file where you can + discover all of the collections. The collections are found simply by + finding a reference to one through another object in the file (eg. + variable-length datatype elements, etc). + </P> + + <br> + <div align=center> + <table class=format> + <caption> + A Global Heap Collection + </caption> + + <tr> + <th>byte</th> + <th>byte</th> + <th>byte</th> + <th>byte</th> + </tr> + + <tr> + <td colspan=4>Signature</td> + </tr> + + <tr> + <td>Version</td> + <td colspan=3>Reserved (zero)</td> + </tr> + + <tr> + <td colspan=4>Collection Size<sup>L</sup></td> + </tr> + + <tr> + <td colspan=4><br>Global Heap Object 1<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Global Heap Object 2<br><br></td> + </tr> + + <tr> + <td colspan=4><br>...<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Global Heap Object <em>N</em><br><br></td> + </tr> + + <tr> + <td colspan=4><br>Global Heap Object 0 (free space)<br><br></td> + </tr> + </table> + + <table class=note> + <tr><td> + (Items marked with an 'L' the above table are + <br> + of the size specified in "Size of Lengths.") + </td></tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Signature</td> + <td> + <P>The ASCII character string "<code>GCOL</code>" + is used to indicate the + beginning of a collection. This gives file consistency + checking utilities a better chance of reconstructing a + damaged file. + </P> + </td> + </tr> + + <tr> + <td>Version</td> + <td> + <P>Each collection has its own version number so that new + collections can be added to old files. This document + describes version one (1) of the collections (there is no + version zero (0)). + </P> + </td> + </tr> + + <tr> + <td>Collection Size</td> + <td> + <P>This is the size in bytes of the entire collection + including this field. The default (and minimum) + collection size is 4096 bytes which is a typical file + system block size. This allows for 127 16-byte heap + objects plus their overhead (the collection header of 16 bytes + and the 16 bytes of information about each heap object). + </P> + </td> + </tr> + + <tr> + <td>Global Heap Object 1 through <em>N</em></td> + <td> + <P>The objects are stored in any order with no + intervening unused space. + </P> + </td> + </tr> + + <tr> + <td>Global Heap Object 0</td> + <td> + <P>Global Heap Object 0 (zero), when present, represents the free + space in the collection. Free space always appears at the end of + the collection. If the free space is too small to store the header + for Object 0 (described below) then the header is implied and the + collection contains no free space. + </P> + </td> + </table> + </div> + + <br> + <div align=center> + <table class=format> + <caption> + Global Heap Object + </caption> + + <tr> + <th>byte</th> + <th>byte</th> + <th>byte</th> + <th>byte</th> + </tr> + + <tr> + <td colspan=2>Heap Object ID</td> + <td colspan=2>Reference Count</td> + </tr> + + <tr> + <td colspan=4>Reserved</td> + </tr> + + <tr> + <td colspan=4>Object Size<sup>L</sup></td> + </tr> + + <tr> + <td colspan=4><br>Object Data<br><br></td> + </tr> + </table> + + <table class=note> + <tr><td> + (Items marked with an 'L' the above table are + <br> + of the size specified in "Size of Lengths.") + </td></tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Heap Object ID</td> + <td> + <P>Each object has a unique identification number within a + collection. The identification numbers are chosen so that + new objects have the smallest value possible with the + exception that the identifier <code>0</code> always refers to the + object which represents all free space within the + collection. + </P> + </td> + </tr> + + <tr> + <td>Reference Count</td> + <td> + <P>All heap objects have a reference count field. An + object which is referenced from some other part of the + file will have a positive reference count. The reference + count for Object 0 is always zero. + </P> + </td> + </tr> + + <tr> + <td>Reserved</td> + <td> + <P>Zero padding to align next field on an 8-byte boundary. + </P> + </td> + </tr> + + <tr> + <td>Object Size</td> + <td> + <P>This is the size of the object data stored for the object. + The actual storage space allocated for the object data is rounded + up to a multiple of eight. + </P> + </td> + </tr> + + <tr> + <td>Object Data</td> + <td> + <P>The object data is treated as a one-dimensional array + of bytes to be interpreted by the caller. + </P> + </td> + </tr> + </table> + </div> + + <h3><a name="FreeSpaceIndex">Disk Format: Level 1F - Free-space Index</a></h3> + + <p>The free-space index is a collection of blocks of data, + dispersed throughout the file, which are currently not used by + any file objects. + + <p>The super block contains a pointer to root of the free-space description; + that pointer is currently required to be the + <A href="#UndefinedAddress">undefined address</A>. + + <p>The format of the free-space index is not defined at this time. + +<!-- + <p>The Free-space Index is a collection of blocks of data, + dispersed throughout the file, which are currently not used by + any file objects. The blocks of data are indexed by a B-tree of + their length within the file. + + + <p>Each B-tree page is composed of the following entries and + B-tree management information, organized as follows: + + <p> + <center> + <table border cellpadding=4 width="80%"> + <caption align=bottom> + <B>HDF5 Free-space Heap Page</B> + </caption> + + <tr align=center> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + + <tr align=center> + <td colspan=4>Signature</td> + <tr align=center> + <td colspan=4>B-tree Left-link Offset</td> + <tr align=center> + <td colspan=4><br>Length of Free-block #1<br> <br></td> + <tr align=center> + <td colspan=4><br>Offset of Free-block #1<br> <br></td> + <tr align=center> + <td colspan=4>.<br>.<br>.<br></td> + <tr align=center> + <td colspan=4><br>Length of Free-block #n<br> <br></td> + <tr align=center> + <td colspan=4><br>Offset of Free-block #n<br> <br></td> + <tr align=center> + <td colspan=4>"High" Offset</td> + <tr align=center> + <td colspan=4>Right-link Offset</td> + </table> + </center> + + <p> + <dl> + <dt> The elements of the free-space heap page are described below: + <dd> + <dl> + <dt>Signature: (4 bytes) + <dd>The ASCII character string <code>FREE</code> + is used to indicate the + beginning of a free-space heap B-tree page. This gives + file consistency checking utilities a better chance of + reconstructing a damaged file. + + <dt>B-tree Left-link Offset: (<offset> bytes) + <dd>This value is used to indicate the offset of all offsets + in the B-link-tree which are smaller than the value of the + offset in entry #1. This value is also used to indicate a + leaf node in the B-link-tree by being set to all ones. + + <dt>Length of Free-block #n: (<length> bytes) + <dd>This value indicates the length of an unused block in + the file. + + <dt>Offset of Free-block #n: (<offset> bytes) + <dd>This value indicates the offset in the file of an + unused block in the file. + + <dt>"High" Offset: (4-bytes) + <dd>This offset is used as the upper bound on offsets + contained within a page when the page has been split. + + <dt>Right-link Offset: (<offset> bytes) + <dd>This value is used to indicate the offset of the next + child to the right of the parent of this group + page. When there is no node to the right, this value is + all zeros. + </dl> + </dl> + + <p>The algorithms for searching and inserting objects in the + B-tree pages are described fully in the Lehman and Yao paper, + which should be read to provide a full description of the + B-tree's usage. +--> + + <BR> + <HR> + + <h2><a name="DataObject">Disk Format: Level 2 - Data Objects </a></h2> + + <P>Data objects contain the real information in the file. These + objects compose the scientific data and other information which + are generally thought of as "data" by the end-user. All the + other information in the file is provided as a framework for + these data objects. + </P> + + <P>A data object is composed of header information and data + information. The header information contains the information + needed to interpret the data information for the data object as + well as additional "metadata" or pointers to additional + "metadata" used to describe or annotate each data object. + </P> + + <h3><a name="ObjectHeader"> + Disk Format: Level 2A - Data Object Headers</a></h3> + + <P>The header information of an object is designed to encompass + all the information about an object, except for the data itself. + This information includes + the dataspace, datatype, information about how the data + is stored on disk (in external files, compressed, broken up in + blocks, etc.), as well as other information used by the library + to speed up access to the data objects or maintain a file's + integrity. Information stored by user applications as attributes + is also stored in the object's header. The header of each object is + not necessarily located immediately prior to the object's data in the + file and in fact may be located in any position in the file. The order + of the messages in an object header is not significant. + </P> + + <P>Header messages are aligned on 8-byte boundaries. + </P> + + <br> + <div align=center> + <table class=format> + <caption> + Object Headers + </caption> + + <tr> + <th>byte</th> + <th>byte</th> + <th>byte</th> + <th>byte</th> + </tr> + + <tr> + <td>Version</td> + <td>Reserved (zero)</td> + <td colspan=2>Number of Header Messages</td> + </tr> + + <tr> + <td colspan=4>Object Reference Count</td> + </tr> + + <tr> + <td colspan=4>Object Header Size</td> + </tr> + + <tr> + <td colspan=2>Header Message Type #1</td> + <td colspan=2>Size of Header Message Data #1</td> + </tr> + + <tr> + <td>Header Message #1 Flags</td> + <td colspan=3>Reserved (zero)</td> + </tr> + + <tr> + <td colspan=4><br>Header Message Data #1<br><br></td> + </tr> + + <tr> + <td colspan=4>.<br>.<br>.<br></td> + </tr> + + <tr> + <td colspan=2>Header Message Type #n</td> + <td colspan=2>Size of Header Message Data #n</td> + </tr> + + <tr> + <td>Header Message #n Flags</td> + <td colspan=3>Reserved (zero)</td> + </tr> + + <tr> + <td colspan=4><br>Header Message Data #n<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Version</td> + <td> + <P>This value is used to determine the format of the + information in the object header. When the format of the + information in the object header is changed, the version number + is incremented and can be used to determine how the + information in the object header is formatted. This + document describes version one (1) (there was no version + zero (0)). + </P> + </td> + </tr> + + <tr> + <td>Number of Header Messages</td> + <td> + <P>This value determines the number of messages listed in + object headers for this object. This value includes the messages + in continuation messages for this object. + </P> + </td> + </tr> + + <tr> + <td>Object Reference Count</td> + <td> + <P>This value specifies the number of "hard links" to this object + within the current file. References to the object from external + files, "soft links" in this file and object references in this + file are not tracked. + </P> + </td> + </tr> + + <tr> + <td>Object Header Size</td> + <td> + <P>This value specifies the number of bytes of header message data + following this length field that contain object header messages + for this object header. This value does not include the size of + object header continuation blocks for this object elsewhere in the + file. + </P> + </td> + </tr> + + <tr> + <td>Header Message Type</td> + <td> + <P>This value specifies the type of information included in the + following header message data. The header message types for the + pre-defined header messages are included in sections below. + </P> + </td> + </tr> + + <tr> + <td>Size of Header Message Data</td> + <td> + <P>This value specifies the number of bytes of header + message data following the header message type and length + information for the current message. The size includes + padding bytes to make the message a multiple of eight + bytes. + </P> + </td> + </tr> + + <tr> + <td>Header Message Flags</td> + <td> + <P>This is a bit field with the following definition: + <table class=list> + <tr> + <th width="30%">Bit</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>0</code></td> + <td>If set, the message data is constant. This is used + for messages like the datatype message of a dataset. + </td> + </tr> + <tr> + <td align=center><code>1</code></td> + <td>If set, the message is stored in the global heap. + The Header Message Data field contains a Shared Object + message and the Size of Header Message Data field + contains the size of that Shared Object message. + </td> + </tr> + <tr> + <td align=center><code>2-7</code></td> + <td>Reserved</td> + </tr> + </table> + </P> + </td> + </tr> + + <tr> + <td>Header Message Data</td> + <td> + <P>The format and length of this field is determined by the + header message type and size respectively. Some header + message types do not require any data and this information + can be eliminated by setting the length of the message to + zero. The data is padded with enough zeros to make the + size a multiple of eight. + </P> + </td> + </tr> + </table> + </div> + + <P>The header message types and the message data associated with + them compose the critical "metadata" about each object. Some + header messages are required for each object while others are + optional. Some optional header messages may also be repeated + several times in the header itself, the requirements and number + of times allowed in the header will be noted in each header + message description below. + </P> + + <P>The following is a list of currently defined header messages: + </P> + + <hr> + <h4><a name="NILMessage">Name: NIL</a></h4> + + <P class=item><B>Header Message Type: </B>0x0000 + </P> + <P class=item><B>Length:</B> varies + </P> + <P class=item><B>Status:</B> Optional, may be repeated. + </P> + <P class=item><B>Purpose and Description:</B> The NIL message is used to indicate a + message which is to be ignored when reading the header messages for a + data object. [Possibly one which has been deleted for some reason.] + </P> + <P class=item><B>Format of Data:</B> Unspecified. + </P> + + <hr> + <h4><a name="SimpleDataSpace">Name: Simple Dataspace</a></h4> + + <P class=item><B>Header Message Type: </B>0x0001 + </P> + <P class=item><B>Length:</B> Varies according to the number of dimensions, + as described in the following table. + </P> + <P class=item><B>Status:</B> Required for dataset objects, may not be + repeated. + </P> + <P class=item><B>Description:</B> The simple dataspace message describes the + number of dimensions (i.e. "rank") and size of each dimension that the + data object has. This message is only used for datasets which have a + simple, rectilinear grid layout; datasets requiring a more complex + layout (irregularly structured or unstructured grids, etc.) must use + the <em>Complex Dataspace</em> message for expressing the space the + dataset inhabits. <i>(Note: The <em>Complex Dataspace</em> + functionality is not yet implemented and it is not described in this + document.)</i> + </P> + + <P class=item><B>Format of Data:</B> + <br> + <div align=center> + <table class=format> + <caption> + Simple Dataspace Message + </caption> + + <tr> + <th>byte</th> + <th>byte</th> + <th>byte</th> + <th>byte</th> + </tr> + + <tr> + <td>Version</td> + <td>Dimensionality</td> + <td>Flags</td> + <td>Reserved</td> + </tr> + + <tr> + <td colspan=4>Reserved</td> + </tr> + + <tr> + <td colspan=4>Dimension #1 Size<sup>L</sup></td> + <tr> + <td colspan=4>.<br>.<br>.<br></td> + <tr> + <td colspan=4>Dimension #n Size<sup>L</sup></td> + <tr> + <td colspan=4>Dimension #1 Maximum Size<sup>L</sup></td> + <tr> + <td colspan=4>.<br>.<br>.<br></td> + <tr> + <td colspan=4>Dimension #n Maximum Size<sup>L</sup></td> + <tr> + <td colspan=4>Permutation Index #1<sup>L</sup></td> + <tr> + <td colspan=4>.<br>.<br>.<br></td> + <tr> + <td colspan=4>Permutation Index #n<sup>L</sup></td> + </table> + + <table class=note> + <tr><td> + (Items marked with an 'L' the above table are + <br> + of the size specified in "Size of Lengths.") + </td></tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Version</td> + <td> + <P>This value is used to determine the format of the + Simple Dataspace Message. When the format of the + information in the message is changed, the version number + is incremented and can be used to determine how the + information in the object header is formatted. This + document describes version one (1) (there was no version + zero (0)). + </P> + </td> + </tr> + + <tr> + <td>Dimensionality</td> + <td> + <P>This value is the number of dimensions that the data + object has. + </P> + </td> + </tr> + + <tr> + <td>Flags</td> + <td> + <P>This field is used to store flags to indicate the + presence of parts of this message. Bit 0 (the least + significant bit) is used to indicate that maximum + dimensions are present. Bit 1 is used to indicate that + permutation indices are present. + </P> + </td> + </tr> + + <tr> + <td>Dimension #n Size</td> + <td> + <P>This value is the current size of the dimension of the + data as stored in the file. The first dimension stored in + the list of dimensions is the slowest changing dimension + and the last dimension stored is the fastest changing + dimension. + </P> + </td> + </tr> + + <tr> + <td>Dimension #n Maximum Size</td> + <td> + <P>This value is the maximum size of the dimension of the + data as stored in the file. This value may be the special + "<A href="#UnlimitedDim">unlimited</A>" size which indicates + that the data may expand along this dimension indefinitely. + If these values are not stored, the maximum size of each + dimension is assumed to be the dimension's current size. + </P> + </td> + </tr> + + <tr> + <td>Permutation Index #n</td> + <td> + <P>This value is the index permutation used to map + each dimension from the canonical representation to an + alternate axis for each dimension. If these values are + not stored, the first dimension stored in the list of + dimensions is the slowest changing dimension and the last + dimension stored is the fastest changing dimension. + </P> + </td> + </tr> + </table> + </div> + + </P> + +<!-- + <hr> + <h4><a name="DataSpaceMessage">Name: Complex Dataspace (Fiber Bundle?)</a></h4> + <b>Header Message Type: </b>0x0002<br> + <b>Length:</b> varies<br> + + <b>Status:</b> One of the <em>Simple Dataspace</em> or + <em>Complex Dataspace</em> messages is required (but not both) and may + not be repeated.<br> <b>Purpose and Description:</b> The + <em>Dataspace</em> message describes space that the dataset is + mapped onto in a more comprehensive way than the <em>Simple + Dimensionality</em> message is capable of handling. The + dataspace of a dataset encompasses the type of coordinate system + used to locate the dataset's elements as well as the structure and + regularity of the coordinate system. The dataspace also + describes the number of dimensions which the dataset inhabits as + well as a possible higher dimensional space in which the dataset + is located within. + + <br> + <b>Format of Data:</b> + + <center> + <table border cellpadding=4 width="80%"> + <caption align=bottom> + <B>HDF5 Dataspace Message Layout</B> + </caption> + + <tr align=center> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + + <tr align=center> + <td colspan=4>Mesh Type</td> + <tr align=center> + <td colspan=4>Logical Dimensionality</td> + </table> + </center> + + <p> + <dl> + <dt>The elements of the dimensionality message are described below: + <dd> + <dl> + <dt>Mesh Type: (unsigned 32-bit integer) + <dd>This value indicates whether the grid is + polar/spherical/cartesion, + structured/unstructured and regular/irregular. <br> + The mesh type value is broken up as follows: <br> + + <P> + <center> + <table border cellpadding=4 width="80%"> + <caption align=bottom> + <B>HDF5 Mesh-type Layout</B> + </caption> + + <tr align=center> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + + <tr align=center> + <td colspan=1>Mesh Embedding</td> + <td colspan=1>Coordinate System</td> + <td colspan=1>Structure</td> + <td colspan=1>Regularity</td> + </table> + </center> + The following are the definitions of mesh-type bytes: + <dl> + <dt>Mesh Embedding + <dd>This value indicates whether the dataset dataspace + is located within + another dataspace or not: + <dl> <dl> + <dt><STANDALONE> + <dd>The dataset mesh is self-contained and is not + embedded in another mesh. + <dt><EMBEDDED> + <dd>The dataset's dataspace is located within + another dataspace, as + described in information below. + </dl> </dl> + <dt>Coordinate System + <dd>This value defines the type of coordinate system + used for the mesh: + <dl> <dl> + <dt><POLAR> + <dd>The last two dimensions are in polar + coordinates, higher dimensions are + cartesian. + <dt><SPHERICAL> + <dd>The last three dimensions are in spherical + coordinates, higher dimensions + are cartesian. + <dt><CARTESIAN> + <dd>All dimensions are in cartesian coordinates. + </dl> </dl> + <dt>Structure + <dd>This value defines the locations of the grid-points + on the axes: + <dl> <dl> + <dt><STRUCTURED> + <dd>All grid-points are on integral, sequential + locations, starting from 0. + <dt><UNSTRUCTURED> + <dd>Grid-points locations in each dimension are + explicitly defined and + may be of any numeric datatype. + </dl> </dl> + <dt>Regularity + <dd>This value defines the locations of the dataset + points on the grid: + <dl> <dl> + <dt><REGULAR> + <dd>All dataset elements are located at the + grid-points defined. + <dt><IRREGULAR> + <dd>Each dataset element has a particular + grid-location defined. + </dl> </dl> + </dl> + <p>The following grid combinations are currently allowed: + <dl> <dl> + <dt><POLAR-STRUCTURED-REGULAR> + <dt><SPHERICAL-STRUCTURED-REGULAR> + <dt><CARTESIAN-STRUCTURED-REGULAR> + <dt><POLAR-UNSTRUCTURED-REGULAR> + <dt><SPHERICAL-UNSTRUCTURED-REGULAR> + <dt><CARTESIAN-UNSTRUCTURED-REGULAR> + <dt><CARTESIAN-UNSTRUCTURED-IRREGULAR> + </dl> </dl> + All of the above grid types can be embedded within another + dataspace. + <br> <br> + <dt>Logical Dimensionality: (unsigned 32-bit integer) + <dd>This value is the number of dimensions that the dataset occupies. + + <P> + <center> + <table border cellpadding=4 width="80%"> + <caption align=bottom> + <B>HDF5 Dataspace Embedded Dimensionality Information</B> + </caption> + + <tr align=center> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + + <tr align=center> + <td colspan=4>Embedded Dimensionality</td> + <tr align=center> + <td colspan=4>Embedded Dimension Size #1</td> + <tr align=center> + <td colspan=4>.<br>.<br>.<br></td> + <tr align=center> + <td colspan=4>Embedded Dimension Size #n</td> + <tr align=center> + <td colspan=4>Embedded Origin Location #1</td> + <tr align=center> + <td colspan=4>.<br>.<br>.<br></td> + <tr align=center> + <td colspan=4>Embedded Origin Location #n</td> + </table> + </center> + + <dt>Embedded Dimensionality: (unsigned 32-bit integer) + <dd>This value is the number of dimensions of the space the + dataset is located + within. i.e. a planar dataset located within a 3-D space, + or a 3-D dataset + which is a subset of another 3-D space, etc. + <dt>Embedded Dimension Size: (unsigned 32-bit integer) + <dd>These values are the sizes of the dimensions of the + embedded dataspace + that the dataset is located within. + <dt>Embedded Origin Location: (unsigned 32-bit integer) + <dd>These values comprise the location of the dataset's + origin within the embedded dataspace. + </dl> + </dl> + [Comment: need some way to handle different orientations of the + dataset dataspace + within the embedded dataspace]<br> + + <P> + <center> + <table border cellpadding=4 width="80%"> + <caption align=bottom> + <B>HDF5 Dataspace Structured/Regular Grid Information</B> + </caption> + + <tr align=center> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + + <tr align=center> + <td colspan=4>Logical Dimension Size #1</td> + <tr align=center> + <td colspan=4>Logical Dimension Maximum #1</td> + <tr align=center> + <td colspan=4>.<br>.<br>.<br></td> + <tr align=center> + <td colspan=4>Logical Dimension Size #n</td> + <tr align=center> + <td colspan=4>Logical Dimension Maximum #n</td> + </table> + </center> + + <p> + <dl> + <dt>The elements of the dimensionality message are described below: + <dd> + <dl> + <dt>Logical Dimension Size #n: (unsigned 32-bit integer) + <dd>This value is the current size of the dimension of the + data as stored in + the file. The first dimension stored in the list of + dimensions is the slowest + changing dimension and the last dimension stored is the + fastest changing + dimension. + <dt>Logical Dimension Maximum #n: (unsigned 32-bit integer) + <dd>This value is the maximum size of the dimension of the + data as stored in + the file. This value may be the special value + <UNLIMITED> which + indicates that the data may expand along this dimension + indefinitely. + </dl> + </dl> + <P> + <center> + <table border cellpadding=4 width="80%"> + <caption align=bottom> + <B>HDF5 Dataspace Structured/Irregular Grid Information</B> + </caption> + + <tr align=center> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + + <tr align=center> + <td colspan=4># of Grid Points in Dimension #1</td> + <tr align=center> + <td colspan=4>.<br>.<br>.<br></td> + <tr align=center> + <td colspan=4># of Grid Points in Dimension #n</td> + <tr align=center> + <td colspan=4>Datatype of Grid Point Locations</td> + <tr align=center> + <td colspan=4>Location of Grid Points in Dimension #1</td> + <tr align=center> + <td colspan=4>.<br>.<br>.<br></td> + <tr align=center> + <td colspan=4>Location of Grid Points in Dimension #n</td> + </table> + </center> + + <P> + <center> + <table border cellpadding=4 width="80%"> + <caption align=bottom> + <B>HDF5 Dataspace Unstructured Grid Information</B> + </caption> + + <tr align=center> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + + <tr align=center> + <td colspan=4># of Grid Points</td> + <tr align=center> + <td colspan=4>Datatype of Grid Point Locations</td> + <tr align=center> + <td colspan=4>Grid Point Locations<br>.<br>.<br></td> + </table> + </center> +--> + + <hr> + <h4><a name="ReservedMessage_0002">Name: Reserved - Not Assigned Yet</a></h4> + <b>Header Message Type:</b> 0x0002<BR> + <b>Length:</b> N/A<BR> + <b>Status:</b> N/A<BR> + <b>Format of Data:</b> N/A<BR> + + <p><b>Purpose and Description:</b> This message type was skipped during + the initial specification of the file format and may be used in a + future expansion to the format. + + + <hr> + <h4><a name="DataTypeMessage">Name: Datatype</a></h4> + + <P class=item><B>Header Message Type:</B> 0x0003 + </P> + <P class=item><B>Length:</B> variable + </P> + <P class=item><B>Status:</B> Required for dataset or named datatype objects, + may not be repeated. + </P> + + <P class=item><B>Description:</B> The datatype message defines the datatype + for each element of a dataset. A datatype can describe an atomic type + like a fixed- or floating-point type or a compound type like a C + struct. + Datatypes messages are stored + as a list of datatype classes and + their associated properties. + </P> + + <P class=item2>Datatype messages that are part of a dataset object, + do not describe how elements are related to one another, the dataspace + message is used for that purpose. Datatype messages that are part of + a named datatype message describe an "abstract" datatype that can be + used by other objects in the file. + </P> + + <P class=item><B>Format of Data:</B> + <br> + <div align=center> + <table class=format> + <caption> + Datatype Message + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td>Class and Version</td> + <td>Class Bit Field, Bits 0-7</td> + <td>Class Bit Field, Bits 8-15</td> + <td>Class Bit Field, Bits 16-23</td> + </tr> + + <tr> + <td colspan=4>Size</td> + </tr> + + <tr> + <td colspan=4><br><br>Properties<br><br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Class and Version</td> + <td> + <P>The version of the datatype message and the datatype's class + information are packed together in this field. The version + number is packed in the top 4 bits of the field and the class + is contained in the bottom 4 bits. + </P> + <P>The version number information is used for changes in the + format of the datatype message and is described here: + <table class=list> + <tr> + <th width="30%">Version</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>0</code></td> + <td>Never used + </td> + </tr> + <tr> + <td align=center><code>1</code></td> + <td>Used by early versions of the library to encode + compound datatypes with explicit array fields. + See the compound datatype description below for + further details. + </td> + </tr> + <tr> + <td align=center><code>2</code></td> + <td>The current version used by the library. + </td> + </tr> + </table> + </P> + <P>The class of the datatype determines the format for the class + bit field and properties portion of the datatype message, which + are described below. The + following classes are currently defined: + <table width=100% class=list> + <tr> + <th width="30%">Value</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>0</code></td> + <td>Fixed-Point</td> + </tr> + + <tr> + <td align=center><code>1</code></td> + <td>Floating-Point</td> + </tr> + + <tr> + <td align=center><code>2</code></td> + <td>Time</td> + </tr> + + <tr> + <td align=center><code>3</code></td> + <td>String</td> + </tr> + + <tr> + <td align=center><code>4</code></td> + <td>Bitfield</td> + </tr> + + <tr> + <td align=center><code>5</code></td> + <td>Opaque</td> + </tr> + + <tr> + <td align=center><code>6</code></td> + <td>Compound</td> + </tr> + + <tr> + <td align=center><code>7</code></td> + <td>Reference</td> + </tr> + + <tr> + <td align=center><code>8</code></td> + <td>Enumerated</td> + </tr> + + <tr> + <td align=center><code>9</code></td> + <td>Variable-Length</td> + </tr> + + <tr> + <td align=center><code>10</code></td> + <td>Array</td> + </tr> + </table> + </P> + </td> + </tr> + + <tr> + <td>Class Bit Fields</td> + <td> + <P>The information in these bit fields is specific to each datatype + class and is described below. All bits not defined for a + datatype class are set to zero. + </P> + </td> + </tr> + + <tr> + <td>Size</td> + <td> + <P>The size of the datatype in bytes. + </P> + </td> + </tr> + + <tr> + <td>Properties</td> + <td> + <P>This variable-sized field encodes information specific to each + datatype class and is described below. If there is no + property information specified for a datatype class, the size + of this field is zero. + </P> + </td> + </tr> + + </table> + </div> + </P> + + <P>Class specific information for Fixed-Point Numbers (Class 0): + + <br> + <div align=center> + <table class=desc> + <caption> + Bit Field Description + </caption> + + <tr> + <th width="10%">Bits</th> + <th>Meaning</th> + </tr> + + <tr> + <td>0</td> + <td><b>Byte Order.</b> If zero, byte order is little-endian; + otherwise, byte order is big endian.</td> + </tr> + + <tr> + <td>1, 2</td> + <td><b>Padding type.</b> Bit 1 is the lo_pad type and bit 2 + is the hi_pad type. If a datum has unused bits at either + end, then the lo_pad or hi_pad bit is copied to those + locations.</td> + </tr> + + <tr> + <td>3</td> + <td><b>Signed.</b> If this bit is set then the fixed-point + number is in 2's complement form.</td> + </tr> + + <tr> + <td>4-23</td> + <td>Reserved (zero).</td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=format> + <caption> + Property Descriptions + </caption> + + <tr> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + </tr> + + <tr> + <td colspan=2>Bit Offset</td> + <td colspan=2>Bit Precision</td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Bit Offset</td> + <td> + <P>The bit offset of the first significant bit of the fixed-point + value within the datatype. The bit offset specifies the number + of bits "to the right of" the value. + </P> + </td> + </tr> + + <tr> + <td>Bit Precision</td> + <td> + <P>The number of bits of precision of the fixed-point value + within the datatype. + </P> + </td> + </tr> + + </table> + </div> + </P> + + <P>Class specific information for Floating-Point Numbers (Class 1): + + <br> + <div align=center> + <table class=desc> + <caption> + Bit Field Description + </caption> + + <tr> + <th width="10%">Bits</th> + <th>Meaning</th> + </tr> + + <tr> + <td>0</td> + <td><b>Byte Order.</b> If zero, byte order is little-endian; + otherwise, byte order is big endian.</td> + </tr> + + <tr> + <td>1, 2, 3</td> + <td><b>Padding type.</b> Bit 1 is the low bits pad type, bit 2 + is the high bits pad type, and bit 3 is the internal bits + pad type. If a datum has unused bits at either end or between + the sign bit, exponent, or mantissa, then the value of bit + 1, 2, or 3 is copied to those locations.</td> + </tr> + + <tr> + <td>4-5</td> + <td><b>Normalization.</b> The value can be 0 if there is no + normalization, 1 if the most significant bit of the + mantissa is always set (except for 0.0), and 2 if the most + signficant bit of the mantissa is not stored but is + implied to be set. The value 3 is reserved and will not + appear in this field.</td> + </tr> + + <tr> + <td>6-7</td> + <td>Reserved (zero).</td> + </tr> + + <tr> + <td>8-15</td> + <td><b>Sign Location.</b> This is the bit position of the sign + bit. Bits are numbered with the least significant bit zero.</td> + </tr> + + <tr> + <td>16-23</td> + <td>Reserved (zero).</td> + </tr> + + </table> + </div> + + <br> + <div align=center> + <table class=format> + <caption> + Property Descriptions + </caption> + + <tr> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + </tr> + + <tr> + <td colspan=2>Bit Offset</td> + <td colspan=2>Bit Precision</td> + </tr> + + <tr> + <td>Exponent Location</td> + <td>Exponent Size</td> + <td>Mantissa Location</td> + <td>Mantissa Size</td> + </tr> + + <tr> + <td colspan=4>Exponent Bias</td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Bit Offset</td> + <td> + <P>The bit offset of the first significant bit of the floating-point + value within the datatype. The bit offset specifies the number + of bits "to the right of" the value. + </P> + </td> + </tr> + + <tr> + <td>Bit Precision</td> + <td> + <P>The number of bits of precision of the floating-point value + within the datatype. + </P> + </td> + </tr> + + <tr> + <td>Exponent Location</td> + <td> + <P>The bit position of the exponent field. Bits are numbered with + the least significant bit number zero. + </P> + </td> + </tr> + + <tr> + <td>Exponent Size</td> + <td> + <P>The size of the exponent field in bits. + </P> + </td> + </tr> + + <tr> + <td>Mantissa Location</td> + <td> + <P>The bit position of the mantissa field. Bits are numbered with + the least significant bit number zero. + </P> + </td> + </tr> + + <tr> + <td>Mantissa Size</td> + <td> + <P>The size of the mantissa field in bits. + </P> + </td> + </tr> + + <tr> + <td>Exponent Bias</td> + <td> + <P>The bias of the exponent field. + </P> + </td> + </tr> + + </table> + </div> + </P> + + <P>Class specific information for Time (Class 2): + + <br> + <div align=center> + <table class=desc> + <caption> + Bit Field Description + </caption> + + <tr> + <th width="10%">Bits</th> + <th>Meaning</th> + </tr> + + <tr> + <td>0</td> + <td><b>Byte Order.</b> If zero, byte order is little-endian; + otherwise, byte order is big endian.</td> + </tr> + + <tr> + <td>1-23</td> + <td>Reserved (zero).</td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=format> + <caption> + Property Descriptions + </caption> + + <tr> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + </tr> + + <tr> + <td colspan=2>Bit Precision</td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Bit Precision</td> + <td> + <P>The number of bits of precision of the time value. + </P> + </td> + </tr> + + </table> + </div> + </P> + + <P>Class specific information for Strings (Class 3): + + <br> + <div align=center> + <table class=desc> + <caption> + Bit Field Description + </caption> + + <tr> + <th width="10%">Bits</th> + <th>Meaning</th> + </tr> + + <tr> + <td>0-3</td> + <td><b>Padding type.</b> This four-bit value determines the + type of padding to use for the string. The values are: + + <table width=100% class=list> + <tr> + <th width="30%">Value</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>0</code></td> + <td>Null Terminate: A zero byte marks the end of the + string and is guaranteed to be present after + converting a long string to a short string. When + converting a short string to a long string the value is + padded with additional null characters as necessary. + </td> + </tr> + + <tr> + <td align=center><code>1</code></td> + <td>Null Pad: Null characters are added to the end of + the value during conversions from short values to long + values but conversion in the opposite direction simply + truncates the value. + </td> + </tr> + + <tr> + <td align=center><code>2</code></td> + <td>Space Pad: Space characters are added to the end of + the value during conversions from short values to long + values but conversion in the opposite direction simply + truncates the value. This is the Fortran + representation of the string. + </td> + </tr> + + <tr> + <td align=center><code>3-15</code></td> + <td>Reserved + </td> + </tr> + </table> + </tr> + + <tr> + <td>4-7</td> + <td><b>Character Set.</b> The character set to use for + encoding the string. The only character set supported is + the 8-bit ASCII (zero) so no translations have been defined + yet.</td> + </tr> + + <tr> + <td>8-23</td> + <td>Reserved (zero).</td> + </tr> + </table> + </div> + + <P>There are no properties defined for the string class. + </P> + </P> + + <P>Class specific information for Bitfields (Class 4): + + <br> + <div align=center> + <table class=desc> + <caption> + Bit Field Description + </caption> + + <tr> + <th width="10%">Bits</th> + <th>Meaning</th> + </tr> + + <tr> + <td>0</td> + <td><b>Byte Order.</b> If zero, byte order is little-endian; + otherwise, byte order is big endian.</td> + </tr> + + <tr> + <td>1, 2</td> + <td><b>Padding type.</b> Bit 1 is the lo_pad type and bit 2 + is the hi_pad type. If a datum has unused bits at either + end, then the lo_pad or hi_pad bit is copied to those + locations.</td> + </tr> + + <tr> + <td>3-23</td> + <td>Reserved (zero).</td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=format> + <caption> + Property Description + </caption> + + <tr> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + </tr> + + <tr> + <td colspan=2>Bit Offset</td> + <td colspan=2>Bit Precision</td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Bit Offset</td> + <td> + <P>The bit offset of the first significant bit of the bitfield + within the datatype. The bit offset specifies the number + of bits "to the right of" the value. + </P> + </td> + </tr> + + <tr> + <td>Bit Precision</td> + <td> + <P>The number of bits of precision of the bitfield + within the datatype. + </P> + </td> + </tr> + </table> + </div> + </P> + + <P>Class specific information for Opaque (Class 5): + + <br> + <div align=center> + <table class=desc> + <caption> + Bit Field Description + </caption> + + <tr> + <th width="10%">Bits</th> + <th>Meaning</th> + </tr> + + <tr> + <td>0-7</td> + <td>Length of ASCII tag in bytes.</td> + </tr> + + <tr> + <td>8-23</td> + <td>Reserved (zero).</td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=format> + <caption> + Property Description + </caption> + + <tr> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + </tr> + + <tr> + <td colspan=4><br>ASCII Tag<br> + <br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>ASCII Tag</td> + <td> + <P>This NUL-terminated string provides a description for the + opaque type. It is NUL-padded to a multiple of 8 bytes. + </P> + </td> + </tr> + </table> + </div> + </P> + + <P>Class specific information for Compound (Class 6): + + <br> + <div align=center> + <table class=desc> + <caption> + Bit Field Description + </caption> + + <tr> + <th width="10%">Bits</th> + <th>Meaning</th> + </tr> + + <tr> + <td>0-15</td> + <td><b>Number of Members.</b> This field contains the number + of members defined for the compound datatype. The member + definitions are listed in the Properties field of the data + type message. + </tr> + + <tr> + <td>15-23</td> + <td>Reserved (zero).</td> + </tr> + </table> + </div> + </P> + + <P>The Properties field of a compound datatype is a list of the + member definitions of the compound datatype. The member + definitions appear one after another with no intervening bytes. + The member types are described with a recursive datatype + message. + + <P>Note that the property descriptions are different for different + versions of the datatype version. Additionally note that the version + 0 properties are deprecated and have been replaced with the version + 1 properties in versions of the HDF5 library from the 1.4 release + onward. + + <br> + <div align=center> + <table class=format> + <caption> + Properties Description for Datatype Version 1 + </caption> + + <tr> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + </tr> + + <tr> + <td colspan=4><br>Name<br><br></td> + </tr> + + <tr> + <td colspan=4>Byte Offset of Member</td> + </tr> + + <tr> + <td>Dimensionality</td> + <td colspan=3>Reserved (zero)</td> + </tr> + + <tr> + <td colspan=4>Dimension Permutation</td> + </tr> + + <tr> + <td colspan=4>Reserved (zero)</td> + </tr> + + <tr> + <td colspan=4>Dimension #1 Size (required)</td> + </tr> + + <tr> + <td colspan=4>Dimension #2 Size (required)</td> + </tr> + + <tr> + <td colspan=4>Dimension #3 Size (required)</td> + </tr> + + <tr> + <td colspan=4>Dimension #4 Size (required)</td> + </tr> + + <tr> + <td colspan=4><br>Member Type Message<br><br></td> + </tr> + + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Name</td> + <td> + <P>This NUL-terminated string provides a description for the + opaque type. It is NUL-padded to a multiple of 8 bytes. + </P> + </td> + </tr> + + <tr> + <td>Byte Offset of Member</td> + <td> + <P>This is the byte offset of the member within the datatype. + </P> + </td> + </tr> + + <tr> + <td>Dimensionality</td> + <td> + <P>If set to zero, this field indicates a scalar member. If set + to a value greater than zero, this field indicates that the + member is an array of values. For array members, the size of + the array is indicated by the 'Size of Dimension n' field in + this message. + </P> + </td> + </tr> + + <tr> + <td>Dimension Permutation</td> + <td> + <P>This field was intended to allow an array field to have + it's dimensions permuted, but this was never implemented. + This field should always be set to zero. + </P> + </td> + </tr> + + <tr> + <td>Dimension #n Size</td> + <td> + <P>This field is the size of a dimension of the array field as + stored in the file. The first dimension stored in the list of + dimensions is the slowest changing dimension and the last + dimension stored is the fastest changing dimension. + </P> + </td> + </tr> + + <tr> + <td>Member Type Message</td> + <td> + <P>This field is a datatype message describing the datatype of + the member. + </P> + </td> + </tr> + + </table> + </div> + + <br> + <div align=center> + <table class=format> + <caption> + Properties Description for Datatype Version 2 + </caption> + + <tr> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + </tr> + + <tr> + <td colspan=4><br>Name<br><br></td> + </tr> + + <tr> + <td colspan=4>Byte Offset of Member</td> + </tr> + + <tr> + <td colspan=4><br>Member Type Message<br><br></td> + </tr> + + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Name</td> + <td> + <P>This NUL-terminated string provides a description for the + opaque type. It is NUL-padded to a multiple of 8 bytes. + </P> + </td> + </tr> + + <tr> + <td>Byte Offset of Member</td> + <td> + <P>This is the byte offset of the member within the datatype. + </P> + </td> + </tr> + + <tr> + <td>Member Type Message</td> + <td> + <P>This field is a datatype message describing the datatype of + the member. + </P> + </td> + </tr> + + </table> + </div> + </P> + + <P>Class specific information for Reference (Class 7): + + <br> + <div align=center> + <table class=desc> + <caption> + Bit Field Description + </caption> + + <tr> + <th width="10%">Bits</th> + <th>Meaning</th> + </tr> + + <tr> + <td>0-3</td> + <td><b>Type.</b> This four-bit value contains the type of reference + described. The values defined are: + + <table width=100% class=list> + <tr> + <th width="30%">Value</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>0</code></td> + <td>Object Reference: A reference to another object in this + HDF5 file. + </td> + </tr> + + <tr> + <td align=center><code>1</code></td> + <td>Dataset Region Reference: A reference to a region within + a dataset in this HDF5 file. + </td> + </tr> + + <tr> + <td align=center><code>2</code></td> + <td>Internal Reference: A reference to a region within the + current dataset. (Not currently implemented) + </td> + </tr> + + <tr> + <td align=center><code>3-15</code></td> + <td>Reserved + </td> + </tr> + </table> + + </td> + </tr> + + <tr> + <td>15-23</td> + <td>Reserved (zero).</td> + </tr> + </table> + </div> + + <P>There are no properties defined for the reference class. + </P> + </P> + + <P>Class specific information for Enumeration (Class 8): + + <br> + <div align=center> + <table class=desc> + <caption> + Bit Field Description + </caption> + + <tr> + <th width="10%">Bits</th> + <th>Meaning</th> + </tr> + + <tr> + <td>0-15</td> + <td><b>Number of Members.</b> The number of name/value + pairs defined for the enumeration type.</td> + </tr> + + <tr> + <td>16-23</td> + <td>Reserved (zero).</td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=format> + <caption> + Property Description + </caption> + + <tr> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + </tr> + + <tr> + <td colspan=4><br>Base Type<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Names<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Values<br><br></td> + </tr> + + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Base Type</td> + <td> + <P>Each enumeration type is based on some parent type, usually an + integer. The information for that parent type is described + recursively by this field. + </P> + </td> + </tr> + + <tr> + <td>Names</td> + <td> + <P>The name for each name/value pair. Each name is stored as a null + terminated ASCII string in a multiple of eight bytes. The names + are in no particular order. + </P> + </td> + </tr> + + <tr> + <td>Values</td> + <td> + <P>The list of values in the same order as the names. The values + are packed (no inter-value padding) and the size of each value + is determined by the parent type. + </P> + </td> + </tr> + + </table> + </div> + </P> + + + <P>Class specific information for Variable-Length (Class 9): + + <br> + <div align=center> + <table class=desc> + <caption> + Bit Field Description + </caption> + + <tr> + <th width="10%">Bits</th> + <th>Meaning</th> + </tr> + + <tr> + <td>0-3</td> + <td><b>Type.</b> This four-bit value contains the type of + variable-length datatype described. The values defined are: + + <table width=100% class=list> + <tr> + <th width="30%">Value</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>0</code></td> + <td>Sequence: A variable-length sequence of any sequence of + data. Variable-length sequences do not have padding or + character set information. + </td> + </tr> + + <tr> + <td align=center><code>1</code></td> + <td>String: A variable-length sequence of characters. + Variable-length strings have padding and character set + information. + </td> + </tr> + + <tr> + <td align=center><code>2-15</code></td> + <td>Reserved + </td> + </tr> + </table> + + </td> + </tr> + + <tr> + <td>4-7</td> + <td><b>Padding type.</b> (variable-length string only) + This four-bit value determines the type of padding + used for variable-length strings. The values are the same + as for the string padding type, as follows: + <table width=100% class=list> + <tr> + <th width="30%">Value</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>0</code></td> + <td>Null terminate: A zero byte marks the end of a string + and is guaranteed to be present after converting a long + string to a short string. When converting a short string + to a long string, the value is padded with additional null + characters as necessary. + </td> + </tr> + + <tr> + <td align=center><code>1</code></td> + <td>Null pad: Null characters are added to the end of the + value during conversion from a short string to a longer + string. Conversion from a long string to a shorter string + simply truncates the value. + </td> + </tr> + + <tr> + <td align=center><code>2</code></td> + <td>Space pad: Space characters are added to the end of the + value during conversion from a short string to a longer + string. Conversion from a long string to a shorter string + simply truncates the value. This is the Fortran + representation of the string. + </td> + </tr> + + <tr> + <td align=center><code>3-15</code></td> + <td>Reserved + </td> + </tr> + </table> + + This value is set to zero for variable-length sequences. + + </td> + </tr> + + <tr> + <td>8-11</td> + <td><b>Character Set.</b> (variable-length string only) + This four-bit value specifies the character set + to be used for encoding the string: + <table width=100% class=list> + <tr> + <th width="30%">Value</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>0</code></td> + <td>ASCII: As of this writing (July 2003, Release 1.6.0), + 8-bit ASCII is the only character set supported. Therefore, + no translations have been defined. + </td> + </tr> + + <tr> + <td align=center><code>1-15</code></td> + <td>Reserved + </td> + </tr> + </table> + + This value is set to zero for variable-length sequences. + + </td> + </tr> + + <tr> + <td>12-23</td> + <td>Reserved (zero).</td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=format> + <caption> + Property Description + </caption> + + <tr> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + </tr> + + <tr> + <td colspan=4><br>Base Type<br><br></td> + </tr> + + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Base Type</td> + <td> + <P>Each variable-length type is based on some parent type. The + information for that parent type is described recursively by + this field. + </P> + </td> + </tr> + + </table> + </div> + </P> + + <P>Class specific information for Array (Class 10): + + <P>There are no bit fields defined for the array class. + </P> + + <P>Note that the dimension information defined in the property for this + datatype class is independent of dataspace information for a dataset. + The dimension information here describes the dimensionality of the + information within a data element (or a component of an element, if the + array datatype is nested within another datatype) and the dataspace for a + dataset describes the location of the elements in a dataset. + </P> + + <br> + <div align=center> + <table class=format> + <caption> + Property Description + </caption> + + <tr> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + <th width="25%">Byte</th> + </tr> + + <tr> + <td>Dimensionality</td> + <td colspan=3>Reserved (zero)</td> + </tr> + + <tr> + <td colspan=4>Dimension #1 Size</td> + </tr> + <tr> + <td colspan=4>.<br>.<br>.<br></td> + </tr> + <tr> + <td colspan=4>Dimension #n Size</td> + </tr> + + <tr> + <td colspan=4>Permutation Index #1</td> + </tr> + <tr> + <td colspan=4>.<br>.<br>.<br></td> + </tr> + <tr> + <td colspan=4>Permutation Index #n</td> + </tr> + + <tr> + <td colspan=4><br>Base Type<br><br></td> + </tr> + + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Dimensionality</td> + <td> + <P>This value is the number of dimensions that the array has. + </P> + </td> + </tr> + + <tr> + <td>Dimension #n Size</td> + <td> + <P>This value is the size of the dimension of the array + as stored in the file. The first dimension stored in + the list of dimensions is the slowest changing dimension + and the last dimension stored is the fastest changing + dimension. + </P> + </td> + </tr> + + <tr> + <td>Permutation Index #n</td> + <td> + <P>This value is the index permutation used to map + each dimension from the canonical representation to an + alternate axis for each dimension. Currently, dimension + permutations are not supported and these indices should be set + to the index position minus one (i.e. the first dimension should + be set to 0, the second dimension should be set to 1, etc.) + </P> + </td> + </tr> + + <tr> + <td>Base Type</td> + <td> + <P>Each array type is based on some parent type. The + information for that parent type is described recursively by + this field. + </P> + </td> + </tr> + + </table> + </div> + + </P> + + <hr> + <h4><a name="OldFillValueMessage">Name: Data Storage - Fill Value (Old)</a></h4> + + <P class=item><B>Header Message Type:</B> 0x0004 + </P> + <P class=item><B>Length:</B> varies + </P> + <P class=item><B>Status:</B> Optional, may not be repeated. + </P> + + <P class=item><B>Description:</B> The fill value message stores a single + data value which is returned to the application when an uninitialized + data element is read from a dataset. The fill value is interpreted + with the same datatype as the dataset. If no fill value message is + present then a fill value of all zero bytes is assumed. + </P> + + <P class=item2>This fill value message is deprecated in favor of the "new" + fill value message (Message Type 0x0005) and is only written to the + file for forward compatibility with versions of the HDF5 library before + the 1.6.0 version. Additionally, it only appears for datasets with a + user defined fill value (as opposed to the library default fill value + or an explicitly set "undefined" fill value). + </P> + + <P class=item><B>Format of Data:</B> + <br> + <div align=center> + <table class=format> + <caption> + Fill Value Message (Old) + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td colspan=4>Size</td> + </tr> + + <tr> + <td colspan=4><br>Fill Value<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Size</td> + <td> + <P>This is the size of the Fill Value field in bytes. + </P> + </td> + </tr> + + <tr> + <td>Fill Value</td> + <td> + <P>The fill value. The bytes of the fill value are interpreted + using the same datatype as for the dataset. + </P> + </td> + </tr> + </table> + </div> + </P> + + <hr> + <h4><a name="FillValueMessage">Name: Data Storage - Fill Value </a></h4> + + <P class=item><B>Header Message Type:</B> 0x0005 + </P> + <P class=item><B>Length:</B> varies + </P> + <P class=item><B>Status:</B> Required for dataset objects, may not be repeated. + </P> + + <P class=item><B>Description:</B> The fill value message stores a single + data value which is returned to the application when an uninitialized + data element is read from a dataset. The fill value is interpreted + with the same datatype as the dataset. + </P> + + <P class=item><B>Format of Data:</B> + <br> + <div align=center> + <table class=format> + <caption> + Fill Value Message + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td>Version</td> + <td>Space Allocation Time</td> + <td>Fill Value Write Time</td> + <td>Fill Value Defined</td> + </tr> + + <tr> + <td colspan=4>Size</td> + </tr> + + <tr> + <td colspan=4><br>Fill Value<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Version</td> + <td> + <P>The version number information is used for changes in the + format of the fill value message and is described here: + <table class=list> + <tr> + <th width="30%">Version</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>0</code></td> + <td>Never used + </td> + </tr> + <tr> + <td align=center><code>1</code></td> + <td>Used by version 1.6.x of the library to encode + fill values. In this version, the Size field is + always present. + </td> + </tr> + <tr> + <td align=center><code>2</code></td> + <td>The current version used by the library (version + 1.7.3 or later). In this version, the Size and + Fill Value fields are + only present if the Fill Value Defined field is set + to 1. + </td> + </tr> + </table> + </P> + </td> + </tr> + + <tr> + <td>Space Allocation Time</td> + <td> + <P>When the storage space for the dataset's raw data will be + allocated. The allowed values are: + <table class=list> + <tr> + <th width="30%">Value</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>1</code></td> + <td>Early allocation. Storage space for the entire dataset + should be allocated in the file when the dataset is + created. + </td> + </tr> + <tr> + <td align=center><code>2</code></td> + <td>Late allocation. Storage space for the entire dataset + should not be allocated until the dataset is written + to. + </td> + </tr> + <tr> + <td align=center><code>3</code></td> + <td>Incremental allocation. Storage space for the + dataset should not be allocated until the portion + of the dataset is written to. This is currently + used in conjunction with chunked data storage for + datasets. + </td> + </tr> + </table> + </P> + </td> + </tr> + + <tr> + <td>Fill Value Write Time</td> + <td> + <P>At the time that storage space for the dataset's raw data is + allocated, this value indicates whether the fill value should + be written to the raw data storage elements. The allowed values + are: + <table class=list> + <tr> + <th width="30%">Value</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>0</code></td> + <td>On allocation. The fill value is always written to + the raw data storage when the storage space is allocated. + </td> + </tr> + <tr> + <td align=center><code>1</code></td> + <td>Never. The fill value should never be written to + the raw data storage. + </td> + </tr> + <tr> + <td align=center><code>2</code></td> + <td>Fill value written if set by user. The fill value + will be written to the raw data storage when the storage + space is allocated only if the user explicitly set + the fill value. If the fill value is the library + default or is undefined, it will not be written to + the raw data storage. + </td> + </tr> + </table> + </P> + </td> + </tr> + + <tr> + <td>Fill Value Defined</td> + <td> + <P>This value indicates if a fill value is defined for this + dataset. If this value is 0, the fill value is undefined. + If this value is 1, a fill value is defined for this dataset. + For version 2 or later of the fill value message, this value + controls the presence of the Size field. + </P> + </td> + </tr> + + <tr> + <td>Size</td> + <td> + <P>This is the size of the Fill Value field in bytes. This field + is not present if the Version field is >1 and the Fill Value + Defined field is set to 0. + </P> + </td> + </tr> + + <tr> + <td>Fill Value</td> + <td> + <P>The fill value. The bytes of the fill value are interpreted + using the same datatype as for the dataset. This field is + not present if the Version field is >1 and the Fill Value + Defined field is set to 0. + </P> + </td> + </tr> + </table> + </div> + </P> + +<!-- + <hr> + <h4><a name="CompactDataStorageMessage">Name: Data Storage - Compact</a></h4> + + <b>Header Message Type:</b> 0x0006<br> + <b>Length:</b> varies<br> + <b>Status:</b> Optional, may not be repeated.<br> + + <p>This message indicates that the data for the data object is + stored within the current HDF file by including the actual + data as the header data for this message. The data is + stored internally in + the <em>normal format</em>, i.e. in one chunk, uncompressed, etc. + + <P>Note that one and only one of the <em>Data Storage</em> headers can be + stored for each data object. + + <P><b>Format of Data:</b> The message data is actually composed + of dataset data, so the format will be determined by the dataset + format. +--> + + <hr> + <h4><a name="ReservedMessage_0006">Name: Reserved - Not Assigned Yet</a></h4> + <P class=item><B>Header Message Type:</B> 0x0006</P> + <P class=item><B>Length:</B> N/A</P> + <P class=item><B>Status:</B> N/A</P> + <P class=item><B>Format of Data:</B> N/A</P> + + <P class=item><B>Purpose and Description:</B> This message type was skipped during + the initial specification of the file format and may be used in a + future expansion to the format.</P> + + <hr> + <h4><a name="ExternalFileListMessage">Name: Data Storage - + External Data Files</a></h4> + <P class=item><B>Header Message Type:</B> 0x0007 </P> + <P class=item><B>Length:</B> varies</P> + <P class=item><B>Status:</B> Optional, may not be repeated.</P> + + <P class=item><B>Purpose and Description:</B> The external object message + indicates that the data for an object is stored outside the HDF5 + file. The filename of the object is stored as a Universal + Resource Location (URL) of the actual filename containing the + data. An external file list record also contains the byte offset + of the start of the data within the file and the amount of space + reserved in the file for that data.</P> + + <P class=item><B>Format of Data:</B> + <br> + <div align=center> + <table class=format> + <caption> + External File List Message + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td>Version</td> + <td colspan=3>Reserved</td> + </tr> + + <tr> + <td colspan=2>Allocated Slots</td> + <td colspan=2>Used Slots</td> + </tr> + + <tr> + <td colspan=4><br>Heap Address<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Slot Definitions...<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Version</td> + <td> + <P>The version number information is used for changes in the format of External File + List Message and is described here: + <table class=list> + <tr> + <th width="30%">Version</th> + <th align=left>Description</th> + </tr> + <tr> + <td align=center><code>0</code></td> + <td>Never used. + </tr> + <tr> + <td align=center><code>1</code></td> + <td>The current version used by the library. + </tr> + </table> + </P> + </td> + </tr> + + <tr> + <td>Reserved</td> + <td> + <P>This field is reserved for future use.</P> + </td> + </tr> + + <tr> + <td>Allocated Slots</td> + <td> + <P>The total number of slots allocated in the message. Its value must be at least as + large as the value contained in the Used Slots field. (The current library simply + uses the number of Used Slots for this message)</P> + </td> + </tr> + + <tr> + <td>Used Slots</td> + <td> + <P>The number of initial slots which contains valid information.</P> + </td> + </tr> + + <tr> + <td>Heap Address</td> + <td> + <P>This is the address of a local heap which contains the names for the external + files (The local heap information can be found in Disk Format Level 1D in this + document). The name at offset zero in the heap is always the empty string.</P> + </td> + </tr> + + <tr> + <td>Slot Definitions</td> + <td> + <P>The slot definitions are stored in order according to the array addresses they + represent.</P> + </td> + </tr> + + </table> + </div> + + <br> + <div align=center> + <table class=format> + <caption> + External File List Slot + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td colspan=4><br>Name Offset(<size> bytes)<br><br></td> + </tr> + + <tr> + <td colspan=4><br>File Offset(<size> bytes)<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Size<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Name Offset(<size> bytes)</td> + <td> + <P>The byte offset within the local name heap for the name + of the file. File names are stored as a URL which has a + protocol name, a host name, a port number, and a file + name: + <code><em>protocol</em>:<em>port</em>//<em>host</em>/<em>file</em></code>. + If the protocol is omitted then "file:" is assumed. If + the port number is omitted then a default port for that + protocol is used. If both the protocol and the port + number are omitted then the colon can also be omitted. If + the double slash and host name are omitted then + "localhost" is assumed. The file name is the only + mandatory part, and if the leading slash is missing then + it is relative to the application's current working + directory (the use of relative names is not + recommended).</P> + </td> + </tr> + + <tr> + <td>File Offset(<size> bytes)</td> + <td> + <P>This is the byte offset to the start of the data in the + specified file. For files that contain data for a single + dataset this will usually be zero.</P> + </td> + </tr> + + <tr> + <td>Size</td> + <td> + <P>This is the total number of bytes reserved in the + specified file for raw data storage. For a file that + contains exactly one complete dataset which is not + extendable, the size will usually be the exact size of the + dataset. However, by making the size larger one allows + HDF5 to extend the dataset. The size can be set to a value + larger than the entire file since HDF5 will read zeros + past the end of the file without failing.</P> + </td> + </tr> + </table> + </div> + + + <hr> + <h4><a name="LayoutMessage">Name: Data Storage - Layout</a></h4> + + <P class=item><B>Header Message Type:</B> 0x0008</P> + <P class=item><B>Length:</B> varies</P> + <P class=item><B>Status:</B> Required for datasets, may not be repeated.</P> + + <P class=item><B>Purpose and Description:</B> Data layout describes how the + elements of a multi-dimensional array are arranged in the linear + address space of the file. Three types of data layout are + supported: + + <ol> + <li>Contiguous: The array can be stored in one contiguous area of the file. + The layout requires that the size of the array be constant and + does not permit chunking, compression, checksums, encryption, + etc. The message stores the total size of the array and the + offset of an element from the beginning of the storage area is + computed as in C. + + <li>Chunked: The array domain can be regularly decomposed into chunks and + each chunk is allocated separately. This layout supports + arbitrary element traversals, compression, encryption, and + checksums, and the chunks can be distributed across external + raw data files (these features are described in other + messages). The message stores the size of a chunk instead of + the size of the entire array; the size of the entire array can + be calculated by traversing the B-tree that stores the chunk + addresses. + + <li>Compact: The array can be stored in one contiguous block, as part of + this object header message (this is called "compact" storage below). + </ol> + + <P class=item><B>Format of Data:</B> + <br> + <div align=center> + <table class=format> + <caption> + Data Layout Message (Versions 1 and 2) + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td>Version</td> + <td>Dimensionality</td> + <td>Layout Class</td> + <td>Reserved</td> + </tr> + + <tr> + <td colspan=4>Reserved</td> + </tr> + + <tr> + <td colspan=4><br>Address<br><br></td> + </tr> + + <tr> + <td colspan=4>Dimension 0 (4-bytes)</td> + </tr> + + <tr> + <td colspan=4>Dimension 1 (4-bytes)</td> + </tr> + + <tr> + <td colspan=4>...</td> + </tr> + + <tr> + <td colspan=4>Dataset Element Size <em>(optional)</em></td> + </tr> + + <tr> + <td colspan=4>Compact Data Size (4-bytes)</td> + </tr> + + <tr> + <td colspan=4><br>Compact Data...<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Version</td> + <td> + <P>The version number information is used for changes in the format of the data + layout message and is described here:</P> + <table class=list> + <tr> + <th width="30%">Version</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>0</code></td> + <td>Never used.</td> + </tr> + + <tr> + <td align=center><code>1</code></td> + <td>Used by version 1.4 and before of the library to encode layout information. + Data space is always allocated when the data set is created.</td> + </tr> + + <tr> + <td align=center><code>2</code></td> + <td>Used by version 1.6.x of the library to encode layout information. + Data space is allocated only when it is necessary.</td> + </tr> + </table> + </td> + </tr> + + <tr> + <td>Dimensionality</td> + <td><P>An array has a fixed dimensionality. This field + specifies the number of dimension size fields later in the + message.</P></td> + </tr> + + <tr> + <td>Layout Class</td> + <td><P>The layout class specifies how the other fields of the + layout message are to be interpreted. A value of one + indicates contiguous storage, a value of two indicates chunked storage, + while a value of zero indicates compact storage. Other values will be defined + in the future.</P></td> + </tr> + + <tr> + <td>Address</td> + <td><P>For contiguous storage, this is the address of the first + byte of storage. For chunked storage this is the address + of the B-tree that is used to look up the addresses of the + chunks. This field is not present for compact storage. + If the version for this message is set to 2, the address + may have the "undefined address" value, to indicate that + storage has not yet been allocated for this array.</P></td> + </tr> + + <tr> + <td>Dimensions</td> + <td><P>For contiguous and compact storage the dimensions define + the entire size of the array while for chunked storage they define + the size of a single chunk. In all cases, they are in units of + array elements (not bytes). The first dimension stored in the list + of dimensions is the slowest changing dimension and the last + dimension stored is the fastest changing dimension. + </P> + </td> + </tr> + + <tr> + <td>Dataset Element Size</td> + <td><P>The size of a dataset element, in bytes. This field is only + present for chunked storage. + </P> + </td> + </tr> + + <tr> + <td>Compact Data Size</td> + <td><P>This field is only present for compact data storage. + It contains the size of the raw data for the dataset array.</P></td> + + <tr> + <td>Compact Data</td> + <td><P>This field is only present for compact data storage. + It contains the raw data for the dataset array.</P></td> + </tr> + </table> + </div> + + <br> + <P>Version 3 of this message re-structured the format into specific + properties that are required for each layout class. + + <br> + <div align=center> + <table class=format> + <caption> + <B>Data Layout Message (Version 3)</B> + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td>Version</td> + <td>Layout Class</td> + <td colspan=2 bgcolor=#DDDDDD> </td> + </tr> + + <tr> + <td colspan=4><br>Properties<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Version</td> + <td> + <P>The version number information is used for changes in the format of layout message + and is described here:</P> + <table class=list> + <tr> + <th width="30%">Version</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>3</code></td> + <td>Used by the version 1.6.3 and later of the library to store properties + for each layout class.</td> + </tr> + </table> + </td> + </tr> + + <tr> + <td>Layout Class</td> + <td><P>The layout class specifies how the other fields of the layout message are to be + interpreted. A value of one indicates contiguous storage, a value of two + indicates chunked storage, while a value of zero indicates compact storage.</P></td> + </tr> + + <tr> + <td>Properties</td> + <td><P>This variable-sized field encodes information specific to each + layout class and is described below. If there is no property + information specified for a layout class, the size of this field + is zero bytes.</P></td> + </tr> + </table> + </div> + + <br> + <P>Class-specific information for compact layout (Class 0): (Note: The dimensionality information + is in the Dataspace message) + + <br> + <div align=center> + <table class=format> + <caption> + Property Descriptions + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td colspan=2>Size</td> + <td colspan=2 bgcolor=#DDDDDD> </td> + </tr> + + <tr> + <td colspan=4><br>Raw Data...<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Size</td> + <td><P>This field contains the size of the raw data for the dataset array.</P></td> + </tr> + + <tr> + <td>Raw Data</td> + <td><P>This field contains the raw data for the dataset array.</P></td> + </tr> + </table> + </div> + + <br> + <P>Class-specific information for contiguous layout (Class 1): (Note: The dimensionality information + is in the Dataspace message) + + <br> + <div align=center> + <table class=format> + <caption> + Property Descriptions + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td colspan=4><br>Address<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Size<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Address</td> + <td><P>This is the address of the first byte of raw data storage. + The address may have the "undefined address" value, to indicate + that storage has not yet been allocated for this array.</P></td> + </tr> + + <tr> + <td>Size</td> + <td><P>This field contains the size allocated to store the raw data.</P></td> + </tr> + </table> + </div> + + <br> + <P>Class-specific information for chunked layout (Class 2): + + <br> + <div align=center> + <table class=format> + <caption> + Property Descriptions + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td>Dimensionality</td> + <td colspan=3 bgcolor=#DDDDDD> </td> + </tr> + + <tr> + <td colspan=4><br>Address<br><br></td> + </tr> + + <tr> + <td colspan=4>Dimension 0 (4-bytes)</td> + </tr> + + <tr> + <td colspan=4>Dimension 1 (4-bytes)</td> + </tr> + + <tr> + <td colspan=4>...</td> + </tr> + + <tr> + <td colspan=4>Dataset Element Size</td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Dimensionality</td> + <td><P>A chunk has a fixed dimensionality. This field specifies + the number of dimension size fields later in the message.</P></td> + </tr> + + <tr> + <td>Address</td> + <td><P>This is the address of the B-tree that is used to look up the addresses of the + chunks. The address may have the "undefined address" value, to indicate that + storage has not yet been allocated for this array.</P></td> + </tr> + + <tr> + <td>Dimensions</td> + <td><P>These values define the dimension size of a single chunk, in + units of array elements (not bytes). The first dimension stored in + the list of dimensions is the slowest changing dimension and the + last dimension stored is the fastest changing dimension. + </P> + </td> + </tr> + + <tr> + <td>Dataset Element Size</td> + <td><P>The size of a dataset element, in bytes. + </P> + </td> + </tr> + </table> + </div> + + <hr> + <h4><a name="ReservedMessage_0009">Name: Reserved - Not Assigned Yet</a></h4> + <P class=item><B>Header Message Type:</B> 0x0009</P> + <P class=item><B>Length:</B> N/A</P> + <P class=item><B>Status:</B> N/A</P> + <P class=item><B>Format of Data:</B> N/A</P> + + <P class=item><B>Purpose and Description:</B> This message type was skipped during the initial + specification of the file format and may be used in a future expansion to the format. + + <hr> + <h4><a name="ReservedMessage_000A">Name: Reserved - Not Assigned Yet</a></h4> + <P class=item><B>Header Message Type:</B> 0x0009</P> + <P class=item><B>Length:</B> N/A</P> + <P class=item><B>Status:</B> N/A</P> + <P class=item><B>Format of Data:</B> N/A</P> + + <P class=item><B>Purpose and Description:</B> This message type was skipped during the initial + specification of the file format and may be used in a future expansion to the format. + + <hr> + <h4><a name="FilterMessage">Name: Data Storage - Filter Pipeline</a></h4> + <P class=item><B>Header Message Type:</B> 0x000B</P> + <P class=item><B>Length:</B> varies</P> + <P class=item><B>Status:</B> Optional, may not be repeated.</P> + + <P class=item><B>Description:</B> This message describes the + filter pipeline which should be applied to the data stream by + providing filter identification numbers, flags, a name, and + client data.</P> + + <P class=item><B>Format of Data:</B> + <br> + <div align=center> + <table class=format> + <caption> + Filter Pipeline Message + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td>Version</td> + <td>Number of Filters</td> + <td colspan=2>Reserved</td> + </tr> + + <tr> + <td colspan=4>Reserved</td> + </tr> + + <tr> + <td colspan=4><br>Filter List<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Version</td> + <td><P>The version number for this message. This document + describes version 1.</P></td> + </tr> + + <tr> + <td>Number of Filters</td> + <td><P>The total number of filters described by this + message. The maximum possible number of filters in a + message is 32.</P></td> + </tr> + + <tr> + <td>Filter List</td> + <td><P>A description of each filter. A filter description + appears in the next table.</P></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=format> + <caption> + Filter Description + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td colspan=2>Filter Identification</td> + <td colspan=2>Name Length</td> + </tr> + + <tr> + <td colspan=2>Flags</td> + <td colspan=2>Number of Values for Client Data</td> + </tr> + + <tr> + <td colspan=4><br>Name<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Client Data<br><br></td> + </tr> + + <tr> + <td colspan=4>Padding</td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Filter Identification</td> + <td> + <p> + This value, often referred to as a filter identifier, + is designed to be a unique identifier for the filter. + Values from zero through 32,767 are reserved for filters + supported by The HDF Group in the HDF5 library and for + filters requested and supported by third parties. + Filters supported by The HDF Group are documented immediately + below. Information on 3rd-party filters can be found at + <a href="/services/contributions.html#filters"> + <code>https://support.hdfgroup.org/services/contributions.html#filters</code></a>. + <a href="#Footnote1Change"><sup><small>1</small></sup></a> + <p> + To request a filter identifier, please contact + The HDF Group’s Help Desk at + <img src="Graphics/help.png" valign="center" height=14>. + You will be asked to provide the following information: + <ol> + <li>Contact information for the developer requesting the + new identifier + <li>A short description of the new filter + <li>Links to any relevant information, including licensing + information + </ol> + <p> + Values from 32768 to 65535 are reserved for non-distributed uses + (for example, internal company usage) or for application usage + when testing a feature. The HDF Group does not track or document + the use of the filters with identifiers from this range. + + <p> + The filters currently in library version 1.6.5 are + listed below: + <table class=list> + <tr> + <th width="30%">Identification</th> + <th align=left>Name</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>1</code></td> + <td>deflate</td> + <td>GZIP deflate compression</td> + </tr> + + <tr> + <td align=center><code>2</code></td> + <td>shuffle</td> + <td>Data element shuffling</td> + </tr> + + <tr> + <td align=center><code>3</code></td> + <td>fletcher32</td> + <td>Fletcher32 checksum</td> + </tr> + + <tr> + <td align=center><code>4</code></td> + <td>szip</td> + <td>SZIP compression</td> + </tr> + </table> + </P></td> + </tr> + + <tr> + <td>Name Length</td> + <td><P>Each filter has an optional null-terminated ASCII name + and this field holds the length of the name including the + null termination padded with nulls to be a multiple of + eight. If the filter has no name then a value of zero is + stored in this field.</P></td> + </tr> + + <tr> + <td>Flags</td> + <td><P>The flags indicate certain properties for a filter. The + bit values defined so far are:</P> + <table class=list> + <tr> + <th width="30%">Value</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>bit 1</code></td> + <td>If set then the filter is an optional filter. + During output, if an optional filter fails it will be + silently removed from the pipeline.</td> + </tr> + </table> + </td> + </tr> + + <tr> + <td>Client Data Number of Values</td> + <td><P>Each filter can store a few integer values to control + how the filter operates. The number of entries in the + Client Data array is stored in this field.</P></td> + </tr> + + <tr> + <td>Name</td> + <td><P>If the Name Length field is non-zero then it will + contain the size of this field, a multiple of eight. This + field contains a null-terminated, ASCII character + string to serve as a comment/name for the filter.</P></td> + </tr> + + <tr> + <td>Client Data</td> + <td><P>This is an array of four-byte integers which will be + passed to the filter function. The Client Data Number of + Values determines the number of elements in the array.</P></td> + </tr> + + <tr> + <td>Padding</td> + <td><P>Four bytes of zeros are added to the message at this + point if the Client Data Number of Values field contains + an odd number.</P></td> + </tr> + </table> + </div> + <p> + <hr align="left" width="50"> + <a name="Footnote1Change"><sup>1</sup></a>If you are reading + an earlier version of this document, this link may have changed. + If the link does not work, use the latest version of this document + on <a href="https://support.hdfgroup.org">The HDF Group</a>’s website, + <a href="/HDF5/doc/H5.format.html"> + <code>https://support.hdfgroup.org/HDF5/doc/H5.format.html</code></a>; + the link there will always be correct. + <small><a href="#FilterMessage">(Return)</a> + </P> + + <hr> + <h4><a name="AttributeMessage">Name: Attribute</a></h4> + <P class=item><B>Header Message Type:</B> 0x000C + <P class=item><B>Length:</B> varies + <P class=item><B>Status:</B> Optional, may be repeated. + + <P class=item><B>Description:</B> The <em>Attribute</em> + message is used to list objects in the HDF file which are used + as attributes, or "metadata" about the current object. An + attribute is a small dataset; it has a name, a datatype, a data + space, and raw data. Since attributes are stored in the object + header they must be relatively small (<64KB) and can be + associated with any type of object which has an object header + (groups, datasets, named types and spaces, etc.). + + <P class=item2>Note: Attributes on an object must have unique names. (The HDF5 library + currently enforces this by causing the creation of an attribute with + a duplicate name to fail). Attributes on different objects may have the + same name, however. + + <P class=item><B>Format of Data:</B> + <br> + <div align=center> + <table class=format> + <caption> + Attribute Message (Version 1) + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td>Version</td> + <td>Reserved</td> + <td colspan=2>Name Size</td> + </tr> + + <tr> + <td colspan=2>Datatype Size</td> + <td colspan=2>Dataspace Size</td> + </tr> + + <tr> + <td colspan=4><br>Name<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Datatype<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Dataspace<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Data<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Version</td> + <td><P>The version number information is used for changes in the format of the + attribute message and is described here:</P> + <table class=list> + <tr> + <th width="30%">Version</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>0</code></td> + <td>Never used.</td> + </tr> + + <tr> + <td align=center><code>1</code></td> + <td>Used by the library before version 1.6 to encode attribute message. + This version does not support shared data type.</td> + </tr> + </table> + </td> + </tr> + + <tr> + <td>Reserved</td> + <td><P>This field is reserved for later use and is set to + zero.</P></td> + </tr> + + <tr> + <td>Name Size</td> + <td><P>The length of the attribute name in bytes including the + null terminator. Note that the Name field below may + contain additional padding not represented by this + field.</P></td> + </tr> + + <tr> + <td>Datatype Size</td> + <td><P>The length of the datatype description in the Datatype + field below. Note that the Datatype field may contain + additional padding not represented by this field.</P></td> + </tr> + + <tr> + <td>Dataspace Size</td> + <td><P>The length of the dataspace description in the Dataspace + field below. Note that the Dataspace field may contain + additional padding not represented by this field.</P></td> + </tr> + + <tr> + <td>Name</td> + <td><P>The null-terminated attribute name. This field is + padded with additional null characters to make it a + multiple of eight bytes.</P></td> + </tr> + + <tr> + <td>Datatype</td> + <td><P>The datatype description follows the same format as + described for the datatype object header message. This + field is padded with additional zero bytes to make it a + multiple of eight bytes.</P></td> + </tr> + + <tr> + <td>Dataspace</td> + <td><P>The dataspace description follows the same format as + described for the dataspace object header message. This + field is padded with additional zero bytes to make it a + multiple of eight bytes.</P></td> + </tr> + + <tr> + <td>Data</td> + <td><P>The raw data for the attribute. The size is determined + from the datatype and dataspace descriptions. This + field is <em>not</em> padded with additional bytes.</P></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=format> + <caption> + Attribute Message (Version 2) + </caption> + + <tr align=center> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td>Version</td> + <td>Flag</td> + <td colspan=2>Name Size</td> + </tr> + + <tr> + <td colspan=2>Type Size</td> + <td colspan=2>Space Size</td> + </tr> + + <tr> + <td colspan=4><br>Name<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Type<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Space<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Data<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Version</td> + <td><P>The version number information is used for changes in the format of the + attribute message and is described here:</P> + <table class=list width="90%"> + <tr> + <th width="30%">Version</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>2</code></td> + <td>Used by the library of version 1.6.x and after to encode attribute message. + This version supports shared data type. The fields of name, type, and space + are not padded with additional bytes of zero.</td> + </tr> + </table> + </td> + </tr> + + <tr> + <td>Flag</td> + <td><P>This field indicates whether the data type of this attribute is shared:</P> + <table class=list width="90%"> + <tr> + <th width="30%">Value</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>0</code></td> + <td>Datatype is <em>not</em> shared.</td> + </tr> + + <tr> + <td align=center><code>1</code></td> + <td>Datatype is shared.</td> + </tr> + </table> + </td> + </tr> + + <tr> + <td>Name Size</td> + <td><P>The length of the attribute name in bytes including the + null terminator.</P></td> + </tr> + + <tr> + <td>Datatype Size</td> + <td><P>The length of the datatype description in the Datatype + field below.</P></td> + </tr> + + <tr> + <td>Dataspace Size</td> + <td><P>The length of the dataspace description in the Dataspace + field below.</P></td> + </tr> + + <tr> + <td>Name</td> + <td><P>The null-terminated attribute name. This field is <em>not</em> + padded with additional bytes.</P></td> + </tr> + + <tr> + <td>Datatype</td> + <td><P>The datatype description follows the same format as + described for the datatype object header message. This + field is <em>not</em> padded with additional bytes.</P></td> + </tr> + + <tr> + <td>Dataspace</td> + <td><P>The dataspace description follows the same format as + described for the dataspace object header message. This + field is <em>not</em> padded with additional bytes.</P></td> + </tr> + + <tr> + <td>Data</td> + <td><P>The raw data for the attribute. The size is determined + from the datatype and dataspace descriptions. This + field is <em>not</em> padded with additional zero + bytes.</P></td> + </tr> + </table> + </div> + + <hr> + <h4><a name="CommentMessage">Name: Object Comment</a></h4> + + <P class=item><B>Header Message Type:</B> 0x000D</P> + <P class=item><B>Length:</B> varies</P> + <P class=item><B>Status:</B> Optional, may not be repeated.</P> + + <P class=item><B>Description:</B> The object comment is + designed to be a short description of an object. An object comment + is a sequence of non-zero (<code>\0</code>) ASCII characters with no other + formatting included by the library.</P> + + <P class=item><B>Format of Data:</B> + <br> + <div align=center> + <table class=format> + <caption> + Name Message + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td colspan=4><br>Comment<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Name</td> + <td>A null terminated ASCII character string.</td> + </tr> + </table> + </div> + + <hr> + <h4><a name="OldModifiedMessage">Name: Object Modification Date & Time (Old)</a></h4> + + <P class=item><B>Header Message Type:</B> 0x000E</P> + <P class=item><B>Length:</B> fixed</P> + <P class=item><B>Status:</B> Optional, may not be repeated.</P> + + <P class=item><B>Description:</B> The object modification date + and time is a timestamp which indicates (using ISO-8601 date and + time format) the last modification of an object. The time is + updated when any object header message changes according to the + system clock where the change was posted. + + <br><br>This modification time message is deprecated in favor of the "new" + modification time message (Message Type 0x0012) and is no longer written + to the file in versions of the HDF5 library after the 1.6.0 version. + </P> + + <P class=item><B>Format of Data:</B> + <br> + <div align=center> + <table class=format> + <caption> + Modification Time Message + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td colspan=4>Year</td> + </tr> + + <tr> + <td colspan=2>Month</td> + <td colspan=2>Day of Month</td> + </tr> + + <tr> + <td colspan=2>Hour</td> + <td colspan=2>Minute</td> + </tr> + + <tr> + <td colspan=2>Second</td> + <td colspan=2>Reserved</td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Year</td> + <td><P>The four-digit year as an ASCII string. For example, + <code>1998</code>. All fields of this message should be interpreted + as coordinated universal time (UTC)</P></td> + </tr> + + <tr> + <td>Month</td> + <td><P>The month number as a two digit ASCII string where + January is <code>01</code> and December is <code>12</code>.</P></td> + </tr> + + <tr> + <td>Day of Month</td> + <td><P>The day number within the month as a two digit ASCII + string. The first day of the month is <code>01</code>.</P></td> + </tr> + + <tr> + <td>Hour</td> + <td><P>The hour of the day as a two digit ASCII string where + midnight is <code>00</code> and 11:00pm is <code>23</code>.</P></td> + </tr> + + <tr> + <td>Minute</td> + <td><P>The minute of the hour as a two digit ASCII string where + the first minute of the hour is <code>00</code> and + the last is <code>59</code>.</P></td> + </tr> + + <tr> + <td>Second</td> + <td><P>The second of the minute as a two digit ASCII string + where the first second of the minute is <code>00</code> + and the last is <code>59</code>.</P></td> + </tr> + + <tr> + <td>Reserved</td> + <td><P>This field is reserved and should always be zero.</P></td> + </tr> + </table> + </div> + + <hr> + <h4><a name="SharedMessage">Name: Shared Object Message</a></h4> + <P class=item><B>Header Message Type:</B> 0x000F</P> + <P class=item><B>Length:</B> Fixed</P> + <P class=item><B>Status:</B> Optional, may be repeated.</P> + + <P class=item><B>Description:</B> A constant message can be shared among + several object headers. A <em>Shared Object</em> Message contains the address of + the object message to be shared. Care must be exercised to prevent cycles when a + message of one object header points to a message in some other object header. + Starting from Version 2 of the Shared Object Message, the <em>Flags</em> + field becomes unused. + </P> + + <P class=item><B>Format of Data:</B> + <br> + <div align=center> + <table class=format> + <caption> + Shared Object Message (Version 1) + </caption> + + <tr> + <th width="25%">byte</td> + <th width="25%">byte</td> + <th width="25%">byte</td> + <th width="25%">byte</td> + </tr> + + <tr> + <td>Version</td> + <td>Flags</td> + <td colspan=2>Reserved</td> + </tr> + + <tr> + <td colspan=4>Reserved</td> + </tr> + + <tr> + <td colspan=4><br>Pointer<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Version</td> + <td><P>The version number is used when there are changes in the format + of a shared object message and is described here:</P> + <table class=list> + <tr> + <th width="30%">Version</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>0</code></td> + <td>Never used.</td> + </tr> + + <tr> + <td align=center><code>1</code></td> + <td>Used by the library before version 1.6.1. In this version, + the Flags field is used to indicate whether the actual message is + stored in the global heap (never implemented). The Pointer field + either contains the the header message address in the global heap + (never implemented) or the address of the shared object header.</td> + </tr> + </table> + </tr> + + <tr> + <td>Flags</td> + <td><P>The Shared Message message points to a message which is + shared among multiple object headers. The Flags field + describes the type of sharing:</P> + <table class=list> + <tr> + <th width="30%">Bit</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>0</code></td> + <td>If this bit is clear then the actual message is the + first message in some other object header; otherwise + the actual message is stored in the global heap (never + implemented).</td> + </tr> + + <tr> + <td align=center><code>2-7</code></td> + <td>Reserved (always zero)</td> + </tr> + </table> + </td> + </tr> + + <tr> + <td>Pointer</td> + <td><P>The address of the object header + containing the message to be shared.</P></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=format> + <caption> + Shared Object Message (Version 2) + </caption> + + <tr> + <th width="25%">byte</td> + <th width="25%">byte</td> + <th width="25%">byte</td> + <th width="25%">byte</td> + </tr> + + <tr> + <td>Version</td> + <td>Flags</td> + <td colspan=2 bgcolor=#DDDDDD> </td> + </tr> + + <tr> + <td colspan=4><br>Pointer<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Version</td> + <td><P>The version number is used when there are changes in the format + of a shared object message and is described here:</P> + <table class=list> + <tr> + <th width="30%">Version</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>2</code></td> + <td>Used by the library of version 1.6.1 and after. In this version, + The Flags field is not used and the Pointer field contains the address + of the object header containing the message to be shared. </td> + </tr> + </table> + </tr> + + <tr> + <td>Flags</td> + <td><P>Unused.</P></td> + </tr> + + <tr> + <td>Pointer</td> + <td><P>The address of the object header + containing the message to be shared.</P></td> + </tr> + </table> + </div> + + + <hr> + <h4><a name="ContinuationMessage">Name: Object Header Continuation</a></h4> + <P class=item><B>Header Message Type:</B> 0x0010</P> + <P class=item><B>Length:</B> fixed</P> + <P class=item><B>Status:</B> Optional, may be repeated.</P> + <P class=item><B>Description:</B> The object header continuation is the location + in the file of more header messages for the current data object. This can be + used when header blocks become too large or are likely to change over time.</P> + + <P class=item><B>Format of Data:</B> + <br> + <div align=center> + <table class=format> + <caption> + Object Header Continuation Message + </caption> + + <tr> + <th width=25%>byte</th> + <th width=25%>byte</th> + <th width=25%>byte</th> + <th width=25%>byte</th> + </tr> + + <tr> + <td colspan=4><br>Offset<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Length<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width=30%>Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Offset</td> + <td><P>This value is the offset in bytes from the beginning of the file where the + header continuation information is located.</P></td> + </tr> + + <tr> + <td>Length</td> + <td><P>This value is the length in bytes of the header continuation information in + the file.</P></td> + </tr> + </table> + </div> + + <hr> + <h4><a name="SymbolTableMessage">Name: Group Message</a></h4> + <P class=item><B>Header Message Type:</B> 0x0011</P> + <P class=item><B>Length:</B> fixed</P> + <P class=item><B>Status:</B> Required for groups, may not be repeated.</P> + <P class=item><B>Description:</B> Each group has a B-tree and a + name heap which are pointed to by this message.</P> + <P class=item><B>Format of data:</B> + + <br> + <div align=center> + <table class=format> + <caption> + <B>Group Message</B> + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td colspan=4><br>B-tree Address<br><br></td> + </tr> + + <tr> + <td colspan=4><br>Heap Address<br><br></td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width=30%>Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>B-tree Address</td> + <td><P>This value is the offset in bytes from the beginning of the file + where the B-tree is located.</P></td> + </tr> + + <tr> + <td>Heap Address</td> + <td><P>This value is the offset in bytes from the beginning of the file + where the group name heap is located.</P></td> + </tr> + </table> + </div> + + <hr> + <h4><a name="ModifiedMessage">Name: Object Modification Date & Time</a></h4> + + <P class=item><B>Header Message Type:</B> 0x0012 </P> + <P class=item><B>Length:</B> Fixed </P> + <P class=item><B>Status:</B> Optional, may not be repeated. </P> + + <P class=item><B>Description:</B> The object modification date + and time is a timestamp which indicates the last modification of an object. + The time is updated when any object header message changes according to the + system clock where the change was posted. + </P> + + <P class=item><B>Format of Data:</B> + <div align=center> + <table class=format> + <caption> + Modification Time Message + </caption> + + <tr> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + <th width="25%">byte</th> + </tr> + + <tr> + <td>Version</td> + <td colspan=3>Reserved</td> + </tr> + + <tr> + <td colspan=4>Seconds After Epoch</td> + </tr> + </table> + </div> + + <br> + <div align=center> + <table class=desc> + <tr> + <th width="30%">Field Name</th> + <th>Description</th> + </tr> + + <tr> + <td>Version</td> + <td><P>The version number is used for changes in the format of Object Modification Time + and is described here:</P> + <table class=list> + <tr> + <th width="30%">Version</th> + <th align=left>Description</th> + </tr> + + <tr> + <td align=center><code>0</code></td> + <td>Never used.</td> + </tr> + + <tr> + <td align=center><code>1</code></td> + <td>Used by Version 1.6.1 and after of the library to encode time. In + this version, the time is the seconds after Epoch.</td> + </tr> + </table> + </td> + </tr> + + <tr> + <td>Reserved</td> + <td><P>This field is reserved and should always be zero.</P></td> + </tr> + + <tr> + <td>Seconds After Epoch</td> + <td><P>The number of seconds since 0 hours, 0 minutes, 0 seconds, + January 1, 1970, Coordinated Universal Time.</P></td> + </tr> + </table> + </div> + +<hr> +<h3><a name="DataStorage">Disk Format: Level 2b - Data Object Data Storage</a></h3> +<P>The data for an object is stored separately from the header +information in the file and may not actually be located in the HDF5 file +itself if the header indicates that the data is stored externally. The +information for each record in the object is stored according to the +dimensionality of the object (indicated in the dimensionality header message). +Multi-dimensional data is stored in C order [same as current scheme], i.e. the +"last" dimension changes fastest. +<P>Data whose elements are composed of simple number-types are stored in +native-endian IEEE format, unless they are specifically defined as being stored +in a different machine format with the architecture-type information from the +number-type header message. This means that each architecture will need to +[potentially] byte-swap data values into the internal representation for that +particular machine. +<P> Data with a variable-length datatype is stored in the global heap +of the HDF5 file. Global heap identifiers are stored in the +data object storage. +<P>Data whose elements are composed of pointer number-types are stored in several +different ways depending on the particular pointer type involved. Simple +pointers are just stored as the dataset offset of the object being pointed to with the +size of the pointer being the same number of bytes as offsets in the file. +Dataset region references are stored as a heap-ID which points to the following +information within the file-heap: an offset of the object pointed to, number-type +information (same format as header message), dimensionality information (same +format as header message), sub-set start and end information (i.e. a coordinate +location for each), and field start and end names (i.e. a [pointer to the] +string indicating the first field included and a [pointer to the] string name +for the last field). + +<P>Data of a compound datatype is stored as a contiguous stream of the items +in the structure, with each item formatted according to its datatype.</p> + +<hr> +<h3><a name="Appendix">Appendix</a></h3> +<P>Definitions of various terms used in this document. +</P> +<P>The <A name="UndefinedAddress">"undefined address"</A> for a file is a +file address with all bits set, i.e. <code>0xffff...ff</code>. +<P>The <A name="UnlimitedDim">"unlimited size"</A> for a size is a +value with all bits set, i.e. <code>0xffff...ff</code>. + +</body> +</html> |