summaryrefslogtreecommitdiffstats
path: root/doxygen/examples
diff options
context:
space:
mode:
authorGerd Heber <gheber@hdfgroup.org>2021-04-26 19:07:29 (GMT)
committerGitHub <noreply@github.com>2021-04-26 19:07:29 (GMT)
commit1d680fe04c545d601678d876ad22dea7bfc31edd (patch)
treef84a97c149df9ccf4409c5bfea04a7316c019e8a /doxygen/examples
parent12082e728de7481144db7ea3cca6f38acf0a7cac (diff)
downloadhdf5-1d680fe04c545d601678d876ad22dea7bfc31edd.zip
hdf5-1d680fe04c545d601678d876ad22dea7bfc31edd.tar.gz
hdf5-1d680fe04c545d601678d876ad22dea7bfc31edd.tar.bz2
Merge doxygen2 into develop (#553)
* Fixed warnings and started H5Epublic.h. * Include H5FD* headers to correctly resolve references. * Doxygen2 (#330) * H5Eauto_is_v2. * Added a few more calls. * Added a few more H5E calls. * First cut of H5E v2. * Added the deprecated v1 calls. * Updated spacing. * Once more. * Taking some inspiration from Eigen3. * Add doxygen for the assigned functions: H5Pregister1,H5Pinsert1,H5Pen… (#352) * Add doxygen for the assigned functions: H5Pregister1,H5Pinsert1,H5Pencode1, H5Pget_filter_by_id1,H5Pget_version, H5Pset_file_space,H5Pget_file_space. Someone already adds H5Pget_filter1. Also fixs an extra parameter 'close' call back function for HPregister2. * doxygen work. fixs format by using clang-format. * doxgen work for H5Pregister1 etc. Addressed Barbara and Gerd's comments. For Quincey's comments, since we are not supposed to change the source code. I leave this to future improvements. * added documentation for H5P APIs (#350) * add documenation for H5Pget_buffer,H5Pget_data_transform,H5Pget_edc_check,H5Pget_hyper_vector_size,H5Pget_preserve,H5Pget_type_conv_cb,H5Pget_vlen_mem_manager,H5Pset_btree_ratios * format corrections * fixed grammer * fixed herr_t * Better name. * A fresh look. * add doxygen to H5Ppublic.h * use attention instead of warning * Add doxygen comments in H5Ppublic.h (#375) * Add doxygen comments in H5Ppublic.h * H5Pset_meta_block_size * H5Pset_metadata_read_attempts * H5Pset_multi_type * H5Pset_object_flush_cb * H5Pset_sieve_buf_size * H5Pset_small_data_block_size * H5Pset_all_coll_metadata_ops * H5Pget_all_coll_metadata_ops * Add DOXYGEN_EXAMPLES_DIR to src/CMakeLists.txt * Fix clang-format errors * Fix filenames in doxygen/examples * add doxygen to H5Ppublic.h (#378) * add doxygen to H5Ppublic.h * use attention instead of warning Co-authored-by: Kimmy Mu <kmu@hdfgroup.org> * Revert "add doxygen to H5Ppublic.h (#378)" This reverts commit 2ee1821b138a5c00b15ea57ce9e950367480f5f2. * Updated Doxygen variables. * I forgot to copy two images. * Enable desktop search by default. * Add my assigned Doxygen documentation. * Remove whitespace at EOL. Appease clang-format. * Addressed Chris' comments. * Added an alias for asynchronous functions. * One space is enough for all of us. * Slightly restructured RM page. * address some issues * reformatting * Style external links. * reformatting * reformatting * Added "Metadata Caching in HDF5" as a technical note example. * Revise this soon! * Added specification examples. * Fixed references. * Added H5AC cache image stuff and file format study. * Added older FMT versions. Where did 1.0 go? * Updated C/C++ note and replaced ambiguous labels. * Reformat source with clang v10.0.1. * Added the VFL technical note. * Added what I believe might be called version 1.0 of the format. * Added the remaining specs. * Added H5Z callback documentation and fixed a few mistakes. * Added dox for deprecated H5G calls and fixed a few snippet blockIDs. * clang-format happy? * Ok? * Bonus track: Deprecated H5D functions. * Carry over the more detailed group description. * Added documentation for the missing and deprecated H5R calls. * Life is easier and less repetitive w/ snippets. Use them! * Eliminate the snippet block ID artifacts in the HTML rendering. * Fixed snippet HTML artifacts and added a few missing calls. * Under 20 H5Ps to go! * Almost complete! * "This is a form of pedantry up with which I will not put." (Churchill) * Let's not waste as much space on bulleted lists! * First complete (?) draft of the Doxygen-based RM. * Completeness check and minor fixes along the way. * Pedantry. * Adding missing H5FD calls checkpoint. * Pedantry. * More pedantry. * Added H5Pset_fapl_log. * First draft of H5ES. * Fixed warnings. * Prep. for map module. * First cut of the map module. * Pedantry. * Possible H5F introduction. * Fix the indentation. * Pedantry. * Ditto. * Thanks to the reviewers for their comments. * Added missing images. * Line numbers are a distraction here. * More examples, references, and clean-up. Don't repeat yourself! * Clang pedantry. * Ditto. * More reviewer comments... * Templatized references and cleaned up \todos. * Committing clang-format changes * Fixed MANIFEST. * Addressed Quincey's comments. (OCPLs) * Fixed a few more \todo items. * Fixed more \todo items. * Added attribute life cycle. * Forgot the examples file. * Committing clang-format changes * Pedantry. * Live and learn! * Added a sample H5D life cycle. * Committing clang-format changes * Pedantry. Co-authored-by: kyang2014 <kyang2014@users.noreply.github.com> Co-authored-by: Scot Breitenfeld <brtnfld@hdfgroup.org> Co-authored-by: Kimmy Mu <kmu@hdfgroup.org> Co-authored-by: Christopher Hogan <ChristopherHogan@users.noreply.github.com> Co-authored-by: jya-kmu <53388330+jya-kmu@users.noreply.github.com> Co-authored-by: David Young <dyoung@hdfgroup.org> Co-authored-by: Larry Knox <lrknox@hdfgroup.org> Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Diffstat (limited to 'doxygen/examples')
-rw-r--r--doxygen/examples/FF-IH_FileGroup.gifbin0 -> 3407 bytes
-rw-r--r--doxygen/examples/FF-IH_FileObject.gifbin0 -> 2136 bytes
-rw-r--r--doxygen/examples/FileFormatSpecChunkDiagram.jpgbin0 -> 29237 bytes
-rw-r--r--doxygen/examples/H5.format.1.0.html4050
-rw-r--r--doxygen/examples/H5.format.1.1.html6439
-rw-r--r--doxygen/examples/H5.format.2.0.html14902
-rw-r--r--doxygen/examples/H5.format.html20400
-rw-r--r--doxygen/examples/H5A_examples.c145
-rw-r--r--doxygen/examples/H5D_examples.c173
-rw-r--r--doxygen/examples/H5F_examples.c187
-rw-r--r--doxygen/examples/H5Pget_metadata_read_attempts.1.c22
-rw-r--r--doxygen/examples/H5Pget_metadata_read_attempts.2.c44
-rw-r--r--doxygen/examples/H5Pget_metadata_read_attempts.3.c44
-rw-r--r--doxygen/examples/H5Pget_object_flush_cb.c41
-rw-r--r--doxygen/examples/H5Pset_metadata_read_attempts.c59
-rw-r--r--doxygen/examples/H5Pset_object_flush_cb.c41
-rw-r--r--doxygen/examples/ImageSpec.html1203
-rw-r--r--doxygen/examples/PaletteExample1.gifbin0 -> 2731 bytes
-rw-r--r--doxygen/examples/Palettes.fm.anc.gifbin0 -> 4748 bytes
-rw-r--r--doxygen/examples/TableSpec.html193
-rw-r--r--doxygen/examples/ThreadSafeLibrary.html787
-rw-r--r--doxygen/examples/VFL.html1601
22 files changed, 50331 insertions, 0 deletions
diff --git a/doxygen/examples/FF-IH_FileGroup.gif b/doxygen/examples/FF-IH_FileGroup.gif
new file mode 100644
index 0000000..b0d76f5
--- /dev/null
+++ b/doxygen/examples/FF-IH_FileGroup.gif
Binary files differ
diff --git a/doxygen/examples/FF-IH_FileObject.gif b/doxygen/examples/FF-IH_FileObject.gif
new file mode 100644
index 0000000..8eba623
--- /dev/null
+++ b/doxygen/examples/FF-IH_FileObject.gif
Binary files differ
diff --git a/doxygen/examples/FileFormatSpecChunkDiagram.jpg b/doxygen/examples/FileFormatSpecChunkDiagram.jpg
new file mode 100644
index 0000000..03fd90a
--- /dev/null
+++ b/doxygen/examples/FileFormatSpecChunkDiagram.jpg
Binary files differ
diff --git a/doxygen/examples/H5.format.1.0.html b/doxygen/examples/H5.format.1.0.html
new file mode 100644
index 0000000..2d3ffbe
--- /dev/null
+++ b/doxygen/examples/H5.format.1.0.html
@@ -0,0 +1,4050 @@
+<html>
+ <head>
+ <title>
+ HDF5 File Format Specification
+ </title>
+ </head>
+ <body bgcolor="#FFFFFF">
+
+ <center>
+ <table border=0 width=90%>
+ <tr>
+ <td valign=top>
+ <ol type=I>
+ <li><a href="#Intro">Introduction</a>
+ <li><a href="#BootBlock">Disk Format Level 0 - File Signature and Super Block</a>
+ <li><a href="#Group">Disk Format Level 1 - File Infrastructure</a>
+ <font size=-2>
+ <ol type=A>
+ <li><a href="#Btrees">Disk Format Level 1A - B-link Trees and B-tree Nodes</a>
+ <li><a href="#SymbolTable">Disk Format Level 1B - Group</a>
+ <li><a href="#SymbolTableEntry">Disk Format Level 1C - Group Entry</a>
+ <li><a href="#LocalHeap">Disk Format Level 1D - Local Heaps</a>
+ <li><a href="#GlobalHeap">Disk Format Level 1E - Global Heap</a>
+ <li><a href="#FreeSpaceIndex">Disk Format Level 1F - Free-space Index</a>
+ </ol>
+ </font>
+ <li><a href="#DataObject">Disk Format Level 2 - Data Objects</a>
+ <font size=-2>
+ <ol type=A>
+ <li><a href="#ObjectHeader">Disk Format Level 2a - Data Object Headers</a>
+ <ol type=1>
+ <li><a href="#NILMessage">Name: NIL</a> <!-- 0x0000 -->
+ <li><a href="#SimpleDataSpace">Name: Simple Dataspace</a> <!-- 0x0001 -->
+<!--
+ <li><a href="#DataSpaceMessage">Name: Complex Dataspace</a> --> <!-- 0x0002 -->
+ <li><a href="#DataTypeMessage">Name: Datatype</a> <!-- 0x0003 -->
+ <li><a href="#FillValueMessage">Name: Data Storage - Fill Value</a> <!-- 0x0004 -->
+ <li><a href="#ReservedMessage_0005">Name: Reserved - not assigned yet</a> <!-- 0x0005 -->
+ </ol>
+ </ol>
+ </font>
+ </ol>
+ </td><td>&nbsp;&nbsp;</td><td valign=top>
+ <ol type=I>
+
+ <li><a href="#DataObject">Disk Format Level 2 - Data Objects</a>
+ <font size=-2><i>(Continued)</i>
+ <ol type=A>
+ <li><a href="#ObjectHeader">Disk Format Level 2a - Data Object Headers</a><i>(Continued)</i>
+ <ol type=1>
+ <li><a href="#CompactDataStorageMessage">Name: Data Storage - Compact</a> <!-- 0x0006 -->
+ <li><a href="#ExternalFileListMessage">Name: Data Storage - External Data Files</a> <!-- 0x0007 -->
+ <li><a href="#LayoutMessage">Name: Data Storage - Layout</a> <!-- 0x0008 -->
+ <li><a href="#ReservedMessage_0009">Name: Reserved - not assigned yet</a> <!-- 0x0009 -->
+ <li><a href="#ReservedMessage_000A">Name: Reserved - not assigned yet</a> <!-- 0x000a -->
+ <li><a href="#FilterMessage">Name: Data Storage - Filter Pipeline</a> <!-- 0x000b -->
+ <li><a href="#AttributeMessage">Name: Attribute</a> <!-- 0x000c -->
+ <li><a href="#NameMessage">Name: Object Name</a> <!-- 0x000d -->
+ <li><a href="#ModifiedMessage">Name: Object Modification Date and Time</a> <!-- 0x000e -->
+ <li><a href="#SharedMessage">Name: Shared Object Message</a> <!-- 0x000f -->
+ <li><a href="#ContinuationMessage">Name: Object Header Continuation</a> <!-- 0x0010 -->
+ <li><a href="#SymbolTableMessage">Name: Group Message</a> <!-- 0x0011 -->
+ </ol>
+ <li><a href="#SharedObjectHeader">Disk Format: Level 2b - Shared Data Object Headers</a>
+ <li><a href="#DataStorage">Disk Format: Level 2c - Data Object Data Storage</a>
+ </ol>
+ </font>
+ </ol>
+</td></tr>
+</table>
+</center>
+
+<br><br>
+
+
+ <h2>Introduction</h2>
+
+ <table align=right width=100>
+ <tr><td>&nbsp;</td><td align=center>
+ <hr>
+ <img src="FF-IH_FileGroup.gif" alt="HDF5 Groups" hspace=15 vspace=15>
+ </td><td>&nbsp;</td></tr>
+ <tr><td>&nbsp;</td><td align=center>
+ <strong>Figure 1:</strong> Relationships among the HDF5 root group, other groups, and objects
+ <hr>
+ </td><td>&nbsp;</td></tr>
+
+ <tr><td>&nbsp;</td><td align=center>
+ <img src="FF-IH_FileObject.gif" alt="HDF5 Objects" hspace=15 vspace=15>
+ </td><td>&nbsp;</td></tr>
+ <tr><td>&nbsp;</td><td align=center>
+ <strong>Figure 2:</strong> HDF5 objects -- datasets, datatypes, or dataspaces
+ <hr>
+ </td><td>&nbsp;</td></tr>
+ </table>
+
+
+ <P>The format of an HDF5 file on disk encompasses several
+ key ideas of the HDF4 and AIO file formats as well as
+ addressing some shortcomings therein. The new format is
+ more self-describing than the HDF4 format and is more
+ uniformly applied to data objects in the file.
+
+ <P>An HDF5 file appears to the user as a directed graph.
+ The nodes of this graph are the higher-level HDF5 objects
+ that are exposed by the HDF5 APIs:
+
+ <ul>
+ <li>Groups
+ <li>Datasets
+ <li>Datatypes
+ <li>Dataspaces
+ </ul>
+
+ <P>At the lowest level, as information is actually written to the disk,
+ an HDF5 file is made up of the following objects:
+ <ul>
+ <li>A super block
+ <li>B-tree nodes (containing either symbol nodes or raw data chunks)
+ <li>Object headers
+
+ <li>Collections
+ <li>Local heaps
+ <li>Free space
+ </ul>
+
+ The HDF5 library uses these lower-level objects to represent the
+ higher-level objects that are then presented to the user or
+ to applications through the APIs.
+ For instance, a group is an object header that contains a message that
+ points to a local heap and to a B-tree which points to symbol nodes.
+ A dataset is an object header that contains messages that describe
+ datatype, space, layout, filters, external files, fill value, etc
+ with the layout message pointing to either a raw data chunk or to a
+ B-tree that points to raw data chunks.
+
+
+ <h3>This Document</h3>
+
+ <p>This document describes the lower-level data objects;
+ the higher-level objects and their properties are described
+ in the <a href="H5.user.html"><cite>HDF5 User's Guide</cite></a>.
+
+
+<!--
+<blockquote>
+<pre>
+
+Elena> NOTE: give reference to the detailed discussion of the B-trees
+Elena> when needed. Right now we do not have specification (only general one)
+Elena> for the Symbol Table B-trees and B-trees used to manage chunked datasets.
+Elena> B-trees
+Elena> General Discussion
+Elena> Object related discussions
+Elena> Symbol Tables
+Elena> Global heap
+Elena> "Free-space object"
+
+
+</pre>
+</blockquote>
+-->
+
+
+
+ <P>Three levels of information comprise the file format.
+ Level 0 contains basic information for identifying and
+ defining information about the file. Level 1 information contains
+ the group information (stored as a B-tree) and is used as the
+ index for all the objects in the file. Level 2 is the rest
+ of the file and contains all of the data objects, with each object
+ partitioned into header information, also known as
+ <em>meta information</em>, and data.
+
+ <p>The sizes of various fields in the following layout tables are
+ determined by looking at the number of columns the field spans
+ in the table. There are three exceptions: (1) The size may be
+ overridden by specifying a size in parentheses, (2) the size of
+ addresses is determined by the <em>Size of Offsets</em> field
+ in the super block, and (3) the size of size fields is determined
+ by the <em>Size of Lengths</em> field in the super block.
+
+
+
+<br><br>
+<br><br>
+
+
+ <h2><a name="BootBlock">
+ Disk Format: Level 0 - File Signature and Super Block</a></h2>
+
+ <P>The super block may begin at certain predefined offsets within
+ the HDF5 file, allowing a block of unspecified content for
+ users to place additional information at the beginning (and
+ end) of the HDF5 file without limiting the HDF5 library's
+ ability to manage the objects within the file itself. This
+ feature was designed to accommodate wrapping an HDF5 file in
+ another file format or adding descriptive information to the
+ file without requiring the modification of the actual file's
+ information. The super block is located by searching for the
+ HDF5 file signature at byte offset 0, byte offset 512 and at
+ successive locations in the file, each a multiple of two of
+ the previous location, i.e. 0, 512, 1024, 2048, etc.
+
+ <P>The super block is composed of a file signature, followed by
+ super block and group version numbers, information
+ about the sizes of offset and length values used to describe
+ items within the file, the size of each group page,
+ and a group entry for the root object in the file.
+
+ <p>
+ <center>
+ <table border align=center cellpadding=4 width="80%">
+ <caption align=top>
+ <B>HDF5 Super Block Layout</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>HDF5 File Signature (8 bytes)<br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td>Version # of Super Block</td>
+ <td>Version # of Global Free-space Storage</td>
+ <td>Version # of Group</td>
+ <td>Reserved</td>
+ </tr>
+
+ <tr align=center>
+ <td>Version # of Shared Header Message Format</td>
+ <td>Size of Offsets</td>
+ <td>Size of Lengths</td>
+ <td>Reserved (zero)</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=2>Group Leaf Node K</td>
+ <td colspan=2>Group Internal Node K</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>File Consistency Flags</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Base Address*</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Address of Global Free-space Heap*</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>End of File Address*</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Driver Information Block Address*</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Root Group Address*</td>
+ </tr>
+ </table>
+
+ <table width="80%" border=0>
+ <tr><td>
+ <div align=right>
+ (Items marked with an asterisk (*) in the above table
+ <br>
+ are of the size specified in "Size of Offsets.")
+ </div>
+ </td></tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>File Signature</td>
+ <td>This field contains a constant value and can be used to
+ quickly identify a file as being an HDF5 file. The
+ constant value is designed to allow easy identification of
+ an HDF5 file and to allow certain types of data corruption
+ to be detected. The file signature of an HDF5 file always
+ contains the following values:
+
+ <br><br><center>
+ <table border align=center cellpadding=4 width="100%">
+ <tr align=center>
+ <td>decimal</td>
+ <td width="8%">137</td>
+ <td width="8%">72</td>
+ <td width="8%">68</td>
+ <td width="8%">70</td>
+ <td width="8%">13</td>
+ <td width="8%">10</td>
+ <td width="8%">26</td>
+ <td width="8%">10</td>
+ </tr>
+
+ <tr align=center>
+ <td>hexadecimal</td>
+ <td width="8%">89</td>
+ <td width="8%">48</td>
+ <td width="8%">44</td>
+ <td width="8%">46</td>
+ <td width="8%">0d</td>
+ <td width="8%">0a</td>
+ <td width="8%">1a</td>
+ <td width="8%">0a</td>
+ </tr>
+
+ <tr align=center>
+ <td>ASCII C Notation</td>
+ <td width="8%">\211</td>
+ <td width="8%">H</td>
+ <td width="8%">D</td>
+ <td width="8%">F</td>
+ <td width="8%">\r</td>
+ <td width="8%">\n</td>
+ <td width="8%">\032</td>
+ <td width="8%">\n</td>
+ </tr>
+ </table>
+ </center>
+ <br>
+
+ This signature both identifies the file as an HDF5 file
+ and provides for immediate detection of common
+ file-transfer problems. The first two bytes distinguish
+ HDF5 files on systems that expect the first two bytes to
+ identify the file type uniquely. The first byte is
+ chosen as a non-ASCII value to reduce the probability
+ that a text file may be misrecognized as an HDF5 file;
+ also, it catches bad file transfers that clear bit
+ 7. Bytes two through four name the format. The CR-LF
+ sequence catches bad file transfers that alter newline
+ sequences. The control-Z character stops file display
+ under MS-DOS. The final line feed checks for the inverse
+ of the CR-LF translation problem. (This is a direct
+ descendent of the PNG file signature.)</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Version Number of the Super Block</td>
+ <td>This value is used to determine the format of the
+ information in the super block. When the format of the
+ information in the super block is changed, the version number
+ is incremented to the next integer and can be used to
+ determine how the information in the super block is
+ formatted.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Version Number of the Global Free-space Heap</td>
+ <td>This value is used to determine the format of the
+ information in the Global Free-space Heap.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Version Number of the Group</td>
+ <td>This value is used to determine the format of the
+ information in the Group. When the format of
+ the information in the Group is changed, the
+ version number is incremented to the next integer and can be
+ used to determine how the information in the Group
+ is formatted.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Version Number of the Shared Header Message Format</td>
+ <td>This value is used to determine the format of the
+ information in a shared object header message, which is
+ stored in the global small-data heap. Since the format
+ of the shared header messages differs from the private
+ header messages, a version number is used to identify changes
+ in the format.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Size of Offsets</td>
+ <td>This value contains the number of bytes used to store
+ addresses in the file. The values for the addresses of
+ objects in the file are offsets relative to a base address,
+ usually the address of the super block signature. This
+ allows a wrapper to be added after the file is created
+ without invalidating the internal offset locations.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Size of Lengths</td>
+ <td>This value contains the number of bytes used to store
+ the size of an object.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Group Leaf Node K</td>
+ <td>Each leaf node of a group B-tree will have at
+ least this many entries but not more than twice this
+ many. If a group has a single leaf node then it
+ may have fewer entries.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Group Internal Node K</td>
+ <td>Each internal node of a group B-tree will have
+ at least K pointers to other nodes but not more than 2K
+ pointers. If the group has only one internal
+ node then it might have fewer than K pointers.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Bytes per B-tree Page</td>
+ <td>This value contains the number of bytes used for symbol
+ pairs per page of the B-trees used in the file. All
+ B-tree pages will have the same size per page.
+ <br>
+ For 32-bit file offsets, 340 objects is the maximum
+ per 4KB page; for 64-bit file offset, 254 objects will fit
+ per 4KB page. In general, the equation is:
+ <br>
+ <code>&nbsp;&nbsp;&nbsp;&lt;<i>number of objects</i>&gt; =
+ <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
+ FLOOR((&lt;<i>page size</i>&gt; - &lt;<i>offset size</i>&gt;) /
+ <br>&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;
+ (&lt;<i>Symbol size</i>&gt; + &lt;<i>offset size</i>&gt;))
+ - 1 </code></td>
+ </tr>
+
+ <tr valign=top>
+ <td>File Consistency Flags</td>
+ <td>This value contains flags to indicate information
+ about the consistency of the information contained
+ within the file. Currently, the following bit flags are
+ defined:
+ <ul>
+ <li>Bit 0 set indicates that the file is opened for
+ write-access.
+ <li>Bit 1 set indicates that the file has
+ been verified for consistency and is guaranteed to be
+ consistent with the format defined in this document.
+ <li>Bits 2-31 are reserved for future use.
+ </ul>
+ Bit 0 should be
+ set as the first action when a file is opened for write
+ access and should be cleared only as the final action
+ when closing a file. Bit 1 should be cleared during
+ normal access to a file and only set after the file's
+ consistency is guaranteed by the library or a
+ consistency utility.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Base Address</td>
+ <td>This is the absolute file address of the first byte of
+ the HDF5 data within the file. The library currently
+ constrains this value to be the absolute file address
+ of the super block itself when creating new files;
+ future versions of the library may provide greater
+ flexibility. Unless otherwise noted,
+ all other file addresses are relative to this base
+ address.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Address of Global Free-space Heap</td>
+ <td>Free-space management is not yet defined in the HDF5
+ file format and is not handled by the library.
+ Currently this field always contains the
+ undefined address <code>0xfff...ff</code>.
+<!--
+ <td>This value contains the relative address of the B-tree
+ used to manage the blocks of data which are unused in the
+ file currently. The free-space heap is used to manage the
+ blocks of bytes at the file-level which become unused when
+ objects are moved within the file.</td>
+-->
+ </tr>
+
+ <tr valign=top>
+ <td>End of File Address</td>
+ <td>This is the relative file address of the first byte past
+ the end of all HDF5 data. It is used to determine whether a
+ file has been accidently truncated and as an address where
+ file data allocation can occur if the free list is not
+ used.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Driver Information Block Address</td>
+ <td>This is the relative file address of the file driver
+ information block which contains driver-specific
+ information needed to reopen the file. If there is no
+ driver information block then this entry should be the
+ undefined address (all bits set).</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Root Group Address</td>
+ <td>This is the address of the root group (described later
+ in this document), which serves as the entry point into
+ the group graph.</td>
+ </tr>
+ </table>
+ </center>
+
+
+ <p>The <em>file driver information block</em> is an optional region of the
+ file which contains information needed by the file driver in
+ order to reopen a file. The format of the file driver information
+ block is:
+
+ <p>
+ <center>
+ <table border align=center cellpadding=4 width="80%">
+ <caption align=top>
+ <B>Driver Information Block</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td>Version</td>
+ <td colspan=3>Reserved (zero)</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Driver Information Size (4 bytes)</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Driver Identification (8 bytes)<br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br><br>Driver Information<br><br><br></td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Version</td>
+ <td>The version number of the driver information block. The
+ file format documented here is version zero.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Driver Information Size</td>
+ <td>The size in bytes of the Driver Information part of this
+ structure.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Driver Identification</td>
+ <td>This is an eight-byte ASCII string without null
+ termination which identifies the driver and version number
+ of the Driver Information block. The predefined drivers
+ supplied with the HDF5 library are identified by the
+ letters <code>NCSA</code> followed by the first four characters of
+ the driver name. If the Driver Information block is not
+ the original version then the last letter(s) of the
+ identification will be replaced by a version number in
+ ASCII.
+ For example, the various versions of the <em>family driver</em>
+ will be identified by <code>NCSAfami</code>, <code>NCSAfam0</code>,
+ <code>NCSAfam1</code>, etc.
+ (<code>NCSAfami</code> is simply <code>NCSAfamily</code> truncated
+ to eight characters. Subsequent identifiers will be created by
+ substituting sequential numerical values for the final character,
+ starting with zero.)
+ <p>
+ Identification for user-defined drivers
+ is arbitrary but should be unique.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Driver Information</td>
+ <td>Driver information is stored in a format defined by the
+ file driver and encoded/decoded by the driver callbacks
+ invoked from the <code>H5FD_sb_encode</code> and
+ <code>H5FD_sb_decode</code> functions.</td>
+ </tr>
+ </table>
+ </center>
+
+
+ <br><br>
+ <br><br>
+
+
+ <h2><a name="Group">
+ Disk Format: Level 1 - File Infrastructure</a></h2>
+ <h3><a name="Btrees">Disk Format: Level 1A - B-link Trees and B-tree Nodes</a></h3>
+
+ <p>B-link trees allow flexible storage for objects which tend to grow
+ in ways that cause the object to be stored discontiguously. B-trees
+ are described in various algorithms books including "Introduction to
+ Algorithms" by Thomas H. Cormen, Charles E. Leiserson, and Ronald
+ L. Rivest. The B-link tree, in which the sibling nodes at a
+ particular level in the tree are stored in a doubly-linked list,
+ is described in the "Efficient Locking for Concurrent Operations
+ on B-trees" paper by Phillip Lehman and S. Bing Yao as published
+ in the <em>ACM Transactions on Database Systems</em>, Vol. 6,
+ No. 4, December 1981.
+
+ <p>The B-link trees implemented by the file format contain one more
+ key than the number of children. In other words, each child
+ pointer out of a B-tree node has a left key and a right key.
+ The pointers out of internal nodes point to sub-trees while
+ the pointers out of leaf nodes point to symbol nodes and
+ raw data chunks.
+ Aside from that difference, internal nodes and leaf nodes
+ are identical.
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <B>B-tree Nodes</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+
+ <tr align=center>
+ <td colspan=4>Node Signature</td>
+
+ <tr align=center>
+ <td>Node Type</td>
+ <td>Node Level</td>
+ <td colspan=2>Entries Used</td>
+
+ <tr align=center>
+ <td colspan=4>Address of Left Sibling</td>
+
+ <tr align=center>
+ <td colspan=4>Address of Right Sibling</td>
+
+ <tr align=center>
+ <td colspan=4>Key 0 (variable size)</td>
+
+ <tr align=center>
+ <td colspan=4>Address of Child 0</td>
+
+ <tr align=center>
+ <td colspan=4>Key 1 (variable size)</td>
+
+ <tr align=center>
+ <td colspan=4>Address of Child 1</td>
+
+ <tr align=center>
+ <td colspan=4>...</td>
+
+ <tr align=center>
+ <td colspan=4>Key 2<em>K</em> (variable size)</td>
+
+ <tr align=center>
+ <td colspan=4>Address of Child 2<em>K</em></td>
+
+ <tr align=center>
+ <td colspan=4>Key 2<em>K</em>+1 (variable size)</td>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Node Signature</td>
+ <td>The ASCII character string <code>TREE</code> is
+ used to indicate the
+ beginning of a B-link tree node. This gives file
+ consistency checking utilities a better chance of
+ reconstructing a damaged file.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Node Type</td>
+ <td>Each B-link tree points to a particular type of data.
+ This field indicates the type of data as well as
+ implying the maximum degree <em>K</em> of the tree and
+ the size of each Key field.
+ <br>
+ <dl compact>
+ <dt>0
+ <dd>This tree points to group nodes.
+ <dt>1
+ <dd>This tree points to a new data chunk.
+ </dl>
+ </td>
+ </tr>
+
+ <tr valign=top>
+ <td>Node Level</td>
+ <td>The node level indicates the level at which this node
+ appears in the tree (leaf nodes are at level zero). Not
+ only does the level indicate whether child pointers
+ point to sub-trees or to data, but it can also be used
+ to help file consistency checking utilities reconstruct
+ damanged trees.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Entries Used</td>
+ <td>This determines the number of children to which this
+ node points. All nodes of a particular type of tree
+ have the same maximum degree, but most nodes will point
+ to less than that number of children. The valid child
+ pointers and keys appear at the beginning of the node
+ and the unused pointers and keys appear at the end of
+ the node. The unused pointers and keys have undefined
+ values.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Address of Left Sibling</td>
+ <td>This is the file address of the left sibling of the
+ current node relative to the super block. If the current
+ node is the left-most node at this level then this field
+ is the undefined address (all bits set).</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Address of Right Sibling</td>
+ <td>This is the file address of the right sibling of the
+ current node relative to the super block. If the current
+ node is the right-most node at this level then this
+ field is the undefined address (all bits set).</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Keys and Child Pointers</td>
+ <td>Each tree has 2<em>K</em>+1 keys with 2<em>K</em>
+ child pointers interleaved between the keys. The number
+ of keys and child pointers actually containing valid
+ values is determined by the <em>Entries Used</em> field. If
+ that field is <em>N</em> then the B-link tree contains
+ <em>N</em> child pointers and <em>N</em>+1 keys.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Key</td>
+ <td>The format and size of the key values is determined by
+ the type of data to which this tree points. The keys are
+ ordered and are boundaries for the contents of the child
+ pointer; that is, the key values represented by child
+ <em>N</em> fall between Key <em>N</em> and Key
+ <em>N</em>+1. Whether the interval is open or closed on
+ each end is determined by the type of data to which the
+ tree points.
+ <p>
+ The format of the key depends on the node type.
+ For nodes of node type 1, the key is formatted as follows:
+ <center>
+ <table>
+ <tr valign=top align=left>
+ <td width=40%>Bytes 1-4</td>
+ <td>Size of chunk in bytes.</td>
+ <tr valign=top align=left></tr>
+ <td>Bytes 4-8</td>
+ <td>Filter mask, a 32-bit bitfield indicating which
+ filters have been applied to that chunk.</td>
+ </tr><tr valign=top align=left>
+ <td><i>N</i> fields of 8 bytes each</td>
+ <td>A 64-bit index indicating the offset of the
+ chunk within the dataset where <i>N</i> is the number
+ of dimensions of the dataset. For example, if
+ a chunk in a 3-dimensional dataset begins at the
+ position <code>[5,5,5]</code>, there will be three
+ such 8-bit indices, each with the value of
+ <code>5</code>.</td>
+ </tr>
+ </table>
+ </center>
+ <p>
+ For nodes of node type 0, the key is formatted as follows:
+ <center>
+ <table>
+ <tr valign=top align=left>
+ <td width=40%>A single field of <i>Size of Lengths</i>
+ bytes</td>
+ <td>Indicates the byte offset into the local heap
+ for the first object name in the subtree which
+ that key describes.</td>
+ </tr>
+ </table>
+ </center>
+ </td>
+ </tr>
+
+ <tr valign=top>
+ <td>Child Pointers</td>
+ <td>The tree node contains file addresses of subtrees or
+ data depending on the node level. Nodes at Level 0 point
+ to data addresses, either data chunk or group nodes.
+ Nodes at non-zero levels point to other nodes of the
+ same B-tree.</td>
+ </tr>
+ </table>
+ </center>
+
+<p>
+ Each B-tree node looks like this:
+
+ <center>
+ <table>
+ <tr valign=top align=center>
+ <td>key[0]</td><td>&nbsp;&nbsp;</td>
+ <td>child[0]</td><td>&nbsp;&nbsp;</td>
+ <td>key[1]</td><td>&nbsp;&nbsp;</td>
+ <td>child[1]</td><td>&nbsp;&nbsp;</td>
+ <td>key[2]</td><td>&nbsp;&nbsp;</td>
+ <td>...</td><td>&nbsp;&nbsp;</td>
+ <td>...</td><td>&nbsp;&nbsp;</td>
+ <td>key[<i>N</i>-1]</td><td>&nbsp;&nbsp;</td>
+ <td>child[<i>N</i>-1]</td><td>&nbsp;&nbsp;</td>
+ <td>key[<i>N</i>]</td>
+ </tr>
+ </table>
+ </center>
+
+ where child[<i>i</i>] is a pointer to a sub-tree (at a level
+ above Level 0) or to data (at Level 0).
+ Each key[<i>i</i>] describes an <i>item</i> stored by the B-tree
+ (a chunk or an object of a group node). The range of values
+ represented by child[<i>i</i>] are indicated by key[<i>i</i>]
+ and key[<i>i</i>+1].
+
+
+ <p>The following question must next be answered:
+ "Is the value described by key[<i>i</i>] contained in
+ child[<i>i</i>-1] or in child[<i>i</i>]?"
+ The answer depends on the type of tree.
+ In trees for groups (node type 0) the object described by
+ key[<i>i</i>] is the greatest object contained in
+ child[<i>i</i>-1] while in chunk trees (node type 1) the
+ chunk described by key[<i>i</i>] is the least chunk in
+ child[<i>i</i>].
+
+ <p>That means that key[0] for group trees is sometimes unused;
+ it points to offset zero in the heap, which is always the
+ empty string and compares as "less-than" any valid object name.
+
+ <p>And key[<i>N</i>] for chunk trees is sometimes unused;
+ it contains a chunk offset which compares as "greater-than"
+ any other chunk offset and has a chunk byte size of zero
+ to indicate that it is not actually allocated.
+
+
+ <h3><a name="SymbolTable">Disk Format: Level 1B - Group and Symbol Nodes</a></h3>
+
+ <p>A group is an object internal to the file that allows
+ arbitrary nesting of objects (including other groups).
+ A group maps a set of names to a set of file
+ address relative to the base address. Certain meta data
+ for an object to which the group points can be duplicated
+ in the group symbol table in addition to the object header.
+
+ <p>An HDF5 object name space can be stored hierarchically by
+ partitioning the name into components and storing each
+ component in a group. The group entry for a
+ non-ultimate component points to the group containing
+ the next component. The group entry for the last
+ component points to the object being named.
+
+ <p>A group is a collection of group nodes pointed
+ to by a B-link tree. Each group node contains entries
+ for one or more symbols. If an attempt is made to add a
+ symbol to an already full group node containing
+ 2<em>K</em> entries, then the node is split and one node
+ contains <em>K</em> symbols and the other contains
+ <em>K</em>+1 symbols.
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <B>Group Node (A Leaf of a B-tree)</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+
+ <tr align=center>
+ <td colspan=4>Node Signature</td>
+
+ <tr align=center>
+ <td>Version Number</td>
+ <td>Reserved for Future Use</td>
+ <td colspan=2>Number of Symbols</td>
+
+ <tr align=center>
+ <td colspan=4><br><br>Group Entries<br><br><br></td>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Node Signature</td>
+ <td>The ASCII character string <code>SNOD</code> is
+ used to indicate the
+ beginning of a group node. This gives file
+ consistency checking utilities a better chance of
+ reconstructing a damaged file.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Version Number</td>
+ <td>The version number for the group node. This
+ document describes version 1.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Number of Symbols</td>
+ <td>Although all group nodes have the same length,
+ most contain fewer than the maximum possible number of
+ symbol entries. This field indicates how many entries
+ contain valid data. The valid entries are packed at the
+ beginning of the group node while the remaining
+ entries contain undefined values.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Group Entries</td>
+ <td>Each symbol has an entry in the group node.
+ The format of the entry is described below.</td>
+ </tr>
+ </table>
+ </center>
+
+ <h3><a name="SymbolTableEntry">
+ Disk Format: Level 1C - Group Entry </a></h3>
+
+ <p>Each group entry in a group node is designed
+ to allow for very fast browsing of stored objects.
+ Toward that design goal, the group entries
+ include space for caching certain constant meta data from the
+ object header.
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <B>Group Entry</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Name Offset (&lt;size&gt; bytes)</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Object Header Address</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Cache Type</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Reserved</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br><br>Scratch-pad Space (16 bytes)<br><br><br></td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Name Offset</td>
+ <td>This is the byte offset into the group local
+ heap for the name of the object. The name is null
+ terminated.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Object Header Address</td>
+ <td>Every object has an object header which serves as a
+ permanent location for the object's meta data. In addition
+ to appearing in the object header, some meta data can be
+ cached in the scratch-pad space.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Cache Type</td>
+ <td>The cache type is determined from the object header.
+ It also determines the format for the scratch-pad space.
+ <br>
+ <dl compact>
+ <dt>0
+ <dd>No data is cached by the group entry. This
+ is guaranteed to be the case when an object header
+ has a link count greater than one.
+
+ <dt>1
+ <dd>Object header meta data is cached in the group
+ entry. This implies that the group
+ entry refers to another group.
+
+ <dt>2
+ <dd>The entry is a symbolic link. The first four bytes
+ of the scratch-pad space are the offset into the local
+ heap for the link value. The object header address
+ will be undefined.
+
+ <dt><em>N</em>
+ <dd>Other cache values can be defined later and
+ libraries that do not understand the new values will
+ still work properly.
+ </dl>
+ </td>
+ </tr>
+
+ <tr valign=top>
+ <td>Reserved</td>
+ <td>These four bytes are present so that the scratch-pad
+ space is aligned on an eight-byte boundary. They are
+ always set to zero.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Scratch-pad Space</td>
+ <td>This space is used for different purposes, depending
+ on the value of the Cache Type field. Any meta-data
+ about a dataset object represented in the scratch-pad
+ space is duplicated in the object header for that
+ dataset. This meta data can include the datatype
+ and the size of the dataspace for a dataset whose datatype
+ is atomic and whose dataspace is fixed and less than
+ four dimensions.
+ Furthermore, no data is cached in the group
+ entry scratch-pad space if the object header for
+ the group entry has a link count greater than
+ one.</td>
+ </tr>
+ </table>
+ </center>
+
+ <h4>Format of the Scratch-pad Space</h4>
+
+ <p>The group entry scratch-pad space is formatted
+ according to the value in the Cache Type field.
+
+ <p>If the Cache Type field contains the value zero
+ (<code>0</code>) then no information is
+ stored in the scratch-pad space.
+
+ <p>If the Cache Type field contains the value one
+ (<code>1</code>), then the scratch-pad space
+ contains cached meta data for another object header
+ in the following format:
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <B>Object Header Scratch-pad Format</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+
+ <tr align=center>
+ <td colspan=4>Address of B-tree</td>
+
+ <tr align=center>
+ <td colspan=4>Address of Name Heap</td>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Address of B-tree</td>
+ <td>This is the file address for the root of the
+ group's B-tree.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Address of Name Heap</td>
+ <td>This is the file address for the group's local
+ heap, in which are stored the symbol names.</td>
+ </tr>
+ </table>
+ </center>
+
+
+ <p>If the Cache Type field contains the value two
+ (<code>2</code>), then the scratch-pad space
+ contains cached meta data for another symbolic link
+ in the following format:
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <B>Symbolic Link Scratch-pad Format</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Offset to Link Value</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Offset to Link Value</td>
+ <td>The value of a symbolic link (that is, the name of the
+ thing to which it points) is stored in the local heap.
+ This field is the 4-byte offset into the local heap for
+ the start of the link value, which is null terminated.</td>
+ </tr>
+ </table>
+ </center>
+
+ <h3><a name="LocalHeap">Disk Format: Level 1D - Local Heaps</a></h3>
+
+ <p>A heap is a collection of small heap objects. Objects can be
+ inserted and removed from the heap at any time.
+ The address of a heap does not change once the heap is created.
+ References to objects are stored in the group table;
+ the names of those objects are stored in the local heap.
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Local Heaps</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Heap Signature</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Reserved (zero)</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Data Segment Size</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Offset to Head of Free-list (&lt;size&gt; bytes)</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Address of Data Segment</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Heap Signature</td>
+ <td>The ASCII character string <code>HEAP</code>
+ is used to indicate the
+ beginning of a heap. This gives file consistency
+ checking utilities a better chance of reconstructing a
+ damaged file.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Data Segment Size</td>
+ <td>The total amount of disk memory allocated for the heap
+ data. This may be larger than the amount of space
+ required by the object stored in the heap. The extra
+ unused space holds a linked list of free blocks.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Offset to Head of Free-list</td>
+ <td>This is the offset within the heap data segment of the
+ first free block (or all 0xff bytes if there is no free
+ block). The free block contains &lt;size&gt; bytes that
+ are the offset of the next free chunk (or all 0xff bytes
+ if this is the last free chunk) followed by &lt;size&gt;
+ bytes that store the size of this free chunk.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Address of Data Segment</td>
+ <td>The data segment originally starts immediately after
+ the heap header, but if the data segment must grow as a
+ result of adding more objects, then the data segment may
+ be relocated, in its entirety, to another part of the
+ file.</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>Objects within the heap should be aligned on an 8-byte boundary.
+
+ <h3><a name="GlobalHeap">Disk Format: Level 1E - Global Heap</a></h3>
+
+ <p>Each HDF5 file has a global heap which stores various types of
+ information which is typically shared between datasets. The
+ global heap was designed to satisfy these goals:
+
+ <ol type="A">
+ <li>Repeated access to a heap object must be efficient without
+ resulting in repeated file I/O requests. Since global heap
+ objects will typically be shared among several datasets, it is
+ probable that the object will be accessed repeatedly.
+
+ <br><br>
+ <li>Collections of related global heap objects should result in
+ fewer and larger I/O requests. For instance, a dataset of
+ void pointers will have a global heap object for each
+ pointer. Reading the entire set of void pointer objects
+ should result in a few large I/O requests instead of one small
+ I/O request for each object.
+
+ <br><br>
+ <li>It should be possible to remove objects from the global heap
+ and the resulting file hole should be eligible to be reclaimed
+ for other uses.
+ <br><br>
+ </ol>
+
+ <p>The implementation of the heap makes use of the memory
+ management already available at the file level and combines that
+ with a new top-level object called a <em>collection</em> to
+ achieve Goal B. The global heap is the set of all collections.
+ Each global heap object belongs to exactly one collection and
+ each collection contains one or more global heap objects. For
+ the purposes of disk I/O and caching, a collection is treated as
+ an atomic object.
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <B>A Global Heap Collection</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Magic Number</td>
+ </tr>
+
+ <tr align=center>
+ <td>Version</td>
+ <td colspan=3>Reserved</td>
+ </td>
+
+ <tr align=center>
+ <td colspan=4>Collection Size</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Global Heap Object 1
+ <i>(described below)</i><br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Global Heap Object 2<br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>...<br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Global Heap Object <em>N</em><br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Global Heap Object 0 (free space)<br><br></td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Magic Number</td>
+ <td>The magic number for global heap collections are the
+ four bytes <code>G</code>, <code>C</code>, <code>O</code>,
+ and <code>L</code>.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Version</td>
+ <td>Each collection has its own version number so that new
+ collections can be added to old files. This document
+ describes version zero of the collections.
+ </tr>
+
+ <tr valign=top>
+ <td>Collection Data Size</td>
+ <td>This is the size in bytes of the entire collection
+ including this field. The default (and minimum)
+ collection size is 4096 bytes which is a typical file
+ system block size and which allows for 170 16-byte heap
+ objects plus their overhead.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Object 1 through <em>N</em></td>
+ <td>The objects are stored in any order with no
+ intervening unused space.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Object 0</td>
+ <td>Object 0 (zero), when present, represents the free space in
+ the collection. Free space always appears at the end of
+ the collection. If the free space is too small to store
+ the header for Object 0 (described below) then the
+ header is implied and the collection contains no free space.
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <B>Global Heap Object</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=2>Object ID</td>
+ <td colspan=2>Reference Count</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Reserved</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Object Data Size</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Object Data<br><br></td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Object ID</td>
+ <td>Each object has a unique identification number within a
+ collection. The identification numbers are chosen so that
+ new objects have the smallest value possible with the
+ exception that the identifier <code>0</code> always refers to the
+ object which represents all free space within the
+ collection.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Reference Count</td>
+ <td>All heap objects have a reference count field. An
+ object which is referenced from some other part of the
+ file will have a positive reference count. The reference
+ count for Object 0 is always zero.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Reserved</td>
+ <td>Zero padding to align next field on an 8-byte
+ boundary.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Object Size</td> <td>This is the size of the the fields
+ above plus the object data stored for the object. The
+ actual storage size is rounded up to a multiple of
+ eight.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Object Data</td>
+ <td>The object data is treated as a one-dimensional array
+ of bytes to be interpreted by the caller.</td>
+ </tr>
+ </table>
+ </center>
+
+ <h3><a name="FreeSpaceIndex">Disk Format: Level 1F - Free-space Heap</a></h3>
+
+ <p>The Free-space Index is a collection of blocks of data,
+ dispersed throughout the file, which are currently not used by
+ any file objects.
+
+ <p>The super block contains a pointer to root of the free-space description;
+ that pointer is currently (i.e., in HDF5 Release 1.2) required
+ to be the undefined address <code>0xfff...ff</code>.
+
+ <p>The free-sapce index is not otherwise publicly defined at this time.
+
+
+ <!--
+ <p>The Free-space Index is a collection of blocks of data,
+ dispersed throughout the file, which are currently not used by
+ any file objects. The blocks of data are indexed by a B-tree of
+ their length within the file.
+
+
+ <p>Each B-tree page is composed of the following entries and
+ B-tree management information, organized as follows:
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=bottom>
+ <B>HDF5 Free-space Heap Page</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+
+ <tr align=center>
+ <td colspan=4>Free-space Heap Signature</td>
+ <tr align=center>
+ <td colspan=4>B-tree Left-link Offset</td>
+ <tr align=center>
+ <td colspan=4><br>Length of Free-block #1<br> <br></td>
+ <tr align=center>
+ <td colspan=4><br>Offset of Free-block #1<br> <br></td>
+ <tr align=center>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr align=center>
+ <td colspan=4><br>Length of Free-block #n<br> <br></td>
+ <tr align=center>
+ <td colspan=4><br>Offset of Free-block #n<br> <br></td>
+ <tr align=center>
+ <td colspan=4>"High" Offset</td>
+ <tr align=center>
+ <td colspan=4>Right-link Offset</td>
+ </table>
+ </center>
+
+ <p>
+ <dl>
+ <dt> The elements of the free-space heap page are described below:
+ <dd>
+ <dl>
+ <dt>Free-space Heap Signature: (4 bytes)
+ <dd>The ASCII character string <code>FREE</code>
+ is used to indicate the
+ beginning of a free-space heap B-tree page. This gives
+ file consistency checking utilities a better chance of
+ reconstructing a damaged file.
+
+ <dt>B-tree Left-link Offset: (&lt;offset&gt; bytes)
+ <dd>This value is used to indicate the offset of all offsets
+ in the B-link-tree which are smaller than the value of the
+ offset in entry #1. This value is also used to indicate a
+ leaf node in the B-link-tree by being set to all ones.
+
+ <dt>Length of Free-block #n: (&lt;length&gt; bytes)
+ <dd>This value indicates the length of an unused block in
+ the file.
+
+ <dt>Offset of Free-block #n: (&lt;offset&gt; bytes)
+ <dd>This value indicates the offset in the file of an
+ unused block in the file.
+
+ <dt>"High" Offset: (4-bytes)
+ <dd>This offset is used as the upper bound on offsets
+ contained within a page when the page has been split.
+
+ <dt>Right-link Offset: (&lt;offset&gt; bytes)
+ <dd>This value is used to indicate the offset of the next
+ child to the right of the parent of this group
+ page. When there is no node to the right, this value is
+ all zeros.
+ </dl>
+ </dl>
+
+ <p>The algorithms for searching and inserting objects in the
+ B-tree pages are described fully in the Lehman and Yao paper,
+ which should be read to provide a full description of the
+ B-tree's usage.
+-->
+
+
+<br><br>
+<br><br>
+
+
+ <h2><a name="DataObject">Disk Format: Level 2 - Data Objects </a></h2>
+
+ <p>Data objects contain the real information in the file. These
+ objects compose the scientific data and other information which
+ are generally thought of as "data" by the end-user. All the
+ other information in the file is provided as a framework for
+ these data objects.
+
+ <p>A data object is composed of header information and data
+ information. The header information contains the information
+ needed to interpret the data information for the data object as
+ well as additional "meta-data" or pointers to additional
+ "meta-data" used to describe or annotate each data object.
+
+ <h3><a name="ObjectHeader">
+ Disk Format: Level 2a - Data Object Headers</a></h3>
+
+ <p>The header information of an object is designed to encompass
+ all the information about an object which would be desired to be
+ known, except for the data itself. This information includes
+ the dimensionality, number-type, information about how the data
+ is stored on disk (in external files, compressed, broken up in
+ blocks, etc.), as well as other information used by the library
+ to speed up access to the data objects or maintain a file's
+ integrity. The header of each object is not necessarily located
+ immediately prior to the object's data in the file and in fact
+ may be located in any position in the file.
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <B>Object Headers</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=1 width="25%">Version # of Object Header</td>
+ <td colspan=1 width="25%">Reserved</td>
+ <td colspan=2 width="50%">Number of Header Messages</td>
+ </tr>
+ <tr align=center>
+ <td colspan=4>Object Reference Count</td>
+ </tr>
+ <tr align=center>
+ <td colspan=4><br>Total Object Header Size<br><br></td>
+ </tr>
+ <tr align=center>
+ <td colspan=2>Header Message Type #1</td>
+ <td colspan=2>Size of Header Message Data #1</td>
+ </tr>
+ <tr align=center>
+ <td>Flags</td>
+ <td colspan=3>Reserved</td>
+ </tr>
+ <tr align=center>
+ <td colspan=4><br>Header Message Data #1<br><br></td>
+ </tr>
+ <tr align=center>
+ <td colspan=4>.<br>.<br>.<br></td>
+ </tr>
+ <tr align=center>
+ <td colspan=2>Header Message Type #n</td>
+ <td colspan=2>Size of Header Message Data #n</td>
+ </tr>
+ <tr align=center>
+ <td>Flags</td>
+ <td colspan=3>Reserved</td>
+ </tr>
+ <tr align=center>
+ <td colspan=4><br>Header Message Data #n<br><br></td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Version number of the object header</td>
+ <td>This value is used to determine the format of the
+ information in the object header. When the format of the
+ information in the object header is changed, the version number
+ is incremented and can be used to determine how the
+ information in the object header is formatted.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Reserved</td>
+ <td>Always set to zero.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Number of header messages</td>
+ <td>This value determines the number of messages listed in
+ this object header. This provides a fast way for software
+ to prepare storage for the messages in the header.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Object Reference Count</td>
+ <td>This value specifies the number of references to this
+ object within the current file. References to the
+ data object from external files are not tracked.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Total Object Header Size</td>
+ <td>This value specifies the total number of bytes of header
+ message data following this length field for the current
+ message as well as any continuation data located elsewhere
+ in the file.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Header Message Type</td>
+ <td>The header message type specifies the type of
+ information included in the header message data following
+ the type along with a small amount of other information.
+ Bit 15 of the message type is set if the message is
+ constant (constant messages cannot be changed since they
+ may be cached in group entries throughout the
+ file). The header message types for the pre-defined
+ header messages will be included in further discussion
+ below.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Size of Header Message Data</td>
+ <td>This value specifies the number of bytes of header
+ message data following the header message type and length
+ information for the current message. The size includes
+ padding bytes to make the message a multiple of eight
+ bytes.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Flags</td>
+ <td>This is a bit field with the following definition:
+ <dl>
+ <dt><code>0</code>
+ <dd>If set, the message data is constant. This is used
+ for messages like the datatype message of a dataset.
+ <dt><code>1</code>
+ <dd>If set, the message is stored in the global heap and
+ the Header Message Data field contains a Shared Object
+ message and the Size of Header Message Data field
+ contains the size of that Shared Object message.
+ <dt><code>2-7</code>
+ <dd>Reserved
+ </dl>
+ </td>
+
+ <tr valign=top>
+ <td>Header Message Data</td>
+ <td>The format and length of this field is determined by the
+ header message type and size respectively. Some header
+ message types do not require any data and this information
+ can be eliminated by setting the length of the message to
+ zero. The data is padded with enough zeros to make the
+ size a multiple of eight.</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>The header message types and the message data associated with
+ them compose the critical "meta-data" about each object. Some
+ header messages are required for each object while others are
+ optional. Some optional header messages may also be repeated
+ several times in the header itself, the requirements and number
+ of times allowed in the header will be noted in each header
+ message description below.
+
+ <P>The following is a list of currently defined header messages:
+
+ <hr>
+ <h4><a name="NILMessage">Name: NIL</a></h4>
+ <b>Type: </b>0x0000<br>
+ <b>Length:</b> varies<br>
+ <b>Status:</b> Optional, may be repeated.<br>
+ <b>Purpose and Description:</b> The NIL message is used to
+ indicate a message
+ which is to be ignored when reading the header messages for a data object.
+ [Probably one which has been deleted for some reason.]<br>
+ <b>Format of Data:</b> Unspecified.<br>
+
+<!-- Delete examples throughout doc
+ <b>Examples:</b> None.
+-->
+
+
+ <hr>
+ <h4><a name="SimpleDataSpace">Name: Simple Dataspace</a></h4>
+
+ <b>Type: </b>0x0001<br>
+ <b>Length:</b> Varies according to the number of dimensions,
+ as described in the following table<br>
+ <b>Status:</b> The <em>Simple Dataspace</em> message is required
+ and may not be repeated. This message is currently used with
+ datasets and named dataspaces.<br>
+
+ <p>The <em>Simple Dataspace</em> message describes the number
+ of dimensions and size of each dimension that the data object
+ has. This message is only used for datasets which have a
+ simple, rectilinear grid layout; datasets requiring a more
+ complex layout (irregularly structured or unstructured grids, etc.)
+ must use the <em>Complex Dataspace</em> message for expressing
+ the space the dataset inhabits.
+ <i>(Note: The <em>Complex Dataspace</em> functionality is
+ not yet implemented (as of HDF5 Release 1.2). It is not described
+ in this document.)</i>
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Simple Dataspace Message</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td>Version</td>
+ <td>Dimensionality</td>
+ <td>Flags</td>
+ <td>Reserved</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Reserved</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Dimension Size #1 (&lt;size&gt; bytes)</td>
+ <tr align=center>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr align=center>
+ <td colspan=4>Dimension Size #n (&lt;size&gt; bytes)</td>
+ <tr align=center>
+ <td colspan=4>Dimension Maximum #1 (&lt;size&gt; bytes)</td>
+ <tr align=center>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr align=center>
+ <td colspan=4>Dimension Maximum #n (&lt;size&gt; bytes)</td>
+ <tr align=center>
+ <td colspan=4>Permutation Index #1</td>
+ <tr align=center>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr align=center>
+ <td colspan=4>Permutation Index #n</td>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Version </td>
+ <td>This value is used to determine the format of the
+ Simple Dataspace Message. When the format of the
+ information in the message is changed, the version number
+ is incremented and can be used to determine how the
+ information in the object header is formatted.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Dimensionality</td>
+ <td>This value is the number of dimensions that the data
+ object has.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Flags</td>
+ <td>This field is used to store flags to indicate the
+ presence of parts of this message. Bit 0 (the least
+ significant bit) is used to indicate that maximum
+ dimensions are present. Bit 1 is used to indicate that
+ permutation indices are present for each dimension.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Dimension Size #n (&lt;size&gt; bytes)</td>
+ <td>This value is the current size of the dimension of the
+ data as stored in the file. The first dimension stored in
+ the list of dimensions is the slowest changing dimension
+ and the last dimension stored is the fastest changing
+ dimension.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Dimension Maximum #n (&lt;size&gt; bytes)</td>
+ <td>This value is the maximum size of the dimension of the
+ data as stored in the file. This value may be the special
+ value &lt;UNLIMITED&gt; (all bits set) which indicates
+ that the data may expand along this dimension
+ indefinitely. If these values are not stored, the maximum
+ value of each dimension is assumed to be the same as the
+ current size value.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Permutation Index #n (4 bytes)</td>
+ <td>This value is the index permutation used to map
+ each dimension from the canonical representation to an
+ alternate axis for each dimension. If these values are
+ not stored, the first dimension stored in the list of
+ dimensions is the slowest changing dimension and the last
+ dimension stored is the fastest changing dimension.</td>
+ </tr>
+ </table>
+ </center>
+
+<!-- Delete examples throughout doc
+ <h4>Examples</h4>
+ <dl>
+ <dt> Example #1
+ <dd>A sample 640 horizontally by 480 vertically raster image
+ dimension header. The number of dimensions would be set to 2
+ and the first dimension's size and maximum would both be set
+ to 480. The second dimension's size and maximum would both be
+ set to 640
+.
+ <dt>Example #2
+ <dd>A sample 4 dimensional scientific dataset which is composed
+ of 30x24x3 slabs of data being written out in an unlimited
+ series every several minutes as timestep data (currently there
+ are five slabs). The number of dimensions is 4. The first
+ dimension size is 5 and its maximum is &lt;UNLIMITED&gt;. The
+ second through fourth dimension's size and maximum value are
+ set to 3, 24, and 30 respectively.
+
+ <dt>Example #3
+ <dd>A sample unlimited length text string, currently of length
+ 83. The number of dimensions is 1, the size of the first
+ dimension is 83 and the maximum of the first dimension is set
+ to &lt;UNLIMITED&gt;, allowing further text data to be
+ appended to the string or possibly the string to be replaced
+ with another string of a different size. (This could also be
+ stored as a scalar dataset with number-type set to "string")
+ </dl>
+-->
+
+<!-- DELETE ENTIRE DATASPACE SECTION -->
+<!--
+ <hr>
+ <h4><a name="DataSpaceMessage">Name: Complex Dataspace (Fiber Bundle?)</a></h4>
+ <b>Type: </b>0x0002<br>
+ <b>Length:</b> varies<br>
+
+ <b>Status:</b> One of the <em>Simple Dataspace</em> or
+ <em>Complex Dataspace</em> messages is required (but not both) and may
+ not be repeated.<br> <b>Purpose and Description:</b> The
+ <em>Dataspace</em> message describes space that the dataset is
+ mapped onto in a more comprehensive way than the <em>Simple
+ Dimensionality</em> message is capable of handling. The
+ dataspace of a dataset encompasses the type of coordinate system
+ used to locate the dataset's elements as well as the structure and
+ regularity of the coordinate system. The dataspace also
+ describes the number of dimensions which the dataset inhabits as
+ well as a possible higher dimensional space in which the dataset
+ is located within.
+
+ <br>
+ <b>Format of Data:</b>
+
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=bottom>
+ <B>HDF5 Dataspace Message Layout</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+
+ <tr align=center>
+ <td colspan=4>Mesh Type</td>
+ <tr align=center>
+ <td colspan=4>Logical Dimensionality</td>
+ </table>
+ </center>
+
+ <p>
+ <dl>
+ <dt>The elements of the dimensionality message are described below:
+ <dd>
+ <dl>
+ <dt>Mesh Type: (unsigned 32-bit integer)
+ <dd>This value indicates whether the grid is
+ polar/spherical/cartesion,
+ structured/unstructured and regular/irregular. <br>
+ The mesh type value is broken up as follows: <br>
+
+ <P>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=bottom>
+ <B>HDF5 Mesh-type Layout</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+
+ <tr align=center>
+ <td colspan=1>Mesh Embedding</td>
+ <td colspan=1>Coordinate System</td>
+ <td colspan=1>Structure</td>
+ <td colspan=1>Regularity</td>
+ </table>
+ </center>
+ The following are the definitions of mesh-type bytes:
+ <dl>
+ <dt>Mesh Embedding
+ <dd>This value indicates whether the dataset dataspace
+ is located within
+ another dataspace or not:
+ <dl> <dl>
+ <dt>&lt;STANDALONE&gt;
+ <dd>The dataset mesh is self-contained and is not
+ embedded in another mesh.
+ <dt>&lt;EMBEDDED&gt;
+ <dd>The dataset's dataspace is located within
+ another dataspace, as
+ described in information below.
+ </dl> </dl>
+ <dt>Coordinate System
+ <dd>This value defines the type of coordinate system
+ used for the mesh:
+ <dl> <dl>
+ <dt>&lt;POLAR&gt;
+ <dd>The last two dimensions are in polar
+ coordinates, higher dimensions are
+ cartesian.
+ <dt>&lt;SPHERICAL&gt;
+ <dd>The last three dimensions are in spherical
+ coordinates, higher dimensions
+ are cartesian.
+ <dt>&lt;CARTESIAN&gt;
+ <dd>All dimensions are in cartesian coordinates.
+ </dl> </dl>
+ <dt>Structure
+ <dd>This value defines the locations of the grid-points
+ on the axes:
+ <dl> <dl>
+ <dt>&lt;STRUCTURED&gt;
+ <dd>All grid-points are on integral, sequential
+ locations, starting from 0.
+ <dt>&lt;UNSTRUCTURED&gt;
+ <dd>Grid-points locations in each dimension are
+ explicitly defined and
+ may be of any numeric datatype.
+ </dl> </dl>
+ <dt>Regularity
+ <dd>This value defines the locations of the dataset
+ points on the grid:
+ <dl> <dl>
+ <dt>&lt;REGULAR&gt;
+ <dd>All dataset elements are located at the
+ grid-points defined.
+ <dt>&lt;IRREGULAR&gt;
+ <dd>Each dataset element has a particular
+ grid-location defined.
+ </dl> </dl>
+ </dl>
+ <p>The following grid combinations are currently allowed:
+ <dl> <dl>
+ <dt>&lt;POLAR-STRUCTURED-REGULAR&gt;
+ <dt>&lt;SPHERICAL-STRUCTURED-REGULAR&gt;
+ <dt>&lt;CARTESIAN-STRUCTURED-REGULAR&gt;
+ <dt>&lt;POLAR-UNSTRUCTURED-REGULAR&gt;
+ <dt>&lt;SPHERICAL-UNSTRUCTURED-REGULAR&gt;
+ <dt>&lt;CARTESIAN-UNSTRUCTURED-REGULAR&gt;
+ <dt>&lt;CARTESIAN-UNSTRUCTURED-IRREGULAR&gt;
+ </dl> </dl>
+ All of the above grid types can be embedded within another
+ dataspace.
+ <br> <br>
+ <dt>Logical Dimensionality: (unsigned 32-bit integer)
+ <dd>This value is the number of dimensions that the dataset occupies.
+
+ <P>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=bottom>
+ <B>HDF5 Dataspace Embedded Dimensionality Information</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+
+ <tr align=center>
+ <td colspan=4>Embedded Dimensionality</td>
+ <tr align=center>
+ <td colspan=4>Embedded Dimension Size #1</td>
+ <tr align=center>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr align=center>
+ <td colspan=4>Embedded Dimension Size #n</td>
+ <tr align=center>
+ <td colspan=4>Embedded Origin Location #1</td>
+ <tr align=center>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr align=center>
+ <td colspan=4>Embedded Origin Location #n</td>
+ </table>
+ </center>
+
+ <dt>Embedded Dimensionality: (unsigned 32-bit integer)
+ <dd>This value is the number of dimensions of the space the
+ dataset is located
+ within. i.e. a planar dataset located within a 3-D space,
+ or a 3-D dataset
+ which is a subset of another 3-D space, etc.
+ <dt>Embedded Dimension Size: (unsigned 32-bit integer)
+ <dd>These values are the sizes of the dimensions of the
+ embedded dataspace
+ that the dataset is located within.
+ <dt>Embedded Origin Location: (unsigned 32-bit integer)
+ <dd>These values comprise the location of the dataset's
+ origin within the embedded dataspace.
+ </dl>
+ </dl>
+ [Comment: need some way to handle different orientations of the
+ dataset dataspace
+ within the embedded dataspace]<br>
+
+ <P>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=bottom>
+ <B>HDF5 Dataspace Structured/Regular Grid Information</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+
+ <tr align=center>
+ <td colspan=4>Logical Dimension Size #1</td>
+ <tr align=center>
+ <td colspan=4>Logical Dimension Maximum #1</td>
+ <tr align=center>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr align=center>
+ <td colspan=4>Logical Dimension Size #n</td>
+ <tr align=center>
+ <td colspan=4>Logical Dimension Maximum #n</td>
+ </table>
+ </center>
+
+ <p>
+ <dl>
+ <dt>The elements of the dimensionality message are described below:
+ <dd>
+ <dl>
+ <dt>Logical Dimension Size #n: (unsigned 32-bit integer)
+ <dd>This value is the current size of the dimension of the
+ data as stored in
+ the file. The first dimension stored in the list of
+ dimensions is the slowest
+ changing dimension and the last dimension stored is the
+ fastest changing
+ dimension.
+ <dt>Logical Dimension Maximum #n: (unsigned 32-bit integer)
+ <dd>This value is the maximum size of the dimension of the
+ data as stored in
+ the file. This value may be the special value
+ &lt;UNLIMITED&gt; which
+ indicates that the data may expand along this dimension
+ indefinitely.
+ </dl>
+ </dl>
+ <P>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=bottom>
+ <B>HDF5 Dataspace Structured/Irregular Grid Information</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+
+ <tr align=center>
+ <td colspan=4># of Grid Points in Dimension #1</td>
+ <tr align=center>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr align=center>
+ <td colspan=4># of Grid Points in Dimension #n</td>
+ <tr align=center>
+ <td colspan=4>Datatype of Grid Point Locations</td>
+ <tr align=center>
+ <td colspan=4>Location of Grid Points in Dimension #1</td>
+ <tr align=center>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr align=center>
+ <td colspan=4>Location of Grid Points in Dimension #n</td>
+ </table>
+ </center>
+
+ <P>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=bottom>
+ <B>HDF5 Dataspace Unstructured Grid Information</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+
+ <tr align=center>
+ <td colspan=4># of Grid Points</td>
+ <tr align=center>
+ <td colspan=4>Datatype of Grid Point Locations</td>
+ <tr align=center>
+ <td colspan=4>Grid Point Locations<br>.<br>.<br></td>
+ </table>
+ </center>
+
+ <h4><a name="DataSpaceExample">Examples:</a></h4>
+ Need some good examples, this is complex!
+-->
+
+
+ <hr>
+ <h4><a name="DataTypeMessage">Name: Datatype</a></h4>
+
+ <b>Type:</b> 0x0003<br>
+ <b>Length:</b> variable<br>
+ <b>Status:</b> One required per dataset or named datatype<br>
+
+ <p>The datatype message defines the datatype for each data point
+ of a dataset. A datatype can describe an atomic type like a
+ fixed- or floating-point type or a compound type like a C
+ struct. A datatype does not, however, describe how data points
+ are combined to produce a dataset. Datatypes are stored on disk
+ as a datatype message, which is a list of datatype classes and
+ their associated properties.
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Datatype Message</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td>Type Class and Version</td>
+ <td colspan=3>Class Bit Field</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Size in Bytes (4 bytes)</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br><br>Properties<br><br><br></td>
+ </tr>
+ </table>
+ </center>
+
+ <p>The Class Bit Field and Properties fields vary depending
+ on the Type Class, which is the low-order four bits of the Type
+ Class and Version field (the high-order four bits are the
+ version, which should be set to the value one). The type class
+ is one of 0 (fixed-point number), 1 (floating-point number),
+ 2 (date and time), 3 (text string), 4 (bit field), 5 (opaque),
+ 6 (compound), 7 (reference), 8 (enumeration), or 9 (variable-length).
+ The Class Bit Field is zero and the size of the
+ Properties field is zero except for the cases noted here.
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Bit Field for Fixed-point Numbers (Class 0)</b>
+ </caption>
+
+ <tr align=center>
+ <th width="10%">Bits</th>
+ <th width="90%">Meaning</th>
+ </tr>
+
+ <tr valign=top>
+ <td>0</td>
+ <td><b>Byte Order.</b> If zero, byte order is little-endian;
+ otherwise, byte order is big endian.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>1, 2</td>
+ <td><b>Padding type.</b> Bit 1 is the lo_pad type and bit 2
+ is the hi_pad type. If a datum has unused bits at either
+ end, then the lo_pad or hi_pad bit is copied to those
+ locations.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>3</td>
+ <td><b>Signed.</b> If this bit is set then the fixed-point
+ number is in 2's complement form.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>4-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Properties for Fixed-point Numbers (Class 0)</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=2>Bit Offset</td>
+ <td colspan=2>Bit Precision</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Bit Field for Floating-point Numbers (Class 1)</b>
+ </caption>
+
+ <tr align=center>
+ <th width="10%">Bits</th>
+ <th width="90%">Meaning</th>
+ </tr>
+
+ <tr valign=top>
+ <td>0</td>
+ <td><b>Byte Order.</b> If zero, byte order is little-endian;
+ otherwise, byte order is big endian.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>1, 2, 3</td>
+ <td><b>Padding type.</b> Bit 1 is the low bits pad type, bit 2
+ is the high bits pad type, and bit 3 is the internal bits
+ pad type. If a datum has unused bits at either or between
+ the sign bit, exponent, or mantissa, then the value of bit
+ 1, 2, or 3 is copied to those locations.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>4-5</td>
+ <td><b>Normalization.</b> The value can be 0 if there is no
+ normalization, 1 if the most significant bit of the
+ mantissa is always set (except for 0.0), and 2 if the most
+ signficant bit of the mantissa is not stored but is
+ implied to be set. The value 3 is reserved and will not
+ appear in this field.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>6-7</td>
+ <td>Reserved (zero).</td>
+ </tr>
+
+ <tr valign=top>
+ <td>8-15</td>
+ <td><b>Sign.</b> This is the bit position of the sign
+ bit.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>16-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Properties for Floating-point Numbers (Class 1)</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=2>Bit Offset</td>
+ <td colspan=2>Bit Precision</td>
+ </tr>
+
+ <tr align=center>
+ <td>Exponent Location</td>
+ <td>Exponent Size in Bits</td>
+ <td>Mantissa Location</td>
+ <td>Mantissa Size in Bits</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Exponent Bias</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Bit Field for Strings (Class 3)</b>
+ </caption>
+
+ <tr align=center>
+ <th width="10%">Bits</th>
+ <th width="90%">Meaning</th>
+ </tr>
+
+ <tr valign=top>
+ <td>0-3</td>
+ <td><b>Padding type.</b> This four-bit value determines the
+ type of padding to use for the string. The values are:
+
+ <dl>
+ <dt><code>0</code> Null terminate.
+ <dd>A zero byte marks the end of the string and is
+ guaranteed to be present after converting a long
+ string to a short string. When converting a short
+ string to a long string the value is padded with
+ additional null characters as necessary.
+
+ <br><br>
+ <dt><code>1</code> Null pad.
+ <dd>Null characters are added to the end of the value
+ during conversions from short values to long values
+ but conversion in the opposite direction simply
+ truncates the value.
+
+ <br><br>
+ <dt><code>2</code> Space pad.
+ <dd>Space characters are added to the end of the value
+ during conversions from short values to long values
+ but conversion in the opposite direction simply
+ truncates the value. This is the Fortran
+ representation of the string.
+
+ <br><br>
+ <dt><code>3-15</code> Reserved.
+ <dd>These values are reserved for future use.
+ </dl>
+ </tr>
+
+ <tr valign=top>
+ <td>4-7</td>
+ <td><b>Character Set.</b> The character set to use for
+ encoding the string. The only character set supported is
+ the 8-bit ASCII (zero) so no translations have been defined
+ yet.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>8-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Bit Field for Bitfield Types (Class 4)</b>
+ </caption>
+
+ <tr align=center>
+ <th width="10%">Bits</th>
+ <th width="90%">Meaning</th>
+ </tr>
+
+ <tr valign=top>
+ <td>0</td>
+ <td><b>Byte Order.</b> If zero, byte order is little-endian;
+ otherwise, byte order is big endian.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>1, 2</td>
+ <td><b>Padding type.</b> Bit 1 is the lo_pad type and bit 2
+ is the hi_pad type. If a datum has unused bits at either
+ end, then the lo_pad or hi_pad bit is copied to those
+ locations.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>3-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Properties for Bitfield Types (Class 4)</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=2>Bit Offset</td>
+ <td colspan=2>Bit Precision</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Bit Field for Opaque Types (Class 5)</b>
+ </caption>
+
+ <tr align=center>
+ <th width="10%">Bits</th>
+ <th width="90%">Meaning</th>
+ </tr>
+
+ <tr valign=top>
+ <td>0-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Properties for Opaque Types (Class 5)</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Null-terminated ASCII Tag<br>
+ (multiple of 8 bytes)<br><br></td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Bit Field for Compound Types (Class 6)</b>
+ </caption>
+
+ <tr align=center>
+ <th width="10%">Bits</th>
+ <th width="90%">Meaning</th>
+ </tr>
+
+ <tr valign=top>
+ <td>0-15</td>
+ <td><b>Number of Members.</b> This field contains the number
+ of members defined for the compound datatype. The member
+ definitions are listed in the Properties field of the data
+ type message.
+ </tr>
+
+ <tr valign=top>
+ <td>15-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>The Properties field of a compound datatype is a list of the
+ member definitions of the compound datatype. The member
+ definitions appear one after another with no intervening bytes.
+ The member types are described with a recursive datatype
+ message.
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Properties for Compound Types (Class 6)</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br><br>Name (null terminated, multiple of
+ eight bytes)<br><br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Byte Offset of Member in Compound Instance</td>
+ </tr>
+
+ <tr align=center>
+ <td>Dimensionality</td>
+ <td colspan=3>reserved</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Dimension Permutation</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Reserved</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Size of Dimension 0 (required)</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Size of Dimension 1 (required)</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Size of Dimension 2 (required)</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Size of Dimension 3 (required)</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br><br>Member Type Message<br><br><br></td>
+ </tr>
+
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Bit Field for Enumeration Types (Class 8)</b>
+ </caption>
+
+ <tr align=center>
+ <th width="10%">Bits</th>
+ <th width="90%">Meaning</th>
+ </tr>
+
+ <tr valign=top>
+ <td>0-15</td>
+ <td><b>Number of Members.</b> The number of name/value
+ pairs defined for the enumeration type.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>16-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Properties for Enumeration Types (Class 8)</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Parent Type<br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Names<br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Values<br><br></td>
+ </tr>
+
+ </table>
+ </center>
+
+ <center>
+ <table border=0 cellpadding=4 width="80%">
+ <tr align=left valign=top>
+ <td valign=top width=20%>Parent Type:</td>
+ <td valign=top>Each enumeration type is based on some parent type,
+ usually an integer. The information for that parent type is
+ described recursively by this field.</td>
+ </tr><tr align=left valign=top>
+ <td valign=top>Names:</td>
+ <td valign=top>The name for each name/value pair. Each name is
+ stored as a null terminated ASCII string in a multiple of
+ eight bytes. The names are in no particular order.</td>
+ </tr><tr align=left valign=top>
+ <td valign=top>Values:</td>
+ <td valign=top>The list of values in the same order as the names.
+ The values are packed (no inter-value padding) and the
+ size of each value is determined by the parent type.</td>
+ </tr>
+ </table>
+ </center>
+
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Bit Field for Variable-length Types (Class 9)</b>
+ </caption>
+
+ <tr align=center>
+ <th width="10%">Bits</th>
+ <th width="90%">Meaning</th>
+ </tr>
+
+ <tr valign=top>
+ <td>0-3</td>
+ <td><dl><dt><b>Type</b></dt>
+ <dt>0 Variable-length sequence</dt>
+ <dd>This variable-length datatype can be of any sequence
+ of data. Variable-length sequences do not have padding
+ or character set information.</dd>
+ <dt>1 Variable-length string</dt>
+ <dd>This variable-length datatype is composed of a series of
+ characters. Variable-length strings have padding and
+ character set information.</dd></dl>
+ </td>
+ </tr>
+
+ <tr valign=top>
+ <td>4-7</td>
+ <td><dl><dt><b>Padding type</b> (variable-length string only)</dt>
+ <dd>This four-bit value determines the type of padding
+ used for variable-length strings. The values are the same
+ as for the string padding type, as follows:</dd>
+ <dt>0 Null terminate</dt>
+ <dd>A zero byte marks the end of a string and is guaranteed
+ to be present after converting a long string to a short
+ string. When converting a short string to a long string,
+ the value is padded with additional null characters
+ as necessary.
+ <dt>1 Null pad</dt>
+ <dd>Null characters are added to the end of the value
+ during conversion from a short string to a longer string.
+ Conversion from a long string to a shorter string
+ simply truncates the value.</dd>
+ <dt>2 Space pad</dt>
+ <dd>Space characters are added to the end of the value
+ during conversion from a short string to a longer string.
+ Conversion from a long string to a shorter string simply
+ truncates the value.
+ This is the Fortran representation of the string.
+ </dd>
+ <dt>3-15 Reserved</dt>
+ <dd>These values are reserved for future use.</dd></dl>
+ </td>
+ </tr>
+
+ <tr valign=top>
+ <td>8-11</td>
+ <td><dl><dt><b>Character set</b> (variable-length string only)</dt>
+ <dd>This four-bit value specifies the character set
+ to be used for encoding the string.</dd>
+ <dt>0 8-bit ASCII</dt>
+ <dd>As of this writing (July 2002, Release 1.4.4),
+ 8-bit ASCII is the only character set supported.
+ Therefore, no translations have been defined.</dd></dl>
+ </td>
+ </tr>
+
+ <tr valign=top>
+ <td>12-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Properties for Variable-length Types (Class 9)</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Parent Type<br><br></td>
+ </tr>
+
+ </table>
+ </center>
+
+ <center>
+ <table border=0 cellpadding=4 width="80%">
+ <tr align=left valign=top>
+ <td valign=top width=20%>Parent Type:</td>
+ <td valign=top>Each variable-length type is based on
+ some parent type. The information for that parent type is
+ described recursively by this field.</td>
+ </tr>
+ </table>
+ </center>
+
+
+
+ <p>
+
+<!--
+ <p>Datatype examples are <a href="Datatypes.html">here</a>.
+-->
+
+
+ <hr>
+ <h4><a name="FillValueMessage">Name: Data Storage - Fill Value</a></h4>
+ <b>Type:</b> 0x0004<br>
+ <b>Length:</b> varies<br>
+ <b>Status:</b> Optional, may not be repeated.<br>
+
+ <p>The fill value message stores a single data point value which
+ is returned to the application when an uninitialized data point
+ is read from the dataset. The fill value is interpretted with
+ the same datatype as the dataset. If no fill value message is
+ present then a fill value of all zero is assumed.
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Fill Value Message</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Size (4 bytes)</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Fill Value<br><br></td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Size (4 bytes)</td>
+ <td>This is the size of the Fill Value field in bytes.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Fill Value</td>
+ <td>The fill value. The bytes of the fill value are
+ interpreted using the same datatype as for the dataset.</td>
+ </tr>
+ </table>
+ </center>
+
+ <hr>
+ <h4><a name="ReservedMessage_0005">Name: Reserved - Not Assigned Yet</a></h4>
+ <b>Type:</b> 0x0005<br>
+ <b>Length:</b> N/A<br>
+ <b>Status:</b> N/A<br>
+
+
+
+ <hr>
+ <h4><a name="CompactDataStorageMessage">Name: Data Storage - Compact</a></h4>
+
+ <b>Type:</b> 0x0006<br>
+ <b>Length:</b> varies<br>
+ <b>Status:</b> Optional, may not be repeated.<br>
+
+ <p>This message indicates that the data for the data object is
+ stored within the current HDF file by including the actual
+ data as the header data for this message. The data is
+ stored internally in
+ the <em>normal format</em>, i.e. in one chunk, uncompressed, etc.
+
+ <P>Note that one and only one of the <em>Data Storage</em> headers can be
+ stored for each data object.
+
+ <P><b>Format of Data:</b> The message data is actually composed
+ of dataset data, so the format will be determined by the dataset
+ format.
+
+<!-- Delete examples throughout doc
+ <h4><a name="CompactDataStorageExample">Examples:</a></h4>
+ [very straightforward]
+-->
+
+ <hr>
+ <h4><a name="ExternalFileListMessage">Name: Data Storage -
+ External Data Files</a></h4>
+ <b>Type:</b> 0x0007<BR>
+ <b>Length:</b> varies<BR>
+ <b>Status:</b> Optional, may not be repeated.<BR>
+
+ <p><b>Purpose and Description:</b> The external object message
+ indicates that the data for an object is stored outside the HDF5
+ file. The filename of the object is stored as a Universal
+ Resource Location (URL) of the actual filename containing the
+ data. An external file list record also contains the byte offset
+ of the start of the data within the file and the amount of space
+ reserved in the file for that data.
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>External File List Message</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td>Version</td>
+ <td colspan=3>Reserved</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=2>Allocated Slots</td>
+ <td colspan=2>Used Slots</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Heap Address<br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Slot Definitions...<br><br></td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Version </td>
+ <td>This value is used to determine the format of the
+ External File List Message. When the format of the
+ information in the message is changed, the version number
+ is incremented and can be used to determine how the
+ information in the object header is formatted.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Reserved</td>
+ <td>This field is reserved for future use.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Allocated Slots</td>
+ <td>The total number of slots allocated in the message. Its
+ value must be at least as large as the value contained in
+ the Used Slots field.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Used Slots</td>
+ <td>The number of initial slots which contain valid
+ information. The remaining slots are zero filled.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Heap Address</td>
+ <td>This is the address of a local name heap which contains
+ the names for the external files. The name at offset zero
+ in the heap is always the empty string.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Slot Definitions</td>
+ <td>The slot definitions are stored in order according to
+ the array addresses they represent. If more slots have
+ been allocated than what has been used then the defined
+ slots are all at the beginning of the list.</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>External File List Slot</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Name Offset (&lt;size&gt; bytes)<br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>File Offset (&lt;size&gt; bytes)<br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Size<br><br></td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Name Offset (&lt;size&gt; bytes)</td>
+ <td>The byte offset within the local name heap for the name
+ of the file. File names are stored as a URL which has a
+ protocol name, a host name, a port number, and a file
+ name:
+ <code><em>protocol</em>:<em>port</em>//<em>host</em>/<em>file</em></code>.
+ If the protocol is omitted then "file:" is assumed. If
+ the port number is omitted then a default port for that
+ protocol is used. If both the protocol and the port
+ number are omitted then the colon can also be omitted. If
+ the double slash and host name are omitted then
+ "localhost" is assumed. The file name is the only
+ mandatory part, and if the leading slash is missing then
+ it is relative to the application's current working
+ directory (the use of relative names is not
+ recommended).</td>
+ </tr>
+
+ <tr valign=top>
+ <td>File Offset (&lt;size&gt; bytes)</td>
+ <td>This is the byte offset to the start of the data in the
+ specified file. For files that contain data for a single
+ dataset this will usually be zero.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Size</td>
+ <td>This is the total number of bytes reserved in the
+ specified file for raw data storage. For a file that
+ contains exactly one complete dataset which is not
+ extendable, the size will usually be the exact size of the
+ dataset. However, by making the size larger one allows
+ HDF5 to extend the dataset. The size can be set to a value
+ larger than the entire file since HDF5 will read zeros
+ past the end of the file without failing.</td>
+ </tr>
+ </table>
+ </center>
+
+
+ <hr>
+ <h4><a name="LayoutMessage">Name: Data Storage - Layout</a></h4>
+
+ <b>Type:</b> 0x0008<BR>
+ <b>Length:</b> varies<BR>
+ <b>Status:</b> Required for datasets, may not be repeated.
+
+ <p><b>Purpose and Description:</b> Data layout describes how the
+ elements of a multi-dimensional array are arranged in the linear
+ address space of the file. Two types of data layout are
+ supported:
+
+ <ol>
+ <li>The array can be stored in one contiguous area of the file.
+ The layout requires that the size of the array be constant and
+ does not permit chunking, compression, checksums, encryption,
+ etc. The message stores the total size of the array and the
+ offset of an element from the beginning of the storage area is
+ computed as in C.
+
+ <li>The array domain can be regularly decomposed into chunks and
+ each chunk is allocated separately. This layout supports
+ arbitrary element traversals, compression, encryption, and
+ checksums, and the chunks can be distributed across external
+ raw data files (these features are described in other
+ messages). The message stores the size of a chunk instead of
+ the size of the entire array; the size of the entire array can
+ be calculated by traversing the B-tree that stores the chunk
+ addresses.
+ </ol>
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <B>Data Layout Message</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td>Version</td>
+ <td>Dimensionality</td>
+ <td>Layout Class</td>
+ <td>Reserved</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Reserved</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Address<br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Dimension 0 (4-bytes)</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Dimension 1 (4-bytes)</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>...</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Version</td>
+ <td>A version number for the layout message. This
+ documentation describes version one.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Dimensionality</td>
+ <td>An array has a fixed dimensionality. This field
+ specifies the number of dimension size fields later in the
+ message.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Layout Class</td>
+ <td>The layout class specifies how the other fields of the
+ layout message are to be interpreted. A value of one
+ indicates contiguous storage while a value of two
+ indicates chunked storage. Other values will be defined
+ in the future.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Address</td>
+ <td>For contiguous storage, this is the address of the first
+ byte of storage. For chunked storage this is the address
+ of the B-tree that is used to look up the addresses of the
+ chunks.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Dimensions</td>
+ <td>For contiguous storage the dimensions define the entire
+ size of the array while for chunked storage they define
+ the size of a single chunk.</td>
+ </tr>
+ </table>
+ </center>
+
+
+ <hr>
+ <h4><a name="ReservedMessage_0009">Name: Reserved - Not Assigned Yet</a></h4>
+ <b>Type:</b> 0x0009<BR>
+ <b>Length:</b> N/A<BR>
+ <b>Status:</b> N/A<BR>
+ <b>Purpose and Description:</b> N/A<BR>
+ <b>Format of Data:</b> N/A
+
+ <hr>
+ <h4><a name="ReservedMessage_000A">Name: Reserved - Not Assigned Yet</a></h4>
+ <b>Type:</b> 0x000A<BR>
+ <b>Length:</b> N/A<BR>
+ <b>Status:</b> N/A<BR>
+ <b>Purpose and Description:</b> N/A<BR>
+ <b>Format of Data:</b> N/A
+
+ <hr>
+ <h4><a name="FilterMessage">Name: Data Storage - Filter Pipeline</a></h4>
+ <b>Type:</b> 0x000B<BR>
+ <b>Length:</b> varies<BR>
+ <b>Status:</b> Optional, may not be repeated.
+
+ <p><b>Purpose and Description:</b> This message describes the
+ filter pipeline which should be applied to the data stream by
+ providing filter identification numbers, flags, a name, an
+ client data.
+
+ <p>
+ <center>
+ <table border align=center cellpadding=4 witdh="80%">
+ <caption align=top>
+ <b>Filter Pipeline Message</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td>Version</td>
+ <td>Number of Filters</td>
+ <td colspan=2>Reserved</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Reserved</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Filter List<br><br></td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Version</td>
+ <td>The version number for this message. This document
+ describes version one.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Number of Filters</td>
+ <td>The total number of filters described by this
+ message. The maximum possible number of filters in a
+ message is 32.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Filter List</td>
+ <td>A description of each filter. A filter description
+ appears in the next table.</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table border align=center cellpadding=4 witdh="80%">
+ <caption align=top>
+ <b>Filter Pipeline Message</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=2>Filter Identification</td>
+ <td colspan=2>Name Length</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=2>Flags</td>
+ <td colspan=2>Client Data Number of Values</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Name<br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Client Data<br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Padding</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Filter Identification</td>
+ <td>This is a unique (except in the case of testing)
+ identifier for the filter. Values from zero through 255
+ are reserved for filters defined by the NCSA HDF5
+ library. Values 256 through 511 have been set aside for
+ use when developing/testing new filters. The remaining
+ values are allocated to specific filters by contacting the
+ <a href="mailto:hdf5dev@ncsa.uiuc.edu">HDF5 Development
+ Team</a>.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Name Length</td>
+ <td>Each filter has an optional null-terminated ASCII name
+ and this field holds the length of the name including the
+ null termination padded with nulls to be a multiple of
+ eight. If the filter has no name then a value of zero is
+ stored in this field.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Flags</td>
+ <td>The flags indicate certain properties for a filter. The
+ bit values defined so far are:
+
+ <dl>
+ <dt><code>bit 1</code>
+ <dd>If set then the filter is an optional filter.
+ During output, if an optional filter fails it will be
+ silently removed from the pipeline.
+ </dl>
+ </tr>
+
+ <tr valign=top>
+ <td>Client Data Number of Values</td>
+ <td>Each filter can store a few integer values to control
+ how the filter operates. The number of entries in the
+ Client Data array is stored in this field.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Name</td>
+ <td>If the Name Length field is non-zero then it will
+ contain the size of this field, a multiple of eight. This
+ field contains a null-terminated, ASCII character
+ string to serve as a comment/name for the filter.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Client Data</td>
+ <td>This is an array of four-byte integers which will be
+ passed to the filter function. The Client Data Number of
+ Values determines the number of elements in the
+ array.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Padding</td>
+ <td>Four bytes of zeros are added to the message at this
+ point if the Client Data Number of Values field contains
+ an odd number.</td>
+ </tr>
+ </table>
+ </center>
+
+ <hr>
+ <h4><a name="AttributeMessage">Name: Attribute</a></h4>
+ <b>Type:</b> 0x000C<BR>
+ <b>Length:</b> varies<BR>
+ <b>Status:</b> Optional, may be repeated.<BR>
+
+ <p><b>Purpose and Description:</b> The <em>Attribute</em>
+ message is used to list objects in the HDF file which are used
+ as attributes, or "meta-data" about the current object. An
+ attribute is a small dataset; it has a name, a datatype, a data
+ space, and raw data. Since attributes are stored in the object
+ header they must be relatively small (<64kb) and can be
+ associated with any type of object which has an object header
+ (groups, datasets, named types and spaces, etc.).
+
+ <p>
+ <center>
+ <table border align=center cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Attribute Message</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td>Version</td>
+ <td>Reserved</td>
+ <td colspan=2>Name Size</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=2>Type Size</td>
+ <td colspan=2>Space Size</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Name<br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Type<br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Space<br><br></td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Data<br><br></td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Version</td>
+ <td>Version number for the message. This document describes
+ version 1 of attribute messages.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Reserved</td>
+ <td>This field is reserved for later use and is set to
+ zero.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Name Size</td>
+ <td>The length of the attribute name in bytes including the
+ null terminator. Note that the Name field below may
+ contain additional padding not represented by this
+ field.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Type Size</td>
+ <td>The length of the datatype description in the Type
+ field below. Note that the Type field may contain
+ additional padding not represented by this field.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Space Size</td>
+ <td>The length of the dataspace description in the Space
+ field below. Note that the Space field may contain
+ additional padding not represented by this field.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Name</td>
+ <td>The null-terminated attribute name. This field is
+ padded with additional null characters to make it a
+ multiple of eight bytes.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Type</td>
+ <td>The datatype description follows the same format as
+ described for the datatype object header message. This
+ field is padded with additional zero bytes to make it a
+ multiple of eight bytes.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Space</td>
+ <td>The dataspace description follows the same format as
+ described for the dataspace object header message. This
+ field is padded with additional zero bytes to make it a
+ multiple of eight bytes.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Data</td>
+ <td>The raw data for the attribute. The size is determined
+ from the datatype and dataspace descriptions. This
+ field is <em>not</em> padded with additional zero
+ bytes.</td>
+ </tr>
+ </table>
+ </center>
+
+ <hr>
+ <h4><a name="NameMessage">Name: Object Name</a></h4>
+
+ <p><b>Type:</b> 0x000D<br>
+ <b>Length:</b> varies<br>
+ <b>Status:</b> Optional, may not be repeated.
+
+ <p><b>Purpose and Description:</b> The object name or comment is
+ designed to be a short description of an object. An object name
+ is a sequence of non-zero (<code>\0</code>) ASCII characters with no other
+ formatting included by the library.
+
+ <p>
+ <center>
+ <table border align=center cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Name Message</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Name<br><br></td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Name</td>
+ <td>A null terminated ASCII character string.</td>
+ </tr>
+ </table>
+ </center>
+
+ <hr>
+ <h4><a name="ModifiedMessage">Name: Object Modification Date &amp; Time</a></h4>
+
+ <p><b>Type:</b> 0x000E<br>
+ <b>Length:</b> fixed<br>
+ <b>Status:</b> Optional, may not be repeated.
+
+ <p><b>Purpose and Description:</b> The object modification date
+ and time is a timestamp which indicates (using ISO-8601 date and
+ time format) the last modification of an object. The time is
+ updated when any object header message changes according to the
+ system clock where the change was posted.
+
+ <p>
+ <center>
+ <table border align=center cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Modification Time Message</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Year</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=2>Month</td>
+ <td colspan=2>Day of Month</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=2>Hour</td>
+ <td colspan=2>Minute</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=2>Second</td>
+ <td colspan=2>Reserved</td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Year</td>
+ <td>The four-digit year as an ASCII string. For example,
+ <code>1998</code>. All fields of this message should be interpreted
+ as coordinated universal time (UTC)</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Month</td>
+ <td>The month number as a two digit ASCII string where
+ January is <code>01</code> and December is <code>12</code>.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Day of Month</td>
+ <td>The day number within the month as a two digit ASCII
+ string. The first day of the month is <code>01</code>.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Hour</td>
+ <td>The hour of the day as a two digit ASCII string where
+ midnight is <code>00</code> and 11:00pm is <code>23</code>.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Minute</td>
+ <td>The minute of the hour as a two digit ASCII string where
+ the first minute of the hour is <code>00</code> and
+ the last is <code>59</code>.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Second</td>
+ <td>The second of the minute as a two digit ASCII string
+ where the first second of the minute is <code>00</code>
+ and the last is <code>59</code>.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Reserved</td>
+ <td>This field is reserved and should always be zero.</td>
+ </tr>
+ </table>
+ </center>
+
+ <hr>
+ <h4><a name="SharedMessage">Name: Shared Object Message</a></h4>
+ <b>Type:</b> 0x000F<br>
+ <b>Length:</b> 4 Bytes<br>
+ <b>Status:</b> Optional, may be repeated.
+
+ <p>A constant message can be shared among several object headers
+ by writing that message in the global heap and having the object
+ headers all point to it. The pointing is accomplished with a
+ Shared Object message which is understood directly by the object
+ header layer of the library. It is also possible to have a
+ message of one object header point to a message in some other
+ object header, but care must be exercised to prevent cycles.
+
+ <p>If a message is shared, then the message appears in the global
+ heap and its message ID appears in the Header Message Type
+ field of the object header. Also, the Flags field in the object
+ header for that message will have bit two set (the
+ <code>H5O_FLAG_SHARED</code> bit). The message body in the
+ object header will be that of a Shared Object message defined
+ here and not that of the pointed-to message.
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=top>
+ <b>Shared Message Message</b>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</td>
+ <th width="25%">byte</td>
+ <th width="25%">byte</td>
+ <th width="25%">byte</td>
+ </tr>
+
+ <tr align=center>
+ <td>Version</td>
+ <td>Flags</td>
+ <td colspan=2>Reserved</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4>Reserved</td>
+ </tr>
+
+ <tr align=center>
+ <td colspan=4><br>Pointer<br><br></td>
+ </tr>
+ </table>
+ </center>
+
+ <p>
+ <center>
+ <table align=center width="80%">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th width="70%">Description</th>
+ </tr>
+
+ <tr valign=top>
+ <td>Version</td>
+ <td>The version number for the message. This document
+ describes version one of shared messages.</td>
+ </tr>
+
+ <tr valign=top>
+ <td>Flags</td>
+ <td>The Shared Message message points to a message which is
+ shared among multiple object headers. The Flags field
+ describes the type of sharing:
+
+ <dl>
+ <dt><code>Bit 0</code>
+ <dd>If this bit is clear then the actual message is the
+ first message in some other object header; otherwise
+ the actual message is stored in the global heap.
+
+ <dt><code>Bits 2-7</code>
+ <dd>Reserved (always zero)
+ </dl>
+ </tr>
+
+ <tr valign=top>
+ <td>Pointer</td>
+ <td>This field points to the actual message. The format of
+ the pointer depends on the value of the Flags field. If
+ the actual message is in the global heap then the pointer
+ is the file address of the global heap collection that
+ holds the message, and a four-byte index into that
+ collection. Otherwise the pointer is a group entry
+ that points to some other object header.</td>
+ </tr>
+ </table>
+ </center>
+
+
+<hr>
+<h4><a name="ContinuationMessage">Name: Object Header Continuation</a></h4>
+<b>Type:</b> 0x0010<BR>
+<b>Length:</b> fixed<BR>
+<b>Status:</b> Optional, may be repeated.<BR>
+<b>Purpose and Description:</b> The object header continuation is the location
+in the file of more header messages for the current data object. This can be
+used when header blocks are large, or likely to change over time.<BR>
+<b>Format of Data:</b><p>
+ The object header continuation is formatted as follows (assuming a 4-byte
+length &amp; offset are being used in the current file):
+
+<P>
+<center>
+<table border cellpadding=4 width=60%>
+<caption align=bottom>
+<B>HDF5 Object Header Continuation Message Layout</B>
+</caption>
+
+<tr align=center>
+<th width=25%>byte</th>
+<th width=25%>byte</th>
+<th width=25%>byte</th>
+<th width=25%>byte</th>
+
+<tr align=center>
+<td colspan=4>Header Continuation Offset</td>
+<tr align=center>
+<td colspan=4>Header Continuation Length</td>
+</table>
+</center>
+
+<P>
+<dl>
+<dt>The elements of the Header Continuation Message are described below:
+<dd>
+<dl>
+<dt>Header Continuation Offset: (&lt;offset&gt; bytes)
+<dd>This value is the offset in bytes from the beginning of the file where the
+header continuation information is located.
+<dt>Header Continuation Length: (&lt;length&gt; bytes)
+<dd>This value is the length in bytes of the header continuation information in
+the file.
+</dl>
+</dl>
+
+<!-- Delete examples throughout doc
+<h4><a name="ContinuationExample">Examples:</a></h4>
+ [straightforward]
+-->
+
+<hr>
+<h4><a name="SymbolTableMessage">Name: Group Message</a></h4>
+<b>Type:</b> 0x0011<BR>
+<b>Length:</b> fixed<BR>
+<b>Status:</b> Required for groups, may not be repeated.<BR>
+<b>Purpose and Description:</b> Each group has a B-tree and a
+name heap which are pointed to by this message.<BR>
+<b>Format of data:</b>
+<p>The group message is formatted as follows:
+
+<p>
+<center>
+<table border cellpadding=4 width="80%">
+<caption align=bottom>
+<b>HDF5 Object Header Group Message Layout</b>
+</caption>
+
+<tr align=center>
+<th width="25%">byte</th>
+<th width="25%">byte</th>
+<th width="25%">byte</th>
+<th width="25%">byte</th>
+
+<tr align=center>
+<td colspan=4>B-tree Address</td>
+
+<tr align=center>
+<td colspan=4>Heap Address</td>
+</table>
+</center>
+
+<P>
+<dl>
+<dt>The elements of the Group Message are described below:
+<dd>
+<dl>
+<dt>B-tree Address (&lt;offset&gt; bytes)
+<dd>This value is the offset in bytes from the beginning of the file
+where the B-tree is located.
+<dt>Heap Address (&lt;offset&gt; bytes)
+<dd>This value is the offset in bytes from the beginning of the file
+where the group name heap is located.
+</dl>
+</dl>
+
+<h3><a name="SharedObjectHeader">Disk Format: Level 2b - Shared Data Object Headers</a></h3>
+<P>In order to share header messages between several dataset objects, object
+header messages may be placed into the global heap. Since these
+messages require additional information beyond the basic object header message
+information, the format of the shared message is detailed below.
+
+<BR> <BR>
+<center>
+<table border cellpadding=4 width=60%>
+<caption align=bottom>
+<B>HDF5 Shared Object Header Message</B>
+</caption>
+
+<tr align=center>
+<th width=25%>byte</th>
+<th width=25%>byte</th>
+<th width=25%>byte</th>
+<th width=25%>byte</th>
+
+<tr align=center>
+<td colspan=4>Reference Count of Shared Header Message</td>
+<tr align=center>
+<td colspan=4><br> Shared Object Header Message<br> <br></td>
+</table>
+</center>
+
+<p>
+<dl>
+<dt> The elements of the shared object header message are described below:
+<dd>
+<dl>
+<dt>Reference Count of Shared Header Message: (32-bit unsigned integer)
+<dd>This value is used to keep a count of the number of dataset objects which
+refer to this message from their dataset headers. When this count reaches zero,
+the shared message header may be removed from the global heap.
+<dt>Shared Object Header Message: (various lengths)
+<dd>The data stored for the shared object header message is formatted in the
+same way as the private object header messages described in the object header
+description earlier in this document and begins with the header message Type.
+</dl>
+</dl>
+
+
+<h3><a name="DataStorage">Disk Format: Level 2c - Data Object Data Storage</a></h3>
+<P>The data for an object is stored separately from the header
+information in the file and may not actually be located in the HDF5 file
+itself if the header indicates that the data is stored externally. The
+information for each record in the object is stored according to the
+dimensionality of the object (indicated in the dimensionality header message).
+Multi-dimensional data is stored in C order [same as current scheme], i.e. the
+"last" dimension changes fastest.
+<P>Data whose elements are composed of simple number-types are stored in
+native-endian IEEE format, unless they are specifically defined as being stored
+in a different machine format with the architecture-type information from the
+number-type header message. This means that each architecture will need to
+[potentially] byte-swap data values into the internal representation for that
+particular machine.
+<P> Data with a "variable" sized number-type is stored in a data heap
+internal to the HDF5 file. Global heap identifiers are stored in the
+data object storage.
+<P>Data whose elements are composed of pointer number-types are stored in several
+different ways depending on the particular pointer type involved. Simple
+pointers are just stored as the dataset offset of the object being pointed to with the
+size of the pointer being the same number of bytes as offsets in the file.
+Partial-object pointers are stored as a heap-ID which points to the following
+information within the file-heap: an offset of the object pointed to, number-type
+information (same format as header message), dimensionality information (same
+format as header message), sub-set start and end information (i.e. a coordinate
+location for each), and field start and end names (i.e. a [pointer to the]
+string indicating the first field included and a [pointer to the] string name
+for the last field).
+
+<P>Data of a compound datatype is stored as a contiguous stream of the items
+in the structure, with each item formatted according to its datatype.
+
+</body>
+</html>
diff --git a/doxygen/examples/H5.format.1.1.html b/doxygen/examples/H5.format.1.1.html
new file mode 100644
index 0000000..ebbbe8e
--- /dev/null
+++ b/doxygen/examples/H5.format.1.1.html
@@ -0,0 +1,6439 @@
+<html>
+ <head>
+ <title>
+ HDF5 File Format Specification Version 1.1
+ </title>
+
+<STYLE TYPE="text/css">
+
+P { text-indent: 2em}
+P.item { margin-left: 2em; text-indent: -2em}
+P.item2 { margin-left: 2em; text-indent: 2em}
+
+TABLE.format { border:solid; border-collapse:collapse; caption-side:top; text-align:center; width:80%;}
+TABLE.format TH { border:ridge; padding:4px; width:25%;}
+TABLE.format TD { border:ridge; padding:4px; }
+TABLE.format CAPTION { font-weight:bold; font-size:larger;}
+
+TABLE.note {border:none; text-align:right; width:80%;}
+
+TABLE.desc { border:solid; border-collapse:collapse; caption-size:top; text-align:left; width:80%;}
+TABLE.desc TR { vertical-align:top;}
+TABLE.desc TH { border-style:ridge; font-size:larger; padding:4px; text-decoration:underline;}
+TABLE.desc TD { border-style:ridge; padding:4px; }
+TABLE.desc CAPTION { font-weight:bold; font-size:larger;}
+
+TABLE.list { border:none; }
+TABLE.list TR { vertical-align:top;}
+TABLE.list TH { border:none; text-decoration:underline;}
+TABLE.list TD { border:none; }
+
+</STYLE>
+</head>
+ <body>
+
+ <center>
+ <table border=0 width=90%>
+ <tr>
+ <td valign=top>
+ <ol type=I>
+ <li><a href="#Intro">Introduction</a>
+ <li><a href="#FileMetaData">Disk Format Level 0 - File Metadata</a>
+ <font size=-2>
+ <ol type=A>
+ <li><a href="#SuperBlock">Disk Format Level 0A - File Signature and Super Block</a>
+ <li><a href="#DriverInfo">Disk Format Level 0B - File Driver Info</a>
+ </ol>
+ </font>
+ <li><a href="#FileInfra">Disk Format Level 1 - File Infrastructure</a>
+ <font size=-2>
+ <ol type=A>
+ <li><a href="#Btrees">Disk Format Level 1A - B-link Trees and B-tree Nodes</a>
+ <li><a href="#SymbolTable">Disk Format Level 1B - Group</a>
+ <li><a href="#SymbolTableEntry">Disk Format Level 1C - Group Entry</a>
+ <li><a href="#LocalHeap">Disk Format Level 1D - Local Heaps</a>
+ <li><a href="#GlobalHeap">Disk Format Level 1E - Global Heap</a>
+ <li><a href="#FreeSpaceIndex">Disk Format Level 1F - Free-space Index</a>
+ </ol>
+ </font>
+ <li><a href="#DataObject">Disk Format Level 2 - Data Objects</a>
+ <font size=-2>
+ <ol type=A>
+ <li><a href="#ObjectHeader">Disk Format Level 2a - Data Object Headers</a>
+ <ol type=1>
+ <li><a href="#NILMessage">Name: NIL</a> <!-- 0x0000 -->
+ <li><a href="#SimpleDataSpace">Name: Simple Dataspace</a> <!-- 0x0001 -->
+<!-- <li><a href="#DataSpaceMessage">Name: Complex Dataspace</a> --> <!-- 0x0002 -->
+ <li><a href="#ReservedMessage_0002">Name: Reserved - not assigned yet</a> <!-- 0x0002 -->
+ <li><a href="#DataTypeMessage">Name: Datatype</a> <!-- 0x0003 -->
+ <li><a href="#OldFillValueMessage">Name: Data Storage - Fill Value (Old)</a> <!-- 0x0004 -->
+ <li><a href="#FillValueMessage">Name: Data Storage - Fill Value</a> <!-- 0x0005 -->
+ </ol>
+ </ol>
+ </font>
+ </ol>
+ </td><td>&nbsp;&nbsp;</td><td valign=top>
+ <ol type=I start=4>
+
+ <li><a href="#DataObject">Disk Format Level 2 - Data Objects</a>
+ <font size=-2><i>(Continued)</i>
+ <ol type=A>
+ <li><a href="#ObjectHeader">Disk Format Level 2a - Data Object Headers</a><i>(Continued)</i>
+ <ol type=1 start=6>
+<!-- <li><a href="#CompactDataStorageMessage">Name: Data Storage - Compact</a> --> <!-- 0x0006 -->
+ <li><a href="#ReservedMessage_0006">Name: Reserved - not assigned yet</a> <!-- 0x0006 -->
+ <li><a href="#ExternalFileListMessage">Name: Data Storage - External Data Files</a> <!-- 0x0007 -->
+ <li><a href="#LayoutMessage">Name: Data Storage - Layout</a> <!-- 0x0008 -->
+ <li><a href="#ReservedMessage_0009">Name: Reserved - not assigned yet</a> <!-- 0x0009 -->
+ <li><a href="#ReservedMessage_000A">Name: Reserved - not assigned yet</a> <!-- 0x000a -->
+ <li><a href="#FilterMessage">Name: Data Storage - Filter Pipeline</a> <!-- 0x000b -->
+ <li><a href="#AttributeMessage">Name: Attribute</a> <!-- 0x000c -->
+ <li><a href="#CommentMessage">Name: Object Comment</a> <!-- 0x000d -->
+ <li><a href="#OldModifiedMessage">Name: Object Modification Date and Time (Old)</a> <!-- 0x000e -->
+ <li><a href="#SharedMessage">Name: Shared Object Message</a> <!-- 0x000f -->
+ <li><a href="#ContinuationMessage">Name: Object Header Continuation</a> <!-- 0x0010 -->
+ <li><a href="#SymbolTableMessage">Name: Group Message</a> <!-- 0x0011 -->
+ <li><a href="#ModifiedMessage">Name: Object Modification Date and Time</a> <!-- 0x0012 -->
+ </ol>
+ <li><a href="#DataStorage">Disk Format: Level 2b - Data Object Data Storage</a>
+ </ol>
+ </font>
+ <LI><A href="#Appendix">Appendix</A>
+ </ol>
+</td></tr>
+</table>
+</center>
+
+ <BR>
+ <HR>
+
+
+ <h2>Introduction</h2>
+
+ <table align=right width=100>
+ <tr><td>&nbsp;</td><td align=center>
+ <hr>
+ <img src="FF-IH_FileGroup.gif" alt="HDF5 Groups" hspace=15 vspace=15>
+ </td><td>&nbsp;</td></tr>
+ <tr><td>&nbsp;</td><td align=center>
+ <strong>Figure 1:</strong> Relationships among the HDF5 root group, other groups, and objects
+ <hr>
+ </td><td>&nbsp;</td></tr>
+
+ <tr><td>&nbsp;</td><td align=center>
+ <img src="FF-IH_FileObject.gif" alt="HDF5 Objects" hspace=15 vspace=15>
+ </td><td>&nbsp;</td></tr>
+ <tr><td>&nbsp;</td><td align=center>
+ <strong>Figure 2:</strong> HDF5 objects -- datasets, datatypes, or dataspaces
+ <hr>
+ </td><td>&nbsp;</td></tr>
+ </table>
+
+
+ <P>The format of an HDF5 file on disk encompasses several
+ key ideas of the HDF4 and AIO file formats as well as
+ addressing some shortcomings therein. The new format is
+ more self-describing than the HDF4 format and is more
+ uniformly applied to data objects in the file.
+
+ <P>An HDF5 file appears to the user as a directed graph.
+ The nodes of this graph are the higher-level HDF5 objects
+ that are exposed by the HDF5 APIs:
+
+ <ul>
+ <li>Groups
+ <li>Datasets
+ <li>Named datatypes
+ </ul>
+
+ <P>At the lowest level, as information is actually written to the disk,
+ an HDF5 file is made up of the following objects:
+ <ul>
+ <li>A super block
+ <li>B-tree nodes (containing either symbol nodes or raw data chunks)
+ <li>Object headers
+ <li>A global heap
+ <li>Local heaps
+ <li>Free space
+ </ul>
+
+ <P>The HDF5 library uses these low-level objects to represent the
+ higher-level objects that are then presented to the user or
+ to applications through the APIs.
+ For instance, a group is an object header that contains a message that
+ points to a local heap and to a B-tree which points to symbol nodes.
+ A dataset is an object header that contains messages that describe
+ datatype, space, layout, filters, external files, fill value, etc
+ with the layout message pointing to either a raw data chunk or to a
+ B-tree that points to raw data chunks.
+
+
+ <h3>This Document</h3>
+
+ <p>This document describes the lower-level data objects;
+ the higher-level objects and their properties are described
+ in the <a href="H5.user.html"><cite>HDF5 User's Guide</cite></a>.
+
+ <P>Three levels of information comprise the file format.
+ Level 0 contains basic information for identifying and
+ defining information about the file. Level 1 information contains
+ the information about the pieces of a file shared by many objects
+ in the file (such as a B-trees and heaps). Level 2 is the rest
+ of the file and contains all of the data objects, with each object
+ partitioned into header information, also known as
+ <em>metadata</em>, and data.
+
+ <p>The sizes of various fields in the following layout tables are
+ determined by looking at the number of columns the field spans
+ in the table. There are three exceptions: (1) The size may be
+ overridden by specifying a size in parentheses, (2) the size of
+ addresses is determined by the <em>Size of Offsets</em> field
+ in the super block and is indicated in this document with a
+ superscripted 'O', and (3) the size of length fields is determined
+ by the <em>Size of Lengths</em> field in the super block and is
+ indicated in this document with a superscripted 'L'.
+
+ <P>Values for all fields in this document should be treated as unsigned
+ integers, unless otherwise noted in the description of a field.
+ Additionally, all metadata fields are stored in little-endian byte
+ order.
+ </P>
+
+ <BR>
+ <HR>
+
+ <h2><a name="FileMetaData">
+ Disk Format: Level 0 - File Metadata</a></h2>
+
+ <H3><A name="SuperBlock">
+ Disk Format: Level 0A - File Signature and Super Block</A></H3>
+
+ <P>The super block may begin at certain predefined offsets within
+ the HDF5 file, allowing a block of unspecified content for
+ users to place additional information at the beginning (and
+ end) of the HDF5 file without limiting the HDF5 library's
+ ability to manage the objects within the file itself. This
+ feature was designed to accommodate wrapping an HDF5 file in
+ another file format or adding descriptive information to the
+ file without requiring the modification of the actual file's
+ information. The super block is located by searching for the
+ HDF5 file signature at byte offset 0, byte offset 512 and at
+ successive locations in the file, each a multiple of two of
+ the previous location, i.e. 0, 512, 1024, 2048, etc.
+
+ <P>The super block is composed of a file signature, followed by
+ super block and group version numbers, information
+ about the sizes of offset and length values used to describe
+ items within the file, the size of each group page,
+ and a group entry for the root object in the file.
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ HDF5 Super Block Layout
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>HDF5 File Signature (8 bytes)<br><br></td>
+ </tr>
+
+ <tr>
+ <td>Version # of Super Block</td>
+ <td>Version # of Global Free-space Storage</td>
+ <td>Version # of Root Group Symbol Table Entry</td>
+ <td>Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td>Version # of Shared Header Message Format</td>
+ <td>Size of Offsets</td>
+ <td>Size of Lengths</td>
+ <td>Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan=2>Group Leaf Node K</td>
+ <td colspan=2>Group Internal Node K</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>File Consistency Flags</td>
+ </tr>
+
+ <tr>
+ <td colspan=2 style="border:dotted;">Indexed Storage Internal Node K<sup>1</sup></td>
+ <td colspan=2 style="border:dotted;">Reserved (zero)<sup>1</sup></td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Base Address<sup>O</sup></td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Address of Global Free-space Heap<sup>O</sup></td>
+ </tr>
+
+ <tr>
+ <td colspan=4>End of File Address<sup>O</sup></td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Driver Information Block Address<sup>O</sup></td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Root Group Symbol Table Entry</td>
+ </tr>
+ </table>
+
+ <table class=note>
+ <tr><td>
+ (Items marked with an 'O' the above table are
+ <br>
+ of the size specified in "Size of Offsets.")
+ </td></tr>
+ <tr><td>
+ (Items marked with an '1' the above table are
+ <br>
+ new in version 1 of the superblock)
+ </td></tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>HDF5 File Signature</td>
+ <td>
+ <P>This field contains a constant value and can be used to
+ quickly identify a file as being an HDF5 file. The
+ constant value is designed to allow easy identification of
+ an HDF5 file and to allow certain types of data corruption
+ to be detected. The file signature of an HDF5 file always
+ contains the following values:
+ </P>
+
+ <center>
+ <table border align=center cellpadding=4>
+ <tr align=center>
+ <td align=right>Decimal:</td>
+ <td width="8%">137</td>
+ <td width="8%">72</td>
+ <td width="8%">68</td>
+ <td width="8%">70</td>
+ <td width="8%">13</td>
+ <td width="8%">10</td>
+ <td width="8%">26</td>
+ <td width="8%">10</td>
+ </tr>
+
+ <tr align=center>
+ <td align=right>Hexadecimal:</td>
+ <td>89</td>
+ <td>48</td>
+ <td>44</td>
+ <td>46</td>
+ <td>0d</td>
+ <td>0a</td>
+ <td>1a</td>
+ <td>0a</td>
+ </tr>
+
+ <tr align=center>
+ <td align=right>ASCII C Notation:</td>
+ <td>\211</td>
+ <td>H</td>
+ <td>D</td>
+ <td>F</td>
+ <td>\r</td>
+ <td>\n</td>
+ <td>\032</td>
+ <td>\n</td>
+ </tr>
+ </table>
+ </center>
+ <br>
+
+ <P>This signature both identifies the file as an HDF5 file
+ and provides for immediate detection of common
+ file-transfer problems. The first two bytes distinguish
+ HDF5 files on systems that expect the first two bytes to
+ identify the file type uniquely. The first byte is
+ chosen as a non-ASCII value to reduce the probability
+ that a text file may be misrecognized as an HDF5 file;
+ also, it catches bad file transfers that clear bit
+ 7. Bytes two through four name the format. The CR-LF
+ sequence catches bad file transfers that alter newline
+ sequences. The control-Z character stops file display
+ under MS-DOS. The final line feed checks for the inverse
+ of the CR-LF translation problem. (This is a direct
+ descendent of the <A href="http://www.libpng.org/pub/png/spec/PNG-Rationale.html#R.PNG-file-signature">PNG</A> file
+ signature.)
+ </P>
+
+ <P><EM>This field is present in version 0+ of the superblock.</EM>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Version Number of the Super Block</td>
+ <td>
+ <P>This value is used to determine the format of the
+ information in the super block. When the format of the
+ information in the super block is changed, the version number
+ is incremented to the next integer and can be used to
+ determine how the information in the super block is
+ formatted.
+ </P>
+
+ <P>Values of 0 and 1 are defined for this field.
+ </P>
+
+ <P><EM>This field is present in version 0+ of the superblock.</EM>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Version Number of the File Free-space Information</td>
+ <td>
+ <P>This value is used to determine the format of the
+ information in the File Free-space Information.
+ </P>
+ <P>The only value currently valid in this field is '0', which
+ indicates that the free space index is formatted as described
+ <A href="#FreeSpaceIndex">below</A>.
+ </P>
+
+ <P><EM>This field is present in version 0+ of the superblock.</EM>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Version Number of the Root Group Symbol Table Entry</td>
+ <td>
+ <P>This value is used to determine the format of the
+ information in the Root Group Symbol Table Entry. When the
+ format of the information in that field is changed, the
+ version number is incremented to the next integer and can be
+ used to determine how the information in the field
+ is formatted.
+ </P>
+ <P>The only value currently valid in this field is '0', which
+ indicates that the root group symbol table entry is formatted as
+ described <A href="#SymbolTableEntry">below</A>.
+ </P>
+
+ <P><EM>This field is present in version 0+ of the superblock.</EM>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Version Number of the Shared Header Message Format</td>
+ <td>
+ <P>This value is used to determine the format of the
+ information in a shared object header message. Since the format
+ of the shared header messages differs from the other private
+ header messages, a version number is used to identify changes
+ in the format.
+ </P>
+ <P>The only value currently valid in this field is '0', which
+ indicates that shared header messages are formatted as
+ described <A href="#SharedMessage">below</A>.
+ </P>
+
+ <P><EM>This field is present in version 0+ of the superblock.</EM>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Size of Offsets</td>
+ <td>
+ <P>This value contains the number of bytes used to store
+ addresses in the file. The values for the addresses of
+ objects in the file are offsets relative to a base address,
+ usually the address of the super block signature. This
+ allows a wrapper to be added after the file is created
+ without invalidating the internal offset locations.
+ </P>
+
+ <P><EM>This field is present in version 0+ of the superblock.</EM>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Size of Lengths</td>
+ <td>
+ <P>This value contains the number of bytes used to store
+ the size of an object.
+ </P>
+
+ <P><EM>This field is present in version 0+ of the superblock.</EM>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Group Leaf Node K</td>
+ <td>
+ <P>Each leaf node of a group B-tree will have at
+ least this many entries but not more than twice this
+ many. If a group has a single leaf node then it
+ may have fewer entries.
+ </P>
+ <P>This value must be greater than zero.
+ </P>
+ <P>See the <A href="#Btrees">description</A> of B-trees below.
+ </P>
+
+ <P><EM>This field is present in version 0+ of the superblock.</EM>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Group Internal Node K</td>
+ <td>
+ <P>Each internal node of a group B-tree will have at
+ least this many entries but not more than twice this
+ many. If the group has only one internal
+ node then it might have fewer entries.
+ </P>
+ <P>This value must be greater than zero.
+ </P>
+ <P>See the <A href="#Btrees">description</A> of B-trees below.
+ </P>
+
+ <P><EM>This field is present in version 0+ of the superblock.</EM>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>File Consistency Flags</td>
+ <td>
+ <P>This value contains flags to indicate information
+ about the consistency of the information contained
+ within the file. Currently, the following bit flags are
+ defined:
+ <ul>
+ <li>Bit 0 set indicates that the file is opened for
+ write-access.
+ <li>Bit 1 set indicates that the file has
+ been verified for consistency and is guaranteed to be
+ consistent with the format defined in this document.
+ <li>Bits 2-31 are reserved for future use.
+ </ul>
+ Bit 0 should be
+ set as the first action when a file is opened for write
+ access and should be cleared only as the final action
+ when closing a file. Bit 1 should be cleared during
+ normal access to a file and only set after the file's
+ consistency is guaranteed by the library or a
+ consistency utility.
+ </P>
+
+ <P><EM>This field is present in version 0+ of the superblock.</EM>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Indexed Storage Internal Node K</td>
+ <td>
+ <P>Each internal node of a indexed storage B-tree will have at
+ least this many entries but not more than twice this
+ many. If the group has only one internal
+ node then it might have fewer entries.
+ </P>
+ <P>This value must be greater than zero.
+ </P>
+ <P>See the <A href="#Btrees">description</A> of B-trees below.
+ </P>
+
+ <P><EM>This field is present in version 1+ of the superblock.</EM>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Base Address</td>
+ <td>
+ <P>This is the absolute file address of the first byte of
+ the HDF5 data within the file. The library currently
+ constrains this value to be the absolute file address
+ of the super block itself when creating new files;
+ future versions of the library may provide greater
+ flexibility. When opening an existing file and this address does
+ not match the offset of the superblock, the library assumes
+ that the entire contents of the HDF5 file have been adjusted in
+ the file and adjusts the base address and end of file address to
+ reflect their new positions in the file. Unless otherwise noted,
+ all other file addresses are relative to this base
+ address.
+ </P>
+
+ <P><EM>This field is present in version 0+ of the superblock.</EM>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Address of Global Free-space Index</td>
+ <td>
+ <P>Free-space management is not yet defined in the HDF5
+ file format and is not handled by the library.
+ Currently this field always contains the
+ <A href="#UndefinedAddress">undefined address</A>.
+ </P>
+
+ <P><EM>This field is present in version 0+ of the superblock.</EM>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>End of File Address</td>
+ <td>
+ <P>This is the absolute file address of the first byte past
+ the end of all HDF5 data. It is used to determine whether a
+ file has been accidently truncated and as an address where
+ file data allocation can occur if space from the free list is
+ not used.
+ </P>
+
+ <P><EM>This field is present in version 0+ of the superblock.</EM>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Driver Information Block Address</td>
+ <td>
+ <P>This is the relative file address of the file driver
+ information block which contains driver-specific
+ information needed to reopen the file. If there is no
+ driver information block then this entry should be the
+ <A href="#UndefinedAddress">undefined address</A>.
+ </P>
+
+ <P><EM>This field is present in version 0+ of the superblock.</EM>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Root Group Symbol Table Entry</td>
+ <td>
+ <P>This is the <A href="#SymbolTableEntry">symbol table entry</A>
+ of the root group, which serves as the entry point into
+ the group graph for the file.
+ </P>
+
+ <P><EM>This field is present in version 0+ of the superblock.</EM>
+ </P>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <H3><A name="DriverInfo">
+ Disk Format: Level 0B - File Driver Info</A></H3>
+
+ <p>The <em>file driver information block</em> is an optional region of the
+ file which contains information needed by the file driver in
+ order to reopen a file. The format of the file driver information
+ block is:
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Driver Information Block
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan=3>Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Driver Information Size (4 bytes)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Driver Identification (8 bytes)<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br><br>Driver Information (<em>n</em> bytes)<br><br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>
+ <P>The version number of the driver information block. The
+ file format documented here is version zero.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Driver Information Size</td>
+ <td>
+ <P>The size in bytes of the Driver Information part of this
+ structure.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Driver Identification</td>
+ <td>
+ <P>This is an eight-byte ASCII string without null
+ termination which identifies the driver and version number
+ of the Driver Information block. The predefined drivers
+ supplied with the HDF5 library are identified by the
+ letters <code>NCSA</code> followed by the first four characters of
+ the driver name. If the Driver Information block is not
+ the original version then the last letter(s) of the
+ identification will be replaced by a version number in
+ ASCII.
+ </P>
+ <P>
+ For example, the various versions of the <em>multi driver</em>
+ will be identified by <code>NCSAmult</code>.
+ (<code>NCSAmult</code> is simply <code>NCSAmulti</code> truncated
+ to eight characters. Subsequent identifiers will be created by
+ substituting sequential numerical values for the final character,
+ starting with zero.) <em>multi driver</em> is the only default driver that
+ is encoded in this field.
+ </P>
+ <P>
+ Identification for user-defined drivers
+ is eight-byte long and arbitrary but should be unique and avoid
+ the four character prefix "NCSA".
+ </P>
+ </td>
+ </tr>
+
+ <tr valign=top>
+ <td>Driver Information</td>
+ <td>Driver information is encoded/decoded in a format defined by the
+ file driver. <em>multi driver</em> is the only default driver that has driver
+ information stored in this field. Its format is explained in the
+ following block.</td>
+ </tr>
+ </table>
+ </div>
+
+ <BR>
+ <P><em>Multi driver</em> has the following format:</P>
+
+ <div align=center>
+ <table class=format>
+ <caption>
+ Multi Driver Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Member Mapping</td>
+ <td>Member Mapping</td>
+ <td>Member Mapping</td>
+ <td>Member Mapping</td>
+ </tr>
+
+ <tr>
+ <td>Member Mapping</td>
+ <td>Member Mapping</td>
+ <td>Reserved</td>
+ <td>Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Address of Member File 1<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>End of Address for Member File 1<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Address of Member File 2<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>End of Address for Member File 2<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>... ...<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Name of Member File 1<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Name of Member File 2<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>... ...<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Member Mapping</td>
+ <td><P><em>Multi driver</em> enables different types of HDF5 data and
+ metadata to be written to separate files. These files are viewed by the
+ library as a single virtual HDF5 file with a single file address.
+ It allows maximal 6 files to be created.
+ In sequence, these <em>Member Mapping</em> fields are for super block,
+ B-tree, raw data, global heap, local heap,
+ and object header. More than one type of data can be written to the
+ same file.</P>
+ <P>These <em>Member Mapping</em> fields are integer values from 1 to 6
+ indicating how the data can be mapped to or merged with another type of
+ data.
+ <table class=list>
+ <tr>
+ <th width="30%">Member Mapping</th>
+ <th align=left>Description</th>
+ </tr>
+ <tr>
+ <td align=center>1</td>
+ <td>The super block data.</td>
+ </tr>
+ <tr>
+ <td align=center>2</td>
+ <td>The B-tree data.</td>
+ </tr>
+ <tr>
+ <td align=center>3</td>
+ <td>The raw data.</td>
+ </tr>
+ <tr>
+ <td align=center>4</td>
+ <td>The global heap data.</td>
+ </tr>
+ <tr>
+ <td align=center>5</td>
+ <td>The local heap data.</td>
+ </tr>
+ <tr>
+ <td align=center>6</td>
+ <td>The object header data.</td>
+ </tr>
+ </table></P>
+ For example, if the third field has the value 3 and all the rest have the
+ value 1, it means there are two files, one for raw data, one for super block,
+ B-tree, global heap, local heap, and object header.
+ </td>
+ </tr>
+
+ <tr>
+ <td>Reserved</td>
+ <td><P>These fields are reserved and should always be zero.</P></td>
+ </tr>
+
+ <tr>
+ <td>Address of Member File</td>
+ <td><P>Specifies the virtual address. A normally eight-byte integer with
+ the value from <em>0</em> (zero) to maximal value,
+ at which the member file starts.</P></td>
+ </tr>
+
+ <tr>
+ <td>End of Address for Member File</td>
+ <td><P>The end of allocated address for the member file. A normally eight-byte
+ integer value.</P></td>
+ </tr>
+
+ <tr>
+ <td>Name of Member File</td>
+ <td><P>The null-terminated name of member file. Its length should be multiples of
+ 8 bytes. Additional bytes will be padded with <em>NULL</em>s. The default naming
+ convention is <em>%%s-X.h5</em>, where <em>X</em> is one of the letters
+ <em>s</em> (for super block), <em>b</em> (for B-tree), <em>r</em> (for raw data),
+ <em>g</em> (for global heap), <em>l</em> (for local heap), and <em>o</em> (for
+ object header). The name for the whole HDF5 file will substitute the <em>%s</em>
+ in the string.
+ </P>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <BR>
+ <HR>
+
+ <h2><a name="FileInfra">
+ Disk Format: Level 1 - File Infrastructure</a></h2>
+ <h3><a name="Btrees">Disk Format: Level 1A - B-link Trees and B-tree Nodes</a></h3>
+
+ <p>B-link trees allow flexible storage for objects which tend to grow
+ in ways that cause the object to be stored discontiguously. B-trees
+ are described in various algorithms books including "Introduction to
+ Algorithms" by Thomas H. Cormen, Charles E. Leiserson, and Ronald
+ L. Rivest. The B-link tree, in which the sibling nodes at a
+ particular level in the tree are stored in a doubly-linked list,
+ is described in the "Efficient Locking for Concurrent Operations
+ on B-trees" paper by Phillip Lehman and S. Bing Yao as published
+ in the <cite>ACM Transactions on Database Systems</cite>, Vol. 6,
+ No. 4, December 1981.
+
+ <p>The B-link trees implemented by the file format contain one more
+ key than the number of children. In other words, each child
+ pointer out of a B-tree node has a left key and a right key.
+ The pointers out of internal nodes point to sub-trees while
+ the pointers out of leaf nodes point to symbol nodes and
+ raw data chunks.
+ Aside from that difference, internal nodes and leaf nodes
+ are identical.
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ B-tree Nodes
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+
+ <tr>
+ <td colspan=4>Signature</td>
+
+ <tr>
+ <td>Node Type</td>
+ <td>Node Level</td>
+ <td colspan=2>Entries Used</td>
+
+ <tr>
+ <td colspan=4>Address of Left Sibling<sup>O</sup></td>
+
+ <tr>
+ <td colspan=4>Address of Right Sibling<sup>O</sup></td>
+
+ <tr>
+ <td colspan=4>Key 0 (variable size)</td>
+
+ <tr>
+ <td colspan=4>Address of Child 0<sup>O</sup></td>
+
+ <tr>
+ <td colspan=4>Key 1 (variable size)</td>
+
+ <tr>
+ <td colspan=4>Address of Child 1<sup>O</sup></td>
+
+ <tr>
+ <td colspan=4>...</td>
+
+ <tr>
+ <td colspan=4>Key 2<em>K</em> (variable size)</td>
+
+ <tr>
+ <td colspan=4>Address of Child 2<em>K</em><sup>O</sup></td>
+
+ <tr>
+ <td colspan=4>Key 2<em>K</em>+1 (variable size)</td>
+ </table>
+
+ <table class=note>
+ <tr><td>
+ (Items marked with an 'O' the above table are
+ <br>
+ of the size specified in "Size of Offsets.")
+ </td></tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Signature</td>
+ <td>
+ <P>The ASCII character string "<code>TREE</code>" is
+ used to indicate the
+ beginning of a B-link tree node. This gives file
+ consistency checking utilities a better chance of
+ reconstructing a damaged file.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Node Type</td>
+ <td>
+ <P>Each B-link tree points to a particular type of data.
+ This field indicates the type of data as well as
+ implying the maximum degree <em>K</em> of the tree and
+ the size of each Key field.
+ </P>
+
+ <table class=list>
+ <tr>
+ <th width="30%">Node Type</th>
+ <th align=left>Description</th>
+ </tr>
+ <tr>
+ <td align=center>0</td>
+ <td>This tree points to group nodes.</td>
+ </tr>
+ <tr>
+ <td align=center>1</td>
+ <td>This tree points to raw data chunk nodes.</td>
+ </tr>
+ </table>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Node Level</td>
+ <td>
+ <P>The node level indicates the level at which this node
+ appears in the tree (leaf nodes are at level zero). Not
+ only does the level indicate whether child pointers
+ point to sub-trees or to data, but it can also be used
+ to help file consistency checking utilities reconstruct
+ damanged trees.
+ </P>
+ </td>
+ </tr>
+
+ <tr valign=top>
+ <td>Entries Used</td>
+ <td>
+ <P>This determines the number of children to which this
+ node points. All nodes of a particular type of tree
+ have the same maximum degree, but most nodes will point
+ to less than that number of children. The valid child
+ pointers and keys appear at the beginning of the node
+ and the unused pointers and keys appear at the end of
+ the node. The unused pointers and keys have undefined
+ values.
+ </P>
+ </td>
+ </tr>
+
+ <tr valign=top>
+ <td>Address of Left Sibling</td>
+ <td>
+ <P>This is the relative file address of the left sibling of
+ the current node. If the current
+ node is the left-most node at this level then this field
+ is the <A href="#UndefinedAddress">undefined address</A>.
+ </P>
+ </td>
+ </tr>
+
+ <tr valign=top>
+ <td>Address of Right Sibling</td>
+ <td>
+ <P>This is the relative file address of the right sibling of
+ the current node. If the current
+ node is the right-most node at this level then this
+ field is the <A href="#UndefinedAddress">undefined address</A>.
+ </P>
+ </td>
+ </tr>
+
+ <tr valign=top>
+ <td>Keys and Child Pointers</td>
+ <td>
+ <P>Each tree has 2<em>K</em>+1 keys with 2<em>K</em>
+ child pointers interleaved between the keys. The number
+ of keys and child pointers actually containing valid
+ values is determined by the node's <em>Entries Used</em> field.
+ If that field is <em>N</em> then the B-link tree contains
+ <em>N</em> child pointers and <em>N</em>+1 keys.
+ </P>
+ </td>
+ </tr>
+
+ <tr valign=top>
+ <td>Key</td>
+ <td>
+ <P>The format and size of the key values is determined by
+ the type of data to which this tree points. The keys are
+ ordered and are boundaries for the contents of the child
+ pointer; that is, the key values represented by child
+ <em>N</em> fall between Key <em>N</em> and Key
+ <em>N</em>+1. Whether the interval is open or closed on
+ each end is determined by the type of data to which the
+ tree points.
+ </P>
+
+ <P>
+ The format of the key depends on the node type.
+ For nodes of node type 0 (group nodes), the key is formatted as
+ follows:
+ <center>
+ <table class=list>
+ <tr>
+ <td width=30%>A single field of <i>Size of Lengths</i>
+ bytes:</td>
+ <td>Indicates the byte offset into the local heap
+ for the first object name in the subtree which
+ that key describes.
+ </td>
+ </tr>
+ </table>
+ </center>
+ </P>
+
+ <P>
+ For nodes of node type 1 (chunked raw data nodes), the key is
+ formatted as follows:
+ <center>
+ <table class=list>
+ <tr>
+ <td width=30%>Bytes 1-4:</td>
+ <td>Size of chunk in bytes.</td>
+ </tr>
+ <tr>
+ <td>Bytes 4-8:</td>
+ <td>Filter mask, a 32-bit bitfield indicating which
+ filters have been skipped for this chunk. Each filter
+ has an index number in the pipeline (starting at 0, with
+ the first filter to apply) and if that filter is skipped,
+ the bit corresponding to it's index is set.</td>
+ </tr>
+ <tr>
+ <td><em>N</em> 64-bit fields:</td>
+ <td>A 64-bit index indicating the offset of the
+ chunk within the dataset where <i>N</i> is the number
+ of dimensions of the dataset. For example, if
+ a chunk in a 3-dimensional dataset begins at the
+ position <code>[5,5,5]</code>, there will be three
+ such 64-bit indices, each with the value of
+ <code>5</code>.</td>
+ </tr>
+ </table>
+ </center>
+ </P>
+ </td>
+ </tr>
+
+ <tr valign=top>
+ <td>Child Pointer</td>
+ <td>
+ <P>The tree node contains file addresses of subtrees or
+ data depending on the node level. Nodes at Level 0 point
+ to data addresses, either raw data chunk or group nodes.
+ Nodes at non-zero levels point to other nodes of the
+ same B-tree.
+ </P>
+ <P>For raw data chunk nodes, the child pointer is the address
+ of a single raw data chunk. For group nodes, the child pointer
+ points to a <A href="#SymbolTable">symbol table</A>, which contains
+ information for multiple symbol table entries.
+ </P>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <p>
+ Conceptually, each B-tree node looks like this:
+ <center>
+ <table>
+ <tr valign=top align=center>
+ <td>key[0]</td><td>&nbsp;</td>
+ <td>child[0]</td><td>&nbsp;</td>
+ <td>key[1]</td><td>&nbsp;</td>
+ <td>child[1]</td><td>&nbsp;</td>
+ <td>key[2]</td><td>&nbsp;</td>
+ <td>...</td><td>&nbsp;</td>
+ <td>...</td><td>&nbsp;</td>
+ <td>key[<i>N</i>-1]</td><td>&nbsp;</td>
+ <td>child[<i>N</i>-1]</td><td>&nbsp;</td>
+ <td>key[<i>N</i>]</td>
+ </tr>
+ </table>
+ </center>
+ <br>
+
+ where child[<i>i</i>] is a pointer to a sub-tree (at a level
+ above Level 0) or to data (at Level 0).
+ Each key[<i>i</i>] describes an <i>item</i> stored by the B-tree
+ (a chunk or an object of a group node). The range of values
+ represented by child[<i>i</i>] is indicated by key[<i>i</i>]
+ and key[<i>i</i>+1].
+
+
+ <p>The following question must next be answered:
+ "Is the value described by key[<i>i</i>] contained in
+ child[<i>i</i>-1] or in child[<i>i</i>]?"
+ The answer depends on the type of tree.
+ In trees for groups (node type 0) the object described by
+ key[<i>i</i>] is the greatest object contained in
+ child[<i>i</i>-1] while in chunk trees (node type 1) the
+ chunk described by key[<i>i</i>] is the least chunk in
+ child[<i>i</i>].
+
+ <p>That means that key[0] for group trees is sometimes unused;
+ it points to offset zero in the heap, which is always the
+ empty string and compares as "less-than" any valid object name.
+
+ <p>And key[<i>N</i>] for chunk trees is sometimes unused;
+ it contains a chunk offset which compares as "greater-than"
+ any other chunk offset and has a chunk byte size of zero
+ to indicate that it is not actually allocated.
+
+
+ <h3><a name="SymbolTable">Disk Format: Level 1B - Group and Symbol Nodes</a></h3>
+
+ <p>A group is an object internal to the file that allows
+ arbitrary nesting of objects within the file (including other groups).
+ A group maps a set of names in the group to a set of relative
+ file addresses where objects with those names are located in
+ the file. Certain metadata for an object to which the group points
+ can be cached in the group's symbol table in addition to the
+ object's header.
+
+ <p>An HDF5 object name space can be stored hierarchically by
+ partitioning the name into components and storing each
+ component in a group. The group entry for a
+ non-ultimate component points to the group containing
+ the next component. The group entry for the last
+ component points to the object being named.
+
+ <p>A group is a collection of group nodes pointed
+ to by a B-link tree. Each group node contains entries
+ for one or more symbols. If an attempt is made to add a
+ symbol to an already full group node containing
+ 2<em>K</em> entries, then the node is split and one node
+ contains <em>K</em> symbols and the other contains
+ <em>K</em>+1 symbols.
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Group Node (A Leaf of a B-tree)
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+
+ <tr>
+ <td colspan=4>Signature</td>
+
+ <tr>
+ <td>Version Number</td>
+ <td>Reserved (0)</td>
+ <td colspan=2>Number of Symbols</td>
+
+ <tr>
+ <td colspan=4><br><br>Group Entries<br><br><br></td>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Signature</td>
+ <td>
+ <P>The ASCII character string "<code>SNOD</code>" is
+ used to indicate the
+ beginning of a group node. This gives file
+ consistency checking utilities a better chance of
+ reconstructing a damaged file.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Version Number</td>
+ <td>
+ <P>The version number for the group node. This
+ document describes version 1. (There is no version '0'
+ of the group node)
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Number of Symbols</td>
+ <td>
+ <P>Although all group nodes have the same length,
+ most contain fewer than the maximum possible number of
+ symbol entries. This field indicates how many entries
+ contain valid data. The valid entries are packed at the
+ beginning of the group node while the remaining
+ entries contain undefined values.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Group Entries</td>
+ <td>
+ <P>Each symbol has an entry in the group node.
+ The format of the entry is described below.
+ There are 2<EM>K</EM> entries in each group node, where
+ <EM>K</EM> is the "Group Leaf Node K" value from the
+ <A href="#SuperBlock">super block</A>.
+ </P>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <h3><a name="SymbolTableEntry">
+ Disk Format: Level 1C - Group Entry </a></h3>
+
+ <p>Each group entry in a group node is designed
+ to allow for very fast browsing of stored objects.
+ Toward that design goal, the group entries
+ include space for caching certain constant metadata from the
+ object header.
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Group Entry
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=4>Name Offset<sup>O</sup></td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Object Header Address<sup>O</sup></td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Cache Type</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br><br>Scratch-pad Space (16 bytes)<br><br><br></td>
+ </tr>
+ </table>
+
+ <table class=note>
+ <tr><td>
+ (Items marked with an 'O' the above table are
+ <br>
+ of the size specified in "Size of Offsets.")
+ </td></tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Name Offset</td>
+ <td>
+ <P>This is the byte offset into the group local
+ heap for the name of the object. The name is null
+ terminated.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Object Header Address</td>
+ <td>
+ <P>Every object has an object header which serves as a
+ permanent location for the object's metadata. In addition
+ to appearing in the object header, some metadata can be
+ cached in the scratch-pad space.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Cache Type</td>
+ <td>
+ <P>The cache type is determined from the object header.
+ It also determines the format for the scratch-pad space:
+ <br>
+ <table class=list>
+ <tr align=left>
+ <th>Type:</th>
+ <th>Description:</th>
+ </tr>
+ <tr>
+ <td width="10%" align=center>0</td>
+ <td>No data is cached by the group entry. This
+ is guaranteed to be the case when an object header
+ has a link count greater than one.
+ </td>
+ </tr>
+ <tr>
+ <td align=center>1</td>
+ <td>Object header metadata is cached in the group
+ entry. This implies that the group
+ entry refers to another group.
+ </td>
+ </tr>
+ <tr>
+ <td align=center>2</td>
+ <td>The entry is a symbolic link. The first four bytes
+ of the scratch-pad space are the offset into the local
+ heap for the link value. The object header address
+ will be undefined.
+ </td>
+ </tr>
+ <tr>
+ <td align=center><em>N</em></td>
+ <td>Other cache values can be defined later and
+ libraries that do not understand the new values will
+ still work properly.
+ </td>
+ </tr>
+ </table>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Reserved</td>
+ <td>
+ <P>These four bytes are present so that the scratch-pad
+ space is aligned on an eight-byte boundary. They are
+ always set to zero.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Scratch-pad Space</td>
+ <td>
+ <P>This space is used for different purposes, depending
+ on the value of the Cache Type field. Any metadata
+ about a dataset object represented in the scratch-pad
+ space is duplicated in the object header for that
+ dataset. This metadata can include the datatype
+ and the size of the dataspace for a dataset whose datatype
+ is atomic and whose dataspace is fixed and less than
+ four dimensions.
+ </P>
+ <P>
+ Furthermore, no data is cached in the group
+ entry scratch-pad space if the object header for
+ the group entry has a link count greater than
+ one.
+ </P>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <h4>Format of the Scratch-pad Space</h4>
+
+ <p>The group entry scratch-pad space is formatted
+ according to the value in the Cache Type field.
+
+ <p>If the Cache Type field contains the value zero
+ <code>(0)</code> then no information is
+ stored in the scratch-pad space.
+
+ <p>If the Cache Type field contains the value one
+ <code>(1)</code>, then the scratch-pad space
+ contains cached metadata for another object header
+ in the following format:
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Object Header Scratch-pad Format
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+
+ <tr>
+ <td colspan=4>Address of B-tree<sup>O</sup></td>
+
+ <tr>
+ <td colspan=4>Address of Name Heap<sup>O</sup></td>
+ </table>
+
+ <table class=note>
+ <tr><td>
+ (Items marked with an 'O' the above table are
+ <br>
+ of the size specified in "Size of Offsets.")
+ </td></tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Address of B-tree</td>
+ <td>
+ <P>This is the file address for the root of the
+ group's B-tree.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Address of Name Heap</td>
+ <td>
+ <P>This is the file address for the group's local
+ heap, in which are stored the group's symbol names.
+ </P>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+ <P>If the Cache Type field contains the value two
+ <code>(2)</code>, then the scratch-pad space
+ contains cached metadata for another symbolic link
+ in the following format:
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Symbolic Link Scratch-pad Format
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=4>Offset to Link Value</td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Offset to Link Value</td>
+ <td>
+ <P>The value of a symbolic link (that is, the name of the
+ thing to which it points) is stored in the local heap.
+ This field is the 4-byte offset into the local heap for
+ the start of the link value, which is null terminated.
+ </P>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <h3><a name="LocalHeap">Disk Format: Level 1D - Local Heaps</a></h3>
+
+ <P>A heap is a collection of small heap objects. Objects can be
+ inserted and removed from the heap at any time.
+ The address of a heap does not change once the heap is created.
+ References to objects are stored in the group table;
+ the names of those objects are stored in the local heap.
+ </P>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Local Heap
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=4>Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan=3>Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Data Segment Size<sup>L</sup></td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Offset to Head of Free-list<sup>L</sup></td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Address of Data Segment<sup>O</sup></td>
+ </tr>
+ </table>
+
+ <table class=note>
+ <tr><td>
+ (Items marked with an 'L' the above table are
+ <br>
+ of the size specified in "Size of Lengths.")
+ </td></tr>
+ <tr><td>
+ (Items marked with an 'O' the above table are
+ <br>
+ of the size specified in "Size of Offsets.")
+ </td></tr>
+ </table>
+ </div>
+
+ <p>
+ <center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Signature</td>
+ <td>
+ <P>The ASCII character string "<code>HEAP</code>"
+ is used to indicate the
+ beginning of a heap. This gives file consistency
+ checking utilities a better chance of reconstructing a
+ damaged file.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>
+ <P>Each local heap has its own version number so that new
+ heaps can be added to old files. This document
+ describes version zero (0) of the local heap.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Data Segment Size</td>
+ <td>
+ <P>The total amount of disk memory allocated for the heap
+ data. This may be larger than the amount of space
+ required by the objects stored in the heap. The extra
+ unused space in the heap holds a linked list of free blocks.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Offset to Head of Free-list</td>
+ <td>
+ <P>This is the offset within the heap data segment of the
+ first free block (or the
+ <A href="#UndefinedAddress">undefined address</A> if there is no
+ free block). The free block contains "Size of Lengths" bytes that
+ are the offset of the next free block (or the
+ value '1' if this is the
+ last free block) followed by "Size of Lengths" bytes that store
+ the size of this free block. The size of the free block includes
+ the space used to store the offset of the next free block and
+ the of the current block, making the minimum size of a free block
+ 2 * "Size of Lengths".
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Address of Data Segment</td>
+ <td>
+ <P>The data segment originally starts immediately after
+ the heap header, but if the data segment must grow as a
+ result of adding more objects, then the data segment may
+ be relocated, in its entirety, to another part of the
+ file.
+ </P>
+ </td>
+ </tr>
+ </table>
+ </center>
+
+ <p>Objects within the heap should be aligned on an 8-byte boundary.
+
+ <h3><a name="GlobalHeap">Disk Format: Level 1E - Global Heap</a></h3>
+
+ <P>Each HDF5 file has a global heap which stores various types of
+ information which is typically shared between datasets. The
+ global heap was designed to satisfy these goals:
+
+ <ol type="A">
+ <li>Repeated access to a heap object must be efficient without
+ resulting in repeated file I/O requests. Since global heap
+ objects will typically be shared among several datasets, it is
+ probable that the object will be accessed repeatedly.
+ <li>Collections of related global heap objects should result in
+ fewer and larger I/O requests. For instance, a dataset of
+ object references will have a global heap object for each
+ reference. Reading the entire set of object references
+ should result in a few large I/O requests instead of one small
+ I/O request for each reference.
+ <li>It should be possible to remove objects from the global heap
+ and the resulting file hole should be eligible to be reclaimed
+ for other uses.
+ </ol>
+ </P>
+
+ <P>The implementation of the heap makes use of the memory
+ management already available at the file level and combines that
+ with a new top-level object called a <em>collection</em> to
+ achieve Goal B. The global heap is the set of all collections.
+ Each global heap object belongs to exactly one collection and
+ each collection contains one or more global heap objects. For
+ the purposes of disk I/O and caching, a collection is treated as
+ an atomic object.
+ </P>
+
+ <P>The HDF5 library creates global heap collections as needed, so there may
+ be multiple collections throughout the file. The set of all of them is
+ abstractly called the "global heap", although they don't actually link
+ to each other, and there is no global place in the file where you can
+ discover all of the collections. The collections are found simply by
+ finding a reference to one through another object in the file (eg.
+ variable-length datatype elements, etc).
+ </P>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ A Global Heap Collection
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=4>Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan=3>Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Collection Size<sup>L</sup></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Global Heap Object 1<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Global Heap Object 2<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>...<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Global Heap Object <em>N</em><br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Global Heap Object 0 (free space)<br><br></td>
+ </tr>
+ </table>
+
+ <table class=note>
+ <tr><td>
+ (Items marked with an 'L' the above table are
+ <br>
+ of the size specified in "Size of Lengths.")
+ </td></tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Signature</td>
+ <td>
+ <P>The ASCII character string "<code>GCOL</code>"
+ is used to indicate the
+ beginning of a collection. This gives file consistency
+ checking utilities a better chance of reconstructing a
+ damaged file.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>
+ <P>Each collection has its own version number so that new
+ collections can be added to old files. This document
+ describes version one (1) of the collections (there is no
+ version zero (0)).
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Collection Size</td>
+ <td>
+ <P>This is the size in bytes of the entire collection
+ including this field. The default (and minimum)
+ collection size is 4096 bytes which is a typical file
+ system block size. This allows for 127 16-byte heap
+ objects plus their overhead (the collection header of 16 bytes
+ and the 16 bytes of information about each heap object).
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Global Heap Object 1 through <em>N</em></td>
+ <td>
+ <P>The objects are stored in any order with no
+ intervening unused space.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Global Heap Object 0</td>
+ <td>
+ <P>Global Heap Object 0 (zero), when present, represents the free
+ space in the collection. Free space always appears at the end of
+ the collection. If the free space is too small to store the header
+ for Object 0 (described below) then the header is implied and the
+ collection contains no free space.
+ </P>
+ </td>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Global Heap Object
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=2>Heap Object ID</td>
+ <td colspan=2>Reference Count</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Object Size<sup>L</sup></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Object Data<br><br></td>
+ </tr>
+ </table>
+
+ <table class=note>
+ <tr><td>
+ (Items marked with an 'L' the above table are
+ <br>
+ of the size specified in "Size of Lengths.")
+ </td></tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Heap Object ID</td>
+ <td>
+ <P>Each object has a unique identification number within a
+ collection. The identification numbers are chosen so that
+ new objects have the smallest value possible with the
+ exception that the identifier <code>0</code> always refers to the
+ object which represents all free space within the
+ collection.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Reference Count</td>
+ <td>
+ <P>All heap objects have a reference count field. An
+ object which is referenced from some other part of the
+ file will have a positive reference count. The reference
+ count for Object 0 is always zero.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Reserved</td>
+ <td>
+ <P>Zero padding to align next field on an 8-byte boundary.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Object Size</td>
+ <td>
+ <P>This is the size of the object data stored for the object.
+ The actual storage space allocated for the object data is rounded
+ up to a multiple of eight.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Object Data</td>
+ <td>
+ <P>The object data is treated as a one-dimensional array
+ of bytes to be interpreted by the caller.
+ </P>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <h3><a name="FreeSpaceIndex">Disk Format: Level 1F - Free-space Index</a></h3>
+
+ <p>The free-space index is a collection of blocks of data,
+ dispersed throughout the file, which are currently not used by
+ any file objects.
+
+ <p>The super block contains a pointer to root of the free-space description;
+ that pointer is currently required to be the
+ <A href="#UndefinedAddress">undefined address</A>.
+
+ <p>The format of the free-space index is not defined at this time.
+
+<!--
+ <p>The Free-space Index is a collection of blocks of data,
+ dispersed throughout the file, which are currently not used by
+ any file objects. The blocks of data are indexed by a B-tree of
+ their length within the file.
+
+
+ <p>Each B-tree page is composed of the following entries and
+ B-tree management information, organized as follows:
+
+ <p>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=bottom>
+ <B>HDF5 Free-space Heap Page</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+
+ <tr align=center>
+ <td colspan=4>Signature</td>
+ <tr align=center>
+ <td colspan=4>B-tree Left-link Offset</td>
+ <tr align=center>
+ <td colspan=4><br>Length of Free-block #1<br> <br></td>
+ <tr align=center>
+ <td colspan=4><br>Offset of Free-block #1<br> <br></td>
+ <tr align=center>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr align=center>
+ <td colspan=4><br>Length of Free-block #n<br> <br></td>
+ <tr align=center>
+ <td colspan=4><br>Offset of Free-block #n<br> <br></td>
+ <tr align=center>
+ <td colspan=4>"High" Offset</td>
+ <tr align=center>
+ <td colspan=4>Right-link Offset</td>
+ </table>
+ </center>
+
+ <p>
+ <dl>
+ <dt> The elements of the free-space heap page are described below:
+ <dd>
+ <dl>
+ <dt>Signature: (4 bytes)
+ <dd>The ASCII character string <code>FREE</code>
+ is used to indicate the
+ beginning of a free-space heap B-tree page. This gives
+ file consistency checking utilities a better chance of
+ reconstructing a damaged file.
+
+ <dt>B-tree Left-link Offset: (&lt;offset&gt; bytes)
+ <dd>This value is used to indicate the offset of all offsets
+ in the B-link-tree which are smaller than the value of the
+ offset in entry #1. This value is also used to indicate a
+ leaf node in the B-link-tree by being set to all ones.
+
+ <dt>Length of Free-block #n: (&lt;length&gt; bytes)
+ <dd>This value indicates the length of an unused block in
+ the file.
+
+ <dt>Offset of Free-block #n: (&lt;offset&gt; bytes)
+ <dd>This value indicates the offset in the file of an
+ unused block in the file.
+
+ <dt>"High" Offset: (4-bytes)
+ <dd>This offset is used as the upper bound on offsets
+ contained within a page when the page has been split.
+
+ <dt>Right-link Offset: (&lt;offset&gt; bytes)
+ <dd>This value is used to indicate the offset of the next
+ child to the right of the parent of this group
+ page. When there is no node to the right, this value is
+ all zeros.
+ </dl>
+ </dl>
+
+ <p>The algorithms for searching and inserting objects in the
+ B-tree pages are described fully in the Lehman and Yao paper,
+ which should be read to provide a full description of the
+ B-tree's usage.
+-->
+
+ <BR>
+ <HR>
+
+ <h2><a name="DataObject">Disk Format: Level 2 - Data Objects </a></h2>
+
+ <P>Data objects contain the real information in the file. These
+ objects compose the scientific data and other information which
+ are generally thought of as "data" by the end-user. All the
+ other information in the file is provided as a framework for
+ these data objects.
+ </P>
+
+ <P>A data object is composed of header information and data
+ information. The header information contains the information
+ needed to interpret the data information for the data object as
+ well as additional "metadata" or pointers to additional
+ "metadata" used to describe or annotate each data object.
+ </P>
+
+ <h3><a name="ObjectHeader">
+ Disk Format: Level 2A - Data Object Headers</a></h3>
+
+ <P>The header information of an object is designed to encompass
+ all the information about an object, except for the data itself.
+ This information includes
+ the dataspace, datatype, information about how the data
+ is stored on disk (in external files, compressed, broken up in
+ blocks, etc.), as well as other information used by the library
+ to speed up access to the data objects or maintain a file's
+ integrity. Information stored by user applications as attributes
+ is also stored in the object's header. The header of each object is
+ not necessarily located immediately prior to the object's data in the
+ file and in fact may be located in any position in the file. The order
+ of the messages in an object header is not significant.
+ </P>
+
+ <P>Header messages are aligned on 8-byte boundaries.
+ </P>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Object Headers
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Reserved (zero)</td>
+ <td colspan=2>Number of Header Messages</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Object Reference Count</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Object Header Size</td>
+ </tr>
+
+ <tr>
+ <td colspan=2>Header Message Type #1</td>
+ <td colspan=2>Size of Header Message Data #1</td>
+ </tr>
+
+ <tr>
+ <td>Header Message #1 Flags</td>
+ <td colspan=3>Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Header Message Data #1<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4>.<br>.<br>.<br></td>
+ </tr>
+
+ <tr>
+ <td colspan=2>Header Message Type #n</td>
+ <td colspan=2>Size of Header Message Data #n</td>
+ </tr>
+
+ <tr>
+ <td>Header Message #n Flags</td>
+ <td colspan=3>Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Header Message Data #n<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>
+ <P>This value is used to determine the format of the
+ information in the object header. When the format of the
+ information in the object header is changed, the version number
+ is incremented and can be used to determine how the
+ information in the object header is formatted. This
+ document describes version one (1) (there was no version
+ zero (0)).
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Number of Header Messages</td>
+ <td>
+ <P>This value determines the number of messages listed in
+ object headers for this object. This value includes the messages
+ in continuation messages for this object.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Object Reference Count</td>
+ <td>
+ <P>This value specifies the number of "hard links" to this object
+ within the current file. References to the object from external
+ files, "soft links" in this file and object references in this
+ file are not tracked.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Object Header Size</td>
+ <td>
+ <P>This value specifies the number of bytes of header message data
+ following this length field that contain object header messages
+ for this object header. This value does not include the size of
+ object header continuation blocks for this object elsewhere in the
+ file.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Header Message Type</td>
+ <td>
+ <P>This value specifies the type of information included in the
+ following header message data. The header message types for the
+ pre-defined header messages are included in sections below.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Size of Header Message Data</td>
+ <td>
+ <P>This value specifies the number of bytes of header
+ message data following the header message type and length
+ information for the current message. The size includes
+ padding bytes to make the message a multiple of eight
+ bytes.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Header Message Flags</td>
+ <td>
+ <P>This is a bit field with the following definition:
+ <table class=list>
+ <tr>
+ <th width="30%">Bit</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>0</code></td>
+ <td>If set, the message data is constant. This is used
+ for messages like the datatype message of a dataset.
+ </td>
+ </tr>
+ <tr>
+ <td align=center><code>1</code></td>
+ <td>If set, the message is stored in the global heap.
+ The Header Message Data field contains a Shared Object
+ message and the Size of Header Message Data field
+ contains the size of that Shared Object message.
+ </td>
+ </tr>
+ <tr>
+ <td align=center><code>2-7</code></td>
+ <td>Reserved</td>
+ </tr>
+ </table>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Header Message Data</td>
+ <td>
+ <P>The format and length of this field is determined by the
+ header message type and size respectively. Some header
+ message types do not require any data and this information
+ can be eliminated by setting the length of the message to
+ zero. The data is padded with enough zeros to make the
+ size a multiple of eight.
+ </P>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <P>The header message types and the message data associated with
+ them compose the critical "metadata" about each object. Some
+ header messages are required for each object while others are
+ optional. Some optional header messages may also be repeated
+ several times in the header itself, the requirements and number
+ of times allowed in the header will be noted in each header
+ message description below.
+ </P>
+
+ <P>The following is a list of currently defined header messages:
+ </P>
+
+ <hr>
+ <h4><a name="NILMessage">Name: NIL</a></h4>
+
+ <P class=item><B>Header Message Type: </B>0x0000
+ </P>
+ <P class=item><B>Length:</B> varies
+ </P>
+ <P class=item><B>Status:</B> Optional, may be repeated.
+ </P>
+ <P class=item><B>Purpose and Description:</B> The NIL message is used to indicate a
+ message which is to be ignored when reading the header messages for a
+ data object. [Possibly one which has been deleted for some reason.]
+ </P>
+ <P class=item><B>Format of Data:</B> Unspecified.
+ </P>
+
+ <hr>
+ <h4><a name="SimpleDataSpace">Name: Simple Dataspace</a></h4>
+
+ <P class=item><B>Header Message Type: </B>0x0001
+ </P>
+ <P class=item><B>Length:</B> Varies according to the number of dimensions,
+ as described in the following table.
+ </P>
+ <P class=item><B>Status:</B> Required for dataset objects, may not be
+ repeated.
+ </P>
+ <P class=item><B>Description:</B> The simple dataspace message describes the
+ number of dimensions (i.e. "rank") and size of each dimension that the
+ data object has. This message is only used for datasets which have a
+ simple, rectilinear grid layout; datasets requiring a more complex
+ layout (irregularly structured or unstructured grids, etc.) must use
+ the <em>Complex Dataspace</em> message for expressing the space the
+ dataset inhabits. <i>(Note: The <em>Complex Dataspace</em>
+ functionality is not yet implemented and it is not described in this
+ document.)</i>
+ </P>
+
+ <P class=item><B>Format of Data:</B>
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Simple Dataspace Message
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Dimensionality</td>
+ <td>Flags</td>
+ <td>Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Dimension #1 Size<sup>L</sup></td>
+ <tr>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr>
+ <td colspan=4>Dimension #n Size<sup>L</sup></td>
+ <tr>
+ <td colspan=4>Dimension #1 Maximum Size<sup>L</sup></td>
+ <tr>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr>
+ <td colspan=4>Dimension #n Maximum Size<sup>L</sup></td>
+ <tr>
+ <td colspan=4>Permutation Index #1<sup>L</sup></td>
+ <tr>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr>
+ <td colspan=4>Permutation Index #n<sup>L</sup></td>
+ </table>
+
+ <table class=note>
+ <tr><td>
+ (Items marked with an 'L' the above table are
+ <br>
+ of the size specified in "Size of Lengths.")
+ </td></tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>
+ <P>This value is used to determine the format of the
+ Simple Dataspace Message. When the format of the
+ information in the message is changed, the version number
+ is incremented and can be used to determine how the
+ information in the object header is formatted. This
+ document describes version one (1) (there was no version
+ zero (0)).
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Dimensionality</td>
+ <td>
+ <P>This value is the number of dimensions that the data
+ object has.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Flags</td>
+ <td>
+ <P>This field is used to store flags to indicate the
+ presence of parts of this message. Bit 0 (the least
+ significant bit) is used to indicate that maximum
+ dimensions are present. Bit 1 is used to indicate that
+ permutation indices are present.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Dimension #n Size</td>
+ <td>
+ <P>This value is the current size of the dimension of the
+ data as stored in the file. The first dimension stored in
+ the list of dimensions is the slowest changing dimension
+ and the last dimension stored is the fastest changing
+ dimension.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Dimension #n Maximum Size</td>
+ <td>
+ <P>This value is the maximum size of the dimension of the
+ data as stored in the file. This value may be the special
+ "<A href="#UnlimitedDim">unlimited</A>" size which indicates
+ that the data may expand along this dimension indefinitely.
+ If these values are not stored, the maximum size of each
+ dimension is assumed to be the dimension's current size.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Permutation Index #n</td>
+ <td>
+ <P>This value is the index permutation used to map
+ each dimension from the canonical representation to an
+ alternate axis for each dimension. If these values are
+ not stored, the first dimension stored in the list of
+ dimensions is the slowest changing dimension and the last
+ dimension stored is the fastest changing dimension.
+ </P>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ </P>
+
+<!--
+ <hr>
+ <h4><a name="DataSpaceMessage">Name: Complex Dataspace (Fiber Bundle?)</a></h4>
+ <b>Header Message Type: </b>0x0002<br>
+ <b>Length:</b> varies<br>
+
+ <b>Status:</b> One of the <em>Simple Dataspace</em> or
+ <em>Complex Dataspace</em> messages is required (but not both) and may
+ not be repeated.<br> <b>Purpose and Description:</b> The
+ <em>Dataspace</em> message describes space that the dataset is
+ mapped onto in a more comprehensive way than the <em>Simple
+ Dimensionality</em> message is capable of handling. The
+ dataspace of a dataset encompasses the type of coordinate system
+ used to locate the dataset's elements as well as the structure and
+ regularity of the coordinate system. The dataspace also
+ describes the number of dimensions which the dataset inhabits as
+ well as a possible higher dimensional space in which the dataset
+ is located within.
+
+ <br>
+ <b>Format of Data:</b>
+
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=bottom>
+ <B>HDF5 Dataspace Message Layout</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+
+ <tr align=center>
+ <td colspan=4>Mesh Type</td>
+ <tr align=center>
+ <td colspan=4>Logical Dimensionality</td>
+ </table>
+ </center>
+
+ <p>
+ <dl>
+ <dt>The elements of the dimensionality message are described below:
+ <dd>
+ <dl>
+ <dt>Mesh Type: (unsigned 32-bit integer)
+ <dd>This value indicates whether the grid is
+ polar/spherical/cartesion,
+ structured/unstructured and regular/irregular. <br>
+ The mesh type value is broken up as follows: <br>
+
+ <P>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=bottom>
+ <B>HDF5 Mesh-type Layout</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+
+ <tr align=center>
+ <td colspan=1>Mesh Embedding</td>
+ <td colspan=1>Coordinate System</td>
+ <td colspan=1>Structure</td>
+ <td colspan=1>Regularity</td>
+ </table>
+ </center>
+ The following are the definitions of mesh-type bytes:
+ <dl>
+ <dt>Mesh Embedding
+ <dd>This value indicates whether the dataset dataspace
+ is located within
+ another dataspace or not:
+ <dl> <dl>
+ <dt>&lt;STANDALONE&gt;
+ <dd>The dataset mesh is self-contained and is not
+ embedded in another mesh.
+ <dt>&lt;EMBEDDED&gt;
+ <dd>The dataset's dataspace is located within
+ another dataspace, as
+ described in information below.
+ </dl> </dl>
+ <dt>Coordinate System
+ <dd>This value defines the type of coordinate system
+ used for the mesh:
+ <dl> <dl>
+ <dt>&lt;POLAR&gt;
+ <dd>The last two dimensions are in polar
+ coordinates, higher dimensions are
+ cartesian.
+ <dt>&lt;SPHERICAL&gt;
+ <dd>The last three dimensions are in spherical
+ coordinates, higher dimensions
+ are cartesian.
+ <dt>&lt;CARTESIAN&gt;
+ <dd>All dimensions are in cartesian coordinates.
+ </dl> </dl>
+ <dt>Structure
+ <dd>This value defines the locations of the grid-points
+ on the axes:
+ <dl> <dl>
+ <dt>&lt;STRUCTURED&gt;
+ <dd>All grid-points are on integral, sequential
+ locations, starting from 0.
+ <dt>&lt;UNSTRUCTURED&gt;
+ <dd>Grid-points locations in each dimension are
+ explicitly defined and
+ may be of any numeric datatype.
+ </dl> </dl>
+ <dt>Regularity
+ <dd>This value defines the locations of the dataset
+ points on the grid:
+ <dl> <dl>
+ <dt>&lt;REGULAR&gt;
+ <dd>All dataset elements are located at the
+ grid-points defined.
+ <dt>&lt;IRREGULAR&gt;
+ <dd>Each dataset element has a particular
+ grid-location defined.
+ </dl> </dl>
+ </dl>
+ <p>The following grid combinations are currently allowed:
+ <dl> <dl>
+ <dt>&lt;POLAR-STRUCTURED-REGULAR&gt;
+ <dt>&lt;SPHERICAL-STRUCTURED-REGULAR&gt;
+ <dt>&lt;CARTESIAN-STRUCTURED-REGULAR&gt;
+ <dt>&lt;POLAR-UNSTRUCTURED-REGULAR&gt;
+ <dt>&lt;SPHERICAL-UNSTRUCTURED-REGULAR&gt;
+ <dt>&lt;CARTESIAN-UNSTRUCTURED-REGULAR&gt;
+ <dt>&lt;CARTESIAN-UNSTRUCTURED-IRREGULAR&gt;
+ </dl> </dl>
+ All of the above grid types can be embedded within another
+ dataspace.
+ <br> <br>
+ <dt>Logical Dimensionality: (unsigned 32-bit integer)
+ <dd>This value is the number of dimensions that the dataset occupies.
+
+ <P>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=bottom>
+ <B>HDF5 Dataspace Embedded Dimensionality Information</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+
+ <tr align=center>
+ <td colspan=4>Embedded Dimensionality</td>
+ <tr align=center>
+ <td colspan=4>Embedded Dimension Size #1</td>
+ <tr align=center>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr align=center>
+ <td colspan=4>Embedded Dimension Size #n</td>
+ <tr align=center>
+ <td colspan=4>Embedded Origin Location #1</td>
+ <tr align=center>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr align=center>
+ <td colspan=4>Embedded Origin Location #n</td>
+ </table>
+ </center>
+
+ <dt>Embedded Dimensionality: (unsigned 32-bit integer)
+ <dd>This value is the number of dimensions of the space the
+ dataset is located
+ within. i.e. a planar dataset located within a 3-D space,
+ or a 3-D dataset
+ which is a subset of another 3-D space, etc.
+ <dt>Embedded Dimension Size: (unsigned 32-bit integer)
+ <dd>These values are the sizes of the dimensions of the
+ embedded dataspace
+ that the dataset is located within.
+ <dt>Embedded Origin Location: (unsigned 32-bit integer)
+ <dd>These values comprise the location of the dataset's
+ origin within the embedded dataspace.
+ </dl>
+ </dl>
+ [Comment: need some way to handle different orientations of the
+ dataset dataspace
+ within the embedded dataspace]<br>
+
+ <P>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=bottom>
+ <B>HDF5 Dataspace Structured/Regular Grid Information</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+
+ <tr align=center>
+ <td colspan=4>Logical Dimension Size #1</td>
+ <tr align=center>
+ <td colspan=4>Logical Dimension Maximum #1</td>
+ <tr align=center>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr align=center>
+ <td colspan=4>Logical Dimension Size #n</td>
+ <tr align=center>
+ <td colspan=4>Logical Dimension Maximum #n</td>
+ </table>
+ </center>
+
+ <p>
+ <dl>
+ <dt>The elements of the dimensionality message are described below:
+ <dd>
+ <dl>
+ <dt>Logical Dimension Size #n: (unsigned 32-bit integer)
+ <dd>This value is the current size of the dimension of the
+ data as stored in
+ the file. The first dimension stored in the list of
+ dimensions is the slowest
+ changing dimension and the last dimension stored is the
+ fastest changing
+ dimension.
+ <dt>Logical Dimension Maximum #n: (unsigned 32-bit integer)
+ <dd>This value is the maximum size of the dimension of the
+ data as stored in
+ the file. This value may be the special value
+ &lt;UNLIMITED&gt; which
+ indicates that the data may expand along this dimension
+ indefinitely.
+ </dl>
+ </dl>
+ <P>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=bottom>
+ <B>HDF5 Dataspace Structured/Irregular Grid Information</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+
+ <tr align=center>
+ <td colspan=4># of Grid Points in Dimension #1</td>
+ <tr align=center>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr align=center>
+ <td colspan=4># of Grid Points in Dimension #n</td>
+ <tr align=center>
+ <td colspan=4>Datatype of Grid Point Locations</td>
+ <tr align=center>
+ <td colspan=4>Location of Grid Points in Dimension #1</td>
+ <tr align=center>
+ <td colspan=4>.<br>.<br>.<br></td>
+ <tr align=center>
+ <td colspan=4>Location of Grid Points in Dimension #n</td>
+ </table>
+ </center>
+
+ <P>
+ <center>
+ <table border cellpadding=4 width="80%">
+ <caption align=bottom>
+ <B>HDF5 Dataspace Unstructured Grid Information</B>
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+
+ <tr align=center>
+ <td colspan=4># of Grid Points</td>
+ <tr align=center>
+ <td colspan=4>Datatype of Grid Point Locations</td>
+ <tr align=center>
+ <td colspan=4>Grid Point Locations<br>.<br>.<br></td>
+ </table>
+ </center>
+-->
+
+ <hr>
+ <h4><a name="ReservedMessage_0002">Name: Reserved - Not Assigned Yet</a></h4>
+ <b>Header Message Type:</b> 0x0002<BR>
+ <b>Length:</b> N/A<BR>
+ <b>Status:</b> N/A<BR>
+ <b>Format of Data:</b> N/A<BR>
+
+ <p><b>Purpose and Description:</b> This message type was skipped during
+ the initial specification of the file format and may be used in a
+ future expansion to the format.
+
+
+ <hr>
+ <h4><a name="DataTypeMessage">Name: Datatype</a></h4>
+
+ <P class=item><B>Header Message Type:</B> 0x0003
+ </P>
+ <P class=item><B>Length:</B> variable
+ </P>
+ <P class=item><B>Status:</B> Required for dataset or named datatype objects,
+ may not be repeated.
+ </P>
+
+ <P class=item><B>Description:</B> The datatype message defines the datatype
+ for each element of a dataset. A datatype can describe an atomic type
+ like a fixed- or floating-point type or a compound type like a C
+ struct.
+ Datatypes messages are stored
+ as a list of datatype classes and
+ their associated properties.
+ </P>
+
+ <P class=item2>Datatype messages that are part of a dataset object,
+ do not describe how elements are related to one another, the dataspace
+ message is used for that purpose. Datatype messages that are part of
+ a named datatype message describe an "abstract" datatype that can be
+ used by other objects in the file.
+ </P>
+
+ <P class=item><B>Format of Data:</B>
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Datatype Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Class and Version</td>
+ <td>Class Bit Field, Bits 0-7</td>
+ <td>Class Bit Field, Bits 8-15</td>
+ <td>Class Bit Field, Bits 16-23</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Size</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br><br>Properties<br><br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Class and Version</td>
+ <td>
+ <P>The version of the datatype message and the datatype's class
+ information are packed together in this field. The version
+ number is packed in the top 4 bits of the field and the class
+ is contained in the bottom 4 bits.
+ </P>
+ <P>The version number information is used for changes in the
+ format of the datatype message and is described here:
+ <table class=list>
+ <tr>
+ <th width="30%">Version</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>0</code></td>
+ <td>Never used
+ </td>
+ </tr>
+ <tr>
+ <td align=center><code>1</code></td>
+ <td>Used by early versions of the library to encode
+ compound datatypes with explicit array fields.
+ See the compound datatype description below for
+ further details.
+ </td>
+ </tr>
+ <tr>
+ <td align=center><code>2</code></td>
+ <td>The current version used by the library.
+ </td>
+ </tr>
+ </table>
+ </P>
+ <P>The class of the datatype determines the format for the class
+ bit field and properties portion of the datatype message, which
+ are described below. The
+ following classes are currently defined:
+ <table width=100% class=list>
+ <tr>
+ <th width="30%">Value</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>0</code></td>
+ <td>Fixed-Point</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>1</code></td>
+ <td>Floating-Point</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>2</code></td>
+ <td>Time</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>3</code></td>
+ <td>String</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>4</code></td>
+ <td>Bitfield</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>5</code></td>
+ <td>Opaque</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>6</code></td>
+ <td>Compound</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>7</code></td>
+ <td>Reference</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>8</code></td>
+ <td>Enumerated</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>9</code></td>
+ <td>Variable-Length</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>10</code></td>
+ <td>Array</td>
+ </tr>
+ </table>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Class Bit Fields</td>
+ <td>
+ <P>The information in these bit fields is specific to each datatype
+ class and is described below. All bits not defined for a
+ datatype class are set to zero.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Size</td>
+ <td>
+ <P>The size of the datatype in bytes.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Properties</td>
+ <td>
+ <P>This variable-sized field encodes information specific to each
+ datatype class and is described below. If there is no
+ property information specified for a datatype class, the size
+ of this field is zero.
+ </P>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+ </P>
+
+ <P>Class specific information for Fixed-Point Numbers (Class 0):
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <caption>
+ Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td>0</td>
+ <td><b>Byte Order.</b> If zero, byte order is little-endian;
+ otherwise, byte order is big endian.</td>
+ </tr>
+
+ <tr>
+ <td>1, 2</td>
+ <td><b>Padding type.</b> Bit 1 is the lo_pad type and bit 2
+ is the hi_pad type. If a datum has unused bits at either
+ end, then the lo_pad or hi_pad bit is copied to those
+ locations.</td>
+ </tr>
+
+ <tr>
+ <td>3</td>
+ <td><b>Signed.</b> If this bit is set then the fixed-point
+ number is in 2's complement form.</td>
+ </tr>
+
+ <tr>
+ <td>4-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Property Descriptions
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=2>Bit Offset</td>
+ <td colspan=2>Bit Precision</td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Bit Offset</td>
+ <td>
+ <P>The bit offset of the first significant bit of the fixed-point
+ value within the datatype. The bit offset specifies the number
+ of bits "to the right of" the value.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Bit Precision</td>
+ <td>
+ <P>The number of bits of precision of the fixed-point value
+ within the datatype.
+ </P>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+ </P>
+
+ <P>Class specific information for Floating-Point Numbers (Class 1):
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <caption>
+ Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td>0</td>
+ <td><b>Byte Order.</b> If zero, byte order is little-endian;
+ otherwise, byte order is big endian.</td>
+ </tr>
+
+ <tr>
+ <td>1, 2, 3</td>
+ <td><b>Padding type.</b> Bit 1 is the low bits pad type, bit 2
+ is the high bits pad type, and bit 3 is the internal bits
+ pad type. If a datum has unused bits at either end or between
+ the sign bit, exponent, or mantissa, then the value of bit
+ 1, 2, or 3 is copied to those locations.</td>
+ </tr>
+
+ <tr>
+ <td>4-5</td>
+ <td><b>Normalization.</b> The value can be 0 if there is no
+ normalization, 1 if the most significant bit of the
+ mantissa is always set (except for 0.0), and 2 if the most
+ signficant bit of the mantissa is not stored but is
+ implied to be set. The value 3 is reserved and will not
+ appear in this field.</td>
+ </tr>
+
+ <tr>
+ <td>6-7</td>
+ <td>Reserved (zero).</td>
+ </tr>
+
+ <tr>
+ <td>8-15</td>
+ <td><b>Sign Location.</b> This is the bit position of the sign
+ bit. Bits are numbered with the least significant bit zero.</td>
+ </tr>
+
+ <tr>
+ <td>16-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Property Descriptions
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=2>Bit Offset</td>
+ <td colspan=2>Bit Precision</td>
+ </tr>
+
+ <tr>
+ <td>Exponent Location</td>
+ <td>Exponent Size</td>
+ <td>Mantissa Location</td>
+ <td>Mantissa Size</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Exponent Bias</td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Bit Offset</td>
+ <td>
+ <P>The bit offset of the first significant bit of the floating-point
+ value within the datatype. The bit offset specifies the number
+ of bits "to the right of" the value.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Bit Precision</td>
+ <td>
+ <P>The number of bits of precision of the floating-point value
+ within the datatype.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Exponent Location</td>
+ <td>
+ <P>The bit position of the exponent field. Bits are numbered with
+ the least significant bit number zero.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Exponent Size</td>
+ <td>
+ <P>The size of the exponent field in bits.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Mantissa Location</td>
+ <td>
+ <P>The bit position of the mantissa field. Bits are numbered with
+ the least significant bit number zero.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Mantissa Size</td>
+ <td>
+ <P>The size of the mantissa field in bits.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Exponent Bias</td>
+ <td>
+ <P>The bias of the exponent field.
+ </P>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+ </P>
+
+ <P>Class specific information for Time (Class 2):
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <caption>
+ Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td>0</td>
+ <td><b>Byte Order.</b> If zero, byte order is little-endian;
+ otherwise, byte order is big endian.</td>
+ </tr>
+
+ <tr>
+ <td>1-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Property Descriptions
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=2>Bit Precision</td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Bit Precision</td>
+ <td>
+ <P>The number of bits of precision of the time value.
+ </P>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+ </P>
+
+ <P>Class specific information for Strings (Class 3):
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <caption>
+ Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td>0-3</td>
+ <td><b>Padding type.</b> This four-bit value determines the
+ type of padding to use for the string. The values are:
+
+ <table width=100% class=list>
+ <tr>
+ <th width="30%">Value</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>0</code></td>
+ <td>Null Terminate: A zero byte marks the end of the
+ string and is guaranteed to be present after
+ converting a long string to a short string. When
+ converting a short string to a long string the value is
+ padded with additional null characters as necessary.
+ </td>
+ </tr>
+
+ <tr>
+ <td align=center><code>1</code></td>
+ <td>Null Pad: Null characters are added to the end of
+ the value during conversions from short values to long
+ values but conversion in the opposite direction simply
+ truncates the value.
+ </td>
+ </tr>
+
+ <tr>
+ <td align=center><code>2</code></td>
+ <td>Space Pad: Space characters are added to the end of
+ the value during conversions from short values to long
+ values but conversion in the opposite direction simply
+ truncates the value. This is the Fortran
+ representation of the string.
+ </td>
+ </tr>
+
+ <tr>
+ <td align=center><code>3-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table>
+ </tr>
+
+ <tr>
+ <td>4-7</td>
+ <td><b>Character Set.</b> The character set to use for
+ encoding the string. The only character set supported is
+ the 8-bit ASCII (zero) so no translations have been defined
+ yet.</td>
+ </tr>
+
+ <tr>
+ <td>8-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+ </table>
+ </div>
+
+ <P>There are no properties defined for the string class.
+ </P>
+ </P>
+
+ <P>Class specific information for Bitfields (Class 4):
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <caption>
+ Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td>0</td>
+ <td><b>Byte Order.</b> If zero, byte order is little-endian;
+ otherwise, byte order is big endian.</td>
+ </tr>
+
+ <tr>
+ <td>1, 2</td>
+ <td><b>Padding type.</b> Bit 1 is the lo_pad type and bit 2
+ is the hi_pad type. If a datum has unused bits at either
+ end, then the lo_pad or hi_pad bit is copied to those
+ locations.</td>
+ </tr>
+
+ <tr>
+ <td>3-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=2>Bit Offset</td>
+ <td colspan=2>Bit Precision</td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Bit Offset</td>
+ <td>
+ <P>The bit offset of the first significant bit of the bitfield
+ within the datatype. The bit offset specifies the number
+ of bits "to the right of" the value.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Bit Precision</td>
+ <td>
+ <P>The number of bits of precision of the bitfield
+ within the datatype.
+ </P>
+ </td>
+ </tr>
+ </table>
+ </div>
+ </P>
+
+ <P>Class specific information for Opaque (Class 5):
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <caption>
+ Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td>0-7</td>
+ <td>Length of ASCII tag in bytes.</td>
+ </tr>
+
+ <tr>
+ <td>8-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>ASCII Tag<br>
+ <br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>ASCII Tag</td>
+ <td>
+ <P>This NUL-terminated string provides a description for the
+ opaque type. It is NUL-padded to a multiple of 8 bytes.
+ </P>
+ </td>
+ </tr>
+ </table>
+ </div>
+ </P>
+
+ <P>Class specific information for Compound (Class 6):
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <caption>
+ Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td>0-15</td>
+ <td><b>Number of Members.</b> This field contains the number
+ of members defined for the compound datatype. The member
+ definitions are listed in the Properties field of the data
+ type message.
+ </tr>
+
+ <tr>
+ <td>15-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+ </table>
+ </div>
+ </P>
+
+ <P>The Properties field of a compound datatype is a list of the
+ member definitions of the compound datatype. The member
+ definitions appear one after another with no intervening bytes.
+ The member types are described with a recursive datatype
+ message.
+
+ <P>Note that the property descriptions are different for different
+ versions of the datatype version. Additionally note that the version
+ 0 properties are deprecated and have been replaced with the version
+ 1 properties in versions of the HDF5 library from the 1.4 release
+ onward.
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Properties Description for Datatype Version 1
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Name<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Byte Offset of Member</td>
+ </tr>
+
+ <tr>
+ <td>Dimensionality</td>
+ <td colspan=3>Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Dimension Permutation</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Dimension #1 Size (required)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Dimension #2 Size (required)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Dimension #3 Size (required)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Dimension #4 Size (required)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Member Type Message<br><br></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Name</td>
+ <td>
+ <P>This NUL-terminated string provides a description for the
+ opaque type. It is NUL-padded to a multiple of 8 bytes.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Byte Offset of Member</td>
+ <td>
+ <P>This is the byte offset of the member within the datatype.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Dimensionality</td>
+ <td>
+ <P>If set to zero, this field indicates a scalar member. If set
+ to a value greater than zero, this field indicates that the
+ member is an array of values. For array members, the size of
+ the array is indicated by the 'Size of Dimension n' field in
+ this message.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Dimension Permutation</td>
+ <td>
+ <P>This field was intended to allow an array field to have
+ it's dimensions permuted, but this was never implemented.
+ This field should always be set to zero.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Dimension #n Size</td>
+ <td>
+ <P>This field is the size of a dimension of the array field as
+ stored in the file. The first dimension stored in the list of
+ dimensions is the slowest changing dimension and the last
+ dimension stored is the fastest changing dimension.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Member Type Message</td>
+ <td>
+ <P>This field is a datatype message describing the datatype of
+ the member.
+ </P>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Properties Description for Datatype Version 2
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Name<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Byte Offset of Member</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Member Type Message<br><br></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Name</td>
+ <td>
+ <P>This NUL-terminated string provides a description for the
+ opaque type. It is NUL-padded to a multiple of 8 bytes.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Byte Offset of Member</td>
+ <td>
+ <P>This is the byte offset of the member within the datatype.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Member Type Message</td>
+ <td>
+ <P>This field is a datatype message describing the datatype of
+ the member.
+ </P>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+ </P>
+
+ <P>Class specific information for Reference (Class 7):
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <caption>
+ Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td>0-3</td>
+ <td><b>Type.</b> This four-bit value contains the type of reference
+ described. The values defined are:
+
+ <table width=100% class=list>
+ <tr>
+ <th width="30%">Value</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>0</code></td>
+ <td>Object Reference: A reference to another object in this
+ HDF5 file.
+ </td>
+ </tr>
+
+ <tr>
+ <td align=center><code>1</code></td>
+ <td>Dataset Region Reference: A reference to a region within
+ a dataset in this HDF5 file.
+ </td>
+ </tr>
+
+ <tr>
+ <td align=center><code>2</code></td>
+ <td>Internal Reference: A reference to a region within the
+ current dataset. (Not currently implemented)
+ </td>
+ </tr>
+
+ <tr>
+ <td align=center><code>3-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td>15-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+ </table>
+ </div>
+
+ <P>There are no properties defined for the reference class.
+ </P>
+ </P>
+
+ <P>Class specific information for Enumeration (Class 8):
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <caption>
+ Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td>0-15</td>
+ <td><b>Number of Members.</b> The number of name/value
+ pairs defined for the enumeration type.</td>
+ </tr>
+
+ <tr>
+ <td>16-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Base Type<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Names<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Values<br><br></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Base Type</td>
+ <td>
+ <P>Each enumeration type is based on some parent type, usually an
+ integer. The information for that parent type is described
+ recursively by this field.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Names</td>
+ <td>
+ <P>The name for each name/value pair. Each name is stored as a null
+ terminated ASCII string in a multiple of eight bytes. The names
+ are in no particular order.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Values</td>
+ <td>
+ <P>The list of values in the same order as the names. The values
+ are packed (no inter-value padding) and the size of each value
+ is determined by the parent type.
+ </P>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+ </P>
+
+
+ <P>Class specific information for Variable-Length (Class 9):
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <caption>
+ Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td>0-3</td>
+ <td><b>Type.</b> This four-bit value contains the type of
+ variable-length datatype described. The values defined are:
+
+ <table width=100% class=list>
+ <tr>
+ <th width="30%">Value</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>0</code></td>
+ <td>Sequence: A variable-length sequence of any sequence of
+ data. Variable-length sequences do not have padding or
+ character set information.
+ </td>
+ </tr>
+
+ <tr>
+ <td align=center><code>1</code></td>
+ <td>String: A variable-length sequence of characters.
+ Variable-length strings have padding and character set
+ information.
+ </td>
+ </tr>
+
+ <tr>
+ <td align=center><code>2-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td>4-7</td>
+ <td><b>Padding type.</b> (variable-length string only)
+ This four-bit value determines the type of padding
+ used for variable-length strings. The values are the same
+ as for the string padding type, as follows:
+ <table width=100% class=list>
+ <tr>
+ <th width="30%">Value</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>0</code></td>
+ <td>Null terminate: A zero byte marks the end of a string
+ and is guaranteed to be present after converting a long
+ string to a short string. When converting a short string
+ to a long string, the value is padded with additional null
+ characters as necessary.
+ </td>
+ </tr>
+
+ <tr>
+ <td align=center><code>1</code></td>
+ <td>Null pad: Null characters are added to the end of the
+ value during conversion from a short string to a longer
+ string. Conversion from a long string to a shorter string
+ simply truncates the value.
+ </td>
+ </tr>
+
+ <tr>
+ <td align=center><code>2</code></td>
+ <td>Space pad: Space characters are added to the end of the
+ value during conversion from a short string to a longer
+ string. Conversion from a long string to a shorter string
+ simply truncates the value. This is the Fortran
+ representation of the string.
+ </td>
+ </tr>
+
+ <tr>
+ <td align=center><code>3-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table>
+
+ This value is set to zero for variable-length sequences.
+
+ </td>
+ </tr>
+
+ <tr>
+ <td>8-11</td>
+ <td><b>Character Set.</b> (variable-length string only)
+ This four-bit value specifies the character set
+ to be used for encoding the string:
+ <table width=100% class=list>
+ <tr>
+ <th width="30%">Value</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>0</code></td>
+ <td>ASCII: As of this writing (July 2003, Release 1.6.0),
+ 8-bit ASCII is the only character set supported. Therefore,
+ no translations have been defined.
+ </td>
+ </tr>
+
+ <tr>
+ <td align=center><code>1-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table>
+
+ This value is set to zero for variable-length sequences.
+
+ </td>
+ </tr>
+
+ <tr>
+ <td>12-23</td>
+ <td>Reserved (zero).</td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Base Type<br><br></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Base Type</td>
+ <td>
+ <P>Each variable-length type is based on some parent type. The
+ information for that parent type is described recursively by
+ this field.
+ </P>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+ </P>
+
+ <P>Class specific information for Array (Class 10):
+
+ <P>There are no bit fields defined for the array class.
+ </P>
+
+ <P>Note that the dimension information defined in the property for this
+ datatype class is independent of dataspace information for a dataset.
+ The dimension information here describes the dimensionality of the
+ information within a data element (or a component of an element, if the
+ array datatype is nested within another datatype) and the dataspace for a
+ dataset describes the location of the elements in a dataset.
+ </P>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td>Dimensionality</td>
+ <td colspan=3>Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Dimension #1 Size</td>
+ </tr>
+ <tr>
+ <td colspan=4>.<br>.<br>.<br></td>
+ </tr>
+ <tr>
+ <td colspan=4>Dimension #n Size</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Permutation Index #1</td>
+ </tr>
+ <tr>
+ <td colspan=4>.<br>.<br>.<br></td>
+ </tr>
+ <tr>
+ <td colspan=4>Permutation Index #n</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Base Type<br><br></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Dimensionality</td>
+ <td>
+ <P>This value is the number of dimensions that the array has.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Dimension #n Size</td>
+ <td>
+ <P>This value is the size of the dimension of the array
+ as stored in the file. The first dimension stored in
+ the list of dimensions is the slowest changing dimension
+ and the last dimension stored is the fastest changing
+ dimension.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Permutation Index #n</td>
+ <td>
+ <P>This value is the index permutation used to map
+ each dimension from the canonical representation to an
+ alternate axis for each dimension. Currently, dimension
+ permutations are not supported and these indices should be set
+ to the index position minus one (i.e. the first dimension should
+ be set to 0, the second dimension should be set to 1, etc.)
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Base Type</td>
+ <td>
+ <P>Each array type is based on some parent type. The
+ information for that parent type is described recursively by
+ this field.
+ </P>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ </P>
+
+ <hr>
+ <h4><a name="OldFillValueMessage">Name: Data Storage - Fill Value (Old)</a></h4>
+
+ <P class=item><B>Header Message Type:</B> 0x0004
+ </P>
+ <P class=item><B>Length:</B> varies
+ </P>
+ <P class=item><B>Status:</B> Optional, may not be repeated.
+ </P>
+
+ <P class=item><B>Description:</B> The fill value message stores a single
+ data value which is returned to the application when an uninitialized
+ data element is read from a dataset. The fill value is interpreted
+ with the same datatype as the dataset. If no fill value message is
+ present then a fill value of all zero bytes is assumed.
+ </P>
+
+ <P class=item2>This fill value message is deprecated in favor of the "new"
+ fill value message (Message Type 0x0005) and is only written to the
+ file for forward compatibility with versions of the HDF5 library before
+ the 1.6.0 version. Additionally, it only appears for datasets with a
+ user defined fill value (as opposed to the library default fill value
+ or an explicitly set "undefined" fill value).
+ </P>
+
+ <P class=item><B>Format of Data:</B>
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Fill Value Message (Old)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=4>Size</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Fill Value<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Size</td>
+ <td>
+ <P>This is the size of the Fill Value field in bytes.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Fill Value</td>
+ <td>
+ <P>The fill value. The bytes of the fill value are interpreted
+ using the same datatype as for the dataset.
+ </P>
+ </td>
+ </tr>
+ </table>
+ </div>
+ </P>
+
+ <hr>
+ <h4><a name="FillValueMessage">Name: Data Storage - Fill Value </a></h4>
+
+ <P class=item><B>Header Message Type:</B> 0x0005
+ </P>
+ <P class=item><B>Length:</B> varies
+ </P>
+ <P class=item><B>Status:</B> Required for dataset objects, may not be repeated.
+ </P>
+
+ <P class=item><B>Description:</B> The fill value message stores a single
+ data value which is returned to the application when an uninitialized
+ data element is read from a dataset. The fill value is interpreted
+ with the same datatype as the dataset.
+ </P>
+
+ <P class=item><B>Format of Data:</B>
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Fill Value Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Space Allocation Time</td>
+ <td>Fill Value Write Time</td>
+ <td>Fill Value Defined</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Size</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Fill Value<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>
+ <P>The version number information is used for changes in the
+ format of the fill value message and is described here:
+ <table class=list>
+ <tr>
+ <th width="30%">Version</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>0</code></td>
+ <td>Never used
+ </td>
+ </tr>
+ <tr>
+ <td align=center><code>1</code></td>
+ <td>Used by version 1.6.x of the library to encode
+ fill values. In this version, the Size field is
+ always present.
+ </td>
+ </tr>
+ <tr>
+ <td align=center><code>2</code></td>
+ <td>The current version used by the library (version
+ 1.7.3 or later). In this version, the Size and
+ Fill Value fields are
+ only present if the Fill Value Defined field is set
+ to 1.
+ </td>
+ </tr>
+ </table>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Space Allocation Time</td>
+ <td>
+ <P>When the storage space for the dataset's raw data will be
+ allocated. The allowed values are:
+ <table class=list>
+ <tr>
+ <th width="30%">Value</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>1</code></td>
+ <td>Early allocation. Storage space for the entire dataset
+ should be allocated in the file when the dataset is
+ created.
+ </td>
+ </tr>
+ <tr>
+ <td align=center><code>2</code></td>
+ <td>Late allocation. Storage space for the entire dataset
+ should not be allocated until the dataset is written
+ to.
+ </td>
+ </tr>
+ <tr>
+ <td align=center><code>3</code></td>
+ <td>Incremental allocation. Storage space for the
+ dataset should not be allocated until the portion
+ of the dataset is written to. This is currently
+ used in conjunction with chunked data storage for
+ datasets.
+ </td>
+ </tr>
+ </table>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Fill Value Write Time</td>
+ <td>
+ <P>At the time that storage space for the dataset's raw data is
+ allocated, this value indicates whether the fill value should
+ be written to the raw data storage elements. The allowed values
+ are:
+ <table class=list>
+ <tr>
+ <th width="30%">Value</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>0</code></td>
+ <td>On allocation. The fill value is always written to
+ the raw data storage when the storage space is allocated.
+ </td>
+ </tr>
+ <tr>
+ <td align=center><code>1</code></td>
+ <td>Never. The fill value should never be written to
+ the raw data storage.
+ </td>
+ </tr>
+ <tr>
+ <td align=center><code>2</code></td>
+ <td>Fill value written if set by user. The fill value
+ will be written to the raw data storage when the storage
+ space is allocated only if the user explicitly set
+ the fill value. If the fill value is the library
+ default or is undefined, it will not be written to
+ the raw data storage.
+ </td>
+ </tr>
+ </table>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Fill Value Defined</td>
+ <td>
+ <P>This value indicates if a fill value is defined for this
+ dataset. If this value is 0, the fill value is undefined.
+ If this value is 1, a fill value is defined for this dataset.
+ For version 2 or later of the fill value message, this value
+ controls the presence of the Size field.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Size</td>
+ <td>
+ <P>This is the size of the Fill Value field in bytes. This field
+ is not present if the Version field is >1 and the Fill Value
+ Defined field is set to 0.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Fill Value</td>
+ <td>
+ <P>The fill value. The bytes of the fill value are interpreted
+ using the same datatype as for the dataset. This field is
+ not present if the Version field is >1 and the Fill Value
+ Defined field is set to 0.
+ </P>
+ </td>
+ </tr>
+ </table>
+ </div>
+ </P>
+
+<!--
+ <hr>
+ <h4><a name="CompactDataStorageMessage">Name: Data Storage - Compact</a></h4>
+
+ <b>Header Message Type:</b> 0x0006<br>
+ <b>Length:</b> varies<br>
+ <b>Status:</b> Optional, may not be repeated.<br>
+
+ <p>This message indicates that the data for the data object is
+ stored within the current HDF file by including the actual
+ data as the header data for this message. The data is
+ stored internally in
+ the <em>normal format</em>, i.e. in one chunk, uncompressed, etc.
+
+ <P>Note that one and only one of the <em>Data Storage</em> headers can be
+ stored for each data object.
+
+ <P><b>Format of Data:</b> The message data is actually composed
+ of dataset data, so the format will be determined by the dataset
+ format.
+-->
+
+ <hr>
+ <h4><a name="ReservedMessage_0006">Name: Reserved - Not Assigned Yet</a></h4>
+ <P class=item><B>Header Message Type:</B> 0x0006</P>
+ <P class=item><B>Length:</B> N/A</P>
+ <P class=item><B>Status:</B> N/A</P>
+ <P class=item><B>Format of Data:</B> N/A</P>
+
+ <P class=item><B>Purpose and Description:</B> This message type was skipped during
+ the initial specification of the file format and may be used in a
+ future expansion to the format.</P>
+
+ <hr>
+ <h4><a name="ExternalFileListMessage">Name: Data Storage -
+ External Data Files</a></h4>
+ <P class=item><B>Header Message Type:</B> 0x0007 </P>
+ <P class=item><B>Length:</B> varies</P>
+ <P class=item><B>Status:</B> Optional, may not be repeated.</P>
+
+ <P class=item><B>Purpose and Description:</B> The external object message
+ indicates that the data for an object is stored outside the HDF5
+ file. The filename of the object is stored as a Universal
+ Resource Location (URL) of the actual filename containing the
+ data. An external file list record also contains the byte offset
+ of the start of the data within the file and the amount of space
+ reserved in the file for that data.</P>
+
+ <P class=item><B>Format of Data:</B>
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ External File List Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan=3>Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan=2>Allocated Slots</td>
+ <td colspan=2>Used Slots</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Heap Address<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Slot Definitions...<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>
+ <P>The version number information is used for changes in the format of External File
+ List Message and is described here:
+ <table class=list>
+ <tr>
+ <th width="30%">Version</th>
+ <th align=left>Description</th>
+ </tr>
+ <tr>
+ <td align=center><code>0</code></td>
+ <td>Never used.
+ </tr>
+ <tr>
+ <td align=center><code>1</code></td>
+ <td>The current version used by the library.
+ </tr>
+ </table>
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Reserved</td>
+ <td>
+ <P>This field is reserved for future use.</P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Allocated Slots</td>
+ <td>
+ <P>The total number of slots allocated in the message. Its value must be at least as
+ large as the value contained in the Used Slots field. (The current library simply
+ uses the number of Used Slots for this message)</P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Used Slots</td>
+ <td>
+ <P>The number of initial slots which contains valid information.</P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Heap Address</td>
+ <td>
+ <P>This is the address of a local heap which contains the names for the external
+ files (The local heap information can be found in Disk Format Level 1D in this
+ document). The name at offset zero in the heap is always the empty string.</P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Slot Definitions</td>
+ <td>
+ <P>The slot definitions are stored in order according to the array addresses they
+ represent.</P>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ External File List Slot
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Name Offset(&lt;size&gt; bytes)<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>File Offset(&lt;size&gt; bytes)<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Size<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Name Offset(&lt;size&gt; bytes)</td>
+ <td>
+ <P>The byte offset within the local name heap for the name
+ of the file. File names are stored as a URL which has a
+ protocol name, a host name, a port number, and a file
+ name:
+ <code><em>protocol</em>:<em>port</em>//<em>host</em>/<em>file</em></code>.
+ If the protocol is omitted then "file:" is assumed. If
+ the port number is omitted then a default port for that
+ protocol is used. If both the protocol and the port
+ number are omitted then the colon can also be omitted. If
+ the double slash and host name are omitted then
+ "localhost" is assumed. The file name is the only
+ mandatory part, and if the leading slash is missing then
+ it is relative to the application's current working
+ directory (the use of relative names is not
+ recommended).</P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>File Offset(&lt;size&gt; bytes)</td>
+ <td>
+ <P>This is the byte offset to the start of the data in the
+ specified file. For files that contain data for a single
+ dataset this will usually be zero.</P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Size</td>
+ <td>
+ <P>This is the total number of bytes reserved in the
+ specified file for raw data storage. For a file that
+ contains exactly one complete dataset which is not
+ extendable, the size will usually be the exact size of the
+ dataset. However, by making the size larger one allows
+ HDF5 to extend the dataset. The size can be set to a value
+ larger than the entire file since HDF5 will read zeros
+ past the end of the file without failing.</P>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+ <hr>
+ <h4><a name="LayoutMessage">Name: Data Storage - Layout</a></h4>
+
+ <P class=item><B>Header Message Type:</B> 0x0008</P>
+ <P class=item><B>Length:</B> varies</P>
+ <P class=item><B>Status:</B> Required for datasets, may not be repeated.</P>
+
+ <P class=item><B>Purpose and Description:</B> Data layout describes how the
+ elements of a multi-dimensional array are arranged in the linear
+ address space of the file. Three types of data layout are
+ supported:
+
+ <ol>
+ <li>Contiguous: The array can be stored in one contiguous area of the file.
+ The layout requires that the size of the array be constant and
+ does not permit chunking, compression, checksums, encryption,
+ etc. The message stores the total size of the array and the
+ offset of an element from the beginning of the storage area is
+ computed as in C.
+
+ <li>Chunked: The array domain can be regularly decomposed into chunks and
+ each chunk is allocated separately. This layout supports
+ arbitrary element traversals, compression, encryption, and
+ checksums, and the chunks can be distributed across external
+ raw data files (these features are described in other
+ messages). The message stores the size of a chunk instead of
+ the size of the entire array; the size of the entire array can
+ be calculated by traversing the B-tree that stores the chunk
+ addresses.
+
+ <li>Compact: The array can be stored in one contiguous block, as part of
+ this object header message (this is called "compact" storage below).
+ </ol>
+
+ <P class=item><B>Format of Data:</B>
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Data Layout Message (Versions 1 and 2)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Dimensionality</td>
+ <td>Layout Class</td>
+ <td>Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Address<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Dimension 0 (4-bytes)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Dimension 1 (4-bytes)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>...</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Dataset Element Size <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Compact Data Size (4-bytes)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Compact Data...<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>
+ <P>The version number information is used for changes in the format of the data
+ layout message and is described here:</P>
+ <table class=list>
+ <tr>
+ <th width="30%">Version</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>0</code></td>
+ <td>Never used.</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>1</code></td>
+ <td>Used by version 1.4 and before of the library to encode layout information.
+ Data space is always allocated when the data set is created.</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>2</code></td>
+ <td>Used by version 1.6.x of the library to encode layout information.
+ Data space is allocated only when it is necessary.</td>
+ </tr>
+ </table>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Dimensionality</td>
+ <td><P>An array has a fixed dimensionality. This field
+ specifies the number of dimension size fields later in the
+ message.</P></td>
+ </tr>
+
+ <tr>
+ <td>Layout Class</td>
+ <td><P>The layout class specifies how the other fields of the
+ layout message are to be interpreted. A value of one
+ indicates contiguous storage, a value of two indicates chunked storage,
+ while a value of zero indicates compact storage. Other values will be defined
+ in the future.</P></td>
+ </tr>
+
+ <tr>
+ <td>Address</td>
+ <td><P>For contiguous storage, this is the address of the first
+ byte of storage. For chunked storage this is the address
+ of the B-tree that is used to look up the addresses of the
+ chunks. This field is not present for compact storage.
+ If the version for this message is set to 2, the address
+ may have the "undefined address" value, to indicate that
+ storage has not yet been allocated for this array.</P></td>
+ </tr>
+
+ <tr>
+ <td>Dimensions</td>
+ <td><P>For contiguous and compact storage the dimensions define
+ the entire size of the array while for chunked storage they define
+ the size of a single chunk. In all cases, they are in units of
+ array elements (not bytes). The first dimension stored in the list
+ of dimensions is the slowest changing dimension and the last
+ dimension stored is the fastest changing dimension.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Dataset Element Size</td>
+ <td><P>The size of a dataset element, in bytes. This field is only
+ present for chunked storage.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Compact Data Size</td>
+ <td><P>This field is only present for compact data storage.
+ It contains the size of the raw data for the dataset array.</P></td>
+
+ <tr>
+ <td>Compact Data</td>
+ <td><P>This field is only present for compact data storage.
+ It contains the raw data for the dataset array.</P></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <P>Version 3 of this message re-structured the format into specific
+ properties that are required for each layout class.
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ <B>Data Layout Message (Version 3)</B>
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Layout Class</td>
+ <td colspan=2 bgcolor=#DDDDDD>&nbsp;</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Properties<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>
+ <P>The version number information is used for changes in the format of layout message
+ and is described here:</P>
+ <table class=list>
+ <tr>
+ <th width="30%">Version</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>3</code></td>
+ <td>Used by the version 1.6.3 and later of the library to store properties
+ for each layout class.</td>
+ </tr>
+ </table>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Layout Class</td>
+ <td><P>The layout class specifies how the other fields of the layout message are to be
+ interpreted. A value of one indicates contiguous storage, a value of two
+ indicates chunked storage, while a value of zero indicates compact storage.</P></td>
+ </tr>
+
+ <tr>
+ <td>Properties</td>
+ <td><P>This variable-sized field encodes information specific to each
+ layout class and is described below. If there is no property
+ information specified for a layout class, the size of this field
+ is zero bytes.</P></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <P>Class-specific information for compact layout (Class 0): (Note: The dimensionality information
+ is in the Dataspace message)
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Property Descriptions
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=2>Size</td>
+ <td colspan=2 bgcolor=#DDDDDD>&nbsp;</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Raw Data...<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Size</td>
+ <td><P>This field contains the size of the raw data for the dataset array.</P></td>
+ </tr>
+
+ <tr>
+ <td>Raw Data</td>
+ <td><P>This field contains the raw data for the dataset array.</P></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <P>Class-specific information for contiguous layout (Class 1): (Note: The dimensionality information
+ is in the Dataspace message)
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Property Descriptions
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Address<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Size<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Address</td>
+ <td><P>This is the address of the first byte of raw data storage.
+ The address may have the "undefined address" value, to indicate
+ that storage has not yet been allocated for this array.</P></td>
+ </tr>
+
+ <tr>
+ <td>Size</td>
+ <td><P>This field contains the size allocated to store the raw data.</P></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <P>Class-specific information for chunked layout (Class 2):
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Property Descriptions
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Dimensionality</td>
+ <td colspan=3 bgcolor=#DDDDDD>&nbsp;</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Address<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Dimension 0 (4-bytes)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Dimension 1 (4-bytes)</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>...</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Dataset Element Size</td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Dimensionality</td>
+ <td><P>A chunk has a fixed dimensionality. This field specifies
+ the number of dimension size fields later in the message.</P></td>
+ </tr>
+
+ <tr>
+ <td>Address</td>
+ <td><P>This is the address of the B-tree that is used to look up the addresses of the
+ chunks. The address may have the "undefined address" value, to indicate that
+ storage has not yet been allocated for this array.</P></td>
+ </tr>
+
+ <tr>
+ <td>Dimensions</td>
+ <td><P>These values define the dimension size of a single chunk, in
+ units of array elements (not bytes). The first dimension stored in
+ the list of dimensions is the slowest changing dimension and the
+ last dimension stored is the fastest changing dimension.
+ </P>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Dataset Element Size</td>
+ <td><P>The size of a dataset element, in bytes.
+ </P>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <hr>
+ <h4><a name="ReservedMessage_0009">Name: Reserved - Not Assigned Yet</a></h4>
+ <P class=item><B>Header Message Type:</B> 0x0009</P>
+ <P class=item><B>Length:</B> N/A</P>
+ <P class=item><B>Status:</B> N/A</P>
+ <P class=item><B>Format of Data:</B> N/A</P>
+
+ <P class=item><B>Purpose and Description:</B> This message type was skipped during the initial
+ specification of the file format and may be used in a future expansion to the format.
+
+ <hr>
+ <h4><a name="ReservedMessage_000A">Name: Reserved - Not Assigned Yet</a></h4>
+ <P class=item><B>Header Message Type:</B> 0x0009</P>
+ <P class=item><B>Length:</B> N/A</P>
+ <P class=item><B>Status:</B> N/A</P>
+ <P class=item><B>Format of Data:</B> N/A</P>
+
+ <P class=item><B>Purpose and Description:</B> This message type was skipped during the initial
+ specification of the file format and may be used in a future expansion to the format.
+
+ <hr>
+ <h4><a name="FilterMessage">Name: Data Storage - Filter Pipeline</a></h4>
+ <P class=item><B>Header Message Type:</B> 0x000B</P>
+ <P class=item><B>Length:</B> varies</P>
+ <P class=item><B>Status:</B> Optional, may not be repeated.</P>
+
+ <P class=item><B>Description:</B> This message describes the
+ filter pipeline which should be applied to the data stream by
+ providing filter identification numbers, flags, a name, and
+ client data.</P>
+
+ <P class=item><B>Format of Data:</B>
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Filter Pipeline Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Number of Filters</td>
+ <td colspan=2>Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Filter List<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td><P>The version number for this message. This document
+ describes version 1.</P></td>
+ </tr>
+
+ <tr>
+ <td>Number of Filters</td>
+ <td><P>The total number of filters described by this
+ message. The maximum possible number of filters in a
+ message is 32.</P></td>
+ </tr>
+
+ <tr>
+ <td>Filter List</td>
+ <td><P>A description of each filter. A filter description
+ appears in the next table.</P></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Filter Description
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=2>Filter Identification</td>
+ <td colspan=2>Name Length</td>
+ </tr>
+
+ <tr>
+ <td colspan=2>Flags</td>
+ <td colspan=2>Number of Values for Client Data</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Name<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Client Data<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Padding</td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Filter Identification</td>
+ <td>
+ <p>
+ This value, often referred to as a filter identifier,
+ is designed to be a unique identifier for the filter.
+ Values from zero through 32,767 are reserved for filters
+ supported by The HDF Group in the HDF5 library and for
+ filters requested and supported by third parties.
+ Filters supported by The HDF Group are documented immediately
+ below. Information on 3rd-party filters can be found at
+ <a href="/services/contributions.html#filters">
+ <code>https://support.hdfgroup.org/services/contributions.html#filters</code></a>.
+ <a href="#Footnote1Change"><sup><small>1</small></sup></a>
+ <p>
+ To request a filter identifier, please contact
+ The HDF Group&rsquo;s Help Desk at
+ <img src="Graphics/help.png" valign="center" height=14>.
+ You will be asked to provide the following information:
+ <ol>
+ <li>Contact information for the developer requesting the
+ new identifier
+ <li>A short description of the new filter
+ <li>Links to any relevant information, including licensing
+ information
+ </ol>
+ <p>
+ Values from 32768 to 65535 are reserved for non-distributed uses
+ (for example, internal company usage) or for application usage
+ when testing a feature. The HDF Group does not track or document
+ the use of the filters with identifiers from this range.
+
+ <p>
+ The filters currently in library version 1.6.5 are
+ listed below:
+ <table class=list>
+ <tr>
+ <th width="30%">Identification</th>
+ <th align=left>Name</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>1</code></td>
+ <td>deflate</td>
+ <td>GZIP deflate compression</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>2</code></td>
+ <td>shuffle</td>
+ <td>Data element shuffling</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>3</code></td>
+ <td>fletcher32</td>
+ <td>Fletcher32 checksum</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>4</code></td>
+ <td>szip</td>
+ <td>SZIP compression</td>
+ </tr>
+ </table>
+ </P></td>
+ </tr>
+
+ <tr>
+ <td>Name Length</td>
+ <td><P>Each filter has an optional null-terminated ASCII name
+ and this field holds the length of the name including the
+ null termination padded with nulls to be a multiple of
+ eight. If the filter has no name then a value of zero is
+ stored in this field.</P></td>
+ </tr>
+
+ <tr>
+ <td>Flags</td>
+ <td><P>The flags indicate certain properties for a filter. The
+ bit values defined so far are:</P>
+ <table class=list>
+ <tr>
+ <th width="30%">Value</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>bit 1</code></td>
+ <td>If set then the filter is an optional filter.
+ During output, if an optional filter fails it will be
+ silently removed from the pipeline.</td>
+ </tr>
+ </table>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Client Data Number of Values</td>
+ <td><P>Each filter can store a few integer values to control
+ how the filter operates. The number of entries in the
+ Client Data array is stored in this field.</P></td>
+ </tr>
+
+ <tr>
+ <td>Name</td>
+ <td><P>If the Name Length field is non-zero then it will
+ contain the size of this field, a multiple of eight. This
+ field contains a null-terminated, ASCII character
+ string to serve as a comment/name for the filter.</P></td>
+ </tr>
+
+ <tr>
+ <td>Client Data</td>
+ <td><P>This is an array of four-byte integers which will be
+ passed to the filter function. The Client Data Number of
+ Values determines the number of elements in the array.</P></td>
+ </tr>
+
+ <tr>
+ <td>Padding</td>
+ <td><P>Four bytes of zeros are added to the message at this
+ point if the Client Data Number of Values field contains
+ an odd number.</P></td>
+ </tr>
+ </table>
+ </div>
+ <p>
+ <hr align="left" width="50">
+ <a name="Footnote1Change"><sup>1</sup></a>If you are reading
+ an earlier version of this document, this link may have changed.
+ If the link does not work, use the latest version of this document
+ on <a href="https://support.hdfgroup.org">The HDF Group</a>&rsquo;s website,
+ <a href="/HDF5/doc/H5.format.html">
+ <code>https://support.hdfgroup.org/HDF5/doc/H5.format.html</code></a>;
+ the link there will always be correct.
+ <small><a href="#FilterMessage">(Return)</a>
+ </P>
+
+ <hr>
+ <h4><a name="AttributeMessage">Name: Attribute</a></h4>
+ <P class=item><B>Header Message Type:</B> 0x000C
+ <P class=item><B>Length:</B> varies
+ <P class=item><B>Status:</B> Optional, may be repeated.
+
+ <P class=item><B>Description:</B> The <em>Attribute</em>
+ message is used to list objects in the HDF file which are used
+ as attributes, or "metadata" about the current object. An
+ attribute is a small dataset; it has a name, a datatype, a data
+ space, and raw data. Since attributes are stored in the object
+ header they must be relatively small (<64KB) and can be
+ associated with any type of object which has an object header
+ (groups, datasets, named types and spaces, etc.).
+
+ <P class=item2>Note: Attributes on an object must have unique names. (The HDF5 library
+ currently enforces this by causing the creation of an attribute with
+ a duplicate name to fail). Attributes on different objects may have the
+ same name, however.
+
+ <P class=item><B>Format of Data:</B>
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Attribute Message (Version 1)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Reserved</td>
+ <td colspan=2>Name Size</td>
+ </tr>
+
+ <tr>
+ <td colspan=2>Datatype Size</td>
+ <td colspan=2>Dataspace Size</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Name<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Datatype<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Dataspace<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Data<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td><P>The version number information is used for changes in the format of the
+ attribute message and is described here:</P>
+ <table class=list>
+ <tr>
+ <th width="30%">Version</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>0</code></td>
+ <td>Never used.</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>1</code></td>
+ <td>Used by the library before version 1.6 to encode attribute message.
+ This version does not support shared data type.</td>
+ </tr>
+ </table>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Reserved</td>
+ <td><P>This field is reserved for later use and is set to
+ zero.</P></td>
+ </tr>
+
+ <tr>
+ <td>Name Size</td>
+ <td><P>The length of the attribute name in bytes including the
+ null terminator. Note that the Name field below may
+ contain additional padding not represented by this
+ field.</P></td>
+ </tr>
+
+ <tr>
+ <td>Datatype Size</td>
+ <td><P>The length of the datatype description in the Datatype
+ field below. Note that the Datatype field may contain
+ additional padding not represented by this field.</P></td>
+ </tr>
+
+ <tr>
+ <td>Dataspace Size</td>
+ <td><P>The length of the dataspace description in the Dataspace
+ field below. Note that the Dataspace field may contain
+ additional padding not represented by this field.</P></td>
+ </tr>
+
+ <tr>
+ <td>Name</td>
+ <td><P>The null-terminated attribute name. This field is
+ padded with additional null characters to make it a
+ multiple of eight bytes.</P></td>
+ </tr>
+
+ <tr>
+ <td>Datatype</td>
+ <td><P>The datatype description follows the same format as
+ described for the datatype object header message. This
+ field is padded with additional zero bytes to make it a
+ multiple of eight bytes.</P></td>
+ </tr>
+
+ <tr>
+ <td>Dataspace</td>
+ <td><P>The dataspace description follows the same format as
+ described for the dataspace object header message. This
+ field is padded with additional zero bytes to make it a
+ multiple of eight bytes.</P></td>
+ </tr>
+
+ <tr>
+ <td>Data</td>
+ <td><P>The raw data for the attribute. The size is determined
+ from the datatype and dataspace descriptions. This
+ field is <em>not</em> padded with additional bytes.</P></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Attribute Message (Version 2)
+ </caption>
+
+ <tr align=center>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Flag</td>
+ <td colspan=2>Name Size</td>
+ </tr>
+
+ <tr>
+ <td colspan=2>Type Size</td>
+ <td colspan=2>Space Size</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Name<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Type<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Space<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Data<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td><P>The version number information is used for changes in the format of the
+ attribute message and is described here:</P>
+ <table class=list width="90%">
+ <tr>
+ <th width="30%">Version</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>2</code></td>
+ <td>Used by the library of version 1.6.x and after to encode attribute message.
+ This version supports shared data type. The fields of name, type, and space
+ are not padded with additional bytes of zero.</td>
+ </tr>
+ </table>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Flag</td>
+ <td><P>This field indicates whether the data type of this attribute is shared:</P>
+ <table class=list width="90%">
+ <tr>
+ <th width="30%">Value</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>0</code></td>
+ <td>Datatype is <em>not</em> shared.</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>1</code></td>
+ <td>Datatype is shared.</td>
+ </tr>
+ </table>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Name Size</td>
+ <td><P>The length of the attribute name in bytes including the
+ null terminator.</P></td>
+ </tr>
+
+ <tr>
+ <td>Datatype Size</td>
+ <td><P>The length of the datatype description in the Datatype
+ field below.</P></td>
+ </tr>
+
+ <tr>
+ <td>Dataspace Size</td>
+ <td><P>The length of the dataspace description in the Dataspace
+ field below.</P></td>
+ </tr>
+
+ <tr>
+ <td>Name</td>
+ <td><P>The null-terminated attribute name. This field is <em>not</em>
+ padded with additional bytes.</P></td>
+ </tr>
+
+ <tr>
+ <td>Datatype</td>
+ <td><P>The datatype description follows the same format as
+ described for the datatype object header message. This
+ field is <em>not</em> padded with additional bytes.</P></td>
+ </tr>
+
+ <tr>
+ <td>Dataspace</td>
+ <td><P>The dataspace description follows the same format as
+ described for the dataspace object header message. This
+ field is <em>not</em> padded with additional bytes.</P></td>
+ </tr>
+
+ <tr>
+ <td>Data</td>
+ <td><P>The raw data for the attribute. The size is determined
+ from the datatype and dataspace descriptions. This
+ field is <em>not</em> padded with additional zero
+ bytes.</P></td>
+ </tr>
+ </table>
+ </div>
+
+ <hr>
+ <h4><a name="CommentMessage">Name: Object Comment</a></h4>
+
+ <P class=item><B>Header Message Type:</B> 0x000D</P>
+ <P class=item><B>Length:</B> varies</P>
+ <P class=item><B>Status:</B> Optional, may not be repeated.</P>
+
+ <P class=item><B>Description:</B> The object comment is
+ designed to be a short description of an object. An object comment
+ is a sequence of non-zero (<code>\0</code>) ASCII characters with no other
+ formatting included by the library.</P>
+
+ <P class=item><B>Format of Data:</B>
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Name Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Comment<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Name</td>
+ <td>A null terminated ASCII character string.</td>
+ </tr>
+ </table>
+ </div>
+
+ <hr>
+ <h4><a name="OldModifiedMessage">Name: Object Modification Date &amp; Time (Old)</a></h4>
+
+ <P class=item><B>Header Message Type:</B> 0x000E</P>
+ <P class=item><B>Length:</B> fixed</P>
+ <P class=item><B>Status:</B> Optional, may not be repeated.</P>
+
+ <P class=item><B>Description:</B> The object modification date
+ and time is a timestamp which indicates (using ISO-8601 date and
+ time format) the last modification of an object. The time is
+ updated when any object header message changes according to the
+ system clock where the change was posted.
+
+ <br><br>This modification time message is deprecated in favor of the "new"
+ modification time message (Message Type 0x0012) and is no longer written
+ to the file in versions of the HDF5 library after the 1.6.0 version.
+ </P>
+
+ <P class=item><B>Format of Data:</B>
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Modification Time Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=4>Year</td>
+ </tr>
+
+ <tr>
+ <td colspan=2>Month</td>
+ <td colspan=2>Day of Month</td>
+ </tr>
+
+ <tr>
+ <td colspan=2>Hour</td>
+ <td colspan=2>Minute</td>
+ </tr>
+
+ <tr>
+ <td colspan=2>Second</td>
+ <td colspan=2>Reserved</td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Year</td>
+ <td><P>The four-digit year as an ASCII string. For example,
+ <code>1998</code>. All fields of this message should be interpreted
+ as coordinated universal time (UTC)</P></td>
+ </tr>
+
+ <tr>
+ <td>Month</td>
+ <td><P>The month number as a two digit ASCII string where
+ January is <code>01</code> and December is <code>12</code>.</P></td>
+ </tr>
+
+ <tr>
+ <td>Day of Month</td>
+ <td><P>The day number within the month as a two digit ASCII
+ string. The first day of the month is <code>01</code>.</P></td>
+ </tr>
+
+ <tr>
+ <td>Hour</td>
+ <td><P>The hour of the day as a two digit ASCII string where
+ midnight is <code>00</code> and 11:00pm is <code>23</code>.</P></td>
+ </tr>
+
+ <tr>
+ <td>Minute</td>
+ <td><P>The minute of the hour as a two digit ASCII string where
+ the first minute of the hour is <code>00</code> and
+ the last is <code>59</code>.</P></td>
+ </tr>
+
+ <tr>
+ <td>Second</td>
+ <td><P>The second of the minute as a two digit ASCII string
+ where the first second of the minute is <code>00</code>
+ and the last is <code>59</code>.</P></td>
+ </tr>
+
+ <tr>
+ <td>Reserved</td>
+ <td><P>This field is reserved and should always be zero.</P></td>
+ </tr>
+ </table>
+ </div>
+
+ <hr>
+ <h4><a name="SharedMessage">Name: Shared Object Message</a></h4>
+ <P class=item><B>Header Message Type:</B> 0x000F</P>
+ <P class=item><B>Length:</B> Fixed</P>
+ <P class=item><B>Status:</B> Optional, may be repeated.</P>
+
+ <P class=item><B>Description:</B> A constant message can be shared among
+ several object headers. A <em>Shared Object</em> Message contains the address of
+ the object message to be shared. Care must be exercised to prevent cycles when a
+ message of one object header points to a message in some other object header.
+ Starting from Version 2 of the Shared Object Message, the <em>Flags</em>
+ field becomes unused.
+ </P>
+
+ <P class=item><B>Format of Data:</B>
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Shared Object Message (Version 1)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</td>
+ <th width="25%">byte</td>
+ <th width="25%">byte</td>
+ <th width="25%">byte</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td colspan=2>Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Pointer<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td><P>The version number is used when there are changes in the format
+ of a shared object message and is described here:</P>
+ <table class=list>
+ <tr>
+ <th width="30%">Version</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>0</code></td>
+ <td>Never used.</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>1</code></td>
+ <td>Used by the library before version 1.6.1. In this version,
+ the Flags field is used to indicate whether the actual message is
+ stored in the global heap (never implemented). The Pointer field
+ either contains the the header message address in the global heap
+ (never implemented) or the address of the shared object header.</td>
+ </tr>
+ </table>
+ </tr>
+
+ <tr>
+ <td>Flags</td>
+ <td><P>The Shared Message message points to a message which is
+ shared among multiple object headers. The Flags field
+ describes the type of sharing:</P>
+ <table class=list>
+ <tr>
+ <th width="30%">Bit</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>0</code></td>
+ <td>If this bit is clear then the actual message is the
+ first message in some other object header; otherwise
+ the actual message is stored in the global heap (never
+ implemented).</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>2-7</code></td>
+ <td>Reserved (always zero)</td>
+ </tr>
+ </table>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Pointer</td>
+ <td><P>The address of the object header
+ containing the message to be shared.</P></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Shared Object Message (Version 2)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</td>
+ <th width="25%">byte</td>
+ <th width="25%">byte</td>
+ <th width="25%">byte</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td colspan=2 bgcolor=#DDDDDD>&nbsp;</td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Pointer<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td><P>The version number is used when there are changes in the format
+ of a shared object message and is described here:</P>
+ <table class=list>
+ <tr>
+ <th width="30%">Version</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>2</code></td>
+ <td>Used by the library of version 1.6.1 and after. In this version,
+ The Flags field is not used and the Pointer field contains the address
+ of the object header containing the message to be shared. </td>
+ </tr>
+ </table>
+ </tr>
+
+ <tr>
+ <td>Flags</td>
+ <td><P>Unused.</P></td>
+ </tr>
+
+ <tr>
+ <td>Pointer</td>
+ <td><P>The address of the object header
+ containing the message to be shared.</P></td>
+ </tr>
+ </table>
+ </div>
+
+
+ <hr>
+ <h4><a name="ContinuationMessage">Name: Object Header Continuation</a></h4>
+ <P class=item><B>Header Message Type:</B> 0x0010</P>
+ <P class=item><B>Length:</B> fixed</P>
+ <P class=item><B>Status:</B> Optional, may be repeated.</P>
+ <P class=item><B>Description:</B> The object header continuation is the location
+ in the file of more header messages for the current data object. This can be
+ used when header blocks become too large or are likely to change over time.</P>
+
+ <P class=item><B>Format of Data:</B>
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Object Header Continuation Message
+ </caption>
+
+ <tr>
+ <th width=25%>byte</th>
+ <th width=25%>byte</th>
+ <th width=25%>byte</th>
+ <th width=25%>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Offset<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Length<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width=30%>Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Offset</td>
+ <td><P>This value is the offset in bytes from the beginning of the file where the
+ header continuation information is located.</P></td>
+ </tr>
+
+ <tr>
+ <td>Length</td>
+ <td><P>This value is the length in bytes of the header continuation information in
+ the file.</P></td>
+ </tr>
+ </table>
+ </div>
+
+ <hr>
+ <h4><a name="SymbolTableMessage">Name: Group Message</a></h4>
+ <P class=item><B>Header Message Type:</B> 0x0011</P>
+ <P class=item><B>Length:</B> fixed</P>
+ <P class=item><B>Status:</B> Required for groups, may not be repeated.</P>
+ <P class=item><B>Description:</B> Each group has a B-tree and a
+ name heap which are pointed to by this message.</P>
+ <P class=item><B>Format of data:</B>
+
+ <br>
+ <div align=center>
+ <table class=format>
+ <caption>
+ <B>Group Message</B>
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>B-tree Address<br><br></td>
+ </tr>
+
+ <tr>
+ <td colspan=4><br>Heap Address<br><br></td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width=30%>Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>B-tree Address</td>
+ <td><P>This value is the offset in bytes from the beginning of the file
+ where the B-tree is located.</P></td>
+ </tr>
+
+ <tr>
+ <td>Heap Address</td>
+ <td><P>This value is the offset in bytes from the beginning of the file
+ where the group name heap is located.</P></td>
+ </tr>
+ </table>
+ </div>
+
+ <hr>
+ <h4><a name="ModifiedMessage">Name: Object Modification Date &amp; Time</a></h4>
+
+ <P class=item><B>Header Message Type:</B> 0x0012 </P>
+ <P class=item><B>Length:</B> Fixed </P>
+ <P class=item><B>Status:</B> Optional, may not be repeated. </P>
+
+ <P class=item><B>Description:</B> The object modification date
+ and time is a timestamp which indicates the last modification of an object.
+ The time is updated when any object header message changes according to the
+ system clock where the change was posted.
+ </P>
+
+ <P class=item><B>Format of Data:</B>
+ <div align=center>
+ <table class=format>
+ <caption>
+ Modification Time Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan=3>Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan=4>Seconds After Epoch</td>
+ </tr>
+ </table>
+ </div>
+
+ <br>
+ <div align=center>
+ <table class=desc>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td><P>The version number is used for changes in the format of Object Modification Time
+ and is described here:</P>
+ <table class=list>
+ <tr>
+ <th width="30%">Version</th>
+ <th align=left>Description</th>
+ </tr>
+
+ <tr>
+ <td align=center><code>0</code></td>
+ <td>Never used.</td>
+ </tr>
+
+ <tr>
+ <td align=center><code>1</code></td>
+ <td>Used by Version 1.6.1 and after of the library to encode time. In
+ this version, the time is the seconds after Epoch.</td>
+ </tr>
+ </table>
+ </td>
+ </tr>
+
+ <tr>
+ <td>Reserved</td>
+ <td><P>This field is reserved and should always be zero.</P></td>
+ </tr>
+
+ <tr>
+ <td>Seconds After Epoch</td>
+ <td><P>The number of seconds since 0 hours, 0 minutes, 0 seconds,
+ January 1, 1970, Coordinated Universal Time.</P></td>
+ </tr>
+ </table>
+ </div>
+
+<hr>
+<h3><a name="DataStorage">Disk Format: Level 2b - Data Object Data Storage</a></h3>
+<P>The data for an object is stored separately from the header
+information in the file and may not actually be located in the HDF5 file
+itself if the header indicates that the data is stored externally. The
+information for each record in the object is stored according to the
+dimensionality of the object (indicated in the dimensionality header message).
+Multi-dimensional data is stored in C order [same as current scheme], i.e. the
+"last" dimension changes fastest.
+<P>Data whose elements are composed of simple number-types are stored in
+native-endian IEEE format, unless they are specifically defined as being stored
+in a different machine format with the architecture-type information from the
+number-type header message. This means that each architecture will need to
+[potentially] byte-swap data values into the internal representation for that
+particular machine.
+<P> Data with a variable-length datatype is stored in the global heap
+of the HDF5 file. Global heap identifiers are stored in the
+data object storage.
+<P>Data whose elements are composed of pointer number-types are stored in several
+different ways depending on the particular pointer type involved. Simple
+pointers are just stored as the dataset offset of the object being pointed to with the
+size of the pointer being the same number of bytes as offsets in the file.
+Dataset region references are stored as a heap-ID which points to the following
+information within the file-heap: an offset of the object pointed to, number-type
+information (same format as header message), dimensionality information (same
+format as header message), sub-set start and end information (i.e. a coordinate
+location for each), and field start and end names (i.e. a [pointer to the]
+string indicating the first field included and a [pointer to the] string name
+for the last field).
+
+<P>Data of a compound datatype is stored as a contiguous stream of the items
+in the structure, with each item formatted according to its datatype.</p>
+
+<hr>
+<h3><a name="Appendix">Appendix</a></h3>
+<P>Definitions of various terms used in this document.
+</P>
+<P>The <A name="UndefinedAddress">"undefined address"</A> for a file is a
+file address with all bits set, i.e. <code>0xffff...ff</code>.
+<P>The <A name="UnlimitedDim">"unlimited size"</A> for a size is a
+value with all bits set, i.e. <code>0xffff...ff</code>.
+
+</body>
+</html>
diff --git a/doxygen/examples/H5.format.2.0.html b/doxygen/examples/H5.format.2.0.html
new file mode 100644
index 0000000..3653489
--- /dev/null
+++ b/doxygen/examples/H5.format.2.0.html
@@ -0,0 +1,14902 @@
+<!DOCTYPE HTML>
+<html>
+ <head>
+ <title>
+ HDF5 File Format Specification Version 2.0
+ </title>
+
+<style>
+h1 { display: block;
+ margin-top: 24px;
+ margin-bottom: 24px;
+ margin-left: 0px;
+ margin-right: 0px;
+ text-indent: 0px;
+ }
+
+h2 { display: block;
+ margin-top: 8x;
+ margin-bottom: 8px;
+ margin-left: 0px;
+ margin-right: 0px;
+ text-indent: 0px;
+ }
+<!-- A horizontal rule (<hr />) should be placed on the line above
+each h2 tag. The h2 tags are used on the main sections along with
+the hr tags. -->
+
+h3 { display: block;
+ margin-top: 8px;
+ margin-bottom: 8px;
+ margin-left: 0px;
+ margin-right: 0px;
+ text-indent: 0px;
+ }
+
+h4 { display: block;
+ margin-top: 8px;
+ margin-bottom: 8px;
+ margin-left: 0px;
+ margin-right: 0px;
+ text-indent: 0px;
+ }
+
+p { display: block;
+ margin-top: 8px;
+ margin-bottom: 8px;
+ margin-left: 0px;
+ margin-right: 0px;
+ text-indent: 0px;
+ }
+<!--
+p.item { margin-left: 2em;
+ text-indent: -2em
+ } -->
+<!-- p.item2 { margin-left: 2em; text-indent: 2em} -->
+
+table.format { border:solid;
+ border-collapse:collapse;
+ caption-side:top;
+ text-align:center;
+ width:80%;
+ }
+table.format th { border:ridge;
+ padding:4px;
+ width:25%;
+ }
+table.format td { border:ridge;
+ padding:4px;
+ }
+table.format caption { font-weight:bold;
+ font-size:larger;
+ }
+
+table.note {border:none;
+ text-align:right;
+ width:80%;
+ }
+
+table.desc { border:solid;
+ border-collapse:collapse;
+ caption-size:top;
+ text-align:left;
+ width:80%;
+ }
+table.desc tr { vertical-align:top;
+ }
+table.desc th { border-style:ridge;
+ font-size:larger;
+ padding:4px;
+ <!-- text-decoration:underline; -->
+ }
+table.desc td { border-style:ridge;
+ <!-- padding: 4px; -->
+ vertical-align:text-top;
+ }
+table.desc caption { font-weight:bold;
+ font-size:larger;
+ }
+
+table.list { border:none;
+ width:100%
+ }
+table.list tr { vertical-align:text-top;
+ }
+table.list th { border:none;
+ text-decoration:underline;
+ vertical-align:text-top;
+ }
+table.list td { border:none;
+ vertical-align:text-top;
+ }
+
+table.msgdesc { border:none;
+ text-align:left;
+ width: 80%
+ }
+table.msgdesc tr { vertical-align:text-top;
+ border-spacing:0;
+ padding:0; }
+table.msgdesc th { border:none;
+ text-decoration:underline;
+ vertical-align:text-top; }
+table.msgdesc td { border:none;
+ vertical-align:text-top;
+ }
+
+table.list80 { border:none;
+ width:80%
+ }
+table.list80 tr { vertical-align:text-top;
+ }
+table.list80 th { border:none;
+ text-decoration:underline;
+ vertical-align:text-top;
+ }
+table.list80 td { border:none;
+ vertical-align:text-top;
+ }
+
+table.glossary { border:none;
+ text-align:left;
+ width: 80%
+ }
+table.glossary tr { vertical-align:text-top;
+ border-spacing:0;
+ padding:0; }
+table.glossary th { border:none;
+ text-align:left;
+ text-decoration:underline;
+ vertical-align:text-top; }
+table.glossary td { border:none;
+ text-align:left;
+ vertical-align:text-top;
+ }
+
+div { page-break-inside:avoid;
+ page-break-after:auto
+ }
+
+</style>
+
+ <center>
+ <table border="0" width="90%">
+ <tr>
+ <td valign="top">
+ <ol type="I">
+ <li><a href="#Intro">Introduction</a></li>
+ <font size="-1">
+ <ol type="A">
+ <li><a href="#ThisDocument">This Document</a></li>
+ <li><a href="#ChangesForHdf5_1_10">Changes for HDF5 1.10</a></li>
+ </ol>
+ </font>
+
+ <li><a href="#FileMetaData">Disk Format: Level 0 - File Metadata</a></li>
+ <font size="-1">
+ <ol type="A">
+ <li><a href="#Superblock">Disk Format: Level 0A - Format Signature and Superblock</a></li>
+ <li><a href="#DriverInfo">Disk Format: Level 0B - File Driver Info</a></li>
+ <li><a href="#SuperblockExt">Disk Format: Level 0C - Superblock Extension</a></li>
+ </ol>
+ </font>
+ <li><a href="#FileInfra">Disk Format: Level 1 - File Infrastructure</a></li>
+ <font size="-1">
+ <ol type="A">
+ <li><a href="#Btrees">Disk Format: Level 1A - B-trees and B-tree
+ Nodes</a></li>
+ <ol type="1">
+ <li><a href="#V1Btrees">Disk Format: Level 1A1 - Version 1
+ B-trees (B-link Trees)</a></li>
+ <li><a href="#V2Btrees">Disk Format: Level 1A2 - Version 2
+ B-trees</a></li>
+ </ol>
+ <li><a href="#SymbolTable">Disk Format: Level 1B - Group Symbol Table Nodes</a></li>
+ <li><a href="#SymbolTableEntry">Disk Format: Level 1C - Symbol Table Entry</a></li>
+ <li><a href="#LocalHeap">Disk Format: Level 1D - Local Heaps</a></li>
+ <li><a href="#GlobalHeap">Disk Format: Level 1E - Global Heap</a></li>
+ <li><a href="#FractalHeap">Disk Format: Level 1F - Fractal Heap</a></li>
+ <li><a href="#FreeSpaceManager">Disk Format: Level 1G - Free-space Manager</a></li>
+ <li><a href="#SOHMTable">Disk Format: Level 1H - Shared Object Header Message Table</a></li>
+ </ol>
+ </font>
+ <li><a href="#DataObject">Disk Format: Level 2 - Data Objects</a></li>
+ <font size="-1">
+ <ol type="A">
+ <li><a href="#ObjectHeader">Disk Format: Level 2A - Data Object Headers</a></li>
+ <ol type="1">
+ <li><a href="#ObjectHeaderPrefix">Disk Format: Level 2A1 - Data Object Header Prefix</a></li>
+ <ol type="a">
+ <li><a href="#V1ObjectHeaderPrefix">Version 1 Data Object Header Prefix</a></li>
+ <li><a href="#V2ObjectHeaderPrefix">Version 2 Data Object Header Prefix</a></li>
+ </ol>
+ <li><a href="#ObjectHeaderMessages">Disk Format: Level 2A2 - Data Object Header Messages</a></li>
+ <ol type="a">
+ <li><a href="#NILMessage">The NIL Message</a></li> <!-- 0x0000 -->
+ <li><a href="#DataspaceMessage">The Dataspace Message</a></li> <!-- 0x0001 -->
+ <li><a href="#LinkInfoMessage">The Link Info Message</a></li> <!-- 0x0002 -->
+ </ol>
+ </ol>
+ </ol>
+ </font>
+ </ol>
+ </td>
+
+ <td>&nbsp;</td>
+
+ <td valign="top">
+ <ol type="I" start="4">
+ <li><a href="#DataObject">Disk Format: Level 2 - Data
+ Objects</a><font size="-1"><i> (Continued)</i></li>
+ <ol type="A">
+ <li><a href="#ObjectHeader">Disk Format: Level 2A - Data Object
+ Headers</a><i> (Continued)</i></li>
+ <ol type="1" start="2">
+ <li><a href="#ObjectHeaderMessages">Disk Format: Level 2A2 -
+ Data Object Header Messages</a><i> (Continued)</i></li>
+ <ol type="a" start="4">
+ <li><a href="#DatatypeMessage">The Datatype Message</a></li> <!-- 0x0003 -->
+ <li><a href="#OldFillValueMessage">The Data Storage -
+ Fill Value (Old) Message</a></li> <!-- 0x0004 -->
+ <li><a href="#FillValueMessage">The Data Storage -
+ Fill Value Message</a></li> <!-- 0x0005 -->
+ <li><a href="#LinkMessage">The Link Message</a></li> <!-- 0x0006 -->
+ <li><a href="#ExternalFileListMessage">The Data Storage -
+ External Data Files Message</a></li> <!-- 0x0007 -->
+ <li><a href="#LayoutMessage">The Data Storage -
+ Layout Message</a></li> <!-- 0x0008 -->
+ <li><a href="#BogusMessage">The Bogus Message</a></li> <!-- 0x0009 -->
+ <li><a href="#GroupInfoMessage">The Group Info
+ Message</a></li> <!-- 0x000a -->
+ <li><a href="#FilterMessage">The Data Storage -
+ Filter Pipeline Message</a></li> <!-- 0x000b -->
+ <li><a href="#AttributeMessage">The Attribute
+ Message</a></li> <!-- 0x000c -->
+ <li><a href="#CommentMessage">The Object Comment
+ Message</a></li> <!-- 0x000d -->
+ <li><a href="#OldModificationTimeMessage">The Object
+ Modification Time (Old) Message</a></li> <!-- 0x000e -->
+ <li><a href="#SOHMTableMessage">The Shared Message
+ Table Message</a></li> <!-- 0x000f -->
+ <li><a href="#ContinuationMessage">The Object Header
+ Continuation Message</a></li> <!-- 0x0010 -->
+ <li><a href="#SymbolTableMessage">The Symbol
+ Table Message</a></li> <!-- 0x0011 -->
+ <li><a href="#ModificationTimeMessage">The Object
+ Modification Time Message</a></li> <!-- 0x0012 -->
+ <li><a href="#BtreeKValuesMessage">The B-tree
+ &lsquo;K&rsquo; Values Message</a></li> <!-- 0x0013 -->
+ <li><a href="#DrvInfoMessage">The Driver Info
+ Message</a></li> <!-- 0x0014 -->
+ <li><a href="#AinfoMessage">The Attribute Info
+ Message</a></li> <!-- 0x0015 -->
+ <li><a href="#RefCountMessage">The Object Reference
+ Count Message</a></li> <!-- 0x0016 -->
+ <li><a href="#FsinfoMessage">The File Space Info
+ Message</a></li> <!-- 0x0018 -->
+ </ol>
+ </ol>
+ <li><a href="#DataStorage">Disk Format: Level 2B - Data Object Data Storage</a></li>
+ </ol>
+ </font>
+ <li><a href="#AppendixA">Appendix A: Definitions</a></li>
+ <li><a href="#AppendixB">Appendix B: File Memory Allocation Types</a></li>
+ </ol>
+</td></tr>
+</table>
+</center>
+
+
+
+<br />
+<br />
+<hr />
+<a name="Intro"><h2>I. Introduction</h2></a>
+
+ <table align="right" width="100">
+ <tr><td>&nbsp;</td><td align="center">
+ <hr />
+ <img src="FF-IH_FileGroup.gif" alt="HDF5 Groups" hspace="15" vspace="15">
+ </td><td>&nbsp;</td></tr>
+ <tr><td>&nbsp;</td><td align="center">
+ <strong>Figure 1:</strong> Relationships among the HDF5 root group, other groups, and objects
+ <hr />
+ </td><td>&nbsp;</td></tr>
+
+ <tr><td>&nbsp;</td><td align="center">
+ <img src="FF-IH_FileObject.gif" alt="HDF5 Objects" hspace="15" vspace="15">
+ </td><td>&nbsp;</td></tr>
+ <tr><td>&nbsp;</td><td align="center">
+ <strong>Figure 2:</strong> HDF5 objects -- datasets, datatypes, or dataspaces
+ <hr />
+ </td><td>&nbsp;</td></tr>
+ </table>
+
+
+ <p>The format of an HDF5 file on disk encompasses several
+ key ideas of the HDF4 and AIO file formats as well as
+ addressing some shortcomings therein. The new format is
+ more self-describing than the HDF4 format and is more
+ uniformly applied to data objects in the file.</p>
+
+ <p>An HDF5 file appears to the user as a directed graph.
+ The nodes of this graph are the higher-level HDF5 objects
+ that are exposed by the HDF5 APIs:</p>
+
+ <ul>
+ <li>Groups</li>
+ <li>Datasets</li>
+ <li>Committed (formerly Named) datatypes</li>
+ </ul>
+
+ <p>At the lowest level, as information is actually written to the disk,
+ an HDF5 file is made up of the following objects:</p>
+ <ul>
+ <li>A superblock</li>
+ <li>B-tree nodes</li>
+ <li>Heap blocks</li>
+ <li>Object headers</li>
+ <li>Object data</li>
+ <li>Free space</li>
+ </ul>
+
+ <p>The HDF5 Library uses these low-level objects to represent the
+ higher-level objects that are then presented to the user or
+ to applications through the APIs. For instance, a group is an
+ object header that contains a message that points to a local
+ heap (for storing the links to objects in the group) and to a
+ B-tree (which indexes the links). A dataset is an object header
+ that contains messages that describe datatype, dataspace, layout,
+ filters, external files, fill value, and other elements with the
+ layout message pointing to either a raw data chunk or to a
+ B-tree that points to raw data chunks.</p>
+
+
+<br />
+<a name="ThisDocument"><h3>I.A. This Document</h3></a>
+
+ <p>This document describes the lower-level data objects;
+ the higher-level objects and their properties are described
+ in the <a href="UG/HDF5_Users_Guide-Responsive HTML5/index.html"><cite>HDF5 User&rsquo;s Guide</cite></a>.</p>
+
+ <p>Three levels of information comprise the file format.
+ Level 0 contains basic information for identifying and
+ defining information about the file. Level 1 information contains
+ the information about the pieces of a file shared by many objects
+ in the file (such as a B-trees and heaps). Level 2 is the rest
+ of the file and contains all of the data objects, with each object
+ partitioned into header information, also known as
+ <em>metadata</em>, and data.</p>
+
+ <p>The sizes of various fields in the following layout tables are
+ determined by looking at the number of columns the field spans
+ in the table. There are three exceptions: (1) The size may be
+ overridden by specifying a size in parentheses, (2) the size of
+ addresses is determined by the <em>Size of Offsets</em> field
+ in the superblock and is indicated in this document with a
+ superscripted &lsquo;O&rsquo;, and (3) the size of length fields is determined
+ by the <em>Size of Lengths</em> field in the superblock and is
+ indicated in this document with a superscripted &lsquo;L&rsquo;.</p>
+
+ <p>Values for all fields in this document should be treated as unsigned
+ integers, unless otherwise noted in the description of a field.
+ Additionally, all metadata fields are stored in little-endian byte
+ order.
+ </p>
+
+ <p>All checksums used in the format are computed with the
+ <a href="http://www.burtleburtle.net/bob/hash/doobs.html">Jenkins&rsquo;
+ lookup3</a> algorithm.
+ </p>
+
+ <p>Whenever a bit flag or field is mentioned for an entry, bits are
+ numbered from the lowest bit position in the entry.
+ </p>
+
+ <p>Various tables in this document aligned with &ldquo;This space inserted
+ only to align table nicely&rdquo;. These entries in the table are just
+ to make the table presentation nicer and do not represent any values
+ or padding in the file.
+ </p>
+
+
+<br />
+<a name="ChangesForHdf5_1_10"><h3>I.B. Changes for HDF5 1.10</h3></a>
+
+ <p>As of October 2015, changes in the file format for HDF5 1.10
+ have not yet been finalized.</p>
+
+
+
+<br />
+<br />
+<hr />
+<h2><a name="FileMetaData">
+II. Disk Format: Level 0 - File Metadata</a></h2>
+
+<br />
+<h3><a name="Superblock">
+II.A. Disk Format: Level 0A - Format Signature and Superblock</a></h3>
+
+ <p>The superblock may begin at certain predefined offsets within
+ the HDF5 file, allowing a block of unspecified content for
+ users to place additional information at the beginning (and
+ end) of the HDF5 file without limiting the HDF5 Library&rsquo;s
+ ability to manage the objects within the file itself. This
+ feature was designed to accommodate wrapping an HDF5 file in
+ another file format or adding descriptive information to an HDF5
+ file without requiring the modification of the actual file&rsquo;s
+ information. The superblock is located by searching for the
+ HDF5 format signature at byte offset 0, byte offset 512, and at
+ successive locations in the file, each a multiple of two of
+ the previous location; in other words, at these byte offsets:
+ 0, 512, 1024, 2048, and so on.</p>
+
+ <p>The superblock is composed of the format signature, followed by a
+ superblock version number and information that is specific to each
+ version of the superblock.
+ Currently, there are three versions of the superblock format.
+ Version 0 is the default format, while version 1 is basically the same
+ as version 0 with additional information when a non-default B-tree &lsquo;K&rsquo;
+ value is stored. Version 2 is the latest format, with some fields
+ eliminated or compressed and with superblock extension and checksum
+ support.</p>
+
+ <p>Version 0 and 1 of the superblock are described below:</p>
+
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Superblock (Versions 0 and 1)
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Format Signature (8 bytes)<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td>Version # of Superblock</td>
+ <td>Version # of File&rsquo;s Free Space Storage</td>
+ <td>Version # of Root Group Symbol Table Entry</td>
+ <td>Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td>Version # of Shared Header Message Format</td>
+ <td>Size of Offsets</td>
+ <td>Size of Lengths</td>
+ <td>Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Group Leaf Node K</td>
+ <td colspan="2">Group Internal Node K</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">File Consistency Flags</td>
+ </tr>
+
+ <tr>
+ <td colspan="2" style="border:dotted;">Indexed Storage Internal Node K<sup>1</sup></td>
+ <td colspan="2" style="border:dotted;">Reserved (zero)<sup>1</sup></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Base Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of File Free space Info<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />End of File Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Driver Information Block Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Root Group Symbol Table Entry</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in &ldquo;Size of Offsets.&rdquo;)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with a &lsquo;1&rsquo; in the above table are
+ new in version 1 of the superblock)
+ </td></tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Format Signature</p></td>
+ <td><p>This field contains a constant value and can be used to
+ quickly identify a file as being an HDF5 file. The
+ constant value is designed to allow easy identification of
+ an HDF5 file and to allow certain types of data corruption
+ to be detected. The file signature of an HDF5 file always
+ contains the following values:</p>
+ <center>
+ <table border align="center" cellpadding="4">
+ <tr align="center">
+ <td align="right">Decimal:</td>
+ <td width="8%">137</td>
+ <td width="8%">72</td>
+ <td width="8%">68</td>
+ <td width="8%">70</td>
+ <td width="8%">13</td>
+ <td width="8%">10</td>
+ <td width="8%">26</td>
+ <td width="8%">10</td>
+ </tr>
+
+ <tr align="center">
+ <td align="right">Hexadecimal:</td>
+ <td>89</td>
+ <td>48</td>
+ <td>44</td>
+ <td>46</td>
+ <td>0d</td>
+ <td>0a</td>
+ <td>1a</td>
+ <td>0a</td>
+ </tr>
+
+ <tr align="center">
+ <td align="right">ASCII C Notation:</td>
+ <td>\211</td>
+ <td>H</td>
+ <td>D</td>
+ <td>F</td>
+ <td>\r</td>
+ <td>\n</td>
+ <td>\032</td>
+ <td>\n</td>
+ </tr>
+ </table>
+ </center>
+ <p>This signature both identifies the file as an HDF5 file
+ and provides for immediate detection of common
+ file-transfer problems. The first two bytes distinguish
+ HDF5 files on systems that expect the first two bytes to
+ identify the file type uniquely. The first byte is
+ chosen as a non-ASCII value to reduce the probability
+ that a text file may be misrecognized as an HDF5 file;
+ also, it catches bad file transfers that clear bit
+ 7. Bytes two through four name the format. The CR-LF
+ sequence catches bad file transfers that alter newline
+ sequences. The control-Z character stops file display
+ under MS-DOS. The final line feed checks for the inverse
+ of the CR-LF translation problem. (This is a direct
+ descendent of the
+ <a href="http://www.libpng.org/pub/png/spec/iso/index-object.html#5PNG-file-signature">PNG</a> file
+ signature.)</p>
+ <p><em>This field is present in version 0+ of the superblock.</em>
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Version Number of the Superblock</p></td>
+ <td><p>This value is used to determine the format of the
+ information in the superblock. When the format of the
+ information in the superblock is changed, the version number
+ is incremented to the next integer and can be used to
+ determine how the information in the superblock is
+ formatted.</p>
+
+ <p>Values of 0, 1 and 2 are defined for this field. (The format
+ of version 2 is described below, not here)
+ </p>
+
+ <p><em>This field is present in version 0+ of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version Number of the File&rsquo;s Free Space
+ Information</p></td>
+ <td>
+ <p>This value is used to determine the format of the
+ file&rsquo;s free space information.
+ </p>
+ <p>The only value currently valid in this field is &lsquo;0&rsquo;, which
+ indicates that the file&rsquo;s free space is as described
+ <a href="#FreeSpaceManager">below</a>.
+ </p>
+
+ <p><em>This field is present in version 0 and 1 of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version Number of the Root Group Symbol Table
+ Entry</p></td>
+ <td><p>This value is used to determine the format of the
+ information in the Root Group Symbol Table Entry. When the
+ format of the information in that field is changed, the
+ version number is incremented to the next integer and can be
+ used to determine how the information in the field
+ is formatted.</p>
+ <p>The only value currently valid in this field is &lsquo;0&rsquo;,
+ which indicates that the root group symbol table entry is
+ formatted as described <a href="#SymbolTableEntry">below</a>.</p>
+ <p><em>This field is present in version 0 and 1 of the
+ superblock.</em></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version Number of the Shared Header Message Format</p></td>
+ <td><p>This value is used to determine the format of the
+ information in a shared object header message. Since the format
+ of the shared header messages differs from the other private
+ header messages, a version number is used to identify changes
+ in the format.
+ </p>
+ <p>The only value currently valid in this field is &lsquo;0&rsquo;, which
+ indicates that shared header messages are formatted as
+ described <a href="#ObjectHeaderMessages">below</a>.
+ </p>
+
+ <p><em>This field is present in version 0 and 1 of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Offsets</p></td>
+ <td><p>This value contains the number of bytes used to store
+ addresses in the file. The values for the addresses of
+ objects in the file are offsets relative to a base address,
+ usually the address of the superblock signature. This
+ allows a wrapper to be added after the file is created
+ without invalidating the internal offset locations.
+ </p>
+
+ <p><em>This field is present in version 0+ of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Lengths</p></td>
+ <td><p>This value contains the number of bytes used to store
+ the size of an object.
+ </p>
+ <p><em>This field is present in version 0+ of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Group Leaf Node K</p></td>
+ <td>
+ <p>Each leaf node of a group B-tree will have at
+ least this many entries but not more than twice this
+ many. If a group has a single leaf node then it
+ may have fewer entries.
+ </p>
+ <p>This value must be greater than zero.
+ </p>
+ <p>See the <a href="#Btrees">description</a> of B-trees below.
+ </p>
+
+ <p><em>This field is present in version 0 and 1 of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Group Internal Node K</p></td>
+ <td>
+ <p>Each internal node of a group B-tree will have at
+ least this many entries but not more than twice this
+ many. If the group has only one internal
+ node then it might have fewer entries.
+ </p>
+ <p>This value must be greater than zero.
+ </p>
+ <p>See the <a href="#Btrees">description</a> of B-trees below.
+ </p>
+
+ <p><em>This field is present in version 0 and 1 of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>File Consistency Flags</p></td>
+ <td>
+ <p>This value contains flags to indicate information
+ about the consistency of the information contained
+ within the file. Currently, the following bit flags are
+ defined:
+ <ul>
+ <li>Bit 0 set indicates that the file is opened for
+ write-access.</li>
+ <li>Bit 1 set indicates that the file has
+ been verified for consistency and is guaranteed to be
+ consistent with the format defined in this document.</li>
+ <li>Bits 2-31 are reserved for future use.</li>
+ </ul>
+ Bit 0 should be
+ set as the first action when a file is opened for write
+ access and should be cleared only as the final action
+ when closing a file. Bit 1 should be cleared during
+ normal access to a file and only set after the file&rsquo;s
+ consistency is guaranteed by the library or a
+ consistency utility.
+ </p>
+
+ <p><em>This field is present in version 0+ of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Indexed Storage Internal Node K</p></td>
+ <td>
+ <p>Each internal node of an indexed storage B-tree will have at
+ least this many entries but not more than twice this
+ many. If the index storage B-tree has only one internal
+ node then it might have fewer entries.
+ </p>
+ <p>This value must be greater than zero.
+ </p>
+ <p>See the <a href="#Btrees">description</a> of B-trees below.
+ </p>
+
+ <p><em>This field is present in version 1 of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Base Address</p></td>
+ <td>
+ <p>This is the absolute file address of the first byte of
+ the HDF5 data within the file. The library currently
+ constrains this value to be the absolute file address
+ of the superblock itself when creating new files;
+ future versions of the library may provide greater
+ flexibility. When opening an existing file and this address does
+ not match the offset of the superblock, the library assumes
+ that the entire contents of the HDF5 file have been adjusted in
+ the file and adjusts the base address and end of file address to
+ reflect their new positions in the file. Unless otherwise noted,
+ all other file addresses are relative to this base
+ address.
+ </p>
+
+ <p><em>This field is present in version 0+ of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address of Global Free-space Index</p></td>
+ <td>
+ <p>The file&rsquo;s free space is not persistent for version 0 and 1 of
+ the superblock.
+ Currently this field always contains the
+ <a href="#UndefinedAddress">undefined address</a>.
+ </p>
+
+ <p><em>This field is present in version 0 and 1 of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>End of File Address</p></td>
+ <td>
+ <p>This is the absolute file address of the first byte past
+ the end of all HDF5 data. It is used to determine whether a
+ file has been accidently truncated and as an address where
+ file data allocation can occur if space from the free list is
+ not used.
+ </p>
+
+ <p><em>This field is present in version 0+ of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Driver Information Block Address</p></td>
+ <td>
+ <p>This is the relative file address of the file driver
+ information block which contains driver-specific
+ information needed to reopen the file. If there is no
+ driver information block then this entry should be the
+ <a href="#UndefinedAddress">undefined address</a>.
+ </p>
+
+ <p><em>This field is present in version 0 and 1 of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Root Group Symbol Table Entry</p></td>
+ <td>
+ <p>This is the <a href="#SymbolTableEntry">symbol table entry</a>
+ of the root group, which serves as the entry point into
+ the group graph for the file.
+ </p>
+
+ <p><em>This field is present in version 0 and 1 of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <p>Version 2 of the superblock is described below:</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Superblock (Version 2)
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Format Signature (8 bytes)<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td>Version # of Superblock</td>
+ <td>Size of Offsets</td>
+ <td>Size of Lengths</td>
+ <td>File Consistency Flags</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Base Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Superblock Extension Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />End of File Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Root Group Object Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Superblock Checksum</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in &ldquo;Size of Offsets.&rdquo;)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Format Signature</p></td>
+ <td>
+ <p>This field is the same as described for versions 0 and 1 of the
+ superblock.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Version Number of the Superblock</p></td>
+ <td>
+ <p>This field has a value of 2 and has the same meaning as for
+ versions 0 and 1.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Offsets</p></td>
+ <td>
+ <p>This field is the same as described for versions 0 and 1 of the
+ superblock.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Lengths</p></td>
+ <td>
+ <p>This field is the same as described for versions 0 and 1 of the
+ superblock.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>File Consistency Flags</p></td>
+ <td>
+ <p>This field is the same as described for versions 0 and 1 except
+ that it is smaller (the number of reserved bits has been reduced
+ from 30 to 6).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Base Address</p></td>
+ <td>
+ <p>This field is the same as described for versions 0 and 1 of the
+ superblock.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Superblock Extension Address</p></td>
+ <td>
+ <p>The field is the address of the object header for the
+ <a href="#SuperblockExt">superblock extension</a>.
+ If there is no extension then this entry should be the
+ <a href="#UndefinedAddress">undefined address</a>.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>End of File Address</p></td>
+ <td>
+ <p>This field is the same as described for versions 0 and 1 of the
+ superblock.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Root Group Object Header Address</p></td>
+ <td>
+ <p>This is the address of
+ the <a href="#DataObject">root group object header</a>,
+ which serves as the entry point into the group graph for the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Superblock Checksum</p></td>
+ <td>
+ <p>The checksum for the superblock.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+<br />
+<h3><a name="DriverInfo">
+II.B. Disk Format: Level 0B - File Driver Info</a></h3>
+
+ <p>The <b>driver information block</b> is an optional region of the
+ file which contains information needed by the file driver
+ to reopen a file. The format is described below:</p>
+
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Driver Information Block
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3">Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Driver Information Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Driver Identification (8 bytes)<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br /><br />Driver Information (<em>variable size</em>)<br /><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number of the Driver Information Block.
+ This document describes version 0.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Driver Information Size</p></td>
+ <td>
+ <p>The size in bytes of the <em>Driver Information</em> field.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Driver Identification</p></td>
+ <td>
+ <p>This is an eight-byte ASCII string without null
+ termination which identifies the driver and/or version number
+ of the Driver Information Block. The predefined driver encoded
+ in this field by the HDF5 Library is identified by the
+ letters <code>NCSA</code> followed by the first four characters of
+ the driver name. If the Driver Information block is not
+ the original version then the last letter(s) of the
+ identification will be replaced by a version number in
+ ASCII, starting with 0.
+ </p>
+ <p>
+ Identification for user-defined drivers is also eight-byte long.
+ It can be arbitrary but should be unique to avoid
+ the four character prefix &ldquo;NCSA&rdquo;.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Driver Information</p></td>
+ <td>Driver information is stored in a format defined by the
+ file driver (see description below).</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ The two drivers encoded in the <em>Driver Identification</em> field are as follows:
+ <ul>
+ <li>
+ Multi driver:
+ <p>
+ The identifier for this driver is &ldquo;NCSAmulti&rdquo;.
+ This driver provides a mechanism for segregating raw data and different types of metadata
+ into multiple files.
+ These files are viewed by the library as a single virtual HDF5 file with a single file address.
+ A maximum of 6 files will be created for the following data:
+ superblock, B-tree, raw data, global heap, local heap, and object header.
+ More than one type of data can be written to the same file.
+ </p></li>
+ <li>
+ Family driver
+ <p>
+ The identifier for this driver is &ldquo;NCSAfami&rdquo; and is encoded in this field for library version 1.8 and after.
+ This driver is designed for systems that do not support files larger than 2 gigabytes
+ by splitting the HDF5 file address space across several smaller files.
+ It does nothing to segregate metadata and raw data;
+ they are mixed in the address space just as they would be in a single contiguous file.
+ </p></li>
+ </ul>
+ <p>The format of the <em>Driver Information</em> field for the
+ above two drivers are described below:</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Multi Driver Information
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Member Mapping</td>
+ <td>Member Mapping</td>
+ <td>Member Mapping</td>
+ <td>Member Mapping</td>
+ </tr>
+
+ <tr>
+ <td>Member Mapping</td>
+ <td>Member Mapping</td>
+ <td>Reserved</td>
+ <td>Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Member File 1<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />End of Address for Member File 1<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Member File 2<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />End of Address for Member File 2<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />... ...<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Member File N<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />End of Address for Member File N<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name of Member File 1 <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name of Member File 2 <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />... ...<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name of Member File N <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Member Mapping</p></td>
+ <td><p>These fields are integer values from 1 to 6
+ indicating how the data can be mapped to or merged with another type of
+ data.
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Member Mapping</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+ <tr>
+ <td align="center">1</td>
+ <td>The superblock data.</td>
+ </tr>
+ <tr>
+ <td align="center">2</td>
+ <td>The B-tree data.</td>
+ </tr>
+ <tr>
+ <td align="center">3</td>
+ <td>The raw data.</td>
+ </tr>
+ <tr>
+ <td align="center">4</td>
+ <td>The global heap data.</td>
+ </tr>
+ <tr>
+ <td align="center">5</td>
+ <td>The local heap data.</td>
+ </tr>
+ <tr>
+ <td align="center">6</td>
+ <td>The object header data.</td>
+ </tr>
+ </table></p>
+ <p>For example, if the third field has the value 3 and all the rest have the
+ value 1, it means there are two files: one for raw data, and one for superblock,
+ B-tree, global heap, local heap, and object header.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Reserved</p></td>
+ <td><p>These fields are reserved and should always be zero.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Address of Member File N</p></td>
+ <td><p>This field Specifies the virtual address at which the member file starts.</p>
+ <p>N is the number of member files.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>End of Address for Member File N</p></td>
+ <td><p>This field is the end of the allocated address for the member file.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Name of Member File N</p></td>
+ <td><p>This field is the null-terminated name of the member file and
+ its length should be multiples of 8 bytes.
+ Additional bytes will be padded with <em>NULL</em>s. The default naming
+ convention is <em>%s-X.h5</em>, where <em>X</em> is one of the letters
+ <em>s</em> (for superblock), <em>b</em> (for B-tree), <em>r</em> (for raw data),
+ <em>g</em> (for global heap), <em>l</em> (for local heap), and <em>o</em> (for
+ object header). The name of the whole HDF5 file will substitute the <em>%s</em>
+ in the string.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Family Driver Information
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="8"><br />Size of Member File<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Size of Member File</p></td>
+ <td><p>This field is the size of the member file in the family of files.</p></td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h3><a name="SuperblockExt">
+II.C. Disk Format: Level 0C - Superblock Extension</a></h3>
+
+ <p>The <em>superblock extension</em> is used to store superblock metadata
+ which is either optional, or added after the version of the superblock
+ was defined. Superblock extensions may only exist when version 2+ of
+ superblock is used. A superblock extension is an object header which may
+ hold the following messages:</p>
+ <ul>
+ <li>
+ <a href="#SOHMTableMessage">Shared Message Table message</a> containing
+ information to locate the master table of shared object header message
+ indices.</li>
+ <li>
+ <a href="#BtreeKValuesMessage">B-tree &lsquo;K&rsquo; Values message</a> containing
+ non-default B-tree &lsquo;K&rsquo; values.</li>
+ <li>
+ <a href="#DrvInfoMessage">Driver Info message</a> containing information
+ needed by the file driver in order to reopen a file.
+ See also the
+ <a href="#DriverInfo">&ldquo;Disk Format: Level 0B - File Driver
+ Info&rdquo;</a> section above.</li>
+ <li>
+ <a href="#FsinfoMessage">File Space Info message</a> containing
+ information about file space handling in the file.</li>
+ </ul>
+
+
+
+<br />
+<br />
+<hr />
+<h2><a name="FileInfra">
+III. Disk Format: Level 1 - File Infrastructure</a></h2>
+
+<br />
+<h3><a name="Btrees">
+III.A. Disk Format: Level 1A - B-trees and B-tree Nodes</a></h3>
+
+ <p>B-trees allow flexible storage for objects which tend to grow
+ in ways that cause the object to be stored discontiguously. B-trees
+ are described in various algorithms books including &ldquo;Introduction to
+ Algorithms&rdquo; by Thomas H. Cormen, Charles E. Leiserson, and Ronald
+ L. Rivest. B-trees are used in several places in the HDF5 file format,
+ when an index is needed for another data structure.</p>
+
+ <p>The version 1 B-tree structure described below is the original index
+ structure, but are limited by some bugs in our implementation (mainly in
+ how they handle deleting records). The version 1 B-trees are being phased
+ out in favor of the version 2 B-trees described below, although both
+ types of structures may be found in the same file, depending on
+ application settings when creating the file.</p>
+
+<br />
+<h4><a name="V1Btrees">
+III.A.1. Disk Format: Level 1A1 - Version 1 B-trees (B-link Trees)</a></h4>
+
+ <p>Version 1 B-trees in HDF5 files an implementation of the B-link tree,
+ in which the sibling nodes at a particular level in the tree are stored
+ in a doubly-linked list, is described in the &ldquo;Efficient Locking for
+ Concurrent Operations on B-trees&rdquo; paper by Phillip Lehman and S. Bing Yao
+ as published in the <cite>ACM Transactions on Database Systems</cite>,
+ Vol. 6, No. 4, December 1981.</p>
+
+ <p>The B-link trees implemented by the file format contain one more
+ key than the number of children. In other words, each child
+ pointer out of a B-tree node has a left key and a right key.
+ The pointers out of internal nodes point to sub-trees while
+ the pointers out of leaf nodes point to symbol nodes and
+ raw data chunks.
+ Aside from that difference, internal nodes and leaf nodes
+ are identical.</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ B-link Tree Nodes
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Node Type</td>
+ <td>Node Level</td>
+ <td colspan="2">Entries Used</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Left Sibling<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Right Sibling<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Key 0 (variable size)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Child 0<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Key 1 (variable size)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Child 1<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Key 2<em>K</em> (variable size)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Child 2<em>K</em><sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Key 2<em>K</em>+1 (variable size)</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>TREE</code>&rdquo; is
+ used to indicate the
+ beginning of a B-link tree node. This gives file
+ consistency checking utilities a better chance of
+ reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Node Type</p></td>
+ <td>
+ <p>Each B-link tree points to a particular type of data.
+ This field indicates the type of data as well as
+ implying the maximum degree <em>K</em> of the tree and
+ the size of each Key field.
+
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Node Type</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+ <tr>
+ <td align="center">0</td>
+ <td>This tree points to group nodes.</td>
+ </tr>
+ <tr>
+ <td align="center">1</td>
+ <td>This tree points to raw data chunk nodes.</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Node Level</p></td>
+ <td>
+ <p>The node level indicates the level at which this node
+ appears in the tree (leaf nodes are at level zero). Not
+ only does the level indicate whether child pointers
+ point to sub-trees or to data, but it can also be used
+ to help file consistency checking utilities reconstruct
+ damaged trees.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Entries Used</p></td>
+ <td>
+ <p>This determines the number of children to which this
+ node points. All nodes of a particular type of tree
+ have the same maximum degree, but most nodes will point
+ to less than that number of children. The valid child
+ pointers and keys appear at the beginning of the node
+ and the unused pointers and keys appear at the end of
+ the node. The unused pointers and keys have undefined
+ values.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Address of Left Sibling</p></td>
+ <td>
+ <p>This is the relative file address of the left sibling of
+ the current node. If the current
+ node is the left-most node at this level then this field
+ is the <a href="#UndefinedAddress">undefined address</a>.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Address of Right Sibling</p></td>
+ <td>
+ <p>This is the relative file address of the right sibling of
+ the current node. If the current
+ node is the right-most node at this level then this
+ field is the <a href="#UndefinedAddress">undefined address</a>.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Keys and Child Pointers</p></td>
+ <td>
+ <p>Each tree has 2<em>K</em>+1 keys with 2<em>K</em>
+ child pointers interleaved between the keys. The number
+ of keys and child pointers actually containing valid
+ values is determined by the node&rsquo;s <em>Entries Used</em> field.
+ If that field is <em>N</em> then the B-link tree contains
+ <em>N</em> child pointers and <em>N</em>+1 keys.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Key</p></td>
+ <td>
+ <p>The format and size of the key values is determined by
+ the type of data to which this tree points. The keys are
+ ordered and are boundaries for the contents of the child
+ pointer; that is, the key values represented by child
+ <em>N</em> fall between Key <em>N</em> and Key
+ <em>N</em>+1. Whether the interval is open or closed on
+ each end is determined by the type of data to which the
+ tree points.
+ </p>
+
+ <p>
+ The format of the key depends on the node type.
+ For nodes of node type 0 (group nodes), the key is formatted as
+ follows:
+
+ <table class="list">
+ <tr>
+ <td width="20%">A single field of <i>Size of Lengths</i>
+ bytes:</td>
+ <td width="80%">Indicates the byte offset into the local heap
+ for the first object name in the subtree which
+ that key describes.
+ </td>
+ </tr>
+ </table>
+ </p>
+
+
+ <p>
+ For nodes of node type 1 (chunked raw data nodes), the key is
+ formatted as follows:
+
+ <table class="list">
+ <tr>
+ <td width="20%">Bytes 1-4:</td>
+ <td width="80%">Size of chunk in bytes.</td>
+ </tr>
+ <tr>
+ <td>Bytes 4-8:</td>
+ <td>Filter mask, a 32-bit bit field indicating which
+ filters have been skipped for this chunk. Each filter
+ has an index number in the pipeline (starting at 0, with
+ the first filter to apply) and if that filter is skipped,
+ the bit corresponding to its index is set.</td>
+ </tr>
+ <tr>
+ <td>(<em>D + 1</em>) 64-bit fields:</td>
+ <td>The offset of the
+ chunk within the dataset where <i>D</i> is the number
+ of dimensions of the dataset, and the last value is the
+ offset within the dataset&rsquo;s datatype and should always be
+ zero. For example, if
+ a chunk in a 3-dimensional dataset begins at the
+ position <code>[5,5,5]</code>, there will be three
+ such 64-bit values, each with the value of
+ <code>5</code>, followed by a <code>0</code> value.</td>
+ </tr>
+ </table>
+ </p>
+
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Child Pointer</p></td>
+ <td>
+ <p>The tree node contains file addresses of subtrees or
+ data depending on the node level. Nodes at Level 0 point
+ to data addresses, either raw data chunks or group nodes.
+ Nodes at non-zero levels point to other nodes of the
+ same B-tree.
+ </p>
+ <p>For raw data chunk nodes, the child pointer is the address
+ of a single raw data chunk. For group nodes, the child pointer
+ points to a <a href="#SymbolTable">symbol table</a>, which contains
+ information for multiple symbol table entries.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <p>
+ Conceptually, each B-tree node looks like this:</p>
+ <center>
+ <table>
+ <tr valign="top" align="center">
+ <td>key[0]</td><td>&nbsp;</td>
+ <td>child[0]</td><td>&nbsp;</td>
+ <td>key[1]</td><td>&nbsp;</td>
+ <td>child[1]</td><td>&nbsp;</td>
+ <td>key[2]</td><td>&nbsp;</td>
+ <td>...</td><td>&nbsp;</td>
+ <td>...</td><td>&nbsp;</td>
+ <td>key[<i>N</i>-1]</td><td>&nbsp;</td>
+ <td>child[<i>N</i>-1]</td><td>&nbsp;</td>
+ <td>key[<i>N</i>]</td>
+ </tr>
+ </table>
+ </center>
+ <br />
+
+ where child[<i>i</i>] is a pointer to a sub-tree (at a level
+ above Level 0) or to data (at Level 0).
+ Each key[<i>i</i>] describes an <i>item</i> stored by the B-tree
+ (a chunk or an object of a group node). The range of values
+ represented by child[<i>i</i>] is indicated by key[<i>i</i>]
+ and key[<i>i</i>+1].
+
+
+ <p>The following question must next be answered:
+ &ldquo;Is the value described by key[<i>i</i>] contained in
+ child[<i>i</i>-1] or in child[<i>i</i>]?&rdquo;
+ The answer depends on the type of tree.
+ In trees for groups (node type 0) the object described by
+ key[<i>i</i>] is the greatest object contained in
+ child[<i>i</i>-1] while in chunk trees (node type 1) the
+ chunk described by key[<i>i</i>] is the least chunk in
+ child[<i>i</i>].</p>
+
+ <p>That means that key[0] for group trees is sometimes unused;
+ it points to offset zero in the heap, which is always the
+ empty string and compares as &ldquo;less-than&rdquo; any valid object name.</p>
+
+ <p>And key[<i>N</i>] for chunk trees is sometimes unused;
+ it contains a chunk offset which compares as &ldquo;greater-than&rdquo;
+ any other chunk offset and has a chunk byte size of zero
+ to indicate that it is not actually allocated.</p>
+
+<br />
+<h4><a name="V2Btrees">
+III.A.2. Disk Format: Level 1A2 - Version 2 B-trees</a></h4>
+
+ <p>Version 2 B-trees are &ldquo;traditional&rdquo; B-trees, with one major difference.
+ Instead of just using a simple pointer (or address in the file) to a
+ child of an internal node, the pointer to the child node contains two
+ additional pieces of information: the number of records in the child
+ node itself, and the total number of records in the child node and
+ all its descendants. Storing this additional information allows fast
+ array-like indexing to locate the n<sup>th</sup> record in the B-tree.</p>
+
+ <p>The entry into a version 2 B-tree is a header which contains global
+ information about the structure of the B-tree. The <em>root node
+ address</em>
+ field in the header points to the B-tree root node, which is either an
+ internal or leaf node, depending on the value in the header&rsquo;s
+ <em>depth</em> field. An internal node consists of records plus
+ pointers to further leaf or internal nodes in the tree. A leaf node
+ consists of solely of records. The format of the records depends on
+ the B-tree type (stored in the header).</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Version 2 B-tree Header
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+ <tr>
+ <td>Version</td>
+ <td>Type</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Node Size</td>
+ </tr>
+ <tr>
+ <td colspan="2">Record Size</td>
+ <td colspan="2">Depth</td>
+ </tr>
+ <tr>
+ <td>Split Percent</td>
+ <td>Merge Percent</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Root Node Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="2">Number of Records in Root Node</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Total Number of Records in B-tree<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>BTHD</code>&rdquo; is
+ used to indicate the header of a version 2 B-link tree node.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number for this B-tree header. This document
+ describes version 0.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Type</p></td>
+ <td>
+ <p>This field indicates the type of B-tree:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+ <tr>
+ <td align="center">0</td>
+ <td>A &ldquo;testing&rdquo; B-tree, this value should <em>not</em> be
+ used for storing records in actual HDF5 files.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">1</td>
+ <td>This B-tree is used for indexing indirectly accessed,
+ non-filtered &lsquo;huge&rsquo; fractal heap objects.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">2</td>
+ <td>This B-tree is used for indexing indirectly accessed,
+ filtered &lsquo;huge&rsquo; fractal heap objects.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">3</td>
+ <td>This B-tree is used for indexing directly accessed,
+ non-filtered &lsquo;huge&rsquo; fractal heap objects.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">4</td>
+ <td>This B-tree is used for indexing directly accessed,
+ filtered &lsquo;huge&rsquo; fractal heap objects.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">5</td>
+ <td>This B-tree is used for indexing the &lsquo;name&rsquo; field for
+ links in indexed groups.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">6</td>
+ <td>This B-tree is used for indexing the &lsquo;creation order&rsquo;
+ field for links in indexed groups.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">7</td>
+ <td>This B-tree is used for indexing shared object header
+ messages.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">8</td>
+ <td>This B-tree is used for indexing the &lsquo;name&rsquo; field for
+ indexed attributes.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">9</td>
+ <td>This B-tree is used for indexing the &lsquo;creation order&rsquo;
+ field for indexed attributes.
+ </td>
+ </tr>
+ </table></p>
+ <p>The format of records for each type is described below.</p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Node Size</p></td>
+ <td>
+ <p>This is the size in bytes of all B-tree nodes.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Record Size</p></td>
+ <td>
+ <p>This field is the size in bytes of the B-tree record.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Depth</p></td>
+ <td>
+ <p>This is the depth of the B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Split Percent</p></td>
+ <td>
+ <p>The percent full that a node needs to increase above before it
+ is split.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Merge Percent</p></td>
+ <td>
+ <p>The percent full that a node needs to be decrease below before it
+ is split.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Root Node Address</p></td>
+ <td>
+ <p>This is the address of the root B-tree node. A B-tree with
+ no records will have the <a href="#UndefinedAddress">undefined
+ address</a> in this field.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Number of Records in Root Node</p></td>
+ <td>
+ <p>This is the number of records in the root node.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Total Number of Records in B-tree</p></td>
+ <td>
+ <p>This is the total number of records in the entire B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the B-tree header.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Version 2 B-tree Internal Node
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+ <tr>
+ <td>Version</td>
+ <td>Type</td>
+ <td colspan="2">Records 0, 1, 2...N-1 <em>(variable size)</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Child Node Pointer 0<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Number of Records N<sub>0</sub> for Child Node 0 <em>(variable size)</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Total Number of Records for Child Node 0 <em>(optional, variable size)</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Child Node Pointer 1<sup>O</sup><br /><br /></td>
+ </tr>
+ <td colspan="4"><br />Number of Records N<sub>1</sub> for Child Node 1 <em>(variable size)</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Total Number of Records for Child Node 1 <em>(optional, variable size)</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Child Node Pointer N<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Number of Records N<sub>n</sub> for Child Node N <em>(variable size)</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Total Number of Records for Child Node N <em>(optional, variable size)</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+ </div>
+
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>BTIN</code>&rdquo; is
+ used to indicate the internal node of a B-link tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number for this B-tree internal node.
+ This document describes version 0.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Type</p></td>
+ <td>
+ <p>This field is the type of the B-tree node. It should always
+ be the same as the B-tree type in the header.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Records</p></td>
+ <td>
+ <p>The size of this field is determined by the number of records
+ for this node and the record size (from the header). The format
+ of records depends on the type of B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Child Node Pointer</p></td>
+ <td>
+ <p>This field is the address of the child node pointed to by the
+ internal node.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Records in Child Node</p></td>
+ <td>
+ <p>This is the number of records in the child node pointed to by
+ the corresponding <em>Node Pointer</em>.
+ </p>
+ <p>The number of bytes used to store this field is determined by
+ the maximum possible number of records able to be stored in the
+ child node.
+ </p>
+ <p>
+ The maximum number of records in a child node is computed
+ in the following way:
+
+ <ul>
+ <li>Subtract the fixed size overhead for
+ the child node (for example, its signature, version,
+ checksum, and so on and <em>one</em> pointer triplet
+ of information for the child node (because there is one
+ more pointer triplet than records in each internal node))
+ from the size of nodes for the B-tree. </li>
+ <li>Divide that result by the size of a record plus the
+ pointer triplet of information stored to reach each
+ child node from this node.
+ </ul>
+
+ </p>
+ <p>
+ Note that leaf nodes do not encode any
+ child pointer triplets, so the maximum number of records in a
+ leaf node is just the node size minus the leaf node overhead,
+ divided by the record size.
+ </p>
+ <p>
+ Also note that the first level of internal nodes above the
+ leaf nodes do not encode the <em>Total Number of Records in Child
+ Node</em> value in the child pointer triplets (since it is the
+ same as the <em>Number of Records in Child Node</em>), so the
+ maximum number of records in these nodes is computed with the
+ equation above, but using (<em>Child Pointer</em>, <em>Number of
+ Records in Child Node</em>) pairs instead of triplets.
+ </p>
+ <p>
+ The number of
+ bytes used to encode this field is the least number of bytes
+ required to encode the maximum number of records in a child
+ node value for the child nodes below this level
+ in the B-tree.
+ </p>
+ <p>
+ For example, if the maximum number of child records is
+ 123, one byte will be used to encode these values in this
+ node; if the maximum number of child records is
+ 20000, two bytes will be used to encode these values in this
+ node; and so on. The maximum number of bytes used to
+ encode these values is 8 (in other words, an unsigned
+ 64-bit integer).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Total Number of Records in Child Node</p></td>
+ <td>
+ <p>This is the total number of records for the node pointed to by
+ the corresponding <em>Node Pointer</em> and all its children.
+ This field exists only in nodes whose depth in the B-tree node
+ is greater than 1 (in other words, the &ldquo;twig&rdquo;
+ internal nodes, just above leaf nodes, do not store this
+ field in their child node pointers).
+ </p>
+ <p>The number of bytes used to store this field is determined by
+ the maximum possible number of records able to be stored in the
+ child node and its descendants.
+ </p>
+ <p>
+ The maximum possible number of records able to be stored in a
+ child node and its descendants is computed iteratively, in the
+ following way: The maximum number of records in a leaf node
+ is computed, then that value is used to compute the maximum
+ possible number of records in the first level of internal nodes
+ above the leaf nodes. Multiplying these two values together
+ determines the maximum possible number of records in child node
+ pointers for the level of nodes two levels above leaf nodes.
+ This process is continued up to any level in the B-tree.
+ </p>
+ <p>
+ The number of bytes used to encode this value is computed in
+ the same way as for the <em>Number of Records in Child Node</em>
+ field.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for this node.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Version 2 B-tree Leaf Node
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+ <tr>
+ <td>Version</td>
+ <td>Type</td>
+ <td colspan="2">Record 0, 1, 2...N-1 <em>(variable size)</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>BTLF</code>&ldquo; is
+ used to indicate the leaf node of a version 2 B-link tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number for this B-tree leaf node.
+ This document describes version 0.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Type</p></td>
+ <td>
+ <p>This field is the type of the B-tree node. It should always
+ be the same as the B-tree type in the header.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Records</p></td>
+ <td>
+ <p>The size of this field is determined by the number of records
+ for this node and the record size (from the header). The format
+ of records depends on the type of B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for this node.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <p>The record layout for each stored (in other words, non-testing)
+ B-tree type is as follows:</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Version 2 B-tree, Type 1 Record Layout - Indirectly Accessed, Non-Filtered,
+ &lsquo;Huge&rsquo; Fractal Heap Objects
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Huge Object Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Huge Object Length<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Huge Object ID<sup>L</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Huge Object Address</p></td>
+ <td>
+ <p>The address of the huge object in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Huge Object Length</p></td>
+ <td>
+ <p>The length of the huge object in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Huge Object ID</p></td>
+ <td>
+ <p>The heap ID for the huge object.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Version 2 B-tree, Type 2 Record Layout - Indirectly Accessed, Filtered,
+ &lsquo;Huge&rsquo; Fractal Heap Objects
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Filtered Huge Object Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Filtered Huge Object Length<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Filter Mask</td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Filtered Huge Object Memory Size<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Huge Object ID<sup>L</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Filtered Huge Object Address</p></td>
+ <td>
+ <p>The address of the filtered huge object in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filtered Huge Object Length</p></td>
+ <td>
+ <p>The length of the filtered huge object in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filter Mask</p></td>
+ <td>
+ <p>A 32-bit bit field indicating which filters have been skipped for
+ this chunk. Each filter has an index number in the pipeline
+ (starting at 0, with the first filter to apply) and if that
+ filter is skipped, the bit corresponding to its index is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filtered Huge Object Memory Size</p></td>
+ <td>
+ <p>The size of the de-filtered huge object in memory.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Huge Object ID</p></td>
+ <td>
+ <p>The heap ID for the huge object.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Version 2 B-tree, Type 3 Record Layout - Directly Accessed, Non-Filtered,
+ &lsquo;Huge&rsquo; Fractal Heap Objects
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Huge Object Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Huge Object Length<sup>L</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Huge Object Address</p></td>
+ <td>
+ <p>The address of the huge object in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Huge Object Length</p></td>
+ <td>
+ <p>The length of the huge object in the file.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Version 2 B-tree, Type 4 Record Layout - Directly Accessed, Filtered,
+ &lsquo;Huge&rsquo; Fractal Heap Objects
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Filtered Huge Object Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Filtered Huge Object Length<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Filter Mask</td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Filtered Huge Object Memory Size<sup>L</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Filtered Huge Object Address</p></td>
+ <td>
+ <p>The address of the filtered huge object in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filtered Huge Object Length</p></td>
+ <td>
+ <p>The length of the filtered huge object in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filter Mask</p></td>
+ <td>
+ <p>A 32-bit bit field indicating which filters have been skipped for
+ this chunk. Each filter has an index number in the pipeline
+ (starting at 0, with the first filter to apply) and if that
+ filter is skipped, the bit corresponding to its index is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filtered Huge Object Memory Size</p></td>
+ <td>
+ <p>The size of the de-filtered huge object in memory.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Version 2 B-tree, Type 5 Record Layout - Link Name for Indexed Group
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Hash of Name</td>
+ </tr>
+ <tr>
+ <td colspan="4">ID <em>(bytes 1-4)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="3">ID <em>(bytes 5-7)</em></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Hash</p></td>
+ <td>
+ <p>This field is hash value of the name for the link. The hash
+ value is the Jenkins&rsquo; lookup3 checksum algorithm applied to
+ the link&rsquo;s name.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>ID</p></td>
+ <td>
+ <p>This is a 7-byte sequence of bytes and is the heap ID for the
+ link record in the group&rsquo;s fractal heap.</p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Version 2 B-tree, Type 6 Record Layout - Creation Order for Indexed Group
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Creation Order <em>(8 bytes)</em><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">ID <em>(bytes 1-4)</em></td>
+ </tr>
+ <tr>
+ <td colspan="3">ID <em>(bytes 5-7)</em></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Creation Order</p></td>
+ <td>
+ <p>This field is the creation order value for the link.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>ID</p></td>
+ <td>
+ <p>This is a 7-byte sequence of bytes and is the heap ID for the
+ link record in the group&rsquo;s fractal heap.</p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Version 2 B-tree, Type 7 Record Layout - Shared Object Header Messages (Sub-Type 0 - Message in Heap)
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan>Message Location</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Hash</td>
+ </tr>
+ <tr>
+ <td colspan="4">Reference Count</td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Heap ID <em>(8 bytes)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Message Location</p></td>
+ <td>
+ <p>This field Indicates the location where the message is stored:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+ <tr>
+ <td align="center">0</td>
+ <td>Shared message is stored in shared message index heap.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">1</td>
+ <td>Shared message is stored in object header.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Hash</p></td>
+ <td>
+ <p>This field is hash value of the shared message. The hash
+ value is the Jenkins&rsquo; lookup3 checksum algorithm applied to
+ the shared message.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Reference Count</p></td>
+ <td>
+ <p>The number of objects which reference this message.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Heap ID</p></td>
+ <td>
+ <p>This is an 8-byte sequence of bytes and is the heap ID for the
+ shared message in the shared message index&rsquo;s fractal heap.</p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Version 2 B-tree, Type 7 Record Layout - Shared Object Header Messages (Sub-Type 1 - Message in Object Header)
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan>Message Location</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Hash</td>
+ </tr>
+ <tr>
+ <td>Reserved (zero)</td>
+ <td>Message Type</td>
+ <td colspan="2">Object Header Index</td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Object Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Message Location</p></td>
+ <td>
+ <p>This field Indicates the location where the message is stored:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+ <tr>
+ <td align="center">0</td>
+ <td>Shared message is stored in shared message index heap.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">1</td>
+ <td>Shared message is stored in object header.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Hash</p></td>
+ <td>
+ <p>This field is hash value of the shared message. The hash
+ value is the Jenkins&rsquo; lookup3 checksum algorithm applied to
+ the shared message.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Message Type</p></td>
+ <td>
+ <p>The object header message type of the shared message.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Header Index</p></td>
+ <td>
+ <p>This field indicates that the shared message is the n<sup>th</sup> message
+ of its type in the specified object header.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Header Address</p></td>
+ <td>
+ <p>The address of the object header containing the shared message.</p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Version 2 B-tree, Type 8 Record Layout - Attribute Name for Indexed Attributes
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Heap ID <em>(8 bytes)</em><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan>Message Flags</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Creation Order</td>
+ </tr>
+ <tr>
+ <td colspan="4">Hash of Name</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Heap ID</p></td>
+ <td>
+ <p>This is an 8-byte sequence of bytes and is the heap ID for the
+ attribute in the object&rsquo;s attribute fractal heap.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Message Flags</p></td>
+ <td><p>The object header message flags for the attribute message.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Creation Order</p></td>
+ <td>
+ <p>This field is the creation order value for the attribute.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Hash</p></td>
+ <td>
+ <p>This field is hash value of the name for the attribute. The hash
+ value is the Jenkins&rsquo; lookup3 checksum algorithm applied to
+ the attribute&rsquo;s name.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Version 2 B-tree, Type 9 Record Layout- Creation Order for Indexed Attributes
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Heap ID <em>(8 bytes)</em><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan>Message Flags</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Creation Order</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Heap ID</p></td>
+ <td>
+ <p>This is an 8-byte sequence of bytes and is the heap ID for the
+ attribute in the object&rsquo;s attribute fractal heap.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Message Flags</p></td>
+ <td>
+ <p>The object header message flags for the attribute message.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Creation Order</p></td>
+ <td>
+ <p>This field is the creation order value for the attribute.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+<br />
+<h3><a name="SymbolTable">
+III.B. Disk Format: Level 1B - Group Symbol Table Nodes</a></h3>
+
+ <p>A group is an object internal to the file that allows
+ arbitrary nesting of objects within the file (including other groups).
+ A group maps a set of link names in the group to a set of relative
+ file addresses of objects in the file. Certain metadata for an object to
+ which the group points can be cached in the group&rsquo;s symbol table entry in
+ addition to being in the object&rsquo;s header.</p>
+
+ <p>An HDF5 object name space can be stored hierarchically by
+ partitioning the name into components and storing each
+ component as a link in a group. The link for a
+ non-ultimate component points to the group containing
+ the next component. The link for the last
+ component points to the object being named.</p>
+
+ <p>One implementation of a group is a collection of symbol table nodes
+ indexed by a B-link tree. Each symbol table node contains entries
+ for one or more links. If an attempt is made to add a link to an already
+ full symbol table node containing 2<em>K</em> entries, then the node is
+ split and one node contains <em>K</em> symbols and the other contains
+ <em>K</em>+1 symbols.</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Symbol Table Node (A Leaf of a B-link tree)
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version Number</td>
+ <td>Reserved (zero)</td>
+ <td colspan="2">Number of Symbols</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br /><br />Group Entries<br /><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>SNOD</code>&rdquo; is
+ used to indicate the
+ beginning of a symbol table node. This gives file
+ consistency checking utilities a better chance of
+ reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version Number</p></td>
+ <td>
+ <p>The version number for the symbol table node. This
+ document describes version 1. (There is no version &lsquo;0&rsquo;
+ of the symbol table node)
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Entries</p></td>
+ <td>
+ <p>Although all symbol table nodes have the same length,
+ most contain fewer than the maximum possible number of
+ link entries. This field indicates how many entries
+ contain valid data. The valid entries are packed at the
+ beginning of the symbol table node while the remaining
+ entries contain undefined values.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Symbol Table Entries</p></td>
+ <td>
+ <p>Each link has an entry in the symbol table node.
+ The format of the entry is described below.
+ There are 2<em>K</em> entries in each group node, where
+ <em>K</em> is the &ldquo;Group Leaf Node K&rdquo; value from the
+ <a href="#Superblock">superblock</a>.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h3><a name="SymbolTableEntry">
+III.C. Disk Format: Level 1C - Symbol Table Entry </a></h3>
+
+ <p>Each symbol table entry in a symbol table node is designed
+ to allow for very fast browsing of stored objects.
+ Toward that design goal, the symbol table entries
+ include space for caching certain constant metadata from the
+ object header.</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Symbol Table Entry
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Link Name Offset<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Object Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Cache Type</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br /><br />Scratch-pad Space (16 bytes)<br /><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Link Name Offset</p></td>
+ <td>
+ <p>This is the byte offset into the group&rsquo;s local
+ heap for the name of the link. The name is null
+ terminated.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Header Address</p></td>
+ <td>
+ <p>Every object has an object header which serves as a
+ permanent location for the object&rsquo;s metadata. In addition
+ to appearing in the object header, some of the object&rsquo;s metadata
+ can be cached in the scratch-pad space.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Cache Type</p></td>
+ <td>
+ <p>The cache type is determined from the object header.
+ It also determines the format for the scratch-pad space:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Type</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+ <tr>
+ <td align="center">0</td>
+ <td>No data is cached by the group entry. This
+ is guaranteed to be the case when an object header
+ has a link count greater than one.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">1</td>
+ <td>Group object header metadata is cached in the
+ scratch-pad space. This implies that the symbol table
+ entry refers to another group.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">2</td>
+ <td>The entry is a symbolic link. The first four bytes
+ of the scratch-pad space are the offset into the local
+ heap for the link value. The object header address
+ will be undefined.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Reserved</p></td>
+ <td>
+ <p>These four bytes are present so that the scratch-pad
+ space is aligned on an eight-byte boundary. They are
+ always set to zero.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Scratch-pad Space</p></td>
+ <td>
+ <p>This space is used for different purposes, depending
+ on the value of the Cache Type field. Any metadata
+ about an object represented in the scratch-pad
+ space is duplicated in the object header for that
+ object.
+ </p>
+ <p>
+ Furthermore, no data is cached in the group
+ entry scratch-pad space if the object header for
+ the object has a link count greater than one.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h4>Format of the Scratch-pad Space</h4>
+
+ <p>The symbol table entry scratch-pad space is formatted
+ according to the value in the Cache Type field.</p>
+
+ <p>If the Cache Type field contains the value zero
+ <code>(0)</code> then no information is
+ stored in the scratch-pad space.</p>
+
+ <p>If the Cache Type field contains the value one
+ <code>(1)</code>, then the scratch-pad space
+ contains cached metadata for another object header
+ in the following format:</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Object Header Scratch-pad Format
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of B-tree<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Name Heap<sup>O</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Address of B-tree</p></td>
+ <td>
+ <p>This is the file address for the root of the
+ group&rsquo;s B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address of Name Heap</p></td>
+ <td>
+ <p>This is the file address for the group&rsquo;s local
+ heap, in which are stored the group&rsquo;s symbol names.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+ <br />
+ <p>If the Cache Type field contains the value two
+ <code>(2)</code>, then the scratch-pad space
+ contains cached metadata for a symbolic link
+ in the following format:</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Symbolic Link Scratch-pad Format
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Offset to Link Value</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Offset to Link Value</p></td>
+ <td>
+ <p>The value of a symbolic link (that is, the name of the
+ thing to which it points) is stored in the local heap.
+ This field is the 4-byte offset into the local heap for
+ the start of the link value, which is null terminated.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h3><a name="LocalHeap">
+III.D. Disk Format: Level 1D - Local Heaps</a></h3>
+
+ <p>A local heap is a collection of small pieces of data that are particular
+ to a single object in the HDF5 file. Objects can be
+ inserted and removed from the heap at any time.
+ The address of a heap does not change once the heap is created.
+ For example, a group stores addresses of objects in symbol table nodes
+ with the names of links stored in the group&rsquo;s local heap.
+ </p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Local Heap
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Data Segment Size<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Offset to Head of Free-list<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Data Segment<sup>O</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>HEAP</code>&rdquo;
+ is used to indicate the
+ beginning of a heap. This gives file consistency
+ checking utilities a better chance of reconstructing a
+ damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>Each local heap has its own version number so that new
+ heaps can be added to old files. This document
+ describes version zero (0) of the local heap.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data Segment Size</p></td>
+ <td>
+ <p>The total amount of disk memory allocated for the heap
+ data. This may be larger than the amount of space
+ required by the objects stored in the heap. The extra
+ unused space in the heap holds a linked list of free blocks.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Offset to Head of Free-list</p></td>
+ <td>
+ <p>This is the offset within the heap data segment of the
+ first free block (or the
+ <a href="#UndefinedAddress">undefined address</a> if there is no
+ free block). The free block contains &ldquo;Size of Lengths&rdquo; bytes that
+ are the offset of the next free block (or the
+ value &lsquo;1&rsquo; if this is the
+ last free block) followed by &ldquo;Size of Lengths&rdquo; bytes that store
+ the size of this free block. The size of the free block includes
+ the space used to store the offset of the next free block and
+ the size of the current block, making the minimum size of a free
+ block 2 * &ldquo;Size of Lengths&rdquo;.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address of Data Segment</p></td>
+ <td>
+ <p>The data segment originally starts immediately after
+ the heap header, but if the data segment must grow as a
+ result of adding more objects, then the data segment may
+ be relocated, in its entirety, to another part of the
+ file.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <p>Objects within a local heap should be aligned on an 8-byte boundary.</p>
+
+<br />
+<h3><a name="GlobalHeap">
+III.E. Disk Format: Level 1E - Global Heap</a></h3>
+
+ <p>Each HDF5 file has a global heap which stores various types of
+ information which is typically shared between datasets. The
+ global heap was designed to satisfy these goals:</p>
+
+ <ol type="A">
+ <li>Repeated access to a heap object must be efficient without
+ resulting in repeated file I/O requests. Since global heap
+ objects will typically be shared among several datasets, it is
+ probable that the object will be accessed repeatedly.</li>
+ <li>Collections of related global heap objects should result in
+ fewer and larger I/O requests. For instance, a dataset of
+ object references will have a global heap object for each
+ reference. Reading the entire set of object references
+ should result in a few large I/O requests instead of one small
+ I/O request for each reference.</li>
+ <li>It should be possible to remove objects from the global heap
+ and the resulting file hole should be eligible to be reclaimed
+ for other uses.</li>
+ </ol>
+
+
+ <p>The implementation of the heap makes use of the memory management
+ already available at the file level and combines that with a new
+ object called a <em>collection</em> to achieve goal B. The global heap
+ is the set of all collections. Each global heap object belongs to
+ exactly one collection and each collection contains one or more global
+ heap objects. For the purposes of disk I/O and caching, a collection is
+ treated as an atomic object, addressing goal A.
+ </p>
+
+ <p>When a global heap object is deleted from a collection (which occurs
+ when its reference count falls to zero), objects located after the
+ deleted object in the collection are packed down toward the beginning
+ of the collection and the collection&rsquo;s global heap object 0 is created
+ (if possible) or its size is increased to account for the recently
+ freed space. There are no gaps between objects in each collection,
+ with the possible exception of the final space in the collection, if
+ it is not large enough to hold the header for the collection&rsquo;s global
+ heap object 0. These features address goal C.
+ </p>
+
+ <p>The HDF5 Library creates global heap collections as needed, so there may
+ be multiple collections throughout the file. The set of all of them is
+ abstractly called the &ldquo;global heap&rdquo;, although they do not actually link
+ to each other, and there is no global place in the file where you can
+ discover all of the collections. The collections are found simply by
+ finding a reference to one through another object in the file. For
+ example, data of variable-length datatype elements is stored in the
+ global heap and is accessed via a global heap ID. The format for
+ global heap IDs is described at the end of this section.
+ </p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ A Global Heap Collection
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Collection Size<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Global Heap Object 1<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Global Heap Object 2<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />...<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Global Heap Object <em>N</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Global Heap Object 0 (free space)<br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>GCOL</code>&rdquo;
+ is used to indicate the
+ beginning of a collection. This gives file consistency
+ checking utilities a better chance of reconstructing a
+ damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>Each collection has its own version number so that new
+ collections can be added to old files. This document
+ describes version one (1) of the collections (there is no
+ version zero (0)).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Collection Size</p></td>
+ <td>
+ <p>This is the size in bytes of the entire collection
+ including this field. The default (and minimum)
+ collection size is 4096 bytes which is a typical file
+ system block size. This allows for 127 16-byte heap
+ objects plus their overhead (the collection header of 16 bytes
+ and the 16 bytes of information about each heap object).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Global Heap Object 1 through <em>N</em></p></td>
+ <td>
+ <p>The objects are stored in any order with no
+ intervening unused space.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Global Heap Object 0</p></td>
+ <td>
+ <p>Global Heap Object 0 (zero), when present, represents the free
+ space in the collection. Free space always appears at the end of
+ the collection. If the free space is too small to store the header
+ for Object 0 (described below) then the header is implied and the
+ collection contains no free space.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Global Heap Object
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="2">Heap Object Index</td>
+ <td colspan="2">Reference Count</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Object Size<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Object Data<br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Heap Object Index</p></td>
+ <td>
+ <p>Each object has a unique identification number within a
+ collection. The identification numbers are chosen so that
+ new objects have the smallest value possible with the
+ exception that the identifier <code>0</code> always refers to the
+ object which represents all free space within the
+ collection.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Reference Count</p></td>
+ <td>
+ <p>All heap objects have a reference count field. An
+ object which is referenced from some other part of the
+ file will have a positive reference count. The reference
+ count for Object 0 is always zero.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Reserved</p></td>
+ <td>
+ <p>Zero padding to align next field on an 8-byte boundary.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Size</p></td>
+ <td>
+ <p>This is the size of the object data stored for the object.
+ The actual storage space allocated for the object data is rounded
+ up to a multiple of eight.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Data</p></td>
+ <td>
+ <p>The object data is treated as a one-dimensional array
+ of bytes to be interpreted by the caller.
+ </p>
+ </td>
+ </tr>
+ </table>
+
+ </div>
+
+ <br />
+ <p>
+ The format for the ID used to locate an object in the global heap is
+ described here:</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Global Heap ID
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Collection Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Object Index</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Collection Address</p></td>
+ <td>
+ <p>This field is the address of the global heap collection
+ where the data object is stored.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>ID</p></td>
+ <td>
+ <p>This field is the index of the data object within the
+ global heap collection.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+<br />
+<h3><a name="FractalHeap">
+III.F. Disk Format: Level 1F - Fractal Heap</a></h3>
+
+ <p>
+ Each fractal heap consists of a header and zero or more direct and
+ indirect blocks (described below). The header contains general
+ information as well as
+ initialization parameters for the doubling table. The <em>Root
+ Block Address</em> in the header points to the first direct or
+ indirect block in the heap.
+ </p>
+
+ <p>
+ Fractal heaps are based on a data structure called a <em>doubling
+ table</em>. A doubling table provides a mechanism for quickly
+ extending an array-like data structure that minimizes the number of
+ empty blocks in the heap, while retaining very fast lookup of any
+ element within the array. More information on fractal heaps and
+ doubling tables can be found in the RFC
+ &ldquo;<a href="Supplements/FractalHeap/PrivateHeap.pdf">Private
+ Heaps in HDF5</a>.&rdquo;
+ </p>
+
+ <p>
+ The fractal heap implements the doubling table structure with
+ indirect and direct blocks.
+ Indirect blocks in the heap do not actually contain data for
+ objects in the heap, their &ldquo;size&rdquo; is abstract -
+ they represent the indexing structure for locating the
+ direct blocks in the doubling table.
+ Direct blocks
+ contain the actual data for objects stored in the heap.
+ </p>
+
+ <p>
+ All indirect blocks have a constant number of block entries in each
+ row, called the <em>width</em> of the doubling table (stored in
+ the heap header).
+
+ The number
+ of rows for each indirect block in the heap is determined by the
+ size of the block that the indirect block represents in the
+ doubling table (calculation of this is shown below) and is
+ constant, except for the &ldquo;root&rdquo;
+ indirect block, which expands and shrinks its number of rows as
+ needed.
+ </p>
+
+ <p>
+ Blocks in the first <em>two</em> rows of an indirect block
+ are <em>Starting Block Size</em> number of bytes in size,
+ and the blocks in each subsequent row are twice the size of
+ the blocks in the previous row. In other words, blocks in
+ the third row are twice the <em>Starting Block Size</em>,
+ blocks in the fourth row are four times the
+ <em>Starting Block Size</em>, and so on. Entries for
+ blocks up to the <em>Maximum Direct Block Size</em> point to
+ direct blocks, and entries for blocks greater than that size
+ point to further indirect blocks (which have their own
+ entries for direct and indirect blocks).
+ </p>
+
+ <p>
+ The number of rows of blocks, <em>nrows</em>, in an
+ indirect block of size <em>iblock_size</em> is given by the
+ following expression:
+ <br /> <br />
+ <em>nrows</em> = (log<sub>2</sub>(<em>iblock_size</em>) -
+ log<sub>2</sub>(<em>&lt;Starting Block Size&gt;</em> *
+ <em>&lt;Width&gt;</em>)) + 1
+ </p>
+
+ <p>
+ The maximum number of rows of direct blocks, <em>max_dblock_rows</em>,
+ in any indirect block of a fractal heap is given by the
+ following expression:
+ <br /> <br />
+ <em>max_dblock_rows</em> =
+ (log<sub>2</sub>(<em>&lt;Max. Direct Block Size&gt;</em>) -
+ log<sub>2</sub>(<em>&lt;Starting Block Size&gt;</em>)) + 2
+ </p>
+
+ <p>
+ Using the computed values for <em>nrows</em> and
+ <em>max_dblock_rows</em>, along with the <em>Width</em> of the
+ doubling table, the number of direct and indirect block entries
+ (<em>K</em> and <em>N</em> in the indirect block description, below)
+ in an indirect block can be computed:
+ <br /> <br />
+ <em>K</em> = MIN(<em>nrows</em>, <em>max_dblock_rows</em>) *
+ <em>Width</em>
+
+ <br /> <br />
+ If <em>nrows</em> is less than or equal to <em>max_dblock_rows</em>,
+ <em>N</em> is 0. Otherwise, <em>N</em> is simply computed:
+ <br /> <br />
+ <em>N</em> = <em>K</em> - (<em>max_dblock_rows</em> *
+ <em>Width</em>)
+ </p>
+
+ <p>
+ The size indirect blocks on disk is determined by the number
+ of rows in the indirect block (computed above). The size of direct
+ blocks on disk is exactly the size of the block in the doubling
+ table.
+ </p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Fractal Heap Header
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Heap ID Length</td>
+ <td colspan="2">I/O Filters&rsquo; Encoded Length</td>
+ </tr>
+
+ <tr>
+ <td>Flags</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Maximum Size of Managed Objects</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Next Huge Object ID<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />v2 B-tree Address of Huge Objects<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Amount of Free Space in Managed Blocks<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Managed Block Free Space Manager<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Amount of Managed Space in Heap<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Amount of Allocated Managed Space in Heap<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Offset of Direct Block Allocation Iterator in Managed Space<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Number of Managed Objects in Heap<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Size of Huge Objects in Heap<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Number of Huge Objects in Heap<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Size of Tiny Objects in Heap<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Number of Tiny Objects in Heap<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Table Width</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Starting Block Size<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Maximum Direct Block Size<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Maximum Heap Size</td>
+ <td colspan="2">Starting # of Rows in Root Indirect Block</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Root Block<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Current # of Rows in Root Indirect Block</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Size of Filtered Root Direct Block <em>(optional)</em><sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">I/O Filter Mask<em> (optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">I/O Filter Information<em> (optional, variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="40%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>FRHP</code>&rdquo;
+ is used to indicate the
+ beginning of a fractal heap header. This gives file consistency
+ checking utilities a better chance of reconstructing a
+ damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Heap ID Length</p></td>
+ <td>
+ <p>This is the length in bytes of heap object IDs for this heap.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>I/O Filters&rsquo; Encoded Length</p></td>
+ <td>
+ <p>This is the size in bytes of the encoded <em>I/O Filter Information</em>.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td>
+ <p>This field is the heap status flag and is a bit field
+ indicating additional information about the fractal heap.
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit(s)</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, the ID value to use for huge object has wrapped
+ around. If the value for the <em>Next Huge Object ID</em>
+ has wrapped around, each new huge object inserted into the
+ heap will require a search for an ID value.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>If set, the direct blocks in the heap are checksummed.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2-7</code></td>
+ <td>Reserved</td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Maximum Size of Managed Objects</p></td>
+ <td>
+ <p>This is the maximum size of managed objects allowed in the heap.
+ Objects greater than this this are &lsquo;huge&rsquo; objects and will be
+ stored in the file directly, rather than in a direct block for
+ the heap.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Next Huge Object ID</p></td>
+ <td>
+ <p>This is the next ID value to use for a huge object in the heap.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>v2 B-tree Address of Huge Objects</p></td>
+ <td>
+ <p>This is the address of the <a href="#V2Btrees">v2 B-tree</a>
+ used to track huge objects in the heap. The type of records
+ stored in the <em>v2 B-tree</em> will
+ be determined by whether the address & length of a huge object
+ can fit into a heap ID (if yes, it is a &ldquo;directly&rdquo; accessed
+ huge object) and whether there is a filter used on objects
+ in the heap.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Amount of Free Space in Managed Blocks</p></td>
+ <td>
+ <p>This is the total amount of free space in managed direct blocks
+ (in bytes).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address of Managed Block Free Space Manager</p></td>
+ <td>
+ <p>This is the address of the
+ <em><a href="#FreeSpaceManager">Free-space Manager</a></em> for
+ managed blocks.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Amount of Managed Space in Heap</p></td>
+ <td>
+ <p>This is the total amount of managed space in the heap (in bytes),
+ essentially the upper bound of the heap&rsquo;s linear address space.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Amount of Allocated Managed Space in Heap</p></td>
+ <td>
+ <p>This is the total amount of managed space (in bytes) actually
+ allocated in
+ the heap. This can be less than the <em>Amount of Managed Space
+ in Heap</em> field, if some direct blocks in the heap&rsquo;s linear
+ address space are not allocated.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Offset of Direct Block Allocation Iterator in Managed Space</p></td>
+ <td>
+ <p>This is the linear heap offset where the next direct
+ block should be allocated at (in bytes). This may be less than
+ the <em>Amount of Managed Space in Heap</em> value because the
+ heap&rsquo;s address space is increased by a &ldquo;row&rdquo; of direct blocks
+ at a time, rather than by single direct block increments.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Managed Objects in Heap</p></td>
+ <td>
+ <p>This is the number of managed objects in the heap.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Huge Objects in Heap</p></td>
+ <td>
+ <p>This is the total size of huge objects in the heap (in bytes).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Huge Objects in Heap</p></td>
+ <td>
+ <p>This is the number of huge objects in the heap.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Tiny Objects in Heap</p></td>
+ <td>
+ <p>This is the total size of tiny objects that are packed in heap
+ IDs (in bytes).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Tiny Objects in Heap</p></td>
+ <td>
+ <p>This is the number of tiny objects that are packed in heap IDs.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Table Width</p></td>
+ <td>
+ <p>This is the number of columns in the doubling table for managed
+ blocks. This value must be a power of two.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Starting Block Size</p></td>
+ <td>
+ <p>This is the starting block size to use in the doubling table for
+ managed blocks (in bytes). This value must be a power of two.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Maximum Direct Block Size</p></td>
+ <td>
+ <p>This is the maximum size allowed for a managed direct block.
+ Objects inserted into the heap that are larger than this value
+ (less the # of bytes of direct block prefix/suffix)
+ are stored as &lsquo;huge&rsquo; objects. This value must be a power of
+ two.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Maximum Heap Size</p></td>
+ <td>
+ <p>This is the maximum size of the heap&rsquo;s linear address space for
+ managed objects (in bytes). The value stored is the log2 of
+ the actual value, that is: the # of bits of the address space.
+ &lsquo;Huge&rsquo; and &lsquo;tiny&rsquo; objects are not counted in this value, since
+ they do not store objects in the linear address space of the
+ heap.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Starting # of Rows in Root Indirect Block</p></td>
+ <td>
+ <p>This is the starting number of rows for the root indirect block.
+ A value of 0 indicates that the root indirect block will have
+ the maximum number of rows needed to address the heap&rsquo;s <em>Maximum
+ Heap Size</em>.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address of Root Block</p></td>
+ <td>
+ <p>This is the address of the root block for the heap. It can
+ be the <a href="#UndefinedAddress">undefined address</a> if
+ there is no data in the heap. It either points to a direct
+ block (if the <em>Current # of Rows in the Root Indirect Block</em>
+ value is 0), or an indirect block.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Current # of Rows in Root Indirect Block</p></td>
+ <td>
+ <p>This is the current number of rows in the root indirect block.
+ A value of 0 indicates that <em>Address of Root Block</em>
+ points to direct block instead of indirect block.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Filtered Root Direct Block</p></td>
+ <td>
+ <p>This is the size of the root direct block, if filters are
+ applied to heap objects (in bytes). This field is only
+ stored in the header if the <em>I/O Filters&rsquo; Encoded Length</em>
+ is greater than 0.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>I/O Filter Mask</p></td>
+ <td>
+ <p>This is the filter mask for the root direct block, if filters
+ are applied to heap objects. This mask has the same format as
+ that used for the filter mask in chunked raw data records in a
+ <a href="#V1Btrees">v1 B-tree</a>.
+ This field is only
+ stored in the header if the <em>I/O Filters&rsquo; Encoded Length</em>
+ is greater than 0.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>I/O Filter Information</p></td>
+ <td>
+ <p>This is the I/O filter information encoding direct blocks and
+ huge objects, if filters are applied to heap objects. This
+ field is encoded as a <a href="#FilterMessage">Filter Pipeline</a>
+ message.
+ The size of this field is determined by <em>I/O Filters&rsquo;
+ Encoded Length</em>.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the header.</p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Fractal Heap Direct Block
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Heap Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Block Offset <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Object Data <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>FHDB</code>&rdquo;
+ is used to indicate the
+ beginning of a fractal heap direct block. This gives file consistency
+ checking utilities a better chance of reconstructing a
+ damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Heap Header Address</p></td>
+ <td>
+ <p>This is the address for the fractal heap header that this
+ block belongs to. This field is principally used for file
+ integrity checking.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Block Offset</p></td>
+ <td>
+ <p>This is the offset of the block within the fractal heap&rsquo;s
+ address space (in bytes). The number of bytes used to encode
+ this field is the <em>Maximum Heap Size</em> (in the heap&rsquo;s
+ header) divided by 8 and rounded up to the next highest integer,
+ for values that are not a multiple of 8. This value is
+ principally used for file integrity checking.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the direct block.</p>
+ <p>This field is only present if bit 1 of <em>Flags</em> in the
+ heap&rsquo;s header is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Data</p></td>
+ <td>
+ <p>This section of the direct block stores the actual data for
+ objects in the heap. The size of this section is determined by
+ the direct block&rsquo;s size minus the size of the other fields
+ stored in the direct block (for example, the <em>Signature</em>,
+ <em>Version</em>, and others including the <em>Checksum</em> if it is
+ present).
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Fractal Heap Indirect Block
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Heap Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Block Offset <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Child Direct Block #0 Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Size of Filtered Direct Block #0 <em>(optional)</em> <sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Filter Mask for Direct Block #0 <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Child Direct Block #1 Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Size of Filtered Direct Block #1 <em>(optional)</em><sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Filter Mask for Direct Block #1 <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Child Direct Block #K-1 Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Size of Filtered Direct Block #K-1 <em>(optional)</em><sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Filter Mask for Direct Block #K-1 <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Child Indirect Block #0 Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Child Indirect Block #1 Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Child Indirect Block #N-1 Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>FHIB</code>&rdquo; is used to
+ indicate the beginning of a fractal heap indirect block. This
+ gives file consistency checking utilities a better chance of
+ reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Heap Header Address</p></td>
+ <td>
+ <p>This is the address for the fractal heap header that this
+ block belongs to. This field is principally used for file
+ integrity checking.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Block Offset</p></td>
+ <td>
+ <p>This is the offset of the block within the fractal heap&rsquo;s
+ address space (in bytes). The number of bytes used to encode
+ this field is the <em>Maximum Heap Size</em> (in the heap&rsquo;s
+ header) divided by 8 and rounded up to the next highest integer,
+ for values that are not a multiple of 8. This value is
+ principally used for file integrity checking.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Child Direct Block #K Address</p></td>
+ <td>
+ <p>This field is the address of the child direct block.
+ The size of the [uncompressed] direct block can be computed by
+ its offset in the heap&rsquo;s linear address space.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Filtered Direct Block #K</p></td>
+ <td>
+ <p>This is the size of the child direct block after passing through
+ the I/O filters defined for this heap (in bytes). If no I/O
+ filters are present for this heap, this field is not present.
+ </p>
+ </td>
+ </tr>
+ <tr>
+ <td><p>Filter Mask for Direct Block #K</p></td>
+ <td>
+ <p>This is the I/O filter mask for the filtered direct block.
+ This mask has the same format as that used for the filter mask
+ in chunked raw data records in a <a href="#V1Btrees">v1 B-tree</a>.
+ If no I/O filters are present for this heap, this field is not
+ present.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Child Indirect Block #N Address</p></td>
+ <td>
+ <p>This field is the address of the child indirect block.
+ The size of the indirect block can be computed by
+ its offset in the heap&rsquo;s linear address space.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the indirect block.</p>
+ </td>
+ </tr>
+
+ </table>
+
+ </div>
+
+ <br />
+ <p>An object in the fractal heap is identified by means of a fractal heap ID,
+ which encodes information to locate the object in the heap.
+ Currently, the fractal heap stores an object in one of three ways,
+ depending on the object&rsquo;s size:</p>
+
+ <div align="center">
+ <table class="list80">
+ <tr>
+ <th width="20%">Type</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center">Tiny</td>
+ <td>
+ <p>When an object is small enough to be encoded in the heap ID, the
+ object&rsquo;s data is embedded in the fractal heap ID itself. There are
+ 2 sub-types for this type of object: normal and extended. The
+ sub-type for tiny heap IDs depends on whether the heap ID is large
+ enough to store objects greater than 16 bytes or not. If the
+ heap ID length is 18 bytes or smaller, the &lsquo;normal&rsquo; tiny heap ID
+ form is used. If the heap ID length is greater than 18 bytes in
+ length, the &ldquo;extented&rdquo; form is used. See format description below
+ for both sub-types.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center">Huge</td>
+ <td>
+ <p>When the size of an object is larger than <em>Maximum Size of
+ Managed Objects</em> in the <em>Fractal Heap Header</em>, the
+ object&rsquo;s data is stored on its own in the file and the object
+ is tracked/indexed via a version 2 B-tree. All huge objects
+ for a particular fractal heap use the same v2 B-tree. All huge
+ objects for a particular fractal heap use the same format for
+ their huge object IDs.
+ </p>
+
+ <p>Depending on whether the IDs for a heap are large enough to hold
+ the object&rsquo;s retrieval information and whether I/O pipeline filters
+ are applied to the heap&rsquo;s objects, 4 sub-types are derived for
+ huge object IDs for this heap:</p>
+
+ <div align="center">
+ <table class="list">
+ <tr>
+ <th align="left" width="35%">Sub-type</th>
+ <th align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="left">Directly accessed, non-filtered</td>
+ <td>
+ <p>The object&rsquo;s address and length are embedded in the
+ fractal heap ID itself and the object is directly accessed
+ from them. This allows the object to be accessed without
+ resorting to the B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td align="left">Directly accessed, filtered</td>
+ <td>
+ <p>The filtered object&rsquo;s address, length, filter mask and
+ de-filtered size are embedded in the fractal heap ID itself
+ and the object is accessed directly with them. This allows
+ the object to be accessed without resorting to the B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td align="left">Indirectly accessed, non-filtered</td>
+ <td>
+ <p>The object is located by using a B-tree key embedded in
+ the fractal heap ID to retrieve the address and length from
+ the version 2 B-tree for huge objects. Then, the address
+ and length are used to access the object.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td align="left">Indirectly accessed, filtered</td>
+ <td>
+ <p>The object is located by using a B-tree key embedded in
+ the fractal heap ID to retrieve the filtered object&rsquo;s
+ address, length, filter mask and de-filtered size from the
+ version 2 B-tree for huge objects. Then, this information
+ is used to access the object.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center">Managed</td>
+ <td>
+ <p>When the size of an object does not meet the above two
+ conditions, the object is stored and managed via the direct and
+ indirect blocks based on the doubling table.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+ <p>The specific format for each type of heap ID is described below:
+ </p>
+
+ <div align="center">
+ <table class="format">
+ <caption>Fractal Heap ID for Tiny Objects (sub-type 1 - &lsquo;Normal&rsquo;)
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version, Type & Length</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Data <em>(variable size)</em></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version, Type & Length</p></td>
+ <td>
+ <p>This is a bit field with the following definition:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6-7</code></td>
+ <td>The current version of ID format. This document
+ describes version 0.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4-5</code></td>
+ <td>The ID type. Tiny objects have a value of <code>2</code>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>0-3</code></td>
+ <td>The length of the tiny object. The value stored
+ is one less than the actual length (since zero-length
+ objects are not allowed to be stored in the heap).
+ For example, an object of actual length 1 has an
+ encoded length of 0, an object of actual length 2
+ has an encoded length of 1, and so on.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data</p></td>
+ <td>
+ <p>This is the data for the object.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>Fractal Heap ID for Tiny Objects (sub-type 2 - &lsquo;Extended&rsquo;)
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version, Type & Length</td>
+ <td>Extended Length</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Data <em>(variable size)</em></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version, Type & Length</p></td>
+ <td>
+ <p>This is a bit field with the following definition:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6-7</code></td>
+ <td>The current version of ID format. This document
+ describes version 0.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4-5</code></td>
+ <td>The ID type. Tiny objects have a value of <code>2</code>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>0-3</code></td>
+ <td>These 4 bits, together with the next byte, form an
+ unsigned 12-bit integer for holding the length of the
+ object. These 4-bits are bits 8-11 of the 12-bit integer.
+ See description for the <em>Extended Length</em> field below.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Extended Length</p></td>
+ <td>
+ <p>This byte, together with the 4 bits in the previous byte,
+ forms an unsigned 12-bit integer for holding the length of
+ the tiny object. These 8 bits are bits 0-7 of the 12-bit
+ integer formed. The value stored is one less than the actual
+ length (since zero-length objects are not allowed to be
+ stored in the heap). For example, an object of actual length
+ 1 has an encoded length of 0, an object of actual length
+ 2 has an encoded length of 1, and so on.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data</p></td>
+ <td>
+ <p>This is the data for the object.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>Fractal Heap ID for Huge Objects (sub-type 1 & 2): indirectly accessed, non-filtered/filtered
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version & Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />v2 B-tree Key<sup>L</sup><em> (variable size)</em><br /><br /></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version & Type</p></td>
+ <td>
+ <p>This is a bit field with the following definition:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6-7</code></td>
+ <td>The current version of ID format. This document
+ describes version 0.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4-5</code></td>
+ <td>The ID type. Huge objects have a value of <code>1</code>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>0-3</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>v2 B-tree Key</p></td>
+ <td><p>This field is the B-tree key for retrieving the information
+ from the version 2 B-tree for huge objects needed to access the
+ object. See the description of <a href="#V2Btrees">v2 B-tree</a>
+ records sub-type 1 & 2 for a description of the fields. New key
+ values are derived from <em>Next Huge Object ID</em> in the
+ <em>Fractal Heap Header</em>.</p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>Fractal Heap ID for Huge Objects (sub-type 3): directly accessed, non-filtered
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version & Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address <sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Length <sup>L</sup><br /><br /></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version & Type</p></td>
+ <td>
+ <p>This is a bit field with the following definition:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6-7</code></td>
+ <td>The current version of ID format. This document
+ describes version 0.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4-5</code></td>
+ <td>The ID type. Huge objects have a value of <code>1</code>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>0-3</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>This field is the address of the object in the file.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Length</p></td>
+ <td><p>This field is the length of the object in the file.</p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>Fractal Heap ID for Huge Objects (sub-type 4): directly accessed, filtered
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version & Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address <sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Length <sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Filter Mask</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />De-filtered Size <sup>L</sup><br /><br /></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td>
+ </tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>(Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version & Type</p></td>
+ <td>
+ <p>This is a bit field with the following definition:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6-7</code></td>
+ <td>The current version of ID format. This document
+ describes version 0.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4-5</code></td>
+ <td>The ID type. Huge objects have a value of <code>1</code>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>0-3</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>This field is the address of the filtered object in the file.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Length</p></td>
+ <td><p>This field is the length of the filtered object in the file.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filter Mask</p></td>
+ <td><p>This field is the I/O pipeline filter mask for the
+ filtered object in the file.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filtered Size</p></td>
+ <td><p>This field is the size of the de-filtered object in the file.</p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>Fractal Heap ID for Managed Objects
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version & Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Offset <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Length <em>(variable size)</em></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version & Type</p></td>
+ <td><p>This is a bit field with the following definition:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6-7</code></td>
+ <td>The current version of ID format. This document
+ describes version 0.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4-5</code></td>
+ <td>The ID type. Managed objects have a value of <code>0</code>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>0-3</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Offset</p></td>
+ <td><p>This field is the offset of the object in the heap.
+ This field&rsquo;s size is the minimum number of bytes
+ necessary to encode the <em>Maximum Heap Size</em> value
+ (from the <em>Fractal Heap Header</em>). For example, if the
+ value of the <em>Maximum Heap Size</em> is less than 256 bytes,
+ this field is 1 byte in length, a <em>Maximum Heap Size</em>
+ of 256-65535 bytes uses a 2 byte length, and so on.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Length</p></td>
+ <td><p>This field is the length of the object in the heap. It
+ is determined by taking the minimum value of <em>Maximum
+ Direct Block Size</em> and <em>Maximum Size of Managed
+ Objects</em> in the <em>Fractal Heap Header</em>. Again,
+ the minimum number of bytes needed to encode that value is
+ used for the size of this field.</p></td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h3><a name="FreeSpaceManager">
+III.G. Disk Format: Level 1G - Free-space Manager</a></h3>
+
+ <p>
+ Free-space managers are used to describe space within a heap or
+ the entire HDF5 file that is not currently used for that heap or
+ file.
+ </p>
+
+ <p>
+ The <em>free-space manager header</em> contains metadata information
+ about the space being tracked, along with the address of the list
+ of <em>free space sections</em> which actually describes the free
+ space. The header records information about free-space sections being
+ tracked, creation parameters for handling free-space sections of a
+ client, and section information used to locate the collection of
+ free-space sections.
+ </p>
+
+ <p>
+ The <em>free-space section list</em> stores a collection of
+ free-space sections that is specific to each <em>client</em> of the
+ free-space manager.
+
+ For example, the fractal heap is a client of the free space manager
+ and uses it to track unused space within the heap. There are 4
+ types of section records for the fractal heap, each of which has
+ its own format, listed below.
+ </p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Free-space Manager Header
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Client ID</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Total Space Tracked<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Total Number of Sections<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Number of Serialized Sections<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Number of Un-Serialized Sections<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Number of Section Classes</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Shrink Percent</td>
+ <td colspan="2">Expand Percent</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Size of Address Space</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Maximum Section Size <sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Serialized Section List<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Size of Serialized Section List Used<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Allocated Size of Serialized Section List<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="35%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>FSHD</code>&rdquo; is used to
+ indicate the beginning of the Free-space Manager Header.
+ This gives file consistency checking utilities a better chance of
+ reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This is the version number for the Free-space Manager Header
+ and this document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Client ID</p></td>
+ <td>
+ <p>This is the client ID for identifying the user of this
+ free-space manager:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">ID</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Fractal heap
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>File
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2+</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Total Space Tracked</p></td>
+ <td>
+ <p>This is the total amount of free space being tracked, in bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Total Number of Sections</p></td>
+ <td>
+ <p>This is the total number of free-space sections being tracked.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Serialized Sections</p></td>
+ <td>
+ <p>This is the number of serialized free-space sections being
+ tracked.
+ </p>
+ </td>
+ </tr>
+ <tr>
+ <td><p>Number of Un-Serialized Sections</p></td>
+ <td>
+ <p>This is the number of un-serialized free-space sections being
+ managed. Un-serialized sections are created by the free-space
+ client when the list of sections is read in.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Section Classes</p></td>
+ <td>
+ <p>This is the number of section classes handled by this free space
+ manager for the free-space client.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Shrink Percent</p></td>
+ <td>
+ <p>This is the percent of current size to shrink the allocated
+ serialized free-space section list.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Expand Percent</p></td>
+ <td>
+ <p>This is the percent of current size to expand the allocated
+ serialized free-space section list.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Address Space</p></td>
+ <td>
+ <p>This is the size of the address space that free-space sections
+ are within. This is stored as the log<sub>2</sub> of the
+ actual value (in other words, the number of bits required
+ to store values within that address space).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Maximum Section Size</p></td>
+ <td>
+ <p>This is the maximum size of a section to be tracked.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address of Serialized Section List</p></td>
+ <td>
+ <p>This is the address where the serialized free-space section
+ list is stored.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Serialized Section List Used</p></td>
+ <td>
+ <p>This is the size of the serialized free-space section
+ list used (in bytes). This value must be less than
+ or equal to the <em>allocated size of serialized section
+ list</em>, below.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Allocated Size of Serialized Section List</p></td>
+ <td>
+ <p>This is the size of serialized free-space section list
+ actually allocated (in bytes).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the free-space manager header.</p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <p>The free-space sections being managed are stored in a
+ <em>free-space section list</em>, described below. The sections
+ in the free-space section list are stored in the following way:
+ a count of the number of sections describing a particular size of
+ free space and the size of the free-space described (in bytes),
+ followed by a list of section description records; then another
+ section count and size, followed by the list of section
+ descriptions for that size; and so on.</p>
+
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Free-space Section List
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Free-space Manager Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Number of Section Records in Set #0 <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Size of Free-space Section Described in Record Set #0 <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #0 Section Record #0 Offset<em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="1">Record Set #0 Section Record #0 Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #0 Section Record #0 Data <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #0 Section Record #K-1 Offset<em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="1">Record Set #0 Section Record #K-1 Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #0 Section Record #K-1 Data <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Number of Section Records in Set #1 <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Size of Free-space Section Described in Record Set #1 <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #1 Section Record #0 Offset<em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="1">Record Set #1 Section Record #0 Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #1 Section Record #0 Data <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #1 Section Record #K-1 Offset<em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="1">Record Set #1 Section Record #K-1 Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #1 Section Record #K-1 Data <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><strong>...</strong></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><strong>...</strong></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Number of Section Records in Set #N-1 <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Size of Free-space Section Described in Record Set #N-1 <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #N-1 Section Record #0 Offset<em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="1">Record Set #N-1 Section Record #0 Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #N-1 Section Record #0 Data <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #N-1 Section Record #K-1 Offset<em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="1">Record Set #N-1 Section Record #K-1 Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #N-1 Section Record #K-1 Data <em>(variable size)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="35%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>FSSE</code>&rdquo; is used to
+ indicate the beginning of the Free-space Section Information.
+ This gives file consistency checking utilities a better chance of
+ reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This is the version number for the Free-space Section List
+ and this document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Free-space Manager Header Address</p></td>
+ <td>
+ <p>This is the address of the <em>Free-space Manager Header</em>.
+ This field is principally used for file
+ integrity checking.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Section Records for Set #N</p></td>
+ <td>
+ <p>This is the number of free-space section records for set #N.
+ The length of this field is the minimum number of bytes needed
+ to store the <em>number of serialized sections</em> (from the
+ <em>free-space manager header</em>).
+ </p>
+
+ <p>
+ The number of sets of free-space section records is
+ determined by the <em>size of serialized section list</em> in
+ the <em>free-space manager header</em>.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Section Size for Record Set #N</p></td>
+ <td>
+ <p>This is the size (in bytes) of the free-space section described
+ for <em>all</em> the section records in set #N.
+ </p>
+
+ <p>
+ The length of this field is the minimum number of bytes needed
+ to store the <em>maximum section size</em> (from the
+ <em>free-space manager header</em>).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Record Set #N Section #K Offset</p></td>
+ <td>
+ <p>This is the offset (in bytes) of the free-space section within
+ the client for the free-space manager.
+ </p>
+
+ <p>
+ The length of this field is the minimum number of bytes needed
+ to store the <em>size of address space</em> (from the
+ <em>free-space manager header</em>).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Record Set #N Section #K Type</p></td>
+ <td>
+ <p>This is the type of the section record, used to decode the
+ <em>record set #N section #K data</em> information. The defined
+ record type for <em>file</em> client is:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Type</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>File&rsquo;s section (a range of actual bytes in file)
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1+</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table></p>
+
+ <p>The defined record types for a <em>fractal heap</em> client are:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Type</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Fractal heap &ldquo;single&rdquo; section
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Fractal heap &ldquo;first row&rdquo; section
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Fractal heap &ldquo;normal row&rdquo; section
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Fractal heap &ldquo;indirect&rdquo; section
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>4+</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Record Set #N Section #K Data</p></td>
+ <td>
+ <p>This is the section-type specific information for each record
+ in the record set, described below.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the <em>Free-space Section List</em>.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <p>
+ The section-type specific data for each free-space section record is
+ described below:
+ </p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ File&rsquo;s Section Data Record
+ </caption>
+
+ <tr>
+ <td colspan="4"><em>No additional record data stored</em></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Fractal Heap &ldquo;Single&rdquo; Section Data Record
+ </caption>
+
+ <tr>
+ <td colspan="4"><em>No additional record data stored</em></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Fractal Heap &ldquo;First Row&rdquo; Section Data Record
+ </caption>
+
+ <tr>
+ <td colspan="4"><em>Same format as &ldquo;indirect&rdquo; section data</em></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Fractal Heap &ldquo;Normal Row&rdquo; Section Data Record
+ </caption>
+
+ <tr>
+ <td colspan="4"><em>No additional record data stored</em></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Fractal Heap &ldquo;Indirect&rdquo; Section Data Record
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Fractal Heap Indirect Block Offset <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Block Start Row</td>
+ <td colspan="2">Block Start Column</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Number of Blocks</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Fractal Heap Block Offset</p></td>
+ <td>
+ <p>The offset of the indirect block in the fractal heap&rsquo;s address
+ space containing the empty blocks.
+ </p>
+ <p>
+ The number of bytes used to encode this field is the minimum
+ number of bytes needed to encode values for the <em>Maximum
+ Heap Size</em> (in the fractal heap&rsquo;s header).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Block Start Row</p></td>
+ <td>
+ <p>This is the row that the empty blocks start in.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Block Start Column</p></td>
+ <td>
+ <p>This is the column that the empty blocks start in.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Blocks</p></td>
+ <td>
+ <p>This is the number of empty blocks covered by the section.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h3><a name="SOHMTable">
+III.H. Disk Format: Level 1H - Shared Object Header Message Table</a></h3>
+
+ <p>
+ The <em>shared object header message table</em> is used to locate
+ object
+ header messages that are shared between two or more object headers
+ in the file. Shared object header messages are stored and indexed
+ in the file in one of two ways: indexed sequentially in a
+ <em>shared header message list</em> or indexed with a v2 B-tree.
+ The shared messages themselves are either stored in a fractal
+ heap (when two or more objects share the message), or remain in an
+ object&rsquo;s header (when only one object uses the message currently,
+ but the message can be shared in the future).
+ </p>
+
+ <p>
+ The <em>shared object header message table</em>
+ contains a list of shared message index headers. Each index header
+ records information about the version of the index format, the index
+ storage type, flags for the message types indexed, the number of
+ messages in the index, the address where the index resides,
+ and the fractal heap address if shared messages are stored there.
+ </p>
+
+ <p>
+ Each index can be either a list or a v2 B-tree and may transition
+ between those two forms as the number of messages in the index
+ varies. Each shared message record contains information used to
+ locate the shared message from either a fractal heap or an object
+ header. The types of messages that can be shared are: <em>Dataspace,
+ Datatype, Fill Value, Filter Pipeline and Attribute</em>.
+ </p>
+
+ <p>
+ The <em>shared object header message table</em> is pointed to
+ from a <a href="#SOHMTableMessage">shared message table</a> message
+ in the superblock extension for a file. This message stores the
+ version of the table format, along with the number of index headers
+ in the table.
+ </p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Shared Object Header Message Table
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version for index #0</td>
+ <td>Index Type for index #0</td>
+ <td colspan="2">Message Type Flags for index #0</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Minimum Message Size for index #0</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">List Cutoff for index #0</td>
+ <td colspan="2">v2 B-tree Cutoff for index #0</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Number of Messages for index #0</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Index Address<sup>O</sup> for index #0<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Fractal Heap Address<sup>O</sup> for index #0<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td>Version for index #N-1</td>
+ <td>Index Type for index #N-1</td>
+ <td colspan="2">Message Type Flags for index #N-1</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Minimum Message Size for index #N-1</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">List Cutoff for index #N-1</td>
+ <td colspan="2">v2 B-tree Cutoff for index #N-1</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Number of Messages for index #N-1</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Index Address<sup>O</sup> for index #N-1<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Fractal Heap Address<sup>O</sup> for index #N-1<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="35%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>SMTB</code>&rdquo; is used to
+ indicate the beginning of the Shared Object Header Message table.
+ This gives file consistency checking utilities a better chance of
+ reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version for index #N</p></td>
+ <td>
+ <p>This is the version number for the list of shared object header message
+ indexes and this document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Index Type for index #N</p></td>
+ <td>
+ <p>The type of index can be an unsorted list or a v2 B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Message Type Flags for index #N</p></td>
+ <td>
+ <p>This field indicates the type of messages tracked in the index,
+ as follows:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bits</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, the index tracks <em>Dataspace Messages</em>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>If set, the message tracks <em>Datatype Messages</em>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>If set, the message tracks <em>Fill Value Messages</em>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>If set, the message tracks <em>Filter Pipeline Messages</em>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>If set, the message tracks <em>Attribute Messages</em>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>5-15</code></td>
+ <td>Reserved (zero).
+ </td>
+ </tr>
+ </table></p>
+
+
+ <p>
+ An index can track more than one type of message, but each type
+ of message can only by in one index.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Minimum Message Size for index #N</p></td>
+ <td>
+ <p>This is the message size sharing threshold for the index.
+ If the encoded size of the message is less than this value, the
+ message is not shared.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>List Cutoff for index #N</p></td>
+ <td>
+ <p>This is the cutoff value for the indexing of messages to
+ switch from a list to a v2 B-tree. If the number of messages
+ is greater than this value, the index should be a v2 B-tree.
+ </p>
+ </td>
+ </tr>
+ <tr>
+ <td><p>v2 B-tree Cutoff for index #N</p></td>
+ <td>
+ <p>This is is the cutoff value for the indexing of messages to
+ switch from a v2 B-tree back to a list. If the number of
+ messages is less than this value, the index should be a list.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Messages for index #N</p></td>
+ <td>
+ <p>The number of shared messages being tracked for the index.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Index Address for index #N</p></td>
+ <td>
+ <p>This field is the address of the list or v2 B-tree where the
+ index nodes reside.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fractal Heap Address for index #N</p></td>
+ <td>
+ <p>This field is the address of the fractal heap if shared messages
+ are stored there.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the table.</p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <p>
+ Shared messages are indexed either with a <em>shared message record
+ list</em>, described below, or using a v2 B-tree (using record type 7).
+ The number of records in the <em>shared message record list</em> is
+ determined in the index&rsquo;s entry in the <em>shared object header message
+ table</em>.
+ </p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Shared Message Record List
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Shared Message Record #0</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Shared Message Record #1</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Shared Message Record #N-1</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>SMLI</code>&rdquo; is used to
+ indicate the beginning of a list of index nodes.
+ This gives file consistency checking utilities a better chance of
+ reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Shared Message Record #N</p></td>
+ <td>
+ <p>The record for locating the shared message, either in the
+ fractal heap for the index, or an object header (see format for
+ <em>index nodes</em> below).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the list.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <p>
+ The record for each shared message in an index is stored in one of the
+ following forms:
+ </p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Shared Message Record, for messages stored in a fractal heap
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Message Location</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Hash Value</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reference Count</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Fractal Heap ID<br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Message Location</p></td>
+ <td>
+ <p>This has a value of 0 indicating that the message is stored in
+ the heap.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Hash Value</p></td>
+ <td>
+ <p>This is the hash value for the message.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Reference Count</p></td>
+ <td>
+ <p>This is the number of times the message is used in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fractal Heap ID</p></td>
+ <td>
+ <p>This is an 8-byte fractal heap ID for the message as stored in
+ the fractal heap for the index.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Shared Message Record, for messages stored in an object header
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Message Location</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Hash Value</td>
+ </tr>
+
+ <tr>
+ <td>Reserved</td>
+ <td>Message Type</td>
+ <td colspan="2">Creation Index</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Object Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Message Location</p></td>
+ <td>
+ <p>This has a value of 1 indicating that the message is stored in
+ an object header.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Hash Value</p></td>
+ <td>
+ <p>This is the hash value for the message.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Message Type</p></td>
+ <td>
+ <p>This is the message type in the object header.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Creation Index</p></td>
+ <td>
+ <p>This is the creation index of the message within the object
+ header.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Header Address</p></td>
+ <td>
+ <p>This is the address of the object header where the message is
+ located.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+
+<br />
+<br />
+<hr />
+<h2><a name="DataObject">
+IV. Disk Format: Level 2 - Data Objects </a></h2>
+
+ <p>Data objects contain the &ldquo;real&rdquo; user-visible information in the file.
+ These objects compose the scientific data and other information which
+ are generally thought of as &ldquo;data&rdquo; by the end-user. All the
+ other information in the file is provided as a framework for
+ storing and accessing these data objects.
+ </p>
+
+ <p>A data object is composed of header and data
+ information. The header information contains the information
+ needed to interpret the data information for the object as
+ well as additional &ldquo;metadata&rdquo; or pointers to additional
+ &ldquo;metadata&rdquo; used to describe or annotate each object.
+ </p>
+
+<br />
+<h3><a name="ObjectHeader">
+IV.A. Disk Format: Level 2A - Data Object Headers</a></h3>
+
+ <p>The header information of an object is designed to encompass
+ all of the information about an object, except for the data itself.
+ This information includes the dataspace, the datatype, information
+ about how the data is stored on disk (in external files, compressed,
+ broken up in blocks, and so on), as well as other information used
+ by the library to speed up access to the data objects or maintain
+ a file&rsquo;s integrity. Information stored by user applications
+ as attributes is also stored in the object&rsquo;s header. The header
+ of each object is not necessarily located immediately prior to the
+ object&rsquo;s data in the file and in fact may be located in any
+ position in the file. The order of the messages in an object header
+ is not significant.</p>
+
+ <p>Object headers are composed of a prefix and a set of messages. The
+ prefix contains the information needed to interpret the messages and
+ a small amount of metadata about the object, and the messages contain
+ the majority of the metadata about the object.
+ </p>
+
+<br />
+<h3><a name="ObjectHeaderPrefix">
+IV.A.1. Disk Format: Level 2A1 - Data Object Header Prefix</a></h3>
+
+<br />
+<h4><a name="V1ObjectHeaderPrefix">
+IV.A.1.a. Version 1 Data Object Header Prefix</a></h4>
+
+ <p>Header messages are aligned on 8-byte boundaries for version 1
+ object headers.
+ </p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Version 1 Object Header
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Reserved (zero)</td>
+ <td colspan="2">Total Number of Header Messages</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Object Reference Count</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Object Header Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Header Message Type #1</td>
+ <td colspan="2">Size of Header Message Data #1</td>
+ </tr>
+
+ <tr>
+ <td>Header Message #1 Flags</td>
+ <td colspan="3">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Header Message Data #1<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Header Message Type #n</td>
+ <td colspan="2">Size of Header Message Data #n</td>
+ </tr>
+
+ <tr>
+ <td>Header Message #n Flags</td>
+ <td colspan="3">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Header Message Data #n<br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This value is used to determine the format of the
+ information in the object header. When the format of the
+ object header is changed, the version number
+ is incremented and can be used to determine how the
+ information in the object header is formatted. This
+ is version one (1) (there was no version zero (0)) of the
+ object header.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Total Number of Header Messages</p></td>
+ <td>
+ <p>This value determines the total number of messages listed in
+ object headers for this object. This value includes the messages
+ in continuation messages for this object.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Reference Count</p></td>
+ <td>
+ <p>This value specifies the number of &ldquo;hard links&rdquo; to this object
+ within the current file. References to the object from external
+ files, &ldquo;soft links&rdquo; in this file and object references in this
+ file are not tracked.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Header Size</p></td>
+ <td>
+ <p>This value specifies the number of bytes of header message data
+ following this length field that contain object header messages
+ for this object header. This value does not include the size of
+ object header continuation blocks for this object elsewhere in the
+ file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Type</p></td>
+ <td>
+ <p>This value specifies the type of information included in the
+ following header message data. The message types for
+ header messages are defined in sections below.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Header Message #n Data</p></td>
+ <td>
+ <p>This value specifies the number of bytes of header
+ message data following the header message type and length
+ information for the current message. The size includes
+ padding bytes to make the message a multiple of eight
+ bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Flags</p></td>
+ <td>
+ <p>This is a bit field with the following definition:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, the message data is constant. This is used
+ for messages like the datatype message of a dataset.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>If set, the message is <em>shared</em> and stored
+ in another location than the object header. The Header
+ Message Data field contains a Shared Message
+ (described in the <a href="#ObjectHeaderMessages">Data Object Header Messages</a>
+ section below)
+ and the Size of Header Message Data field
+ contains the size of that Shared Message.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>If set, the message should not be shared.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>If set, the HDF5 decoder should fail to open this object
+ if it does not understand the message&rsquo;s type and the file
+ is open with permissions allowing write access to the file.
+ (Normally, unknown messages can just be ignored by HDF5
+ decoders)
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>If set, the HDF5 decoder should set bit 5 of this
+ message&rsquo;s flags (in other words, this bit field)
+ if it does not understand the message&rsquo;s type
+ and the object is modified in any way. (Normally,
+ unknown messages can just be ignored by HDF5
+ decoders)
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>5</code></td>
+ <td>If set, this object was modified by software that did not
+ understand this message.
+ (Normally, unknown messages should just be ignored by HDF5
+ decoders) (Can be used to invalidate an index or a similar
+ feature)
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>6</code></td>
+ <td>If set, this message is shareable.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>7</code></td>
+ <td>If set, the HDF5 decoder should always fail to open this
+ object if it does not understand the message&rsquo;s type (whether
+ it is open for read-only or read-write access). (Normally,
+ unknown messages can just be ignored by HDF5 decoders)
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Data</p></td>
+ <td>
+ <p>The format and length of this field is determined by the
+ header message type and size respectively. Some header
+ message types do not require any data and this information
+ can be eliminated by setting the length of the message to
+ zero. The data is padded with enough zeroes to make the
+ size a multiple of eight.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h4><a name="V2ObjectHeaderPrefix">
+IV.A.1.b. Version 2 Data Object Header Prefix</a></h4>
+
+ <p>Note that the &ldquo;total number of messages&rdquo; field has been dropped from
+ the data object header prefix in this version. The number of messages
+ in the data object header is just determined by the messages encountered
+ in all the object header blocks.</p>
+
+ <p>Note also that the fields and messages in this version of data object
+ headers have <em>no</em> alignment or padding bytes inserted - they are
+ stored packed together.</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Version 2 Object Header
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Access time <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Modification Time <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Change Time <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Birth Time <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Maximum # of compact attributes <em>(optional)</em></td>
+ <td colspan="2">Minimum # of dense attributes <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td>Size of Chunk #0 <em>(variable size)</em></td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td>Header Message Type #1</td>
+ <td colspan="2">Size of Header Message Data #1</td>
+ <td>Header Message #1 Flags</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Header Message #1 Creation Order <em>(optional)</em></td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Header Message Data #1<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+
+ <tr>
+ <td>Header Message Type #n</td>
+ <td colspan="2">Size of Header Message Data #n</td>
+ <td>Header Message #n Flags</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Header Message #n Creation Order <em>(optional)</em></td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Header Message Data #n<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Gap <em>(optional, variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>OHDR</code>&rdquo;
+ is used to indicate the
+ beginning of an object header. This gives file consistency
+ checking utilities a better chance of reconstructing a
+ damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This field has a value of 2 indicating version 2 of the object header.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td>
+ <p>This field is a bit field indicating additional information
+ about the object header.
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit(s)</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0-1</code></td>
+ <td>This two bit field determines the size of the
+ <em>Size of Chunk #0</em> field. The values are:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>The <em>Size of Chunk #0</em> field is 1 byte.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>The <em>Size of Chunk #0</em> field is 2 bytes.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>The <em>Size of Chunk #0</em> field is 4 bytes.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>The <em>Size of Chunk #0</em> field is 8 bytes.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>If set, attribute creation order is tracked.</td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>If set, attribute creation order is indexed.</td>
+ </tr>
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>If set, non-default attribute storage phase change
+ values are stored.</td>
+ </tr>
+ <tr>
+ <td align="center"><code>5</code></td>
+ <td>If set, access, modification, change and birth times
+ are stored.</td>
+ </tr>
+ <tr>
+ <td align="center"><code>6-7</code></td>
+ <td>Reserved</td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Access Time</p></td>
+ <td>
+ <p>This 32-bit value represents the number of seconds after the
+ UNIX epoch when the object&rsquo;s raw data was last accessed
+ (in other words, read or written).
+ </p>
+ <p>This field is present if bit 5 of <em>flags</em> is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Modification Time</p></td>
+ <td>
+ <p>This 32-bit value represents the number of seconds after
+ the UNIX epoch when the object&rsquo;s raw data was last
+ modified (in other words, written).
+ </p>
+ <p>This field is present if bit 5 of <em>flags</em> is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Change Time</p></td>
+ <td>
+ <p>This 32-bit value represents the number of seconds after the
+ UNIX epoch when the object&rsquo;s metadata was last changed.
+ </p>
+ <p>This field is present if bit 5 of <em>flags</em> is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Birth Time</p></td>
+ <td>
+ <p>This 32-bit value represents the number of seconds after the
+ UNIX epoch when the object was created.
+ </p>
+ <p>This field is present if bit 5 of <em>flags</em> is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Maximum # of compact attributes</p></td>
+ <td>
+ <p>This is the maximum number of attributes to store in the compact
+ format before switching to the indexed format.
+ </p>
+ <p>This field is present if bit 4 of <em>flags</em> is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Minimum # of dense attributes</p></td>
+ <td>
+ <p>This is the minimum number of attributes to store in the indexed
+ format before switching to the compact format.
+ </p>
+ <p>This field is present if bit 4 of <em>flags</em> is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Chunk #0</p></td>
+ <td>
+ <p>
+ This unsigned value specifies the number of bytes of header
+ message data following this field that contain object header
+ information.
+ </p>
+ <p>
+ This value does not include the size of object header
+ continuation blocks for this object elsewhere in the file.
+ </p>
+ <p>
+ The length of this field varies depending on bits 0 and 1 of
+ the <em>flags</em> field.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Type</p></td>
+ <td>
+ <p>Same format as version 1 of the object header, described above.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Header Message #n Data</p></td>
+ <td>
+ <p>This value specifies the number of bytes of header
+ message data following the header message type and length
+ information for the current message. The size of messages
+ in this version does <em>not</em> include any padding bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Flags</p></td>
+ <td>
+ <p>Same format as version 1 of the object header, described above.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Creation Order</p></td>
+ <td>
+ <p>This field stores the order that a message of a given type
+ was created in.
+ </p>
+ <p>This field is present if bit 2 of <em>flags</em> is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Data</p></td>
+ <td>
+ <p>Same format as version 1 of the object header, described above.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Gap</p></td>
+ <td>
+ <p>A gap in an object header chunk is inferred by the end of the
+ messages for the chunk before the beginning of the chunk&rsquo;s
+ checksum. Gaps are always smaller than the size of an
+ object header message prefix (message type + message size +
+ message flags).
+ </p>
+ <p>Gaps are formed when a message (typically an attribute message)
+ in an earlier chunk is deleted and a message from a later
+ chunk that does not quite fit into the free space is moved
+ into the earlier chunk.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the object header chunk.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <p>The header message types and the message data associated with
+ them compose the critical &ldquo;metadata&rdquo; about each object. Some
+ header messages are required for each object while others are
+ optional. Some optional header messages may also be repeated
+ several times in the header itself, the requirements and number
+ of times allowed in the header will be noted in each header
+ message description below.
+ </p>
+
+
+<br />
+<h3><a name="ObjectHeaderMessages">
+IV.A.2. Disk Format: Level 2A2 - Data Object Header Messages</a></h3>
+
+ <p>Data object header messages are small pieces of metadata that are
+ stored in the data object header for each object in an HDF5 file.
+ Data object header messages provide the metadata required to describe
+ an object and its contents, as well as optional pieces of metadata
+ that annotate the meaning or purpose of the object.
+ </p>
+
+ <p>Data object header messages are either stored directly in the data
+ object header for the object or are shared between multiple objects
+ in the file. When a message is shared, a flag in the <em>Message Flags</em>
+ indicates that the actual <em>Message Data</em>
+ portion of that message is stored in another location (such as another
+ data object header, or a heap in the file) and the <em>Message Data</em>
+ field contains the information needed to locate the actual information
+ for the message.
+ </p>
+
+ <p>
+ The format of shared message data is described here:</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Shared Message (Version 1)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Type</td>
+ <td colspan="2">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number is used when there are changes in the format
+ of a shared object message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Never used.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Used by the library before version 1.6.1.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Type</p></td>
+ <td><p>The type of shared message location:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Message stored in another object&rsquo;s header (a <em>committed</em>
+ message).
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>The address of the object header
+ containing the message to be shared.</p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Shared Message (Version 2)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Type</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number is used when there are changes in the format
+ of a shared object message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Used by the library of version 1.6.1 and after.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Type</p></td>
+ <td><p>The type of shared message location:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Message stored in another object&rsquo;s header (a <em>committed</em>
+ message).
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>The address of the object header
+ containing the message to be shared.</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Shared Message (Version 3)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Type</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Location <em>(variable size)</em></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number indicates changes in the format of shared
+ object message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Used by the library of version 1.8 and after. In this
+ version, the <em>Type</em> field can indicate that
+ the message is stored in the fractal heap.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Type</p></td>
+ <td><p>The type of shared message location:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Message is not shared and is not shareable.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Message stored in file&rsquo;s <em>shared object header message</em>
+ heap (a <em>shared</em> message).
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Message stored in another object&rsquo;s header (a <em>committed</em>
+ message).
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Message stored is not shared, but is sharable.
+ </td>
+ </tr>
+
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Location</p></td>
+ <td><p>This field contains either a <em>Size of Offsets</em>-bytes
+ address of the object header
+ containing the message to be shared, or an 8-byte fractal heap ID
+ for the message in the file&rsquo;s <em>shared object header message</em>
+ heap.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+ <p>The following is a list of currently defined header messages:
+ </p>
+
+<br />
+<h4><a name="NILMessage">IV.A.2.a. The NIL Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> NIL</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0000</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may be repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>The NIL message is used to indicate a message which is to be
+ ignored when reading the header messages for a data object.
+ [Possibly one which has been deleted for some reason.]
+ </td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> Unspecified</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+
+<br />
+<h4><a name="DataspaceMessage">IV.A.2.b. The Dataspace Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Dataspace</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0001</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies according to the number of
+ dimensions, as described in the following table.</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Required for dataset objects;
+ may not be repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>The dataspace message describes the number of dimensions (in
+ other words, &ldquo;rank&rdquo;) and size of each dimension that
+ the data object has. This message is only used for datasets which
+ have a simple, rectilinear, array-like layout; datasets requiring
+ a more complex layout are not yet supported.
+ </td>
+ </tr>
+
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Dataspace Message - Version 1
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Dimensionality</td>
+ <td>Flags</td>
+ <td>Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Dimension #1 Size<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Dimension #n Size<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Dimension #1 Maximum Size<sup>L</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Dimension #n Maximum Size<sup>L</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Permutation Index #1<sup>L</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Permutation Index #n<sup>L</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This value is used to determine the format of the
+ Dataspace Message. When the format of the
+ information in the message is changed, the version number
+ is incremented and can be used to determine how the
+ information in the object header is formatted. This
+ document describes version one (1) (there was no version
+ zero (0)).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimensionality</p></td>
+ <td>
+ <p>This value is the number of dimensions that the data
+ object has.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td>
+ <p>This field is used to store flags to indicate the
+ presence of parts of this message. Bit 0 (the least
+ significant bit) is used to indicate that maximum
+ dimensions are present. Bit 1 is used to indicate that
+ permutation indices are present.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Size</p></td>
+ <td>
+ <p>This value is the current size of the dimension of the
+ data as stored in the file. The first dimension stored in
+ the list of dimensions is the slowest changing dimension
+ and the last dimension stored is the fastest changing
+ dimension.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Maximum Size</p></td>
+ <td>
+ <p>This value is the maximum size of the dimension of the
+ data as stored in the file. This value may be the special
+ &ldquo;<a href="#UnlimitedDim">unlimited</a>&rdquo; size which indicates
+ that the data may expand along this dimension indefinitely.
+ If these values are not stored, the maximum size of each
+ dimension is assumed to be the dimension&rsquo;s current size.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Permutation Index #n</p></td>
+ <td>
+ <p>This value is the index permutation used to map
+ each dimension from the canonical representation to an
+ alternate axis for each dimension. If these values are
+ not stored, the first dimension stored in the list of
+ dimensions is the slowest changing dimension and the last
+ dimension stored is the fastest changing dimension.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+
+ <br />
+ <p>Version 2 of the dataspace message dropped the optional
+ permutation index value support, as it was never implemented in the
+ HDF5 Library:</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Dataspace Message - Version 2
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Dimensionality</td>
+ <td>Flags</td>
+ <td>Type</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Dimension #1 Size<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Dimension #n Size<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Dimension #1 Maximum Size<sup>L</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Dimension #n Maximum Size<sup>L</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This value is used to determine the format of the
+ Dataspace Message. This field should be &lsquo;2&rsquo; for version 2
+ format messages.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimensionality</p></td>
+ <td>
+ <p>This value is the number of dimensions that the data object has.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td>
+ <p>This field is used to store flags to indicate the
+ presence of parts of this message. Bit 0 (the least
+ significant bit) is used to indicate that maximum
+ dimensions are present.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Type</p></td>
+ <td>
+ <p>This field indicates the type of the dataspace:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>A <em>scalar</em> dataspace; in other words,
+ a dataspace with a single, dimensionless element.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>A <em>simple</em> dataspace; in other words,
+ a dataspace with a rank > 0 and an appropriate # of
+ dimensions.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>A <em>null</em> dataspace; in other words,
+ a dataspace with no elements.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Size</p></td>
+ <td>
+ <p>This value is the current size of the dimension of the
+ data as stored in the file. The first dimension stored in
+ the list of dimensions is the slowest changing dimension
+ and the last dimension stored is the fastest changing
+ dimension.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Maximum Size</p></td>
+ <td>
+ <p>This value is the maximum size of the dimension of the
+ data as stored in the file. This value may be the special
+ &ldquo;<a href="#UnlimitedDim">unlimited</a>&rdquo; size which indicates
+ that the data may expand along this dimension indefinitely.
+ If these values are not stored, the maximum size of each
+ dimension is assumed to be the dimension&rsquo;s current size.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+
+<!--
+<br />
+<h4><a name="DataSpaceMessage">Header Message Name: Complex Dataspace (Fiber Bundle?)</a></h4>
+
+ <!-- start msgdesc table --
+ <center>
+ <table class="msgdesc">
+ <p><b>Header Message Name: ???????</b></td></tr>
+ <b>Header Message Type: </b>0x0002<br />
+ <b>Length:</b> Varies</td></tr>
+
+ <b>Status:</b> One of the <em>Simple Dataspace</em> or
+ <em>Complex Dataspace</em> messages is required (but not both) and may
+ not be repeated.<br /> <b>Description:</b> The
+ <em>Dataspace</em> message describes space that the dataset is
+ mapped onto in a more comprehensive way than the <em>Simple
+ Dimensionality</em> message is capable of handling. The
+ dataspace of a dataset encompasses the type of coordinate system
+ used to locate the dataset&rsquo;s elements as well as the structure and
+ regularity of the coordinate system. The dataspace also
+ describes the number of dimensions which the dataset inhabits as
+ well as a possible higher dimensional space in which the dataset
+ is located within.
+
+ <br />
+ <p><b>Format of Data:</b></p>
+
+ <center>
+ <table border cellpadding="4" width="80%">
+ <caption align="bottom">
+ <b>HDF5 Dataspace Message Layout</b>
+ </caption>
+
+ <tr align="center">
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align="center">
+ <td colspan="4">Mesh Type</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Logical Dimensionality</td>
+ </tr>
+ </table>
+ </center>
+
+ <br />
+ <dl>
+ <dt>The elements of the dimensionality message are described below:
+ <dd>
+ <dl>
+ <dt>Mesh Type: (unsigned 32-bit integer)
+ <dd>This value indicates whether the grid is
+ polar/spherical/cartesion,
+ structured/unstructured and regular/irregular. <br />
+ The mesh type value is broken up as follows: <br />
+
+ <br />
+ <center>
+ <table border cellpadding="4" width="80%">
+ <caption align="bottom">
+ <b>HDF5 Mesh-type Layout</b>
+ </caption>
+
+ <tr align="center">
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align="center">
+ <td colspan="1">Mesh Embedding</td>
+ <td colspan="1">Coordinate System</td>
+ <td colspan="1">Structure</td>
+ <td colspan="1">Regularity</td>
+ </tr>
+ </table>
+ </center>
+ The following are the definitions of mesh-type bytes:
+ <dl>
+ <dt>Mesh Embedding
+ <dd>This value indicates whether the dataset dataspace
+ is located within
+ another dataspace or not:
+ <dl> <dl>
+ <dt>&lt;STANDALONE&gt;
+ <dd>The dataset mesh is self-contained and is not
+ embedded in another mesh.
+ <dt>&lt;EMBEDDED&gt;
+ <dd>The dataset&rsquo;s dataspace is located within
+ another dataspace, as
+ described in information below.
+ </dl> </dl>
+ <dt>Coordinate System
+ <dd>This value defines the type of coordinate system
+ used for the mesh:
+ <dl> <dl>
+ <dt>&lt;POLAR&gt;
+ <dd>The last two dimensions are in polar
+ coordinates, higher dimensions are
+ cartesian.
+ <dt>&lt;SPHERICAL&gt;
+ <dd>The last three dimensions are in spherical
+ coordinates, higher dimensions
+ are cartesian.
+ <dt>&lt;CARTESIAN&gt;
+ <dd>All dimensions are in cartesian coordinates.
+ </dl> </dl>
+ <dt>Structure
+ <dd>This value defines the locations of the grid-points
+ on the axes:
+ <dl> <dl>
+ <dt>&lt;STRUCTURED&gt;
+ <dd>All grid-points are on integral, sequential
+ locations, starting from 0.
+ <dt>&lt;UNSTRUCTURED&gt;
+ <dd>Grid-points locations in each dimension are
+ explicitly defined and
+ may be of any numeric datatype.
+ </dl> </dl>
+ <dt>Regularity
+ <dd>This value defines the locations of the dataset
+ points on the grid:
+ <dl> <dl>
+ <dt>&lt;REGULAR&gt;
+ <dd>All dataset elements are located at the
+ grid-points defined.
+ <dt>&lt;IRREGULAR&gt;
+ <dd>Each dataset element has a particular
+ grid-location defined.
+ </dl> </dl>
+ </dl>
+ <p>The following grid combinations are currently allowed:</p>
+ <dl> <dl>
+ <dt>&lt;POLAR-STRUCTURED-REGULAR&gt;
+ <dt>&lt;SPHERICAL-STRUCTURED-REGULAR&gt;
+ <dt>&lt;CARTESIAN-STRUCTURED-REGULAR&gt;
+ <dt>&lt;POLAR-UNSTRUCTURED-REGULAR&gt;
+ <dt>&lt;SPHERICAL-UNSTRUCTURED-REGULAR&gt;
+ <dt>&lt;CARTESIAN-UNSTRUCTURED-REGULAR&gt;
+ <dt>&lt;CARTESIAN-UNSTRUCTURED-IRREGULAR&gt;
+ </dl> </dl>
+ All of the above grid types can be embedded within another
+ dataspace.
+ <br /> <br />
+ <dt>Logical Dimensionality: (unsigned 32-bit integer)
+ <dd>This value is the number of dimensions that the dataset occupies.
+
+ <br />
+ <center>
+ <table border cellpadding="4" width="80%">
+ <caption align="bottom">
+ <b>HDF5 Dataspace Embedded Dimensionality Information</b>
+ </caption>
+
+ <tr align="center">
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align="center">
+ <td colspan="4">Embedded Dimensionality</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Embedded Dimension Size #1</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Embedded Dimension Size #n</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Embedded Origin Location #1</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Embedded Origin Location #n</td>
+ </tr>
+ </table>
+ </center>
+
+ <dt>Embedded Dimensionality: (unsigned 32-bit integer)
+ <dd>This value is the number of dimensions of the space the
+ dataset is located within: in other words, a planar dataset
+ located within a 3-D space, a 3-D dataset
+ which is a subset of another 3-D space, and so on.
+ <dt>Embedded Dimension Size: (unsigned 32-bit integer)
+ <dd>These values are the sizes of the dimensions of the
+ embedded dataspace
+ that the dataset is located within.
+ <dt>Embedded Origin Location: (unsigned 32-bit integer)
+ <dd>These values comprise the location of the dataset&rsquo;s
+ origin within the embedded dataspace.
+ </dl>
+ </dl>
+ [Comment: need some way to handle different orientations of the
+ dataset dataspace
+ within the embedded dataspace]<br />
+
+ <br />
+ <center>
+ <table border cellpadding="4" width="80%">
+ <caption align="bottom">
+ <b>HDF5 Dataspace Structured/Regular Grid Information</b>
+ </caption>
+
+ <tr align="center">
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align="center">
+ <td colspan="4">Logical Dimension Size #1</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Logical Dimension Maximum #1</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Logical Dimension Size #n</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Logical Dimension Maximum #n</td>
+ </tr>
+ </table>
+ </center>
+
+ <br />
+ <dl>
+ <dt>The elements of the dimensionality message are described below:
+ <dd>
+ <dl>
+ <dt>Logical Dimension Size #n: (unsigned 32-bit integer)
+ <dd>This value is the current size of the dimension of the
+ data as stored in
+ the file. The first dimension stored in the list of
+ dimensions is the slowest
+ changing dimension and the last dimension stored is the
+ fastest changing
+ dimension.
+ <dt>Logical Dimension Maximum #n: (unsigned 32-bit integer)
+ <dd>This value is the maximum size of the dimension of the
+ data as stored in
+ the file. This value may be the special value
+ &lt;UNLIMITED&gt; which
+ indicates that the data may expand along this dimension
+ indefinitely.
+ </dl>
+ </dl>
+ <br />
+ <center>
+ <table border cellpadding="4" width="80%">
+ <caption align="bottom">
+ <b>HDF5 Dataspace Structured/Irregular Grid Information</b>
+ </caption>
+
+ <tr align="center">
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align="center">
+ <td colspan="4"># of Grid Points in Dimension #1</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr align="center">
+ <td colspan="4"># of Grid Points in Dimension #n</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Datatype of Grid Point Locations</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Location of Grid Points in Dimension #1</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Location of Grid Points in Dimension #n</td>
+ </tr>
+ </table>
+ </center>
+
+ <br />
+ <center>
+ <table border cellpadding="4" width="80%">
+ <caption align="bottom">
+ <b>HDF5 Dataspace Unstructured Grid Information</b>
+ </caption>
+
+ <tr align="center">
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align="center">
+ <td colspan="4"># of Grid Points</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Datatype of Grid Point Locations</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Grid Point Locations<br />.<br />.<br /></td>
+ </tr>
+ </table>
+ </center>
+-->
+
+<br />
+<h4><a name="LinkInfoMessage">IV.A.2.c. The Link Info Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Link Info</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x002 </td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies </td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated. </td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>The link info message tracks variable information about the
+ current state of the links for a &ldquo;new style&rdquo;
+ group&rsquo;s behavior. Variable information will be stored in
+ this message and constant information will be stored in the
+ <a href="#GroupInfoMessage">Group Info</a> message.
+ </td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Link Info
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Maximum Creation Index <em>(8 bytes, optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Fractal Heap Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of v2 B-tree for Name Index<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of v2 B-tree for Creation Order Index<sup>O</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number for this message. This document describes
+ version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>This field determines various optional aspects of the link
+ info message:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, creation order for the links is tracked.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>If set, creation order for the links is indexed.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2-7</code></td>
+ <td>Reserved</td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Maximum Creation Index</p></td>
+ <td><p>This 64-bit value is the maximum creation order index value
+ stored for a link in this group.</p>
+ <p>This field is present if bit 0 of <em>flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fractal Heap Address</p></td>
+ <td>
+ <p>
+ This is the address of the fractal heap to store dense links.
+ Each link stored in the fractal heap is stored as a
+ <a href="#LinkMessage">Link Message</a>.
+ </p>
+ <p>
+ If there are no links in the group, or the group&rsquo;s links
+ are stored &ldquo;compactly&rdquo; (as object header messages), this
+ value will be the <a href="#UndefinedAddress">undefined
+ address</a>.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address of v2 B-tree for Name Index</p></td>
+ <td><p>This is the address of the version 2 B-tree to index names of links.</p>
+ <p>If there are no links in the group, or the group&rsquo;s links
+ are stored &ldquo;compactly&rdquo; (as object header messages), this
+ value will be the <a href="#UndefinedAddress">undefined
+ address</a>.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address of v2 B-tree for Creation Order Index</p></td>
+ <td><p>This is the address of the version 2 B-tree to index creation order of links.</p>
+ <p>If there are no links in the group, or the group&rsquo;s links
+ are stored &ldquo;compactly&rdquo; (as object header messages), this
+ value will be the <a href="#UndefinedAddress">undefined
+ address</a>.
+ </p>
+ <p>This field exists if bit 1 of <em>flags</em> is set.</p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+<br />
+<h4><a name="DatatypeMessage">IV.A.2.d. The Datatype Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Datatype</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0003
+ </td></tr>
+ <tr><td colspan="2"><b>Length:</b> Variable</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Required for dataset or committed
+ datatype (formerly named datatype) objects; may not be repeated.
+ </td></tr>
+ <tr><td><b>Description:</b></td>
+ <td><p>The datatype message defines the datatype for each element
+ of a dataset or a common datatype for sharing between multiple
+ datasets. A datatype can describe an atomic type like a fixed-
+ or floating-point type or more complex types like a C struct
+ (compound datatype), array (array datatype) or C++ vector
+ (variable-length datatype).</p>
+ <p>Datatype messages that are part of a dataset object do not
+ describe how elements are related to one another; the dataspace
+ message is used for that purpose. Datatype messages that are part of
+ a committed datatype (formerly named datatype) message describe
+ a common datatype that can be shared by multiple datasets in the
+ file.</p>
+ </td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Datatype Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Class and Version</td>
+ <td>Class Bit Field, Bits 0-7</td>
+ <td>Class Bit Field, Bits 8-15</td>
+ <td>Class Bit Field, Bits 16-23</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br /><br />Properties<br /><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Class and Version</p></td>
+ <td>
+ <p>The version of the datatype message and the datatype&rsquo;s class
+ information are packed together in this field. The version
+ number is packed in the top 4 bits of the field and the class
+ is contained in the bottom 4 bits.
+ </p>
+ <p>The version number information is used for changes in the
+ format of the datatype message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Never used
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Used by early versions of the library to encode
+ compound datatypes with explicit array fields.
+ See the compound datatype description below for
+ further details.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Used when an array datatype needs to be encoded.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Used when a VAX byte-ordered type needs to be
+ encoded. Packs various other datatype classes more
+ efficiently also.
+ </td>
+ </tr>
+ </table></p>
+
+ <p>The class of the datatype determines the format for the class
+ bit field and properties portion of the datatype message, which
+ are described below. The
+ following classes are currently defined:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Fixed-Point</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Floating-Point</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Time</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>String</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>Bit field</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>5</code></td>
+ <td>Opaque</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6</code></td>
+ <td>Compound</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>7</code></td>
+ <td>Reference</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>8</code></td>
+ <td>Enumerated</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>9</code></td>
+ <td>Variable-Length</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>10</code></td>
+ <td>Array</td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Class Bit Fields</p></td>
+ <td>
+ <p>The information in these bit fields is specific to each datatype
+ class and is described below. All bits not defined for a
+ datatype class are set to zero.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size</p></td>
+ <td>
+ <p>The size of a datatype element in bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Properties</p></td>
+ <td>
+ <p>This variable-sized sequence of bytes encodes information
+ specific to each datatype class and is described for each class
+ below. If there is no property information specified for a
+ datatype class, the size of this field is zero bytes.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+ <br />
+ <p>Class specific information for Fixed-Point Numbers (Class 0):</p>
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fixed-point Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0</p></td>
+ <td><p><b>Byte Order.</b> If zero, byte order is little-endian;
+ otherwise, byte order is big endian.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>1, 2</p></td>
+ <td><p><b>Padding type.</b> Bit 1 is the lo_pad bit and bit 2
+ is the hi_pad bit. If a datum has unused bits at either
+ end, then the lo_pad or hi_pad bit is copied to those
+ locations.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>3</p></td>
+ <td><p><b>Signed.</b> If this bit is set then the fixed-point
+ number is in 2&rsquo;s complement form.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>4-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Fixed-Point Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="2">Bit Offset</td>
+ <td colspan="2">Bit Precision</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Bit Offset</p></td>
+ <td>
+ <p>The bit offset of the first significant bit of the fixed-point
+ value within the datatype. The bit offset specifies the number
+ of bits &ldquo;to the right of&rdquo; the value (which are set to the
+ lo_pad bit value).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Bit Precision</p></td>
+ <td>
+ <p>The number of bits of precision of the fixed-point value
+ within the datatype. This value, combined with the datatype
+ element&rsquo;s size and the Bit Offset field specifies the number
+ of bits &ldquo;to the left of&rdquo; the value (which are set to the
+ hi_pad bit value).
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+ <br />
+ <p>Class specific information for Floating-Point Numbers (Class 1):</p>
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Floating-Point Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0, 6</p></td>
+ <td><p><b>Byte Order.</b> These two non-contiguous bits specify the
+ &ldquo;endianness&rdquo; of the bytes in the datatype element.
+ <table class="list">
+ <tr>
+ <th width="10%" align="center">Bit 6</th>
+ <th width="10%" align="center">Bit 0</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td align="center"><code>0</code></td>
+ <td>Byte order is little-endian
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td align="center"><code>1</code></td>
+ <td>Byte order is big-endian
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td align="center"><code>0</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td align="center"><code>1</code></td>
+ <td>Byte order is VAX-endian
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>1, 2, 3</p></td>
+ <td><p><b>Padding type.</b> Bit 1 is the low bits pad type, bit 2
+ is the high bits pad type, and bit 3 is the internal bits
+ pad type. If a datum has unused bits at either end or between
+ the sign bit, exponent, or mantissa, then the value of bit
+ 1, 2, or 3 is copied to those locations.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>4-5</p></td>
+ <td><p><b>Mantissa Normalization.</b> This 2-bit bit field specifies
+ how the most significant bit of the mantissa is managed.
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>No normalization
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>The most significant bit of the mantissa is always set
+ (except for 0.0).
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>The most significant bit of the mantissa is not stored,
+ but is implied to be set.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>7</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+
+ <tr>
+ <td><p>8-15</p></td>
+ <td><p><b>Sign Location.</b> This is the bit position of the sign
+ bit. Bits are numbered with the least significant bit zero.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>16-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Floating-Point Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="2">Bit Offset</td>
+ <td colspan="2">Bit Precision</td>
+ </tr>
+
+ <tr>
+ <td>Exponent Location</td>
+ <td>Exponent Size</td>
+ <td>Mantissa Location</td>
+ <td>Mantissa Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Exponent Bias</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Bit Offset</p></td>
+ <td>
+ <p>The bit offset of the first significant bit of the floating-point
+ value within the datatype. The bit offset specifies the number
+ of bits &ldquo;to the right of&rdquo; the value.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Bit Precision</p></td>
+ <td>
+ <p>The number of bits of precision of the floating-point value
+ within the datatype.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Exponent Location</p></td>
+ <td>
+ <p>The bit position of the exponent field. Bits are numbered with
+ the least significant bit number zero.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Exponent Size</p></td>
+ <td>
+ <p>The size of the exponent field in bits.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Mantissa Location</p></td>
+ <td>
+ <p>The bit position of the mantissa field. Bits are numbered with
+ the least significant bit number zero.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Mantissa Size</p></td>
+ <td>
+ <p>The size of the mantissa field in bits.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Exponent Bias</p></td>
+ <td>
+ <p>The bias of the exponent field.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+ <br />
+ <p>Class specific information for Time (Class 2):</p>
+
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Time Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0</p></td>
+ <td><p><b>Byte Order.</b> If zero, byte order is little-endian;
+ otherwise, byte order is big endian.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>1-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Time Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="2">Bit Precision</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Bit Precision</p></td>
+ <td>
+ <p>The number of bits of precision of the time value.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+ <br />
+ <p>Class specific information for Strings (Class 3):</p>
+
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ String Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0-3</p></td>
+ <td><p><b>Padding type.</b> This four-bit value determines the
+ type of padding to use for the string. The values are:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Null Terminate: A zero byte marks the end of the
+ string and is guaranteed to be present after
+ converting a long string to a short string. When
+ converting a short string to a long string the value is
+ padded with additional null characters as necessary.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Null Pad: Null characters are added to the end of
+ the value during conversions from short values to long
+ values but conversion in the opposite direction simply
+ truncates the value.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Space Pad: Space characters are added to the end of
+ the value during conversions from short values to long
+ values but conversion in the opposite direction simply
+ truncates the value. This is the Fortran
+ representation of the string.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>4-7</p></td>
+ <td><p><b>Character Set.</b> The character set used to
+ encode the string.
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>ASCII character set encoding
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>UTF-8 character set encoding
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>8-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <p>There are no properties defined for the string class.
+ </p>
+
+
+ <p>Class specific information for bit fields (Class 4):</p>
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Bitfield Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0</p></td>
+ <td><p><b>Byte Order.</b> If zero, byte order is little-endian;
+ otherwise, byte order is big endian.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>1, 2</p></td>
+ <td><p><b>Padding type.</b> Bit 1 is the lo_pad type and bit 2
+ is the hi_pad type. If a datum has unused bits at either
+ end, then the lo_pad or hi_pad bit is copied to those
+ locations.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>3-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Bit Field Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="2">Bit Offset</td>
+ <td colspan="2">Bit Precision</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Bit Offset</p></td>
+ <td>
+ <p>The bit offset of the first significant bit of the bit field
+ within the datatype. The bit offset specifies the number
+ of bits &ldquo;to the right of&rdquo; the value.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Bit Precision</p></td>
+ <td>
+ <p>The number of bits of precision of the bit field
+ within the datatype.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+ <br />
+ <p>Class specific information for Opaque (Class 5):</p>
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Opaque Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0-7</p></td>
+ <td><p>Length of ASCII tag in bytes.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>8-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Opaque Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />ASCII Tag<br />
+ <br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>ASCII Tag</p></td>
+ <td>
+ <p>This NUL-terminated string provides a description for the
+ opaque type. It is NUL-padded to a multiple of 8 bytes.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+ <br />
+ <p>Class specific information for Compound (Class 6):</p>
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Compound Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0-15</p></td>
+ <td><p><b>Number of Members.</b> This field contains the number
+ of members defined for the compound datatype. The member
+ definitions are listed in the Properties field of the data
+ type message.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>16-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+
+ <p>The Properties field of a compound datatype is a list of the
+ member definitions of the compound datatype. The member
+ definitions appear one after another with no intervening bytes.
+ The member types are described with a (recursively) encoded datatype
+ message.</p>
+
+ <p>Note that the property descriptions are different for different
+ versions of the datatype version. Additionally note that the version
+ 0 datatype encoding is deprecated and has been replaced with later
+ encodings in versions of the HDF5 Library from the 1.4 release
+ onward.</p>
+
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Compound Properties Description for Datatype Version 1
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Byte Offset of Member</td>
+ </tr>
+
+ <tr>
+ <td>Dimensionality</td>
+ <td colspan="3">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension Permutation</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension #1 Size (required)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension #2 Size (required)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension #3 Size (required)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension #4 Size (required)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Member Type Message<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td>
+ <p>This NUL-terminated string provides a description for the
+ opaque type. It is NUL-padded to a multiple of 8 bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Byte Offset of Member</p></td>
+ <td>
+ <p>This is the byte offset of the member within the datatype.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimensionality</p></td>
+ <td>
+ <p>If set to zero, this field indicates a scalar member. If set
+ to a value greater than zero, this field indicates that the
+ member is an array of values. For array members, the size of
+ the array is indicated by the &lsquo;Size of Dimension n&rsquo; field in
+ this message.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension Permutation</p></td>
+ <td>
+ <p>This field was intended to allow an array field to have
+ its dimensions permuted, but this was never implemented.
+ This field should always be set to zero.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Size</p></td>
+ <td>
+ <p>This field is the size of a dimension of the array field as
+ stored in the file. The first dimension stored in the list of
+ dimensions is the slowest changing dimension and the last
+ dimension stored is the fastest changing dimension.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Member Type Message</p></td>
+ <td>
+ <p>This field is a datatype message describing the datatype of
+ the member.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Compound Properties Description for Datatype Version 2
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Byte Offset of Member</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Member Type Message<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td>
+ <p>This NUL-terminated string provides a description for the
+ opaque type. It is NUL-padded to a multiple of 8 bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Byte Offset of Member</p></td>
+ <td>
+ <p>This is the byte offset of the member within the datatype.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Member Type Message</p></td>
+ <td>
+ <p>This field is a datatype message describing the datatype of
+ the member.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Compound Properties Description for Datatype Version 3
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Byte Offset of Member <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Member Type Message<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td><p>This NUL-terminated string provides a description for the
+ opaque type. It is <em>not</em> NUL-padded to a multiple of 8
+ bytes.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Byte Offset of Member</p></td>
+ <td><p>This is the byte offset of the member within the datatype.
+ The field size is the minimum number of bytes necessary,
+ based on the size of the datatype element. For example, a
+ datatype element size of less than 256 bytes uses a 1 byte
+ length, a datatype element size of 256-65535 bytes uses a
+ 2 byte length, and so on.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Member Type Message</p></td>
+ <td><p>This field is a datatype message describing the datatype of
+ the member.</p></td>
+ </tr>
+
+ </table>
+ </div>
+
+
+ <br />
+ <p>Class specific information for Reference (Class 7):</p>
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Reference Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0-3</p></td>
+ <td><p><b>Type.</b> This four-bit value contains the type of reference
+ described. The values defined are:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Object Reference: A reference to another object in this
+ HDF5 file.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Dataset Region Reference: A reference to a region within
+ a dataset in this HDF5 file.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>4-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <p>There are no properties defined for the reference class.
+ </p>
+
+
+ <br />
+ <p>Class specific information for Enumeration (Class 8):</p>
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Enumeration Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0-15</p></td>
+ <td><p><b>Number of Members.</b> The number of name/value
+ pairs defined for the enumeration type.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>16-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Enumeration Property Description for Datatype Versions 1 & 2
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Base Type<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Names<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Values<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Base Type</p></td>
+ <td>
+ <p>Each enumeration type is based on some parent type, usually an
+ integer. The information for that parent type is described
+ recursively by this field.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Names</p></td>
+ <td>
+ <p>The name for each name/value pair. Each name is stored as a null
+ terminated ASCII string in a multiple of eight bytes. The names
+ are in no particular order.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Values</p></td>
+ <td>
+ <p>The list of values in the same order as the names. The values
+ are packed (no inter-value padding) and the size of each value
+ is determined by the parent type.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Enumeration Property Description for Datatype Version 3
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Base Type<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Names<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Values<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Base Type</p></td>
+ <td>
+ <p>Each enumeration type is based on some parent type, usually an
+ integer. The information for that parent type is described
+ recursively by this field.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Names</p></td>
+ <td>
+ <p>The name for each name/value pair. Each name is stored as a null
+ terminated ASCII string, <em>not</em> padded to a multiple of
+ eight bytes. The names are in no particular order.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Values</p></td>
+ <td>
+ <p>The list of values in the same order as the names. The values
+ are packed (no inter-value padding) and the size of each value
+ is determined by the parent type.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+
+ <br />
+ <p>Class specific information for Variable-Length (Class 9):</p>
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Variable-Length Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0-3</p></td>
+ <td><p><b>Type.</b> This four-bit value contains the type of
+ variable-length datatype described. The values defined are:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Sequence: A variable-length sequence of any datatype.
+ Variable-length sequences do not have padding or
+ character set information.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>String: A variable-length sequence of characters.
+ Variable-length strings have padding and character set
+ information.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>4-7</p></td>
+ <td><p><b>Padding type.</b> (variable-length string only)
+ This four-bit value determines the type of padding
+ used for variable-length strings. The values are the same
+ as for the string padding type, as follows:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Null terminate: A zero byte marks the end of a string
+ and is guaranteed to be present after converting a long
+ string to a short string. When converting a short string
+ to a long string, the value is padded with additional null
+ characters as necessary.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Null pad: Null characters are added to the end of the
+ value during conversion from a short string to a longer
+ string. Conversion from a long string to a shorter string
+ simply truncates the value.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Space pad: Space characters are added to the end of the
+ value during conversion from a short string to a longer
+ string. Conversion from a long string to a shorter string
+ simply truncates the value. This is the Fortran
+ representation of the string.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table></p>
+
+ <p>This value is set to zero for variable-length sequences.</p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>8-11</p></td>
+ <td><p><b>Character Set.</b> (variable-length string only)
+ This four-bit value specifies the character set
+ to be used for encoding the string:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>ASCII character set encoding
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>UTF-8 character set encoding
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table></p>
+
+ <p>This value is set to zero for variable-length sequences.</p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>12-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Variable-Length Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Base Type<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="10%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Base Type</p></td>
+ <td>
+ <p>Each variable-length type is based on some parent type. The
+ information for that parent type is described recursively by
+ this field.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+ <br />
+ <p>Class specific information for Array (Class 10):</p>
+
+ <p>There are no bit fields defined for the array class.
+ </p>
+
+ <p>Note that the dimension information defined in the property for this
+ datatype class is independent of dataspace information for a dataset.
+ The dimension information here describes the dimensionality of the
+ information within a data element (or a component of an element, if the
+ array datatype is nested within another datatype) and the dataspace for a
+ dataset describes the size and locations of the elements in a dataset.
+ </p>
+
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Array Property Description for Datatype Version 2
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td>Dimensionality</td>
+ <td colspan="3">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension #1 Size</td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Dimension #n Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Permutation Index #1</td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Permutation Index #n</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Base Type<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Dimensionality</p></td>
+ <td>
+ <p>This value is the number of dimensions that the array has.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Size</p></td>
+ <td>
+ <p>This value is the size of the dimension of the array
+ as stored in the file. The first dimension stored in
+ the list of dimensions is the slowest changing dimension
+ and the last dimension stored is the fastest changing
+ dimension.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Permutation Index #n</p></td>
+ <td>
+ <p>This value is the index permutation used to map
+ each dimension from the canonical representation to an
+ alternate axis for each dimension. Currently, dimension
+ permutations are not supported, and these indices should
+ be set to the index position minus one. In other words,
+ the first dimension should be set to 0, the second dimension
+ should be set to 1, and so on.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Base Type</p></td>
+ <td>
+ <p>Each array type is based on some parent type. The
+ information for that parent type is described recursively by
+ this field.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Array Property Description for Datatype Version 3
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td>Dimensionality</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension #1 Size</td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Dimension #n Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Base Type<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Dimensionality</p></td>
+ <td>
+ <p>This value is the number of dimensions that the array has.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Size</p></td>
+ <td>
+ <p>This value is the size of the dimension of the array
+ as stored in the file. The first dimension stored in
+ the list of dimensions is the slowest changing dimension
+ and the last dimension stored is the fastest changing
+ dimension.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Base Type</p></td>
+ <td>
+ <p>Each array type is based on some parent type. The
+ information for that parent type is described recursively by
+ this field.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+
+<br />
+<h4><a name="OldFillValueMessage">IV.A.2.e. The Data Storage -
+Fill Value (Old) Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Fill Value
+ (old)</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0004</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td><p>The fill value message stores a single data value which
+ is returned to the application when an uninitialized data element
+ is read from a dataset. The fill value is interpreted with the
+ same datatype as the dataset. If no fill value message is present
+ then a fill value of all zero bytes is assumed.</p>
+ <p>This fill value message is deprecated in favor of the
+ &ldquo;new&rdquo; fill value message (Message Type 0x0005) and
+ is only written to the file for forward compatibility with
+ versions of the HDF5 Library before the 1.6.0 version.
+ Additionally, it only appears for datasets with a user-defined
+ fill value (as opposed to the library default fill value or an
+ explicitly set &ldquo;undefined&rdquo; fill value).</p>
+ </td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Fill Value Message (Old)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Fill Value <em>(optional, variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Size</p></td>
+ <td>
+ <p>This is the size of the Fill Value field in bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fill Value</p></td>
+ <td>
+ <p>The fill value. The bytes of the fill value are interpreted
+ using the same datatype as for the dataset.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+<br />
+<h4><a name="FillValueMessage">IV.A.2.f. The Data Storage -
+Fill Value Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Fill
+ Value</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0005</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Required for dataset objects;
+ may not be repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>The fill value message stores a single data value which is
+ returned to the application when an uninitialized data element
+ is read from a dataset. The fill value is interpreted with the
+ same datatype as the dataset.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Fill Value Message - Versions 1 & 2
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Space Allocation Time</td>
+ <td>Fill Value Write Time</td>
+ <td>Fill Value Defined</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Size <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Fill Value <em>(optional, variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number information is used for changes in the
+ format of the fill value message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Never used
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Initial version of this message.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>In this version, the Size and Fill Value fields are
+ only present if the Fill Value Defined field is set
+ to 1.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>This version packs the other fields in the message
+ more efficiently than version 2.
+ </td>
+ </tr>
+ </table></p>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Space Allocation Time</p></td>
+ <td>
+ <p>When the storage space for the dataset&rsquo;s raw data will be
+ allocated. The allowed values are:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Not used.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Early allocation. Storage space for the entire dataset
+ should be allocated in the file when the dataset is
+ created.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Late allocation. Storage space for the entire dataset
+ should not be allocated until the dataset is written
+ to.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Incremental allocation. Storage space for the
+ dataset should not be allocated until the portion
+ of the dataset is written to. This is currently
+ used in conjunction with chunked data storage for
+ datasets.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fill Value Write Time</p></td>
+ <td>
+ <p>At the time that storage space for the dataset&rsquo;s raw data is
+ allocated, this value indicates whether the fill value should
+ be written to the raw data storage elements. The allowed values
+ are:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>On allocation. The fill value is always written to
+ the raw data storage when the storage space is allocated.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Never. The fill value should never be written to
+ the raw data storage.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Fill value written if set by user. The fill value
+ will be written to the raw data storage when the storage
+ space is allocated only if the user explicitly set
+ the fill value. If the fill value is the library
+ default or is undefined, it will not be written to
+ the raw data storage.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fill Value Defined</p></td>
+ <td>
+ <p>This value indicates if a fill value is defined for this
+ dataset. If this value is 0, the fill value is undefined.
+ If this value is 1, a fill value is defined for this dataset.
+ For version 2 or later of the fill value message, this value
+ controls the presence of the Size and Fill Value fields.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size</p></td>
+ <td>
+ <p>This is the size of the Fill Value field in bytes. This field
+ is not present if the Version field is greater than 1,
+ and the Fill Value Defined field is set to 0.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fill Value</p></td>
+ <td>
+ <p>The fill value. The bytes of the fill value are interpreted
+ using the same datatype as for the dataset. This field is
+ not present if the Version field is greater than 1,
+ and the Fill Value Defined field is set to 0.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Fill Value Message - Version 3
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Size <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Fill Value <em>(optional, variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number information is used for changes in the
+ format of the fill value message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Never used
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Initial version of this message.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>In this version, the Size and Fill Value fields are
+ only present if the Fill Value Defined field is set
+ to 1.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>This version packs the other fields in the message
+ more efficiently than version 2.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td>
+ <p>When the storage space for the dataset&rsquo;s raw data will be
+ allocated. The allowed values are:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bits</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0-1</code></td>
+ <td>Space Allocation Time, with the same
+ values as versions 1 and 2 of the message.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2-3</code></td>
+ <td>Fill Value Write Time, with the same
+ values as versions 1 and 2 of the message.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>Fill Value Undefined, indicating that the fill
+ value has been marked as &ldquo;undefined&rdquo; for this dataset.
+ Bits 4 and 5 cannot both be set.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>5</code></td>
+ <td>Fill Value Defined, with the same values as
+ versions 1 and 2 of the message.
+ Bits 4 and 5 cannot both be set.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>6-7</code></td>
+ <td>Reserved (zero).
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size</p></td>
+ <td>
+ <p>This is the size of the Fill Value field in bytes. This field
+ is not present if the Version field is greater than 1,
+ and the Fill Value Defined flag is set to 0.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fill Value</p></td>
+ <td>
+ <p>The fill value. The bytes of the fill value are interpreted
+ using the same datatype as for the dataset. This field is
+ not present if the Version field is greater than 1,
+ and the Fill Value Defined flag is set to 0.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+<br />
+<h4><a name="LinkMessage">IV.A.2.g. The Link Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Link</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0006</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies </td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may be
+ repeated. </td></tr>
+ <tr><td><b>Description:</b></td>
+ <td><p>This message encodes the information for a link in a
+ group&rsquo;s object header, when the group is storing its links
+ &ldquo;compactly&rdquo;, or in the group&rsquo;s fractal heap,
+ when the group is storing its links &ldquo;densely&rdquo;.</p>
+ <p>A group is storing its links compactly when the fractal heap
+ address in the <em><a href="#LinkInfoMessage">Link Info
+ Message</a></em> is set to the &ldquo;undefined address&rdquo;
+ value.</p></td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Link Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td>Link type <em>(optional)</em></td>
+ <td bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Creation Order <em>(8 bytes, optional)</em><br /><br /></td>
+ </tr>
+ <tr>
+ <td>Link Name Character Set <em>(optional)</em></td>
+ <td>Length of Link Name (variable size)</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Link Name (variable size)</td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Link Information (variable size)<br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This document describes version 1.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>This field contains information about the link and controls
+ the presence of other fields below.
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bits</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0-1</code></td>
+ <td>Determines the size of the <em>Length of Link Name</em>
+ field.
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>The size of the <em>Length of Link Name</em>
+ field is 1 byte.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>The size of the <em>Length of Link Name</em>
+ field is 2 bytes.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>The size of the <em>Length of Link Name</em>
+ field is 4 bytes.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>The size of the <em>Length of Link Name</em>
+ field is 8 bytes.
+ </td>
+ </tr>
+ </table>
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Creation Order Field Present: if set, the <em>Creation
+ Order</em> field is present. If not set, creation order
+ information is not stored for links in this group.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Link Type Field Present: if set, the link is not
+ a hard link and the <em>Link Type</em> field is present.
+ If not set, the link is a hard link.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>Link Name Character Set Field Present: if set, the
+ link name is not represented with the ASCII character
+ set and the <em>Link Name Character Set</em> field is
+ present. If not set, the link name is represented with
+ the ASCII character set.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>5-7</code></td>
+ <td>Reserved (zero).
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Link type</p></td>
+ <td><p>This is the link class type and can be one of the following
+ values:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>A hard link (should never be stored in the file)
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>A soft link.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2-63</code></td>
+ <td>Reserved for future HDF5 internal use.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>64</code></td>
+ <td>An external link.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>65-255</code></td>
+ <td>Reserved, but available for user-defined link types.
+ </td>
+ </tr>
+ </table></p>
+
+ <p>This field is present if bit 3 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Creation Order</p></td>
+ <td><p>This 64-bit value is an index of the link&rsquo;s creation time within
+ the group. Values start at 0 when the group is created an increment
+ by one for each link added to the group. Removing a link from a
+ group does not change existing links&rsquo; creation order field.
+ </p>
+ <p>This field is present if bit 2 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Link Name Character Set</p></td>
+ <td><p>This is the character set for encoding the link&rsquo;s name:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>ASCII character set encoding (this should never be stored
+ in the file)
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>UTF-8 character set encoding
+ </td>
+ </tr>
+ </table></p>
+
+ <p>This field is present if bit 4 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Length of link name</p></td>
+ <td><p>This is the length of the link&rsquo;s name. The size of this field
+ depends on bits 0 and 1 of <em>Flags</em>.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Link name</p></td>
+ <td><p>This is the name of the link, non-NULL terminated.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Link information</p></td>
+ <td><p>The format of this field depends on the <em>link type</em>.</p>
+ <p>For <b>hard</b> links, the field is formatted as follows:
+
+ <table class="list">
+ <tr>
+ <td width="20%"><i>Size of Offsets</i> bytes:</td>
+ <td width="80%">The address of the object header for the object that the
+ link points to.
+ </td>
+ </tr>
+ </table>
+ </p>
+
+ <p>
+ For <b>soft</b> links, the field is formatted as follows:
+
+ <table class="list">
+ <tr>
+ <td width="20%">Bytes 1-2:</td>
+ <td width="80%">Length of soft link value.</td>
+ </tr>
+ <tr>
+ <td><em>Length of soft link value</em> bytes:</td>
+ <td>A non-NULL-terminated string storing the value of the
+ soft link.
+ </td>
+ </tr>
+ </table>
+ </p>
+
+ <p>
+ For <b>external</b> links, the field is formatted as follows:
+
+ <table class="list">
+ <tr>
+ <td width="20%">Bytes 1-2:</td>
+ <td width="80%">Length of external link value.</td>
+ </tr>
+ <tr>
+ <td><em>Length of external link value</em> bytes:</td>
+ <td>The first byte contains the version number in the
+ upper 4 bits and flags in the lower 4 bits for the external
+ link. Both version and flags are defined to be zero in
+ this document. The remaining bytes consist of two
+ NULL-terminated strings, with no padding between them.
+ The first string is the name of the HDF5 file containing
+ the object linked to and the second string is the full path
+ to the object linked to, within the HDF5 file&rsquo;s
+ group hierarchy.
+ </td>
+ </tr>
+ </table>
+ </p>
+
+ <p>
+ For <b>user-defined</b> links, the field is formatted as follows:
+
+ <table class="list">
+ <tr>
+ <td width="20%">Bytes 1-2:</td>
+ <td width="80%">Length of user-defined data.</td>
+ </tr>
+ <tr>
+ <td><em>Length of user-defined link value</em> bytes:</td>
+ <td>The data supplied for the user-defined link type.</td>
+ </tr>
+ </table>
+ </p>
+
+ </td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h4><a name="ExternalFileListMessage">IV.A.2.h. The Data Storage -
+External Data Files Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> External
+ Data Files</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0007</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>The external data storage message indicates that the data
+ for an object is stored outside the HDF5 file. The filename of
+ the object is stored as a Universal Resource Location (URL) of
+ the actual filename containing the data. An external file list
+ record also contains the byte offset of the start of the data
+ within the file and the amount of space reserved in the file
+ for that data.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ External File List Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Allocated Slots</td>
+ <td colspan="2">Used Slots</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Heap Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Slot Definitions...<br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number information is used for changes in the format of
+ External Data Storage Message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Never used.</td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>The current version used by the library.</td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Allocated Slots</p></td>
+ <td>
+ <p>The total number of slots allocated in the message. Its value must be at least as
+ large as the value contained in the Used Slots field. (The current library simply
+ uses the number of Used Slots for this message)</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Used Slots</p></td>
+ <td>
+ <p>The number of initial slots which contains valid information.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Heap Address</p></td>
+ <td>
+ <p>This is the address of a local heap which contains the names for the external
+ files (The local heap information can be found in Disk Format Level 1D in this
+ document). The name at offset zero in the heap is always the empty string.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Slot Definitions</p></td>
+ <td>
+ <p>The slot definitions are stored in order according to the array addresses they
+ represent.</p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ External File List Slot
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name Offset in Local Heap<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Offset in External Data File<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Data Size in External File<sup>L</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Name Offset in Local Heap</p></td>
+ <td>
+ <p>The byte offset within the local name heap for the name
+ of the file. File names are stored as a URL which has a
+ protocol name, a host name, a port number, and a file
+ name:
+ <code><em>protocol</em>:<em>port</em>//<em>host</em>/<em>file</em></code>.
+ If the protocol is omitted then &ldquo;file:&rdquo; is assumed. If
+ the port number is omitted then a default port for that
+ protocol is used. If both the protocol and the port
+ number are omitted then the colon can also be omitted. If
+ the double slash and host name are omitted then
+ &ldquo;localhost&rdquo; is assumed. The file name is the only
+ mandatory part, and if the leading slash is missing then
+ it is relative to the application&rsquo;s current working
+ directory (the use of relative names is not
+ recommended).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Offset in External Data File</p></td>
+ <td>
+ <p>This is the byte offset to the start of the data in the
+ specified file. For files that contain data for a single
+ dataset this will usually be zero.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data Size in External File</p></td>
+ <td>
+ <p>This is the total number of bytes reserved in the
+ specified file for raw data storage. For a file that
+ contains exactly one complete dataset which is not
+ extendable, the size will usually be the exact size of the
+ dataset. However, by making the size larger one allows
+ HDF5 to extend the dataset. The size can be set to a value
+ larger than the entire file since HDF5 will read zeroes
+ past the end of the file without failing.</p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+<br />
+<h4><a name="LayoutMessage">IV.A.2.i. The Data Storage - Layout
+Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Data Storage -
+ Layout</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0008</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Required for datasets; may not
+ be repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>Data layout describes how the elements of a multi-dimensional
+ array are stored in the HDF5 file. Three types of data layout
+ are supported:
+ <ol>
+ <li>Contiguous: The array is stored in one contiguous area of
+ the file. This layout requires that the size of the array be
+ constant: data manipulations such as chunking, compression,
+ checksums, or encryption are not permitted. The message stores
+ the total storage size of the array. The offset of an element
+ from the beginning of the storage area is computed as in a C
+ array.</li>
+ <li>Chunked: The array domain is regularly decomposed into
+ chunks, and each chunk is allocated and stored separately. This
+ layout supports arbitrary element traversals, compression,
+ encryption, and checksums. (these features are described
+ in other messages). The message stores the size of a chunk
+ instead of the size of the entire array; the storage size of
+ the entire array can be calculated by traversing the B-tree
+ that stores the chunk addresses.</li>
+ <li>Compact: The array is stored in one contiguous block, as
+ part of this object header message.</li>
+ </ol></td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Data Layout Message (Versions 1 and 2)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Dimensionality</td>
+ <td>Layout Class</td>
+ <td>Reserved <em>(zero)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reserved <em>(zero)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Data Address<sup>O</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension 0 Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension 1 Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension #n Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dataset Element Size <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Compact Data Size <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Compact Data... <em>(variable size, optional)</em><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number information is used for changes in the format of the data
+ layout message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Never used.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Used by version 1.4 and before of the library to encode layout information.
+ Data space is always allocated when the data set is created.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Used by version 1.6.x of the library to encode layout information.
+ Data space is allocated only when it is necessary.</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimensionality</p></td>
+ <td><p>An array has a fixed dimensionality. This field
+ specifies the number of dimension size fields later in the
+ message. The value stored for chunked storage is 1 greater than
+ the number of dimensions in the dataset&rsquo;s dataspace.
+ For example, 2 is stored for a 1 dimensional dataset.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Layout Class</p></td>
+ <td><p>The layout class specifies the type of storage for the data
+ and how the other fields of the layout message are to be
+ interpreted.
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Compact Storage
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Contiguous Storage
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Chunked Storage
+ </td>
+ </tr>
+ </table>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data Address</p></td>
+ <td><p>For contiguous storage, this is the address of the raw
+ data in the file. For chunked storage this is the address
+ of the v1 B-tree that is used to look up the addresses of the
+ chunks. This field is not present for compact storage.
+ If the version for this message is greater than 1, the address
+ may have the &ldquo;undefined address&rdquo; value, to indicate that
+ storage has not yet been allocated for this array.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Size</p></td>
+ <td><p>For contiguous and compact storage the dimensions define
+ the entire size of the array while for chunked storage they define
+ the size of a single chunk. In all cases, they are in units of
+ array elements (not bytes). The first dimension stored in the list
+ of dimensions is the slowest changing dimension and the last
+ dimension stored is the fastest changing dimension.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dataset Element Size</p></td>
+ <td><p>The size of a dataset element, in bytes. This field is only
+ present for chunked storage.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Compact Data Size</p></td>
+ <td><p>This field is only present for compact data storage.
+ It contains the size of the raw data for the dataset array, in
+ bytes.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Compact Data</p></td>
+ <td><p>This field is only present for compact data storage.
+ It contains the raw data for the dataset array.</p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <p>Version 3 of this message re-structured the format into specific
+ properties that are required for each layout class.</p>
+
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ <b>Data Layout Message (Version 3)</b>
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Layout Class</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Properties <em>(variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number information is used for changes in the format of layout message
+ and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Used by the version 1.6.3 and later of the library to store properties
+ for each layout class.</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Layout Class</p></td>
+ <td><p>The layout class specifies the type of storage for the data
+ and how the other fields of the layout message are to be
+ interpreted.
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Compact Storage
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Contiguous Storage
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Chunked Storage
+ </td>
+ </tr>
+ </table>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Properties</p></td>
+ <td><p>This variable-sized field encodes information specific to each
+ layout class and is described below. If there is no property
+ information specified for a layout class, the size of this field
+ is zero bytes.</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <p>Class-specific information for compact layout (Class 0): (Note: The dimensionality information
+ is in the Dataspace message)</p>
+
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Compact Storage Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="2">Size</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Raw Data... <em>(variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Size</p></td>
+ <td><p>This field contains the size of the raw data for the dataset
+ array, in bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Raw Data</p></td>
+ <td><p>This field contains the raw data for the dataset array.</p></td>
+ </tr>
+ </table>
+ </div>
+
+
+ <br />
+ <p>Class-specific information for contiguous layout (Class 1): (Note: The dimensionality information
+ is in the Dataspace message)</p>
+
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Contiguous Storage Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Size<sup>L</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>This is the address of the raw data in the file.
+ The address may have the &ldquo;undefined address&rdquo; value, to indicate
+ that storage has not yet been allocated for this array.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Size</p></td>
+ <td><p>This field contains the size allocated to store the raw data,
+ in bytes.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+ <br />
+ <p>Class-specific information for chunked layout (Class 2):</p>
+
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Chunked Storage Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Dimensionality</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension 0 Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension 1 Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension #n Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dataset Element Size</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Dimensionality</p></td>
+ <td><p>A chunk has a fixed dimensionality. This field specifies
+ the number of dimension size fields later in the message.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>This is the address of the v1 B-tree that is used to look up the
+ addresses of the chunks that actually store portions of the array
+ data. The address may have the &ldquo;undefined address&rdquo; value, to
+ indicate that storage has not yet been allocated for this array.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Size</p></td>
+ <td><p>These values define the dimension size of a single chunk, in
+ units of array elements (not bytes). The first dimension stored in
+ the list of dimensions is the slowest changing dimension and the
+ last dimension stored is the fastest changing dimension.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dataset Element Size</p></td>
+ <td><p>The size of a dataset element, in bytes.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h4><a name="BogusMessage">IV.A.2.j. The Bogus Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Bogus</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0009</td></tr>
+ <tr><td colspan="2"><b>Length:</b> 4 bytes</td></tr>
+ <tr><td colspan="2"><b>Status:</b> For testing only; should never
+ be stored in a valid file.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>This message is used for testing the HDF5 Library&rsquo;s
+ response to an &ldquo;unknown&rdquo; message type and should
+ never be encountered in a valid HDF5 file.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Bogus Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Bogus Value</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Bogus Value</p></td>
+ <td>
+ <p>This value should always be: <code>0xdeadbeef</code>.</p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h4><a name="GroupInfoMessage">IV.A.2.k. The Group Info Message
+</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Group Info</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x000A</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td><p>This message stores information for the constants defining
+ a &ldquo;new style&rdquo; group&rsquo;s behavior. Constant
+ information will be stored in this message and variable
+ information will be stored in the
+ <a href="#LinkInfoMessage">Link Info</a> message.</p>
+ <p>Note: the &ldquo;estimated entry&rdquo; information below is
+ used when determining the size of the object header for the
+ group when it is created.</p></td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Group Info Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td colspan="2">Link Phase Change: Maximum Compact Value <em>(optional)</em></td>
+ </tr>
+ <tr>
+ <td colspan="2">Link Phase Change: Minimum Dense Value <em>(optional)</em></td>
+ <td colspan="2">Estimated Number of Entries <em>(optional)</em></td>
+ </tr>
+ <tr>
+ <td colspan="2">Estimated Link Name Length of Entries <em>(optional)</em></td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>This is the group information flag with the following definition:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, link phase change values are stored.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>If set, the estimated entry information is non-default
+ and is stored.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2-7</code></td>
+ <td>Reserved</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Link Phase Change: Maximum Compact Value</p></td>
+ <td><p>The is the maximum number of links to store &ldquo;compactly&rdquo; (in
+ the group&rsquo;s object header).</p>
+ <p>This field is present if bit 0 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Link Phase Change: Minimum Dense Value</p></td>
+ <td><p>This is the minimum number of links to store &ldquo;densely&rdquo; (in
+ the group&rsquo;s fractal heap). The fractal heap&rsquo;s address is
+ located in the <a href="#LinkInfoMessage">Link Info</a>
+ message.</p>
+ <p>This field is present if bit 0 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Estimated Number of Entries</p></td>
+ <td><p>This is the estimated number of entries in groups.</p>
+ <p>If this field is not present, the default value of <code>4</code>
+ will be used for the estimated number of group entries.</p>
+ <p>This field is present if bit 1 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Estimated Link Name Length of Entries</p></td>
+ <td><p>This is the estimated length of entry name.</p>
+ <p>If this field is not present, the default value of <code>8</code>
+ will be used for the estimated link name length of group entries.</p>
+ <p>This field is present if bit 1 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+ </p>
+
+<br />
+<h4><a name="FilterMessage">IV.A.2.l. The Data Storage - Filter
+Pipeline Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b>
+ Data Storage - Filter Pipeline</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x000B</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td><p>This message describes the filter pipeline which should
+ be applied to the data stream by providing filter identification
+ numbers, flags, a name, and client data.</p>
+ <p>This message may be present in the object headers of both
+ dataset and group objects. For datasets, it specifies the
+ filters to apply to raw data. For groups, it specifies the
+ filters to apply to the group&rsquo;s fractal heap. Currently,
+ only datasets using chunked data storage use the filter
+ pipeline on their raw data.</p></td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Filter Pipeline Message - Version 1
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Number of Filters</td>
+ <td colspan="2">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Filter Description List <em>(variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This table
+ describes version 1.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Filters</p></td>
+ <td><p>The total number of filters described in this
+ message. The maximum possible number of filters in a
+ message is 32.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Filter Description List</p></td>
+ <td><p>A description of each filter. A filter description
+ appears in the next table.</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Filter Description
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="2">Filter Identification Value</td>
+ <td colspan="2">Name Length</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Flags</td>
+ <td colspan="2">Number Client Data Values</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name <em>(variable size, optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Client Data <em>(variable size, optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Padding <em>(variable size, optional)</em></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Filter Identification Value</p></td>
+ <td>
+ <p>
+ This value, often referred to as a filter identifier,
+ is designed to be a unique identifier for the filter.
+ Values from zero through 32,767 are reserved for filters
+ supported by The HDF Group in the HDF5 Library and for
+ filters requested and supported by third parties.
+ Filters supported by The HDF Group are documented immediately
+ below. Information on 3rd-party filters can be found at
+ The HDF Group&rsquo;s
+ <a href="http://www.hdfgroup.org/services/contributions.html">
+ Contributions</a> page.</p>
+
+ <p>
+ To request a filter identifier, please contact
+ The HDF Group&rsquo;s Help Desk at
+ <img src="Graphics/help.png" valign="middle" height="14"
+ alt="The HDF Group Help Desk">.
+ You will be asked to provide the following information:</p>
+ <ol>
+ <li>Contact information for the developer requesting the
+ new identifier</li>
+ <li>A short description of the new filter</li>
+ <li>Links to any relevant information, including licensing
+ information</li>
+ </ol>
+ <p>
+ Values from 32768 to 65535 are reserved for non-distributed uses
+ (for example, internal company usage) or for application usage
+ when testing a feature. The HDF Group does not track or document
+ the use of the filters with identifiers from this range.</p>
+
+ <p>
+ The filters currently in library version 1.8.0 are
+ listed below:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Identification</th>
+ <th width="15%" align="left">Name</th>
+ <th width="65%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>N/A</td>
+ <td>Reserved</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>deflate</td>
+ <td>GZIP deflate compression</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>shuffle</td>
+ <td>Data element shuffling</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>fletcher32</td>
+ <td>Fletcher32 checksum</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>szip</td>
+ <td>SZIP compression</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>5</code></td>
+ <td>nbit</td>
+ <td>N-bit packing</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6</code></td>
+ <td>scaleoffset</td>
+ <td>Scale and offset encoded values</td>
+ </tr>
+ </table>
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Name Length</p></td>
+ <td><p>Each filter has an optional null-terminated ASCII name
+ and this field holds the length of the name including the
+ null termination padded with nulls to be a multiple of
+ eight. If the filter has no name then a value of zero is
+ stored in this field.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>The flags indicate certain properties for a filter. The
+ bit values defined so far are:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set then the filter is an optional filter.
+ During output, if an optional filter fails it will be
+ silently skipped in the pipeline.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1-15</code></td>
+ <td>Reserved (zero)</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Client Data Values</p></td>
+ <td><p>Each filter can store integer values to control
+ how the filter operates. The number of entries in the
+ <em>Client Data</em> array is stored in this field.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td><p>If the <em>Name Length</em> field is non-zero then it will
+ contain the size of this field, padded to a multiple of eight. This
+ field contains a null-terminated, ASCII character
+ string to serve as a comment/name for the filter.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Client Data</p></td>
+ <td><p>This is an array of four-byte integers which will be
+ passed to the filter function. The <em>Client Data Number</em> of
+ Values determines the number of elements in the array.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Padding</p></td>
+ <td><p>Four bytes of zeroes are added to the message at this
+ point if the Client Data Number of Values field contains
+ an odd number.</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Filter Pipeline Message - Version 2
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Number of Filters</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Filter Description List <em>(variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This table
+ describes version 2.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Filters</p></td>
+ <td><p>The total number of filters described in this
+ message. The maximum possible number of filters in a
+ message is 32.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Filter Description List</p></td>
+ <td><p>A description of each filter. A filter description
+ appears in the next table.</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Filter Description
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="2">Filter Identification Value</td>
+ <td colspan="2">Name Length <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Flags</td>
+ <td colspan="2">Number Client Data Values</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name <em>(variable size, optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Client Data <em>(variable size, optional)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Filter Identification Value</p></td>
+ <td>
+ <p>
+ This value, often referred to as a filter identifier,
+ is designed to be a unique identifier for the filter.
+ Values from zero through 32,767 are reserved for filters
+ supported by The HDF Group in the HDF5 Library and for
+ filters requested and supported by third parties.
+ Filters supported by The HDF Group are documented immediately
+ below. Information on 3rd-party filters can be found at
+ The HDF Group&rsquo;s
+ <a href="http://www.hdfgroup.org/services/contributions.html">
+ Contributions</a> page.</p>
+
+ <p>
+ To request a filter identifier, please contact
+ The HDF Group&rsquo;s Help Desk at
+ <img src="Graphics/help.png" valign="middle" height="14"
+ alt="The HDF Group Help Desk">.
+ You will be asked to provide the following information:</p>
+ <ol>
+ <li>Contact information for the developer requesting the
+ new identifier</li>
+ <li>A short description of the new filter</li>
+ <li>Links to any relevant information, including licensing
+ information</li>
+ </ol>
+ <p>
+ Values from 32768 to 65535 are reserved for non-distributed uses
+ (for example, internal company usage) or for application usage
+ when testing a feature. The HDF Group does not track or document
+ the use of the filters with identifiers from this range.</p>
+
+ <p>
+ The filters currently in library version 1.8.0 are
+ listed below:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Identification</th>
+ <th width="15%" align="left">Name</th>
+ <th width="65%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>N/A</td>
+ <td>Reserved</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>deflate</td>
+ <td>GZIP deflate compression</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>shuffle</td>
+ <td>Data element shuffling</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>fletcher32</td>
+ <td>Fletcher32 checksum</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>szip</td>
+ <td>SZIP compression</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>5</code></td>
+ <td>nbit</td>
+ <td>N-bit packing</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6</code></td>
+ <td>scaleoffset</td>
+ <td>Scale and offset encoded values</td>
+ </tr>
+ </table>
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Name Length</p></td>
+ <td><p>Each filter has an optional null-terminated ASCII name
+ and this field holds the length of the name including the
+ null termination padded with nulls to be a multiple of
+ eight. If the filter has no name then a value of zero is
+ stored in this field.</p>
+ <p>Filters with IDs less than 256 (in other words, filters
+ that are defined in this format documentation) do not store
+ the <em>Name Length</em> or <em>Name</em> fields.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>The flags indicate certain properties for a filter. The
+ bit values defined so far are:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set then the filter is an optional filter.
+ During output, if an optional filter fails it will be
+ silently skipped in the pipeline.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1-15</code></td>
+ <td>Reserved (zero)</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Client Data Values</p></td>
+ <td><p>Each filter can store integer values to control
+ how the filter operates. The number of entries in the
+ <em>Client Data</em> array is stored in this field.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td><p>If the <em>Name Length</em> field is non-zero then it will
+ contain the size of this field, <em>not</em> padded to a multiple
+ of eight. This field contains a <em>non-</em>null-terminated,
+ ASCII character string to serve as a comment/name for the filter.
+ </p>
+ <p>Filters that are defined in this format documentation
+ such as deflate and shuffle do not store the <em>Name
+ Length</em> or <em>Name</em> fields.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Client Data</p></td>
+ <td><p>This is an array of four-byte integers which will be
+ passed to the filter function. The Client Data Number of
+ Values</em> determines the number of elements in the array.</p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h4><a name="AttributeMessage">IV.A.2.m. The Attribute Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Attribute</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x000C</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td><p>The <em>Attribute</em> message is used to store objects
+ in the HDF5 file which are used as attributes, or
+ &ldquo;metadata&rdquo; about the current object. An attribute
+ is a small dataset; it has a name, a datatype, a dataspace, and
+ raw data. Since attributes are stored in the object header, they
+ should be relatively small (in other words, less than 64KB).
+ They can be associated with any type of object which has an
+ object header (groups, datasets, or committed (named)
+ datatypes).</p>
+ <p>In 1.8.x versions of the library, attributes can be larger
+ than 64KB. See the
+ <a href="UG/HDF5_Users_Guide-Responsive%20HTML5/index.html#t=HDF5_Users_Guide%2FAttributes%2FHDF5_Attributes.htm%3Frhtocid%3Dtoc8.2_1%23TOC_8_5_Special_Issuesbc-13">
+ &ldquo;Special Issues&rdquo;</a> section of the Attributes chapter
+ in the <cite>HDF5 User&rsquo;s Guide</cite> for more information.</p>
+ <p>Note: Attributes on an object must have unique names:
+ the HDF5 Library currently enforces this by causing the
+ creation of an attribute with a duplicate name to fail.
+ Attributes on different objects may have the same name,
+ however.</p></td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Attribute Message (Version 1)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Reserved (zero)</td>
+ <td colspan="2">Name Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Datatype Size</td>
+ <td colspan="2">Dataspace Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Datatype <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Dataspace <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Data <em>(variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number information is used for changes in the format of the
+ attribute message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Never used.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Used by the library before version 1.6 to encode attribute message.
+ This version does not support shared datatypes.</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Name Size</p></td>
+ <td><p>The length of the attribute name in bytes including the
+ null terminator. Note that the <em>Name</em> field below may
+ contain additional padding not represented by this
+ field.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Datatype Size</p></td>
+ <td><p>The length of the datatype description in the <em>Datatype</em>
+ field below. Note that the <em>Datatype</em> field may contain
+ additional padding not represented by this field.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Dataspace Size</p></td>
+ <td><p>The length of the dataspace description in the <em>Dataspace</em>
+ field below. Note that the <em>Dataspace</em> field may contain
+ additional padding not represented by this field.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td><p>The null-terminated attribute name. This field is
+ padded with additional null characters to make it a
+ multiple of eight bytes.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Datatype</p></td>
+ <td><p>The datatype description follows the same format as
+ described for the datatype object header message. This
+ field is padded with additional zero bytes to make it a
+ multiple of eight bytes.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Dataspace</p></td>
+ <td><p>The dataspace description follows the same format as
+ described for the dataspace object header message. This
+ field is padded with additional zero bytes to make it a
+ multiple of eight bytes.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Data</p></td>
+ <td><p>The raw data for the attribute. The size is determined
+ from the datatype and dataspace descriptions. This
+ field is <em>not</em> padded with additional bytes.</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Attribute Message (Version 2)
+ </caption>
+
+ <tr align="center">
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td colspan="2">Name Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Datatype Size</td>
+ <td colspan="2">Dataspace Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Datatype <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Dataspace <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Data <em>(variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number information is used for changes in the
+ format of the attribute message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Used by the library of version 1.6.x and after to encode
+ attribute messages.
+ This version supports shared datatypes. The fields of
+ name, datatype, and dataspace are not padded with
+ additional bytes of zero.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>This bit field contains extra information about
+ interpreting the attribute message:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, datatype is shared.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>If set, dataspace is shared.</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Name Size</p></td>
+ <td><p>The length of the attribute name in bytes including the
+ null terminator.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Datatype Size</p></td>
+ <td><p>The length of the datatype description in the <em>Datatype</em>
+ field below.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Dataspace Size</p></td>
+ <td><p>The length of the dataspace description in the <em>Dataspace</em>
+ field below.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td><p>The null-terminated attribute name. This field is <em>not</em>
+ padded with additional bytes.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Datatype</p></td>
+ <td><p>The datatype description follows the same format as
+ described for the datatype object header message.
+ </p>
+ <p>If the
+ <em>Flag</em> field indicates this attribute&rsquo;s datatype is
+ shared, this field will contain a &ldquo;shared message&rdquo; encoding
+ instead of the datatype encoding.
+ </p>
+ <p>This field is <em>not</em> padded with additional bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dataspace</p></td>
+ <td><p>The dataspace description follows the same format as
+ described for the dataspace object header message.
+ </p>
+ <p>If the
+ <em>Flag</em> field indicates this attribute&rsquo;s dataspace is
+ shared, this field will contain a &ldquo;shared message&rdquo; encoding
+ instead of the dataspace encoding.
+ </p>
+ <p>This field is <em>not</em> padded with additional bytes.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data</p></td>
+ <td><p>The raw data for the attribute. The size is determined
+ from the datatype and dataspace descriptions.
+ </p>
+ <p>This field is <em>not</em> padded with additional zero bytes.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Attribute Message (Version 3)
+ </caption>
+
+ <tr align="center">
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td colspan="2">Name Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Datatype Size</td>
+ <td colspan="2">Dataspace Size</td>
+ </tr>
+
+ <tr>
+ <td>Name Character Set Encoding</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Datatype <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Dataspace <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Data <em>(variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number information is used for changes in the
+ format of the attribute message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Used by the library of version 1.8.x and after to
+ encode attribute messages.
+ This version supports attributes with non-ASCII names.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>This bit field contains extra information about
+ interpreting the attribute message:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, datatype is shared.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>If set, dataspace is shared.</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Name Size</p></td>
+ <td><p>The length of the attribute name in bytes including the
+ null terminator.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Datatype Size</p></td>
+ <td><p>The length of the datatype description in the <em>Datatype</em>
+ field below.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Dataspace Size</p></td>
+ <td><p>The length of the dataspace description in the <em>Dataspace</em>
+ field below.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Name Character Set Encoding</p></td>
+ <td><p>The character set encoding for the attribute&rsquo;s name:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>ASCII character set encoding
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>UTF-8 character set encoding
+ </td>
+ </tr>
+ </table>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td><p>The null-terminated attribute name. This field is <em>not</em>
+ padded with additional bytes.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Datatype</p></td>
+ <td><p>The datatype description follows the same format as
+ described for the datatype object header message.
+ </p>
+ <p>If the
+ <em>Flag</em> field indicates this attribute&rsquo;s datatype is
+ shared, this field will contain a &ldquo;shared message&rdquo; encoding
+ instead of the datatype encoding.
+ </p>
+ <p>This field is <em>not</em> padded with additional bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dataspace</p></td>
+ <td><p>The dataspace description follows the same format as
+ described for the dataspace object header message.
+ </p>
+ <p>If the
+ <em>Flag</em> field indicates this attribute&rsquo;s dataspace is
+ shared, this field will contain a &ldquo;shared message&rdquo; encoding
+ instead of the dataspace encoding.
+ </p>
+ <p>This field is <em>not</em> padded with additional bytes.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data</p></td>
+ <td><p>The raw data for the attribute. The size is determined
+ from the datatype and dataspace descriptions.
+ </p>
+ <p>This field is <em>not</em> padded with additional zero bytes.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h4><a name="CommentMessage">IV.A.2.n. The Object Comment
+Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Object
+ Comment</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x000D</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>The object comment is designed to be a short description of
+ an object. An object comment is a sequence of non-zero
+ (<code>\0</code>) ASCII characters with no other formatting
+ included by the library.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Name Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Comment <em>(variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td><p>A null terminated ASCII character string.</p></td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h4><a name="OldModificationTimeMessage">IV.A.2.o. The Object
+Modification Time (Old) Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Object
+ Modification Time (Old)</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x000E</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Fixed</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td><p>The object modification date and time is a timestamp
+ which indicates (using ISO-8601 date and time format) the last
+ modification of an object. The time is updated when any object
+ header message changes according to the system clock where the
+ change was posted. All fields of this message should be
+ interpreted as coordinated universal time (UTC).</p>
+ <p>This modification time message is deprecated in favor of
+ the &ldquo;new&rdquo; <a href="#ModificationTimeMessage">Object
+ Modification Time</a> message and is no longer written to the
+ file in versions of the HDF5 Library after the 1.6.0
+ version.</p></td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Modification Time Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Year</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Month</td>
+ <td colspan="2">Day of Month</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Hour</td>
+ <td colspan="2">Minute</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Second</td>
+ <td colspan="2">Reserved</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Year</p></td>
+ <td><p>The four-digit year as an ASCII string. For example,
+ <code>1998</code>.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Month</p></td>
+ <td><p>The month number as a two digit ASCII string where
+ January is <code>01</code> and December is <code>12</code>.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Day of Month</p></td>
+ <td><p>The day number within the month as a two digit ASCII
+ string. The first day of the month is <code>01</code>.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Hour</p></td>
+ <td><p>The hour of the day as a two digit ASCII string where
+ midnight is <code>00</code> and 11:00pm is <code>23</code>.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Minute</p></td>
+ <td><p>The minute of the hour as a two digit ASCII string where
+ the first minute of the hour is <code>00</code> and
+ the last is <code>59</code>.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Second</p></td>
+ <td><p>The second of the minute as a two digit ASCII string
+ where the first second of the minute is <code>00</code>
+ and the last is <code>59</code>.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Reserved</p></td>
+ <td><p>This field is reserved and should always be zero.</p></td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h4><a name="SOHMTableMessage">IV.A.2.p. The Shared Message Table
+Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Shared Message
+ Table</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x000F</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Fixed</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>This message is used to locate the table of shared object
+ header message (SOHM) indexes. Each index consists of information
+ to find the shared messages from either the heap or object header.
+ This message is <em>only</em> found in the superblock
+ extension.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Shared Message Table Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Shared Object Header Message Table Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td>Number of Indices</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This document describes version 0.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Shared Object Header Message Table Address</p></td>
+ <td><p>This field is the address of the master table for shared
+ object header message indexes.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Indices</p></td>
+ <td><p>This field is the number of indices in the master table.
+ </p></td>
+ </tr>
+
+ </table>
+ </div>
+
+<br />
+<h4><a name="ContinuationMessage">IV.A.2.q. The Object Header
+Continuation Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Object Header
+ Continuation</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0010</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Fixed</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>The object header continuation is the location in the file
+ of a block containing more header messages for the current data
+ object. This can be used when header blocks become too large or
+ are likely to change over time.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Object Header Continuation Message
+ </caption>
+
+ <tr>
+ <th width=25%>byte</th>
+ <th width=25%>byte</th>
+ <th width=25%>byte</th>
+ <th width=25%>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Offset<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Length<sup>L</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Offset</p></td>
+ <td><p>This value is the address in the file where the
+ header continuation block is located.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Length</p></td>
+ <td><p>This value is the length in bytes of the header continuation
+ block in the file.</p></td>
+ </tr>
+ </table>
+ </div>
+ <br />
+
+ <p>The format of the header continuation block that this message points
+ to depends on the version of the object header that the message is
+ contained within.
+ </p>
+
+ <p>
+ Continuation blocks for version 1 object headers have no special
+ formatting information; they are merely a list of object header
+ message info sequences (type, size, flags, reserved bytes and data
+ for each message sequence). See the description
+ of <a href="#V1ObjectHeaderPrefix">Version 1 Data Object Header Prefix.</a>
+ </p>
+
+ <p>Continuation blocks for version 2 object headers <em>do</em> have
+ special formatting information as described here
+ (see also the description of
+ <a href="#V2ObjectHeaderPrefix">Version 2 Data Object Header Prefix.</a>):
+ </p>
+ <div align="center">
+ <table class="format">
+ <caption>
+ Version 2 Object Header Continuation Block
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+ <tr>
+ <td>Header Message Type #1</td>
+ <td colspan="2">Size of Header Message Data #1</td>
+ <td>Header Message #1 Flags</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Header Message #1 Creation Order <em>(optional)</em></td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Header Message Data #1<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+
+ <tr>
+ <td>Header Message Type #n</td>
+ <td colspan="2">Size of Header Message Data #n</td>
+ <td>Header Message #n Flags</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Header Message #n Creation Order <em>(optional)</em></td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Header Message Data #n<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Gap <em>(optional, variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>OCHK</code>&rdquo;
+ is used to indicate the
+ beginning of an object header continuation block. This gives file
+ consistency checking utilities a better chance of reconstructing a
+ damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Type</p></td>
+ <td>
+ <p>Same format as version 1 of the object header, described above.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Header Message #n Data</p></td>
+ <td>
+ <p>Same format as version 1 of the object header, described above.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Flags</p></td>
+ <td>
+ <p>Same format as version 1 of the object header, described above.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Creation Order</p></td>
+ <td>
+ <p>This field stores the order that a message of a given type
+ was created in.</p>
+ <p>This field is present if bit 2 of <em>flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Data</p></td>
+ <td>
+ <p>Same format as version 1 of the object header, described above.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Gap</p></td>
+ <td>
+ <p>A gap in an object header chunk is inferred by the end of the
+ messages for the chunk before the beginning of the chunk&rsquo;s
+ checksum. Gaps are always smaller than the size of an
+ object header message prefix (message type + message size +
+ message flags).</p>
+ <p>Gaps are formed when a message (typically an attribute message)
+ in an earlier chunk is deleted and a message from a later
+ chunk that does not quite fit into the free space is moved
+ into the earlier chunk.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the object header chunk.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h4><a name="SymbolTableMessage">IV.A.2.r. The Symbol Table
+Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Symbol Table
+ Message</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0011</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Fixed</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Required for
+ &ldquo;old style&rdquo; groups; may not be repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>Each &ldquo;old style&rdquo; group has a v1 B-tree and a
+ local heap for storing symbol table entries, which are located
+ with this message.</td></tr>
+ <tr><td colspan="2"><b>Format of data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ <b>Symbol Table Message</b>
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />v1 B-tree Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Local Heap Address<sup>O</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>v1 B-tree Address</p></td>
+ <td><p>This value is the address of the v1 B-tree containing the
+ symbol table entries for the group.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Local Heap Address</p></td>
+ <td><p>This value is the address of the local heap containing
+ the link names for the symbol table entries for the group.</p></td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h4><a name="ModificationTimeMessage">IV.A.2.s. The Object
+Modification Time Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Object
+ Modification Time</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0012</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Fixed</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>The object modification time is a timestamp which indicates
+ the time of the last modification of an object. The time is
+ updated when any object header message changes according to
+ the system clock where the change was posted.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Modification Time Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Seconds After UNIX Epoch</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number is used for changes in the format of Object Modification Time
+ and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Never used.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Used by Version 1.6.1 and after of the library to encode time. In
+ this version, the time is the seconds after Epoch.</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Seconds After UNIX Epoch</p></td>
+ <td><p>A 32-bit unsigned integer value that stores the number of
+ seconds since 0 hours, 0 minutes, 0 seconds, January 1, 1970,
+ Coordinated Universal Time.</p></td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h4><a name="BtreeKValuesMessage">IV.A.2.t. The B-tree
+&lsquo;K&rsquo; Values Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> B-tree
+ &lsquo;K&rsquo; Values</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0013</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Fixed</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>This message retrieves non-default &lsquo;K&rsquo; values
+ for internal and leaf nodes of a group or indexed storage v1
+ B-trees. This message is <em>only</em> found in the superblock
+ extension.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ B-tree &lsquo;K&rsquo; Values Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="2">Indexed Storage Internal Node K</td>
+ <td bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Group Internal Node K</td>
+ <td colspan="2">Group Leaf Node K</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This document describes
+ version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Indexed Storage Internal Node K</p></td>
+ <td><p>This is the node &lsquo;K&rsquo; value for each internal node of an
+ indexed storage v1 B-tree. See the description of this field
+ in version 0 and 1 of the superblock as well the section on
+ v1 B-trees.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Group Internal Node K</p></td>
+ <td><p>This is the node &lsquo;K&rsquo; value for each internal node of a group
+ v1 B-tree. See the description of this field in version 0 and
+ 1 of the superblock as well as the section on v1 B-trees.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Group Leaf Node K</p></td>
+ <td><p>This is the node &lsquo;K&rsquo; value for each leaf node of a group v1
+ B-tree. See the description of this field in version 0 and 1
+ of the superblock as well as the section on v1 B-trees.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+<br />
+<h4><a name="DrvInfoMessage">IV.A.2.u. The Driver Info
+Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Driver
+ Info</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0014</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+
+ <tr><td>
+ <b>Description:</b></td>
+ <td>This message contains information needed by the file driver
+ to reopen a file. This message is <em>only</em> found in the
+ superblock extension: see the <a href="#SuperblockExt">
+ &ldquo;Disk Format: Level 0C - Superblock Extension&rdquo;</a>
+ section for more information. For more information on the fields
+ in the driver info message, see the <a href="#DriverInfo">
+ &ldquo;Disk Format : Level 0B - File Driver Info&rdquo;</a>
+ section; those who use the multi and family file drivers will
+ find this section particularly helpful.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Driver Info Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Driver Identification</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Driver Information Size</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br /><br />Driver Information <em>(variable size)</em><br /><br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This document describes
+ version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Driver Identification</p></td>
+ <td><p>This is an eight-byte ASCII string without null termination which
+ identifies the driver.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Driver Information Size</p></td>
+ <td><p>The size in bytes of the <em>Driver Information</em> field of this
+ message.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Driver Information</p></td>
+ <td><p>Driver information is stored in a format defined by the file driver.</p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+<br />
+<h4><a name="AinfoMessage">IV.A.2.v. The Attribute Info
+Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Attribute
+ Info</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0015</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>This message stores information about the attributes on an
+ object, such as the maximum creation index for the attributes
+ created and the location of the attribute storage when the
+ attributes are stored &ldquo;densely&rdquo;.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Attribute Info Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td colspan="2">Maximum Creation Index <em>(optional)</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Fractal Heap Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Attribute Name v2 B-tree Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Attribute Creation Order v2 B-tree Address<sup>O</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This document describes
+ version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>This is the attribute index information flag with the
+ following definition:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, creation order for attributes is tracked.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>If set, creation order for attributes is indexed.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2-7</code></td>
+ <td>Reserved</td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Maximum Creation Index</p></td>
+ <td><p>The is the maximum creation order index value for the
+ attributes on the object.</p>
+ <p>This field is present if bit 0 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fractal Heap Address</p></td>
+ <td><p>This is the address of the fractal heap to store dense
+ attributes.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Attribute Name v2 B-tree Address</p></td>
+ <td><p>This is the address of the version 2 B-tree to index the
+ names of densely stored attributes.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Attribute Creation Order v2 B-tree Address</p></td>
+ <td><p>This is the address of the version 2 B-tree to index the
+ creation order of densely stored attributes.</p>
+ <p>This field is present if bit 1 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+<br />
+<h4><a name="RefCountMessage">IV.A.2.w. The Object Reference
+Count Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Object Reference
+ Count</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0016</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Fixed</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>This message stores the number of hard links (in groups or
+ objects) pointing to an object: in other words, its
+ <em>reference count</em>.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Object Reference Count
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reference count</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This document describes
+ version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Reference Count</p></td>
+ <td><p>The unsigned 32-bit integer is the reference count for the
+ object. This message is only present in &ldquo;version 2&rdquo;
+ (or later) object headers, and if not present those object
+ header versions, the reference count for the object is assumed
+ to be 1.</p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+<br />
+<h4><a name="FsinfoMessage">IV.A.2.x. The File Space Info
+Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> File Space
+ Info</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0018</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Fixed</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td>
+ <b>Description:</b></td>
+ <td>This message stores the file space management strategy (see
+ description below) that the library uses in handling file space
+ request for the file. It also contains the free-space section
+ threshold used by the library&rsquo;s free-space managers for
+ the file. If the strategy is 1, this message also contains the
+ addresses of the file&rsquo;s free-space managers which track
+ free space for each type of file space allocation. There are
+ six basic types of file space allocation: superblock, B-tree,
+ raw data, global heap, local heap, and object header. See the
+ description of <a href="#FreeSpaceManager">Free-space
+ Manager</a> as well the description of allocation types in
+ <a href="#AppendixB">Appendix B</a>.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ File Space Info
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Strategy</td>
+ <td colspan="2">Threshold<sup>L</sup></td>
+ </tr>
+ <tr>
+ <td colspan="4">Super-block Free-space Manager Address<sup>O</sup></td>
+ </tr>
+ <tr>
+ <td colspan="4">B-tree Free-space Manager Address<sup>O</sup></td>
+ </tr>
+ <tr>
+ <td colspan="4">Raw Data Free-space Manager Address<sup>O</sup></td>
+ </tr>
+ <tr>
+ <td colspan="4">Global Heap Free-space Manager Address<sup>O</sup></td>
+ </tr>
+ <tr>
+ <td colspan="4">Local Heap Free-space Manager Address<sup>O</sup></td>
+ </tr>
+ <tr>
+ <td colspan="4">Object Header Free-space Manager Address<sup>O</sup></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Offsets&rdquo; field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are of the size
+ specified in &ldquo;Size of Lengths&rdquo; field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>This is the version number of this message. This document describes
+ version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Strategy</p></td>
+ <td><p>This is the file space management strategy for the file.
+ There are four types of strategies:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>With this strategy, the HDF5 Library&rsquo;s free-space managers track the
+ free space that results from the manipulation of HDF5 objects
+ in the HDF5 file. The free space information is saved when the
+ file is closed, and reloaded when the file is reopened.
+ <br />
+ When space is needed for file metadata or raw data,
+ the HDF5 Library first requests space from the library&rsquo;s free-space
+ managers. If the request is not satisfied, the library requests space
+ from the aggregators. If the request is still not satisfied,
+ the library requests space from the virtual file driver.
+ That is, the library will use all of the mechanisms for allocating
+ space.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>This is the HDF5 Library&rsquo;s default file space management strategy.
+ With this strategy, the library&rsquo;s free-space managers track the free space
+ that results from the manipulation of HDF5 objects in the HDF5 file.
+ The free space information is NOT saved when the file is closed and
+ the free space that exists upon file closing becomes unaccounted
+ space in the file.
+ <br />
+ As with strategy #1, the library will try all of the mechanisms
+ for allocating space. When space is needed for file metadata or
+ raw data, the library first requests space from the free-space
+ managers. If the request is not satisfied, the library requests
+ space from the aggregators. If the request is still not satisfied,
+ the library requests space from the virtual file driver.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>With this strategy, the HDF5 Library does not track free space that results
+ from the manipulation of HDF5 objects in the HDF5 file and
+ the free space becomes unaccounted space in the file.
+ <br />
+ When space is needed for file metadata or raw data,
+ the library first requests space from the aggregators.
+ If the request is not satisfied, the library requests space from
+ the virtual file driver.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>With this strategy, the HDF5 Library does not track free space that results
+ from the manipulation of HDF5 objects in the HDF5 file and
+ the free space becomes unaccounted space in the file.
+ <br />
+ When space is needed for file metadata or raw data,
+ the library requests space from the virtual file driver.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Threshold</p></td>
+ <td><p>This is the free-space section threshold.
+ The library&rsquo;s free-space managers will track only
+ free-space sections with size greater than or equal to
+ <em>threshold</em>. The default is to track free-space
+ sections of all sizes.</p>
+ </td>
+ </tr>
+ <tr>
+ <td><p>Superblock Free-space Manager Address</p></td>
+ <td><p>This is the address of the free-space manager for
+ H5FD_MEM_SUPER allocation type.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>B-tree Free-space Manager Address</p></td>
+ <td><p>This is the address of the free-space manager for
+ H5FD_MEM_BTREE allocation type.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Raw Data Free-space Manager Address</p></td>
+ <td><p>This is the address of the free-space manager for
+ H5FD_MEM_DRAW allocation type.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Global Heap Free-space Manager Address</p></td>
+ <td><p>This is the address of the free-space manager for
+ H5FD_MEM_GHEAP allocation type.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Local Heap Free-space Manager Address</p></td>
+ <td><p>This is the address of the free-space manager for
+ H5FD_MEM_LHEAP allocation type.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Header Free-space Manager Address</p></td>
+ <td><p>This is the address of the free-space manager for
+ H5FD_MEM_OHDR allocation type.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+ <br />
+
+
+<br />
+<h3><a name="DataStorage">
+IV.B. Disk Format: Level 2B - Data Object Data Storage</a></h3>
+
+<p>The data for an object is stored separately from its header
+ information in the file and may not actually be located in the HDF5 file
+ itself if the header indicates that the data is stored externally. The
+ information for each record in the object is stored according to the
+ dimensionality of the object (indicated in the dataspace header message).
+ Multi-dimensional array data is stored in C order; in other words, the
+ &ldquo;last&rdquo; dimension changes fastest.</p>
+
+<p>Data whose elements are composed of atomic datatypes are stored in IEEE
+ format, unless they are specifically defined as being stored in a different
+ machine format with the architecture-type information from the datatype
+ header message. This means that each architecture will need to [potentially]
+ byte-swap data values into the internal representation for that particular
+ machine.</p>
+
+<p> Data with a variable-length datatype is stored in the global heap
+ of the HDF5 file. Global heap identifiers are stored in the
+ data object storage.</p>
+
+<p>Data whose elements are composed of reference datatypes are stored in
+ several different ways depending on the particular reference type involved.
+ Object pointers are just stored as the offset of the object header being
+ pointed to with the size of the pointer being the same number of bytes as
+ offsets in the file.</p>
+
+<p>Dataset region references are stored as a heap-ID which points to
+the following information within the file-heap: an offset of the object
+pointed to, number-type information (same format as header message),
+dimensionality information (same format as header message), sub-set start
+and end information (in other words, a coordinate location for each),
+and field start and end names (in other words, a [pointer to the] string
+indicating the first field included and a [pointer to the] string name
+for the last field). </p>
+
+<p>Data of a compound datatype is stored as a contiguous stream of the items
+ in the structure, with each item formatted according to its datatype.</p>
+
+
+
+<br />
+<br />
+<hr />
+<h2><a name="AppendixA">
+V. Appendix A: Definitions</a></h2>
+
+<p>Definitions of various terms used in this document are included in
+this section.</p>
+
+ <div align="center">
+ <table class="glossary">
+ <tr>
+ <th width="20%">Term</th>
+ <th>Definition</th>
+ </tr>
+
+ <tr>
+ <td>Undefined Address</td>
+ <td>The <a name="UndefinedAddress">undefined
+ address</a> for a file is a file address with all bits
+ set: in other words, <code>0xffff...ff</code>.</td>
+ </tr>
+
+ <tr>
+ <td>Unlimited Size</td>
+ <td>The <a name="UnlimitedDim">unlimited size</a>
+ for a size is a value with all bits set: in other words,
+ <code>0xffff...ff</code>.</td>
+ </tr>
+
+ </table>
+ </div>
+
+
+
+<br />
+<br />
+<hr />
+<h2><a name="AppendixB">
+VI. Appendix B: File Memory Allocation Types</a></h2>
+
+<p>There are six basic types of file memory allocation as follows:
+</p>
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Basic Allocation Type</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_SUPER</td>
+ <td>File memory allocated for <em>Superblock.</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_BTREE</td>
+ <td>File memory allocated for <em>B-tree.</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_DRAW</td>
+ <td>File memory allocated for raw data.</td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_GHEAP</td>
+ <td>File memory allocated for <em>Global Heap.</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_LHEAP</td>
+ <td>File memory allocated for <em>Local Heap.</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_OHDR</td>
+ <td>File memory allocated for <em>Object Header.</em></td>
+ </tr>
+ </table>
+ </div>
+
+<p>There are other file memory allocation types that are mapped to the
+above six basic allocation types because they are similar in nature.
+The mapping is listed in the following table:
+</p>
+
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Basic Allocation Type</th>
+ <th>Mapping of Allocation Types to Basic Allocation Types</th>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_SUPER</td>
+ <td><em>none</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_BTREE</td>
+ <td>H5FD_MEM_SOHM_INDEX</td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_DRAW</td>
+ <td>H5FD_MEM_FHEAP_HUGE_OBJ</td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_GHEAP</td>
+ <td><em>none</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_LHEAP</td>
+ <td>H5FD_MEM_FHEAP_DBLOCK, H5FD_MEM_FSPACE_SINFO</td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_OHDR</td>
+ <td>H5FD_MEM_FHEAP_HDR, H5FD_MEM_FHEAP_IBLOCK, H5FD_MEM_FSPACE_HDR, H5FD_MEM_SOHM_TABLE</td>
+ </tr>
+ </table>
+ </div>
+
+<p>Allocation types that are mapped to basic allocation types are described below:
+</p>
+
+ <div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Allocation Type</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_FHEAP_HDR</td>
+ <td>File memory allocated for <em>Fractal Heap Header.</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_FHEAP_DBLOCK</td>
+ <td>File memory allocated for <em>Fractal Heap Direct Blocks.</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_FHEAP_IBLOCK</td>
+ <td>File memory allocated for <em>Fractal Heap Indirect Blocks.</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_FHEAP_HUGE_OBJ</td>
+ <td>File memory allocated for huge objects in the fractal heap.</td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_FSPACE_HDR</td>
+ <td>File memory allocated for <em>Free-space Manager Header.</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_FSPACE_SINFO</td>
+ <td>File memory allocated for <em>Free-space Section List</em> of the free-space manager.</td>
+ </tr>
+ <tr>
+ <td>H5FD_MEM_SOHM_TABLE</td>
+ <td>File memory allocated for <em>Shared Object Header Message Table.</em></td>
+ </tr>
+ <tr>
+ <td>H5FD_MEM_SOHM_INDEX</td>
+ <td>File memory allocated for <em>Shared Message Record List.</em></td>
+ </tr>
+ </table>
+ </div>
+</body>
+</html>
diff --git a/doxygen/examples/H5.format.html b/doxygen/examples/H5.format.html
new file mode 100644
index 0000000..e16805f
--- /dev/null
+++ b/doxygen/examples/H5.format.html
@@ -0,0 +1,20400 @@
+<html>
+ <head>
+ <title>
+ HDF5 File Format Specification Version 3.0
+ </title>
+
+ <style>
+ h1 { display: block;
+ margin-top: 24px;
+ margin-bottom: 24px;
+ margin-left: 0px;
+ margin-right: 0px;
+ text-indent: 0px;
+ font-size: 300%;
+ }
+
+ h2 { display: block;
+ margin-top: 60px;
+ margin-bottom: 8px;
+ margin-left: 0px;
+ margin-right: 0px;
+ text-indent: 0px;
+ border-style: solid;
+ border-top-style: medium;
+ border-top-color: #A9A9A9;
+ border-bottom: none;
+ border-left: none;
+ border-right: none;
+ font-size: 250%;
+ }
+
+ h3 { display: block;
+ margin-top: 40px;
+ margin-bottom: 8px;
+ margin-left: 0px;
+ margin-right: 0px;
+ text-indent: 0px;
+ font-size: 200%;
+ }
+
+ h4 { display: block;
+ margin-top: 32px;
+ margin-bottom: 8px;
+ margin-left: 0px;
+ margin-right: 0px;
+ text-indent: 0px;
+ font-size: 150%;
+ }
+
+ p { display: block;
+ margin-top: 8px;
+ margin-bottom: 8px;
+ margin-left: 0px;
+ margin-right: 0px;
+ text-indent: 0px;
+ font-size: 100%;
+ }
+ <!--
+ p.item { margin-left: 2em;
+ text-indent: -2em
+ } -->
+ <!-- p.item2 { margin-left: 2em; text-indent: 2em} -->
+
+ table.format { border:solid;
+ border-collapse:collapse;
+ caption-side:top;
+ text-align:center;
+ width:80%;
+ }
+ table.format th { border:ridge;
+ padding:4px;
+ width:25%;
+ }
+ table.format td { border:ridge;
+ padding:4px;
+ }
+ table.format caption { font-weight:bold;
+ font-size:larger;
+ }
+
+ table.note {border:none;
+ text-align:right;
+ width:80%;
+ }
+
+ table.desc { border:solid;
+ border-collapse:collapse;
+ caption-size:top;
+ text-align:left;
+ width:80%;
+ }
+ table.desc tr { vertical-align:top;
+ }
+ table.desc th { border-style:ridge;
+ font-size:larger;
+ padding:4px;
+ <!-- text-decoration:underline; -->
+ }
+ table.desc td { border-style:ridge;
+ <!-- padding: 4px; -->
+ vertical-align:text-top;
+ }
+ table.desc caption { font-weight:bold;
+ font-size:larger;
+ }
+
+ table.list { border:none;
+ width:100%
+ }
+ table.list tr { vertical-align:text-top;
+ }
+ table.list th { border:none;
+ text-decoration:underline;
+ vertical-align:text-top;
+ }
+ table.list td { border:none;
+ vertical-align:text-top;
+ }
+
+ table.msgdesc { border:none;
+ text-align:left;
+ width: 80%
+ }
+ table.msgdesc tr { vertical-align:text-top;
+ border-spacing:0;
+ padding:0; }
+ table.msgdesc th { border:none;
+ text-decoration:underline;
+ vertical-align:text-top; }
+ table.msgdesc td { border:none;
+ vertical-align:text-top;
+ }
+
+ table.list80 { border:none;
+ width:80%
+ }
+ table.list80 tr { vertical-align:text-top;
+ }
+ table.list80 th { border:none;
+ text-decoration:underline;
+ vertical-align:text-top;
+ }
+ table.list80 td { border:none;
+ vertical-align:text-top;
+ }
+
+ table.glossary { border:none;
+ text-align:left;
+ width: 80%
+ }
+ table.glossary tr { vertical-align:text-top;
+ border-spacing:0;
+ padding:0; }
+ table.glossary th { border:none;
+ text-align:left;
+ text-decoration:underline;
+ vertical-align:text-top; }
+ table.glossary td { border:none;
+ text-align:left;
+ vertical-align:text-top;
+ }
+
+ div { page-break-inside:avoid;
+ page-break-after:auto
+ }
+
+ </style>
+
+ <!-- #BeginLibraryItem "/ed_libs/styles_Format.lbi" -->
+ <!--
+ * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
+ * Copyright by The HDF Group. *
+ * Copyright by the Board of Trustees of the University of Illinois. *
+ * All rights reserved. *
+ * *
+ * This file is part of HDF5. The full HDF5 copyright notice, including *
+ * terms governing use, modification, and redistribution, is contained in *
+ * the files COPYING and Copyright.html. COPYING can be found at the root *
+ * of the source code distribution tree; Copyright.html can be found at the *
+ * root level of an installed copy of the electronic HDF5 document set and *
+ * is linked from the top-level documents page. It can also be found at *
+ * http://www.hdfgroup.org/HDF5/doc/Copyright.html. If you do not have *
+ * access to either file, you may request a copy from help@hdfgroup.org. *
+ * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
+ -->
+ <!-- #EndLibraryItem --><!-- #BeginLibraryItem "/ed_libs/NavBar_ADevG.lbi" -->
+ </head>
+ <body>
+ <!-- #EndLibraryItem -->
+
+ <center>
+ <table border="0" width="90%">
+ <tr>
+ <td valign="top">
+ <ol type="I">
+ <li><a href="#Intro">Introduction</a></li>
+ <font size="-1">
+ <ol type="A">
+ <li><a href="#ThisDocument">This Document</a></li>
+ <li><a href="#ChangesForHdf5_1_12">Changes for HDF5 1.12</a></li>
+ <li><a href="#ChangesForHdf5_1_10">Changes for HDF5 1.10</a></li>
+ </ol>
+ </font>
+
+ <li><a href="#FileMetaData">Disk Format: Level 0 - File Metadata</a></li>
+ <font size="-1">
+ <ol type="A">
+ <li><a href="#Superblock">Disk Format: Level 0A - Format Signature
+ and Superblock</a></li>
+ <li><a href="#DriverInfo">Disk Format: Level 0B - File Driver
+ Info</a></li>
+ <li><a href="#SuperblockExt">Disk Format: Level 0C - Superblock
+ Extension</a></li>
+ </ol>
+ </font>
+ <li><a href="#FileInfra">Disk Format: Level 1 - File Infrastructure</a></li>
+ <font size="-1">
+ <ol type="A">
+ <li><a href="#Btrees">Disk Format: Level 1A - B-trees and B-tree
+ Nodes</a>
+ <ol type="1">
+ <li><a href="#V1Btrees">Disk Format: Level 1A1 - Version 1
+ B-trees</a></li>
+ <li><a href="#V2Btrees">Disk Format: Level 1A2 - Version 2
+ B-trees</a></li>
+ </ol>
+ </li>
+ <li><a href="#SymbolTable">Disk Format: Level 1B - Group Symbol
+ Table Nodes</a></li>
+ <li><a href="#SymbolTableEntry">Disk Format: Level 1C - Symbol
+ Table Entry</a></li>
+ <li><a href="#LocalHeap">Disk Format: Level 1D - Local Heaps</a></li>
+ <li><a href="#GlobalHeap">Disk Format: Level 1E - Global Heap</a></li>
+ <li><a href="#GlobalHeapVDS">Disk Format: Level 1F - Global Heap
+ Block for Virtual Datasets</a></li>
+ <li><a href="#FractalHeap">Disk Format: Level 1G - Fractal Heap</a></li>
+ <li><a href="#FreeSpaceManager">Disk Format: Level 1H - Free-space
+ Manager</a></li>
+ <li><a href="#SOHMTable">Disk Format: Level 1I - Shared Object
+ Header Message Table</a></li>
+ </ol>
+ </font>
+ <li><a href="#DataObject">Disk Format: Level 2 - Data Objects</a></li>
+ <font size="-1">
+ <ol type="A">
+ <li><a href="#ObjectHeader">Disk Format: Level 2A - Data Object Headers</a></li>
+ <ol type="1">
+ <li><a href="#ObjectHeaderPrefix">Disk Format: Level 2A1 -
+ Data Object Header Prefix</a>
+ <ol type="a">
+ <li><a href="#V1ObjectHeaderPrefix">Version 1 Data
+ Object Header Prefix</a></li>
+ <li><a href="#V2ObjectHeaderPrefix">Version 2 Data
+ Object Header Prefix</a></li>
+ </ol>
+ </li>
+ <li><a href="#ObjectHeaderMessages">Disk Format: Level 2A2 -
+ Data Object Header Messages</a></li>
+ <ol type="a">
+ <li><a href="#NILMessage">The NIL Message</a></li> <!-- 0x0000 -->
+ <li><a href="#DataspaceMessage">The Dataspace Message</a></li> <!-- 0x0001 -->
+ <li><a href="#LinkInfoMessage">The Link Info Message</a></li> <!-- 0x0002 -->
+ <li><a href="#DatatypeMessage">The Datatype Message</a></li> <!-- 0x0003 -->
+ <li><a href="#OldFillValueMessage">The Data Storage -
+ Fill Value (Old) Message</a></li> <!-- 0x0004 -->
+ </ol>
+ </ol>
+ </ol>
+ </font>
+ </ol>
+ </td>
+
+ <td>&nbsp;</td>
+
+ <td valign="top">
+ <ol type="I" start="4">
+ <li><a href="#DataObject">Disk Format: Level 2 - Data
+ Objects</a><font size="-1"><i> (Continued)</i></li>
+ <ol type="A">
+ <li><a href="#ObjectHeader">Disk Format: Level 2A - Data Object
+ Headers</a><i> (Continued)</i>
+ <ol type="1" start="2">
+ <li><a href="#ObjectHeaderMessages">Disk Format: Level 2A2 -
+ Data Object Header Messages</a><i> (Continued)</i></li>
+ <ol type="a" start="6">
+ <li><a href="#FillValueMessage">The Data Storage -
+ Fill Value Message</a></li> <!-- 0x0005 -->
+ <li><a href="#LinkMessage">The Link Message</a></li> <!-- 0x0006 -->
+ <li><a href="#ExternalFileListMessage">The Data Storage -
+ External Data Files Message</a></li> <!-- 0x0007 -->
+ <li><a href="#LayoutMessage">The Data Layout Message</a></li> <!-- 0x0008 -->
+ <li><a href="#BogusMessage">The Bogus Message</a></li> <!-- 0x0009 -->
+ <li><a href="#GroupInfoMessage">The Group Info
+ Message</a></li> <!-- 0x000a -->
+ <li><a href="#FilterMessage">The Data Storage -
+ Filter Pipeline Message</a></li> <!-- 0x000b -->
+ <li><a href="#AttributeMessage">The Attribute
+ Message</a></li> <!-- 0x000c -->
+ <li><a href="#CommentMessage">The Object Comment
+ Message</a></li> <!-- 0x000d -->
+ <li><a href="#OldModificationTimeMessage">The Object
+ Modification Time (Old) Message</a></li> <!-- 0x000e -->
+ <li><a href="#SOHMTableMessage">The Shared Message
+ Table Message</a></li> <!-- 0x000f -->
+ <li><a href="#ContinuationMessage">The Object Header
+ Continuation Message</a></li> <!-- 0x0010 -->
+ <li><a href="#SymbolTableMessage">The Symbol
+ Table Message</a></li> <!-- 0x0011 -->
+ <li><a href="#ModificationTimeMessage">The Object
+ Modification Time Message</a></li> <!-- 0x0012 -->
+ <li><a href="#BtreeKValuesMessage">The B-tree
+ &lsquo;K&rsquo; Values Message</a></li> <!-- 0x0013 -->
+ <li><a href="#DrvInfoMessage">The Driver Info
+ Message</a></li> <!-- 0x0014 -->
+ <li><a href="#AinfoMessage">The Attribute Info
+ Message</a></li> <!-- 0x0015 -->
+ <li><a href="#RefCountMessage">The Object Reference
+ Count Message</a></li> <!-- 0x0016 -->
+ <li><a href="#FsinfoMessage">The File Space Info
+ Message</a></li> <!-- 0x0017 -->
+ </ol>
+ </ol>
+ </li>
+ <li><a href="#DataStorage">Disk Format: Level 2B - Data Object Data Storage</a></li>
+ </ol>
+ </font>
+ <li><a href="#AppendixA">Appendix A: Definitions</a></li>
+ <li><a href="#AppendixB">Appendix B: File Space Allocation
+ Types</a></li>
+ <li><a href="#AppendixC">
+ Appendix C: Types of Indexes for Dataset Chunks</a></li>
+ <font size="-1">
+ <ol type="A">
+ <li><a href="#SingleChunk">The Single Chunk Index</a></li>
+ <li><a href="#Implicit">The Implicit Index</a></li>
+ <li><a href="#FixedArray">The Fixed Array Index</a></li>
+ <li><a href="#ExtensibleArray">The Extensible Array Index</a></li>
+ <li><a href="#AppendV2Btrees">The Version 2 B-trees Index</a></li>
+ </ol>
+ </font>
+ <li><a href="#AppendixD">
+ Appendix D: Encoding for Dataspace and Reference</a></li>
+ <font size="-1">
+ <ol type="A">
+ <li><a href="#DataspaceEncode">Dataspace Encoding</a></li>
+ <li><a href="#ReferenceEncodeRV">Reference Encoding (Revised)</a></li>
+ <li><a href="#ReferenceEncodeDP">Reference Encoding (Backward Compatibility)</a></li>
+ </ol>
+ </font>
+ </ol>
+ </td></tr>
+ </table>
+ </center>
+
+
+ <a name="Intro"><h2>I. Introduction</h2></a>
+
+ <table align="right" width="100">
+ <tr><td>&nbsp;</td><td align="center">
+ <hr />
+ <img src="FF-IH_FileGroup.gif" alt="HDF5 Groups" hspace="15" vspace="15">
+ </td><td>&nbsp;</td></tr>
+ <tr><td>&nbsp;</td><td align="center">
+ <strong>Figure 1:</strong> Relationships among the HDF5 root group, other groups, and objects
+ <hr />
+ </td><td>&nbsp;</td></tr>
+
+ <tr><td>&nbsp;</td><td align="center">
+ <img src="FF-IH_FileObject.gif" alt="HDF5 Objects" hspace="15" vspace="15">
+ </td><td>&nbsp;</td></tr>
+ <tr><td>&nbsp;</td><td align="center">
+ <strong>Figure 2:</strong> HDF5 objects -- datasets, datatypes, or dataspaces
+ <hr />
+ </td><td>&nbsp;</td></tr>
+ </table>
+
+
+ <p>The format of an HDF5 file on disk encompasses several
+ key ideas of the HDF4 and AIO file formats as well as
+ addressing some shortcomings therein. The new format is
+ more self-describing than the HDF4 format and is more
+ uniformly applied to data objects in the file.</p>
+
+ <p>An HDF5 file appears to the user as a directed graph.
+ The nodes of this graph are the higher-level HDF5 objects
+ that are exposed by the HDF5 APIs:</p>
+
+ <ul>
+ <li>Groups</li>
+ <li>Datasets</li>
+ <li>Committed (formerly Named) datatypes</li>
+ </ul>
+
+ <p>At the lowest level, as information is actually written to the disk,
+ an HDF5 file is made up of the following objects:</p>
+ <ul>
+ <li>A superblock</li>
+ <li>B-tree nodes</li>
+ <li>Heap blocks</li>
+ <li>Object headers</li>
+ <li>Object data</li>
+ <li>Free space</li>
+ </ul>
+
+ <p>The HDF5 Library uses these low-level objects to represent the
+ higher-level objects that are then presented to the user or
+ to applications through the APIs. For instance, a group is an
+ object header that contains a message that points to a local
+ heap (for storing the links to objects in the group) and to a
+ B-tree (which indexes the links). A dataset is an object header
+ that contains messages that describe the datatype, dataspace,
+ layout, filters, external files, fill value, and other elements
+ with the layout message pointing to either a raw data chunk or
+ to a B-tree that points to raw data chunks.</p>
+
+
+ <a name="ThisDocument"><h3>I.A. This Document</h3></a>
+
+ <p>This document describes the lower-level data objects;
+ the higher-level objects and their properties are described
+ in the <a href="UG/HDF5_Users_Guide-Responsive HTML5/index.html"><cite>HDF5 User&rsquo;s Guide</cite></a>.</p>
+
+ <p>Three levels of information comprise the file format.
+ Level 0 contains basic information for identifying and
+ defining information about the file. Level 1 information contains
+ the information about the pieces of a file shared by many objects
+ in the file (such as B-trees and heaps). Level 2 is the rest
+ of the file and contains all of the data objects with each object
+ partitioned into header information, also known as
+ <em>metadata</em>, and data.</p>
+
+ <p>The various components of the lower-level data objects are
+ described in pairs of tables. The first table shows the format
+ layout, and the second table describes the fields. The titles
+ of format layout tables begin with &ldquo;Layout&rdquo;. The
+ titles of the tables where the fields are described begin with
+ &ldquo;Fields&rdquo;. For example, the table that describes the
+ format of the <a href="#V2Btrees">version 2 B-tree header</a> has
+ a title of &ldquo;Layout: Version 2 B-tree Header&rdquo;, and the
+ fields in the version 2 B-tree header are described in the table
+ titled &ldquo;Fields: Version 2 B-tree Header&rdquo;.
+
+ <p>The sizes of various fields in the following layout tables are
+ determined by looking at the number of columns the field spans
+ in the table. There are exceptions: </p>
+ <ul>
+ <li> The size may be overridden by specifying a size in
+ parentheses</li>
+ <li> The size of addresses is determined by the
+ <em><a href="#SizeOfOffsetsV0">Size of Offsets</a></em> field
+ in the superblock and is indicated in this document with a
+ superscripted &lsquo;O&rsquo;</li>
+ <li> The size of length fields is determined by the
+ <em><a href="#SizeOfLengthsV0">Size of Lengths</a></em> field in
+ the superblock and is indicated in this document with a
+ superscripted &lsquo;L&rsquo;</li>
+ </ul>
+
+ <p>Values for all fields in this document should be treated as unsigned
+ integers, unless otherwise noted in the description of a field.
+ Additionally, all metadata fields are stored in little-endian byte
+ order.
+ </p>
+
+ <p>All checksums used in the format are computed with the
+ <a href="http://www.burtleburtle.net/bob/hash/doobs.html">Jenkins&rsquo;
+ lookup3</a> algorithm.
+ </p>
+
+ <p>Whenever a bit flag or field is mentioned for an entry, bits are
+ numbered from the lowest bit position in the entry.
+ </p>
+
+ <p>Various format tables in this document have cells with
+ &ldquo;This space inserted only to align table nicely&rdquo;. These
+ entries in the table are just to make the table presentation nicer
+ and do not represent any values or padding in the file.
+ </p>
+
+ <a name="ChangesForHdf5_1_12">
+ <h3>I.B. Changes for HDF5 1.12</h3></a>
+ <p>The following sections have been
+ changed or added for the 1.12 release:</p>
+ <ul>
+ <li>Under <a href="#DatatypeMessage">&ldquo;The Datatype Message&rdquo;</a>,
+ in the Description for &ldquo;Fields:Datatype Message&rdquo;,
+ version 4 was added and Reference class (7) of the datatype was updated to describe version 4.</li>
+ <li><a href="#AppendixD">
+ &ldquo;Appendix D: Encoding for Dataspace and Reference&rdquo;</a>
+ was added. </li>
+ </ul>
+
+
+ <a name="ChangesForHdf5_1_10">
+ <h3>I.C. Changes for HDF5 1.10</h3></a>
+
+ <p>The following sections have been
+ changed or added for the 1.10 release:</p>
+ <ul>
+ <li>In the <a href="#Superblock">
+ &ldquo;Disk Format: Level 0A - Format Signature and
+ Superblock&rdquo;</a> section, version 3 of the superblock was
+ added. </li>
+ <li>In the <a href="#SuperblockExt">
+ &ldquo;Disk Format: Level 0C - Superblock Extension&rdquo;</a>
+ section, a link to the Data Storage message was added. </li>
+ <li>In the <a href="#V2Btrees">
+ &ldquo;Disk Format: Level 1A2 - Version 2 B-trees&rdquo;</a>
+ section, additional B-tree types were added. Tables that
+ describe the <a href="#V2BtreesType10">type 10</a> and
+ <a href="#V2BtreesType11">11</a> record layouts were added at
+ the end of the section.</li>
+ <li>The <a href="#GlobalHeapVDS">&ldquo;Disk Format: Level 1F -
+ Global Heap Block for Virtual Datasets&rdquo;</a> was added.
+ </li>
+ <li><a href="#LayoutMessage">
+ &ldquo;The Data Layout Message&rdquo;</a> section was changed.
+ The name was changed, and <a href="#DataLayoutV4">version 4</a>
+ of the data layout message was added for the virtual type.</li>
+ <li>The <a href="#FsinfoMessage">
+ &ldquo;The File Space Info Message&rdquo;</a> header message
+ type was added.</li>
+ <li><a href="#AppendixC">
+ &ldquo;Appendix C: Types of Indexes for Dataset Chunks&rdquo;</a>
+ was added. Five indexing types were added.</li>
+ </ul>
+
+
+
+ <h2><a name="FileMetaData">
+ II. Disk Format: Level 0 - File Metadata</a></h2>
+
+
+
+ <h3><a name="Superblock">
+ II.A. Disk Format: Level 0A - Format Signature and Superblock</a></h3>
+
+ <p>The superblock may begin at certain predefined offsets within
+ the HDF5 file, allowing a block of unspecified content for
+ users to place additional information at the beginning (and
+ end) of the HDF5 file without limiting the HDF5 Library&rsquo;s
+ ability to manage the objects within the file itself. This
+ feature was designed to accommodate wrapping an HDF5 file in
+ another file format or adding descriptive information to an HDF5
+ file without requiring the modification of the actual file&rsquo;s
+ information. The superblock is located by searching for the
+ HDF5 format signature at byte offset 0, byte offset 512, and at
+ successive locations in the file, each a multiple of two of
+ the previous location; in other words, at these byte offsets:
+ 0, 512, 1024, 2048, and so on.</p>
+
+ <p>The superblock is composed of the format signature, followed by a
+ superblock version number and information that is specific to each
+ version of the superblock.
+
+ <p>Currently, there are four versions of the superblock format:
+ <ul>
+ <li>Version 0 is the default format.</li>
+ <li>Version 1 is the same as version 0 but with the
+ &ldquo;<em>Indexed Storage Internal Node K</em>&rdquo; field
+ for storing non-default B-tree &lsquo;K&rsquo; value.</li>
+ <li>Version 2 has some fields eliminated and compressed from
+ superblock format versions 0 and 1. It has added checksum support
+ and superblock extension to store additional superblock
+ metadata.</li>
+ <li>Version 3 is the same as version 2 except that the field
+ &ldquo;<em>File Consistency Flags</em>&rdquo; is used for file
+ locking. This format version will enable support for the latest
+ version.</li>
+ </ul>
+
+ <p>Versions 0 and 1 of the superblock are described below:</p>
+
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Superblock (Versions 0 and 1)
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Format Signature
+ <em>(8 bytes)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td>Version # of Superblock</td>
+ <td>Version # of File&rsquo;s Free Space Storage</td>
+ <td>Version # of Root Group Symbol Table Entry</td>
+ <td>Reserved <em>(zero)</em></td>
+ </tr>
+
+ <tr>
+ <td>Version Number of Shared Header Message Format</td>
+ <td>Size of Offsets</td>
+ <td>Size of Lengths</td>
+ <td>Reserved <em>(zero)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Group Leaf Node K</td>
+ <td colspan="2">Group Internal Node K</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">File Consistency Flags</td>
+ </tr>
+
+ <tr>
+ <td colspan="2" style="border:dotted;">Indexed Storage Internal Node K<sup>1</sup></td>
+ <td colspan="2" style="border:dotted;">Reserved
+ <em>(zero)</em><sup>1</sup></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Base Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of File Free space Info<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />End of File Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Driver Information Block Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Root Group Symbol Table Entry</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with a &lsquo;1&rsquo; in the above table are
+ new in version 1 of the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Superblock (Versions 0 and 1)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Format Signature</p></td>
+ <td><p>This field contains a constant value and can be used to
+ quickly identify a file as being an HDF5 file. The
+ constant value is designed to allow easy identification of
+ an HDF5 file and to allow certain types of data corruption
+ to be detected. The file signature of an HDF5 file always
+ contains the following values:</p>
+ <center>
+ <table border align="center" cellpadding="4">
+ <tr align="center">
+ <td align="right">Decimal:</td>
+ <td width="8%">137</td>
+ <td width="8%">72</td>
+ <td width="8%">68</td>
+ <td width="8%">70</td>
+ <td width="8%">13</td>
+ <td width="8%">10</td>
+ <td width="8%">26</td>
+ <td width="8%">10</td>
+ </tr>
+
+ <tr align="center">
+ <td align="right">Hexadecimal:</td>
+ <td>89</td>
+ <td>48</td>
+ <td>44</td>
+ <td>46</td>
+ <td>0d</td>
+ <td>0a</td>
+ <td>1a</td>
+ <td>0a</td>
+ </tr>
+
+ <tr align="center">
+ <td align="right">ASCII C Notation:</td>
+ <td>\211</td>
+ <td>H</td>
+ <td>D</td>
+ <td>F</td>
+ <td>\r</td>
+ <td>\n</td>
+ <td>\032</td>
+ <td>\n</td>
+ </tr>
+ </table>
+ </center>
+ <p>This signature both identifies the file as an HDF5 file
+ and provides for immediate detection of common
+ file-transfer problems. The first two bytes distinguish
+ HDF5 files on systems that expect the first two bytes to
+ identify the file type uniquely. The first byte is
+ chosen as a non-ASCII value to reduce the probability
+ that a text file may be misrecognized as an HDF5 file;
+ also, it catches bad file transfers that clear bit
+ 7. Bytes two through four name the format. The CR-LF
+ sequence catches bad file transfers that alter newline
+ sequences. The control-Z character stops file display
+ under MS-DOS. The final line feed checks for the inverse
+ of the CR-LF translation problem. (This is a direct
+ descendent of the
+ <a href="http://www.libpng.org/pub/png/spec/iso/index-object.html#5PNG-file-signature">PNG</a> file
+ signature.)</p>
+ <p><em>This field is present in version 0+ of the superblock.</em>
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Version Number of the Superblock</p></td>
+ <td><p>This value is used to determine the format of the
+ information in the superblock. When the format of the
+ information in the superblock is changed, the version number
+ is incremented to the next integer and can be used to
+ determine how the information in the superblock is
+ formatted.</p>
+
+ <p>Values of 0, 1 and 2 are defined for this field (the
+ format of version 2 is described below, not here).
+ </p>
+
+ <p><em>This field is present in version 0+ of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version Number of the File&rsquo;s Free Space
+ Information</p></td>
+ <td>
+ <p>This value is used to determine the format of the
+ file&rsquo;s free space information.
+ </p>
+ <p>The only value currently valid in this field is &lsquo;0&rsquo;, which
+ indicates that the file&rsquo;s free space is as described
+ <a href="#FreeSpaceManager">below</a>.
+ </p>
+
+ <p><em>This field is present in versions 0 and 1 of the
+ superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version Number of the Root Group Symbol Table
+ Entry</p></td>
+ <td><p>This value is used to determine the format of the
+ information in the Root Group Symbol Table Entry. When the
+ format of the information in that field is changed, the
+ version number is incremented to the next integer and can be
+ used to determine how the information in the field
+ is formatted.</p>
+ <p>The only value currently valid in this field is &lsquo;0&rsquo;,
+ which indicates that the root group symbol table entry is
+ formatted as described <a href="#SymbolTableEntry">below</a>.</p>
+ <p><em>This field is present in version 0 and 1 of the
+ superblock.</em></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version Number of the Shared Header Message Format</p></td>
+ <td><p>This value is used to determine the format of the
+ information in a shared object header message. Since the format
+ of the shared header messages differs from the other private
+ header messages, a version number is used to identify changes
+ in the format.
+ </p>
+ <p>The only value currently valid in this field is &lsquo;0&rsquo;, which
+ indicates that shared header messages are formatted as
+ described <a href="#ObjectHeaderMessages">below</a>.
+ </p>
+
+ <p><em>This field is present in version 0 and 1 of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p><a name="SizeOfOffsetsV0">Size of Offsets</a></p></td>
+ <td><p>This value contains the number of bytes used to store
+ addresses in the file. The values for the addresses of
+ objects in the file are offsets relative to a base address,
+ usually the address of the superblock signature. This
+ allows a wrapper to be added after the file is created
+ without invalidating the internal offset locations.
+ </p>
+
+ <p><em>This field is present in version 0+ of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p><a name="SizeOfLengthsV0">Size of Lengths</a></p></td>
+ <td><p>This value contains the number of bytes used to store
+ the size of an object.
+ </p>
+ <p><em>This field is present in version 0+ of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Group Leaf Node K</p></td>
+ <td>
+ <p>Each leaf node of a group B-tree will have at
+ least this many entries but not more than twice this
+ many. If a group has a single leaf node then it
+ may have fewer entries.
+ </p>
+ <p>This value must be greater than zero.
+ </p>
+ <p>See the <a href="#Btrees">description</a> of B-trees below.
+ </p>
+
+ <p><em>This field is present in version 0 and 1 of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Group Internal Node K</p></td>
+ <td>
+ <p>Each internal node of a group B-tree will have at
+ least this many entries but not more than twice this
+ many. If the group has only one internal
+ node then it might have fewer entries.
+ </p>
+ <p>This value must be greater than zero.
+ </p>
+ <p>See the <a href="#Btrees">description</a> of B-trees below.
+ </p>
+
+ <p><em>This field is present in version 0 and 1 of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>File Consistency Flags</p></td>
+ <td>
+ <p>This field is unused and should be ignored.
+ </p>
+ <p><em>This field is present in version 0+ of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Indexed Storage Internal Node K</p></td>
+ <td>
+ <p>Each internal node of an indexed storage B-tree will have at
+ least this many entries but not more than twice this
+ many. If the index storage B-tree has only one internal
+ node then it might have fewer entries.
+ </p>
+ <p>This value must be greater than zero.
+ </p>
+ <p>See the <a href="#Btrees">description</a> of B-trees below.
+ </p>
+
+ <p><em>This field is present in version 1 of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Base Address</p></td>
+ <td>
+ <p>This is the absolute file address of the first byte of
+ the HDF5 data within the file. The library currently
+ constrains this value to be the absolute file address
+ of the superblock itself when creating new files;
+ future versions of the library may provide greater
+ flexibility. When opening an existing file and this address does
+ not match the offset of the superblock, the library assumes
+ that the entire contents of the HDF5 file have been adjusted in
+ the file and adjusts the base address and end of file address to
+ reflect their new positions in the file. Unless otherwise noted,
+ all other file addresses are relative to this base
+ address.
+ </p>
+
+ <p><em>This field is present in version 0+ of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address of Global Free-space Index</p></td>
+ <td>
+ <p>The file&rsquo;s free space is not persistent for version 0 and 1 of
+ the superblock.
+ Currently this field always contains the
+ <a href="#UndefinedAddress">undefined address</a>.
+ </p>
+
+ <p><em>This field is present in version 0 and 1 of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>End of File Address</p></td>
+ <td>
+ <p>This is the absolute file address of the first byte past
+ the end of all HDF5 data. It is used to determine whether a
+ file has been accidently truncated and as an address where
+ file data allocation can occur if space from the free list is
+ not used.
+ </p>
+
+ <p><em>This field is present in version 0+ of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Driver Information Block Address</p></td>
+ <td>
+ <p>This is the relative file address of the file driver
+ information block which contains driver-specific
+ information needed to reopen the file. If there is no
+ driver information block then this entry should be the
+ <a href="#UndefinedAddress">undefined address</a>.
+ </p>
+
+ <p><em>This field is present in version 0 and 1 of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Root Group Symbol Table Entry</p></td>
+ <td>
+ <p>This is the <a href="#SymbolTableEntry">symbol table entry</a>
+ of the root group, which serves as the entry point into
+ the group graph for the file.
+ </p>
+
+ <p><em>This field is present in version 0 and 1 of the superblock.</em>
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <br />
+ <p>Versions 2 and 3 of the superblock are described below:</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Superblock (Versions 2 and 3)
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Format Signature
+ <em>(8 bytes)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td>Version # of Superblock</td>
+ <td>Size of Offsets</td>
+ <td>Size of Lengths</td>
+ <td>File Consistency Flags</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Base Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Superblock Extension Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />End of File Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Root Group Object Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Superblock Checksum</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Superblock (Versions 2 and 3)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Format Signature</p></td>
+ <td>
+ <p>This field is the same as described for versions 0 and 1 of the
+ superblock.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Version Number of the Superblock</p></td>
+ <td>
+ <p>This field has a value of 2 and has the same meaning as for
+ versions 0 and 1.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Offsets</p></td>
+ <td>
+ <p>This field is the same as described for
+ <a href="#SizeOfOffsetsV0">versions 0 and 1</a> of the
+ superblock.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Lengths</p></td>
+ <td>
+ <p>This field is the same as described for
+ <a href="#SizeOfLengthsV0">versions 0 and 1</a> of the
+ superblock.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>File Consistency Flags</p></td>
+
+ <td>
+ <p>For superblock version
+ 2: This field is unused and should be ignored.</p>
+ <p>For superblock version
+ 3: This value contains flags to ensure file consistency for
+ file locking. Currently, the following bit flags are defined:
+ <ul>
+ <li>Bit 0 if set indicates that the file has been opened for
+ write access.</li>
+ <li>Bit 1 is reserved for future use.</li>
+ <li>Bit 2 if set indicates that the file has been opened for
+ single-writer/multiple-reader (SWMR) write access.</li>
+ <li>Bits 3-7 are reserved for future use.</li>
+ </ul>
+ <p>
+ Bit 0 should be set as the first action when a file has been
+ opened for write access. Bit 2 should be set when a file
+ has been opened for SWMR write access. These two bits should
+ be cleared only as the final action when closing a file.
+ </p>
+ <p><em>This field is present in version 0+ of the superblock.</em>
+ </p>
+ <p><em>The size of this
+ field has been reduced from 4 bytes in superblock format
+ versions 0 and 1 to 1 byte.</em>
+ </p>
+ </td>
+
+ </tr>
+
+ <tr>
+ <td><p>Base Address</p></td>
+ <td>
+ <p>This field is the same as described for versions 0 and
+ 1 of the superblock.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Superblock Extension Address</p></td>
+ <td>
+ <p>The field is the address of the object header for the
+ <a href="#SuperblockExt">superblock extension</a>.
+ If there is no extension then this entry should be the
+ <a href="#UndefinedAddress">undefined address</a>.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>End of File Address</p></td>
+ <td>
+ <p>This field is the same as described for versions 0 and 1 of the
+ superblock.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Root Group Object Header Address</p></td>
+ <td>
+ <p>This is the address of
+ the <a href="#DataObject">root group object header</a>,
+ which serves as the entry point into the group graph for the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Superblock Checksum</p></td>
+ <td>
+ <p>The checksum for the superblock.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+
+ <h3><a name="DriverInfo">
+ II.B. Disk Format: Level 0B - File Driver Info</a></h3>
+
+ <p>The <b>driver information block</b> is an optional region of the
+ file which contains information needed by the file driver
+ to reopen a file. The format is described below:</p>
+
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Driver Information Block
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3">Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Driver Information Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Driver Identification
+ <em>(8 bytes)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br /><br />Driver Information
+ <em>(variable size)</em><br /><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Driver Information Block
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number of the Driver Information Block.
+ This document describes version 0.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Driver Information Size</p></td>
+ <td>
+ <p>The size in bytes of the <em>Driver Information</em> field.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Driver Identification</p></td>
+ <td>
+ <p>This is an eight-byte ASCII string without null
+ termination which identifies the driver and/or version number
+ of the Driver Information Block. The predefined driver encoded
+ in this field by the HDF5 Library is identified by the
+ letters <code>NCSA</code> followed by the first four characters of
+ the driver name. If the Driver Information block is not
+ the original version then the last letter(s) of the
+ identification will be replaced by a version number in
+ ASCII, starting with 0.
+ </p>
+ <p>
+ Identification for user-defined drivers is also eight-byte long.
+ It can be arbitrary but should be unique to avoid
+ the four character prefix &ldquo;NCSA&rdquo;.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Driver Information</p></td>
+ <td>Driver information is stored in a format defined by the
+ file driver (see description below).</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <p>The two drivers encoded in the <em>Driver Identification</em>
+ field are as follows:</p>
+ <ul>
+ <li>
+ Multi driver:
+ <p>
+ The identifier for this driver is &ldquo;NCSAmulti&rdquo;.
+ This driver provides a mechanism for segregating raw data and different types of metadata
+ into multiple files.
+ These files are viewed by the library as a single virtual HDF5 file with a single file address.
+ A maximum of 6 files will be created for the following data:
+ superblock, B-tree, raw data, global heap, local heap, and object header.
+ More than one type of data can be written to the same file.
+ </p></li>
+ <li>
+ Family driver
+ <p>
+ The identifier for this driver is &ldquo;NCSAfami&rdquo; and is encoded in this field for library version 1.8 and after.
+ This driver is designed for systems that do not support files larger than 2 gigabytes
+ by splitting the HDF5 file address space across several smaller files.
+ It does nothing to segregate metadata and raw data;
+ they are mixed in the address space just as they would be in a single contiguous file.
+ </p></li>
+ </ul>
+ <p>The format of the <em>Driver Information</em> field for the
+ above two drivers are described below:</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Multi Driver Information
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Member Mapping</td>
+ <td>Member Mapping</td>
+ <td>Member Mapping</td>
+ <td>Member Mapping</td>
+ </tr>
+
+ <tr>
+ <td>Member Mapping</td>
+ <td>Member Mapping</td>
+ <td>Reserved</td>
+ <td>Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Member File 1<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />End of Address for Member File 1<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Member File 2<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />End of Address for Member File 2<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />... ...<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Member File N<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />End of Address for Member File N<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name of Member File 1
+ <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name of Member File 2
+ <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />... ...<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name of Member File N
+ <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Multi Driver Information
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Member Mapping</p></td>
+ <td><p>These fields are integer values from 1 to 6
+ indicating how the data can be mapped to or merged with another type of
+ data.
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Member Mapping</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+ <tr>
+ <td align="center">1</td>
+ <td>The superblock data.</td>
+ </tr>
+ <tr>
+ <td align="center">2</td>
+ <td>The B-tree data.</td>
+ </tr>
+ <tr>
+ <td align="center">3</td>
+ <td>The raw data.</td>
+ </tr>
+ <tr>
+ <td align="center">4</td>
+ <td>The global heap data.</td>
+ </tr>
+ <tr>
+ <td align="center">5</td>
+ <td>The local heap data.</td>
+ </tr>
+ <tr>
+ <td align="center">6</td>
+ <td>The object header data.</td>
+ </tr>
+ </table></p>
+ <p>For example, if the third field has the value 3 and all the rest have the
+ value 1, it means there are two files: one for raw data, and one for superblock,
+ B-tree, global heap, local heap, and object header.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Reserved</p></td>
+ <td><p>These fields are reserved and should always be zero.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Address of Member File N</p></td>
+ <td><p>This field Specifies the virtual address at which the member file starts.</p>
+ <p>N is the number of member files.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>End of Address for Member File N</p></td>
+ <td><p>This field is the end of the allocated address for the member file.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Name of Member File N</p></td>
+ <td><p>This field is the null-terminated name of the member file and
+ its length should be multiples of 8 bytes.
+ Additional bytes will be padded with <em>NULL</em>s. The default naming
+ convention is <em>%s-X.h5</em>, where <em>X</em> is one of the letters
+ <em>s</em> (for superblock), <em>b</em> (for B-tree), <em>r</em> (for raw data),
+ <em>g</em> (for global heap), <em>l</em> (for local heap), and <em>o</em> (for
+ object header). The name of the whole HDF5 file will substitute the <em>%s</em>
+ in the string.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Family Driver Information
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="8"><br />Size of Member File<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Family Driver Information
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Size of Member File</p></td>
+ <td><p>This field is the size of the member file in the family of files.</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <h3><a name="SuperblockExt">
+ II.C. Disk Format: Level 0C - Superblock Extension</a></h3>
+
+ <p>The <em>superblock extension</em> is used to store superblock metadata
+ which is either optional, or added after the version of the superblock
+ was defined. Superblock extensions may only exist when version 2
+ or later of the superblock is used. A superblock extension is an object
+ header which may hold the following messages:</p>
+ <ul>
+ <li>
+ <a href="#SOHMTableMessage">Shared Message Table message</a> containing
+ information to locate the master table of shared object header message
+ indices.</li>
+ <li>
+ <a href="#BtreeKValuesMessage">B-tree &lsquo;K&rsquo; Values message</a> containing
+ non-default B-tree &lsquo;K&rsquo; values.</li>
+ <li>
+ <a href="#DrvInfoMessage">Driver Info message</a> containing information
+ needed by the file driver in order to reopen a file.
+ See also the
+ <a href="#DriverInfo">&ldquo;Disk Format: Level 0B - File Driver
+ Info&rdquo;</a> section above.</li>
+ <li>
+ <a href="#FsinfoMessage">File Space Info message</a> containing
+ information about file space handling in the file.</li>
+ </ul>
+
+
+
+ <h2><a name="FileInfra">
+ III. Disk Format: Level 1 - File Infrastructure</a></h2>
+
+ <h3><a name="Btrees">
+ III.A. Disk Format: Level 1A - B-trees and B-tree Nodes</a></h3>
+
+ <p>B-trees allow flexible storage for objects which tend to grow
+ in ways that cause the object to be stored discontiguously. B-trees
+ are described in various algorithms books including &ldquo;Introduction to
+ Algorithms&rdquo; by Thomas H. Cormen, Charles E. Leiserson, and Ronald
+ L. Rivest. B-trees are used in several places in the HDF5 file format,
+ when an index is needed for another data structure.</p>
+
+ <p>The version 1 B-tree structure described below is the original
+ index structure. The version 1 B-trees are being phased out in
+ favor of the version 2 B-trees described below. Note that both
+ types of structures may be found in the same file depending on
+ the application settings when creating the file.</p>
+
+ <h4><a name="V1Btrees">
+ III.A.1. Disk Format: Level 1A1 - Version 1 B-trees</a></h4>
+
+ <p>Version 1 B-trees in HDF5 files are an implementation of the
+ B-link tree. The sibling nodes at a particular level in
+ the tree are stored in a doubly-linked list. See the
+ &ldquo;Efficient Locking for Concurrent Operations on B-trees&rdquo;
+ paper by Phillip Lehman and S. Bing Yao as published in the
+ <cite>ACM Transactions on Database Systems</cite>, Vol. 6, No. 4,
+ December 1981.</p>
+
+ <p>The B-trees implemented by the file format contain one more
+ key than the number of children. In other words, each child
+ pointer out of a B-tree node has a left key and a right key.
+ The pointers out of internal nodes point to sub-trees while
+ the pointers out of leaf nodes point to symbol nodes and
+ raw data chunks.
+ Aside from that difference, internal nodes and leaf nodes
+ are identical.</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: B-tree Nodes
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Node Type</td>
+ <td>Node Level</td>
+ <td colspan="2">Entries Used</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Left Sibling<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Right Sibling<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Key 1 <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Child 1<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Key 2 <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Child 2<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Key 2<em>K</em> <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Child 2<em>K</em><sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Key 2<em>K</em>+1
+ <em>(variable size)</em></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: B-tree Nodes
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>TREE</code>&rdquo;
+ is used to indicate the beginning of a B-tree node. This
+ gives file consistency checking utilities a better chance
+ of reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Node Type</p></td>
+ <td>
+ <p>Each B-tree points to a particular type of data.
+ This field indicates the type of data as well as
+ implying the maximum degree <em>K</em> of the tree and
+ the size of each Key field.
+
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Node Type</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+ <tr>
+ <td align="center">0</td>
+ <td>This tree points to group nodes.</td>
+ </tr>
+ <tr>
+ <td align="center">1</td>
+ <td>This tree points to raw data chunk nodes.</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Node Level</p></td>
+ <td>
+ <p>The node level indicates the level at which this node
+ appears in the tree (leaf nodes are at level zero). Not
+ only does the level indicate whether child pointers
+ point to sub-trees or to data, but it can also be used
+ to help file consistency checking utilities reconstruct
+ damaged trees.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Entries Used</p></td>
+ <td>
+ <p>This determines the number of children to which this
+ node points. All nodes of a particular type of tree
+ have the same maximum degree, but most nodes will point
+ to less than that number of children. The valid child
+ pointers and keys appear at the beginning of the node
+ and the unused pointers and keys appear at the end of
+ the node. The unused pointers and keys have undefined
+ values.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Address of Left Sibling</p></td>
+ <td>
+ <p>This is the relative file address of the left sibling of
+ the current node. If the current
+ node is the left-most node at this level then this field
+ is the <a href="#UndefinedAddress">undefined address</a>.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Address of Right Sibling</p></td>
+ <td>
+ <p>This is the relative file address of the right sibling of
+ the current node. If the current
+ node is the right-most node at this level then this
+ field is the <a href="#UndefinedAddress">undefined address</a>.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Keys and Child Pointers</p></td>
+ <td>
+ <p>Each tree has 2<em>K</em>+1 keys with 2<em>K</em>
+ child pointers interleaved between the keys. The number
+ of keys and child pointers actually containing valid
+ values is determined by the node&rsquo;s <em>Entries
+ Used</em> field. If that field is <em>N</em>, then the
+ B-tree contains <em>N</em> child pointers and
+ <em>N</em>+1 keys.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Key</p></td>
+ <td>
+ <p>The format and size of the key values is determined by
+ the type of data to which this tree points. The keys are
+ ordered and are boundaries for the contents of the child
+ pointer; that is, the key values represented by child
+ <em>N</em> fall between Key <em>N</em> and Key
+ <em>N</em>+1. Whether the interval is open or closed on
+ each end is determined by the type of data to which the
+ tree points.
+ </p>
+
+ <p>
+ The format of the key depends on the node type.
+ For nodes of node type 0 (group nodes), the key is formatted as
+ follows:
+
+ <table class="list">
+ <tr>
+ <td width="20%">A single field of
+ <i><a href="#SizeOfLengthsV0">Size of Lengths</a></i>
+ bytes:</td>
+ <td width="80%">Indicates the byte offset into the local heap
+ for the first object name in the subtree which
+ that key describes.
+ </td>
+ </tr>
+ </table>
+ </p>
+
+
+ <p>
+ For nodes of node type 1 (chunked raw data nodes), the key is
+ formatted as follows:
+
+ <table class="list">
+ <tr>
+ <td width="20%">Bytes 1-4:</td>
+ <td width="80%">Size of chunk in bytes.</td>
+ </tr>
+ <tr>
+ <td>Bytes 4-8:</td>
+ <td>Filter mask, a 32-bit bit field indicating which
+ filters have been skipped for this chunk. Each filter
+ has an index number in the pipeline (starting at 0, with
+ the first filter to apply) and if that filter is skipped,
+ the bit corresponding to its index is set.</td>
+ </tr>
+ <tr>
+ <td>(<em>D + 1</em>) 64-bit fields:</td>
+ <td>The offset of the
+ chunk within the dataset where <i>D</i> is the number
+ of dimensions of the dataset, and the last value is the
+ offset within the dataset&rsquo;s datatype and should
+ always be zero. For example, if
+ a chunk in a 3-dimensional dataset begins at the
+ position <code>[5,5,5]</code>, there will be three
+ such 64-bit values, each with the value of
+ <code>5</code>, followed by a <code>0</code> value.</td>
+ </tr>
+ </table>
+ </p>
+
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Child Pointer</p></td>
+ <td>
+ <p>The tree node contains file addresses of subtrees or
+ data depending on the node level. Nodes at Level 0 point
+ to data addresses, either raw data chunks or group nodes.
+ Nodes at non-zero levels point to other nodes of the
+ same B-tree.
+ </p>
+ <p>For raw data chunk nodes, the child pointer is the address
+ of a single raw data chunk. For group nodes, the child pointer
+ points to a <a href="#SymbolTable">symbol table</a>, which contains
+ information for multiple symbol table entries.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <p>
+ Conceptually, each B-tree node looks like this:</p>
+ <center>
+ <table>
+ <tr valign="top" align="center">
+ <td>key[0]</td><td>&nbsp;</td>
+ <td>child[0]</td><td>&nbsp;</td>
+ <td>key[1]</td><td>&nbsp;</td>
+ <td>child[1]</td><td>&nbsp;</td>
+ <td>key[2]</td><td>&nbsp;</td>
+ <td>...</td><td>&nbsp;</td>
+ <td>...</td><td>&nbsp;</td>
+ <td>key[<i>N</i>-1]</td><td>&nbsp;</td>
+ <td>child[<i>N</i>-1]</td><td>&nbsp;</td>
+ <td>key[<i>N</i>]</td>
+ </tr>
+ </table>
+ </center>
+ <br />
+
+ where child[<i>i</i>] is a pointer to a sub-tree (at a level
+ above Level 0) or to data (at Level 0).
+ Each key[<i>i</i>] describes an <i>item</i> stored by the B-tree
+ (a chunk or an object of a group node). The range of values
+ represented by child[<i>i</i>] is indicated by key[<i>i</i>]
+ and key[<i>i</i>+1].
+
+
+ <p>The following question must next be answered:
+ &ldquo;Is the value described by key[<i>i</i>] contained in
+ child[<i>i</i>-1] or in child[<i>i</i>]?&rdquo;
+ The answer depends on the type of tree.
+ In trees for groups (node type 0), the object described by
+ key[<i>i</i>] is the greatest object contained in
+ child[<i>i</i>-1] while in chunk trees (node type 1) the
+ chunk described by key[<i>i</i>] is the least chunk in
+ child[<i>i</i>].</p>
+
+ <p>That means that key[0] for group trees is sometimes unused;
+ it points to offset zero in the heap, which is always the
+ empty string and compares as &ldquo;less-than&rdquo; any valid
+ object name.</p>
+
+ <p>And key[<i>N</i>] for chunk trees is sometimes unused;
+ it contains a chunk offset which compares as &ldquo;greater-than&rdquo;
+ any other chunk offset and has a chunk byte size of zero
+ to indicate that it is not actually allocated.</p>
+
+ <h4><a name="V2Btrees">
+ III.A.2. Disk Format: Level 1A2 - Version 2 B-trees</a></h4>
+
+ <p>Version 2 (v2) B-trees are &ldquo;traditional&rdquo; B-trees
+ with one major difference. Instead of just using a simple pointer
+ (or address in the file) to a child of an internal node, the pointer
+ to the child node contains two additional pieces of information:
+ the number of records in the child node itself, and the total number
+ of records in the child node and all its descendants. Storing this
+ additional information allows fast array-like indexing to locate
+ the n<sup>th</sup> record in the B-tree.</p>
+
+ <p>The entry into a version 2 B-tree is a header which contains global
+ information about the structure of the B-tree. The <em>root node
+ address</em>
+ field in the header points to the B-tree root node, which is either an
+ internal or leaf node, depending on the value in the header&rsquo;s
+ <em>depth</em> field. An internal node consists of records plus
+ pointers to further leaf or internal nodes in the tree. A leaf node
+ consists of solely of records. The format of the records depends on
+ the B-tree type (stored in the header).</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 B-tree Header
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+ <tr>
+ <td>Version</td>
+ <td>Type</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Node Size</td>
+ </tr>
+ <tr>
+ <td colspan="2">Record Size</td>
+ <td colspan="2">Depth</td>
+ </tr>
+ <tr>
+ <td>Split Percent</td>
+ <td>Merge Percent</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Root Node Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="2">Number of Records in Root Node</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Total Number of Records in B-tree<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 B-tree Header
+ </caption>
+
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>BTHD</code>&rdquo;
+ is used to indicate the header of a version 2 (v2) B-tree
+ node.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number for this B-tree header. This document
+ describes version 0.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Type</p></td>
+ <td>
+ <p>This field indicates the type of B-tree:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+ <tr>
+ <td align="center">0</td>
+ <td>This B-tree is used for testing only. This
+ value should <em>not</em> be used for storing
+ records in actual HDF5 files.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">1</td>
+ <td>This B-tree is used for indexing indirectly accessed,
+ non-filtered &lsquo;huge&rsquo; fractal heap objects.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">2</td>
+ <td>This B-tree is used for indexing indirectly accessed,
+ filtered &lsquo;huge&rsquo; fractal heap objects.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">3</td>
+ <td>This B-tree is used for indexing directly accessed,
+ non-filtered &lsquo;huge&rsquo; fractal heap objects.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">4</td>
+ <td>This B-tree is used for indexing directly accessed,
+ filtered &lsquo;huge&rsquo; fractal heap objects.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">5</td>
+ <td>This B-tree is used for indexing the &lsquo;name&rsquo; field for
+ links in indexed groups.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">6</td>
+ <td>This B-tree is used for indexing the &lsquo;creation order&rsquo;
+ field for links in indexed groups.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">7</td>
+ <td>This B-tree is used for indexing shared object header
+ messages.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">8</td>
+ <td>This B-tree is used for indexing the &lsquo;name&rsquo; field for
+ indexed attributes.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">9</td>
+ <td>This B-tree is used for indexing the &lsquo;creation order&rsquo;
+ field for indexed attributes.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center">10</td>
+ <td>This B-tree is used for indexing chunks of
+ datasets with no filters and with more than one
+ dimension of unlimited extent.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center">11</td>
+ <td>This B-tree is used for indexing chunks of
+ datasets with filters and more than one dimension
+ of unlimited extent.
+ </td>
+ </tr>
+ </table></p>
+ <p>The format of records for each type is described below.</p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Node Size</p></td>
+ <td>
+ <p>This is the size in bytes of all B-tree nodes.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Record Size</p></td>
+ <td>
+ <p>This field is the size in bytes of the B-tree record.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Depth</p></td>
+ <td>
+ <p>This is the depth of the B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Split Percent</p></td>
+ <td>
+ <p>The percent full that a node needs to increase above before it
+ is split.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Merge Percent</p></td>
+ <td>
+ <p>The percent full that a node needs to be decrease below before it
+ is split.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Root Node Address</p></td>
+ <td>
+ <p>This is the address of the root B-tree node. A B-tree with
+ no records will have the <a href="#UndefinedAddress">undefined
+ address</a> in this field.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Number of Records in Root Node</p></td>
+ <td>
+ <p>This is the number of records in the root node.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Total Number of Records in B-tree</p></td>
+ <td>
+ <p>This is the total number of records in the entire B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr valign="top">
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the B-tree header.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 B-tree Internal Node
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+ <tr>
+ <td>Version</td>
+ <td>Type</td>
+ <td colspan="2">Records 0, 1, 2...N-1 <em>(variable size)</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Child Node Pointer 0<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Number of Records N<sub>0</sub> for Child
+ Node 0 <em>(variable size)</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Total Number of Records for Child Node 0
+ <em>(optional, variable size)</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Child Node Pointer 1<sup>O</sup><br /><br /></td>
+ </tr>
+ <td colspan="4"><br />Number of Records N<sub>1</sub> for
+ Child Node 1 <em>(variable size)</em></td>
+</tr>
+<tr>
+ <td colspan="4"><br />Total Number of Records for Child Node 1
+ <em>(optional, variable size)</em></td>
+</tr>
+<tr>
+ <td colspan="4">...</td>
+</tr>
+<tr>
+ <td colspan="4"><br />Child Node Pointer N<sup>O</sup><br /><br /></td>
+</tr>
+<tr>
+ <td colspan="4"><br />Number of Records N<sub>n</sub> for
+ Child Node N <em>(variable size)</em></td>
+</tr>
+<tr>
+ <td colspan="4"><br />Total Number of Records for Child Node N
+ <em>(optional, variable size)</em></td>
+</tr>
+<tr>
+ <td colspan="4">Checksum</td>
+</tr>
+</table>
+
+<table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+</table>
+</div>
+
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 B-tree Internal Node
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>BTIN</code>&rdquo; is
+ used to indicate the internal node of a B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number for this B-tree internal node.
+ This document describes version 0.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Type</p></td>
+ <td>
+ <p>This field is the type of the B-tree node. It should always
+ be the same as the B-tree type in the header.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Records</p></td>
+ <td>
+ <p>The size of this field is determined by the number of records
+ for this node and the record size (from the header). The format
+ of records depends on the type of B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Child Node Pointer</p></td>
+ <td>
+ <p>This field is the address of the child node pointed to by the
+ internal node.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Records in Child Node</p></td>
+ <td>
+ <p>This is the number of records in the child node pointed to by
+ the corresponding <em>Node Pointer</em>.
+ </p>
+ <p>The number of bytes used to store this field is determined by
+ the maximum possible number of records able to be stored in the
+ child node.
+ </p>
+ <p>
+ The maximum number of records in a child node is computed
+ in the following way:
+
+ <ul>
+ <li>Subtract the fixed size overhead for
+ the child node (for example, its signature, version,
+ checksum, and so on and <em>one</em> pointer triplet
+ of information for the child node (because there is one
+ more pointer triplet than records in each internal node))
+ from the size of nodes for the B-tree. </li>
+ <li>Divide that result by the size of a record plus the
+ pointer triplet of information stored to reach each
+ child node from this node.</li>
+ </ul>
+
+ </p>
+ <p>
+ Note that leaf nodes do not encode any
+ child pointer triplets, so the maximum number of records in a
+ leaf node is just the node size minus the leaf node overhead,
+ divided by the record size.
+ </p>
+ <p>
+ Also note that the first level of internal nodes above the
+ leaf nodes do not encode the <em>Total Number of Records in Child
+ Node</em> value in the child pointer triplets (since it is the
+ same as the <em>Number of Records in Child Node</em>), so the
+ maximum number of records in these nodes is computed with the
+ equation above, but using (<em>Child Pointer</em>, <em>Number of
+ Records in Child Node</em>) pairs instead of triplets.
+ </p>
+ <p>
+ The number of
+ bytes used to encode this field is the least number of bytes
+ required to encode the maximum number of records in a child
+ node value for the child nodes below this level
+ in the B-tree.
+ </p>
+ <p>
+ For example, if the maximum number of child records is
+ 123, one byte will be used to encode these values in this
+ node; if the maximum number of child records is
+ 20000, two bytes will be used to encode these values in this
+ node; and so on. The maximum number of bytes used to
+ encode these values is 8 (in other words, an unsigned
+ 64-bit integer).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Total Number of Records in Child Node</p></td>
+ <td>
+ <p>This is the total number of records for the node pointed to by
+ the corresponding <em>Node Pointer</em> and all its children.
+ This field exists only in nodes whose depth in the B-tree node
+ is greater than 1 (in other words, the &ldquo;twig&rdquo;
+ internal nodes, just above leaf nodes, do not store this
+ field in their child node pointers).
+ </p>
+ <p>The number of bytes used to store this field is determined by
+ the maximum possible number of records able to be stored in the
+ child node and its descendants.
+ </p>
+ <p>
+ The maximum possible number of records able to be stored in a
+ child node and its descendants is computed iteratively, in the
+ following way: The maximum number of records in a leaf node
+ is computed, then that value is used to compute the maximum
+ possible number of records in the first level of internal nodes
+ above the leaf nodes. Multiplying these two values together
+ determines the maximum possible number of records in child node
+ pointers for the level of nodes two levels above leaf nodes.
+ This process is continued up to any level in the B-tree.
+ </p>
+ <p>
+ The number of bytes used to encode this value is computed in
+ the same way as for the <em>Number of Records in Child Node</em>
+ field.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for this node.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 B-tree Leaf Node
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+ <tr>
+ <td>Version</td>
+ <td>Type</td>
+ <td colspan="2">Record 0, 1, 2...N-1 <em>(variable size)</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 B-tree Leaf Node
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>BTLF</code>&ldquo;
+ is used to indicate the leaf node of a version 2 (v2) B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number for this B-tree leaf node.
+ This document describes version 0.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Type</p></td>
+ <td>
+ <p>This field is the type of the B-tree node. It should always
+ be the same as the B-tree type in the header.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Records</p></td>
+ <td>
+ <p>The size of this field is determined by the number of records
+ for this node and the record size (from the header). The format
+ of records depends on the type of B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for this node.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<p>The record layout for each stored (in other words, non-testing)
+ B-tree type is as follows:</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 B-tree, Type 1 Record Layout - Indirectly
+ Accessed, Non-filtered, &lsquo;Huge&rsquo; Fractal Heap Objects
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Huge Object Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Huge Object Length<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Huge Object ID<sup>L</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 B-tree, Type 1 Record Layout - Indirectly
+ Accessed, Non-filtered, &lsquo;Huge&rsquo; Fractal Heap Objects
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Huge Object Address</p></td>
+ <td>
+ <p>The address of the huge object in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Huge Object Length</p></td>
+ <td>
+ <p>The length of the huge object in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Huge Object ID</p></td>
+ <td>
+ <p>The heap ID for the huge object.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 B-tree, Type 2 Record Layout - Indirectly
+ Accessed, Filtered, &lsquo;Huge&rsquo; Fractal Heap Objects
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Filtered Huge Object Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Filtered Huge Object Length<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Filter Mask</td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Filtered Huge Object Memory Size<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Huge Object ID<sup>L</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 B-tree, Type 2 Record Layout - Indirectly
+ Accessed, Filtered, &lsquo;Huge&rsquo; Fractal Heap Objects
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Filtered Huge Object Address</p></td>
+ <td>
+ <p>The address of the filtered huge object in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filtered Huge Object Length</p></td>
+ <td>
+ <p>The length of the filtered huge object in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filter Mask</p></td>
+ <td>
+ <p>A 32-bit bit field indicating which filters have been skipped for
+ this chunk. Each filter has an index number in the pipeline
+ (starting at 0, with the first filter to apply) and if that
+ filter is skipped, the bit corresponding to its index is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filtered Huge Object Memory Size</p></td>
+ <td>
+ <p>The size of the de-filtered huge object in memory.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Huge Object ID</p></td>
+ <td>
+ <p>The heap ID for the huge object.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 B-tree, Type 3 Record Layout - Directly
+ Accessed, Non-filtered, &lsquo;Huge&rsquo; Fractal Heap Objects
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Huge Object Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Huge Object Length<sup>L</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 B-tree, Type 3 Record Layout - Directly
+ Accessed, Non-filtered, &lsquo;Huge&rsquo; Fractal Heap Objects
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Huge Object Address</p></td>
+ <td>
+ <p>The address of the huge object in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Huge Object Length</p></td>
+ <td>
+ <p>The length of the huge object in the file.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 B-tree, Type 4 Record Layout - Directly
+ Accessed, Filtered, &lsquo;Huge&rsquo; Fractal Heap Objects
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Filtered Huge Object Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Filtered Huge Object Length<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Filter Mask</td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Filtered Huge Object Memory Size<sup>L</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 B-tree, Type 4 Record Layout - Directly
+ Accessed, Filtered, &lsquo;Huge&rsquo; Fractal Heap Objects
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Filtered Huge Object Address</p></td>
+ <td>
+ <p>The address of the filtered huge object in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filtered Huge Object Length</p></td>
+ <td>
+ <p>The length of the filtered huge object in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filter Mask</p></td>
+ <td>
+ <p>A 32-bit bit field indicating which filters have been skipped for
+ this chunk. Each filter has an index number in the pipeline
+ (starting at 0, with the first filter to apply) and if that
+ filter is skipped, the bit corresponding to its index is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filtered Huge Object Memory Size</p></td>
+ <td>
+ <p>The size of the de-filtered huge object in memory.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 B-tree, Type 5 Record Layout - Link Name
+ for Indexed Group
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Hash of Name</td>
+ </tr>
+ <tr>
+ <td colspan="4">ID <em>(bytes 1-4)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="3">ID <em>(bytes 5-7)</em></td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 B-tree, Type 5 Record Layout - Link Name
+ for Indexed Group
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Hash</p></td>
+ <td>
+ <p>This field is hash value of the name for the link. The hash
+ value is the Jenkins&rsquo; lookup3 checksum algorithm applied to
+ the link&rsquo;s name.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>ID</p></td>
+ <td>
+ <p>This is a 7-byte sequence of bytes and is the heap ID for the
+ link record in the group&rsquo;s fractal heap.</p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 B-tree, Type 6 Record Layout - Creation
+ Order for Indexed Group
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Creation Order
+ <em>(8 bytes)</em><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">ID <em>(bytes 1-4)</em></td>
+ </tr>
+ <tr>
+ <td colspan="3">ID <em>(bytes 5-7)</em></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 B-tree, Type 6 Record Layout - Creation
+ Order for Indexed Group
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Creation Order</p></td>
+ <td>
+ <p>This field is the creation order value for the link.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>ID</p></td>
+ <td>
+ <p>This is a 7-byte sequence of bytes and is the heap ID for the
+ link record in the group&rsquo;s fractal heap.</p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 B-tree, Type 7 Record Layout - Shared
+ Object Header Messages (Sub-type 0 - Message in Heap)
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan>Message Location</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Hash</td>
+ </tr>
+ <tr>
+ <td colspan="4">Reference Count</td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Heap ID <em>(8 bytes)</em><br /><br /></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 B-tree, Type 7 Record Layout - Shared
+ Object Header Messages (Sub-type 0 - Message in Heap)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Message Location</p></td>
+ <td>
+ <p>This field Indicates the location where the message is stored:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+ <tr>
+ <td align="center">0</td>
+ <td>Shared message is stored in shared message index heap.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">1</td>
+ <td>Shared message is stored in object header.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Hash</p></td>
+ <td>
+ <p>This field is hash value of the shared message. The hash
+ value is the Jenkins&rsquo; lookup3 checksum algorithm applied to
+ the shared message.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Reference Count</p></td>
+ <td>
+ <p>The number of objects which reference this message.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Heap ID</p></td>
+ <td>
+ <p>This is an 8-byte sequence of bytes and is the heap ID for the
+ shared message in the shared message index&rsquo;s fractal heap.</p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 B-tree, Type 7 Record Layout - Shared
+ Object Header Messages (Sub-type 1 - Message in Object Header)
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan>Message Location</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Hash</td>
+ </tr>
+ <tr>
+ <td>Reserved (zero)</td>
+ <td>Message Type</td>
+ <td colspan="2">Object Header Index</td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Object Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 B-tree, Type 7 Record Layout - Shared
+ Object Header Messages (Sub-type 1 - Message in Object Header)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Message Location</p></td>
+ <td>
+ <p>This field Indicates the location where the message is stored:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+ <tr>
+ <td align="center">0</td>
+ <td>Shared message is stored in shared message index heap.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">1</td>
+ <td>Shared message is stored in object header.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Hash</p></td>
+ <td>
+ <p>This field is hash value of the shared message. The hash
+ value is the Jenkins&rsquo; lookup3 checksum algorithm applied to
+ the shared message.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Message Type</p></td>
+ <td>
+ <p>The object header message type of the shared message.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Header Index</p></td>
+ <td>
+ <p>This field indicates that the shared message is the n<sup>th</sup> message
+ of its type in the specified object header.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Header Address</p></td>
+ <td>
+ <p>The address of the object header containing the shared message.</p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 B-tree, Type 8 Record Layout - Attribute
+ Name for Indexed Attributes
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Heap ID <em>(8 bytes)</em><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan>Message Flags</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Creation Order</td>
+ </tr>
+ <tr>
+ <td colspan="4">Hash of Name</td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 B-tree, Type 8 Record Layout - Attribute
+ Name for Indexed Attributes
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Heap ID</p></td>
+ <td>
+ <p>This is an 8-byte sequence of bytes and is the heap ID for the
+ attribute in the object&rsquo;s attribute fractal heap.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Message Flags</p></td>
+ <td><p>The object header message flags for the attribute message.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Creation Order</p></td>
+ <td>
+ <p>This field is the creation order value for the attribute.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Hash</p></td>
+ <td>
+ <p>This field is hash value of the name for the attribute. The hash
+ value is the Jenkins&rsquo; lookup3 checksum algorithm applied to
+ the attribute&rsquo;s name.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 B-tree, Type 9 Record Layout - Creation
+ Order for Indexed Attributes
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Heap ID <em>(8 bytes)</em><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan>Message Flags</td>
+ <td colspan="3" bgcolor="#DDDDDD">
+ <em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Creation Order</td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 B-tree, Type 9 Record Layout - Creation
+ Order for Indexed Attributes
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Heap ID</p></td>
+ <td>
+ <p>This is an 8-byte sequence of bytes and is the heap ID for the
+ attribute in the object&rsquo;s attribute fractal heap.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Message Flags</p></td>
+ <td>
+ <p>The object header message flags for the attribute message.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Creation Order</p></td>
+ <td>
+ <p>This field is the creation order value for the attribute.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<a name="V2BtType10"></a>
+ <div align="center">
+ <table class="format">
+ <caption>
+ <a name="V2BtreesType10"></a>
+ Layout: Version 2 B-tree, Type 10 Record Layout -
+ Non-filtered Dataset Chunks
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Dimension 0 Scaled Offset
+ <em>(8 bytes)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Dimension 1 Scaled Offset
+ <em>(8 bytes)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />...<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Dimension #n Scaled Offset
+ <em>(8 bytes)</em><br /><br /></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 B-tree, Type 10 Record Layout -
+ Non-filtered Dataset Chunks
+</caption>
+<tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+</tr>
+
+<tr>
+ <td><p>Address</p></td>
+ <td>
+ <p>This field is the address of the dataset chunk in the file.</p>
+ </td>
+</tr>
+
+<tr>
+ <td><p>Dimension #n Scaled Offset</p></td>
+ <td>
+ <p>This field is the scaled offset of the chunk within the
+ dataset. <em>n</em> is the number of dimensions for the
+ dataset. The first scaled offset stored in the list is for
+ the slowest changing dimension, and the last scaled offset
+ stored is for the fastest changing dimension. Scaled offset
+ is calculated by dividing the chunk dimension sizes into
+ the chunk offsets.</p>
+ </td>
+</tr>
+
+</table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ <a name="V2BtreesType11"></a>
+ Layout: Version 2 B-tree, Type 11 Record Layout - Filtered
+ Dataset Chunks
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Chunk Size
+ <em>(variable size; at most 8 bytes)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Filter Mask</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Dimension 0 Scaled Offset
+ <em>(8 bytes)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Dimension 1 Scaled Offset
+ <em>(8 bytes)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />...<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Dimension #n Scaled Offset
+ <em>(8 bytes)</em><br /><br /></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 B-tree, Type 11 Record Layout - Filtered
+ Dataset Chunks
+</caption>
+<tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+</tr>
+
+<tr>
+ <td><p>Address</p></td>
+ <td>
+ <p>This field is the address of the dataset chunk in the file.</p>
+ </td>
+</tr>
+
+<tr>
+ <td><p>Chunk Size</p></td>
+ <td>
+ <p>This field is the size of the dataset chunk in bytes.</p>
+ </td>
+</tr>
+
+<tr>
+ <td><p>Filter Mask</p></td>
+ <td>
+ <p>This field is the filter mask which indicates the filter
+ to skip for the dataset chunk. Each filter has an index
+ number in the pipeline and if that filter is skipped,
+ the bit corresponding to its index is set.</p>
+ </td>
+</tr>
+
+<tr>
+ <td><p>Dimension #n Scaled Offset</p></td>
+ <td>
+ <p>This field is the scaled offset of the chunk within
+ the dataset. <em>n</em> is the number of dimensions for
+ the dataset. The first scaled offset stored in the list
+ is for the slowest changing dimension, and the last scaled
+ offset stored is for the fastest changing dimension.</p>
+ </td>
+</tr>
+
+</table>
+</div>
+
+<h3><a name="SymbolTable">
+ III.B. Disk Format: Level 1B - Group Symbol Table Nodes</a></h3>
+
+<p>A group is an object internal to the file that allows
+ arbitrary nesting of objects within the file (including other
+ groups). A group maps a set of link names in the group to a set
+ of relative file addresses of objects in the file. Certain metadata
+ for an object to which the group points can be cached in the
+ group&rsquo;s symbol table entry in addition to being in the
+ object&rsquo;s header.</p>
+
+<p>An HDF5 object name space can be stored hierarchically by
+ partitioning the name into components and storing each
+ component as a link in a group. The link for a
+ non-ultimate component points to the group containing
+ the next component. The link for the last
+ component points to the object being named.</p>
+
+<p>One implementation of a group is a collection of symbol table
+ nodes indexed by a B-tree. Each symbol table node contains entries
+ for one or more links. If an attempt is made to add a link to an
+ already full symbol table node containing 2<em>K</em> entries, then
+ the node is split and one node contains <em>K</em> symbols and the
+ other contains <em>K</em>+1 symbols.</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Symbol Table Node (A Leaf of a B-tree)
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version Number</td>
+ <td>Reserved <em>(zero)</em></td>
+ <td colspan="2">Number of Symbols</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br /><br />Group Entries<br /><br /><br /></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Symbol Table Node (A Leaf of a B-tree)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>SNOD</code>&rdquo; is
+ used to indicate the
+ beginning of a symbol table node. This gives file
+ consistency checking utilities a better chance of
+ reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version Number</p></td>
+ <td>
+ <p>The version number for the symbol table node. This
+ document describes version 1. (There is no version &lsquo;0&rsquo;
+ of the symbol table node)
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Entries</p></td>
+ <td>
+ <p>Although all symbol table nodes have the same length,
+ most contain fewer than the maximum possible number of
+ link entries. This field indicates how many entries
+ contain valid data. The valid entries are packed at the
+ beginning of the symbol table node while the remaining
+ entries contain undefined values.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Symbol Table Entries</p></td>
+ <td>
+ <p>Each link has an entry in the symbol table node.
+ The format of the entry is described below.
+ There are 2<em>K</em> entries in each group node, where
+ <em>K</em> is the &ldquo;Group Leaf Node K&rdquo; value from the
+ <a href="#Superblock">superblock</a>.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<h3><a name="SymbolTableEntry">
+ III.C. Disk Format: Level 1C - Symbol Table Entry </a></h3>
+
+<p>Each symbol table entry in a symbol table node is designed
+ to allow for very fast browsing of stored objects.
+ Toward that design goal, the symbol table entries
+ include space for caching certain constant metadata from the
+ object header.</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Symbol Table Entry
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Link Name Offset<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Object Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Cache Type</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reserved <em>(zero)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br /><br />Scratch-pad Space
+ <em>(16 bytes)</em><br /><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Symbol Table Entry
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Link Name Offset</p></td>
+ <td>
+ <p>This is the byte offset into the group&rsquo;s local
+ heap for the name of the link. The name is null
+ terminated.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Header Address</p></td>
+ <td>
+ <p>Every object has an object header which serves as a
+ permanent location for the object&rsquo;s metadata. In addition
+ to appearing in the object header, some of the object&rsquo;s metadata
+ can be cached in the scratch-pad space.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Cache Type</p></td>
+ <td>
+ <p>The cache type is determined from the object header.
+ It also determines the format for the scratch-pad space:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Type</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+ <tr>
+ <td align="center">0</td>
+ <td>No data is cached by the group entry. This
+ is guaranteed to be the case when an object header
+ has a link count greater than one.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">1</td>
+ <td>Group object header metadata is cached in the
+ scratch-pad space. This implies that the symbol table
+ entry refers to another group.
+ </td>
+ </tr>
+ <tr>
+ <td align="center">2</td>
+ <td>The entry is a symbolic link. The first four bytes
+ of the scratch-pad space are the offset into the local
+ heap for the link value. The object header address
+ will be undefined.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Reserved</p></td>
+ <td>
+ <p>These four bytes are present so that the scratch-pad
+ space is aligned on an eight-byte boundary. They are
+ always set to zero.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Scratch-pad Space</p></td>
+ <td>
+ <p>This space is used for different purposes, depending
+ on the value of the Cache Type field. Any metadata
+ about an object represented in the scratch-pad
+ space is duplicated in the object header for that
+ object.
+ </p>
+ <p>
+ Furthermore, no data is cached in the group
+ entry scratch-pad space if the object header for
+ the object has a link count greater than one.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<h4>Format of the Scratch-pad Space</h4>
+
+<p>The symbol table entry scratch-pad space is formatted
+ according to the value in the Cache Type field.</p>
+
+<p>If the Cache Type field contains the value zero
+ <code>(0)</code> then no information is
+ stored in the scratch-pad space.</p>
+
+<p>If the Cache Type field contains the value one
+ <code>(1)</code>, then the scratch-pad space
+ contains cached metadata for another object header
+ in the following format:</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Object Header Scratch-pad Format
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of B-tree<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Name Heap<sup>O</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Object Header Scratch-pad Format
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Address of B-tree</p></td>
+ <td>
+ <p>This is the file address for the root of the
+ group&rsquo;s B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address of Name Heap</p></td>
+ <td>
+ <p>This is the file address for the group&rsquo;s local
+ heap, in which are stored the group&rsquo;s symbol names.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+
+<br />
+<br />
+<br />
+<p>If the Cache Type field contains the value two
+ <code>(2)</code>, then the scratch-pad space
+ contains cached metadata for a symbolic link
+ in the following format:</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Symbolic Link Scratch-pad Format
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Offset to Link Value</td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Symbolic Link Scratch-pad Format
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Offset to Link Value</p></td>
+ <td>
+ <p>The value of a symbolic link (that is, the name of the
+ thing to which it points) is stored in the local heap.
+ This field is the 4-byte offset into the local heap for
+ the start of the link value, which is null terminated.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<h3><a name="LocalHeap">
+ III.D. Disk Format: Level 1D - Local Heaps</a></h3>
+
+<p>A local heap is a collection of small pieces of data that are particular
+ to a single object in the HDF5 file. Objects can be
+ inserted and removed from the heap at any time.
+ The address of a heap does not change once the heap is created.
+ For example, a group stores addresses of objects in symbol table nodes
+ with the names of links stored in the group&rsquo;s local heap.
+</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Local Heap
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3">Reserved <em>(zero)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Data Segment Size<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Offset to Head of Free-list<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Data Segment<sup>O</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Local Heap
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>HEAP</code>&rdquo;
+ is used to indicate the
+ beginning of a heap. This gives file consistency
+ checking utilities a better chance of reconstructing a
+ damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>Each local heap has its own version number so that new
+ heaps can be added to old files. This document
+ describes version zero (0) of the local heap.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data Segment Size</p></td>
+ <td>
+ <p>The total amount of disk memory allocated for the heap
+ data. This may be larger than the amount of space
+ required by the objects stored in the heap. The extra
+ unused space in the heap holds a linked list of free blocks.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Offset to Head of Free-list</p></td>
+ <td>
+ <p>This is the offset within the heap data segment of the
+ first free block (or the
+ <a href="#UndefinedAddress">undefined address</a> if there is no
+ free block). The free block contains
+ <a href="#SizeOfLengthsV0">Size of Lengths</a> bytes that
+ are the offset of the next free block (or the
+ value &lsquo;1&rsquo; if this is the
+ last free block) followed by Size of Lengths bytes that store
+ the size of this free block. The size of the free block includes
+ the space used to store the offset of the next free block and
+ the size of the current block, making the minimum size of a free
+ block 2 * Size of Lengths.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address of Data Segment</p></td>
+ <td>
+ <p>The data segment originally starts immediately after
+ the heap header, but if the data segment must grow as a
+ result of adding more objects, then the data segment may
+ be relocated, in its entirety, to another part of the
+ file.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<p>Objects within a local heap should be aligned on an 8-byte boundary.</p>
+
+<h3><a name="GlobalHeap">
+ III.E. Disk Format: Level 1E - Global Heap</a></h3>
+
+<p>Each HDF5 file has a global heap which stores various types of
+ information which is typically shared between datasets. The
+ global heap was designed to satisfy these goals:</p>
+
+<ol type="A">
+ <li>Repeated access to a heap object must be efficient without
+ resulting in repeated file I/O requests. Since global heap
+ objects will typically be shared among several datasets, it is
+ probable that the object will be accessed repeatedly.</li>
+ <li>Collections of related global heap objects should result in
+ fewer and larger I/O requests. For instance, a dataset of
+ object references will have a global heap object for each
+ reference. Reading the entire set of object references
+ should result in a few large I/O requests instead of one small
+ I/O request for each reference.</li>
+ <li>It should be possible to remove objects from the global heap
+ and the resulting file hole should be eligible to be reclaimed
+ for other uses.</li>
+</ol>
+
+
+<p>The implementation of the heap makes use of the memory management
+ already available at the file level and combines that with a new
+ object called a <em>collection</em> to achieve goal B. The global heap
+ is the set of all collections. Each global heap object belongs to
+ exactly one collection, and each collection contains one or more global
+ heap objects. For the purposes of disk I/O and caching, a collection is
+ treated as an atomic object, addressing goal A.
+</p>
+
+<p>When a global heap object is deleted from a collection (which
+ occurs when its reference count falls to zero), objects located
+ after the deleted object in the collection are packed down toward
+ the beginning of the collection, and the collection&rsquo;s
+ global heap object 0 is created (if possible), or its size is
+ increased to account for the recently freed space. There are
+ no gaps between objects in each collection, with the possible
+ exception of the final space in the collection, if it is not
+ large enough to hold the header for the collection&rsquo;s
+ global heap object 0. These features address goal C.
+</p>
+
+<p>The HDF5 Library creates global heap collections as needed, so there may
+ be multiple collections throughout the file. The set of all of them is
+ abstractly called the &ldquo;global heap&rdquo;, although they do not actually link
+ to each other, and there is no global place in the file where you can
+ discover all of the collections. The collections are found simply by
+ finding a reference to one through another object in the file. For
+ example, data of variable-length datatype elements is stored in the
+ global heap and is accessed via a global heap ID. The format for
+ global heap IDs is described at the end of this section.
+</p>
+
+<p>For more information on global heaps for virtual datasets, see
+ <a href="#GlobalHeapVDS">&ldquo;Disk Format: Level 1F - Global Heap
+ Block for Virtual Datasets.&rdquo;</a></p>
+<br />
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: A Global Heap Collection
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Collection Size<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Global Heap Object 1<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Global Heap Object 2<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />...<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Global Heap Object <em>N</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Global Heap Object 0 (free space)<br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: A Global Heap Collection
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>GCOL</code>&rdquo;
+ is used to indicate the
+ beginning of a collection. This gives file consistency
+ checking utilities a better chance of reconstructing a
+ damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>Each collection has its own version number so that new
+ collections can be added to old files. This document
+ describes version one (1) of the collections (there is no
+ version zero (0)).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Collection Size</p></td>
+ <td>
+ <p>This is the size in bytes of the entire collection
+ including this field. The default (and minimum)
+ collection size is 4096 bytes which is a typical file
+ system block size. This allows for 127 16-byte heap
+ objects plus their overhead (the collection header of 16 bytes
+ and the 16 bytes of information about each heap object).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Global Heap Object 1 through <em>N</em></p></td>
+ <td>
+ <p>The objects are stored in any order with no
+ intervening unused space.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Global Heap Object 0</p></td>
+ <td>
+ <p>Global Heap Object 0 (zero), when present, represents the free
+ space in the collection. Free space always appears at the end of
+ the collection. If the free space is too small to store the header
+ for Object 0 (described below) then the header is implied and is not
+ written.
+ <p>
+ The field <em>Object Size</em> for Object 0 indicates the
+ amount of possible free space in the collection including the 16-byte
+ header size of Object 0.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Global Heap Object
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="2">Heap Object Index</td>
+ <td colspan="2">Reference Count</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Object Size<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Object Data<br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Global Heap Object
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Heap Object Index</p></td>
+ <td>
+ <p>Each object has a unique identification number within a
+ collection. The identification numbers are chosen so that
+ new objects have the smallest value possible with the
+ exception that the identifier <code>0</code> always refers to the
+ object which represents all free space within the
+ collection.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Reference Count</p></td>
+ <td>
+ <p>All heap objects have a reference count field. An
+ object which is referenced from some other part of the
+ file will have a positive reference count. The reference
+ count for Object 0 is always zero.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Reserved</p></td>
+ <td>
+ <p>Zero padding to align next field on an 8-byte boundary.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Size</p></td>
+ <td>
+ <p>This is the size of the object data stored for the object.
+ The actual storage space allocated for the object data is rounded
+ up to a multiple of eight.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Data</p></td>
+ <td>
+ <p>The object data is treated as a one-dimensional array
+ of bytes to be interpreted by the caller.
+ </p>
+ </td>
+ </tr>
+ </table>
+
+</div>
+
+<br />
+<br />
+<br />
+<p>
+ <a name="GlobalHeapID"></a>
+ The format for the ID used to locate an object in the global heap is
+ described here:</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Global Heap ID
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Collection Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Object Index</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Global Heap ID
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Collection Address</p></td>
+ <td>
+ <p>This field is the address of the global heap collection
+ where the data object is stored.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>ID</p></td>
+ <td>
+ <p>This field is the index of the data object within the
+ global heap collection.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+
+
+<h3><a name="GlobalHeapVDS"> III.F. Disk Format: Level 1F - Global
+ Heap Block for Virtual Datasets</a></h3>
+
+<p>The layout for the global heap block used with virtual datasets is
+ described below. For more information on global heaps, see
+ <a href="#GlobalHeap"></a>&ldquo;Disk Format: Level 1E - Global Heap.&rdquo;</p>
+
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Global Heap Block for Virtual Dataset
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted
+ only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Num Entries<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Source Filename #1 <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Source Dataset #1 <em>(variable
+ size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Source Selection #1 <em>(variable
+ size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Virtual Selection #1 <em>(variable
+ size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Source Filename #n <em>(variable
+ size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Source Dataset #n <em>(variable
+ size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Source Selection #n <em>(variable
+ size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Virtual Selection #n <em>(variable
+ size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Global Heap Block for Virtual Dataset
+ </caption>
+ <tr>
+ <th width="40%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number for the block; the value is 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Num Entries</p></td>
+ <td><p>The number of entries in the block.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Source Filename #n</p></td>
+ <td>
+ <p>The source file name where the source dataset is located.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Source Dataset #n</p></td>
+ <td><p>The source dataset name that is mapped to the
+ virtual dataset.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Source Selection #n</p></td>
+ <td>
+ <p>The <a href="#DataspaceSEL">dataspace selection</a> in the
+ source dataset that is mapped to the virtual selection.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Virtual Selection #n</p></td>
+ <td>
+ <p>This is the <a href="#DataspaceSEL">dataspace selection</a> in the virtual dataset that is
+ mapped to the source selection.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the block.</p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+<br>
+
+<h3><a name="FractalHeap">
+ III.G. Disk Format: Level 1G - Fractal Heap</a></h3>
+
+<p>
+ Each fractal heap consists of a header and zero or more direct and
+ indirect blocks (described below). The header contains general
+ information as well as
+ initialization parameters for the doubling table. The <em>Address
+ of Root Block</em> field in the header points to the first direct or
+ indirect block in the heap.
+</p>
+
+<p>
+ Fractal heaps are based on a data structure called a <em>doubling
+ table</em>. A doubling table provides a mechanism for quickly
+ extending an array-like data structure that minimizes the number of
+ empty blocks in the heap, while retaining very fast lookup of any
+ element within the array. More information on fractal heaps and
+ doubling tables can be found in the RFC
+ &ldquo;<a href="Supplements/FractalHeap/PrivateHeap.pdf">Private
+ Heaps in HDF5</a>.&rdquo;
+</p>
+
+<p>
+ The fractal heap implements the doubling table structure with
+ indirect and direct blocks.
+ Indirect blocks in the heap do not actually contain data for
+ objects in the heap, their &ldquo;size&rdquo; is abstract -
+ they represent the indexing structure for locating the
+ direct blocks in the doubling table.
+ Direct blocks
+ contain the actual data for objects stored in the heap.
+</p>
+
+<p>
+ All indirect blocks have a constant number of block entries in each
+ row, called the <em>width</em> of the doubling table
+ (see <em>Table Width</em> field in the header).
+
+ The number
+ of rows for each indirect block in the heap is determined by the
+ size of the block that the indirect block represents in the
+ doubling table (calculation of this is shown below) and is
+ constant, except for the &ldquo;root&rdquo;
+ indirect block, which expands and shrinks its number of rows as
+ needed.
+</p>
+
+<p>
+ Blocks in the first <em>two</em> rows of an indirect block
+ are <em>Starting Block Size</em> number of bytes in size.
+ For example, if the row <em>width</em> of the doubling table is 4,
+ then the first eight block entries in the
+ indirect block are <em>Starting Block Size</em> number of bytes in size.
+ The blocks in each subsequent row are twice the size of
+ the blocks in the previous row. In other words, blocks in
+ the third row are twice the <em>Starting Block Size</em>,
+ blocks in the fourth row are four times the
+ <em>Starting Block Size</em>, and so on. Entries for
+ blocks up to the <em>Maximum Direct Block Size</em> point to
+ direct blocks, and entries for blocks greater than that size
+ point to further indirect blocks (which have their own
+ entries for direct and indirect blocks).
+ <em>Starting Block Size</em> and
+ <em>Maximum Direct Block Size</em> are fields
+ stored in the header.
+</p>
+
+<p>
+ The number of rows of blocks, <em>nrows</em>, in an
+ indirect block is calculated by the following expression:
+ <br /> <br />
+ <em>nrows</em> = (log<sub>2</sub>(<em>block_size</em>) -
+ log<sub>2</sub>(<em>&lt;Starting Block Size&gt;</em>)) + 1
+</p>
+where <em>block_size</em> is the size of the block that the indirect block
+represents in the doubling table.
+For example, to represent a block with <em>block_size</em> equals to 1024,
+and <em>Starting Block Size</em> equals to 256,
+three rows are needed.
+<p>
+ The maximum number of rows of direct blocks, <em>max_dblock_rows</em>,
+ in any indirect block of a fractal heap is given by the
+ following expression:
+ <br /> <br />
+ <em>max_dblock_rows</em> =
+ (log<sub>2</sub>(<em>&lt;Maximum Direct Block Size&gt;</em>) -
+ log<sub>2</sub>(<em>&lt;Starting Block Size&gt;</em>)) + 2
+</p>
+<p>
+ Using the computed values for <em>nrows</em> and
+ <em>max_dblock_rows</em>, along with the <em>width</em> of the
+ doubling table, the number of direct and indirect block entries
+ (<em>K</em> and <em>N</em> in the indirect block description, below)
+ in an indirect block can be computed:
+ <br /> <br />
+ <em>K</em> = MIN(<em>nrows</em>, <em>max_dblock_rows</em>) *
+ <em>&lt;Table Width&gt;</em>
+
+ <br /> <br />
+ If <em>nrows</em> is less than or equal to <em>max_dblock_rows</em>,
+ <em>N</em> is 0. Otherwise, <em>N</em> is simply computed:
+ <br /> <br />
+ <em>N</em> = <em>K</em> - (<em>max_dblock_rows</em> *
+ <em>&lt;Table Width&gt;</em>)
+</p>
+
+<p>
+ The size of indirect blocks on disk is determined by the number
+ of rows in the indirect block (computed above). The size of direct
+ blocks on disk is exactly the size of the block in the doubling
+ table.
+</p>
+<br>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fractal Heap Header
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Heap ID Length</td>
+ <td colspan="2">I/O Filters&rsquo; Encoded Length</td>
+ </tr>
+
+ <tr>
+ <td>Flags</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Maximum Size of Managed Objects</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Next Huge Object ID<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />v2 B-tree Address of Huge Objects<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Amount of Free Space in Managed Blocks<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Managed Block Free Space Manager<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Amount of Managed Space in Heap<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Amount of Allocated Managed Space in Heap<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Offset of Direct Block Allocation Iterator in Managed Space<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Number of Managed Objects in Heap<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Size of Huge Objects in Heap<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Number of Huge Objects in Heap<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Size of Tiny Objects in Heap<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Number of Tiny Objects in Heap<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Table Width</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Starting Block Size<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Maximum Direct Block Size<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Maximum Heap Size</td>
+ <td colspan="2">Starting # of Rows in Root Indirect Block</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Root Block<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Current # of Rows in Root Indirect Block</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Size of Filtered Root Direct Block <em>(optional)</em><sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">I/O Filter Mask<em> (optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">I/O Filter Information<em> (optional, variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fractal Heap Header
+ </caption>
+ <tr>
+ <th width="40%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>FRHP</code>&rdquo;
+ is used to indicate the
+ beginning of a fractal heap header. This gives file consistency
+ checking utilities a better chance of reconstructing a
+ damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Heap ID Length</p></td>
+ <td>
+ <p>This is the length in bytes of heap object IDs for this heap.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>I/O Filters&rsquo; Encoded Length</p></td>
+ <td>
+ <p>This is the size in bytes of the encoded <em>I/O Filter Information</em>.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td>
+ <p>This field is the heap status flag and is a bit field
+ indicating additional information about the fractal heap.
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit(s)</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, the ID value to use for huge object has wrapped
+ around. If the value for the <em>Next Huge Object ID</em>
+ has wrapped around, each new huge object inserted into the
+ heap will require a search for an ID value.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>If set, the direct blocks in the heap are checksummed.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2-7</code></td>
+ <td>Reserved</td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Maximum Size of Managed Objects</p></td>
+ <td>
+ <p>This is the maximum size of managed objects allowed in the heap.
+ Objects greater than this this are &lsquo;huge&rsquo; objects and will be
+ stored in the file directly, rather than in a direct block for
+ the heap.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Next Huge Object ID</p></td>
+ <td>
+ <p>This is the next ID value to use for a huge object in the heap.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>v2 B-tree Address of Huge Objects</p></td>
+ <td>
+ <p>This is the address of the <a href="#V2Btrees">v2 B-tree</a>
+ used to track huge objects in the heap. The type of records
+ stored in the <em>v2 B-tree</em> will
+ be determined by whether the address and length of a huge object
+ can fit into a heap ID (if yes, it is a &ldquo;directly&rdquo; accessed
+ huge object) and whether there is a filter used on objects
+ in the heap.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Amount of Free Space in Managed Blocks</p></td>
+ <td>
+ <p>This is the total amount of free space in managed direct blocks
+ (in bytes).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address of Managed Block Free Space Manager</p></td>
+ <td>
+ <p>This is the address of the
+ <em><a href="#FreeSpaceManager">Free-space Manager</a></em> for
+ managed blocks.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Amount of Managed Space in Heap</p></td>
+ <td>
+ <p>This is the total amount of managed space in the heap (in bytes),
+ essentially the upper bound of the heap&rsquo;s linear address space.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Amount of Allocated Managed Space in Heap</p></td>
+ <td>
+ <p>This is the total amount of managed space (in bytes) actually
+ allocated in
+ the heap. This can be less than the <em>Amount of Managed Space
+ in Heap</em> field, if some direct blocks in the heap&rsquo;s linear
+ address space are not allocated.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Offset of Direct Block Allocation Iterator in Managed Space</p></td>
+ <td>
+ <p>This is the linear heap offset where the next direct
+ block should be allocated at (in bytes). This may be less than
+ the <em>Amount of Managed Space in Heap</em> value because the
+ heap&rsquo;s address space is increased by a &ldquo;row&rdquo; of direct blocks
+ at a time, rather than by single direct block increments.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Managed Objects in Heap</p></td>
+ <td>
+ <p>This is the number of managed objects in the heap.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Huge Objects in Heap</p></td>
+ <td>
+ <p>This is the total size of huge objects in the heap (in bytes).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Huge Objects in Heap</p></td>
+ <td>
+ <p>This is the number of huge objects in the heap.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Tiny Objects in Heap</p></td>
+ <td>
+ <p>This is the total size of tiny objects that are packed in heap
+ IDs (in bytes).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Tiny Objects in Heap</p></td>
+ <td>
+ <p>This is the number of tiny objects that are packed in heap IDs.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Table Width</p></td>
+ <td>
+ <p>This is the number of columns in the doubling table for managed
+ blocks. This value must be a power of two.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Starting Block Size</p></td>
+ <td>
+ <p>This is the starting block size to use in the doubling table for
+ managed blocks (in bytes). This value must be a power of two.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Maximum Direct Block Size</p></td>
+ <td>
+ <p>This is the maximum size allowed for a managed direct block.
+ Objects inserted into the heap that are larger than this value
+ (less the number of bytes of direct block prefix/suffix)
+ are stored as &lsquo;huge&rsquo; objects. This value must be a power of
+ two.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Maximum Heap Size</p></td>
+ <td>
+ <p>This is the maximum size of the heap&rsquo;s linear address space for
+ managed objects (in bytes). The value stored is the log2 of
+ the actual value, that is: the number of bits of the address space.
+ &lsquo;Huge&rsquo; and &lsquo;tiny&rsquo; objects are not counted in this value, since
+ they do not store objects in the linear address space of the
+ heap.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Starting # of Rows in Root Indirect Block</p></td>
+ <td>
+ <p>This is the starting number of rows for the root indirect block.
+ A value of 0 indicates that the root indirect block will have
+ the maximum number of rows needed to address the heap&rsquo;s <em>Maximum
+ Heap Size</em>.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address of Root Block</p></td>
+ <td>
+ <p>This is the address of the root block for the heap. It can
+ be the <a href="#UndefinedAddress">undefined address</a> if
+ there is no data in the heap. It either points to a direct
+ block (if the <em>Current # of Rows in the Root Indirect
+ Block</em> value is 0), or an indirect block.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Current # of Rows in Root Indirect Block</p></td>
+ <td>
+ <p>This is the current number of rows in the root indirect block.
+ A value of 0 indicates that <em>Address of Root Block</em>
+ points to direct block instead of indirect block.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Filtered Root Direct Block</p></td>
+ <td>
+ <p>This is the size of the root direct block, if filters are
+ applied to heap objects (in bytes). This field is only
+ stored in the header if the <em>I/O Filters&rsquo; Encoded Length</em>
+ is greater than 0.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>I/O Filter Mask</p></td>
+ <td>
+ <p>This is the filter mask for the root direct block, if filters
+ are applied to heap objects. This mask has the same format as
+ that used for the filter mask in chunked raw data records in a
+ <a href="#V1Btrees">v1 B-tree</a>.
+ This field is only
+ stored in the header if the <em>I/O Filters&rsquo; Encoded Length</em>
+ is greater than 0.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>I/O Filter Information</p></td>
+ <td>
+ <p>This is the I/O filter information encoding direct blocks and
+ huge objects, if filters are applied to heap objects. This
+ field is encoded as a <a href="#FilterMessage">Filter Pipeline</a>
+ message.
+ The size of this field is determined by <em>I/O Filters&rsquo;
+ Encoded Length</em>.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the header.</p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fractal Heap Direct Block
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Heap Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Block Offset <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Object Data <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fractal Heap Direct Block
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>FHDB</code>&rdquo;
+ is used to indicate the
+ beginning of a fractal heap direct block. This gives file consistency
+ checking utilities a better chance of reconstructing a
+ damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Heap Header Address</p></td>
+ <td>
+ <p>This is the address for the fractal heap header that this
+ block belongs to. This field is principally used for file
+ integrity checking.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Block Offset</p></td>
+ <td>
+ <p>This is the offset of the block within the fractal heap&rsquo;s
+ address space (in bytes). The number of bytes used to encode
+ this field is the <em>Maximum Heap Size</em> (in the heap&rsquo;s
+ header) divided by 8 and rounded up to the next highest integer,
+ for values that are not a multiple of 8. This value is
+ principally used for file integrity checking.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the direct block.</p>
+ <p>This field is only present if bit 1 of <em>Flags</em> in the
+ heap&rsquo;s header is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Data</p></td>
+ <td>
+ <p>This section of the direct block stores the actual data for
+ objects in the heap. The size of this section is determined by
+ the direct block&rsquo;s size minus the size of the other fields
+ stored in the direct block (for example, the <em>Signature</em>,
+ <em>Version</em>, and others including the <em>Checksum</em> if it is
+ present).
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fractal Heap Indirect Block
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Heap Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Block Offset <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Child Direct Block #0 Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Size of Filtered Direct Block #0 <em>(optional)</em> <sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Filter Mask for Direct Block #0 <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Child Direct Block #1 Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Size of Filtered Direct Block #1 <em>(optional)</em><sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Filter Mask for Direct Block #1 <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Child Direct Block #K-1 Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Size of Filtered Direct Block #K-1 <em>(optional)</em><sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Filter Mask for Direct Block #K-1 <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Child Indirect Block #0 Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Child Indirect Block #1 Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Child Indirect Block #N-1 Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fractal Heap Indirect Block
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>FHIB</code>&rdquo; is used to
+ indicate the beginning of a fractal heap indirect block. This
+ gives file consistency checking utilities a better chance of
+ reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Heap Header Address</p></td>
+ <td>
+ <p>This is the address for the fractal heap header that this
+ block belongs to. This field is principally used for file
+ integrity checking.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Block Offset</p></td>
+ <td>
+ <p>This is the offset of the block within the fractal heap&rsquo;s
+ address space (in bytes). The number of bytes used to encode
+ this field is the <em>Maximum Heap Size</em> (in the heap&rsquo;s
+ header) divided by 8 and rounded up to the next highest integer,
+ for values that are not a multiple of 8. This value is
+ principally used for file integrity checking.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Child Direct Block #K Address</p></td>
+ <td>
+ <p>This field is the address of the child direct block.
+ The size of the [uncompressed] direct block can be computed by
+ its offset in the heap&rsquo;s linear address space.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Filtered Direct Block #K</p></td>
+ <td>
+ <p>This is the size of the child direct block after passing through
+ the I/O filters defined for this heap (in bytes). If no I/O
+ filters are present for this heap, this field is not present.
+ </p>
+ </td>
+ </tr>
+ <tr>
+ <td><p>Filter Mask for Direct Block #K</p></td>
+ <td>
+ <p>This is the I/O filter mask for the filtered direct block.
+ This mask has the same format as that used for the filter mask
+ in chunked raw data records in a <a href="#V1Btrees">v1 B-tree</a>.
+ If no I/O filters are present for this heap, this field is not
+ present.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Child Indirect Block #N Address</p></td>
+ <td>
+ <p>This field is the address of the child indirect block.
+ The size of the indirect block can be computed by
+ its offset in the heap&rsquo;s linear address space.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the indirect block.</p>
+ </td>
+ </tr>
+
+ </table>
+
+</div>
+
+<br />
+<p>An object in the fractal heap is identified by means of a fractal heap ID,
+ which encodes information to locate the object in the heap.
+ Currently, the fractal heap stores an object in one of three ways,
+ depending on the object&rsquo;s size:</p>
+
+<div align="center">
+ <table class="list80">
+ <tr>
+ <th width="20%">Type</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center">Tiny</td>
+ <td>
+ <p>When an object is small enough to be encoded in the
+ heap ID, the object&rsquo;s data is embedded in the fractal
+ heap ID itself. There are two sub-types for this type of
+ object: normal and extended. The sub-type for tiny heap
+ IDs depends on whether the heap ID is large enough to
+ store objects greater than 16 bytes or not. If the
+ heap ID length is 18 bytes or smaller, the
+ &lsquo;normal&rsquo; tiny heap ID form is used. If the
+ heap ID length is greater than 18 bytes in length, the
+ &ldquo;extended&rdquo; form is used. See the format
+ description below for both sub-types.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center">Huge</td>
+ <td>
+ <p>When the size of an object is larger than <em>Maximum Size of
+ Managed Objects</em> in the <em>Fractal Heap Header</em>, the
+ object&rsquo;s data is stored on its own in the file and the object
+ is tracked/indexed via a version 2 B-tree. All huge objects
+ for a particular fractal heap use the same v2 B-tree. All huge
+ objects for a particular fractal heap use the same format for
+ their huge object IDs.
+ </p>
+
+ <p>Depending on whether the IDs for a heap are large enough to hold
+ the object&rsquo;s retrieval information and whether I/O pipeline filters
+ are applied to the heap&rsquo;s objects, 4 sub-types are derived for
+ huge object IDs for this heap:</p>
+
+ <div align="center">
+ <table class="list">
+ <tr>
+ <th align="left" width="35%">Sub-type</th>
+ <th align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="left">Directly accessed, non-filtered</td>
+ <td>
+ <p>The object&rsquo;s address and length are embedded in the
+ fractal heap ID itself and the object is directly accessed
+ from them. This allows the object to be accessed without
+ resorting to the B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td align="left">Directly accessed, filtered</td>
+ <td>
+ <p>The filtered object&rsquo;s address, length, filter mask and
+ de-filtered size are embedded in the fractal heap ID itself
+ and the object is accessed directly with them. This allows
+ the object to be accessed without resorting to the B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td align="left">Indirectly accessed, non-filtered</td>
+ <td>
+ <p>The object is located by using a B-tree key embedded in
+ the fractal heap ID to retrieve the address and length from
+ the version 2 B-tree for huge objects. Then, the address
+ and length are used to access the object.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td align="left">Indirectly accessed, filtered</td>
+ <td>
+ <p>The object is located by using a B-tree key embedded in
+ the fractal heap ID to retrieve the filtered object&rsquo;s
+ address, length, filter mask and de-filtered size from the
+ version 2 B-tree for huge objects. Then, this information
+ is used to access the object.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center">Managed</td>
+ <td>
+ <p>When the size of an object does not meet the above two
+ conditions, the object is stored and managed via the direct and
+ indirect blocks based on the doubling table.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+
+<br />
+<p>The specific format for each type of heap ID is described below:
+</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fractal Heap ID for Tiny Objects (Sub-type 1 -
+ &lsquo;Normal&rsquo;)
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version, Type, and Length</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Data <em>(variable size)</em></td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fractal Heap ID for Tiny Objects (Sub-type 1 -
+ &lsquo;Normal&rsquo;)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version, Type, and Length</p></td>
+ <td>
+ <p>This is a bit field with the following definition:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6-7</code></td>
+ <td>The current version of ID format. This document
+ describes version 0.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4-5</code></td>
+ <td>The ID type. Tiny objects have a value of <code>2</code>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>0-3</code></td>
+ <td>The length of the tiny object. The value stored
+ is one less than the actual length (since zero-length
+ objects are not allowed to be stored in the heap).
+ For example, an object of actual length 1 has an
+ encoded length of 0, an object of actual length 2
+ has an encoded length of 1, and so on.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data</p></td>
+ <td>
+ <p>This is the data for the object.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fractal Heap ID for Tiny Objects (Sub-type 2 -
+ &lsquo;Extended&rsquo;)
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version, Type, and Length</td>
+ <td>Extended Length</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Data <em>(variable size)</em></td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fractal Heap ID for Tiny Objects (Sub-type 2 -
+ &lsquo;Extended&rsquo;)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version, Type, and Length</p></td>
+ <td>
+ <p>This is a bit field with the following definition:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6-7</code></td>
+ <td>The current version of ID format. This document
+ describes version 0.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4-5</code></td>
+ <td>The ID type. Tiny objects have a value of <code>2</code>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>0-3</code></td>
+ <td>These 4 bits, together with the next byte, form an
+ unsigned 12-bit integer for holding the length of the
+ object. These 4-bits are bits 8-11 of the 12-bit integer.
+ See description for the <em>Extended Length</em> field below.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Extended Length</p></td>
+ <td>
+ <p>This byte, together with the 4 bits in the previous byte,
+ forms an unsigned 12-bit integer for holding the length of
+ the tiny object. These 8 bits are bits 0-7 of the 12-bit
+ integer formed. The value stored is one less than the actual
+ length (since zero-length objects are not allowed to be
+ stored in the heap). For example, an object of actual length
+ 1 has an encoded length of 0, an object of actual length
+ 2 has an encoded length of 1, and so on.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data</p></td>
+ <td>
+ <p>This is the data for the object.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fractal Heap ID for Huge Objects (Sub-types 1 and 2):
+ Indirectly Accessed, Non-filtered/Filtered
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version and Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted
+ only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />v2 B-tree Key<sup>L</sup><em> (variable size)</em><br /><br /></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fractal Heap ID for Huge Objects (Sub-types 1 and 2):
+ Indirectly Accessed, Non-filtered/Filtered
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version and Type</p></td>
+ <td>
+ <p>This is a bit field with the following definition:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6-7</code></td>
+ <td>The current version of ID format. This document
+ describes version 0.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4-5</code></td>
+ <td>The ID type. Huge objects have a value of <code>1</code>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>0-3</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>v2 B-tree Key</p></td>
+ <td><p>This field is the B-tree key for retrieving the information
+ from the version 2 B-tree for huge objects needed to access the
+ object. See the description of <a href="#V2Btrees">v2 B-tree</a>
+ records sub-types 1 and 2 for a description of the fields. New key
+ values are derived from <em>Next Huge Object ID</em> in the
+ <em>Fractal Heap Header</em>.</p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fractal Heap ID for Huge Objects (Sub-type 3):
+ Directly Accessed, Non-filtered
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version and Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address <sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Length <sup>L</sup><br /><br /></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fractal Heap ID for Huge Objects (Sub-type 3):
+ Directly Accessed, Non-filtered
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version and Type</p></td>
+ <td>
+ <p>This is a bit field with the following definition:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6-7</code></td>
+ <td>The current version of ID format. This document
+ describes version 0.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4-5</code></td>
+ <td>The ID type. Huge objects have a value of <code>1</code>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>0-3</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>This field is the address of the object in the file.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Length</p></td>
+ <td><p>This field is the length of the object in the file.</p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fractal Heap ID for Huge Objects (Sub-type 4):
+ Directly Accessed, Filtered
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version and Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address <sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Length <sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Filter Mask</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />De-filtered Size <sup>L</sup><br /><br /></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fractal Heap ID for Huge Objects (Sub-type 4):
+ Directly Accessed, Filtered
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version and Type</p></td>
+ <td>
+ <p>This is a bit field with the following definition:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6-7</code></td>
+ <td>The current version of ID format. This document
+ describes version 0.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4-5</code></td>
+ <td>The ID type. Huge objects have a value of <code>1</code>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>0-3</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>This field is the address of the filtered object in the file.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Length</p></td>
+ <td><p>This field is the length of the filtered object in the file.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filter Mask</p></td>
+ <td><p>This field is the I/O pipeline filter mask for the
+ filtered object in the file.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filtered Size</p></td>
+ <td><p>This field is the size of the de-filtered object in the file.</p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fractal Heap ID for Managed Objects
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version and Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Offset <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Length <em>(variable size)</em></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fractal Heap ID for Managed Objects
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version and Type</p></td>
+ <td><p>This is a bit field with the following definition:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6-7</code></td>
+ <td>The current version of ID format. This document
+ describes version 0.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4-5</code></td>
+ <td>The ID type. Managed objects have a value of <code>0</code>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>0-3</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Offset</p></td>
+ <td><p>This field is the offset of the object in the heap.
+ This field&rsquo;s size is the minimum number of bytes
+ necessary to encode the <em>Maximum Heap Size</em> value
+ (from the <em>Fractal Heap Header</em>). For example, if the
+ value of the <em>Maximum Heap Size</em> is less than 256 bytes,
+ this field is 1 byte in length, a <em>Maximum Heap Size</em>
+ of 256-65535 bytes uses a 2 byte length, and so on.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Length</p></td>
+ <td><p>This field is the length of the object in the heap. It
+ is determined by taking the minimum value of <em>Maximum
+ Direct Block Size</em> and <em>Maximum Size of Managed
+ Objects</em> in the <em>Fractal Heap Header</em>. Again,
+ the minimum number of bytes needed to encode that value is
+ used for the size of this field.</p></td>
+ </tr>
+ </table>
+</div>
+
+<h3><a name="FreeSpaceManager">
+ III.H. Disk Format: Level 1H - Free-space Manager</a></h3>
+
+<p>
+ Free-space managers are used to describe space within a heap or
+ the entire HDF5 file that is not currently used for that heap or
+ file.
+</p>
+
+<p>
+ The <em>free-space manager header</em> contains metadata information
+ about the space being tracked, along with the address of the list
+ of <em>free space sections</em> which actually describes the free
+ space. The header records information about free-space sections being
+ tracked, creation parameters for handling free-space sections of a
+ client, and section information used to locate the collection of
+ free-space sections.
+</p>
+
+<p>
+ The <em>free-space section list</em> stores a collection of
+ free-space sections that is specific to each <em>client</em> of the
+ free-space manager.
+
+ For example, the fractal heap is a client of the free space manager
+ and uses it to track unused space within the heap. There are 4
+ types of section records for the fractal heap, each of which has
+ its own format, listed below.
+</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Free-space Manager Header
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Client ID</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Total Space Tracked<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Total Number of Sections<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Number of Serialized Sections<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Number of Un-Serialized Sections<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Number of Section Classes</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Shrink Percent</td>
+ <td colspan="2">Expand Percent</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Size of Address Space</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Maximum Section Size <sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of Serialized Section List<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Size of Serialized Section List Used<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Allocated Size of Serialized Section List<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Free-space Manager Header
+ </caption>
+ <tr>
+ <th width="35%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>FSHD</code>&rdquo;
+ is used to indicate the beginning of the Free-space Manager
+ Header. This gives file consistency checking utilities a
+ better chance of reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This is the version number for the Free-space Manager Header
+ and this document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Client ID</p></td>
+ <td>
+ <p>This is the client ID for identifying the user of this
+ free-space manager:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">ID</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Fractal heap
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>File
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2+</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Total Space Tracked</p></td>
+ <td>
+ <p>This is the total amount of free space being tracked, in bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Total Number of Sections</p></td>
+ <td>
+ <p>This is the total number of free-space sections being tracked.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Serialized Sections</p></td>
+ <td>
+ <p>This is the number of serialized free-space sections being
+ tracked.
+ </p>
+ </td>
+ </tr>
+ <tr>
+ <td><p>Number of Un-Serialized Sections</p></td>
+ <td>
+ <p>This is the number of un-serialized free-space sections being
+ managed. Un-serialized sections are created by the free-space
+ client when the list of sections is read in.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Section Classes</p></td>
+ <td>
+ <p>This is the number of section classes handled by this free space
+ manager for the free-space client.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Shrink Percent</p></td>
+ <td>
+ <p>This is the percent of current size to shrink the allocated
+ serialized free-space section list.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Expand Percent</p></td>
+ <td>
+ <p>This is the percent of current size to expand the allocated
+ serialized free-space section list.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Address Space</p></td>
+ <td>
+ <p>This is the size of the address space that free-space sections
+ are within. This is stored as the log<sub>2</sub> of the
+ actual value (in other words, the number of bits required
+ to store values within that address space).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Maximum Section Size</p></td>
+ <td>
+ <p>This is the maximum size of a section to be tracked.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address of Serialized Section List</p></td>
+ <td>
+ <p>This is the address where the serialized free-space section
+ list is stored.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Serialized Section List Used</p></td>
+ <td>
+ <p>This is the size of the serialized free-space section
+ list used (in bytes). This value must be less than
+ or equal to the <em>allocated size of serialized section
+ list</em>, below.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Allocated Size of Serialized Section List</p></td>
+ <td>
+ <p>This is the size of serialized free-space section list
+ actually allocated (in bytes).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the free-space manager header.</p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<p>The free-space sections being managed are stored in a
+ <em>free-space section list</em>, described below. The sections
+ in the free-space section list are stored in the following way:
+ a count of the number of sections describing a particular size of
+ free space and the size of the free-space described (in bytes),
+ followed by a list of section description records; then another
+ section count and size, followed by the list of section
+ descriptions for that size; and so on.</p>
+
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Free-space Section List
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Free-space Manager Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Number of Section Records in Set #0 <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Size of Free-space Section Described in Record Set #0 <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #0 Section Record #0 Offset<em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="1">Record Set #0 Section Record #0 Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #0 Section Record #0 Data <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #0 Section Record #K-1 Offset<em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="1">Record Set #0 Section Record #K-1 Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #0 Section Record #K-1 Data <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Number of Section Records in Set #1 <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Size of Free-space Section Described in Record Set #1 <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #1 Section Record #0 Offset<em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="1">Record Set #1 Section Record #0 Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #1 Section Record #0 Data <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #1 Section Record #K-1 Offset<em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="1">Record Set #1 Section Record #K-1 Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #1 Section Record #K-1 Data <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><strong>...</strong></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><strong>...</strong></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Number of Section Records in Set #N-1 <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Size of Free-space Section Described in Record Set #N-1 <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #N-1 Section Record #0 Offset<em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="1">Record Set #N-1 Section Record #0 Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #N-1 Section Record #0 Data <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #N-1 Section Record #K-1 Offset<em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="1">Record Set #N-1 Section Record #K-1 Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Record Set #N-1 Section Record #K-1 Data <em>(variable size)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Free-space Section List
+ </caption>
+ <tr>
+ <th width="35%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>FSSE</code>&rdquo;
+ is used to indicate the beginning of the Free-space Section
+ Information. This gives file consistency checking utilities
+ a better chance of reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This is the version number for the Free-space Section List
+ and this document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Free-space Manager Header Address</p></td>
+ <td>
+ <p>This is the address of the <em>Free-space Manager Header</em>.
+ This field is principally used for file
+ integrity checking.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Section Records for Set #N</p></td>
+ <td>
+ <p>This is the number of free-space section records for set #N.
+ The length of this field is the minimum number of bytes needed
+ to store the <em>number of serialized sections</em> (from the
+ <em>free-space manager header</em>).
+ </p>
+
+ <p>
+ The number of sets of free-space section records is
+ determined by the <em>size of serialized section list</em> in
+ the <em>free-space manager header</em>.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Section Size for Record Set #N</p></td>
+ <td>
+ <p>This is the size (in bytes) of the free-space section described
+ for <em>all</em> the section records in set #N.
+ </p>
+
+ <p>
+ The length of this field is the minimum number of bytes needed
+ to store the <em>maximum section size</em> (from the
+ <em>free-space manager header</em>).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Record Set #N Section #K Offset</p></td>
+ <td>
+ <p>This is the offset (in bytes) of the free-space section within
+ the client for the free-space manager.
+ </p>
+
+ <p>
+ The length of this field is the minimum number of bytes needed
+ to store the <em>size of address space</em> (from the
+ <em>free-space manager header</em>).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Record Set #N Section #K Type</p></td>
+ <td>
+ <p>This is the type of the section record, used to decode the
+ <em>record set #N section #K data</em> information. The defined
+ record type for <em>file</em> client is:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Type</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>File&rsquo;s section (a range of actual bytes in file)
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1+</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table></p>
+
+ <p>The defined record types for a <em>fractal heap</em> client are:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Type</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Fractal heap &ldquo;single&rdquo; section
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Fractal heap &ldquo;first row&rdquo; section
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Fractal heap &ldquo;normal row&rdquo; section
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Fractal heap &ldquo;indirect&rdquo; section
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>4+</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Record Set #N Section #K Data</p></td>
+ <td>
+ <p>This is the section-type specific information for each record
+ in the record set, described below.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the <em>Free-space Section List</em>.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<p>
+ The section-type specific data for each free-space section record is
+ described below:
+</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: File&rsquo;s Section Data Record
+ </caption>
+
+ <tr>
+ <td colspan="4"><em>No additional record data stored</em></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fractal Heap &ldquo;Single&rdquo; Section Data Record
+ </caption>
+
+ <tr>
+ <td colspan="4"><em>No additional record data stored</em></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fractal Heap &ldquo;First Row&rdquo; Section Data
+ Record
+ </caption>
+
+ <tr>
+ <td colspan="4"><em>Same format as &ldquo;indirect&rdquo;
+ section data</em></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fractal Heap &ldquo;Normal Row&rdquo; Section Data
+ Record
+ </caption>
+
+ <tr>
+ <td colspan="4"><em>No additional record data stored</em></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fractal Heap &ldquo;Indirect&rdquo; Section
+ Data Record
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Fractal Heap Indirect Block Offset <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Block Start Row</td>
+ <td colspan="2">Block Start Column</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Number of Blocks</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fractal Heap &ldquo;Indirect&rdquo; Section
+ Data Record
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Fractal Heap Block Offset</p></td>
+ <td>
+ <p>The offset of the indirect block in the fractal heap&rsquo;s address
+ space containing the empty blocks.
+ </p>
+ <p>
+ The number of bytes used to encode this field is the minimum
+ number of bytes needed to encode values for the <em>Maximum
+ Heap Size</em> (in the fractal heap&rsquo;s header).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Block Start Row</p></td>
+ <td>
+ <p>This is the row that the empty blocks start in.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Block Start Column</p></td>
+ <td>
+ <p>This is the column that the empty blocks start in.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Blocks</p></td>
+ <td>
+ <p>This is the number of empty blocks covered by the section.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<h3><a name="SOHMTable">
+ III.I. Disk Format: Level 1I - Shared Object Header Message Table</a></h3>
+
+<p>
+ The <em>shared object header message table</em> is used to locate
+ object
+ header messages that are shared between two or more object headers
+ in the file. Shared object header messages are stored and indexed
+ in the file in one of two ways: indexed sequentially in a
+ <em>shared header message list</em> or indexed with a v2 B-tree.
+ The shared messages themselves are either stored in a fractal
+ heap (when two or more objects share the message), or remain in an
+ object&rsquo;s header (when only one object uses the message currently,
+ but the message can be shared in the future).
+</p>
+
+<p>
+ The <em>shared object header message table</em>
+ contains a list of shared message index headers. Each index header
+ records information about the version of the index format, the index
+ storage type, flags for the message types indexed, the number of
+ messages in the index, the address where the index resides,
+ and the fractal heap address if shared messages are stored there.
+</p>
+
+<p>
+ Each index can be either a list or a v2 B-tree and may transition
+ between those two forms as the number of messages in the index
+ varies. Each shared message record contains information used to
+ locate the shared message from either a fractal heap or an object
+ header. The types of messages that can be shared are: <em>Dataspace,
+ Datatype, Fill Value, Filter Pipeline and Attribute</em>.
+</p>
+
+<p>
+ The <em>shared object header message table</em> is pointed to
+ from a <a href="#SOHMTableMessage">shared message table</a> message
+ in the superblock extension for a file. This message stores the
+ version of the table format, along with the number of index headers
+ in the table.
+</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Shared Object Header Message Table
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version for index #0</td>
+ <td>Index Type for index #0</td>
+ <td colspan="2">Message Type Flags for index #0</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Minimum Message Size for index #0</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">List Cutoff for index #0</td>
+ <td colspan="2">v2 B-tree Cutoff for index #0</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Number of Messages for index #0</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Index Address<sup>O</sup> for index #0<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Fractal Heap Address<sup>O</sup> for index #0<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td>Version for index #N-1</td>
+ <td>Index Type for index #N-1</td>
+ <td colspan="2">Message Type Flags for index #N-1</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Minimum Message Size for index #N-1</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">List Cutoff for index #N-1</td>
+ <td colspan="2">v2 B-tree Cutoff for index #N-1</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Number of Messages for index #N-1</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Index Address<sup>O</sup> for index #N-1<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Fractal Heap Address<sup>O</sup> for index #N-1<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Shared Object Header Message Table
+ </caption>
+ <tr>
+ <th width="35%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>SMTB</code>&rdquo;
+ is used to indicate the beginning of the Shared Object
+ Header Message table. This gives file consistency checking
+ utilities a better chance of reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version for index #N</p></td>
+ <td>
+ <p>This is the version number for the list of shared object header message
+ indexes and this document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Index Type for index #N</p></td>
+ <td>
+ <p>The type of index can be an unsorted list or a v2 B-tree.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Message Type Flags for index #N</p></td>
+ <td>
+ <p>This field indicates the type of messages tracked in the index,
+ as follows:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bits</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, the index tracks <em>Dataspace Messages</em>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>If set, the message tracks <em>Datatype Messages</em>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>If set, the message tracks <em>Fill Value Messages</em>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>If set, the message tracks <em>Filter Pipeline Messages</em>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>If set, the message tracks <em>Attribute Messages</em>.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>5-15</code></td>
+ <td>Reserved (zero).
+ </td>
+ </tr>
+ </table></p>
+
+
+ <p>
+ An index can track more than one type of message, but each type
+ of message can only by in one index.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Minimum Message Size for index #N</p></td>
+ <td>
+ <p>This is the message size sharing threshold for the index.
+ If the encoded size of the message is less than this value, the
+ message is not shared.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>List Cutoff for index #N</p></td>
+ <td>
+ <p>This is the cutoff value for the indexing of messages to
+ switch from a list to a v2 B-tree. If the number of messages
+ is greater than this value, the index should be a v2 B-tree.
+ </p>
+ </td>
+ </tr>
+ <tr>
+ <td><p>v2 B-tree Cutoff for index #N</p></td>
+ <td>
+ <p>This is the cutoff value for the indexing of messages
+ to switch from a v2 B-tree back to a list. If the number
+ of messages is less than this value, the index should be
+ a list.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Messages for index #N</p></td>
+ <td>
+ <p>The number of shared messages being tracked for the index.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Index Address for index #N</p></td>
+ <td>
+ <p>This field is the address of the list or v2 B-tree where the
+ index nodes reside.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fractal Heap Address for index #N</p></td>
+ <td>
+ <p>This field is the address of the fractal heap if shared messages
+ are stored there.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the table.</p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<p>
+ Shared messages are indexed either with a <em>shared message record
+ list</em>, described below, or using a v2 B-tree (using record type 7).
+ The number of records in the <em>shared message record list</em> is
+ determined in the index&rsquo;s entry in the <em>shared object header message
+ table</em>.
+</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Shared Message Record List
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Shared Message Record #0</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Shared Message Record #1</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Shared Message Record #N-1</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Shared Message Record List
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>SMLI</code>&rdquo;
+ is used to indicate the beginning of a list of index nodes.
+ This gives file consistency checking utilities a better
+ chance of reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Shared Message Record #N</p></td>
+ <td>
+ <p>The record for locating the shared message, either in the
+ fractal heap for the index, or an object header (see format for
+ <em>index nodes</em> below).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the list.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<p>
+ The record for each shared message in an index is stored in one
+ of the following forms:
+</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Shared Message Record for Messages Stored in a
+ Fractal Heap
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Message Location</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Hash Value</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reference Count</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Fractal Heap ID<br /><br /></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Shared Message Record for Messages Stored in a
+ Fractal Heap
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Message Location</p></td>
+ <td>
+ <p>This has a value of 0 indicating that the message is stored in
+ the heap.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Hash Value</p></td>
+ <td>
+ <p>This is the hash value for the message.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Reference Count</p></td>
+ <td>
+ <p>This is the number of times the message is used in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fractal Heap ID</p></td>
+ <td>
+ <p>This is an 8-byte fractal heap ID for the message as stored in
+ the fractal heap for the index.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Shared Message Record for Messages Stored in an
+ Object Header
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Message Location</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Hash Value</td>
+ </tr>
+
+ <tr>
+ <td>Reserved</td>
+ <td>Message Type</td>
+ <td colspan="2">Creation Index</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Object Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Shared Message Record for Messages Stored in an
+ Object Header
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Message Location</p></td>
+ <td>
+ <p>This has a value of 1 indicating that the message is stored in
+ an object header.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Hash Value</p></td>
+ <td>
+ <p>This is the hash value for the message.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Message Type</p></td>
+ <td>
+ <p>This is the message type in the object header.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Creation Index</p></td>
+ <td>
+ <p>This is the creation index of the message within the object
+ header.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Header Address</p></td>
+ <td>
+ <p>This is the address of the object header where the message is
+ located.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<h2><a name="DataObject">
+ IV. Disk Format: Level 2 - Data Objects </a></h2>
+
+<p>Data objects contain the &ldquo;real&rdquo; user-visible information in the file.
+ These objects compose the scientific data and other information which
+ are generally thought of as &ldquo;data&rdquo; by the end-user. All the
+ other information in the file is provided as a framework for
+ storing and accessing these data objects.
+</p>
+
+<p>A data object is composed of header and data
+ information. The header information contains the information
+ needed to interpret the data information for the object as
+ well as additional &ldquo;metadata&rdquo; or pointers to additional
+ &ldquo;metadata&rdquo; used to describe or annotate each object.
+</p>
+
+<h3><a name="ObjectHeader">
+ IV.A. Disk Format: Level 2A - Data Object Headers</a></h3>
+
+<p>The header information of an object is designed to encompass
+ all of the information about an object, except for the data itself.
+ This information includes the dataspace, the datatype, information
+ about how the data is stored on disk (in external files, compressed,
+ broken up in blocks, and so on), as well as other information used
+ by the library to speed up access to the data objects or maintain
+ a file&rsquo;s integrity. Information stored by user applications
+ as attributes is also stored in the object&rsquo;s header. The header
+ of each object is not necessarily located immediately prior to the
+ object&rsquo;s data in the file and in fact may be located in any
+ position in the file. The order of the messages in an object header
+ is not significant.</p>
+
+<p>Object headers are composed of a prefix and a set of messages. The
+ prefix contains the information needed to interpret the messages and
+ a small amount of metadata about the object, and the messages contain
+ the majority of the metadata about the object.
+</p>
+
+<h3><a name="ObjectHeaderPrefix">
+ IV.A.1. Disk Format: Level 2A1 - Data Object Header Prefix</a></h3>
+
+
+
+<h4><a name="V1ObjectHeaderPrefix">
+ IV.A.1.a. Version 1 Data Object Header Prefix</a></h4>
+
+<p>Header messages are aligned on 8-byte boundaries for version 1
+ object headers.
+</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 1 Object Header
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Reserved (zero)</td>
+ <td colspan="2">Total Number of Header Messages</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Object Reference Count</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Object Header Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Header Message Type #1</td>
+ <td colspan="2">Size of Header Message Data #1</td>
+ </tr>
+
+ <tr>
+ <td>Header Message #1 Flags</td>
+ <td colspan="3">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Header Message Data #1<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Header Message Type #n</td>
+ <td colspan="2">Size of Header Message Data #n</td>
+ </tr>
+
+ <tr>
+ <td>Header Message #n Flags</td>
+ <td colspan="3">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Header Message Data #n<br /><br /></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 1 Object Header
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This value is used to determine the format of the
+ information in the object header. When the format of the
+ object header is changed, the version number
+ is incremented and can be used to determine how the
+ information in the object header is formatted. This
+ is version one (1) (there was no version zero (0)) of the
+ object header.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Total Number of Header Messages</p></td>
+ <td>
+ <p>This value determines the total number of messages listed in
+ object headers for this object. This value includes the messages
+ in continuation messages for this object.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Reference Count</p></td>
+ <td>
+ <p>This value specifies the number of &ldquo;hard links&rdquo; to this object
+ within the current file. References to the object from external
+ files, &ldquo;soft links&rdquo; in this file and object references in this
+ file are not tracked.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Object Header Size</p></td>
+ <td>
+ <p>This value specifies the number of bytes of header message data
+ following this length field that contain object header messages
+ for this object header. This value does not include the size of
+ object header continuation blocks for this object elsewhere in the
+ file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Type</p></td>
+ <td>
+ <p>This value specifies the type of information included in the
+ following header message data. The message types for
+ header messages are defined in sections below.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Header Message #n Data</p></td>
+ <td>
+ <p>This value specifies the number of bytes of header
+ message data following the header message type and length
+ information for the current message. The size includes
+ padding bytes to make the message a multiple of eight
+ bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Flags</p></td>
+ <td>
+ <p>This is a bit field with the following definition:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, the message data is constant. This is used
+ for messages like the datatype message of a dataset.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>If set, the message is <em>shared</em> and stored
+ in another location than the object header. The Header
+ Message Data field contains a Shared Message
+ (described in the <a href="#ObjectHeaderMessages">Data Object Header Messages</a>
+ section below)
+ and the Size of Header Message Data field
+ contains the size of that Shared Message.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>If set, the message should not be shared.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>If set, the HDF5 decoder should fail to open this object
+ if it does not understand the message&rsquo;s type and the file
+ is open with permissions allowing write access to the file.
+ (Normally, unknown messages can just be ignored by HDF5
+ decoders)
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>If set, the HDF5 decoder should set bit 5 of this
+ message&rsquo;s flags (in other words, this bit field)
+ if it does not understand the message&rsquo;s type
+ and the object is modified in any way. (Normally,
+ unknown messages can just be ignored by HDF5
+ decoders)
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>5</code></td>
+ <td>If set, this object was modified by software that did not
+ understand this message.
+ (Normally, unknown messages should just be ignored by HDF5
+ decoders) (Can be used to invalidate an index or a similar
+ feature)
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>6</code></td>
+ <td>If set, this message is shareable.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>7</code></td>
+ <td>If set, the HDF5 decoder should always fail to open this
+ object if it does not understand the message&rsquo;s type (whether
+ it is open for read-only or read-write access). (Normally,
+ unknown messages can just be ignored by HDF5 decoders)
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Data</p></td>
+ <td>
+ <p>The format and length of this field is determined by the
+ header message type and size respectively. Some header
+ message types do not require any data and this information
+ can be eliminated by setting the length of the message to
+ zero. The data is padded with enough zeroes to make the
+ size a multiple of eight.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<h4><a name="V2ObjectHeaderPrefix">
+ IV.A.1.b. Version 2 Data Object Header Prefix</a></h4>
+
+<p>Note that the &ldquo;total number of messages&rdquo; field has been dropped from
+ the data object header prefix in this version. The number of messages
+ in the data object header is just determined by the messages encountered
+ in all the object header blocks.</p>
+
+<p>Note also that the fields and messages in this version of data object
+ headers have <em>no</em> alignment or padding bytes inserted - they are
+ stored packed together.</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 Object Header
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Access time <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Modification Time <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Change Time <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Birth Time <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Maximum # of compact attributes <em>(optional)</em></td>
+ <td colspan="2">Minimum # of dense attributes <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td>Size of Chunk #0 <em>(variable size)</em></td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td>Header Message Type #1</td>
+ <td colspan="2">Size of Header Message Data #1</td>
+ <td>Header Message #1 Flags</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Header Message #1 Creation Order <em>(optional)</em></td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Header Message Data #1<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+
+ <tr>
+ <td>Header Message Type #n</td>
+ <td colspan="2">Size of Header Message Data #n</td>
+ <td>Header Message #n Flags</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Header Message #n Creation Order <em>(optional)</em></td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Header Message Data #n<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Gap <em>(optional, variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 Object Header
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>OHDR</code>&rdquo;
+ is used to indicate the beginning of an object header. This
+ gives file consistency checking utilities a better chance
+ of reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This field has a value of 2 indicating version 2 of the object header.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td>
+ <p>This field is a bit field indicating additional information
+ about the object header.
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit(s)</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0-1</code></td>
+ <td>This two bit field determines the size of the
+ <em>Size of Chunk #0</em> field. The values are:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>The <em>Size of Chunk #0</em> field is 1 byte.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>The <em>Size of Chunk #0</em> field is 2 bytes.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>The <em>Size of Chunk #0</em> field is 4 bytes.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>The <em>Size of Chunk #0</em> field is 8 bytes.
+ </td>
+ </tr>
+ </table>
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>If set, attribute creation order is tracked.</td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>If set, attribute creation order is indexed.</td>
+ </tr>
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>If set, non-default attribute storage phase change
+ values are stored.</td>
+ </tr>
+ <tr>
+ <td align="center"><code>5</code></td>
+ <td>If set, access, modification, change and birth times
+ are stored.</td>
+ </tr>
+ <tr>
+ <td align="center"><code>6-7</code></td>
+ <td>Reserved</td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Access Time</p></td>
+ <td>
+ <p>This 32-bit value represents the number of seconds after the
+ UNIX epoch when the object&rsquo;s raw data was last accessed
+ (in other words, read or written).
+ </p>
+ <p>This field is present if bit 5 of <em>flags</em> is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Modification Time</p></td>
+ <td>
+ <p>This 32-bit value represents the number of seconds after
+ the UNIX epoch when the object&rsquo;s raw data was last
+ modified (in other words, written).
+ </p>
+ <p>This field is present if bit 5 of <em>flags</em> is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Change Time</p></td>
+ <td>
+ <p>This 32-bit value represents the number of seconds after the
+ UNIX epoch when the object&rsquo;s metadata was last changed.
+ </p>
+ <p>This field is present if bit 5 of <em>flags</em> is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Birth Time</p></td>
+ <td>
+ <p>This 32-bit value represents the number of seconds after the
+ UNIX epoch when the object was created.
+ </p>
+ <p>This field is present if bit 5 of <em>flags</em> is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Maximum # of compact attributes</p></td>
+ <td>
+ <p>This is the maximum number of attributes to store in the compact
+ format before switching to the indexed format.
+ </p>
+ <p>This field is present if bit 4 of <em>flags</em> is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Minimum # of dense attributes</p></td>
+ <td>
+ <p>This is the minimum number of attributes to store in the indexed
+ format before switching to the compact format.
+ </p>
+ <p>This field is present if bit 4 of <em>flags</em> is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Chunk #0</p></td>
+ <td>
+ <p>
+ This unsigned value specifies the number of bytes of header
+ message data following this field that contain object header
+ information.
+ </p>
+ <p>
+ This value does not include the size of object header
+ continuation blocks for this object elsewhere in the file.
+ </p>
+ <p>
+ The length of this field varies depending on bits 0 and 1 of
+ the <em>flags</em> field.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Type</p></td>
+ <td>
+ <p>Same format as version 1 of the object header, described above.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Header Message #n Data</p></td>
+ <td>
+ <p>This value specifies the number of bytes of header
+ message data following the header message type and length
+ information for the current message. The size of messages
+ in this version does <em>not</em> include any padding bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Flags</p></td>
+ <td>
+ <p>Same format as version 1 of the object header, described above.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Creation Order</p></td>
+ <td>
+ <p>This field stores the order that a message of a given type
+ was created in.
+ </p>
+ <p>This field is present if bit 2 of <em>flags</em> is set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Data</p></td>
+ <td>
+ <p>Same format as version 1 of the object header, described above.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Gap</p></td>
+ <td>
+ <p>A gap in an object header chunk is inferred by the end of the
+ messages for the chunk before the beginning of the chunk&rsquo;s
+ checksum. Gaps are always smaller than the size of an
+ object header message prefix (message type + message size +
+ message flags).
+ </p>
+ <p>Gaps are formed when a message (typically an attribute message)
+ in an earlier chunk is deleted and a message from a later
+ chunk that does not quite fit into the free space is moved
+ into the earlier chunk.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the object header chunk.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<p>The header message types and the message data associated with
+ them compose the critical &ldquo;metadata&rdquo; about each object. Some
+ header messages are required for each object while others are
+ optional. Some optional header messages may also be repeated
+ several times in the header itself, the requirements and number
+ of times allowed in the header will be noted in each header
+ message description below.
+</p>
+
+
+<h3><a name="ObjectHeaderMessages">
+ IV.A.2. Disk Format: Level 2A2 - Data Object Header Messages</a></h3>
+
+<p>Data object header messages are small pieces of metadata that are
+ stored in the data object header for each object in an HDF5 file.
+ Data object header messages provide the metadata required to describe
+ an object and its contents, as well as optional pieces of metadata
+ that annotate the meaning or purpose of the object.
+</p>
+
+<p>Data object header messages are either stored directly in the data
+ object header for the object or are shared between multiple objects
+ in the file. When a message is shared, a flag in the <em>Message Flags</em>
+ indicates that the actual <em>Message Data</em>
+ portion of that message is stored in another location (such as another
+ data object header, or a heap in the file) and the <em>Message Data</em>
+ field contains the information needed to locate the actual information
+ for the message.
+</p>
+
+<p>
+ The format of shared message data is described here:</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Shared Message (Version 1)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Type</td>
+ <td colspan="2">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Shared Message (Version 1)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number is used when there are changes in the format
+ of a shared object message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Never used.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Used by the library before version 1.6.1.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Type</p></td>
+ <td><p>The type of shared message location:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Message stored in another object&rsquo;s header (a <em>committed</em>
+ message).
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>The address of the object header
+ containing the message to be shared.</p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+
+ <caption>
+ Layout: Shared Message (Version 2)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Type</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Shared Message (Version 2)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number is used when there are changes in the format
+ of a shared object message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Used by the library of version 1.6.1 and after.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Type</p></td>
+ <td><p>The type of shared message location:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Message stored in another object&rsquo;s header (a <em>committed</em>
+ message).
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>The address of the object header
+ containing the message to be shared.</p></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Shared Message (Version 3)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Type</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Location <em>(variable size)</em></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Shared Message (Version 3)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number indicates changes in the format of shared
+ object message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Used by the library of version 1.8 and after. In this
+ version, the <em>Type</em> field can indicate that
+ the message is stored in the fractal heap.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Type</p></td>
+ <td><p>The type of shared message location:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Message is not shared and is not shareable.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Message stored in file&rsquo;s <em>shared object header message</em>
+ heap (a <em>shared</em> message).
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Message stored in another object&rsquo;s header (a <em>committed</em>
+ message).
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Message stored is not shared, but is sharable.
+ </td>
+ </tr>
+
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Location</p></td>
+ <td><p>This field contains either a <a href="#SizeOfOffsetsV0">
+ <em>Size of Offsets</em></a>-bytes address of the object header
+ containing the message to be shared, or an 8-byte fractal heap
+ ID for the message in the file&rsquo;s <em>shared object header
+ message</em> heap.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+
+<p>The following is a list of currently defined header messages:
+</p>
+
+<h4><a name="NILMessage">IV.A.2.a. The NIL Message</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> NIL</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0000</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may be repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>The NIL message is used to indicate a message which is to be
+ ignored when reading the header messages for a data object.
+ [Possibly one which has been deleted for some reason.]
+ </td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> Unspecified</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+
+<h4><a name="DataspaceMessage">IV.A.2.b. The Dataspace Message</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Dataspace</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0001</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies according to the number of
+ dimensions, as described in the following table.</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Required for dataset objects;
+ may not be repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>The dataspace message describes the number of dimensions (in
+ other words, &ldquo;rank&rdquo;) and size of each dimension that
+ the data object has. This message is only used for datasets which
+ have a simple, rectilinear, array-like layout; datasets requiring
+ a more complex layout are not yet supported.
+ </td>
+ </tr>
+
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Dataspace Message - Version 1
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Dimensionality</td>
+ <td>Flags</td>
+ <td>Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reserved</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Dimension #1 Size<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Dimension #n Size<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Dimension #1 Maximum Size<sup>L</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Dimension #n Maximum Size<sup>L</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Permutation Index #1<sup>L</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Permutation Index #n<sup>L</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Dataspace Message - Version 1
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This value is used to determine the format of the
+ Dataspace Message. When the format of the
+ information in the message is changed, the version number
+ is incremented and can be used to determine how the
+ information in the object header is formatted. This
+ document describes version one (1) (there was no version
+ zero (0)).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimensionality</p></td>
+ <td>
+ <p>This value is the number of dimensions that the data
+ object has.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td>
+ <p>This field is used to store flags to indicate the
+ presence of parts of this message. Bit 0 (the least
+ significant bit) is used to indicate that maximum
+ dimensions are present. Bit 1 is used to indicate that
+ permutation indices are present.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Size</p></td>
+ <td>
+ <p>This value is the current size of the dimension of the
+ data as stored in the file. The first dimension stored in
+ the list of dimensions is the slowest changing dimension
+ and the last dimension stored is the fastest changing
+ dimension.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Maximum Size</p></td>
+ <td>
+ <p>This value is the maximum size of the dimension of the
+ data as stored in the file. This value may be the special
+ &ldquo;<a href="#UnlimitedDim">unlimited</a>&rdquo; size which indicates
+ that the data may expand along this dimension indefinitely.
+ If these values are not stored, the maximum size of each
+ dimension is assumed to be the dimension&rsquo;s current size.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Permutation Index #n</p></td>
+ <td>
+ <p>This value is the index permutation used to map
+ each dimension from the canonical representation to an
+ alternate axis for each dimension. If these values are
+ not stored, the first dimension stored in the list of
+ dimensions is the slowest changing dimension and the last
+ dimension stored is the fastest changing dimension.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+
+
+<br />
+<p>Version 2 of the dataspace message dropped the optional
+ permutation index value support, as it was never implemented in the
+ HDF5 Library:</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Dataspace Message - Version 2
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Dimensionality</td>
+ <td>Flags</td>
+ <td>Type</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Dimension #1 Size<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Dimension #n Size<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Dimension #1 Maximum Size<sup>L</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Dimension #n Maximum Size<sup>L</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Dataspace Message - Version 2
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This value is used to determine the format of the
+ Dataspace Message. This field should be &lsquo;2&rsquo; for version 2
+ format messages.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimensionality</p></td>
+ <td>
+ <p>This value is the number of dimensions that the data object has.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td>
+ <p>This field is used to store flags to indicate the
+ presence of parts of this message. Bit 0 (the least
+ significant bit) is used to indicate that maximum
+ dimensions are present.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Type</p></td>
+ <td>
+ <p>This field indicates the type of the dataspace:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>A <em>scalar</em> dataspace; in other words,
+ a dataspace with a single, dimensionless element.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>A <em>simple</em> dataspace; in other words,
+ a dataspace with a rank greater than 0 and an
+ appropriate number of dimensions.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>A <em>null</em> dataspace; in other words,
+ a dataspace with no elements.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Size</p></td>
+ <td>
+ <p>This value is the current size of the dimension of the
+ data as stored in the file. The first dimension stored in
+ the list of dimensions is the slowest changing dimension
+ and the last dimension stored is the fastest changing
+ dimension.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Maximum Size</p></td>
+ <td>
+ <p>This value is the maximum size of the dimension of the
+ data as stored in the file. This value may be the special
+ &ldquo;<a href="#UnlimitedDim">unlimited</a>&rdquo; size which indicates
+ that the data may expand along this dimension indefinitely.
+ If these values are not stored, the maximum size of each
+ dimension is assumed to be the dimension&rsquo;s current size.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+
+
+<!--
+ <h4><a name="DataSpaceMessage">Header Message Name: Complex Dataspace (Fiber Bundle?)</a></h4>
+
+ <!-- start msgdesc table --
+ <center>
+ <table class="msgdesc">
+ <p><b>Header Message Name: ???????</b></td></tr>
+<b>Header Message Type: </b>0x0002<br />
+<b>Length:</b> Varies</td></tr>
+
+<b>Status:</b> One of the <em>Simple Dataspace</em> or
+<em>Complex Dataspace</em> messages is required (but not both) and may
+not be repeated.<br /> <b>Description:</b> The
+<em>Dataspace</em> message describes space that the dataset is
+mapped onto in a more comprehensive way than the <em>Simple
+ Dimensionality</em> message is capable of handling. The
+dataspace of a dataset encompasses the type of coordinate system
+used to locate the dataset&rsquo;s elements as well as the structure and
+regularity of the coordinate system. The dataspace also
+describes the number of dimensions which the dataset inhabits as
+well as a possible higher dimensional space in which the dataset
+is located within.
+
+<br />
+<p><b>Format of Data:</b></p>
+
+<center>
+ <table border cellpadding="4" width="80%">
+ <caption align="bottom">
+ <b>HDF5 Dataspace Message Layout</b>
+ </caption>
+
+ <tr align="center">
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align="center">
+ <td colspan="4">Mesh Type</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Logical Dimensionality</td>
+ </tr>
+ </table>
+</center>
+
+<br />
+<dl>
+ <dt>The elements of the dimensionality message are described below:
+ <dd>
+ <dl>
+ <dt>Mesh Type: (unsigned 32-bit integer)
+ <dd>This value indicates whether the grid is
+ polar/spherical/cartesion,
+ structured/unstructured and regular/irregular. <br />
+ The mesh type value is broken up as follows: <br />
+
+ <br />
+ <center>
+ <table border cellpadding="4" width="80%">
+ <caption align="bottom">
+ <b>HDF5 Mesh-type Layout</b>
+ </caption>
+
+ <tr align="center">
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align="center">
+ <td colspan="1">Mesh Embedding</td>
+ <td colspan="1">Coordinate System</td>
+ <td colspan="1">Structure</td>
+ <td colspan="1">Regularity</td>
+ </tr>
+ </table>
+ </center>
+ The following are the definitions of mesh-type bytes:
+ <dl>
+ <dt>Mesh Embedding
+ <dd>This value indicates whether the dataset dataspace
+ is located within
+ another dataspace or not:
+ <dl> <dl>
+ <dt>&lt;STANDALONE&gt;
+ <dd>The dataset mesh is self-contained and is not
+ embedded in another mesh.
+ <dt>&lt;EMBEDDED&gt;
+ <dd>The dataset&rsquo;s dataspace is located within
+ another dataspace, as
+ described in information below.
+ </dl> </dl>
+ <dt>Coordinate System
+ <dd>This value defines the type of coordinate system
+ used for the mesh:
+ <dl> <dl>
+ <dt>&lt;POLAR&gt;
+ <dd>The last two dimensions are in polar
+ coordinates, higher dimensions are
+ cartesian.
+ <dt>&lt;SPHERICAL&gt;
+ <dd>The last three dimensions are in spherical
+ coordinates, higher dimensions
+ are cartesian.
+ <dt>&lt;CARTESIAN&gt;
+ <dd>All dimensions are in cartesian coordinates.
+ </dl> </dl>
+ <dt>Structure
+ <dd>This value defines the locations of the grid-points
+ on the axes:
+ <dl> <dl>
+ <dt>&lt;STRUCTURED&gt;
+ <dd>All grid-points are on integral, sequential
+ locations, starting from 0.
+ <dt>&lt;UNSTRUCTURED&gt;
+ <dd>Grid-points locations in each dimension are
+ explicitly defined and
+ may be of any numeric datatype.
+ </dl> </dl>
+ <dt>Regularity
+ <dd>This value defines the locations of the dataset
+ points on the grid:
+ <dl> <dl>
+ <dt>&lt;REGULAR&gt;
+ <dd>All dataset elements are located at the
+ grid-points defined.
+ <dt>&lt;IRREGULAR&gt;
+ <dd>Each dataset element has a particular
+ grid-location defined.
+ </dl> </dl>
+ </dl>
+ <p>The following grid combinations are currently allowed:</p>
+ <dl> <dl>
+ <dt>&lt;POLAR-STRUCTURED-REGULAR&gt;
+ <dt>&lt;SPHERICAL-STRUCTURED-REGULAR&gt;
+ <dt>&lt;CARTESIAN-STRUCTURED-REGULAR&gt;
+ <dt>&lt;POLAR-UNSTRUCTURED-REGULAR&gt;
+ <dt>&lt;SPHERICAL-UNSTRUCTURED-REGULAR&gt;
+ <dt>&lt;CARTESIAN-UNSTRUCTURED-REGULAR&gt;
+ <dt>&lt;CARTESIAN-UNSTRUCTURED-IRREGULAR&gt;
+ </dl> </dl>
+ All of the above grid types can be embedded within another
+ dataspace.
+ <br /> <br />
+ <dt>Logical Dimensionality: (unsigned 32-bit integer)
+ <dd>This value is the number of dimensions that the dataset occupies.
+
+ <br />
+ <center>
+ <table border cellpadding="4" width="80%">
+ <caption align="bottom">
+ <b>HDF5 Dataspace Embedded Dimensionality Information</b>
+ </caption>
+
+ <tr align="center">
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align="center">
+ <td colspan="4">Embedded Dimensionality</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Embedded Dimension Size #1</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Embedded Dimension Size #n</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Embedded Origin Location #1</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Embedded Origin Location #n</td>
+ </tr>
+ </table>
+ </center>
+
+ <dt>Embedded Dimensionality: (unsigned 32-bit integer)
+ <dd>This value is the number of dimensions of the space the
+ dataset is located within: in other words, a planar dataset
+ located within a 3-D space, a 3-D dataset
+ which is a subset of another 3-D space, and so on.
+ <dt>Embedded Dimension Size: (unsigned 32-bit integer)
+ <dd>These values are the sizes of the dimensions of the
+ embedded dataspace
+ that the dataset is located within.
+ <dt>Embedded Origin Location: (unsigned 32-bit integer)
+ <dd>These values comprise the location of the dataset&rsquo;s
+ origin within the embedded dataspace.
+ </dl>
+</dl>
+[Comment: need some way to handle different orientations of the
+dataset dataspace
+within the embedded dataspace]<br />
+
+<br />
+<center>
+ <table border cellpadding="4" width="80%">
+ <caption align="bottom">
+ <b>HDF5 Dataspace Structured/Regular Grid Information</b>
+ </caption>
+
+ <tr align="center">
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align="center">
+ <td colspan="4">Logical Dimension Size #1</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Logical Dimension Maximum #1</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Logical Dimension Size #n</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Logical Dimension Maximum #n</td>
+ </tr>
+ </table>
+</center>
+
+<br />
+<dl>
+ <dt>The elements of the dimensionality message are described below:
+ <dd>
+ <dl>
+ <dt>Logical Dimension Size #n: (unsigned 32-bit integer)
+ <dd>This value is the current size of the dimension of the
+ data as stored in
+ the file. The first dimension stored in the list of
+ dimensions is the slowest
+ changing dimension and the last dimension stored is the
+ fastest changing
+ dimension.
+ <dt>Logical Dimension Maximum #n: (unsigned 32-bit integer)
+ <dd>This value is the maximum size of the dimension of the
+ data as stored in
+ the file. This value may be the special value
+ &lt;UNLIMITED&gt; which
+ indicates that the data may expand along this dimension
+ indefinitely.
+ </dl>
+</dl>
+<br />
+<center>
+ <table border cellpadding="4" width="80%">
+ <caption align="bottom">
+ <b>HDF5 Dataspace Structured/Irregular Grid Information</b>
+ </caption>
+
+ <tr align="center">
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align="center">
+ <td colspan="4"># of Grid Points in Dimension #1</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr align="center">
+ <td colspan="4"># of Grid Points in Dimension #n</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Datatype of Grid Point Locations</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Location of Grid Points in Dimension #1</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Location of Grid Points in Dimension #n</td>
+ </tr>
+ </table>
+</center>
+
+<br />
+<center>
+ <table border cellpadding="4" width="80%">
+ <caption align="bottom">
+ <b>HDF5 Dataspace Unstructured Grid Information</b>
+ </caption>
+
+ <tr align="center">
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr align="center">
+ <td colspan="4"># of Grid Points</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Datatype of Grid Point Locations</td>
+ </tr>
+ <tr align="center">
+ <td colspan="4">Grid Point Locations<br />.<br />.<br /></td>
+ </tr>
+ </table>
+</center>
+-->
+
+<h4><a name="LinkInfoMessage">IV.A.2.c. The Link Info Message</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Link Info</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x002 </td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies </td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated. </td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>The link info message tracks variable information about the
+ current state of the links for a &ldquo;new style&rdquo;
+ group&rsquo;s behavior. Variable information will be stored in
+ this message and constant information will be stored in the
+ <a href="#GroupInfoMessage">Group Info</a> message.
+ </td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Link Info
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Maximum Creation Index <em>(8 bytes, optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Fractal Heap Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of v2 B-tree for Name Index<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address of v2 B-tree for Creation Order Index<sup>O</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Link Info
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number for this message. This document describes
+ version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>This field determines various optional aspects of the link
+ info message:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, creation order for the links is tracked.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>If set, creation order for the links is indexed.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2-7</code></td>
+ <td>Reserved</td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Maximum Creation Index</p></td>
+ <td><p>This 64-bit value is the maximum creation order index value
+ stored for a link in this group.</p>
+ <p>This field is present if bit 0 of <em>flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fractal Heap Address</p></td>
+ <td>
+ <p>
+ This is the address of the fractal heap to store dense links.
+ Each link stored in the fractal heap is stored as a
+ <a href="#LinkMessage">Link Message</a>.
+ </p>
+ <p>
+ If there are no links in the group, or the group&rsquo;s links
+ are stored &ldquo;compactly&rdquo; (as object header messages), this
+ value will be the <a href="#UndefinedAddress">undefined
+ address</a>.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address of v2 B-tree for Name Index</p></td>
+ <td><p>This is the address of the version 2 B-tree to index names of links.</p>
+ <p>If there are no links in the group, or the group&rsquo;s links
+ are stored &ldquo;compactly&rdquo; (as object header messages), this
+ value will be the <a href="#UndefinedAddress">undefined
+ address</a>.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address of v2 B-tree for Creation Order Index</p></td>
+ <td><p>This is the address of the version 2 B-tree to index creation order of links.</p>
+ <p>If there are no links in the group, or the group&rsquo;s links
+ are stored &ldquo;compactly&rdquo; (as object header messages), this
+ value will be the <a href="#UndefinedAddress">undefined
+ address</a>.
+ </p>
+ <p>This field exists if bit 1 of <em>flags</em> is set.</p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+
+<h4><a name="DatatypeMessage">IV.A.2.d. The Datatype Message</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Datatype</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0003
+ </td></tr>
+ <tr><td colspan="2"><b>Length:</b> Variable</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Required for dataset or committed
+ datatype (formerly named datatype) objects; may not be repeated.
+ </td></tr>
+ <tr><td><b>Description:</b></td>
+ <td><p>The datatype message defines the datatype for each element
+ of a dataset or a common datatype for sharing between multiple
+ datasets. A datatype can describe an atomic type like a fixed-
+ or floating-point type or more complex types like a C struct
+ (compound datatype), array (array datatype), or C++ vector
+ (variable-length datatype).</p>
+ <p>Datatype messages that are part of a dataset object do not
+ describe how elements are related to one another; the dataspace
+ message is used for that purpose. Datatype messages that are part of
+ a committed datatype (formerly named datatype) message describe
+ a common datatype that can be shared by multiple datasets in the
+ file.</p>
+ </td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Datatype Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Class and Version</td>
+ <td>Class Bit Field, Bits 0-7</td>
+ <td>Class Bit Field, Bits 8-15</td>
+ <td>Class Bit Field, Bits 16-23</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br /><br />Properties<br /><br /><br /></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Datatype Message
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Class and Version</p></td>
+ <td>
+ <p>The version of the datatype message and the datatype&rsquo;s class
+ information are packed together in this field. The version
+ number is packed in the top 4 bits of the field and the class
+ is contained in the bottom 4 bits.
+ </p>
+ <p>The version number information is used for changes in the
+ format of the datatype message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Never used
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Used by early versions of the library to encode
+ compound datatypes with explicit array fields.
+ See the compound datatype description below for
+ further details.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Used when an array datatype needs to be encoded.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Used when a VAX byte-ordered type needs to be
+ encoded. Packs various other datatype classes more
+ efficiently also.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>Used to encode the revised reference datatype.
+ </td>
+ </tr>
+ </table></p>
+
+ <p>The class of the datatype determines the format for the class
+ bit field and properties portion of the datatype message, which
+ are described below. The
+ following classes are currently defined:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td><a href="#ClassFixedPoint">Fixed-Point</a></td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td><a href="#ClassFloatingPoint">Floating-Point</a></td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td> <a href="#ClassTime">Time</a></td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td><a href="#ClassString">String</a></td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td><a href="#ClassBitField">Bit field</a></td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>5</code></td>
+ <td><a href="#ClassOpaque">Opaque</a></td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6</code></td>
+ <td><a href="#ClassCompound">Compound</a></td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>7</code></td>
+ <td><a href="#ClassReference">Reference</a></td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>8</code></td>
+ <td><a href="#ClassEnum">Enumerated</a></td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>9</code></td>
+ <td><a href="#ClassVarLen">Variable-Length</a></td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>10</code></td>
+ <td><a href="#ClassArray">Array</a></td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Class Bit Fields</p></td>
+ <td>
+ <p>The information in these bit fields is specific to each datatype
+ class and is described below. All bits not defined for a
+ datatype class are set to zero.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size</p></td>
+ <td>
+ <p>The size of a datatype element in bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Properties</p></td>
+ <td>
+ <p>This variable-sized sequence of bytes encodes information
+ specific to each datatype class and is described for each class
+ below. If there is no property information specified for a
+ datatype class, the size of this field is zero bytes.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+
+<br />
+<br />
+<a name="ClassFixedPoint"></a>
+ <p>Class specific information for the Fixed-point Numbers class
+ (Class 0):</p>
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Bits: Fixed-point Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0</p></td>
+ <td><p><b>Byte Order.</b> If zero, byte order is little-endian;
+ otherwise, byte order is big endian.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>1, 2</p></td>
+ <td><p><b>Padding type.</b> Bit 1 is the lo_pad bit and bit 2
+ is the hi_pad bit. If a datum has unused bits at either
+ end, then the lo_pad or hi_pad bit is copied to those
+ locations.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>3</p></td>
+ <td><p><b>Signed.</b> If this bit is set then the fixed-point
+ number is in 2&rsquo;s complement form.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>4-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fixed-point Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="2">Bit Offset</td>
+ <td colspan="2">Bit Precision</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fixed-point Property Description
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Bit Offset</p></td>
+ <td>
+ <p>The bit offset of the first significant bit of the fixed-point
+ value within the datatype. The bit offset specifies the number
+ of bits &ldquo;to the right of&rdquo; the value (which are set to the
+ lo_pad bit value).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Bit Precision</p></td>
+ <td>
+ <p>The number of bits of precision of the fixed-point value
+ within the datatype. This value, combined with the datatype
+ element&rsquo;s size and the Bit Offset field specifies the number
+ of bits &ldquo;to the left of&rdquo; the value (which are set to the
+ hi_pad bit value).
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+ <br />
+ <br />
+ <a name="ClassFloatingPoint"></a>
+ <p>Class specific information for the Floating-point Numbers class
+ (Class 1):</p>
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Bits: Floating-point Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0, 6</p></td>
+ <td><p><b>Byte Order.</b> These two non-contiguous bits specify the
+ &ldquo;endianness&rdquo; of the bytes in the datatype element.
+ <table class="list">
+ <tr>
+ <th width="10%" align="center">Bit 6</th>
+ <th width="10%" align="center">Bit 0</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td align="center"><code>0</code></td>
+ <td>Byte order is little-endian
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td align="center"><code>1</code></td>
+ <td>Byte order is big-endian
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td align="center"><code>0</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td align="center"><code>1</code></td>
+ <td>Byte order is VAX-endian
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>1, 2, 3</p></td>
+ <td><p><b>Padding type.</b> Bit 1 is the low bits pad type, bit 2
+ is the high bits pad type, and bit 3 is the internal bits
+ pad type. If a datum has unused bits at either end or between
+ the sign bit, exponent, or mantissa, then the value of bit
+ 1, 2, or 3 is copied to those locations.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>4-5</p></td>
+ <td><p><b>Mantissa Normalization.</b> This 2-bit bit field specifies
+ how the most significant bit of the mantissa is managed.
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>No normalization
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>The most significant bit of the mantissa is always set
+ (except for 0.0).
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>The most significant bit of the mantissa is not stored,
+ but is implied to be set.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>7</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+
+ <tr>
+ <td><p>8-15</p></td>
+ <td><p><b>Sign Location.</b> This is the bit position of the sign
+ bit. Bits are numbered with the least significant bit zero.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>16-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Floating-point Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="2">Bit Offset</td>
+ <td colspan="2">Bit Precision</td>
+ </tr>
+
+ <tr>
+ <td>Exponent Location</td>
+ <td>Exponent Size</td>
+ <td>Mantissa Location</td>
+ <td>Mantissa Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Exponent Bias</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Floating-point Property Description
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Bit Offset</p></td>
+ <td>
+ <p>The bit offset of the first significant bit of the floating-point
+ value within the datatype. The bit offset specifies the number
+ of bits &ldquo;to the right of&rdquo; the value.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Bit Precision</p></td>
+ <td>
+ <p>The number of bits of precision of the floating-point value
+ within the datatype.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Exponent Location</p></td>
+ <td>
+ <p>The bit position of the exponent field. Bits are numbered with
+ the least significant bit number zero.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Exponent Size</p></td>
+ <td>
+ <p>The size of the exponent field in bits.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Mantissa Location</p></td>
+ <td>
+ <p>The bit position of the mantissa field. Bits are numbered with
+ the least significant bit number zero.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Mantissa Size</p></td>
+ <td>
+ <p>The size of the mantissa field in bits.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Exponent Bias</p></td>
+ <td>
+ <p>The bias of the exponent field.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+ <br />
+ <br />
+ <a name="ClassTime"></a>
+ <p>Class specific information for the Time class (Class 2):</p>
+
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Bits: Time Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0</p></td>
+ <td><p><b>Byte Order.</b> If zero, byte order is little-endian;
+ otherwise, byte order is big endian.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>1-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Time Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="2">Bit Precision</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Time Property Description
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Bit Precision</p></td>
+ <td>
+ <p>The number of bits of precision of the time value.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+ <br />
+ <a name="ClassString"></a>
+ <p>Class specific information for the Strings class (Class 3):</p>
+
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Bits: String Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0-3</p></td>
+ <td><p><b>Padding type.</b> This four-bit value determines the
+ type of padding to use for the string. The values are:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Null Terminate: A zero byte marks the end of the
+ string and is guaranteed to be present after
+ converting a long string to a short string. When
+ converting a short string to a long string the value is
+ padded with additional null characters as necessary.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Null Pad: Null characters are added to the end of
+ the value during conversions from short values to long
+ values but conversion in the opposite direction simply
+ truncates the value.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Space Pad: Space characters are added to the end of
+ the value during conversions from short values to long
+ values but conversion in the opposite direction simply
+ truncates the value. This is the Fortran
+ representation of the string.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>4-7</p></td>
+ <td><p><b>Character Set.</b> The character set used to
+ encode the string.
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>ASCII character set encoding
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>UTF-8 character set encoding
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>8-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <p>There are no properties defined for the string class.
+ </p>
+
+ <br />
+ <br />
+ <a name="ClassBitField"></a>
+ <p>Class specific information for the Bit Fields class (Class 4):</p>
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Bits: Bitfield Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0</p></td>
+ <td><p><b>Byte Order.</b> If zero, byte order is little-endian;
+ otherwise, byte order is big endian.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>1, 2</p></td>
+ <td><p><b>Padding type.</b> Bit 1 is the lo_pad type and bit 2
+ is the hi_pad type. If a datum has unused bits at either
+ end, then the lo_pad or hi_pad bit is copied to those
+ locations.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>3-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Bit Field Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="2">Bit Offset</td>
+ <td colspan="2">Bit Precision</td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Bit Field Property Description
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Bit Offset</p></td>
+ <td>
+ <p>The bit offset of the first significant bit of the bit field
+ within the datatype. The bit offset specifies the number
+ of bits &ldquo;to the right of&rdquo; the value.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Bit Precision</p></td>
+ <td>
+ <p>The number of bits of precision of the bit field
+ within the datatype.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+ <br />
+ <br />
+ <a name="ClassOpaque"></a>
+ <p>Class specific information for the Opaque class (Class 5):</p>
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Bits: Opaque Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0-7</p></td>
+ <td><p>Length of ASCII tag in bytes.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>8-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Opaque Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />ASCII Tag<br />
+ <br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Opaque Property Description
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>ASCII Tag</p></td>
+ <td>
+ <p>This NUL-terminated string provides a description for the
+ opaque type. It is NUL-padded to a multiple of 8 bytes.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+ <br />
+ <br />
+ <a name="ClassCompound"></a>
+ <p>Class specific information for the Compound class (Class 6):</p>
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Bits: Compound Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0-15</p></td>
+ <td><p><b>Number of Members.</b> This field contains the number
+ of members defined for the compound datatype. The member
+ definitions are listed in the Properties field of the data
+ type message.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>16-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+
+ <p>The Properties field of a compound datatype is a list of the
+ member definitions of the compound datatype. The member
+ definitions appear one after another with no intervening bytes.
+ The member types are described with a (recursively) encoded datatype
+ message.</p>
+
+ <p>Note that the property descriptions are different for different
+ versions of the datatype version. Additionally note that the version
+ 0 datatype encoding is deprecated and has been replaced with later
+ encodings in versions of the HDF5 Library from the 1.4 release
+ onward.</p>
+
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Compound Properties Description for Datatype Version 1
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Byte Offset of Member</td>
+ </tr>
+
+ <tr>
+ <td>Dimensionality</td>
+ <td colspan="3">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension Permutation</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension #1 Size (required)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension #2 Size (required)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension #3 Size (required)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension #4 Size (required)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Member Type Message<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Compound Properties Description for Datatype Version 1
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td>
+ <p>This NUL-terminated string provides a description for the
+ opaque type. It is NUL-padded to a multiple of 8 bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Byte Offset of Member</p></td>
+ <td>
+ <p>This is the byte offset of the member within the datatype.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimensionality</p></td>
+ <td>
+ <p>If set to zero, this field indicates a scalar member. If set
+ to a value greater than zero, this field indicates that the
+ member is an array of values. For array members, the size of
+ the array is indicated by the &lsquo;Size of Dimension n&rsquo; field in
+ this message.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension Permutation</p></td>
+ <td>
+ <p>This field was intended to allow an array field to have
+ its dimensions permuted, but this was never implemented.
+ This field should always be set to zero.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Size</p></td>
+ <td>
+ <p>This field is the size of a dimension of the array field as
+ stored in the file. The first dimension stored in the list of
+ dimensions is the slowest changing dimension and the last
+ dimension stored is the fastest changing dimension.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Member Type Message</p></td>
+ <td>
+ <p>This field is a datatype message describing the datatype of
+ the member.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Compound Properties Description for Datatype Version 2
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Byte Offset of Member</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Member Type Message<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Compound Properties Description for Datatype Version 2
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td>
+ <p>This NUL-terminated string provides a description for the
+ opaque type. It is NUL-padded to a multiple of 8 bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Byte Offset of Member</p></td>
+ <td>
+ <p>This is the byte offset of the member within the datatype.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Member Type Message</p></td>
+ <td>
+ <p>This field is a datatype message describing the datatype of
+ the member.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+ <br />
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Compound Properties Description for Datatype Version 3
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Byte Offset of Member <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Member Type Message<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Compound Properties Description for Datatype Version 3
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td><p>This NUL-terminated string provides a description for the
+ opaque type. It is <em>not</em> NUL-padded to a multiple of 8
+ bytes.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Byte Offset of Member</p></td>
+ <td><p>This is the byte offset of the member within the datatype.
+ The field size is the minimum number of bytes necessary,
+ based on the size of the datatype element. For example, a
+ datatype element size of less than 256 bytes uses a 1 byte
+ length, a datatype element size of 256-65535 bytes uses a
+ 2 byte length, and so on.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Member Type Message</p></td>
+ <td><p>This field is a datatype message describing the datatype of
+ the member.</p></td>
+ </tr>
+
+ </table>
+ </div>
+
+
+ <br />
+ <br />
+ <a name="ClassReference"></a>
+ <p>Class specific information for the Reference class (Class 7):</p>
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Bits: Reference Bit Field Description for Datatype Version &lt; 4
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0-3</p></td>
+ <td><p><b>Type.</b> This four-bit value contains the reference types which are supported for
+ backward compatibility. The values defined are:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Object Reference (H5R_OBJECT1): A reference to another object in this
+ HDF5 file.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Dataset Region Reference (H5R_DATASET_REGION1): A reference to a region within
+ a dataset in this HDF5 file.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>4-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Bits: Reference Bit Field Description for Datatype Version 4
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0-3</p></td>
+ <td><p><b>Type.</b> This four-bit value contains the revised reference types.
+ The values defined are:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Object Reference (H5R_OBJECT2): A reference to another object
+ in this file or an external file.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Dataset Region Reference (H5R_DATASET_REGION2): A reference to a region within
+ a dataset in this file or an external file.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>Attribute Reference (H5R_ATTR): A reference to an attribute attached to an
+ object in this file or an external file.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>5-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>4-7</p></td>
+ <td><p><b>Version.</b> This four-bit value contains the version for encoding
+ the revised reference types. The values defined are:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Unused
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>The version for encoding the revised reference types: Object Reference (2),
+ Dataset Region Reference (3) and Attribute Reference (4).
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>8-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <p>There are no properties defined for the reference class.
+ </p>
+
+
+ <br />
+ <br />
+ <a name="ClassEnum"></a>
+ <p>Class specific information for the Enumeration class (Class 8):</p>
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Bits: Enumeration Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0-15</p></td>
+ <td><p><b>Number of Members.</b> The number of name/value
+ pairs defined for the enumeration type.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>16-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Enumeration Property Description for Datatype Versions
+ 1 and 2
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Base Type<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Names<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Values<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Enumeration Property Description for Datatype Versions
+ 1 and 2
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Base Type</p></td>
+ <td>
+ <p>Each enumeration type is based on some parent type, usually an
+ integer. The information for that parent type is described
+ recursively by this field.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Names</p></td>
+ <td>
+ <p>The name for each name/value pair. Each name is stored as a null
+ terminated ASCII string in a multiple of eight bytes. The names
+ are in no particular order.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Values</p></td>
+ <td>
+ <p>The list of values in the same order as the names. The values
+ are packed (no inter-value padding) and the size of each value
+ is determined by the parent type.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Enumeration Property Description for Datatype Version 3
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Base Type<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Names<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Values<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Enumeration Property Description for Datatype Version 3
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Base Type</p></td>
+ <td>
+ <p>Each enumeration type is based on some parent type, usually an
+ integer. The information for that parent type is described
+ recursively by this field.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Names</p></td>
+ <td>
+ <p>The name for each name/value pair. Each name is stored as a null
+ terminated ASCII string, <em>not</em> padded to a multiple of
+ eight bytes. The names are in no particular order.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Values</p></td>
+ <td>
+ <p>The list of values in the same order as the names. The values
+ are packed (no inter-value padding) and the size of each value
+ is determined by the parent type.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+
+ <br />
+ <a name="ClassVarLen"></a>
+ <p>Class specific information for the Variable-length class (Class 9):</p>
+
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Bits: Variable-length Bit Field Description
+ </caption>
+
+ <tr>
+ <th width="10%">Bits</th>
+ <th>Meaning</th>
+ </tr>
+
+ <tr>
+ <td><p>0-3</p></td>
+ <td><p><b>Type.</b> This four-bit value contains the type of
+ variable-length datatype described. The values defined are:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Sequence: A variable-length sequence of any datatype.
+ Variable-length sequences do not have padding or
+ character set information.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>String: A variable-length sequence of characters.
+ Variable-length strings have padding and character set
+ information.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>4-7</p></td>
+ <td><p><b>Padding type.</b> (variable-length string only)
+ This four-bit value determines the type of padding
+ used for variable-length strings. The values are the same
+ as for the string padding type, as follows:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Null terminate: A zero byte marks the end of a string
+ and is guaranteed to be present after converting a long
+ string to a short string. When converting a short string
+ to a long string, the value is padded with additional null
+ characters as necessary.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Null pad: Null characters are added to the end of the
+ value during conversion from a short string to a longer
+ string. Conversion from a long string to a shorter string
+ simply truncates the value.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Space pad: Space characters are added to the end of the
+ value during conversion from a short string to a longer
+ string. Conversion from a long string to a shorter string
+ simply truncates the value. This is the Fortran
+ representation of the string.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table></p>
+
+ <p>This value is set to zero for variable-length sequences.</p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>8-11</p></td>
+ <td><p><b>Character Set.</b> (variable-length string only)
+ This four-bit value specifies the character set
+ to be used for encoding the string:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>ASCII character set encoding
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>UTF-8 character set encoding
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2-15</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table></p>
+
+ <p>This value is set to zero for variable-length sequences.</p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>12-23</p></td>
+ <td><p>Reserved (zero).</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Variable-length Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Base Type<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Variable-length Property Description
+ </caption>
+ <tr>
+ <th width="10%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Base Type</p></td>
+ <td>
+ <p>Each variable-length type is based on some parent type. The
+ information for that parent type is described recursively by
+ this field.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+ <br />
+ <br />
+ <a name="ClassArray"></a>
+ <p>Class specific information for the Array class (Class 10):</p>
+
+ <p>There are no bit fields defined for the array class.
+ </p>
+
+ <p>Note that the dimension information defined in the property for this
+ datatype class is independent of dataspace information for a dataset.
+ The dimension information here describes the dimensionality of the
+ information within a data element (or a component of an element, if the
+ array datatype is nested within another datatype) and the dataspace for a
+ dataset describes the size and locations of the elements in a dataset.
+ </p>
+
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Array Property Description for Datatype Version 2
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td>Dimensionality</td>
+ <td colspan="3">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension #1 Size</td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Dimension #n Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Permutation Index #1</td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Permutation Index #n</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Base Type<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Array Property Description for Datatype Version 2
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Dimensionality</p></td>
+ <td>
+ <p>This value is the number of dimensions that the array has.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Size</p></td>
+ <td>
+ <p>This value is the size of the dimension of the array
+ as stored in the file. The first dimension stored in
+ the list of dimensions is the slowest changing dimension
+ and the last dimension stored is the fastest changing
+ dimension.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Permutation Index #n</p></td>
+ <td>
+ <p>This value is the index permutation used to map
+ each dimension from the canonical representation to an
+ alternate axis for each dimension. Currently, dimension
+ permutations are not supported, and these indices should
+ be set to the index position minus one. In other words,
+ the first dimension should be set to 0, the second dimension
+ should be set to 1, and so on.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Base Type</p></td>
+ <td>
+ <p>Each array type is based on some parent type. The
+ information for that parent type is described recursively by
+ this field.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Array Property Description for Datatype Version 3
+ </caption>
+
+ <tr>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ <th width="25%">Byte</th>
+ </tr>
+
+ <tr>
+ <td>Dimensionality</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension #1 Size</td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Dimension #n Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Base Type<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Array Property Description for Datatype Version 3
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Dimensionality</p></td>
+ <td>
+ <p>This value is the number of dimensions that the array has.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Size</p></td>
+ <td>
+ <p>This value is the size of the dimension of the array
+ as stored in the file. The first dimension stored in
+ the list of dimensions is the slowest changing dimension
+ and the last dimension stored is the fastest changing
+ dimension.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Base Type</p></td>
+ <td>
+ <p>Each array type is based on some parent type. The
+ information for that parent type is described recursively by
+ this field.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+
+
+ <h4><a name="OldFillValueMessage">IV.A.2.e. The Data Storage -
+ Fill Value (Old) Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Fill Value
+ (old)</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0004</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td><p>The fill value message stores a single data value which
+ is returned to the application when an uninitialized data element
+ is read from a dataset. The fill value is interpreted with the
+ same datatype as the dataset. If no fill value message is present
+ then a fill value of all zero bytes is assumed.</p>
+ <p>This fill value message is deprecated in favor of the
+ &ldquo;new&rdquo; fill value message (Message Type 0x0005) and
+ is only written to the file for forward compatibility with
+ versions of the HDF5 Library before the 1.6.0 version.
+ Additionally, it only appears for datasets with a user-defined
+ fill value (as opposed to the library default fill value or an
+ explicitly set &ldquo;undefined&rdquo; fill value).</p>
+ </td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fill Value Message (Old)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Fill Value <em>(optional, variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fill Value Message (Old)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Size</p></td>
+ <td>
+ <p>This is the size of the Fill Value field in bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fill Value</p></td>
+ <td>
+ <p>The fill value. The bytes of the fill value are interpreted
+ using the same datatype as for the dataset.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+ <h4><a name="FillValueMessage">IV.A.2.f. The Data Storage -
+ Fill Value Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Fill
+ Value</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0005</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Required for dataset objects;
+ may not be repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>The fill value message stores a single data value which is
+ returned to the application when an uninitialized data element
+ is read from a dataset. The fill value is interpreted with the
+ same datatype as the dataset.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fill Value Message - Versions 1 and 2
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Space Allocation Time</td>
+ <td>Fill Value Write Time</td>
+ <td>Fill Value Defined</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Size <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Fill Value <em>(optional, variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fill Value Message - Versions 1 and 2
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number information is used for changes in the
+ format of the fill value message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Never used
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Initial version of this message.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>In this version, the Size and Fill Value fields are
+ only present if the Fill Value Defined field is set
+ to 1.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>This version packs the other fields in the message
+ more efficiently than version 2.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Space Allocation Time</p></td>
+ <td>
+ <p>When the storage space for the dataset&rsquo;s raw data will be
+ allocated. The allowed values are:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Not used.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Early allocation. Storage space for the entire dataset
+ should be allocated in the file when the dataset is
+ created.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Late allocation. Storage space for the entire dataset
+ should not be allocated until the dataset is written
+ to.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Incremental allocation. Storage space for the
+ dataset should not be allocated until the portion
+ of the dataset is written to. This is currently
+ used in conjunction with chunked data storage for
+ datasets.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fill Value Write Time</p></td>
+ <td>
+ <p>At the time that storage space for the dataset&rsquo;s raw data is
+ allocated, this value indicates whether the fill value should
+ be written to the raw data storage elements. The allowed values
+ are:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>On allocation. The fill value is always written to
+ the raw data storage when the storage space is allocated.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Never. The fill value should never be written to
+ the raw data storage.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Fill value written if set by user. The fill value
+ will be written to the raw data storage when the storage
+ space is allocated only if the user explicitly set
+ the fill value. If the fill value is the library
+ default or is undefined, it will not be written to
+ the raw data storage.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fill Value Defined</p></td>
+ <td>
+ <p>This value indicates if a fill value is defined for this
+ dataset. If this value is 0, the fill value is undefined.
+ If this value is 1, a fill value is defined for this dataset.
+ For version 2 or later of the fill value message, this value
+ controls the presence of the Size and Fill Value fields.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size</p></td>
+ <td>
+ <p>This is the size of the Fill Value field in bytes. This field
+ is not present if the Version field is greater than 1,
+ and the Fill Value Defined field is set to 0.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fill Value</p></td>
+ <td>
+ <p>The fill value. The bytes of the fill value are interpreted
+ using the same datatype as for the dataset. This field is
+ not present if the Version field is greater than 1,
+ and the Fill Value Defined field is set to 0.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fill Value Message - Version 3
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Size <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Fill Value <em>(optional, variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fill Value Message - Version 3
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number information is used for changes in the
+ format of the fill value message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Never used
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Initial version of this message.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>In this version, the Size and Fill Value fields are
+ only present if the Fill Value Defined field is set
+ to 1.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>This version packs the other fields in the message
+ more efficiently than version 2.
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td>
+ <p>When the storage space for the dataset&rsquo;s raw data will be
+ allocated. The allowed values are:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bits</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0-1</code></td>
+ <td>Space Allocation Time, with the same
+ values as versions 1 and 2 of the message.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2-3</code></td>
+ <td>Fill Value Write Time, with the same
+ values as versions 1 and 2 of the message.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>Fill Value Undefined, indicating that the fill
+ value has been marked as &ldquo;undefined&rdquo; for this dataset.
+ Bits 4 and 5 cannot both be set.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>5</code></td>
+ <td>Fill Value Defined, with the same values as
+ versions 1 and 2 of the message.
+ Bits 4 and 5 cannot both be set.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>6-7</code></td>
+ <td>Reserved (zero).
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size</p></td>
+ <td>
+ <p>This is the size of the Fill Value field in bytes. This field
+ is not present if the Version field is greater than 1,
+ and the Fill Value Defined flag is set to 0.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fill Value</p></td>
+ <td>
+ <p>The fill value. The bytes of the fill value are interpreted
+ using the same datatype as for the dataset. This field is
+ not present if the Version field is greater than 1,
+ and the Fill Value Defined flag is set to 0.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+ <h4><a name="LinkMessage">IV.A.2.g. The Link Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Link</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0006</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies </td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may be
+ repeated. </td></tr>
+ <tr><td><b>Description:</b></td>
+ <td><p>This message encodes the information for a link in a
+ group&rsquo;s object header, when the group is storing its links
+ &ldquo;compactly&rdquo;, or in the group&rsquo;s fractal heap,
+ when the group is storing its links &ldquo;densely&rdquo;.</p>
+ <p>A group is storing its links compactly when the fractal heap
+ address in the <em><a href="#LinkInfoMessage">Link Info
+ Message</a></em> is set to the &ldquo;undefined address&rdquo;
+ value.</p></td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Link Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td>Link type <em>(optional)</em></td>
+ <td bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Creation Order <em>(8 bytes, optional)</em><br /><br /></td>
+ </tr>
+ <tr>
+ <td>Link Name Character Set <em>(optional)</em></td>
+ <td>Length of Link Name (variable size)</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Link Name (variable size)</td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Link Information (variable size)<br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Link Message
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This document describes version 1.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>This field contains information about the link and controls
+ the presence of other fields below.
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bits</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0-1</code></td>
+ <td>Determines the size of the <em>Length of Link Name</em>
+ field.
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>The size of the <em>Length of Link Name</em>
+ field is 1 byte.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>The size of the <em>Length of Link Name</em>
+ field is 2 bytes.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>The size of the <em>Length of Link Name</em>
+ field is 4 bytes.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>The size of the <em>Length of Link Name</em>
+ field is 8 bytes.
+ </td>
+ </tr>
+ </table>
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Creation Order Field Present: if set, the <em>Creation
+ Order</em> field is present. If not set, creation order
+ information is not stored for links in this group.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Link Type Field Present: if set, the link is not
+ a hard link and the <em>Link Type</em> field is present.
+ If not set, the link is a hard link.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>Link Name Character Set Field Present: if set, the
+ link name is not represented with the ASCII character
+ set and the <em>Link Name Character Set</em> field is
+ present. If not set, the link name is represented with
+ the ASCII character set.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>5-7</code></td>
+ <td>Reserved (zero).
+ </td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Link type</p></td>
+ <td><p>This is the link class type and can be one of the following
+ values:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>A hard link (should never be stored in the file)
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>A soft link.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2-63</code></td>
+ <td>Reserved for future HDF5 internal use.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>64</code></td>
+ <td>An external link.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>65-255</code></td>
+ <td>Reserved, but available for user-defined link types.
+ </td>
+ </tr>
+ </table></p>
+
+ <p>This field is present if bit 3 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Creation Order</p></td>
+ <td><p>This 64-bit value is an index of the link&rsquo;s creation time within
+ the group. Values start at 0 when the group is created an increment
+ by one for each link added to the group. Removing a link from a
+ group does not change existing links&rsquo; creation order field.
+ </p>
+ <p>This field is present if bit 2 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Link Name Character Set</p></td>
+ <td><p>This is the character set for encoding the link&rsquo;s name:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>ASCII character set encoding (this should never be stored
+ in the file)
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>UTF-8 character set encoding
+ </td>
+ </tr>
+ </table></p>
+
+ <p>This field is present if bit 4 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Length of link name</p></td>
+ <td><p>This is the length of the link&rsquo;s name. The size of this field
+ depends on bits 0 and 1 of <em>Flags</em>.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Link name</p></td>
+ <td><p>This is the name of the link, non-NULL terminated.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Link information</p></td>
+ <td><p>The format of this field depends on the <em>link type</em>.</p>
+ <p>For <b>hard</b> links, the field is formatted as follows:
+
+ <table class="list">
+ <tr>
+ <td width="20%"><i><a href="#SizeOfOffsetsV0">
+ Size of Offsets</a></i> bytes:</td>
+ <td width="80%">The address of the object header for the object that the
+ link points to.
+ </td>
+ </tr>
+ </table>
+ </p>
+
+ <p>
+ For <b>soft</b> links, the field is formatted as follows:
+
+ <table class="list">
+ <tr>
+ <td width="20%">Bytes 1-2:</td>
+ <td width="80%">Length of soft link value.</td>
+ </tr>
+ <tr>
+ <td><em>Length of soft link value</em> bytes:</td>
+ <td>A non-NULL-terminated string storing the value of the
+ soft link.
+ </td>
+ </tr>
+ </table>
+ </p>
+
+ <p>
+ For <b>external</b> links, the field is formatted as follows:
+
+ <table class="list">
+ <tr>
+ <td width="20%">Bytes 1-2:</td>
+ <td width="80%">Length of external link value.</td>
+ </tr>
+ <tr>
+ <td><em>Length of external link value</em> bytes:</td>
+ <td>The first byte contains the version number in the
+ upper 4 bits and flags in the lower 4 bits for the external
+ link. Both version and flags are defined to be zero in
+ this document. The remaining bytes consist of two
+ NULL-terminated strings, with no padding between them.
+ The first string is the name of the HDF5 file containing
+ the object linked to and the second string is the full path
+ to the object linked to, within the HDF5 file&rsquo;s
+ group hierarchy.
+ </td>
+ </tr>
+ </table>
+ </p>
+
+ <p>
+ For <b>user-defined</b> links, the field is formatted as follows:
+
+ <table class="list">
+ <tr>
+ <td width="20%">Bytes 1-2:</td>
+ <td width="80%">Length of user-defined data.</td>
+ </tr>
+ <tr>
+ <td><em>Length of user-defined link value</em> bytes:</td>
+ <td>The data supplied for the user-defined link type.</td>
+ </tr>
+ </table>
+ </p>
+
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <h4><a name="ExternalFileListMessage">IV.A.2.h. The Data Storage -
+ External Data Files Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> External
+ Data Files</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0007</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>The external data storage message indicates that the data
+ for an object is stored outside the HDF5 file. The filename of
+ the object is stored as a Universal Resource Location (URL) of
+ the actual filename containing the data. An external file list
+ record also contains the byte offset of the start of the data
+ within the file and the amount of space reserved in the file
+ for that data.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: External File List Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Allocated Slots</td>
+ <td colspan="2">Used Slots</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Heap Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Slot Definitions...<br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: External File List Message
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number information is used for changes in the format of
+ External Data Storage Message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Never used.</td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>The current version used by the library.</td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Allocated Slots</p></td>
+ <td>
+ <p>The total number of slots allocated in the message. Its value must be at least as
+ large as the value contained in the Used Slots field. (The current library simply
+ uses the number of Used Slots for this message)</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Used Slots</p></td>
+ <td>
+ <p>The number of initial slots which contains valid information.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Heap Address</p></td>
+ <td>
+ <p>This is the address of a local heap which contains the names for the external
+ files (The local heap information can be found in Disk Format Level 1D in this
+ document). The name at offset zero in the heap is always the empty string.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Slot Definitions</p></td>
+ <td>
+ <p>The slot definitions are stored in order according to the array addresses they
+ represent.</p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: External File List Slot
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name Offset in Local Heap<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Offset in External Data File<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Data Size in External File<sup>L</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: External File List Slot
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Name Offset in Local Heap</p></td>
+ <td>
+ <p>The byte offset within the local name heap for the name
+ of the file. File names are stored as a URL which has a
+ protocol name, a host name, a port number, and a file
+ name:
+ <code><em>protocol</em>:<em>port</em>//<em>host</em>/<em>file</em></code>.
+ If the protocol is omitted then &ldquo;file:&rdquo; is assumed. If
+ the port number is omitted then a default port for that
+ protocol is used. If both the protocol and the port
+ number are omitted then the colon can also be omitted. If
+ the double slash and host name are omitted then
+ &ldquo;localhost&rdquo; is assumed. The file name is the only
+ mandatory part, and if the leading slash is missing then
+ it is relative to the application&rsquo;s current working
+ directory (the use of relative names is not
+ recommended).
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Offset in External Data File</p></td>
+ <td>
+ <p>This is the byte offset to the start of the data in the
+ specified file. For files that contain data for a single
+ dataset this will usually be zero.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data Size in External File</p></td>
+ <td>
+ <p>This is the total number of bytes reserved in the
+ specified file for raw data storage. For a file that
+ contains exactly one complete dataset which is not
+ extendable, the size will usually be the exact size of the
+ dataset. However, by making the size larger one allows
+ HDF5 to extend the dataset. The size can be set to a value
+ larger than the entire file since HDF5 will read zeroes
+ past the end of the file without failing.</p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+ <h4><a name="LayoutMessage">IV.A.2.i. The Data Layout Message</a></h4>
+
+ <!-- start msgdesc table -->
+ <center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Data Layout</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0008</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Required for datasets; may not
+ be repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>The Data Layout message
+ describes how the elements of a multi-dimensional array are stored
+ in the HDF5 file. Four types of data layout are supported:
+ <ol>
+ <li>Contiguous: The array is stored in one contiguous area of
+ the file. This layout requires that the size of the array be
+ constant: data manipulations such as chunking, compression,
+ checksums, or encryption are not permitted. The message stores
+ the total storage size of the array. The offset of an element
+ from the beginning of the storage area is computed as in a C
+ array.</li>
+ <li>Chunked: The array domain is regularly decomposed into
+ chunks, and each chunk is allocated and stored separately. This
+ layout supports arbitrary element traversals, compression,
+ encryption, and checksums (these features are described
+ in other messages). The message stores the size of a chunk
+ instead of the size of the entire array; the storage size of
+ the entire array can be calculated by traversing the chunk index
+ that stores the chunk addresses.</li>
+ <li>Compact: The array is stored in one contiguous block as
+ part of this object header message.</li>
+ <li>Virtual: This is only supported for version 4 of the Data
+ Layout message. The message stores information that is used to
+ locate the global heap collection containing the Virtual Dataset
+ (VDS) mapping information. The mapping associates the VDS to
+ the source dataset elements that are stored across a collection
+ of HDF5 files.</li>
+ </ol></td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+ </table></center>
+ <!-- end msgdesc table -->
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Data Layout Message (Versions 1 and 2)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Dimensionality</td>
+ <td>Layout Class</td>
+ <td>Reserved <em>(zero)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reserved <em>(zero)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Data Address<sup>O</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension 1 Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension 2 Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension #n Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dataset Element Size <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Compact Data Size <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Compact Data... <em>(variable size, optional)</em><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Data Layout Message (Versions 1 and 2)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number information is used for changes in the format of the data
+ layout message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Never used.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Used by version 1.4 and before of the library to encode layout information.
+ Data space is always allocated when the data set is created.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Used by version 1.6.[0,1,2] of the library to encode layout information.
+ Data space is allocated only when it is necessary.</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimensionality</p></td>
+ <td><p>An array has a fixed dimensionality. This field
+ specifies the number of dimension size fields later in the
+ message. The value stored for chunked storage is 1 greater than
+ the number of dimensions in the dataset&rsquo;s dataspace.
+ For example, 2 is stored for a 1 dimensional dataset.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Layout Class</p></td>
+ <td><p>The layout class specifies the type of storage for the data
+ and how the other fields of the layout message are to be
+ interpreted.
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Compact Storage
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Contiguous Storage
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Chunked Storage
+ </td>
+ </tr>
+ </table>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data Address</p></td>
+ <td><p>For contiguous storage, this is the address of the raw
+ data in the file. For chunked storage this is the address
+ of the <a href="#V1Btrees">v1 B-tree</a> that is used to look up the addresses of the
+ chunks. This field is not present for compact storage.
+ If the version for this message is greater than 1, the address
+ may have the &ldquo;undefined address&rdquo; value, to indicate that
+ storage has not yet been allocated for this array.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Size</p></td>
+ <td><p>For contiguous and compact storage the dimensions define
+ the entire size of the array while for chunked storage they define
+ the size of a single chunk. In all cases, they are in units of
+ array elements (not bytes). The first dimension stored in the list
+ of dimensions is the slowest changing dimension and the last
+ dimension stored is the fastest changing dimension.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dataset Element Size</p></td>
+ <td><p>The size of a dataset element, in bytes. This field is only
+ present for chunked storage.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Compact Data Size</p></td>
+ <td><p>This field is only present for compact data storage.
+ It contains the size of the raw data for the dataset array, in
+ bytes.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Compact Data</p></td>
+ <td><p>This field is only present for compact data storage.
+ It contains the raw data for the dataset array.</p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <p>Version 3 of this message re-structured the format into specific
+ properties that are required for each layout class.</p>
+
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Data Layout Message (Version 3)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Layout Class</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Properties <em>(variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Data Layout Message (Version 3)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The version number information is used for changes in the format of layout message
+ and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Used by the version 1.6.3 and later of the library to store properties
+ for each layout class.</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Layout Class</p></td>
+ <td><p>The layout class specifies the type of storage for the data
+ and how the other fields of the layout message are to be
+ interpreted.
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Compact Storage
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Contiguous Storage
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Chunked Storage
+ </td>
+ </tr>
+ </table>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Properties</p></td>
+ <td><p>This variable-sized field encodes information specific to each
+ layout class and is described below. If there is no property
+ information specified for a layout class, the size of this field
+ is zero bytes.</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <a name="CompactStorage"></a>
+ <p>Class-specific information for compact storage (layout class 0): (Note: The dimensionality information
+ is in the Dataspace message)</p>
+
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Compact Storage Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="2">Size</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Raw Data... <em>(variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Compact Storage Property Description
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Size</p></td>
+ <td><p>This field contains the size of the raw data for the dataset
+ array, in bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Raw Data</p></td>
+ <td><p>This field contains the raw data for the dataset array.</p></td>
+ </tr>
+ </table>
+ </div>
+
+
+ <br />
+ <a name="ContiguousStorage"></a>
+ <p>Class-specific information for contiguous storage (layout class 1):
+ (Note: The dimensionality information is in the Dataspace message)</p>
+
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Contiguous Storage Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Size<sup>L</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Contiguous Storage Property Description
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>This is the address of the raw data in the file.
+ The address may have the &ldquo;undefined address&rdquo; value, to indicate
+ that storage has not yet been allocated for this array.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Size</p></td>
+ <td><p>This field contains the size allocated to store the raw data,
+ in bytes.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+ <br />
+ <p>Class-specific information for chunked storage (layout class 2):</p>
+
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Chunked Storage Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Dimensionality</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension 0 Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension 1 Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension #n Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dataset Element Size</td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Chunked Storage Property Description
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Dimensionality</p></td>
+ <td><p>A chunk has a fixed dimensionality. This field specifies
+ the number of dimension size fields later in the message.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>This is the address of the <a href="#V1Btrees">v1 B-tree</a>
+ that is used to look up the
+ addresses of the chunks that actually store portions of the array
+ data. The address may have the &ldquo;undefined address&rdquo; value, to
+ indicate that storage has not yet been allocated for this array.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Size</p></td>
+ <td><p>These values define the dimension size of a single chunk, in
+ units of array elements (not bytes). The first dimension stored in
+ the list of dimensions is the slowest changing dimension and the
+ last dimension stored is the fastest changing dimension.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dataset Element Size</p></td>
+ <td><p>The size of a dataset element, in bytes.
+ </p>
+ </td>
+ </tr>
+ </table>
+ </div>
+
+
+ <br />
+
+ <p><a name="DataLayoutV4">
+ Version 4</a> of this message is similar to version 3 but has
+ additional information for the virtual layout class as well as
+ indexing information for the chunked layout class.</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Data Layout Message (Version 4)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Layout Class</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted
+ only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Properties <em>(variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Data Layout Message (Version 4)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>The value for this field is 4 and is used by version 1.10.0
+ and later of the library to store properties for each layout
+ class and indexing information for the chunked layout.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Layout Class</p></td>
+ <td><p>The layout class specifies the type of storage for the data
+ and how the other fields of the layout message are to be
+ interpreted.
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Compact Storage
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Contiguous Storage
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Chunked Storage
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Virtual Storage
+ </td>
+ </tr>
+ </table>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Properties</p></td>
+ <td><p>This variable-sized field encodes information specific to a
+ layout class as follows:
+ <table class="list">
+ <tr>
+ <th align="left" width="20%">Layout Class</th>
+ <th align="left" width="80%">Description</th>
+ </tr>
+
+ <tr>
+ <td align="left">Compact Storage</td>
+ <td>See <a href="#CompactStorage">Compact Storage
+ Property Description</i></a> for the version 3
+Data Layout message.
+</td>
+</tr>
+
+<tr>
+ <td align="left">Contiguous Storage</td>
+ <td>See <a href="#ContiguousStorage">Contiguous Storage
+ Property Description</i></a> for the version 3
+Data Layout message.
+</td>
+</tr>
+
+<tr>
+ <td align="left">Chunked Storage</td>
+ <td>See <a href="#ChunkedStorage">Chunked Storage
+ Property Description</i></a> below.
+</td>
+</tr>
+
+<tr>
+ <td align="left">Virtual Storage</td>
+ <td>See <a href="#VirtualStorage">Virtual Storage
+ Property Description</i></a> below.
+</td>
+</tr>
+</table>
+
+</p></td>
+</tr>
+</table>
+</div>
+
+<br />
+<a name="ChunkedStorage"></a>
+<p>Class-specific information for chunked storage (layout
+ class 2):</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Chunked Storage Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Flags</td>
+ <td>Dimensionality</td>
+ <td>Dimension Size Encoded Length</td>
+ <td colspan="1" bgcolor="#DDDDDD"><em>This space inserted to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension 0 Size <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension 1 Size <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">...</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Dimension #n Size <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td>Chunk Indexing Type</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Indexing Type Information <em>(variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup><br /><br /></td>
+ </tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Chunked Storage Property Description
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>This is the chunked layout feature flag:</p>
+
+ <table class="list">
+ <tr>
+ <th width="55%" align="left">Value</th>
+ <th width="45%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="left"><code>DONT_FILTER_PARTIAL_BOUND_CHUNKS (bit 0)</code></td>
+ <td>Do not apply filter to a partial edge chunk.
+
+ </td>
+ </tr>
+
+ <tr>
+ <td align="left"><code>SINGLE_INDEX_WITH_FILTER (bit 1)</code></td>
+ <td>A filtered chunk for <i>Single Chunk</i> indexing.
+ </td>
+ </tr>
+
+ </table>
+
+ </td>
+
+ </tr>
+
+ <tr>
+ <td><p>Dimensionality</p></td>
+ <td><p>A chunk has fixed dimension. This field specifies
+ the number of <em>Dimension Size</em> fields later in the message.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension Size Encoded Length</p></td>
+ <td>
+ <p>This is the size in bytes used to encode <em>Dimension Size</em>.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dimension #n Size</p></td>
+ <td><p>These values define the dimension size of a single chunk, in
+ units of array elements (not bytes). The first dimension stored in
+ the list of dimensions is the slowest changing dimension and the
+ last dimension stored is the fastest changing dimension.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Chunk Indexing Type</p></td>
+ <td><p>There are five indexing types used to look up addresses
+ of the chunks. For more information on each type, see
+ <a href="#AppendixC">&ldquo;Appendix C: Types of Indexes for
+ Dataset Chunks.&rdquo;</a>
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td><a href="#SingleChunk"><i>Single Chunk</i></a> indexing type.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td><a href="#Implicit"><i>Implicit</i></a> indexing type.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td><a href="#FixedArray"><i>Fixed Array</i></a> indexing type.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td><a href="#ExtensibleArray"><i>Extensible Array</i></a> indexing type.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>5</code></td>
+ <td><a href="#V2Btrees"><i>Version 2 B-tree</i></a> indexing type.
+ </td>
+ </tr>
+
+ </table>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Indexing Type Information</p></td>
+ <td><p>This variable-sized field encodes information specific to
+ an indexing type. More information on what is encoded with
+ each type can be found below this table.
+ <ul>
+ <li>See <a href="#IndexInfoSingle"><i>Single Chunk</i></a> below.</li>
+ <li>See <a href="#IndexInfoImplicit"><i>Implicit</i></a> below.</li>
+ <li>See <a href="#IndexInfoFixed"><i>Fixed Array</i></a> below.</li>
+ <li>See <a href="#IndexInfoExtensible"><i>Extensible Array</i></a> below.</li>
+ <li>See <a href="#IndexInfoV2Btrees"><i>Version 2 B-tree</i></a> below.</li>
+ </ul>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>This is the address specific to an indexing type.
+ The address may be undefined if the chunk or index storage is not allocated yet.
+ <table class="list">
+ <tr>
+ <th width="40%" align="left">Value</th>
+ <th width="60%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="left"><i>Single Chunk index</i></td>
+ <td align="left">Address of the single chunk.</td>
+ </td>
+ </tr>
+
+ <tr>
+ <td align="left"><i>Implicit index</i></td>
+ <td align="left">Address of the array of dataset chunks.</td>
+</td>
+</tr>
+
+<tr>
+ <td align="left"><i>Fixed Array index</i></td>
+ <td align="left">Address of the index.</td>
+</tr>
+
+<tr>
+ <td align="left"><i>Extensible Array index</i></td>
+ <td align="left">Address of the index.</td>
+</td>
+</tr>
+
+<tr>
+ <td align="left"><i>Version 2 B-tree index</i></td>
+ <td align="left">Address of the index.</td>
+</td>
+</tr>
+
+</table>
+
+</p>
+</td>
+</tr>
+
+</table>
+</div>
+
+<br />
+
+<ol>
+ <li>
+ <a name="IndexInfoSingle"></a>
+ Index-specific information for <i>Single Chunk</i>:
+ </li>
+
+ <p>The following information exists only when the chunk is filtered.
+ In other words, when <code>DONT_FILTER_PARTIAL_BOUND_CHUNKS</code>
+ (bit 0) is enabled in the field <em>flags</em>.</p>
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Single Chunk Indexing Information
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Size of filtered chunk<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Filters for chunk</td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="55%">&nbsp;</td>
+ <td width="45%"> <!-- width is slightly different: these
+ tables are part of an ordered list; see <ol> tags. -->
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Single Chunk Indexing Information
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Size of filtered chunk</p></td>
+ <td><p>This field is the size of a filtered chunk.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Filters for chunk</p></td>
+ <td><p>This field contains filters for the chunk.</p></td>
+ </tr>
+ </table>
+ </div>
+</p>
+
+<br />
+
+<li>
+ <a name="IndexInfoImplicit"></a>
+ Index-specific information for <i>Implicit</i>:
+</li>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Implicit Indexing Information
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4" bgcolor="#DDDDDD">
+ <em>No specific indexing information</em></td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<li>
+ <a name="IndexInfoFixed"></a>
+ Index-specific information for <i>Fixed Array</i>:
+</li>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fixed Array Indexing Information
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="1">Page Bits</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fixed Array Indexing Information
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Page Bits</p></td>
+ <td><p>This field contains the number of bits needed to store the
+ maximum number of elements in a data block page.</p></td>
+ </tr>
+
+ </table>
+</div>
+</p>
+
+<br />
+<li>
+ <a name="IndexInfoExtensible"></a>
+ Index-specific information for <i>Extensible Array</i>:
+</li>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Extensible Array Indexing Information
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Max Bits</td>
+ <td>Index Elements</td>
+ <td>Min Pointers</td>
+ <td>Min Elements</td>
+ </tr>
+
+ <td colspan="2">Page Bits</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+</tr>
+
+</table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Extensible Array Indexing Information
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Max Bits</p></td>
+ <td><p>This field contains the number of bits needed to store the maximum number of elements
+ in the array.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Index Elements</p></td>
+ <td><p>This field contains the number of elements to store in the
+ index block.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Min Pointers</p></td>
+ <td><p>This field contains the minimum number of data block pointers
+ for a superblock.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Min Elements</p></td>
+ <td><p>This field contains the minimum number of elements per data block.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Page Bits</p></td>
+ <td><p>This field contains the number of bits needed to store the
+ maximum number of elements in a data block page.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+</p>
+<br />
+
+<li>
+ <a name="IndexInfoV2Btrees"></a>
+ Index-specific information for <i>Version 2 B-tree</i>:
+</li>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 B-tree Indexing Information
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Node Size</td>
+ </tr>
+
+ <tr>
+ <td>Split Percent</td>
+ <td>Merge Percent</td>
+ <td colspan="2" bgcolor="#DDDDDD">
+ <em>This space inserted only to align table nicely</em></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 B-tree Indexing Information
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Node Size</p></td>
+ <td><p>This field is the size in bytes of a B-tree node.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Split Percent</p></td>
+ <td><p>This field is the percentage full of a B-tree node at which to split the node.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Merge Percent</p></td>
+ <td><p>This field is the percentage full of a B-tree node at which to merge the node.</p></td>
+ </tr>
+ </table>
+</div>
+</ol>
+
+
+
+<br />
+<a name="VirtualStorage"></a>
+<p>
+ Class-specific information for virtual storage (layout class 3):</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Virtual Storage Property Description
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Index</td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Virtual Storage Property Description
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>This is the address of the global heap collection where
+ the VDS mapping entries are stored.
+ See <a href="#GlobalHeapVDS">&ldquo;Disk Format: Level 1F -
+ Global Heap Block for Virtual Datasets.&rdquo;</a>
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Index</p></td>
+ <td><p>This is the index of the data object within the global heap collection.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<h4><a name="BogusMessage">IV.A.2.j. The Bogus Message</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Bogus</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0009</td></tr>
+ <tr><td colspan="2"><b>Length:</b> 4 bytes</td></tr>
+ <tr><td colspan="2"><b>Status:</b> For testing only; should never
+ be stored in a valid file.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>This message is used for testing the HDF5 Library&rsquo;s
+ response to an &ldquo;unknown&rdquo; message type and should
+ never be encountered in a valid HDF5 file.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Bogus Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Bogus Value</td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Bogus Message
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Bogus Value</p></td>
+ <td>
+ <p>This value should always be: <code>0xdeadbeef</code>.</p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<h4><a name="GroupInfoMessage">IV.A.2.k. The Group Info Message
+</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Group Info</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x000A</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td><p>This message stores information for the constants defining
+ a &ldquo;new style&rdquo; group&rsquo;s behavior. Constant
+ information will be stored in this message and variable
+ information will be stored in the
+ <a href="#LinkInfoMessage">Link Info</a> message.</p>
+ <p>Note: the &ldquo;estimated entry&rdquo; information below is
+ used when determining the size of the object header for the
+ group when it is created.</p></td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Group Info Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td colspan="2">Link Phase Change: Maximum Compact Value <em>(optional)</em></td>
+ </tr>
+ <tr>
+ <td colspan="2">Link Phase Change: Minimum Dense Value <em>(optional)</em></td>
+ <td colspan="2">Estimated Number of Entries <em>(optional)</em></td>
+ </tr>
+ <tr>
+ <td colspan="2">Estimated Link Name Length of Entries <em>(optional)</em></td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Group Info Message
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>This is the group information flag with the following definition:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, link phase change values are stored.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>If set, the estimated entry information is non-default
+ and is stored.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2-7</code></td>
+ <td>Reserved</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Link Phase Change: Maximum Compact Value</p></td>
+ <td><p>The is the maximum number of links to store &ldquo;compactly&rdquo; (in
+ the group&rsquo;s object header).</p>
+ <p>This field is present if bit 0 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Link Phase Change: Minimum Dense Value</p></td>
+ <td><p>This is the minimum number of links to store &ldquo;densely&rdquo; (in
+ the group&rsquo;s fractal heap). The fractal heap&rsquo;s address is
+ located in the <a href="#LinkInfoMessage">Link Info</a>
+ message.</p>
+ <p>This field is present if bit 0 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Estimated Number of Entries</p></td>
+ <td><p>This is the estimated number of entries in groups.</p>
+ <p>If this field is not present, the default value of <code>4</code>
+ will be used for the estimated number of group entries.</p>
+ <p>This field is present if bit 1 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Estimated Link Name Length of Entries</p></td>
+ <td><p>This is the estimated length of entry name.</p>
+ <p>If this field is not present, the default value of <code>8</code>
+ will be used for the estimated link name length of group entries.</p>
+ <p>This field is present if bit 1 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+<!-- </p> -->
+
+<h4><a name="FilterMessage">IV.A.2.l. The Data Storage - Filter
+ Pipeline Message</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b>
+ Data Storage - Filter Pipeline</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x000B</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td><p>This message describes the filter pipeline which should
+ be applied to the data stream by providing filter identification
+ numbers, flags, a name, and client data.</p>
+ <p>This message may be present in the object headers of both
+ dataset and group objects. For datasets, it specifies the
+ filters to apply to raw data. For groups, it specifies the
+ filters to apply to the group&rsquo;s fractal heap. Currently,
+ only datasets using chunked data storage use the filter
+ pipeline on their raw data.</p></td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Filter Pipeline Message - Version 1
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Number of Filters</td>
+ <td colspan="2">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reserved (zero)</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Filter Description List <em>(variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Filter Pipeline Message - Version 1
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This table
+ describes version 1.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Filters</p></td>
+ <td><p>The total number of filters described in this
+ message. The maximum possible number of filters in a
+ message is 32.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Filter Description List</p></td>
+ <td><p>A description of each filter. A filter description
+ appears in the next table.</p></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Filter Description - Version 1
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="2">Filter Identification Value</td>
+ <td colspan="2">Name Length</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Flags</td>
+ <td colspan="2">Number Client Data Values</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name <em>(variable size, optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Client Data <em>(variable size, optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Padding <em>(variable size, optional)</em></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Filter Description - Version 1
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Filter Identification Value</p></td>
+ <td>
+ <p>
+ This value, often referred to as a filter identifier,
+ is designed to be a unique identifier for the filter.
+ Values from zero through 32,767 are reserved for filters
+ supported by The HDF Group in the HDF5 Library and for
+ filters requested and supported by third parties.
+ Filters supported by The HDF Group are documented immediately
+ below. Information on 3rd-party filters can be found at
+ The HDF Group&rsquo;s
+ <a href="http://www.hdfgroup.org/services/contributions.html">
+ Contributions</a> page.</p>
+
+ <p>
+ To request a filter identifier, please contact
+ The HDF Group&rsquo;s Help Desk at
+ <img src="Graphics/help.png" valign="middle" height="14"
+ alt="The HDF Group Help Desk">.
+ You will be asked to provide the following information:</p>
+ <ol>
+ <li>Contact information for the developer requesting the
+ new identifier</li>
+ <li>A short description of the new filter</li>
+ <li>Links to any relevant information, including licensing
+ information</li>
+ </ol>
+ <p>
+ Values from 32768 to 65535 are reserved for non-distributed uses
+ (for example, internal company usage) or for application usage
+ when testing a feature. The HDF Group does not track or document
+ the use of the filters with identifiers from this range.</p>
+
+ <p>
+ The filters currently in library version 1.8.0 are
+ listed below:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Identification</th>
+ <th width="15%" align="left">Name</th>
+ <th width="65%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>N/A</td>
+ <td>Reserved</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>deflate</td>
+ <td>GZIP deflate compression</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>shuffle</td>
+ <td>Data element shuffling</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>fletcher32</td>
+ <td>Fletcher32 checksum</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>szip</td>
+ <td>SZIP compression</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>5</code></td>
+ <td>nbit</td>
+ <td>N-bit packing</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6</code></td>
+ <td>scaleoffset</td>
+ <td>Scale and offset encoded values</td>
+ </tr>
+ </table>
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Name Length</p></td>
+ <td><p>Each filter has an optional null-terminated ASCII name
+ and this field holds the length of the name including the
+ null termination padded with nulls to be a multiple of
+ eight. If the filter has no name then a value of zero is
+ stored in this field.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>The flags indicate certain properties for a filter. The
+ bit values defined so far are:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set then the filter is an optional filter.
+ During output, if an optional filter fails it will be
+ silently skipped in the pipeline.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1-15</code></td>
+ <td>Reserved (zero)</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Client Data Values</p></td>
+ <td><p>Each filter can store integer values to control
+ how the filter operates. The number of entries in the
+ <em>Client Data</em> array is stored in this field.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td><p>If the <em>Name Length</em> field is non-zero then it will
+ contain the size of this field, padded to a multiple of eight. This
+ field contains a null-terminated, ASCII character string to serve
+ as a comment/name for the filter.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Client Data</p></td>
+ <td><p>This is an array of four-byte integers which will be
+ passed to the filter function. The <em>Client Data Number</em> of
+ Values determines the number of elements in the array.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Padding</p></td>
+ <td><p>Four bytes of zeroes are added to the message at this
+ point if the Client Data Number of Values field contains
+ an odd number.</p></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Filter Pipeline Message - Version 2
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Number of Filters</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Filter Description List <em>(variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Filter Pipeline Message - Version 2
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This table
+ describes version 2.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Filters</p></td>
+ <td><p>The total number of filters described in this
+ message. The maximum possible number of filters in a
+ message is 32.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Filter Description List</p></td>
+ <td><p>A description of each filter. A filter description
+ appears in the next table.</p></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Filter Description - Version 2
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="2">Filter Identification Value</td>
+ <td colspan="2">Name Length <em>(optional)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Flags</td>
+ <td colspan="2">Number Client Data Values</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name <em>(variable size, optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Client Data <em>(variable size, optional)</em><br /><br /></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Filter Description - Version 2
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Filter Identification Value</p></td>
+ <td>
+ <p>
+ This value, often referred to as a filter identifier,
+ is designed to be a unique identifier for the filter.
+ Values from zero through 32,767 are reserved for filters
+ supported by The HDF Group in the HDF5 Library and for
+ filters requested and supported by third parties.
+ Filters supported by The HDF Group are documented immediately
+ below. Information on 3rd-party filters can be found at
+ The HDF Group&rsquo;s
+ <a href="http://www.hdfgroup.org/services/contributions.html">
+ Contributions</a> page.</p>
+
+ <p>
+ To request a filter identifier, please contact
+ The HDF Group&rsquo;s Help Desk at
+ <img src="Graphics/help.png" valign="middle" height="14"
+ alt="The HDF Group Help Desk">.
+ You will be asked to provide the following information:</p>
+ <ol>
+ <li>Contact information for the developer requesting the
+ new identifier</li>
+ <li>A short description of the new filter</li>
+ <li>Links to any relevant information, including licensing
+ information</li>
+ </ol>
+ <p>
+ Values from 32768 to 65535 are reserved for non-distributed uses
+ (for example, internal company usage) or for application usage
+ when testing a feature. The HDF Group does not track or document
+ the use of the filters with identifiers from this range.</p>
+
+ <p>
+ The filters currently in library version 1.8.0 are
+ listed below:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Identification</th>
+ <th width="15%" align="left">Name</th>
+ <th width="65%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>N/A</td>
+ <td>Reserved</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>deflate</td>
+ <td>GZIP deflate compression</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>shuffle</td>
+ <td>Data element shuffling</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>fletcher32</td>
+ <td>Fletcher32 checksum</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>szip</td>
+ <td>SZIP compression</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>5</code></td>
+ <td>nbit</td>
+ <td>N-bit packing</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>6</code></td>
+ <td>scaleoffset</td>
+ <td>Scale and offset encoded values</td>
+ </tr>
+ </table>
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Name Length</p></td>
+ <td><p>Each filter has an optional null-terminated ASCII name
+ and this field holds the length of the name including the
+ null termination padded with nulls to be a multiple of
+ eight. If the filter has no name then a value of zero is
+ stored in this field.</p>
+ <p>Filters with IDs less than 256 (in other words, filters
+ that are defined in this format documentation) do not store
+ the <em>Name Length</em> or <em>Name</em> fields.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>The flags indicate certain properties for a filter. The
+ bit values defined so far are:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set then the filter is an optional filter.
+ During output, if an optional filter fails it will be
+ silently skipped in the pipeline.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1-15</code></td>
+ <td>Reserved (zero)</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Client Data Values</p></td>
+ <td><p>Each filter can store integer values to control
+ how the filter operates. The number of entries in the
+ <em>Client Data</em> array is stored in this field.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td><p>If the <em>Name Length</em> field is non-zero, then it will
+ contain the size of this field, <em>not</em> padded to a multiple
+ of eight. This field contains a <em>non-</em>null-terminated,
+ ASCII character string to serve as a comment/name for the filter.
+ </p>
+ <p>Filters that are defined in this format documentation
+ such as deflate and shuffle do not store the <em>Name
+ Length</em> or <em>Name</em> fields.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Client Data</p></td>
+ <td><p>This is an array of four-byte integers which will be
+ passed to the filter function. The Client Data Number of
+ Values</em> determines the number of elements in the array.</p>
+</td>
+</tr>
+</table>
+</div>
+
+<h4><a name="AttributeMessage">IV.A.2.m. The Attribute Message</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Attribute</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x000C</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td><p>The <em>Attribute</em> message is used to store objects
+ in the HDF5 file which are used as attributes, or
+ &ldquo;metadata&rdquo; about the current object. An attribute
+ is a small dataset; it has a name, a datatype, a dataspace, and
+ raw data. Since attributes are stored in the object header, they
+ should be relatively small (in other words, less than 64KB).
+ They can be associated with any type of object which has an
+ object header (groups, datasets, or committed (named)
+ datatypes).</p>
+ <p>In 1.8.x versions of the library, attributes can be larger
+ than 64KB. See the
+ <a href="UG/HDF5_Users_Guide-Responsive%20HTML5/index.html#t=HDF5_Users_Guide%2FAttributes%2FHDF5_Attributes.htm%3Frhtocid%3Dtoc8.2_1%23TOC_8_5_Special_Issuesbc-13">
+ &ldquo;Special Issues&rdquo;</a> section of the Attributes chapter
+ in the <cite>HDF5 User&rsquo;s Guide</cite> for more information.</p>
+ <p>Note: Attributes on an object must have unique names:
+ the HDF5 Library currently enforces this by causing the
+ creation of an attribute with a duplicate name to fail.
+ Attributes on different objects may have the same name,
+ however.</p></td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Attribute Message (Version 1)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Reserved (zero)</td>
+ <td colspan="2">Name Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Datatype Size</td>
+ <td colspan="2">Dataspace Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Datatype <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Dataspace <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Data <em>(variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Attribute Message (Version 1)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number information is used for changes in the format of the
+ attribute message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Never used.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Used by the library before version 1.6 to encode attribute message.
+ This version does not support shared datatypes.</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Name Size</p></td>
+ <td><p>The length of the attribute name in bytes including the
+ null terminator. Note that the <em>Name</em> field below may
+ contain additional padding not represented by this
+ field.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Datatype Size</p></td>
+ <td><p>The length of the datatype description in the <em>Datatype</em>
+ field below. Note that the <em>Datatype</em> field may contain
+ additional padding not represented by this field.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Dataspace Size</p></td>
+ <td><p>The length of the dataspace description in the <em>Dataspace</em>
+ field below. Note that the <em>Dataspace</em> field may contain
+ additional padding not represented by this field.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td><p>The null-terminated attribute name. This field is
+ padded with additional null characters to make it a
+ multiple of eight bytes.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Datatype</p></td>
+ <td><p>The datatype description follows the same format as
+ described for the datatype object header message. This
+ field is padded with additional zero bytes to make it a
+ multiple of eight bytes.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Dataspace</p></td>
+ <td><p>The dataspace description follows the same format as
+ described for the dataspace object header message. This
+ field is padded with additional zero bytes to make it a
+ multiple of eight bytes.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Data</p></td>
+ <td><p>The raw data for the attribute. The size is determined
+ from the datatype and dataspace descriptions. This
+ field is <em>not</em> padded with additional bytes.</p></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Attribute Message (Version 2)
+ </caption>
+
+ <tr align="center">
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td colspan="2">Name Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Datatype Size</td>
+ <td colspan="2">Dataspace Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Datatype <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Dataspace <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Data <em>(variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Attribute Message (Version 2)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number information is used for changes in the
+ format of the attribute message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Used by the library of version 1.6.x and after to encode
+ attribute messages.
+ This version supports shared datatypes. The fields of
+ name, datatype, and dataspace are not padded with
+ additional bytes of zero.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>This bit field contains extra information about
+ interpreting the attribute message:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, datatype is shared.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>If set, dataspace is shared.</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Name Size</p></td>
+ <td><p>The length of the attribute name in bytes including the
+ null terminator.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Datatype Size</p></td>
+ <td><p>The length of the datatype description in the <em>Datatype</em>
+ field below.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Dataspace Size</p></td>
+ <td><p>The length of the dataspace description in the <em>Dataspace</em>
+ field below.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td><p>The null-terminated attribute name. This field is <em>not</em>
+ padded with additional bytes.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Datatype</p></td>
+ <td><p>The datatype description follows the same format as
+ described for the datatype object header message.
+ </p>
+ <p>If the
+ <em>Flag</em> field indicates this attribute&rsquo;s datatype is
+ shared, this field will contain a &ldquo;shared message&rdquo; encoding
+ instead of the datatype encoding.
+ </p>
+ <p>This field is <em>not</em> padded with additional bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dataspace</p></td>
+ <td><p>The dataspace description follows the same format as
+ described for the dataspace object header message.
+ </p>
+ <p>If the
+ <em>Flag</em> field indicates this attribute&rsquo;s dataspace is
+ shared, this field will contain a &ldquo;shared message&rdquo; encoding
+ instead of the dataspace encoding.
+ </p>
+ <p>This field is <em>not</em> padded with additional bytes.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data</p></td>
+ <td><p>The raw data for the attribute. The size is determined
+ from the datatype and dataspace descriptions.
+ </p>
+ <p>This field is <em>not</em> padded with additional zero bytes.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Attribute Message (Version 3)
+ </caption>
+
+ <tr align="center">
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td colspan="2">Name Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Datatype Size</td>
+ <td colspan="2">Dataspace Size</td>
+ </tr>
+
+ <tr>
+ <td>Name Character Set Encoding</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Name <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Datatype <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Dataspace <em>(variable size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Data <em>(variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Attribute Message (Version 3)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number information is used for changes in the
+ format of the attribute message and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Used by the library of version 1.8.x and after to
+ encode attribute messages.
+ This version supports attributes with non-ASCII names.
+ </td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>This bit field contains extra information about
+ interpreting the attribute message:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, datatype is shared.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>If set, dataspace is shared.</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Name Size</p></td>
+ <td><p>The length of the attribute name in bytes including the
+ null terminator.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Datatype Size</p></td>
+ <td><p>The length of the datatype description in the <em>Datatype</em>
+ field below.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Dataspace Size</p></td>
+ <td><p>The length of the dataspace description in the <em>Dataspace</em>
+ field below.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Name Character Set Encoding</p></td>
+ <td><p>The character set encoding for the attribute&rsquo;s name:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>ASCII character set encoding
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>UTF-8 character set encoding
+ </td>
+ </tr>
+ </table>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td><p>The null-terminated attribute name. This field is <em>not</em>
+ padded with additional bytes.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Datatype</p></td>
+ <td><p>The datatype description follows the same format as
+ described for the datatype object header message.
+ </p>
+ <p>If the
+ <em>Flag</em> field indicates this attribute&rsquo;s datatype is
+ shared, this field will contain a &ldquo;shared message&rdquo; encoding
+ instead of the datatype encoding.
+ </p>
+ <p>This field is <em>not</em> padded with additional bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dataspace</p></td>
+ <td><p>The dataspace description follows the same format as
+ described for the dataspace object header message.
+ </p>
+ <p>If the
+ <em>Flag</em> field indicates this attribute&rsquo;s dataspace is
+ shared, this field will contain a &ldquo;shared message&rdquo; encoding
+ instead of the dataspace encoding.
+ </p>
+ <p>This field is <em>not</em> padded with additional bytes.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data</p></td>
+ <td><p>The raw data for the attribute. The size is determined
+ from the datatype and dataspace descriptions.
+ </p>
+ <p>This field is <em>not</em> padded with additional zero bytes.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<h4><a name="CommentMessage">IV.A.2.n. The Object Comment
+ Message</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Object
+ Comment</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x000D</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>The object comment is designed to be a short description of
+ an object. An object comment is a sequence of non-zero
+ (<code>\0</code>) ASCII characters with no other formatting
+ included by the library.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Object Comment Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Comment <em>(variable size)</em><br /><br /></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Object Comment Message
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Name</p></td>
+ <td><p>A null terminated ASCII character string.</p></td>
+ </tr>
+ </table>
+</div>
+
+<h4><a name="OldModificationTimeMessage">IV.A.2.o. The Object
+ Modification Time (Old) Message</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Object
+ Modification Time (Old)</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x000E</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Fixed</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td><p>The object modification date and time is a timestamp
+ which indicates (using ISO-8601 date and time format) the last
+ modification of an object. The time is updated when any object
+ header message changes according to the system clock where the
+ change was posted. All fields of this message should be
+ interpreted as coordinated universal time (UTC).</p>
+ <p>This modification time message is deprecated in favor of
+ the &ldquo;new&rdquo; <a href="#ModificationTimeMessage">Object
+ Modification Time</a> message and is no longer written to the
+ file in versions of the HDF5 Library after the 1.6.0
+ version.</p></td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Modification Time Message (Old)
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Year</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Month</td>
+ <td colspan="2">Day of Month</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Hour</td>
+ <td colspan="2">Minute</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Second</td>
+ <td colspan="2">Reserved</td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Modification Time Message (Old)
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Year</p></td>
+ <td><p>The four-digit year as an ASCII string. For example,
+ <code>1998</code>.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Month</p></td>
+ <td><p>The month number as a two digit ASCII string where
+ January is <code>01</code> and December is <code>12</code>.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Day of Month</p></td>
+ <td><p>The day number within the month as a two digit ASCII
+ string. The first day of the month is <code>01</code>.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Hour</p></td>
+ <td><p>The hour of the day as a two digit ASCII string where
+ midnight is <code>00</code> and 11:00pm is <code>23</code>.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Minute</p></td>
+ <td><p>The minute of the hour as a two digit ASCII string where
+ the first minute of the hour is <code>00</code> and
+ the last is <code>59</code>.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Second</p></td>
+ <td><p>The second of the minute as a two digit ASCII string
+ where the first second of the minute is <code>00</code>
+ and the last is <code>59</code>.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Reserved</p></td>
+ <td><p>This field is reserved and should always be zero.</p></td>
+ </tr>
+ </table>
+</div>
+
+<h4><a name="SOHMTableMessage">IV.A.2.p. The Shared Message Table
+ Message</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Shared Message
+ Table</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x000F</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Fixed</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>This message is used to locate the table of shared object
+ header message (SOHM) indexes. Each index consists of information
+ to find the shared messages from either the heap or object header.
+ This message is <em>only</em> found in the superblock
+ extension.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Shared Message Table Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Shared Object Header Message Table Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td>Number of Indices</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Shared Message Table Message
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This document describes version 0.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Shared Object Header Message Table Address</p></td>
+ <td><p>This field is the address of the master table for shared
+ object header message indexes.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Number of Indices</p></td>
+ <td><p>This field is the number of indices in the master table.
+ </p></td>
+ </tr>
+
+ </table>
+</div>
+
+<h4><a name="ContinuationMessage">IV.A.2.q. The Object Header
+ Continuation Message</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Object Header
+ Continuation</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0010</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Fixed</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>The object header continuation is the location in the file
+ of a block containing more header messages for the current data
+ object. This can be used when header blocks become too large or
+ are likely to change over time.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Object Header Continuation Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Offset<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Length<sup>L</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Object Header Continuation Message
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Offset</p></td>
+ <td><p>This value is the address in the file where the
+ header continuation block is located.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Length</p></td>
+ <td><p>This value is the length in bytes of the header continuation
+ block in the file.</p></td>
+ </tr>
+ </table>
+</div>
+<br />
+
+<p>The format of the header continuation block that this message points
+ to depends on the version of the object header that the message is
+ contained within.
+</p>
+
+<p>
+ Continuation blocks for version 1 object headers have no special
+ formatting information; they are merely a list of object header
+ message info sequences (type, size, flags, reserved bytes and data
+ for each message sequence). See the description
+ of <a href="#V1ObjectHeaderPrefix">Version 1 Data Object Header Prefix.</a>
+</p>
+
+<p>Continuation blocks for version 2 object headers <em>do</em> have
+ special formatting information as described here
+ (see also the description of
+ <a href="#V2ObjectHeaderPrefix">Version 2 Data Object Header Prefix.</a>):
+</p>
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 Object Header Continuation Block
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+ <tr>
+ <td>Header Message Type #1</td>
+ <td colspan="2">Size of Header Message Data #1</td>
+ <td>Header Message #1 Flags</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Header Message #1 Creation Order <em>(optional)</em></td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Header Message Data #1<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+
+ <tr>
+ <td>Header Message Type #n</td>
+ <td colspan="2">Size of Header Message Data #n</td>
+ <td>Header Message #n Flags</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Header Message #n Creation Order <em>(optional)</em></td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Header Message Data #n<br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Gap <em>(optional, variable size)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 Object Header Continuation Block
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>OCHK</code>&rdquo;
+ is used to indicate the beginning of an object header
+ continuation block. This gives file consistency checking
+ utilities a better chance of reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Type</p></td>
+ <td>
+ <p>Same format as version 1 of the object header, described above.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Header Message #n Data</p></td>
+ <td>
+ <p>Same format as version 1 of the object header, described above.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Flags</p></td>
+ <td>
+ <p>Same format as version 1 of the object header, described above.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Creation Order</p></td>
+ <td>
+ <p>This field stores the order that a message of a given type
+ was created in.</p>
+ <p>This field is present if bit 2 of <em>flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Message #n Data</p></td>
+ <td>
+ <p>Same format as version 1 of the object header, described above.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Gap</p></td>
+ <td>
+ <p>A gap in an object header chunk is inferred by the end of the
+ messages for the chunk before the beginning of the chunk&rsquo;s
+ checksum. Gaps are always smaller than the size of an
+ object header message prefix (message type + message size +
+ message flags).</p>
+ <p>Gaps are formed when a message (typically an attribute message)
+ in an earlier chunk is deleted and a message from a later
+ chunk that does not quite fit into the free space is moved
+ into the earlier chunk.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>This is the checksum for the object header chunk.
+ </p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<h4><a name="SymbolTableMessage">IV.A.2.r. The Symbol Table
+ Message</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Symbol Table
+ Message</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0011</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Fixed</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Required for
+ &ldquo;old style&rdquo; groups; may not be repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>Each &ldquo;old style&rdquo; group has a v1 B-tree and a
+ local heap for storing symbol table entries, which are located
+ with this message.</td></tr>
+ <tr><td colspan="2"><b>Format of data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Symbol Table Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />v1 B-tree Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Local Heap Address<sup>O</sup><br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Symbol Table Message
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>v1 B-tree Address</p></td>
+ <td><p>This value is the address of the v1 B-tree containing the
+ symbol table entries for the group.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Local Heap Address</p></td>
+ <td><p>This value is the address of the local heap containing
+ the link names for the symbol table entries for the group.</p></td>
+ </tr>
+ </table>
+</div>
+
+<h4><a name="ModificationTimeMessage">IV.A.2.s. The Object
+ Modification Time Message</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Object
+ Modification Time</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0012</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Fixed</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>The object modification time is a timestamp which indicates
+ the time of the last modification of an object. The time is
+ updated when any object header message changes according to
+ the system clock where the change was posted.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Modification Time Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3">Reserved <em>(zero)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Seconds After UNIX Epoch</td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Modification Time Message
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number is used for changes in the format of Object Modification Time
+ and is described here:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Never used.</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Used by Version 1.6.1 and after of the library to encode time. In
+ this version, the time is the seconds after Epoch.</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Seconds After UNIX Epoch</p></td>
+ <td><p>A 32-bit unsigned integer value that stores the number of
+ seconds since 0 hours, 0 minutes, 0 seconds, January 1, 1970,
+ Coordinated Universal Time.</p></td>
+ </tr>
+ </table>
+</div>
+
+<h4><a name="BtreeKValuesMessage">IV.A.2.t. The B-tree
+ &lsquo;K&rsquo; Values Message</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> B-tree
+ &lsquo;K&rsquo; Values</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0013</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Fixed</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>This message retrieves non-default &lsquo;K&rsquo; values
+ for internal and leaf nodes of a group or indexed storage v1
+ B-trees. This message is <em>only</em> found in the superblock
+ extension.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: B-tree &lsquo;K&rsquo; Values Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="2">Indexed Storage Internal Node K</td>
+ <td bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Group Internal Node K</td>
+ <td colspan="2">Group Leaf Node K</td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: B-tree &lsquo;K&rsquo; Values Message
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This document describes
+ version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Indexed Storage Internal Node K</p></td>
+ <td><p>This is the node &lsquo;K&rsquo; value for each internal node of an
+ indexed storage v1 B-tree. See the description of this field
+ in version 0 and 1 of the superblock as well the section on
+ v1 B-trees.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Group Internal Node K</p></td>
+ <td><p>This is the node &lsquo;K&rsquo; value for each internal node of a group
+ v1 B-tree. See the description of this field in version 0 and
+ 1 of the superblock as well as the section on v1 B-trees.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Group Leaf Node K</p></td>
+ <td><p>This is the node &lsquo;K&rsquo; value for each leaf node of a group v1
+ B-tree. See the description of this field in version 0 and 1
+ of the superblock as well as the section on v1 B-trees.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<h4><a name="DrvInfoMessage">IV.A.2.u. The Driver Info
+ Message</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Driver
+ Info</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0014</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+
+ <tr><td>
+ <b>Description:</b></td>
+ <td>This message contains information needed by the file driver
+ to reopen a file. This message is <em>only</em> found in the
+ superblock extension: see the <a href="#SuperblockExt">
+ &ldquo;Disk Format: Level 0C - Superblock Extension&rdquo;</a>
+ section for more information. For more information on the fields
+ in the driver info message, see the <a href="#DriverInfo">
+ &ldquo;Disk Format: Level 0B - File Driver Info&rdquo;</a>
+ section; those who use the multi and family file drivers will
+ find this section particularly helpful.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Driver Info Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Driver Identification</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Driver Information Size</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br /><br />Driver Information <em>(variable size)</em><br /><br /><br /></td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Driver Info Message
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This document describes
+ version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Driver Identification</p></td>
+ <td><p>This is an eight-byte ASCII string without null termination which
+ identifies the driver.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Driver Information Size</p></td>
+ <td><p>The size in bytes of the <em>Driver Information</em> field of this
+ message.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Driver Information</p></td>
+ <td><p>Driver information is stored in a format defined by the file driver.</p>
+ </td>
+ </tr>
+ </table>
+</div>
+
+<h4><a name="AinfoMessage">IV.A.2.v. The Attribute Info
+ Message</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Attribute
+ Info</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0015</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Varies</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>This message stores information about the attributes on an
+ object, such as the maximum creation index for the attributes
+ created and the location of the attribute storage when the
+ attributes are stored &ldquo;densely&rdquo;.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Attribute Info Message
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Flags</td>
+ <td colspan="2">Maximum Creation Index <em>(optional)</em></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Fractal Heap Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Attribute Name v2 B-tree Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Attribute Creation Order v2 B-tree Address<sup>O</sup> <em>(optional)</em><br /><br /></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Attribute Info Message
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This document describes
+ version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>This is the attribute index information flag with the
+ following definition:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, creation order for attributes is tracked.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>If set, creation order for attributes is indexed.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2-7</code></td>
+ <td>Reserved</td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Maximum Creation Index</p></td>
+ <td><p>The is the maximum creation order index value for the
+ attributes on the object.</p>
+ <p>This field is present if bit 0 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Fractal Heap Address</p></td>
+ <td><p>This is the address of the fractal heap to store dense
+ attributes.
+ Each attribute stored in the fractal heap is described by
+ the <a href="#AttributeMessage">Attribute Message.</a>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Attribute Name v2 B-tree Address</p></td>
+ <td><p>This is the address of the version 2 B-tree to index the
+ names of densely stored attributes.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Attribute Creation Order v2 B-tree Address</p></td>
+ <td><p>This is the address of the version 2 B-tree to index the
+ creation order of densely stored attributes.</p>
+ <p>This field is present if bit 1 of <em>Flags</em> is set.</p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<h4><a name="RefCountMessage">IV.A.2.w. The Object Reference
+ Count Message</a></h4>
+
+<!-- start msgdesc table -->
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> Object Reference
+ Count</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0016</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Fixed</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td><b>Description:</b></td>
+ <td>This message stores the number of hard links (in groups or
+ objects) pointing to an object: in other words, its
+ <em>reference count</em>.</td></tr>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<!-- end msgdesc table -->
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Object Reference Count
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reference count</td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Object Reference Count
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for this message. This document describes
+ version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Reference Count</p></td>
+ <td><p>The unsigned 32-bit integer is the reference count for the
+ object. This message is only present in &ldquo;version 2&rdquo;
+ (or later) object headers, and if not present those object
+ header versions, the reference count for the object is assumed
+ to be 1.</p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+
+<h4><a name="FsinfoMessage">IV.A.2.x. The File Space Info
+ Message</a></h4>
+
+<center>
+ <table class="msgdesc">
+ <tr><td colspan="2"><b>Header Message Name:</b> File Space
+ Info</td></tr>
+ <tr><td colspan="2"><b>Header Message Type:</b> 0x0017</td></tr>
+ <tr><td colspan="2"><b>Length:</b> Fixed</td></tr>
+ <tr><td colspan="2"><b>Status:</b> Optional; may not be
+ repeated.</td></tr>
+ <tr><td>
+ <b>Description:</b></td>
+ <td>This message stores the file space management information
+ that the library uses in handling file space
+ requests for the file. Version 0 of the message is used for release 1.10.0 only.
+ Version 1 of the message is used for release 1.10.1+.
+ There is no File Space Info message before release 1.10 as the library does
+ not track file space across multiple file opens.
+ <p>
+ Note that version 0 is deprecated starting release 1.10.1.
+ That means when the 1.10.1+ library opens an HDF5 file with a version 0 message,
+ the library will decode and map the message to version 1.
+ On file close, it will encode the message as a version 1 message.
+ <p>
+ The library uses the following three mechanisms to manage file space in an HDF5 file:
+ <ul>
+ <li> Free-space managers
+ <br> They track free-space sections of various sizes in the file that are not currently
+ allocated. Each free-space manager corresponds to a file space type.
+ There are two main groups of file space types: metadata and raw data.
+ Metadata is further divided into five types: superblock, B-tree, global heap,
+ local heap, and object header.
+ See the description of <a href="#FreeSpaceManager">Free-space
+ Manager</a> as well the description of file space allocation types in
+ <a href="#AppendixB">Appendix B</a>
+ </li>
+ <li> Aggregators
+ <br> The library manages two aggregators, one for metadata and one for raw data.
+ Aggregator is a contiguous block of free-space in the file.
+ The size of each aggregator is tunable via public routines
+ <code>H5Pset_meta_block_size</code> and <code>H5Pset_small_data_block_size</code> respectively.
+ </li>
+ <li> Virtual file drivers
+ <br> The library's virtual file driver interface dispatches requests for additional
+ space to the allocation routine of the file driver associated with the file.
+ For example, if the sec2 file driver is being used, its allocation routine will
+ increase the size of the file to service the requests.
+ </li>
+ </ul>
+ <p>
+ For release 1.10.0, the library derives the following four file space strategies
+ based on the mechanisms:
+ <ul>
+ <li>H5F_FILE_SPACE_ALL
+ <ul>
+ <li>Mechanisms used: free-space managers, aggregators, and virtual file drivers</li>
+ <li>Does not persist free-space across file opens</li>
+ <li>This strategy is the library default</li>
+ </ul>
+ </li>
+ <li>H5F_FILE_SPACE_ALL_PERSIST</li>
+ <ul>
+ <li>Mechanisms used: free-space managers, aggregators, and virtual file drivers</li>
+ <li>Persist free-space across file opens</li>
+ </ul>
+ <li>H5F_FILE_SPACE_AGGR_VFD</li>
+ <ul>
+ <li>Mechanisms used: aggregators and virtual file drivers</li>
+ <li>Does not persist free-space across file opens</li>
+ </ul>
+ <li>H5F_FILE_SPACE_VFD</li>
+ <ul>
+ <li>Mechanisms used: virtual file drivers</li>
+ <li>Does not persist free-space across file opens</li>
+ </ul>
+ </ul>
+ For release 1.10.1+, the free-space manager mechanism is modified to handle paged aggregation
+ which aggregates small metadata and raw data allocations into constant-sized well-aligned pages
+ to allow efficient I/O accesses.
+ With the support of this feature, the library derives the following four file space strategies:
+ <ul>
+ <li>H5F_FSPACE_STRATEGY_FSM_AGGR </li>
+ <ul>
+ <li>Mechanisms used: free-space managers, aggregators, and virtual file drivers</li>
+ <li>This strategy is the library default</li>
+ </ul>
+ <li>H5F_FSPACE_STRATEGY_PAGE</li>
+ <ul>
+ <li>Mechanisms used: free-space managers with embedded paged aggregation and virtual file drivers</li>
+ </ul>
+ <li>H5F_FSPACE_STRATEGY_AGGR</li>
+ <ul>
+ <li>Mechanisms used: aggregators and virtual file drivers</li>
+ </ul>
+ <li>H5F_FSPACE_STRATEGY_NONE</li>
+ <ul>
+ <li>Mechanisms used: virtual file drivers</li>
+ </ul>
+ </ul>
+ The default is not persisting free-space across file opens for the above four strategies.
+ User can use the public routine <code>H5Pset_file_space_strategy</code> to request
+ persisting free-space.
+ </td></tr>
+ <p>
+ <tr><td colspan="2"><b>Format of Data:</b> See the tables
+ below.</td></tr>
+</table></center>
+<p>
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: File Space Info - Version 0
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Strategy</td>
+ <td colspan="2">Threshold<sup>L</sup></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Free-space manager address<sup>O</sup> for H5FD_MEM_SUPER<br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Free-space manager address<sup>0</sup> for H5FD_MEM_BTREE<br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Free-space manager address<sup>0</sup> for H5FD_MEM_DRAW<br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Free-space manager address<sup>0</sup> for H5FD_MEM_GHEAP<br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Free-space manager address<sup>0</sup> for H5FD_MEM_LHEAP<br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Free-space manager address<sup>0</sup> for H5FD_MEM_OHDR<br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: File Space Info
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>This is version 0 of this message.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Strategy</p></td>
+ <td><p>This is the file space strategy used to manage file space.
+ There are four types:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>H5F_FILE_SPACE_ALL_PERSIST</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>H5F_FILE_SPACE_ALL</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>H5F_FILE_SPACE_AGGR_VFD</td>
+ </tr>
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>H5F_FILE_SPACE_VFD</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Threshold</p></td>
+ <td><p>This is the smallest free-space section size that the
+ free-space manager will track.
+ </td>
+ </tr>
+ <tr>
+ <td><p>Free-space manager addresses</p></td>
+ <td><p>These are the six free-space manager addresses for the
+ six file space allocation types:
+ <ul>
+ <li>H5FD_MEM_SUPER</li>
+ <li>H5FD_MEM_BTREE</li>
+ <li>H5FD_MEM_DRAW</li>
+ <li>H5FD_MEM_GHEAP</li>
+ <li>H5FD_MEM_LHEAP</li>
+ <li>H5FD_MEM_OHDR</li>
+ </ul>
+ Note that these six fields exist only if the value for the field
+ &ldquo;<em>Strategy</em>&rdquo; is H5F_FILE_SPACE_ALL_PERSIST.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+ <br />
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: File Space Info - Version 1
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Strategy</td>
+ <td>Persisting free-space</td>
+ <td colspan="1" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Free-space Section Threshold<sup>L</sup></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">File Space Page Size</td>
+ </tr>
+
+ <tr>
+ <td colspan="2">Page-end Metadata threshold</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />EOA<sup>0</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup> of small-sized free-space manager for H5FD_MEM_SUPER<br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup> of small-sized free-space manager for H5FD_MEM_BTREE<br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup> of small-sized free-space manager for H5FM_MEM_DRAW<br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup> of small-sized free-space manager for H5FD_MEM_GHEAP<br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup> of small-sized free-space manager for H5FD_MEM_LHEAP<br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup> of small-sized free-space manager for H5FD_MEM_OHDR<br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup> of large-sized free-space manager for H5FD_MEM_SUPER<br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup> of large-sized free-space manager for H5FD_MEM_BTREE<br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup> of large-sized free-space manager for H5FM_MEM_DRAW<br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup> of large-sized free-space manager for H5FD_MEM_GHEAP<br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup> of large-sized free-space manager for H5FD_MEM_LHEAP<br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup> of large-sized free-space manager for H5FD_MEM_OHDR<br /><br /></td>
+ </tr>
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: File Space Info
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>This is version 1 of this message.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Strategy</p></td>
+ <td><p>This is the file space strategy used to manage file space.
+ There are four types:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>H5F_FSPACE_STRATEGY_FSM_AGGR</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>H5F_FSPACE_STRATEGY_PAGE</td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>H5F_FSPACE_STRATEGY_AGGR</td>
+ </tr>
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>H5F_FSPACE_STRATEGY_NONE</td>
+ </tr>
+ </table></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Persisting free-space</p></td>
+ <td><p>True or false in persisting free-space.
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Free-space Section Threshold</p></td>
+ <td><p>This is the smallest free-space section size that the
+ free-space manager will track.
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>File space page size</p></td>
+ <td><p>This is the file space page size, which is used when the paged aggregation feature
+ is enabled.
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Page-end metadata threshold</p></td>
+ <td><p>This is the smallest free-space section size at the end of a page that
+ the free-space manager will track. This is used when the paged aggregation feature
+ is enabled.
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>EOA</p></td>
+ <td><p>The EOA before the allocation of free-space manager header and section info for the
+ self-referential free-space managers when persisting free-space.
+ <br>
+ Note that self-referential free-space managers are managers that involve file space
+ allocation for the managers' free-space header and section info.
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Addresses of small-sized free-space managers</p></td>
+ <td><p>These are the addresses of the six small-sized free-space managers for
+ the six file space allocation types:
+ </p>
+ <ul>
+ <li>H5FD_MEM_SUPER</li>
+ <li>H5FD_MEM_BTREE</li>
+ <li>H5FD_MEM_DRAW</li>
+ <li>H5FD_MEM_GHEAP</li>
+ <li>H5FD_MEM_LHEAP</li>
+ <li>H5FD_MEM_OHDR</li>
+ </ul>
+ Note that these six fields exist only if the value for the field
+ &ldquo;<em>Persisting free-space</em>&rdquo; is true.
+</ul>
+</td>
+</tr>
+
+<tr>
+ <td><p>Addresses of large-sized free-space managers</p></td>
+ <td><p>These are the addresses of the six large-sized free-space managers for
+ the six file space allocation types:
+ </p>
+ <ul>
+ <li>H5FD_MEM_SUPER</li>
+ <li>H5FD_MEM_BTREE</li>
+ <li>H5FD_MEM_DRAW</li>
+ <li>H5FD_MEM_GHEAP</li>
+ <li>H5FD_MEM_LHEAP</li>
+ <li>H5FD_MEM_OHDR</li>
+ </ul>
+ Note that these six fields exist only if the value for the field
+ &ldquo;<em>Persisting free-space</em>&rdquo; is true.
+</ul>
+</td>
+</tr>
+
+</table>
+</div>
+
+<h3><a name="DataStorage">
+ IV.B. Disk Format: Level 2B - Data Object Data Storage</a></h3>
+
+<p>The data for an object is stored separately from its header
+ information in the file and may not actually be located in the HDF5 file
+ itself if the header indicates that the data is stored externally. The
+ information for each record in the object is stored according to the
+ dimensionality of the object (indicated in the dataspace header message).
+ Multi-dimensional array data is stored in C order; in other words, the
+ &ldquo;last&rdquo; dimension changes fastest.</p>
+
+<p>Data whose elements are composed of atomic datatypes are stored in IEEE
+ format, unless they are specifically defined as being stored in a different
+ machine format with the architecture-type information from the datatype
+ header message. This means that each architecture will need to [potentially]
+ byte-swap data values into the internal representation for that particular
+ machine.</p>
+
+<p> Data with a variable-length datatype is stored in the global heap
+ of the HDF5 file. Global heap identifiers are stored in the
+ data object storage.</p>
+
+<p>Data whose elements are composed of reference datatypes are stored in
+ several different ways depending on the particular reference type involved.
+ Object pointers are just stored as the offset of the object header being
+ pointed to with the size of the pointer being the same number of bytes as
+ offsets in the file.</p>
+
+<p>Dataset region references are stored as a heap-ID which points to
+ the following information within the file-heap: an offset of the object
+ pointed to, number-type information (same format as header message),
+ dimensionality information (same format as header message), sub-set start
+ and end information (in other words, a coordinate location for each),
+ and field start and end names (in other words, a [pointer to the] string
+ indicating the first field included and a [pointer to the] string name
+ for the last field). </p>
+
+<p>Data of a compound datatype is stored as a contiguous stream of the items
+ in the structure, with each item formatted according to its datatype.
+<p>
+ Description of datatypes for variable-length, references and compound classes can be found
+ in <a href="#DatatypeMessage">Datatype Message</a>.
+<p>
+ Information about global heap and heap ID can be found in <a href="#GlobalHeap">Global Heap</a>.
+<p>
+ For reference datatype,
+ see also the encoding description for <a href="#ReferenceEncodeRV">Reference Encoding (Revised) </a> and
+ <a href="#ReferenceEncodeDP">Reference Encoding (Backward Compatibility)</a> in Appendix D.
+</p>
+
+<h2><a name="AppendixA">
+ V. Appendix A: Definitions</a></h2>
+
+<p>Definitions of various terms used in this document are included in
+ this section.</p>
+
+<div align="center">
+ <table class="glossary">
+ <tr>
+ <th width="20%">Term</th>
+ <th>Definition</th>
+ </tr>
+
+ <tr>
+ <td>Undefined Address</td>
+ <td>The <a name="UndefinedAddress">undefined
+ address</a> for a file is a file address with all bits
+ set: in other words, <code>0xffff...ff</code>.</td>
+ </tr>
+
+ <tr>
+ <td>Unlimited Size</td>
+ <td>The <a name="UnlimitedDim">unlimited size</a>
+ for a size is a value with all bits set: in other words,
+ <code>0xffff...ff</code>.</td>
+ </tr>
+
+ </table>
+</div>
+
+
+<h2><a name="AppendixB">
+ VI. Appendix B: File Space Allocation Types</a></h2>
+
+<p>There are six basic types of file space allocation as follows:
+</p>
+<div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Basic Allocation Type</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_SUPER</td>
+ <td>File space allocated for <em>Superblock.</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_BTREE</td>
+ <td>File space allocated for <em>B-tree.</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_DRAW</td>
+ <td>File space allocated for <em>raw data</em>.</td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_GHEAP</td>
+ <td>File space allocated for <em>Global Heap.</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_LHEAP</td>
+ <td>File space allocated for <em>Local Heap.</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_OHDR</td>
+ <td>File space allocated for <em>Object Header.</em></td>
+ </tr>
+ </table>
+</div>
+
+<br />
+<p>There are other file space allocation types that are mapped to the
+ above six basic types because they are similar in nature.
+ The mapping and the corresponding description are listed in the following two tables:
+</p>
+
+<div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Basic Allocation Type</th>
+ <th>Mapping of Allocation Types to Basic Allocation Types</th>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_SUPER</td>
+ <td><em>none</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_BTREE</td>
+ <td>H5FD_MEM_SOHM_INDEX</td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_DRAW</td>
+ <td>H5FD_MEM_FHEAP_HUGE_OBJ</td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_GHEAP</td>
+ <td><em>none</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_LHEAP</td>
+ <td>H5FD_MEM_FHEAP_DBLOCK, H5FD_MEM_FSPACE_SINFO</td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_OHDR</td>
+ <td>H5FD_MEM_FHEAP_HDR, H5FD_MEM_FHEAP_IBLOCK, H5FD_MEM_FSPACE_HDR, H5FD_MEM_SOHM_TABLE</td>
+ </tr>
+ </table>
+</div>
+
+<br />
+</p>
+
+<div align="center">
+ <table class="desc">
+ <tr>
+ <th width="30%">Allocation Type</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_FHEAP_HDR</td>
+ <td>File space allocated for <em>Fractal Heap Header.</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_FHEAP_DBLOCK</td>
+ <td>File space allocated for <em>Fractal Heap Direct Blocks.</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_FHEAP_IBLOCK</td>
+ <td>File space allocated for <em>Fractal Heap Indirect Blocks.</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_FHEAP_HUGE_OBJ</td>
+ <td>File space allocated for huge objects in the fractal heap.</td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_FSPACE_HDR</td>
+ <td>File space allocated for <em>Free-space Manager Header.</em></td>
+ </tr>
+
+ <tr>
+ <td>H5FD_MEM_FSPACE_SINFO</td>
+ <td>File space allocated for <em>Free-space Section List</em> of the free-space manager.</td>
+ </tr>
+ <tr>
+ <td>H5FD_MEM_SOHM_TABLE</td>
+ <td>File space allocated for <em>Shared Object Header Message Table.</em></td>
+ </tr>
+ <tr>
+ <td>H5FD_MEM_SOHM_INDEX</td>
+ <td>File space allocated for <em>Shared Message Record List.</em></td>
+ </tr>
+ </table>
+</div>
+
+<h2><a name="AppendixC"> VII. Appendix C:
+ Types of Indexes for Dataset Chunks</a></h2>
+
+<p>For an HDF5 file without the latest format enabled, the library
+ uses the <a href="#V1Btrees">Version 1 B-tree</a> to index dataset
+ chunks.</p>
+
+<p>For an HDF5 file with the latest format enabled, the library uses
+ one of the following five indexing types depending on a chunked
+ dataset&rsquo;s dimension specification and the way it is extended.
+</p>
+
+<a name="SingleChunk">
+ <h3>VII.A. The Single Chunk Index</h3></a>
+
+<p>The <i>Single Chunk</i> index can be used when the dataset fulfills
+ the following condition:</p>
+
+<ul>
+ <li>the current, maximum, and chunk dimension sizes are all the same</li>
+</ul>
+
+<p>The dataset has only one chunk, and the address of the single
+ chunk is stored in the version 4 <i>Data Layout</i> message.
+ See the <a href="#ChunkedStorage">Chunked Storage Property
+ Description</i></a> layout and field description tables.</p>
+
+<a name="Implicit">
+ <h3>VII.B. The Implicit Index</h3></a>
+
+<p>The <i>Implicit</i> index can be used when the dataset fulfills
+ the following conditions:</p>
+
+<ul>
+ <li>fixed maximum dimension sizes</li>
+ <li>no filter applied to the dataset</li>
+ <li>the timing for the space allocation of the dataset chunks is
+ <code>H5P_ALLOC_TIME_EARLY</code></li>
+</ul>
+
+<p>Since the dataset&rsquo;s dimension sizes are known and storage space
+ is to be allocated early, an array of dataset chunks are allocated
+ based on the maximum dimension sizes when the dataset is created.
+ The base address of the array is stored in the version 4
+ <i>Data Layout</i> message. See the
+ <a href="#ChunkedStorage">Chunked Storage Property
+ Description</i></a> layout and field description tables.
+</p>
+
+<p>When accessing a dataset chunk with a specified offset, the
+ address of the chunk in the array is computed as below:</p>
+
+<dir><p><code>base address + (size of a chunk in bytes * chunk index
+ associated with the offset)</code></p></dir>
+
+<p>A chunk index starts at 0 and increases according to the
+ fastest changing dimension, then the next fastest, and so on.
+ <a name="ChunkIndex"></a>
+ The chunk index for a dataset chunk offset is computed as below:
+ <ol>
+ <li>Calculate the scaled offset for each dimension in
+ <code>scaled_offset</code>:
+ <br />
+ <pre>
+ scaled_offset = chunk_offset/chunk_dims
+ </pre></li>
+ <li>Calculate the # of chunks for each dimension in
+ <code>nchunks</code>:
+ <br />
+ <pre>
+ nchunks = (curr_dims + chunk_dims - 1)/chunk_dims
+ </pre></li>
+
+ <li>Calculate the down chunks for each dimension in
+ <code>down_chunks</code>:
+ <br />
+ <pre>
+ /* n is the # of dimensions */
+ for(i = (int)(n-1), acc = 1; i >= 0; i--) {
+ down_chunks[i] = acc;
+ acc *= nchunks[i];
+ }
+ </pre>
+ </li>
+
+ <li>Calculate the chunk index in <code>chunk_index</code>:
+ <br />
+ <pre>
+ /* n is the # of dimensions */
+ for(u = 0, chunk_index = 0; u < n; u++)
+ chunk_index += down_chunks[u] * scaled_offset[u];
+ </pre>
+ </li>
+ </ol>
+<p>
+ For example, for a 2-dimensional dataset with
+ <code>curr_dims[4,5]</code> and <code>chunk_dims[3,2]</code>,
+ there will be a total of 6 chunks, with 3 chunks in the fastest
+ changing dimension and 2 chunks in the slowest changing dimension.
+ See the figure below.
+ The chunk index for the chunk offset <code>[3,4]</code>
+ is computed as below:
+ <ol>
+ <code>
+ <li>scaled_offset[0] = 1, scaled_offset[1] = 2</li>
+ <li>nchunks[0] = 2, nchunks[1] = 3</li>
+ <li>down_chunks[0] = 3, down_chunks[1] = 1</li>
+ <li>chunk_index = 5</li>
+ </code>
+ </ol>
+
+
+ <table align="center" width="400" border="0">
+ <tr valign="center" align="center">
+ <td>
+ <hr size="2"/>
+ <img height="250" src="FileFormatSpecChunkDiagram.jpg"
+ alt="Chunk Diagram"></td>
+ </tr>
+ <tr valign="top" align="center">
+ <td>
+ <hr size="1" />
+ <b>Figure 3. Implicit index chunk diagram </b>
+ <hr size="2"/></td>
+ </tr>
+ </table>
+
+
+
+
+
+ <a name="FixedArray">
+ <h3>VII.C. The Fixed Array Index</h3></a>
+
+<p>The <i>Fixed Array</i> index can be used when the dataset fulfills
+ the following condition:</p>
+<ul>
+ <li>fixed maximum dimension sizes</li>
+</ul>
+
+<p>Since the maximum number of chunks is known, an array of
+ in-file-on-disk addresses based on the maximum number of chunks is
+ allocated when data is written to the dataset. To access a dataset
+ chunk with a specified offset, the
+ <a href="#ChunkIndex">chunk index</i></a> associated with the offset
+is calculated. The index is mapped into the array to locate the
+disk address for the chunk.</p>
+
+<p>The Fixed Array (FA) index structure provides space and speed
+ improvements in locating chunks over index structures that handle
+ more dynamic data accesses like a
+ <a href="#AppendV2Btrees">Version 2 B-tree</a> index.
+ The entry into the Fixed Array is the Fixed Array header which
+ contains metadata about the entries stored in the array. The
+ header contains a pointer to a data block which stores the array
+ of entries that describe the dataset chunks. For greater efficiency,
+ the array will be divided into multiple pages if the number of
+ entries exceeds a threshold value. The space for the data block
+ and possibly data block pages are allocated as a single contiguous
+ block of space.</p>
+
+<p>The content of the data block depends on whether paging is
+ activated or not. When paging is not used, elements that describe
+ the chunks are stored in the data block. If paging is turned on,
+ the data block contains a bitmap indicating which pages are
+ initialized. Then subsequent data block pages will contain the
+ entries that describe the chunks.</p>
+
+<p>An entry describes either a filtered or non-filtered dataset
+ chunk. The formats for both element types are described below.
+</p>
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fixed Array Header
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Client ID</td>
+ <td>Entry Size</td>
+ <td>Page Bits</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Max Num
+ Entries<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Data Block
+ Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fixed Array Header
+ </caption>
+ <tr>
+ <th width="40%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>FAHD</code>&rdquo;
+ is used to indicate the beginning of a Fixed Array header.
+ This gives file consistency checking utilities a better
+ chance of reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Client ID</p></td>
+ <td>
+ <p>The ID for identifying the client of the
+ Fixed Array:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">ID</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Non-filtered dataset chunks
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Filtered dataset chunks
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2+</code></td>
+ <td>Reserved
+ </td>
+ </tr>
+ </table>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Entry Size</p></td>
+ <td>
+ <p>The size in bytes of an entry in the Fixed Array.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Page Bits</p></td>
+ <td>
+ <p>The number of bits needed to store the maximum
+ number of entries in a
+ <a href="#FADataBlockPage">data block page.</a></p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Max Num Entries</p></td>
+ <td>
+ <p>The maximum number of entries in the Fixed
+ Array.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data Block Address</p></td>
+ <td>
+ <p>The address of the data block in the Fixed Array.
+ </p>
+ </td>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>The checksum for the header.</p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Fixed Array Data Block
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Client ID</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted
+ only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Page Bitmap <em>(variable size and
+ optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Elements <em>(variable size and
+ optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fixed Array Data Block
+ </caption>
+ <tr>
+ <th width="40%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>FADB</code>&rdquo;
+ is used to indicate the beginning of a Fixed Array data
+ block. This gives file consistency checking utilities a
+ better chance of reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Client ID</p></td>
+ <td>
+ <p>The ID for identifying the client of the
+ Fixed Array:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">ID</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Non-filtered dataset chunks
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Filtered dataset chunks
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2+</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Address</p></td>
+ <td>
+ <p>The address of the Fixed Array header. Principally used
+ for file integrity checking.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Page Bitmap</p></td>
+ <td><p>A bitmap indicating which data block pages are initialized.</p>
+ <p>Exists only if the data block is paged.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Elements</p></td>
+ <td>
+ <p>Contains the elements stored in the data block
+ and exists only if the data block is not paged.
+ There are two element types:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">ID</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td><a href="#FaNonFilterChunk">Non-filtered
+ dataset chunks</i></a>
+</td>
+</tr>
+<tr>
+ <td align="center"><code>1</code></td>
+ <td><a href="#FaFilterChunk">Filtered dataset
+ chunks</i></a>
+</td>
+</tr>
+</table>
+</p>
+</td>
+</tr>
+
+<tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>The checksum for the Fixed Array data block.</p>
+ </td>
+</tr>
+
+</table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption><a name="FADataBlockPage">
+ Layout: Fixed Array Data Block Page</a>
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Elements <em>(variable
+ size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Fixed Array Data Block Page
+ </caption>
+ <tr>
+ <th width="40%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Elements</p></td>
+ <td>
+ <p>Contains the elements stored in the data block page.
+ There are two element types:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">ID</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td><a href="#FaNonFilterChunk">Non-filtered dataset chunks</i></a>
+</td>
+</tr>
+<tr>
+ <td align="center"><code>1</code></td>
+ <td><a href="#FaFilterChunk">Filtered dataset chunks</i></a>
+</td>
+</tr>
+</table>
+</p>
+</td>
+</tr>
+
+<tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>The checksum for a Fixed Array data block page.</p>
+ </td>
+</tr>
+
+</table>
+</div>
+
+<br />
+<br />
+<br />
+<a name="FaNonFilterChunk"></a>
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Data Block Element for Non-filtered Dataset Chunk
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Data Block Element for Non-filtered Dataset Chunk
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>The address of the dataset chunk in the file.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+<!-- </p> -->
+
+<br />
+<br />
+<br />
+<a name="FaFilterChunk"></a>
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Data Block Element for Filtered Dataset Chunk
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Chunk Size <em>(variable size; at most
+ 8 bytes)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Filter Mask</td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Data Block Element for Filtered Dataset Chunk
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>The address of the dataset chunk in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Chunk Size</p></td>
+ <td><p>The size of the dataset chunk in bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filter Mask</p></td>
+ <td><p>Indicates the filter to skip for the dataset chunk. Each
+ filter has an index number in the pipeline; if that filter is
+ skipped, the bit corresponding to its index is set.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<a name="ExtensibleArray">
+ <h3>VII.D. The Extensible Array Index</h3></a>
+
+<p>The <i>Extensible Array</i> index can be used when the dataset
+ fulfills the following condition:</p>
+
+<ul>
+ <li>only one dimension of unlimited extent</li>
+</ul>
+
+<p>The Extensible Array (EA) is a data structure that is used as a
+ chunk index in datasets where the dataspace has a single
+ unlimited dimension. In other words, one dimension is set to
+ <code>H5S_UNLIMITED</code>, and the other dimensions are any number
+ of fixed-size dimensions. The idea behind the extensible array is
+ that a particular data object can be located via a lightweight
+ indexing structure of fixed depth for a given address space. This
+ indexing structure requires only a few (2-3) file operations per
+ element lookup and gives good cache performance. Unlike the B-tree
+ structure, the extensible array is optimized for appends. Where a
+ B-tree would always add at the rightmost node under these
+ circumstances, either creating a deep tree (version 1) or requiring
+ expensive rebalances to correct (version 2), the extensible array
+ has already mapped out a pre-balanced internal structure. This
+ optimized internal structure is instantiated as needed when chunk
+ records are inserted into the structure.</p>
+
+
+
+<!--
+
+ <p>A description of the rationale that leads to the present
+ implementation of the extensible array can be found at
+ <a href="https://svn.hdfgroup.org/hdf5doc/trunk/projects/1_10_alpha/ReviseChunks/skip_lists">
+ https://svn.hdfgroup.org/hdf5doc/trunk/projects/1_10_alpha/ReviseChunks/skip_lists</a>.
+ </p>
+
+<p>The current implementation differs from the data structure
+ described in that reference in some ways, but the basic idea is the
+ same.</p>
+
+-->
+
+
+
+<p>An Extensible Array consists of a header, an index block,
+ secondary blocks, data blocks, and (optional) data block pages. The
+ general scheme is that the index block is used to reference a
+ secondary block, which is, in turn, used to reference the data block
+ page where the chunk information is stored. The data blocks will
+ be paged for efficiency when their size passes a threshold value.
+ These pages are laid out contiguously on the disk after the data
+ block, are initialized as needed, and are tracked via bitmaps
+ stored in the secondary block. The number of secondary and data
+ blocks/pages in a chunk index varies as they are allocated as
+ needed and the first few are (conceptually) stored in parent
+ elements as an optimization.</p>
+
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Extensible Array Header
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Client ID</td>
+ <td>Element Size</td>
+ <td>Max Nelmts Bits</td>
+ </tr>
+
+ <tr>
+ <td>Index Blk Elmts</td>
+ <td>Data Blk Min Elmts</td>
+ <td>Secondary Blk Min Data Ptrs</td>
+ <td>Max Data Blk Page Nelmts Bits</td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Num Secondary Blks<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Secondary Blk Size<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Num Data Blks<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Data Blk Size<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Max Index Set<sup>L</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Num Elements<sup>L</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Index Block Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;L&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfLengthsV0">Size
+ of Lengths</a> field in the superblock.)
+ </td></tr>
+ <tr>
+ <td>&nbsp;</td>
+ <td>
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Extensible Array Header
+ </caption>
+ <tr>
+ <th width="40%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>EAHD</code>&rdquo;
+ is used to indicate the beginning of an Extensible Array
+ header. This gives file consistency checking utilities a
+ better chance of reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Client ID</p></td>
+ <td>
+ <p>The ID for identifying the client of the
+ Fixed Array:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">ID</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Non-filtered dataset chunks
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Filtered dataset chunks
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2+</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Element Size</p></td>
+ <td>
+ <p>The size in bytes of an element in the Extensible Array.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Max Nelmts Bits</p></td>
+ <td>
+ <p>The number of bits needed to store the
+ maximum number of elements in the Extensible Array.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Index Blk Elmts</p></td>
+ <td>
+ <p>The number of elements to store in the index block.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data Blk Min Elmts</p></td>
+ <td>
+ <p>The minimum number of elements per data block.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Secondary Blk Min Data Ptrs</p></td>
+ <td>
+ <p>The minimum number of data block pointers for a
+ secondary block.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Max Dblk Page Nelmts Bits</p></td>
+ <td>
+ <p>The number of bits needed to store the maximum number
+ of elements in a data block page.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Num Secondary Blks</p></td>
+ <td>
+ <p>The number of secondary blocks created.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Secondary Blk Size</p></td>
+ <td>
+ <p>The size of the secondary blocks created.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Num Data Blks</p></td>
+ <td>
+ <p>The number of data blocks created.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data Blk Size</p></td>
+ <td>
+ <p>The size of the data blocks created.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Max Index Set</p></td>
+ <td>
+ <p>The maximum index set.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Num Elmts</p></td>
+ <td>
+ <p>The number of elements realized.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Index Block Address</p></td>
+ <td>
+ <p>The address of the index block.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>The checksum for the header.</p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Extensible Array Index Block
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Client ID</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted
+ only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Elements <em>(variable size and
+ optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Data Block Addresses <em>(variable
+ size and optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Secondary Block Addresses <em>(variable
+ size and optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Extensible Array Index Block
+ </caption>
+ <tr>
+ <th width="40%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>EAIB</code>&rdquo;
+ is used to indicate the beginning of an Extensible Array
+ Index Block. This gives file consistency checking utilities
+ a better chance of reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Client ID</p></td>
+ <td>
+ <p>The client ID for identifying the user of the
+ Extensible Array:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">ID</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Non-filtered dataset chunks
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Filtered dataset chunks
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2+</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Address</p></td>
+ <td>
+ <p>The address of the Extensible Array header. Principally
+ used for file integrity checking.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Elements</p></td>
+ <td>
+ <p>Contains the elements that are stored directly in
+ the index block. An optimization to avoid unnecessary
+ secondary blocks.
+ <br />
+ <br />
+ There are two element types:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">ID</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td><a href="#EaNonFilterChunk">Non-filtered dataset chunks</i></a>
+</td>
+</tr>
+<tr>
+ <td align="center"><code>1</code></td>
+ <td><a href="#EaFilterChunk">Filtered dataset chunks</i></a>
+</td>
+</tr>
+</table>
+</p>
+</td>
+</tr>
+
+<tr>
+ <td><p>Data Block Addresses</p></td>
+ <td>
+ <p>Contains the addresses of the data blocks
+ that are stored directly in the Index Block. An
+ optimization to avoid unnecessary secondary blocks.</p>
+ </td>
+</tr>
+
+<tr>
+ <td><p>Secondary Block Addresses</p></td>
+ <td>
+ <p>Contains the addresses of the secondary
+ blocks.</p>
+ </td>
+</tr>
+
+<tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>The checksum for the Extensible Array Index Block.</p>
+ </td>
+</tr>
+
+</table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Extensible Array Secondary Block
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Client ID</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted
+ only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Block Offset <em>(variable
+ size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Page Bitmap <em>(variable size and
+ optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Data Block Addresses <em>(variable
+ size and optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Extensible Array Secondary Block
+ </caption>
+ <tr>
+ <th width="40%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>EASB</code>&rdquo;
+ is used to indicate the beginning of an Extensible Array
+ Secondary Block. This gives file consistency checking utilities
+ a better chance of reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Client ID</p></td>
+ <td>
+ <p>The ID for identifying the client of the
+ Extensible Array:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">ID</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Non-filtered dataset chunks
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Filtered dataset chunks
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2+</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Address</p></td>
+ <td>
+ <p>The address of the Extensible Array header. Principally
+ used for file integrity checking.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Block Offset</p></td>
+ <td>
+ <p>Stores the offset of the block in the array.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Page Bitmap</p></td>
+ <td>
+ <p>A bitmap indicating which
+ data block pages are initialized.
+ <p>
+ Exists only if the data block is paged.
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Data Block Addresses</p></td>
+ <td>
+ <p>Contains the addresses of the data blocks
+ referenced by this secondary block.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>The checksum for the Extensible Array
+ Secondary Block.</p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Extensible Array Data Block
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Signature</td>
+ </tr>
+
+ <tr>
+ <td>Version</td>
+ <td>Client ID</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted
+ only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Header Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Block Offset <em>(variable
+ size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Elements <em>(variable size and
+ optional)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Extensible Array Data Block
+ </caption>
+ <tr>
+ <th width="40%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Signature</p></td>
+ <td>
+ <p>The ASCII character string &ldquo;<code>EADB</code>&rdquo;
+ is used to indicate the beginning of an Extensible Array
+ data block. This gives file consistency checking utilities
+ a better chance of reconstructing a damaged file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td>
+ <p>This document describes version 0.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Client ID</p></td>
+ <td>
+ <p>The ID for identifying the client of the
+ Extensible Array:
+
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">ID</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Non-filtered dataset chunks
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Filtered dataset chunks
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>2+</code></td>
+ <td>Reserved.
+ </td>
+ </tr>
+ </table>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Header Address</p></td>
+ <td>
+ <p>The address of the Extensible Array header. Principally
+ used for file integrity checking.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Block Offset</p></td>
+ <td>
+ <p>The offset of the block in the array.
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Elements</p></td>
+ <td>
+ <p>Contains the elements stored in the data block and
+ exists only if the data block is not paged.
+ <br />
+ <br />
+ There are two element types:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">ID</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td><a href="#EaNonFilterChunk">Non-filtered dataset chunks</i></a>
+</td>
+</tr>
+<tr>
+ <td align="center"><code>1</code></td>
+ <td><a href="#EaFilterChunk">Filtered dataset chunks</i></a>
+</td>
+</tr>
+</table>
+</p>
+</td>
+</tr>
+
+<tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>The checksum for the Extensible Array data block.</p>
+ </td>
+</tr>
+
+</table>
+</div>
+
+<br />
+<br />
+<br />
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Extensible Array Data Block Page
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Elements <em>(variable
+ size)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Checksum</td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Extensible Array Data Block Page
+ </caption>
+ <tr>
+ <th width="40%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Elements</p></td>
+ <td>
+ <p>Contains the elements stored in the data block
+ page.</p>
+ <p>
+ There are two element types:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">ID</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td><a href="#EaNonFilterChunk">Non-filtered dataset chunks</i></a>
+</td>
+</tr>
+<tr>
+ <td align="center"><code>1</code></td>
+ <td><a href="#EaFilterChunk">Filtered dataset chunks</i></a>
+</td>
+</tr>
+</table>
+</p>
+</td>
+</tr>
+
+<tr>
+ <td><p>Checksum</p></td>
+ <td>
+ <p>The checksum for an Extensible Array data block
+ page.</p>
+ </td>
+</tr>
+
+</table>
+</div>
+
+<br />
+<br />
+<br />
+<a name="EaNonFilterChunk"></a>
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Data Block Element for Non-filtered Dataset Chunk
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr><td>
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Data Block Element for Non-filtered Dataset Chunk
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>The address of the dataset chunk in the file.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+</p>
+
+<br />
+<br />
+<br />
+<a name="EaFilterChunk"></a>
+<div align="center">
+ <table class="format">
+ <caption>
+ Layout: Data Block Element for Filtered Dataset Chunk
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Chunk Size<em> (variable size; at
+ most 8 bytes)</em><br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Filter Mask</td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+</div>
+
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Data Block Element for Filtered Dataset Chunk
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Address</p></td>
+ <td><p>The address of the dataset chunk in the file.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Chunk Size</p></td>
+ <td><p>The size of the dataset chunk in bytes.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Filter Mask</p></td>
+ <td><p>Indicates the filter to skip for the dataset chunk.
+ Each filter has an index number in the pipeline; if that
+ filter is skipped, the bit corresponding to its index is set.
+ </p>
+ </td>
+ </tr>
+
+ </table>
+</div>
+
+<a name="AppendV2Btrees">
+ <h3>VII.E. The Version 2 B-trees Index</h3></a>
+
+<p>The <i>Version 2 B-trees</i> index can be used when the dataset
+ fulfills the following condition:</p>
+
+<ul>
+ <li>more than one dimension of unlimited extent</li>
+</ul>
+
+<p>Version 2 B-trees can be used to index various objects in the
+ library. See <a href="#V2Btrees">&ldquo;Version 2 B-trees&rdquo;</a>
+ for more information. The B-tree types <a href="#V2BtType10">10</a>
+ and <a href="#V2BtreesType11">11</a> record layouts are for
+ indexing dataset chunks.</p>
+
+<h2><a name="AppendixD"> VIII. Appendix D:
+ Encoding for dataspace and reference</a></h2>
+
+<a name="DataspaceEncode">
+ <h3>VIII.A. Dataspace Encoding </h3></a>
+<i>H5Sencode</i> is a public routine that encodes a dataspace description into a buffer while
+<i>H5Sdecode</i> is the corresponding routine that decodes the description encoded in the buffer.
+<p>
+ See the reference manual description for these two public routines.
+
+ <br />
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Dataspace Description for H5Sencode/H5Sdecode
+ </caption>
+
+ <tr>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ <th>byte</th>
+ </tr>
+
+ <tr>
+ <td>Dataspace ID</td>
+ <td>Encode Version</td>
+ <td>Size of Size</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted
+ only to align table nicely</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Size of Extent
+ <br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br /><br />Dataspace Message
+ <em>(variable size)</em>
+ <br /><br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br /><br />Dataspace Selection
+ <em>(variable size)</em>
+ <br /><br /></td>
+ </tr>
+
+ </table>
+
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Dataspace Description for H5Sencode/H5Sdecode
+ </caption>
+ <tr>
+ <th width="40%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Dataspace ID</p></td>
+ <td>
+ <p>The datspace message ID which is 1.</p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Encode Version</p></td>
+ <td>
+ <p>H5S_ENCODE_VERSION which is 0.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Size</p></td>
+ <td>
+ <p>The number of bytes used to store the size of an object.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Size of Extent</p></td>
+ <td>
+ <p>Size of the dataspace message.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dataspace Message</p></td>
+ <td>
+ <p>The dataspace message information. See
+ <a href="#DataspaceMessage">Dataspace Message.</a></p>
+</p>
+</td>
+</tr>
+
+<tr>
+ <td><p>Dataspace Selection</p></td>
+ <td>
+ <p>The dataspace selection information. See
+ <a href="#DataspaceSEL">Dataspace Selection.</a></p>
+ </td>
+</tr>
+
+</table>
+</div>
+
+
+<br />
+<br />
+<br />
+<a name="DataspaceSEL"></a>
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Dataspace Selection
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Selection Type</td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Selection Info (<em>variable
+ size</em>)<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Dataspace Selection
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Selection Type</p></td>
+ <td>
+ <p>There are 4 types of selection:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>H5S_SEL_NONE: Nothing selected
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>H5S_SEL_POINTS: Sequence of points selected
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>H5S_SEL_HYPER: Hyperslab selected
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>H5S_SEL_ALL: Entire extent selected
+ </td>
+ </tr>
+ </table>
+ </td>
+
+ </tr>
+
+ <tr>
+ <td><p>Selection Info</p></td>
+ <td>
+ <p>There are 4 types of selection info:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>Selection info for <a href="#SelNONE">H5S_SEL_NONE</a>
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>Selection info for <a href="#SelPOINTS">H5S_SEL_POINTS</a>
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>Selection info for <a href="#SelHYPER">H5S_SEL_HYPER</a>
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>Selection for <a href="#SelALL">H5S_SEL_ALL</a>
+ </td>
+ </tr>
+ </table>
+ </td>
+
+ </tr>
+ </table>
+ </div>
+
+
+ <br />
+ <br />
+ <br />
+<a name="SelNONE"/></a>
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Selection Info for H5S_SEL_NONE
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Version</td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Reserved <em>(zero, 8 bytes)</em><br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Selection Info for H5S_SEL_NONE
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for the H5S_SEL_NONE Selection Info.
+ The value is 1.</p></td>
+ </tr>
+ </table>
+ </div>
+
+
+ <br />
+ <br />
+ <br />
+ <a name="SelPOINTS"></a>
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Selection Info for H5S_SEL_POINTS
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Version</td>
+ </tr>
+ <tr>
+ <td colspan="4"><br /><br />Points Selection Info <em>(variable size)</em>
+ <br /><br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Selection Info for H5S_SEL_POINTS
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for the H5S_SEL_POINTS Selection Info.
+ The value is either 1 or 2.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Points Selection Info</p></td>
+ <td><p>Depending on <em>version</em>:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>See <a href="#SelPOINTSV1">Version 1 Points Selection Info</a>
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>See <a href="#SelPOINTSV2">Version 2 Points Selection Info</a>
+ </td>
+ </tr>
+
+ </table>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <br />
+ <a name="SelPOINTSV1"></a>
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 1 Points Selection Info
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reserved <em>(zero)</em></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Length</td>
+ </tr>
+ <tr>
+ <td colspan="4">Rank</td>
+ </tr>
+ <tr>
+ <td colspan="4">Num Points</td>
+ </tr>
+ <tr>
+ <td colspan="4">Point #1: coordinate #1</td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Point #1: coordinate #u</td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Point #n: coordinate #1</td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Point #n: coordinate #u</td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 1 Points Selection Info
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Length</p></td>
+ <td><p>The size in bytes from <em>Length</em> to the end of the
+ selection info.</td>
+ </tr>
+
+ <tr>
+ <td><p>Rank</p></td>
+ <td><p>The number of dimensions.</p></td>
+ </tr>
+ <tr>
+ <td><p>Num Points</p></td>
+ <td><p>The number of points in the selection.</p></td>
+ </tr>
+ <tr>
+ <td><p>Point #n: coordinate #u</p></td>
+ <td><p>The array of points in the selection.
+ <p>The points selected are #1 to #n where n is <em>Num Points</em>.
+ <p>The list of coordinates for each point are #1 to #u where u is
+ <em>Rank</em>.</p></td>
+ </tr>
+ </table>
+ </div>
+
+
+ <br />
+ <br />
+ <br />
+ <a name="SelPOINTSV2"></a>
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 Points Selection Info
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="1">Encode Size</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted only to align table nicely</em>
+ </td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Rank</td>
+ </tr>
+ <tr>
+ <td colspan="4">Num Points<p>(2, 4 or 8 bytes)<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Point #1: coordinate #1<p>(2, 4 or 8 bytes)<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Point #1: coordinate #u<p>(2, 4 or 8 bytes)<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Point #n: coordinate #1 <p>(2, 4 or 8 bytes)<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Point #n: coordinate #u<p>(2, 4 or 8 bytes)<br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 Points Selection Info
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Encode Size</td>
+ <td><p>The size for encoding the points selection info which can be 2, 4 or 8 bytes.
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Rank</p></td>
+ <td><p>The number of dimensions.</p></td>
+ </tr>
+ <tr>
+ <td><p>Num Points</p></td>
+ <td><p>The number of points in the selection.
+ <p>The field <em>Encode Size</em> indicates the size of this field</p></td>
+ </tr>
+ <tr>
+ <td><p>Point #n: coordinate #u</p></td>
+ <td><p>The array of points in the selection.
+ <p>The points selected are #1 to #n where n is <em>Num Points</em>.
+ <p>The list of coordinates for each point are #1 to #u where u is
+ <em>Rank</em>.
+ <p>The field <em>Encode Size</em> indicates the size of this field</p></td>
+ </tr>
+ </table>
+ </div>
+
+
+ <br />
+ <br />
+ <br />
+ <a name="SelHYPER"></a>
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Selection Info for H5S_SEL_HYPER
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Version</td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Hyperslab Selection Info
+ (<em>variable size</em>)<br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Selection Info for H5S_SEL_HYPER
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for the H5S_SEL_HYPER selection info.
+ The value is 1, 2 or 3.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Hyperslab Selection Info</p></td>
+ <td><p>Depending on <em>version</em>:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Version</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>1</code></td>
+ <td>See <a href="#SelHYPERV1">Version 1 Hyperslab Selection Info</a>.
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>See <a href="#SelHYPERV2">Version 2 Hyperslab Selection Info</a>
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>See <a href="#SelHYPERV3">Version 3 Hyperslab Selection Info</a>
+ </td>
+ </tr>
+ </table>
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <br />
+ <a name="SelHYPERV1"></a>
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 1 Hyperslab Selection Info
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Reserved</td>
+ </tr>
+ <tr>
+ <td colspan="4">Length</td>
+ </tr>
+ <tr>
+ <td colspan="4">Rank</td>
+ </tr>
+ <tr>
+ <td colspan="4">Num Blocks</td>
+ </tr>
+ <tr>
+ <td colspan="4">Starting Offset #1 for Block #1</td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Starting Offset #n for Block #1</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Ending Offset #1 for Block #1</td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Ending Offset #n for Block #1</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Starting Offset #1 for Block #u</td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Starting Offset #n for Block #u</td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Ending Offset #1 for Block #u</em></td>
+</tr>
+<tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+</tr>
+<tr>
+ <td colspan="4">Ending Offset #n for Block #u</td>
+</tr>
+
+</table>
+</div>
+
+<br />
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 1 Hyperslab Selection Info
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Length</p></td>
+ <td><p>The size in bytes from the field <em>Rank</em> to the
+ end of the Selection Info.</td>
+ </tr>
+
+ <tr>
+ <td><p>Rank</p></td>
+ <td><p>The number of dimensions in the dataspace.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Num Blocks</p></td>
+ <td><p>The number of blocks in the selection.</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Starting Offset #n for Block #u</p></td>
+ <td><p>The offset #n of the starting element in block #u.
+ <p>#n is from 1 to <em>Rank</em>.
+ <p>#u is from 1 to <em>Num Blocks</em> moving from the fastest
+ changing dimension to the slowest changing dimension.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Ending Offset #n for Block #u</p></td>
+ <td><p>The offset #n of the ending element in block #u.
+ <p>#n is from 1 to <em>Rank</em>.
+ <p>#u is from 1 to <em>Num Blocks</em> moving from the fastest
+ changing dimension to the slowest changing dimension.
+ </p></td>
+ </tr>
+
+ </table>
+</div>
+
+<br />
+<br />
+<br />
+<a name="SelHYPERV2"></a>
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 2 Hyperslab Selection Info
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Flags</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted
+ only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Length</td>
+ </tr>
+ <tr>
+ <td colspan="4">Rank</td>
+ </tr>
+ <tr>
+ <td colspan="4">Start #1 <em>(8 bytes)</em><p></td>
+ </tr>
+ <tr>
+ <td colspan="4">Stride #1 <em>(8 bytes)</em><p></td>
+ </tr>
+ <tr>
+ <td colspan="4">Count #1 <em>(8 bytes)</em><p></td>
+ </tr>
+ <tr>
+ <td colspan="4">Block #1 <em>(8 bytes)</em><p></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Start #n <em>(8 bytes)</em><p></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Stride #n <em>(8 bytes)</em><p></td>
+ </tr>
+ <tr>
+ <td colspan="4">Count #n <em>(8 bytes)</em><p></td>
+ </tr>
+ <tr>
+ <td colspan="4">Block #n <em>(8 bytes)</em><p></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 2 Hyperslab Selection Info
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>This is a bit field with the following definition.
+ Currently, this is always set to 0x1.
+ <p>
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, it a a regular hyperslab, otherwise, irregular.
+ </td>
+ </tr>
+
+ </table>
+ </td>
+
+ </tr>
+
+ <tr>
+ <td><p>Length</p></td>
+ <td><p>The size in bytes from the field <em>Rank</em> to the
+ end of the Selection Info.</td>
+ </tr>
+
+ <tr>
+ <td><p>Rank</p></td>
+ <td><p>The number of dimensions in the dataspace.</td>
+ </tr>
+
+ <tr>
+ <td><p>Start #n</p></td>
+ <td><p>The offset of the starting element in the block.
+ <p>#n is from 1 to <em>Rank</em>.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Stride #n</p></td>
+ <td><p>The number of elements to move in each dimension.
+ <p>#n is from 1 to <em>Rank</em>.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Count #n</p></td>
+ <td><p>The number of blocks to select in each dimension.
+ <p>#n is from 1 to <em>Rank</em>.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Block #n</p></td>
+ <td><p>The size (in elements) of each block in each dimension.
+ <p>#n is from 1 to <em>Rank</em>.
+ </p></td>
+ </tr>
+ </table>
+ </div>
+
+
+
+
+ <br />
+ <br />
+ <br />
+ <a name="SelHYPERV3"></a>
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 3 Hyperslab Selection Info
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Flags</td>
+ <td>Encode Size</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted
+ only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan="4">Rank</td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Regular/Irregular Hyperslab Selection Info
+ <p><em>(variable size)</em><br /><br/></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 3 Hyperslab Selection Info
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>This is a bit field with the following definition:
+ <p>
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, it is a regular hyperslab, otherwise, irregular.
+ </td>
+ </tr>
+
+ </table>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Encode Size</p></td>
+ <td><p>The size for encoding hyperslab selection info, which can 2, 4 or 8 bytes.</td>
+ </tr>
+
+ <tr>
+ <td><p>Rank</p></td>
+ <td><p>The number of dimensions in the dataspace.</td>
+ </tr>
+
+ <tr>
+ <td><p>Regular/Irregular Hyperslab Selection Info</p></td>
+ <td><p>This is the selection info for version 3 hyperslab which can be regular or irregular.
+ <p>If bit 0 of the field <em>Flags</em> is set,
+ See <a href="#SelHYPERV3REG">Version 3 Regular Hyperslab Selection Info</a>
+ <p>Otherwise, see <a href="#SelHYPERV3IRREG">Version 3 Irregular Hyperslab Selection Info</a>
+ </td>
+
+ </tr>
+
+ </table>
+ </div>
+
+
+ <br />
+ <br />
+ <br />
+ <a name="SelHYPERV3REG"></a>
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 3 Regular Hyperslab Selection Info
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Start #1 <p><em>(2, 4 or 8 bytes)</em><p></td>
+ </tr>
+ <tr>
+ <td colspan="4">Stride #1 <p><em>(2, 4 or 8 bytes)</em><p></td>
+ </tr>
+ <tr>
+ <td colspan="4">Count #1 <p><em>(2, 4 or 8 bytes)</em><p></td>
+ </tr>
+ <tr>
+ <td colspan="4">Block #1 <p><em>(2, 4 or 8 bytes)</em><p></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Start #n <p><em>(2, 4 or 8 bytes)</em><p></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Stride #n <p><em>(2, 4 or 8 bytes)</em><p></td>
+ </tr>
+ <tr>
+ <td colspan="4">Count #n <p><em>(2, 4 or 8 bytes)</em><p></td>
+ </tr>
+ <tr>
+ <td colspan="4">Block #n <p><em>(2, 4 or 8 bytes)</em><p></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 3 Regular Hyperslab Selection Info
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Start #n</p></td>
+ <td><p>The offset of the starting element in the block.
+ <p>#n is from 1 to <em>Rank</em>.
+ <p>The field <em>Encode Size</em> indicates the size of this field.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Stride #n</p></td>
+ <td><p>The number of elements to move in each dimension.
+ <p>#n is from 1 to <em>Rank</em>.
+ <p>The field <em>Encode Size</em> indicates the size of this field.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Count #n</p></td>
+ <td><p>The number of blocks to select in each dimension.
+ <p>#n is from 1 to <em>Rank</em>.
+ <p>The field <em>Encode Size</em> indicates the size of this field.
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Block #n</p></td>
+ <td><p>The size (in elements) of each block in each dimension.
+ <p>#n is from 1 to <em>Rank</em>.
+ <p>The field <em>Encode Size</em> indicates the size of this field.
+ </p></td>
+ </tr>
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <br />
+ <a name="SelHYPERV3IRREG"></a>
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Version 3 Irregular Hyperslab Selection Info
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Num Blocks<p><em>(2, 4 or 8 bytes)</em><p></td>
+ </tr>
+ <tr>
+ <td colspan="4">Starting Offset #1 for Block #1<p><em>(2, 4 or 8 bytes)</em><p></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Starting Offset #n for Block #1<p><em>(2, 4 or 8 bytes)</em><p></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Ending Offset #1 for Block #1<p><em>(2, 4 or 8 bytes)</em><p></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Ending Offset #n for Block #1<p><em>(2, 4 or 8 bytes)</em><p></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Starting Offset #1 for Block #u<p><em>(2, 4 or 8 bytes)</em><p></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Starting Offset #n for Block #u<p><em>(2, 4 or 8 bytes)</em><p></td>
+ </tr>
+
+ <tr>
+ <td colspan="4">Ending Offset #1 for Block #u<p><em>(2, 4 or 8 bytes)</em><p></td>
+ </tr>
+ <tr>
+ <td colspan="4">.<br />.<br />.<br /></td>
+ </tr>
+ <tr>
+ <td colspan="4">Ending Offset #n for Block #u<p><em>(2, 4 or 8 bytes)</em><p></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Version 3 Irregular Hyperslab Selection Info
+ </caption>
+
+ <tr>
+ <td><p>Num Blocks</p></td>
+ <td><p>The number of blocks in the selection.
+ <p>The field <em>Encode Size</em> indicates the size of this field</p></td>
+ </tr>
+
+ <tr>
+ <td><p>Starting Offset #n for Block #u</p></td>
+ <td><p>The offset #n of the starting element in block #u.
+ <p>#n is from 1 to <em>Rank</em>.
+ <p>#u is from 1 to <em>Num Blocks</em> moving from the fastest
+ changing dimension to the slowest changing dimension.
+ <p>The field <em>Encode Size</em> indicates the size of this field
+ </p></td>
+ </tr>
+
+ <tr>
+ <td><p>Ending Offset #n for Block #u</p></td>
+ <td><p>The offset #n of the ending element in block #u.
+ <p>#n is from 1 to <em>Rank</em>.
+ <p>#u is from 1 to <em>Num Blocks</em> moving from the fastest
+ changing dimension to the slowest changing dimension.
+ <p>The field <em>Encode Size</em> indicates the size of this field
+ </p></td>
+ </tr>
+
+ </table>
+ </div>
+
+
+ <br />
+ <br />
+ <br />
+ <a name="SelALL"></a>
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Selection Info for H5S_SEL_ALL
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4">Version</td>
+ </tr>
+ <tr>
+ <td colspan="4"><br />Reserved <em>(zero,
+ 8 bytes)</em><br /><br /></td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Selection Info for H5S_SEL_ALL
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Version</p></td>
+ <td><p>The version number for the H5S_SEL_ALL Selection Info;
+ the value is 1.</p></td>
+ </tr>
+ </table>
+ </div>
+
+ <a name="ReferenceEncodeRV">
+ <h3>VIII.B. Reference Encoding (Revised)</h3></a>
+ <p>
+ <br />
+ For the following reference type,
+ the Reference Header and Reference Block are stored together as the dataset's raw data:
+ <ul>
+ <li>Object Reference (H5R_OBJECT2) (without reference to an external file)</li>
+ </ul>
+ <p>
+ For the following reference types,
+ the Reference Header plus the <a href="#GlobalHeapID">Global Heap ID</a> are stored
+ as the dataset's raw data in the file.
+ The global heap ID is used to locate the Reference Block stored in the global heap:
+ <ul>
+ <li>Object Reference (H5R_OBJECT2) (with reference to an external file)</li>
+ <li>Dataset Region Reference (H5R_DATASET_REGION2) (with/without reference to an external file)</li>
+ <li>Attribute Reference (H5R_ATTR) (with/without reference to an external file)</li>
+ </ul>
+ <br />
+ <br />
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Reference Header
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Reference Type</td>
+ <td>Flags</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted
+ only to align table nicely</em></td>
+ </tr>
+
+ </table>
+
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Reference Header
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Reference Type</p></td>
+ <td>
+ <p>There are 3 types of references:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Value</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>2</code></td>
+ <td>H5R_OBJECT2: Object Reference
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>3</code></td>
+ <td>H5R_DATASET_REGION2: Dataset Region Reference
+ </td>
+ </tr>
+
+ <tr>
+ <td align="center"><code>4</code></td>
+ <td>H5R_ATTR: Attribute Reference
+ </td>
+ </tr>
+
+ </table>
+
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Flags</p></td>
+ <td><p>This field describes the reference:
+ <table class="list">
+ <tr>
+ <th width="20%" align="center">Bit</th>
+ <th width="80%" align="left">Description</th>
+ </tr>
+
+ <tr>
+ <td align="center"><code>0</code></td>
+ <td>If set, the reference is to an external file.
+ </td>
+ </tr>
+ <tr>
+ <td align="center"><code>1-7</code></td>
+ <td>Reserved</td>
+ </tr>
+ </table></p>
+
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <br />
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Reference Block
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td>Token Size</td>
+ <td colspan="3" bgcolor="#DDDDDD"><em>This space inserted
+ only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan=4><br /><br />Token
+ <em>(variable size)</em><br /> <br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan=2>Length of External File Name</td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted
+ only to align table nicely</em></td>
+ </tr>
+ <tr>
+ <td colspan=4><br /><br />External File Name
+ <em>(variable size)</em><br /><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan=4>Size of Dataspace Selection</td>
+ </tr>
+ <tr>
+ <td colspan=4>Rank of Dataspace Selection</td>
+ </tr>
+ <tr>
+ <td colspan=4><br /><br />Dataspace Selection Information
+ <em>(variable size)</em><br /><br /> <br /></td>
+</td>
+</tr>
+<tr>
+ <td colspan=2>Length of Attribute Name </td>
+ <td colspan="2" bgcolor="#DDDDDD"><em>This space inserted
+ only to align table nicely</em></td>
+</tr>
+<tr>
+ <td colspan=4><br /><br />Attribute Name
+ <em>(variable size)</em><br /><br /><br /></td>
+</tr>
+
+</table>
+
+</div>
+
+<br />
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Reference Block
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Token size</p></td>
+ <td><p>This is the size of the token for the object.
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Token</p></td>
+ <td>
+ <p>
+ This is the token for the object.
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Length fo External File Name</p></td>
+ <td><p>This is the length for the external file name.
+ <p>This field exists if bit 0 of <em>flags</em> is set.</p>
+ </p>
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>External File Name</p></td>
+ <td><p>This is the name of the external file being referenced.</p>
+</p>
+<p>This field exists if bit 0 of <em>flags</em> is set.</p>
+</td>
+</tr>
+
+<tr>
+ <td><p>Dataspace Selection Information</p></td>
+ <td><p>See <a href="#DataspaceSEL">Dataspace Selection.</a></p>
+</p>
+<p>This field exists if the <em>Reference Type</em> is H5R_DATASET_REGION2.</p>
+</td>
+</tr>
+
+<tr>
+ <td><p>Length of Attribute Name</p></td>
+ <td><p>This is the length of the attribute name.
+ <p>This field exists if the <em>Reference Type</em> is H5R_ATTRIBUTE.</p>
+ </td>
+</tr>
+
+<tr>
+ <td><p>Attribute Name</p></td>
+ <td><p>This is the name of the attribute being referenced.
+ <p>This field exists if the <em>Reference Type</em> is H5R_ATTRIBUTE.</p>
+ </td>
+</tr>
+
+</table>
+</div>
+
+<br />
+<br />
+<br />
+
+
+<a name="ReferenceEncodeDP">
+ <h3>VIII.C. Reference Encoding (Backward Compatibility)</h3></a>
+<p>
+ <br />
+ The two references described below are maintained to preserve compatibility with previous versions of the library.
+<p>
+ For the following reference type,
+ the reference encoding is stored as the dataset's raw data in the file:
+ <ul>
+ <li>Object Reference (H5R_OBJECT1)</li>
+ </ul>
+<p>
+ For the following reference type,
+ the <a href="#GlobalHeapID">Global Heap ID</a> is stored as the dataset's raw data in the file.
+ The global heap ID is used to locate the reference encoding
+ stored in the global heap:
+ <ul>
+ <li>Dataset Region Reference (H5R_DATASET_REGION1)</li>
+ </ul>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Reference for H5R_OBJECT1
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Object Address<sup>O</sup><br /><br /></td>
+ </tr>
+
+ </table>
+
+ <table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+ </table>
+
+ </div>
+
+ <br />
+ <br />
+ <div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Reference for H5R_OBJECT1
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Object Address</p></td>
+ <td>
+ <p>Address of the object being referenced
+ </td>
+ </tr>
+
+ </table>
+ </div>
+
+ <br />
+ <br />
+ <br />
+
+ <div align="center">
+ <table class="format">
+ <caption>
+ Layout: Reference for H5R_DATASET_REGION1
+ </caption>
+
+ <tr>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ <th width="25%">byte</th>
+ </tr>
+
+ <tr>
+ <td colspan="4"><br />Object Address<sup>O</sup><br /><br /></td>
+ </tr>
+ <tr>
+ <td colspan=4><br /><br />Dataspace Selection Information
+ <em>(variable size)</em><br /><br /> <br /></td>
+</td>
+</tr>
+
+</table>
+
+<table class="note">
+ <tr>
+ <td width="60%">&nbsp;</td>
+ <td width="40%">
+ (Items marked with an &lsquo;O&rsquo; in the above table are
+ of the size specified in the <a href="#SizeOfOffsetsV0">Size
+ of Offsets</a> field in the superblock.)
+ </td></tr>
+</table>
+
+</div>
+
+<br />
+<br />
+<div align="center">
+ <table class="desc">
+ <caption>
+ Fields: Reference for H5R_DATASET_REGION1
+ </caption>
+ <tr>
+ <th width="30%">Field Name</th>
+ <th>Description</th>
+ </tr>
+
+ <tr>
+ <td><p>Object Address</p></td>
+ <td><p>This is the address of the object being referenced.
+ </td>
+ </tr>
+
+ <tr>
+ <td><p>Dataspace Selection Information</p></td>
+ <td><p>This is the dataspace selection for the object being referenced.
+ See <a href="#DataspaceSEL">Dataspace Selection.</a></p>
+</p>
+</td>
+</tr>
+
+</table>
+</div>
+
+<br />
+<br />
+<br />
+
+
+</body>
+</html>
diff --git a/doxygen/examples/H5A_examples.c b/doxygen/examples/H5A_examples.c
new file mode 100644
index 0000000..f332efa
--- /dev/null
+++ b/doxygen/examples/H5A_examples.c
@@ -0,0 +1,145 @@
+/* -*- c-file-style: "stroustrup" -*- */
+
+#include "hdf5.h"
+
+#include <stdio.h>
+#include <stdlib.h>
+
+int
+main(void)
+{
+ int ret_val = EXIT_SUCCESS;
+
+ //! <!-- [create] -->
+ {
+ __label__ fail_acpl, fail_attr, fail_file;
+ hid_t file, acpl, fspace, attr;
+
+ unsigned mode = H5F_ACC_TRUNC;
+ char file_name[] = "f1.h5";
+ // attribute names can be arbitrary Unicode strings
+ char attr_name[] = "Χαρακτηριστικό";
+
+ if ((file = H5Fcreate(file_name, mode, H5P_DEFAULT, H5P_DEFAULT)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_file;
+ }
+ if ((acpl = H5Pcreate(H5P_ATTRIBUTE_CREATE)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_acpl;
+ }
+ // use UTF-8 encoding for the attribute name
+ if (H5Pset_char_encoding(acpl, H5T_CSET_UTF8) < 0) {
+ ret_val = EXIT_FAILURE;
+ goto fail_fspace;
+ }
+ // create a scalar (singleton) attribute
+ if ((fspace = H5Screate(H5S_SCALAR)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_fspace;
+ }
+ // create an attribute on the root group
+ if ((attr = H5Acreate2(file, attr_name, H5T_STD_I32LE, fspace, acpl, H5P_DEFAULT)) ==
+ H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_attr;
+ }
+
+ H5Aclose(attr);
+fail_attr:
+ H5Sclose(fspace);
+fail_fspace:
+ H5Pclose(acpl);
+fail_acpl:
+ H5Fclose(file);
+fail_file:;
+ }
+ //! <!-- [create] -->
+
+ //! <!-- [read] -->
+ {
+ __label__ fail_attr, fail_file;
+ hid_t file, attr;
+
+ unsigned mode = H5F_ACC_RDONLY;
+ char file_name[] = "f1.h5";
+ char attr_name[] = "Χαρακτηριστικό";
+ int value;
+
+ if ((file = H5Fopen(file_name, mode, H5P_DEFAULT)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_file;
+ }
+ if ((attr = H5Aopen(file, attr_name, H5P_DEFAULT)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_attr;
+ }
+ // read the attribute value
+ if (H5Aread(attr, H5T_NATIVE_INT, &value) < 0)
+ ret_val = EXIT_FAILURE;
+
+ // do something w/ the attribute value
+
+ H5Aclose(attr);
+fail_attr:
+ H5Fclose(file);
+fail_file:;
+ }
+ //! <!-- [read] -->
+
+ //! <!-- [update] -->
+ {
+ __label__ fail_attr, fail_file;
+ hid_t file, attr;
+
+ unsigned mode = H5F_ACC_RDWR;
+ char file_name[] = "f1.h5";
+ char attr_name[] = "Χαρακτηριστικό";
+ int value = 1234;
+
+ if ((file = H5Fopen(file_name, mode, H5P_DEFAULT)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_file;
+ }
+ if ((attr = H5Aopen(file, attr_name, H5P_DEFAULT)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_attr;
+ }
+ // update the attribute value
+ if (H5Awrite(attr, H5T_NATIVE_INT, &value) < 0)
+ ret_val = EXIT_FAILURE;
+
+ H5Aclose(attr);
+fail_attr:
+ H5Fclose(file);
+fail_file:;
+ }
+ //! <!-- [update] -->
+
+ //! <!-- [delete] -->
+ {
+ __label__ fail_attr, fail_file;
+ hid_t file;
+
+ unsigned mode = H5F_ACC_RDWR;
+ char file_name[] = "f1.h5";
+ char attr_name[] = "Χαρακτηριστικό";
+
+ if ((file = H5Fopen(file_name, mode, H5P_DEFAULT)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_file;
+ }
+ // delete the attribute
+ if (H5Adelete(file, attr_name) < 0) {
+ ret_val = EXIT_FAILURE;
+ goto fail_attr;
+ }
+
+fail_attr:
+ H5Fclose(file);
+fail_file:;
+ }
+ //! <!-- [delete] -->
+
+ return ret_val;
+}
diff --git a/doxygen/examples/H5D_examples.c b/doxygen/examples/H5D_examples.c
new file mode 100644
index 0000000..aad057d
--- /dev/null
+++ b/doxygen/examples/H5D_examples.c
@@ -0,0 +1,173 @@
+/* -*- c-file-style: "stroustrup" -*- */
+
+#include "hdf5.h"
+
+#include <stdio.h>
+#include <stdlib.h>
+
+int
+main(void)
+{
+ int ret_val = EXIT_SUCCESS;
+
+ //! <!-- [create] -->
+ {
+ __label__ fail_lcpl, fail_dset, fail_file;
+ hid_t file, lcpl, fspace, dset;
+
+ unsigned mode = H5F_ACC_TRUNC;
+ char file_name[] = "d1.h5";
+ // link names can be arbitrary Unicode strings
+ char dset_name[] = "σύνολο/δεδομένων";
+
+ if ((file = H5Fcreate(file_name, mode, H5P_DEFAULT, H5P_DEFAULT)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_file;
+ }
+ if ((lcpl = H5Pcreate(H5P_LINK_CREATE)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_lcpl;
+ }
+ // use UTF-8 encoding for link names
+ if (H5Pset_char_encoding(lcpl, H5T_CSET_UTF8) < 0) {
+ ret_val = EXIT_FAILURE;
+ goto fail_fspace;
+ }
+ // create intermediate groups as needed
+ if (H5Pset_create_intermediate_group(lcpl, 1) < 0) {
+ ret_val = EXIT_FAILURE;
+ goto fail_fspace;
+ }
+ // create a 1D dataspace
+ if ((fspace = H5Screate_simple(1, (hsize_t[]){10}, NULL)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_fspace;
+ }
+ // create a 32-bit integer dataset
+ if ((dset = H5Dcreate2(file, dset_name, H5T_STD_I32LE, fspace, lcpl, H5P_DEFAULT, H5P_DEFAULT)) ==
+ H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_dset;
+ }
+
+ H5Dclose(dset);
+fail_dset:
+ H5Sclose(fspace);
+fail_fspace:
+ H5Pclose(lcpl);
+fail_lcpl:
+ H5Fclose(file);
+fail_file:;
+ }
+ //! <!-- [create] -->
+
+ //! <!-- [read] -->
+ {
+ __label__ fail_dset, fail_file;
+ hid_t file, dset;
+
+ unsigned mode = H5F_ACC_RDONLY;
+ char file_name[] = "d1.h5";
+ // assume a priori knowledge of dataset name and size
+ char dset_name[] = "σύνολο/δεδομένων";
+ int elts[10];
+
+ if ((file = H5Fopen(file_name, mode, H5P_DEFAULT)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_file;
+ }
+ if ((dset = H5Dopen2(file, dset_name, H5P_DEFAULT)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_dset;
+ }
+ // read all dataset elements
+ if (H5Dread(dset, H5T_NATIVE_INT, H5S_ALL, H5S_ALL, H5P_DEFAULT, elts) < 0)
+ ret_val = EXIT_FAILURE;
+
+ // do something w/ the dataset elements
+
+ H5Dclose(dset);
+fail_dset:
+ H5Fclose(file);
+fail_file:;
+ }
+ //! <!-- [read] -->
+
+ //! <!-- [update] -->
+ {
+ __label__ fail_update, fail_fspace, fail_dset, fail_file;
+ hid_t file, dset, fspace;
+
+ unsigned mode = H5F_ACC_RDWR;
+ char file_name[] = "d1.h5";
+ char dset_name[] = "σύνολο/δεδομένων";
+ int new_elts[6][2] = {{-1, 1}, {-2, 2}, {-3, 3}, {-4, 4}, {-5, 5}, {-6, 6}};
+
+ if ((file = H5Fopen(file_name, mode, H5P_DEFAULT)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_file;
+ }
+ if ((dset = H5Dopen2(file, dset_name, H5P_DEFAULT)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_dset;
+ }
+ // get the dataset's dataspace
+ if ((fspace = H5Dget_space(dset)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_fspace;
+ }
+ // select the first 5 elements in odd positions
+ if (H5Sselect_hyperslab(fspace, H5S_SELECT_SET, (hsize_t[]){1}, (hsize_t[]){2}, (hsize_t[]){5},
+ NULL) < 0) {
+ ret_val = EXIT_FAILURE;
+ goto fail_update;
+ }
+
+ // (implicitly) select and write the first 5 elements of the second column of NEW_ELTS
+ if (H5Dwrite(dset, H5T_NATIVE_INT, H5S_ALL, fspace, H5P_DEFAULT, new_elts) < 0)
+ ret_val = EXIT_FAILURE;
+
+fail_update:
+ H5Sclose(fspace);
+fail_fspace:
+ H5Dclose(dset);
+fail_dset:
+ H5Fclose(file);
+fail_file:;
+ }
+ //! <!-- [update] -->
+
+ //! <!-- [delete] -->
+ {
+ __label__ fail_delete, fail_file;
+ hid_t file;
+
+ unsigned mode = H5F_ACC_RDWR;
+ char file_name[] = "d1.h5";
+ char group_name[] = "σύνολο";
+ char dset_name[] = "σύνολο/δεδομένων";
+
+ if ((file = H5Fopen(file_name, mode, H5P_DEFAULT)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_file;
+ }
+ // delete (unlink) the dataset
+ if (H5Ldelete(file, dset_name, H5P_DEFAULT) < 0) {
+ ret_val = EXIT_FAILURE;
+ goto fail_delete;
+ }
+ // the previous call deletes (unlinks) only the dataset
+ if (H5Ldelete(file, group_name, H5P_DEFAULT) < 0) {
+ ret_val = EXIT_FAILURE;
+ goto fail_delete;
+ }
+
+fail_delete:
+ H5Fclose(file);
+fail_file:;
+ }
+
+ //! <!-- [delete] -->
+
+ return ret_val;
+}
diff --git a/doxygen/examples/H5F_examples.c b/doxygen/examples/H5F_examples.c
new file mode 100644
index 0000000..a7ce6fb
--- /dev/null
+++ b/doxygen/examples/H5F_examples.c
@@ -0,0 +1,187 @@
+/* -*- c-file-style: "stroustrup" -*- */
+
+#include "hdf5.h"
+
+#include <stdio.h>
+#include <stdlib.h>
+
+int
+main(void)
+{
+ int ret_val = EXIT_SUCCESS;
+
+ //! <!-- [life_cycle] -->
+ {
+ __label__ fail_fapl, fail_fcpl, fail_file;
+ hid_t fcpl, fapl, file;
+
+ if ((fcpl = H5Pcreate(H5P_FILE_CREATE)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_fcpl;
+ }
+ else {
+ // adjust the file creation properties
+ }
+
+ if ((fapl = H5Pcreate(H5P_FILE_ACCESS)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_fapl;
+ }
+ else {
+ // adjust the file access properties
+ }
+
+ unsigned mode = H5F_ACC_EXCL;
+ char name[] = "f1.h5";
+
+ if ((file = H5Fcreate(name, mode, fcpl, fapl)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_file;
+ }
+
+ // do something useful with FILE
+
+ H5Fclose(file);
+fail_file:
+ H5Pclose(fapl);
+fail_fapl:
+ H5Pclose(fcpl);
+fail_fcpl:;
+ }
+ //! <!-- [life_cycle] -->
+
+ //! <!-- [life_cycle_w_open] -->
+ {
+ __label__ fail_fapl, fail_file;
+ hid_t fapl, file;
+
+ if ((fapl = H5Pcreate(H5P_FILE_ACCESS)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_fapl;
+ }
+ else {
+ // adjust the file access properties
+ }
+
+ unsigned mode = H5F_ACC_RDWR;
+ char name[] = "f1.h5";
+
+ if ((file = H5Fopen(name, mode, fapl)) == H5I_INVALID_HID) {
+ ret_val = EXIT_FAILURE;
+ goto fail_file;
+ }
+
+ // do something useful with FILE
+
+ H5Fclose(file);
+fail_file:
+ H5Pclose(fapl);
+fail_fapl:;
+ }
+ //! <!-- [life_cycle_w_open] -->
+
+ //! <!-- [minimal] -->
+ {
+ unsigned mode = H5F_ACC_TRUNC;
+ char name[] = "f11.h5";
+
+ hid_t file = H5Fcreate(name, mode, H5P_DEFAULT, H5P_DEFAULT);
+ if (file != H5I_INVALID_HID)
+ H5Fclose(file);
+ else
+ ret_val = EXIT_FAILURE;
+ }
+ //! <!-- [minimal] -->
+
+ //! <!-- [open] -->
+ {
+ unsigned mode = H5F_ACC_RDONLY;
+ char name[] = "f11.h5";
+
+ hid_t file = H5Fopen(name, mode, H5P_DEFAULT);
+ if (file != H5I_INVALID_HID)
+ H5Fclose(file);
+ else
+ ret_val = EXIT_FAILURE;
+ }
+ //! <!-- [open] -->
+
+ //! <!-- [flush] -->
+ {
+ unsigned mode = H5F_ACC_RDWR;
+ char name[] = "f11.h5";
+
+ hid_t file = H5Fopen(name, mode, H5P_DEFAULT);
+ if (file != H5I_INVALID_HID) {
+ int step;
+ for (step = 0; step < 1000; ++step) {
+
+ // do important work & flush every 20 steps
+
+ if (step % 20 == 0) {
+ if (H5Fflush(file, H5F_SCOPE_LOCAL) < 0) {
+ perror("H5Fflush failed.");
+ ret_val = EXIT_FAILURE;
+ break;
+ }
+ }
+ }
+
+ if (H5Fclose(file) < 0)
+ perror("H5Fclose failed.");
+ }
+ else
+ ret_val = EXIT_FAILURE;
+ }
+ //! <!-- [flush] -->
+
+ //! <!-- [libver_bounds] -->
+ {
+ unsigned mode = H5F_ACC_RDWR;
+ char name[] = "f11.h5";
+
+ hid_t file = H5Fopen(name, mode, H5P_DEFAULT);
+ if (file != H5I_INVALID_HID) {
+ if (H5Fset_libver_bounds(file, H5F_LIBVER_EARLIEST, H5F_LIBVER_V18) >= 0) {
+
+ // object creation will not exceed HDF5 version 1.8.x
+ }
+ else
+ perror("H5Fset_libver_bounds failed.");
+
+ if (H5Fclose(file) < 0)
+ perror("H5Fclose failed.");
+ }
+ else
+ ret_val = EXIT_FAILURE;
+ }
+ //! <!-- [libver_bounds] -->
+
+ //! <!-- [mount] -->
+ {
+ hid_t file = H5Fopen("f11.h5", H5F_ACC_RDWR, H5P_DEFAULT);
+ if (file != H5I_INVALID_HID) {
+ hid_t group, child;
+ if ((group = H5Gcreate1(file, "mount_point", H5P_DEFAULT)) != H5I_INVALID_HID) {
+ if ((child = H5Fopen("f1.h5", H5F_ACC_RDONLY, H5P_DEFAULT)) != H5I_INVALID_HID) {
+ if (H5Fmount(group, ".", child, H5P_DEFAULT) >= 0) {
+
+ // do something useful w/ the mounted file
+ }
+ else {
+ ret_val = EXIT_FAILURE;
+ perror("H5Fmount failed.");
+ }
+ H5Fclose(child);
+ }
+ H5Gclose(group);
+ }
+ H5Fclose(file);
+ }
+ else
+ ret_val = EXIT_FAILURE;
+ }
+ //! <!-- [mount] -->
+
+ return ret_val;
+}
diff --git a/doxygen/examples/H5Pget_metadata_read_attempts.1.c b/doxygen/examples/H5Pget_metadata_read_attempts.1.c
new file mode 100644
index 0000000..da325c0
--- /dev/null
+++ b/doxygen/examples/H5Pget_metadata_read_attempts.1.c
@@ -0,0 +1,22 @@
+/* Get a copy of file access property list */
+fapl = H5Pcreate(H5P_FILE_ACCESS);
+
+/* Retrieve the # of read attempts from the file access property list */
+H5Pget_metadata_read_attempts(fapl, &attempts);
+
+/*
+ * The value returned in "attempts" will be 1 (default for non-SWMR access).
+ */
+
+/* Set the # of read attempts to 20 */
+H5Pset_metadata_read_attempts(fapl, 20);
+
+/* Retrieve the # of read attempts from the file access property list */
+H5Pget_metadata_read_attempts(fapl, &attempts);
+
+/*
+ * The value returned in "attempts" will be 20 as set.
+ */
+
+/* Close the property list */
+H5Pclose(fapl);
diff --git a/doxygen/examples/H5Pget_metadata_read_attempts.2.c b/doxygen/examples/H5Pget_metadata_read_attempts.2.c
new file mode 100644
index 0000000..2cd12db
--- /dev/null
+++ b/doxygen/examples/H5Pget_metadata_read_attempts.2.c
@@ -0,0 +1,44 @@
+/* Open the file with SWMR access and default file access property list */
+fid = H5Fopen(FILE, (H5F_ACC_RDONLY | H5F_ACC_SWMR_READ), H5P_DEFAULT);
+
+/* Get the file's file access roperty list */
+file_fapl = H5Fget_access_plist(fid);
+
+/* Retrieve the # of read attempts from the file's file access property list */
+H5Pget_metadata_read_attempts(file_fapl, &attempts);
+
+/*
+ * The value returned in "attempts" will be 100 (default for SWMR access).
+ */
+
+/* Close the property list */
+H5Pclose(file_fapl);
+
+/* Close the file */
+H5Fclose(fid);
+
+/* Create a copy of file access property list */
+fapl = H5Pcreate(H5P_FILE_ACCESS);
+
+/* Set the # of read attempts */
+H5Pset_metadata_read_attempts(fapl, 20);
+
+/* Open the file with SWMR access and the non-default file access property list */
+fid = H5Fopen(FILE, (H5F_ACC_RDONLY | H5F_ACC_SWMR_READ), fapl);
+
+/* Get the file's file access roperty list */
+file_fapl = H5Fget_access_plist(fid);
+
+/* Retrieve the # of read attempts from the file's file access property list */
+H5Pget_metadata_read_attempts(file_fapl, &attempts);
+
+/*
+ * The value returned in "attempts" will be 20.
+ */
+
+/* Close the property lists */
+H5Pclose(file_fapl);
+H5Pclose(fapl);
+
+/* Close the file */
+H5Fclose(fid);
diff --git a/doxygen/examples/H5Pget_metadata_read_attempts.3.c b/doxygen/examples/H5Pget_metadata_read_attempts.3.c
new file mode 100644
index 0000000..4b5ea3a
--- /dev/null
+++ b/doxygen/examples/H5Pget_metadata_read_attempts.3.c
@@ -0,0 +1,44 @@
+/* Open the file with non-SWMR access and default file access property list */
+fid = H5Fopen(FILE, H5F_ACC_RDONLY, H5P_DEFAULT);
+
+/* Get the file's file access roperty list */
+file_fapl = H5Fget_access_plist(fid);
+
+/* Retrieve the # of read attempts from the file's file access property list */
+H5Pget_metadata_read_attempts(file_fapl, &attempts);
+
+/*
+ * The value returned in "attempts" will be 1 (default for non-SWMR access).
+ */
+
+/* Close the property list */
+H5Pclose(file_fapl);
+
+/* Close the file */
+H5Fclose(fid);
+
+/* Create a copy of file access property list */
+fapl = H5Pcreate(H5P_FILE_ACCESS);
+
+/* Set the # of read attempts */
+H5Pset_metadata_read_attempts(fapl, 20);
+
+/* Open the file with non-SWMR access and the non-default file access property list */
+fid = H5Fopen(FILE, H5F_ACC_RDONLY, fapl);
+
+/* Get the file's file access roperty list */
+file_fapl = H5Fget_access_plist(fid);
+
+/* Retrieve the # of read attempts from the file's file access property list */
+H5Pget_metadata_read_attempts(file_fapl, &attempts);
+
+/*
+ * The value returned in "attempts" will be 1 (default for non-SWMR access).
+ */
+
+/* Close the property lists */
+H5Pclose(file_fapl);
+H5Pclose(fapl);
+
+/* Close the file */
+H5Fclose(fid);
diff --git a/doxygen/examples/H5Pget_object_flush_cb.c b/doxygen/examples/H5Pget_object_flush_cb.c
new file mode 100644
index 0000000..d18f3df
--- /dev/null
+++ b/doxygen/examples/H5Pget_object_flush_cb.c
@@ -0,0 +1,41 @@
+hid_t fapl_id;
+unsigned counter;
+H5F_object_flush_t *ret_cb;
+unsigned * ret_counter;
+
+/* Create a copy of the file access property list */
+fapl_id = H5Pcreate(H5P_FILE_ACCESS);
+
+/* Set up the object flush property values */
+/* flush_cb: callback function to invoke when an object flushes (see below) */
+/* counter: user data to pass along to the callback function */
+H5Pset_object_flush_cb(fapl_id, flush_cb, &counter);
+
+/* Open the file */
+file_id = H5Fopen(FILE, H5F_ACC_RDWR, H5P_DEFAULT);
+
+/* Get the file access property list for the file */
+fapl = H5Fget_access_plist(file_id);
+
+/* Retrieve the object flush property values for the file */
+H5Pget_object_flush_cb(fapl, &ret_cb, &ret_counter);
+/* ret_cb will point to flush_cb() */
+/* ret_counter will point to counter */
+
+/*
+.
+.
+.
+.
+.
+.
+*/
+
+/* The callback function for the object flush property */
+static herr_t
+flush_cb(hid_t obj_id, void *_udata)
+{
+ unsigned *flush_ct = (unsigned *)_udata;
+ ++(*flush_ct);
+ return 0;
+}
diff --git a/doxygen/examples/H5Pset_metadata_read_attempts.c b/doxygen/examples/H5Pset_metadata_read_attempts.c
new file mode 100644
index 0000000..7c2f65d
--- /dev/null
+++ b/doxygen/examples/H5Pset_metadata_read_attempts.c
@@ -0,0 +1,59 @@
+//! [SWMR Access]
+/* Create a copy of file access property list */
+fapl = H5Pcreate(H5P_FILE_ACCESS);
+
+/* Set the # of read attempts */
+H5Pset_metadata_read_attempts(fapl, 20);
+
+/* Open the file with SWMR access and the non-default file access property list */
+fid = H5Fopen(FILE, (H5F_ACC_RDONLY | H5F_ACC_SWMR_READ), fapl);
+
+/* Get the file's file access roperty list */
+file_fapl = H5Fget_access_plist(fid);
+
+/* Retrieve the # of read attempts from the file's file access property list */
+H5Pget_metadata_read_attempts(file_fapl, &attempts);
+
+/*
+ * The value returned in "attempts" will be 20.
+ * The library will use 20 as the number of read attempts
+ * when reading checksummed metadata in the file
+ */
+
+/* Close the property list */
+H5Pclose(fapl);
+H5Pclose(file_fapl);
+
+/* Close the file */
+H5Fclose(fid);
+//! [SWMR Access]
+
+//! [non-SWMR Access]
+/* Create a copy of file access property list */
+fapl = H5Pcreate(H5P_FILE_ACCESS);
+
+/* Set the # of read attempts */
+H5Pset_metadata_read_attempts(fapl, 20);
+
+/* Open the file with SWMR access and the non-default file access property list */
+fid = H5Fopen(FILE, H5F_ACC_RDONLY, fapl);
+
+/* Get the file's file access roperty list */
+file_fapl = H5Fget_access_plist(fid);
+
+/* Retrieve the # of read attempts from the file's file access property list */
+H5Pget_metadata_read_attempts(file_fapl, &attempts);
+
+/*
+ * The value returned in "attempts" will be 1 (default for non-SWMR access).
+ * The library will use 1 as the number of read attempts
+ * when reading checksummed metadata in the file
+ */
+
+/* Close the property lists */
+H5Pclose(fapl);
+H5Pclose(file_fapl);
+
+/* Close the file */
+H5Fclose(fid);
+//! [non-SWMR Access]
diff --git a/doxygen/examples/H5Pset_object_flush_cb.c b/doxygen/examples/H5Pset_object_flush_cb.c
new file mode 100644
index 0000000..1dfa90d
--- /dev/null
+++ b/doxygen/examples/H5Pset_object_flush_cb.c
@@ -0,0 +1,41 @@
+hid_t file_id, fapl_id;
+hid_t dataset_id, dapl_id;
+unsigned counter;
+
+/* Create a copy of the file access property list */
+fapl_id = H5Pcreate(H5P_FILE_ACCESS);
+
+/* Set up the object flush property values */
+/* flush_cb: callback function to invoke when an object flushes (see below) */
+/* counter: user data to pass along to the callback function */
+H5Pset_object_flush_cb(fapl_id, flush_cb, &counter);
+
+/* Open the file */
+file_id = H5Fopen(FILE, H5F_ACC_RDWR, H5P_DEFAULT);
+
+/* Create a group */
+gid = H5Gcreate2(fid, “group”, H5P_DEFAULT, H5P_DEFAULT_H5P_DEFAULT);
+
+/* Open a dataset */
+dataset_id = H5Dopen2(file_id, DATASET, H5P_DEFAULT);
+
+/* The flush will invoke flush_cb() with counter */
+H5Dflush(dataset_id);
+/* counter will be equal to 1 */
+
+/* ... */
+
+/* The flush will invoke flush_cb() with counter */
+H5Gflush(gid);
+/* counter will be equal to 2 */
+
+/* ... */
+
+/* The callback function for object flush property */
+static herr_t
+flush_cb(hid_t obj_id, void *_udata)
+{
+ unsigned *flush_ct = (unsigned *)_udata;
+ ++(*flush_ct);
+ return 0;
+}
diff --git a/doxygen/examples/ImageSpec.html b/doxygen/examples/ImageSpec.html
new file mode 100644
index 0000000..1b700ff
--- /dev/null
+++ b/doxygen/examples/ImageSpec.html
@@ -0,0 +1,1203 @@
+<!doctype html public "-//w3c//dtd html 4.0 transitional//en">
+<html>
+<head>
+ <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
+ <meta name="GENERATOR" content="Mozilla/4.72 [en] (WinNT; U) [Netscape]">
+ <title>Image Specification</title>
+
+The HDF5 specification defines the standard objects and storage for the
+standard HDF5 objects. (For information about the HDF5 library, model and
+specification, see the HDF documentation.)&nbsp; This document is an additional
+specification do define a standard profile for how to store image data
+in HDF5. Image data in HDF5 is stored as HDF5 datasets with standard attributes
+to define the properties of the image.
+<p>This specification is primarily concerned with two dimensional raster
+data similar to HDF4 Raster Images.&nbsp; Specifications for storing other
+types of imagery will be covered in other documents.
+<p>This specification defines:
+<ul>
+<li>
+Standard storage and attributes for an Image dataset (<a href="#Sect1">Section
+1</a>)</li>
+
+<li>
+Standard storage and attributes for Palettes (<a href="#sect2">Section
+2</a>)</li>
+
+<li>
+Standard for associating Palettes with Images. (<a href="#Sect3">Section
+3</a>)</li>
+</ul>
+
+<h2>
+<a NAME="Sect1"></a>1. HDF5 Image Specification</h2>
+
+<h3>
+1.1 Overview</h3>
+Image data is stored as an HDF5 dataset with values of HDF5 class Integer
+or Float.&nbsp; A common example would be a two dimensional dataset, with
+elements of class Integer, e.g., a two dimensional array of unsigned 8
+bit integers.&nbsp; However, this specification does not limit the dimensions
+or number type that may be used for an Image.
+<p>The dataset for an image is distinguished from other datasets by giving
+it an attribute "CLASS=IMAGE".&nbsp; In addition, the Image dataset may
+have an optional attribute "PALETTE" that is an array of object references
+for zero or more palettes. The Image dataset may have additional attributes
+to describe the image data, as defined in <a href="#Sect1.2">Section 1.2</a>.
+<p>A Palette is an HDF5 dataset which contains color map information.&nbsp;
+A Pallet dataset has an attribute "CLASS=PALETTE" and other attributes
+indicating the type and size of the palette, as defined in <a href="#sect2">Section
+2.1</a>.&nbsp; A Palette is an independent object, which can be shared
+among several Image datasets.
+<h3>
+<a NAME="Sect1.2"></a>1.2&nbsp; Image Attributes</h3>
+The attributes for the Image are scalars unless otherwise noted.&nbsp;
+The length of String valued attributes should be at least the number of
+characters. Optionally, String valued attributes may be stored in a String
+longer than the minimum, in which case it must be zero terminated or null
+padded.&nbsp; "Required" attributes must always be used. "Optional" attributes
+must be used when required.
+<br>&nbsp;
+<h4>
+Attributes</h4>
+
+<dl>
+<dt>
+Attribute name="<b>CLASS</b>" (Required)</dt>
+
+<dd>
+This attribute is type H5T_C_S1, with size 5.</dd>
+
+<dd>
+For all Images, the value of this attribute is "IMAGE".</dd>
+
+<dd>
+</dd>
+
+<dd>
+This attribute identifies this data set as intended to be interpreted as
+an image that conforms to the specifications on this page.</dd>
+</dl>
+
+<dt>
+Attribute name="<b>PALETTE</b>"</dt>
+
+<dl>
+<dd>
+A Image dataset within an HDF5 file may optionally specify an array of
+palettes to be viewed with. The dataset will have an attribute field called
+"<b>PALETTE</b>" which contains a one-dimensional array of object reference
+pointers (HDF5 datatype H5T_STD_REF_OBJ) which refer to palettes in the
+file. The palette datasets must conform to the Palette specification in
+<a href="#sect2">section
+2 below</a>. The first palette in this array will be the default palette
+that the data may be viewed with.</dd>
+</dl>
+
+<dl>
+<dt>
+</dt>
+
+<dt>
+Attribute name="<b>IMAGE_SUBCLASS</b>"</dt>
+
+<dd>
+If present, the value of this attribute indicates the type of Palette that
+should be used with the Image.&nbsp; This attribute is a scalar of type
+H5T_C_S1, with size according to the string plus one.&nbsp; The values
+are:</dd>
+
+<dl>
+<dt>
+"IMAGE_GRAYSCALE" (length 15)</dt>
+
+<dd>
+A grayscale image</dd>
+
+<dt>
+"IMAGE_BITMAP" (length 12)</dt>
+
+<dd>
+A bit map image</dd>
+
+<dt>
+"IMAGE_TRUECOLOR" (length 15)</dt>
+
+<dd>
+A truecolor image</dd>
+
+<dt>
+"IMAGE_INDEXED" (length 13)</dt>
+
+<dd>
+An indexed image</dd>
+
+<dd>
+</dd>
+</dl>
+
+<dt>
+Attribute name="<b>INTERLACE_MODE</b>"</dt>
+
+<dd>
+For images with more than one component for each pixel, this optional attribute
+specifies the layout of the data. The values are type H5T_C_S1 of length
+15. See <a href="#Section1.3">section 1.3</a> for information about the
+storage layout for data.</dd>
+
+<dd>
+"INTERLACE_PIXEL" (default): the component value for a pixel are contiguous.</dd>
+
+<dd>
+"INTERLACE_PLANE": each component is stored as a plane.</dd>
+
+<dt>
+</dt>
+
+<dt>
+Attribute name="<b>DISPLAY_ORIGIN</b>"</dt>
+
+<dd>
+This optional attribute indicates the intended orientation of the data
+on a two-dimensional raster display.&nbsp; The value indicates which corner
+the pixel at (0, 0) should be viewed.&nbsp; The values are type H5T_C_S1
+of length 2. If DISPLAY_ORIGIN is not set, the orientation is undefined.</dd>
+
+<dd>
+"UL": (0,0) is at the upper left.</dd>
+
+<dd>
+"LL": (0,0) is at the lower left.</dd>
+
+<dd>
+"UR": (0,0) is at the upper right.</dd>
+
+<dd>
+"LR": (0,0) is at the lower right.</dd>
+</dl>
+
+<dt>
+Attribute name="<b>IMAGE_WHITE_IS_ZERO</b>"</dt>
+
+<dl>
+<dd>
+This attribute is of type H5T_NATIVE_UCHAR.&nbsp; 0 = false, 1 = true .&nbsp;
+This is used for images with IMAGE_SUBCLASS="IMAGE_GRAYSCALE" or "IMAGE_BITMAP".</dd>
+</dl>
+
+<dl>
+<dt>
+Attribute name="<b>IMAGE_MINMAXRANGE</b>"</dt>
+
+<dd>
+If present, this attribute is an array of two numbers, of the same HDF5
+datatype as the data.&nbsp; The first element is the minimum value of the
+data, and the second is the maximum.&nbsp; This is used for images with
+IMAGE_SUBCLASS="IMAGE_GRAYSCALE", "IMAGE_BITMAP" or "IMAGE_INDEXED".</dd>
+</dl>
+
+<dt>
+Attribute name="<b>IMAGE_BACKGROUNDINDEX</b>"</dt>
+
+<dl>
+<dd>
+If set, this attribute indicates the index value that should be interpreted
+as the "background color".&nbsp; This attribute is HDF5 type H5T_NATIVE_UINT.</dd>
+</dl>
+
+<dt>
+Attribute name="<b>IMAGE_TRANSPARENCY</b>"</dt>
+
+<dl>
+<dd>
+If set, this attribute indicates the index value that should be interpreted
+as the "transparent color".&nbsp; This attribute is HDF5 type H5T_NATIVE_UINT.&nbsp;
+This attribute may not be used for IMAGE_SUBCLASS="IMAGE_TRUE_COLOR".</dd>
+</dl>
+
+<dt>
+Attribute name="<b>IMAGE_ASPECTRATIO</b>"</dt>
+
+<dl>
+<dd>
+If set, this attribute indicates the aspect ratio.</dd>
+</dl>
+
+<dt>
+Attribute name="<b>IMAGE_COLORMODEL</b>"</dt>
+
+<dl>
+<dd>
+If set, this attribute indicates the color model of Palette that should
+be used with the Image.&nbsp; This attribute is of type H5T_C_S1, with
+size 3, 4, or 5.&nbsp; The value is one of the color models described in
+the Palette specification in <a href="#sect2.2">section 2.2 below</a>.&nbsp;
+This attribute may be used only for IMAGE_SUBCLASS="IMAGE_TRUECOLOR" or
+"IMAGE_INDEXED".</dd>
+</dl>
+
+<dt>
+Attribute name="<b>IMAGE_GAMMACORRECTION</b>"</dt>
+
+<dl>
+<dd>
+If set, this attribute gives the Gamma correction.&nbsp; The attribute
+is type H5T_NATIVE_FLOAT.&nbsp; This attribute may be used only for IMAGE_SUBCLASS="IMAGE_TRUECOLOR"
+or "IMAGE_INDEXED".</dd>
+</dl>
+Attribute name="<b>IMAGE_VERSION</b>" (Required)
+<dl>
+<dd>
+This attribute is of type H5T_C_S1, with size corresponding to the length
+of the version string.&nbsp; This attribute identifies the version number
+of this specification to which it conforms.&nbsp; The current version number
+is "1.2".</dd>
+
+<br>&nbsp;
+<p>&nbsp;
+<br>&nbsp;
+<br>&nbsp;
+<center><table BORDER=2 BGCOLOR="#FFFFFF" >
+<caption><b>Table 1. Attributes of an Image Dataset</b></caption>
+
+<tr>
+<td><b>Attribute Name</b></td>
+
+<td><b>(R = Required</b>
+<br><b>O= Optional)</b></td>
+
+<td><b>Type</b></td>
+
+<td><b>String Size</b></td>
+
+<td><b>Value</b></td>
+</tr>
+
+<tr>
+<td>CLASS</td>
+
+<td>R</td>
+
+<td>String</td>
+
+<td>5</td>
+
+<td>"IMAGE"</td>
+</tr>
+
+<tr>
+<td>PALETTE</td>
+
+<td>O</td>
+
+<td>Array Object References</td>
+
+<td></td>
+
+<td>&lt;references to Palette datasets><sup>1</sup></td>
+</tr>
+
+<tr>
+<td>IMAGE_SUBCLASS</td>
+
+<td>O<sup>2</sup></td>
+
+<td>String</td>
+
+<td>15,&nbsp;
+<br>12,&nbsp;
+<br>15,
+<br>13</td>
+
+<td>
+<dt>
+"IMAGE_GRAYSCALE",</dt>
+
+<dt>
+"IMAGE_BITMAP",</dt>
+
+<dt>
+"IMAGE_TRUECOLOR",</dt>
+
+<dt>
+"IMAGE_INDEXED"</dt>
+</td>
+</tr>
+
+<tr>
+<td>INTERLACE_MODE</td>
+
+<td>O<sup>3,6</sup></td>
+
+<td>String</td>
+
+<td>15</td>
+
+<td>The layout of components if more than one component per pixel.</td>
+</tr>
+
+<tr>
+<td>DISPLAY_ORIGIN</td>
+
+<td>O</td>
+
+<td>String</td>
+
+<td>2</td>
+
+<td>If set, indicates the intended location of the pixel (0,0).</td>
+</tr>
+
+<tr>
+<td>IMAGE_WHITE_IS_ZERO</td>
+
+<td>O<sup>3,4</sup></td>
+
+<td>Unsigned Integer</td>
+
+<td></td>
+
+<td>0 = false, 1 = true</td>
+</tr>
+
+<tr>
+<td>IMAGE_MINMAXRANGE</td>
+
+<td>O<sup>3,5</sup></td>
+
+<td>Array [2] &lt;same datatype as data values></td>
+
+<td></td>
+
+<td>The (&lt;minimum>, &lt;maximum>) value of the data.</td>
+</tr>
+
+<tr>
+<td>IMAGE_BACKGROUNDINDEX</td>
+
+<td>O<sup>3</sup></td>
+
+<td>Unsigned Integer</td>
+
+<td></td>
+
+<td>The index of the background color.</td>
+</tr>
+
+<tr>
+<td>IMAGE_TRANSPARENCY</td>
+
+<td>O<sup>3,5</sup></td>
+
+<td>Unsigned Integer</td>
+
+<td></td>
+
+<td>The index of the transparent color.</td>
+</tr>
+
+<tr>
+<td>IMAGE_ASPECTRATIO</td>
+
+<td>O<sup>3,4</sup></td>
+
+<td>Unsigned Integer</td>
+
+<td></td>
+
+<td>The aspect ratio.</td>
+</tr>
+
+<tr>
+<td>IMAGE_COLORMODEL</td>
+
+<td>O<sup>3,6</sup></td>
+
+<td>String</td>
+
+<td>3, 4, or 5</td>
+
+<td>The color model, as defined below in the Palette specification for
+attribute <b>PAL_COLORMODEL</b>.</td>
+</tr>
+
+<tr>
+<td>IMAGE_GAMMACORRECTION</td>
+
+<td>O<sup>3,6</sup></td>
+
+<td>Float</td>
+
+<td></td>
+
+<td>The gamma correction.</td>
+</tr>
+
+<tr>
+<td>IMAGE_VERSION</td>
+
+<td>R</td>
+
+<td>String</td>
+
+<td>3</td>
+
+<td>"1.2"</td>
+</tr>
+</table></center>
+
+<dl><font size=-1>1.&nbsp; The first element of the array is the default
+Palette.</font>
+<br><font size=-1>2.&nbsp; This attribute is <b>required</b> for images
+that use one of the standard color map types listed.</font>
+<br><font size=-1>3. This attribute is <b>required</b> if set for the source
+image, in the case that the image is translated from another file into
+HDF5.</font>
+<br><font size=-1>4.&nbsp; This applies to:&nbsp; IMAGE_SUBCLASS="IMAGE_GRAYSCALE"
+or "IMAGE_BITMAP".</font>
+<br><font size=-1>5.&nbsp; This applies to:&nbsp; IMAGE_SUBCLASS="IMAGE_GRAYSCALE",
+"IMAGE_BITMAP", or "IMAGE_INDEXED".</font>
+<br><font size=-1>6.&nbsp; This applies to: IMAGE_SUBCLASS="IMAGE_TRUECOLOR",
+or "IMAGE_INDEXED".</font></dl>
+</dl>
+Table 2 summarizes the standard attributes for an Image datasets using
+the common sub-classes. R means that the attribute listed on the leftmost
+column is Required for the image subclass on the first row, O means that
+the attribute is Optional for that subclass and N that the attribute cannot
+be applied to that subclass. The two first rows show the only required
+attributes
+for all subclasses.
+<br>&nbsp;
+<table BORDER WIDTH="100%" >
+<caption><b>Table 2a. Applicability of Attributes to IMAGE sub-classes</b></caption>
+
+<tr>
+<td WIDTH="20%"><b>IMAGE_SUBCLASS</b><sup>1</sup></td>
+
+<td WIDTH="20%"><b>IMAGE_GRAYSCALE</b></td>
+
+<td WIDTH="20%"><b>IMAGE_BITMAP</b></td>
+</tr>
+
+<tr>
+<td WIDTH="20%">CLASS</td>
+
+<td WIDTH="20%">R</td>
+
+<td WIDTH="20%">R</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">IMAGE_VERSION</td>
+
+<td WIDTH="20%">R</td>
+
+<td WIDTH="20%">R</td>
+</tr>
+
+<tr>
+<td>INTERLACE_MODE</td>
+
+<td>N</td>
+
+<td>N</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">IMAGE_WHITE_IS_ZERO</td>
+
+<td WIDTH="20%">R</td>
+
+<td WIDTH="20%">R</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">IMAGE_MINMAXRANGE</td>
+
+<td WIDTH="20%">O</td>
+
+<td WIDTH="20%">O</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">IMAGE_BACKGROUNDINDEX</td>
+
+<td WIDTH="20%">O</td>
+
+<td WIDTH="20%">O</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">IMAGE_TRANSPARENCY</td>
+
+<td WIDTH="20%">O</td>
+
+<td WIDTH="20%">O</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">IMAGE_ASPECTRATIO</td>
+
+<td WIDTH="20%">O</td>
+
+<td WIDTH="20%">O</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">IMAGE_COLORMODEL</td>
+
+<td WIDTH="20%">N</td>
+
+<td WIDTH="20%">N</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">IMAGE_GAMMACORRECTION</td>
+
+<td WIDTH="20%">N</td>
+
+<td WIDTH="20%">N</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">PALETTE</td>
+
+<td WIDTH="20%">O</td>
+
+<td WIDTH="20%">O</td>
+</tr>
+
+<tr>
+<td>DISPLAY_ORIGIN</td>
+
+<td>O</td>
+
+<td>O</td>
+</tr>
+</table>
+
+<blockquote>&nbsp;</blockquote>
+
+<table BORDER WIDTH="100%" >
+<caption><b>Table 2b. Applicability of Attributes to IMAGE sub-classes</b></caption>
+
+<tr>
+<td WIDTH="20%"><b>IMAGE_SUBCLASS</b></td>
+
+<td WIDTH="20%"><b>IMAGE_TRUECOLOR</b></td>
+
+<td><b>IMAGE_INDEXED</b></td>
+</tr>
+
+<tr>
+<td WIDTH="20%">CLASS</td>
+
+<td WIDTH="20%">R</td>
+
+<td>R</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">IMAGE_VERSION</td>
+
+<td WIDTH="20%">R</td>
+
+<td>R</td>
+</tr>
+
+<tr>
+<td>INTERLACE_MODE</td>
+
+<td>R</td>
+
+<td>N</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">IMAGE_WHITE_IS_ZERO</td>
+
+<td WIDTH="20%">N</td>
+
+<td>N</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">IMAGE_MINMAXRANGE</td>
+
+<td WIDTH="20%">N</td>
+
+<td>O</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">IMAGE_BACKGROUNDINDEX</td>
+
+<td WIDTH="20%">N</td>
+
+<td>O</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">IMAGE_TRANSPARENCY</td>
+
+<td WIDTH="20%">N</td>
+
+<td>O</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">IMAGE_ASPECTRATIO</td>
+
+<td WIDTH="20%">O</td>
+
+<td>O</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">IMAGE_COLORMODEL</td>
+
+<td WIDTH="20%">O</td>
+
+<td>O</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">IMAGE_GAMMACORRECTION</td>
+
+<td WIDTH="20%">O</td>
+
+<td>O</td>
+</tr>
+
+<tr>
+<td WIDTH="20%">PALETTE</td>
+
+<td WIDTH="20%">O</td>
+
+<td>O</td>
+</tr>
+
+<tr>
+<td>DISPLAY_ORIGIN</td>
+
+<td>O</td>
+
+<td>O</td>
+</tr>
+</table>
+
+<h3>
+<a NAME="Section1.3"></a>1.3 Storage Layout and Properties for Images</h3>
+In the case of an image with more than one component per pixel (e.g., Red,
+Green, and Blue), the data may be arranged in one of two ways.&nbsp; Following
+HDF4 terminology, the data may be interlaced by pixel or by plane, which
+should be indicated by the INTERLACE_MODE&nbsp; attribute.&nbsp; In both
+cases, the dataset will have a dataspace with three dimensions, height,
+width, and components.&nbsp; The interlace modes specify different orders
+for the dimensions.
+<br>&nbsp;
+<table BORDER COLS=2 WIDTH="100%" >
+<caption><b>Table 3. Storage of multiple component image data.</b></caption>
+
+<tr>
+<td><b>Interlace Mode</b></td>
+
+<td><b>Dimensions in the Dataspace</b></td>
+</tr>
+
+<tr>
+<td>INTERLACE_PIXEL</td>
+
+<td>[height][width][pixel components]</td>
+</tr>
+
+<tr>
+<td>INTERLACE_PLANE</td>
+
+<td>[pixel components][height][width]</td>
+</tr>
+</table>
+
+<p>For example, consider a 5 (rows) by 10 (column) image, with Red, Green,
+and Blue components.&nbsp; Each component is an unsigned byte. In HDF5,
+the datatype would be declared as an unsigned 8 bit integer.&nbsp; For
+pixel interlace, the dataspace would be a three dimensional array, with
+dimensions: [10][5][3].&nbsp; For plane interleave, the dataspace would
+be three dimensions: [3][10][5].
+<p>In the case of images with only one component, the dataspace may be
+either a two dimensional array, or a three dimensional array with the third
+dimension of size 1.&nbsp; For example, a 5 by 10 image with 8 bit color
+indexes would be an HDF5 dataset with type unsigned 8 bit integer.&nbsp;
+The dataspace could be either a two dimensional array, with dimensions
+[10][5], or three dimensions, with dimensions either [10][5][1] or [1][10][5].
+<p>Image datasets may be stored with any chunking or compression properties
+supported by HDF5.
+<p><b>A note concerning compatibility with HDF5 GR interface: </b>An Image
+dataset is stored as an HDF5 dataset.&nbsp; It is important to note that
+the order of the dimensions is the same as for any other HDF5 dataset.&nbsp;
+For a two dimensional image that is to be stored as a series of horizontal
+scan lines, with the scan lines contiguous (i.e., the fastest changing
+dimension is 'width'), the image will have a dataspace with <i>dim[0] =
+height</i> and <i>dim[1]</i> = <i>width</i>.&nbsp; This is completely consistent
+with all other HDF5 datasets.
+<p>Users familiar with HDF4 should be cautioned that <i>this is not the
+same as HDF4</i>, and specifically is not consistent with what the HDF4
+GR interface does.
+<br>&nbsp;
+<h2>
+<a NAME="sect2"></a>2.&nbsp; HDF5 Palette Specification</h2>
+
+<h3>
+2.1 Overview</h3>
+A palette is the means by which color is applied to an image and is also
+referred to as a color lookup table. It is a table in which every row contains
+the numerical representation of a particular color. In the example of an
+8 bit standard RGB color model palette, this numerical representation of
+a color is presented as a triplet specifying the intensity of red, green,
+and blue components that make up each color.
+<center>
+<p><img SRC="Palettes.fm.anc.gif" ></center>
+
+<p>In this example, the color component numeric type is an 8 bit unsigned
+integer. While this is most common and recommended for general use, other
+component color numeric datatypes, such as a 16 bit unsigned integer ,
+may be used. This type is specified as the type attribute of the palette
+dataset. (see H5Tget_type(), H5Tset_type())
+<p>The minimum and maximum values of the component color numeric are specified
+as attribute of the palette dataset. See below (attribute PAL_MINMAXNUMERIC).
+If these attributes do not exist, it is assumed that the range of values
+will fill the space of the color numeric type. i.e. with an 8 bit unsigned
+integer, the valid range would be 0 to 255 for each color component.
+<p>The HDF5 palette specification additionally allows for color models
+beyond RGB. YUV, HSV, CMY, CMYK, YCbCr color models are supported, and
+may be specified as a color model attribute of the palette dataset. <i>(see
+"Palette Attributes" for details)</i>.
+<p>In HDF 4 and earlier, palettes were limited to 256 colors. The HDF5
+palette specification allows for palettes of varying length. The length
+is specified as the number of rows of the palette dataset.
+<br>&nbsp;
+<br>&nbsp;
+<table BORDER COLS=1 WIDTH="100%" BGCOLOR="#666666" >
+<tr>
+<td><font color="#FFFFFF">Important Note: The specification of the Indexed
+Palette will change substantially in the next version.&nbsp; The Palette
+described here is <i>denigrated</i> and is not supported.</font></td>
+</tr>
+</table>
+
+<br>&nbsp;
+<table BORDER COLS=1 WIDTH="100%" BGCOLOR="#CCCCCC" >
+<tr>
+<td><i>Denigrated</i>
+<p>In a standard palette, the color entries are indexed directly. HDF5
+supports the notion of a range index table. Such a table defines an ascending
+ordered list of ranges that map dataset values to the palette. If a range
+index table exists for the palette, the PAL_TYPE attribute will be set
+to "RANGEINDEX", and the PAL_RANGEINDEX attribute will contain an object
+reference to a range index table array. If not, the PAL_TYPE attribute
+either does not exist, or will be set to "STANDARD".
+<p>The range index table array consists of a one dimensional array with
+the same length as the palette dataset - 1. Ideally, the range index would
+be of the same type as the dataset it refers to, however this is not a
+requirement.
+<p><b>Example 2: A range index array of type floating point</b>
+<center>
+<p><img SRC="PaletteExample1.gif" ></center>
+
+<p>The range index array attribute defines the "<i>to</i>" of the range.
+Notice that the range index array attribute is one less entry in size than
+the palette. The first entry of 0.1259, specifies that all values below
+and up to 0.1259 inclusive, will map to the first palette entry. The second
+entry signifies that all values greater than 0.1259 up to 0.3278 inclusive,
+will map to the second palette entry, etc. All value greater than the last
+range index array attribute (100000) map to the last entry in the palette.</td>
+</tr>
+</table>
+
+<h3>
+<a NAME="sect2.2"></a>2.2. Palette Attributes</h3>
+A palette exists in an HDF file as an independent data set with accompanying
+attributes.&nbsp; The Palette attributes are scalars except where noted
+otherwise.&nbsp; String values should have size the length of the string
+value plus one.&nbsp; "Required" attributes must be used.&nbsp; "Optional"
+attributes must be used when required.
+<p>These attributes are defined as follows:
+<dl>
+<dt>
+Attribute name="<b>CLASS</b>" (Required)</dt>
+
+<dd>
+This attribute is of type H5T_C_S1, with size 7.</dd>
+
+<dd>
+For all palettes, the value of this attribute is "PALETTE". This attribute
+identifies this palette data set as a palette that conforms to the specifications
+on this page.</dd>
+
+<dt>
+Attribute name="<b>PAL_COLORMODEL</b>" (Required)</dt>
+
+<dd>
+This attribute is of type H5T_C_S1, with size 3, 4, or 5.</dd>
+
+<dd>
+Possible values for this are "RGB", "YUV", "CMY", "CMYK", "YCbCr", "HSV".</dd>
+
+<dd>
+This defines the color model that the entries in the palette data set represent.</dd>
+
+<dl>
+<dt>
+"RGB"</dt>
+
+<dd>
+Each color index contains a triplet where the the first value defines the
+red component, second defines the green component, and the third the blue
+component.</dd>
+
+<dt>
+"CMY"</dt>
+
+<dd>
+Each color index contains a triplet where the the first value defines the
+cyan component, second defines the magenta component, and the third the
+yellow component.</dd>
+
+<dt>
+"CMYK"</dt>
+
+<dd>
+Each color index contains a quadruplet where the the first value defines
+the cyan component, second defines the magenta component, the third the
+yellow component, and the forth the black component.</dd>
+
+<dt>
+"YCbCr"</dt>
+
+<dd>
+Class Y encoding model. Each color index contains a triplet where the the
+first value defines the luminance, second defines the Cb Chromonance, and
+the third the Cr Chromonance.</dd>
+
+<dt>
+"YUV"</dt>
+
+<dd>
+Composite encoding color model. Each color index contains a triplet where
+the the first value defines the luminance component, second defines the
+chromonance component, and the third the value component.</dd>
+
+<dt>
+"HSV"</dt>
+
+<dd>
+Each color index contains a triplet where the the first value defines the
+hue component, second defines the saturation component, and the third the
+value component. The hue component defines the hue spectrum with a low
+value representing magenta/red progressing to a high value which would
+represent blue/magenta, passing through yellow, green, cyan. A low value
+for the saturation component means less color saturation than a high value.
+A low value for <i>value</i> will be darker than a high value.</dd>
+
+<dd>
+</dd>
+</dl>
+
+<dt>
+Attribute name="<b>PAL_TYPE</b>" (Required)</dt>
+
+<dd>
+This attribute is of type H5T_C_S1, with size 9 or 10.</dd>
+
+<dd>
+The current supported values for this attribute are : "STANDARD8" or "RANGEINDEX"</dd>
+
+<dd>
+A PAL_TYPE of "STANDARD8" defines a palette dataset such that the first
+entry defines index 0, the second entry defines index 1, etc. up until
+the length of the palette - 1. This assumes an image dataset with direct
+indexes into the palette.</dd>
+</dl>
+
+<dl>&nbsp;
+<table BORDER COLS=1 WIDTH="100%" BGCOLOR="#CCCCCC" >
+<tr>
+<td><i>Denigrated</i>
+<p>If the PAL_TYPE is set to "RANGEINDEX", there will be an additional
+attribute with a name of "<b>PAL_RANGEINDEX</b>",&nbsp; (See example 2
+for more details)</td>
+</tr>
+</table>
+
+<table BORDER COLS=1 WIDTH="100%" BGCOLOR="#CCCCCC" >
+<tr>
+<td>
+<dt>
+Attribute name="<b>PAL_RANGEINDEX</b>"&nbsp;&nbsp; <i>(Denigrated)</i></dt>
+
+<dl>
+<dd>
+The <b>PAL_RANGEINDEX</b> attribute contains an HDF object reference (HDF5
+datatype H5T_STD_REF_OBJ) pointer which specifies a range index array in
+the file to be used for color lookups for the palette.&nbsp; (Only for
+PAL_TYPE="RANGEINDEX")</dd>
+</dl>
+</td>
+</tr>
+</table>
+
+<dt>
+Attribute name="<b>PAL_MINMAXNUMERIC</b>"</dt>
+
+<dl>
+<dt>
+If present, this attribute is an array of two numbers, of the same HDF5
+datatype as the palette elements or color numerics.</dt>
+
+<br>They specify the minimum and maximum values of the color numeric components.
+For example, if the palette was an RGB of type Float, the color numeric
+range for Red, Green, and Blue could be set to be between 0.0 and 1.0.
+The intensity of the color guns would then be scaled accordingly to be
+between this minimum and maximum attribute.</dl>
+Attribute name="<b>PAL_VERSION</b>"&nbsp; (Required)
+<dl>This attribute is of type H5T_C_S1, with size corresponding to the
+length of the version string.&nbsp; This attribute identifies the version
+number of this specification to which it conforms.&nbsp; The current version
+is "1.2".</dl>
+
+<center><table BORDER=2 BGCOLOR="#FFFFFF" >
+<caption><b>Table 4. Attributes of a Palette Dataset</b></caption>
+
+<tr>
+<td><b>Attribute Name</b></td>
+
+<td><b>(R = Required,</b>
+<br><b>O = Optional)</b></td>
+
+<td><b>Type</b></td>
+
+<td><b>String Size</b></td>
+
+<td><b>Value</b></td>
+</tr>
+
+<tr>
+<td>CLASS</td>
+
+<td>R</td>
+
+<td>String</td>
+
+<td>
+<center>7</center>
+</td>
+
+<td>"PALETTE"</td>
+</tr>
+
+<tr>
+<td>PAL_COLORMODEL</td>
+
+<td>R</td>
+
+<td>String</td>
+
+<td>
+<center>3, 4, or 5</center>
+</td>
+
+<td>Color Model:&nbsp; "RGB", YUV", "CMY", "CMYK", "YCbCr", or "HSV"</td>
+</tr>
+
+<tr>
+<td>PAL_TYPE</td>
+
+<td>R</td>
+
+<td>String</td>
+
+<td>
+<center>9</center>
+
+<p><br>
+<table BORDER COLS=1 WIDTH="100%" BGCOLOR="#CCCCCC" >
+<tr>
+<td>or 10</td>
+</tr>
+</table>
+</td>
+
+<td>"STANDARD8"&nbsp;
+<table BORDER COLS=1 WIDTH="100%" BGCOLOR="#CCCCCC" >
+<tr>
+<td>or "RANGEINDEX" <i>(Denigrated)</i></td>
+</tr>
+</table>
+</td>
+</tr>
+
+<tr>
+<td>
+<table BORDER COLS=1 WIDTH="100%" BGCOLOR="#CCCCCC" >
+<tr>
+<td><i>Denigrated</i>
+<br>RANGE_INDEX</td>
+</tr>
+</table>
+</td>
+
+<td></td>
+
+<td>
+<table BORDER COLS=1 WIDTH="100%" BGCOLOR="#CCCCCC" >
+<tr>
+<td>Object Reference&nbsp;</td>
+</tr>
+</table>
+</td>
+
+<td></td>
+
+<td>
+<table BORDER COLS=1 WIDTH="100%" BGCOLOR="#CCCCCC" >
+<tr>
+<td>&lt;Object Reference to Dataset of range index values></td>
+</tr>
+</table>
+</td>
+</tr>
+
+<tr>
+<td>PAL_MINMAXNUMERIC</td>
+
+<td>O</td>
+
+<td>Array[2] of &lt;same datatype as palette></td>
+
+<td></td>
+
+<td>The first value is the &lt;Minimum value for color values>, the second
+value is &lt;Maximum value for color values><sup>2</sup></td>
+</tr>
+
+<tr>
+<td>PAL_VERSION</td>
+
+<td>R</td>
+
+<td>String</td>
+
+<td>4</td>
+
+<td>"1.2"</td>
+</tr>
+</table></center>
+
+<dl>&nbsp;
+<table BORDER COLS=1 WIDTH="100%" BGCOLOR="#CCCCCC" >
+<tr>
+<td><font size=-1>1.&nbsp; The RANGE_INDEX attribute is required if the
+PAL_TYPE is "RANGEINDEX".&nbsp; Otherwise, the RANGE_INDEX attribute should
+be omitted. (Range index is denigrated.)</font></td>
+</tr>
+</table>
+<font size=-1>2.&nbsp; The minimum and maximum are optional.&nbsp; If not
+set, the range is assumed to the maximum range of the number type.&nbsp;
+If one of these attributes is set, then both should be set.&nbsp; The value
+of the minimum must be less than or equal to the value of the maximum.</font></dl>
+</dl>
+Table 5 summarized the uses of the standard attributes for a palette dataset.
+R means that the attribute listed on the leftmost column is Required for
+the palette type on the first row, O means that the attribute is Optional
+for that type and N that the attribute cannot be applied to that type.
+The four first rows show the attributes that are always required&nbsp;
+for the two palette types.
+<br>&nbsp;
+<br>&nbsp;
+<table BORDER WIDTH="100%" >
+<caption><b>Table 5. Applicability of Attributes</b></caption>
+
+<tr>
+<td WIDTH="33%"><b>PAL_TYPE</b></td>
+
+<td WIDTH="33%"><b>STANDARD8</b></td>
+
+<td WIDTH="34%"><b>RANGEINDEX</b></td>
+</tr>
+
+<tr>
+<td WIDTH="33%">CLASS</td>
+
+<td WIDTH="33%">R</td>
+
+<td WIDTH="34%">R</td>
+</tr>
+
+<tr>
+<td WIDTH="33%">PAL_VERSION</td>
+
+<td WIDTH="33%">R</td>
+
+<td WIDTH="34%">R</td>
+</tr>
+
+<tr>
+<td WIDTH="33%">PAL_COLORMODEL</td>
+
+<td WIDTH="33%">R</td>
+
+<td WIDTH="34%">R</td>
+</tr>
+
+<tr>
+<td WIDTH="33%">RANGE_INDEX</td>
+
+<td WIDTH="33%">N</td>
+
+<td WIDTH="34%">R</td>
+</tr>
+
+<tr>
+<td WIDTH="33%">PAL_MINMAXNUMERIC</td>
+
+<td WIDTH="33%">O</td>
+
+<td WIDTH="34%">O</td>
+</tr>
+</table>
+
+<h3>
+2.3. Storage Layout for Palettes</h3>
+The values of the Palette are stored as a dataset.&nbsp; The datatype can
+be any HDF 5 atomic numeric type.&nbsp; The dataset will have dimensions
+(<tt>nentries</tt>&nbsp; by&nbsp; <tt>ncomponents</tt>), where '<tt>nentries</tt>'
+is the number of colors (usually 256) and '<tt>ncomponents'</tt> is the
+number of values per color (3 for <b>RGB</b>, 4 for <b>CMYK</b>, etc.)
+<br>&nbsp;
+<h2>
+<a NAME="Sect3"></a>3.&nbsp; Consistency and Correlation of Image and Palette
+Attributes</h2>
+The objects in this specification are an extension to the base HDF5 specification
+and library.&nbsp; They are accessible with the standard HDF5 library,
+but the semantics of the objects are not enforced by the base library.&nbsp;
+For example, it is perfectly possible to add an attribute called <b>IMAGE</b>
+to <i>any</i> dataset, or to include an object reference to <i>any</i>
+HDF5 dataset in a <b>PALETTE</b> attribute.&nbsp; This would be a valid
+HDF5 file, but not conformant to this specification.&nbsp; The rules defined
+in this specification must be implemented with appropriate software, and
+applications must use conforming software to assure correctness.
+<p>The Image and Palette specifications include several redundant standard
+attributes, such as the <b>IMAGE_COLORMODEL</b> and the <b>PAL_COLORMODEL</b>.&nbsp;
+These attributes are informative not normative, in that it is acceptable
+to attach a Palette to an Image dataset even if their attributes do not
+match.&nbsp; Software is not required to enforce consistency, and files
+may contain mismatched associations of Images and Palettes.&nbsp; In all
+cases, it is up to applications to determine what kinds of images and color
+models can be supported.
+<p>For example, an Image that was created from a file with an "RGB" may
+have a "YUV" Palette in its <b>PALETTE</b> attribute array.&nbsp; This
+would be a legal HDF5 file and also conforms to this specification, although
+it may or may not be correct for a given application.</p>
+
+</body>
+</html>
diff --git a/doxygen/examples/PaletteExample1.gif b/doxygen/examples/PaletteExample1.gif
new file mode 100644
index 0000000..8694d9d
--- /dev/null
+++ b/doxygen/examples/PaletteExample1.gif
Binary files differ
diff --git a/doxygen/examples/Palettes.fm.anc.gif b/doxygen/examples/Palettes.fm.anc.gif
new file mode 100644
index 0000000..d344c03
--- /dev/null
+++ b/doxygen/examples/Palettes.fm.anc.gif
Binary files differ
diff --git a/doxygen/examples/TableSpec.html b/doxygen/examples/TableSpec.html
new file mode 100644
index 0000000..474176e
--- /dev/null
+++ b/doxygen/examples/TableSpec.html
@@ -0,0 +1,193 @@
+<html>
+<head>
+ <title>HDF5 Table Specification</title>
+</head>
+
+The HDF5 specification defines the standard objects and storage for the
+standard HDF5 objects. (For information about the HDF5 library, model and
+specification, see the HDF documentation.)&nbsp; This document is an additional
+specification do define a standard profile for how to store tables in HDF5.
+Table data in HDF5 is stored as HDF5 datasets with standard attributes to define
+the properties of the tables.
+
+<h2>
+1. Overview</h2>
+A generic table is a sequence of records, each record has a name and a type.
+Table data is stored as an HDF5 one dimensional compound dataset.&nbsp; A table
+is defined as a collection of records whose values are stored in fixed-length
+fields. All records have the same structure and all values in each field have
+the same data type.
+<p>The dataset for a table is distinguished from other datasets by giving
+it an attribute &quot;CLASS=TABLE&quot;.&nbsp;&nbsp;
+Optional attributes allow the storage of a title for the Table and for
+each column, and a fill value for each column.
+<h2>
+2.&nbsp; Table Attributes</h2>
+The attributes for the Table are strings.&nbsp;They are written with the <a href="RM_H5LT.html#H5LTset_attribute_string"><code>H5LTset_attribute_string</code></a>
+Lite API function.&nbsp; "Required" attributes must always be used. "Optional" attributes
+must be used when required.
+<br>&nbsp;
+<h4>
+Attributes</h4>
+
+<dl>
+<dt>
+Attribute name="<b>CLASS</b>" (Required)</dt>
+
+<dd>
+This attribute is type H5T_C_S1, with size 5.</dd>
+
+<dd>
+For all Tables, the value of this attribute is &quot;TABLE&quot;.</dd>
+
+<dd>
+This attribute identifies this data set as intended to be interpreted as Table that conforms to the specifications on this page.</dd>
+</dl>
+
+<dl>
+Attribute name="<b>VERSION</b>" (Required)
+
+<dd>
+This attribute is of type H5T_C_S1, with size corresponding to the length
+of the version string.&nbsp; This attribute identifies the version number
+of this specification to which it conforms.&nbsp; The current version number
+is &quot;0.2&quot;.</dd>
+
+</dl>
+
+<dl>
+<dt>
+Attribute name="<b>TITLE</b>" (Optional)</dt>
+
+<dd>
+The <b>TITLE</b> is an optional String that is to be used as the
+informative title of the whole table.
+The <b>TITLE</b> is set with the parameter <code> table_title</code> of the function
+<a href="RM_H5TB.html#H5TBmake_table"> <code> H5TBmake_table</code></a>.&nbsp;</dd>
+</dl>
+
+<dl>
+<dt>
+Attribute name="<b>FIELD_(n)_NAME</b>" (Required)</dt>
+
+<dd>
+The <b>FIELD_(n)_NAME</b> is an optional String that is to be used as the
+informative title of column <b>n</b> of the table.
+For each of the fields the word FIELD_ is concatenated with
+ the zero based field (n) index together with the name of the field.</dd>
+
+</dl>
+<dl>
+<dt>
+Attribute name="<b>FIELD_(n)_FILL</b>" (Optional)</dt>
+
+<dd>
+The <b>FIELD_(n)_FILL</b> is an optional String that is the fill value for
+column <b>n</b> of the table.
+For each of the fields the word FIELD_ is concatenated with
+ the zero based field (n) index together with the fill value, if present.
+This value is written only when a fill value is defined for the table.</dd>
+
+</dl>
+
+<dl>
+
+<br>&nbsp;
+<center><table BORDER=2 BGCOLOR="#FFFFFF" >
+<caption><b>Table 1. Attributes of an Image Dataset</b></caption>
+
+<tr>
+<td><b>Attribute Name</b></td>
+
+<td><b>(R = Required</b>
+<br><b>O= Optional)</b></td>
+
+<td><b>Type</b></td>
+
+<td><b>String Size</b></td>
+
+<td><b>Value</b></td>
+</tr>
+
+<tr>
+<td>CLASS</td>
+
+<td>R</td>
+
+<td>String</td>
+
+<td>5</td>
+
+<td>&quot;TABLE&quot;</td>
+</tr>
+
+<tr>
+<td>VERSION</td>
+
+<td>R</td>
+
+<td>String</td>
+
+<td>3</td>
+
+<td>&quot;0.2&quot;</td>
+</tr>
+
+<tr>
+<td>TITLE</td>
+
+<td>O</td>
+
+<td>String</td>
+
+<td>&nbsp;</td>
+
+<td>
+
+<tr>
+<td>FIELD_(n)_NAME</td>
+
+<td>R</td>
+
+<td>String</td>
+
+<td>&nbsp;</td>
+
+<td>
+&nbsp;
+
+<tr>
+<td>FIELD_(n)_FILL</td>
+
+<td>O*</td>
+
+<td>String</td>
+
+<td>&nbsp;</td>
+
+<td>
+&nbsp;
+</table>
+</center>
+
+ </dl>
+<p>
+<center>
+&nbsp;
+</center>
+<i>* </i>The attribute FIELD_(n)_FILL is written to the table if a fill value is
+specified on the creation of the Table. Otherwise, it is not.<p>The following
+section of code shows the calls necessary to the creation of a table.
+
+<p><code>/* Create a new HDF5 file using default properties. */<br>
+ file_id = H5Fcreate( &quot;my_table.h5&quot;, H5F_ACC_TRUNC, H5P_DEFAULT, H5P_DEFAULT );</code> </p>
+
+<p><code>/* Call the make table function */<br>
+</code> <code>H5TBmake_table( "Table Title", file_id, "Table1", NFIELDS, NRECORDS, dst_size,&nbsp;<br>
+ field_names, dst_offset, field_type,&nbsp;<br>
+ chunk_size, fill_data, compress, p_data )&nbsp;</code> </p>
+
+<p><code> /* Close the file. */<br>
+ status = H5Fclose( file_id );</code> </p>
+
+</body>
diff --git a/doxygen/examples/ThreadSafeLibrary.html b/doxygen/examples/ThreadSafeLibrary.html
new file mode 100644
index 0000000..8daf386
--- /dev/null
+++ b/doxygen/examples/ThreadSafeLibrary.html
@@ -0,0 +1,787 @@
+<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"
+ "http://www.w3.org/TR/REC-html40/loose.dtd">
+<html lang="en-US">
+<head>
+ <title>Thread Safe Library</title>
+</head>
+
+<h1>1. Library header files and conditional compilation</h1>
+
+<p>
+The following code is placed at the beginning of H5private.h:
+</p>
+
+<blockquote>
+ <pre>
+ #ifdef H5_HAVE_THREADSAFE
+ #include &lt;pthread.h&gt;
+ #endif
+ </pre>
+</blockquote>
+
+<p>
+<code>H5_HAVE_THREADSAFE</code> is defined when the HDF-5 library is
+compiled with the --enable-threadsafe configuration option. In general,
+code for the non-threadsafe version of HDF-5 library are placed within
+the <code>#else</code> part of the conditional compilation. The exception
+to this rule are the changes to the <code>FUNC_ENTER</code> (in
+H5private.h), <code>HRETURN</code> and <code>HRETURN_ERROR</code> (in
+H5Eprivate.h) macros (see section 3.2).
+</p>
+
+
+<h1>2. Global variables/structures</h1>
+
+<h2>2.1 Global library initialization variable</h2>
+
+<p>
+In the threadsafe implementation, the global library initialization
+variable <code>H5_libinit_g</code> is changed to a global structure
+consisting of the variable with its associated lock (locks are explained
+in section 4.1):
+</p>
+
+<blockquote>
+ <pre>
+ hbool_t H5_libinit_g = FALSE;
+ </pre>
+</blockquote>
+
+<p>
+becomes
+</p>
+
+<blockquote>
+ <pre>
+ H5_api_t H5_g;
+ </pre>
+</blockquote>
+
+<p>
+where <code>H5_api_t</code> is
+</p>
+
+<blockquote>
+ <pre>
+ typedef struct H5_api_struct {
+ H5_mutex_t init_lock; /* API entrance mutex */
+ hbool_t H5_libinit_g;
+ } H5_api_t;
+ </pre>
+</blockquote>
+
+<p>
+All former references to <code>H5_libinit_g</code> in the library are now
+made using the macro <code>H5_INIT_GLOBAL</code>. If the threadsafe
+library is to be used, the macro is set to <code>H5_g.H5_libinit_g</code>
+instead.
+</p>
+
+<h2>2.2 Global serialization variable</h2>
+
+<p>
+A new global boolean variable <code>H5_allow_concurrent_g</code> is used
+to determine if multiple threads are allowed to an API call
+simultaneously. This is set to <code>FALSE</code>.
+</p>
+
+<p>
+All APIs that are allowed to do so have their own local variable that
+shadows the global variable and is set to <code>TRUE</code>. In phase 1,
+no such APIs exist.
+</p>
+
+<p>
+It is defined in <code>H5.c</code> as follows:
+</p>
+
+<blockquote>
+ <pre>
+ hbool_t H5_allow_concurrent_g = FALSE;
+ </pre>
+</blockquote>
+
+<h2>2.3 Global thread initialization variable</h2>
+
+<p>
+The global variable <code>H5_first_init_g</code> of type
+<code>pthread_once_t</code> is used to allow only the first thread in the
+application process to call an initialization function using
+<code>pthread_once</code>. All subsequent calls to
+<code>pthread_once</code> by any thread are disregarded.
+</p>
+
+<p>
+The call sets up the mutex in the global structure <code>H5_g</code> (see
+section 3.1) via an initialization function
+<code>H5_first_thread_init</code>. The first thread initialization
+function is described in section 4.2.
+</p>
+
+<p>
+<code>H5_first_init_g</code> is defined in <code>H5.c</code> as follows:
+</p>
+
+<blockquote>
+ <pre>
+ pthread_once_t H5_first_init_g = PTHREAD_ONCE_INIT;
+ </pre>
+</blockquote>
+
+<h2>2.4 Global key for per-thread error stacks</h2>
+
+<p>
+A global pthread-managed key <code>H5_errstk_key_g</code> is used to
+allow pthreads to maintain a separate error stack (of type
+<code>H5E_t</code>) for each thread. This is defined in <code>H5.c</code>
+as:
+</p>
+
+<blockquote>
+ <pre>
+ pthread_key_t H5_errstk_key_g;
+ </pre>
+</blockquote>
+
+<p>
+Error stack management is described in section 4.3.
+</p>
+
+<h2>2.5 Global structure and key for thread cancellation prevention</h2>
+
+<p>
+We need to preserve the thread cancellation status of each thread
+individually by using a key <code>H5_cancel_key_g</code>. The status is
+preserved using a structure (of type <code>H5_cancel_t</code>) which
+maintains the cancellability state of the thread before it entered the
+library and a count (which works very much like the recursive lock
+counter) which keeps track of the number of API calls the thread makes
+within the library.
+</p>
+
+<p>
+The structure is defined in <code>H5private.h</code> as:
+</p>
+
+<blockquote>
+ <pre>
+ /* cancelability structure */
+ typedef struct H5_cancel_struct {
+ int previous_state;
+ unsigned int cancel_count;
+ } H5_cancel_t;
+ </pre>
+</blockquote>
+
+<p>
+Thread cancellation is described in section 4.4.
+</p>
+
+
+<h1>3. Changes to Macro expansions</h1>
+
+<h2>3.1 Changes to FUNC_ENTER</h2>
+
+<p>
+The <code>FUNC_ENTER</code> macro is now extended to include macro calls
+to initialize first threads, disable cancellability and wraps a lock
+operation around the checking of the global initialization flag. It
+should be noted that the cancellability should be disabled before
+acquiring the lock on the library. Doing so otherwise would allow the
+possibility that the thread be cancelled just after it has acquired the
+lock on the library and in that scenario, if the cleanup routines are not
+properly set, the library would be permanently locked out.
+</p>
+
+<p>
+The additional macro code and new macro definitions can be found in
+Appendix E.1 to E.5. The changes are made in <code>H5private.h</code>.
+</p>
+
+<h2>3.2 Changes to HRETURN and HRETURN_ERROR</h2>
+
+<p>
+The <code>HRETURN</code> and <code>HRETURN_ERROR</code> macros are the
+counterparts to the <code>FUNC_ENTER</code> macro described in section
+3.1. <code>FUNC_LEAVE</code> makes a macro call to <code>HRETURN</code>,
+so it is also covered here.
+</p>
+
+<p>
+The basic changes to these two macros involve adding macro calls to call
+an unlock operation and re-enable cancellability if necessary. It should
+be noted that the cancellability should be re-enabled only after the
+thread has released the lock to the library. The consequence of doing
+otherwise would be similar to that described in section 3.1.
+</p>
+
+<p>
+The additional macro code and new macro definitions can be found in
+Appendix E.9 to E.9. The changes are made in <code>H5Eprivate.h</code>.
+</p>
+
+<h1>4. Implementation of threadsafe functionality</h1>
+
+<h2>4.1 Recursive Locks</h2>
+
+<p>
+A recursive mutex lock m allows a thread t1 to successfully lock m more
+than once without blocking t1. Another thread t2 will block if t2 tries
+to lock m while t1 holds the lock to m. If t1 makes k lock calls on m,
+then it also needs to make k unlock calls on m before it releases the
+lock.
+</p>
+
+<p>
+Our implementation of recursive locks is built on top of a pthread mutex
+lock (which is not recursive). It makes use of a pthread condition
+variable to have unsuccessful threads wait on the mutex. Waiting threads
+are awaken by a signal from the final unlock call made by the thread
+holding the lock.
+</p>
+
+<p>
+Recursive locks are defined to be the following type
+(<code>H5private.h</code>):
+</p>
+
+<blockquote>
+ <pre>
+ typedef struct H5_mutex_struct {
+ pthread_t owner_thread; /* current lock owner */
+ pthread_mutex_t atomic_lock; /* lock for atomicity of new mechanism */
+ pthread_cond_t cond_var; /* condition variable */
+ unsigned int lock_count;
+ } H5_mutex_t;
+ </pre>
+</blockquote>
+
+<p>
+Detailed implementation code can be found in Appendix A. The
+implementation changes are made in <code>H5TS.c</code>.
+</p>
+
+<h2>4.2 First thread initialization</h2>
+
+<p>
+Because the mutex lock associated with a recursive lock cannot be
+statically initialized, a mechanism is required to initialize the
+recursive lock associated with <code>H5_g</code> so that it can be used
+for the first time.
+</p>
+
+<p>
+The pthreads library allows this through the pthread_once call which as
+described in section 3.3 allows only the first thread accessing the
+library in an application to initialize <code>H5_g</code>.
+</p>
+
+<p>
+In addition to initializing <code>H5_g</code>, it also initializes the
+key (see section 3.4) for use with per-thread error stacks (see section
+4.3).
+</p>
+
+<p>
+The first thread initialization mechanism is implemented as the function
+call <code>H5_first_thread_init()</code> in <code>H5TS.c</code>. This is
+described in appendix B.
+</p>
+
+<h2>4.3 Per-thread error stack management</h2>
+
+<p>
+Pthreads allows individual threads to access dynamic and persistent
+per-thread data through the use of keys. Each key is associated with
+a table that maps threads to data items. Keys can be initialized by
+<code>pthread_key_create()</code> in pthreads (see sections 3.4 and 4.2).
+Per-thread data items are accessed using a key through the
+<code>pthread_getspecific()</code> and <code>pthread_setspecific()</code>
+calls to read and write to the association table respectively.
+</p>
+
+<p>
+Per-thread error stacks are accessed through the key
+<code>H5_errstk_key_g</code> which is initialized by the first thread
+initialization call (see section 4.2).
+</p>
+
+<p>
+In the non-threadsafe version of the library, there is a global stack
+variable <code>H5E_stack_g[1]</code> which is no longer defined in the
+threadsafe version. At the same time, the macro call to gain access to
+the error stack <code>H5E_get_my_stack</code> is changed from:
+</p>
+
+<blockquote>
+ <pre>
+ #define H5E_get_my_stack() (H5E_stack_g+0)
+ </pre>
+</blockquote>
+
+<p>
+to:
+</p>
+
+<blockquote>
+ <pre>
+ #define H5E_get_my_stack() H5E_get_stack()
+ </pre>
+</blockquote>
+
+<p>
+where <code>H5E_get_stack()</code> is a surrogate function that does the
+following operations:
+</p>
+
+<ol>
+ <li>if a thread is attempting to get an error stack for the first
+ time, the error stack is dynamically allocated for the thread and
+ associated with <code>H5_errstk_key_g</code> using
+ <code>pthread_setspecific()</code>. The way we detect if it is the
+ first time is through <code>pthread_getspecific()</code> which
+ returns <code>NULL</code> if no previous value is associated with
+ the thread using the key.</li>
+
+ <li>if <code>pthread_getspecific()</code> returns a non-null value,
+ then that is the pointer to the error stack associated with the
+ thread and the stack can be used as usual.</li>
+</ol>
+
+<p>
+A final change to the error reporting routines is as follows; the current
+implementation reports errors to always be detected at thread 0. In the
+threadsafe implementation, this is changed to report the number returned
+by a call to <code>pthread_self()</code>.
+</p>
+
+<p>
+The change in code (reflected in <code>H5Eprint</code> of file
+<code>H5E.c</code>) is as follows:
+</p>
+
+<blockquote>
+ <pre>
+ #ifdef H5_HAVE_THREADSAFE
+ fprintf (stream, "HDF5-DIAG: Error detected in thread %d."
+ ,pthread_self());
+ #else
+ fprintf (stream, "HDF5-DIAG: Error detected in thread 0.");
+ #endif
+ </pre>
+</blockquote>
+
+<p>
+Code for <code>H5E_get_stack()</code> can be found in Appendix C. All the
+above changes were made in <code>H5E.c</code>.
+</p>
+
+<h2>4.4 Thread Cancellation safety</h2>
+
+<p>
+To prevent thread cancellations from killing a thread while it is in the
+library, we maintain per-thread information about the cancellability
+status of the thread before it entered the library so that we can restore
+that same status when the thread leaves the library.
+</p>
+
+<p>
+By <i>enter</i> and <i>leave</i> the library, we mean the points when a
+thread makes an API call from a user application and the time that API
+call returns. Other API or callback function calls made from within that
+API call are considered <i>within</i> the library.
+</p>
+
+<p>
+Because other API calls may be made from within the first API call, we
+need to maintain a counter to determine which was the first and
+correspondingly the last return.
+</p>
+
+<p>
+When a thread makes an API call, the macro <code>H5_API_SET_CANCEL</code>
+calls the worker function <code>H5_cancel_count_inc()</code> which does
+the following:
+</p>
+
+<ol>
+ <li>if this is the first time the thread has entered the library,
+ a new cancellability structure needs to be assigned to it.</li>
+ <li>if the thread is already within the library when the API call is
+ made, then cancel_count is simply incremented. Otherwise, we set
+ the cancellability state to <code>PTHREAD_CANCEL_DISABLE</code>
+ while storing the previous state into the cancellability structure.
+ <code>cancel_count</code> is also incremented in this case.</li>
+</ol>
+
+<p>
+When a thread leaves an API call, the macro
+<code>H5_API_UNSET_CANCEL</code> calls the worker function
+<code>H5_cancel_count_dec()</code> which does the following:
+</p>
+
+<ol>
+ <li>if <code>cancel_count</code> is greater than 1, indicating that the
+ thread is not yet about to leave the library, then
+ <code>cancel_count</code> is simply decremented.</li>
+ <li>otherwise, we reset the cancellability state back to its original
+ state before it entered the library and decrement the count (back
+ to zero).</li>
+</ol>
+
+<p>
+<code>H5_cancel_count_inc</code> and <code>H5_cancel_count_dec</code> are
+described in Appendix D and may be found in <code>H5TS.c</code>.
+</p>
+
+<h1>5. Test programs</h1>
+
+<p>
+Except where stated, all tests involve 16 simultaneous threads that make
+use of HDF-5 API calls without any explicit synchronization typically
+required in a non-threadsafe environment.
+</p>
+
+<h2>5.1 Data set create and write</h2>
+
+<p>
+The test program sets up 16 threads to simultaneously create 16
+different datasets named from <i>zero</i> to <i>fifteen</i> for a single
+file and then writing an integer value into that dataset equal to the
+dataset's named value.
+</p>
+
+<p>
+The main thread would join with all 16 threads and attempt to match the
+resulting HDF-5 file with expected results - that each dataset contains
+the correct value (0 for <i>zero</i>, 1 for <i>one</i> etc ...) and all
+datasets were correctly created.
+</p>
+
+<p>
+The test is implemented in the file <code>ttsafe_dcreate.c</code>.
+</p>
+
+<h2>5.2 Test on error stack</h2>
+
+<p>
+The error stack test is one in which 16 threads simultaneously try to
+create datasets with the same name. The result, when properly serialized,
+should be equivalent to 16 attempts to create the dataset with the same
+name.
+</p>
+
+<p>
+The error stack implementation runs correctly if it reports 15 instances
+of the dataset name conflict error and finally generates a correct HDF-5
+containing that single dataset. Each thread should report its own stack
+of errors with a thread number associated with it.
+</p>
+
+<p>
+The test is implemented in the file <code>ttsafe_error.c</code>.
+</p>
+
+<h2>5.3 Test on cancellation safety</h2>
+
+<p>
+The main idea in thread cancellation safety is as follows; a child thread
+is spawned to create and write to a dataset. Following that, it makes a
+<code>H5Diterate</code> call on that dataset which activates a callback
+function.
+</p>
+
+<p>
+A deliberate barrier is invoked at the callback function which waits for
+both the main and child thread to arrive at that point. After that
+happens, the main thread proceeds to make a thread cancel call on the
+child thread while the latter sleeps for 3 seconds before proceeding to
+write a new value to the dataset.
+</p>
+
+<p>
+After the iterate call, the child thread logically proceeds to wait
+another 3 seconds before writing another newer value to the dataset.
+</p>
+
+<p>
+The test is correct if the main thread manages to read the second value
+at the end of the test. This means that cancellation did not take place
+until the end of the iteration call despite of the 3 second wait within
+the iteration callback and the extra dataset write operation.
+Furthermore, the cancellation should occur before the child can proceed
+to write the last value into the dataset.
+</p>
+
+<h2>5.4 Test on attribute creation</h2>
+
+<p>
+A main thread makes 16 threaded calls to <code>H5Acreate</code> with a
+generated name for each attribute. Sixteen attributes should be created
+for the single dataset in random (chronological) order and receive values
+depending on its generated attribute name (e.g. <i>attrib010</i> would
+receive the value 10).
+</p>
+
+<p>
+After joining with all child threads, the main thread proceeds to read
+each attribute by generated name to see if the value tallies. Failure is
+detected if the attribute name does not exist (meaning they were never
+created) or if the wrong values were read back.
+</p>
+
+<h1>A. Recursive Lock implementation code</h1>
+
+<blockquote>
+ <pre>
+ void H5_mutex_init(H5_mutex_t *H5_mutex)
+ {
+ H5_mutex-&gt;owner_thread = NULL;
+ pthread_mutex_init(&amp;H5_mutex-&gt;atomic_lock, NULL);
+ pthread_cond_init(&amp;H5_mutex-&gt;cond_var, NULL);
+ H5_mutex-&gt;lock_count = 0;
+ }
+
+ void H5_mutex_lock(H5_mutex_t *H5_mutex)
+ {
+ pthread_mutex_lock(&amp;H5_mutex-&gt;atomic_lock);
+
+ if (pthread_equal(pthread_self(), H5_mutex-&gt;owner_thread)) {
+ /* already owned by self - increment count */
+ H5_mutex-&gt;lock_count++;
+ } else {
+ if (H5_mutex-&gt;owner_thread == NULL) {
+ /* no one else has locked it - set owner and grab lock */
+ H5_mutex-&gt;owner_thread = pthread_self();
+ H5_mutex-&gt;lock_count = 1;
+ } else {
+ /* if already locked by someone else */
+ while (1) {
+ pthread_cond_wait(&amp;H5_mutex-&gt;cond_var, &amp;H5_mutex-&gt;atomic_lock);
+
+ if (H5_mutex-&gt;owner_thread == NULL) {
+ H5_mutex-&gt;owner_thread = pthread_self();
+ H5_mutex-&gt;lock_count = 1;
+ break;
+ } /* else do nothing and loop back to wait on condition*/
+ }
+ }
+ }
+
+ pthread_mutex_unlock(&amp;H5_mutex-&gt;atomic_lock);
+ }
+
+ void H5_mutex_unlock(H5_mutex_t *H5_mutex)
+ {
+ pthread_mutex_lock(&amp;H5_mutex-&gt;atomic_lock);
+ H5_mutex-&gt;lock_count--;
+
+ if (H5_mutex-&gt;lock_count == 0) {
+ H5_mutex-&gt;owner_thread = NULL;
+ pthread_cond_signal(&amp;H5_mutex-&gt;cond_var);
+ }
+ pthread_mutex_unlock(&amp;H5_mutex-&gt;atomic_lock);
+ }
+ </pre>
+</blockquote>
+
+<h1>B. First thread initialization</h1>
+
+<blockquote>
+ <pre>
+ void H5_first_thread_init(void)
+ {
+ /* initialize global API mutex lock */
+ H5_g.H5_libinit_g = FALSE;
+ H5_g.init_lock.owner_thread = NULL;
+ pthread_mutex_init(&amp;H5_g.init_lock.atomic_lock, NULL);
+ pthread_cond_init(&amp;H5_g.init_lock.cond_var, NULL);
+ H5_g.init_lock.lock_count = 0;
+
+ /* initialize key for thread-specific error stacks */
+ pthread_key_create(&amp;H5_errstk_key_g, NULL);
+
+ /* initialize key for thread cancellability mechanism */
+ pthread_key_create(&amp;H5_cancel_key_g, NULL);
+ }
+ </pre>
+</blockquote>
+
+
+<h1>C. Per-thread error stack acquisition</h1>
+
+<blockquote>
+ <pre>
+ H5E_t *H5E_get_stack(void)
+ {
+ H5E_t *estack;
+
+ if (estack = pthread_getspecific(H5_errstk_key_g)) {
+ return estack;
+ } else {
+ /* no associated value with current thread - create one */
+ estack = (H5E_t *)malloc(sizeof(H5E_t));
+ pthread_setspecific(H5_errstk_key_g, (void *)estack);
+ return estack;
+ }
+ }
+ </pre>
+</blockquote>
+
+<h1>D. Thread cancellation mechanisms</h1>
+
+<blockquote>
+ <pre>
+ void H5_cancel_count_inc(void)
+ {
+ H5_cancel_t *cancel_counter;
+
+ if (cancel_counter = pthread_getspecific(H5_cancel_key_g)) {
+ /* do nothing here */
+ } else {
+ /*
+ * first time thread calls library - create new counter and
+ * associate with key
+ */
+ cancel_counter = (H5_cancel_t *)malloc(sizeof(H5_cancel_t));
+ cancel_counter-&gt;cancel_count = 0;
+ pthread_setspecific(H5_cancel_key_g, (void *)cancel_counter);
+ }
+
+ if (cancel_counter-&gt;cancel_count == 0) {
+ /* thread entering library */
+ pthread_setcancelstate(PTHREAD_CANCEL_DISABLE,
+ &amp;(cancel_counter-&gt;previous_state));
+ }
+
+ cancel_counter-&gt;cancel_count++;
+ }
+
+ void H5_cancel_count_dec(void)
+ {
+ H5_cancel_t *cancel_counter = pthread_getspecific(H5_cancel_key_g);
+
+ if (cancel_counter-&gt;cancel_count == 1)
+ pthread_setcancelstate(cancel_counter-&gt;previous_state, NULL);
+
+ cancel_counter-&gt;cancel_count--;
+ }
+ </pre>
+</blockquote>
+
+<h1>E. Macro expansion codes</h1>
+
+<h2>E.1 <code>FUNC_ENTER</code></h2>
+
+<blockquote>
+ <pre>
+ /* Initialize the library */ \
+ H5_FIRST_THREAD_INIT \
+ H5_API_UNSET_CANCEL \
+ H5_API_LOCK_BEGIN \
+ if (!(H5_INIT_GLOBAL)) { \
+ H5_INIT_GLOBAL = TRUE; \
+ if (H5_init_library() &lt; 0) { \
+ HRETURN_ERROR (H5E_FUNC, H5E_CANTINIT, err, \
+ "library initialization failed"); \
+ } \
+ } \
+ H5_API_LOCK_END \
+ :
+ :
+ :
+ </pre>
+</blockquote>
+
+<h2>E.2 <code>H5_FIRST_THREAD_INIT</code></h2>
+
+<blockquote>
+ <pre>
+ /* Macro for first thread initialization */
+ #define H5_FIRST_THREAD_INIT \
+ pthread_once(&amp;H5_first_init_g, H5_first_thread_init);
+ </pre>
+</blockquote>
+
+
+<h2>E.3 <code>H5_API_UNSET_CANCEL</code></h2>
+
+<blockquote>
+ <pre>
+ #define H5_API_UNSET_CANCEL \
+ if (H5_IS_API(FUNC)) { \
+ H5_cancel_count_inc(); \
+ }
+ </pre>
+</blockquote>
+
+
+<h2>E.4 <code>H5_API_LOCK_BEGIN</code></h2>
+
+<blockquote>
+ <pre>
+ #define H5_API_LOCK_BEGIN \
+ if (H5_IS_API(FUNC)) { \
+ H5_mutex_lock(&amp;H5_g.init_lock);
+ </pre>
+</blockquote>
+
+
+<h2>E.5 <code>H5_API_LOCK_END</code></h2>
+
+<blockquote>
+ <pre>
+ #define H5_API_LOCK_END }
+ </pre>
+</blockquote>
+
+
+<h2>E.6 <code>HRETURN</code> and <code>HRETURN_ERROR</code></h2>
+
+<blockquote>
+ <pre>
+ :
+ :
+ H5_API_UNLOCK_BEGIN \
+ H5_API_UNLOCK_END \
+ H5_API_SET_CANCEL \
+ return ret_val; \
+ }
+ </pre>
+</blockquote>
+
+<h2>E.7 <code>H5_API_UNLOCK_BEGIN</code></h2>
+
+<blockquote>
+ <pre>
+ #define H5_API_UNLOCK_BEGIN \
+ if (H5_IS_API(FUNC)) { \
+ H5_mutex_unlock(&amp;H5_g.init_lock);
+ </pre>
+</blockquote>
+
+<h2>E.8 <code>H5_API_UNLOCK_END</code></h2>
+
+<blockquote>
+ <pre>
+ #define H5_API_UNLOCK_END }
+ </pre>
+</blockquote>
+
+
+<h2>E.9 <code>H5_API_SET_CANCEL</code></h2>
+
+<blockquote>
+ <pre>
+ #define H5_API_SET_CANCEL \
+ if (H5_IS_API(FUNC)) { \
+ H5_cancel_count_dec(); \
+ }
+ </pre>
+</blockquote>
+
+<h2>By Chee Wai Lee</h2>
+<h4>By Bill Wendling</h4>
+
+</body>
+</html>
diff --git a/doxygen/examples/VFL.html b/doxygen/examples/VFL.html
new file mode 100644
index 0000000..9776f96
--- /dev/null
+++ b/doxygen/examples/VFL.html
@@ -0,0 +1,1601 @@
+<HTML>
+<HEAD>
+<!-- This HTML file has been created by texi2html 1.51
+ from VFL.texi on 18 November 1999 -->
+
+<TITLE>HDF5 Virtual File Layer</TITLE>
+</HEAD>
+
+
+<!--
+ * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
+ * Copyright by The HDF Group. *
+ * Copyright by the Board of Trustees of the University of Illinois. *
+ * All rights reserved. *
+ * *
+ * This file is part of HDF5. The full HDF5 copyright notice, including *
+ * terms governing use, modification, and redistribution, is contained in *
+ * the files COPYING and Copyright.html. COPYING can be found at the root *
+ * of the source code distribution tree; Copyright.html can be found at the *
+ * root level of an installed copy of the electronic HDF5 document set and *
+ * is linked from the top-level documents page. It can also be found at *
+ * http://hdfgroup.org/HDF5/doc/Copyright.html. If you do not have *
+ * access to either file, you may request a copy from help@hdfgroup.org. *
+ * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * *
+ -->
+
+
+<BODY>
+
+<strong>Revision History</strong>
+<p>Initial document, 18 November 1999.</p>
+
+<p>Updated on 10/24/00, Quincey Koziol</p>
+
+<p>Added the section &ldquo;Programming Note for C++ Developers Using C
+Functions,&rdquo; 08/23/2012, Mark Evans
+
+
+
+<P>
+<P><HR><P>
+<H1>Table of Contents</H1>
+<UL>
+<LI><A NAME="TOC1" HREF="#SEC1">Introduction</A>
+<LI><A NAME="TOC2" HREF="#SEC2">Using a File Driver</A>
+<UL>
+<LI><A NAME="TOC3" HREF="#SEC3">Driver Header Files</A>
+<LI><A NAME="TOC4" HREF="#SEC4">Creating and Opening Files</A>
+<LI><A NAME="TOC5" HREF="#SEC5">Performing I/O</A>
+<LI><A NAME="TOC6" HREF="#SEC6">File Driver Interchangeability</A>
+</UL>
+<LI><A NAME="TOC7" HREF="#SEC7">Implementation of a Driver</A>
+<UL>
+<LI><A NAME="TOC8" HREF="#SEC8">Mode Functions</A>
+<LI><A NAME="TOC9" HREF="#SEC9">File Functions</A>
+<UL>
+<LI><A NAME="TOC10" HREF="#SEC10">Opening Files</A>
+<LI><A NAME="TOC11" HREF="#SEC11">Closing Files</A>
+<LI><A NAME="TOC12" HREF="#SEC12">File Keys</A>
+<LI><A NAME="TOC13" HREF="#SEC13">Saving Modes Across Opens</A>
+</UL>
+<LI><A NAME="TOC14" HREF="#SEC14">Address Space Functions</A>
+<UL>
+<LI><A NAME="TOC15" HREF="#SEC15">Userblock and Superblock</A>
+<LI><A NAME="TOC16" HREF="#SEC16">Allocation of Format Regions</A>
+<LI><A NAME="TOC17" HREF="#SEC17">Freeing Format Regions</A>
+<LI><A NAME="TOC18" HREF="#SEC18">Querying Address Range</A>
+</UL>
+<LI><A NAME="TOC19" HREF="#SEC19">Data Functions</A>
+<UL>
+<LI><A NAME="TOC20" HREF="#SEC20">Contiguous I/O Functions</A>
+<LI><A NAME="TOC21" HREF="#SEC21">Flushing Cached Data</A>
+</UL>
+<LI><A NAME="TOC22" HREF="#SEC22">Optimization Functions</A>
+<LI><A NAME="TOC23" HREF="#SEC23">Registration of a Driver</A>
+ <ul>
+ <li><a name="TOCProgNote" href="#SECProgNote">
+ Programming Note for C++ Developers Using C Functions</a>
+ </li>
+ </ul>
+<LI><A NAME="TOC24" HREF="#SEC24">Querying Driver Information</A>
+</UL>
+<LI><A NAME="TOC25" HREF="#SEC25">Miscellaneous</A>
+</UL>
+<P><HR><P>
+
+
+<H1><A NAME="SEC1" HREF="#TOC1">Introduction</A></H1>
+
+<P>
+The HDF5 file format describes how HDF5 data structures and dataset raw
+data are mapped to a linear <STRONG>format address space</STRONG> and the HDF5
+library implements that bidirectional mapping in terms of an
+API. However, the HDF5 format specifications do <EM>not</EM> indicate how
+the format address space is mapped onto storage and HDF (version 5 and
+earlier) simply mapped the format address space directly onto a single
+file by convention.
+
+</P>
+<P>
+Since early versions of HDF5 it became apparent that users want the ability to
+map the format address space onto different types of storage (a single file,
+multiple files, local memory, global memory, network distributed global
+memory, a network protocol, <I>etc</I>.) with various types of maps. For
+instance, some users want to be able to handle very large format address
+spaces on operating systems that support only 2GB files by partitioning the
+format address space into equal-sized parts each served by a separate
+file. Other users want the same multi-file storage capability but want to
+partition the address space according to purpose (raw data in one file, object
+headers in another, global heap in a third, <I>etc.</I>) in order to improve I/O
+speeds.
+
+</P>
+<P>
+In fact, the number of storage variations is probably larger than the
+number of methods that the HDF5 team is capable of implementing and
+supporting. Therefore, a <STRONG>Virtual File Layer</STRONG> API is being
+implemented which will allow application teams or departments to design
+and implement their own mapping between the HDF5 format address space
+and storage, with each mapping being a separate <STRONG>file driver</STRONG>
+(possibly written in terms of other file drivers). The HDF5 team will
+provide a small set of useful file drivers which will also serve as
+examples for those who which to write their own:
+
+</P>
+<DL COMPACT>
+
+<DT><CODE>H5FD_SEC2</CODE>
+<DD>
+This is the default driver which uses Posix file-system functions like
+<CODE>read</CODE> and <CODE>write</CODE> to perform I/O to a single file. All I/O
+requests are unbuffered although the driver does optimize file seeking
+operations to some extent.
+
+<DT><CODE>H5FD_STDIO</CODE>
+<DD>
+This driver uses functions from <TT>`stdio.h'</TT> to perform buffered I/O
+to a single file.
+
+<DT><CODE>H5FD_CORE</CODE>
+<DD>
+This driver performs I/O directly to memory and can be used to create small
+temporary files that never exist on permanent storage. This type of storage is
+generally very fast since the I/O consists only of memory-to-memory copy
+operations.
+
+<DT><CODE>H5FD_MPIIO</CODE>
+<DD>
+This is the driver of choice for accessing files in parallel using MPI and
+MPI-IO. It is only predefined if the library is compiled with parallel I/O
+support.
+
+<DT><CODE>H5FD_FAMILY</CODE>
+<DD>
+Large format address spaces are partitioned into more manageable pieces and
+sent to separate storage locations using an underlying driver of the user's
+choice. The <CODE>h5repart</CODE> tool can be used to change the sizes of the
+family members when stored as files or to convert a family of files to a
+single file or vice versa.
+
+<DT><CODE>H5FD_SPLIT</CODE>
+<DD>
+The format address space is split into meta data and raw data and each is
+mapped onto separate storage using underlying drivers of the user's
+choice. The meta data storage can be read by itself (for limited
+functionality) or both files can be accessed together.
+</DL>
+
+
+
+<H1><A NAME="SEC2" HREF="#TOC2">Using a File Driver</A></H1>
+
+<P>
+Most application writers will use a driver defined by the HDF5 library or
+contributed by another programming team. This chapter describes how existing
+drivers are used.
+
+</P>
+
+
+
+<H2><A NAME="SEC3" HREF="#TOC3">Driver Header Files</A></H2>
+
+<P>
+Each file driver is defined in its own public header file which should
+be included by any application which plans to use that driver. The
+predefined drivers are in header files whose names begin with
+<SAMP>`H5FD'</SAMP> followed by the driver name and <SAMP>`.h'</SAMP>. The <TT>`hdf5.h'</TT>
+header file includes all the predefined driver header files.
+
+</P>
+<P>
+Once the appropriate header file is included a symbol of the form
+<SAMP>`H5FD_'</SAMP> followed by the upper-case driver name will be the driver
+identification number.<A NAME="DOCF1" HREF="#FOOT1">(1)</A> However, the
+value may change if the library is closed (<I>e.g.</I>, by calling
+<CODE>H5close</CODE>) and the symbol is referenced again.
+
+</P>
+
+
+<H2><A NAME="SEC4" HREF="#TOC4">Creating and Opening Files</A></H2>
+
+<P>
+In order to create or open a file one must define the method by which the
+storage is accessed<A NAME="DOCF2" HREF="#FOOT2">(2)</A> and does so by creating a file access property list<A NAME="DOCF3" HREF="#FOOT3">(3)</A> which is passed to the <CODE>H5Fcreate</CODE> or
+<CODE>H5Fopen</CODE> function. A default file access property list is created by
+calling <CODE>H5Pcreate</CODE> and then the file driver information is inserted by
+calling a driver initialization function such as <CODE>H5Pset_fapl_family</CODE>:
+
+</P>
+
+<PRE>
+hid_t fapl = H5Pcreate(H5P_FILE_ACCESS);
+size_t member_size = 100*1024*1024; /*100MB*/
+H5Pset_fapl_family(fapl, member_size, H5P_DEFAULT);
+hid_t file = H5Fcreate("foo%05d.h5", H5F_ACC_TRUNC, H5P_DEFAULT, fapl);
+H5Pclose(fapl);
+</PRE>
+
+<P>
+Each file driver will have its own initialization function
+whose name is <CODE>H5Pset_fapl_</CODE> followed by the driver name and which
+takes a file access property list as the first argument followed by
+additional driver-dependent arguments.
+
+</P>
+<P>
+An alternative to using the driver initialization function is to set the
+driver directly using the <CODE>H5Pset_driver</CODE> function.<A NAME="DOCF4" HREF="#FOOT4">(4)</A> Its second argument is the file driver identifier, which may
+have a different numeric value from run to run depending on the order in which
+the file drivers are registered with the library. The third argument
+encapsulates the additional arguments of the driver initialization
+function. This method only works if the file driver writer has made the
+driver-specific property list structure a public datatype, which is
+often not the case.
+
+</P>
+
+<PRE>
+hid_t fapl = H5Pcreate(H5P_FILE_ACCESS);
+static H5FD_family_fapl_t fa = {100*1024*1024, H5P_DEFAULT};
+H5Pset_driver(fapl, H5FD_FAMILY, &#38;fa);
+hid_t file = H5Fcreate("foo.h5", H5F_ACC_TRUNC, H5P_DEFAULT, fapl);
+H5Pclose(fapl);
+</PRE>
+
+<P>
+It is also possible to query the file driver information from a file access
+property list by calling <CODE>H5Pget_driver</CODE> to determine the driver and then
+calling a driver-defined query function to obtain the driver information:
+
+</P>
+
+<PRE>
+hid_t driver = H5Pget_driver(fapl);
+if (H5FD_SEC2==driver) {
+ /*nothing further to get*/
+} else if (H5FD_FAMILY==driver) {
+ hid_t member_fapl;
+ haddr_t member_size;
+ H5Pget_fapl_family(fapl, &#38;member_size, &#38;member_fapl);
+} else if (....) {
+ ....
+}
+</PRE>
+
+
+
+<H2><A NAME="SEC5" HREF="#TOC5">Performing I/O</A></H2>
+
+<P>
+The <CODE>H5Dread</CODE> and <CODE>H5Dwrite</CODE> functions transfer data between
+application memory and the file. They both take an optional data transfer
+property list which has some general driver-independent properties and
+optional driver-defined properties. An application will typically perform I/O
+in one of three styles via the <CODE>H5Dread</CODE> or <CODE>H5Dwrite</CODE> function:
+
+</P>
+<P>
+Like file access properties in the previous section, data transfer properties
+can be set using a driver initialization function or a general purpose
+function. For example, to set the MPI-IO driver to use independent access for
+I/O operations one would say:
+
+</P>
+
+<PRE>
+hid_t dxpl = H5Pcreate(H5P_DATA_XFER);
+H5Pset_dxpl_mpio(dxpl, H5FD_MPIO_INDEPENDENT);
+H5Dread(dataset, type, mspace, fspace, buffer, dxpl);
+H5Pclose(dxpl);
+</PRE>
+
+<P>
+The alternative is to initialize a driver defined C <CODE>struct</CODE> and pass it
+to the <CODE>H5Pset_driver</CODE> function:
+
+</P>
+
+<PRE>
+hid_t dxpl = H5Pcreate(H5P_DATA_XFER);
+static H5FD_mpio_dxpl_t dx = {H5FD_MPIO_INDEPENDENT};
+H5Pset_driver(dxpl, H5FD_MPIO, &#38;dx);
+H5Dread(dataset, type, mspace, fspace, buffer, dxpl);
+</PRE>
+
+<P>
+The transfer propery list can be queried in a manner similar to the file
+access property list: the driver provides a function (or functions) to return
+various information about the transfer property list:
+
+</P>
+
+<PRE>
+hid_t driver = H5Pget_driver(dxpl);
+if (H5FD_MPIO==driver) {
+ H5FD_mpio_xfer_t xfer_mode;
+ H5Pget_dxpl_mpio(dxpl, &#38;xfer_mode);
+} else {
+ ....
+}
+</PRE>
+
+
+
+<H2><A NAME="SEC6" HREF="#TOC6">File Driver Interchangeability</A></H2>
+
+<P>
+The HDF5 specifications describe two things: the mapping of data onto a linear
+<STRONG>format address space</STRONG> and the C API which performs the mapping.
+However, the mapping of the format address space onto storage intentionally
+falls outside the scope of the HDF5 specs. This is a direct result of the fact
+that it is not generally possible to store information about how to access
+storage inside the storage itself. For instance, given only the file name
+<TT>`/arborea/1225/work/f%03d'</TT> the HDF5 library is unable to tell whether the
+name refers to a file on the local file system, a family of files on the local
+file system, a file on host <SAMP>`arborea'</SAMP> port 1225, a family of files on a
+remote system, <I>etc</I>.
+
+</P>
+<P>
+Two ways which library could figure out where the storage is located are:
+storage access information can be provided by the user, or the library can try
+all known file access methods. This implementation uses the former method.
+
+</P>
+<P>
+In general, if a file was created with one driver then it isn't possible to
+open it with another driver. There are of course exceptions: a file created
+with MPIO could probably be opened with the sec2 driver, any file created
+by the sec2 driver could be opened as a family of files with one member,
+<I>etc</I>. In fact, sometimes a file must not only be opened with the same
+driver but also with the same driver properties. The predefined drivers are
+written in such a way that specifying the correct driver is sufficient for
+opening a file.
+
+</P>
+
+
+<H1><A NAME="SEC7" HREF="#TOC7">Implementation of a Driver</A></H1>
+
+<P>
+A driver is simply a collection of functions and data structures which are
+registered with the HDF5 library at runtime. The functions fall into these
+categories:
+
+</P>
+
+<UL>
+<LI>Functions which operate on modes
+
+<LI>Functions which operate on files
+
+<LI>Functions which operate on the address space
+
+<LI>Functions which operate on data
+
+<LI>Functions for driver initialization
+
+<LI>Optimization functions
+
+</UL>
+
+
+
+<H2><A NAME="SEC8" HREF="#TOC8">Mode Functions</A></H2>
+
+<P>
+Some drivers need information about file access and data transfers which are
+very specific to the driver. The information is usually implemented as a pair
+of pointers to C structs which are allocated and initialized as part of an
+HDF5 property list and passed down to various driver functions. There are two
+classes of settings: file access modes that describe how to access the file
+through the driver, and data transfer modes which are settings that control
+I/O operations. Each file opened by a particular driver may have a different
+access mode; each dataset I/O request for a particular file may have a
+different data transfer mode.
+
+</P>
+<P>
+Since each driver has its own particular requirements for various settings,
+each driver is responsible for defining the mode structures that it
+needs. Higher layers of the library treat the structures as opaque but must be
+able to copy and free them. Thus, the driver provides either the size of the
+structure or a pair of function pointers for each of the mode types.
+
+</P>
+<P>
+<STRONG>Example:</STRONG> The family driver needs to know how the format address
+space is partitioned and the file access property list to use for the
+family members.
+
+</P>
+
+<PRE>
+/* Driver-specific file access properties */
+typedef struct H5FD_family_fapl_t {
+ hsize_t memb_size; /*size of each member */
+ hid_t memb_fapl_id; /*file access property list of each memb*/
+} H5FD_family_fapl_t;
+
+/* Driver specific data transfer properties */
+typedef struct H5FD_family_dxpl_t {
+ hid_t memb_dxpl_id; /*data xfer property list of each memb */
+} H5FD_family_dxpl_t;
+</PRE>
+
+<P>
+In order to copy or free one of these structures the member file access
+or data transfer properties must also be copied or freed. This is done
+by providing a copy and close function for each structure:
+
+</P>
+<P>
+<STRONG>Example:</STRONG> The file access property list copy and close functions
+for the family driver:
+
+</P>
+
+<PRE>
+static void *
+H5FD_family_fapl_copy(const void *_old_fa)
+{
+ const H5FD_family_fapl_t *old_fa = (const H5FD_family_fapl_t*)_old_fa;
+ H5FD_family_fapl_t *new_fa = malloc(sizeof(H5FD_family_fapl_t));
+ assert(new_fa);
+
+ memcpy(new_fa, old_fa, sizeof(H5FD_family_fapl_t));
+ new_fa-&#62;memb_fapl_id = H5Pcopy(old_fa-&#62;memb_fapl_id);
+ return new_fa;
+}
+
+static herr_t
+H5FD_family_fapl_free(void *_fa)
+{
+ H5FD_family_fapl_t *fa = (H5FD_family_fapl_t*)_fa;
+ H5Pclose(fa-&#62;memb_fapl_id);
+ free(fa);
+ return 0;
+}
+</PRE>
+
+<P>
+Generally when a file is created or opened the file access properties
+for the driver are copied into the file pointer which is returned and
+they may be modified from their original value (for instance, the file
+family driver modifies the member size property when opening an existing
+family). In order to support the <CODE>H5Fget_access_plist</CODE> function the
+driver must provide a <CODE>fapl_get</CODE> callback which creates a copy of
+the driver-specific properties based on a particular file.
+
+</P>
+<P>
+<STRONG>Example:</STRONG> The file family driver copies the member size file
+access property list into the return value:
+
+</P>
+
+<PRE>
+static void *
+H5FD_family_fapl_get(H5FD_t *_file)
+{
+ H5FD_family_t *file = (H5FD_family_t*)_file;
+ H5FD_family_fapl_t *fa = calloc(1, sizeof(H5FD_family_fapl_t*));
+
+ fa-&#62;memb_size = file-&#62;memb_size;
+ fa-&#62;memb_fapl_id = H5Pcopy(file-&#62;memb_fapl_id);
+ return fa;
+}
+</PRE>
+
+
+
+<H2><A NAME="SEC9" HREF="#TOC9">File Functions</A></H2>
+
+<P>
+The higher layers of the library expect files to have a name and allow the
+file to be accessed in various modes. The driver must be able to create a new
+file, replace an existing file, or open an existing file. Opening or creating
+a file should return a handle, a pointer to a specialization of the
+<CODE>H5FD_t</CODE> struct, which allows read-only or read-write access and which
+will be passed to the other driver functions as they are
+called.<A NAME="DOCF5" HREF="#FOOT5">(5)</A>
+
+</P>
+
+<PRE>
+typedef struct {
+ /* Public fields */
+ H5FD_class_t *cls; /*class data defined below*/
+
+ /* Private fields -- driver-defined */
+
+} H5FD_t;
+</PRE>
+
+<P>
+<STRONG>Example:</STRONG> The family driver requires handles to the underlying
+storage, the size of the members for this particular file (which might be
+different than the member size specified in the file access property list if
+an existing file family is being opened), the name used to open the file in
+case additional members must be created, and the flags to use for creating
+those additional members. The <CODE>eoa</CODE> member caches the size of the format
+address space so the family members don't have to be queried in order to find
+it.
+
+</P>
+
+<PRE>
+/* The description of a file belonging to this driver. */
+typedef struct H5FD_family_t {
+ H5FD_t pub; /*public stuff, must be first */
+ hid_t memb_fapl_id; /*file access property list for members */
+ hsize_t memb_size; /*maximum size of each member file */
+ int nmembs; /*number of family members */
+ int amembs; /*number of member slots allocated */
+ H5FD_t **memb; /*dynamic array of member pointers */
+ haddr_t eoa; /*end of allocated addresses */
+ char *name; /*name generator printf format */
+ unsigned flags; /*flags for opening additional members */
+} H5FD_family_t;
+</PRE>
+
+<P>
+<STRONG>Example:</STRONG> The sec2 driver needs to keep track of the underlying Unix
+file descriptor and also the end of format address space and current Unix file
+size. It also keeps track of the current file position and last operation
+(read, write, or unknown) in order to optimize calls to <CODE>lseek</CODE>. The
+<CODE>device</CODE> and <CODE>inode</CODE> fields are defined on Unix in order to uniquely
+identify the file and will be discussed below.
+
+</P>
+
+<PRE>
+typedef struct H5FD_sec2_t {
+ H5FD_t pub; /*public stuff, must be first */
+ int fd; /*the unix file */
+ haddr_t eoa; /*end of allocated region */
+ haddr_t eof; /*end of file; current file size*/
+ haddr_t pos; /*current file I/O position */
+ int op; /*last operation */
+ dev_t device; /*file device number */
+ ino_t inode; /*file i-node number */
+} H5FD_sec2_t;
+</PRE>
+
+
+
+<H3><A NAME="SEC10" HREF="#TOC10">Opening Files</A></H3>
+
+<P>
+All drivers must define a function for opening/creating a file. This
+function should have a prototype which is:
+
+</P>
+<P>
+<DL>
+<DT><U>Function:</U> static H5FD_t * <B>open</B> <I>(const char *<VAR>name</VAR>, unsigned <VAR>flags</VAR>, hid_t <VAR>fapl</VAR>, haddr_t <VAR>maxaddr</VAR>)</I>
+<DD><A NAME="IDX1"></A>
+
+</P>
+<P>
+The file name <VAR>name</VAR> and file access property list <VAR>fapl</VAR> are
+the same as were specified in the <CODE>H5Fcreate</CODE> or <CODE>H5Fopen</CODE>
+call. The <VAR>flags</VAR> are the same as in those calls also except the
+flag <CODE>H5F_ACC_CREATE</CODE> is also present if the call was to
+<CODE>H5Fcreate</CODE> and they are documented in the <TT>`H5Fpublic.h'</TT>
+file. The <VAR>maxaddr</VAR> argument is the maximum format address that the
+driver should be prepared to handle (the minimum address is always
+zero).
+</DL>
+
+</P>
+<P>
+<STRONG>Example:</STRONG> The sec2 driver opens a Unix file with the requested name
+and saves information which uniquely identifies the file (the Unix device
+number and inode).
+
+</P>
+
+<PRE>
+static H5FD_t *
+H5FD_sec2_open(const char *name, unsigned flags, hid_t fapl_id/*unused*/,
+ haddr_t maxaddr)
+{
+ unsigned o_flags;
+ int fd;
+ struct stat sb;
+ H5FD_sec2_t *file=NULL;
+
+ /* Check arguments */
+ if (!name || !*name) return NULL;
+ if (0==maxaddr || HADDR_UNDEF==maxaddr) return NULL;
+ if (ADDR_OVERFLOW(maxaddr)) return NULL;
+
+ /* Build the open flags */
+ o_flags = (H5F_ACC_RDWR &#38; flags) ? O_RDWR : O_RDONLY;
+ if (H5F_ACC_TRUNC &#38; flags) o_flags |= O_TRUNC;
+ if (H5F_ACC_CREAT &#38; flags) o_flags |= O_CREAT;
+ if (H5F_ACC_EXCL &#38; flags) o_flags |= O_EXCL;
+
+ /* Open the file */
+ if ((fd=open(name, o_flags, 0666))&#60;0) return NULL;
+ if (fstat(fd, &#38;sb)&#60;0) {
+ close(fd);
+ return NULL;
+ }
+
+ /* Create the new file struct */
+ file = calloc(1, sizeof(H5FD_sec2_t));
+ file-&#62;fd = fd;
+ file-&#62;eof = sb.st_size;
+ file-&#62;pos = HADDR_UNDEF;
+ file-&#62;op = OP_UNKNOWN;
+ file-&#62;device = sb.st_dev;
+ file-&#62;inode = sb.st_ino;
+
+ return (H5FD_t*)file;
+}
+</PRE>
+
+
+
+<H3><A NAME="SEC11" HREF="#TOC11">Closing Files</A></H3>
+
+<P>
+Closing a file simply means that all cached data should be flushed to the next
+lower layer, the file should be closed at the next lower layer, and all
+file-related data structures should be freed. All information needed by the
+close function is already present in the file handle.
+
+</P>
+<P>
+<DL>
+<DT><U>Function:</U> static herr_t <B>close</B> <I>(H5FD_t *<VAR>file</VAR>)</I>
+<DD><A NAME="IDX2"></A>
+
+</P>
+<P>
+The <VAR>file</VAR> argument is the handle which was returned by the <CODE>open</CODE>
+function, and the <CODE>close</CODE> should free only memory associated with the
+driver-specific part of the handle (the public parts will have already been released by HDF5's virtual file layer).
+</DL>
+
+</P>
+<P>
+<STRONG>Example:</STRONG> The sec2 driver just closes the underlying Unix file,
+making sure that the actual file size is the same as that known to the
+library by writing a zero to the last file position it hasn't been
+written by some previous operation (which happens in the same code which
+flushes the file contents and is shown below).
+
+</P>
+
+<PRE>
+static herr_t
+H5FD_sec2_close(H5FD_t *_file)
+{
+ H5FD_sec2_t *file = (H5FD_sec2_t*)_file;
+
+ if (H5FD_sec2_flush(_file)&#60;0) return -1;
+ if (close(file-&#62;fd)&#60;0) return -1;
+ free(file);
+ return 0;
+}
+</PRE>
+
+
+
+<H3><A NAME="SEC12" HREF="#TOC12">File Keys</A></H3>
+
+<P>
+Occasionally an application will attempt to open a single file more than one
+time in order to obtain multiple handles to the file. HDF5 allows the files to
+share information<A NAME="DOCF6" HREF="#FOOT6">(6)</A> but in order to
+accomplish this HDF5 must be able to tell when two names refer to the same
+file. It does this by associating a driver-defined key with each file opened
+by a driver and comparing the key for an open request with the keys for all
+other files currently open by the same driver.
+
+</P>
+<P>
+<DL>
+<DT><U>Function:</U> const int <B>cmp</B> <I>(const H5FD_t *<VAR>f1</VAR>, const H5FD_t *<VAR>f2</VAR>)</I>
+<DD><A NAME="IDX3"></A>
+
+</P>
+<P>
+The driver may provide a function which compares two files <VAR>f1</VAR> and
+<VAR>f2</VAR> belonging to the same driver and returns a negative, positive, or
+zero value <I>a la</I> the <CODE>strcmp</CODE> function.<A NAME="DOCF7" HREF="#FOOT7">(7)</A> If this
+function is not provided then HDF5 assumes that all calls to the <CODE>open</CODE>
+callback return unique files regardless of the arguments and it is up to the
+application to avoid doing this if that assumption is incorrect.
+</DL>
+
+</P>
+<P>
+Each time a file is opened the library calls the <CODE>cmp</CODE> function to
+compare that file with all other files currently open by the same driver and
+if one of them matches (at most one can match) then the file which was just
+opened is closed and the previously opened file is used instead.
+
+</P>
+<P>
+Opening a file twice with incompatible flags will result in failure. For
+instance, opening a file with the truncate flag is a two step process which
+first opens the file without truncation so keys can be compared, and if no
+matching file is found already open then the file is closed and immediately
+reopened with the truncation flag set (if a matching file is already open then
+the truncating open will fail).
+
+</P>
+<P>
+<STRONG>Example:</STRONG> The sec2 driver uses the Unix device and i-node as the
+key. They were initialized when the file was opened.
+
+</P>
+
+<PRE>
+static int
+H5FD_sec2_cmp(const H5FD_t *_f1, const H5FD_t *_f2)
+{
+ const H5FD_sec2_t *f1 = (const H5FD_sec2_t*)_f1;
+ const H5FD_sec2_t *f2 = (const H5FD_sec2_t*)_f2;
+
+ if (f1-&#62;device &#60; f2-&#62;device) return -1;
+ if (f1-&#62;device &#62; f2-&#62;device) return 1;
+
+ if (f1-&#62;inode &#60; f2-&#62;inode) return -1;
+ if (f1-&#62;inode &#62; f2-&#62;inode) return 1;
+
+ return 0;
+}
+</PRE>
+
+
+
+<H3><A NAME="SEC13" HREF="#TOC13">Saving Modes Across Opens</A></H3>
+
+<P>
+Some drivers may also need to store certain information in the file superblock
+in order to be able to reliably open the file at a later date. This is done by
+three functions: one to determine how much space will be necessary to store
+the information in the superblock, one to encode the information, and one to
+decode the information. These functions are optional, but if any one is
+defined then the other two must also be defined.
+
+</P>
+<P>
+<DL>
+<DT><U>Function:</U> static hsize_t <B>sb_size</B> <I>(H5FD_t *<VAR>file</VAR>)</I>
+<DD><A NAME="IDX4"></A>
+<DT><U>Function:</U> static herr_t <B>sb_encode</B> <I>(H5FD_t *<VAR>file</VAR>, char *<VAR>name</VAR>, unsigned char *<VAR>buf</VAR>)</I>
+<DD><A NAME="IDX5"></A>
+<DT><U>Function:</U> static herr_t <B>sb_decode</B> <I>(H5FD_t *<VAR>file</VAR>, const char *<VAR>name</VAR>, const unsigned char *<VAR>buf</VAR>)</I>
+<DD><A NAME="IDX6"></A>
+
+</P>
+<P>
+The <CODE>sb_size</CODE> function returns the number of bytes necessary to encode
+information needed later if the file is reopened. The <CODE>sb_encode</CODE>
+function encodes information from the file into buffer <VAR>buf</VAR>
+allocated by the caller. It also writes an 8-character (plus null
+termination) into the <CODE>name</CODE> argument, which should be a unique
+identification for the driver. The <CODE>sb_decode</CODE> function looks at
+the <VAR>name</VAR>
+
+</P>
+<P>
+ decodes
+data from the buffer <VAR>buf</VAR> and updates the <VAR>file</VAR> argument with the new information,
+advancing <VAR>*p</VAR> in the process.
+</DL>
+
+</P>
+<P>
+The part of this which is somewhat tricky is that the file must be readable
+before the superblock information is decoded. File access modes fall outside
+the scope of the HDF5 file format, but they are placed inside the boot block
+for convenience.<A NAME="DOCF8" HREF="#FOOT8">(8)</A>
+
+</P>
+<P>
+<STRONG>Example:</STRONG> <EM>To be written later.</EM>
+
+</P>
+
+
+<H2><A NAME="SEC14" HREF="#TOC14">Address Space Functions</A></H2>
+
+<P>
+HDF5 does not assume that a file is a linear address space of bytes. Instead,
+the library will call functions to allocate and free portions of the HDF5
+format address space, which in turn map onto functions in the file driver to
+allocate and free portions of file address space. The library tells the file
+driver how much format address space it wants to allocate and the driver
+decides what format address to use and how that format address is mapped onto
+the file address space. Usually the format address is chosen so that the file
+address can be calculated in constant time for data I/O operations (which are
+always specified by format addresses).
+
+</P>
+
+
+
+<H3><A NAME="SEC15" HREF="#TOC15">Userblock and Superblock</A></H3>
+
+<P>
+The HDF5 format allows an optional userblock to appear before the actual HDF5
+data in such a way that if the userblock is <STRONG>sucked out</STRONG> of the file and
+everything remaining is shifted downward in the file address space, then the
+file is still a valid HDF5 file. The userblock size can be zero or any
+multiple of two greater than or equal to 512 and the file superblock begins
+immediately after the userblock.
+
+</P>
+<P>
+HDF5 allocates space for the userblock and superblock by calling an
+allocation function defined below, which must return a chunk of memory at
+format address zero on the first call.
+
+</P>
+
+
+<H3><A NAME="SEC16" HREF="#TOC16">Allocation of Format Regions</A></H3>
+
+<P>
+The library makes many types of allocation requests:
+
+</P>
+<DL COMPACT>
+
+<DT><CODE>H5FD_MEM_SUPER</CODE>
+<DD>
+An allocation request for the userblock and/or superblock.
+<DT><CODE>H5FD_MEM_BTREE</CODE>
+<DD>
+An allocation request for a node of a B-tree.
+<DT><CODE>H5FD_MEM_DRAW</CODE>
+<DD>
+An allocation request for the raw data of a dataset.
+<DT><CODE>H5FD_MEM_META</CODE>
+<DD>
+An allocation request for the raw data of a dataset which
+the user has indicated will be relatively small.
+<DT><CODE>H5FD_MEM_GROUP</CODE>
+<DD>
+An allocation request for a group leaf node (internal nodes of the group tree
+are allocated as H5MF_BTREE).
+<DT><CODE>H5FD_MEM_GHEAP</CODE>
+<DD>
+An allocation request for a global heap collection. Global heaps are used to
+store certain types of references such as dataset region references. The set
+of all global heap collections can become quite large.
+<DT><CODE>H5FD_MEM_LHEAP</CODE>
+<DD>
+An allocation request for a local heap. Local heaps are used to store the
+names which are members of a group. The combined size of all local heaps is a
+function of the number of object names in the file.
+<DT><CODE>H5FD_MEM_OHDR</CODE>
+<DD>
+An allocation request for (part of) an object header. Object headers are
+relatively small and include meta information about objects (like the data
+space and type of a dataset) and attributes.
+</DL>
+
+<P>
+When a chunk of memory is freed the library adds it to a free list and
+allocation requests are satisfied from the free list before requesting memory
+from the file driver. Each type of allocation request enumerated above has its
+own free list, but the file driver can specify that certain object types can
+share a free list. It does so by providing an array which maps a request type
+to a free list. If any value of the map is <CODE>H5MF_DEFAULT</CODE> (zero) then the
+object's own free list is used. The special value <CODE>H5MF_NOLIST</CODE> indicates
+that the library should not attempt to maintain a free list for that
+particular object type, instead calling the file driver each time an object of
+that type is freed.
+
+</P>
+<P>
+Mappings predefined in the <TT>`H5FDpublic.h'</TT> file are:
+<DL COMPACT>
+
+<DT><CODE>H5FD_FLMAP_SINGLE</CODE>
+<DD>
+All memory usage types are mapped to a single free list.
+<DT><CODE>H5FD_FLMAP_DICHOTOMY</CODE>
+<DD>
+Memory usage is segregated into meta data and raw data for the purposes of
+memory management.
+<DT><CODE>H5FD_FLMAP_DEFAULT</CODE>
+<DD>
+Each memory usage type has its own free list.
+</DL>
+
+<P>
+<STRONG>Example:</STRONG> To make a map that manages object headers on one free list
+and everything else on another free list one might initialize the map with the
+following code: (the use of <CODE>H5FD_MEM_SUPER</CODE> is arbitrary)
+
+</P>
+
+<PRE>
+H5FD_mem_t mt, map[H5FD_MEM_NTYPES];
+
+for (mt=0; mt&#60;H5FD_MEM_NTYPES; mt++) {
+ map[mt] = (H5FD_MEM_OHDR==mt) ? mt : H5FD_MEM_SUPER;
+}
+</PRE>
+
+<P>
+If an allocation request cannot be satisfied from the free list then one of
+two things happen. If the driver defines an allocation callback then it is
+used to allocate space; otherwise new memory is allocated from the end of the
+format address space by incrementing the end-of-address marker.
+
+</P>
+<P>
+<DL>
+<DT><U>Function:</U> static haddr_t <B>alloc</B> <I>(H5FD_t *<VAR>file</VAR>, H5MF_type_t <VAR>type</VAR>, hsize_t <VAR>size</VAR>)</I>
+<DD><A NAME="IDX7"></A>
+
+</P>
+<P>
+The <VAR>file</VAR> argument is the file from which space is to be allocated,
+<VAR>type</VAR> is the type of memory being requested (from the list above) without
+being mapped according to the freelist map and <VAR>size</VAR> is the number of
+bytes being requested. The library is allowed to allocate large chunks of
+storage and manage them in a layer above the file driver (although the current
+library doesn't do that). The allocation function should return a format
+address for the first byte allocated. The allocated region extends from that
+address for <VAR>size</VAR> bytes. If the request cannot be honored then the
+undefined address value is returned (<CODE>HADDR_UNDEF</CODE>). The first call to
+this function for a file which has never had memory allocated <EM>must</EM>
+return a format address of zero or <CODE>HADDR_UNDEF</CODE> since this is how the
+library allocates space for the userblock and/or superblock.
+</DL>
+
+</P>
+
+<P>
+<STRONG>Example:</STRONG> <EM>To be written later.</EM>
+
+</P>
+
+
+<H3><A NAME="SEC17" HREF="#TOC17">Freeing Format Regions</A></H3>
+
+<P>
+When the library is finished using a certain region of the format address
+space it will return the space to the free list according to the type of
+memory being freed and the free list map described above. If the free list has
+been disabled for a particular memory usage type (according to the free list
+map) and the driver defines a <CODE>free</CODE> callback then it will be
+invoked. The <CODE>free</CODE> callback is also invoked for all entries on the free
+list when the file is closed.
+
+</P>
+<P>
+<DL>
+<DT><U>Function:</U> static herr_t <B>free</B> <I>(H5FD_t *<VAR>file</VAR>, H5MF_type_t <VAR>type</VAR>, haddr_t <VAR>addr</VAR>, hsize_t <VAR>size</VAR>)</I>
+<DD><A NAME="IDX8"></A>
+
+</P>
+<P>
+The <VAR>file</VAR> argument is the file for which space is being freed; <VAR>type</VAR>
+is the type of object being freed (from the list above) without being mapped
+according to the freelist map; <VAR>addr</VAR> is the first format address to free;
+and <VAR>size</VAR> is the size in bytes of the region being freed. The region
+being freed may refer to just part of the region originally allocated and/or
+may cross allocation boundaries provided all regions being freed have the same
+usage type. However, the library will never attempt to free regions which have
+already been freed or which have never been allocated.
+</DL>
+
+</P>
+<P>
+A driver may choose to not define the <CODE>free</CODE> function, in which case
+format addresses will be leaked. This isn't normally a huge problem since the
+library contains a simple free list of its own and freeing parts of the format
+address space is not a common occurrence.
+
+</P>
+<P>
+<STRONG>Example:</STRONG> <EM>To be written later.</EM>
+
+</P>
+
+
+<H3><A NAME="SEC18" HREF="#TOC18">Querying Address Range</A></H3>
+
+<P>
+Each file driver must have some mechanism for setting and querying the end of
+address, or <STRONG>EOA</STRONG>, marker. The EOA marker is the first format address
+after the last format address ever allocated. If the last part of the
+allocated address range is freed then the driver may optionally decrease the
+eoa marker.
+
+</P>
+<P>
+<DL>
+<DT><U>Function:</U> static haddr_t <B>get_eoa</B> <I>(H5FD_t *<VAR>file</VAR>)</I>
+<DD><A NAME="IDX9"></A>
+
+</P>
+<P>
+This function returns the current value of the EOA marker for the specified
+file.
+</DL>
+
+</P>
+<P>
+<STRONG>Example:</STRONG> The sec2 driver just returns the current eoa marker value
+which is cached in the file structure:
+
+</P>
+
+<PRE>
+static haddr_t
+H5FD_sec2_get_eoa(H5FD_t *_file)
+{
+ H5FD_sec2_t *file = (H5FD_sec2_t*)_file;
+ return file-&#62;eoa;
+}
+</PRE>
+
+<P>
+The eoa marker is initially zero when a file is opened and the library may set
+it to some other value shortly after the file is opened (after the superblock
+is read and the saved eoa marker is determined) or when allocating additional
+memory in the absence of an <CODE>alloc</CODE> callback (described above).
+
+</P>
+<P>
+<STRONG>Example:</STRONG> The sec2 driver simply caches the eoa marker in the file
+structure and does not extend the underlying Unix file. When the file is
+flushed or closed then the Unix file size is extended to match the eoa marker.
+
+</P>
+
+<PRE>
+static herr_t
+H5FD_sec2_set_eoa(H5FD_t *_file, haddr_t addr)
+{
+ H5FD_sec2_t *file = (H5FD_sec2_t*)_file;
+ file-&#62;eoa = addr;
+ return 0;
+}
+</PRE>
+
+
+
+<H2><A NAME="SEC19" HREF="#TOC19">Data Functions</A></H2>
+
+<P>
+These functions operate on data, transferring a region of the format address
+space between memory and files.
+
+</P>
+
+
+
+<H3><A NAME="SEC20" HREF="#TOC20">Contiguous I/O Functions</A></H3>
+
+<P>
+A driver must specify two functions to transfer data from the library to the
+file and vice versa.
+
+</P>
+<P>
+<DL>
+<DT><U>Function:</U> static herr_t <B>read</B> <I>(H5FD_t *<VAR>file</VAR>, H5FD_mem_t <VAR>type</VAR>, hid_t <VAR>dxpl</VAR>, haddr_t <VAR>addr</VAR>, hsize_t <VAR>size</VAR>, void *<VAR>buf</VAR>)</I>
+<DD><A NAME="IDX10"></A>
+<DT><U>Function:</U> static herr_t <B>write</B> <I>(H5FD_t *<VAR>file</VAR>, H5FD_mem_t <VAR>type</VAR>, hid_t <VAR>dxpl</VAR>, haddr_t <VAR>addr</VAR>, hsize_t <VAR>size</VAR>, const void *<VAR>buf</VAR>)</I>
+<DD><A NAME="IDX11"></A>
+
+</P>
+<P>
+The <CODE>read</CODE> function reads data from file <VAR>file</VAR> beginning at address
+<VAR>addr</VAR> and continuing for <VAR>size</VAR> bytes into the buffer <VAR>buf</VAR>
+supplied by the caller. The <CODE>write</CODE> function transfers data in the
+opposite direction. Both functions take a data transfer property list
+<VAR>dxpl</VAR> which indicates the fine points of how the data is to be
+transferred and which comes directly from the <CODE>H5Dread</CODE> or
+<CODE>H5Dwrite</CODE> function. Both functions receive <VAR>type</VAR> of
+data being written, which may allow a driver to tune it's behavior for
+different kinds of data.
+</DL>
+
+</P>
+<P>
+Both functions should return a negative value if they fail to transfer the
+requested data, or non-negative if they succeed. The library will never
+attempt to read from unallocated regions of the format address space.
+
+</P>
+<P>
+<STRONG>Example:</STRONG> The sec2 driver just makes system calls. It tries not to
+call <CODE>lseek</CODE> if the current operation is the same as the previous
+operation and the file position is correct. It also fills the output buffer
+with zeros when reading between the current EOF and EOA markers and restarts
+system calls which were interrupted.
+
+</P>
+
+<PRE>
+static herr_t
+H5FD_sec2_read(H5FD_t *_file, H5FD_mem_t type/*unused*/, hid_t dxpl_id/*unused*/,
+ haddr_t addr, hsize_t size, void *buf/*out*/)
+{
+ H5FD_sec2_t *file = (H5FD_sec2_t*)_file;
+ ssize_t nbytes;
+
+ assert(file &#38;&#38; file-&#62;pub.cls);
+ assert(buf);
+
+ /* Check for overflow conditions */
+ if (REGION_OVERFLOW(addr, size)) return -1;
+ if (addr+size&#62;file-&#62;eoa) return -1;
+
+ /* Seek to the correct location */
+ if ((addr!=file-&#62;pos || OP_READ!=file-&#62;op) &#38;&#38;
+ file_seek(file-&#62;fd, (file_offset_t)addr, SEEK_SET)&#60;0) {
+ file-&#62;pos = HADDR_UNDEF;
+ file-&#62;op = OP_UNKNOWN;
+ return -1;
+ }
+
+ /*
+ * Read data, being careful of interrupted system calls, partial results,
+ * and the end of the file.
+ */
+ while (size&#62;0) {
+ do nbytes = read(file-&#62;fd, buf, size);
+ while (-1==nbytes &#38;&#38; EINTR==errno);
+ if (-1==nbytes) {
+ /* error */
+ file-&#62;pos = HADDR_UNDEF;
+ file-&#62;op = OP_UNKNOWN;
+ return -1;
+ }
+ if (0==nbytes) {
+ /* end of file but not end of format address space */
+ memset(buf, 0, size);
+ size = 0;
+ }
+ assert(nbytes&#62;=0);
+ assert((hsize_t)nbytes&#60;=size);
+ size -= (hsize_t)nbytes;
+ addr += (haddr_t)nbytes;
+ buf = (char*)buf + nbytes;
+ }
+
+ /* Update current position */
+ file-&#62;pos = addr;
+ file-&#62;op = OP_READ;
+ return 0;
+}
+</PRE>
+
+<P>
+<STRONG>Example:</STRONG> The sec2 <CODE>write</CODE> callback is similar except it updates
+the file EOF marker when extending the file.
+
+</P>
+
+
+<H3><A NAME="SEC21" HREF="#TOC21">Flushing Cached Data</A></H3>
+
+<P>
+Some drivers may desire to cache data in memory in order to make larger I/O
+requests to the underlying file and thus improving bandwidth. Such drivers
+should register a cache flushing function so that the library can insure that
+data has been flushed out of the drivers in response to the application
+calling <CODE>H5Fflush</CODE>.
+
+</P>
+<P>
+<DL>
+<DT><U>Function:</U> static herr_t <B>flush</B> <I>(H5FD_t *<VAR>file</VAR>)</I>
+<DD><A NAME="IDX12"></A>
+
+</P>
+<P>
+Flush all data for file <VAR>file</VAR> to storage.
+</DL>
+
+</P>
+<P>
+<STRONG>Example:</STRONG> The sec2 driver doesn't cache any data but it also doesn't
+extend the Unix file as agressively as it should. Therefore, when finalizing a
+file it should write a zero to the last byte of the allocated region so that
+when reopening the file later the EOF marker will be at least as large as the
+EOA marker saved in the superblock (otherwise HDF5 will refuse to open the
+file, claiming that the data appears to be truncated).
+
+</P>
+
+<PRE>
+static herr_t
+H5FD_sec2_flush(H5FD_t *_file)
+{
+ H5FD_sec2_t *file = (H5FD_sec2_t*)_file;
+
+ if (file-&#62;eoa&#62;file-&#62;eof) {
+ if (-1==file_seek(file-&#62;fd, file-&#62;eoa-1, SEEK_SET)) return -1;
+ if (write(file-&#62;fd, "", 1)!=1) return -1;
+ file-&#62;eof = file-&#62;eoa;
+ file-&#62;pos = file-&#62;eoa;
+ file-&#62;op = OP_WRITE;
+ }
+
+ return 0;
+}
+</PRE>
+
+
+
+<H2><A NAME="SEC22" HREF="#TOC22">Optimization Functions</A></H2>
+
+<P>
+The library is capable of performing several generic optimizations on I/O, but
+these types of optimizations may not be appropriate for a given VFL driver.
+</P>
+
+<P>
+Each driver may provide a query function to allow the library to query whether
+to enable these optimizations. If a driver lacks a query function, the library
+will disable all types of optimizations which can be queried.
+</P>
+
+<P>
+<DL>
+<DT><U>Function:</U> static herr_t <B>query</B> <I>(const H5FD_t *<VAR>file</VAR>, unsigned long *<VAR>flags</VAR>)</I>
+<DD><A NAME="IDX17"></A>
+</P>
+<P>
+This function is called by the library to query which optimizations to enable
+for I/O to this driver. These are the flags which are currently defined:
+
+<UL>
+<DL>
+<DT>H5FD_FEAT_AGGREGATE_METADATA (0x00000001)
+<DD>Defining the H5FD_FEAT_AGGREGATE_METADATA for a VFL driver means that
+the library will attempt to allocate a larger block for metadata and
+then sub-allocate each metadata request from that larger block.
+<DT>H5FD_FEAT_ACCUMULATE_METADATA (0x00000002)
+<DD>Defining the H5FD_FEAT_ACCUMULATE_METADATA for a VFL driver means that
+the library will attempt to cache metadata as it is written to the file
+and build up a larger block of metadata to eventually pass to the VFL
+'write' routine.
+<DT>H5FD_FEAT_DATA_SIEVE (0x00000004)
+<DD>Defining the H5FD_FEAT_DATA_SIEVE for a VFL driver means that
+the library will attempt to cache raw data as it is read from/written to
+a file in a "data sieve" buffer. See Rajeev Thakur's papers:
+ <UL>
+ <DL>
+ <DT>http://www.mcs.anl.gov/~thakur/papers/romio-coll.ps.gz
+ <DT>http://www.mcs.anl.gov/~thakur/papers/mpio-high-perf.ps.gz
+ </DL>
+ </UL>
+</DL>
+</UL>
+</P>
+
+</DL>
+</P>
+
+<H2><A NAME="SEC23" HREF="#TOC23">Registration of a Driver</A></H2>
+
+<P>
+Before a driver can be used the HDF5 library needs to be told of its
+existence. This is done by registering the driver, which results in a driver
+identification number. Instead of passing many arguments to the registration
+function, the driver information is entered into a structure and the address
+of the structure is passed to the registration function where it is
+copied. This allows the HDF5 API to be extended while providing backward
+compatibility at the source level.
+
+</P>
+<P>
+<DL>
+<DT><U>Function:</U> hid_t <B>H5FDregister</B> <I>(H5FD_class_t *<VAR>cls</VAR>)</I>
+<DD><A NAME="IDX13"></A>
+
+</P>
+<P>
+The driver described by struct <VAR>cls</VAR> is registered with the library and an
+ID number for the driver is returned.
+</DL>
+
+</P>
+<P>
+The <CODE>H5FD_class_t</CODE> type is a struct with the following fields:
+
+</P>
+<DL COMPACT>
+
+<DT><CODE>const char *name</CODE>
+<DD>
+A pointer to a constant, null-terminated driver name to be used for debugging
+purposes.
+<DT><CODE>size_t fapl_size</CODE>
+<DD>
+The size in bytes of the file access mode structure or zero if the driver
+supplies a copy function or doesn't define the structure.
+<DT><CODE>void *(*fapl_copy)(const void *fapl)</CODE>
+<DD>
+An optional function which copies a driver-defined file access mode structure.
+This field takes precedence over <CODE>fm_size</CODE> when both are defined.
+<DT><CODE>void (*fapl_free)(void *fapl)</CODE>
+<DD>
+An optional function to free the driver-defined file access mode structure. If
+null, then the library calls the C <CODE>free</CODE> function to free the
+structure.
+<DT><CODE>size_t dxpl_size</CODE>
+<DD>
+The size in bytes of the data transfer mode structure or zero if the driver
+supplies a copy function or doesn't define the structure.
+<DT><CODE>void *(*dxpl_copy)(const void *dxpl)</CODE>
+<DD>
+An optional function which copies a driver-defined data transfer mode
+structure. This field takes precedence over <CODE>xm_size</CODE> when both are
+defined.
+<DT><CODE>void (*dxpl_free)(void *dxpl)</CODE>
+<DD>
+An optional function to free the driver-defined data transfer mode
+structure. If null, then the library calls the C <CODE>free</CODE> function to
+free the structure.
+<DT><CODE>H5FD_t *(*open)(const char *name, unsigned flags, hid_t fapl, haddr_t maxaddr)</CODE>
+<DD>
+The function which opens or creates a new file.
+<DT><CODE>herr_t (*close)(H5FD_t *file)</CODE>
+<DD>
+The function which ends access to a file.
+<DT><CODE>int (*cmp)(const H5FD_t *f1, const H5FD_t *f2)</CODE>
+<DD>
+An optional function to determine whether two open files have the same key. If
+this function is not present then the library assumes that two files will
+never be the same.
+<DT><CODE>int (*query)(const H5FD_t *f, unsigned long *flags)</CODE>
+<DD>
+An optional function to determine which library optimizations a driver can
+support.
+<DT><CODE>haddr_t (*alloc)(H5FD_t *file, H5FD_mem_t type, hsize_t size)</CODE>
+<DD>
+An optional function to allocate space in the file.
+<DT><CODE>herr_t (*free)(H5FD_t *file, H5FD_mem_t type, haddr_t addr, hsize_t size)</CODE>
+<DD>
+An optional function to free space in the file.
+<DT><CODE>haddr_t (*get_eoa)(H5FD_t *file)</CODE>
+<DD>
+A function to query how much of the format address space has been allocated.
+<DT><CODE>herr_t (*set_eoa)(H5FD_t *file, haddr_t)</CODE>
+<DD>
+A function to set the end of address space.
+<DT><CODE>haddr_t (*get_eof)(H5FD_t *file)</CODE>
+<DD>
+A function to return the current end-of-file marker value.
+<DT><CODE>herr_t (*read)(H5FD_t *file, H5FD_mem_t type, hid_t dxpl, haddr_t addr, hsize_t size, void *buffer)</CODE>
+<DD>
+A function to read data from a file.
+<DT><CODE>herr_t (*write)(H5FD_t *file, H5FD_mem_t type, hid_t dxpl, haddr_t addr, hsize_t size, const void *buffer)</CODE>
+<DD>
+A function to write data to a file.
+<DT><CODE>herr_t (*flush)(H5FD_t *file)</CODE>
+<DD>
+A function which flushes cached data to the file.
+<DT><CODE>H5FD_mem_t fl_map[H5FD_MEM_NTYPES]</CODE>
+<DD>
+An array which maps a file allocation request type to a free list.
+</DL>
+
+<P>
+<STRONG>Example:</STRONG> The sec2 driver would be registered as:
+
+</P>
+
+<PRE>
+static const H5FD_class_t H5FD_sec2_g = {
+ "sec2", /*name */
+ MAXADDR, /*maxaddr */
+ NULL, /*sb_size */
+ NULL, /*sb_encode */
+ NULL, /*sb_decode */
+ 0, /*fapl_size */
+ NULL, /*fapl_get */
+ NULL, /*fapl_copy */
+ NULL, /*fapl_free */
+ 0, /*dxpl_size */
+ NULL, /*dxpl_copy */
+ NULL, /*dxpl_free */
+ H5FD_sec2_open, /*open */
+ H5FD_sec2_close, /*close */
+ H5FD_sec2_cmp, /*cmp */
+ H5FD_sec2_query, /*query */
+ NULL, /*alloc */
+ NULL, /*free */
+ H5FD_sec2_get_eoa, /*get_eoa */
+ H5FD_sec2_set_eoa, /*set_eoa */
+ H5FD_sec2_get_eof, /*get_eof */
+ H5FD_sec2_read, /*read */
+ H5FD_sec2_write, /*write */
+ H5FD_sec2_flush, /*flush */
+ H5FD_FLMAP_SINGLE, /*fl_map */
+};
+
+hid_t
+H5FD_sec2_init(void)
+{
+ if (!H5FD_SEC2_g) {
+ H5FD_SEC2_g = H5FDregister(&#38;H5FD_sec2_g);
+ }
+ return H5FD_SEC2_g;
+}
+</PRE>
+
+<P>
+A driver can be removed from the library by unregistering it
+
+</P>
+<P>
+<DL>
+<DT><U>Function:</U> herr_t <B>H5Dunregister</B> <I>(hid_t <VAR>driver</VAR>)</I>
+<DD><A NAME="IDX14"></A>
+Where <VAR>driver</VAR> is the ID number returned when the driver was registered.
+</DL>
+
+</P>
+<P>
+Unregistering a driver makes it unusable for creating new file access or data
+transfer property lists but doesn't affect any property lists or files that
+already use that driver.
+
+</P>
+
+
+
+
+<H3><A NAME="SECProgNote" HREF="#TOCProgNote">Programming Note
+for C++ Developers Using C Functions</A></H3>
+
+<p>If a C routine that takes a function pointer as an argument is
+called from within C++ code, the C routine should be returned from
+normally. </p>
+
+<p>Examples of this kind of routine include callbacks such as
+<code>H5Pset_elink_cb</code> and <code>H5Pset_type_conv_cb</code>
+and functions such as <code>H5Tconvert</code> and
+<code>H5Ewalk2</code>.</p>
+
+<p>Exiting the routine in its normal fashion allows the HDF5 C
+Library to clean up its work properly. In other words, if the C++
+application jumps out of the routine back to the C++
+&ldquo;catch&rdquo; statement, the library is not given the
+opportunity to close any temporary data structures that were set
+up when the routine was called. The C++ application should save
+some state as the routine is started so that any problem that
+occurs might be diagnosed.</p>
+
+
+
+
+
+
+
+<H2><A NAME="SEC24" HREF="#TOC24">Querying Driver Information</A></H2>
+
+<P>
+<DL>
+<DT><U>Function:</U> void * <B>H5Pget_driver_data</B> <I>(hid_t <VAR>fapl</VAR>)</I>
+<DD><A NAME="IDX15"></A>
+<DT><U>Function:</U> void * <B>H5Pget_driver_data</B> <I>(hid_t <VAR>fxpl</VAR>)</I>
+<DD><A NAME="IDX16"></A>
+
+</P>
+<P>
+This function is intended to be used by driver functions, not applications.
+It returns a pointer directly into the file access property list
+<CODE><VAR>fapl</VAR></CODE> which is a copy of the driver's file access mode originally
+provided to the <CODE>H5Pset_driver</CODE> function. If its argument is a data
+transfer property list <CODE>fxpl</CODE> then it returns a pointer to the
+driver-specific data transfer information instead.
+</DL>
+
+</P>
+
+
+
+<H1><A NAME="SEC25" HREF="#TOC25">Miscellaneous</A></H1>
+
+<P>
+The various private <CODE>H5F_low_*</CODE> functions will be replaced by public
+<CODE>H5FD*</CODE> functions so they can be called from drivers.
+
+</P>
+<P>
+All private functions <CODE>H5F_addr_*</CODE> which operate on addresses will be
+renamed as public functions by removing the first underscore so they can be
+called by drivers.
+
+</P>
+<P>
+The <CODE>haddr_t</CODE> address data type will be passed by value throughout the
+library. The original intent was that this type would eventually be a union of
+file address types for the various drivers and may become quite large, but
+that was back when drivers were part of HDF5. It will become an alias for an
+unsigned integer type (32 or 64 bits depending on how the library was
+configured).
+
+</P>
+<P>
+The various <CODE>H5F*.c</CODE> driver files will be renamed <CODE>H5FD*.c</CODE> and each
+will have a corresponding header file. All driver functions except the
+initializer and API will be declared static.
+
+</P>
+<P>
+This documentation didn't cover optimization functions which would be useful
+to drivers like MPI-IO. Some drivers may be able to perform data pipeline
+operations more efficiently than HDF5 and need to be given a chance to
+override those parts of the pipeline. The pipeline would be designed to call
+various H5FD optimization functions at various points which return one of
+three values: the operation is not implemented by the driver, the operation is
+implemented but failed in a non-recoverable manner, the operation is
+implemented and succeeded.
+
+</P>
+<P>
+Various parts of HDF5 check the only the top-level file driver and do
+something special if it is the MPI-IO driver. However, we might want to be
+able to put the MPI-IO driver under other drivers such as the raw part of a
+split driver or under a debug driver whose sole purpose is to accumulate
+statistics as it passes all requests through to the MPI-IO driver. Therefore
+we will probably need a function which takes a format address and or object
+type and returns the driver which would have been used at the lowest level to
+process the request.
+
+</P>
+
+<P><HR><P>
+<H1>Footnotes</H1>
+<H3><A NAME="FOOT1" HREF="#DOCF1">(1)</A></H3>
+<P>The driver name is by convention and might
+not apply to drivers which are not distributed with HDF5.
+<H3><A NAME="FOOT2" HREF="#DOCF2">(2)</A></H3>
+<P>The access method also indicates how to translate
+the storage name to a storage server such as a file, network protocol, or
+memory.
+<H3><A NAME="FOOT3" HREF="#DOCF3">(3)</A></H3>
+<P>The term
+"<EM>file</EM> access property list" is a misnomer since storage isn't
+required to be a file.
+<H3><A NAME="FOOT4" HREF="#DOCF4">(4)</A></H3>
+<P>This
+function is overloaded to operate on data transfer property lists also, as
+described below.
+<H3><A NAME="FOOT5" HREF="#DOCF5">(5)</A></H3>
+<P>Read-only access is only appropriate when opening an existing
+file.
+<H3><A NAME="FOOT6" HREF="#DOCF6">(6)</A></H3>
+<P>For instance, writing data to one handle will cause
+the data to be immediately visible on the other handle.
+<H3><A NAME="FOOT7" HREF="#DOCF7">(7)</A></H3>
+<P>The ordering is
+arbitrary as long as it's consistent within a particular file driver.
+<H3><A NAME="FOOT8" HREF="#DOCF8">(8)</A></H3>
+<P>File access modes do not describe data, but rather
+describe how the HDF5 format address space is mapped to the underlying
+file(s). Thus, in general the mapping must be known before the file superblock
+can be read. However, the user usually knows enough about the mapping for the
+superblock to be readable and once the superblock is read the library can fill
+in the missing parts of the mapping.
+<P><HR><P>
+
+<?php include("../ed_libs/Footer2.htm"); ?>
+
+</BODY>
+</HTML>