diff options
| author | Frank Baker <fbaker@hdfgroup.org> | 2005-07-19 17:28:56 (GMT) |
|---|---|---|
| committer | Frank Baker <fbaker@hdfgroup.org> | 2005-07-19 17:28:56 (GMT) |
| commit | 794ba0a251af47b8e3c60afa2fe92d267e2a6b55 (patch) | |
| tree | f24cea3b81ff02fa3f31c0a1c4e80fa10f4393c0 /doc/html/H5.format.html | |
| parent | d2e92fd23610c3ccdddbbc55484e54a5a21a9252 (diff) | |
| download | hdf5-794ba0a251af47b8e3c60afa2fe92d267e2a6b55.zip hdf5-794ba0a251af47b8e3c60afa2fe92d267e2a6b55.tar.gz hdf5-794ba0a251af47b8e3c60afa2fe92d267e2a6b55.tar.bz2 | |
[svn-r11084]
Description:
All HDF5 user documentation has been moved to a separate hdf5doc/
repository, managed under Subversion.
With this 'cvs commit', all files are stripped from hdf5/doc/.
THIS CHANGE IS APPLIED ONLY TO THE HDF5 DEVELOPMENT BRANCH,
post Release 1.6.x; it is not applied to the release branches.
Diffstat (limited to 'doc/html/H5.format.html')
| -rw-r--r-- | doc/html/H5.format.html | 5956 |
1 files changed, 0 insertions, 5956 deletions
diff --git a/doc/html/H5.format.html b/doc/html/H5.format.html deleted file mode 100644 index 8d2ba87..0000000 --- a/doc/html/H5.format.html +++ /dev/null @@ -1,5956 +0,0 @@ -<html> - <head> - <title> - HDF5 File Format Specification - </title> - -<STYLE TYPE="text/css"> - -P { text-indent: 2em} -P.item { margin-left: 2em; text-indent: -2em} -P.item2 { margin-left: 2em; text-indent: 2em} - -TABLE.format { border:solid; border-collapse:collapse; caption-side:top; text-align:center; width:80%;} -TABLE.format TH { border:ridge; padding:4px; width:25%;} -TABLE.format TD { border:ridge; padding:4px; } -TABLE.format CAPTION { font-weight:bold; font-size:larger;} - -TABLE.note {border:none; text-align:right; width:80%;} - -TABLE.desc { border:solid; border-collapse:collapse; caption-size:top; text-align:left; width:80%;} -TABLE.desc TR { vertical-align:top;} -TABLE.desc TH { border-style:ridge; font-size:larger; padding:4px; text-decoration:underline;} -TABLE.desc TD { border-style:ridge; padding:4px; } -TABLE.desc CAPTION { font-weight:bold; font-size:larger;} - -TABLE.list { border:none; } -TABLE.list TR { vertical-align:top;} -TABLE.list TH { border:none; text-decoration:underline;} -TABLE.list TD { border:none; } - -</STYLE> - -<!-- #BeginLibraryItem "/ed_libs/styles_Format.lbi" --> -<!-- - * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * - * Copyright by the Board of Trustees of the University of Illinois. * - * All rights reserved. * - * * - * This file is part of HDF5. The full HDF5 copyright notice, including * - * terms governing use, modification, and redistribution, is contained in * - * the files COPYING and Copyright.html. COPYING can be found at the root * - * of the source code distribution tree; Copyright.html can be found at the * - * root level of an installed copy of the electronic HDF5 document set and * - * is linked from the top-level documents page. It can also be found at * - * http://hdf.ncsa.uiuc.edu/HDF5/doc/Copyright.html. If you do not have * - * access to either file, you may request a copy from hdfhelp@ncsa.uiuc.edu. * - * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * * - --> - -<link href="ed_styles/FormatElect.css" rel="stylesheet" type="text/css"> -<!-- #EndLibraryItem --></head> - <body bgcolor="#FFFFFF"> - - -<!-- #BeginLibraryItem "/ed_libs/NavBar_ADevG.lbi" --><hr> -<center> -<table border=0 width=98%> -<tr><td valign=top align=left> - <a href="index.html">HDF5 documents and links</a> <br> - <a href="H5.intro.html">Introduction to HDF5</a> <br> - <!-- - <a href="Glossary.html">Glossary</a><br> - --> -</td> -<td valign=top align=right> - <a href="http://hdf.ncsa.uiuc.edu/HDF5/doc/UG/index.html">HDF5 User's Guide</a> <br> - <a href="RM_H5Front.html">HDF5 Reference Manual</a> <br> - <a href="ADGuide.html">HDF5 Application Developer's Guide</a> <br> -</td></tr> -</table> -</center> -<hr> -<!-- #EndLibraryItem --><center><h1>HDF5 File Format Specification</h1></center> - - <center> - <table border=0 width=90%> - <tr> - <td valign=top> - <ol type=I> - <li><a href="#Intro">Introduction</a> - <li><a href="#FileMetaData">Disk Format Level 0 - File Metadata</a> - <font size=-2> - <ol type=A> - <li><a href="#SuperBlock">Disk Format Level 0A - File Signature and Super Block</a> - <li><a href="#DriverInfo">Disk Format Level 0B - File Driver Info</a> - </ol> - </font> - <li><a href="#FileInfra">Disk Format Level 1 - File Infrastructure</a> - <font size=-2> - <ol type=A> - <li><a href="#Btrees">Disk Format Level 1A - B-link Trees and B-tree Nodes</a> - <li><a href="#SymbolTable">Disk Format Level 1B - Group</a> - <li><a href="#SymbolTableEntry">Disk Format Level 1C - Group Entry</a> - <li><a href="#LocalHeap">Disk Format Level 1D - Local Heaps</a> - <li><a href="#GlobalHeap">Disk Format Level 1E - Global Heap</a> - <li><a href="#FreeSpaceIndex">Disk Format Level 1F - Free-space Index</a> - </ol> - </font> - <li><a href="#DataObject">Disk Format Level 2 - Data Objects</a> - <font size=-2> - <ol type=A> - <li><a href="#ObjectHeader">Disk Format Level 2a - Data Object Headers</a> - <ol type=1> - <li><a href="#NILMessage">Name: NIL</a> <!-- 0x0000 --> - <li><a href="#SimpleDataSpace">Name: Simple Dataspace</a> <!-- 0x0001 --> -<!-- <li><a href="#DataSpaceMessage">Name: Complex Dataspace</a> --> <!-- 0x0002 --> - <li><a href="#ReservedMessage_0002">Name: Reserved - not assigned yet</a> <!-- 0x0002 --> - <li><a href="#DataTypeMessage">Name: Datatype</a> <!-- 0x0003 --> - <li><a href="#OldFillValueMessage">Name: Data Storage - Fill Value (Old)</a> <!-- 0x0004 --> - <li><a href="#FillValueMessage">Name: Data Storage - Fill Value</a> <!-- 0x0005 --> - </ol> - </ol> - </font> - </ol> - </td><td> </td><td valign=top> - <ol type=I start=4> - - <li><a href="#DataObject">Disk Format Level 2 - Data Objects</a> - <font size=-2><i>(Continued)</i> - <ol type=A> - <li><a href="#ObjectHeader">Disk Format Level 2a - Data Object Headers</a><i>(Continued)</i> - <ol type=1 start=6> -<!-- <li><a href="#CompactDataStorageMessage">Name: Data Storage - Compact</a> --> <!-- 0x0006 --> - <li><a href="#ReservedMessage_0006">Name: Reserved - not assigned yet</a> <!-- 0x0006 --> - <li><a href="#ExternalFileListMessage">Name: Data Storage - External Data Files</a> <!-- 0x0007 --> - <li><a href="#LayoutMessage">Name: Data Storage - Layout</a> <!-- 0x0008 --> - <li><a href="#ReservedMessage_0009">Name: Reserved - not assigned yet</a> <!-- 0x0009 --> - <li><a href="#ReservedMessage_000A">Name: Reserved - not assigned yet</a> <!-- 0x000a --> - <li><a href="#FilterMessage">Name: Data Storage - Filter Pipeline</a> <!-- 0x000b --> - <li><a href="#AttributeMessage">Name: Attribute</a> <!-- 0x000c --> - <li><a href="#CommentMessage">Name: Object Comment</a> <!-- 0x000d --> - <li><a href="#OldModifiedMessage">Name: Object Modification Date and Time (Old)</a> <!-- 0x000e --> - <li><a href="#SharedMessage">Name: Shared Object Message</a> <!-- 0x000f --> - <li><a href="#ContinuationMessage">Name: Object Header Continuation</a> <!-- 0x0010 --> - <li><a href="#SymbolTableMessage">Name: Group Message</a> <!-- 0x0011 --> - <li><a href="#ModifiedMessage">Name: Object Modification Date and Time</a> <!-- 0x0012 --> - </ol> - <li><a href="#SharedObjectHeader">Disk Format: Level 2b - Shared Data Object Headers</a> - <li><a href="#DataStorage">Disk Format: Level 2c - Data Object Data Storage</a> - </ol> - </font> - <LI><A href="#Appendix">Appendix</A> - </ol> -</td></tr> -</table> -</center> - - <BR> - <HR> - - - <h2>Introduction</h2> - - <table align=right width=100> - <tr><td> </td><td align=center> - <hr> - <img src="FF-IH_FileGroup.gif" alt="HDF5 Groups" hspace=15 vspace=15> - </td><td> </td></tr> - <tr><td> </td><td align=center> - <strong>Figure 1:</strong> Relationships among the HDF5 root group, other groups, and objects - <hr> - </td><td> </td></tr> - - <tr><td> </td><td align=center> - <img src="FF-IH_FileObject.gif" alt="HDF5 Objects" hspace=15 vspace=15> - </td><td> </td></tr> - <tr><td> </td><td align=center> - <strong>Figure 2:</strong> HDF5 objects -- datasets, datatypes, or dataspaces - <hr> - </td><td> </td></tr> - </table> - - - <P>The format of an HDF5 file on disk encompasses several - key ideas of the HDF4 and AIO file formats as well as - addressing some shortcomings therein. The new format is - more self-describing than the HDF4 format and is more - uniformly applied to data objects in the file. - - <P>An HDF5 file appears to the user as a directed graph. - The nodes of this graph are the higher-level HDF5 objects - that are exposed by the HDF5 APIs: - - <ul> - <li>Groups - <li>Datasets - <li>Named datatypes - </ul> - - <P>At the lowest level, as information is actually written to the disk, - an HDF5 file is made up of the following objects: - <ul> - <li>A super block - <li>B-tree nodes (containing either symbol nodes or raw data chunks) - <li>Object headers - <li>A global heap - <li>Local heaps - <li>Free space - </ul> - - <P>The HDF5 library uses these low-level objects to represent the - higher-level objects that are then presented to the user or - to applications through the APIs. - For instance, a group is an object header that contains a message that - points to a local heap and to a B-tree which points to symbol nodes. - A dataset is an object header that contains messages that describe - datatype, space, layout, filters, external files, fill value, etc - with the layout message pointing to either a raw data chunk or to a - B-tree that points to raw data chunks. - - - <h3>This Document</h3> - - <p>This document describes the lower-level data objects; - the higher-level objects and their properties are described - in the <a href="H5.user.html"><cite>HDF5 User's Guide</cite></a>. - - <P>Three levels of information comprise the file format. - Level 0 contains basic information for identifying and - defining information about the file. Level 1 information contains - the information about the pieces of a file shared by many objects - in the file (such as a B-trees and heaps). Level 2 is the rest - of the file and contains all of the data objects, with each object - partitioned into header information, also known as - <em>metadata</em>, and data. - - <p>The sizes of various fields in the following layout tables are - determined by looking at the number of columns the field spans - in the table. There are three exceptions: (1) The size may be - overridden by specifying a size in parentheses, (2) the size of - addresses is determined by the <em>Size of Offsets</em> field - in the super block and is indicated in this document with a - superscripted 'O', and (3) the size of length fields is determined - by the <em>Size of Lengths</em> field in the super block and is - indicated in this document with a superscripted 'L'. - - <P>Values for all fields in this document should be treated as unsigned - integers, unless otherwise noted in the description of a field. - Additionally, all metadata fields are stored in little-endian byte - order. - </P> - - <BR> - <HR> - - <h2><a name="FileMetaData"> - Disk Format: Level 0 - File Metadata</a></h2> - - <H3><A name="SuperBlock"> - Disk Format: Level 0A - File Signature and Super Block</A></H3> - - <P>The super block may begin at certain predefined offsets within - the HDF5 file, allowing a block of unspecified content for - users to place additional information at the beginning (and - end) of the HDF5 file without limiting the HDF5 library's - ability to manage the objects within the file itself. This - feature was designed to accommodate wrapping an HDF5 file in - another file format or adding descriptive information to the - file without requiring the modification of the actual file's - information. The super block is located by searching for the - HDF5 file signature at byte offset 0, byte offset 512 and at - successive locations in the file, each a multiple of two of - the previous location, i.e. 0, 512, 1024, 2048, etc. - - <P>The super block is composed of a file signature, followed by - super block and group version numbers, information - about the sizes of offset and length values used to describe - items within the file, the size of each group page, - and a group entry for the root object in the file. - - <br> - <div align=center> - <table class=format> - <caption> - HDF5 Super Block Layout - </caption> - - <tr> - <th>byte</th> - <th>byte</th> - <th>byte</th> - <th>byte</th> - </tr> - - <tr> - <td colspan=4><br>HDF5 File Signature (8 bytes)<br><br></td> - </tr> - - <tr> - <td>Version # of Super Block</td> - <td>Version # of Global Free-space Storage</td> - <td>Version # of Root Group Symbol Table Entry</td> - <td>Reserved (zero)</td> - </tr> - - <tr> - <td>Version # of Shared Header Message Format</td> - <td>Size of Offsets</td> - <td>Size of Lengths</td> - <td>Reserved (zero)</td> - </tr> - - <tr> - <td colspan=2>Group Leaf Node K</td> - <td colspan=2>Group Internal Node K</td> - </tr> - - <tr> - <td colspan=4>File Consistency Flags</td> - </tr> - - <tr> - <td colspan=2 style="border:dotted;">Indexed Storage Internal Node K<sup>1</sup></td> - <td colspan=2 style="border:dotted;">Reserved (zero)<sup>1</sup></td> - </tr> - - <tr> - <td colspan=4>Base Address<sup>O</sup></td> - </tr> - - <tr> - <td colspan=4>Address of Global Free-space Heap<sup>O</sup></td> - </tr> - - <tr> - <td colspan=4>End of File Address<sup>O</sup></td> - </tr> - - <tr> - <td colspan=4>Driver Information Block Address<sup>O</sup></td> - </tr> - - <tr> - <td colspan=4>Root Group Symbol Table Entry</td> - </tr> - </table> - - <table class=note> - <tr><td> - (Items marked with an 'O' the above table are - <br> - of the size specified in "Size of Offsets.") - </td></tr> - <tr><td> - (Items marked with an '1' the above table are - <br> - new in version 1 of the superblock) - </td></tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>HDF5 File Signature</td> - <td> - <P>This field contains a constant value and can be used to - quickly identify a file as being an HDF5 file. The - constant value is designed to allow easy identification of - an HDF5 file and to allow certain types of data corruption - to be detected. The file signature of an HDF5 file always - contains the following values: - </P> - - <center> - <table border align=center cellpadding=4> - <tr align=center> - <td align=right>Decimal:</td> - <td width="8%">137</td> - <td width="8%">72</td> - <td width="8%">68</td> - <td width="8%">70</td> - <td width="8%">13</td> - <td width="8%">10</td> - <td width="8%">26</td> - <td width="8%">10</td> - </tr> - - <tr align=center> - <td align=right>Hexadecimal:</td> - <td>89</td> - <td>48</td> - <td>44</td> - <td>46</td> - <td>0d</td> - <td>0a</td> - <td>1a</td> - <td>0a</td> - </tr> - - <tr align=center> - <td align=right>ASCII C Notation:</td> - <td>\211</td> - <td>H</td> - <td>D</td> - <td>F</td> - <td>\r</td> - <td>\n</td> - <td>\032</td> - <td>\n</td> - </tr> - </table> - </center> - <br> - - <P>This signature both identifies the file as an HDF5 file - and provides for immediate detection of common - file-transfer problems. The first two bytes distinguish - HDF5 files on systems that expect the first two bytes to - identify the file type uniquely. The first byte is - chosen as a non-ASCII value to reduce the probability - that a text file may be misrecognized as an HDF5 file; - also, it catches bad file transfers that clear bit - 7. Bytes two through four name the format. The CR-LF - sequence catches bad file transfers that alter newline - sequences. The control-Z character stops file display - under MS-DOS. The final line feed checks for the inverse - of the CR-LF translation problem. (This is a direct - descendent of the <A href="http://www.libpng.org/pub/png/spec/PNG-Rationale.html#R.PNG-file-signature">PNG</A> file - signature.) - </P> - - <P><EM>This field is present in version 0+ of the superblock.</EM> - </P> - </td> - </tr> - - <tr> - <td>Version Number of the Super Block</td> - <td> - <P>This value is used to determine the format of the - information in the super block. When the format of the - information in the super block is changed, the version number - is incremented to the next integer and can be used to - determine how the information in the super block is - formatted. - </P> - - <P>Values of 0 and 1 are defined for this field. - </P> - - <P><EM>This field is present in version 0+ of the superblock.</EM> - </P> - </td> - </tr> - - <tr> - <td>Version Number of the File Free-space Information</td> - <td> - <P>This value is used to determine the format of the - information in the File Free-space Information. - </P> - <P>The only value currently valid in this field is '0', which - indicates that the free space index is formatted as described - <A href="#FreeSpaceIndex">below</A>. - </P> - - <P><EM>This field is present in version 0+ of the superblock.</EM> - </P> - </td> - </tr> - - <tr> - <td>Version Number of the Root Group Symbol Table Entry</td> - <td> - <P>This value is used to determine the format of the - information in the Root Group Symbol Table Entry. When the - format of the information in that field is changed, the - version number is incremented to the next integer and can be - used to determine how the information in the field - is formatted. - </P> - <P>The only value currently valid in this field is '0', which - indicates that the root group symbol table entry is formatted as - described <A href="#SymbolTableEntry">below</A>. - </P> - - <P><EM>This field is present in version 0+ of the superblock.</EM> - </P> - </td> - </tr> - - <tr> - <td>Version Number of the Shared Header Message Format</td> - <td> - <P>This value is used to determine the format of the - information in a shared object header message, which is - stored in the global small-data heap. Since the format - of the shared header messages differs from the private - header messages, a version number is used to identify changes - in the format. - </P> - <P>The only value currently valid in this field is '0', which - indicates that shared header messages are formatted as - described <A href="#SharedObjectHeader">below</A>. - </P> - - <P><EM>This field is present in version 0+ of the superblock.</EM> - </P> - </td> - </tr> - - <tr> - <td>Size of Offsets</td> - <td> - <P>This value contains the number of bytes used to store - addresses in the file. The values for the addresses of - objects in the file are offsets relative to a base address, - usually the address of the super block signature. This - allows a wrapper to be added after the file is created - without invalidating the internal offset locations. - </P> - - <P><EM>This field is present in version 0+ of the superblock.</EM> - </P> - </td> - </tr> - - <tr> - <td>Size of Lengths</td> - <td> - <P>This value contains the number of bytes used to store - the size of an object. - </P> - - <P><EM>This field is present in version 0+ of the superblock.</EM> - </P> - </td> - </tr> - - <tr> - <td>Group Leaf Node K</td> - <td> - <P>Each leaf node of a group B-tree will have at - least this many entries but not more than twice this - many. If a group has a single leaf node then it - may have fewer entries. - </P> - <P>This value must be greater than zero. - </P> - <P>See the <A href="#Btrees">description</A> of B-trees below. - </P> - - <P><EM>This field is present in version 0+ of the superblock.</EM> - </P> - </td> - </tr> - - <tr> - <td>Group Internal Node K</td> - <td> - <P>Each internal node of a group B-tree will have at - least this many entries but not more than twice this - many. If the group has only one internal - node then it might have fewer entries. - </P> - <P>This value must be greater than zero. - </P> - <P>See the <A href="#Btrees">description</A> of B-trees below. - </P> - - <P><EM>This field is present in version 0+ of the superblock.</EM> - </P> - </td> - </tr> - - <tr> - <td>File Consistency Flags</td> - <td> - <P>This value contains flags to indicate information - about the consistency of the information contained - within the file. Currently, the following bit flags are - defined: - <ul> - <li>Bit 0 set indicates that the file is opened for - write-access. - <li>Bit 1 set indicates that the file has - been verified for consistency and is guaranteed to be - consistent with the format defined in this document. - <li>Bits 2-31 are reserved for future use. - </ul> - Bit 0 should be - set as the first action when a file is opened for write - access and should be cleared only as the final action - when closing a file. Bit 1 should be cleared during - normal access to a file and only set after the file's - consistency is guaranteed by the library or a - consistency utility. - </P> - - <P><EM>This field is present in version 0+ of the superblock.</EM> - </P> - </td> - </tr> - - <tr> - <td>Indexed Storage Internal Node K</td> - <td> - <P>Each internal node of a indexed storage B-tree will have at - least this many entries but not more than twice this - many. If the group has only one internal - node then it might have fewer entries. - </P> - <P>This value must be greater than zero. - </P> - <P>See the <A href="#Btrees">description</A> of B-trees below. - </P> - - <P><EM>This field is present in version 1+ of the superblock.</EM> - </P> - </td> - </tr> - - <tr> - <td>Base Address</td> - <td> - <P>This is the absolute file address of the first byte of - the HDF5 data within the file. The library currently - constrains this value to be the absolute file address - of the super block itself when creating new files; - future versions of the library may provide greater - flexibility. When opening an existing file and this address does - not match the offset of the superblock, the library assumes - that the entire contents of the HDF5 file have been adjusted in - the file and adjusts the base address and end of file address to - reflect their new positions in the file. Unless otherwise noted, - all other file addresses are relative to this base - address. - </P> - - <P><EM>This field is present in version 0+ of the superblock.</EM> - </P> - </td> - </tr> - - <tr> - <td>Address of Global Free-space Index</td> - <td> - <P>Free-space management is not yet defined in the HDF5 - file format and is not handled by the library. - Currently this field always contains the - <A href="#UndefinedAddress">undefined address</A>. - </P> - - <P><EM>This field is present in version 0+ of the superblock.</EM> - </P> - </td> - </tr> - - <tr> - <td>End of File Address</td> - <td> - <P>This is the absolute file address of the first byte past - the end of all HDF5 data. It is used to determine whether a - file has been accidently truncated and as an address where - file data allocation can occur if space from the free list is - not used. - </P> - - <P><EM>This field is present in version 0+ of the superblock.</EM> - </P> - </td> - </tr> - - <tr> - <td>Driver Information Block Address</td> - <td> - <P>This is the relative file address of the file driver - information block which contains driver-specific - information needed to reopen the file. If there is no - driver information block then this entry should be the - <A href="#UndefinedAddress">undefined address</A>. - </P> - - <P><EM>This field is present in version 0+ of the superblock.</EM> - </P> - </td> - </tr> - - <tr> - <td>Root Group Symbol Table Entry</td> - <td> - <P>This is the <A href="#SymbolTableEntry">symbol table entry</A> - of the root group, which serves as the entry point into - the group graph for the file. - </P> - - <P><EM>This field is present in version 0+ of the superblock.</EM> - </P> - </td> - </tr> - </table> - </div> - - <H3><A name="DriverInfo"> - Disk Format: Level 0B - File Driver Info</A></H3> - - <p>The <em>file driver information block</em> is an optional region of the - file which contains information needed by the file driver in - order to reopen a file. The format of the file driver information - block is: - - <br> - <div align=center> - <table class=format> - <caption> - Driver Information Block - </caption> - - <tr> - <th>byte</th> - <th>byte</th> - <th>byte</th> - <th>byte</th> - </tr> - - <tr> - <td>Version</td> - <td colspan=3>Reserved (zero)</td> - </tr> - - <tr> - <td colspan=4>Driver Information Size (4 bytes)</td> - </tr> - - <tr> - <td colspan=4><br>Driver Identification (8 bytes)<br><br></td> - </tr> - - <tr> - <td colspan=4><br><br>Driver Information (<em>n</em> bytes)<br><br><br></td> - </tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Version</td> - <td> - <P>The version number of the driver information block. The - file format documented here is version zero. - </P> - </td> - </tr> - - <tr> - <td>Driver Information Size</td> - <td> - <P>The size in bytes of the Driver Information part of this - structure. - </P> - </td> - </tr> - - <tr> - <td>Driver Identification</td> - <td> - <P>This is an eight-byte ASCII string without null - termination which identifies the driver and version number - of the Driver Information block. The predefined drivers - supplied with the HDF5 library are identified by the - letters <code>NCSA</code> followed by the first four characters of - the driver name. If the Driver Information block is not - the original version then the last letter(s) of the - identification will be replaced by a version number in - ASCII. - </P> - <P> - For example, the various versions of the <em>family driver</em> - will be identified by <code>NCSAfami</code>, <code>NCSAfam0</code>, - <code>NCSAfam1</code>, etc. - (<code>NCSAfami</code> is simply <code>NCSAfamily</code> truncated - to eight characters. Subsequent identifiers will be created by - substituting sequential numerical values for the final character, - starting with zero.) - </P> - <P> - Identification for user-defined drivers - is arbitrary but should be unique and avoid the four character - prefix "NCSA". - </P> - </td> - </tr> - - <tr valign=top> - <td>Driver Information</td> - <td>Driver information is stored in a format defined by the - file driver and encoded/decoded by the driver callbacks - invoked from the <code>H5FD_sb_encode</code> and - <code>H5FD_sb_decode</code> functions.</td> - </tr> - </table> - </div> - - <BR> - <HR> - - <h2><a name="FileInfra"> - Disk Format: Level 1 - File Infrastructure</a></h2> - <h3><a name="Btrees">Disk Format: Level 1A - B-link Trees and B-tree Nodes</a></h3> - - <p>B-link trees allow flexible storage for objects which tend to grow - in ways that cause the object to be stored discontiguously. B-trees - are described in various algorithms books including "Introduction to - Algorithms" by Thomas H. Cormen, Charles E. Leiserson, and Ronald - L. Rivest. The B-link tree, in which the sibling nodes at a - particular level in the tree are stored in a doubly-linked list, - is described in the "Efficient Locking for Concurrent Operations - on B-trees" paper by Phillip Lehman and S. Bing Yao as published - in the <cite>ACM Transactions on Database Systems</cite>, Vol. 6, - No. 4, December 1981. - - <p>The B-link trees implemented by the file format contain one more - key than the number of children. In other words, each child - pointer out of a B-tree node has a left key and a right key. - The pointers out of internal nodes point to sub-trees while - the pointers out of leaf nodes point to symbol nodes and - raw data chunks. - Aside from that difference, internal nodes and leaf nodes - are identical. - - <br> - <div align=center> - <table class=format> - <caption> - B-tree Nodes - </caption> - - <tr> - <th>byte</th> - <th>byte</th> - <th>byte</th> - <th>byte</th> - - <tr> - <td colspan=4>Signature</td> - - <tr> - <td>Node Type</td> - <td>Node Level</td> - <td colspan=2>Entries Used</td> - - <tr> - <td colspan=4>Address of Left Sibling<sup>O</sup></td> - - <tr> - <td colspan=4>Address of Right Sibling<sup>O</sup></td> - - <tr> - <td colspan=4>Key 0 (variable size)</td> - - <tr> - <td colspan=4>Address of Child 0<sup>O</sup></td> - - <tr> - <td colspan=4>Key 1 (variable size)</td> - - <tr> - <td colspan=4>Address of Child 1<sup>O</sup></td> - - <tr> - <td colspan=4>...</td> - - <tr> - <td colspan=4>Key 2<em>K</em> (variable size)</td> - - <tr> - <td colspan=4>Address of Child 2<em>K</em><sup>O</sup></td> - - <tr> - <td colspan=4>Key 2<em>K</em>+1 (variable size)</td> - </table> - - <table class=note> - <tr><td> - (Items marked with an 'O' the above table are - <br> - of the size specified in "Size of Offsets.") - </td></tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Signature</td> - <td> - <P>The ASCII character string "<code>TREE</code>" is - used to indicate the - beginning of a B-link tree node. This gives file - consistency checking utilities a better chance of - reconstructing a damaged file. - </P> - </td> - </tr> - - <tr> - <td>Node Type</td> - <td> - <P>Each B-link tree points to a particular type of data. - This field indicates the type of data as well as - implying the maximum degree <em>K</em> of the tree and - the size of each Key field. - </P> - - <table class=list> - <tr> - <th width="30%">Node Type</th> - <th align=left>Description</th> - </tr> - <tr> - <td align=center>0</td> - <td>This tree points to group nodes.</td> - </tr> - <tr> - <td align=center>1</td> - <td>This tree points to raw data chunk nodes.</td> - </tr> - </table> - </td> - </tr> - - <tr> - <td>Node Level</td> - <td> - <P>The node level indicates the level at which this node - appears in the tree (leaf nodes are at level zero). Not - only does the level indicate whether child pointers - point to sub-trees or to data, but it can also be used - to help file consistency checking utilities reconstruct - damanged trees. - </P> - </td> - </tr> - - <tr valign=top> - <td>Entries Used</td> - <td> - <P>This determines the number of children to which this - node points. All nodes of a particular type of tree - have the same maximum degree, but most nodes will point - to less than that number of children. The valid child - pointers and keys appear at the beginning of the node - and the unused pointers and keys appear at the end of - the node. The unused pointers and keys have undefined - values. - </P> - </td> - </tr> - - <tr valign=top> - <td>Address of Left Sibling</td> - <td> - <P>This is the relative file address of the left sibling of - the current node. If the current - node is the left-most node at this level then this field - is the <A href="#UndefinedAddress">undefined address</A>. - </P> - </td> - </tr> - - <tr valign=top> - <td>Address of Right Sibling</td> - <td> - <P>This is the relative file address of the right sibling of - the current node. If the current - node is the right-most node at this level then this - field is the <A href="#UndefinedAddress">undefined address</A>. - </P> - </td> - </tr> - - <tr valign=top> - <td>Keys and Child Pointers</td> - <td> - <P>Each tree has 2<em>K</em>+1 keys with 2<em>K</em> - child pointers interleaved between the keys. The number - of keys and child pointers actually containing valid - values is determined by the node's <em>Entries Used</em> field. - If that field is <em>N</em> then the B-link tree contains - <em>N</em> child pointers and <em>N</em>+1 keys. - </P> - </td> - </tr> - - <tr valign=top> - <td>Key</td> - <td> - <P>The format and size of the key values is determined by - the type of data to which this tree points. The keys are - ordered and are boundaries for the contents of the child - pointer; that is, the key values represented by child - <em>N</em> fall between Key <em>N</em> and Key - <em>N</em>+1. Whether the interval is open or closed on - each end is determined by the type of data to which the - tree points. - </P> - - <P> - The format of the key depends on the node type. - For nodes of node type 0 (group nodes), the key is formatted as - follows: - <center> - <table class=list> - <tr> - <td width=30%>A single field of <i>Size of Lengths</i> - bytes:</td> - <td>Indicates the byte offset into the local heap - for the first object name in the subtree which - that key describes. - </td> - </tr> - </table> - </center> - </P> - - <P> - For nodes of node type 1 (chunked raw data nodes), the key is - formatted as follows: - <center> - <table class=list> - <tr> - <td width=30%>Bytes 1-4:</td> - <td>Size of chunk in bytes.</td> - </tr> - <tr> - <td>Bytes 4-8:</td> - <td>Filter mask, a 32-bit bitfield indicating which - filters have been skipped for this chunk. Each filter - has an index number in the pipeline (starting at 0, with - the first filter to apply) and if that filter is skipped, - the bit corresponding to it's index is set.</td> - </tr> - <tr> - <td><em>N</em> 64-bit fields:</td> - <td>A 64-bit index indicating the offset of the - chunk within the dataset where <i>N</i> is the number - of dimensions of the dataset. For example, if - a chunk in a 3-dimensional dataset begins at the - position <code>[5,5,5]</code>, there will be three - such 64-bit indices, each with the value of - <code>5</code>.</td> - </tr> - </table> - </center> - </P> - </td> - </tr> - - <tr valign=top> - <td>Child Pointer</td> - <td> - <P>The tree node contains file addresses of subtrees or - data depending on the node level. Nodes at Level 0 point - to data addresses, either raw data chunk or group nodes. - Nodes at non-zero levels point to other nodes of the - same B-tree. - </P> - <P>For raw data chunk nodes, the child pointer is the address - of a single raw data chunk. For group nodes, the child pointer - points to a <A href="#SymbolTable">symbol table</A>, which contains - information for multiple symbol table entries. - </P> - </td> - </tr> - </table> - </div> - - <p> - Conceptually, each B-tree node looks like this: - <center> - <table> - <tr valign=top align=center> - <td>key[0]</td><td> </td> - <td>child[0]</td><td> </td> - <td>key[1]</td><td> </td> - <td>child[1]</td><td> </td> - <td>key[2]</td><td> </td> - <td>...</td><td> </td> - <td>...</td><td> </td> - <td>key[<i>N</i>-1]</td><td> </td> - <td>child[<i>N</i>-1]</td><td> </td> - <td>key[<i>N</i>]</td> - </tr> - </table> - </center> - <br> - - where child[<i>i</i>] is a pointer to a sub-tree (at a level - above Level 0) or to data (at Level 0). - Each key[<i>i</i>] describes an <i>item</i> stored by the B-tree - (a chunk or an object of a group node). The range of values - represented by child[<i>i</i>] is indicated by key[<i>i</i>] - and key[<i>i</i>+1]. - - - <p>The following question must next be answered: - "Is the value described by key[<i>i</i>] contained in - child[<i>i</i>-1] or in child[<i>i</i>]?" - The answer depends on the type of tree. - In trees for groups (node type 0) the object described by - key[<i>i</i>] is the greatest object contained in - child[<i>i</i>-1] while in chunk trees (node type 1) the - chunk described by key[<i>i</i>] is the least chunk in - child[<i>i</i>]. - - <p>That means that key[0] for group trees is sometimes unused; - it points to offset zero in the heap, which is always the - empty string and compares as "less-than" any valid object name. - - <p>And key[<i>N</i>] for chunk trees is sometimes unused; - it contains a chunk offset which compares as "greater-than" - any other chunk offset and has a chunk byte size of zero - to indicate that it is not actually allocated. - - - <h3><a name="SymbolTable">Disk Format: Level 1B - Group and Symbol Nodes</a></h3> - - <p>A group is an object internal to the file that allows - arbitrary nesting of objects within the file (including other groups). - A group maps a set of names in the group to a set of relative - file addresses where objects with those names are located in - the file. Certain metadata for an object to which the group points - can be cached in the group's symbol table in addition to the - object's header. - - <p>An HDF5 object name space can be stored hierarchically by - partitioning the name into components and storing each - component in a group. The group entry for a - non-ultimate component points to the group containing - the next component. The group entry for the last - component points to the object being named. - - <p>A group is a collection of group nodes pointed - to by a B-link tree. Each group node contains entries - for one or more symbols. If an attempt is made to add a - symbol to an already full group node containing - 2<em>K</em> entries, then the node is split and one node - contains <em>K</em> symbols and the other contains - <em>K</em>+1 symbols. - - <br> - <div align=center> - <table class=format> - <caption> - Group Node (A Leaf of a B-tree) - </caption> - - <tr> - <th>byte</th> - <th>byte</th> - <th>byte</th> - <th>byte</th> - - <tr> - <td colspan=4>Signature</td> - - <tr> - <td>Version Number</td> - <td>Reserved (0)</td> - <td colspan=2>Number of Symbols</td> - - <tr> - <td colspan=4><br><br>Group Entries<br><br><br></td> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Signature</td> - <td> - <P>The ASCII character string "<code>SNOD</code>" is - used to indicate the - beginning of a group node. This gives file - consistency checking utilities a better chance of - reconstructing a damaged file. - </P> - </td> - </tr> - - <tr> - <td>Version Number</td> - <td> - <P>The version number for the group node. This - document describes version 1. (There is no version '0' - of the group node) - </P> - </td> - </tr> - - <tr> - <td>Number of Symbols</td> - <td> - <P>Although all group nodes have the same length, - most contain fewer than the maximum possible number of - symbol entries. This field indicates how many entries - contain valid data. The valid entries are packed at the - beginning of the group node while the remaining - entries contain undefined values. - </P> - </td> - </tr> - - <tr> - <td>Group Entries</td> - <td> - <P>Each symbol has an entry in the group node. - The format of the entry is described below. - There are 2<EM>K</EM> entries in each group node, where - <EM>K</EM> is the "Group Leaf Node K" value from the - <A href="#SuperBlock">super block</A>. - </P> - </td> - </tr> - </table> - </div> - - <h3><a name="SymbolTableEntry"> - Disk Format: Level 1C - Group Entry </a></h3> - - <p>Each group entry in a group node is designed - to allow for very fast browsing of stored objects. - Toward that design goal, the group entries - include space for caching certain constant metadata from the - object header. - - <br> - <div align=center> - <table class=format> - <caption> - Group Entry - </caption> - - <tr> - <th>byte</th> - <th>byte</th> - <th>byte</th> - <th>byte</th> - </tr> - - <tr> - <td colspan=4>Name Offset<sup>O</sup></td> - </tr> - - <tr> - <td colspan=4>Object Header Address<sup>O</sup></td> - </tr> - - <tr> - <td colspan=4>Cache Type</td> - </tr> - - <tr> - <td colspan=4>Reserved</td> - </tr> - - <tr> - <td colspan=4><br><br>Scratch-pad Space (16 bytes)<br><br><br></td> - </tr> - </table> - - <table class=note> - <tr><td> - (Items marked with an 'O' the above table are - <br> - of the size specified in "Size of Offsets.") - </td></tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Name Offset</td> - <td> - <P>This is the byte offset into the group local - heap for the name of the object. The name is null - terminated. - </P> - </td> - </tr> - - <tr> - <td>Object Header Address</td> - <td> - <P>Every object has an object header which serves as a - permanent location for the object's metadata. In addition - to appearing in the object header, some metadata can be - cached in the scratch-pad space. - </P> - </td> - </tr> - - <tr> - <td>Cache Type</td> - <td> - <P>The cache type is determined from the object header. - It also determines the format for the scratch-pad space: - <br> - <table class=list> - <tr align=left> - <th>Type:</th> - <th>Description:</th> - </tr> - <tr> - <td width="10%" align=center>0</td> - <td>No data is cached by the group entry. This - is guaranteed to be the case when an object header - has a link count greater than one. - </td> - </tr> - <tr> - <td align=center>1</td> - <td>Object header metadata is cached in the group - entry. This implies that the group - entry refers to another group. - </td> - </tr> - <tr> - <td align=center>2</td> - <td>The entry is a symbolic link. The first four bytes - of the scratch-pad space are the offset into the local - heap for the link value. The object header address - will be undefined. - </td> - </tr> - <tr> - <td align=center><em>N</em></td> - <td>Other cache values can be defined later and - libraries that do not understand the new values will - still work properly. - </td> - </tr> - </table> - </P> - </td> - </tr> - - <tr> - <td>Reserved</td> - <td> - <P>These four bytes are present so that the scratch-pad - space is aligned on an eight-byte boundary. They are - always set to zero. - </P> - </td> - </tr> - - <tr> - <td>Scratch-pad Space</td> - <td> - <P>This space is used for different purposes, depending - on the value of the Cache Type field. Any metadata - about a dataset object represented in the scratch-pad - space is duplicated in the object header for that - dataset. This metadata can include the datatype - and the size of the dataspace for a dataset whose datatype - is atomic and whose dataspace is fixed and less than - four dimensions. - </P> - <P> - Furthermore, no data is cached in the group - entry scratch-pad space if the object header for - the group entry has a link count greater than - one. - </P> - </td> - </tr> - </table> - </div> - - <h4>Format of the Scratch-pad Space</h4> - - <p>The group entry scratch-pad space is formatted - according to the value in the Cache Type field. - - <p>If the Cache Type field contains the value zero - <code>(0)</code> then no information is - stored in the scratch-pad space. - - <p>If the Cache Type field contains the value one - <code>(1)</code>, then the scratch-pad space - contains cached metadata for another object header - in the following format: - - <br> - <div align=center> - <table class=format> - <caption> - Object Header Scratch-pad Format - </caption> - - <tr> - <th>byte</th> - <th>byte</th> - <th>byte</th> - <th>byte</th> - - <tr> - <td colspan=4>Address of B-tree<sup>O</sup></td> - - <tr> - <td colspan=4>Address of Name Heap<sup>O</sup></td> - </table> - - <table class=note> - <tr><td> - (Items marked with an 'O' the above table are - <br> - of the size specified in "Size of Offsets.") - </td></tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Address of B-tree</td> - <td> - <P>This is the file address for the root of the - group's B-tree. - </P> - </td> - </tr> - - <tr> - <td>Address of Name Heap</td> - <td> - <P>This is the file address for the group's local - heap, in which are stored the group's symbol names. - </P> - </td> - </tr> - </table> - </div> - - - <P>If the Cache Type field contains the value two - <code>(2)</code>, then the scratch-pad space - contains cached metadata for another symbolic link - in the following format: - - <br> - <div align=center> - <table class=format> - <caption> - Symbolic Link Scratch-pad Format - </caption> - - <tr> - <th>byte</th> - <th>byte</th> - <th>byte</th> - <th>byte</th> - </tr> - - <tr> - <td colspan=4>Offset to Link Value</td> - </tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Offset to Link Value</td> - <td> - <P>The value of a symbolic link (that is, the name of the - thing to which it points) is stored in the local heap. - This field is the 4-byte offset into the local heap for - the start of the link value, which is null terminated. - </P> - </td> - </tr> - </table> - </div> - - <h3><a name="LocalHeap">Disk Format: Level 1D - Local Heaps</a></h3> - - <P>A heap is a collection of small heap objects. Objects can be - inserted and removed from the heap at any time. - The address of a heap does not change once the heap is created. - References to objects are stored in the group table; - the names of those objects are stored in the local heap. - </P> - - <br> - <div align=center> - <table class=format> - <caption> - Local Heap - </caption> - - <tr> - <th>byte</th> - <th>byte</th> - <th>byte</th> - <th>byte</th> - </tr> - - <tr> - <td colspan=4>Signature</td> - </tr> - - <tr> - <td>Version</td> - <td colspan=3>Reserved (zero)</td> - </td> - - <tr> - <td colspan=4>Data Segment Size<sup>L</sup></td> - </tr> - - <tr> - <td colspan=4>Offset to Head of Free-list<sup>L</sup></td> - </tr> - - <tr> - <td colspan=4>Address of Data Segment<sup>O</sup></td> - </tr> - </table> - - <table class=note> - <tr><td> - (Items marked with an 'L' the above table are - <br> - of the size specified in "Size of Lengths.") - </td></tr> - <tr><td> - (Items marked with an 'O' the above table are - <br> - of the size specified in "Size of Offsets.") - </td></tr> - </table> - </div> - - <p> - <center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Signature</td> - <td> - <P>The ASCII character string "<code>HEAP</code>" - is used to indicate the - beginning of a heap. This gives file consistency - checking utilities a better chance of reconstructing a - damaged file. - </P> - </td> - </tr> - - <tr> - <td>Version</td> - <td> - <P>Each local heap has its own version number so that new - heaps can be added to old files. This document - describes version zero (0) of the local heap. - </P> - </td> - </tr> - - <tr> - <td>Data Segment Size</td> - <td> - <P>The total amount of disk memory allocated for the heap - data. This may be larger than the amount of space - required by the objects stored in the heap. The extra - unused space in the heap holds a linked list of free blocks. - </P> - </td> - </tr> - - <tr> - <td>Offset to Head of Free-list</td> - <td> - <P>This is the offset within the heap data segment of the - first free block (or the - <A href="#UndefinedAddress">undefined address</A> if there is no - free block). The free block contains "Size of Lengths" bytes that - are the offset of the next free block (or the - value '1' if this is the - last free block) followed by "Size of Lengths" bytes that store - the size of this free block. The size of the free block includes - the space used to store the offset of the next free block and - the of the current block, making the minimum size of a free block - 2 * "Size of Lengths". - </P> - </td> - </tr> - - <tr> - <td>Address of Data Segment</td> - <td> - <P>The data segment originally starts immediately after - the heap header, but if the data segment must grow as a - result of adding more objects, then the data segment may - be relocated, in its entirety, to another part of the - file. - </P> - </td> - </tr> - </table> - </center> - - <p>Objects within the heap should be aligned on an 8-byte boundary. - - <h3><a name="GlobalHeap">Disk Format: Level 1E - Global Heap</a></h3> - - <P>Each HDF5 file has a global heap which stores various types of - information which is typically shared between datasets. The - global heap was designed to satisfy these goals: - - <ol type="A"> - <li>Repeated access to a heap object must be efficient without - resulting in repeated file I/O requests. Since global heap - objects will typically be shared among several datasets, it is - probable that the object will be accessed repeatedly. - <li>Collections of related global heap objects should result in - fewer and larger I/O requests. For instance, a dataset of - object references will have a global heap object for each - reference. Reading the entire set of object references - should result in a few large I/O requests instead of one small - I/O request for each reference. - <li>It should be possible to remove objects from the global heap - and the resulting file hole should be eligible to be reclaimed - for other uses. - </ol> - </P> - - <P>The implementation of the heap makes use of the memory - management already available at the file level and combines that - with a new top-level object called a <em>collection</em> to - achieve Goal B. The global heap is the set of all collections. - Each global heap object belongs to exactly one collection and - each collection contains one or more global heap objects. For - the purposes of disk I/O and caching, a collection is treated as - an atomic object. - </P> - - <P>The HDF5 library creates global heap collections as needed, so there may - be multiple collections throughout the file. The set of all of them is - abstractly called the "global heap", although they don't actually link - to each other, and there is no global place in the file where you can - discover all of the collections. The collections are found simply by - finding a reference to one through another object in the file (eg. - variable-length datatype elements, etc). - </P> - - <br> - <div align=center> - <table class=format> - <caption> - A Global Heap Collection - </caption> - - <tr> - <th>byte</th> - <th>byte</th> - <th>byte</th> - <th>byte</th> - </tr> - - <tr> - <td colspan=4>Signature</td> - </tr> - - <tr> - <td>Version</td> - <td colspan=3>Reserved (zero)</td> - </td> - - <tr> - <td colspan=4>Collection Size<sup>L</sup></td> - </tr> - - <tr> - <td colspan=4><br>Global Heap Object 1<br><br></td> - </tr> - - <tr> - <td colspan=4><br>Global Heap Object 2<br><br></td> - </tr> - - <tr> - <td colspan=4><br>...<br><br></td> - </tr> - - <tr> - <td colspan=4><br>Global Heap Object <em>N</em><br><br></td> - </tr> - - <tr> - <td colspan=4><br>Global Heap Object 0 (free space)<br><br></td> - </tr> - </table> - - <table class=note> - <tr><td> - (Items marked with an 'L' the above table are - <br> - of the size specified in "Size of Lengths.") - </td></tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Signature</td> - <td> - <P>The ASCII character string "<code>GCOL</code>" - is used to indicate the - beginning of a collection. This gives file consistency - checking utilities a better chance of reconstructing a - damaged file. - </P> - </td> - </tr> - - <tr> - <td>Version</td> - <td> - <P>Each collection has its own version number so that new - collections can be added to old files. This document - describes version one (1) of the collections (there is no - version zero (0)). - </P> - </td> - </tr> - - <tr> - <td>Collection Size</td> - <td> - <P>This is the size in bytes of the entire collection - including this field. The default (and minimum) - collection size is 4096 bytes which is a typical file - system block size. This allows for 127 16-byte heap - objects plus their overhead (the collection header of 16 bytes - and the 16 bytes of information about each heap object). - </P> - </td> - </tr> - - <tr> - <td>Global Heap Object 1 through <em>N</em></td> - <td> - <P>The objects are stored in any order with no - intervening unused space. - </P> - </td> - </tr> - - <tr> - <td>Global Heap Object 0</td> - <td> - <P>Global Heap Object 0 (zero), when present, represents the free - space in the collection. Free space always appears at the end of - the collection. If the free space is too small to store the header - for Object 0 (described below) then the header is implied and the - collection contains no free space. - </P> - </td> - </table> - </div> - - <br> - <div align=center> - <table class=format> - <caption> - Global Heap Object - </caption> - - <tr> - <th>byte</th> - <th>byte</th> - <th>byte</th> - <th>byte</th> - </tr> - - <tr> - <td colspan=2>Heap Object ID</td> - <td colspan=2>Reference Count</td> - </tr> - - <tr> - <td colspan=4>Reserved</td> - </tr> - - <tr> - <td colspan=4>Object Size<sup>L</sup></td> - </tr> - - <tr> - <td colspan=4><br>Object Data<br><br></td> - </tr> - </table> - - <table class=note> - <tr><td> - (Items marked with an 'L' the above table are - <br> - of the size specified in "Size of Lengths.") - </td></tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Heap Object ID</td> - <td> - <P>Each object has a unique identification number within a - collection. The identification numbers are chosen so that - new objects have the smallest value possible with the - exception that the identifier <code>0</code> always refers to the - object which represents all free space within the - collection. - </P> - </td> - </tr> - - <tr> - <td>Reference Count</td> - <td> - <P>All heap objects have a reference count field. An - object which is referenced from some other part of the - file will have a positive reference count. The reference - count for Object 0 is always zero. - </P> - </td> - </tr> - - <tr> - <td>Reserved</td> - <td> - <P>Zero padding to align next field on an 8-byte boundary. - </P> - </td> - </tr> - - <tr> - <td>Object Size</td> - <td> - <P>This is the size of the object data stored for the object. - The actual storage space allocated for the object data is rounded - up to a multiple of eight. - </P> - </td> - </tr> - - <tr> - <td>Object Data</td> - <td> - <P>The object data is treated as a one-dimensional array - of bytes to be interpreted by the caller. - </P> - </td> - </tr> - </table> - </div> - - <h3><a name="FreeSpaceIndex">Disk Format: Level 1F - Free-space Index</a></h3> - - <p>The free-space index is a collection of blocks of data, - dispersed throughout the file, which are currently not used by - any file objects. - - <p>The super block contains a pointer to root of the free-space description; - that pointer is currently required to be the - <A href="#UndefinedAddress">undefined address</A>. - - <p>The format of the free-space index is not defined at this time. - -<!-- - <p>The Free-space Index is a collection of blocks of data, - dispersed throughout the file, which are currently not used by - any file objects. The blocks of data are indexed by a B-tree of - their length within the file. - - - <p>Each B-tree page is composed of the following entries and - B-tree management information, organized as follows: - - <p> - <center> - <table border cellpadding=4 width="80%"> - <caption align=bottom> - <B>HDF5 Free-space Heap Page</B> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - - <tr align=center> - <td colspan=4>Signature</td> - <tr align=center> - <td colspan=4>B-tree Left-link Offset</td> - <tr align=center> - <td colspan=4><br>Length of Free-block #1<br> <br></td> - <tr align=center> - <td colspan=4><br>Offset of Free-block #1<br> <br></td> - <tr align=center> - <td colspan=4>.<br>.<br>.<br></td> - <tr align=center> - <td colspan=4><br>Length of Free-block #n<br> <br></td> - <tr align=center> - <td colspan=4><br>Offset of Free-block #n<br> <br></td> - <tr align=center> - <td colspan=4>"High" Offset</td> - <tr align=center> - <td colspan=4>Right-link Offset</td> - </table> - </center> - - <p> - <dl> - <dt> The elements of the free-space heap page are described below: - <dd> - <dl> - <dt>Signature: (4 bytes) - <dd>The ASCII character string <code>FREE</code> - is used to indicate the - beginning of a free-space heap B-tree page. This gives - file consistency checking utilities a better chance of - reconstructing a damaged file. - - <dt>B-tree Left-link Offset: (<offset> bytes) - <dd>This value is used to indicate the offset of all offsets - in the B-link-tree which are smaller than the value of the - offset in entry #1. This value is also used to indicate a - leaf node in the B-link-tree by being set to all ones. - - <dt>Length of Free-block #n: (<length> bytes) - <dd>This value indicates the length of an unused block in - the file. - - <dt>Offset of Free-block #n: (<offset> bytes) - <dd>This value indicates the offset in the file of an - unused block in the file. - - <dt>"High" Offset: (4-bytes) - <dd>This offset is used as the upper bound on offsets - contained within a page when the page has been split. - - <dt>Right-link Offset: (<offset> bytes) - <dd>This value is used to indicate the offset of the next - child to the right of the parent of this group - page. When there is no node to the right, this value is - all zeros. - </dl> - </dl> - - <p>The algorithms for searching and inserting objects in the - B-tree pages are described fully in the Lehman and Yao paper, - which should be read to provide a full description of the - B-tree's usage. ---> - - <BR> - <HR> - - <h2><a name="DataObject">Disk Format: Level 2 - Data Objects </a></h2> - - <P>Data objects contain the real information in the file. These - objects compose the scientific data and other information which - are generally thought of as "data" by the end-user. All the - other information in the file is provided as a framework for - these data objects. - </P> - - <P>A data object is composed of header information and data - information. The header information contains the information - needed to interpret the data information for the data object as - well as additional "metadata" or pointers to additional - "metadata" used to describe or annotate each data object. - </P> - - <h3><a name="ObjectHeader"> - Disk Format: Level 2A - Data Object Headers</a></h3> - - <P>The header information of an object is designed to encompass - all the information about an object, except for the data itself. - This information includes - the dataspace, datatype, information about how the data - is stored on disk (in external files, compressed, broken up in - blocks, etc.), as well as other information used by the library - to speed up access to the data objects or maintain a file's - integrity. Information stored by user applications as attributes - is also stored in the object's header. The header of each object is - not necessarily located immediately prior to the object's data in the - file and in fact may be located in any position in the file. The order - of the messages in an object header is not significant. - </P> - - <P>Header messages are aligned on 8-byte boundaries. - </P> - - <br> - <div align=center> - <table class=format> - <caption> - Object Headers - </caption> - - <tr> - <th>byte</th> - <th>byte</th> - <th>byte</th> - <th>byte</th> - </tr> - - <tr> - <td>Version</td> - <td>Reserved (zero)</td> - <td colspan=2>Number of Header Messages</td> - </tr> - - <tr> - <td colspan=4>Object Reference Count</td> - </tr> - - <tr> - <td colspan=4>Object Header Size</td> - </tr> - - <tr> - <td colspan=2>Header Message Type #1</td> - <td colspan=2>Size of Header Message Data #1</td> - </tr> - - <tr> - <td>Header Message #1 Flags</td> - <td colspan=3>Reserved (zero)</td> - </tr> - - <tr> - <td colspan=4><br>Header Message Data #1<br><br></td> - </tr> - - <tr> - <td colspan=4>.<br>.<br>.<br></td> - </tr> - - <tr> - <td colspan=2>Header Message Type #n</td> - <td colspan=2>Size of Header Message Data #n</td> - </tr> - - <tr> - <td>Header Message #n Flags</td> - <td colspan=3>Reserved (zero)</td> - </tr> - - <tr> - <td colspan=4><br>Header Message Data #n<br><br></td> - </tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Version</td> - <td> - <P>This value is used to determine the format of the - information in the object header. When the format of the - information in the object header is changed, the version number - is incremented and can be used to determine how the - information in the object header is formatted. This - document describes version one (1) (there was no version - zero (0)). - </P> - </td> - </tr> - - <tr> - <td>Number of Header Messages</td> - <td> - <P>This value determines the number of messages listed in - object headers for this object. This value includes the messages - in continuation messages for this object. - </P> - </td> - </tr> - - <tr> - <td>Object Reference Count</td> - <td> - <P>This value specifies the number of "hard links" to this object - within the current file. References to the object from external - files, "soft links" in this file and object references in this - file are not tracked. - </P> - </td> - </tr> - - <tr> - <td>Object Header Size</td> - <td> - <P>This value specifies the number of bytes of header message data - following this length field that contain object header messages - for this object header. This value does not include the size of - object header continuation blocks for this object elsewhere in the - file. - </P> - </td> - </tr> - - <tr> - <td>Header Message Type</td> - <td> - <P>This value specifies the type of information included in the - following header message data. The header message types for the - pre-defined header messages are included in sections below. - </P> - </td> - </tr> - - <tr> - <td>Size of Header Message Data</td> - <td> - <P>This value specifies the number of bytes of header - message data following the header message type and length - information for the current message. The size includes - padding bytes to make the message a multiple of eight - bytes. - </P> - </td> - </tr> - - <tr> - <td>Header Message Flags</td> - <td> - <P>This is a bit field with the following definition: - <table class=list> - <tr> - <th width="30%">Bit</th> - <th align=left>Description</th> - </tr> - - <tr> - <td align=center><code>0</code></td> - <td>If set, the message data is constant. This is used - for messages like the datatype message of a dataset. - </td> - </tr> - <tr> - <td align=center><code>1</code></td> - <td>If set, the message is stored in the global heap. - The Header Message Data field contains a Shared Object - message and the Size of Header Message Data field - contains the size of that Shared Object message. - </td> - </tr> - <tr> - <td align=center><code>2-7</code></td> - <td>Reserved</td> - </tr> - </table> - </P> - </td> - </tr> - - <tr> - <td>Header Message Data</td> - <td> - <P>The format and length of this field is determined by the - header message type and size respectively. Some header - message types do not require any data and this information - can be eliminated by setting the length of the message to - zero. The data is padded with enough zeros to make the - size a multiple of eight. - </P> - </td> - </tr> - </table> - </div> - - <P>The header message types and the message data associated with - them compose the critical "metadata" about each object. Some - header messages are required for each object while others are - optional. Some optional header messages may also be repeated - several times in the header itself, the requirements and number - of times allowed in the header will be noted in each header - message description below. - </P> - - <P>The following is a list of currently defined header messages: - </P> - - <hr> - <h4><a name="NILMessage">Name: NIL</a></h4> - - <P class=item><B>Header Message Type: </B>0x0000 - </P> - <P class=item><B>Length:</B> varies - </P> - <P class=item><B>Status:</B> Optional, may be repeated. - </P> - <P class=item><B>Purpose and Description:</B> The NIL message is used to indicate a - message which is to be ignored when reading the header messages for a - data object. [Possibly one which has been deleted for some reason.] - </P> - <P class=item><B>Format of Data:</B> Unspecified. - </P> - - <hr> - <h4><a name="SimpleDataSpace">Name: Simple Dataspace</a></h4> - - <P class=item><B>Header Message Type: </B>0x0001 - </P> - <P class=item><B>Length:</B> Varies according to the number of dimensions, - as described in the following table. - </P> - <P class=item><B>Status:</B> Required for dataset objects, may not be - repeated. - </P> - <P class=item><B>Description:</B> The simple dataspace message describes the - number of dimensions (i.e. "rank") and size of each dimension that the - data object has. This message is only used for datasets which have a - simple, rectilinear grid layout; datasets requiring a more complex - layout (irregularly structured or unstructured grids, etc.) must use - the <em>Complex Dataspace</em> message for expressing the space the - dataset inhabits. <i>(Note: The <em>Complex Dataspace</em> - functionality is not yet implemented and it is not described in this - document.)</i> - </P> - - <P class=item><B>Format of Data:</B> - <br> - <div align=center> - <table class=format> - <caption> - Simple Dataspace Message - </caption> - - <tr> - <th>byte</th> - <th>byte</th> - <th>byte</th> - <th>byte</th> - </tr> - - <tr> - <td>Version</td> - <td>Dimensionality</td> - <td>Flags</td> - <td>Reserved</td> - </tr> - - <tr> - <td colspan=4>Reserved</td> - </tr> - - <tr> - <td colspan=4>Dimension #1 Size<sup>L</sup></td> - <tr> - <td colspan=4>.<br>.<br>.<br></td> - <tr> - <td colspan=4>Dimension #n Size<sup>L</sup></td> - <tr> - <td colspan=4>Dimension #1 Maximum Size<sup>L</sup></td> - <tr> - <td colspan=4>.<br>.<br>.<br></td> - <tr> - <td colspan=4>Dimension #n Maximum Size<sup>L</sup></td> - <tr> - <td colspan=4>Permutation Index #1<sup>L</sup></td> - <tr> - <td colspan=4>.<br>.<br>.<br></td> - <tr> - <td colspan=4>Permutation Index #n<sup>L</sup></td> - </table> - - <table class=note> - <tr><td> - (Items marked with an 'L' the above table are - <br> - of the size specified in "Size of Lengths.") - </td></tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Version</td> - <td> - <P>This value is used to determine the format of the - Simple Dataspace Message. When the format of the - information in the message is changed, the version number - is incremented and can be used to determine how the - information in the object header is formatted. This - document describes version one (1) (there was no version - zero (0)). - </P> - </td> - </tr> - - <tr> - <td>Dimensionality</td> - <td> - <P>This value is the number of dimensions that the data - object has. - </P> - </td> - </tr> - - <tr> - <td>Flags</td> - <td> - <P>This field is used to store flags to indicate the - presence of parts of this message. Bit 0 (the least - significant bit) is used to indicate that maximum - dimensions are present. Bit 1 is used to indicate that - permutation indices are present. - </P> - </td> - </tr> - - <tr> - <td>Dimension #n Size</td> - <td> - <P>This value is the current size of the dimension of the - data as stored in the file. The first dimension stored in - the list of dimensions is the slowest changing dimension - and the last dimension stored is the fastest changing - dimension. - </P> - </td> - </tr> - - <tr> - <td>Dimension #n Maximum Size</td> - <td> - <P>This value is the maximum size of the dimension of the - data as stored in the file. This value may be the special - "<A href="#UnlimitedDim">unlimited</A>" size which indicates - that the data may expand along this dimension indefinitely. - If these values are not stored, the maximum size of each - dimension is assumed to be the dimension's current size. - </P> - </td> - </tr> - - <tr> - <td>Permutation Index #n</td> - <td> - <P>This value is the index permutation used to map - each dimension from the canonical representation to an - alternate axis for each dimension. If these values are - not stored, the first dimension stored in the list of - dimensions is the slowest changing dimension and the last - dimension stored is the fastest changing dimension. - </P> - </td> - </tr> - </table> - </div> - - </P> - -<!-- - <hr> - <h4><a name="DataSpaceMessage">Name: Complex Dataspace (Fiber Bundle?)</a></h4> - <b>Header Message Type: </b>0x0002<br> - <b>Length:</b> varies<br> - - <b>Status:</b> One of the <em>Simple Dataspace</em> or - <em>Complex Dataspace</em> messages is required (but not both) and may - not be repeated.<br> <b>Purpose and Description:</b> The - <em>Dataspace</em> message describes space that the dataset is - mapped onto in a more comprehensive way than the <em>Simple - Dimensionality</em> message is capable of handling. The - dataspace of a dataset encompasses the type of coordinate system - used to locate the dataset's elements as well as the structure and - regularity of the coordinate system. The dataspace also - describes the number of dimensions which the dataset inhabits as - well as a possible higher dimensional space in which the dataset - is located within. - - <br> - <b>Format of Data:</b> - - <center> - <table border cellpadding=4 width="80%"> - <caption align=bottom> - <B>HDF5 Dataspace Message Layout</B> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - - <tr align=center> - <td colspan=4>Mesh Type</td> - <tr align=center> - <td colspan=4>Logical Dimensionality</td> - </table> - </center> - - <p> - <dl> - <dt>The elements of the dimensionality message are described below: - <dd> - <dl> - <dt>Mesh Type: (unsigned 32-bit integer) - <dd>This value indicates whether the grid is - polar/spherical/cartesion, - structured/unstructured and regular/irregular. <br> - The mesh type value is broken up as follows: <br> - - <P> - <center> - <table border cellpadding=4 width="80%"> - <caption align=bottom> - <B>HDF5 Mesh-type Layout</B> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - - <tr align=center> - <td colspan=1>Mesh Embedding</td> - <td colspan=1>Coordinate System</td> - <td colspan=1>Structure</td> - <td colspan=1>Regularity</td> - </table> - </center> - The following are the definitions of mesh-type bytes: - <dl> - <dt>Mesh Embedding - <dd>This value indicates whether the dataset dataspace - is located within - another dataspace or not: - <dl> <dl> - <dt><STANDALONE> - <dd>The dataset mesh is self-contained and is not - embedded in another mesh. - <dt><EMBEDDED> - <dd>The dataset's dataspace is located within - another dataspace, as - described in information below. - </dl> </dl> - <dt>Coordinate System - <dd>This value defines the type of coordinate system - used for the mesh: - <dl> <dl> - <dt><POLAR> - <dd>The last two dimensions are in polar - coordinates, higher dimensions are - cartesian. - <dt><SPHERICAL> - <dd>The last three dimensions are in spherical - coordinates, higher dimensions - are cartesian. - <dt><CARTESIAN> - <dd>All dimensions are in cartesian coordinates. - </dl> </dl> - <dt>Structure - <dd>This value defines the locations of the grid-points - on the axes: - <dl> <dl> - <dt><STRUCTURED> - <dd>All grid-points are on integral, sequential - locations, starting from 0. - <dt><UNSTRUCTURED> - <dd>Grid-points locations in each dimension are - explicitly defined and - may be of any numeric datatype. - </dl> </dl> - <dt>Regularity - <dd>This value defines the locations of the dataset - points on the grid: - <dl> <dl> - <dt><REGULAR> - <dd>All dataset elements are located at the - grid-points defined. - <dt><IRREGULAR> - <dd>Each dataset element has a particular - grid-location defined. - </dl> </dl> - </dl> - <p>The following grid combinations are currently allowed: - <dl> <dl> - <dt><POLAR-STRUCTURED-REGULAR> - <dt><SPHERICAL-STRUCTURED-REGULAR> - <dt><CARTESIAN-STRUCTURED-REGULAR> - <dt><POLAR-UNSTRUCTURED-REGULAR> - <dt><SPHERICAL-UNSTRUCTURED-REGULAR> - <dt><CARTESIAN-UNSTRUCTURED-REGULAR> - <dt><CARTESIAN-UNSTRUCTURED-IRREGULAR> - </dl> </dl> - All of the above grid types can be embedded within another - dataspace. - <br> <br> - <dt>Logical Dimensionality: (unsigned 32-bit integer) - <dd>This value is the number of dimensions that the dataset occupies. - - <P> - <center> - <table border cellpadding=4 width="80%"> - <caption align=bottom> - <B>HDF5 Dataspace Embedded Dimensionality Information</B> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - - <tr align=center> - <td colspan=4>Embedded Dimensionality</td> - <tr align=center> - <td colspan=4>Embedded Dimension Size #1</td> - <tr align=center> - <td colspan=4>.<br>.<br>.<br></td> - <tr align=center> - <td colspan=4>Embedded Dimension Size #n</td> - <tr align=center> - <td colspan=4>Embedded Origin Location #1</td> - <tr align=center> - <td colspan=4>.<br>.<br>.<br></td> - <tr align=center> - <td colspan=4>Embedded Origin Location #n</td> - </table> - </center> - - <dt>Embedded Dimensionality: (unsigned 32-bit integer) - <dd>This value is the number of dimensions of the space the - dataset is located - within. i.e. a planar dataset located within a 3-D space, - or a 3-D dataset - which is a subset of another 3-D space, etc. - <dt>Embedded Dimension Size: (unsigned 32-bit integer) - <dd>These values are the sizes of the dimensions of the - embedded dataspace - that the dataset is located within. - <dt>Embedded Origin Location: (unsigned 32-bit integer) - <dd>These values comprise the location of the dataset's - origin within the embedded dataspace. - </dl> - </dl> - [Comment: need some way to handle different orientations of the - dataset dataspace - within the embedded dataspace]<br> - - <P> - <center> - <table border cellpadding=4 width="80%"> - <caption align=bottom> - <B>HDF5 Dataspace Structured/Regular Grid Information</B> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - - <tr align=center> - <td colspan=4>Logical Dimension Size #1</td> - <tr align=center> - <td colspan=4>Logical Dimension Maximum #1</td> - <tr align=center> - <td colspan=4>.<br>.<br>.<br></td> - <tr align=center> - <td colspan=4>Logical Dimension Size #n</td> - <tr align=center> - <td colspan=4>Logical Dimension Maximum #n</td> - </table> - </center> - - <p> - <dl> - <dt>The elements of the dimensionality message are described below: - <dd> - <dl> - <dt>Logical Dimension Size #n: (unsigned 32-bit integer) - <dd>This value is the current size of the dimension of the - data as stored in - the file. The first dimension stored in the list of - dimensions is the slowest - changing dimension and the last dimension stored is the - fastest changing - dimension. - <dt>Logical Dimension Maximum #n: (unsigned 32-bit integer) - <dd>This value is the maximum size of the dimension of the - data as stored in - the file. This value may be the special value - <UNLIMITED> which - indicates that the data may expand along this dimension - indefinitely. - </dl> - </dl> - <P> - <center> - <table border cellpadding=4 width="80%"> - <caption align=bottom> - <B>HDF5 Dataspace Structured/Irregular Grid Information</B> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - - <tr align=center> - <td colspan=4># of Grid Points in Dimension #1</td> - <tr align=center> - <td colspan=4>.<br>.<br>.<br></td> - <tr align=center> - <td colspan=4># of Grid Points in Dimension #n</td> - <tr align=center> - <td colspan=4>Datatype of Grid Point Locations</td> - <tr align=center> - <td colspan=4>Location of Grid Points in Dimension #1</td> - <tr align=center> - <td colspan=4>.<br>.<br>.<br></td> - <tr align=center> - <td colspan=4>Location of Grid Points in Dimension #n</td> - </table> - </center> - - <P> - <center> - <table border cellpadding=4 width="80%"> - <caption align=bottom> - <B>HDF5 Dataspace Unstructured Grid Information</B> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - - <tr align=center> - <td colspan=4># of Grid Points</td> - <tr align=center> - <td colspan=4>Datatype of Grid Point Locations</td> - <tr align=center> - <td colspan=4>Grid Point Locations<br>.<br>.<br></td> - </table> - </center> ---> - - <hr> - <h4><a name="ReservedMessage_0002">Name: Reserved - Not Assigned Yet</a></h4> - <b>Header Message Type:</b> 0x0002<BR> - <b>Length:</b> N/A<BR> - <b>Status:</b> N/A<BR> - <b>Format of Data:</b> N/A<BR> - - <p><b>Purpose and Description:</b> This message type was skipped during - the initial specification of the file format and may be used in a - future expansion to the format. - - - <hr> - <h4><a name="DataTypeMessage">Name: Datatype</a></h4> - - <P class=item><B>Header Message Type:</B> 0x0003 - </P> - <P class=item><B>Length:</B> variable - </P> - <P class=item><B>Status:</B> Required for dataset or named datatype objects, - may not be repeated. - </P> - - <P class=item><B>Description:</B> The datatype message defines the datatype - for each element of a dataset. A datatype can describe an atomic type - like a fixed- or floating-point type or a compound type like a C - struct. - Datatypes messages are stored - as a list of datatype classes and - their associated properties. - </P> - - <P class=item2>Datatype messages that are part of a dataset object, - do not describe how elements are related to one another, the dataspace - message is used for that purpose. Datatype messages that are part of - a named datatype message describe an "abstract" datatype that can be - used by other objects in the file. - </P> - - <P class=item><B>Format of Data:</B> - <br> - <div align=center> - <table class=format> - <caption> - Datatype Message - </caption> - - <tr> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - </tr> - - <tr> - <td>Class and Version</td> - <td>Class Bit Field, Bits 0-7</td> - <td>Class Bit Field, Bits 8-15</td> - <td>Class Bit Field, Bits 16-23</td> - </tr> - - <tr> - <td colspan=4>Size</td> - </tr> - - <tr> - <td colspan=4><br><br>Properties<br><br><br></td> - </tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Class and Version</td> - <td> - <P>The version of the datatype message and the datatype's class - information are packed together in this field. The version - number is packed in the top 4 bits of the field and the class - is contained in the bottom 4 bits. - </P> - <P>The version number information is used for changes in the - format of the datatype message and is described here: - <table class=list> - <tr> - <th width="30%">Version</th> - <th align=left>Description</th> - </tr> - - <tr> - <td align=center><code>0</code></td> - <td>Never used - </td> - </tr> - <tr> - <td align=center><code>1</code></td> - <td>Used by early versions of the library to encode - compound datatypes with explicit array fields. - See the compound datatype description below for - further details. - </td> - </tr> - <tr> - <td align=center><code>2</code></td> - <td>The current version used by the library. - </td> - </tr> - </table> - </P> - <P>The class of the datatype determines the format for the class - bit field and properties portion of the datatype message, which - are described below. The - following classes are currently defined: - <table width=100% class=list> - <tr> - <th width="30%">Value</th> - <th align=left>Description</th> - </tr> - - <tr> - <td align=center><code>0</code></td> - <td>Fixed-Point</td> - </tr> - - <tr> - <td align=center><code>1</code></td> - <td>Floating-Point</td> - </tr> - - <tr> - <td align=center><code>2</code></td> - <td>Time</td> - </tr> - - <tr> - <td align=center><code>3</code></td> - <td>String</td> - </tr> - - <tr> - <td align=center><code>4</code></td> - <td>Bitfield</td> - </tr> - - <tr> - <td align=center><code>5</code></td> - <td>Opaque</td> - </tr> - - <tr> - <td align=center><code>6</code></td> - <td>Compound</td> - </tr> - - <tr> - <td align=center><code>7</code></td> - <td>Reference</td> - </tr> - - <tr> - <td align=center><code>8</code></td> - <td>Enumerated</td> - </tr> - - <tr> - <td align=center><code>9</code></td> - <td>Variable-Length</td> - </tr> - - <tr> - <td align=center><code>10</code></td> - <td>Array</td> - </tr> - </table> - </P> - </td> - </tr> - - <tr> - <td>Class Bit Fields</td> - <td> - <P>The information in these bit fields is specific to each datatype - class and is described below. All bits not defined for a - datatype class are set to zero. - </P> - </td> - </tr> - - <tr> - <td>Size</td> - <td> - <P>The size of the datatype in bytes. - </P> - </td> - </tr> - - <tr> - <td>Properties</td> - <td> - <P>This variable-sized field encodes information specific to each - datatype class and is described below. If there is no - property information specified for a datatype class, the size - of this field is zero. - </P> - </td> - </tr> - - </table> - </div> - </P> - - <P>Class specific information for Fixed-Point Numbers (Class 0): - - <br> - <div align=center> - <table class=desc> - <caption> - Bit Field Description - </caption> - - <tr> - <th width="10%">Bits</th> - <th>Meaning</th> - </tr> - - <tr> - <td>0</td> - <td><b>Byte Order.</b> If zero, byte order is little-endian; - otherwise, byte order is big endian.</td> - </tr> - - <tr> - <td>1, 2</td> - <td><b>Padding type.</b> Bit 1 is the lo_pad type and bit 2 - is the hi_pad type. If a datum has unused bits at either - end, then the lo_pad or hi_pad bit is copied to those - locations.</td> - </tr> - - <tr> - <td>3</td> - <td><b>Signed.</b> If this bit is set then the fixed-point - number is in 2's complement form.</td> - </tr> - - <tr> - <td>4-23</td> - <td>Reserved (zero).</td> - </tr> - </table> - </div> - - <br> - <div align=center> - <table class=format> - <caption> - Property Descriptions - </caption> - - <tr> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - </tr> - - <tr> - <td colspan=2>Bit Offset</td> - <td colspan=2>Bit Precision</td> - </tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Bit Offset</td> - <td> - <P>The bit offset of the first significant bit of the fixed-point - value within the datatype. The bit offset specifies the number - of bits "to the right of" the value. - </P> - </td> - </tr> - - <tr> - <td>Bit Precision</td> - <td> - <P>The number of bits of precision of the fixed-point value - within the datatype. - </P> - </td> - </tr> - - </table> - </div> - </P> - - <P>Class specific information for Floating-Point Numbers (Class 1): - - <br> - <div align=center> - <table class=desc> - <caption> - Bit Field Description - </caption> - - <tr> - <th width="10%">Bits</th> - <th>Meaning</th> - </tr> - - <tr> - <td>0</td> - <td><b>Byte Order.</b> If zero, byte order is little-endian; - otherwise, byte order is big endian.</td> - </tr> - - <tr> - <td>1, 2, 3</td> - <td><b>Padding type.</b> Bit 1 is the low bits pad type, bit 2 - is the high bits pad type, and bit 3 is the internal bits - pad type. If a datum has unused bits at either end or between - the sign bit, exponent, or mantissa, then the value of bit - 1, 2, or 3 is copied to those locations.</td> - </tr> - - <tr> - <td>4-5</td> - <td><b>Normalization.</b> The value can be 0 if there is no - normalization, 1 if the most significant bit of the - mantissa is always set (except for 0.0), and 2 if the most - signficant bit of the mantissa is not stored but is - implied to be set. The value 3 is reserved and will not - appear in this field.</td> - </tr> - - <tr> - <td>6-7</td> - <td>Reserved (zero).</td> - </tr> - - <tr> - <td>8-15</td> - <td><b>Sign Location.</b> This is the bit position of the sign - bit. Bits are numbered with the least significant bit zero.</td> - </tr> - - <tr> - <td>16-23</td> - <td>Reserved (zero).</td> - </tr> - - </table> - </div> - - <br> - <div align=center> - <table class=format> - <caption> - Property Descriptions - </caption> - - <tr> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - </tr> - - <tr> - <td colspan=2>Bit Offset</td> - <td colspan=2>Bit Precision</td> - </tr> - - <tr> - <td>Exponent Location</td> - <td>Exponent Size</td> - <td>Mantissa Location</td> - <td>Mantissa Size</td> - </tr> - - <tr> - <td colspan=4>Exponent Bias</td> - </tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Bit Offset</td> - <td> - <P>The bit offset of the first significant bit of the floating-point - value within the datatype. The bit offset specifies the number - of bits "to the right of" the value. - </P> - </td> - </tr> - - <tr> - <td>Bit Precision</td> - <td> - <P>The number of bits of precision of the floating-point value - within the datatype. - </P> - </td> - </tr> - - <tr> - <td>Exponent Location</td> - <td> - <P>The bit position of the exponent field. Bits are numbered with - the least significant bit number zero. - </P> - </td> - </tr> - - <tr> - <td>Exponent Size</td> - <td> - <P>The size of the exponent field in bits. - </P> - </td> - </tr> - - <tr> - <td>Mantissa Location</td> - <td> - <P>The bit position of the mantissa field. Bits are numbered with - the least significant bit number zero. - </P> - </td> - </tr> - - <tr> - <td>Mantissa Size</td> - <td> - <P>The size of the mantissa field in bits. - </P> - </td> - </tr> - - <tr> - <td>Exponent Bias</td> - <td> - <P>The bias of the exponent field. - </P> - </td> - </tr> - - </table> - </div> - </P> - - <P>Class specific information for Time (Class 2): - - <br> - <div align=center> - <table class=desc> - <caption> - Bit Field Description - </caption> - - <tr> - <th width="10%">Bits</th> - <th>Meaning</th> - </tr> - - <tr> - <td>0</td> - <td><b>Byte Order.</b> If zero, byte order is little-endian; - otherwise, byte order is big endian.</td> - </tr> - - <tr> - <td>1-23</td> - <td>Reserved (zero).</td> - </tr> - </table> - </div> - - <br> - <div align=center> - <table class=format> - <caption> - Property Descriptions - </caption> - - <tr> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - </tr> - - <tr> - <td colspan=2>Bit Precision</td> - </tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Bit Precision</td> - <td> - <P>The number of bits of precision of the time value. - </P> - </td> - </tr> - - </table> - </div> - </P> - - <P>Class specific information for Strings (Class 3): - - <br> - <div align=center> - <table class=desc> - <caption> - Bit Field Description - </caption> - - <tr> - <th width="10%">Bits</th> - <th>Meaning</th> - </tr> - - <tr> - <td>0-3</td> - <td><b>Padding type.</b> This four-bit value determines the - type of padding to use for the string. The values are: - - <table width=100% class=list> - <tr> - <th width="30%">Value</th> - <th align=left>Description</th> - </tr> - - <tr> - <td align=center><code>0</code></td> - <td>Null Terminate: A zero byte marks the end of the - string and is guaranteed to be present after - converting a long string to a short string. When - converting a short string to a long string the value is - padded with additional null characters as necessary. - </td> - </tr> - - <tr> - <td align=center><code>1</code></td> - <td>Null Pad: Null characters are added to the end of - the value during conversions from short values to long - values but conversion in the opposite direction simply - truncates the value. - </td> - </tr> - - <tr> - <td align=center><code>2</code></td> - <td>Space Pad: Space characters are added to the end of - the value during conversions from short values to long - values but conversion in the opposite direction simply - truncates the value. This is the Fortran - representation of the string. - </td> - </tr> - - <tr> - <td align=center><code>3-15</code></td> - <td>Reserved - </td> - </tr> - </table> - </tr> - - <tr> - <td>4-7</td> - <td><b>Character Set.</b> The character set to use for - encoding the string. The only character set supported is - the 8-bit ASCII (zero) so no translations have been defined - yet.</td> - </tr> - - <tr> - <td>8-23</td> - <td>Reserved (zero).</td> - </tr> - </table> - </div> - - <P>There are no properties defined for the string class. - </P> - </P> - - <P>Class specific information for Bitfields (Class 4): - - <br> - <div align=center> - <table class=desc> - <caption> - Bit Field Description - </caption> - - <tr> - <th width="10%">Bits</th> - <th>Meaning</th> - </tr> - - <tr> - <td>0</td> - <td><b>Byte Order.</b> If zero, byte order is little-endian; - otherwise, byte order is big endian.</td> - </tr> - - <tr> - <td>1, 2</td> - <td><b>Padding type.</b> Bit 1 is the lo_pad type and bit 2 - is the hi_pad type. If a datum has unused bits at either - end, then the lo_pad or hi_pad bit is copied to those - locations.</td> - </tr> - - <tr> - <td>3-23</td> - <td>Reserved (zero).</td> - </tr> - </table> - </div> - - <br> - <div align=center> - <table class=format> - <caption> - Property Description - </caption> - - <tr> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - </tr> - - <tr> - <td colspan=2>Bit Offset</td> - <td colspan=2>Bit Precision</td> - </tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Bit Offset</td> - <td> - <P>The bit offset of the first significant bit of the bitfield - within the datatype. The bit offset specifies the number - of bits "to the right of" the value. - </P> - </td> - </tr> - - <tr> - <td>Bit Precision</td> - <td> - <P>The number of bits of precision of the bitfield - within the datatype. - </P> - </td> - </tr> - </table> - </div> - </P> - - <P>Class specific information for Opaque (Class 5): - - <br> - <div align=center> - <table class=desc> - <caption> - Bit Field Description - </caption> - - <tr> - <th width="10%">Bits</th> - <th>Meaning</th> - </tr> - - <tr> - <td>0-7</td> - <td>Length of ASCII tag in bytes.</td> - </tr> - - <tr> - <td>8-23</td> - <td>Reserved (zero).</td> - </tr> - </table> - </div> - - <br> - <div align=center> - <table class=format> - <caption> - Property Description - </caption> - - <tr> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - </tr> - - <tr> - <td colspan=4><br>ASCII Tag<br> - <br></td> - </tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>ASCII Tag</td> - <td> - <P>This NUL-terminated string provides a description for the - opaque type. It is NUL-padded to a multiple of 8 bytes. - </P> - </td> - </tr> - </table> - </div> - </P> - - <P>Class specific information for Compound (Class 6): - - <br> - <div align=center> - <table class=desc> - <caption> - Bit Field Description - </caption> - - <tr> - <th width="10%">Bits</th> - <th>Meaning</th> - </tr> - - <tr> - <td>0-15</td> - <td><b>Number of Members.</b> This field contains the number - of members defined for the compound datatype. The member - definitions are listed in the Properties field of the data - type message. - </tr> - - <tr> - <td>15-23</td> - <td>Reserved (zero).</td> - </tr> - </table> - </div> - </P> - - <P>The Properties field of a compound datatype is a list of the - member definitions of the compound datatype. The member - definitions appear one after another with no intervening bytes. - The member types are described with a recursive datatype - message. - - <P>Note that the property descriptions are different for different - versions of the datatype version. Additionally note that the version - 0 properties are deprecated and have been replaced with the version - 1 properties in versions of the HDF5 library from the 1.4 release - onward. - - <br> - <div align=center> - <table class=format> - <caption> - Properties Description for Datatype Version 1 - </caption> - - <tr> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - </tr> - - <tr> - <td colspan=4><br>Name<br><br></td> - </tr> - - <tr> - <td colspan=4>Byte Offset of Member</td> - </tr> - - <tr> - <td>Dimensionality</td> - <td colspan=3>Reserved (zero)</td> - </tr> - - <tr> - <td colspan=4>Dimension Permutation</td> - </tr> - - <tr> - <td colspan=4>Reserved (zero)</td> - </tr> - - <tr> - <td colspan=4>Dimension #1 Size (required)</td> - </tr> - - <tr> - <td colspan=4>Dimension #2 Size (required)</td> - </tr> - - <tr> - <td colspan=4>Dimension #3 Size (required)</td> - </tr> - - <tr> - <td colspan=4>Dimension #4 Size (required)</td> - </tr> - - <tr> - <td colspan=4><br>Member Type Message<br><br></td> - </tr> - - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Name</td> - <td> - <P>This NUL-terminated string provides a description for the - opaque type. It is NUL-padded to a multiple of 8 bytes. - </P> - </td> - </tr> - - <tr> - <td>Byte Offset of Member</td> - <td> - <P>This is the byte offset of the member within the datatype. - </P> - </td> - </tr> - - <tr> - <td>Dimensionality</td> - <td> - <P>If set to zero, this field indicates a scalar member. If set - to a value greater than zero, this field indicates that the - member is an array of values. For array members, the size of - the array is indicated by the 'Size of Dimension n' field in - this message. - </P> - </td> - </tr> - - <tr> - <td>Dimension Permutation</td> - <td> - <P>This field was intended to allow an array field to have - it's dimensions permuted, but this was never implemented. - This field should always be set to zero. - </P> - </td> - </tr> - - <tr> - <td>Dimension #n Size</td> - <td> - <P>This field is the size of a dimension of the array field as - stored in the file. The first dimension stored in the list of - dimensions is the slowest changing dimension and the last - dimension stored is the fastest changing dimension. - </P> - </td> - </tr> - - <tr> - <td>Member Type Message</td> - <td> - <P>This field is a datatype message describing the datatype of - the member. - </P> - </td> - </tr> - - </table> - </div> - - <br> - <div align=center> - <table class=format> - <caption> - Properties Description for Datatype Version 2 - </caption> - - <tr> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - </tr> - - <tr> - <td colspan=4><br>Name<br><br></td> - </tr> - - <tr> - <td colspan=4>Byte Offset of Member</td> - </tr> - - <tr> - <td colspan=4><br>Member Type Message<br><br></td> - </tr> - - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Name</td> - <td> - <P>This NUL-terminated string provides a description for the - opaque type. It is NUL-padded to a multiple of 8 bytes. - </P> - </td> - </tr> - - <tr> - <td>Byte Offset of Member</td> - <td> - <P>This is the byte offset of the member within the datatype. - </P> - </td> - </tr> - - <tr> - <td>Member Type Message</td> - <td> - <P>This field is a datatype message describing the datatype of - the member. - </P> - </td> - </tr> - - </table> - </div> - </P> - - <P>Class specific information for Reference (Class 7): - - <br> - <div align=center> - <table class=desc> - <caption> - Bit Field Description - </caption> - - <tr> - <th width="10%">Bits</th> - <th>Meaning</th> - </tr> - - <tr> - <td>0-3</td> - <td><b>Type.</b> This four-bit value contains the type of reference - described. The values defined are: - - <table width=100% class=list> - <tr> - <th width="30%">Value</th> - <th align=left>Description</th> - </tr> - - <tr> - <td align=center><code>0</code></td> - <td>Object Reference: A reference to another object in this - HDF5 file. - </td> - </tr> - - <tr> - <td align=center><code>1</code></td> - <td>Dataset Region Reference: A reference to a region within - a dataset in this HDF5 file. - </td> - </tr> - - <tr> - <td align=center><code>2</code></td> - <td>Internal Reference: A reference to a region within the - current dataset. (Not currently implemented) - </td> - </tr> - - <tr> - <td align=center><code>3-15</code></td> - <td>Reserved - </td> - </tr> - </table> - - </td> - </tr> - - <tr> - <td>15-23</td> - <td>Reserved (zero).</td> - </tr> - </table> - </div> - - <P>There are no properties defined for the reference class. - </P> - </P> - - <P>Class specific information for Enumeration (Class 8): - - <br> - <div align=center> - <table class=desc> - <caption> - Bit Field Description - </caption> - - <tr> - <th width="10%">Bits</th> - <th>Meaning</th> - </tr> - - <tr> - <td>0-15</td> - <td><b>Number of Members.</b> The number of name/value - pairs defined for the enumeration type.</td> - </tr> - - <tr> - <td>16-23</td> - <td>Reserved (zero).</td> - </tr> - </table> - </div> - - <br> - <div align=center> - <table class=format> - <caption> - Property Description - </caption> - - <tr> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - </tr> - - <tr> - <td colspan=4><br>Base Type<br><br></td> - </tr> - - <tr> - <td colspan=4><br>Names<br><br></td> - </tr> - - <tr> - <td colspan=4><br>Values<br><br></td> - </tr> - - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Base Type</td> - <td> - <P>Each enumeration type is based on some parent type, usually an - integer. The information for that parent type is described - recursively by this field. - </P> - </td> - </tr> - - <tr> - <td>Names</td> - <td> - <P>The name for each name/value pair. Each name is stored as a null - terminated ASCII string in a multiple of eight bytes. The names - are in no particular order. - </P> - </td> - </tr> - - <tr> - <td>Values</td> - <td> - <P>The list of values in the same order as the names. The values - are packed (no inter-value padding) and the size of each value - is determined by the parent type. - </P> - </td> - </tr> - - </table> - </div> - </P> - - - <P>Class specific information for Variable-Length (Class 9): - - <br> - <div align=center> - <table class=desc> - <caption> - Bit Field Description - </caption> - - <tr> - <th width="10%">Bits</th> - <th>Meaning</th> - </tr> - - <tr> - <td>0-3</td> - <td><b>Type.</b> This four-bit value contains the type of - variable-length datatype described. The values defined are: - - <table width=100% class=list> - <tr> - <th width="30%">Value</th> - <th align=left>Description</th> - </tr> - - <tr> - <td align=center><code>0</code></td> - <td>Sequence: A variable-length sequence of any sequence of - data. Variable-length sequences do not have padding or - character set information. - </td> - </tr> - - <tr> - <td align=center><code>1</code></td> - <td>String: A variable-length sequence of characters. - Variable-length strings have padding and character set - information. - </td> - </tr> - - <tr> - <td align=center><code>2-15</code></td> - <td>Reserved - </td> - </tr> - </table> - - </td> - </tr> - - <tr> - <td>4-7</td> - <td><b>Padding type.</b> (variable-length string only) - This four-bit value determines the type of padding - used for variable-length strings. The values are the same - as for the string padding type, as follows: - <table width=100% class=list> - <tr> - <th width="30%">Value</th> - <th align=left>Description</th> - </tr> - - <tr> - <td align=center><code>0</code></td> - <td>Null terminate: A zero byte marks the end of a string - and is guaranteed to be present after converting a long - string to a short string. When converting a short string - to a long string, the value is padded with additional null - characters as necessary. - </td> - </tr> - - <tr> - <td align=center><code>1</code></td> - <td>Null pad: Null characters are added to the end of the - value during conversion from a short string to a longer - string. Conversion from a long string to a shorter string - simply truncates the value. - </td> - </tr> - - <tr> - <td align=center><code>2</code></td> - <td>Space pad: Space characters are added to the end of the - value during conversion from a short string to a longer - string. Conversion from a long string to a shorter string - simply truncates the value. This is the Fortran - representation of the string. - </td> - </tr> - - <tr> - <td align=center><code>3-15</code></td> - <td>Reserved - </td> - </tr> - </table> - - This value is set to zero for variable-length sequences. - - </td> - </tr> - - <tr> - <td>8-11</td> - <td><b>Character Set.</b> (variable-length string only) - This four-bit value specifies the character set - to be used for encoding the string: - <table width=100% class=list> - <tr> - <th width="30%">Value</th> - <th align=left>Description</th> - </tr> - - <tr> - <td align=center><code>0</code></td> - <td>ASCII: As of this writing (July 2003, Release 1.6.0), - 8-bit ASCII is the only character set supported. Therefore, - no translations have been defined. - </td> - </tr> - - <tr> - <td align=center><code>1-15</code></td> - <td>Reserved - </td> - </tr> - </table> - - This value is set to zero for variable-length sequences. - - </td> - </tr> - - <tr> - <td>12-23</td> - <td>Reserved (zero).</td> - </tr> - </table> - </div> - - <br> - <div align=center> - <table class=format> - <caption> - Property Description - </caption> - - <tr> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - </tr> - - <tr> - <td colspan=4><br>Base Type<br><br></td> - </tr> - - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Base Type</td> - <td> - <P>Each variable-length type is based on some parent type. The - information for that parent type is described recursively by - this field. - </P> - </td> - </tr> - - </table> - </div> - </P> - - <P>Class specific information for Array (Class 10): - - <P>There are no bit fields defined for the array class. - </P> - - <P>Note that the dimension information defined in the property for this - datatype class is independent of dataspace information for a dataset. - The dimension information here describes the dimensionality of the - information within a data element (or a component of an element, if the - array datatype is nested within another datatype) and the dataspace for a - dataset describes the location of the elements in a dataset. - </P> - - <br> - <div align=center> - <table class=format> - <caption> - Property Description - </caption> - - <tr> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - <th width="25%">Byte</th> - </tr> - - <tr> - <td>Dimensionality</td> - <td colspan=3>Reserved (zero)</td> - </tr> - - <tr> - <td colspan=4>Dimension #1 Size</td> - </tr> - <tr> - <td colspan=4>.<br>.<br>.<br></td> - </tr> - <tr> - <td colspan=4>Dimension #n Size</td> - </tr> - - <tr> - <td colspan=4>Permutation Index #1</td> - </tr> - <tr> - <td colspan=4>.<br>.<br>.<br></td> - </tr> - <tr> - <td colspan=4>Permutation Index #n</td> - </tr> - - <tr> - <td colspan=4><br>Base Type<br><br></td> - </tr> - - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Dimensionality</td> - <td> - <P>This value is the number of dimensions that the array has. - </P> - </td> - </tr> - - <tr> - <td>Dimension #n Size</td> - <td> - <P>This value is the size of the dimension of the array - as stored in the file. The first dimension stored in - the list of dimensions is the slowest changing dimension - and the last dimension stored is the fastest changing - dimension. - </P> - </td> - </tr> - - <tr> - <td>Permutation Index #n</td> - <td> - <P>This value is the index permutation used to map - each dimension from the canonical representation to an - alternate axis for each dimension. Currently, dimension - permutations are not supported and these indices should be set - to the index position minus one (i.e. the first dimension should - be set to 0, the second dimension should be set to 1, etc.) - </P> - </td> - </tr> - - <tr> - <td>Base Type</td> - <td> - <P>Each array type is based on some parent type. The - information for that parent type is described recursively by - this field. - </P> - </td> - </tr> - - </table> - </div> - - </P> - - <hr> - <h4><a name="OldFillValueMessage">Name: Data Storage - Fill Value (Old)</a></h4> - - <P class=item><B>Header Message Type:</B> 0x0004 - </P> - <P class=item><B>Length:</B> varies - </P> - <P class=item><B>Status:</B> Optional, may not be repeated. - </P> - - <P class=item><B>Description:</B> The fill value message stores a single - data value which is returned to the application when an uninitialized - data element is read from a dataset. The fill value is interpreted - with the same datatype as the dataset. If no fill value message is - present then a fill value of all zero bytes is assumed. - </P> - - <P class=item2>This fill value message is deprecated in favor of the "new" - fill value message (Message Type 0x0005) and is only written to the - file for forward compatibility with versions of the HDF5 library before - the 1.6.0 version. Additionally, it only appears for datasets with a - user defined fill value (as opposed to the library default fill value - or an explicitly set "undefined" fill value). - </P> - - <P class=item><B>Format of Data:</B> - <br> - <div align=center> - <table class=format> - <caption> - Fill Value Message (Old) - </caption> - - <tr> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - </tr> - - <tr> - <td colspan=4>Size</td> - </tr> - - <tr> - <td colspan=4><br>Fill Value<br><br></td> - </tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Size</td> - <td> - <P>This is the size of the Fill Value field in bytes. - </P> - </td> - </tr> - - <tr> - <td>Fill Value</td> - <td> - <P>The fill value. The bytes of the fill value are interpreted - using the same datatype as for the dataset. - </P> - </td> - </tr> - </table> - </div> - </P> - - <hr> - <h4><a name="FillValueMessage">Name: Data Storage - Fill Value </a></h4> - - <P class=item><B>Header Message Type:</B> 0x0005 - </P> - <P class=item><B>Length:</B> varies - </P> - <P class=item><B>Status:</B> Required for dataset objects, may not be repeated. - </P> - - <P class=item><B>Description:</B> The fill value message stores a single - data value which is returned to the application when an uninitialized - data element is read from a dataset. The fill value is interpreted - with the same datatype as the dataset. - </P> - - <P class=item><B>Format of Data:</B> - <br> - <div align=center> - <table class=format> - <caption> - Fill Value Message - </caption> - - <tr> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - </tr> - - <tr> - <td>Version</td> - <td>Space Allocation Time</td> - <td>Fill Value Write Time</td> - <td>Fill Value Defined</td> - </tr> - - <tr> - <td colspan=4>Size</td> - </tr> - - <tr> - <td colspan=4><br>Fill Value<br><br></td> - </tr> - </table> - </div> - - <br> - <div align=center> - <table class=desc> - <tr> - <th width="30%">Field Name</th> - <th>Description</th> - </tr> - - <tr> - <td>Version</td> - <td> - <P>The version number information is used for changes in the - format of the fill value message and is described here: - <table class=list> - <tr> - <th width="30%">Version</th> - <th align=left>Description</th> - </tr> - - <tr> - <td align=center><code>0</code></td> - <td>Never used - </td> - </tr> - <tr> - <td align=center><code>1</code></td> - <td>Used by version 1.6.x of the library to encode - fill values. In this version, the Size field is - always present. - </td> - </tr> - <tr> - <td align=center><code>2</code></td> - <td>The current version used by the library (version - 1.7.3 or later). In this version, the Size and - Fill Value fields are - only present if the Fill Value Defined field is set - to 1. - </td> - </tr> - </table> - </P> - </td> - </tr> - - <tr> - <td>Space Allocation Time</td> - <td> - <P>When the storage space for the dataset's raw data will be - allocated. The allowed values are: - <table class=list> - <tr> - <th width="30%">Value</th> - <th align=left>Description</th> - </tr> - - <tr> - <td align=center><code>1</code></td> - <td>Early allocation. Storage space for the entire dataset - should be allocated in the file when the dataset is - created. - </td> - </tr> - <tr> - <td align=center><code>2</code></td> - <td>Late allocation. Storage space for the entire dataset - should not be allocated until the dataset is written - to. - </td> - </tr> - <tr> - <td align=center><code>3</code></td> - <td>Incremental allocation. Storage space for the - dataset should not be allocated until the portion - of the dataset is written to. This is currently - used in conjunction with chunked data storage for - datasets. - </td> - </tr> - </table> - </P> - </td> - </tr> - - <tr> - <td>Fill Value Write Time</td> - <td> - <P>At the time that storage space for the dataset's raw data is - allocated, this value indicates whether the fill value should - be written to the raw data storage elements. The allowed values - are: - <table class=list> - <tr> - <th width="30%">Value</th> - <th align=left>Description</th> - </tr> - - <tr> - <td align=center><code>0</code></td> - <td>On allocation. The fill value is always written to - the raw data storage when the storage space is allocated. - </td> - </tr> - <tr> - <td align=center><code>1</code></td> - <td>Never. The fill value should never be written to - the raw data storage. - </td> - </tr> - <tr> - <td align=center><code>2</code></td> - <td>Fill value written if set by user. The fill value - will be written to the raw data storage when the storage - space is allocated only if the user explicitly set - the fill value. If the fill value is the library - default or is undefined, it will not be written to - the raw data storage. - </td> - </tr> - </table> - </P> - </td> - </tr> - - <tr> - <td>Fill Value Defined</td> - <td> - <P>This value indicates if a fill value is defined for this - dataset. If this value is 0, the fill value is undefined. - If this value is 1, a fill value is defined for this dataset. - For version 2 or later of the fill value message, this value - controls the presence of the Size field. - </P> - </td> - </tr> - - <tr> - <td>Size</td> - <td> - <P>This is the size of the Fill Value field in bytes. This field - is not present if the Version field is >1 and the Fill Value - Defined field is set to 0. - </P> - </td> - </tr> - - <tr> - <td>Fill Value</td> - <td> - <P>The fill value. The bytes of the fill value are interpreted - using the same datatype as for the dataset. This field is - not present if the Version field is >1 and the Fill Value - Defined field is set to 0. - </P> - </td> - </tr> - </table> - </div> - </P> - -<!-- - <hr> - <h4><a name="CompactDataStorageMessage">Name: Data Storage - Compact</a></h4> - - <b>Header Message Type:</b> 0x0006<br> - <b>Length:</b> varies<br> - <b>Status:</b> Optional, may not be repeated.<br> - - <p>This message indicates that the data for the data object is - stored within the current HDF file by including the actual - data as the header data for this message. The data is - stored internally in - the <em>normal format</em>, i.e. in one chunk, uncompressed, etc. - - <P>Note that one and only one of the <em>Data Storage</em> headers can be - stored for each data object. - - <P><b>Format of Data:</b> The message data is actually composed - of dataset data, so the format will be determined by the dataset - format. ---> - - <hr> - <h4><a name="ReservedMessage_0006">Name: Reserved - Not Assigned Yet</a></h4> - <b>Header Message Type:</b> 0x0006<BR> - <b>Length:</b> N/A<BR> - <b>Status:</b> N/A<BR> - <b>Format of Data:</b> N/A<BR> - - <p><b>Purpose and Description:</b> This message type was skipped during - the initial specification of the file format and may be used in a - future expansion to the format. - - <hr> - <h4><a name="ExternalFileListMessage">Name: Data Storage - - External Data Files</a></h4> - <b>Header Message Type:</b> 0x0007<BR> - <b>Length:</b> varies<BR> - <b>Status:</b> Optional, may not be repeated.<BR> - - <p><b>Purpose and Description:</b> The external object message - indicates that the data for an object is stored outside the HDF5 - file. The filename of the object is stored as a Universal - Resource Location (URL) of the actual filename containing the - data. An external file list record also contains the byte offset - of the start of the data within the file and the amount of space - reserved in the file for that data. - - <p> - <center> - <table border cellpadding=4 width="80%"> - <caption align=top> - <b>External File List Message</b> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - </tr> - - <tr align=center> - <td>Version</td> - <td colspan=3>Reserved</td> - </tr> - - <tr align=center> - <td colspan=2>Allocated Slots</td> - <td colspan=2>Used Slots</td> - </tr> - - <tr align=center> - <td colspan=4><br>Heap Address<br><br></td> - </tr> - - <tr align=center> - <td colspan=4><br>Slot Definitions...<br><br></td> - </tr> - </table> - </center> - - <p> - <center> - <table align=center width="80%"> - <tr align=left> - <th width="30%"><U><font size=+1>Field Name</font></U></th> - <th><U><font size=+1>Description</font></U></th> - </tr> - - <tr valign=top> - <td>Version </td> - <td>This value is used to determine the format of the - External File List Message. When the format of the - information in the message is changed, the version number - is incremented and can be used to determine how the - information in the object header is formatted.</td> - </tr> - - <tr valign=top> - <td>Reserved</td> - <td>This field is reserved for future use.</td> - </tr> - - <tr valign=top> - <td>Allocated Slots</td> - <td>The total number of slots allocated in the message. Its - value must be at least as large as the value contained in - the Used Slots field.</td> - </tr> - - <tr valign=top> - <td>Used Slots</td> - <td>The number of initial slots which contain valid - information. The remaining slots are zero filled.</td> - </tr> - - <tr valign=top> - <td>Heap Address</td> - <td>This is the address of a local name heap which contains - the names for the external files. The name at offset zero - in the heap is always the empty string.</td> - </tr> - - <tr valign=top> - <td>Slot Definitions</td> - <td>The slot definitions are stored in order according to - the array addresses they represent. If more slots have - been allocated than what has been used then the defined - slots are all at the beginning of the list.</td> - </tr> - </table> - </center> - - <p> - <center> - <table border cellpadding=4 width="80%"> - <caption align=top> - <b>External File List Slot</b> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - </tr> - - <tr align=center> - <td colspan=4><br>Name Offset (<size> bytes)<br><br></td> - </tr> - - <tr align=center> - <td colspan=4><br>File Offset (<size> bytes)<br><br></td> - </tr> - - <tr align=center> - <td colspan=4><br>Size<br><br></td> - </tr> - </table> - </center> - - <p> - <center> - <table align=center width="80%"> - <tr align=left> - <th width="30%"><U><font size=+1>Field Name</font></U></th> - <th><U><font size=+1>Description</font></U></th> - </tr> - - <tr valign=top> - <td>Name Offset (<size> bytes)</td> - <td>The byte offset within the local name heap for the name - of the file. File names are stored as a URL which has a - protocol name, a host name, a port number, and a file - name: - <code><em>protocol</em>:<em>port</em>//<em>host</em>/<em>file</em></code>. - If the protocol is omitted then "file:" is assumed. If - the port number is omitted then a default port for that - protocol is used. If both the protocol and the port - number are omitted then the colon can also be omitted. If - the double slash and host name are omitted then - "localhost" is assumed. The file name is the only - mandatory part, and if the leading slash is missing then - it is relative to the application's current working - directory (the use of relative names is not - recommended).</td> - </tr> - - <tr valign=top> - <td>File Offset (<size> bytes)</td> - <td>This is the byte offset to the start of the data in the - specified file. For files that contain data for a single - dataset this will usually be zero.</td> - </tr> - - <tr valign=top> - <td>Size</td> - <td>This is the total number of bytes reserved in the - specified file for raw data storage. For a file that - contains exactly one complete dataset which is not - extendable, the size will usually be the exact size of the - dataset. However, by making the size larger one allows - HDF5 to extend the dataset. The size can be set to a value - larger than the entire file since HDF5 will read zeros - past the end of the file without failing.</td> - </tr> - </table> - </center> - - - <hr> - <h4><a name="LayoutMessage">Name: Data Storage - Layout</a></h4> - - <b>Header Message Type:</b> 0x0008<BR> - <b>Length:</b> varies<BR> - <b>Status:</b> Required for datasets, may not be repeated. - - <p><b>Purpose and Description:</b> Data layout describes how the - elements of a multi-dimensional array are arranged in the linear - address space of the file. Three types of data layout are - supported: - - <ol> - <li>The array can be stored in one contiguous area of the file. - The layout requires that the size of the array be constant and - does not permit chunking, compression, checksums, encryption, - etc. The message stores the total size of the array and the - offset of an element from the beginning of the storage area is - computed as in C. - - <li>The array domain can be regularly decomposed into chunks and - each chunk is allocated separately. This layout supports - arbitrary element traversals, compression, encryption, and - checksums, and the chunks can be distributed across external - raw data files (these features are described in other - messages). The message stores the size of a chunk instead of - the size of the entire array; the size of the entire array can - be calculated by traversing the B-tree that stores the chunk - addresses. - - <li>The array can be stored in one contiguous block, as part of - this object header message (this is called "compact" storage below). - </ol> - - <P>Version 3 of this message re-structured the format into specific - properties that are required for each layout class. - - <p> - <center> - <table border cellpadding=4 width="80%"> - <caption align=top> - <B>Data Layout Message, Versions 1 and 2</B> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - </tr> - - <tr align=center> - <td>Version</td> - <td>Dimensionality</td> - <td>Layout Class</td> - <td>Reserved</td> - </tr> - - <tr align=center> - <td colspan=4>Reserved</td> - </tr> - - <tr align=center> - <td colspan=4><br>Address<br><br></td> - </tr> - - <tr align=center> - <td colspan=4>Dimension 0 (4-bytes)</td> - </tr> - - <tr align=center> - <td colspan=4>Dimension 1 (4-bytes)</td> - </tr> - - <tr align=center> - <td colspan=4>...</td> - </tr> - - <tr align=center> - <td colspan=4>Compact Data Size (4-bytes)</td> - </tr> - - <tr align=center> - <td colspan=4>Compact Data</td> - </tr> - - <tr align=center> - <td colspan=4>...</td> - </tr> - </table> - </center> - - <p> - <center> - <table align=center width="80%"> - <tr align=left> - <th width="30%"><U><font size=+1>Field Name</font></U></th> - <th><U><font size=+1>Description</font></U></th> - </tr> - - <tr valign=top> - <td>Version</td> - <td>A version number for the layout message. This value can be - either 1 or 2.</td> - </tr> - - <tr valign=top> - <td>Dimensionality</td> - <td>An array has a fixed dimensionality. This field - specifies the number of dimension size fields later in the - message.</td> - </tr> - - <tr valign=top> - <td>Layout Class</td> - <td>The layout class specifies how the other fields of the - layout message are to be interpreted. A value of one - indicates contiguous storage, a value of two - indicates chunked storage, - while a value of zero - indicates compact storage. Other values will be defined - in the future.</td> - </tr> - - <tr valign=top> - <td>Address</td> - <td>For contiguous storage, this is the address of the first - byte of storage. For chunked storage this is the address - of the B-tree that is used to look up the addresses of the - chunks. This field is not present for compact storage. - If the version for this message is set to 2, the address - may have the "undefined address" value, to indicate that - storage has not yet been allocated for this array.</td> - </tr> - - <tr valign=top> - <td>Dimensions</td> - <td>For contiguous storage the dimensions define the entire - size of the array while for chunked storage they define - the size of a single chunk.</td> - </tr> - - <tr valign=top> - <td>Compact Data Size</td> - <td>This field is only present for compact data storage. - It contains the size of the raw data for the dataset array.</td> - - <tr valign=top> - <td>Compact Data</td> - <td>This field is only present for compact data storage. - It contains the raw data for the dataset array.</td> - </tr> - </table> - </center> - - <p> - <center> - <table border cellpadding=4 width="80%"> - <caption align=top> - <B>Data Layout Message, Version 3</B> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - </tr> - - <tr align=center> - <td>Version</td> - <td>Layout Class</td> - </tr> - - <tr align=center> - <td colspan=4>Properties</td> - </tr> - - </table> - </center> - - <p> - <center> - <table align=center width="80%"> - <tr align=left> - <th width="30%"><U><font size=+1>Field Name</font></U></th> - <th><U><font size=+1>Description</font></U></th> - </tr> - - <tr valign=top> - <td>Version</td> - <td>A version number for the layout message. This value can be - either 1, 2 or 3.</td> - </tr> - - <tr valign=top> - <td>Layout Class</td> - <td>The layout class specifies how the other fields of the - layout message are to be interpreted. A value of one - indicates contiguous storage, a value of two - indicates chunked storage, - while a value of three - indicates compact storage.</td> - </tr> - - <tr valign=top> - <td>Properties</td> - <td>This variable-sized field encodes information specific to each - layout class and is described below. If there is no property - information specified for a layout class, the size of this field - is zero bytes.</td> - </tr> - - </table> - </center> - - <P>Class-specific information for contiguous layout (Class 0): - - <p> - <center> - <table border cellpadding=4 width="80%"> - <caption align=top> - <B>Property Descriptions</B> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - </tr> - - <tr align=center> - <td colspan=4><br>Address<br><br></td> - </tr> - - <tr align=center> - <td colspan=4><br>Size<br><br></td> - </tr> - </table> - </center> - - <p> - <center> - <table align=center width="80%"> - <tr align=left> - <th width="30%"><U><font size=+1>Field Name</font></U></th> - <th><U><font size=+1>Description</font></U></th> - </tr> - - <tr valign=top> - <td>Address</td> - <td>This is the address of the first byte of raw data storage. - The address may have the "undefined address" value, to indicate - that storage has not yet been allocated for this array.</td> - </tr> - - <tr valign=top> - <td>Size</td> - <td>This field contains the size allocated to store the raw data.</td> - </table> - </center> - - <P>Class-specific information for chunked layout (Class 1): - - <p> - <center> - <table border cellpadding=4 width="80%"> - <caption align=top> - <B>Property Descriptions</B> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - </tr> - - <tr align=center> - <td>Dimensionality</td> - </tr> - - <tr align=center> - <td colspan=4><br>Address<br><br></td> - </tr> - - <tr align=center> - <td colspan=4>Dimension 0 (4-bytes)</td> - </tr> - - <tr align=center> - <td colspan=4>Dimension 1 (4-bytes)</td> - </tr> - - <tr align=center> - <td colspan=4>...</td> - </tr> - </table> - </center> - - <p> - <center> - <table align=center width="80%"> - <tr align=left> - <th width="30%"><U><font size=+1>Field Name</font></U></th> - <th><U><font size=+1>Description</font></U></th> - </tr> - - <tr valign=top> - <td>Dimensionality</td> - <td>A chunk has a fixed dimensionality. This field - specifies the number of dimension size fields later in the - message.</td> - </tr> - - <tr valign=top> - <td>Address</td> - <td>This is the address - of the B-tree that is used to look up the addresses of the - chunks. - The address - may have the "undefined address" value, to indicate that - storage has not yet been allocated for this array.</td> - </tr> - - <tr valign=top> - <td>Dimensions</td> - <td>The dimension sizes define the size of a single chunk.</td> - </tr> - </table> - </center> - - <P>Class-specific information for compact layout (Class 2): - - <p> - <center> - <table border cellpadding=4 width="80%"> - <caption align=top> - <B>Property Descriptions</B> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - </tr> - - <tr align=center> - <td colspan=2>Size</td> - </tr> - - <tr align=center> - <td colspan=4>Raw Data</td> - </tr> - - <tr align=center> - <td colspan=4>...</td> - </tr> - </table> - </center> - - <p> - <center> - <table align=center width="80%"> - <tr align=left> - <th width="30%"><U><font size=+1>Field Name</font></U></th> - <th><U><font size=+1>Description</font></U></th> - </tr> - - <tr valign=top> - <td>Size</td> - <td>This field contains the size of the raw data for the dataset array.</td> - - <tr valign=top> - <td>Raw Data</td> - <td>This field contains the raw data for the dataset array.</td> - </tr> - </table> - </center> - - - <hr> - <h4><a name="ReservedMessage_0009">Name: Reserved - Not Assigned Yet</a></h4> - <b>Header Message Type:</b> 0x0009<BR> - <b>Length:</b> N/A<BR> - <b>Status:</b> N/A<BR> - <b>Format of Data:</b> N/A<BR> - - <p><b>Purpose and Description:</b> This message type was skipped during - the initial specification of the file format and may be used in a - future expansion to the format. - - <hr> - <h4><a name="ReservedMessage_000A">Name: Reserved - Not Assigned Yet</a></h4> - <b>Header Message Type:</b> 0x000A<BR> - <b>Length:</b> N/A<BR> - <b>Status:</b> N/A<BR> - <b>Format of Data:</b> N/A<BR> - - <p><b>Purpose and Description:</b> This message type was skipped during - the initial specification of the file format and may be used in a - future expansion to the format. - - <hr> - <h4><a name="FilterMessage">Name: Data Storage - Filter Pipeline</a></h4> - <b>Header Message Type:</b> 0x000B<BR> - <b>Length:</b> varies<BR> - <b>Status:</b> Optional, may not be repeated. - - <p><b>Purpose and Description:</b> This message describes the - filter pipeline which should be applied to the data stream by - providing filter identification numbers, flags, a name, an - client data. - - <p> - <center> - <table border align=center cellpadding=4 width="80%"> - <caption align=top> - <b>Filter Pipeline Message</b> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - </tr> - - <tr align=center> - <td>Version</td> - <td>Number of Filters</td> - <td colspan=2>Reserved</td> - </tr> - - <tr align=center> - <td colspan=4>Reserved</td> - </tr> - - <tr align=center> - <td colspan=4><br>Filter List<br><br></td> - </tr> - </table> - </center> - - <p> - <center> - <table align=center width="80%"> - <tr align=left> - <th width="30%"><U><font size=+1>Field Name</font></U></th> - <th><U><font size=+1>Description</font></U></th> - </tr> - - <tr valign=top> - <td>Version</td> - <td>The version number for this message. This document - describes version one.</td> - </tr> - - <tr valign=top> - <td>Number of Filters</td> - <td>The total number of filters described by this - message. The maximum possible number of filters in a - message is 32.</td> - </tr> - - <tr valign=top> - <td>Filter List</td> - <td>A description of each filter. A filter description - appears in the next table.</td> - </tr> - </table> - </center> - - <p> - <center> - <table border align=center cellpadding=4 width="80%"> - <caption align=top> - <b>Filter Pipeline Message</b> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - </tr> - - <tr align=center> - <td colspan=2>Filter Identification</td> - <td colspan=2>Name Length</td> - </tr> - - <tr align=center> - <td colspan=2>Flags</td> - <td colspan=2>Client Data Number of Values</td> - </tr> - - <tr align=center> - <td colspan=4><br>Name<br><br></td> - </tr> - - <tr align=center> - <td colspan=4><br>Client Data<br><br></td> - </tr> - - <tr align=center> - <td colspan=4>Padding</td> - </tr> - </table> - </center> - - <p> - <center> - <table align=center width="80%"> - <tr align=left> - <th width="30%"><U><font size=+1>Field Name</font></U></th> - <th><U><font size=+1>Description</font></U></th> - </tr> - - <tr valign=top> - <td>Filter Identification</td> - <td>This is a unique (except in the case of testing) - identifier for the filter. Values from zero through 255 - are reserved for filters defined by the NCSA HDF5 - library. Values 256 through 511 have been set aside for - use when developing/testing new filters. The remaining - values are allocated to specific filters by contacting the - <a href="mailto:hdf5dev@ncsa.uiuc.edu">HDF5 Development - Team</a>.</td> - </tr> - - <tr valign=top> - <td>Name Length</td> - <td>Each filter has an optional null-terminated ASCII name - and this field holds the length of the name including the - null termination padded with nulls to be a multiple of - eight. If the filter has no name then a value of zero is - stored in this field.</td> - </tr> - - <tr valign=top> - <td>Flags</td> - <td>The flags indicate certain properties for a filter. The - bit values defined so far are: - - <dl> - <dt><code>bit 1</code> - <dd>If set then the filter is an optional filter. - During output, if an optional filter fails it will be - silently removed from the pipeline. - </dl> - </tr> - - <tr valign=top> - <td>Client Data Number of Values</td> - <td>Each filter can store a few integer values to control - how the filter operates. The number of entries in the - Client Data array is stored in this field.</td> - </tr> - - <tr valign=top> - <td>Name</td> - <td>If the Name Length field is non-zero then it will - contain the size of this field, a multiple of eight. This - field contains a null-terminated, ASCII character - string to serve as a comment/name for the filter.</td> - </tr> - - <tr valign=top> - <td>Client Data</td> - <td>This is an array of four-byte integers which will be - passed to the filter function. The Client Data Number of - Values determines the number of elements in the - array.</td> - </tr> - - <tr valign=top> - <td>Padding</td> - <td>Four bytes of zeros are added to the message at this - point if the Client Data Number of Values field contains - an odd number.</td> - </tr> - </table> - </center> - - <hr> - <h4><a name="AttributeMessage">Name: Attribute</a></h4> - <b>Header Message Type:</b> 0x000C<BR> - <b>Length:</b> varies<BR> - <b>Status:</b> Optional, may be repeated.<BR> - - <p><b>Purpose and Description:</b> The <em>Attribute</em> - message is used to list objects in the HDF file which are used - as attributes, or "metadata" about the current object. An - attribute is a small dataset; it has a name, a datatype, a data - space, and raw data. Since attributes are stored in the object - header they must be relatively small (<64KB) and can be - associated with any type of object which has an object header - (groups, datasets, named types and spaces, etc.). - - <p>Note: Attributes on an object must have unique names. (The HDF5 library - currently enforces this by causing the creation of an attribute with - a duplicate name to fail) - Attributes on different objects may have the same name, however. - - <p> - <center> - <table border align=center cellpadding=4 width="80%"> - <caption align=top> - <b>Attribute Message</b> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - </tr> - - <tr align=center> - <td>Version</td> - <td>Reserved</td> - <td colspan=2>Name Size</td> - </tr> - - <tr align=center> - <td colspan=2>Type Size</td> - <td colspan=2>Space Size</td> - </tr> - - <tr align=center> - <td colspan=4><br>Name<br><br></td> - </tr> - - <tr align=center> - <td colspan=4><br>Type<br><br></td> - </tr> - - <tr align=center> - <td colspan=4><br>Space<br><br></td> - </tr> - - <tr align=center> - <td colspan=4><br>Data<br><br></td> - </tr> - </table> - </center> - - <p> - <center> - <table align=center width="80%"> - <tr align=left> - <th width="30%"><U><font size=+1>Field Name</font></U></th> - <th><U><font size=+1>Description</font></U></th> - </tr> - - <tr valign=top> - <td>Version</td> - <td>Version number for the message. This document describes - version 1 of attribute messages.</td> - </tr> - - <tr valign=top> - <td>Reserved</td> - <td>This field is reserved for later use and is set to - zero.</td> - </tr> - - <tr valign=top> - <td>Name Size</td> - <td>The length of the attribute name in bytes including the - null terminator. Note that the Name field below may - contain additional padding not represented by this - field.</td> - </tr> - - <tr valign=top> - <td>Type Size</td> - <td>The length of the datatype description in the Type - field below. Note that the Type field may contain - additional padding not represented by this field.</td> - </tr> - - <tr valign=top> - <td>Space Size</td> - <td>The length of the dataspace description in the Space - field below. Note that the Space field may contain - additional padding not represented by this field.</td> - </tr> - - <tr valign=top> - <td>Name</td> - <td>The null-terminated attribute name. This field is - padded with additional null characters to make it a - multiple of eight bytes.</td> - </tr> - - <tr valign=top> - <td>Type</td> - <td>The datatype description follows the same format as - described for the datatype object header message. This - field is padded with additional zero bytes to make it a - multiple of eight bytes.</td> - </tr> - - <tr valign=top> - <td>Space</td> - <td>The dataspace description follows the same format as - described for the dataspace object header message. This - field is padded with additional zero bytes to make it a - multiple of eight bytes.</td> - </tr> - - <tr valign=top> - <td>Data</td> - <td>The raw data for the attribute. The size is determined - from the datatype and dataspace descriptions. This - field is <em>not</em> padded with additional zero - bytes.</td> - </tr> - </table> - </center> - - <hr> - <h4><a name="CommentMessage">Name: Object Comment</a></h4> - - <p><b>Header Message Type:</b> 0x000D<br> - <b>Length:</b> varies<br> - <b>Status:</b> Optional, may not be repeated. - - <p><b>Purpose and Description:</b> The object comment is - designed to be a short description of an object. An object comment - is a sequence of non-zero (<code>\0</code>) ASCII characters with no other - formatting included by the library. - - <p> - <center> - <table border align=center cellpadding=4 width="80%"> - <caption align=top> - <b>Name Message</b> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - </tr> - - <tr align=center> - <td colspan=4><br>Comment<br><br></td> - </tr> - </table> - </center> - - <p> - <center> - <table align=center width="80%"> - <tr align=left> - <th width="30%"><U><font size=+1>Field Name</font></U></th> - <th><U><font size=+1>Description</font></U></th> - </tr> - - <tr valign=top> - <td>Name</td> - <td>A null terminated ASCII character string.</td> - </tr> - </table> - </center> - - <hr> - <h4><a name="OldModifiedMessage">Name: Object Modification Date & Time (Old)</a></h4> - - <p><b>Header Message Type:</b> 0x000E<br> - <b>Length:</b> fixed<br> - <b>Status:</b> Optional, may not be repeated. - - <p><b>Purpose and Description:</b> The object modification date - and time is a timestamp which indicates (using ISO-8601 date and - time format) the last modification of an object. The time is - updated when any object header message changes according to the - system clock where the change was posted. - - <p>This modification time message is deprecated in favor of the "new" - modification time message (Message Type 0x0012) and is no longer written - to the file in versions of the HDF5 library after the 1.6.0 version. - </p> - - - <p> - <center> - <table border align=center cellpadding=4 width="80%"> - <caption align=top> - <b>Modification Time Message</b> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - </tr> - - <tr align=center> - <td colspan=4>Year</td> - </tr> - - <tr align=center> - <td colspan=2>Month</td> - <td colspan=2>Day of Month</td> - </tr> - - <tr align=center> - <td colspan=2>Hour</td> - <td colspan=2>Minute</td> - </tr> - - <tr align=center> - <td colspan=2>Second</td> - <td colspan=2>Reserved</td> - </tr> - </table> - </center> - - <p> - <center> - <table align=center width="80%"> - <tr align=left> - <th width="30%"><U><font size=+1>Field Name</font></U></th> - <th><U><font size=+1>Description</font></U></th> - </tr> - - <tr valign=top> - <td>Year</td> - <td>The four-digit year as an ASCII string. For example, - <code>1998</code>. All fields of this message should be interpreted - as coordinated universal time (UTC)</td> - </tr> - - <tr valign=top> - <td>Month</td> - <td>The month number as a two digit ASCII string where - January is <code>01</code> and December is <code>12</code>.</td> - </tr> - - <tr valign=top> - <td>Day of Month</td> - <td>The day number within the month as a two digit ASCII - string. The first day of the month is <code>01</code>.</td> - </tr> - - <tr valign=top> - <td>Hour</td> - <td>The hour of the day as a two digit ASCII string where - midnight is <code>00</code> and 11:00pm is <code>23</code>.</td> - </tr> - - <tr valign=top> - <td>Minute</td> - <td>The minute of the hour as a two digit ASCII string where - the first minute of the hour is <code>00</code> and - the last is <code>59</code>.</td> - </tr> - - <tr valign=top> - <td>Second</td> - <td>The second of the minute as a two digit ASCII string - where the first second of the minute is <code>00</code> - and the last is <code>59</code>.</td> - </tr> - - <tr valign=top> - <td>Reserved</td> - <td>This field is reserved and should always be zero.</td> - </tr> - </table> - </center> - - <hr> - <h4><a name="SharedMessage">Name: Shared Object Message</a></h4> - <b>Header Message Type:</b> 0x000F<br> - <b>Length:</b> 4 Bytes<br> - <b>Status:</b> Optional, may be repeated. - - <p>A constant message can be shared among several object headers - by writing that message in the global heap and having the object - headers all point to it. The pointing is accomplished with a - Shared Object message which is understood directly by the object - header layer of the library. It is also possible to have a - message of one object header point to a message in some other - object header, but care must be exercised to prevent cycles. - - <p>If a message is shared, then the message appears in the global - heap and its message ID appears in the Header Message Type - field of the object header. Also, the Flags field in the object - header for that message will have bit two set (the - <code>H5O_FLAG_SHARED</code> bit). The message body in the - object header will be that of a Shared Object message defined - here and not that of the pointed-to message. - - <p> - <center> - <table border cellpadding=4 width="80%"> - <caption align=top> - <b>Shared Message Message</b> - </caption> - - <tr align=center> - <th width="25%">byte</td> - <th width="25%">byte</td> - <th width="25%">byte</td> - <th width="25%">byte</td> - </tr> - - <tr align=center> - <td>Version</td> - <td>Flags</td> - <td colspan=2>Reserved</td> - </tr> - - <tr align=center> - <td colspan=4>Reserved</td> - </tr> - - <tr align=center> - <td colspan=4><br>Pointer<br><br></td> - </tr> - </table> - </center> - - <p> - <center> - <table align=center width="80%"> - <tr align=left> - <th width="30%"><U><font size=+1>Field Name</font></U></th> - <th><U><font size=+1>Description</font></U></th> - </tr> - - <tr valign=top> - <td>Version</td> - <td>The version number for the message. This document - describes version one of shared messages.</td> - </tr> - - <tr valign=top> - <td>Flags</td> - <td>The Shared Message message points to a message which is - shared among multiple object headers. The Flags field - describes the type of sharing: - - <dl> - <dt><code>Bit 0</code> - <dd>If this bit is clear then the actual message is the - first message in some other object header; otherwise - the actual message is stored in the global heap. - - <dt><code>Bits 2-7</code> - <dd>Reserved (always zero) - </dl> - </tr> - - <tr valign=top> - <td>Pointer</td> - <td>This field points to the actual message. The format of - the pointer depends on the value of the Flags field. If - the actual message is in the global heap then the pointer - is the file address of the global heap collection that - holds the message, and a four-byte index into that - collection. Otherwise the pointer is a group entry - that points to some other object header.</td> - </tr> - </table> - </center> - - -<hr> -<h4><a name="ContinuationMessage">Name: Object Header Continuation</a></h4> -<b>Header Message Type:</b> 0x0010<BR> -<b>Length:</b> fixed<BR> -<b>Status:</b> Optional, may be repeated.<BR> -<b>Purpose and Description:</b> The object header continuation is the location -in the file of more header messages for the current data object. This can be -used when header blocks are large, or likely to change over time.<BR> -<b>Format of Data:</b><p> - The object header continuation is formatted as follows (assuming a 4-byte -length & offset are being used in the current file): - -<P> -<center> -<table border cellpadding=4 width=60%> -<caption align=bottom> -<B>HDF5 Object Header Continuation Message Layout</B> -</caption> - -<tr align=center> -<th width=25%>byte</th> -<th width=25%>byte</th> -<th width=25%>byte</th> -<th width=25%>byte</th> - -<tr align=center> -<td colspan=4>Header Continuation Offset</td> -<tr align=center> -<td colspan=4>Header Continuation Length</td> -</table> -</center> - -<P> -<dl> -<dt>The elements of the Header Continuation Message are described below: -<dd> -<dl> -<dt>Header Continuation Offset: (<offset> bytes) -<dd>This value is the offset in bytes from the beginning of the file where the -header continuation information is located. -<dt>Header Continuation Length: (<length> bytes) -<dd>This value is the length in bytes of the header continuation information in -the file. -</dl> -</dl> - -<hr> -<h4><a name="SymbolTableMessage">Name: Group Message</a></h4> -<b>Header Message Type:</b> 0x0011<BR> -<b>Length:</b> fixed<BR> -<b>Status:</b> Required for groups, may not be repeated.<BR> -<b>Purpose and Description:</b> Each group has a B-tree and a -name heap which are pointed to by this message.<BR> -<b>Format of data:</b> -<p>The group message is formatted as follows: - -<p> -<center> -<table border cellpadding=4 width="80%"> -<caption align=bottom> -<b>HDF5 Object Header Group Message Layout</b> -</caption> - -<tr align=center> -<th width="25%">byte</th> -<th width="25%">byte</th> -<th width="25%">byte</th> -<th width="25%">byte</th> - -<tr align=center> -<td colspan=4>B-tree Address</td> - -<tr align=center> -<td colspan=4>Heap Address</td> -</table> -</center> - -<P> -<dl> -<dt>The elements of the Group Message are described below: -<dd> -<dl> -<dt>B-tree Address (<offset> bytes) -<dd>This value is the offset in bytes from the beginning of the file -where the B-tree is located. -<dt>Heap Address (<offset> bytes) -<dd>This value is the offset in bytes from the beginning of the file -where the group name heap is located. -</dl> -</dl> - - <hr> - <h4><a name="ModifiedMessage">Name: Object Modification Date & Time</a></h4> - - <P class=item><B>Header Message Type:</B> 0x0012 - </P> - <P class=item><B>Length:</B> Fixed - </P> - <P class=item><B>Status:</B> Optional, may not be repeated. - </P> - - <P class=item><B>Description:</B> The object modification date - and time is a timestamp which indicates - the last modification of an object. The time is - updated when any object header message changes according to the - system clock where the change was posted. - </P> - - <p> - <center> - <table border align=center cellpadding=4 width="80%"> - <caption align=top> - <b>Modification Time Message</b> - </caption> - - <tr align=center> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - <th width="25%">byte</th> - </tr> - - <tr align=center> - <td colspan=1>Version</td> - <td colspan=3>Reserved</td> - </tr> - - <tr align=center> - <td colspan=4>Seconds After Epoch</td> - </tr> - </table> - </center> - - <p> - <center> - <table align=center width="80%"> - <tr align=left> - <th width="30%"><U><font size=+1>Field Name</font></U></th> - <th><U><font size=+1>Description</font></U></th> - </tr> - - <tr valign=top> - <td>Version</td> - <td>The version number for the message. This document - describes version one of the new modification time message.</td> - </tr> - - <tr valign=top> - <td>Reserved</td> - <td>This field is reserved and should always be zero.</td> - </tr> - - <tr valign=top> - <td>Seconds After Epoch</td> - <td>The number of seconds since 0 hours, 0 - minutes, 0 seconds, January 1, 1970, Coordinated Universal Time. - </tr> - </table> - </center> - -<h3><a name="SharedObjectHeader">Disk Format: Level 2b - Shared Data Object Headers</a></h3> -<P>In order to share header messages between several dataset objects, object -header messages may be placed into the global heap. Since these -messages require additional information beyond the basic object header message -information, the format of the shared message is detailed below. - -<BR> <BR> -<center> -<table border cellpadding=4 width=60%> -<caption align=bottom> -<B>HDF5 Shared Object Header Message</B> -</caption> - -<tr align=center> -<th width=25%>byte</th> -<th width=25%>byte</th> -<th width=25%>byte</th> -<th width=25%>byte</th> - -<tr align=center> -<td colspan=4>Reference Count of Shared Header Message</td> -<tr align=center> -<td colspan=4><br> Shared Object Header Message<br> <br></td> -</table> -</center> - -<p> -<dl> -<dt> The elements of the shared object header message are described below: -<dd> -<dl> -<dt>Reference Count of Shared Header Message: (32-bit unsigned integer) -<dd>This value is used to keep a count of the number of dataset objects which -refer to this message from their dataset headers. When this count reaches zero, -the shared message header may be removed from the global heap. -<dt>Shared Object Header Message: (various lengths) -<dd>The data stored for the shared object header message is formatted in the -same way as the private object header messages described in the object header -description earlier in this document and begins with the header message Type. -</dl> -</dl> - - -<h3><a name="DataStorage">Disk Format: Level 2c - Data Object Data Storage</a></h3> -<P>The data for an object is stored separately from the header -information in the file and may not actually be located in the HDF5 file -itself if the header indicates that the data is stored externally. The -information for each record in the object is stored according to the -dimensionality of the object (indicated in the dimensionality header message). -Multi-dimensional data is stored in C order [same as current scheme], i.e. the -"last" dimension changes fastest. -<P>Data whose elements are composed of simple number-types are stored in -native-endian IEEE format, unless they are specifically defined as being stored -in a different machine format with the architecture-type information from the -number-type header message. This means that each architecture will need to -[potentially] byte-swap data values into the internal representation for that -particular machine. -<P> Data with a variable-length datatype is stored in the global heap -of the HDF5 file. Global heap identifiers are stored in the -data object storage. -<P>Data whose elements are composed of pointer number-types are stored in several -different ways depending on the particular pointer type involved. Simple -pointers are just stored as the dataset offset of the object being pointed to with the -size of the pointer being the same number of bytes as offsets in the file. -Dataset region references are stored as a heap-ID which points to the following -information within the file-heap: an offset of the object pointed to, number-type -information (same format as header message), dimensionality information (same -format as header message), sub-set start and end information (i.e. a coordinate -location for each), and field start and end names (i.e. a [pointer to the] -string indicating the first field included and a [pointer to the] string name -for the last field). - -<P>Data of a compound datatype is stored as a contiguous stream of the items -in the structure, with each item formatted according to its datatype.</p> - -<h3><a name="Appendix">Appendix</a></h3> -<P>Definitions of various terms used in this document. -</P> -<P>The <A name="UndefinedAddress">"undefined address"</A> for a file is a -file address with all bits set, i.e. <code>0xffff...ff</code>. -<P>The <A name="UnlimitedDim">"unlimited size"</A> for a size is a -value with all bits set, i.e. <code>0xffff...ff</code>. - -<!-- #BeginLibraryItem "/ed_libs/NavBar_ADevG.lbi" --><hr> -<center> -<table border=0 width=98%> -<tr><td valign=top align=left> - <a href="index.html">HDF5 documents and links</a> <br> - <a href="H5.intro.html">Introduction to HDF5</a> <br> - <!-- - <a href="Glossary.html">Glossary</a><br> - --> -</td> -<td valign=top align=right> - <a href="http://hdf.ncsa.uiuc.edu/HDF5/doc/UG/index.html">HDF5 User's Guide</a> <br> - <a href="RM_H5Front.html">HDF5 Reference Manual</a> <br> - <a href="ADGuide.html">HDF5 Application Developer's Guide</a> <br> -</td></tr> -</table> -</center> -<hr> -<!-- #EndLibraryItem --><!-- -<address><a href="mailto:koziol@ncsa.uiuc.edu">Quincey Koziol</a></address> -<address><a href="mailto:matzke@llnl.gov">Robb Matzke</a></address> ---> - - -<!-- #BeginLibraryItem "/ed_libs/Footer.lbi" --><address> -<a href="mailto:hdfhelp@ncsa.uiuc.edu">HDF Help Desk</a> -<br> -Describes HDF5 Release 1.7, the unreleased development branch; working toward HDF5 Release 1.8.0 -</address><!-- #EndLibraryItem --><!-- hhmts start --> -Last modified: 12 July 2004 -<!-- hhmts end --> - -</body> -</html> |