diff options
Diffstat (limited to 'doc/html/References.html')
| -rw-r--r-- | doc/html/References.html | 558 |
1 files changed, 558 insertions, 0 deletions
diff --git a/doc/html/References.html b/doc/html/References.html new file mode 100644 index 0000000..88534e5 --- /dev/null +++ b/doc/html/References.html @@ -0,0 +1,558 @@ +<!DOCTYPE HTML PUBLIC "-//IETF//DTD HTML//EN"> +<html> +<head> +<title>References</title> +</head> + +<body> +<h1>References</h1> + +<h2>1. Introduction</h2> + +This document discusses the kinds of references implemented +(and planned) in HDF5 and the functions implemented (and planned) +to support them. + + +<h2>2. References</h2> + +This section contains an overview of the kinds of references +implemented, or planned for implementation, in HDF5. + +<p> + +<dl> + <dt>Object reference + <dd>Reference to an entire object in the current HDF5 file. + <br> + <em>The only kind of reference currently implemented.</em> + <p> + An object reference points to an entire object in the + current HDF5 file by storing the relative file address + (OID) of the object header for the object pointed to. + The relative file address of an object header is + constant for the life of the object. + An object reference is of a fixed size in the file. + <p> + <dt>Dataset region reference + <dd>Reference to a specific dataset region. + <br> + <em>Not yet implemented.</em> + <p> + A dataset region reference points to a region of a + dataset in the current HDF5 file by storing the OID + of the dataset and the global heap offset of the + region referenced. The region referenced is located + by retrieving the coordinates of the areas in the + region from the global heap. A dataset region + reference is of a variable size in the file. + <p> + <dt>Internal dataset region reference + <dd>Reference to a region within the current dataset. + <br> + <em>Not yet implemented.</em> + <p> + An internal dataset region reference points to a + region of the current dataset by storing the + coordinates of the region. An internal dataset + region reference is of a fixed size in the file. +</dl> + + +<b>Note:</b> All references are treated as soft links for the +purposes of reference counting. The library does not keep track of +reference links and they may dangle if the object they refer to +is deleted, moved, or not yet available. + + +<h2>3. Reference Types</h2> + +This section lists valid HDF5 reference types for use in the +H5R functions. + +<center> +<table> +<tr><th align=left>Reference Type</th><th align=left>Value </th><th align=left>Description</th></tr> +<tr><td><code>H5R_BADTYPE</code></td> + <td align=right><code>-1 </code></td> + <td>Invalid reference type</td></tr> +<tr><td><code>H5R_OBJECT</code></td> + <td align=right><code>0 </code></td> + <td>Object reference</td></tr> +<tr><td><code>H5R_DATASET_REGION</code></td> + <td align=right><code>1 </code></td> + <td>Dataset region reference</td></tr> +<tr><td><code>H5R_INTERNAL</code></td> + <td align=right><code>2 </code></td> + <td>Internal reference</td></tr> +</table> +</center> + + +<h2>4. Functions</h2> + +Four functions, three in the H5R interface and one in the +H5I interface, have been implemented to support references. +The H5I function is also useful outside the context of references. +<p> +<dl> + <dt><em>herr_t</em> <code>H5Rcreate(</code><em>href_t *</em><code>reference,</code> + <em>hid_t</em> <code>loc_id,</code> + <em>const char *</em><code>name,</code> + <em>H5R_type_t</em> <code>type,</code> + <em>hid_t</em> <code>space_id)</code> + <dd><code>H5Rcreate</code> creates an object which is a + particular type of reference (specified with the + <code>type</code> parameter) to some file object and/or + location specified with the <code>space_id</code> parameter. + For dataset region references, the selection specified + in the dataspace is the portion of the dataset which + will be referred to. + <p> + <em>Currently only object references which point to + entire datasets can be created.</em> + <p> + + <dt><em>hid_t</em> <code>H5Rdereference(</code><em>hid_t</em> <code>dset,</code> + <em>H5R_type_t</em> <code>rtype,</code> + <em>href_t *</em><code>ref)</code> + <dd><code>H5Rdereference</code> opens the object referenced + and returns an identifier for that object. + The parameter <code>ref</code> specifies a reference of + type <code>rtype</code> that is stored in the dataset + <code>dset</code>. + <p> + + <dt><em>hid_t</em> <code>H5Rget_region(</code><em>hid_t</em> <code>dataset,</code> + <em>H5R_type_t</em> <code>type,</code> + <em>href_t *</em><code>reference)</code> + <dd><code>H5Rget_region</code> creates a copy of dataspace of + the dataset that is pointed to and defines a selection in + the copy which is the location (or region) pointed to. + The parameter <code>ref</code> specifies a reference of + type <code>rtype</code> that is stored in the dataset + <code>dset</code>. + <p> + <em>This function is not yet implemented.</em> + <p> + + <dt><em>H5I_type_t</em> <code>H5Iget_type(</code><em>hid_t</em> <code>id)</code> + <dd>Returns the type of object referred to by the + identifier <code>id</code>. Valid return values appear + in the following list: + <center> + <table> + <tr><td><code>H5I_BADID</code></td> + <td>Invalid ID</td></tr> + <tr><td><code>H5I_FILE</code></td> + <td>File objects</td></tr> + <tr><td><code>H5I_GROUP</code></td> + <td>Group objects</td></tr> + <tr><td><code>H5I_DATATYPE</code></td> + <td>Data type objects</td></tr> + <tr><td><code>H5I_DATASPACE</code></td> + <td>Dataspace objects</td></tr> + <tr><td><code>H5I_DATASET</code></td> + <td>Dataset objects</td></tr> + <tr><td><code>H5I_ATTR</code></td> + <td>Attribute objects</td></tr> + </table> + </center> + <p> + This function was inspired by the need of users to figure + out which type of object closing function + (<code>H5Dclose</code>, <code>H5Gclose</code>, etc.) + to call after a call to <code>H5Ddereference</code>, + but it is also of general use. + <p> +</dl> + + + +<h2>5. Examples</h2> + +<b>Object Reference Writing Example</b> +<br> +Create a dataset which has links to other datasets as part +of its raw data and write the dataset to the file. +<p> + +<pre> +{ + hid_t file1; + hid_t dataset1; + hid_t datatype, dataspace; + char buf[128]; + href_t link; + href_t data[10][10]; + int rank; + size_t dimsf[2]; + int i, j; + + /* Open the file */ + file1=H5Fopen("example.h5", H5F_ACC_RDWR, H5P_DEFAULT); + + /* Describe the size of the array and create the data space */ + rank=2; + dimsf[0] = 10; + dimsf[1] = 10; + dataspace = H5Screate_simple(rank, dimsf, NULL); + + /* Define datatype */ + datatype = H5Tcopy(H5T_STD_REF_OBJ); + + /* Create a dataset */ + dataset1=H5Dcreate(file1,"Dataset One",datatype,dataspace,H5P_DEFAULT); + + /* Construct array of OIDs for other datasets in the file */ + /* somewhat hokey and artificial, but demonstrates the point */ + for(i=0; i<10; i++) + for(j=0; j<10; i++) + { + sprintf(buf,"/Group/Linked Set %d-%d",i,j); + if(H5Rcreate(&link,file1,buf,H5R_REF_OBJ,-1)>0) + data[i][j]=link; + } /* end for */ + + /* Write the data to the dataset using default transfer properties. */ + H5Dwrite(dataset, H5T_POINTER_OBJECT, H5S_ALL, H5S_ALL, H5P_DEFAULT, data); + + /* Close everything */ + H5Sclose(dataspace); + H5Tclose(datatype); + H5Dclose(dataset1); + H5Fclose(file1); +} +</pre> + + +<b>Object Reference Reading Example</b> +<br> +Open a dataset which has links to other datasets as part of +its raw data and read in those links. +<p> + +<pre> +{ + hid_t file1; + hid_t dataset1, tmp_dset; + href_t data[10][10]; + int i, j; + + /* Open the file */ + file1=H5Fopen("example.h5", H5F_ACC_RDWR, H5P_DEFAULT); + + /* Open the dataset */ + dataset1=H5Dopen(file1,"Dataset One",H5P_DEFAULT); + + /* + * Read the data to the dataset using default transfer properties. + * (we are assuming the dataset is the same and not querying the + * dimensions, etc.) + */ + H5Dread(dataset, H5T_STD_REF_OBJ, H5S_ALL, H5S_ALL, H5P_DEFAULT, data); + + /* Analyze array of OIDs of linked datasets in the file */ + /* somewhat hokey and artificial, but demonstrates the point */ + for(i=0; i<10; i++) + for(j=0; j<10; i++) + { + if((tmp_dset=H5Rdereference(dataset, H5T_STD_REF_OBJ, data[i][j]))>0) + { + <perform operations on linked dataset> + } /* end if */ + H5Dclose(tmp_dset); + } /* end for */ + + + /* Close everything */ + H5Dclose(dataset1); + H5Fclose(file1); +} +</pre> + + +<b>Dataset Region Reference Writing Example</b> +<br> +Create a dataset which has links to other dataset regions +(single elements in this case) as part of its raw data and +write the dataset to the file. +<p> + +<pre> +{ + hid_t file1; + hid_t dataset1, dataset2; + hid_t datatype, dataspace1, dataspace2; + char buf[128]; + href_t link; + href_t data[10][10]; /* HDF5 reference type */ + int rank; + size_t dimsf[2]; + hssize_t start[3],count[3]; + int i, j; + + /* Open the file */ + file1=H5Fopen("example.h5", H5F_ACC_RDWR, H5P_DEFAULT); + + /* Describe the size of the array and create the data space */ + rank=2; + dimsf[0] = 10; + dimsf[1] = 10; + dataspace1 = H5Screate_simple(rank, dimsf, NULL); + + /* Define Dataset Region Reference datatype */ + datatype = H5Tcopy(H5T_STD_REF_DATAREG); + + /* Create a dataset */ + dataset1=H5Dcreate(file1,"Dataset One",datatype,dataspace1,H5P_DEFAULT); + + /* Construct array of OIDs for other datasets in the file */ + /* (somewhat artificial, but demonstrates the point) */ + for(i=0; i<10; i++) + for(j=0; j<10; i++) + { + sprintf(buf,"/Group/Linked Set %d-%d",i,j); + + /* Get the dataspace for the object to point to */ + dataset2=H5Dopen(file1,buf,H5P_DEFAULT); + dataspace2=H5Dget_space(dataspace2); + + /* Select the region to point to */ + /* (could be different region for each pointer) */ + start[0]=5; start[1]=4; start[2]=3; + count[0]=2; count[1]=4; count[2]=1; + H5Sselect_hyperslab(dataspace2,H5S_SELECT_SET,start,NULL,count,NULL); + + if(H5Rcreate(&link,file1,buf,H5R_REF_DATAREG,dataspace2)>0) + /* Store the reference */ + data[i][j]=link; + + H5Sclose(dataspace2); + H5Dclose(dataspace2); + } /* end for */ + + /* Write the data to the dataset using default transfer properties. */ + H5Dwrite(dataset, H5T_STD_REF_DATAREG, H5S_ALL, H5S_ALL, H5P_DEFAULT, data); + + /* Close everything */ + H5Sclose(dataspace); + H5Tclose(datatype); + H5Dclose(dataset1); + H5Fclose(file1); +} +</pre> + + +<b>Dataset Region Reference Reading Example</b> +<br> +Open a dataset which has links to other datasets regions +(single elements in this case) as part of its raw data and +read in those links. +<p> + +<pre> +{ + hid_t file1; + hid_t dataset1, tmp_dset; + hid_t dataspace; + href_t data[10][10]; /* HDF5 reference type */ + int i, j; + + /* Open the file */ + file1=H5Fopen("example.h5", H5F_ACC_RDWR, H5P_DEFAULT); + + /* Open the dataset */ + dataset1=H5Dopen(file1,"Dataset One",H5P_DEFAULT); + + /* + * Read the data to the dataset using default transfer properties. + * (we are assuming the dataset is the same and not querying the + * dimensions, etc.) + */ + H5Dread(dataset, H5T_STD_REF_DATAREG, H5S_ALL, H5S_ALL, H5P_DEFAULT, data); + + /* Analyze array of OIDs of linked datasets in the file */ + /* (somewhat artificial, but demonstrates the point) */ + for(i=0; i<10; i++) + for(j=0; j<10; i++) + { + if((tmp_dset=H5Rdereference(dataset, H5D_STD_REF_DATAREG,data[i][j]))>0) + { + /* Get the dataspace with the pointed to region selected */ + dataspace=H5Rget_space(data[i][j]); + + <perform operation on linked dataset region> + + H5Sclose(dataspace); + } /* end if */ + H5Dclose(tmp_dset); + } /* end for */ + + + /* Close everything */ + H5Dclose(dataset1); + H5Fclose(file1); +} +</pre> + + + +<hr> +<address> +<a href="mailto:hdfhelp@ncsa.uiuc.edu">HDF Help Desk</a> +</address> + +Last modified: 28 October 1998 + + +<h1>Material to Be Omitted!!!</h1> + +Additional material above will also need to be deleted or +commented out. +<p> + +<h2>"Kinds of Reference" Information</h2> + +<dl> + <dt>Dataset Offset Reference + <dd>Reference to a specific byte sequence in a dataset (3) + <dt>Disk Offset Reference + <dd>Reference to a specific byte sequence in a file (3) + <dt>External Object Reference + <dd>Reference to an entire object in another HDF5 file (3) + <dt>External Dataset Region Reference + <dd>Reference an a specific dataset region in another HDF5 file (3) + <dt>External Dataset Offset Reference + <dd>Reference to a specific byte sequence in a dataset in + another HDF5 file (3) + <dt>External Disk Offset Reference + <dd>Reference to a specific byte sequence in another HDF5 file (3) + <dt>Generic Reference + <dd>A reference which may be any of the types defined above. (3) +</dl> + +Notes: +<ul> + <li>1 - Planned for in final release. + <li>2 - Most useful reference types to tackle next + <li>3 - Optional reference types. +</ul> + +<h2>Comments</h2> +<pre> + Reference types are atomic types and may be included as fields in compound + data types. + + There are (at least) three levels of reference strength: + Weak - We allow the user to store any type of reference in an array + of references. (u.e., the array of references in the example above + could be a mix of Object, Dataset Region and Internal references) + Medium - We force the user to stick with a particular type of + reference within a dataset, but the datasets pointed to (with + Object and Dataset Region references) may be of any data type + or dataspace. + Strong - We force the user to stick with a particular type of + reference and Object and Dataset Region references must point to + datasets with the same data type. + Extra Strong - We force the user to stick with a particular type of + reference and Object and Dataset Region references must point to + datasets with the same data type _and_ dataspace. + + The library is currently implemented with "medium" strength references. +</pre> + + +<h2>Reference Type</h2> + +<center> +<table> +<tr><td><code>H5R_MAXTYPE</code></td> + <td align=right><code>3 </code></td> + <td>Highest type in group (invalid as true type)</td></tr> +</table> +</center> + + +<h2>Information Regarding Specific Kinds of References</h2> + +<dl> +<dt>Dataset Offset Reference + <dd>Points to a sequence of bytes within a dataset in + the current HDF5 file by storing the OID of the dataset and the byte + length and offset of the sequence within the dataset. The offset is + the logical byte offset within the dataset, meaning that the data is + de-compressed before returning the sequence of bytes requested. No + interpretation of the data at that location is provided. However, + if the dataset is extendible and the size of the dimensions are changed, + the element(s) that the sequence is located within may vary. Fixed size + in file. + +<dt>Disk Offset Reference + <dd>Points to a sequence of bytes in the current + HDF5 file by storing the byte length and offset of the sequence within + the file, relative to the boot-block (as are all the other high-level + addresses used in the file). The offset is the absolute byte offset + within the file, no interpretation of the data at that location is + provided. Fixed size in file. + +<dt>External Object Reference + <dd>Points to an entire object in another HDF5 file by + storing a global heap offset which points to the URL of the external + file and the OID of the object pointed to. Variable size in file. + +<dt>External Dataset Region Reference + <dd>Points to a region of a dataset in + another HDF5 file by storing a global heap offset which points to the + URL of the external file, OID of the dataset and the coordinates of the + region. Variable size in file. + +<dt>External Dataset Offset Reference + <dd>Points to a sequence of bytes within a + dataset in another HDF5 file by storing a global heap offset which + points to the URL of the external file, the OID of the dataset and the + byte length and offset of the sequence within the dataset. The offset + is the logical byte offset within the dataset, meaning that the data is + de-compressed before returning the sequence of bytes requested. + However, if the dataset is not stored contiguously and the size of the + dimensions are changed, the element(s) that the sequence is located + within may vary. Variable size in file. + +<dt>External Disk Offset Reference + <dd>Points to a sequence of bytes in another + HDF5 file by storing a global heap reference which points to the URL + of the external file and the byte length and offset of the sequence + within the file. The offset is the absolute byte offset within the + file, no interpretation of the data at that location is provided. + Variable size in file. + +<dt>Generic Reference + <dd>A reference which may contain any of the other + references defined above. (Mostly useful for implementing "weak" + strength pointers within the medium strength model we are using) + Variable size in file. + +</dl> + +<h2>Implementation Details</h2> + +<dl> +<dt>File Storage + <dd>In order to efficiently index an array, each element must be the + same size when stored in the dataset on disk. Fixed-sized references + will be stored directly in the dataset array on disk; variable-sized + references will have a fixed-size head offset stored in the array on + disk with a file heap used to store the actual variable-sized + information stored in the heap. + +<dt>Memory Storage + <dd>Each <code>href_t</code> object in memory is a struct containing + a pointer type and union of information required for each + pointer type. + Information in this structure is not designed for users to view. + Non-C APIs may have to mangle this structure in some way, in order + to provide users with access to references in a language-appropriate + way. +</dl> + </body> +</html> |