summaryrefslogtreecommitdiffstats
path: root/doxygen/dox/LearnBasics1.dox
blob: a9b6d0e71fdebddf2e17631a265a4fc783c208ad (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
/** @page LBFileOrg HDF5 File Organization

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics
<hr>

\section secLBFileOrg HDF5 file
An HDF5 file is a container for storing a variety of scientific data and is composed of two primary types of objects: groups and datasets.

\li HDF5 group: a grouping structure containing zero or more HDF5 objects, together with supporting metadata
\li HDF5 dataset: a multidimensional array of data elements, together with supporting metadata

Any HDF5 group or dataset may have an associated attribute list. An HDF5 attribute is a user-defined HDF5 structure
that provides extra information about an HDF5 object.

Working with groups and datasets is similar in many ways to working with directories and files in UNIX. As with UNIX
directories and files, an HDF5 object in an HDF5 file is often referred to by its full path name (also called an absolute path name).

\li <code style="background-color:whitesmoke;">/</code>   signifies the root group.

\li <code style="background-color:whitesmoke;">/foo</code>  signifies a member of the root group called foo.

\li <code style="background-color:whitesmoke;">/foo/zoo</code>  signifies a member of the group foo, which in turn is a member of the root group.

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics

@page LBAPI The HDF5 API

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics
<hr>

\section secLBAPI HDF5 C API
The HDF5 library provides several interfaces, or APIs. These APIs provide routines for creating,
accessing, and manipulating HDF5 files and objects.

The library itself is implemented in C. To facilitate the work of FORTRAN 90, C++ and Java programmers,
HDF5 function wrappers have been developed in each of these languages. This tutorial discusses the use
of the C functions and the FORTRAN wrappers.

All C routines in the HDF5 library begin with a prefix of the form H5*, where * is one or two uppercase
letters indicating the type of object on which the function operates.
The FORTRAN wrappers come in the form of subroutines that begin with h5 and end with _f.
Java routine names begin with “H5*” and are prefixed with “H5.” as the class. Constants are
in the HDF5Constants class and are prefixed with "HDF5Constants.".
The APIs are listed below:
<table>
<tr>
<th><strong>API</strong>
</th>
<th><strong>DESCRIPTION</strong>
</th>
</tr>
<tr>
<th><strong>H5</strong>
</th>
<td>Library Functions: general-purpose H5 functions
</td>
</tr>
<tr>
<th><strong>H5A</strong>
</th>
<td>Annotation Interface: attribute access and manipulation routines
</td>
</tr>
<tr>
<th><strong>H5D</strong>
</th>
<td>Dataset Interface: dataset access and manipulation routines
</td>
</tr>
<tr>
<th><strong>H5E</strong>
</th>
<td>Error Interface: error handling routines
</td>
</tr>
<tr>
<th><strong>H5F</strong>
</th>
<td>File Interface: file access routines
</td>
</tr>
<tr>
<th><strong>H5G</strong>
</th>
<td>Group Interface: group creation and operation routines
</td>
</tr>
<tr>
<th><strong>H5I</strong>
</th>
<td>Identifier Interface: identifier routines
</td>
</tr>
<tr>
<th><strong>H5L</strong>
</th>
<td>Link Interface: link routines
</td>
</tr>
<tr>
<th><strong>H5O</strong>
</th>
<td>Object Interface: object routines
</td>
</tr>
<tr>
<th><strong>H5P</strong>
</th>
<td>Property List Interface: object property list manipulation routines
</td>
</tr>
<tr>
<th><strong>H5R</strong>
</th>
<td>Reference Interface: reference routines
</td>
</tr>
<tr>
<th><strong>H5S</strong>
</th>
<td>Dataspace Interface: dataspace definition and access routines
</td>
</tr>
<tr>
<th><strong>H5T</strong>
</th>
<td>Datatype Interface: datatype creation and manipulation routines
</td>
</tr>
<tr>
<th><strong>H5Z</strong>
</th>
<td>Compression Interface: compression routine(s)
</td>
</tr>
</table>

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics

@page LBProg Programming Issues

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics
<hr>

Keep the following in mind when looking at the example programs included in this tutorial:

\section LBProgAPI APIs vary with different languages
\li C routines begin with the prefix “H5*” where * is a single letter indicating the object on which the operation is to be performed:
<table>
<tr>
<td>File Interface: </td>
<td>#H5Fopen</td>
</tr><tr>
<td>Dataset Interface:</td>
<td>#H5Dopen</td>
</tr>
</table>

\li FORTRAN routines begin with “h5*” and end with “_f”:
<table>
<tr>
<td>File Interface: </td>
<td>h5fopen_f</td>
</tr><tr>
<td>Dataset Interface:</td>
<td>h5dopen_f</td>
</tr>
</table>

\li Java routine names begin with “H5*” and are prefixed with “H5.” as the class. Constants are
in the HDF5Constants class and are prefixed with "HDF5Constants.".:
<table>
<tr>
<td>File Interface: </td>
<td>H5.H5Fopen</td>
</tr><tr>
<td>Dataset Interface:</td>
<td>H5.H5Dopen</td>
</tr>
</table>

\li APIS for languages like C++, Java, and Python use methods associated with specific objects.

\section LBProgTypes HDF5 library has its own defined types
\li #hid_t is used for object handles
\li hsize_t is used for dimensions
\li #herr_t is used for many return values

\section LBProgLang Language specific files must be included in applications
<ul>
<li>
Python:   Add <code>"import h5py / import numpy"</code>
</li>
<li>
C:        Add <code>"#include hdf5.h"</code>
</li>
<li>
FORTRAN:  Add <code>"USE HDF5"</code> and call h5open_f and h5close_f to initialize and close the HDF5 FORTRAN interface
</li>
<li>
Java:     Add <code>"import hdf.hdf5lib.H5;
                     import hdf.hdf5lib.HDF5Constants;"</code>
</li>
</ul>

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics

@page LBFileCreate Creating an HDF5 File

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics
<hr>

An HDF5 file is a binary file containing scientific data and supporting metadata.
\section secLBFileCreate HDF5 File Access
To create an HDF5 file, an application must specify not only a file name, but a file access mode,
a file creation property list, and a file access property list. These terms are described below:
<ul>
<li><strong>File access mode:</strong><br />
When creating a file, the file access mode specifies the action to take if the file already exists:
<ul>
<li>#H5F_ACC_TRUNC specifies that if the file already exists, the current contents will be deleted so
that the application can rewrite the file with new data.
</li>
<li>#H5F_ACC_EXCL specifies that the open will fail if the file already exists. If the file does not
already exist, the file access parameter is ignored.
</li>
</ul>
In either case, the application has both read and write access to the successfully created file.
<br />
Note that there are two different access modes for opening existing files:
<ul>
<li>#H5F_ACC_RDONLY specifies that the application has read access but will not be allowed to write any data.
</li>
<li>#H5F_ACC_RDWR specifies that the application has read and write access.
</li>
</ul>
</li>
<li><strong>File creation property list:</strong><br />The file creation property list is used to
control the file metadata. File metadata contains information about the size of the user-block*,
the size of various file data structures used by the HDF5 library, etc. In this tutorial, the
default file creation property list, #H5P_DEFAULT, is used.<br />
 *The user-block is a fixed-length block of data located at the beginning of the file which is
ignored by the HDF5 library. The user-block may be used to store any data or information found
to be useful to applications.
</li>
<li><strong>File access property list:</strong><br />The file access property list is used to
control different methods of performing I/O on files. It also can be used to control how a file
is closed (whether or not to delay the actual file close until all objects in a file are closed).
The default file access property list, #H5P_DEFAULT, is used in this tutorial.
</li>
</ul>

Please refer to the \ref sec_file section of the \ref UG and \ref H5F section in the \ref RM for
detailed information regarding file access/creation property lists and access modes.

The steps to create and close an HDF5 file are as follows:
<ol>
<li>Specify the file creation and access property lists, if necessary.</li>
<li>Create the file.</li>
<li>Close the file, and if necessary, close the property lists.</li>
</ol>

\section secLBFileExample Programming Example

\subsection subsecLBFileExampleDesc Description
The following example code demonstrates how to create and close an HDF5 file.

<em>C</em>
\code
#include "hdf5.h"
   #define FILE "file.h5"

   int main() {

      hid_t       file_id;   /* file identifier */
      herr_t      status;

      /* Create a new file using default properties. */
      file_id = H5Fcreate(FILE, H5F_ACC_TRUNC, H5P_DEFAULT, H5P_DEFAULT);

      /* Terminate access to the file. */
      status = H5Fclose(file_id); 
   }
\endcode

<em>Fortran</em>
\code
  PROGRAM FILEEXAMPLE

     USE HDF5 ! This module contains all necessary modules 
        
     IMPLICIT NONE

     CHARACTER(LEN=8), PARAMETER :: filename = "filef.h5" ! File name
     INTEGER(HID_T) :: file_id                            ! File identifier
 
     INTEGER     ::   error  ! Error flag
     
!
!    Initialize FORTRAN interface.
!
     CALL h5open_f (error)
     !
     ! Create a new file using default properties.
     ! 
     CALL h5fcreate_f(filename, H5F_ACC_TRUNC_F, file_id, error)

     !
     ! Terminate access to the file.
     !
     CALL h5fclose_f(file_id, error)
!
!    Close FORTRAN interface.
!
     CALL h5close_f(error)
     END PROGRAM FILEEXAMPLE
\endcode

See \ref LBExamples for the examples used in the Learning the Basics tutorial.

For details on compiling an HDF5 application:
[ \ref LBCompiling ]

\subsection subsecLBFileExampleRem Remarks
\li <strong>In C</strong>: The include file <code style="background-color:whitesmoke;">hdf5.h</code> contains definitions and declarations and must be included
in any program that uses the HDF5 library.
<br />
<strong>In FORTRAN</strong>: The module <code style="background-color:whitesmoke;">HDF5</code> contains definitions and declarations and must be used in any
program that uses the HDF5 library. Also note that #H5open MUST be called at the beginning of an HDF5 Fortran
application (prior to any HDF5 calls) to initialize the library and variables. The #H5close call MUST be at
the end of the HDF5 Fortran application.
\li #H5Fcreate creates an HDF5 file and returns the file identifier.<br />
For Fortran, the file creation property list and file access property list are optional. They can be omitted if the
default values are to be used.<br />
The root group is automatically created when a file is created. Every file has a root group and the path name of
the root group is always <code style="background-color:whitesmoke;">/</code>.
\li #H5Fclose terminates access to an HDF5 file.<br />
When an HDF5 file is no longer accessed by a program, #H5Fclose must be called to release the resources used by the file.
This call is mandatory.<br />
Note that if #H5Fclose is called for a file, but one or more objects within the file remain open, those objects will
remain accessible until they are individually closed. This can cause access problems for other users, if objects were
inadvertently left open. A File Access property controls how the file is closed.

\subsection subsecLBFileExampleCont File Contents
The HDF Group has developed tools for examining the contents of HDF5 files. The tool used throughout the HDF5 tutorial
is the HDF5 dumper, <code style="background-color:whitesmoke;">h5dump</code>, which displays the file contents in human-readable form. The output of <code style="background-color:whitesmoke;">h5dump</code> is an ASCII
display formatted according to the HDF5 DDL grammar. This grammar is defined, using Backus-Naur Form, in the
\ref DDLBNF110.

To view the HDF5 file contents, simply type:
\code
h5dump <filename>
\endcode

<table>
<caption>Describe the file contents of file.h5 using a directed graph.</caption>
<tr>
<td>
\image html imgLBFile.gif
</td>
</tr>
</table>

The text description of <code style="background-color:whitesmoke;">file.h5</code>, as generated by <code style="background-color:whitesmoke;">h5dump</code>. The HDF5 file called <code style="background-color:whitesmoke;">file.h5</code>
contains a group called <code style="background-color:whitesmoke;">/</code>, or the root group. (The file called <code style="background-color:whitesmoke;">filef.h5</code>, created by the FORTRAN version of the example,
has the same output except that the filename shown is <code style="background-color:whitesmoke;">filef.h5</code>.)
\code
HDF5 "file.h5" {
         GROUP "/" {
         }
         }
\endcode

\subsection subsecLBFileExampleDDL File Definition in DDL
The simplified DDL file definition for creating an HDF5 file. For simplicity, a simplified DDL is used in this tutorial. A
complete and more rigorous DDL can be found in the \ref DDLBNF110.

The following symbol definitions are used in the DDL:
\code
        ::=               defined as
        <tname>           a token with the name tname
        <a> | <b>         one of <a> or <b>
        <a>*              zero or more occurrences of <a>
\endcode

The simplified DDL for file definition is as follows:
\code
        <file> ::= HDF5 "<file_name>" { <root_group> }

        <root_group> ::= GROUP "/" { <group_attribute>* 
                                        <group_member>* }

        <group_attribute> ::= <attribute>

        <group_member> ::= <group> | <dataset>
\endcode

Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics

@page LBDsetCreate Creating a Dataset
Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics
<hr>

A dataset is a multidimensional array of data elements, together with supporting metadata. To create
a dataset, the application program must specify the location at which to create the dataset, the
dataset name, the datatype and dataspace of the data array, and the property lists.

\section secLBDsetCreateDtype Datatypes
A datatype is a collection of properties, all of which can be stored on disk, and which, when taken as
a whole, provide complete information for data conversion to or from that datatype.

There are two categories of datatypes in HDF5:
<ul>
<li><strong>Pre-defined</strong>:   These datatypes are opened and closed by HDF5.<br />
Pre-defined datatypes can be atomic or composite:
<ul><li>Atomic datatypes cannot be decomposed into smaller datatype units at the API level. For example: integer, float, reference, string.</li>
<li>Composite datatypes are aggregations of one or more datatypes. For example: array, variable length, enumeration, compound.</li></ul>
</li>
<li><strong>Derived</strong>:   These datatypes are created or derived from the pre-defined types.<br />
A simple example of creating a derived datatype is using the string datatype, H5T_C_S1, to create strings of more than one character:<br />
\code
      hid_t strtype;                     // Datatype ID
      herr_t status;

      strtype = H5Tcopy (H5T_C_S1);
      status = H5Tset_size (strtype, 5); // create string of length 5
\endcode
</li>
</ul>

Shown below is the HDF5 pre-defined datatypes.
\code
                                          +--  integer
                                          +--  floating point
                        +---- atomic  ----+--  date and time
                        |                 +--  character string
       HDF5 datatypes --|                 +--  bitfield
                        |                 +--  opaque
                        |
                        +---- compound
\endcode

Some of the HDF5 predefined atomic datatypes are listed below.

<table>
<caption>Examples of HDF5 predefined datatypes</caption>
<tr>
<th><strong>Datatype</strong></th>
<th><strong>Description</strong></th>
</tr>
<tr>
<th><strong>H5T_STD_I32LE</strong></th>
<td>Four-byte, little-endian, signed, two's complement integer</td>
</tr>
<tr>
<th><strong>H5T_STD_U16BE</strong></th>
<td>Two-byte, big-endian, unsigned integer</td>
</tr>
<tr>
<th><strong>H5T_IEEE_F32BE</strong></th>
<td>Four-byte, big-endian, IEEE floating point</td>
</tr>
<tr>
<th><strong>H5T_IEEE_F64LE</strong></th>
<td>Eight-byte, little-endian, IEEE floating point</td>
</tr>
<tr>
<th><strong>H5T_C_S1</strong></th>
<td>One-byte, null-terminated string of eight-bit characters</td>
</tr>
</table>

<table>
<caption>Examples of HDF5 predefined native datatypes</caption>
<tr>
<th><strong>Native Datatype</strong></th>
<th><strong>Corresponding C or FORTRAN Type</strong></th>
</tr>
<tr>
<th span="2"><strong>C</strong></th>
</tr>
<tr>
<th><strong>H5T_NATIVE_INT</strong></th>
<td>int</td>
</tr>
<tr>
<th><strong>H5T_NATIVE_FLOAT</strong></th>
<td>float</td>
</tr>
<tr>
<th><strong>H5T_NATIVE_CHAR</strong></th>
<td>char</td>
</tr>
<tr>
<th><strong>H5T_NATIVE_DOUBLE</strong></th>
<td>double</td>
</tr>
<tr>
<th><strong>H5T_NATIVE_LDOUBLE</strong></th>
<td>long double</td>
</tr>
<tr>
<th span="2"><strong>Fortran</strong></th>
</tr>
<tr>
<th><strong>H5T_NATIVE_INTEGER</strong></th>
<td>integer</td>
</tr>
<tr>
<th><strong>H5T_NATIVE_REAL</strong></th>
<td>real</td>
</tr>
<tr>
<th><strong>H5T_NATIVE_DOUBLE</strong></th>
<td>double precision</td>
</tr>
<tr>
<th><strong>H5T_NATIVE_CHARACTER</strong></th>
<td>character</td>
</tr>
</table>

In this tutorial, we consider only HDF5 predefined integers.

For further information on datatypes, see \ref sec_datatype in the \ref UG, in addition to the \ref LBDatatypes tutorial topic.

\section secLBDsetCreateDspace Datasets and Dataspaces
A dataspace describes the dimensionality of the data array. A dataspace is either a regular N-dimensional
array of data points, called a simple dataspace, or a more general collection of data points organized
in another manner, called a complex dataspace. In this tutorial, we only consider simple dataspaces.

<em>HDF5 dataspaces</em>
\code
                         +-- simple
       HDF5 dataspaces --|
                         +-- complex
\endcode
The dimensions of a dataset can be fixed (unchanging), or they may be unlimited, which means that they are
extensible. A dataspace can also describe a portion of a dataset, making it possible to do partial
I/O operations on selections.

\section secLBDsetCreateProp Property Lists
Property lists are a mechanism for modifying the default behavior when creating or accessing objects. For
more information on property lists see the \ref LBPropsList tutorial topic.

The following property lists can be specified when creating a dataset:
\li Dataset Creation Property List<br />
When creating a dataset, HDF5 allows the user to specify how raw data is organized and/or compressed on
disk. This information is stored in a dataset creation property list and passed to the dataset interface.
The raw data on disk can be stored contiguously (in the same linear way that it is organized in memory),
partitioned into chunks, stored externally, etc. In this tutorial, we use the default dataset creation
property list (contiguous storage layout and no compression). For more information about dataset creation
property lists, see \ref sec_dataset in the \ref UG.
\li Link Creation Property List<br />
The link creation property list governs creation of the link(s) by which a new dataset is accessed and the
creation of any intermediate groups that may be missing.
\li Dataset Access Property List<br />
Dataset access property lists are properties that can be specified when accessing a dataset.

\section secLBDsetCreateSteps Steps to Create a Dataset
To create an empty dataset (no data written) the following steps need to be taken:
<ol>
<li>Obtain the location identifier where the dataset is to be created.</li>
<li>Define or specify the dataset characteristics:
<ol>
<li>Define a datatype or specify a pre-defined datatype.</li>
<li>Define a dataspace.</li>
<li>Specify the property list(s) or use the default.</li>
</ol></li>
<li>Create the dataset.</li>
<li>Close the datatype, the dataspace, and the property list(s) if necessary.</li>
<li>Close the dataset.</li>
</ol>
In HDF5, datatypes and dataspaces are independent objects which are created separately from any dataset
that they might be attached to. Because of this, the creation of a dataset requires the definition of
the datatype and dataspace. In this tutorial, we use the HDF5 predefined datatypes (integer) and consider
only simple dataspaces. Hence, only the creation of dataspace objects is needed.

\section secLBDsetCreateHL High Level APIs
The High Level \ref H5LT include functions that simplify and condense the steps for
creating datasets in HDF5. The examples in the following section use the standard APIs. For a
quick start you may prefer to look at the \ref H5LT at this time.

If you plan to work with images, please look at the High Level \ref H5IM, as well.

\section secLBDsetCreateProg Programming Example

\subsection subsecLBDsetCreateProgDesc Description
See \ref LBExamples for the examples used in the \ref LearnBasics tutorial.

The example shows how to create an empty dataset. It creates a file called <code style="background-color:whitesmoke;">dset.h5</code>
in the C version (<code style="background-color:whitesmoke;">dsetf.h5</code> in Fortran), defines the dataset dataspace, creates a
dataset which is a 4x6 integer array, and then closes the dataspace, the dataset, and the file.

For details on compiling an HDF5 application: [ \ref LBCompiling ]

\subsection subsecLBDsetCreateProgRem Remarks
#H5Screate_simple creates a new simple dataspace and returns a dataspace identifier.
#H5Sclose releases and terminates access to a dataspace.

<em>C</em>
\code
    dataspace_id = H5Screate_simple (rank, dims, maxdims);
    status = H5Sclose (dataspace_id );
\endcode

<em>FORTRAN</em>
\code
    CALL h5screate_simple_f (rank, dims, dataspace_id, hdferr, maxdims=max_dims)
         or
    CALL h5screate_simple_f (rank, dims, dataspace_id, hdferr)

    CALL h5sclose_f (dataspace_id, hdferr)
\endcode

#H5Dcreate creates an empty dataset at the specified location and returns a dataset identifier.
#H5Dclose closes the dataset and releases the resource used by the dataset. This call is mandatory.

<em>C</em>
\code
   dataset_id = H5Dcreate(file_id, "/dset", H5T_STD_I32BE, dataspace_id, H5P_DEFAULT, H5P_DEFAULT, H5P_DEFAULT);
   status = H5Dclose (dataset_id);
\endcode

<em>FORTRAN</em>
\code
   CALL h5dcreate_f (loc_id, name, type_id, dataspace_id, dset_id, hdferr)
   CALL h5dclose_f (dset_id, hdferr)
\endcode

Note that if using the pre-defined datatypes in FORTRAN, then a call must be made to initialize and terminate access to the pre-defined datatypes:
\code
  CALL h5open_f (hdferr)
  CALL h5close_f (hdferr)
\endcode

H5open must be called before any HDF5 library subroutine calls are made;
H5close must be called after the final HDF5 library subroutine call.

See the programming example for an illustration of the use of these calls.

\subsection subsecLBDsetCreateContent File Contents
The contents of the file dset.h5 (dsetf.h5 for FORTRAN) are shown below:
<table>
<caption>Contents of dset.h5 ( dsetf.h5)</caption>
<tr>
<td>
\image html imgLBDsetCreate.gif
</td>
</tr>
</table>
<table>
<tr>
<th>dset.h5 in DDL</th>
<th>dsetf.h5 in DDL</th>
<tr>
<td>
\code
HDF5 "dset.h5" {
GROUP "/" {
   DATASET "dset" {
      DATATYPE { H5T_STD_I32BE }
      DATASPACE { SIMPLE ( 4, 6 ) / ( 4, 6 ) }
      DATA {
         0, 0, 0, 0, 0, 0,
         0, 0, 0, 0, 0, 0,
         0, 0, 0, 0, 0, 0,
         0, 0, 0, 0, 0, 0
      }
   }
}
}
\endcode
</td>
<td>
\code
HDF5 "dsetf.h5" {
GROUP "/" {
   DATASET "dset" {
      DATATYPE { H5T_STD_I32BE }
      DATASPACE { SIMPLE ( 6, 4 ) / ( 6, 4 ) }
      DATA {
         0, 0, 0, 0,
         0, 0, 0, 0,
         0, 0, 0, 0,
         0, 0, 0, 0,
         0, 0, 0, 0,
         0, 0, 0, 0
      }
   }
}
}
\endcode
</td>
</tr>
</table>
Note in above that #H5T_STD_I32BE, a 32-bit Big Endian integer, is an HDF atomic datatype.

\subsection subsecLBDsetCreateProgDDL Dataset Definition in DDL
The following is the simplified DDL dataset definition:
\code
      <dataset> ::= DATASET "<dataset_name>" { <datatype>
                                               <dataspace>
                                               <data>
                                               <dataset_attribute>* }

      <datatype> ::= DATATYPE { <atomic_type> }

      <dataspace> ::= DATASPACE { SIMPLE <current_dims> / <max_dims> }

      <dataset_attribute> ::= <attribute>
\endcode

<hr>
Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics

@page LBDsetRW Reading From and Writing To a Dataset
Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics
<hr>

\section secLBDsetRW Dataset I/O Operation
During a dataset I/O operation, the library transfers raw data between memory and the file. The data in memory
can have a datatype different from that of the file and can also be of a different size (i.e., the data in
memory is a subset of the dataset elements, or vice versa). Therefore, to perform read or write operations,
the application program must specify:
\li The dataset
\li The dataset's datatype in memory
\li The dataset's dataspace in memory
\li The dataset's dataspace in the file
\li The dataset transfer property list<br />
<ul>
<li>(The dataset transfer property list controls various aspects of the I/O operations, such as the number
of processes participating in a collective I/O request or hints to the library to control caching of raw
data. In this tutorial, we use the default dataset transfer property list.)</li>
</ul>
\li The data buffer

The steps to read from or write to a dataset are as follows:
<ol>
<li>Obtain the dataset identifier.</li>
<li>Specify the memory datatype.</li>
<li>Specify the memory dataspace.</li>
<li>Specify the file dataspace.</li>
<li>Specify the transfer properties.</li>
<li>Perform the desired operation on the dataset.</li>
<li>Close the dataset.</li>
<li>Close the dataspace, datatype, and property list if necessary.</li>
</ol>

To read from or write to a dataset, the #H5Dread and #H5Dwrite routines are used.

<em>C</em>
\code
   status = H5Dread (set_id, mem_type_id, mem_space_id, file_space_id, xfer_prp, buf );
   status = H5Dwrite (set_id, mem_type_id, mem_space_id, file_space_id, xfer_prp, buf);
\endcode

<em>Fortran</em>
\code
   CALL h5dread_f(dset_id, mem_type_id, buf, dims, error, &
                     mem_space_id=mspace_id, file_space_id=fspace_id, &
                     xfer_prp=xfer_plist_id)
        or
   CALL h5dread_f(dset_id, mem_type_id, buf, dims,  error)


   CALL h5dwrite_f(dset_id, mem_type_id, buf, dims, error, &
                     mem_space_id=mspace_id, file_space_id=fspace_id, &
                     xfer_prp=xfer_plist_id)
        or
   CALL h5dwrite_f(dset_id, mem_type_id, buf, dims, error)
\endcode

\section secLBDsetRWHL High Level APIs
The High Level \ref H5LT include functions that simplify and condense the steps for creating and
reading datasets. Please be sure to review them, in addition to this tutorial.

\section secLBDsetRWEx Programming Example

\subsection secLBDsetRWExDesc Description
See \ref LBExamples for the examples used in the \ref LearnBasics tutorial.

The example shows how to read and write an existing dataset. It opens the file created in the previous example,
obtains the dataset identifier for the dataset <code style="background-color:whitesmoke;">/dset</code>, writes the dataset to the file, then reads
the dataset back. It then closes the dataset and file.

Note that #H5S_ALL is passed in for both the memory and file dataspace parameters in the read and write calls.
This indicates that the entire dataspace of the dataset will be read or written to. #H5S_ALL by itself does not
necessarily have this meaning. See the \ref RM entry for #H5Dread or #H5Dwrite for more information on using #H5S_ALL.

For details on compiling an HDF5 application:
[ \ref LBCompiling ]

\subsection secLBDsetRWExRem Remarks
#H5Fopen opens an existing file and returns a file identifier.

#H5Dopen opens an existing dataset with the specified name and location.

#H5Dwrite writes raw data from an application buffer to the specified dataset, converting from the datatype and
dataspace of the dataset in memory to the datatype and dataspace of the dataset in the file. Specifying #H5S_ALL
for both the memory and file dataspaces indicates that the entire dataspace of the dataset is to be written to.
#H5S_ALL by itself does not necessarily have this meaning. See the \ref RM entry for #H5Dwrite for more information
on using #H5S_ALL.

#H5Dread reads raw data from the specified dataset to an application buffer, converting from the file datatype and
dataspace to the memory datatype and dataspace. Specifying #H5S_ALL for both the memory and file dataspaces
indicates that the entire dataspace of the dataset is to be read. #H5S_ALL by itself does not necessarily have
this meaning. See the \ref RM entry for #H5Dread for more information on using #H5S_ALL.

\subsection secLBDsetRWExCont File Contents

Shown below is the contents of dset.h5 (created by the C program).

<em>dset.h5 in DDL</em>
\code
      HDF5 "dset.h5" {
      GROUP "/" {
         DATASET "dset" {
            DATATYPE { H5T_STD_I32BE }
            DATASPACE { SIMPLE ( 4, 6 ) / ( 4, 6 ) }
            DATA {
               1, 2, 3, 4, 5, 6,
               7, 8, 9, 10, 11, 12,
               13, 14, 15, 16, 17, 18,
               19, 20, 21, 22, 23, 24
            }
         }
      }
      }
\endcode

Shown below is the contents of dsetf.h5 (created by the FORTRAN program).

<em>dsetf.h5 in DDL</em>
\code
      HDF5 "dsetf.h5" {
      GROUP "/" {
         DATASET "dset" {
            DATATYPE { H5T_STD_I32BE }
            DATASPACE { SIMPLE ( 6, 4 ) / ( 6, 4 ) }
                  DATA {
               1, 7, 13, 19,
               2, 8, 14, 20,
               3, 9, 15, 21,
               4, 10, 16, 22,
               5, 11, 17, 23,
               6, 12, 18, 24
            }
         }
      }
      }
\endcode

<hr>
Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics

@page LBAttrCreate Creating an Attribute
Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics
<hr>

Attributes are small datasets that can be used to describe the nature and/or the intended usage of
the object they are attached to. In this section, we show how to create, read, and write an attribute.

\section secLBAttrCreate Creating an attribute
Creating an attribute is similar to creating a dataset. To create an attribute, the application must
specify the object which the attribute is attached to, the datatype and dataspace of the attribute
data, and the attribute creation property list.

The steps to create an attribute are as follows:
<ol>
<li>Obtain the object identifier that the attribute is to be attached to.</li>
<li>Define the characteristics of the attribute and specify the attribute creation property list.
<ul>
<li>Define the datatype.</li>
<li>Define the dataspace.</li>
<li>Specify the attribute creation property list.</li>
</ul></li>
<li>Create the attribute.</li>
<li>Close the attribute and datatype, dataspace, and attribute creation property list, if necessary.</li>
</ol>

To create and close an attribute, the calling program must use #H5Acreate and #H5Aclose. For example:

<em>C</em>
\code
  attr_id = H5Acreate (dataset_id, "Units", H5T_STD_I32BE, dataspace_id, H5P_DEFAULT, H5P_DEFAULT)
  status = H5Aclose (attr_id);
\endcode

<em>Fortran</em>
\code
  CALL h5acreate_f (dset_id, attr_nam, type_id, space_id, attr_id, &
                    hdferr, creation_prp=creat_plist_id)
  or
  CALL h5acreate_f (dset_id, attr_nam, type_id, space_id, attr_id, hdferr)

  CALL h5aclose_f (attr_id, hdferr)
\endcode

\section secLBAttrCreateRW Reading/Writing an attribute
Attributes may only be read or written as an entire object; no partial I/O is supported. Therefore,
to perform I/O operations on an attribute, the application needs only to specify the attribute and
the attribute's memory datatype.

The steps to read or write an attribute are as follows.
<ol>
<li>Obtain the attribute identifier.</li>
<li>Specify the attribute's memory datatype.</li>
<li>Perform the desired operation.</li>
<li>Close the memory datatype if necessary.</li>
</ol>

To read and/or write an attribute, the calling program must contain the #H5Aread and/or
#H5Awrite routines. For example:

<em>C</em>
\code
  status = H5Aread (attr_id, mem_type_id, buf);
  status = H5Awrite (attr_id, mem_type_id, buf);
\endcode

<em>Fortran</em>
\code
  CALL h5awrite_f (attr_id, mem_type_id, buf, dims, hdferr)  
  CALL h5aread_f (attr_id, mem_type_id, buf, dims, hdferr)
\endcode

\section secLBAttrCreateHL High Level APIs
The High Level \ref H5LT include functions that simplify and condense the steps for creating and
reading datasets. Please be sure to review them, in addition to this tutorial.

\section secLBAttrCreateRWEx Programming Example

\subsection secLBAttrCreateRWExDesc Description
See \ref LBExamples for the examples used in the \ref LearnBasics tutorial.

The example shows how to create and write a dataset attribute. It opens an existing file <code style="background-color:whitesmoke;">dset.h5</code>
in C (<code style="background-color:whitesmoke;">dsetf.h5</code> in FORTRAN), obtains the identifier of the dataset <code style="background-color:whitesmoke;">/dset</code>, defines
the attribute's dataspace, creates the dataset attribute, writes the attribute, and then closes the attribute's
dataspace, attribute, dataset, and file.

For details on compiling an HDF5 application:
[ \ref LBCompiling ]

\subsection secLBAttrCreateRWExRem Remarks
#H5Acreate creates an attribute which is attached to the object specified by the first parameter, and returns an identifier.

#H5Awrite writes the entire attribute, and returns the status of the write.

When an attribute is no longer accessed by a program, #H5Aclose must be called to release the attribute from use.
An #H5Aclose/h5aclose_f call is mandatory.

\subsection secLBAttrCreateRWExCont File Contents

Shown below is the contents and the attribute definition of <code style="background-color:whitesmoke;">dset.h5</code> (created by the C program).

<em>dset.h5 in DDL</em>
\code
HDF5 "dset.h5" {
GROUP "/" {
DATASET "dset" {
DATATYPE { H5T_STD_I32BE }
DATASPACE { SIMPLE ( 4, 6 ) / ( 4, 6 ) }
DATA {
 1, 2, 3, 4, 5, 6,
 7, 8, 9, 10, 11, 12,
 13, 14, 15, 16, 17, 18,
 19, 20, 21, 22, 23, 24
}
ATTRIBUTE "attr" {
 DATATYPE { H5T_STD_I32BE }
 DATASPACE { SIMPLE ( 2 ) / ( 2 ) }
 DATA {
    100, 200
 }
}
}
}
}
\endcode

Shown below is the contents and the attribute definition of <code style="background-color:whitesmoke;">dsetf.h5</code> (created by the FORTRAN program).

<em>dsetf.h5 in DDL</em>
\code
HDF5 "dsetf.h5" {
GROUP "/" {
DATASET "dset" {
DATATYPE { H5T_STD_I32BE }
DATASPACE { SIMPLE ( 6, 4 ) / ( 6, 4 ) }
DATA {
 1, 7, 13, 19,
 2, 8, 14, 20,
 3, 9, 15, 21,
 4, 10, 16, 22,
 5, 11, 17, 23,
 6, 12, 18, 24
}
ATTRIBUTE "attr" {
 DATATYPE { H5T_STD_I32BE }
 DATASPACE { SIMPLE ( 2 ) / ( 2 ) }
 DATA {
    100, 200
 }
}
}
}
}
\endcode

\subsection secLBAttrCreateRWExDDL Attribute Definition in DDL

<em>HDF5 Attribute Definition</em>
\code
<attribute> ::= ATTRIBUTE "<attr_name>" { <datatype>
                       <dataspace>
                       <data>  }
\endcode

<hr>
Navigate back: \ref index "Main" / \ref GettingStarted / \ref LearnBasics

*/