1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.10.0"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>HDF5: Files</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtreedata.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<script type="text/javascript" src="cookie.js"></script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/searchdata.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&dn=expat.txt MIT */
$(function() { init_search(); });
/* @license-end */
</script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
<link href="hdf5doxy.css" rel="stylesheet" type="text/css">
<!-- <link href="hdf5doxy.css" rel="stylesheet" type="text/css"/>
-->
<script type="text/javascript" src="hdf5_navtree_hacks.js"></script>
</head>
<body>
<div style="background:#FFDDDD;font-size:120%;text-align:center;margin:0;padding:5px">Please, help us to better serve our user community by answering the following short survey: <a href="https://www.hdfgroup.org/website-survey/">https://www.hdfgroup.org/website-survey/</a></div>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td id="projectlogo"><img alt="Logo" src="HDFG-logo.png"/></td>
<td id="projectalign" style="padding-left: 0.5em;">
<div id="projectname"><a href="https://www.hdfgroup.org">HDF5</a>
 <span id="projectnumber">1.15.0.2908dd1</span>
</div>
<div id="projectbrief">API Reference</div>
</td>
<td> <div id="MSearchBox" class="MSearchBoxInactive">
<span class="left">
<span id="MSearchSelect" onmouseover="return searchBox.OnSearchSelectShow()" onmouseout="return searchBox.OnSearchSelectHide()"> </span>
<input type="text" id="MSearchField" value="" placeholder="Search" accesskey="S"
onfocus="searchBox.OnSearchFieldFocus(true)"
onblur="searchBox.OnSearchFieldFocus(false)"
onkeyup="searchBox.OnSearchFieldChange(event)"/>
</span><span class="right">
<a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.svg" alt=""/></a>
</span>
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.10.0 -->
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&dn=expat.txt MIT */
var searchBox = new SearchBox("searchBox", "search/",'.html');
/* @license-end */
</script>
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:d3d9a9a6595521f9666a5e94cc830dab83b65699&dn=expat.txt MIT */
$(function(){initNavTree('_files.html',''); initResizable(); });
/* @license-end */
</script>
<div id="doc-content">
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
onmouseover="return searchBox.OnSearchSelectShow()"
onmouseout="return searchBox.OnSearchSelectHide()"
onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>
<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<div id="MSearchResults">
<div class="SRPage">
<div id="SRIndex">
<div id="SRResults"></div>
<div class="SRStatus" id="Loading">Loading...</div>
<div class="SRStatus" id="Searching">Searching...</div>
<div class="SRStatus" id="NoMatches">No Matches</div>
</div>
</div>
</div>
</div>
<div><div class="header">
<div class="headertitle"><div class="title">Files</div></div>
</div><!--header-->
<div class="contents">
<div class="textblock"><h1><a class="anchor" id="secFiles"></a>
Files</h1>
<h2><a class="anchor" id="CB_FreeSpace"></a>
Tracking Free Space in HDF5 Files</h2>
<dl class="section user"><dt>Problem</dt><dd>You sometimes delete objects in HDF5 files and don't have new content to use the free space, but would like to reuse it in the future.</dd></dl>
<dl class="section user"><dt>Solution</dt><dd>At file creation time, set the file space management strategy and persistence of free space tracking information via <a class="el" href="group___f_c_p_l.html#ga167ff65f392ca3b7f1933b1cee1b9f70" title="Sets the file space handling strategy and persisting free-space values for a file creation property l...">H5Pset_file_space_strategy()</a>.</dd></dl>
<dl class="section note"><dt>Note</dt><dd>This feature is only supported in HDF5 1.10.1+.</dd></dl>
<div class="fragment"><div class="line"><span class="lineno"> 15</span> {</div>
<div class="line"><span class="lineno"> 16</span> __label__ fail_fcpl, fail_fapl, fail_file;</div>
<div class="line"><span class="lineno"> 17</span> <a class="code hl_typedef" href="_h5_ipublic_8h.html#a0045db7ff9c22ad35db6ae91662e1943">hid_t</a> fcpl, fapl, file;</div>
<div class="line"><span class="lineno"> 18</span> </div>
<div class="line"><span class="lineno"> 19</span> <span class="keywordflow">if</span> ((fcpl = <a class="code hl_function" href="group___p_l_c_r.html#gaf1b11da01d4d45d788c45f8bc5f0cbfa">H5Pcreate</a>(<a class="code hl_define" href="_h5_ppublic_8h.html#a206f334f1e6c973e1215a3148b45b977">H5P_FILE_CREATE</a>)) < 0) {</div>
<div class="line"><span class="lineno"> 20</span> ret_val = EXIT_FAILURE;</div>
<div class="line"><span class="lineno"> 21</span> <span class="keywordflow">goto</span> fail_fcpl;</div>
<div class="line"><span class="lineno"> 22</span> }</div>
<div class="line"><span class="lineno"> 23</span><span class="preprocessor">#if H5_VERSION_GE(1, 10, 1)</span></div>
<div class="line"><span class="lineno"> 24</span> <span class="keywordflow">if</span> (<a class="code hl_function" href="group___f_c_p_l.html#ga167ff65f392ca3b7f1933b1cee1b9f70">H5Pset_file_space_strategy</a>(fcpl, <a class="code hl_enumvalue" href="_h5_fpublic_8h.html#a9cc492c4b5c936e48716a8dab3691bcca4eb2323fa7feed13452676d57cc27a87">H5F_FSPACE_STRATEGY_FSM_AGGR</a>, 1, 4096) < 0) {</div>
<div class="line"><span class="lineno"> 25</span><span class="preprocessor">#else</span></div>
<div class="line"><span class="lineno"> 26</span><span class="preprocessor">#error HDF5 1.10.1+ required</span></div>
<div class="line"><span class="lineno"> 27</span><span class="preprocessor">#endif</span></div>
<div class="line"><span class="lineno"> 28</span> ret_val = EXIT_FAILURE;</div>
<div class="line"><span class="lineno"> 29</span> <span class="keywordflow">goto</span> fail_fapl;</div>
<div class="line"><span class="lineno"> 30</span> }</div>
<div class="line"><span class="lineno"> 31</span> </div>
<div class="line"><span class="lineno"> 32</span> <span class="keywordflow">if</span> ((fapl = <a class="code hl_function" href="group___p_l_c_r.html#gaf1b11da01d4d45d788c45f8bc5f0cbfa">H5Pcreate</a>(<a class="code hl_define" href="_h5_ppublic_8h.html#a60ec2d4334addfc0eda89614598ee38e">H5P_FILE_ACCESS</a>)) < 0) {</div>
<div class="line"><span class="lineno"> 33</span> ret_val = EXIT_FAILURE;</div>
<div class="line"><span class="lineno"> 34</span> <span class="keywordflow">goto</span> fail_fapl;</div>
<div class="line"><span class="lineno"> 35</span> }</div>
<div class="line"><span class="lineno"> 36</span><span class="preprocessor">#if H5_VERSION_GE(1, 10, 1)</span></div>
<div class="line"><span class="lineno"> 37</span> <span class="keywordflow">if</span> (<a class="code hl_function" href="group___f_a_p_l.html#gacbe1724e7f70cd17ed687417a1d2a910">H5Pset_libver_bounds</a>(fapl, <a class="code hl_enumvalue" href="_h5_fpublic_8h.html#a2d963b599894f684571fbd4d5e8a96a2a33c6cdc401a3a32dbf63d74019fad4b3">H5F_LIBVER_V110</a>, <a class="code hl_define" href="_h5_fpublic_8h.html#a47d050173da48a2b85c730446f0771c0">H5F_LIBVER_LATEST</a>) < 0) {</div>
<div class="line"><span class="lineno"> 38</span><span class="preprocessor">#else</span></div>
<div class="line"><span class="lineno"> 39</span><span class="preprocessor">#error HDF5 1.10.x+ required</span></div>
<div class="line"><span class="lineno"> 40</span><span class="preprocessor">#endif</span></div>
<div class="line"><span class="lineno"> 41</span> ret_val = EXIT_FAILURE;</div>
<div class="line"><span class="lineno"> 42</span> <span class="keywordflow">goto</span> fail_file;</div>
<div class="line"><span class="lineno"> 43</span> }</div>
<div class="line"><span class="lineno"> 44</span> </div>
<div class="line"><span class="lineno"> 45</span> <span class="keywordflow">if</span> ((file = <a class="code hl_function" href="group___h5_f.html#gae64b51ee9ac0781bc4ccc599d98387f4">H5Fcreate</a>(<span class="stringliteral">"free_space.h5"</span>, <a class="code hl_define" href="_h5_fpublic_8h.html#a5a2d6726f9ad8d2bca8df2b817e5ad6a">H5F_ACC_TRUNC</a>, fcpl, fapl)) < 0) {</div>
<div class="line"><span class="lineno"> 46</span> ret_val = EXIT_FAILURE;</div>
<div class="line"><span class="lineno"> 47</span> <span class="keywordflow">goto</span> fail_file;</div>
<div class="line"><span class="lineno"> 48</span> }</div>
<div class="line"><span class="lineno"> 49</span> <a class="code hl_function" href="group___h5_f.html#gac55cd91d80822e4f8c2a7f04ea71b124">H5Fclose</a>(file);</div>
<div class="line"><span class="lineno"> 50</span> </div>
<div class="line"><span class="lineno"> 51</span>fail_file:</div>
<div class="line"><span class="lineno"> 52</span> <a class="code hl_function" href="group___p_l_c_r.html#ga5dce61149211d3ef319452aa598887fb">H5Pclose</a>(fapl);</div>
<div class="line"><span class="lineno"> 53</span>fail_fapl:</div>
<div class="line"><span class="lineno"> 54</span> <a class="code hl_function" href="group___p_l_c_r.html#ga5dce61149211d3ef319452aa598887fb">H5Pclose</a>(fcpl);</div>
<div class="line"><span class="lineno"> 55</span>fail_fcpl:;</div>
<div class="line"><span class="lineno"> 56</span> }</div>
</div><!-- fragment --><dl class="section user"><dt>Discussion</dt><dd>Free space tracking is supported only in HDF5 versions 1.10.x and higher. This has implications for the accessibility of your HDF5 files and should be considered carefully. If compatibility with previous versions of HDF5 must be maintained, space reclamation via <code>h5repack</code> might be an option.<br />
The file space strategy <a class="el" href="_h5_fpublic_8h.html#a9cc492c4b5c936e48716a8dab3691bcca4eb2323fa7feed13452676d57cc27a87">H5F_FSPACE_STRATEGY_FSM_AGGR</a> is not the only option that supports free-space tracking. <a class="el" href="_h5_fpublic_8h.html#a9cc492c4b5c936e48716a8dab3691bccacd625bd864903e71132c9098929f5a0a">H5F_FSPACE_STRATEGY_PAGE</a> is another option, which adds paged allocation and is used most effectively with page buffering.<br />
For an in-depth account of HDF5 file space management, paged-allocation, and page buffering, see the following documents: <ul>
<li><a href="https://docs.hdfgroup.org/hdf5/rfc/FileSpaceManagement.pdf">HDF5 File Space Management</a> </li>
<li><a href="https://docs.hdfgroup.org/hdf5/rfc/paged_aggregation.pdf">HDF5 File Space Management: Paged Aggregation</a> </li>
<li><a href="https://docs.hdfgroup.org/hdf5/rfc/RFC-Page_Buffering.pdf">Page Buffering</a></li>
</ul>
</dd></dl>
<dl class="section user"><dt>See Also</dt><dd>See <a class="el" href="_accessibility.html#CB_MaintainCompat">Maintaining Compatibility with other HDF5 Library Versions</a> for HDF5 compatibility implications.</dd></dl>
<h2><a class="anchor" id="CB_RemoveUnusedSpace"></a>
Removing Unused Space from HDF5 Files</h2>
<dl class="section user"><dt>Problem</dt><dd>Based on estimates or <code>h5stat</code> output you know that a large portion of an HDF5 file consists of free or unaccounted space, and you would like to remove it.</dd></dl>
<h2><a class="anchor" id="CB_UserBlock"></a>
Creating an HDF5 File User Block</h2>
<dl class="section user"><dt>Problem</dt><dd>You would like to include certain ancillary, non-HDF5 content in an HDF5 file such that it can be accessed without the HDF5 library.</dd></dl>
<dl class="section user"><dt>Solution</dt><dd>Use a file creation property list in which the user block size is set via <a class="el" href="group___f_c_p_l.html#ga403bd982a2976c932237b186ed1cff4d" title="Sets user block size.">H5Pset_userblock()</a>. In the example below, we create an 8 MiB user block. <div class="fragment"><div class="line"><span class="lineno"> 60</span> {</div>
<div class="line"><span class="lineno"> 61</span> __label__ fail_fcpl, fail_file;</div>
<div class="line"><span class="lineno"> 62</span> <a class="code hl_typedef" href="_h5_ipublic_8h.html#a0045db7ff9c22ad35db6ae91662e1943">hid_t</a> fcpl, file;</div>
<div class="line"><span class="lineno"> 63</span> </div>
<div class="line"><span class="lineno"> 64</span> <span class="keywordflow">if</span> ((fcpl = <a class="code hl_function" href="group___p_l_c_r.html#gaf1b11da01d4d45d788c45f8bc5f0cbfa">H5Pcreate</a>(<a class="code hl_define" href="_h5_ppublic_8h.html#a206f334f1e6c973e1215a3148b45b977">H5P_FILE_CREATE</a>)) < 0) {</div>
<div class="line"><span class="lineno"> 65</span> ret_val = EXIT_FAILURE;</div>
<div class="line"><span class="lineno"> 66</span> <span class="keywordflow">goto</span> fail_fcpl;</div>
<div class="line"><span class="lineno"> 67</span> }</div>
<div class="line"><span class="lineno"> 68</span> <span class="keywordflow">if</span> (<a class="code hl_function" href="group___f_c_p_l.html#ga403bd982a2976c932237b186ed1cff4d">H5Pset_userblock</a>(fcpl, 8192 * 1024) < 0) {</div>
<div class="line"><span class="lineno"> 69</span> ret_val = EXIT_FAILURE;</div>
<div class="line"><span class="lineno"> 70</span> <span class="keywordflow">goto</span> fail_file;</div>
<div class="line"><span class="lineno"> 71</span> }</div>
<div class="line"><span class="lineno"> 72</span> <span class="keywordflow">if</span> ((file = <a class="code hl_function" href="group___h5_f.html#gae64b51ee9ac0781bc4ccc599d98387f4">H5Fcreate</a>(<span class="stringliteral">"userblock.h5"</span>, <a class="code hl_define" href="_h5_fpublic_8h.html#a5a2d6726f9ad8d2bca8df2b817e5ad6a">H5F_ACC_TRUNC</a>, fcpl, <a class="code hl_define" href="_h5_ppublic_8h.html#afa85e97bfbf9bf1c58e39263846c568f">H5P_DEFAULT</a>)) < 0) {</div>
<div class="line"><span class="lineno"> 73</span> ret_val = EXIT_FAILURE;</div>
<div class="line"><span class="lineno"> 74</span> <span class="keywordflow">goto</span> fail_file;</div>
<div class="line"><span class="lineno"> 75</span> }</div>
<div class="line"><span class="lineno"> 76</span> <a class="code hl_function" href="group___h5_f.html#gac55cd91d80822e4f8c2a7f04ea71b124">H5Fclose</a>(file);</div>
<div class="line"><span class="lineno"> 77</span> </div>
<div class="line"><span class="lineno"> 78</span>fail_file:</div>
<div class="line"><span class="lineno"> 79</span> <a class="code hl_function" href="group___p_l_c_r.html#ga5dce61149211d3ef319452aa598887fb">H5Pclose</a>(fcpl);</div>
<div class="line"><span class="lineno"> 80</span>fail_fcpl:;</div>
<div class="line"><span class="lineno"> 81</span> }</div>
</div><!-- fragment --></dd></dl>
<dl class="section user"><dt>Discussion</dt><dd>The user block begins at offset 0 and must be at least 512 bytes and a power of 2. The HDF5 library ignores any content between the beginning of the file and the end of the user block.<br />
You can add or strip a user block to/from an existing HDF5 file with the <code>h5jam</code>/<code>h5unjam</code> tool, respectively. </dd></dl>
<dl class="section warning"><dt>Warning</dt><dd>If you try to embed content into the user block for use by other applications, pay close attention to how they handle space beyond the last used byte in the user block or the user block in general. In the worst case, applications might try to truncate the rest of the file and destroy the HDF5 portion of the file.</dd></dl>
<dl class="section user"><dt>See Also</dt><dd>References to related recipes </dd></dl>
</div></div><!-- contents -->
</div><!-- PageDoc -->
</div><!-- doc-content -->
<!-- start footer part -->
<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
<ul>
<li class="footer">Generated by
<a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.10.0 </li>
</ul>
</div>
</body>
</html>
|
