1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
|
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="Content-Type" content="text/xhtml;charset=UTF-8"/>
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<meta name="generator" content="Doxygen 1.9.1"/>
<meta name="viewport" content="width=device-width, initial-scale=1"/>
<title>HDF5: Metadata Caching in HDF5</title>
<link href="tabs.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="jquery.js"></script>
<script type="text/javascript" src="dynsections.js"></script>
<link href="navtree.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="resize.js"></script>
<script type="text/javascript" src="navtreedata.js"></script>
<script type="text/javascript" src="navtree.js"></script>
<link href="search/search.css" rel="stylesheet" type="text/css"/>
<script type="text/javascript" src="search/searchdata.js"></script>
<script type="text/javascript" src="search/search.js"></script>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
$(document).ready(function() { init_search(); });
/* @license-end */
</script>
<link href="doxygen.css" rel="stylesheet" type="text/css" />
<link href="hdf5doxy.css" rel="stylesheet" type="text/css">
<!-- <link href="hdf5doxy.css" rel="stylesheet" type="text/css"/>
-->
<script type="text/javascript" src="hdf5_navtree_hacks.js"></script>
</head>
<body>
<div style="background:#FFDDDD;font-size:120%;text-align:center;margin:0;padding:5px">Please, help us to better serve our user community by answering the following short survey: <a href="https://www.hdfgroup.org/website-survey/">https://www.hdfgroup.org/website-survey/</a></div>
<div id="top"><!-- do not remove this div, it is closed by doxygen! -->
<div id="titlearea">
<table cellspacing="0" cellpadding="0">
<tbody>
<tr style="height: 56px;">
<td id="projectlogo"><img alt="Logo" src="HDFG-logo.png"/></td>
<td id="projectalign" style="padding-left: 0.5em;">
<div id="projectname"><a href="https://www.hdfgroup.org">HDF5</a>
 <span id="projectnumber">1.15.0.8253ab9</span>
</div>
<div id="projectbrief">API Reference</div>
</td>
<td> <div id="MSearchBox" class="MSearchBoxInactive">
<span class="left">
<img id="MSearchSelect" src="search/mag_sel.svg"
onmouseover="return searchBox.OnSearchSelectShow()"
onmouseout="return searchBox.OnSearchSelectHide()"
alt=""/>
<input type="text" id="MSearchField" value="Search" accesskey="S"
onfocus="searchBox.OnSearchFieldFocus(true)"
onblur="searchBox.OnSearchFieldFocus(false)"
onkeyup="searchBox.OnSearchFieldChange(event)"/>
</span><span class="right">
<a id="MSearchClose" href="javascript:searchBox.CloseResultsWindow()"><img id="MSearchCloseImg" border="0" src="search/close.svg" alt=""/></a>
</span>
</div>
</td>
</tr>
</tbody>
</table>
</div>
<!-- end header part -->
<!-- Generated by Doxygen 1.9.1 -->
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
var searchBox = new SearchBox("searchBox", "search",false,'Search','.html');
/* @license-end */
</script>
</div><!-- top -->
<div id="side-nav" class="ui-resizable side-nav-resizable">
<div id="nav-tree">
<div id="nav-tree-contents">
<div id="nav-sync" class="sync"></div>
</div>
</div>
<div id="splitbar" style="-moz-user-select:none;"
class="ui-resizable-handle">
</div>
</div>
<script type="text/javascript">
/* @license magnet:?xt=urn:btih:cf05388f2679ee054f2beb29a391d25f4e673ac3&dn=gpl-2.0.txt GPL-v2 */
$(document).ready(function(){initNavTree('_t_n_m_d_c.html',''); initResizable(); });
/* @license-end */
</script>
<div id="doc-content">
<!-- window showing the filter options -->
<div id="MSearchSelectWindow"
onmouseover="return searchBox.OnSearchSelectShow()"
onmouseout="return searchBox.OnSearchSelectHide()"
onkeydown="return searchBox.OnSearchSelectKey(event)">
</div>
<!-- iframe showing the search results (closed by default) -->
<div id="MSearchResultsWindow">
<iframe src="javascript:void(0)" frameborder="0"
name="MSearchResults" id="MSearchResults">
</iframe>
</div>
<div class="PageDoc"><div class="header">
<div class="headertitle">
<div class="title">Metadata Caching in HDF5 </div> </div>
</div><!--header-->
<div class="contents">
<div class="textblock"><dl class="todo"><dt><b><a class="el" href="todo.html#_todo000003">Todo:</a></b></dt><dd>Revise this!</dd></dl>
<h1><a class="anchor" id="intro"></a>
Introduction</h1>
<p>In the 1.6.4 release, we introduced a re-implementation of the metadata cache. That release contained an incomplete version of the cache which could not be controlled via the API. The version in the 1.8 release is more mature and includes new API calls that allow the user program to configure the metadata cache both on file open and at run time.</p>
<p>From the user perspective, the most striking effect of the new cache should be a large reduction in the cache memory requirements when working with complex HDF5 files.</p>
<p>Those working with such files may also notice a reduction in file close time.</p>
<p>Those working with HDF5 files with a simple structure shouldn't notice any particular changes in most cases. In rare cases, there may be a significant improvement in performance.</p>
<p>The remainder of this document contains an architectural overview of the old and new metadata caches, a discussion of algorithms used to automatically adjust cache size to circumstances, and a high-level discussion of the cache configuration controls. It can be safely skipped by anyone who works only with HDF5 files with relatively simple structure (i.e. no huge groups, no datasets with large numbers of chunks, and no objects with large numbers of attributes.)</p>
<p>On the other hand, it is mandatory reading if you want to use something other than the default metadata cache configuration. The documentation on the metadata cache-related API calls will not make much sense without this background.</p>
<h1><a class="anchor" id="oldnew"></a>
Old and New Metadata Cache</h1>
<h2><a class="anchor" id="old"></a>
The Old Metadata Cache</h2>
<p>The old metadata cache indexed the cache with a hash table with no provision for collisions. Instead, collisions were handled by evicting the existing entry to make room for the new entry. Aside from flushes, there was no other mechanism for evicting entries, so the replacement policy could best be described as "Evict on Collision".</p>
<p>As a result, if two frequently used entries hashed to the same location, they would evict each other regularly. To decrease the likelihood of this situation, the default hash table size was set fairly large – slightly more than 10,000. This worked well, but since the size of metadata entries is not bounded, and since entries were only evicted on collision, the large hash table size allowed the cache size to explode when working with HDF5 files with complex structure.</p>
<p>The "Evict on Collision" replacement policy also caused problems with the parallel version of the HDF5 library, as a collision with a dirty entry could force a write in response to a metadata read. Since all metadata writes must be collective in the parallel case while reads need not be, this could cause the library to hang if only some of the processes participated in a metadata read that forced a write. Prior to the implementation of the new metadata cache, we dealt with this issue by maintaining a shadow cache for dirty entries evicted by a read.</p>
<h2><a class="anchor" id="new"></a>
The New Metadata Cache</h2>
<p>The new metadata cache was designed to address the above issues. After implementation, it became evident that the working set size for HDF5 files varies widely depending on both structure and access patterns. Thus it was necessary to add support for cache size adjustment under either automatic or user program control (see section 2.3 for details).</p>
<p>When the cache is operating under direct user program control, it is also possible to temporarily disable evictions from the metadata cache so as to maximize raw data throughput at the expense of allowing the cache to grow without bound until evictions are enabled again.</p>
<p>Structurally, the new metadata cache can be thought of as a heavily modified version of the UNIX buffer cache as described in chapter three of M. J. Bach's "The Design of the UNIX Operating System" In essence, the UNIX buffer cache uses a hash table with chaining to index a pool of fixed-size buffers. It uses the LRU replacement policy to select candidates for eviction.</p>
<p>Since HDF5 metadata entries are not of fixed size and may grow arbitrarily large, the size of the new metadata cache cannot be controlled by setting a maximum number of entries. Instead, the new cache keeps a running sum of the sizes of all entries and attempts to evict entries as necessary to stay within a user-specified maximum size. (Note the use of the word "attempts" here – as will be seen, it is possible for the cache to exceed its currently specified maximum size.) At present, the LRU replacement policy is the only option for selecting candidates for eviction.</p>
<p>Per the standard Unix buffer cache, dirty entries are given two passes through the LRU list before being evicted. The first time they reach the end of the LRU list, they are flushed, marked as clean, and moved to the head of the LRU list. When a clean entry reaches the end of the LRU list, it is simply evicted if space is needed.</p>
<p>The cache cannot evict entries that are locked, and thus it will temporarily grow beyond its maximum size if there are insufficient unlocked entries available for eviction.</p>
<p>In the parallel version of the library, only the cache running under process 0 of the file communicator is allowed to write metadata to file. All the other caches must retain dirty metadata until the process 0 cache tells them that the metadata is clean.</p>
<p>Since all operations modifying metadata must be collective, all caches see the same stream of dirty metadata. This fact is used to allow them to synchronize every n bytes of dirty metadata, where n is a user-configurable value that defaults to 256 KB.</p>
<p>To avoid sending the other caches messages from the future, process 0 must not write any dirty entries until it reaches a synchronization point. When it reaches a synchronization point, it writes entries as needed, and then broadcasts the list of flushed entries to the other caches. The caches on the other processes use this list to mark entries clean before they leave the synchronization point, allowing them to evict those entries as needed.</p>
<p>The caches will also synchronize on a user-initiated flush.</p>
<p>To minimize overhead when running in parallel, the cache maintains a "clean" LRU list in addition to the regular LRU list. This list contains only clean entries and is used as a source of candidates for eviction when flushing dirty entries is not allowed.</p>
<p>Since flushing entries is forbidden most of the time when running in parallel, the caches can be forced to exceed their maximum sizes if they run out of clean entries to evict.</p>
<p>To decrease the likelihood of this event, the new cache allows the user to specify a minimum clean size – which is a minimum total size of all the entries on the clean LRU plus all unused space in the cache.</p>
<p>While the clean LRU list is only maintained in the parallel version of the HDF5 library, the notion of a minimum clean size still applies in the serial case. Here it is used to force a mix of clean and dirty entries in the cache even in the write-only case.</p>
<p>This, in turn, reduces the number of redundant flushes by avoiding the case in which the cache fills with dirty metadata and all entries must be flushed before a clean entry can be evicted to make room for a new entry.</p>
<p>Observe that in both the serial and parallel cases, the maintenance of a minimum clean size modifies the replacement policy, as dirty entries may be flushed earlier than would otherwise be the case so as to maintain the desired amount of clean and/or empty space in the cache.</p>
<p>While the new metadata cache only supports the LRU replacement policy at present, that may change. Support for multiple replacement policies was very much in mind when the cache was designed, as was the ability to switch replacement policies at run time. The situation has been complicated by the later addition of the adaptive cache resizing requirement, as two of the resizing algorithms piggyback on the LRU list. However, if there is a need for additional replacement policies, it shouldn't be too hard to implement them.</p>
<h1><a class="anchor" id="adapt"></a>
Adaptive Cache Resizing in the New Metadata Cache</h1>
<p>As mentioned earlier, the metadata working set size for an HDF5 file varies wildly depending on the structure of the file and the access pattern. For example, a 2MB limit on metadata cache size is excessive for an H5repack of almost all HDF5 files we have tested. However, I have a file submitted by one of our users that will run a 13% hit rate with this cache size and will lock up one of our Linux boxes using the old metadata cache. Increase the new metadata cache size to 4 MB, and the hit rate exceeds 99%.</p>
<p>In this case, the main culprit is a root group with more than 20,000 entries in it. As a result, the root group heap exceeds 1 MB, which tends to crowd out the rest of the metadata in a 2 MB cache</p>
<p>This case and a number of synthetic tests convinced us that we needed to modify the new metadata cache to expand and contract according to need within user-specified bounds.</p>
<p>I was unable to find any previous work on this problem, so I invented solutions as I went along. If you are aware of prior work, please send me references. The closest I was able to come was a group of embedded CPU designers who were turning off sections of their cache to conserve power.</p>
<h2><a class="anchor" id="increasing"></a>
Increasing the Cache Size</h2>
<p>In the context of the HDF5 library, the problem of increasing the cache size as necessary to contain the current working set turns out to involve two rather different issues.</p>
<p>The first of these, which was recognized immediately, is the problem of recognizing long term changes in working set size, and increasing the cache size accordingly, while not reacting to transients.</p>
<p>The second, which I recognized the hard way, is to adjust the cache size for sudden, dramatic increases in working set size caused by requests for large pieces of metadata which may be larger than the current metadata cache size.</p>
<p>The algorithms for handling these situations are discussed below. These problems are largely orthogonal to each other, so both algorithms may be used simultaneously.</p>
<h3><a class="anchor" id="hrtcsi"></a>
Hit Rate Threshold Cache Size Increment</h3>
<p>Perhaps the most obvious heuristic for identifying cases in which the cache is too small involves monitoring the hit rate. If the hit rate is low for a while, and the cache is at its current maximum size, the current maximum cache size is probably too small.</p>
<p>The hit rate threshold algorithm for increasing cache size applies this intuition directly.</p>
<p>Hit rate statistics are collected over a user-specified number of cache accesses. This period is known as an epoch.</p>
<p>At the end of each epoch, the hit rate is computed, and the counters are reset. If the hit rate is below a user-specified threshold and the cache is at its current maximum size, the maximum size of the cache is increased by a user-specified multiple. If required, the new cache maximum size is clipped to stay within the user-specified upper bound on the maximum cache size, and optionally, within a user-specified maximum increment.</p>
<p>My tests indicate that this algorithm works well in most cases. However, in a synthetic test in which hit rate increased slowly with cache size, and load remained steady for many epochs, I observed a case in which cache size increased until the hit rate just exceeded the specified minimum and then stalled. This is a problem, as to avoid volatility, it is necessary to set the minimum hit rate threshold well below the desired hit rate. Thus we may find ourselves with a cache running with a 91% hit rate when we really want it to increase its size until the hit rate is about 99%.</p>
<p>If this case occurs frequently in actual use, I will have to come up with an improved algorithm. Please let me know if you see this behavior. However, I had to work rather hard to create it in my synthetic tests, so I would expect it to be uncommon.</p>
<h3><a class="anchor" id="fcsi"></a>
Flash Cache Size Increment</h3>
<p>A fundamental problem with the above algorithm is that contains the hidden assumption that cache entries are relatively small in comparison to the cache itself. While I knew this assumption was not generally true when I developed the algorithm, I thought that cases, where it failed, would be so rare as to not be worth considering, as even if they did occur, the above algorithm would rectify the situation within an epoch or two.</p>
<p>While it is true that such occurrences are rare, and it is true that the hit rate threshold cache size increment algorithm will rectify the situation eventually, the performance degradation experienced by users while waiting for the epoch to end was so extreme that some way of accelerating response to such situations was essential.</p>
<p>To understand the problem, consider the following use case:</p>
<p>Suppose we create a group, and then repeatedly create a new data set in the group, write some data to it and then close it.</p>
<p>In some versions of the HDF5 file format, the names of the datasets will be stored in a local heap associated with the group, and the space for that heap will be allocated in a single, contiguous chunk. When this local heap is full, we allocate a new chunk twice the size of the old, copy the data from the old local heap into the new, and discard the old local heap.</p>
<p>By default, the minimum metadata cache size is set to 2 MB. Thus in this use case, our hit rate will be fine as long as the local heap is no larger than a little less than 2 MB, as the group related metadata is accessed frequently and never evicted, and the data set related metadata is never accessed once the data set is closed, and thus is evicted smoothly to make room for new data sets.</p>
<p>All this changes abruptly when the local heap finally doubles in size to a value above the slightly less than 2 MB limit. All of a sudden, the local heap is the size of the metadata cache, and the cache must constantly swap it in to access it, and then swap it out to make room for other metadata.</p>
<p>The hit rate threshold-based algorithm for increasing the cache size will fix this problem eventually, but performance will be very bad until it does, as the metadata cache will largely ineffective until its size is increased.</p>
<p>An obvious heuristic for addressing this "big rock in a small pond" issue is to watch for large "incoming rocks", and increase the size of the "pond" if the rock is so big that it will force most of the "water" out of the "pond".</p>
<p>The add space flash cache size increment algorithm applies this intuition directly:</p>
<p>Let x be either the size of a newly inserted entry, a newly loaded entry, or the number of bytes by which the size of an existing entry has been increased (i.e. the size of the "rock").</p>
<p>If x is greater than some user-specified fraction of the current maximum cache size, increase the current maximum cache size by x times some user-specified multiple, less any free space that was in the cache, to begin with. Further, to avoid confusing the other cache size increment/decrement code, start a new epoch.</p>
<p>At present, this algorithm pays no attention to any user-specified limit on the maximum size of any single cache size increase, but it DOES stay within the user-specified upper bound on the maximum cache size.</p>
<p>While it should be easy to see how this algorithm could be fooled into inactivity by a large number of entries that were not quite large enough to cross the threshold, in practice it seems to work reasonably well.</p>
<p>Needless to say, I will revisit the issue should this cease to be the case.</p>
<h2><a class="anchor" id="decreasing"></a>
Decreasing the Cache Size</h2>
<p>Identifying cases in which the maximum cache size is larger than necessary turned out to be more difficult.</p>
<h3><a class="anchor" id="hrtcsr"></a>
Hit Rate Threshold Cache Size Reduction</h3>
<p>One obvious heuristic is to monitor the hit rate and guess that we can safely decrease cache size if the hit rate exceeds some user-supplied threshold (say .99995). The hit rate threshold size decrement algorithm implemented in the new metadata cache implements this intuition as follows:</p>
<p>At the end of each epoch (this is the same epoch that is used in the cache size increment algorithm), the hit rate is compared with the user-specified threshold. If the hit rate exceeds that threshold, the current maximum cache size is decreased by a user-specified factor. If required, the size of the reduction is clipped to stay within a user-specified lower bound on the maximum cache size, and optionally, within a user-specified maximum decrement.</p>
<p>In my synthetic tests, this algorithm works poorly. Even with a very high threshold and a small maximum reduction, it results in cache size oscillations. The size increment code typically increments the maximum cache size above the working set size. This results in a high hit rate, which causes the threshold size decrement code to reduce the maximum cache size below the working set size, which causes the hit rate to crash causing the cycle to repeat. The resulting average hit rate is poor.</p>
<p>It remains to be seen if this behavior will be seen in the field. The algorithm is available for use, but it wouldn't be my first choice. If you use it, please report back.</p>
<h3><a class="anchor" id="acsr"></a>
Ageout Cache Size Reduction</h3>
<p>Another heuristic for dealing with oversized cache conditions is to look for entries that haven't been accessed for a long time, evict them, and reduce the cache size accordingly.</p>
<p>The age out cache size reduction applies this intuition as follows: At the end of each epoch (again the same epoch as used in the cache size increment algorithm), all entries that haven't been accessed for a user-configurable number of epochs (1 - 10 at present) are evicted. The maximum cache size is then reduced to equal the sum of the sizes of the remaining entries. The size of the reduction is clipped to stay within a user-specified lower bound on maximum cache size, and optionally, within a user-specified maximum decrement.</p>
<p>In addition, the user may specify a minimum fraction of the cache which must be empty before the cache size is reduced. Thus if an empty reserve of 0.1 was specified on a 10 MB cache, there would be no cache size reduction unless the eviction of aged out entries resulted in more than 1 MB of empty space. Further, even after the reduction, the cache would be one-tenth empty.</p>
<p>In my synthetic tests, the age out algorithm works rather well, although it is somewhat sensitive to the epoch length and age out period selection.</p>
<h3><a class="anchor" id="awhrtcsr"></a>
Ageout With Hit Rate Threshold Cache Size Reduction</h3>
<p>To address these issues, I combined the hit rate threshold and age out heuristics.</p>
<p>Age out with threshold works just like age out, except that the algorithm is not run unless the hit rate exceeded a user-specified threshold in the previous epoch.</p>
<p>In my synthetic tests, age out with threshold seems to work nicely, with no observed oscillation. Thus I have selected it as the default cache size reduction algorithm.</p>
<p>For those interested in such things, the age out algorithm is implemented by inserting a marker entry at the head of the LRU list at the beginning of each epoch. Entries that haven't been accessed for at least n epochs are simply entries that appear in the LRU list after the n-th marker at the end of an epoch.</p>
<h1><a class="anchor" id="configuring"></a>
Configuring the New Metadata Cache</h1>
<p>Due to a lack of resources, the design work on the automatic cache size adjustment algorithms was done hastily, using primarily synthetic tests. I don't think I spent more than a couple weeks writing and running performance tests – most time went into coding and functional testing.</p>
<p>As a result, while I think the algorithms provided for adaptive cache resizing will work well in actual use, I don't really know (although preliminary results from the field are promising). Fortunately, the issue shouldn't arise for the vast majority of HDF5 users, and those for whom it may arise should be savvy enough to recognize problems and deal with them.</p>
<p>For this latter class of users, I have implemented a number of new API calls allowing the user to select and configure the cache resize algorithms, or to turn them off and control cache size directly from the user program. There are also API calls that allow the user program to monitor hit rate and cache size.</p>
<p>From the user perspective, all the cache configuration data for a given file is contained in an instance of the <a class="el" href="struct_h5_a_c__cache__config__t.html">H5AC_cache_config_t</a> structure – the definition of which is given below:</p>
<div class="fragment"><div class="line"><span class="keyword">typedef</span> <span class="keyword">struct </span><a class="code" href="struct_h5_a_c__cache__config__t.html">H5AC_cache_config_t</a> {</div>
<div class="line"> <span class="comment">/* general configuration fields: */</span></div>
<div class="line"> <span class="keywordtype">int</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#aad880fc4455c253781e8968f2239d56f">version</a>;</div>
<div class="line"> <a class="code" href="_h5public_8h.html#ad470b00eccd2115c707c02de5fa1120d">hbool_t</a> <a class="code" href="struct_h5_a_c__cache__config__t.html#aab04b3e770d1c2decc03f6fd94b64b17">rpt_fcn_enabled</a>;</div>
<div class="line"> <a class="code" href="_h5public_8h.html#ad470b00eccd2115c707c02de5fa1120d">hbool_t</a> <a class="code" href="struct_h5_a_c__cache__config__t.html#a6c03d051d5efb0e64bdca5d0b354a8b1">open_trace_file</a>;</div>
<div class="line"> <a class="code" href="_h5public_8h.html#ad470b00eccd2115c707c02de5fa1120d">hbool_t</a> <a class="code" href="struct_h5_a_c__cache__config__t.html#aced3486ae892a53209b7aae069a34be9">close_trace_file</a>;</div>
<div class="line"> <span class="keywordtype">char</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#a92e7d20eb2b7b353961c64558ddac080">trace_file_name</a>[<a class="code" href="_h5_a_cpublic_8h.html#a717f1f3545cfc3d1b2208c96cc0c3bd3">H5AC__MAX_TRACE_FILE_NAME_LEN</a> + 1];</div>
<div class="line"> <a class="code" href="_h5public_8h.html#ad470b00eccd2115c707c02de5fa1120d">hbool_t</a> <a class="code" href="struct_h5_a_c__cache__config__t.html#a3d6e32c7a9bf8f65848c6840e1de8c6e">evictions_enabled</a>;</div>
<div class="line"> <a class="code" href="_h5public_8h.html#ad470b00eccd2115c707c02de5fa1120d">hbool_t</a> <a class="code" href="struct_h5_a_c__cache__config__t.html#a707198a337ece01a37dfec6e01275d82">set_initial_size</a>;</div>
<div class="line"> <span class="keywordtype">size_t</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#a649236e7dd714855a50f122aa5caca9f">initial_size</a>;</div>
<div class="line"> <span class="keywordtype">double</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#abd805b98f873c1720f34a0ce937838fd">min_clean_fraction</a>;</div>
<div class="line"> <span class="keywordtype">size_t</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#af4728438dee601cb2554d9bf18d78a43">max_size</a>;</div>
<div class="line"> <span class="keywordtype">size_t</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#af99ca22b80e05fd5b3603806348ab647">min_size</a>;</div>
<div class="line"> <span class="keywordtype">long</span> <span class="keywordtype">int</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#ac998e51b01e0eef09d9a29c43f97e4bf">epoch_length</a>;</div>
<div class="line"> </div>
<div class="line"> <span class="comment">/* size increase control fields: */</span></div>
<div class="line"> <span class="keyword">enum</span> <a class="code" href="_h5_cpublic_8h.html#a040d488146ff1ca0a82209e9af3918fa">H5C_cache_incr_mode</a> <a class="code" href="struct_h5_a_c__cache__config__t.html#ae825aaf759060239e92170d20eb97d26">incr_mode</a>;</div>
<div class="line"> <span class="keywordtype">double</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#a20f323fcb4747fc7228d2d74bb965586">lower_hr_threshold</a>;</div>
<div class="line"> <span class="keywordtype">double</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#ac504dff76b24ab9f15536c51aec9fbbb">increment</a>;</div>
<div class="line"> <a class="code" href="_h5public_8h.html#ad470b00eccd2115c707c02de5fa1120d">hbool_t</a> <a class="code" href="struct_h5_a_c__cache__config__t.html#a322f2a955e71ab7ede483e991e76f806">apply_max_increment</a>;</div>
<div class="line"> <span class="keywordtype">size_t</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#ad5a729f1d611f2780679a35b3524052c">max_increment</a>;</div>
<div class="line"> <span class="keyword">enum</span> <a class="code" href="_h5_cpublic_8h.html#aaaa13ca7756d135b7df6d5a6779ee908">H5C_cache_flash_incr_mode</a> <a class="code" href="struct_h5_a_c__cache__config__t.html#a0e25a1dc2c695bea335df0e23ed6363c">flash_incr_mode</a>;</div>
<div class="line"> <span class="keywordtype">double</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#a77b1812e0407c9122db524462a5c9633">flash_multiple</a>;</div>
<div class="line"> <span class="keywordtype">double</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#a95fb1e03a77ef5c109d0c851416ced55">flash_threshold</a>;</div>
<div class="line"> </div>
<div class="line"> <span class="comment">/* size decrease control fields: */</span></div>
<div class="line"> <span class="keyword">enum</span> <a class="code" href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04f">H5C_cache_decr_mode</a> <a class="code" href="struct_h5_a_c__cache__config__t.html#a5df68196b281c19d8ab7da0788566aec">decr_mode</a>;</div>
<div class="line"> <span class="keywordtype">double</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#a84a5ff4ac69196aa27c14f6f796db596">upper_hr_threshold</a>;</div>
<div class="line"> <span class="keywordtype">double</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#a54007d3f2afb718b437f499a5c8b46d9">decrement</a>;</div>
<div class="line"> <a class="code" href="_h5public_8h.html#ad470b00eccd2115c707c02de5fa1120d">hbool_t</a> <a class="code" href="struct_h5_a_c__cache__config__t.html#ac2c89a735413e4c36d6f463c4890ab42">apply_max_decrement</a>;</div>
<div class="line"> <span class="keywordtype">size_t</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#a75e875a61c9da7f82482d0f6fe6e7152">max_decrement</a>;</div>
<div class="line"> <span class="keywordtype">int</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#a7ac41e345300bdecd9943e855d55b71b">epochs_before_eviction</a>;</div>
<div class="line"> <a class="code" href="_h5public_8h.html#ad470b00eccd2115c707c02de5fa1120d">hbool_t</a> <a class="code" href="struct_h5_a_c__cache__config__t.html#a6c5cff8e8a21230cc7f21859d8d6bd22">apply_empty_reserve</a>;</div>
<div class="line"> <span class="keywordtype">double</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#a9c1ae995513b55737aad09e11beff733">empty_reserve</a>;</div>
<div class="line"> </div>
<div class="line"> <span class="comment">/* parallel configuration fields: */</span></div>
<div class="line"> <span class="keywordtype">size_t</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#a8e3c2a2d300b7a8f8d3705fc5e59a3c1">dirty_bytes_threshold</a>;</div>
<div class="line"> <span class="keywordtype">int</span> <a class="code" href="struct_h5_a_c__cache__config__t.html#a83a536128dbb7785b2553c294f33d1fe">metadata_write_strategy</a>;</div>
<div class="line">} <a class="code" href="struct_h5_a_c__cache__config__t.html">H5AC_cache_config_t</a>;</div>
<div class="ttc" id="a_h5_a_cpublic_8h_html_a717f1f3545cfc3d1b2208c96cc0c3bd3"><div class="ttname"><a href="_h5_a_cpublic_8h.html#a717f1f3545cfc3d1b2208c96cc0c3bd3">H5AC__MAX_TRACE_FILE_NAME_LEN</a></div><div class="ttdeci">#define H5AC__MAX_TRACE_FILE_NAME_LEN</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:432</div></div>
<div class="ttc" id="a_h5_cpublic_8h_html_a040d488146ff1ca0a82209e9af3918fa"><div class="ttname"><a href="_h5_cpublic_8h.html#a040d488146ff1ca0a82209e9af3918fa">H5C_cache_incr_mode</a></div><div class="ttdeci">H5C_cache_incr_mode</div><div class="ttdef"><b>Definition:</b> H5Cpublic.h:26</div></div>
<div class="ttc" id="a_h5_cpublic_8h_html_a4f8534794ad9a977185a5d608c0af04f"><div class="ttname"><a href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04f">H5C_cache_decr_mode</a></div><div class="ttdeci">H5C_cache_decr_mode</div><div class="ttdef"><b>Definition:</b> H5Cpublic.h:42</div></div>
<div class="ttc" id="a_h5_cpublic_8h_html_aaaa13ca7756d135b7df6d5a6779ee908"><div class="ttname"><a href="_h5_cpublic_8h.html#aaaa13ca7756d135b7df6d5a6779ee908">H5C_cache_flash_incr_mode</a></div><div class="ttdeci">H5C_cache_flash_incr_mode</div><div class="ttdef"><b>Definition:</b> H5Cpublic.h:34</div></div>
<div class="ttc" id="a_h5public_8h_html_ad470b00eccd2115c707c02de5fa1120d"><div class="ttname"><a href="_h5public_8h.html#ad470b00eccd2115c707c02de5fa1120d">hbool_t</a></div><div class="ttdeci">bool hbool_t</div><div class="ttdef"><b>Definition:</b> H5public.h:249</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html">H5AC_cache_config_t</a></div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:462</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a0e25a1dc2c695bea335df0e23ed6363c"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a0e25a1dc2c695bea335df0e23ed6363c">H5AC_cache_config_t::flash_incr_mode</a></div><div class="ttdeci">enum H5C_cache_flash_incr_mode flash_incr_mode</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:615</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a20f323fcb4747fc7228d2d74bb965586"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a20f323fcb4747fc7228d2d74bb965586">H5AC_cache_config_t::lower_hr_threshold</a></div><div class="ttdeci">double lower_hr_threshold</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:585</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a322f2a955e71ab7ede483e991e76f806"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a322f2a955e71ab7ede483e991e76f806">H5AC_cache_config_t::apply_max_increment</a></div><div class="ttdeci">hbool_t apply_max_increment</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:607</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a3d6e32c7a9bf8f65848c6840e1de8c6e"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a3d6e32c7a9bf8f65848c6840e1de8c6e">H5AC_cache_config_t::evictions_enabled</a></div><div class="ttdeci">hbool_t evictions_enabled</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:526</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a54007d3f2afb718b437f499a5c8b46d9"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a54007d3f2afb718b437f499a5c8b46d9">H5AC_cache_config_t::decrement</a></div><div class="ttdeci">double decrement</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:657</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a5df68196b281c19d8ab7da0788566aec"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a5df68196b281c19d8ab7da0788566aec">H5AC_cache_config_t::decr_mode</a></div><div class="ttdeci">enum H5C_cache_decr_mode decr_mode</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:638</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a649236e7dd714855a50f122aa5caca9f"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a649236e7dd714855a50f122aa5caca9f">H5AC_cache_config_t::initial_size</a></div><div class="ttdeci">size_t initial_size</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:550</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a6c03d051d5efb0e64bdca5d0b354a8b1"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a6c03d051d5efb0e64bdca5d0b354a8b1">H5AC_cache_config_t::open_trace_file</a></div><div class="ttdeci">hbool_t open_trace_file</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:478</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a6c5cff8e8a21230cc7f21859d8d6bd22"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a6c5cff8e8a21230cc7f21859d8d6bd22">H5AC_cache_config_t::apply_empty_reserve</a></div><div class="ttdeci">hbool_t apply_empty_reserve</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:683</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a707198a337ece01a37dfec6e01275d82"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a707198a337ece01a37dfec6e01275d82">H5AC_cache_config_t::set_initial_size</a></div><div class="ttdeci">hbool_t set_initial_size</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:546</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a75e875a61c9da7f82482d0f6fe6e7152"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a75e875a61c9da7f82482d0f6fe6e7152">H5AC_cache_config_t::max_decrement</a></div><div class="ttdeci">size_t max_decrement</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:673</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a77b1812e0407c9122db524462a5c9633"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a77b1812e0407c9122db524462a5c9633">H5AC_cache_config_t::flash_multiple</a></div><div class="ttdeci">double flash_multiple</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:620</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a7ac41e345300bdecd9943e855d55b71b"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a7ac41e345300bdecd9943e855d55b71b">H5AC_cache_config_t::epochs_before_eviction</a></div><div class="ttdeci">int epochs_before_eviction</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:677</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a83a536128dbb7785b2553c294f33d1fe"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a83a536128dbb7785b2553c294f33d1fe">H5AC_cache_config_t::metadata_write_strategy</a></div><div class="ttdeci">int metadata_write_strategy</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:707</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a84a5ff4ac69196aa27c14f6f796db596"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a84a5ff4ac69196aa27c14f6f796db596">H5AC_cache_config_t::upper_hr_threshold</a></div><div class="ttdeci">double upper_hr_threshold</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:643</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a8e3c2a2d300b7a8f8d3705fc5e59a3c1"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a8e3c2a2d300b7a8f8d3705fc5e59a3c1">H5AC_cache_config_t::dirty_bytes_threshold</a></div><div class="ttdeci">size_t dirty_bytes_threshold</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:696</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a92e7d20eb2b7b353961c64558ddac080"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a92e7d20eb2b7b353961c64558ddac080">H5AC_cache_config_t::trace_file_name</a></div><div class="ttdeci">char trace_file_name[1024+1]</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:511</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a95fb1e03a77ef5c109d0c851416ced55"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a95fb1e03a77ef5c109d0c851416ced55">H5AC_cache_config_t::flash_threshold</a></div><div class="ttdeci">double flash_threshold</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:629</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_a9c1ae995513b55737aad09e11beff733"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#a9c1ae995513b55737aad09e11beff733">H5AC_cache_config_t::empty_reserve</a></div><div class="ttdeci">double empty_reserve</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:687</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_aab04b3e770d1c2decc03f6fd94b64b17"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#aab04b3e770d1c2decc03f6fd94b64b17">H5AC_cache_config_t::rpt_fcn_enabled</a></div><div class="ttdeci">hbool_t rpt_fcn_enabled</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:470</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_aad880fc4455c253781e8968f2239d56f"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#aad880fc4455c253781e8968f2239d56f">H5AC_cache_config_t::version</a></div><div class="ttdeci">int version</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:465</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_abd805b98f873c1720f34a0ce937838fd"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#abd805b98f873c1720f34a0ce937838fd">H5AC_cache_config_t::min_clean_fraction</a></div><div class="ttdeci">double min_clean_fraction</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:556</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_ac2c89a735413e4c36d6f463c4890ab42"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#ac2c89a735413e4c36d6f463c4890ab42">H5AC_cache_config_t::apply_max_decrement</a></div><div class="ttdeci">hbool_t apply_max_decrement</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:669</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_ac504dff76b24ab9f15536c51aec9fbbb"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#ac504dff76b24ab9f15536c51aec9fbbb">H5AC_cache_config_t::increment</a></div><div class="ttdeci">double increment</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:595</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_ac998e51b01e0eef09d9a29c43f97e4bf"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#ac998e51b01e0eef09d9a29c43f97e4bf">H5AC_cache_config_t::epoch_length</a></div><div class="ttdeci">long int epoch_length</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:573</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_aced3486ae892a53209b7aae069a34be9"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#aced3486ae892a53209b7aae069a34be9">H5AC_cache_config_t::close_trace_file</a></div><div class="ttdeci">hbool_t close_trace_file</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:499</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_ad5a729f1d611f2780679a35b3524052c"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#ad5a729f1d611f2780679a35b3524052c">H5AC_cache_config_t::max_increment</a></div><div class="ttdeci">size_t max_increment</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:611</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_ae825aaf759060239e92170d20eb97d26"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#ae825aaf759060239e92170d20eb97d26">H5AC_cache_config_t::incr_mode</a></div><div class="ttdeci">enum H5C_cache_incr_mode incr_mode</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:580</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_af4728438dee601cb2554d9bf18d78a43"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#af4728438dee601cb2554d9bf18d78a43">H5AC_cache_config_t::max_size</a></div><div class="ttdeci">size_t max_size</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:565</div></div>
<div class="ttc" id="astruct_h5_a_c__cache__config__t_html_af99ca22b80e05fd5b3603806348ab647"><div class="ttname"><a href="struct_h5_a_c__cache__config__t.html#af99ca22b80e05fd5b3603806348ab647">H5AC_cache_config_t::min_size</a></div><div class="ttdeci">size_t min_size</div><div class="ttdef"><b>Definition:</b> H5ACpublic.h:569</div></div>
</div><!-- fragment --><p> This structure is defined in <code><a class="el" href="_h5_a_cpublic_8h.html">H5ACpublic.h</a></code>. Each field is discussed below and in the associated header comment.</p>
<p>The C API allows you to get and set this structure directly. Unfortunately, the Fortran API has to do this with individual parameters for each of the fields (with the exception of version).</p>
<p>While the API calls are discussed individually in the reference manual, the following high-level discussion of what fields to change for different purposes should be useful.</p>
<h2><a class="anchor" id="gconfig"></a>
General Configuration</h2>
<p>The <code>version</code> field is intended to allow The HDF Group to change the <code><a class="el" href="struct_h5_a_c__cache__config__t.html">H5AC_cache_config_t</a></code> structure without breaking old code. For now, this field should always be set to <code>H5AC__CURR_CACHE_CONFIG_VERSION</code>, even when you are getting the current configuration data from the cache. The library needs the version number to know where fields are located with reference to the supplied base address.</p>
<p>The <a class="el" href="struct_h5_a_c__cache__config__t.html#aab04b3e770d1c2decc03f6fd94b64b17">rpt_fcn_enabled</a> field is a boolean flag that allows you to turn on and off the resize reporting function that reports the activities of the adaptive cache resize code at the end of each epoch – assuming that it is enabled.</p>
<p>The report function is unsupported, so you are on your own if you use it. Since it dumps status data to stdout, you should not attempt to use it with Windows unless you modify the source. You may find it useful if you want to experiment with different adaptive resize configurations. It is also a convenient way of diagnosing poor cache configuration. Finally, if you do lots of runs with identical behavior, you can use it to determine the metadata cache size needed in each phase of your program so you can set the required cache sizes manually.</p>
<p>The trace file fields are also unsupported. They allow one to open and close a trace file in which all calls to the metadata cache are logged in a user-specified file for later analysis. The feature is intended primarily for THG use in debugging or optimizing the metadata cache in cases where users in the field observe obscure failures or poor performance that we cannot re-create in the lab. The trace file will allow us to re-create the exact sequence of cache operations that are triggering the problem.</p>
<p>At present we do not have a playback utility for trace files, although I imagine that we will write one quickly when and if we need it.</p>
<p>To enable the trace file, you load the full path of the desired trace file into <a class="el" href="struct_h5_a_c__cache__config__t.html#a92e7d20eb2b7b353961c64558ddac080">trace_file_name</a>, and set <a class="el" href="struct_h5_a_c__cache__config__t.html#a6c03d051d5efb0e64bdca5d0b354a8b1">open_trace_file</a> to <code>TRUE</code>. In the parallel case, an ASCII representation of the rank of each process is appended to the supplied trace file name to create a unique trace file name for that process.</p>
<p>To close an open trace file, set <a class="el" href="struct_h5_a_c__cache__config__t.html#aced3486ae892a53209b7aae069a34be9">close_trace_file</a> to <code>TRUE</code>.</p>
<p>It must be emphasized that you are on your own if you play with the trace file feature absent a request from The HDF Group. Needless to say, the trace file feature is disabled by default. If you enable it, you will take a large performance hit and generate huge trace files.</p>
<p>The <a class="el" href="struct_h5_a_c__cache__config__t.html#a3d6e32c7a9bf8f65848c6840e1de8c6e">evictions_enabled</a> field is a boolean flag allowing the user to disable the eviction of entries from the metadata cache. Under normal operation conditions, this field will always be set to <code>TRUE</code>.</p>
<p>In rare circumstances, the raw data throughput requirements may be so high that the user wishes to postpone metadata writes so as to reserve I/O throughput for raw data. The <a class="el" href="struct_h5_a_c__cache__config__t.html#a3d6e32c7a9bf8f65848c6840e1de8c6e">evictions_enabled</a> field exists to allow this – although the user is to be warned that the metadata cache will grow without bound while evictions are disabled. Thus evictions should be re-enabled as soon as possible, and it may be wise to monitor cache size and statistics (to see how to enable statistics, see the debugging facilities section below).</p>
<p>Evictions may only be disabled when the automatic cache resize code is disabled as well. Thus to disable evictions, not only must the user set the <a class="el" href="struct_h5_a_c__cache__config__t.html#a3d6e32c7a9bf8f65848c6840e1de8c6e">evictions_enabled</a> field to <code>FALSE</code>, but he must also set <a class="el" href="struct_h5_a_c__cache__config__t.html#ae825aaf759060239e92170d20eb97d26">incr_mode</a> to <a class="el" href="_h5_cpublic_8h.html#a040d488146ff1ca0a82209e9af3918faa6c1241ad99f2c18de01ea9407db860d4">H5C_incr__off</a>, set <a class="el" href="struct_h5_a_c__cache__config__t.html#a0e25a1dc2c695bea335df0e23ed6363c">flash_incr_mode</a> to <a class="el" href="_h5_cpublic_8h.html#aaaa13ca7756d135b7df6d5a6779ee908a12938ef95720e1fd421ce77b1775c692">H5C_flash_incr__off</a>, and set <a class="el" href="struct_h5_a_c__cache__config__t.html#a5df68196b281c19d8ab7da0788566aec">decr_mode</a> to <a class="el" href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04fa2f20b9f9f574b06d8320f138f0e6c319">H5C_decr__off</a>.</p>
<p>To re-enable evictions, just set <a class="el" href="struct_h5_a_c__cache__config__t.html#a3d6e32c7a9bf8f65848c6840e1de8c6e">evictions_enabled</a> back to <code>TRUE</code>.</p>
<p>Before passing on to other subjects, it is worth re-iterating that disabling evictions is an extreme step. Before attempting it, you might consider setting a large cache size manually, and flushing the cache just before high raw data throughput is required. This may yield the desired results without the risks inherent in disabling evictions.</p>
<p>The <a class="el" href="struct_h5_a_c__cache__config__t.html#a707198a337ece01a37dfec6e01275d82">set_initial_size</a> and <a class="el" href="struct_h5_a_c__cache__config__t.html#a649236e7dd714855a50f122aa5caca9f">initial_size</a> fields allow you to specify an initial maximum cache size. If <a class="el" href="struct_h5_a_c__cache__config__t.html#a707198a337ece01a37dfec6e01275d82">set_initial_size</a> is <code>TRUE</code>, <a class="el" href="struct_h5_a_c__cache__config__t.html#a649236e7dd714855a50f122aa5caca9f">initial_size</a> must lie in the interval [<a class="el" href="struct_h5_a_c__cache__config__t.html#af99ca22b80e05fd5b3603806348ab647">min_size</a>, <a class="el" href="struct_h5_a_c__cache__config__t.html#af4728438dee601cb2554d9bf18d78a43">max_size</a>] (see below for a discussion of the <a class="el" href="struct_h5_a_c__cache__config__t.html#af99ca22b80e05fd5b3603806348ab647">min_size</a> and <a class="el" href="struct_h5_a_c__cache__config__t.html#af4728438dee601cb2554d9bf18d78a43">max_size</a> fields).</p>
<p>If you disable the adaptive cache resizing code (done by setting <a class="el" href="struct_h5_a_c__cache__config__t.html#ae825aaf759060239e92170d20eb97d26">incr_mode</a> to <a class="el" href="_h5_cpublic_8h.html#a040d488146ff1ca0a82209e9af3918faa6c1241ad99f2c18de01ea9407db860d4">H5C_incr__off</a>, <a class="el" href="struct_h5_a_c__cache__config__t.html#a0e25a1dc2c695bea335df0e23ed6363c">flash_incr_mode</a> to <a class="el" href="_h5_cpublic_8h.html#aaaa13ca7756d135b7df6d5a6779ee908a12938ef95720e1fd421ce77b1775c692">H5C_flash_incr__off</a>, and <a class="el" href="struct_h5_a_c__cache__config__t.html#a5df68196b281c19d8ab7da0788566aec">decr_mode</a> to <a class="el" href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04fa2f20b9f9f574b06d8320f138f0e6c319">H5C_decr__off</a>), you can use these fields to control maximum cache size manually, as the maximum cache size will remain at the initial size.</p>
<p>Note, that the maximum cache size is only modified when <a class="el" href="struct_h5_a_c__cache__config__t.html#a707198a337ece01a37dfec6e01275d82">set_initial_size</a> is <code>TRUE</code>. This allows the use of configurations specified at compile time to change resize configuration without altering the current maximum size of the cache. Without this feature, an additional call would be required to get the current maximum cache size so as to set the <a class="el" href="struct_h5_a_c__cache__config__t.html#a649236e7dd714855a50f122aa5caca9f">initial_size</a> to the current maximum cache size, and thereby avoid changing it.</p>
<p>The <a class="el" href="struct_h5_a_c__cache__config__t.html#abd805b98f873c1720f34a0ce937838fd">min_clean_fraction</a> sets the current minimum clean size as a fraction of the current max cache size. While this field was originally used only in the parallel version of the library, it now applies to the serial version as well. Its value must lie in the range <code>[0.0, 1.0]</code>. 0.01 is reasonable in the serial case, and 0.3 in the parallel.</p>
<p>A potential interaction, discovered at release 1.8.3, between the enforcement of the <a class="el" href="struct_h5_a_c__cache__config__t.html#abd805b98f873c1720f34a0ce937838fd">min_clean_fraction</a> and the adaptive cache resize code can severely degrade performance. While this interaction is easily dealt with in the serial case by setting <a class="el" href="struct_h5_a_c__cache__config__t.html#abd805b98f873c1720f34a0ce937838fd">min_clean_fraction</a> to 0.01, the problem is more difficult in the parallel case. Please see the Interactions section below for further details.</p>
<p>The <a class="el" href="struct_h5_a_c__cache__config__t.html#af4728438dee601cb2554d9bf18d78a43">max_size</a> and <a class="el" href="struct_h5_a_c__cache__config__t.html#af99ca22b80e05fd5b3603806348ab647">min_size</a> fields specify the range of maximum sizes that may be set for the cache by the automatic resize code. <a class="el" href="struct_h5_a_c__cache__config__t.html#af99ca22b80e05fd5b3603806348ab647">min_size</a> must be less than or equal to <a class="el" href="struct_h5_a_c__cache__config__t.html#af4728438dee601cb2554d9bf18d78a43">max_size</a>, and both must lie in the range <code>[H5C__MIN_MAX_CACHE_SIZE, H5C__MAX_MAX_CACHE_SIZE]</code> – currently [1 KB, 128 MB]. If you routinely run a cache size in the top half of this range, you should increase the hash table size. To do this, modify the <code>H5C__HASH_TABLE_LEN</code> <code>#define</code> in <code>H5Cpkg.h</code> and re-compile. At present, <code>H5C__HASH_TABLE_LEN</code> must be a power of two.</p>
<p>The <code>epoch_length</code> is the number of cache accesses between runs of the adaptive cache size control algorithms. It is ignored if these algorithms are turned off. It must lie in the range <code>[H5C__MIN_AR_EPOCH_LENGTH, H5C__MAX_AR_EPOCH_LENGTH]</code> – currently [100, 1000000]. The above constants are defined in <code>H5Cprivate.h</code>. 50000 is a reasonable value.</p>
<h2><a class="anchor" id="increment"></a>
Increment Configuration</h2>
<p>The <a class="el" href="struct_h5_a_c__cache__config__t.html#ae825aaf759060239e92170d20eb97d26">incr_mode</a> field specifies the cache size increment algorithm used. Its value must be a member of the <a class="el" href="_h5_cpublic_8h.html#a040d488146ff1ca0a82209e9af3918fa">H5C_cache_incr_mode</a> enum type – currently either <a class="el" href="_h5_cpublic_8h.html#a040d488146ff1ca0a82209e9af3918faa6c1241ad99f2c18de01ea9407db860d4">H5C_incr__off</a> or <a class="el" href="_h5_cpublic_8h.html#a040d488146ff1ca0a82209e9af3918faa7b5e2f56b49e6b10c9e0ba88b3ebb894">H5C_incr__threshold</a> (note the double underscores after <code>"incr"</code>). This type is defined in <a class="el" href="_h5_cpublic_8h.html">H5Cpublic.h</a>.</p>
<p>If <a class="el" href="struct_h5_a_c__cache__config__t.html#ae825aaf759060239e92170d20eb97d26">incr_mode</a> is set to <a class="el" href="_h5_cpublic_8h.html#a040d488146ff1ca0a82209e9af3918faa6c1241ad99f2c18de01ea9407db860d4">H5C_incr__off</a>, regular automatic cache size increases are disabled, and the <a class="el" href="struct_h5_a_c__cache__config__t.html#a20f323fcb4747fc7228d2d74bb965586">lower_hr_threshold</a>, <a class="el" href="struct_h5_a_c__cache__config__t.html#ac504dff76b24ab9f15536c51aec9fbbb">increment</a>, <a class="el" href="struct_h5_a_c__cache__config__t.html#a322f2a955e71ab7ede483e991e76f806">apply_max_increment</a>, and <a class="el" href="struct_h5_a_c__cache__config__t.html#ad5a729f1d611f2780679a35b3524052c">max_increment</a>, fields are ignored.</p>
<p>The <a class="el" href="struct_h5_a_c__cache__config__t.html#a0e25a1dc2c695bea335df0e23ed6363c">flash_incr_mode</a> field specifies the flash cache size increment algorithm used. Its value must be a member of the <a class="el" href="_h5_cpublic_8h.html#aaaa13ca7756d135b7df6d5a6779ee908">H5C_cache_flash_incr_mode</a> enum type – currently either <a class="el" href="_h5_cpublic_8h.html#aaaa13ca7756d135b7df6d5a6779ee908a12938ef95720e1fd421ce77b1775c692">H5C_flash_incr__off</a> or <a class="el" href="_h5_cpublic_8h.html#aaaa13ca7756d135b7df6d5a6779ee908a08f42339564848ec4d00e8565715e5e8">H5C_flash_incr__add_space</a> (note the double underscores after <code>"incr"</code>). This type is defined in <a class="el" href="_h5_cpublic_8h.html">H5Cpublic.h</a>.</p>
<p>If <a class="el" href="struct_h5_a_c__cache__config__t.html#a0e25a1dc2c695bea335df0e23ed6363c">flash_incr_mode</a> is set to <a class="el" href="_h5_cpublic_8h.html#aaaa13ca7756d135b7df6d5a6779ee908a12938ef95720e1fd421ce77b1775c692">H5C_flash_incr__off</a>, flash cache size increases are disabled, and the <a class="el" href="struct_h5_a_c__cache__config__t.html#a77b1812e0407c9122db524462a5c9633">flash_multiple</a>, and <a class="el" href="struct_h5_a_c__cache__config__t.html#a95fb1e03a77ef5c109d0c851416ced55">flash_threshold</a>, fields are ignored.</p>
<h3><a class="anchor" id="hrtcsic"></a>
Hit Rate Threshold Cache Size Increase Configuration</h3>
<p>If <a class="el" href="struct_h5_a_c__cache__config__t.html#ae825aaf759060239e92170d20eb97d26">incr_mode</a> is <a class="el" href="_h5_cpublic_8h.html#a040d488146ff1ca0a82209e9af3918faa7b5e2f56b49e6b10c9e0ba88b3ebb894">H5C_incr__threshold</a>, the cache size is increased via the hit rate threshold algorithm. The remaining fields in the section are then used as follows:</p>
<p><a class="el" href="struct_h5_a_c__cache__config__t.html#a20f323fcb4747fc7228d2d74bb965586">lower_hr_threshold</a> is the threshold below which the hit rate must fall to trigger an increase. The value must lie in the range <code>[0.0 - 1.0]</code>. In my tests, a relatively high value seems to work best – 0.9 for example.</p>
<p><a class="el" href="struct_h5_a_c__cache__config__t.html#ac504dff76b24ab9f15536c51aec9fbbb">increment</a> is the factor by which the old maximum cache size is multiplied to obtain an initial new maximum cache size when an increment is needed. The actual change in size may be smaller as required by <a class="el" href="struct_h5_a_c__cache__config__t.html#af4728438dee601cb2554d9bf18d78a43">max_size</a> (above) and <code>max_increment</code> (discussed below). increment must be greater than or equal to 1.0. If you set it to 1.0, you will effectively turn off the increment code. 2.0 is a reasonable value.</p>
<p><a class="el" href="struct_h5_a_c__cache__config__t.html#a322f2a955e71ab7ede483e991e76f806">apply_max_increment</a> and <a class="el" href="struct_h5_a_c__cache__config__t.html#ad5a729f1d611f2780679a35b3524052c">max_increment</a> allow the user to specify a maximum increment. If <a class="el" href="struct_h5_a_c__cache__config__t.html#a322f2a955e71ab7ede483e991e76f806">apply_max_increment</a> is <code>TRUE</code>, the cache size will never be increased by more than the number of bytes specified in <a class="el" href="struct_h5_a_c__cache__config__t.html#ad5a729f1d611f2780679a35b3524052c">max_increment</a> in any single increase.</p>
<h3><a class="anchor" id="fcsic"></a>
Flash Cache Size Increase Configuration</h3>
<p>If <a class="el" href="struct_h5_a_c__cache__config__t.html#a0e25a1dc2c695bea335df0e23ed6363c">flash_incr_mode</a> is set to <a class="el" href="_h5_cpublic_8h.html#aaaa13ca7756d135b7df6d5a6779ee908a08f42339564848ec4d00e8565715e5e8">H5C_flash_incr__add_space</a>, flash cache size increases are enabled. The size of the cache will be increased under the following circumstances:</p>
<p>Let <code>t</code> be the current maximum cache size times the value of the <a class="el" href="struct_h5_a_c__cache__config__t.html#a95fb1e03a77ef5c109d0c851416ced55">flash_threshold</a> field.</p>
<p>Let <code>x</code> be either the size of the newly inserted entry, the size of the newly loaded entry, or the number of bytes added to the size of the entry under consideration for triggering a flash cache size increase.</p>
<p>If <code>t < x</code>, the basic condition for a flash cache size increase is met, and we proceed as follows:</p>
<p>Let <code>space_needed</code> equal <code>x</code> less the amount of free space in the cache.</p>
<p>Further, let <a class="el" href="struct_h5_a_c__cache__config__t.html#ac504dff76b24ab9f15536c51aec9fbbb">increment</a> equal <code>space_needed</code> times the value of the <a class="el" href="struct_h5_a_c__cache__config__t.html#a77b1812e0407c9122db524462a5c9633">flash_multiple</a> field. If <a class="el" href="struct_h5_a_c__cache__config__t.html#ac504dff76b24ab9f15536c51aec9fbbb">increment</a> plus the current cache size is greater than <a class="el" href="struct_h5_a_c__cache__config__t.html#af4728438dee601cb2554d9bf18d78a43">max_size</a> (discussed above), reduce <a class="el" href="struct_h5_a_c__cache__config__t.html#ac504dff76b24ab9f15536c51aec9fbbb">increment</a> so that <a class="el" href="struct_h5_a_c__cache__config__t.html#ac504dff76b24ab9f15536c51aec9fbbb">increment</a> plus the current cache size is equal to <a class="el" href="struct_h5_a_c__cache__config__t.html#af4728438dee601cb2554d9bf18d78a43">max_size</a>.</p>
<p>If the increment is greater than zero, increase the current cache size by <a class="el" href="struct_h5_a_c__cache__config__t.html#ac504dff76b24ab9f15536c51aec9fbbb">increment</a>. To avoid confusing the other cache size increment or decrement algorithms, start a new epoch. Note, however, that we do not cycle the epoch markers if some variant of the age out algorithm is in use.</p>
<p>The use of the <a class="el" href="struct_h5_a_c__cache__config__t.html#a95fb1e03a77ef5c109d0c851416ced55">flash_threshold</a> field is discussed above. It must be a floating-point value in the range of <code>[0.1, 1.0]</code>. 0.25 is a reasonable value.</p>
<p>The use of the <a class="el" href="struct_h5_a_c__cache__config__t.html#a77b1812e0407c9122db524462a5c9633">flash_multiple</a> field is also discussed above. It must be a floating-point value in the range of <code>[0.1, 10.0]</code>. 1.4 is a reasonable value.</p>
<h2><a class="anchor" id="decrement"></a>
Decrement Configuration</h2>
<p>The <a class="el" href="struct_h5_a_c__cache__config__t.html#a5df68196b281c19d8ab7da0788566aec">decr_mode</a> field specifies the cache size decrement algorithm used. Its value must be a member of the <a class="el" href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04f">H5C_cache_decr_mode</a> enum type – currently either <a class="el" href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04fa2f20b9f9f574b06d8320f138f0e6c319">H5C_decr__off</a>, <a class="el" href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04fa7a646a7d80a674a72c25713eb0e6944a">H5C_decr__threshold</a>, <a class="el" href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04fa99d35c253f917810b5863bd84afb26a0">H5C_decr__age_out</a>, or <a class="el" href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04fae833c5cd6228580992471c1c16ad54d2">H5C_decr__age_out_with_threshold</a> (note the double underscores after <code>"decr"</code>). This type is defined in <a class="el" href="_h5_cpublic_8h.html">H5Cpublic.h</a>.</p>
<p>If <a class="el" href="struct_h5_a_c__cache__config__t.html#a5df68196b281c19d8ab7da0788566aec">decr_mode</a> is set to <a class="el" href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04fa2f20b9f9f574b06d8320f138f0e6c319">H5C_decr__off</a>, automatic cache size decreases are disabled, and the remaining fields in the cache size decrease control section are ignored.</p>
<h3><a class="anchor" id="hrtcsdc"></a>
Hit Rate Threshold Cache Size Decrease Configuration</h3>
<p>If <a class="el" href="struct_h5_a_c__cache__config__t.html#a5df68196b281c19d8ab7da0788566aec">decr_mode</a> is <a class="el" href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04fa7a646a7d80a674a72c25713eb0e6944a">H5C_decr__threshold</a>, the cache size is decreased by the threshold algorithm, and the remaining fields of the decrement section are used as follows:</p>
<p><a class="el" href="struct_h5_a_c__cache__config__t.html#a84a5ff4ac69196aa27c14f6f796db596">upper_hr_threshold</a> is the threshold above which the hit rate must rise to trigger cache size reduction. It must be in the range <code>[0.0, 1.0]</code>. In my synthetic tests, very high values like .9995 or .99995 seemed to work best.</p>
<p><a class="el" href="struct_h5_a_c__cache__config__t.html#a54007d3f2afb718b437f499a5c8b46d9">decrement</a> is the factor by which the current maximum cache size is multiplied to obtain a tentative new maximum cache size. It must lie in the range <code>[0.0, 1.0]</code>. Relatively large values like .9 seem to work best in my synthetic tests. Note that the actual size reduction may be smaller as required by <a class="el" href="struct_h5_a_c__cache__config__t.html#af99ca22b80e05fd5b3603806348ab647">min_size</a> and <a class="el" href="struct_h5_a_c__cache__config__t.html#a75e875a61c9da7f82482d0f6fe6e7152">max_decrement</a> (discussed below). <a class="el" href="struct_h5_a_c__cache__config__t.html#ac2c89a735413e4c36d6f463c4890ab42">apply_max_decrement</a> and <a class="el" href="struct_h5_a_c__cache__config__t.html#a75e875a61c9da7f82482d0f6fe6e7152">max_decrement</a> allow the user to specify a maximum decrement. If <a class="el" href="struct_h5_a_c__cache__config__t.html#ac2c89a735413e4c36d6f463c4890ab42">apply_max_decrement</a> is <code>TRUE</code>, the cache size will never be reduced by more than <a class="el" href="struct_h5_a_c__cache__config__t.html#a75e875a61c9da7f82482d0f6fe6e7152">max_decrement</a> bytes in any single reduction.</p>
<p>With the hit rate threshold cache size decrement algorithm, the remaining fields in the section are ignored.</p>
<h3><a class="anchor" id="dacsr"></a>
Ageout Cache Size Reduction</h3>
<p>If <a class="el" href="struct_h5_a_c__cache__config__t.html#a5df68196b281c19d8ab7da0788566aec">decr_mode</a> is <a class="el" href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04fa99d35c253f917810b5863bd84afb26a0">H5C_decr__age_out</a> the cache size is decreased by the ageout algorithm, and the remaining fields of the decrement section are used as follows:</p>
<p><a class="el" href="struct_h5_a_c__cache__config__t.html#a7ac41e345300bdecd9943e855d55b71b">epochs_before_eviction</a> is the number of epochs an entry must reside unaccessed in the cache before it is evicted. This value must lie in the range <code>[1, H5C__MAX_EPOCH_MARKERS]</code>. <code>H5C__MAX_EPOCH_MARKERS</code> is defined in H5Cprivate.h, and is currently set to 10.</p>
<p><a class="el" href="struct_h5_a_c__cache__config__t.html#ac2c89a735413e4c36d6f463c4890ab42">apply_max_decrement</a> and <a class="el" href="struct_h5_a_c__cache__config__t.html#a75e875a61c9da7f82482d0f6fe6e7152">max_decrement</a> are used as in section 2.4.3.1.</p>
<p><a class="el" href="struct_h5_a_c__cache__config__t.html#a6c5cff8e8a21230cc7f21859d8d6bd22">apply_emty_reserve</a> and <a class="el" href="struct_h5_a_c__cache__config__t.html#a9c1ae995513b55737aad09e11beff733">empty_reserve</a> allow the user to specify a minimum empty reserve as discussed in section 2.3.2.2. An empty reserve of 0.05 or 0.1 seems to work well.</p>
<p>The <a class="el" href="struct_h5_a_c__cache__config__t.html#a54007d3f2afb718b437f499a5c8b46d9">decrement</a> and <a class="el" href="struct_h5_a_c__cache__config__t.html#a84a5ff4ac69196aa27c14f6f796db596">upper_hr_threshold</a> fields are ignored in this case.</p>
<h3><a class="anchor" id="dawhrtcsr"></a>
Ageout With Hit Rate Threshold Cache Size Reduction</h3>
<p>If <a class="el" href="struct_h5_a_c__cache__config__t.html#a5df68196b281c19d8ab7da0788566aec">decr_mode</a> is <a class="el" href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04fae833c5cd6228580992471c1c16ad54d2">H5C_decr__age_out_with_threshold</a>, the cache size is decreased by the ageout with hit rate threshold algorithm, and the fields of decrement section are used as per the Ageout algorithm (see 5.3.2) with the exception of <a class="el" href="struct_h5_a_c__cache__config__t.html#a84a5ff4ac69196aa27c14f6f796db596">upper_hr_threshold</a>.</p>
<p>Here, <a class="el" href="struct_h5_a_c__cache__config__t.html#a84a5ff4ac69196aa27c14f6f796db596">upper_hr_threshold</a> is the threshold above which the hit rate must rise to trigger cache size reduction. It must be in the range <code>[0.0, 1.0]</code>. In my synthetic tests, high values like .999 seemed to work well.</p>
<h2><a class="anchor" id="parallel"></a>
Parallel Configuration</h2>
<p>This section is a catch-all for parallel specific configuration data. At present, it has only one field – <a class="el" href="struct_h5_a_c__cache__config__t.html#a8e3c2a2d300b7a8f8d3705fc5e59a3c1">dirty_bytes_threshold</a>.</p>
<p>In PHDF5, all operations that modify metadata must be executed collectively. We used to think that this was enough to ensure consistency across the metadata caches, but since we allow processes to read metadata individually, the order of dirty entries in the LRU list can vary across processes. This, in turn, can change the order in which dirty metadata cache entries reach the bottom of the LRU and are flushed to disk – opening the door to messages from the past and messages from the future bugs.</p>
<p>To prevent this, only the metadata cache on process 0 of the file communicator is allowed to write to file, and then only after entering a sync point with the other caches. After it writes entries to file, it sends the base addresses of the now clean entries to the other caches, so they can mark these entries clean as well, and then leaves the sync point. The other caches mark the specified entries as clean before they leave the sync point as well. (Observe, that since all caches see the same stream of dirty metadata, they will all have the same set of dirty entries upon sync point entry and exit.)</p>
<p>The different caches know when to synchronize by counting the number of bytes of dirty metadata created by the collective operations modifying metadata. Whenever this count exceeds the value specified in the <a class="el" href="struct_h5_a_c__cache__config__t.html#a8e3c2a2d300b7a8f8d3705fc5e59a3c1">dirty_bytes_threshold</a>, they all enter the sync point, and process 0 flushes down to its minimum clean size and sends the list of newly cleaned entries to the other caches.</p>
<p>Needless to say, the value of the <a class="el" href="struct_h5_a_c__cache__config__t.html#a8e3c2a2d300b7a8f8d3705fc5e59a3c1">dirty_bytes_threshold</a> field must be consistent across all the caches operating on a given file.</p>
<p>All dirty metadata can also by flushed under programmatic control via the <a class="el" href="group___h5_f.html#gae686870f0a276c4d06bbc667b2c24124" title="Flushes all buffers associated with a file to storage.">H5Fflush()</a> call. This call must be collective and will reset the dirty data counts on each metadata cache.</p>
<p>Absent calls to <a class="el" href="group___h5_f.html#gae686870f0a276c4d06bbc667b2c24124" title="Flushes all buffers associated with a file to storage.">H5Fflush()</a>, dirty metadata will only be flushed when the <a class="el" href="struct_h5_a_c__cache__config__t.html#a8e3c2a2d300b7a8f8d3705fc5e59a3c1">dirty_bytes_threshold</a> is exceeded, and then only down to the <a class="el" href="struct_h5_a_c__cache__config__t.html#abd805b98f873c1720f34a0ce937838fd">H5AC_cache_config_t.min_clean_fraction</a> "min_clean_fraction". Thus, if a program does all its metadata modifications in one phase, and then doesn't modify metadata thereafter, a residue of dirty metadata will be frozen in the metadata caches for the remainder of the computation – effectively reducing the sizes of the caches.</p>
<p>In the default configuration, the caches will eventually resize themselves to maintain an acceptable hit rate. However, this will take time, and it will increase the application's footprint in memory.</p>
<p>If your application behaves in this manner, you can avoid this by a collective call to <a class="el" href="group___h5_f.html#gae686870f0a276c4d06bbc667b2c24124" title="Flushes all buffers associated with a file to storage.">H5Fflush()</a> immediately after the metadata modification phase.</p>
<h2><a class="anchor" id="interactions"></a>
Interactions</h2>
<p>Evictions may not be disabled unless the automatic cache resize code is disabled as well (by setting <a class="el" href="struct_h5_a_c__cache__config__t.html#a5df68196b281c19d8ab7da0788566aec">decr_mode</a> to <a class="el" href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04fa2f20b9f9f574b06d8320f138f0e6c319">H5C_decr__off</a>, <code>flash_decr_mode</code> to <a class="el" href="_h5_cpublic_8h.html#aaaa13ca7756d135b7df6d5a6779ee908a08f42339564848ec4d00e8565715e5e8">H5C_flash_incr__add_space</a>, and <a class="el" href="struct_h5_a_c__cache__config__t.html#ae825aaf759060239e92170d20eb97d26">incr_mode</a> to <a class="el" href="_h5_cpublic_8h.html#a040d488146ff1ca0a82209e9af3918faa6c1241ad99f2c18de01ea9407db860d4">H5C_incr__off</a>) – thus placing the cache size under the direct control of the user program.</p>
<p>There is no logical necessity for this restriction. It is imposed because it simplifies testing greatly and because I can't see any reason why one would want to disable evictions while the automatic cache size adjustment code was enabled. This restriction can be relaxed if anyone can come up with a good reason to do so.</p>
<p>At present, there are two interactions between the increment and decrement sections of the configuration.</p>
<p>If <a class="el" href="struct_h5_a_c__cache__config__t.html#ae825aaf759060239e92170d20eb97d26">incr_mode</a> is <a class="el" href="_h5_cpublic_8h.html#a040d488146ff1ca0a82209e9af3918faa7b5e2f56b49e6b10c9e0ba88b3ebb894">H5C_incr__threshold</a>, and <a class="el" href="struct_h5_a_c__cache__config__t.html#a5df68196b281c19d8ab7da0788566aec">decr_mode</a> is either <a class="el" href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04fa7a646a7d80a674a72c25713eb0e6944a">H5C_decr__threshold</a> or <a class="el" href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04fae833c5cd6228580992471c1c16ad54d2">H5C_decr__age_out_with_threshold</a>, then <a class="el" href="struct_h5_a_c__cache__config__t.html#a20f323fcb4747fc7228d2d74bb965586">lower_hr_threshold</a> must be strictly less than <a class="el" href="struct_h5_a_c__cache__config__t.html#a84a5ff4ac69196aa27c14f6f796db596">upper_hr_threshold</a>.</p>
<p>Also, if the flash cache size increment code is enabled and is triggered, it will restart the current epoch without calling any other cache size increment or decrement code.</p>
<p>In both the serial and parallel cases, there is the potential for an interaction between the <a class="el" href="struct_h5_a_c__cache__config__t.html#abd805b98f873c1720f34a0ce937838fd">min_clean_fraction</a> and the cache size increment code that can severely degrade performance. Specifically, if the <a class="el" href="struct_h5_a_c__cache__config__t.html#abd805b98f873c1720f34a0ce937838fd">min_clean_fraction</a> is large enough, it is possible that keeping the specified fraction of the cache clean may generate enough flushes to seriously degrade performance even though the hit rate is excellent.</p>
<p>In the serial case, this is easily dealt with by selecting a very small <a class="el" href="struct_h5_a_c__cache__config__t.html#abd805b98f873c1720f34a0ce937838fd">min_clean_fraction</a> – 0.01 for example – as this still avoids the "metadata blizzard" phenomenon that appears when the cache fills with dirty metadata and must then flush all of it before evicting an entry to make space for a new entry.</p>
<p>The problem is more difficult in the parallel case, as the <a class="el" href="struct_h5_a_c__cache__config__t.html#abd805b98f873c1720f34a0ce937838fd">min_clean_fraction</a> is used to ensure that the cache contains clean entries that can be evicted to make space for new entries when metadata writes are forbidden – i.e. between sync points.</p>
<p>This issue was discovered shortly before release 1.8.3 and an automated solution has not been implemented. Should it become an issue for an application, try manually setting the cache size to ~1.5 times the maximum working set size for the application, and leave <a class="el" href="struct_h5_a_c__cache__config__t.html#abd805b98f873c1720f34a0ce937838fd">min_clean_fraction</a> set to 0.3.</p>
<p>You can approximate the working set size of your application via repeated calls to <a class="el" href="group___m_d_c.html#gacda6cbd60d3c50b59f801eba4e5a335f" title="Obtains current metadata cache size data for specified file.">H5Fget_mdc_size()</a> and <a class="el" href="group___m_d_c.html#gabea066c3fd924d2cf868ecee66a7c41f" title="Obtains target file's metadata cache hit rate.">H5Fget_mdc_hit_rate()</a> while running your program with the cache resize code enabled. The maximum value returned by <a class="el" href="group___m_d_c.html#gacda6cbd60d3c50b59f801eba4e5a335f" title="Obtains current metadata cache size data for specified file.">H5Fget_mdc_size()</a> should be a reasonable approximation – particularly if the associated hit rate is good. In the parallel case, there is also an interaction between <code>min_clean_fraction</code> and <a class="el" href="struct_h5_a_c__cache__config__t.html#a8e3c2a2d300b7a8f8d3705fc5e59a3c1">dirty_bytes_threshold</a>. Absent calls to <a class="el" href="group___h5_f.html#gae686870f0a276c4d06bbc667b2c24124" title="Flushes all buffers associated with a file to storage.">H5Fflush()</a> (discussed above), the upper bound on the amount of dirty data in the metadata caches will oscillate between (1 - <a class="el" href="struct_h5_a_c__cache__config__t.html#abd805b98f873c1720f34a0ce937838fd">min_clean_fraction</a>) times current maximum cache size, and that value plus the <a class="el" href="struct_h5_a_c__cache__config__t.html#a8e3c2a2d300b7a8f8d3705fc5e59a3c1">dirty_bytes_threshold</a>. Needless to say, it will be best if the <a class="el" href="struct_h5_a_c__cache__config__t.html#af99ca22b80e05fd5b3603806348ab647">min_size</a>, <a class="el" href="struct_h5_a_c__cache__config__t.html#abd805b98f873c1720f34a0ce937838fd">min_clean_fraction</a>, and the <a class="el" href="struct_h5_a_c__cache__config__t.html#a8e3c2a2d300b7a8f8d3705fc5e59a3c1">dirty_bytes_threshold</a> are chosen so that the cache can't fill with dirty data.</p>
<h2><a class="anchor" id="defaults"></a>
Default Metadata Cache Configuration</h2>
<p>Starting with release 1.8.3, HDF5 provides different default metadata cache configurations depending on whether the library is compiled for serial or parallel.</p>
<p>The default configuration for the serial case is as follows:</p>
<div class="fragment"><div class="line">{</div>
<div class="line"> <span class="comment">/* int version = */</span> H5C__CURR_AUTO_SIZE_CTL_VER,</div>
<div class="line"> <span class="comment">/* hbool_t rpt_fcn_enabled = */</span> FALSE,</div>
<div class="line"> <span class="comment">/* hbool_t open_trace_file = */</span> FALSE,</div>
<div class="line"> <span class="comment">/* hbool_t close_trace_file = */</span> FALSE,</div>
<div class="line"> <span class="comment">/* char trace_file_name[] = */</span> <span class="stringliteral">""</span>,</div>
<div class="line"> <span class="comment">/* hbool_t evictions_enabled = */</span> TRUE,</div>
<div class="line"> <span class="comment">/* hbool_t set_initial_size = */</span> TRUE,</div>
<div class="line"> <span class="comment">/* size_t initial_size = */</span> ( 2 * 1024 * 1024),</div>
<div class="line"> <span class="comment">/* double min_clean_fraction = */</span> 0.01,</div>
<div class="line"> <span class="comment">/* size_t max_size = */</span> (32 * 1024 * 1024),</div>
<div class="line"> <span class="comment">/* size_t min_size = */</span> ( 1 * 1024 * 1024),</div>
<div class="line"> <span class="comment">/* long int epoch_length = */</span> 50000,</div>
<div class="line"> <span class="comment">/* enum H5C_cache_incr_mode incr_mode = */</span> <a class="code" href="_h5_cpublic_8h.html#a040d488146ff1ca0a82209e9af3918faa7b5e2f56b49e6b10c9e0ba88b3ebb894">H5C_incr__threshold</a>,</div>
<div class="line"> <span class="comment">/* double lower_hr_threshold = */</span> 0.9,</div>
<div class="line"> <span class="comment">/* double increment = */</span> 2.0,</div>
<div class="line"> <span class="comment">/* hbool_t apply_max_increment = */</span> TRUE,</div>
<div class="line"> <span class="comment">/* size_t max_increment = */</span> (4 * 1024 * 1024),</div>
<div class="line"> <span class="comment">/* enum H5C_cache_flash_incr_mode */</span></div>
<div class="line"> <span class="comment">/* flash_incr_mode = */</span> <a class="code" href="_h5_cpublic_8h.html#aaaa13ca7756d135b7df6d5a6779ee908a08f42339564848ec4d00e8565715e5e8">H5C_flash_incr__add_space</a>,</div>
<div class="line"> <span class="comment">/* double flash_multiple = */</span> 1.4,</div>
<div class="line"> <span class="comment">/* double flash_threshold = */</span> 0.25,</div>
<div class="line"> <span class="comment">/* enum H5C_cache_decr_mode decr_mode = */</span> <a class="code" href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04fae833c5cd6228580992471c1c16ad54d2">H5C_decr__age_out_with_threshold</a>,</div>
<div class="line"> <span class="comment">/* double upper_hr_threshold = */</span> 0.999,</div>
<div class="line"> <span class="comment">/* double decrement = */</span> 0.9,</div>
<div class="line"> <span class="comment">/* hbool_t apply_max_decrement = */</span> TRUE,</div>
<div class="line"> <span class="comment">/* size_t max_decrement = */</span> (1 * 1024 * 1024),</div>
<div class="line"> <span class="comment">/* int epochs_before_eviction = */</span> 3,</div>
<div class="line"> <span class="comment">/* hbool_t apply_empty_reserve = */</span> TRUE,</div>
<div class="line"> <span class="comment">/* double empty_reserve = */</span> 0.1,</div>
<div class="line"> <span class="comment">/* int dirty_bytes_threshold = */</span> (256 * 1024)</div>
<div class="line">}</div>
<div class="ttc" id="a_h5_cpublic_8h_html_a040d488146ff1ca0a82209e9af3918faa7b5e2f56b49e6b10c9e0ba88b3ebb894"><div class="ttname"><a href="_h5_cpublic_8h.html#a040d488146ff1ca0a82209e9af3918faa7b5e2f56b49e6b10c9e0ba88b3ebb894">H5C_incr__threshold</a></div><div class="ttdeci">@ H5C_incr__threshold</div><div class="ttdef"><b>Definition:</b> H5Cpublic.h:30</div></div>
<div class="ttc" id="a_h5_cpublic_8h_html_a4f8534794ad9a977185a5d608c0af04fae833c5cd6228580992471c1c16ad54d2"><div class="ttname"><a href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04fae833c5cd6228580992471c1c16ad54d2">H5C_decr__age_out_with_threshold</a></div><div class="ttdeci">@ H5C_decr__age_out_with_threshold</div><div class="ttdef"><b>Definition:</b> H5Cpublic.h:52</div></div>
<div class="ttc" id="a_h5_cpublic_8h_html_aaaa13ca7756d135b7df6d5a6779ee908a08f42339564848ec4d00e8565715e5e8"><div class="ttname"><a href="_h5_cpublic_8h.html#aaaa13ca7756d135b7df6d5a6779ee908a08f42339564848ec4d00e8565715e5e8">H5C_flash_incr__add_space</a></div><div class="ttdeci">@ H5C_flash_incr__add_space</div><div class="ttdef"><b>Definition:</b> H5Cpublic.h:38</div></div>
</div><!-- fragment --><p>The default configuration for the parallel case is as follows:</p>
<div class="fragment"><div class="line">{</div>
<div class="line"> <span class="comment">/* int version = */</span> H5C__CURR_AUTO_SIZE_CTL_VER,</div>
<div class="line"> <span class="comment">/* hbool_t rpt_fcn_enabled = */</span> FALSE,</div>
<div class="line"> <span class="comment">/* hbool_t open_trace_file = */</span> FALSE,</div>
<div class="line"> <span class="comment">/* hbool_t close_trace_file = */</span> FALSE,</div>
<div class="line"> <span class="comment">/* char trace_file_name[] = */</span> <span class="stringliteral">""</span>,</div>
<div class="line"> <span class="comment">/* hbool_t evictions_enabled = */</span> TRUE,</div>
<div class="line"> <span class="comment">/* hbool_t set_initial_size = */</span> TRUE,</div>
<div class="line"> <span class="comment">/* size_t initial_size = */</span> ( 2 * 1024 * 1024),</div>
<div class="line"> <span class="comment">/* double min_clean_fraction = */</span> 0.3,</div>
<div class="line"> <span class="comment">/* size_t max_size = */</span> (32 * 1024 * 1024),</div>
<div class="line"> <span class="comment">/* size_t min_size = */</span> ( 1 * 1024 * 1024),</div>
<div class="line"> <span class="comment">/* long int epoch_length = */</span> 50000,</div>
<div class="line"> <span class="comment">/* enum H5C_cache_incr_mode incr_mode = */</span> <a class="code" href="_h5_cpublic_8h.html#a040d488146ff1ca0a82209e9af3918faa7b5e2f56b49e6b10c9e0ba88b3ebb894">H5C_incr__threshold</a>,</div>
<div class="line"> <span class="comment">/* double lower_hr_threshold = */</span> 0.9,</div>
<div class="line"> <span class="comment">/* double increment = */</span> 2.0,</div>
<div class="line"> <span class="comment">/* hbool_t apply_max_increment = */</span> TRUE,</div>
<div class="line"> <span class="comment">/* size_t max_increment = */</span> (4 * 1024 * 1024),</div>
<div class="line"> <span class="comment">/* enum H5C_cache_flash_incr_mode */</span></div>
<div class="line"> <span class="comment">/* flash_incr_mode = */</span> <a class="code" href="_h5_cpublic_8h.html#aaaa13ca7756d135b7df6d5a6779ee908a08f42339564848ec4d00e8565715e5e8">H5C_flash_incr__add_space</a>,</div>
<div class="line"> <span class="comment">/* double flash_multiple = */</span> 1.0,</div>
<div class="line"> <span class="comment">/* double flash_threshold = */</span> 0.25,</div>
<div class="line"> <span class="comment">/* enum H5C_cache_decr_mode decr_mode = */</span> <a class="code" href="_h5_cpublic_8h.html#a4f8534794ad9a977185a5d608c0af04fae833c5cd6228580992471c1c16ad54d2">H5C_decr__age_out_with_threshold</a>,</div>
<div class="line"> <span class="comment">/* double upper_hr_threshold = */</span> 0.999,</div>
<div class="line"> <span class="comment">/* double decrement = */</span> 0.9,</div>
<div class="line"> <span class="comment">/* hbool_t apply_max_decrement = */</span> TRUE,</div>
<div class="line"> <span class="comment">/* size_t max_decrement = */</span> (1 * 1024 * 1024),</div>
<div class="line"> <span class="comment">/* int epochs_before_eviction = */</span> 3,</div>
<div class="line"> <span class="comment">/* hbool_t apply_empty_reserve = */</span> TRUE,</div>
<div class="line"> <span class="comment">/* double empty_reserve = */</span> 0.1,</div>
<div class="line"> <span class="comment">/* int dirty_bytes_threshold = */</span> (256 * 1024)</div>
<div class="line">}</div>
</div><!-- fragment --><p>The default serial configuration should be adequate for most serial HDF5 users.</p>
<p>The same may not be true for the default parallel configuration due to the interaction between the <a class="el" href="struct_h5_a_c__cache__config__t.html#abd805b98f873c1720f34a0ce937838fd">min_clean_fraction</a> and the cache size increase code. See the Interactions section for further details.</p>
<p>Should you need to change the default configuration, it can be found in H5ACprivate.h. Look for the definition of H5AC__DEFAULT_RESIZE_CONFIG.</p>
<h1><a class="anchor" id="controlling"></a>
Controlling the New Metadata Cache Size From Your Program</h1>
<p>You have already seen how <a class="el" href="struct_h5_a_c__cache__config__t.html">H5AC_cache_config_t</a> has facilities that allow you to control the metadata cache size directly. Use <a class="el" href="group___m_d_c.html#gaa67f127242d4aaf244ae8ac4a1fe6a59" title="Obtains current metadata cache configuration for target file.">H5Fget_mdc_config()</a> and <a class="el" href="group___m_d_c.html#ga81bc06be69131484eb04d01511b9c8f8" title="Attempts to configure metadata cache of target file.">H5Fset_mdc_config()</a> to get and set the metadata cache configuration on an open file. Use <a class="el" href="group___f_a_p_l.html#ga3012f7f3310c7d25ada7617896bef1ee" title="Get the current initial metadata cache configuration from the provided file access property list.">H5Pget_mdc_config()</a> and <a class="el" href="group___f_a_p_l.html#gaf234199ad4cf9c708f45893f7f9cd4d3" title="Set the initial metadata cache configuration in the indicated File Access Property List to the suppli...">H5Pset_mdc_config()</a> to get and set the initial metadata cache configuration in a file access property list. Recall that this list contains configuration data used when opening a file.</p>
<p>Use <a class="el" href="group___m_d_c.html#gabea066c3fd924d2cf868ecee66a7c41f" title="Obtains target file's metadata cache hit rate.">H5Fget_mdc_hit_rate()</a> to get the average hit rate since the last time the hit rate stats were reset. This happens automatically at the beginning of each epoch if the adaptive cache resize code is enabled. You can also do it manually with <a class="el" href="group___m_d_c.html#ga6708886c2bb8740327d9078d7840197f" title="Resets hit rate statistics counters for the target file.">H5Freset_mdc_hit_rate_stats()</a>. Be careful about doing this if the adaptive cache resize code is enabled, as you may confuse it.</p>
<p>Use <a class="el" href="group___m_d_c.html#gacda6cbd60d3c50b59f801eba4e5a335f" title="Obtains current metadata cache size data for specified file.">H5Fget_mdc_size()</a> to get metadata cache size data on an open file.</p>
<p>Finally, note that cache size and cache footprint are two different things – in my tests, the cache footprint (as inferred from the UNIX <code>top</code> command) is typically about three times the maximum cache size. I haven't tracked it down yet, but I would guess that most of this is due to the very small typical cache entry size combined with the rather large size of the cache entry header structure. This should be investigated further, but there are other matters of higher priority.</p>
<h1><a class="anchor" id="news"></a>
New Metadata Cache Debugging Facilities</h1>
<p>The new metadata cache has a variety of debugging facilities that may be of use. I doubt that any other than the report function and the trace file will ever be accessible via the API, but they are relatively easy to turn on in the source code.</p>
<p>Note that none of this should be viewed as supported – it is described here on the off chance that you want to use it, but you are on your own if you do. Also, there are no promises as to consistency between versions.</p>
<p>As mentioned above, you can use the <a class="el" href="struct_h5_a_c__cache__config__t.html#aab04b3e770d1c2decc03f6fd94b64b17">rpt_fcn_enabled</a> field of the configuration structure to enable the default reporting function (H5C_def_auto_resize_rpt_fcn() in H5C.c). If this function doesn't work for you, you will have to write your own. In particular, remember that it uses <code>stdout</code>, so it will probably be unhappy under Windows.</p>
<p>Again, remember that this facility is not supported. Further, it is likely to change every time I do any serious work on the cache.</p>
<p>There is also an extensive statistics collection code. Use H5C_COLLECT_CACHE_STATS and H5C_COLLECT_CACHE_ENTRY_STATS in H5Cprivate.h to turn this on. If you also turn on H5AC_DUMP_STATS_ON_CLOSE in H5ACprivate.h, stats will be dumped when you close a file. Alternatively you can call H5C_stats() and H5C_stats__reset() within the library to dump and reset stats. Both of these functions are defined in H5C.c.</p>
<p>Finally, the cache also contains an extensive sanity checking code. Much of this is turned on when you compile in debug mode, but to enable the full suite, turn on H5C_DO_SANITY_CHECKS in H5Cprivate.h.</p>
<h1><a class="anchor" id="trouble"></a>
Trouble Shooting</h1>
<p>Absent major bugs in the cache, the only troubleshooting you should have to do is diagnosing and fixing problems with your cache configuration.</p>
<p>Assuming it runs on your platform (I've only used it under Linux), the reporting function is probably the most convenient diagnosis tool. However, since it is unsupported code, I will not discuss it further beyond directing you to the source (H5C_def_auto_resize_rpt_fcn() in H5C.c).</p>
<p>Absent the reporting function, regular calls to <a class="el" href="group___m_d_c.html#gabea066c3fd924d2cf868ecee66a7c41f" title="Obtains target file's metadata cache hit rate.">H5Fget_mdc_hit_rate()</a> should give you a good idea of the hit rate over time. Remember that the hit rate stats are reset at the end of each epoch (when adaptive cache resizing is enabled), so you should expect some jitter.</p>
<p>Similar calls to <a class="el" href="group___m_d_c.html#gacda6cbd60d3c50b59f801eba4e5a335f" title="Obtains current metadata cache size data for specified file.">H5Fget_mdc_size()</a> should allow you to monitor cache size and the fraction of the current maximum cache size that is actually in use.</p>
<p>If the hit rate is consistently low, and the cache it at its current maximum size, increasing the maximum size is an obvious fix.</p>
<p>If you see hit rate and cache size oscillations, try disabling adaptive cache resizing and setting a fixed cache size a bit greater than the high end of the cache size oscillations you observed.</p>
<p>If the hit rate oscillations don't go away, you are probably looking at a feature of your application that can't be helped without major changes to the cache. Please send along a description of the situation.</p>
<p>If the oscillations do go away, you may be able to come up with a configuration that deals with the situation. If that fails, control the cache size manually, and write to me, so I can try to develop an adaptive resize algorithm that works in your case.</p>
<p>Needless to say, you should give the cache a few epochs to adapt to circumstances. If that is too slow for you, try manual cache size control.</p>
<p>If you find it necessary to disable evictions, you may find it useful to enable the internal statistics collection code mentioned above in the section on debugging facilities.</p>
<p>Amongst many other things, the stats code will report the maximum cache size, and the average successful and unsuccessful search depths in the hash table. If these latter figures are significantly above 1, you should increase the size of the hash table. </p>
</div></div><!-- contents -->
</div><!-- PageDoc -->
</div><!-- doc-content -->
<!-- start footer part -->
<div id="nav-path" class="navpath"><!-- id is needed for treeview function! -->
<ul>
<li class="footer">Generated on Wed Sep 6 2023 00:28:32 for HDF5 by
<a href="http://www.doxygen.org/index.html">
<img class="footer" src="doxygen.png" alt="doxygen"/></a> 1.9.1 </li>
</ul>
</div>
</body>
</html>
|