From 9524dc494f0e62c7b6b82941c34552d2e2bb1283 Mon Sep 17 00:00:00 2001 From: Allen Byrne <50328838+byrnHDF@users.noreply.github.com> Date: Wed, 27 Jan 2021 08:15:34 -0600 Subject: Jan ws changes (#282) * OESS-98 convert plugin option to FetchContent, add tests * Fixes for pkcfg files because of plugin option * OESS-98 fix tools test for plugins * Keep doxygen comments under 100 chars long - format hint * Whitespace * HDFFV-11144 - Reclassify CMake messages * HDFFV-11099/11100 added help text * Reworked switch statement to compare string instead * Fix typo * Whitespace fixes except for H5Tprivate.h typo fix --- src/H5Fpublic.h | 182 ++++++++++++++++++++++++++++-------------------------- src/H5Spkg.h | 6 +- src/H5Spoint.c | 16 ++--- src/H5T.c | 2 +- src/H5TS.c | 3 +- src/H5Tprivate.h | 2 +- src/H5Tref.c | 2 +- src/H5Tvlen.c | 2 +- src/H5VL.c | 3 +- src/H5VLint.c | 2 +- src/H5VLprivate.h | 2 +- 11 files changed, 114 insertions(+), 108 deletions(-) diff --git a/src/H5Fpublic.h b/src/H5Fpublic.h index 9d6d349..dae486f 100644 --- a/src/H5Fpublic.h +++ b/src/H5Fpublic.h @@ -1002,7 +1002,8 @@ H5_DLL ssize_t H5Fget_file_image(hid_t file_id, void *buf_ptr, size_t buf_len); *
open_trace_file
trace_file_name
field should be used to
- * open a trace file for the cache. This field will always be set to 0
in this context.0
in this
+ * context.
* close_trace_file
trace_file_name
open_trace_file
field is
- * set to 1
. This field will always be set to the empty string in this context.1
. This field will always be set to the empty string in this context.evictions_enabled
incr_mode
\c H5C_incr__off: Automatic cache size increase is - * disabled.
\c H5C_incr__threshold: Automatic cache size increase is enabled using the hit rate - * threshold algorithm.
\c H5C_incr__off: Automatic cache size increase + * is disabled.
\c H5C_incr__threshold: Automatic cache size increase is enabled using the hit + * rate threshold algorithm.
*lower_hr_threshold
\c H5C_flash_incr__off: Flash cache size increase is * disabled.
\c H5C_flash_incr__add_space: Flash cache size increase is enabled using the add - * space algorithm.
flash_threshold
flash_multiple
max_size
field above.max_size
field above.
* decr_mode
H5C_decr__off: Automatic cache size decrease is disabled, and - * the remaining decrement fields are ignored.
H5C_decr__threshold: Automatic cache size decrease is - * enabled using the hit rate threshold algorithm.
H5C_decr__age_out: Automatic cache size decrease is - * enabled using the ageout algorithm.
H5C_decr__age_out_with_threshold: Automatic cache size decrease - * is enabled using the ageout with hit rate threshold algorithm
H5C_decr__off: Automatic cache size decrease is + * disabled, and the remaining decrement fields are ignored.
H5C_decr__threshold: Automatic + * cache size decrease is enabled using the hit rate threshold algorithm.
H5C_decr__age_out: + * Automatic cache size decrease is enabled using the ageout algorithm.
+ *H5C_decr__age_out_with_threshold: Automatic cache size decrease is enabled using the ageout + * with hit rate threshold algorithm
*upper_hr_threshold
Synchronization occurs whenever the * number of bytes of dirty metadata created since the last synchronization exceeds this - * limit.
rpt_fcn_enabled
0
). Since resize algorithm activity is
- * reported via stdout, it MUST be set to disabled (0
) on Windows machines.The report function - * is not supported code, and can be expected to change between versions of the library. Use it at your own - * risk.
0
) on Windows machines.The report + * function is not supported code, and can be expected to change between versions of the library. Use + * it at your own risk.
* *open_trace_File
trace_file_name
field should be used to open
* a trace file for the cache.The trace file is a debuging feature that allows the capture of top
- * level metadata cache requests for purposes of debugging and/or optimization. This field should normally be
- * set to 0
, as trace file collection imposes considerable overhead.
This field should only
- * be set to 1
when the trace_file_name
contains the full path of the desired trace
- * file, and either there is no open trace file on the cache, or the close_trace_file
- * field is also 1
.
The trace file feature is unsupported unless used at the direction of - * The HDF Group. It is intended to allow The HDF Group to collect a trace of cache activity in cases of - * occult failures and/or poor performance seen in the field, so as to aid in reproduction in the lab. If you - * use it absent the direction of The HDF Group, you are on your own.
This field should only normally be set to 0
, as trace file collection imposes
+ * considerable overhead.
1
when the trace_file_name
contains
+ * the full path of the desired trace file, and either there is no open trace file on the cache, or the
+ * close_trace_file
field is also 1
.The trace file feature is + * unsupported unless used at the direction of The HDF Group. It is intended to allow The HDF Group to + * collect a trace of cache activity in cases of occult failures and/or poor performance seen in the + * field, so as to aid in reproduction in the lab. If you use it absent the direction of The HDF Group, + * you are on your own.
*close_trace_file
See the
- * above comments on the open_trace_file
field. This field should be set to
- * 0
unless there is an open trace file on the cache that you wish to close.
The trace file - * feature is unsupported unless used at the direction of The HDF Group. It is intended to allow The HDF Group - * to collect a trace of cache activity in cases of occult failures and/or poor performance seen in the field, - * so as to aid in reproduction in the lab. If you use it absent the direction of The HDF Group, you are on - * your own.
open_trace_file
field. This field should be set to
+ * 0
unless there is an open trace file on the cache that you wish to close.The + * trace file feature is unsupported unless used at the direction of The HDF Group. It is intended to + * allow The HDF Group to collect a trace of cache activity in cases of occult failures and/or poor + * performance seen in the field, so as to aid in reproduction in the lab. If you use it absent the + * direction of The HDF Group, you are on your own.
*trace_file_name[]
open_trace_file
field is set
* to 1
.In the parallel case, an ascii representation of the mpi rank of the process - * will be appended to the file name to yield a unique trace file name for each process.
The length of - * the path must not exceed #H5AC__MAX_TRACE_FILE_NAME_LEN characters.
The trace file feature is - * unsupported unless used at the direction of The HDF Group. It is intended to allow The HDF Group - * to collect a trace of cache activity in cases of occult failures and/or poor performance seen in the field, - * so as to aid in reproduction in the lab. If you use it absent the direction of The HDF Group, you are on - * your own.
The + * length of the path must not exceed #H5AC__MAX_TRACE_FILE_NAME_LEN characters.
The trace file + * feature is unsupported unless used at the direction of The HDF Group. It is intended to allow The + * HDF Group to collect a trace of cache activity in cases of occult failures and/or poor performance + * seen in the field, so as to aid in reproduction in the lab. If you use it absent the direction of + * The HDF Group, you are on your own.
*evictions_enabled
1
).In rare circumstances, the raw data throughput
- * requirements may be so high that the user wishes to postpone metadata writes so as to reserve I/O
- * throughput for raw data. The evictions_enabled
field exists to allow this. However, this is an
- * extreme step, and you have no business doing it unless you have read the User Guide section on metadata
- * caching, and have considered all other options carefully.
The evictions_enabled
field
- * may not be set to disabled (0
) unless all adaptive cache resizing code is disabled via the
- * incr_mode
, flash_incr_mode
, and decr_mode
fields.
When this
- * flag is set to disabled (0
), the metadata cache will not attempt to evict entries to make
- * space for new entries, and thus will grow without bound.
Evictions will be re-enabled when
- * this field is set back to 1
. This should be done as soon as possible.
evictions_enabled
field exists to allow this. However,
+ * this is an extreme step, and you have no business doing it unless you have read the User Guide
+ * section on metadata caching, and have considered all other options carefully.The
+ * evictions_enabled
field may not be set to disabled (0
) unless all adaptive
+ * cache resizing code is disabled via the incr_mode
, flash_incr_mode
, and
+ * decr_mode
fields.
When this flag is set to disabled (0
), the
+ * metadata cache will not attempt to evict entries to make space for new entries, and thus will grow
+ * without bound.
Evictions will be re-enabled when this field is set back to 1
.
+ * This should be done as soon as possible.
set_initial_size
initial_size
set_initial_size
is set to 1
, then initial_size
must
* contain the desired initial size in bytes. This value must lie in the closed interval
- * [min_size, max_size]
. (see below)[min_size, max_size]
. (see below)
* min_clean_fraction
The value must lie in the interval [0.0, 1.0]. 0.01 is a good place to start in the serial - * case. In the parallel case, a larger value is needed -- see Metadata Caching in HDF5 in the collection "Advanced - * Topics in HDF5."
max_size
min_size
epoch_length
incr_mode
\c H5C_incr__off: Automatic cache size increase is disabled, - * and the remaining increment fields are ignored.
\c H5C_incr__threshold: Automatic cache size increase - * is enabled using the hit rate threshold algorithm.
\c H5C_incr__threshold: Automatic cache size + * increase is enabled using the hit rate threshold algorithm.
*lower_hr_threshold
When the - * hit rate over an epoch is below this threshold and the cache is full, the maximum size of the - * cache is multiplied by increment (below), and then clipped as necessary to stay within max_size, and - * possibly max_increment.
This field must lie in the interval [0.0, 1.0]. 0.8 or 0.9 is a good starting - * point.
This field must lie in the interval [0.0, 1.0]. 0.8 or 0.9 is a good + * starting point.
*increment
The actual cache size increase will be - * clipped to satisfy the max_size specified in the general configuration, and possibly max_increment - * below.
The parameter must be greater than or equal to 1.0 -- 2.0 is a reasonable value.
If you - * set it to 1.0, you will effectively disable cache size increases.
The parameter must be greater than or equal to 1.0 -- 2.0 is a reasonable + * value.
If you set it to 1.0, you will effectively disable cache size increases.
*apply_max_increment
flash_incr_mode
\c H5C_flash_incr__off: Flash cache size increase is - * disabled.
\c H5C_flash_incr__add_space: Flash cache size increase is enabled using the add - * space algorithm.
\c H5C_flash_incr__add_space: Flash cache size increase is enabled using the add + * space algorithm.
*flash_threshold
At present, this value - * must lie in the range [0.1, 1.0].
flash_multiple
max_size
field above.At present, this field must lie in the - * range [0.1, 10.0].
max_size
field above.At present, this field must lie in + * the range [0.1, 10.0].
*decr_mode
\c H5C_decr__off: Automatic cache size decrease is - * disabled.
\c H5C_decr__threshold: Automatic cache size decrease is enabled using the hit - * rate threshold algorithm.
\c H5C_decr__age_out: Automatic cache size decrease is enabled using the - * ageout algorithm.
\c H5C_decr__age_out_with_threshold: Automatic cache size decrease is enabled using - * the ageout with hit rate threshold algorithm
\c H5C_decr__threshold: Automatic cache size decrease is enabled using the hit + * rate threshold algorithm.
\c H5C_decr__age_out: Automatic cache size decrease is enabled using + * the ageout algorithm.
\c H5C_decr__age_out_with_threshold: Automatic cache size decrease is + * enabled using the ageout with hit rate threshold algorithm
* *upper_hr_threshold
When \c decr_mode is \c H5C_decr__threshold, and the hit rate over a given - * epoch exceeds the supplied threshold, the current maximum cache size is multiplied by decrement to obtain a - * tentative new (and smaller) maximum cache size.
When \c decr_mode is \c - * H5C_decr__age_out_with_threshold, there is no attempt to find and evict aged out entries unless the hit - * rate in the previous epoch exceeded the supplied threshold.
This field must lie in the interval - * [0.0, 1.0].
For \c H5C_incr__threshold, .9995 or .99995 is a good place to start.
For \c - * H5C_decr__age_out_with_threshold, .999 might be more useful.
When \c decr_mode is \c + * H5C_decr__age_out_with_threshold, there is no attempt to find and evict aged out entries unless the + * hit rate in the previous epoch exceeded the supplied threshold.
This field must lie in the + * interval [0.0, 1.0].
For \c H5C_incr__threshold, .9995 or .99995 is a good place to + * start.
For \c H5C_decr__age_out_with_threshold, .999 might be more useful.
*decrement
The actual - * cache size decrease will be clipped to satisfy the min_size specified in the general configuration, and - * possibly max_decrement below.
The parameter must be be in the interval [0.0, 1.0].
If you set - * it to 1.0, you will effectively disable cache size decreases. 0.9 is a reasonable starting point.
The parameter must be be in the interval + * [0.0, 1.0].
If you set it to 1.0, you will effectively disable cache size decreases. 0.9 is a + * reasonable starting point.
*apply_max_decrement
epochs_before_eviction
apply_empty_reserve
empty_reserve
When so directed, the - * ageout based algorithms will not decrease the maximum cache size unless the empty reserve can be - * met.
The parameter must lie in the interval [0.0, 1.0]. 0.1 or 0.05 is a good place to - * start.
The parameter must lie in the interval [0.0, 1.0]. 0.1 or 0.05 is a good place to + * start.
*dirty_bytes_threshold
Synchronization occurs whenever the - * number of bytes of dirty metadata created since the last synchronization exceeds this limit.
This - * field only applies to the parallel case. While it is ignored elsewhere, it can still draw a value out of - * bounds error.
It must be consistant across all caches on any given file.
By default, this field - * is set to 256 KB. It shouldn't be more than half the current maximum cache size times the minimum clean - * fraction.
This field only applies to the parallel case. While it is ignored elsewhere, it can + * still draw a value out of bounds error.
It must be consistant across all caches on any given + * file.
By default, this field is set to 256 KB. It shouldn't be more than half the current + * maximum cache size times the minimum clean fraction.
* * * @@ -1647,7 +1653,7 @@ H5_DLL herr_t H5Fget_info2(hid_t obj_id, H5F_info2_t *file_info); *