summaryrefslogtreecommitdiffstats
path: root/doc/vfd-swmr-user-guide.md
diff options
context:
space:
mode:
authorDavid Young <dyoung@pobox.com>2020-08-25 15:46:36 (GMT)
committerDavid Young <dyoung@pobox.com>2020-08-25 15:46:36 (GMT)
commite1419c872e27a4178e3e6eabf78f48544d7f03ca (patch)
treefecf62b3a1d66511ffc3bc22eb1821885d63271a /doc/vfd-swmr-user-guide.md
parentda4c72a138f0e418d3f72d6f095f5f40c92d9648 (diff)
downloadhdf5-e1419c872e27a4178e3e6eabf78f48544d7f03ca.zip
hdf5-e1419c872e27a4178e3e6eabf78f48544d7f03ca.tar.gz
hdf5-e1419c872e27a4178e3e6eabf78f48544d7f03ca.tar.bz2
Do not use first person singular in the SWMR demos section.
In the `vfd_swmr_create_fapl()` dissection, change the /** **/ comments in the literal code to plain markdown paragraphs. Slightly change wording and markdown elsewhere.
Diffstat (limited to 'doc/vfd-swmr-user-guide.md')
-rw-r--r--doc/vfd-swmr-user-guide.md203
1 files changed, 102 insertions, 101 deletions
diff --git a/doc/vfd-swmr-user-guide.md b/doc/vfd-swmr-user-guide.md
index 894d523..69b7f96 100644
--- a/doc/vfd-swmr-user-guide.md
+++ b/doc/vfd-swmr-user-guide.md
@@ -137,28 +137,26 @@ dataset on three source datasets, each in a different HDF5 file.
## The VFD SWMR demos
-TBD: edit this section
-
-My repository containing the VFD SWMR
-demos is https://bitbucket.hdfgroup.org/scm/~dyoung/swmr-demo.git .
-They are the same two demos that were shown at the webinar.
+The VFD SWMR demos are in a [separate
+repository](https://bitbucket.hdfgroup.org/scm/~dyoung/swmr-demo.git).
Before you build the demos, you will need to install the HDF5 library
-and utilities built from our VFD SWMR branch in your home directory
-somewhere. In the ./configure step, I set the install directory using
-a --prefix=$HOME/path/for/library command-line option, but there may be
-another way. Update the H5CC lines in the Makefiles with the path to
-h5cc. Then you should be able to `make`, `make clean`, etc.
-
-Under gaussians/, two programs are built, `wgaussians` and `rgaussians`.
-If you start both from the same directory in different terminals, you
-should see the "bouncing 2-D Gaussian distributions" in the `rgaussians`
-terminal.
-
-The creation-deletion (`credel`) demo is also run in two terminals. The
-two command lines are given in `credel/README.md`. You need to use the
-`h5ls` installed from the VFD SWMR branch, since only that version has the
-`--poll` option.
+and utilities built from the VFD SWMR branch in your home directory
+somewhere. In the ./configure step, use the command-line option
+`--prefix=$HOME/path/for/library` to set the directory you prefer.
+In the demo Makefiles, update the `H5CC` variable with the path to
+the `h5cc` installed from the VFD SWMR branch. Then you should be
+able to `make` and `make clean` the demos.
+
+Under `gaussians/`, two programs are built, `wgaussians` and
+`rgaussians`. If you start both from the same directory in different
+terminals, you should see the "bouncing 2-D Gaussian distributions"
+in the `rgaussians` terminal.
+
+The creation-deletion (`credel`) demo is also run in two terminals.
+The two command lines are given in `credel/README.md`. You need
+to use the `h5ls` installed from the VFD SWMR branch, since only
+that version has the `--poll` option.
# Developer tips
@@ -191,24 +189,26 @@ vfd_swmr_create_fapl(bool writer, bool only_meta_pages, bool use_vfd_swmr)
H5F_vfd_swmr_config_t config;
hid_t fapl;
-/**
- ** h5_fileaccess() is also a helper routine for the tests.
- ** In your program, you can replace the h5_fileaccess() call with a call to
- ** H5Pcreate(H5P_FILE_ACCESS).
- **/
+```
+
+`h5_fileaccess()` is also a helper routine for the tests. In your
+program, you can replace the `h5_fileaccess()` call with a call to
+`H5Pcreate(H5P_FILE_ACCESS)`.
+```
/* Create file access property list */
if((fapl = h5_fileaccess()) < 0) {
warnx("h5_fileaccess");
return badhid;
}
+```
-/**
- ** VFD SWMR has only been tested with the latest file format. It may
- ** malfunction with older formats, we just don't know. We force the
- ** latest version here.
- **/
+VFD SWMR has only been tested with the latest file format. It may
+malfunction with older formats, we just don't know. We force the
+latest version here.
+
+```
/* FOR NOW: set to use latest format, the "old" parameter is not used */
if(H5Pset_libver_bounds(fapl, H5F_LIBVER_LATEST, H5F_LIBVER_LATEST) < 0) {
warnx("H5Pset_libver_bounds");
@@ -218,84 +218,86 @@ vfd_swmr_create_fapl(bool writer, bool only_meta_pages, bool use_vfd_swmr)
/*
* Set up to open the file with VFD SWMR configured.
*/
+```
+
+VFD SWMR relies on metadata reads and writes to go through the
+page buffer. Note that the default page size is 4096 bytes. This
+call sets the total page buffer size to 4096 bytes. So we have
+effectively created a one-page page buffer! That is adequate for
+testing, but it may not be best for your application.
-/**
- ** VFD SWMR relies on metadata reads and writes to go through the
- ** page buffer. Note that the default page size is 4096 bytes. This
- ** call sets the total page buffer size to 4096 bytes. So we have
- ** effectively created a one-page page buffer! That is adequate for
- ** testing, but it may not be best for your application.
- **
- ** If `only_meta_pages` is true, then the entire page buffer is
- ** dedicated to metadata. That's fine for VFD SWMR.
- **
- ** NOTE WELL: when VFD SWMR is enabled, the meta-/raw-data pages proportion
- ** set by H5Pset_page_buffer_size() does not actually control the
- ** pages reserved for raw data. *All* pages are dedicated to buffering
- ** metadata.
- **/
+If `only_meta_pages` is true, then the entire page buffer is
+dedicated to metadata. That's fine for VFD SWMR.
+*Note well*: when VFD SWMR is enabled, the meta-/raw-data pages proportion
+set by `H5Pset_page_buffer_size()` does not actually control the
+pages reserved for raw data. *All* pages are dedicated to buffering
+metadata.
+
+
+```
/* Enable page buffering */
if(H5Pset_page_buffer_size(fapl, 4096, only_meta_pages ? 100 : 0, 0) < 0) {
warnx("H5Pset_page_buffer_size");
return badhid;
}
+```
-/**
- ** Add VFD SWMR-specific configuration to the file-access property list
- ** (`fapl`) using an H5Pset_vfd_swmr_config() call.
- **
- ** When VFD SWMR is enabled, changes to the HDF5 metadata accumulate in
- ** RAM until a configurable unit of time known as a *tick* has passed.
- ** At the end of each tick, a snapshot of the metadata at the end of
- ** the tick is "published"---that is, made visible to the readers.
- **
- ** The length of a *tick* is configurable in units of 100 milliseconds
- ** using the `tick_len` parameter. Below, `tick_len` is set to `4` to
- ** select a tick length of 400ms.
- **
- ** A snapshot does not persist forever, but it expires after a number
- ** of ticks, given by the *maximum lag*, has passed. Below, `max_lag`
- ** is set to `7` to select a maximum lag of 7 ticks. After a snapshot
- ** has expired, the writer may overwrite it.
- **
- ** When a reader first enters the API, it starts to use, or "selects,"
- ** the metadata in the newest snapshot, and on every subsequent API
- ** entry, if a tick has passed since the last selection, and if new
- ** snapshots are available, then the reader selects the latest.
- **
- ** If a reader spends longer than `max_lag - 1` ticks (2400ms with
- ** the example configuration) inside the HDF5 API, then its snapshot
- ** may expire, resulting in undefined behavior. When a snapshot
- ** expires while the reader is using it, we say that the writer has
- ** "overrun" the reader. The writer cannot currently detect overruns.
- ** Frequently the reader will detect an overrun and force the program
- ** to exit with a diagnostic assertion failure.
- **
- ** The application tells VFD SWMR whether or not to configure for
- ** reading or writing a file by setting the `writer` parameter to
- ** `true` for writing or `false` for reading.
- **
- ** VFD SWMR snapshots are stored in a "shadow file" that is shared
- ** between writer and readers. On a POSIX system, the shadow file
- ** may be placed on any *local* filesystem that the reader and writer
- ** share. The `md_file_path` parameter tells where to put the shadow
- ** file.
- **
- ** The `md_pages_reserved` parameter tells how many pages to reserve
- ** at the beginning of the shadow file for the shadow-file header
- ** and the shadow index. The header has an entire page to itself.
- ** The remaining `md_pages_reserved - 1` pages are reserved for the
- ** shadow index. If the index grows larger than its initial
- ** allocation, then it will move to a new location in the shadow file,
- ** and the initial allocation will be reclaimed. `md_pages_reserved`
- ** must be at least 2.
- **
- ** The `version` parameter tells what version of VFD SWMR configuration
- ** the parameter struct `config` contains. For now, it should be
- ** initialized to `H5F__CURR_VFD_SWMR_CONFIG_VERSION`.
- **/
+Add VFD SWMR-specific configuration to the file-access property list
+(`fapl`) using an `H5Pset_vfd_swmr_config()` call.
+
+When VFD SWMR is enabled, changes to the HDF5 metadata accumulate in
+RAM until a configurable unit of time known as a *tick* has passed.
+At the end of each tick, a snapshot of the metadata at the end of
+the tick is "published"---that is, made visible to the readers.
+
+The length of a *tick* is configurable in units of 100 milliseconds
+using the `tick_len` parameter. Below, `tick_len` is set to `4` to
+select a tick length of 400ms.
+
+A snapshot does not persist forever, but it expires after a number
+of ticks, given by the *maximum lag*, has passed. Below, `max_lag`
+is set to `7` to select a maximum lag of 7 ticks. After a snapshot
+has expired, the writer may overwrite it.
+
+When a reader first enters the API, it starts to use, or "selects,"
+the metadata in the newest snapshot, and on every subsequent API
+entry, if a tick has passed since the last selection, and if new
+snapshots are available, then the reader selects the latest.
+
+If a reader spends longer than `max_lag - 1` ticks (2400ms with
+the example configuration) inside the HDF5 API, then its snapshot
+may expire, resulting in undefined behavior. When a snapshot
+expires while the reader is using it, we say that the writer has
+"overrun" the reader. The writer cannot currently detect overruns.
+Frequently the reader will detect an overrun and force the program
+to exit with a diagnostic assertion failure.
+
+The application tells VFD SWMR whether or not to configure for
+reading or writing a file by setting the `writer` parameter to
+`true` for writing or `false` for reading.
+
+VFD SWMR snapshots are stored in a "shadow file" that is shared
+between writer and readers. On a POSIX system, the shadow file
+may be placed on any *local* filesystem that the reader and writer
+share. The `md_file_path` parameter tells where to put the shadow
+file.
+
+The `md_pages_reserved` parameter tells how many pages to reserve
+at the beginning of the shadow file for the shadow-file header
+and the shadow index. The header has an entire page to itself.
+The remaining `md_pages_reserved - 1` pages are reserved for the
+shadow index. If the index grows larger than its initial
+allocation, then it will move to a new location in the shadow file,
+and the initial allocation will be reclaimed. `md_pages_reserved`
+must be at least 2.
+
+The `version` parameter tells what version of VFD SWMR configuration
+the parameter struct `config` contains. For now, it should be
+initialized to `H5F__CURR_VFD_SWMR_CONFIG_VERSION`.
+
+```
memset(&config, 0, sizeof(config));
config.version = H5F__CURR_VFD_SWMR_CONFIG_VERSION;
@@ -319,8 +321,7 @@ vfd_swmr_create_fapl(bool writer, bool only_meta_pages, bool use_vfd_swmr)
An application may want to use VFD SWMR to create, read, or write
a virtual dataset. Unfortunately, VDS does not work properly with
VFD SWMR at this time. In this section, we describe some workarounds
-that can be used with great care to make VDS and VFD SWMR operate
-simultaneously.
+that can be used with great care to make VDS and VFD SWMR cooperate.
A virtual dataset, when it is read or written, will open files on
an application's behalf in order to access the source datasets