diff options
author | Gerd Heber <gheber@hdfgroup.org> | 2021-04-26 19:07:29 (GMT) |
---|---|---|
committer | GitHub <noreply@github.com> | 2021-04-26 19:07:29 (GMT) |
commit | 1d680fe04c545d601678d876ad22dea7bfc31edd (patch) | |
tree | f84a97c149df9ccf4409c5bfea04a7316c019e8a /src/H5FDlog.h | |
parent | 12082e728de7481144db7ea3cca6f38acf0a7cac (diff) | |
download | hdf5-1d680fe04c545d601678d876ad22dea7bfc31edd.zip hdf5-1d680fe04c545d601678d876ad22dea7bfc31edd.tar.gz hdf5-1d680fe04c545d601678d876ad22dea7bfc31edd.tar.bz2 |
Merge doxygen2 into develop (#553)
* Fixed warnings and started H5Epublic.h.
* Include H5FD* headers to correctly resolve references.
* Doxygen2 (#330)
* H5Eauto_is_v2.
* Added a few more calls.
* Added a few more H5E calls.
* First cut of H5E v2.
* Added the deprecated v1 calls.
* Updated spacing.
* Once more.
* Taking some inspiration from Eigen3.
* Add doxygen for the assigned functions: H5Pregister1,H5Pinsert1,H5Pen… (#352)
* Add doxygen for the assigned functions: H5Pregister1,H5Pinsert1,H5Pencode1, H5Pget_filter_by_id1,H5Pget_version, H5Pset_file_space,H5Pget_file_space. Someone already adds H5Pget_filter1. Also fixs an extra parameter 'close' call back function for HPregister2.
* doxygen work. fixs format by using clang-format.
* doxgen work for H5Pregister1 etc. Addressed Barbara and Gerd's comments.
For Quincey's comments, since we are not supposed to change the source code.
I leave this to future improvements.
* added documentation for H5P APIs (#350)
* add documenation for H5Pget_buffer,H5Pget_data_transform,H5Pget_edc_check,H5Pget_hyper_vector_size,H5Pget_preserve,H5Pget_type_conv_cb,H5Pget_vlen_mem_manager,H5Pset_btree_ratios
* format corrections
* fixed grammer
* fixed herr_t
* Better name.
* A fresh look.
* add doxygen to H5Ppublic.h
* use attention instead of warning
* Add doxygen comments in H5Ppublic.h (#375)
* Add doxygen comments in H5Ppublic.h
* H5Pset_meta_block_size
* H5Pset_metadata_read_attempts
* H5Pset_multi_type
* H5Pset_object_flush_cb
* H5Pset_sieve_buf_size
* H5Pset_small_data_block_size
* H5Pset_all_coll_metadata_ops
* H5Pget_all_coll_metadata_ops
* Add DOXYGEN_EXAMPLES_DIR to src/CMakeLists.txt
* Fix clang-format errors
* Fix filenames in doxygen/examples
* add doxygen to H5Ppublic.h (#378)
* add doxygen to H5Ppublic.h
* use attention instead of warning
Co-authored-by: Kimmy Mu <kmu@hdfgroup.org>
* Revert "add doxygen to H5Ppublic.h (#378)"
This reverts commit 2ee1821b138a5c00b15ea57ce9e950367480f5f2.
* Updated Doxygen variables.
* I forgot to copy two images.
* Enable desktop search by default.
* Add my assigned Doxygen documentation.
* Remove whitespace at EOL. Appease clang-format.
* Addressed Chris' comments.
* Added an alias for asynchronous functions.
* One space is enough for all of us.
* Slightly restructured RM page.
* address some issues
* reformatting
* Style external links.
* reformatting
* reformatting
* Added "Metadata Caching in HDF5" as a technical note example.
* Revise this soon!
* Added specification examples.
* Fixed references.
* Added H5AC cache image stuff and file format study.
* Added older FMT versions. Where did 1.0 go?
* Updated C/C++ note and replaced ambiguous labels.
* Reformat source with clang v10.0.1.
* Added the VFL technical note.
* Added what I believe might be called version 1.0 of the format.
* Added the remaining specs.
* Added H5Z callback documentation and fixed a few mistakes.
* Added dox for deprecated H5G calls and fixed a few snippet blockIDs.
* clang-format happy?
* Ok?
* Bonus track: Deprecated H5D functions.
* Carry over the more detailed group description.
* Added documentation for the missing and deprecated H5R calls.
* Life is easier and less repetitive w/ snippets. Use them!
* Eliminate the snippet block ID artifacts in the HTML rendering.
* Fixed snippet HTML artifacts and added a few missing calls.
* Under 20 H5Ps to go!
* Almost complete!
* "This is a form of pedantry up with which I will not put." (Churchill)
* Let's not waste as much space on bulleted lists!
* First complete (?) draft of the Doxygen-based RM.
* Completeness check and minor fixes along the way.
* Pedantry.
* Adding missing H5FD calls checkpoint.
* Pedantry.
* More pedantry.
* Added H5Pset_fapl_log.
* First draft of H5ES.
* Fixed warnings.
* Prep. for map module.
* First cut of the map module.
* Pedantry.
* Possible H5F introduction.
* Fix the indentation.
* Pedantry.
* Ditto.
* Thanks to the reviewers for their comments.
* Added missing images.
* Line numbers are a distraction here.
* More examples, references, and clean-up. Don't repeat yourself!
* Clang pedantry.
* Ditto.
* More reviewer comments...
* Templatized references and cleaned up \todos.
* Committing clang-format changes
* Fixed MANIFEST.
* Addressed Quincey's comments. (OCPLs)
* Fixed a few more \todo items.
* Fixed more \todo items.
* Added attribute life cycle.
* Forgot the examples file.
* Committing clang-format changes
* Pedantry.
* Live and learn!
* Added a sample H5D life cycle.
* Committing clang-format changes
* Pedantry.
Co-authored-by: kyang2014 <kyang2014@users.noreply.github.com>
Co-authored-by: Scot Breitenfeld <brtnfld@hdfgroup.org>
Co-authored-by: Kimmy Mu <kmu@hdfgroup.org>
Co-authored-by: Christopher Hogan <ChristopherHogan@users.noreply.github.com>
Co-authored-by: jya-kmu <53388330+jya-kmu@users.noreply.github.com>
Co-authored-by: David Young <dyoung@hdfgroup.org>
Co-authored-by: Larry Knox <lrknox@hdfgroup.org>
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Diffstat (limited to 'src/H5FDlog.h')
-rw-r--r-- | src/H5FDlog.h | 405 |
1 files changed, 404 insertions, 1 deletions
diff --git a/src/H5FDlog.h b/src/H5FDlog.h index aa1f3cb..969c091 100644 --- a/src/H5FDlog.h +++ b/src/H5FDlog.h @@ -65,7 +65,410 @@ extern "C" { #endif -H5_DLL hid_t H5FD_log_init(void); +H5_DLL hid_t H5FD_log_init(void); + +/** + * \ingroup FAPL + * + * \brief Sets up the logging virtual file driver (#H5FD_LOG) for use + * + * \fapl_id + * \param[in] logfile Name of the log file + * \param[in] flags Flags specifying the types of logging activity + * \param[in] buf_size The size of the logging buffers, in bytes (see description) + * \returns \herr_t + * + * \details H5Pset_fapl_log() modifies the file access property list to use the + * logging driver, #H5FD_LOG. The logging virtual file driver (VFD) is + * a clone of the standard SEC2 (#H5FD_SEC2) driver with additional + * facilities for logging VFD metrics and activity to a file. + * + * \p logfile is the name of the file in which the logging entries are + * to be recorded. + * + * The actions to be logged are specified in the parameter \p flags + * using the pre-defined constants described in the following + * table. Multiple flags can be set through the use of a logical \c OR + * contained in parentheses. For example, logging read and write + * locations would be specified as + * \Code{(H5FD_LOG_LOC_READ|H5FD_LOG_LOC_WRITE)}. + * + * <table> + * <caption>Table1: Logging Flags</caption> + * <tr> + * <td> + * #H5FD_LOG_LOC_READ + * </td> + * <td rowspan="3"> + * Track the location and length of every read, write, or seek operation. + * </td> + * </tr> + * <tr><td>#H5FD_LOG_LOC_WRITE</td></tr> + * <tr><td>#H5FD_LOG_LOC_SEEK</td></tr> + * <tr> + * <td> + * #H5FD_LOG_LOC_IO + * </td> + * <td> + * Track all I/O locations and lengths. The logical equivalent of the following: + * \Code{(#H5FD_LOG_LOC_READ | #H5FD_LOG_LOC_WRITE | #H5FD_LOG_LOC_SEEK)} + * </td> + * </tr> + * <tr> + * <td> + * #H5FD_LOG_FILE_READ + * </td> + * <td rowspan="2"> + * Track the number of times each byte is read or written. + * </td> + * </tr> + * <tr><td>#H5FD_LOG_FILE_WRITE</td></tr> + * <tr> + * <td> + * #H5FD_LOG_FILE_IO + * </td> + * <td> + * Track the number of times each byte is read and written. The logical + * equivalent of the following: + * \Code{(#H5FD_LOG_FILE_READ | #H5FD_LOG_FILE_WRITE)} + * </td> + * </tr> + * <tr> + * <td> + * #H5FD_LOG_FLAVOR + * </td> + * <td> + * Track the type, or flavor, of information stored at each byte. + * </td> + * </tr> + * <tr> + * <td> + * #H5FD_LOG_NUM_READ + * </td> + * <td rowspan="4"> + * Track the total number of read, write, seek, or truncate operations that occur. + * </td> + * </tr> + * <tr><td>#H5FD_LOG_NUM_WRITE</td></tr> + * <tr><td>#H5FD_LOG_NUM_SEEK</td></tr> + * <tr><td>#H5FD_LOG_NUM_TRUNCATE</td></tr> + * <tr> + * <td> + * #H5FD_LOG_NUM_IO + * </td> + * <td> + * Track the total number of all types of I/O operations. The logical equivalent + * of the following: + * \Code{(#H5FD_LOG_NUM_READ | #H5FD_LOG_NUM_WRITE | #H5FD_LOG_NUM_SEEK | #H5FD_LOG_NUM_TRUNCATE)} + * </td> + * </tr> + * <tr> + * <td> + * #H5FD_LOG_TIME_OPEN + * </td> + * <td rowspan="6"> + * Track the time spent in open, stat, read, write, seek, or close operations. + * </td> + * </tr> + * <tr><td>#H5FD_LOG_TIME_STAT</td></tr> + * <tr><td>#H5FD_LOG_TIME_READ</td></tr> + * <tr><td>#H5FD_LOG_TIME_WRITE</td></tr> + * <tr><td>#H5FD_LOG_TIME_SEEK</td></tr> + * <tr><td>#H5FD_LOG_TIME_CLOSE</td></tr> + * <tr> + * <td> + * #H5FD_LOG_TIME_IO + * </td> + * <td> + * Track the time spent in each of the above operations. The logical equivalent + * of the following: + * \Code{(#H5FD_LOG_TIME_OPEN | #H5FD_LOG_TIME_STAT | #H5FD_LOG_TIME_READ | #H5FD_LOG_TIME_WRITE | + * #H5FD_LOG_TIME_SEEK | #H5FD_LOG_TIME_CLOSE)} + * </td> + * </tr> + * <tr> + * <td> + * #H5FD_LOG_ALLOC + * </td> + * <td> + * Track the allocation of space in the file. + * </td> + * </tr> + * <tr> + * <td> + * #H5FD_LOG_ALL + * </td> + * <td> + * Track everything. The logical equivalent of the following: + * \Code{(#H5FD_LOG_ALLOC | #H5FD_LOG_TIME_IO | #H5FD_LOG_NUM_IO | #H5FD_LOG_FLAVOR | #H5FD_LOG_FILE_IO | + * #H5FD_LOG_LOC_IO)} + * </td> + * </tr> + * </table> + * The logging driver can track the number of times each byte in the file is + * read from or written to (using #H5FD_LOG_FILE_READ and #H5FD_LOG_FILE_WRITE) + * and what kind of data is at that location (e.g., metadata, raw data; using + * #H5FD_LOG_FLAVOR). This information is tracked in internal buffers of size + * buf_size, which must be at least the maximum size in bytes of the file to be + * logged while the log driver is in use.\n + * One buffer of size buf_size will be created for each of #H5FD_LOG_FILE_READ, + * #H5FD_LOG_FILE_WRITE and #H5FD_LOG_FLAVOR when those flags are set; these + * buffers will not grow as the file increases in size. + * + * \par Output: + * This section describes the logging driver (LOG VFD) output.\n + * The table, immediately below, describes output of the various logging driver + * flags and function calls. A list of valid flavor values, describing the type + * of data stored, follows the table. + * <table> + * <caption>Table2: Logging Output</caption> + * <tr> + * <th>Flag</th><th>VFD Call</th><th>Output and Comments</th> + * </th> + * </tr> + * <tr> + * <td>#H5FD_LOG_LOC_READ</td> + * <td>Read</td> + * <td> + * \Code{%10a-%10a (%10Zu bytes) (%s) Read}\n\n + * Start position\n + * End position\n + * Number of bytes\n + * Flavor of read\n\n + * Adds \Code{(\%f s)} and seek time if #H5FD_LOG_TIME_SEEK is also set. + * </td> + * </tr> + * <tr> + * <td>#H5FD_LOG_LOC_READ</td> + * <td>Read Error</td> + * <td> + * \Code{Error! Reading: %10a-%10a (%10Zu bytes)}\n\n + * Same parameters as non-error entry. + * </td> + * </tr> + * <tr> + * <td>#H5FD_LOG_LOC_WRITE</td> + * <td>Write</td> + * <td> + * \Code{%10a-%10a (%10Zu bytes) (%s) Written}\n\n + * Start position\n + * End position\n + * Number of bytes\n + * Flavor of write\n\n + * Adds \Code{(\%f s)} and seek time if #H5FD_LOG_TIME_SEEK is also set. + * </td> + * </tr> + * <tr> + * <td>#H5FD_LOG_LOC_WRITE</td> + * <td>Write Error</td> + * <td> + * \Code{Error! Writing: %10a-%10a (%10Zu bytes)}\n\n + * Same parameters as non-error entry. + * </td> + * </tr> + * <tr> + * <td>#H5FD_LOG_LOC_SEEK</td> + * <td>Read, Write</td> + * <td> + * \Code{Seek: From %10a-%10a}\n\n + * Start position\n + * End position\n\n + * Adds \Code{(\%f s)} and seek time if #H5FD_LOG_TIME_SEEK is also set. + * </td> + * </tr> + * <tr> + * <td>#H5FD_LOG_FILE_READ</td> + * <td>Close</td> + * <td> + * Begins with:\n + * Dumping read I/O information\n\n + * Then, for each range of identical values, there is this line:\n + * \Code{Addr %10-%10 (%10lu bytes) read from %3d times}\n\n + * Start address\n + * End address\n + * Number of bytes\n + * Number of times read\n\n + * Note: The data buffer is scanned and each range of identical values + * gets one entry in the log file to save space and make it easier to read. + * </td> + * </tr> + * <tr> + * <td>#H5FD_LOG_FILE_WRITE</td> + * <td>Close</td> + * <td> + * Begins with:\n + * Dumping read I/O information\n\n + * Then, for each range of identical values, there is this line:\n + * \Code{Addr %10-%10 (%10lu bytes) written to %3d times}\n\n + * Start address\n + * End address\n + * Number of bytes\n + * Number of times written\n\n + * Note: The data buffer is scanned and each range of identical values + * gets one entry in the log file to save space and make it easier to read. + * </td> + * </tr> + * <tr> + * <td>#H5FD_LOG_FLAVOR</td> + * <td>Close</td> + * <td> + * Begins with:\n + * Dumping I/O flavor information\n\n + * Then, for each range of identical values, there is this line:\n + * \Code{Addr %10-%10 (%10lu bytes) flavor is %s}\n\n + * Start address\n + * End address\n + * Number of bytes\n + * Flavor\n\n + * Note: The data buffer is scanned and each range of identical values + * gets one entry in the log file to save space and make it easier to read. + * </td> + * </tr> + * <tr> + * <td>#H5FD_LOG_NUM_READ</td> + * <td>Close</td> + * <td> + * Total number of read operations: \Code{%11u} + * </td> + * </tr> + * <tr> + * <td>#H5FD_LOG_NUM_WRITE</td> + * <td>Close</td> + * <td> + * Total number of write operations: \Code{%11u} + * </td> + * </tr> + * <tr> + * <td>#H5FD_LOG_NUM_SEEK</td> + * <td>Close</td> + * <td> + * Total number of seek operations: \Code{%11u} + * </td> + * </tr> + * <tr> + * <td>#H5FD_LOG_NUM_TRUNCATE</td> + * <td>Close</td> + * <td> + * Total number of truncate operations: \Code{%11u} + * </td> + * </tr> + * <tr> + * <td>#H5FD_LOG_TIME_OPEN</td> + * <td>Open</td> + * <td> + * Open took: \Code{(\%f s)} + * </td> + * </tr> + * <tr> + * <td>#H5FD_LOG_TIME_READ</td> + * <td>Close, Read</td> + * <td> + * Total time in read operations: \Code{\%f s}\n\n + * See also: #H5FD_LOG_LOC_READ + * </td> + * </tr> + * </tr> + * <tr> + * <td>#H5FD_LOG_TIME_WRITE</td> + * <td>Close, Write</td> + * <td> + * Total time in write operations: \Code{\%f s}\n\n + * See also: #H5FD_LOG_LOC_WRITE + * </td> + * </tr> + * <tr> + * <td>#H5FD_LOG_TIME_SEEK</td> + * <td>Close, Read, Write</td> + * <td> + * Total time in write operations: \Code{\%f s}\n\n + * See also: #H5FD_LOG_LOC_SEEK or #H5FD_LOG_LOC_WRITE + * </td> + * </tr> + * <tr> + * <td>#H5FD_LOG_TIME_CLOSE</td> + * <td>Close</td> + * <td> + * Close took: \Code{(\%f s)} + * </td> + * </tr> + * <tr> + * <td>#H5FD_LOG_TIME_STAT</td> + * <td>Open</td> + * <td> + * Stat took: \Code{(\%f s)} + * </td> + * </tr> + * <tr> + * <td>#H5FD_LOG_ALLOC</td> + * <td>Alloc</td> + * <td> + * \Code{%10-%10 (%10Hu bytes) (\%s) Allocated}\n\n + * Start of address space\n + * End of address space\n + * Total size allocation\n + * Flavor of allocation + * </td> + * </tr> + * </table> + * + * \par Flavors: + * The \Emph{flavor} describes the type of stored information. The following + * table lists the flavors that appear in log output and briefly describes each. + * These terms are provided here to aid in the construction of log message + * parsers; a full description is beyond the scope of this document. + * <table> + * <caption>Table3: Flavors of logged data</caption> + * <tr> + * <th>Flavor</th><th>Description</th> + * </th> + * </tr> + * <tr> + * <td>#H5FD_MEM_NOLIST</td> + * <td>Error value</td> + * </tr> + * <tr> + * <td>#H5FD_MEM_DEFAULT</td> + * <td>Value not yet set.\n + * May also be a datatype set in a larger allocation that will be + * suballocated by the library.</td> + * </tr> + * <tr> + * <td>#H5FD_MEM_SUPER</td> + * <td>Superblock data</td> + * </tr> + * <tr> + * <td>#H5FD_MEM_BTREE</td> + * <td>B-tree data</td> + * </tr> + * <tr> + * <td>#H5FD_MEM_DRAW</td> + * <td>Raw data (for example, contents of a dataset)</td> + * </tr> + * <tr> + * <td>#H5FD_MEM_GHEAP</td> + * <td>Global heap data</td> + * </tr> + * <tr> + * <td>#H5FD_MEM_LHEAP</td> + * <td>Local heap data</td> + * </tr> + * <tr> + * <td>#H5FD_MEM_OHDR</td> + * <td>Object header data</td> + * </tr> + * </table> + * + * \version 1.8.7 The flags parameter has been changed from \Code{unsigned int} + * to \Code{unsigned long long}. + * The implementation of the #H5FD_LOG_TIME_OPEN, #H5FD_LOG_TIME_READ, + * #H5FD_LOG_TIME_WRITE, and #H5FD_LOG_TIME_SEEK flags has been finished. + * New flags were added: #H5FD_LOG_NUM_TRUNCATE and #H5FD_LOG_TIME_STAT. + * \version 1.6.0 The \c verbosity parameter has been removed. + * Two new parameters have been added: \p flags of type \Code{unsigned} and + * \p buf_size of type \Code{size_t}. + * \since 1.4.0 + * + */ H5_DLL herr_t H5Pset_fapl_log(hid_t fapl_id, const char *logfile, unsigned long long flags, size_t buf_size); #ifdef __cplusplus |