diff options
| author | Gerd Heber <gheber@hdfgroup.org> | 2021-11-23 14:05:40 (GMT) |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2021-11-23 14:05:40 (GMT) |
| commit | b9e4deec10cc943a7ecb3ac6bc6bd6695b2f33a0 (patch) | |
| tree | 6e9c6b646c9bc28b7e3ff3282b99101edea0508e /doxygen/examples/IOFlow.html | |
| parent | 4b9ca8e1f62c36b8ab5eb803b72df2b0b6fed548 (diff) | |
| download | hdf5-b9e4deec10cc943a7ecb3ac6bc6bd6695b2f33a0.zip hdf5-b9e4deec10cc943a7ecb3ac6bc6bd6695b2f33a0.tar.gz hdf5-b9e4deec10cc943a7ecb3ac6bc6bd6695b2f33a0.tar.bz2 | |
Next batch of Doxygen updates. (#1180)
* Sketch of the H5S life cycle.
* Committing clang-format changes
* Fix H5S_UNLIMITED snafu.
* Updated RM template and RM page.
* Added H5S life cycle.
* Committing clang-format changes
* Added H5T life cycle.
* Committing clang-format changes
* Cleaner layout (?)
* Cleaned the H5F life cycle. Called out unfinished biz.
* Committing clang-format changes
* Remaining life cycle skeletons.
* Committing clang-format changes
* Committing clang-format changes
* Added H5Z life cycle.
* Committing clang-format changes
* Added H5G life cycle.
* Committing clang-format changes
* H5 and H5I life cycle updates.
* Committing clang-format changes
* Added H5PL life cycle.
* Committing clang-format changes
* Added H5L life cycle.
* Committing clang-format changes
* Fix for Chris' comment.
* Add a variable for Doxygen pre-processor definitions.
* Forgot to add the H5M API.
* Clarify the H5Z life cycle.
* Committing clang-format changes
* Add H5Zdevelop.h to Doxygen.in. Added H5I life cycle.
* Committing clang-format changes
* Clarified introduction and fixed missing label declaration.
* Added H5O life cycle.
* Committing clang-format changes
* H5O cleanup, part 1.
* Committing clang-format changes
* Cleaned up some of the endless repetition in H5O.
* Committing clang-format changes
* Cookbook & RFC draft layouts.
* Updated manifest.
* Updated the manifest, the example paths, and sketched the 1st recipe.
* Committing clang-format changes
* Outlined two more recipes.
* Committing clang-format changes
* More recipes and RFCs.
* Committing clang-format changes
* Draft of templatized RFC references.
* Another batch of RFC changes.
* Another batch of RFCs.
* Fixed reference.
* RFCs in reverse chronological order.
* First cut of RFCs.
* Fixed reference.
* Updated recipes.
* Updated recipes.
* More RFCs.
* Updated D*PL comments.
* Added H5P descriptions.
* Committing clang-format changes
* H5R life-cycle snapshot.
* Committing clang-format changes
* H5R life-cycle. Added line numbers to life-cycle examples.
* Committing clang-format changes
* Fixed formatting for H5Dchunk_iter().
* Added comment on collective mode requirement w/ compression.
* Simplified API compat. macro dox.
* More API vers. updates.
* Hide the async macro entrails.
* Latest VFD SWMR RFC.
* Create a tag file for permalinks.
* Added TODOs for metadoc.
* Removed duplication.
* Revised RM landing page.
* Trimmed more duplication.
* Committing clang-format changes
* Revised H5D.
* Committing clang-format changes
* Updated survey link.
* Added Doxygen RM entry template link.
* Added the "Multi-Thread HDF5" RFC.
* Added DOXYGEN_TAG_FILE.
* Added selection I/O RFC.
* Added the VFD Sub-filing RFC.
* Updated meta-documentation and added two old presentations.
* Added a few more RFCs (4).
* Fixed MANIFEST.
* Updated meta-documentation.
* Added Filters technical note.
* Fixed MANIFEST.
* Restore the path stripper.
* Experimental full-text search via Google.
* Better full-text search integration.
* Whoops. Forgot this one.
* Oh boy.
* Make CMake happy.
* Added "Debugging HDF5 Applications" technical note.
* Another batch of RFCs.
Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
Diffstat (limited to 'doxygen/examples/IOFlow.html')
| -rw-r--r-- | doxygen/examples/IOFlow.html | 137 |
1 files changed, 137 insertions, 0 deletions
diff --git a/doxygen/examples/IOFlow.html b/doxygen/examples/IOFlow.html new file mode 100644 index 0000000..6b2c27e --- /dev/null +++ b/doxygen/examples/IOFlow.html @@ -0,0 +1,137 @@ +<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"> +<!-- saved from url=(0064)https://gamma.hdfgroup.org/papers/HISS/030821.IOFlow/IOFlow.html --> +<html><head><meta http-equiv="Content-Type" content="text/html; charset=UTF-8"> + <title>HDF5 Raw I/O Flow Notes</title> + + <meta name="author" content="Quincey Koziol"> +</head> + +<body text="#000000" bgcolor="#FFFFFF"> + +<style type="text/css"> +OL.loweralpha { list-style-type: lower-alpha } +OL.upperroman { list-style-type: upper-roman } +</style> + +<center><h1>HDF5 Raw I/O Flow Notes</h1></center> +<center><h3>Quincey Koziol<br> + koziol@ncsa.uiuc.edu<br> + August 20, 2003 +</h3></center> + +<ol class="upperroman"> + +<li><h3><u>Document's Audience:</u></h3> + +<ul> + <li>Current H5 library designers and knowledgable external developers.</li> +</ul> + +</li><li><h3><u>Background Reading:</u></h3> + +</li><li><h3><u>Introduction:</u></h3> + +<dl> + <dt><strong>What is this document about?</strong></dt> + <dd>This document attempts to supplement the flow charts describing + the flow of control for raw data I/O in the library. + </dd> <br> +</dl> + +</li><li><h3><u>Figures:</u></h3> +<p>The following figures provide the main information:</p> + <table> + <tr><td><img src="IOFlow.gif" alt="High-Level View of Writing Raw Data" style="height:50%;"></td></tr> + <tr><td><img src="IOFlow2.gif" alt="Perform Serial or Parallel I/O" style="height:50%;"></td></tr> + <tr><td><img src="IOFlow3.gif" alt="Gather/Convert/Scatter" style="height:50%;"></td></tr> + </table> + +</li><li><h3><u>Notes From Accompanying Figures:</u></h3> + +<p>This section provides notes to augment the information in the accompanying + figures. +</p> + +<ol> + <li><b>Validate Parameters</b> - Resolve any H5S_ALL parameters + for dataspace selections to actual dataspaces, allocate + conversion buffers, etc. + </li> + + <li><b>Space Allocated in File?</b> - Space may not have been allocated + in the file to store the dataset data, if "late allocation" was chosen + for the allocation time when the dataset was created. + </li> + + <li><b>Allocate & Fill Space</b> - These operations allocate both contiguous + and chunked dataset's space in the file. The chunked dataset space + allocation iterates through all the chunks in the file and allocates + both the B-tree information and the raw data in the file. Because of + the way filters work, fill-values are written out for chunked datasets + as they are allocated, instead of as a separate step. + In parallel + I/O, the chunked dataset allocation can potentially be time-consuming, + since all the raw data in the dataset is allocated from one process. + </li> + + <li><b>Datatype Conversion Needed?</b> - This currently is the deciding + factor between doing "direct I/O" (in serial or parallel) and needing + to perform gather/convert/scatter operations. I believe that MPI + is capable of performing a limited range of type conversions and if so, + we should add support to detect when they can be used. This will + allow more I/O operations to be performed collectively. + </li> + + <li><b>Collective I/O Requested/Allowed?</b> - A user has to both request + that collective I/O occur and also their I/O operation must meet the + requirements that the library sets for supporting collective parallel + I/O: + <ul> + <li>The dataspace must be scalar or simple (which is a no-op really, + since we don't support "complex" dataspaces in the library + currently). + </li> + <li>The selection must be regular. "all" selections + and hyperslab selections that were + made with only one call to H5Sselect_hyperslab() (i.e. not a + hyperslab selection that has been aggregated over multiple + selection calls) are regular. Supporting point and + irregular hyperslab selections are on the "to do" list. + </li> + <li>The dataset must be stored contiguously on disk (as shown in the + figure also). Supporting chunked dataset storage is also + on the "to do" list. + </li> + </ul> + </li> + + <li><b>Build "chunk map"</b> - This step still has some scalability issues + as it creates a data structure that is proportional to the number of + chunks which will be written to, which could potentially be very large. + Building the "chunk map" information incrementally is on the "to do" + list also. + </li> + + <li><b>Perform Chunked I/O</b> - As the figure shows, there is no support + for collective parallel I/O on chunked datasets currently. As noted + earlier, this is on the "to do" list. + </li> + + <li><b>Perform "Direct" Serial I/O</b> - "Direct" serial I/O writes data + from the application's buffer, without any intervening buffer or memory + copies. For maximum efficiency and performance, the elements in the + selections should be adjoining. + </li> + + <li><b>Perform Collective Parallel I/O</b> - This step also writes data + directly from an application buffer, but additionally uses collective + MPI I/O operations to combine the data from each process in the parallel + application in an efficient manner. + </li> +</ol> + +</li></ol> + + + +</body></html> |