summaryrefslogtreecommitdiffstats
path: root/src/H5FDvfd_swmr.c
Commit message (Collapse)AuthorAgeFilesLines
* VFD SWMR: Normalization with develop and cleanup (#1559)Dana Robinson2022-03-311-24/+34
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Brought over plugin and test script changes * Removes remaining register keywords (#1481) * Fixed warnings in the aux process code * Minor fixes from develop * Minor changes from develop, fixed format warnings * Formatted source * Added HD prefix to timespec_get * Cleanup in new files * Removes the MANIFEST file and unused release files (#1497) * Removes the MANIFEST file and unused release files * Updated tar command * checkposix corrections * More checkposix fixes * Ripped out unused instrumentation functionality * Brought over cache tagging changes from develop * Changes to tagged metadata expulsion iteration * Fixed typo * Brought over H5O__free() changes from develop * Brings (unused) parallel page buffer test in line with develop * Moved the functionality in supervise.subr to test_vfd_swmr.sh * Tools VFD parameter updates * Committing clang-format changes * H5F VFD SWMR refactoring * Committing clang-format changes * Misc changes * Acceptance test cleanup * HD prefix * H5_get_option instead of getopt * Some bool fixes * Changes to VFD SWMR acceptance tests * Minor cleanup * Use AC_MSG_RESULT/AC_MSG_ERROR/AC_MSG_WARN instead of echo (#1532) * Merges from develop Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* VFD SWMR: Normalization with develop (#1506)Dana Robinson2022-03-231-21/+7
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Brought over plugin and test script changes * Removes remaining register keywords (#1481) * Fixed warnings in the aux process code * Minor fixes from develop * Minor changes from develop, fixed format warnings * Formatted source * Added HD prefix to timespec_get * Cleanup in new files * Removes the MANIFEST file and unused release files (#1497) * Removes the MANIFEST file and unused release files * Updated tar command * checkposix corrections * More checkposix fixes * Ripped out unused instrumentation functionality * Brought over cache tagging changes from develop * Changes to tagged metadata expulsion iteration * Fixed typo * Brought over H5O__free() changes from develop * Brings (unused) parallel page buffer test in line with develop * Moved the functionality in supervise.subr to test_vfd_swmr.sh * Tools VFD parameter updates * Committing clang-format changes * H5F VFD SWMR refactoring * Committing clang-format changes * Misc changes Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* VFD SWMR: normalization with develop (#1472)Dana Robinson2022-03-061-27/+4
| | | Much normalization with develop. Still needs tools changes wrt VFD plugins.
* VFD SWMR: Aux process changes (#1451)Dana Robinson2022-02-181-1/+0
| | | | | | | | | | | * Removed the aux process submodule * Directly added aux process files after submodule delete * Updated Akadio copyright on appropriate files * Committing clang-format changes Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Updated VFD SWMR's Akadio licenseDana Robinson2022-02-171-0/+1
|
* 1) Core changes for adding NFS/updater support as described in the RFC.vchoi2021-11-171-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | src/H5Pfapl.c src/H5Fvfd_swmr.c src/H5Fpublic.h src/H5Fpkg.h src/H5Fprivate.h 2) For VFD SWMR testing, add private property for checksum generation of metadata files: src/H5Fint.c src/H5Fvfd_swmr.c src/H5Pfapl.c src/H5Fpkg.h src/H5Fprivate.h 3) Fix the following in H5F_vfd_swmr_init() and H5F_vfd_swmr_close_or_flush(): (a) Allocate metadata file index right after metadata file header. (b) Set tick number to 0 when creating header and index for file open case. (c) Remove tick number increment at file close. src/H5Fvfd_swmr.c src/H5Ftest.c 4) To be consistent with the RFC, change the name for field "chksum" to "checksum" in struct H5FD_vfd_swmr_idx_entry_t: src/H5FDprivate.h src/H5FDtest.c src/H5FDvfd_swmr.c src/H5Ftest.c src/H5PB.c 4) Add tests for NFS/updater test/vfd_swmr.c 5) Modify common routine init_vfd_swmr_config() to accept updater_file_path test/vfd_swmr_common.c test/vfd_swmr_common.h 6) Changes to the tests due to the common routine init_vfd_swmr_config(): test/vfd_swmr_addrem_writer.c test/vfd_swmr_attrdset_writer.c test/vfd_swmr_bigset_writer.c test/vfd_swmr_dsetchks_writer.c test/vfd_swmr_dsetops_writer.c test/vfd_swmr_generator.c test/vfd_swmr_gfail_writer.c test/vfd_swmr_gperf_writer.c test/vfd_swmr_group_writer.c test/vfd_swmr_reader.c test/vfd_swmr_remove_reader.c test/vfd_swmr_remove_writer.c test/vfd_swmr_sparse_reader.c test/vfd_swmr_sparse_writer.c test/vfd_swmr_vlstr_reader.c test/vfd_swmr_vlstr_writer.c test/vfd_swmr_writer.c test/page_buffer.c
* VFD SWMR: Removes dedup callback + cleanup (#842)Dana Robinson2021-07-171-328/+317
| | | | | | | | | | | * HDF5-ification of dedup calls * Removes dedup callback from VFD struct * VFD SWMR library code cleanup * Committing clang-format changes Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Brings native H5Fdelete implementation from developDana Robinson2021-05-051-1/+2
|
* Updates license url and history docsDana Robinson2021-03-251-2/+1
|
* Formats the source and updates the gcc warning pragmasDana Robinson2021-03-251-392/+311
|
* Fixes a bug from when the hlog code was removedDana Robinson2021-03-021-1/+2
|
* Switch to hbool_t and TRUE/FALSE in library codeDana Robinson2021-03-021-5/+5
|
* Removed the rest of the hlog codeDana Robinson2021-03-021-40/+3
|
* Switches H5FDvfd_swmr.c memory calls to use H5MMDana Robinson2021-03-021-4/+4
|
* Adds HD prefix to POSIX calls in srcDana Robinson2021-03-021-10/+10
|
* Improve/expand the comments on the file->writer assertions.David Young2020-08-311-6/+5
|
* Delete an unhelpful comment. NFCI.David Young2020-08-311-1/+1
|
* Add some header comments to functions.David Young2020-08-311-0/+8
|
* Rename `api_elapsed_nslots` to `api_elapsed_nbuckets`: possibly clarifiesDavid Young2020-08-311-5/+9
| | | | things, possibly does not? NFCI.
* Add/improve some struct/member documentation per John's request.David Young2020-08-311-1/+9
|
* Fix a typo in a comment. NFCI.David Young2020-08-281-1/+1
|
* Describe the behavior of H5FD_vfd_swmr_dedup() in excruciating detail.David Young2020-08-251-0/+32
|
* Change H5F_open to perform deduplication of open VFDs in a more flexible way,David Young2020-07-301-27/+89
| | | | | | | | | | | | | | | | | | | | | | | | | | using the new routine H5FDdeduplicate(). Simplify H5F_open() a bit, pushing some of the configuration checks into the SMWR VFD. For example, check that page buffering is enabled in H5FD_vfd_swmr_open() instead of in H5F_open(). Compare VFD SWMR configurations in H5FD_vfd_swmr_dedup() instead of in H5F_open(). Clone the default file-access property list at a new file-access property list ID, H5P_FILE_ACCESS_ANY_VFD. The new ID is used to indicate that if the file that's being opened is already open under an existing virtual file, and if that virtual file would not ordinarily be opened with the default FAPL, then it's ok to use that virtual file. Add a new optional method, `dedup`, to H5FD_class_t, and use it to customize a VFD's deduplication. Customize the SWMR VFD's deduplication. Make it honor H5P_FILE_ACCESS_ANY_VFD Embed the VFD SWMR configuration in the H5FD_vfd_swmr_t to facilitate comparison of configuration between new and old SWMR virtual files. In H5F__sfile_search(), match using a pointer comparison instead of H5FD_cmp(), because we will only ever enter H5F__sfile_search() with a deduplicated H5FD_t *.
* Update comment.David Young2020-07-151-10/+3
|
* When H5FD_vfd_swmr_read() reads a page from the shadow file whose checksumDavid Young2020-07-141-0/+10
| | | | | fails, re-load the shadow-file header and log the last-read and current tick number.
* Avoid some casts, change some parameters from H5FD_t to H5FD_vfd_swmr_t whenDavid Young2020-07-141-17/+12
| | | | that's what both the callers and callees expect. NFCI.
* Delete some dead code.David Young2020-07-141-46/+0
|
* Bug fix: avoid leaking memory and other resources, perform the reader-onlyDavid Young2020-07-141-63/+78
| | | | | close activities under the correct condition! Extract the reader-only open and close code into helper routines.
* If the function-enter macro fails, return FAIL. Now the compiler can see thatDavid Young2020-07-141-1/+3
| | | | | | if H5FD__vfd_swmr_header_deserialize() succeeds, then the header that was passed in was actually initialized. This squashes a used-before-initialized warning from GCC.
* Take care to initialize the metadata file descriptor to -1 so that we don't tryDavid Young2020-07-141-0/+1
| | | | to close descriptor 0, later.
* Delete whitespace at EOL. Join consecutive line. NFCI.David Young2020-07-131-77/+76
|
* Delete whitespace at EOL.David Young2020-07-131-1/+1
|
* Be brief: remove gratuitous cast of _xfree(). Use reference to member `pub`David Young2020-07-101-2/+2
| | | | instead of cast to H5FD_t *.
* Make a VFD SWMR writer use the SWMR VFD, to facilitate avoiding conflictingDavid Young2020-07-101-30/+23
| | | | | | multiple opens of the same file with VFD SWMR---i.e., twice for writing, or for reading and for writing. In the long run, this will help me encapsulate more of the SWMR functionality in the VFD, too.
* Straggler from previous commit: when closing the VFD SWMR virtual file, clearDavid Young2020-07-101-8/+12
| | | | | | | | | | the lower virtual file's `exc_owner` field. While I'm here, remove a gratuitous assertion. This is part of a changeset that helps us avoid creating multiple H5F_shared_t for one file when virtual datasets are used with VFD SWMR. The old code for deduplicating VFD SWMR H5F_shared_t instances did not work correctly with VFD SWMR, so we'd end up with multiple H5F_shared_t all active on the same file.
* Add all extant virtual files to a list. Add an "exclusive owner" (`exc_owner`)David Young2020-07-101-0/+2
| | | | | | | | | | | | | | member to all virtual files. Add a routine, H5FD_has_conflict(), that returns true if a new virtual file is identical to an existing virtual file that has an exclusive owner. Establish an exclusive owner for a VFD SWMR virtual file's lower virtual file. Rename bsdqueue.h to H5queue.h and install it, since it's used by H5FDpublic.h. This is part of a changeset that helps us avoid creating multiple H5F_shared_t for one file when virtual datasets are used with VFD SWMR. The old code for deduplicating VFD SWMR H5F_shared_t instances did not work correctly with VFD SWMR, so we'd end up with multiple H5F_shared_t all active on the same file.
* On each SWMR VFD open for reading, keep a histogram for the number ofDavid Young2020-06-161-1/+32
| | | | | ticks elapsed during API calls. Write the histogram to the swmr_stats log outlet when the SWMR VFD closes.
* Delete superfluous casts. Remove superfluous backslashes andDavid Young2020-06-161-19/+17
| | | | | | | | | | | | | | | | | statement-ify, changing HGOTO_ERROR(..., \ ) to HGOTO_ERROR(..., ); Remove blank lines between if-clause and HGOTO_ERROR. Add some curly braces to if-statements where that clarifies things. NFCI.
* In H5FD_vfd_swmr_read(), log on outlet `swmr_read` whether the read isDavid Young2020-05-291-3/+4
| | | | satisfied from the lower file or the shadow file.
* In H5FD_vfd_swmr_read(), do not verify checksums on shadow entriesDavid Young2020-05-131-157/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | that are longer than the buffer that the caller supplied: the checksum usually will fail, but that's not actually a fatal condition, and usually we will have another opportunity to verify the checksum. In H5FD_vfd_swmr_read(), remove a bunch of disused code. In H5FD_vfd_swmr_read(), do not re-read a shadow image that has a bad checksum, because a bad checksum indicates a serious problem (writer outran reader, OS defect, hardware failure) from which H5FD_vfd_swmr_read() cannot recover. Rationale: the writer write(2)s new shadow images before the new index, and the new index before the new header. In H5FD_vfd_swmr_read(), the reader has read(2) both the index and the header in full. POSIX semantics indicate that in these circumstances, the last shadow image write(2) MUST be completely visible when we read(2). That is, the index write(2) & read(2) and the header write(2) & read(2) pair cannot divide a preceding shadow-image write(2). The reader may see a "torn" image at this juncture if, for example, the writer got max_lag ticks ahead of it and reused the storage for this shadow image. Even if the reader "recovers" by re-reading the image until its checksum is correct, it cannot be sure that the image thus read is the right one for the HDF5 address passed to H5FD_vfd_swmr_read(), and it cannot be sure that the image thus read is not stale, because it's operating with an out-of-date shadow index. Add log outlets swmr_read, swmr_read_exception, and swmr_read_err. Log to `swmr_read` on entry to H5FD_vfd_swmr_read(), log to `swmr_read_exception` when checksums are skipped for exceptional conditions (page buffer not configured, buffer shorter than shadow image), and log to `swmr_read_err` when the checksum fails.
* Delete an assertion that doesn't hold true for multipage entries.David Young2020-04-211-3/+0
|
* Add a second bool argument to H5F_vfd_swmr_writer_end_of_tick() thatDavid Young2020-04-211-0/+2
| | | | | | | | | | | | | | | tells whether the call may wait for the reader tick to catch up. Add stub routines vfd_swmr_writer_may_increase_tick_to() and vfd_swmr_reader_did_increase_tick_to() for tests---e.g., vfd_swmr_zoo_writer/_reader---to use to coordinate their tick numbers. vfd_swmr_writer_may_increase_tick_to(new_tick, wait_for_reader) returns true if the writer may increase its tick number to `new_tick` without overrunning the reader. A reader uses vfd_swmr_reader_did_increase_tick_to() to tell a writer that its tick number has increased.
* Fix spelling in a comment.David Young2020-04-081-1/+1
|
* Move the `swmr` log-outlet declaration to H5FDvfd_swmr_private.h and use `swmr`David Young2020-02-281-1/+3
| | | | | as the parent outlet of a new outlet for messages about the motion of the shadow index.
* On the reader, zero the shadow-index entries before filling fields from theDavid Young2020-02-261-1/+1
| | | | | shadow file so that members like `garbage` are not filled with garbage that fools us, later.
* Where n is the number of page-table/shadow-index entries, avoid spending O(n^2)David Young2020-02-261-1/+2
| | | | | | | | time in H5PB_dest(). While we're in H5PB_dest(), mark deleted shadow-index entries as "garbage" and skip the O(n) shadow index-entries copy. Rename shadow index-entry member `moved_to_hdf5_file` to `moved_to_lower_file` while I'm in here---NFCI.
* Delete misleading "Programmer:" field and excessive decoration from theDavid Young2020-02-121-22/+17
| | | | | H5FD_vfd_swmr_write() comment header. Delete a superfluous comment. Delete some whitespace at EOL. NFCI.
* Use an early exit to lower a staircase.David Young2020-02-061-132/+130
|
* Reduce code duplication by using vfd_swmr_pageno_to_mdf_idx_entry() to look upDavid Young2020-02-061-98/+61
| | | | | | | | | | | | | shadow pages. Reduce casts by choosing correct format strings and compatible variable types. Poison writes to addr by making it const. Don't increase addr in the read(2) loop because it's never used afterward. Delete some more dead code. Rename read_ptr as p and declare it much closer to its use. Change its type to `char *` so that no casts are necessary to increase it.
* Delete unused code.David Young2020-02-061-36/+0
|
generated by cgit v0.12 at 2024-07-05 14:25:12 (GMT)