summaryrefslogtreecommitdiffstats
path: root/src/H5FDvfd_swmr.c
Commit message (Collapse)AuthorAgeFilesLines
* 1) Core changes for adding NFS/updater support as described in the RFC.vchoi2021-11-171-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | src/H5Pfapl.c src/H5Fvfd_swmr.c src/H5Fpublic.h src/H5Fpkg.h src/H5Fprivate.h 2) For VFD SWMR testing, add private property for checksum generation of metadata files: src/H5Fint.c src/H5Fvfd_swmr.c src/H5Pfapl.c src/H5Fpkg.h src/H5Fprivate.h 3) Fix the following in H5F_vfd_swmr_init() and H5F_vfd_swmr_close_or_flush(): (a) Allocate metadata file index right after metadata file header. (b) Set tick number to 0 when creating header and index for file open case. (c) Remove tick number increment at file close. src/H5Fvfd_swmr.c src/H5Ftest.c 4) To be consistent with the RFC, change the name for field "chksum" to "checksum" in struct H5FD_vfd_swmr_idx_entry_t: src/H5FDprivate.h src/H5FDtest.c src/H5FDvfd_swmr.c src/H5Ftest.c src/H5PB.c 4) Add tests for NFS/updater test/vfd_swmr.c 5) Modify common routine init_vfd_swmr_config() to accept updater_file_path test/vfd_swmr_common.c test/vfd_swmr_common.h 6) Changes to the tests due to the common routine init_vfd_swmr_config(): test/vfd_swmr_addrem_writer.c test/vfd_swmr_attrdset_writer.c test/vfd_swmr_bigset_writer.c test/vfd_swmr_dsetchks_writer.c test/vfd_swmr_dsetops_writer.c test/vfd_swmr_generator.c test/vfd_swmr_gfail_writer.c test/vfd_swmr_gperf_writer.c test/vfd_swmr_group_writer.c test/vfd_swmr_reader.c test/vfd_swmr_remove_reader.c test/vfd_swmr_remove_writer.c test/vfd_swmr_sparse_reader.c test/vfd_swmr_sparse_writer.c test/vfd_swmr_vlstr_reader.c test/vfd_swmr_vlstr_writer.c test/vfd_swmr_writer.c test/page_buffer.c
* VFD SWMR: Removes dedup callback + cleanup (#842)Dana Robinson2021-07-171-328/+317
| | | | | | | | | | | * HDF5-ification of dedup calls * Removes dedup callback from VFD struct * VFD SWMR library code cleanup * Committing clang-format changes Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
* Brings native H5Fdelete implementation from developDana Robinson2021-05-051-1/+2
|
* Updates license url and history docsDana Robinson2021-03-251-2/+1
|
* Formats the source and updates the gcc warning pragmasDana Robinson2021-03-251-392/+311
|
* Fixes a bug from when the hlog code was removedDana Robinson2021-03-021-1/+2
|
* Switch to hbool_t and TRUE/FALSE in library codeDana Robinson2021-03-021-5/+5
|
* Removed the rest of the hlog codeDana Robinson2021-03-021-40/+3
|
* Switches H5FDvfd_swmr.c memory calls to use H5MMDana Robinson2021-03-021-4/+4
|
* Adds HD prefix to POSIX calls in srcDana Robinson2021-03-021-10/+10
|
* Improve/expand the comments on the file->writer assertions.David Young2020-08-311-6/+5
|
* Delete an unhelpful comment. NFCI.David Young2020-08-311-1/+1
|
* Add some header comments to functions.David Young2020-08-311-0/+8
|
* Rename `api_elapsed_nslots` to `api_elapsed_nbuckets`: possibly clarifiesDavid Young2020-08-311-5/+9
| | | | things, possibly does not? NFCI.
* Add/improve some struct/member documentation per John's request.David Young2020-08-311-1/+9
|
* Fix a typo in a comment. NFCI.David Young2020-08-281-1/+1
|
* Describe the behavior of H5FD_vfd_swmr_dedup() in excruciating detail.David Young2020-08-251-0/+32
|
* Change H5F_open to perform deduplication of open VFDs in a more flexible way,David Young2020-07-301-27/+89
| | | | | | | | | | | | | | | | | | | | | | | | | | using the new routine H5FDdeduplicate(). Simplify H5F_open() a bit, pushing some of the configuration checks into the SMWR VFD. For example, check that page buffering is enabled in H5FD_vfd_swmr_open() instead of in H5F_open(). Compare VFD SWMR configurations in H5FD_vfd_swmr_dedup() instead of in H5F_open(). Clone the default file-access property list at a new file-access property list ID, H5P_FILE_ACCESS_ANY_VFD. The new ID is used to indicate that if the file that's being opened is already open under an existing virtual file, and if that virtual file would not ordinarily be opened with the default FAPL, then it's ok to use that virtual file. Add a new optional method, `dedup`, to H5FD_class_t, and use it to customize a VFD's deduplication. Customize the SWMR VFD's deduplication. Make it honor H5P_FILE_ACCESS_ANY_VFD Embed the VFD SWMR configuration in the H5FD_vfd_swmr_t to facilitate comparison of configuration between new and old SWMR virtual files. In H5F__sfile_search(), match using a pointer comparison instead of H5FD_cmp(), because we will only ever enter H5F__sfile_search() with a deduplicated H5FD_t *.
* Update comment.David Young2020-07-151-10/+3
|
* When H5FD_vfd_swmr_read() reads a page from the shadow file whose checksumDavid Young2020-07-141-0/+10
| | | | | fails, re-load the shadow-file header and log the last-read and current tick number.
* Avoid some casts, change some parameters from H5FD_t to H5FD_vfd_swmr_t whenDavid Young2020-07-141-17/+12
| | | | that's what both the callers and callees expect. NFCI.
* Delete some dead code.David Young2020-07-141-46/+0
|
* Bug fix: avoid leaking memory and other resources, perform the reader-onlyDavid Young2020-07-141-63/+78
| | | | | close activities under the correct condition! Extract the reader-only open and close code into helper routines.
* If the function-enter macro fails, return FAIL. Now the compiler can see thatDavid Young2020-07-141-1/+3
| | | | | | if H5FD__vfd_swmr_header_deserialize() succeeds, then the header that was passed in was actually initialized. This squashes a used-before-initialized warning from GCC.
* Take care to initialize the metadata file descriptor to -1 so that we don't tryDavid Young2020-07-141-0/+1
| | | | to close descriptor 0, later.
* Delete whitespace at EOL. Join consecutive line. NFCI.David Young2020-07-131-77/+76
|
* Delete whitespace at EOL.David Young2020-07-131-1/+1
|
* Be brief: remove gratuitous cast of _xfree(). Use reference to member `pub`David Young2020-07-101-2/+2
| | | | instead of cast to H5FD_t *.
* Make a VFD SWMR writer use the SWMR VFD, to facilitate avoiding conflictingDavid Young2020-07-101-30/+23
| | | | | | multiple opens of the same file with VFD SWMR---i.e., twice for writing, or for reading and for writing. In the long run, this will help me encapsulate more of the SWMR functionality in the VFD, too.
* Straggler from previous commit: when closing the VFD SWMR virtual file, clearDavid Young2020-07-101-8/+12
| | | | | | | | | | the lower virtual file's `exc_owner` field. While I'm here, remove a gratuitous assertion. This is part of a changeset that helps us avoid creating multiple H5F_shared_t for one file when virtual datasets are used with VFD SWMR. The old code for deduplicating VFD SWMR H5F_shared_t instances did not work correctly with VFD SWMR, so we'd end up with multiple H5F_shared_t all active on the same file.
* Add all extant virtual files to a list. Add an "exclusive owner" (`exc_owner`)David Young2020-07-101-0/+2
| | | | | | | | | | | | | | member to all virtual files. Add a routine, H5FD_has_conflict(), that returns true if a new virtual file is identical to an existing virtual file that has an exclusive owner. Establish an exclusive owner for a VFD SWMR virtual file's lower virtual file. Rename bsdqueue.h to H5queue.h and install it, since it's used by H5FDpublic.h. This is part of a changeset that helps us avoid creating multiple H5F_shared_t for one file when virtual datasets are used with VFD SWMR. The old code for deduplicating VFD SWMR H5F_shared_t instances did not work correctly with VFD SWMR, so we'd end up with multiple H5F_shared_t all active on the same file.
* On each SWMR VFD open for reading, keep a histogram for the number ofDavid Young2020-06-161-1/+32
| | | | | ticks elapsed during API calls. Write the histogram to the swmr_stats log outlet when the SWMR VFD closes.
* Delete superfluous casts. Remove superfluous backslashes andDavid Young2020-06-161-19/+17
| | | | | | | | | | | | | | | | | statement-ify, changing HGOTO_ERROR(..., \ ) to HGOTO_ERROR(..., ); Remove blank lines between if-clause and HGOTO_ERROR. Add some curly braces to if-statements where that clarifies things. NFCI.
* In H5FD_vfd_swmr_read(), log on outlet `swmr_read` whether the read isDavid Young2020-05-291-3/+4
| | | | satisfied from the lower file or the shadow file.
* In H5FD_vfd_swmr_read(), do not verify checksums on shadow entriesDavid Young2020-05-131-157/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | that are longer than the buffer that the caller supplied: the checksum usually will fail, but that's not actually a fatal condition, and usually we will have another opportunity to verify the checksum. In H5FD_vfd_swmr_read(), remove a bunch of disused code. In H5FD_vfd_swmr_read(), do not re-read a shadow image that has a bad checksum, because a bad checksum indicates a serious problem (writer outran reader, OS defect, hardware failure) from which H5FD_vfd_swmr_read() cannot recover. Rationale: the writer write(2)s new shadow images before the new index, and the new index before the new header. In H5FD_vfd_swmr_read(), the reader has read(2) both the index and the header in full. POSIX semantics indicate that in these circumstances, the last shadow image write(2) MUST be completely visible when we read(2). That is, the index write(2) & read(2) and the header write(2) & read(2) pair cannot divide a preceding shadow-image write(2). The reader may see a "torn" image at this juncture if, for example, the writer got max_lag ticks ahead of it and reused the storage for this shadow image. Even if the reader "recovers" by re-reading the image until its checksum is correct, it cannot be sure that the image thus read is the right one for the HDF5 address passed to H5FD_vfd_swmr_read(), and it cannot be sure that the image thus read is not stale, because it's operating with an out-of-date shadow index. Add log outlets swmr_read, swmr_read_exception, and swmr_read_err. Log to `swmr_read` on entry to H5FD_vfd_swmr_read(), log to `swmr_read_exception` when checksums are skipped for exceptional conditions (page buffer not configured, buffer shorter than shadow image), and log to `swmr_read_err` when the checksum fails.
* Delete an assertion that doesn't hold true for multipage entries.David Young2020-04-211-3/+0
|
* Add a second bool argument to H5F_vfd_swmr_writer_end_of_tick() thatDavid Young2020-04-211-0/+2
| | | | | | | | | | | | | | | tells whether the call may wait for the reader tick to catch up. Add stub routines vfd_swmr_writer_may_increase_tick_to() and vfd_swmr_reader_did_increase_tick_to() for tests---e.g., vfd_swmr_zoo_writer/_reader---to use to coordinate their tick numbers. vfd_swmr_writer_may_increase_tick_to(new_tick, wait_for_reader) returns true if the writer may increase its tick number to `new_tick` without overrunning the reader. A reader uses vfd_swmr_reader_did_increase_tick_to() to tell a writer that its tick number has increased.
* Fix spelling in a comment.David Young2020-04-081-1/+1
|
* Move the `swmr` log-outlet declaration to H5FDvfd_swmr_private.h and use `swmr`David Young2020-02-281-1/+3
| | | | | as the parent outlet of a new outlet for messages about the motion of the shadow index.
* On the reader, zero the shadow-index entries before filling fields from theDavid Young2020-02-261-1/+1
| | | | | shadow file so that members like `garbage` are not filled with garbage that fools us, later.
* Where n is the number of page-table/shadow-index entries, avoid spending O(n^2)David Young2020-02-261-1/+2
| | | | | | | | time in H5PB_dest(). While we're in H5PB_dest(), mark deleted shadow-index entries as "garbage" and skip the O(n) shadow index-entries copy. Rename shadow index-entry member `moved_to_hdf5_file` to `moved_to_lower_file` while I'm in here---NFCI.
* Delete misleading "Programmer:" field and excessive decoration from theDavid Young2020-02-121-22/+17
| | | | | H5FD_vfd_swmr_write() comment header. Delete a superfluous comment. Delete some whitespace at EOL. NFCI.
* Use an early exit to lower a staircase.David Young2020-02-061-132/+130
|
* Reduce code duplication by using vfd_swmr_pageno_to_mdf_idx_entry() to look upDavid Young2020-02-061-98/+61
| | | | | | | | | | | | | shadow pages. Reduce casts by choosing correct format strings and compatible variable types. Poison writes to addr by making it const. Don't increase addr in the read(2) loop because it's never used afterward. Delete some more dead code. Rename read_ptr as p and declare it much closer to its use. Change its type to `char *` so that no casts are necessary to increase it.
* Delete unused code.David Young2020-02-061-36/+0
|
* Fix a typo in a diagnostic printf and, while I'm here, use the right formatDavid Young2020-02-061-6/+6
| | | | strings.
* Use proper format string.David Young2020-02-061-2/+4
|
* Make H5FD_vfd_swmr_dump_status() take an unsigned page number to be consistentDavid Young2020-02-061-2/+2
| | | | with other code.
* Numerous changes supporting a floating shadow index:David Young2020-02-031-48/+41
| | | | | | | | | | | | | | | | | | | | | | | | | | Add to the H5F_shared_t (!) a new member that tells the index in the shadow file where the index should be written. Allocate shadow filespace for the header and the index separately so that the index can float. Update tests to match the expected original location of the index. Introduce vfd_swmr_enlarge_shadow_index(), a routine that allocates space in the shadow file for a new index that has (up to) twice as many entries as the old index, allocates a new in-core index of the same size, and copies the old in-core index to the new. Call vfd_swmr_enlarge_shadow_index() in H5PB_vfd_swmr__update_index() when the in-core index has too few slots. In the comment at the top of H5FD__vfd_swmr_load_hdr_and_idx(), describe the protocol that it follows, now, when it reads the shadow header and index. Delete some dead code in the function and add a bit of diagnostic code. TBD quiet the diagnostic code. In H5F_vfd_swmr_init(), follow the protocol: write the index, first, then the header. Modify property-list checks and tests to reserve no fewer than two pages at the front of the shadow file for the header and index.
* Remove some dead code and out-of-date comments.David Young2020-01-301-18/+0
|
generated by cgit v0.12 at 2024-08-29 09:59:28 (GMT)