summaryrefslogtreecommitdiffstats
path: root/src/H5PB.c
Commit message (Collapse)AuthorAgeFilesLines
* Updates license url and history docsDana Robinson2021-03-251-1/+1
|
* Formats the source and updates the gcc warning pragmasDana Robinson2021-03-251-1537/+1279
|
* Switch to hbool_t and TRUE/FALSE in library codeDana Robinson2021-03-021-20/+20
|
* Removed the rest of the hlog codeDana Robinson2021-03-021-51/+1
|
* Adds HD prefix to POSIX calls in srcDana Robinson2021-03-021-5/+5
|
* Merge branch 'feature/vfd_swmr' into may-merge-mainzer-to-mineDavid Young2020-07-211-9/+13
|\
| * Only count metadata reads/writes on the page buffer. Neaten the count logs.David Young2020-07-211-9/+13
| |
* | Merge branch 'feature/vfd_swmr' into may-merge-mainzer-to-mineDavid Young2020-07-151-0/+42
|\ \ | |/
| * Log the number of page-buffer accesses at various multiples of the page sizeDavid Young2020-07-151-0/+42
| | | | | | | | with log outlet `pb_access_sizes`.
* | Merge branch 'feature/vfd_swmr' into may-merge-mainzer-to-mineDavid Young2020-06-081-2/+3
|\ \ | |/
| * Only remove shadow-index entries in H5PB__evict_entry() if we're aDavid Young2020-06-081-2/+3
| | | | | | | | | | VFD SWMR writer, so that the reader does not lose track of the real shadow-index content.
* | Merge branch 'feature/vfd_swmr' into may-merge-mainzer-to-mineDavid Young2020-06-031-1/+1
|\ \ | |/
| * Add a label, `addr`, to an HDF5 file address in debug output.David Young2020-06-031-1/+1
| |
* | Merge branch 'feature/vfd_swmr' into may-merge-mainzer-to-mineDavid Young2020-05-261-2/+2
|\ \ | |/
| * We don't write through the metadata accumulator any more. We only writeDavid Young2020-05-261-2/+2
| | | | | | | | through the lower VFD. Update error message to match.
* | Modified page buffer to split entries only where necessary -- specificallymainzer2020-05-211-256/+963
|/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | when handling an I/O request on a metadata entry that has been sub-allocated from a larger file space allocation (i.e. fixed and extensible array), and that crosses at least one page boundary. . This required modifying the metadata cache to provide the type of the metadata cache entry in the current I/O request. For now, this is done with a function call. Once we are sure this works, it may be appropriate to convert this to a macro, or to add a flags parameter to the H5F block read/write calls. Also updated the metadata cache to report whether a read request is speculative -- again via a function call. This allowed me to remove the last address static variable in the H5PB_read() call, which is necessary to support multiple files opened in VFD SWMR mode. Also re-wrote the H5PB_remove_entries() call to handle release of large metadata file space allocations that have been sub-allocated into multiple metadata entries. Also modified the call to H5PB_remove_entries() in H5MF__xfree_impl() to invoke it whenever the page buffer is enabled and the size of the space to be freed is of page size or larger. Tested serial / debug on charis and Jelly. Found a bug in H5MF_xfree_impl(), in which the call to H5PB_remove_entries() is skipped due to HGOTO_DONE calls earlier in the function. While the obvious action is to move the call earlier in the function, best to consult with Vailin first, as there is much going on and it would be best to avoid making the situation worse. If nothing else, there are some error management issues.
* Trim trailing whitespaceQuincey Koziol2020-05-201-2/+2
|
* When an entry changes size, remove it from the tick list and add it backDavid Young2020-05-111-0/+10
| | | | | to keep the cumulative bytes in the tick list up-to-date. That prevents an assertion, later.
* Bug fix: in the freespace manager, use a new routine,David Young2020-04-231-4/+53
| | | | | H5PB_remove_entries(), to remove *all* pages overlapped by the freed space, instead of just the first one.
* Remove gratuitous initialization, comment, and assertions.David Young2020-04-161-6/+2
|
* Log page-buffer reads and writes no matter what storage type. (Used toDavid Young2020-04-161-8/+4
| | | | log only the global heap-type reads and writes.)
* Check whether or not the shadow entry size is different from theDavid Young2020-04-131-2/+15
| | | | | | | | | | | pagebuffer entry size. If the sizes are different, then release the old shadow space (using the correct size) and set the shadow page number to 0 so that H5F_update_vfd_swmr_metadata_file() will allocate new shadow space with the right size. Vailin says that this fixes the bug she found, where a 4096-byte buffer allocated by H5MV_alloc() is released with H5MV_free() as if it was 8192 bytes long.
* Add a new log outlet and change an `fprintf(stderr,` to anDavid Young2020-04-071-1/+2
| | | | `hlog_fast(lengthen_pbentry,`. This quiets one of the VFD SWMR tests.
* Simplify H5PB_vfd_swmr__update_index(), which figures inDavid Young2020-04-071-21/+23
| | | | | | | | | | | | | | | | | my investigation of shadow-file freespace leaks. Copy the tick number from the H5F_shared_t to a temporary variable and use the temporary instead of spelling out `shared->tick_num`. Assert that every entry on the tick list is dirty, since clean entries will not appear on the tick list. Mark each corresponding shadow-index entry dirty, and set its "tick of last change" to the current tick and "tick of last flush" to 0. This makes the scan for entries that do not appear on the tick list more believable. Lower staircase in the loop that scans for shadow-index entries that did not appear on the tick list. Log indicies that are marked for reclamation.
* Declutter: remove superfluous comment and assertion. NFCI.David Young2020-04-071-2/+0
|
* Remove extraneous whitespace. NFCI.David Young2020-04-071-2/+0
|
* After adding a shadow page to the deferred free list, set the entry'sDavid Young2020-04-071-3/+5
| | | | page number to 0 so that we cannot free it again by accident.
* Fix whitespace at end (beginning?) of line. NFCI.David Young2020-03-311-1/+1
|
* Be brief: turn 7 lines of if-else to one line of ?:. Be sensible: don'tDavid Young2020-03-301-10/+1
| | | | assert a pointer we just dereferenced is non-NULL.
* Add a log outlet for pagebuffer I/O and subordinate outlets forDavid Young2020-03-201-2/+12
| | | | | pagebuffer reads and writes: pbio, pbrd, pbwr. In H5PB_read() and H5PB_write(), log only global heap accesses, for now.
* In H5PB__write_meta(), extend a single-page entry when overwriting itDavid Young2020-03-171-51/+55
| | | | | | | | | | | | | | with multiple pages. Update statistics to maintain consistency. Refactor a bit: in H5PB__write_meta(), move code that's in both the if- and else- branch to either before the if-else or after and de-duplicate. In H5PB_vfd_swmr__update_index(), always update the length of a shadow-index entry to the current size of its corresponding page-table entry. In H5PB_vfd_swmr__update_index(), disregard shadow-index entries that await garbage collection.
* Join some lines. NFCI.David Young2020-03-171-3/+1
|
* Fix spelling, alread -> already. NFCI.David Young2020-03-131-2/+2
|
* Update Guide. Switch to a short and informative function name.David Young2020-03-111-5/+3
|
* Don't assert that an H5FD_mem_t read/written through the page buffer is notDavid Young2020-03-061-3/+0
| | | | H5FD_MEM_GHEAP, that's an out-of-date assumption.
* Break metadata reads and writes into up to three pieces, the non-page-alignedDavid Young2020-03-041-2/+134
| | | | | piece in the beginning, 1 or more full pages, and whatever is leftover at the end. Passes all of our tests.
* Shave 44 lines off of H5PB_read() by gathering a bunch of checks into one ifDavid Young2020-02-271-67/+23
| | | | condition, reducing diagnostic checks, single-spacing, etc.
* Don't re-test a condition that's assert()'d at function entry andDavid Young2020-02-261-2/+2
| | | | couldn't change in the mean time, but do be paranoid and re-assert.
* Where n is the number of page-table/shadow-index entries, avoid spending O(n^2)David Young2020-02-261-17/+42
| | | | | | | | time in H5PB_dest(). While we're in H5PB_dest(), mark deleted shadow-index entries as "garbage" and skip the O(n) shadow index-entries copy. Rename shadow index-entry member `moved_to_hdf5_file` to `moved_to_lower_file` while I'm in here---NFCI.
* Improve some comments: fix typos, shorten.David Young2020-02-191-6/+4
|
* Delete an out-of-date comment.David Young2020-02-181-6/+0
|
* Call them log "outlets" not log "sinks".David Young2020-02-101-4/+4
|
* Rename loglib_ / LOGLIB_ to hlog_ / HLOG_, remove the syslog-basedDavid Young2020-02-101-22/+11
| | | | | | | implementation, rename hlog_log() to hlog(), hlog_vlog() to vhlog(), et cetera. Rename hlog_lazy() to hlog_fast(). Define some log sinks and use them in the page buffer and in VFD SWMR.
* NFCI: exit(EXIT_FAILURE) instead of exit(1) for portability.David Young2020-02-031-2/+2
| | | | Delete a comment on a closing curly brace.
* Numerous changes supporting a floating shadow index:David Young2020-02-031-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | Add to the H5F_shared_t (!) a new member that tells the index in the shadow file where the index should be written. Allocate shadow filespace for the header and the index separately so that the index can float. Update tests to match the expected original location of the index. Introduce vfd_swmr_enlarge_shadow_index(), a routine that allocates space in the shadow file for a new index that has (up to) twice as many entries as the old index, allocates a new in-core index of the same size, and copies the old in-core index to the new. Call vfd_swmr_enlarge_shadow_index() in H5PB_vfd_swmr__update_index() when the in-core index has too few slots. In the comment at the top of H5FD__vfd_swmr_load_hdr_and_idx(), describe the protocol that it follows, now, when it reads the shadow header and index. Delete some dead code in the function and add a bit of diagnostic code. TBD quiet the diagnostic code. In H5F_vfd_swmr_init(), follow the protocol: write the index, first, then the header. Modify property-list checks and tests to reserve no fewer than two pages at the front of the shadow file for the header and index.
* Make the first argument to H5PB_vfd_swmr__update_index() and H5F_t instead ofDavid Young2020-02-031-1/+2
| | | | | an H5F_shared_t because the new routine that will relocate the index (which will be in a future commit) has to pass an H5F_t to the filespace allocator.
* In H5PB_remove_entry(), do not call vfd_swmr_mdf_idx_entry_remove(),David Young2020-01-101-29/+24
| | | | | because the H5PB__evict_entry() call should have already done that. Instead, just assert() that the index entry is not present.
* In vfd_swmr_mdf_idx_entry_remove(), give some local variables less wordyDavid Young2020-01-101-14/+12
| | | | | | names. While I am here, do not copy the last element of the index over the element that's being deleted, because in the very next step I'm shifting all elements over by one.
* Change the unwieldy name vfd_swmr_idx_entry_defer_free() toDavid Young2019-12-191-1/+1
| | | | | shadow_image_defer_free(), which is also a better description of what the routine does.
* Use the "tail queue" from queue(3) for the delayed free list of shadow images.David Young2019-12-191-11/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Delete the little-used free-list length, dl_len, and just count up the list entries when diagnostic code needs the length. Extract the code for deferring shadow-image free into a new subroutine, `vfd_swmr_idx_entry_defer_free()`. Rename type `deferred_free_head_t` as `deferred_free_queue_t`. Remove the disused H5F__LL_{REMOVE,PREPEND} macros. Add some diagnostic code and #if 0'd assertions. Change `qsort(ptr, n, sizeof(type), cmp)` to `qsort(ptr, n, sizeof(*ptr), cmp)`. Use a `continue` statement to lower a staircase in H5F_update_vfd_swmr_metadata_file(). Add vfd_swmr_mdf_idx_entry_remove() to delete a shadow index entry and add the image at that entry to a deferred-free list. Call it whenever a page is evicted. Update the comment in H5PB_remove_entry() that asks if we need to remove shadow index entries: now we *do* remove them. Remove shadow index entries in H5PB__evict_entry(). Also mention in the comment that the index-entry removal performed by H5PB__evict_entry() ought to be sufficient.
generated by cgit v0.12 at 2024-11-02 01:12:49 (GMT)