summaryrefslogtreecommitdiffstats
path: root/src
Commit message (Collapse)AuthorAgeFilesLines
* Implement H5VLget_file_type() to return a copy of a datatype with theNeil Fortner2020-05-2034-289/+656
| | | | | | | | | | | location set to be in a file. Only meant to be used by VOL connectors. Implement H5VLpeek_connector_id() to support connectors querying their own IDs. Fix app_ref with connector IDs in a couple places (external VOLs registered as default through ENV should be visible to the application). Modify vlen and reference interfaces to work with arbitrary VOL connectors. Implement file "post open" specific callback, to enable connectors to update their file structs after a wrap context has been set.
* Partial fix for HDFFV-10792Jordan Henderson2020-05-201-4/+45
|
* In H5FD_vfd_swmr_read(), do not verify checksums on shadow entriesDavid Young2020-05-131-157/+63
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | that are longer than the buffer that the caller supplied: the checksum usually will fail, but that's not actually a fatal condition, and usually we will have another opportunity to verify the checksum. In H5FD_vfd_swmr_read(), remove a bunch of disused code. In H5FD_vfd_swmr_read(), do not re-read a shadow image that has a bad checksum, because a bad checksum indicates a serious problem (writer outran reader, OS defect, hardware failure) from which H5FD_vfd_swmr_read() cannot recover. Rationale: the writer write(2)s new shadow images before the new index, and the new index before the new header. In H5FD_vfd_swmr_read(), the reader has read(2) both the index and the header in full. POSIX semantics indicate that in these circumstances, the last shadow image write(2) MUST be completely visible when we read(2). That is, the index write(2) & read(2) and the header write(2) & read(2) pair cannot divide a preceding shadow-image write(2). The reader may see a "torn" image at this juncture if, for example, the writer got max_lag ticks ahead of it and reused the storage for this shadow image. Even if the reader "recovers" by re-reading the image until its checksum is correct, it cannot be sure that the image thus read is the right one for the HDF5 address passed to H5FD_vfd_swmr_read(), and it cannot be sure that the image thus read is not stale, because it's operating with an out-of-date shadow index. Add log outlets swmr_read, swmr_read_exception, and swmr_read_err. Log to `swmr_read` on entry to H5FD_vfd_swmr_read(), log to `swmr_read_exception` when checksums are skipped for exceptional conditions (page buffer not configured, buffer shorter than shadow image), and log to `swmr_read_err` when the checksum fails.
* When an entry changes size, remove it from the tick list and add it backDavid Young2020-05-111-0/+10
| | | | | to keep the cumulative bytes in the tick list up-to-date. That prevents an assertion, later.
* In H5MF__xfree_impl(), always invalidate the page-table entriesDavid Young2020-05-111-20/+53
| | | | | overlapped by the freed region. Previously, the page-table entries were only freed on one success path through _xfree_impl().
* Split VFD_SWMR_TEST_FOR_END_OF_TICK() into VFD_SWMR_ENTER() andDavid Young2020-05-091-12/+25
| | | | | | VFD_SWMR_LEAVE() for use by FUNC_ENTER_API/_LEAVE_API macros. In the macros, don't HGOTO_ERROR(), since that will jump back to the `out` label, but HDONE_ERROR() on error, instead.
* Let the reader-side tick instrumentation know about the first tick readDavid Young2020-04-231-0/+2
| | | | from the shadow file.
* Relax an assertion condition.David Young2020-04-231-1/+1
|
* Bug fix: in the freespace manager, use a new routine,David Young2020-04-233-5/+55
| | | | | H5PB_remove_entries(), to remove *all* pages overlapped by the freed space, instead of just the first one.
* Delete an assertion that doesn't hold true for multipage entries.David Young2020-04-211-3/+0
|
* Prevent a spurious assertion by using the correct index, oops!David Young2020-04-211-1/+1
|
* Straggler from last: provide stub implementationsDavid Young2020-04-211-0/+27
| | | | | of vfd_swmr_writer_may_increase_tick_to() and vfd_swmr_reader_did_increase_tick_to().
* Add a second bool argument to H5F_vfd_swmr_writer_end_of_tick() thatDavid Young2020-04-217-17/+31
| | | | | | | | | | | | | | | tells whether the call may wait for the reader tick to catch up. Add stub routines vfd_swmr_writer_may_increase_tick_to() and vfd_swmr_reader_did_increase_tick_to() for tests---e.g., vfd_swmr_zoo_writer/_reader---to use to coordinate their tick numbers. vfd_swmr_writer_may_increase_tick_to(new_tick, wait_for_reader) returns true if the writer may increase its tick number to `new_tick` without overrunning the reader. A reader uses vfd_swmr_reader_did_increase_tick_to() to tell a writer that its tick number has increased.
* Condense H5F_vfd_swmr_close_or_flush() for readability: removeDavid Young2020-04-211-25/+14
| | | | | | unnecessary comments and trim whitespace. Remove unnecessary backslash line continuations and add semicolons to HGOTO_ERROR() statements. NFCI.
* Retire globals vfd_swmr_writer_g and end_of_tick_g.David Young2020-04-202-24/+2
|
* Stragglers from previous: globally declare some hlog outlets and addDavid Young2020-04-161-1/+4
| | | | declare some new functions.
* Add a log outlet for metadata cache (MDC) invalidations,David Young2020-04-163-39/+82
| | | | | | | | | | | | | | | | | | | | | | | `mdc_invalidation`, and use it to log a message when H5C_evict_or_refresh_all_entries_in_page() does not find any affected entries. Pass a page length to H5C_evict_or_refresh_all_entries_in_page() so that it can assert if a multipage eviction overlaps a single-page entry, which had better not happen. Fix a bug in H5F_vfd_swmr_reader_end_of_tick() and heavily rework it: remove page-table entries and evict/refresh MDC entries that overlap *added* shadow-index entries. Because we didn't do that before, in the zoo test, the reader didn't see all of the changes made by the writer until the writer closed the file: MDC entries covered the new content in the shadow file. In H5F_vfd_swmr_reader_end_of_tick(), log changes to the shadow index with the new outlet, `shadow_index_update`. Convert a some of John's disused diagnostic printfs to an `hlog_fast(eot, ...)` call.
* Delete unused line, shorten a comment, wrap some lines, add missingDavid Young2020-04-161-12/+13
| | | | semicolons and curly braces. NFCI.
* Re-enable reclamation of disused shadow-index entries.David Young2020-04-161-6/+4
|
* Remove unnecessary pointer test: it cannot possibly be NULL.David Young2020-04-161-6/+4
|
* Remove gratuitous initialization, comment, and assertions.David Young2020-04-161-6/+2
|
* Extract a subroutine, H5F_vfd_swmr_process_eot_queue(), from theDavid Young2020-04-162-46/+58
| | | | | | | | | end-of-tick processing macro. H5F_vfd_swmr_process_eot_queue() looks for files due for end-of-tick processing and calls either the reader or writer EOT routine. Always call the reader/writer EOT routines with an actual H5F_t instead of NULL.
* Improve diagnostic logging.David Young2020-04-161-5/+7
|
* Oops, properly the outlet symbol-name prefix with the rest of the symbolDavid Young2020-04-161-6/+8
| | | | | name, so the symbols are not prefixed with `HLOG_PREFIX` but with `hlog_gbl_`.
* When hlog_set_state() changes the state of an outlet, reset eachDavid Young2020-04-161-0/+11
| | | | outlet's cached state so that its state is reevaluated.
* Log page-buffer reads and writes no matter what storage type. (Used toDavid Young2020-04-161-8/+4
| | | | log only the global heap-type reads and writes.)
* Put multiline if-body in curly braces, repair indentation, removeDavid Young2020-04-141-7/+9
| | | | gratuitous backslash line continuation, add missing semicolons.
* Wrap a line before 80 characters. NFCI.David Young2020-04-141-2/+2
|
* Change the prefix for hlog_outlet_t's from log_ to hlog_gbl_ to avoidDavid Young2020-04-131-5/+7
| | | | | namespace pollution. Use a `#define HLOG_PREFIX hlog_gbl_` so that I can change all of the prefixes in one place if need be.
* Add a log outlet, `tick`, and log some changes to the tick number there.David Young2020-04-131-5/+14
| | | | | While I'm here, assert in the reader that the tick number hasn't leapt forward by more than max_lag ticks.
* `tick_num_g` has been gone for a while, so do not refer to it inDavid Young2020-04-131-6/+6
| | | | comments any longer.
* Add clean_shadow_index() for removing shadow-index entries that wereDavid Young2020-04-131-4/+51
| | | | | flushed more than max_lag ticks ago. This conserves space in the shadow file, which would grow great big while the `credel` demo ran.
* Add H5MV split method for alignment.vchoi2020-04-131-24/+57
|
* Check whether or not the shadow entry size is different from theDavid Young2020-04-131-2/+15
| | | | | | | | | | | pagebuffer entry size. If the sizes are different, then release the old shadow space (using the correct size) and set the shadow page number to 0 so that H5F_update_vfd_swmr_metadata_file() will allocate new shadow space with the right size. Vailin says that this fixes the bug she found, where a 4096-byte buffer allocated by H5MV_alloc() is released with H5MV_free() as if it was 8192 bytes long.
* Be brief: remove commas and equal signs from diagnostic log messages.David Young2020-04-081-18/+19
|
* Fix spelling in a comment.David Young2020-04-081-1/+1
|
* Add a new log outlet and change an `fprintf(stderr,` to anDavid Young2020-04-071-1/+2
| | | | `hlog_fast(lengthen_pbentry,`. This quiets one of the VFD SWMR tests.
* Straggler from previous: declare the new shadow_index_reclaim logDavid Young2020-04-071-0/+1
| | | | outlet.
* Simplify H5PB_vfd_swmr__update_index(), which figures inDavid Young2020-04-071-21/+23
| | | | | | | | | | | | | | | | | my investigation of shadow-file freespace leaks. Copy the tick number from the H5F_shared_t to a temporary variable and use the temporary instead of spelling out `shared->tick_num`. Assert that every entry on the tick list is dirty, since clean entries will not appear on the tick list. Mark each corresponding shadow-index entry dirty, and set its "tick of last change" to the current tick and "tick of last flush" to 0. This makes the scan for entries that do not appear on the tick list more believable. Lower staircase in the loop that scans for shadow-index entries that did not appear on the tick list. Log indicies that are marked for reclamation.
* Declutter: remove superfluous comment and assertion. NFCI.David Young2020-04-071-2/+0
|
* Remove extraneous whitespace. NFCI.David Young2020-04-071-2/+0
|
* After adding a shadow page to the deferred free list, set the entry'sDavid Young2020-04-071-3/+5
| | | | page number to 0 so that we cannot free it again by accident.
* Add a log outlet and shorten the name of an existing outlet.David Young2020-04-071-2/+7
|
* Fix the width of an index variable.David Young2020-04-071-1/+1
|
* Make some changes to conserve filespace and ensure that all filespace isDavid Young2020-04-062-2/+8
| | | | | | | | | | | | | | | | eventually reclaimed. Defer reclamation of raw data filespace, only. XXX Deferring only raw-data reclamation isn't *quite* sufficient because XXX a writer could conceivably reuse metadata space as raw-data space XXX before max_lag ticks have elapsed. Readers could see metadata XXX corrupted by raw data. Once a file starts to close, stop deferring reclamation. In H5MF_free_aggrs(), perform all deferred reclamations if the file is closing.
* Improve diagnostic logging.David Young2020-04-021-7/+26
|
* Standardize and shorten diagnostic messages to help my search for theDavid Young2020-04-021-11/+11
| | | | leak of shadow-file space.
* Remove superfluous comment. Remove unnecessary variableDavid Young2020-04-021-3/+3
| | | | | initializations: the compiler will tell us if the variables are used before they're set. NFCI.
* Remove superfluous whitespace, insert whitespace around / operator,David Young2020-04-021-5/+3
| | | | repair for-loop indentation. NFCI.
* Update comments, shorten comments, remove superfluous comments.David Young2020-04-021-14/+5
|
generated by cgit v0.12 at 2024-07-19 20:26:00 (GMT)