summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
* Prepare a bit to let the shadow index "float": drasticallyDavid Young2020-01-101-53/+83
| | | | | | | | | | | | | | simplify H5FD__vfd_swmr_index_deserialize(): reuse h5_retry_init()/h5_retry_next() for retry loops. Don't wait for the fstat(2) to read the correct size, because the read(2) will return short if the file isn't long enough. (This change should save at least one system call, always.) Leave a bunch of comments about the changes that I will have to make so that the shadow index will float. NFCI: do not cast H5MM_malloc() return values, this is not C++.
* Now that we're buffering the removed pages to process at once,David Young2020-01-101-23/+19
| | | | | | | it's not necessary to iterate over pass numbers any more, so just use two loops. While I'm in here, change a comment or formatting here and there.
* H5F_vfd_swmr_reader_end_of_tick(),David Young2020-01-101-133/+98
| | | | | | | | | | | | | | | | | compares both new and old shadow indices and calls H5PB_remove_entry() on each entry that was in the old index but is not in the new. Ever since H5PB_remove_entry() started removing shadow index entries, it has been possible for H5F_vfd_swmr_reader_end_of_tick() to walk past the end of the new shadow index or even to skip entries in the new index. Sometimes an assertion failed when that happened. I have restructured the code in H5F_vfd_swmr_reader_end_of_tick() so that it compares the old and new indices, gathering a list of removed pages, in one step. In the next step, it processes the list of removed pages, calling H5PB_remove_entry() on each page. In the step after that, it notifies the metadata cache of each removed page. This fixes the bug I described, above.
* Simplify nested logic in H5FD_vfd_swmr_get_tick_and_idx().David Young2020-01-101-9/+12
| | | | | assert() our expectation that either a NULL index was passed, or a pointer to the index length was passed in, too.
* Correct some irregular indentation, delete an unused line, split anDavid Young2020-01-101-12/+10
| | | | | | | `if ((p = allocate(...)) == NULL) { }` into two statements, `p = allocate(...); if (p == NULL) { }`, put a semicolon at the end of an HGOTO_ERROR(), remove comments /* end if */, /* end for */ after closing curly braces.
* Make H5FD__vfd_swmr_load_hdr_and_idx() simpler and faster: copy aDavid Young2020-01-101-56/+30
| | | | | | pointer to the metadata index instead of copying the index itself. Use struct assignment instead of copying individual struct members. Lower a staircase.
* After quick-sorting the index, go ahead and assert that there are notDavid Young2020-01-101-5/+2
| | | | | | duplicate entries for the same HDF5 page offset. There's an O(n) cost to that, but it should be no more than the O(n log n) cost of the quicksort.
* In H5F__vfd_swmr_writer__create_index(), use calloc(3) instead ofDavid Young2020-01-101-18/+1
| | | | malloc(3) and a for-loop that zeroes the index.
* In H5PB_remove_entry(), do not call vfd_swmr_mdf_idx_entry_remove(),David Young2020-01-101-29/+24
| | | | | because the H5PB__evict_entry() call should have already done that. Instead, just assert() that the index entry is not present.
* In vfd_swmr_mdf_idx_entry_remove(), give some local variables less wordyDavid Young2020-01-101-14/+12
| | | | | | names. While I am here, do not copy the last element of the index over the element that's being deleted, because in the very next step I'm shifting all elements over by one.
* Straggler from previous: provide uint64_decode().David Young2020-01-101-0/+14
|
* Clean up the code a bit: remove some unnecessary casts and such. NFCI.David Young2020-01-101-4/+4
|
* Change the blah_blah_blah_md_header `index_length` member fromDavid Young2020-01-103-3/+13
| | | | | | | | `uint64_t` to `size_t` because it describes the size of an in-core structure as well as an on-disk one, and `size_t` is wide enough to store the size of any in-core structure, while `uint64_t` may be much too wide. Check that `index_length` is no more than SIZE_MAX after we read it.
* Delete unused constants.David Young2020-01-101-2/+0
|
* Be brief, replace `(H5FS_section_info_t *)node` with `&node->sect_info`.David Young2020-01-101-7/+7
|
* Provide a convenience constant, H5_RETRY_ONE_SECOND.David Young2020-01-101-1/+2
|
* Disable C++ compatibility warnings to follow `develop` on the main fork.David Young2020-01-101-0/+2
|
* Update punchlist a bit with John: add some items, update entries that are nowDavid Young2020-01-091-8/+19
| | | | complete.
* Add a new version of the RFC with some commentary by me.David Young2020-01-081-0/+0
|
* Makes some updates: in-progress, needs unit testing. Insert some whitespaceDavid Young2020-01-061-7/+7
| | | | where it makes things more readable.
* Mention the current situation concerning raw-data flushing.David Young2019-12-311-0/+3
|
* Mention that I fixed the VFD SWMR tests.David Young2019-12-301-3/+3
|
* Don't cast a malloc(3), this is C. Put curly braces around a multiline `if`David Young2019-12-201-5/+5
| | | | statement. Delete an unnecessary backslash line continuation. NFCI.
* Assert that a shadow image is page-aligned since we assume that it is when weDavid Young2019-12-201-0/+2
| | | | compute its base address and free it.
* Delete unnecessary whitespace including invisible whitespace at the end ofDavid Young2019-12-201-4/+5
| | | | lines. Wrap a couple of lines. NFCI.
* Change the unwieldy name vfd_swmr_idx_entry_defer_free() toDavid Young2019-12-193-4/+4
| | | | | shadow_image_defer_free(), which is also a better description of what the routine does.
* Use the "tail queue" from queue(3) for the delayed free list of shadow images.David Young2019-12-196-143/+195
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | Delete the little-used free-list length, dl_len, and just count up the list entries when diagnostic code needs the length. Extract the code for deferring shadow-image free into a new subroutine, `vfd_swmr_idx_entry_defer_free()`. Rename type `deferred_free_head_t` as `deferred_free_queue_t`. Remove the disused H5F__LL_{REMOVE,PREPEND} macros. Add some diagnostic code and #if 0'd assertions. Change `qsort(ptr, n, sizeof(type), cmp)` to `qsort(ptr, n, sizeof(*ptr), cmp)`. Use a `continue` statement to lower a staircase in H5F_update_vfd_swmr_metadata_file(). Add vfd_swmr_mdf_idx_entry_remove() to delete a shadow index entry and add the image at that entry to a deferred-free list. Call it whenever a page is evicted. Update the comment in H5PB_remove_entry() that asks if we need to remove shadow index entries: now we *do* remove them. Remove shadow index entries in H5PB__evict_entry(). Also mention in the comment that the index-entry removal performed by H5PB__evict_entry() ought to be sufficient.
* Remove unnecessary casts. Use the `ptr = calloc(n, sizeof(*ptr))` idiom forDavid Young2019-12-191-4/+5
| | | | specifying the item size to be allocated. NFCI.
* Prepare for some changes I'm committing shortly: pass an `H5F_shared_t *` asDavid Young2019-12-191-10/+10
| | | | the first argument to H5PB__evict_entry() instead of an `H5PB_t *`.
* Make the first parameter of H5PB_remove_entry() non-const to support someDavid Young2019-12-192-2/+2
| | | | changes I will commit shortly.
* There's no need to name parameters in a function declaration, and it makes themDavid Young2019-12-181-2/+2
| | | | very long, so just delete the parameters.
* Allocate regionsin the shadow file of page size or greater size with pageDavid Young2019-12-181-1/+1
| | | | | | | alignment. The VFD SWMR code had always assumed that the regions were aligned to page size. It would blithely round the start addresses of regions to the next lower page. When the region was freed, the freespace manager (H5MV) would suffer an assertion or corruption.
* Tell the tick number in diagnostic messages.David Young2019-12-181-2/+3
|
* Align the shadow images on page boundaries so that they don't overlap. ThisDavid Young2019-12-181-8/+8
| | | | | seems to be the programmer's original intention. It sure makes debugging easier to have non-overlapping shadow images.
* In the faked-up shadow-index entries, assign independent page numbers in theDavid Young2019-12-181-4/+4
| | | | | | | HDF5 file and in the shadow file. I had added assertions that the page numbers were unique, and this caused those assertions to fail. I don't know if I'll keep the assertions, but this is an inexpensive change that makes the test more realistic.
* Allocate and zero using calloc(3) instead of malloc(3)+memset(3).David Young2019-12-181-2/+1
|
* Follow the naming convention enforced by FUNC_ENTER_/FUNC_EXIT_* macros (?)David Young2019-12-181-5/+5
| | | | | where static routines have to have double underscores (__) in their name or else the library asserts false.
* Delete a macro that's not used.David Young2019-12-131-26/+0
|
* Remove disused duplicate of H5F__idx_entry_cmp(). I probably introduced thatDavid Young2019-12-131-34/+0
| | | | | by accident when I merged Vailin's changes that moved some functions from one file to another.
* Remove the ugly casts on calloc() and malloc(). Change some malloc() toDavid Young2019-12-131-6/+6
| | | | calloc() so that all bytes are initialized to something.
* Fix some typos. NFCI.David Young2019-12-132-2/+2
|
* Wrap some comments at <= 80 characters.David Young2019-12-101-6/+7
|
* Shorten the type name `H5F_vfd_swmr_eot_queue_entry_t` to `eot_queue_entry_t`:David Young2019-12-104-102/+57
| | | | | | | people have to read and type this stuff! Use TAILQ_* macros instead of an unnecessary custom implementation of doubly-linked lists.
* Fix typo. NFCI.David Young2019-12-101-1/+1
|
* Remove disused #ifdef QAK code.David Young2019-12-095-72/+0
|
* Be brief: don't cast the last parameter of H5Pset_file_space_strategy(), theDavid Young2019-12-092-21/+17
| | | | constant `1`, to `hsize_t`.
* Don't see a 1GB threshold for tracking free filespace, that workaround for VFDDavid Young2019-12-092-2/+2
| | | | | SWMR was never 100% effective, and now that there is a delay line on filespace frees, it is not necesesary.
* Merge all of my changes from merge-back-to-feature-vfd_swmr-attempt-1,David Young2019-12-091185-87383/+167234
| | | | | | | | including the merge of `hdffv/hdf5/develop`, back to the branch that Vailin and I share. Now I need to put this branch on a fork with a less confusing name than vchoi_fork!
* Fix the FSM bug when setting the FSM threshold to a non-default value.Vailin Choi2019-12-022-1/+6
| | | | | | | Check for smaller or larger section size after merging and shrinking a section, for this case is the section that is smaller than threshold (see H5MF_xfree() in H5MF.c). It is possible for the section to be smaller after merging/shrinking (see H5MF__sect_large_shrink() in H5MFsection.c).
* Update status of the expand/shrink test.David Young2019-11-251-6/+7
|
generated by cgit v0.12 at 2024-09-19 13:57:57 (GMT)