summaryrefslogtreecommitdiffstats
path: root/src/H5Dchunk.c
Commit message (Collapse)AuthorAgeFilesLines
* Partial fix for HDFFV-10792Jordan Henderson2020-02-211-4/+45
|
* Squashed commit of the following:David Young2020-01-311-53/+48
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | commit ff7a9a34de2fcd78f5d32a1532f3d0e224b20d94 Author: Kimmy Mu <kmu@hdfgroup.org> Date: Wed Jan 29 14:12:18 2020 -0600 Merge pull request #2324 in HDFFV/hdf5 from ~KMU/hdf5:hdf5_1_12 to hdf5_1_12 * commit '781fa57f6ee816e037dd12fff02d37ff7ec9c2b1': pick up missing piece from merge Merge pull request #2311 in HDFFV/hdf5 from ~KMU/hdf5:misc to develop remove redundent definition Merge pull request #2311 in HDFFV/hdf5 from ~KMU/hdf5:misc to develop Merge pull request #2311 in HDFFV/hdf5 from ~KMU/hdf5:misc to develop Merge pull request #2315 in HDFFV/hdf5 from ~KMU/hdf5:develop to develop missing prototype warning prototype issue Merge pull request #2306 in HDFFV/hdf5 from ~KMU/hdf5:develop to develop Merge pull request #2300 in HDFFV/hdf5 from ~KMU/hdf5:squashed_cast to develop Merge pull request #2291 in HDFFV/hdf5 from ~KMU/hdf5:bugfix/uninitialized to develop Merge pull request #2295 in HDFFV/hdf5 from ~KMU/hdf5:bugfix/float to develop Merge pull request #2292 in HDFFV/hdf5 from ~KMU/hdf5:bugfix/prototype to develop Merge pull request #2094 in HDFFV/hdf5 from ~KMU/hdf5:bugfix/unused to develop commit 9f3d808c05d2a3b308f32c958c6671fcac0a432c Author: David Young <dyoung@hdfgroup.org> Date: Wed Jan 29 11:43:55 2020 -0600 testpar/t_2Gio.c: Fix a typo that I think was introduced by a previous warnings PR. An array element was assigned to itself---shape[2]Â =Â shape[2];---instead of being assigned to chunk[2]. fortran/src/H5Pf.c: move conditional compilation controlled by H5_NO_DEPRECATED_SYMBOLS outside of a function for readability. fortran/src/H5match_types.c: put a variable's declaration under the same conditional compilation (H5_FORTRAN_HAVE_C_LONG_DOUBLE) as its use. For now, skip compilation of some unused debug dump routines in the JNI. While I'm in the JNI, delete a set-but-unused variable. src/H5Z.c: condition a variable declaration on H5_NO_DEPRECATED_SYMBOLS so that it's not declared but unused or vice versa. test/cache_common.h: add an #include in to get some symbols we need to avoid implicit declaration warnings. test/dsets.c: use a more conventional conditional-compilation syntax. test/dt_arith.c, test/fillval.c: initialize a bunch of uninitialized variables before use. test/vfd.c: pass the expected type of `void **` to posix_memalign(3) instead of `int **`. testpar/t_bigio.c: explicitly compare with 0 instead of using ! when "equal to 0?" is the question not "is false?" Repair some indentation while I'm here. testpar/testpar.h: repair misaligned line-continuation backslashes in a macro that probably should be a function so that we don't have to fiddle with the line continuation to begin with. tools/src/h5repack/h5repack_main.c: fix some compiler fussing about enums. tools/test/perform/pio_engine.c: the compiler fusses if you cast a function call returning double directly to off_t. It's ok if you cast a variable that's a double to off_t, however. Write and use a new function, sqrto(), to avoid the cast warnings. commit 43a4b80b21c0329fbc1f9a0e2854795189f0bb50 Author: David Young <dyoung@hdfgroup.org> Date: Wed Jan 29 10:47:30 2020 -0600 Reduce casts of HDcalloc()/HDmalloc() that -Wc++-compat required. Reduce gratuitous casts---e.g., (size_t)1. Use the right format string for a pointer. In the H5C sanity checks, change a "size increase" variable from ssize_t (too narrow) to int64_t (wide enough). Parenthesize every appearance of `storage` in the macro `H5D_CHUNK_STORAGE_INDEX_CHK(storage)` so that you can pass in an expression like &sc and it works properly. Disallow re-assignment of the `dset` parameter to H5D__chunk_init() because it helped assure me that it's safe to replace the repeating expression `&dset->shared->layout.storage.u.chunk` with `sc` throughout. Replace lengthy expressions such as `&dset->shared->layout.storage.u.chunk` with `sc` throughout several functions in H5Dchunk.c ISTR that the compiler warned that `sc` was declared but unused in a couple of functions, and then I found that `sc` could be used in many places. Maybe the disused `sc` appeared because a bunch of code was copied and pasted, I don't know. Anyway, it's a lot tighter code now that I use `sc`. In H5D__chunk_update_old_edge_chunks() and H5D__chunk_delete() I actually expand `sc` and another temporary variable, `pline`, because they're used only in !defined(NDEBUG) code. This squashes unused-variable warnings in the defined(NDEBUG) configuration. Don't drop the `volatile` qualification with a cast in tools/src/h5import/h5import.c. commit 035bfa0769a4012881fc659b673bacf44d146537 Author: David Young <dyoung@hdfgroup.org> Date: Tue Jan 28 20:35:28 2020 -0600 Reduce differences between my -Werror branch and `develop`: Rename index -> idx, fileno -> fnumber, fileno -> fno to avoid GCC shadowed declaration warnings about index(3). Convert #pragma GCC diagnostic push/pop/ignored to the HDF5 library's H5_GCC_DIAG_OFF()/H5_GCC_DIAG_ON() macros.
* Merge pull request #2300 in HDFFV/hdf5 from ~KMU/hdf5:squashed_cast to developKimmy Mu2020-01-281-17/+14
| | | | | | * commit 'af5c33afabdae2e39bb45eb1b3e9c8366da01145': remove unnecessary stuff squash cast warning fix
* Merge pull request #2094 in HDFFV/hdf5 from ~KMU/hdf5:bugfix/unused to developKimmy Mu2020-01-281-1/+1
| | | | | | | | | | | * commit '145ef3ceee20c28a443bd11507c58858bea3f889': more cleanup change it back pick up missing piece fix unused related warnings removed unused parameter more fix and address comments remove unsed var,function,macro, etc
* Merge pull request #2071 in HDFFV/hdf5 from ~KMU/hdf5:bugfix/intel_warnings ↵Kimmy Mu2020-01-081-8/+6
| | | | | | | | | | | | | | | | | | | | | | to develop * commit '0a2bb11b248df6841daabca3970df5d8504adfc7': address problems from comments fix and address comments change according to previous comments add missing piece remove unnecessary check macro fix intel compile warnings Revert "fix warnings from Intel compiler" Revert "fix warnings and some text alignment" Revert "let hdf5 pick up the right compiler in Intel environment" Revert "fix issues from previous PR comments" Revert "using a different MACRO" using a different MACRO fix issues from previous PR comments let hdf5 pick up the right compiler in Intel environment fix warnings and some text alignment fix warnings from Intel compiler
* Cleanups from PR reviewsQuincey Koziol2020-01-041-1/+0
|
* Refactor all the 'H5VL_*_optional' callbacks to move the type of operation outQuincey Koziol2020-01-041-67/+52
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | of the va_list, so it's at least possible for another connector to know what the operation is and decide whether to implement it or not. Added a new VOL sub-class called "introspect" where callbacks that report information about the connector or container can be placed. Added an 'opt_query' callback to this sub-class, for a connector to report back to the library whether a particular optional callback operation is supported. Also added a 'get_conn_cls' introspection callback, to retrieve the H5VL_class_t of a connector (either the "current" connector, H5VL_GET_CONN_LVL_CURR, or the terminal connector, H5VL_GET_CONN_LVL_TERM). Moved the "post open" operation from a file 'specific' operation to a file 'optional' operation, now that it's possible to detect (with the 'opt_query' introspection callback) whether a VOL connector implements an optional operation, without just returning an error. Added new internal VOL helper routines: H5VL_object_is_native, to determine if an object is in (or is a) native file, and H5VL_file_is_same, to determine if two objects are in (or are) the same terminal VOL connector's container. (And moved the special handling for FILE_IS_EQUAL operation out of internal VOL callback routine into H5VL_file_is_same) Made new dataset 'get' operation for H5Dvlen_get_buf_size, aligning it better with other 'get' operations in API. Fixed several issues with pass-through connectors, which are now passing the 'make check-passthrough-vol' tests again. A bunch of warning and style cleanups as well.
* Fix 2010 compile issueAllen Byrne2019-12-111-1/+1
|
* Revert "Merge branch 'hdf5_1_12' of ↵Jerome Soumagne2019-12-091-15/+8
| | | | | | | https://bitbucket.hdfgroup.org/scm/hdffv/hdf5 into hdf5_1_12" This reverts commit 9f9336a5bd541752f472bab4c93da8de89f862cd, reversing changes made to 437a1919e7ba60fe75a33a466d264183a0255319.
* Replace H5Sselect_adjust_u() and H5Shyper_adjust_s() withNeil Fortner2019-12-061-2/+1
| | | | | H5Sselect_adjust. Implement "adjust_s" callback for all selection types. Add range checking to H5Sselect_adjust().
* Merged HDFFV-10929_2GB_testing to hdf5_1_12Richard Warren2019-12-061-9/+16
|
* Merge pull request #1990 in HDFFV/hdf5 from ~JSOUMAGNE/hdf5:references_1_12 ↵Larry Knox2019-10-091-10/+2
|\ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | to hdf5_1_12 * commit '5d2545ee26d4b7013ed363545705f16a67087549': Fix func enter macro in H5T_ref_reclaim() Update RELEASE.txt for reference changes Fix reference type comparison in h5dump Make wrappers, tests and tools use H5Treclaim() instead of H5Dvlen_reclaim() Add new H5R API that abstracts object, region and attribute reference types Remove ability to loc by ref from H5VL layer Add support for retrieving object name by token Add H5VL_OBJECT_GET_TYPE to get object type Add H5VL_MAX_TOKEN_SIZE and H5VL_token_t Adapt Jerome's "file info" H5VL 'get' query to retrieve container token info. Fix H5VL_blob_get to return size of blob Add 'blob' callbacks to VOL, along with a native implementation to store them in the global heap, and changed the VL datatype conversion code to use blobs.
| * Add new H5R API that abstracts object, region and attribute reference typesJerome Soumagne2019-10-081-10/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Also support references to external files Add new H5T_REF type and type conversion routines Support conversion from H5T_REF_OBJ/DSET_REG to H5T_REF Add H5Treclaim() API to reclaim memory of vlen/reference types Deprecate H5Dvlen_reclaim() Fix H5T_vlen_reclaim() and H5T_reclaim() to use private callback Add H5T_ref_reclaim() Move previous H5R APIs to H5Rdeprec.c Clean up H5Ocopy Separate H5O_copy_expand_ref() to H5Ocopy_ref() Add support for copying new reference types Clean up deprecated routines to go through VOL and same code path Fix return codes in existing trefer.c test Rename trefer.c to trefer_deprec.c trefer.c is for new references Add performance test for trefer Add additional obj_copy_ref test Make use of tokens and blobs to store references Skip blob encoding for object references Start adding new reference examples
* | Merge pull request #1934 in HDFFV/hdf5 from ↵Vailin Choi2019-09-301-36/+168
|/ | | | | | | | ~VCHOI/my_third_fork:bugfix/HDFFV-10585-investigate-slowness-of-regular to develop * commit '7924eee0e5ee0745b784c635042b8633886fb799': (1) Address the feedback from the PR review (2) Add release notes Fix for HDFFV-10585 investigate hyperslab slowness: 1) Improve hyperslab performance when doing I/O from 1-d disjoint file dataspace to 1-d contiguous memory dataspace. 2) Move coding in H5D__chunk_io_init() that is constructing the chunk mappings to a separate routine.
* Updated based on reviews and refactored test codeBinh-Minh Ribler2019-09-011-3/+2
|
* Code cleanup and refactorBinh-Minh Ribler2019-08-291-4/+0
|
* HDFFV-10677 and HDFFV-10661Binh-Minh Ribler2019-08-281-0/+392
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Description: - Added functions to query chunk information: H5Dget_num_chunks(dset_id, fspace_id, *nchunks) Gets the number of written chunks that intersect with the given dataspace. However, in this version, the intersection is not yet completed. Thus, the number of all written chunks will be returned. H5Dget_chunk_info_by_coord(dset_id, *offset, *filter_mask, *addr, *size) Given a chunk's logical coordinates, returns the chunk's filter, address, and size. H5Dget_chunk_info(dset_id, fspace_id, index, *offset, *filter_mask, *addr, *size) Given a chunk's index, returns the chunk's logical coordinates, filter, address, and size. The chunk belongs to a set of chunks that have nonempty intersection with the specified dataspace. However, in this version, the intersection is not yet completed, and the index is of all the written chunks. These functions comply with VOL. - Fixed some oversights found in the library for the tests in chunk_info.c to work correctly. The returned value from a callback function was not checked in H5EA_iterate(), H5FA_iterate(), and H5D__none_idx_iterate(). This oversight caused a callback function to continue iterating even though it's supposed to stop. Platforms tested: Linux/64 (jelly) Linux/64 (platypus) Darwin (osx1011test)
* Progress toward moving the dataset routines to using the 'shared' file pointerQuincey Koziol2019-08-211-11/+14
| | | | instead of the 'top' file pointer.
* Add H5Sselect_shape_same and H5Sselect_intersect_block API routines, alongQuincey Koziol2019-07-301-7/+3
| | | | with tests and minor cleanups and refactorings.
* Add support for GCC9, update warnhist script, and clean up warnings.Quincey Koziol2019-07-031-3/+5
|
* Updated configure & CMake compiler flags for GCC 8.x, along with correspondingQuincey Koziol2019-06-281-2/+2
| | | | | changes to warnhist script (and some extra improvements for condensing C++ and Java warnings), and fixed a bunch of warnings.
* Merge branch 'develop' of ↵Binh-Minh Ribler2019-05-051-93/+279
|\ | | | | | | https://bitbucket.hdfgroup.org/scm/~bmribler/hdf5-bmr into develop
| * Merge remote-tracking branch 'origin/develop' into merge_hyperslab_update_01Quincey Koziol2019-03-171-25/+59
| |\
| * | Core changes to selection code from the hyperslab_updates branch.Quincey Koziol2019-03-161-93/+279
| | |
* | | Code improvementBinh-Minh Ribler2019-05-051-3/+6
| |/ |/| | | | | | | | | | | | | | | | | Description: Fixed potential division by zero occurrences and changed an assert to if statement. Platforms tested: Linux/64 (jelly) Linux/64 (platypus) Darwin (osx1011test)
* | Added an H5MM_memcpy call that checks for buffer overlap.Dana Robinson2019-03-161-21/+21
| |
* | Fix issue with direct chunk write not updating the "last chunk" indexNeil Fortner2019-03-061-3/+37
|/ | | | | cache. Fix issues involving datasets being "no allocated" when they contain cached raw data.
* Merge branch 'develop' into c_stdlib_posix_call_cleanupDana Robinson2019-02-191-70/+53
|\
| * Align develop with incoming hyperslab_updates branch changes.Quincey Koziol2019-02-131-48/+41
| |
| * Fix some collective metadata read issuesJordan Henderson2019-02-121-16/+6
| |
| * Delta reduction against hyperslab_updates branch.Quincey Koziol2019-01-111-6/+6
| |
* | C and POSIX call cleanupDana Robinson2019-01-111-7/+8
|/
* Removed a comment.Binh-Minh Ribler2019-01-031-1/+0
|
* Fixed CVE division-by-zero issuesBinh-Minh Ribler2018-12-231-3/+26
| | | | | | | | | | | | | | | | | Description: Fixed HDFFV-10577 and similar issues found in H5Dchunk.c. All the occurrences are in: H5D__create_chunk_map_single H5D__create_chunk_file_map_hyper H5D__chunk_allocate H5D__chunk_update_old_edge_chunks H5D__chunk_prune_by_extent H5D__chunk_copy_cb H5D__chunk_collective_fill Platforms tested: Linux/64 (jelly) Linux/64 (platypus) Darwin (osx1010test)
* Merge pull request #1335 in HDFFV/hdf5 from ↵Ray Lu2018-11-191-0/+3
|\ | | | | | | | | | | | | | | | | | | | | ~SONGYULU/hdf5_ray:bugfix/HDFFV-10571-cve-2018-17237-divided-by-zero to develop * commit 'c923cdad6e515c842f3795a5b6d754ad94021e09': HDFFV-10571: Minor format changes. HDFFV-10571: Minor change - reformatting the error check. HDFFV-10571: Minor change - adding the error check right after decoding of chunk dimension for safeguard. HDFFV-10571: Minor change - revised the comment to be clearer. HDFFV-10571 Divided by Zero vulnerability. Minor fix: I added an error check to make sure the chunk size is not zero.
| * HDFFV-10571: Minor format changes.Songyu Lu2018-11-151-1/+0
| |
| * HDFFV-10571: Minor change - adding the error check right after decoding of ↵Songyu Lu2018-11-151-4/+4
| | | | | | | | chunk dimension for safeguard.
| * HDFFV-10571: Minor change - revised the comment to be clearer.Songyu Lu2018-11-141-1/+1
| |
| * HDFFV-10571 Divided by Zero vulnerability. Minor fix: I added an error ↵Songyu Lu2018-11-141-0/+4
| | | | | | | | check to make sure the chunk size is not zero.
* | HDFFV-10601: I changed to a better way to calculate the number of chunks in ↵Songyu Lu2018-11-081-5/+3
| | | | | | | | a dataset.
* | HDFFV-10601 Issues with chunk cache hash value calcuation:Songyu Lu2018-11-021-12/+12
|/ | | | | | | | 1. H5D__chunk_hash_val: When the number of chunks in the fastest changing dimension is larger than the number of slots in the hash table, H5D__chunk_hash_val abandons the normal hash value calculation algorithm and simply uses the scaled dimension. This will cause chunks a selection that cuts across chunks in dimensions other than the fastest changing to all have the same hash value, and they will therefore always evict each other from the cache, having an obvious major performance impact. Eliminated the check for the number of slots in this function and always use the full algorithm. 2. H5D__chunk_init: When the scaled dimensions (number of chunks in each dimension) are calculated in H5D__chunk_init, a simple divide ("/") operator is used with the dataset size in elements and the chunk size in elements. While this is fine when the dataset size is an exact multiple of the chunk size, in other cases, since "/" rounds down, it results in a scaled dimension one less than it should (it ignores the partial edge chunk). This has trickle down effects on hash value calculation that can cause excess hash value collisions and therefore performance issues. Changed the calculation to (((dataset_size - 1) / chunk_size) + 1). Tested the build with Autotool and CMake.
* A trivial change to reformat a comment. The purpose is simply to try out Git.Songyu Lu2018-10-311-2/+2
|
* Remainder of vol_normalization changes (dataset, attribute, files, objects).Dana Robinson2018-09-241-37/+36
|
* Removed 'VOL' versions of FUNC_ENTER macros as well as empty functionsDana Robinson2018-09-181-4/+4
| | | | orginally intended to support the full SWMR feature.
* Merge pull request #1134 in HDFFV/hdf5 from ~BMRIBLER/hdf5_bmr_cpp4:develop ↵Binh-Minh Ribler2018-07-161-0/+2
|\ | | | | | | | | | | | | | | | | | | to develop Fixed HDFFV-10481 and HDFFV-10477 * commit '83ca39ba9a2d6852dd7754b533f7839e9d2a0107': Combined macro lines as Dana commented Platforms tested: Linux/64 (jelly) (very minor) Fixed division-by-zero issues Description: Fixed HDFFV-10481 and HDFFV-10477, division by 0. Fixed another occurrence beside what were reported. Also, changed a local variable to avoid an unnecessary cast. Platforms tested: Linux/64 (jelly) Linux/32 (jam) Darwin (osx1010test)
| * Combined macro lines as Dana commentedBinh-Minh Ribler2018-07-161-2/+1
| | | | | | | | | | Platforms tested: Linux/64 (jelly) (very minor)
| * Fixed division-by-zero issuesBinh-Minh Ribler2018-07-161-0/+3
| | | | | | | | | | | | | | | | | | | | | | Description: Fixed HDFFV-10481 and HDFFV-10477, division by 0. Fixed another occurrence beside what were reported. Also, changed a local variable to avoid an unnecessary cast. Platforms tested: Linux/64 (jelly) Linux/32 (jam) Darwin (osx1010test)
* | Merge pull request #1127 in HDFFV/hdf5 from ~JHENDERSON/hdf5:develop to developJordan Henderson2018-07-161-2/+35
|\ \ | |/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'a8d6f100cdddbfc42d0c4abfeb4ceb8788b1b087': Add note about single chunk caching and serial library Add check for actually using the MPI file driver when caching one chunk Fix error message mentioning wrong MPI function used Fix for HDFFV-10509 Revise H5D__mpio_array_gatherv() to not allocate memory needlessly Add test to continually grow and shrink chunks Changes to test with checksum filter as well as deflate filter Eliminate warning about signed to unsigned conversion Remove unused local variable Fix bug in parallel reads of compressed data Add data verification to parallel filtered compound write tests Add seven of fourteen parallel filtered data partial read tests
| * Add note about single chunk caching and serial libraryJordan Henderson2018-07-131-1/+7
| |
| * Add check for actually using the MPI file driver when caching one chunkJordan Henderson2018-07-131-1/+15
| | | | | | Better explain rationale behind chunk caching issue fix
generated by cgit v0.12 at 2024-09-18 02:34:52 (GMT)