summaryrefslogtreecommitdiffstats
path: root/src/H5Dchunk.c
Commit message (Collapse)AuthorAgeFilesLines
* Revert "fix issues from previous PR comments"kmu2019-11-261-4/+4
| | | | This reverts commit d242a900f420b040e364f6c0976c01593e955db3.
* fix issues from previous PR commentskmu2019-11-251-4/+4
|
* fix warnings from Intel compilerkmu2019-11-071-2/+1
|
* Add new H5R API that abstracts object, region and attribute reference typesJerome Soumagne2019-10-081-10/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Also support references to external files Add new H5T_REF type and type conversion routines Support conversion from H5T_REF_OBJ/DSET_REG to H5T_REF Add H5Treclaim() API to reclaim memory of vlen/reference types Deprecate H5Dvlen_reclaim() Fix H5T_vlen_reclaim() and H5T_reclaim() to use private callback Add H5T_ref_reclaim() Move previous H5R APIs to H5Rdeprec.c Clean up H5Ocopy Separate H5O_copy_expand_ref() to H5Ocopy_ref() Add support for copying new reference types Clean up deprecated routines to go through VOL and same code path Fix return codes in existing trefer.c test Rename trefer.c to trefer_deprec.c trefer.c is for new references Add performance test for trefer Add additional obj_copy_ref test Make use of tokens and blobs to store references Skip blob encoding for object references Start adding new reference examples
* (1) Address the feedback from the PR reviewVailin Choi2019-09-261-4/+3
| | | | (2) Add release notes
* Fix for HDFFV-10585 investigate hyperslab slowness:Vailin Choi2019-09-191-36/+169
| | | | | | | 1) Improve hyperslab performance when doing I/O from 1-d disjoint file dataspace to 1-d contiguous memory dataspace. 2) Move coding in H5D__chunk_io_init() that is constructing the chunk mappings to a separate routine.
* Updated based on reviews and refactored test codeBinh-Minh Ribler2019-09-011-3/+2
|
* Code cleanup and refactorBinh-Minh Ribler2019-08-291-4/+0
|
* HDFFV-10677 and HDFFV-10661Binh-Minh Ribler2019-08-281-0/+392
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Description: - Added functions to query chunk information: H5Dget_num_chunks(dset_id, fspace_id, *nchunks) Gets the number of written chunks that intersect with the given dataspace. However, in this version, the intersection is not yet completed. Thus, the number of all written chunks will be returned. H5Dget_chunk_info_by_coord(dset_id, *offset, *filter_mask, *addr, *size) Given a chunk's logical coordinates, returns the chunk's filter, address, and size. H5Dget_chunk_info(dset_id, fspace_id, index, *offset, *filter_mask, *addr, *size) Given a chunk's index, returns the chunk's logical coordinates, filter, address, and size. The chunk belongs to a set of chunks that have nonempty intersection with the specified dataspace. However, in this version, the intersection is not yet completed, and the index is of all the written chunks. These functions comply with VOL. - Fixed some oversights found in the library for the tests in chunk_info.c to work correctly. The returned value from a callback function was not checked in H5EA_iterate(), H5FA_iterate(), and H5D__none_idx_iterate(). This oversight caused a callback function to continue iterating even though it's supposed to stop. Platforms tested: Linux/64 (jelly) Linux/64 (platypus) Darwin (osx1011test)
* Progress toward moving the dataset routines to using the 'shared' file pointerQuincey Koziol2019-08-211-11/+14
| | | | instead of the 'top' file pointer.
* Add H5Sselect_shape_same and H5Sselect_intersect_block API routines, alongQuincey Koziol2019-07-301-7/+3
| | | | with tests and minor cleanups and refactorings.
* Add support for GCC9, update warnhist script, and clean up warnings.Quincey Koziol2019-07-031-3/+5
|
* Updated configure & CMake compiler flags for GCC 8.x, along with correspondingQuincey Koziol2019-06-281-2/+2
| | | | | changes to warnhist script (and some extra improvements for condensing C++ and Java warnings), and fixed a bunch of warnings.
* Merge branch 'develop' of ↵Binh-Minh Ribler2019-05-051-93/+279
|\ | | | | | | https://bitbucket.hdfgroup.org/scm/~bmribler/hdf5-bmr into develop
| * Merge remote-tracking branch 'origin/develop' into merge_hyperslab_update_01Quincey Koziol2019-03-171-25/+59
| |\
| * | Core changes to selection code from the hyperslab_updates branch.Quincey Koziol2019-03-161-93/+279
| | |
* | | Code improvementBinh-Minh Ribler2019-05-051-3/+6
| |/ |/| | | | | | | | | | | | | | | | | Description: Fixed potential division by zero occurrences and changed an assert to if statement. Platforms tested: Linux/64 (jelly) Linux/64 (platypus) Darwin (osx1011test)
* | Added an H5MM_memcpy call that checks for buffer overlap.Dana Robinson2019-03-161-21/+21
| |
* | Fix issue with direct chunk write not updating the "last chunk" indexNeil Fortner2019-03-061-3/+37
|/ | | | | cache. Fix issues involving datasets being "no allocated" when they contain cached raw data.
* Merge branch 'develop' into c_stdlib_posix_call_cleanupDana Robinson2019-02-191-70/+53
|\
| * Align develop with incoming hyperslab_updates branch changes.Quincey Koziol2019-02-131-48/+41
| |
| * Fix some collective metadata read issuesJordan Henderson2019-02-121-16/+6
| |
| * Delta reduction against hyperslab_updates branch.Quincey Koziol2019-01-111-6/+6
| |
* | C and POSIX call cleanupDana Robinson2019-01-111-7/+8
|/
* Removed a comment.Binh-Minh Ribler2019-01-031-1/+0
|
* Fixed CVE division-by-zero issuesBinh-Minh Ribler2018-12-231-3/+26
| | | | | | | | | | | | | | | | | Description: Fixed HDFFV-10577 and similar issues found in H5Dchunk.c. All the occurrences are in: H5D__create_chunk_map_single H5D__create_chunk_file_map_hyper H5D__chunk_allocate H5D__chunk_update_old_edge_chunks H5D__chunk_prune_by_extent H5D__chunk_copy_cb H5D__chunk_collective_fill Platforms tested: Linux/64 (jelly) Linux/64 (platypus) Darwin (osx1010test)
* Merge pull request #1335 in HDFFV/hdf5 from ↵Ray Lu2018-11-191-0/+3
|\ | | | | | | | | | | | | | | | | | | | | ~SONGYULU/hdf5_ray:bugfix/HDFFV-10571-cve-2018-17237-divided-by-zero to develop * commit 'c923cdad6e515c842f3795a5b6d754ad94021e09': HDFFV-10571: Minor format changes. HDFFV-10571: Minor change - reformatting the error check. HDFFV-10571: Minor change - adding the error check right after decoding of chunk dimension for safeguard. HDFFV-10571: Minor change - revised the comment to be clearer. HDFFV-10571 Divided by Zero vulnerability. Minor fix: I added an error check to make sure the chunk size is not zero.
| * HDFFV-10571: Minor format changes.Songyu Lu2018-11-151-1/+0
| |
| * HDFFV-10571: Minor change - adding the error check right after decoding of ↵Songyu Lu2018-11-151-4/+4
| | | | | | | | chunk dimension for safeguard.
| * HDFFV-10571: Minor change - revised the comment to be clearer.Songyu Lu2018-11-141-1/+1
| |
| * HDFFV-10571 Divided by Zero vulnerability. Minor fix: I added an error ↵Songyu Lu2018-11-141-0/+4
| | | | | | | | check to make sure the chunk size is not zero.
* | HDFFV-10601: I changed to a better way to calculate the number of chunks in ↵Songyu Lu2018-11-081-5/+3
| | | | | | | | a dataset.
* | HDFFV-10601 Issues with chunk cache hash value calcuation:Songyu Lu2018-11-021-12/+12
|/ | | | | | | | 1. H5D__chunk_hash_val: When the number of chunks in the fastest changing dimension is larger than the number of slots in the hash table, H5D__chunk_hash_val abandons the normal hash value calculation algorithm and simply uses the scaled dimension. This will cause chunks a selection that cuts across chunks in dimensions other than the fastest changing to all have the same hash value, and they will therefore always evict each other from the cache, having an obvious major performance impact. Eliminated the check for the number of slots in this function and always use the full algorithm. 2. H5D__chunk_init: When the scaled dimensions (number of chunks in each dimension) are calculated in H5D__chunk_init, a simple divide ("/") operator is used with the dataset size in elements and the chunk size in elements. While this is fine when the dataset size is an exact multiple of the chunk size, in other cases, since "/" rounds down, it results in a scaled dimension one less than it should (it ignores the partial edge chunk). This has trickle down effects on hash value calculation that can cause excess hash value collisions and therefore performance issues. Changed the calculation to (((dataset_size - 1) / chunk_size) + 1). Tested the build with Autotool and CMake.
* A trivial change to reformat a comment. The purpose is simply to try out Git.Songyu Lu2018-10-311-2/+2
|
* Remainder of vol_normalization changes (dataset, attribute, files, objects).Dana Robinson2018-09-241-37/+36
|
* Removed 'VOL' versions of FUNC_ENTER macros as well as empty functionsDana Robinson2018-09-181-4/+4
| | | | orginally intended to support the full SWMR feature.
* Merge pull request #1134 in HDFFV/hdf5 from ~BMRIBLER/hdf5_bmr_cpp4:develop ↵Binh-Minh Ribler2018-07-161-0/+2
|\ | | | | | | | | | | | | | | | | | | to develop Fixed HDFFV-10481 and HDFFV-10477 * commit '83ca39ba9a2d6852dd7754b533f7839e9d2a0107': Combined macro lines as Dana commented Platforms tested: Linux/64 (jelly) (very minor) Fixed division-by-zero issues Description: Fixed HDFFV-10481 and HDFFV-10477, division by 0. Fixed another occurrence beside what were reported. Also, changed a local variable to avoid an unnecessary cast. Platforms tested: Linux/64 (jelly) Linux/32 (jam) Darwin (osx1010test)
| * Combined macro lines as Dana commentedBinh-Minh Ribler2018-07-161-2/+1
| | | | | | | | | | Platforms tested: Linux/64 (jelly) (very minor)
| * Fixed division-by-zero issuesBinh-Minh Ribler2018-07-161-0/+3
| | | | | | | | | | | | | | | | | | | | | | Description: Fixed HDFFV-10481 and HDFFV-10477, division by 0. Fixed another occurrence beside what were reported. Also, changed a local variable to avoid an unnecessary cast. Platforms tested: Linux/64 (jelly) Linux/32 (jam) Darwin (osx1010test)
* | Merge pull request #1127 in HDFFV/hdf5 from ~JHENDERSON/hdf5:develop to developJordan Henderson2018-07-161-2/+35
|\ \ | |/ |/| | | | | | | | | | | | | | | | | | | | | | | | | | | * commit 'a8d6f100cdddbfc42d0c4abfeb4ceb8788b1b087': Add note about single chunk caching and serial library Add check for actually using the MPI file driver when caching one chunk Fix error message mentioning wrong MPI function used Fix for HDFFV-10509 Revise H5D__mpio_array_gatherv() to not allocate memory needlessly Add test to continually grow and shrink chunks Changes to test with checksum filter as well as deflate filter Eliminate warning about signed to unsigned conversion Remove unused local variable Fix bug in parallel reads of compressed data Add data verification to parallel filtered compound write tests Add seven of fourteen parallel filtered data partial read tests
| * Add note about single chunk caching and serial libraryJordan Henderson2018-07-131-1/+7
| |
| * Add check for actually using the MPI file driver when caching one chunkJordan Henderson2018-07-131-1/+15
| | | | | | Better explain rationale behind chunk caching issue fix
| * Fix for HDFFV-10509Jordan Henderson2018-07-041-2/+15
| |
* | Fixed HDFFV-10404Binh-Minh Ribler2018-07-131-4/+4
| | | | | | | | | | | | | | | | | | Description: Applied the typo fixes from user's report. The previous pull request couldn't be merged because it was too old, and it was too complicated for me to resolve conflicts. Platform tested: Linux/64 (jelly) - very minor
* | Revert "Merge pull request #1116 in HDFFV/hdf5 from ↵hdftest2018-06-241-2/+2
| | | | | | | | | | | | | | ~HDFTEST/hdf5_hft:hdf5_1_10 to develop" This reverts commit e9f476dad47ce593f13dacb77b1cc664d1f24e7b, reversing changes made to 2ff00b1b937ebe36ac6ddf590c16a4c27fc0b053.
* | Merge branch 'develop' into hdf5_1_10.syncM. Scot Breitenfeld2018-05-081-317/+194
|\ \ | |/
| * Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/hdffv/hdf5 into ↵Quincey Koziol2018-04-291-343/+547
| |\ | | | | | | | | | merge_hyperslab_updates
| | * Merge remote-tracking branch 'origin/develop' into merge_func_enter_volQuincey Koziol2018-03-211-1/+1
| | |\
| | * | Add API context interface and use it throughout the library.Quincey Koziol2018-03-151-306/+187
| | | |
| * | | Merge branch 'develop' of https://bitbucket.hdfgroup.org/scm/hdffv/hdf5 into ↵Quincey Koziol2017-04-111-5/+5
| |\ \ \ | | | | | | | | | | | | | | | merge_hyperslab_updates
generated by cgit v0.12 at 2024-07-25 08:46:19 (GMT)