summaryrefslogtreecommitdiffstats
path: root/src/H5Dchunk.c
Commit message (Collapse)AuthorAgeFilesLines
...
| | | * fix and address commentskmu2019-12-111-6/+5
| | | |
| | | * change according to previous commentskmu2019-12-101-2/+2
| | | |
| | | * Merge branch 'develop' into bugfix/intel_warningskmu2019-12-091-18/+17
| | | |\
| | | * | fix intel compile warningskmu2019-12-041-3/+2
| | | | |
| | * | | Cleanups from PR reviewsQuincey Koziol2019-12-201-1/+0
| | | | |
| | * | | Refactor all the 'H5VL_*_optional' callbacks to move the type of operation outQuincey Koziol2019-12-201-68/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | of the va_list, so it's at least possible for another connector to know what the operation is and decide whether to implement it or not. Added a new VOL sub-class called "introspect" where callbacks that report information about the connector or container can be placed. Added an 'opt_query' callback to this sub-class, for a connector to report back to the library whether a particular optional callback operation is supported. Also added a 'get_conn_cls' introspection callback, to retrieve the H5VL_class_t of a connector (either the "current" connector, H5VL_GET_CONN_LVL_CURR, or the terminal connector, H5VL_GET_CONN_LVL_TERM). Moved the "post open" operation from a file 'specific' operation to a file 'optional' operation, now that it's possible to detect (with the 'opt_query' introspection callback) whether a VOL connector implements an optional operation, without just returning an error. Added new internal VOL helper routines: H5VL_object_is_native, to determine if an object is in (or is a) native file, and H5VL_file_is_same, to determine if two objects are in (or are) the same terminal VOL connector's container. (And moved the special handling for FILE_IS_EQUAL operation out of internal VOL callback routine into H5VL_file_is_same) Made new dataset 'get' operation for H5Dvlen_get_buf_size, aligning it better with other 'get' operations in API. Fixed several issues with pass-through connectors, which are now passing the 'make check-passthrough-vol' tests again. A bunch of warning and style cleanups as well.
| | * | | Fix 2010 compile issuesAllen Byrne2019-12-111-587/+587
| | | |/ | | |/|
| * | | remove unsed var,function,macro, etckmu2019-12-061-15/+2
| |/ /
| * | Revert "Make a squash commit of 'Quiet some warnings by adjusting warnings ↵David Young2019-11-271-18/+17
| |/ | | | | | | level and fixing some code.' (commit 5c911d8baf3)"
| * Merge pull request #2045 in HDFFV/hdf5 from ~NFORTNE2/hdf5_naf:select_adjust ↵Neil Fortner2019-11-261-2/+1
| |\ | | | | | | | | | | | | | | | | | | | | | to develop * commit '4e12984b77cdd7615843d94f8de8d54db27476ac': Move checking for zero offset in selection adjust calls to the selection callbacks. This makes the procedure for checking it consistent across selection types and between _s and _u, ensures it is always is performed even when called within the H5S package, and removes the redundant check that would occur when callins H5S_select_adjust_s() from outside the H5S package. Replace H5Sselect_adjust_u() and H5Shyper_adjust_s() with H5Sselect_adjust. Implement "adjust_s" callback for all selection types. Add range checking to H5Sselect_adjust().
| | * Replace H5Sselect_adjust_u() and H5Shyper_adjust_s() withNeil Fortner2019-11-181-2/+1
| | | | | | | | | | | | | | | H5Sselect_adjust. Implement "adjust_s" callback for all selection types. Add range checking to H5Sselect_adjust().
| * | Move a statement under some declarations since some vintages of VisualDavid Young2019-11-131-1/+2
| | | | | | | | | | | | Studio don't like declarations after statements.
| * | Document H5D__chunk_mem_xfree_wrapper().David Young2019-11-131-0/+5
| | |
| * | Change a signed variable to unsigned to avoid a warning about the signDavid Young2019-11-131-1/+1
| | | | | | | | | | | | being lost by a cast.
| * | Delete variables that are unused or set and unused.David Young2019-11-131-12/+0
| | |
| * | Merge remote-tracking branch 'hdf5/develop' into add-werror-and-squash-some toDavid Young2019-11-121-1/+1
| |\ \ | | |/ | | | | | | see if new warnings-as-errors have cropped up.
| | * Implement H5VLget_file_type() to return a copy of a datatype with theNeil Fortner2019-11-081-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | location set to be in a file. Only meant to be used by VOL connectors. Implement H5VLpeek_connector_id() to support connectors querying their own IDs. Fix app_ref with connector IDs in a couple places (external VOLs registered as default through ENV should be visible to the application). Modify vlen and reference interfaces to work with arbitrary VOL connectors. Implement file "post open" specific callback, to enable connectors to update their file structs after a wrap context has been set.
| * | Change some GCC warnings to errors. Fix code to quiet some warnings.David Young2019-11-051-3/+10
| |/
| * Add new H5R API that abstracts object, region and attribute reference typesJerome Soumagne2019-10-081-10/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Also support references to external files Add new H5T_REF type and type conversion routines Support conversion from H5T_REF_OBJ/DSET_REG to H5T_REF Add H5Treclaim() API to reclaim memory of vlen/reference types Deprecate H5Dvlen_reclaim() Fix H5T_vlen_reclaim() and H5T_reclaim() to use private callback Add H5T_ref_reclaim() Move previous H5R APIs to H5Rdeprec.c Clean up H5Ocopy Separate H5O_copy_expand_ref() to H5Ocopy_ref() Add support for copying new reference types Clean up deprecated routines to go through VOL and same code path Fix return codes in existing trefer.c test Rename trefer.c to trefer_deprec.c trefer.c is for new references Add performance test for trefer Add additional obj_copy_ref test Make use of tokens and blobs to store references Skip blob encoding for object references Start adding new reference examples
* | Partial fix for HDFFV-10792Jordan Henderson2019-10-101-4/+45
|/
* (1) Address the feedback from the PR reviewVailin Choi2019-09-261-4/+3
| | | | (2) Add release notes
* Fix for HDFFV-10585 investigate hyperslab slowness:Vailin Choi2019-09-191-36/+169
| | | | | | | 1) Improve hyperslab performance when doing I/O from 1-d disjoint file dataspace to 1-d contiguous memory dataspace. 2) Move coding in H5D__chunk_io_init() that is constructing the chunk mappings to a separate routine.
* Updated based on reviews and refactored test codeBinh-Minh Ribler2019-09-011-3/+2
|
* Code cleanup and refactorBinh-Minh Ribler2019-08-291-4/+0
|
* HDFFV-10677 and HDFFV-10661Binh-Minh Ribler2019-08-281-0/+392
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Description: - Added functions to query chunk information: H5Dget_num_chunks(dset_id, fspace_id, *nchunks) Gets the number of written chunks that intersect with the given dataspace. However, in this version, the intersection is not yet completed. Thus, the number of all written chunks will be returned. H5Dget_chunk_info_by_coord(dset_id, *offset, *filter_mask, *addr, *size) Given a chunk's logical coordinates, returns the chunk's filter, address, and size. H5Dget_chunk_info(dset_id, fspace_id, index, *offset, *filter_mask, *addr, *size) Given a chunk's index, returns the chunk's logical coordinates, filter, address, and size. The chunk belongs to a set of chunks that have nonempty intersection with the specified dataspace. However, in this version, the intersection is not yet completed, and the index is of all the written chunks. These functions comply with VOL. - Fixed some oversights found in the library for the tests in chunk_info.c to work correctly. The returned value from a callback function was not checked in H5EA_iterate(), H5FA_iterate(), and H5D__none_idx_iterate(). This oversight caused a callback function to continue iterating even though it's supposed to stop. Platforms tested: Linux/64 (jelly) Linux/64 (platypus) Darwin (osx1011test)
* Progress toward moving the dataset routines to using the 'shared' file pointerQuincey Koziol2019-08-211-11/+14
| | | | instead of the 'top' file pointer.
* Add H5Sselect_shape_same and H5Sselect_intersect_block API routines, alongQuincey Koziol2019-07-301-7/+3
| | | | with tests and minor cleanups and refactorings.
* Add support for GCC9, update warnhist script, and clean up warnings.Quincey Koziol2019-07-031-3/+5
|
* Updated configure & CMake compiler flags for GCC 8.x, along with correspondingQuincey Koziol2019-06-281-2/+2
| | | | | changes to warnhist script (and some extra improvements for condensing C++ and Java warnings), and fixed a bunch of warnings.
* Merge branch 'develop' of ↵Binh-Minh Ribler2019-05-051-93/+279
|\ | | | | | | https://bitbucket.hdfgroup.org/scm/~bmribler/hdf5-bmr into develop
| * Merge remote-tracking branch 'origin/develop' into merge_hyperslab_update_01Quincey Koziol2019-03-171-25/+59
| |\
| * | Core changes to selection code from the hyperslab_updates branch.Quincey Koziol2019-03-161-93/+279
| | |
* | | Code improvementBinh-Minh Ribler2019-05-051-3/+6
| |/ |/| | | | | | | | | | | | | | | | | Description: Fixed potential division by zero occurrences and changed an assert to if statement. Platforms tested: Linux/64 (jelly) Linux/64 (platypus) Darwin (osx1011test)
* | Added an H5MM_memcpy call that checks for buffer overlap.Dana Robinson2019-03-161-21/+21
| |
* | Fix issue with direct chunk write not updating the "last chunk" indexNeil Fortner2019-03-061-3/+37
|/ | | | | cache. Fix issues involving datasets being "no allocated" when they contain cached raw data.
* Merge branch 'develop' into c_stdlib_posix_call_cleanupDana Robinson2019-02-191-70/+53
|\
| * Align develop with incoming hyperslab_updates branch changes.Quincey Koziol2019-02-131-48/+41
| |
| * Fix some collective metadata read issuesJordan Henderson2019-02-121-16/+6
| |
| * Delta reduction against hyperslab_updates branch.Quincey Koziol2019-01-111-6/+6
| |
* | C and POSIX call cleanupDana Robinson2019-01-111-7/+8
|/
* Removed a comment.Binh-Minh Ribler2019-01-031-1/+0
|
* Fixed CVE division-by-zero issuesBinh-Minh Ribler2018-12-231-3/+26
| | | | | | | | | | | | | | | | | Description: Fixed HDFFV-10577 and similar issues found in H5Dchunk.c. All the occurrences are in: H5D__create_chunk_map_single H5D__create_chunk_file_map_hyper H5D__chunk_allocate H5D__chunk_update_old_edge_chunks H5D__chunk_prune_by_extent H5D__chunk_copy_cb H5D__chunk_collective_fill Platforms tested: Linux/64 (jelly) Linux/64 (platypus) Darwin (osx1010test)
* Merge pull request #1335 in HDFFV/hdf5 from ↵Ray Lu2018-11-191-0/+3
|\ | | | | | | | | | | | | | | | | | | | | ~SONGYULU/hdf5_ray:bugfix/HDFFV-10571-cve-2018-17237-divided-by-zero to develop * commit 'c923cdad6e515c842f3795a5b6d754ad94021e09': HDFFV-10571: Minor format changes. HDFFV-10571: Minor change - reformatting the error check. HDFFV-10571: Minor change - adding the error check right after decoding of chunk dimension for safeguard. HDFFV-10571: Minor change - revised the comment to be clearer. HDFFV-10571 Divided by Zero vulnerability. Minor fix: I added an error check to make sure the chunk size is not zero.
| * HDFFV-10571: Minor format changes.Songyu Lu2018-11-151-1/+0
| |
| * HDFFV-10571: Minor change - adding the error check right after decoding of ↵Songyu Lu2018-11-151-4/+4
| | | | | | | | chunk dimension for safeguard.
| * HDFFV-10571: Minor change - revised the comment to be clearer.Songyu Lu2018-11-141-1/+1
| |
| * HDFFV-10571 Divided by Zero vulnerability. Minor fix: I added an error ↵Songyu Lu2018-11-141-0/+4
| | | | | | | | check to make sure the chunk size is not zero.
* | HDFFV-10601: I changed to a better way to calculate the number of chunks in ↵Songyu Lu2018-11-081-5/+3
| | | | | | | | a dataset.
* | HDFFV-10601 Issues with chunk cache hash value calcuation:Songyu Lu2018-11-021-12/+12
|/ | | | | | | | 1. H5D__chunk_hash_val: When the number of chunks in the fastest changing dimension is larger than the number of slots in the hash table, H5D__chunk_hash_val abandons the normal hash value calculation algorithm and simply uses the scaled dimension. This will cause chunks a selection that cuts across chunks in dimensions other than the fastest changing to all have the same hash value, and they will therefore always evict each other from the cache, having an obvious major performance impact. Eliminated the check for the number of slots in this function and always use the full algorithm. 2. H5D__chunk_init: When the scaled dimensions (number of chunks in each dimension) are calculated in H5D__chunk_init, a simple divide ("/") operator is used with the dataset size in elements and the chunk size in elements. While this is fine when the dataset size is an exact multiple of the chunk size, in other cases, since "/" rounds down, it results in a scaled dimension one less than it should (it ignores the partial edge chunk). This has trickle down effects on hash value calculation that can cause excess hash value collisions and therefore performance issues. Changed the calculation to (((dataset_size - 1) / chunk_size) + 1). Tested the build with Autotool and CMake.
* A trivial change to reformat a comment. The purpose is simply to try out Git.Songyu Lu2018-10-311-2/+2
|
generated by cgit v0.12 at 2024-07-21 14:29:45 (GMT)