summaryrefslogtreecommitdiffstats
diff options
context:
space:
mode:
authorLarry Knox <lrknox@hdfgroup.org>2021-04-01 22:07:53 (GMT)
committerGitHub <noreply@github.com>2021-04-01 22:07:53 (GMT)
commit5cba1c84e2eacb9c77b698e2b6c316b083663e8c (patch)
tree197b2a54f5b5c82355d5eb10123742f4863aa9c4
parentc38dbaf3e7f0975fd5a7d195d82844da911f9f9a (diff)
downloadhdf5-5cba1c84e2eacb9c77b698e2b6c316b083663e8c.zip
hdf5-5cba1c84e2eacb9c77b698e2b6c316b083663e8c.tar.gz
hdf5-5cba1c84e2eacb9c77b698e2b6c316b083663e8c.tar.bz2
Hdf5 merge issue 488 v112 (#534)
* Snapshot version 1.12 release 1-3. Update version to 1.12.1-4. * First cut of the H5 public API documentation. (#80) * First cut of the H5 public API documentation. * Added H5Z "bonus track." * Applied Quincey's patch. * Added the missing patches from Quincey's original patch. * H5PL (complete) and basic H5VL API documentation. * Added H5I API docs. * Added H5L API docs. * First installment from Elena's H5T batch. * Second installment of Elena's H5T batch. * Final installment of Elena's H5T batch. * Full set of current H5F documentation. (#105) * First cut of the H5 public API documentation. * Added H5Z "bonus track." * Applied Quincey's patch. * Added the missing patches from Quincey's original patch. * H5PL (complete) and basic H5VL API documentation. * Added H5I API docs. * Added H5L API docs. * First installment from Elena's H5T batch. * Second installment of Elena's H5T batch. * Final installment of Elena's H5T batch. * Migrated documentation for SWMR functions. * Catching up on MDC functions. * Integrated the H5F MDC function documentation. * Added MDC and parallel H5F functions. * Slightly updated main page. * Added doxygen/dox/H5AC_cache_config_t.dox to MANIFEST. * Doxygen - added (mostly) beginner functions (#112) * Doxygen - added (mostly) beginner functions * Removed duplicate H5Pset_szip function * Add src/H5module.h to MANIFEST. * close #195. (#196) * Update HDF5PluginMacros.cmake * Update HDF5PluginMacros.cmake * Avoid aligned access for references by decoding into temporary buffer and then copying the result into the actual buffer. Update test to be more thorough with using compound datatype fields everywhere. (#206) * Modify temporary rpath for testing in java example scripts. (#230) * Fix undefined left shifting of negative numbers (#338) Undefined Bahavior Sanitizer errored here about left shifting negative numbers. * Fixes various warnings noticed on Windows (#425) * Fixes various warnings noticed on Windows - Adds a prototype for our implementation of vasprintf - Return type of H5_get_utf16_str() is now non-const - Fixes possible uninitialized return type in Wremove_utf8 - Better isolation of fork() code in accum.c:test_swmr_write_big() - Better isolation of non-zlib code in dsets.c:test_filter_delete() - Removed unused variable in trefer.c:test_reference_cmpnd_obj() * Fixes clang-format issues * Applied clang-tidy readability-non-const-parameter warning fixes auto… (#429) * Automatically applied clang-tidy readability-avoid-const-params-in-decls fixes Removes useless const declarations. * Fixed most readability-non-const-parameter warnings These changes were made automatically by clang-tidy, but I manually reverted the changes related to the H5Z_func_t signature. * Reformat source with clang v10.0.1. Co-authored-by: Larry Knox <lrknox@hdfgroup.org> * Added C++11 override keyword where appropriate (#433) Added H5_OVERRIDE macro for compatibility with both C++11 and older. * Various clang tidy warning fixes (#448) * Fixed clang-tidy bugprone-reserved-identifier warnings * Fixed clang-tidy bugprone-assert-side-effect warnings * Fixed clang-tidy bugprone-copy-constructor-init warning * Fixed clang-tidy readability-redundant-preprocessor warning For error_test.c the removed code was already dead, because it was in the else of an `#if H5_USE_16_API` block. Based on H5Location.h, I think p_get_ref_obj_type was meant to be in `#ifndef DOXYGEN_SHOULD_SKIP_THIS` and an `#endif` was missing. Similarly, in the header, getObjTypeByIdx is only in H5_NO_DEPRECATED_SYMBOLS, not DOXYGEN_SHOULD_SKIP_THIS. * Fixed clang-tidy readability-redundant-string-init warnings * Fixed some clang-tidy performance-type-promotion-in-math-fn warnings * Fixed clang-tidy performance-unnecessary-value-param warnings * Reformat source with clang v10.0.1. Co-authored-by: Larry Knox <lrknox@hdfgroup.org> * Removed checks/workarounds for pre-C++89 compatibility (#449) After 30+ years, just assume that the following exist: - extension-less includes - namespaces - std:: - static_cast - bool * Fixed all clang-tidy bugprone-suspicious-string-compare warnings (#451) * Fixed all clang-tidy bugprone-suspicious-string-compare warnings This change was generated entirely by clang-tidy itself. * Reformat code with clang v10.0.1. Co-authored-by: Larry Knox <lrknox@hdfgroup.org> * Remove 2 functions incorrectly merged from develop in a cherry-pick merge of PR #451. * Purge the buffer used in type conversion. (#263) Some of the uniniitialized bits in the buffer may get carried through all the way to disk, creating a risk for leaks. We observed an msan error during the floating point output conversion. Due to the encoding certain bits could remain untouched during the conversion. In this draft we zero initialize the dbuf used by every convertor. * Fixed HDFFV-10480 (CVE-2018-11206) and HDFFV-11159 (CVE-2018-14033) (#405) * Fixed HDFFV-10480 (CVE-2018-11206) and HDFFV-11159 (CVE-2018-14033) Description Checked against buffer size to prevent segfault, in case of data corruption. + HDFFV-11159 CVE-2018-14033 Buffer over-read in H5O_layout_decode + HDFFV-10480 CVE-2018-11206 Buffer over-read in H5O_fill_new[/old]_decode Platforms tested: Linux/64 (jelly) * Accidentally left in another occurrence of the previous patch from user after a more correct fix was applied, that is the check now accounted for the previous advance of the buffer pointer. Removed it. * Typo * Fixed format issues. * Added test. * Changed arguments to ADD_H5_TEST * Fixing arguments to ADD_H5_TEST again. * Fixing arguments again. * Took out the CMake changes until Allen can help. * Added files: tCVE_2018_11206_fill_old.h5 tCVE_2018_11206_fill_new.h5 * Revert "Took out the CMake changes until Allen can help." This reverts commit c21324d6e0044994c5cd24b0671e7d1dd41096cc. * Revert "Fixing arguments again." This reverts commit 5832a70674339e4b524749adde5a181f8c3a446a. * Revert "Fixing arguments to ADD_H5_TEST again." This reverts commit b45de823c22ce83a388d46466ef7c04b66ff05ed. * Revert "Changed arguments to ADD_H5_TEST" This reverts commit 16719824f57e52158451ddd261788c0dcaa3ec55. * Added first argument to ADD_H5_TEST for HDFFV-10480 fix. * Changed argument 0 to 1 * Revert "Changed argument 0 to 1" This reverts commit b343d6613ba681b43248dd5820e96389984ebcf7. * Revert "Added first argument to ADD_H5_TEST for HDFFV-10480 fix." This reverts commit b8a0f9a9e8ec8e6c6ff38d33195d63edff76a563. * Added first argument and corrected the second. * Updated fixes for HDFFV-10480 and HDFFV-11159/HDFFV-11049 * Improved error messages. * Added description of the current HDF5 branches; added a draft of contribution policy. (#445) * Added description of the current HDF5 branches. * Removed capitalization in from Develop, Release, Feature to reflect the real naming schema * Added a draft of contributions guidance document. * Fixed typos. * Fixed section title. * Fixed typo. * Fixed typos and formatting. * Fixed many typos and simplified the text (e.g., removed testing instructions and left pointers to where to find them, etc.) * Fixed a typo. Please enter the commit message for your changes. Lines starting * Added contributing.md file and rearranged doc entries in alphabetical order. * ddressed Gerd's review comments; found and fixed more typos. * Addressed comments from Larry and Scot. * Fix CMake error message location. (#478) Print error message if Perl is not found. * Committing clang-format changes Co-authored-by: Gerd Heber <gheber@hdfgroup.org> Co-authored-by: bljhdf <58825073+bljhdf@users.noreply.github.com> Co-authored-by: H. Joe Lee <hyoklee@hdfgroup.org> Co-authored-by: Quincey Koziol <quincey@koziol.cc> Co-authored-by: Sean McBride <sean@rogue-research.com> Co-authored-by: Dana Robinson <43805+derobins@users.noreply.github.com> Co-authored-by: Yu Feng <rainwoodman@gmail.com> Co-authored-by: bmribler <39579120+bmribler@users.noreply.github.com> Co-authored-by: epourmal <epourmal@hdfgroup.org> Co-authored-by: github-actions <41898282+github-actions[bot]@users.noreply.github.com>
-rw-r--r--MANIFEST4
-rw-r--r--doc/branches-explained.md41
-rw-r--r--doc/contributing.md87
-rw-r--r--release_docs/RELEASE.txt20
-rw-r--r--src/H5Ofill.c27
-rw-r--r--src/H5Olayout.c29
-rw-r--r--src/H5Tconv.c20
-rw-r--r--test/CMakeTests.cmake4
-rw-r--r--tools/test/h5dump/CMakeTests.cmake6
-rw-r--r--tools/test/h5dump/testh5dump.sh.in35
-rw-r--r--tools/testfiles/tCVE_2018_11206_fill_new.h5bin0 -> 1752 bytes
-rw-r--r--tools/testfiles/tCVE_2018_11206_fill_old.h5bin0 -> 2560 bytes
12 files changed, 241 insertions, 32 deletions
diff --git a/MANIFEST b/MANIFEST
index 0f8a6a0..2bba97f 100644
--- a/MANIFEST
+++ b/MANIFEST
@@ -203,7 +203,9 @@
./config/site-specific/BlankForm
+./doc/branches-explained.md
./doc/code-conventions.md
+./doc/contributing.md
./doxygen/aliases
./doxygen/Doxyfile.in
@@ -2120,6 +2122,8 @@
./tools/testfiles/twithddl.exp
./tools/testfiles/twithddlfile.ddl
./tools/testfiles/twithddlfile.exp
+./tools/testfiles/tCVE_2018_11206_fill_old.h5
+./tools/testfiles/tCVE_2018_11206_fill_new.h5
# h5dump test error files
./tools/test/h5dump/errfiles/filter_fail.err
diff --git a/doc/branches-explained.md b/doc/branches-explained.md
new file mode 100644
index 0000000..22b9c8f
--- /dev/null
+++ b/doc/branches-explained.md
@@ -0,0 +1,41 @@
+# HDF5 Git Branching Model Explained
+
+This document describes current HDF5 branches.
+
+Branches are tested nightly and testing results are available at https://cdash-internal.hdfgroup.org/ and https://cdash.hdfgroup.org/.
+Commits that break daily testing should be fixed by 3:00 pm Central time or reverted.
+We encourage code contributors to check the status of their commits. If you have any questions, please contact help@hdfgroup.org.
+
+## `develop`
+Develop is the main branch whose source code always reflects a state with the latest delivered development changes for the next major release of HDF5.
+This is also considered the integration branch, as **all** new features are integrated into this branch from respective feature branches.
+
+## `Maintenance branches`
+
+Each currently supported release-line of HDF5 (e.g. 1.8.x, 1.10.x, 1.12.x) has a support branch with the name 1_8, 1_10, 1_12.
+Maintenance branches are similar to the develop branch, except the source code in a maintenance branch always reflects a state
+with the latest delivered development changes for the next **maintenance** release of that particular supported release-line of HDF5.
+**Some** new features will be integrated into a release maintenance branch, depending on whether or not those features can be
+introduced in minor releases. Maintenance branches are removed when a release-line is retired from support.
+
+## `feature/*`
+Feature branches are temporary branches used to develop new features in HDF5.
+Feature branches branch off of develop and exist as long as the feature is under development.
+When the feature is complete, the branch is merged back into develop, as well as into any support branches in which the change will be included, and then the feature branch is removed.
+
+## `release/*`
+Release branches are used to prepare a new production release. They are primarily used to allow for last minute dotting of i's and crossing of t's
+(things like setting the release version, finalizing release notes, et cetera) and do not include new development.
+They are created from the maintenance branch at the time of the maintenance release and have
+names 1_8_N, 1_10_N, 1_12_N, where N is the minor release number. Once the release is done it is tagged.
+Patches can be applied to the release branch for patch releases that are treated as "scaled down" maintenance releases as defined by Release coordinator.
+
+## `1.X/master/*` where X is 8, 10 or 12
+These branches are used to tag 1.X.* maintenance releases.
+
+## `inactive/<name>/*`
+These branches are for experimental features that were developed in the past and have not been merged to develop, and are not under active development. The features
+can be out of sync with the develop branch.
+
+This document was last updated on March 16, 2021
+
diff --git a/doc/contributing.md b/doc/contributing.md
new file mode 100644
index 0000000..3f738c8
--- /dev/null
+++ b/doc/contributing.md
@@ -0,0 +1,87 @@
+# How to contribute to HDF5 (Draft)
+
+The HDF Group encourages community members to contribute to the HDF5 project. We accept and are very grateful for any type of contributions
+from small typos and bug fixes to new features. The HDF Group is committed to work with the code contributors and make contribution process simple and enjoyable.
+
+This document describes guiding principles for the HDF5 code contributors and does not pretend to address any possible
+contribution. If in doubt, please do not hesitate to ask us for guidance.
+***Note that no contribution may be accepted unless the donor agrees with the HDF Group software license terms
+found in the COPYING file in the top source directory of every branch.***
+
+
+> We will assume that you are familiar with `git` and `GitHub`. If not, you may go through the GitHub tutorial found at [https://guides.github.com/activities/hello-world/](https://guides.github.com/activities/hello-world/). This tutorial should only take around 10 minutes.
+
+## Table of Contents
+
+* [Workflow](#workflow)
+* [Acceptance criteria for pull request](#criteria)
+* [Check List](#checklist)
+
+# Workflow <A NAME="workflow"></A>
+
+The process for contributing code to HDF5 is as follows:
+
+* Open an issue on [HDF5 GitHub](https://github.com/HDFGroup/hdf5/issues).
+
+> This step is ***required*** unless the change is minor (e.g., typo fix).
+
+* Fork the [HDF5](https://github.com/HDFGroup/hdf5) repository.
+* Make the desired changes to the HDF5 software.
+ * New features should always go to develop branch first and later should be merged to the appropriate maintenance branches.
+ * Bug fixes should go to all appropriate branches (develop and maintenance).
+* Build and test your changes. Detailed instructions on how to build and test HDF5 can be found in the `INSTALL*` files in the `release_docs` directory.
+* Push your changes to GitHub.
+* Issue a pull request and address any code formatting and testing issues reported.
+
+Once a pull request is correctly formatted and passes **ALL** CI tests, it will be reviewed and evaluated by The HDF Group developers and HDF5 community members who can approve pull requests..
+The HDF Group developers will work with you to assure that the pull request satisfies acceptance criteria described in the next section.
+
+# Acceptance criteria for pull request <A NAME="criteria"></A>
+
+We appreciate every contribution we receive, but we may not accept them all. Those that we *do* accept satisfy the following criteria:
+
+* **The pull request has a clear purpose** - What does the pull request address? How does it benefit the HDF5 community?
+If the pull request does not have a clear purpose and benefits it will not be accepted.
+
+* **The pull request is documented** - The HDF5 developers must understand not only *what* a change is doing, but *how* it is doing it.
+ Documenting the code makes it easier for us to understand your patch and will help to maintaine the code in the future.
+
+* **The pull request passes HDF5 regression testing** - Any issue fixed or functionality added should be accompanied by the corresponding
+tests and pass HDF5 regression testing run by The HDF Group. We do not expect you to perform comprehensive testing across multiple platforms
+before we accept the pull request. If the pull request does not pass regression testing after the merge, The HDF Group developers will i
+work with you on the fixes.
+
+* **The pull request does not compromise the principles behind HDF5** - HDF5 has a 100% commitment to backward compatibility.
+ * Any file ever created with HDF5 must be readable by any future version of HDF5.
+ If the purpose of your patch is to modify HDF5 data model or file format,
+ **please** discuss this with us first. File format changes and features required by those changes can be introduced only in a new major release.
+ * HDF5 has a commitment to remaining *machine-independent*; data created on one platform/environment/architecture **must** remain readable by HDF5 on any other.
+ * For binary compatibility no changes are allowed to public APIs and data structures in the maintenance releases; new APIs can be added.
+
+* **New features are documented** - Any new features should have proper documentation; talk to us if you have any questions.
+
+
+# Checklist <A NAME="checklist"></A>
+
+Please make sure that you check the items applicable to your pull request:
+
+* Code
+ * [ ] Does the pull request have a corresponding GitHub issue and clear purpose?
+ * [ ] Does the pull request follow HDF5 best practices (naming conventions, code portability, code structure, etc.)? <<TODO: link to the document>>
+ * [ ] If changes were done to autotools build were they added to CMake and vice versa?
+ * [ ] Is the pull request applicable to any other branches? If yes, which ones? Please document it in the GitHub issue.
+ * [ ] Is the new code sufficiently documented for future maintenance?
+ * [ ] Does the new feature require a change to an existing API? See "API Compatibility Macros" document (https://portal.hdfgroup.org/display/HDF5/API+Compatibility+Macros)
+* Documentation
+ * [ ] Was the change described in the release_docs/RELEASE.txt file?
+ * [ ] Was MANIFEST updated if new files had been added to the source?
+ * [ ] Was the new function documented in the corresponding public header file using Doxygen? <<TODO: link to Doxygen instructions>>
+ * [ ] Was new functionality documented for the HDF5 community (the level of documentation depends on the feature; ask us what would be appropriate)
+* Testing
+ * [ ] Does the pull request have tests?
+ * [ ] Does the pull request affect HDF5 library perfromance?
+
+We want as many contributions as we can get, and we are here to help. Feel free to reach out to us if you have any questions
+
+Thank you for your contribution!
+
diff --git a/release_docs/RELEASE.txt b/release_docs/RELEASE.txt
index 5e8be89..bb40d68 100644
--- a/release_docs/RELEASE.txt
+++ b/release_docs/RELEASE.txt
@@ -337,6 +337,26 @@ Bug Fixes since HDF5-1.12.0 release
===================================
Library
-------
+ - Fixed CVE-2018-17435
+
+ The tool h5dump produced a segfault when the size of a fill value
+ message was corrupted and caused a buffer overflow.
+
+ The problem was fixed by verifying the fill value's size
+ against the buffer size before attempting to access the buffer.
+
+ (BMR - 2021/03/15, HDFFV-10480)
+
+ - Fixed CVE-2018-14033 (same issue as CVE-2020-10811)
+
+ The tool h5dump produced a segfault when the storage size message
+ was corrupted and caused a buffer overflow.
+
+ The problem was fixed by verifying the storage size against the
+ buffer size before attempting to access the buffer.
+
+ (BMR - 2021/03/15, HDFFV-11159/HDFFV-11049)
+
- Remove underscores on header file guards
Header file guards used a variety of underscores at the beginning of the define.
diff --git a/src/H5Ofill.c b/src/H5Ofill.c
index 2882ff4..2314a84 100644
--- a/src/H5Ofill.c
+++ b/src/H5Ofill.c
@@ -195,8 +195,9 @@ H5O__fill_new_decode(H5F_t H5_ATTR_UNUSED *f, H5O_t H5_ATTR_UNUSED *open_oh,
unsigned H5_ATTR_UNUSED mesg_flags, unsigned H5_ATTR_UNUSED *ioflags, size_t p_size,
const uint8_t *p)
{
- H5O_fill_t *fill = NULL;
- void * ret_value = NULL; /* Return value */
+ H5O_fill_t * fill = NULL;
+ const uint8_t *p_end = p + p_size - 1; /* End of the p buffer */
+ void * ret_value = NULL; /* Return value */
FUNC_ENTER_STATIC
@@ -227,8 +228,11 @@ H5O__fill_new_decode(H5F_t H5_ATTR_UNUSED *f, H5O_t H5_ATTR_UNUSED *open_oh,
INT32DECODE(p, fill->size);
if (fill->size > 0) {
H5_CHECK_OVERFLOW(fill->size, ssize_t, size_t);
- if ((size_t)fill->size > p_size)
- HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "destination buffer too small")
+
+ /* Ensure that fill size doesn't exceed buffer size, due to possible data corruption */
+ if (p + fill->size - 1 > p_end)
+ HGOTO_ERROR(H5E_OHDR, H5E_OVERFLOW, NULL, "fill size exceeds buffer size")
+
if (NULL == (fill->buf = H5MM_malloc((size_t)fill->size)))
HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "memory allocation failed for fill value")
H5MM_memcpy(fill->buf, p, (size_t)fill->size);
@@ -310,10 +314,11 @@ static void *
H5O__fill_old_decode(H5F_t *f, H5O_t *open_oh, unsigned H5_ATTR_UNUSED mesg_flags,
unsigned H5_ATTR_UNUSED *ioflags, size_t p_size, const uint8_t *p)
{
- H5O_fill_t *fill = NULL; /* Decoded fill value message */
- htri_t exists = FALSE;
- H5T_t * dt = NULL;
- void * ret_value = NULL; /* Return value */
+ H5O_fill_t * fill = NULL; /* Decoded fill value message */
+ htri_t exists = FALSE;
+ H5T_t * dt = NULL;
+ const uint8_t *p_end = p + p_size - 1; /* End of the p buffer */
+ void * ret_value = NULL; /* Return value */
FUNC_ENTER_STATIC
@@ -334,8 +339,10 @@ H5O__fill_old_decode(H5F_t *f, H5O_t *open_oh, unsigned H5_ATTR_UNUSED mesg_flag
/* Only decode the fill value itself if there is one */
if (fill->size > 0) {
H5_CHECK_OVERFLOW(fill->size, ssize_t, size_t);
- if ((size_t)fill->size > p_size)
- HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL, "destination buffer too small")
+
+ /* Ensure that fill size doesn't exceed buffer size, due to possible data corruption */
+ if (p + fill->size - 1 > p_end)
+ HGOTO_ERROR(H5E_OHDR, H5E_OVERFLOW, NULL, "fill size exceeds buffer size")
/* Get the datatype message */
if ((exists = H5O_msg_exists_oh(open_oh, H5O_DTYPE_ID)) < 0)
diff --git a/src/H5Olayout.c b/src/H5Olayout.c
index 4020b23..651e317 100644
--- a/src/H5Olayout.c
+++ b/src/H5Olayout.c
@@ -17,7 +17,7 @@
* Purpose: Messages related to data layout.
*/
-#define H5D_FRIEND /*suppress error about including H5Dpkg */
+#define H5D_FRIEND /*suppress error about including H5Dpkg */
#include "H5Omodule.h" /* This source code file is part of the H5O module */
#include "H5private.h" /* Generic Functions */
@@ -90,12 +90,13 @@ H5FL_DEFINE(H5O_layout_t);
*/
static void *
H5O__layout_decode(H5F_t *f, H5O_t H5_ATTR_UNUSED *open_oh, unsigned H5_ATTR_UNUSED mesg_flags,
- unsigned H5_ATTR_UNUSED *ioflags, size_t H5_ATTR_UNUSED p_size, const uint8_t *p)
+ unsigned H5_ATTR_UNUSED *ioflags, size_t p_size, const uint8_t *p)
{
- H5O_layout_t *mesg = NULL;
- uint8_t * heap_block = NULL;
- unsigned u;
- void * ret_value = NULL; /* Return value */
+ H5O_layout_t * mesg = NULL;
+ uint8_t * heap_block = NULL;
+ unsigned u;
+ const uint8_t *p_end = p + p_size - 1; /* End of the p buffer */
+ void * ret_value = NULL; /* Return value */
FUNC_ENTER_STATIC
@@ -179,6 +180,10 @@ H5O__layout_decode(H5F_t *f, H5O_t H5_ATTR_UNUSED *open_oh, unsigned H5_ATTR_UNU
if (mesg->type == H5D_COMPACT) {
UINT32DECODE(p, mesg->storage.u.compact.size);
if (mesg->storage.u.compact.size > 0) {
+ /* Ensure that size doesn't exceed buffer size, due to possible data corruption */
+ if (p + mesg->storage.u.compact.size - 1 > p_end)
+ HGOTO_ERROR(H5E_OHDR, H5E_OVERFLOW, NULL, "storage size exceeds buffer size")
+
if (NULL == (mesg->storage.u.compact.buf = H5MM_malloc(mesg->storage.u.compact.size)))
HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, NULL,
"memory allocation failed for compact data buffer")
@@ -198,6 +203,10 @@ H5O__layout_decode(H5F_t *f, H5O_t H5_ATTR_UNUSED *open_oh, unsigned H5_ATTR_UNU
UINT16DECODE(p, mesg->storage.u.compact.size);
if (mesg->storage.u.compact.size > 0) {
+ /* Ensure that size doesn't exceed buffer size, due to possible data corruption */
+ if (p + mesg->storage.u.compact.size - 1 > p_end)
+ HGOTO_ERROR(H5E_OHDR, H5E_OVERFLOW, NULL, "storage size exceeds buffer size")
+
/* Allocate space for compact data */
if (NULL == (mesg->storage.u.compact.buf = H5MM_malloc(mesg->storage.u.compact.size)))
HGOTO_ERROR(H5E_OHDR, H5E_CANTALLOC, NULL,
@@ -887,13 +896,13 @@ done:
} /* end H5O__layout_reset() */
/*-------------------------------------------------------------------------
- * Function: H5O__layout_free
+ * Function: H5O__layout_free
*
- * Purpose: Free's the message
+ * Purpose: Free's the message
*
- * Return: Non-negative on success/Negative on failure
+ * Return: Non-negative on success/Negative on failure
*
- * Programmer: Quincey Koziol
+ * Programmer: Quincey Koziol
* Saturday, March 11, 2000
*
*-------------------------------------------------------------------------
diff --git a/src/H5Tconv.c b/src/H5Tconv.c
index 7b5fce1..0c98633 100644
--- a/src/H5Tconv.c
+++ b/src/H5Tconv.c
@@ -1668,7 +1668,7 @@ H5T__conv_b_b(hid_t src_id, hid_t dst_id, H5T_cdata_t *cdata, size_t nelmts, siz
size_t olap; /*num overlapping elements */
size_t half_size; /*1/2 of total size for swapping*/
uint8_t * s, *sp, *d, *dp; /*source and dest traversal ptrs*/
- uint8_t dbuf[256]; /*temp destination buffer */
+ uint8_t dbuf[256] = {0}; /*temp destination buffer */
size_t msb_pad_offset; /*offset for dest MSB padding */
size_t i;
uint8_t * src_rev = NULL; /*order-reversed source buffer */
@@ -3843,8 +3843,8 @@ H5T__conv_i_i(hid_t src_id, hid_t dst_id, H5T_cdata_t *cdata, size_t nelmts, siz
size_t half_size; /*half the type size */
size_t olap; /*num overlapping elements */
uint8_t * s, *sp, *d, *dp; /*source and dest traversal ptrs*/
- uint8_t * src_rev = NULL; /*order-reversed source buffer */
- uint8_t dbuf[64]; /*temp destination buffer */
+ uint8_t * src_rev = NULL; /*order-reversed source buffer */
+ uint8_t dbuf[64] = {0}; /*temp destination buffer */
size_t first;
ssize_t sfirst; /*a signed version of `first' */
size_t i; /*Local index variables */
@@ -4286,8 +4286,8 @@ H5T__conv_f_f(hid_t src_id, hid_t dst_id, H5T_cdata_t *cdata, size_t nelmts, siz
size_t olap; /*num overlapping elements */
ssize_t bitno = 0; /*bit number */
uint8_t * s, *sp, *d, *dp; /*source and dest traversal ptrs*/
- uint8_t * src_rev = NULL; /*order-reversed source buffer */
- uint8_t dbuf[64]; /*temp destination buffer */
+ uint8_t * src_rev = NULL; /*order-reversed source buffer */
+ uint8_t dbuf[64] = {0}; /*temp destination buffer */
uint8_t tmp1, tmp2; /*temp variables for swapping bytes*/
/* Conversion-related variables */
@@ -4947,7 +4947,7 @@ H5T__conv_s_s(hid_t src_id, hid_t dst_id, H5T_cdata_t *cdata, size_t nelmts, siz
dst_delta = (ssize_t)direction * (ssize_t)(buf_stride ? buf_stride : dst->shared->size);
/* Allocate the overlap buffer */
- if (NULL == (dbuf = (uint8_t *)H5MM_malloc(dst->shared->size)))
+ if (NULL == (dbuf = (uint8_t *)H5MM_calloc(dst->shared->size)))
HGOTO_ERROR(H5E_RESOURCE, H5E_NOSPACE, FAIL, "memory allocation failed for string conversion")
/* The conversion loop. */
@@ -8401,8 +8401,8 @@ H5T__conv_f_i(hid_t src_id, hid_t dst_id, H5T_cdata_t *cdata, size_t nelmts, siz
size_t tsize; /*type size for swapping bytes */
size_t olap; /*num overlapping elements */
uint8_t * s, *sp, *d, *dp; /*source and dest traversal ptrs*/
- uint8_t * src_rev = NULL; /*order-reversed source buffer */
- uint8_t dbuf[64]; /*temp destination buffer */
+ uint8_t * src_rev = NULL; /*order-reversed source buffer */
+ uint8_t dbuf[64] = {0}; /*temp destination buffer */
uint8_t tmp1, tmp2; /*temp variables for swapping bytes*/
/* Conversion-related variables */
@@ -9027,8 +9027,8 @@ H5T__conv_i_f(hid_t src_id, hid_t dst_id, H5T_cdata_t *cdata, size_t nelmts, siz
size_t tsize; /*type size for swapping bytes */
size_t olap; /*num overlapping elements */
uint8_t * s, *sp, *d, *dp; /*source and dest traversal ptrs*/
- uint8_t * src_rev = NULL; /*order-reversed source buffer */
- uint8_t dbuf[64]; /*temp destination buffer */
+ uint8_t * src_rev = NULL; /*order-reversed source buffer */
+ uint8_t dbuf[64] = {0}; /*temp destination buffer */
uint8_t tmp1, tmp2; /*temp variables for swapping bytes*/
/* Conversion-related variables */
diff --git a/test/CMakeTests.cmake b/test/CMakeTests.cmake
index 7de24f0..dfab713 100644
--- a/test/CMakeTests.cmake
+++ b/test/CMakeTests.cmake
@@ -915,9 +915,9 @@ if (ENABLE_EXTENDED_TESTS)
ENVIRONMENT "srcdir=${HDF5_TEST_BINARY_DIR}/H5TEST/flushrefresh_test"
WORKING_DIRECTORY ${HDF5_TEST_BINARY_DIR}/H5TEST/flushrefresh_test
)
+ else ()
+ message (STATUS "Cannot execute TEST flushrefresh - perl not found")
endif ()
-else ()
- message (STATUS "Cannot execute TEST flushrefresh - perl not found")
endif ()
##############################################################################
diff --git a/tools/test/h5dump/CMakeTests.cmake b/tools/test/h5dump/CMakeTests.cmake
index 4b61569..411e9ef 100644
--- a/tools/test/h5dump/CMakeTests.cmake
+++ b/tools/test/h5dump/CMakeTests.cmake
@@ -333,6 +333,8 @@
${HDF5_TOOLS_DIR}/testfiles/tvlstr.h5
${HDF5_TOOLS_DIR}/testfiles/tvms.h5
${HDF5_TOOLS_DIR}/testfiles/t128bit_float.h5
+ ${HDF5_TOOLS_DIR}/testfiles/tCVE_2018_11206_fill_old.h5
+ ${HDF5_TOOLS_DIR}/testfiles/tCVE_2018_11206_fill_new.h5
${HDF5_TOOLS_DIR}/testfiles/zerodim.h5
#STD_REF_OBJ files
${HDF5_TOOLS_DIR}/testfiles/trefer_attr.h5
@@ -1179,6 +1181,10 @@
# test to verify HDFFV-9407: long double full precision
# ADD_H5_GREP_TEST (t128bit_float 1 "1.123456789012345" -m %.35Lg t128bit_float.h5)
+ # test to verify HDFFV-10480: out of bounds read in H5O_fill_new[old]_decode
+ ADD_H5_TEST (tCVE_2018_11206_fill_old 1 tCVE_2018_11206_fill_old.h5)
+ ADD_H5_TEST (tCVE_2018_11206_fill_new 1 tCVE_2018_11206_fill_new.h5)
+
##############################################################################
### P L U G I N T E S T S
##############################################################################
diff --git a/tools/test/h5dump/testh5dump.sh.in b/tools/test/h5dump/testh5dump.sh.in
index f985da5..c4e2fd4 100644
--- a/tools/test/h5dump/testh5dump.sh.in
+++ b/tools/test/h5dump/testh5dump.sh.in
@@ -181,6 +181,8 @@ $SRC_H5DUMP_TESTFILES/tvlenstr_array.h5
$SRC_H5DUMP_TESTFILES/tvlstr.h5
$SRC_H5DUMP_TESTFILES/tvms.h5
$SRC_H5DUMP_TESTFILES/err_attr_dspace.h5
+$SRC_H5DUMP_TESTFILES/tCVE_2018_11206_fill_old.h5
+$SRC_H5DUMP_TESTFILES/tCVE_2018_11206_fill_new.h5
"
LIST_OTHER_TEST_FILES="
@@ -870,6 +872,35 @@ TOOLTEST5() {
fi
}
+# same as TOOLTEST1 but expects h5dump to fail
+#
+TOOLTEST_FAIL() {
+
+ infile=$1
+ expect="$TESTDIR/`basename $1 exp`.ddl"
+ actual="$TESTDIR/`basename $1 .exp`.out"
+
+ # Run test.
+ TESTING $DUMPER $@
+ (
+ cd $TESTDIR
+ $RUNSERIAL $DUMPER_BIN "$@" $infile
+ ) >&$actual
+ RET=$?
+ # Segfault occurred
+ if [ $RET == 139 ] ; then
+ nerrors="`expr $nerrors + 1`"
+ echo "*FAILED - test on $infile failed with segmentation fault"
+ # Should fail but didn't
+ elif [ $RET == 0 ] ; then
+ nerrors="`expr $nerrors + 1`"
+ echo "*FAILED - test on $infile did not fail as expected"
+ else
+ echo " PASSED"
+ fi
+
+}
+
# ADD_HELP_TEST
TOOLTEST_HELP() {
@@ -1448,6 +1479,10 @@ TOOLTEST err_attr_dspace.ddl err_attr_dspace.h5
# test to verify HDFFV-9407: long double full precision
#GREPTEST OUTTXT "1.123456789012345" t128bit_float.ddl -m %.35Lf t128bit_float.h5
+# test to verify HDFFV-10480: out of bounds read in H5O_fill_new[old]_decode
+TOOLTEST_FAIL tCVE_2018_11206_fill_old.h5
+TOOLTEST_FAIL tCVE_2018_11206_fill_new.h5
+
# Clean up temporary files/directories
CLEAN_TESTFILES_AND_TESTDIR
diff --git a/tools/testfiles/tCVE_2018_11206_fill_new.h5 b/tools/testfiles/tCVE_2018_11206_fill_new.h5
new file mode 100644
index 0000000..643591c
--- /dev/null
+++ b/tools/testfiles/tCVE_2018_11206_fill_new.h5
Binary files differ
diff --git a/tools/testfiles/tCVE_2018_11206_fill_old.h5 b/tools/testfiles/tCVE_2018_11206_fill_old.h5
new file mode 100644
index 0000000..7f5b41a
--- /dev/null
+++ b/tools/testfiles/tCVE_2018_11206_fill_old.h5
Binary files differ