diff options
Diffstat (limited to 'release_docs/RELEASE.txt')
| -rw-r--r-- | release_docs/RELEASE.txt | 468 |
1 files changed, 290 insertions, 178 deletions
diff --git a/release_docs/RELEASE.txt b/release_docs/RELEASE.txt index 2050a1c..2b54b44 100644 --- a/release_docs/RELEASE.txt +++ b/release_docs/RELEASE.txt @@ -1,25 +1,25 @@ -HDF5 version 1.8.20 released on 2017-11-28 +HDF5 version 1.8.21 released on 2018-06-04 ================================================================================ INTRODUCTION ============ -This document describes the differences between HDF5-1.8.19 and -HDF5-1.8.20, and contains information on the platforms tested and -known problems in HDF5-1.8.20. +This document describes the differences between HDF5-1.8.20 and +HDF5-1.8.21, and contains information on the platforms tested and +known problems in HDF5-1.8.21. For more details, see the files HISTORY-1_0-1_8_0_rc3.txt and HISTORY-1_8.txt in the release_docs/ directory of the HDF5 source. -Links to the HDF5 1.8.20 source code, documentation, and additional materials +Links to the HDF5 1.8.21 source code, documentation, and additional materials can be found on the HDF5 web page at: https://support.hdfgroup.org/HDF5/ -The HDF5 1.8.20 release can be obtained from: +The HDF5 1.8.21 release can be obtained from: https://support.hdfgroup.org/HDF5/release/obtain518.html -User documentation for 1.8.20 can be accessed directly at this location: +User documentation for 1.8.21 can be accessed directly at this location: https://support.hdfgroup.org/HDF5/doc1.8/ @@ -30,8 +30,8 @@ in 1.8.0?" document: https://support.hdfgroup.org/HDF5/doc/ADGuide/WhatsNew180.html All new and modified APIs are listed in detail in the "HDF5 Software Changes -from Release to Release" document, in the section "Release 1.8.20 (current -release) versus Release 1.8.19 +from Release to Release" document, in the section "Release 1.8.21 (current +release) versus Release 1.8.20 https://support.hdfgroup.org/HDF5/doc1.8/ADGuide/Changes.html @@ -55,252 +55,373 @@ CONTENTS New Features ============ - Tools - ----- - - h5diff + Configuration + ------------- + - CMake + + Change minimum version to 3.10. + + This change removes the need to support a copy of the FindMPI.cmake module, + which has been removed, along with its subfolder in the config/cmake_ext_mod + location. + + (ADB - 2018/03/09) - h5diff has new option enable-error-stack. + - CMake - Updated h5diff with the --enable-error-stack argument, which - enables the display of the hdf5 error stack. This completes the - improvement to the main tools; h5copy, h5diff, h5dump, h5ls and - h5repack. + Add pkg-config file generation - (ADB - 2017/08/30, HDFFV-9774) + Added pkg-config file generation for the C, C++, HL, and HL C++ libraries. + In addition, builds on linux will create h5cXXX scripts that use the pkg-config + files. This is a limited implementation of a script like autotools h5cc. + (ADB - 2018/03/08, HDFFV-4359) + + - CMake + + Refactor use of CMAKE_BUILD_TYPE for new variable, which understands + the type of generator in use. + + Added new configuration macros to use new HDF_BUILD_TYPE variable. This + variable is set correctly for the type of generator being used for the build. + + (ADB - 2018/01/08, HDFFV-10385, HDFFV-10296) C++ API ------- - - The following C++ API wrappers have been added to the C++ Library: + - The following C++ API wrappers have been added to class H5Location + + H5Lcreate_soft: + // Creates a soft link from link_name to target_name. + void link(const char *target_name, const char *link_name,...) + void link(const H5std_string& target_name,...) + + + H5Lcreate_hard: + // Creates a hard link from new_name to curr_name. + void link(const char *curr_name, const Group& new_loc,...) + void link(const H5std_string& curr_name, const Group& new_loc,...) + + // Creates a hard link from new_name to curr_name in the same location. + void link(const char *curr_name, const hid_t same_loc,...) + void link(const H5std_string& curr_name, const hid_t same_loc,...) + + Note: previous version CommonFG::link will be deprecated. - // Creates a binary object description of this datatype. - void DataType::encode() - C API H5Tencode() + + H5Lcopy: + // Copy an object from a group of file to another. + void copyLink(const char *src_name, const Group& dst,...) + void copyLink(const H5std_string& src_name, const Group& dst,...) - // Returns the decoded type from the binary object description. - DataType::decode() - C API H5Tdecode() - ArrayType::decode() - C API H5Tdecode() - CompType::decode() - C API H5Tdecode() - DataType::decode() - C API H5Tdecode() - EnumType::decode() - C API H5Tdecode() - FloatType::decode() - C API H5Tdecode() - IntType::decode() - C API H5Tdecode() - StrType::decode() - C API H5Tdecode() - VarLenType::decode() - C API H5Tdecode() + // Copy an object from a group of file to the same location. + void copyLink(const char *src_name, const char *dst_name,...) + void copyLink(const H5std_string& src_name,...) - // Three overloaded functions to retrieve information about an object - H5Location::getObjectInfo() - H5Oget_info()/H5Oget_info_by_name() + + H5Lmove: + // Rename an object in a group or file to a new location. + void moveLink(const char* src_name, const Group& dst,...) + void moveLink(const H5std_string& src_name, const Group& dst,...) - (BMR - 2017/10/17, HDFFV-10175) + // Rename an object in a group or file to the same location. + void moveLink(const char* src_name, const char* dst_name,...) + void moveLink(const H5std_string& src_name,...) - - New constructors to open existing datatypes added in ArrayType, - CompType, DataType, EnumType, FloatType, IntType, StrType, and - VarLenType. + Note: previous version CommonFG::move will be deprecated. - (BMR - 2017/10/17, HDFFV-10175) + + H5Ldelete: + // Removes the specified link from this location. + void unlink(const char *link_name, + const LinkAccPropList& lapl = LinkAccPropList::DEFAULT) + void unlink(const H5std_string& link_name, + const LinkAccPropList& lapl = LinkAccPropList::DEFAULT) - - A document is added to the HDF5 C++ API Reference Manual to show the - mapping from a C API to C++ wrappers. It can be found from the main - page of the C++ API Reference Manual. + Note: An additional parameter is added to CommonFG::unlink and it + is moved to H5Location. - (BMR - 2017/10/17, HDFFV-10151) + (BMR - 2018/05/11 - HDFFV-10445) + - New property list subclasses - High-Level APIs - --------------- - - H5DOread_chunk + Property list subclasses StrCreatPropList, LinkCreatPropList, and + AttrCreatPropList are added for the C property list classes + H5P_STRING_CREATE, H5P_LINK_CREATE, and H5P_ATTRIBUTE_CREATE. - Users wanted to read compressed data directly from a file without any - processing by the HDF5 data transfer pipeline, just as they were able - to write it directly to a file with H5DOwrite_chunk. + (BMR - 2018/05/11 - HDFFV-10445) - New API function, corresponding to existing function H5DOwrite_chunk. - H5DOread_chunk reads a raw data chunk directly from a chunked dataset - in the file into the application buffer, bypassing the library’s internal - data transfer pipeline, including filters. + - Another argument, LinkCreatPropList& lcpl, is added to the following + functions for the use of link creation property list. + Group createGroup(const char* name, size_t size_hint = 0, + const LinkCreatPropList& lcpl = LinkCreatPropList::DEFAULT) + Group createGroup(const H5std_string& name, size_t size_hint = 0, + const LinkCreatPropList& lcpl = LinkCreatPropList::DEFAULT) + + (BMR - 2018/05/11 - HDFFV-10445) - (VC - 2017/05/02, HDFFV-9934) Support for New Platforms, Languages, and Compilers =================================================== - - Added NAG compiler - + - Added support for Visual Studio 2017 w/ Intel Fortran 18 on Windows 10 x64. -Bug Fixes since HDF5-1.8.19 + +Bug Fixes since HDF5-1.8.20 =========================== - Configuration - ------------- - - cmake + - If an HDF5 file contains a filter pipeline message with a 'number of + filters' field that exceeds the maximum number of allowed filters, + the error handling code will attempt to dereference a NULL pointer. - The hdf5 library used shared szip and zlib, which needlessly required - applications to link with the same szip and zlib libraries. + This issue was reported to The HDF Group as issue #CVE-2017-17505. - Changed the target_link_libraries commands to use the static libs. - Removed improper link duplication of szip and zlib. - Adjusted the link dependencies and the link interface values of - the target_link_libraries commands. + NOTE: The HDF5 C library cannot produce such a file. This condition + should only occur in a corrupt (or deliberately altered) file + or a file created by third-party software. - (ADB - 2017/11/14, HDFFV-10329) + This problem arose because the error handling code assumed that + the 'number of filters' field implied that a dynamic array of that + size had already been created and that the cleanup code should + iterate over that array and clean up each element's resources. If + an error occurred before the array has been allocated, this will + not be true. - - cmake MPI + This has been changed so that the number of filters is set to + zero on errors. Additionally, the filter array traversal in the + error handling code now requires that the filter array not be NULL. - CMake implementation for MPI was problematic and would create incorrect - MPI library references in the hdf5 libraries. + (DER - 2018/02/06, HDFFV-10354) - Reworked the CMake MPI code to properly create CMake targets.Also merged - the latest CMake FindMPI.cmake changes to the local copy. This is necessary - until HDF changes the CMake minimum to 3.9 or greater. + - If an HDF5 file contains a filter pipeline message which contains + a 'number of filters' field that exceeds the actual number of + filters in the message, the HDF5 C library will read off the end of + the read buffer. - (ADB - 2017/11/02, HDFFV-10321) + This issue was reported to The HDF Group as issue #CVE-2017-17506. + NOTE: The HDF5 C library cannot produce such a file. This condition + should only occur in a corrupt (or deliberately altered) file + or a file created by third-party software. - - Fixed Fortran linker flags when using the NAG Fortran compiler (autotools). + The problem was fixed by passing the buffer size with the buffer + and ensuring that the pointer cannot be incremented off the end + of the buffer. A mismatch between the number of filters declared + and the actual number of filters will now invoke normal HDF5 + error handling. - (HDFFV-10037, MSB, 2017/10/21) + (DER - 2018/02/26, HDFFV-10355) - - cmake + - If an HDF5 file contains a malformed compound datatype with a + suitably large offset, the type conversion code can run off + the end of the type conversion buffer, causing a segmentation + fault. - Too many commands for POST_BUILD step caused command line to be - too big on windows. + This issue was reported to The HDF Group as issue #CVE-2017-17507. - Changed foreach of copy command to use a custom command with the - use of the HDFTEST_COPY_FILE macro. + NOTE: The HDF5 C library cannot produce such a file. This condition + should only occur in a corrupt (or deliberately altered) file + or a file created by third-party software. - (ADB - 2017/07/12, HDFFV-10254) + THE HDF GROUP WILL NOT FIX THIS BUG AT THIS TIME + Fixing this problem would involve updating the publicly visible + H5T_conv_t function pointer typedef and versioning the API calls + which use it. We normally only modify the public API during + major releases, so this bug will not be fixed at this time. - Library - ------- - - filter plugin handling in H5PL.c and H5Z.c + (DER - 2018/02/26, HDFFV-10356) + + - If an HDF5 file contains a malformed compound type which contains + a member of size zero, a division by zero error will occur while + processing the type. - It was discovered that the dynamic loading process used by - filter plugins had issues with library dependencies. + This issue was reported to The HDF Group as issue #CVE-2017-17508. - CMake build process changed to use LINK INTERFACE keywords, which - allowed HDF5 C library to make dependent libraries private. The - filter plugin libraries no longer require dependent libraries - (such as szip or zlib) to be available. - (ADB - 2017/11/16, HDFFV-10328) + NOTE: The HDF5 C library cannot produce such a file. This condition + should only occur in a corrupt (or deliberately altered) file + or a file created by third-party software. - - Fix rare object header corruption bug + Checking for zero before dividing fixes the problem. Instead of the + division by zero, the normal HDF5 error handling is invoked. - In certain cases, such as when converting large attributes to dense - storage, an error could occur which would either fail an assertion or - cause file corruption. Fixed and added test. + (DER - 2018/02/26, HDFFV-10357) - (NAF - 2017/11/14, HDFFV-10274) + - If an HDF5 file contains a malformed symbol table node that declares + it contains more symbols than it actually contains, the library + can run off the end of the metadata cache buffer while processing + the symbol table node. - - H5Zfilter_avail in H5Z.c + This issue was reported to The HDF Group as issue #CVE-2017-17509. + + NOTE: The HDF5 C library cannot produce such a file. This condition + should only occur in a corrupt (or deliberately altered) file + or a file created by third-party software. + + Performing bounds checks on the buffer while processing fixes the + problem. Instead of the segmentation fault, the normal HDF5 error + handling is invoked. + + (DER - 2018/03/12, HDFFV-10358) + + + Configuration + ------------- + - Library - The public function checked for plugins, while the private - function did not. + Moved the location of gcc attribute. - Modified H5Zfilter_avail and private function, H5Z_filter_avail. - Moved check for plugin from public to private function. Updated - H5P__set_filter due to change in H5Z_filter_avail. Updated tests. + The gcc attribute(no_sanitize), named as the macro HDF_NO_UBSAN, + was located after the function name. Builds with GCC 7 did not + indicate any problem, but GCC 8 issued errors. Moved the + attribute before the function name, as required. - (ADB - 2017/10/10, HDFFV-10297, HDFFV-10319) + (ADB 2018/05/22, HDFFV-10473) - - Fix H5Sencode bug when num points selected is >2^32 + - CMake - Modified to fail if the 32 bit limit is exceeded when encoding either - offsets or counts in the selection. + Update CMake commands configuration. - (HDFFV-10323, VC, 2017/09/07) + A number of improvements were made to the CMake commands. Most + changes simplify usage or eliminate unused constructs. Also, + some changes support better cross-platform support. - - Fix H5HL_offset_into() + (ADB - 2018/02/01, HDFFV-10398) - (1) Fix H5HL_offset_into() to return error when offset exceeds heap data - block size. - (2) Fix other places in the library that call this routine to detect - error routine. + - CMake - (HDFFV-10216, VC, 2017/09/05) + Correct usage of CMAKE_BUILD_TYPE variable. + + The use of the CMAKE_BUILD_TYPE is incorrect for multi-config + generators (Visual Studio and XCode) and is optional for single + config generators. Created a new macro to check + GLOBAL PROPERTY -> GENERATOR_IS_MULTI_CONFIG + Created two new HDF variable, HDF_BUILD_TYPE and HDF_CFG_BUILD_TYPE. + Defaults for these variables is "Release". + + (ADB - 2018/01/10, HDFFV-10385) + + - CMake + + Add replacement of fortran flags if using static CRT. + + Added TARGET_STATIC_CRT_FLAGS call to HDFUseFortran.cmake file in + config/cmake_ext_mod folder. + + (ADB - 2018/01/08, HDFFV-10334) + + + Library + ------- + - Utility function can not handle lowercase Windows drive letters + + Added call to toupper function for drive letter. + + (ADB - 2017/12/18, HDFFV-10307) Tools ----- - h5repack - h5repack failed to copy a dataset with existing filter. + h5repack changes the chunk parameters when a change of layout is not + specified and a filter is applied. - Reworked code for h5repack and h5diff code in tools library. Added - improved error handling, cleanup of resources and checks of calls. - Modified H5Zfilter_avail and private function, H5Z_filter_avail. - Moved check for plugin from public to private function. Updated - H5P__set_filter due to change in H5Z_filter_avail. Updated tests. - Note, h5repack output display has changed to clarify the individual - steps of the repack process. The output indicates if an operation - applies to all objects. Lines with notation and no information - have been removed. + HDFFV-10297, HDFFV-10319 reworked code for h5repack and h5diff code + in the tools library. The check for an existing layout was incorrectly + placed into an if block and not executed. The check was moved into + the normal path of the function. - (ADB - 2017/10/10, HDFFV-10297, HDFFV-10319) + (ADB - 2018/02/21, HDFFV-10412) - - h5repack + - h5dump - h5repack always set the User Defined filter flag to H5Z_FLAG_MANDATORY. + the tools library will hide the error stack during file open. - Added another parameter to the 'UD=' option to set the flag by default - to '0' or H5Z_FLAG_MANDATORY, the other choice is '1' or H5Z_FLAG_OPTIONAL. + While this is preferable almost always, there are reasons to enable + display of the error stack when a tool will not open a file. Adding an + optional argument to the --enable-error-stack will provide this use case. + As an optional argument it will not affect the operation of the + --enable-error-stack. h5dump is the only tool to implement this change. - (ADB - 2017/08/31, HDFFV-10269) + (ADB - 2018/02/15, HDFFV-10384) - - h5ls + - h5dump - h5ls generated error on stack when it encountered a H5S_NULL - dataspace. + h5dump would output an indented blank line in the filters section. - Adding checks for H5S_NULL before calling H5Sis_simple (located - in the h5tools_dump_mem function) fixed the issue. + h5dump overused the h5tools_simple_prefix function, which is a + function intended to account for the data index (x,y,z) option. + Removed the function call for header information. - (ADB - 2017/08/17, HDFFV-10188) + (ADB - 2018/01/25, HDFFV-10396) - - h5dump + - h5repack - h5dump segfaulted on output of XML file. + h5repack incorrectly searched internal object table for name. - Function that escape'd strings used the full buffer length - instead of just the length of the replacement string in a - strncpy call. Using the correct length fixed the issue. + h5repack would search the table of objects for a name, if the + name did not match it tried to determine if the name without a + leading slash would match. The logic was flawed! The table + stored names(paths) without a leading slash and did a strstr + of the table path to the name. + The assumption was that if there was a difference of one then + it was a match, however "pressure" would match "/pressure" as + well as "/pressure1", "/pressure2", etc. Changed logic to remove + any leading slash and then do a full compare of the name. - (ADB - 2017/08/01, HDFFV-10256) + (ADB - 2018/01/18, HDFFV-10393) - - h5diff + - h5repack - h5diff segfaulted on compare of a NULL variable length string. + h5repack failed to handle command line parameters for customer filters. - Improved h5diff compare of strings by adding a check for - NULL strings and setting the lengths to zero. + User defined filter parameter conversions would fail when integers + were represented on the command line with character strings + larger than 9 characters. Increased local variable array for storing + the current command line parameter to prevent buffer overflows. - (ADB - 2017/07/25, HDFFV-10246) + (ADB - 2018/01/17, HDFFV-10392) - - h5import + - h5diff - h5import crashed trying to import data from a subset of a dataset. + h5diff seg faulted if comparing VL strings against fixed strings. - Improved h5import by adding the SUBSET keyword. h5import understands - to use the Count times the Block as the size of the dimensions. - Added INPUT_B_ORDER keyword to old-style configuration files. - The import from h5dump function expects the binary files to use native - types (FILE '-b' option) in the binary file. + Reworked solution for HDFFV-8625 and HDFFV-8639. Implemented the check + for string objects of same type in the diff_can_type function by + adding an if(tclass1 == H5T_STRING) block. This "if block" moves the + same check that was added for attributes to this function, which is + used by all object types. This function handles complex type structures. + Also added a new test file in h5diffgentest for testing this issue + and removed the temporary files used in the test scripts. - (ADB - 2017/06/15, HDFFV-10219) + (ADB - 2018/01/04, HDFFV-8745) C++ API ------- - - Marked the following functions deprecated because they were moved to - class H5Object: - H5Location::createAttribute() - H5Location::openAttribute() - H5Location::attrExists() - H5Location::removeAttr() - H5Location::renameAttr() - H5Location::getNumAttrs() + - Removal of memory leaks + + A private function was inadvertently called, causing memory leaks. This + is now fixed. - (BMR - 2017/10/17) + (BMR - 2018/04/12 - User reported in email) + + - Changes in exception classes + + Some exception classes are reorganized to reflect the HDF5 object + hierarchy and allow customization. + DataSetIException -> LocationException -> Exception + DataTypeIException -> LocationException -> Exception + GroupIException -> LocationException -> Exception + AttributeIException -> LocationException -> Exception + FileIException -> GroupIException -> LocationException -> Exception + Member functions in H5Location and H5Object now throw specific exceptions + associated with the invoking objects. + + (BMR - 2018/05/11) + + - H5Location::closeObjId is made static + (BMR - 2018/05/11) + + - H5A wrappers in H5Location are removed as they have been in H5Object. + (BMR - 2018/05/11) Supported Platforms @@ -314,10 +435,10 @@ They are built with the configure process unless specified otherwise. Versions 4.9.3, 5.3.0, 6.2.0 PGI C, Fortran, C++ for 64-bit target on x86-64; - Version 16.10-0 + Version 17.10-0 Intel(R) C (icc), C++ (icpc), Fortran (icc) compilers: - Version 17.0.0.196 Build 20160721 + Version 17.0.4.196 Build 20160721 MPICH 3.1.4 compiled with GCC 4.9.3 OpenMPI 2.0.1 compiled with GCC 4.9.3 @@ -341,25 +462,18 @@ They are built with the configure process unless specified otherwise. (emu) Sun Fortran 95 8.6 SunOS_sparc Sun C++ 5.12 SunOS_sparc - Windows 7 Visual Studio 2012 w/ Intel Fortran 15 (cmake) - Visual Studio 2013 w/ Intel Fortran 15 (cmake) - Visual Studio 2015 w/ Intel Fortran 16 (cmake) + Windows 7 Visual Studio 2015 w/ Intel Fortran 16 (cmake) Windows 7 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake) Visual Studio 2013 w/ Intel Fortran 15 (cmake) Visual Studio 2015 w/ Intel Fortran 16 (cmake) Visual Studio 2015 w/ Intel C, Fortran 2017 (cmake) Visual Studio 2015 w/ MSMPI 8 (cmake) - Cygwin(CYGWIN_NT-6.1 2.8.0(0.309/5/3) - gcc and gfortran compilers (GCC 5.4.0) - (cmake and autotools) Windows 10 Visual Studio 2015 w/ Intel Fortran 16 (cmake) - Cygwin(CYGWIN_NT-6.1 2.8.0(0.309/5/3) - gcc and gfortran compilers (GCC 5.4.0) - (cmake and autotools) Windows 10 x64 Visual Studio 2015 w/ Intel Fortran 16 (cmake) + Visual Studio 2017 w/ Intel Fortran 18 (cmake) Mac OS X Mavericks 10.9.5 Apple LLVM version 6.0 (clang-600.0.57) 64-bit gfortran GNU Fortran (GCC) 4.9.2 @@ -401,7 +515,6 @@ Windows 7 Cygwin n y/n n y y y Windows 7 x64 Cygwin n y/n n y y y Windows 10 y y/y n y y y Windows 10 x64 y y/y n y y y -Mac OS X Mavericks 10.9.5 64-bit n y/y n y y y Mac OS X Yosemite 10.10.5 64-bit n y/y n y y y Mac OS X El Capitan 10.11.6 64-bit n y/y n y y y Mac OS Sierra 10.12.6 64-bit n y/y n y y y @@ -411,7 +524,7 @@ CentOS 6.7 Linux 2.6.32 x86_64 Intel n y/y n y y y CentOS 6.7 Linux 2.6.32 x86_64 PGI n y/y n y y y CentOS 7.1 Linux 3.10.0 x86_64 GNU y y/y y y y y CentOS 7.1 Linux 3.10.0 x86_64 Intel n y/y n y y y -Linux 2.6.32-431.11.2.el6.ppc64 n y/n n y y y +Linux 2.6.32-573.18.1.el6.ppc64 n y/n n y y y Platform Shared Shared Shared Thread- C libs F90 libs C++ libs safe @@ -423,7 +536,6 @@ Windows 7 Cygwin n n n y Windows 7 x64 Cygwin n n n y Windows 10 y y y y Windows 10 x64 y y y y -Mac OS X Mavericks 10.9.5 64-bit y n y y Mac OS X Yosemite 10.10.5 64-bit y n y y Mac OS X El Capitan 10.11.6 64-bit y n y y Mac OS Sierra 10.12.6 64-bit y n y y @@ -433,7 +545,7 @@ CentOS 6.7 Linux 2.6.32 x86_64 Intel y y y y CentOS 6.7 Linux 2.6.32 x86_64 PGI y y y y CentOS 7.1 Linux 3.10.0 x86_64 GNU y y y y CentOS 7.1 Linux 3.10.0 x86_64 Intel y y y y -Linux 2.6.32-431.11.2.el6.ppc64 y y y y +Linux 2.6.32-573.18.1.el6.ppc64 y y y y Compiler versions for each platform are listed in the preceding "Supported Platforms" table. |