diff options
author | Larry Knox <lrknox@hdfgroup.org> | 2019-12-24 21:12:08 (GMT) |
---|---|---|
committer | Larry Knox <lrknox@hdfgroup.org> | 2019-12-24 21:12:08 (GMT) |
commit | 5b9cf732caab9daa6ed1e00f2df4f5a792340196 (patch) | |
tree | 4c01c9bd71ba5e7d8d11b474f6896e6ffbf0416d /release_docs/RELEASE.txt | |
parent | 646fc294078f560fc9bef784cb1c4e27cdc51f5b (diff) | |
parent | e0fa36bfe008c03365ce8ea94d705f8fdebccdf0 (diff) | |
download | hdf5-5b9cf732caab9daa6ed1e00f2df4f5a792340196.zip hdf5-5b9cf732caab9daa6ed1e00f2df4f5a792340196.tar.gz hdf5-5b9cf732caab9daa6ed1e00f2df4f5a792340196.tar.bz2 |
Merge pull request #2203 in HDFFV/hdf5 from ~LRKNOX/hdf5_lrk:1.10/master to 1.10/masterhdf5-1_10_6
* commit 'e0fa36bfe008c03365ce8ea94d705f8fdebccdf0': (166 commits)
Update release date strings in README.txt and release_docs/RELEASE.txt
Add release note about using ompio instead of romio when collective writes fail with OpenMPI.
Fixed typos, grammatical errors, etc. Expanded entry for S3 and HDFS VFDs. Moved Mac 10.11 and 10.12 from Supported Platforms to More Platforms Tested.
Add qualification to RELEASE.txt regarding performance improvements
Standalone doesn't use h5test implementation.
Add missing note
Update COPYING file with new HDF5 license.
Update bin/config.guess and config.sub to current version available from git.savannah.gnu.org.
Fix include to correct memory calls - big-endian issue.
Revert BLDLIBDIR value in junit.sh.in.
Corrections to CMake functions
Update version for HDF5 1.10.6 release. Remove autom4te.cache directory that shouldn't have been added.
Remove duplicate instance
Fix merge issue
HDFFV-10979 - fix global name clash
Correct clang search and java include
Latest date first in RELEASE.txt
Update RELEASE.txt with performance improvements and Steven Varga's bugfix.
Update version numbers for shared lib files according to api compatibility report.
Created hdf5_1_10_6 branch for HDF5 1.10.6 release preparation. Set version to 1.10.6-pre1. Switched default maintainer mode to disabled and default build mode to production. Added files produced by autogen.sh to commit except for src/H5config.h.in~ and src/H5public.h~.
...
Diffstat (limited to 'release_docs/RELEASE.txt')
-rw-r--r-- | release_docs/RELEASE.txt | 623 |
1 files changed, 251 insertions, 372 deletions
diff --git a/release_docs/RELEASE.txt b/release_docs/RELEASE.txt index 2192811..f343af1 100644 --- a/release_docs/RELEASE.txt +++ b/release_docs/RELEASE.txt @@ -1,4 +1,4 @@ -HDF5 version 1.10.5 released on 2019-02-25 +HDF5 version 1.10.6 released on 2019-12-23 ================================================================================ @@ -34,7 +34,7 @@ CONTENTS - New Features - Support for new platforms and languages -- Bug Fixes since HDF5-1.10.4 +- Bug Fixes since HDF5-1.10.5 - Supported Platforms - Tested Configuration Features Summary - More Tested Platforms @@ -47,455 +47,318 @@ New Features Configuration: ------------- - - Cross compile on mutrino and other Cray XC40 systems. + - Update CMake for VS2019 support - Added support for CMake options to use CrayLinuxEnvironment, craype-mic-knl - module for building with craype-haswell module for configuration, and - batch scripts in bin/batch for use with sbatch or bsub to run tests in - batch jobs on compute nodes. An instruction file README_HPC describing - the use of these options was added in release_docs. + CMake added support for VS2019 in version 3.15. Changes to the CMake + generator setting required changes to scripts. Also updated version + references in CMake files as necessary. - (LRK - 2019/02/18, TRILABS-34) + (ADB - 2019/11/18, HDFFV-10962) - - Rework CMake command files to fix MPI testing. + - Update CMake options to match new autotools options - Added setup fixture to remove any test generated files and added DEPENDS - to test properties to execute tests in order expected. + Add configure options (autotools - CMake): + enable-asserts HDF5_ENABLE_ASSERTS + enable-symbols HDF5_ENABLE_SYMBOLS + enable-profiling HDF5_ENABLE_PROFILING + enable-optimization HDF5_ENABLE_OPTIMIZATION + In addition NDEBUG is no longer forced defined and relies on the CMake + process. - (ADB - 2019/02/14, TRILABS-111) + (ADB - 2019/10/07, HDFFV-100901, HDFFV-10637, TRILAB-97) - - Disable SZIP or ZLIB options if TGZ files are not available. + - Update CMake tests to use FIXTURES - Changed the TGZ option for SZip and ZLib to disable the options - if the source tar.gz files are not found. + CMake test fixtures allow setup/cleanup tests and other dependency + requirements as properties for tests. This is more flexible for + modern CMake code. - (ADB - 2019/02/05, HDFFV-10697) + (ADB - 2019/07/23, HDFFV-10529) - - Added a new option to enable/disable using pread/pwrite instead of - read/write in the sec2, log, and core VFDs. + - Windows PDB files are always installed - This option is enabled by default when pread/pwrite are detected. + There are build configuration or flag settings for Windows that may not + generate PDB files. If those files are not generated then the install + utility will fail because those PDB files are not found. An optional + variable, DISABLE_PDB_FILES, was added to not install PDB files. - Autotools: --enable-preadwrite - CMake: HDF5_ENABLE_PREADWRITE + (ADB - 2019/07/17, HDFFV-10424) - (DER - 2019/02/03, HDFFV-10696) + - Add mingw CMake support with a toolchain file - - Rework CMake versioning for OSX platforms. + There have been a number of mingw issues that have been linked under + HDFFV-10845. It has been decided to implement the CMake cross-compiling + technique of toolchain files. We will use a linux platform with the mingw + compiler stack for testing. Only the C language is fully supported, and + the error tests are skipped. The C++ language works for static but shared + builds have a shared library issue with the mingw Standard Exception Handling + library, which is not available on Windows. Fortran has a common cross-compile + problem with the fortran configure tests. - Changed the current_version and compatibility_version flags from optional - with HDF5_BUILD_WITH_INSTALL_NAME to always setting the flags for OSX. + (ADB - 2019/07/12, HDFFV-10845, HDFFV-10595) - (ADB - 2019/01/22, HDFFV-10685) + - Windows PDB files are installed incorrectly - - Rework CMake command files to eliminate developer CMP005 warning + For static builds, the PDB files for windows should be installed next + to the static libraries in the lib folder. Also the debug versions of + libraries and PDB files are now correctly built using the default + CMAKE_DEBUG_POSTFIX setting. - Use variables without quotes in if () statements. + (ADB - 2019/07/09, HDFFV-10581) - (ADB - 2019/01/18, TILABS-105) + - Add option to build only shared libs - - Rework CMake configure files to eliminate developer CMP0075 warning + A request was made to prevent building static libraries and only build + shared. A new option was added to CMake, ONLY_SHARED_LIBS, which will + skip building static libraries. Certain utility functions will build with + static libs but are not published. Tests are adjusted to use the correct + libraries depending on SHARED/STATIC settings. - Renamed varname to HDF5_REQUIRED_LIBRARIES as the contents were not - required for configuration. Also moved check includes calls to top of - files. + (ADB - 2019/06/12, HDFFV-10805) - (ADB - 2019/01/03, HDFFV-10546) + - Add options to enable or disable building tools and tests - - Keep stderr and stdout separate in tests + Configure options --enable-tests and --enable-tools were added for + autotools configure. These options are enabled by default, and can be + disabled with either --disable-tests (or tools) or --enable-tests=no + (or --enable-tools=no). Build time is reduced ~20% when tools are + disabled, 35% when tests are disabled, 45% when both are disabled. + Re-enabling them after the initial build requires running configure + again with the option(s) enabled. - Changed test handling of output capture. Tests now keep the stderr - output separate from the stdout output. It is up to the test to decide - which output to check against a reference. Also added the option - to grep for a string in either output. + (LRK - 2019/06/12, HDFFV-9976) - (ADB - 2018/12/12, HDFFV-10632) + - Change tools tests to search the error stack - - Incorrectly installed private header files were removed from - CMake installs. + There are some use cases which can cause the error stack of tools to be + different then the expected output. These tests now use grepTest.cmake; + this was changed to allow the error file to be searched for an expected string. - The CMake build files incorrectly flagged the following header files - as public and installed them. They are private and will no longer be - installed. - - HDF5 library private package files (H5Xpkg.h) - H5Edefin.h - H5Einit.h - H5Eterm.h - H5LTparse.h - h5diff.h - h5tools_dump.h - h5tools.h - h5tools_ref.h - h5tools_str.h - h5tools_utils.h - h5trav.h - - (DER - 2018/10/26, HDFFV-10614, 10609) - - - Autotools installs now install H5FDwindows.h - - This is simply to align the installed header files between the - autotools and CMake. H5FDwindows.h has no functionality on - non-Windows systems. - - (DER - 2018/10/26, HDFFV-10614) + (ADB - 2019/04/15, HDFFV-10741) Library: -------- - - The sec2, log, and core VFDs can now use pread/pwrite instead of - read/write. - - pread and pwrite do not change the file offset, a feature that was - requested by a user working with a multi-threaded application. - - The option to configure this feature is described above. - - (DER - 2019/02/03, HDFFV-10696) - - - Add ability to minimze dataset object headers. - - Creation of many, very small datasets resulted in extensive file bloat - due to extra space in the dataset object headers -- this space is - allocated by default to allow for the insertion of a small number of - attributes within the object header and not require a continuation - block, an unnecessary provision in the target use case. - - Inform the library to expect no attributes on created datasets, and to - allocate the least space possible for the object headers. - NOTE: A continuation block is created if attributes are added to a - 'minimized' dataset, which can reduce performance. - NOTE: Some extra space is allocated for attributes essential to the - correct behavior of the object header (store creation times, e.g.). This - does not violate the design principle, as the space is calculated and - allocated as needed at the time of dataset object header creation -- - unused space is not generated. - New API calls: - H5Fget_dset_no_attrs_hint - H5Fset_dset_no_attrs_hint - H5Pget_dset_no_attrs_hint - H5Pset_dset_no_attrs_hint - - (JOS - 2019/01/04, TRILAB-45) - - - Added new chunk query functions - - The following public functions were added to discover information about - the chunks in an HDF5 file. - herr_t H5Dget_num_chunks(dset_id, fspace_id, *nchunks) - herr_t H5Dget_chunk_info_by_coord(dset_id, *coord, *filter_mask, *addr, *size) - herr_t H5Dget_chunk_info(dset_id, fspace_id, index, *coord, *filter_mask, *addr, *size) - - (BMR - 2018/11/07, HDFFV-10615) - - - Several empty public header files where removed from the distribution - - The following files were empty placeholders. They are for internal - packages that are unlikely to ever have public functionality and have - thus been removed. - - H5Bpublic.h - H5B2public.h - H5FSpublic.h - H5HFpublic.h - H5HGpublic.h - H5HLpublic.h - - They were only installed in CMake builds. - - (DER - 2018/10/26, HDFFV-10614) - - - Parallel Library: - ----------------- - - Changed the default behavior in parallel when reading the same dataset in its entirety - (i.e. H5S_ALL dataset selection) which is being read by all the processes collectively. - The dataset must be contiguous, less than 2GB, and of an atomic datatype. - The new behavior is the HDF5 library will use an MPI_Bcast to pass the data read from - the disk by the root process to the remain processes in the MPI communicator associated - with the HDF5 file. - - (MSB - 2019/01/02, HDFFV-10652) - - - All MPI-1 API calls have been replaced with MPI-2 equivalents. - - This was done to better support OpenMPI, as default builds no longer - include MPI-1 support (as of OpenMPI 4.0). - - (DER - 2018/12/30, HDFFV-10566) - - Fortran Library: - ---------------- - - Added wrappers for dataset object header minimization calls. - (see the note for TRILAB-45, above) - - New API calls: - - h5fget_dset_no_attrs_hint_f - h5fset_dset_no_attrs_hint_f - h5pget_dset_no_attrs_hint_f - h5pset_dset_no_attrs_hint_f - - (DER - 2019/01/09, TRILAB-45) - - - Added new Fortran derived type, c_h5o_info_t, which is interoperable with - C's h5o_info_t. This is needed for callback functions which - pass C's h5o_info_t data type definition. - - (MSB, 2019/01/08, HDFFV-10443) - - - Added new Fortran API, H5gmtime, which converts (C) 'time_t' structure - to Fortran DATE AND TIME storage format. - - (MSB, 2019/01/08, HDFFV-10443) - - - Added new Fortran 'fields' optional parameter to: h5ovisit_f, h5oget_info_by_name_f, - h5oget_info, h5oget_info_by_idx and h5ovisit_by_name_f. - - (MSB, 2019/01/08, HDFFV-10443) + - Added S3 and HDFS Virtual File Drivers (VFDs) to HDF5 + + These new VFDs have been introduced in HDF5-1.10.6. Instructions to + enable them when configuring HDF5 on Linux and Mac may be found at + https://portal.hdfgroup.org/display/HDF5/Virtual+File+Drivers+-+S3+and+HDFS. + + Installing on Windows requires CMake 3.13 and the following additional setup. + Install openssl library (with dev files); + from "Shining Light Productions". msi package preferred. + + PATH should have been updated with the installation dir. + set ENV variable OPENSSL_ROOT_DIR to the installation dir. + set ENV variable OPENSSL_CONF to the cfg file, likely %OPENSSL_ROOT_DIR%\bin\openssl.cfg + Install libcurl library (with dev files); + download the latest released version using git: https://github.com/curl/curl.git + + Open a Visual Studio Command prompt + change to the libcurl root folder + run the "buildconf.bat" batch file + change to the winbuild directory + nmake /f Makefile.vc mode=dll MACHINE=x64 + copy libcurl-vc-x64-release-dll-ipv6-sspi-winssl dir to C:\curl (installation dir) + set ENV variable CURL_ROOT to C:\curl (installation dir) + update PATH ENV variable to %CURL_ROOT%\bin (installation bin dir). + the aws credentials file should be in %USERPROFILE%\.aws folder + set the ENV variable "HDF5_ROS3_TEST_BUCKET_URL=https://s3.us-east-2.amazonaws.com/hdf5ros3" + + (ADB - 2019/09/12, HDFFV-10854) C++ Library: ------------ - - Added new function to the C++ interface - - Added wrapper for H5Ovisit2: - H5Object::visit() + - Added new wrappers for H5Pset/get_create_intermediate_group() + LinkCreatPropList::setCreateIntermediateGroup() + LinkCreatPropList::getCreateIntermediateGroup() - (BMR - 2019/02/14, HDFFV-10532) + (BMR - 2019/04/22, HDFFV-10622) Java Library: ---------------- - - Rewrote the JNI error handling to be much cleaner - - (JTH - 2019/02/12) - - - Add new functions to java interface - - Added wrappers for: - H5Fset_libver_bounds - H5Fget_dset_no_attrs_hint/H5Fset_dset_no_attrs_hint - H5Pget_dset_no_attrs_hint/H5Pset_dset_no_attrs_hint - - (ADB - 2019/01/07, HDFFV-10664) - - - Fix java unit tests when Time is a natural number - - Time substitution in java/test/junit.sh.in doesn't - handle the case when Time is a natural number. Fixed - the regular expression. - - (ADB - 2019/01/07, HDFFV-10674) - - - Duplicate the data read/write functions of Datasets for Attributes. - - Region references could not be displayed for attributes as they could - for datasets. Datasets had overloaded read and write functions for different - datatypes that were not available for attributes. After adding similar - functions, attribute region references work normally. - - (ADB - 2018/12/12, HDFVIEW-4) + - Fixed a failure in JUnit-TestH5P on 32-bit architectures + (JTH - 2019/04/30) - Tools: - ------ - - The h5repart -family-to-sec2 argument was changed to -family-to-single - In order to better support other single-file VFDs which could work with - h5repart, the -family-to-sec2 argument was renamed to -family-to-single. - This is just a name change and the functionality of the argument has not - changed. +Support for new platforms, languages and compilers. +======================================= + - CMake added support for VS2019 in version 3.15. Updated scripts. - The -family-to-sec2 argument has been kept for backwards-compatibility. - This argument should be considered deprecated. + - macOS 10.13.6 Darwin 17.7.0 with Apple clang LLVM version 10.0.0 - (DER - 2018/11/14, HDFFV-10633) + - macOS 10.14.6 Darwin 18.7.0 with Apple clang LLVM version 10.0.1 -Bug Fixes since HDF5-1.10.4 release +Bug Fixes since HDF5-1.10.5 release ================================== Library ------- - - Fix hangs with collective metadata reads during chunked dataset I/O + - Improved performance when creating a large number of small datasets by + retrieving default property values from the API context instead of doing + skip list searches. More work is required to achieve parity with HDF5 1.8. - In the parallel library, it was discovered that when a particular - sequence of operations following a pattern of: + (CJH - 2019/12/10, HDFFV-10658) - "write to chunked dataset" -> "flush file" -> "read from dataset" + - Fixed user-created data access properties not existing in the property list + returned by H5Dget_access_plist. Thanks to Steven Varga for submitting a + reproducer and a patch. - occurred with collective metadata reads enabled, hangs could be - observed due to certain MPI ranks not participating in the collective - metadata reads. + (CJH - 2019/12/9, HDFFV-10934) - To fix the issue, collective metadata reads are now disabled during - chunked dataset raw data I/O. + - Inappropriate linking with deprecated MPI C++ libraries - (JTH - 2019/02/11, HDFFV-10563, HDFFV-10688) + HDF5 does not define *_SKIP_MPICXX in the public headers, so applications + can inadvertently wind up linking to the deprecated MPI C++ wrappers. - - Performance issue when closing an object + MPICH_SKIP_MPICXX and OMPI_SKIP_MPICXX have both been defined in H5public.h + so this should no longer be an issue. HDF5 makes no use of the deprecated + MPI C++ wrappers. - The slow down is due to the search of the "tag_list" to find - out the "corked" status of an object and "uncork" it if so. + (DER - 2019/09/17, HDFFV-10893) - Improve performance by skipping the search of the "tag_list" - if there are no "corked" objects when closing an object. + - fcntl(2)-based file locking incorrectly passed the lock argument struct + instead of a pointer to the struct, causing errors on systems where + flock(2) is not available. - (VC - 2019/02/06) + File locking is used when files are opened to enforce SWMR semantics. A + lock operation takes place on all file opens unless the + HDF5_USE_FILE_LOCKING environment variable is set to the string "FALSE". + flock(2) is preferentially used, with fcntl(2) locks as a backup if + flock(2) is unavailable on a system (if neither is available, the lock + operation fails). On these systems, the file lock will often fail, which + causes HDF5 to not open the file and report an error. - - Uninitialized bytes from a type conversion buffer could be written - to disk in H5Dwrite calls where type conversion takes place - and the type conversion buffer was created by the HDF5 library. + This bug only affects POSIX systems. Win32 builds on Windows use a no-op + locking call which always succeeds. Systems which exhibit this bug will + have H5_HAVE_FCNTL defined but not H5_HAVE_FLOCK in the configure output. - When H5Dwrite is called and datatype conversion must be performed, - the library will create a temporary buffer for type conversion if - one is not provided by the user via H5Pset_buffer. This internal - buffer is allocated via malloc and contains uninitialized data. In - some datatype conversions (float to long double, possibly others), - some of this uninitialized data could be written to disk. + This bug affects HDF5 1.10.0 through 1.10.5. - This was flagged by valgrind in the dtransform test and does not - appear to be a common occurrence (it is flagged in one test out - of the entire HDF5 test suite). + fcntl(2)-based file locking now correctly passes the struct pointer. - Switching to calloc fixed the problem. + (DER - 2019/08/27, HDFFV-10892) - (DER - 2019/02/03, HDFFV-10694) + - Fixed a bug caused by a bad tag value when condensing object header + messages - - There was missing protection against division by zero reported to - The HDF Group as issue #CVE-2018-17434. + There was an assertion failure when moving messages from running a + user test program with library release HDF5 1.10.4. It was because + the tag value (object header's address) was not set up when entering + the library routine H5O__chunk_update_idx(), which eventually + verifies the metadata tag value when protecting the object header. - Protection against division by zero was added to address the issue - #CVE-2018-17434. + The problem was fixed by replacing FUNC_ENTER_PACKAGE in H5O__chunk_update_idx() + with FUNC_ENTER_PACKAGE_TAG(oh->cache_info.addr) to set up the metadata tag. - (BMR - 2019/01/29, HDFFV-10586) + (VC - 2019/08/23, HDFFV-10873) - - The issue CVE-2018-17437 was reported to The HDF Group + - Fixed the test failure from test_metadata_read_retry_info() in + test/swmr.c - Although CVE-2018-17437 reported a memory leak, the actual issue - was invalid read. It was found that the attribute name length - in an attribute message was corrupted, which caused the buffer - pointer to be advanced too far and later caused an invalid read. + The test failure is due to an incorrect number of bins returned for + retry info (info.nbins). The # of bins expected for 101 read attempts + is 3 instead of 2. The routine H5F_set_retries() in src/H5Fint.c + calculates the # of bins by first obtaining the log10 value for + (read attempts - 1). For PGI/19, the log10 value for 100 read attempts + is 1.9999999999999998 instead of 2.00000. When casting the log10 value + to unsigned later on, the decimal part is chopped off causing the test + failure. - A check was added to detect when the attribute name or its length - was corrupted and report the potential of data corruption. + This was fixed by obtaining the rounded integer value (HDceil) for the + log10 value of read attempts first before casting the result to unsigned. - (BMR - 2019/01/29, HDFFV-10588) + (VC - 2019/8/14, HDFFV-10813) - - H5Ewalk did not stop when it was supposed to + - Fixed an issue when creating a file with non-default file space info + together with library high bound setting to H5F_LIBVER_V18. - H5Ewalk was supposed to stop when the callback function stopped - even though the errors in the stack were not all visited, but it - did not. This problem is now fixed. + When setting non-default file space info in fcpl via + H5Pset_file_space_strategy() and then creating a file with both high and + low library bounds set to H5F_LIBVER_V18 in fapl, the library succeeds in + creating the file. File creation should fail because the feature of + setting non-default file space info does not exist in library release 1.8 + or earlier. - (BMR - 2019/01/29, HDFFV-10684) + This was fixed by setting and checking the proper version in the file + space info message based on the library low and high bounds when creating + and opening the HDF5 file. - - Revert H5Oget_info* and H5Ovisit* functions + (VC - 2019/6/25, HDFFV-10808) - In 1.10.3 new H5Oget_info*2 and H5Ovisit*2 functions were - added for performance. Inadvertently, the original functions; - H5Oget_info, - H5Oget_info_by_name, - H5Oget_info_by_idx, - H5Ovisit, - H5Ovisit_by_name - were versioned to H5Oget_info*1 and H5Ovisit*1. This - broke the API compatibility for a maintenance release. The - original functions have been restored. + - Fixed an issue where copying a version 1.8 dataset between files using + H5Ocopy fails due to an incompatible fill version - (ADB - 2019/01/24, HDFFV-10686) + When using the HDF5 1.10.x H5Ocopy() API call to copy a version 1.8 + dataset to a file created with both high and low library bounds set to + H5F_LIBVER_V18, the H5Ocopy() call will fail with the error stack indicating + that the fill value version is out of bounds. - - Fixed a potential invalid memory access and failure that could occur when - decoding an unknown object header message (from a future version of the - library). + This was fixed by changing the fill value message version to H5O_FILL_VERSION_3 + (from H5O_FILL_VERSION_2) for H5F_LIBVER_V18. - (NAF - 2019/01/07) + (VC - 2019/6/14, HDFFV-10800) - - Deleting attributes in dense storage + - Fixed a bug that would cause an error or cause fill values to be + incorrectly read from a chunked dataset using the "single chunk" index if + the data was held in cache and there was no data on disk. - The library aborts with "infinite loop closing library" after - attributes in dense storage are created and then deleted. + (NAF - 2019/03/06) - When deleting the attribute nodes from the name index v2 B-tree, - if an attribute is found in the intermediate B-tree nodes, - which may be merged/redistributed in the process, we need to - free the dynamically allocated spaces for the intermediate - decoded attribute. + - Fixed a bug that could cause an error or cause fill values to be + incorrectly read from a dataset that was written to using H5Dwrite_chunk + if the dataset was not closed after writing. - (VC - 2018/12/26, HDFFV-10659) + (NAF - 2019/03/06, HDFFV-10716) - - There was missing protection against division by zero reported to - The HDF Group as issue #CVE-2018-17233. + - Fixed memory leak in scale offset filter - Protection against division by zero was added to address the issue - #CVE-2018-17233. In addition, several similar occurrences in the same - file were fixed as well. + In a special case where the MinBits is the same as the number of bits in + the datatype's precision, the filter's data buffer was not freed, causing + the memory usage to grow. In general the buffer was freed correctly. The + Minbits are the minimal number of bits to store the data values. Please + see the reference manual for H5Pset_scaleoffset for the details. - (BMR - 2018/12/23, HDFFV-10577) + (RL - 2019/3/4, HDFFV-10705) - - Fixed an issue where the parallel filters tests would fail - if zlib was not available on the system. Until support can - be added in the tests for filters beyond gzip/zlib, the tests - will be skipped if zlib is not available. - (JTH - 2018/12/05) - - - A bug was discovered in the parallel library where an application - would eventually consume all of the available MPI communicators - when continually writing to a compressed dataset in parallel. This - was due to internal copies of an HDF5 File Access Property List, - which each contained a copy of the MPI communicator, not being - closed at the end of each write operation. This problem was - exacerbated by larger numbers of processors. - - (JTH - 2018/12/05, HDFFV-10629) - - - Fortran - -------- - - Fixed issue with Fortran not returning h5o_info_t field values - meta_size%attr%index_size and meta_size%attr%heap_size. - - (MSB, 2019/01/08, HDFFV-10443) - - - Added symbolic links libhdf5_hl_fortran.so to libhdf5hl_fortran.so and - libhdf5_hl_fortran.a to libhdf5hl_fortran.a in hdf5/lib directory for - autotools installs. These were added to match the name of the files - installed by cmake and the general pattern of hl lib files. We will - change the names of the installed lib files to the matching name in - the next major release. - - (LRK - 2019/01/04, HDFFV-10596) - - - Made Fortran specific subroutines PRIVATE in generic procedures. + Configuration + ------------- + - Correct option for default API version - Affected generic procedures were functions in H5A, H5D, H5P, H5R and H5T. + CMake options for default API version are not mutually exclusive. + Change the multiple BOOL options to a single STRING option with the + strings; v16, v18, v110. - (MSB, 2018/12/04, HDFFV-10511) + (ADB - 2019/08/12, HDFFV-10879) + Tools + ----- + - h5repack was fixed to repack datasets with external storage + to other types of storage. - Testing - ------- - - Fixed a test failure in testpar/t_dset.c caused by - the test trying to use the parallel filters feature - on MPI-2 implementations. + New test added to repack files and verify the correct data using h5diff. - (JTH, 2019/2/7) + (JS - 2019/09/25, HDFFV-10408) + (ADB - 2019/10/02, HDFFV-10918) Supported Platforms =================== - Linux 2.6.32-696.16.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) - #1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) - (ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) + Linux 2.6.32-696.20.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23) + #1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23) + (ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23) IBM XL C/C++ V13.1 IBM XL Fortran V15.1 Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++) #1 SMP x86_64 GNU/Linux compilers: - (kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4) + (jelly/kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4) Version 4.9.3, Version 5.2.0 Intel(R) C (icc), C++ (icpc), Fortran (icc) compilers: @@ -506,28 +369,24 @@ Supported Platforms (emu) Sun Fortran 95 8.6 SunOS_sparc Sun C++ 5.12 SunOS_sparc - Windows 7 Visual Studio 2015 w/ Intel Fortran 16 (cmake) + Windows 7 Visual Studio 2015 w/ Intel Fortran 18 (cmake) - Windows 7 x64 Visual Studio 2013 - Visual Studio 2015 w/ Intel Fortran 16 (cmake) - Visual Studio 2015 w/ Intel C, Fortran 2018 (cmake) + Windows 7 x64 Visual Studio 2015 w/ Intel C, Fortran 2018 (cmake) Visual Studio 2015 w/ MSMPI 8 (cmake) Windows 10 Visual Studio 2015 w/ Intel Fortran 18 (cmake) Windows 10 x64 Visual Studio 2015 w/ Intel Fortran 18 (cmake) - Visual Studio 2017 w/ Intel Fortran 18 (cmake) - - Mac OS X Yosemite 10.10.5 Apple clang/clang++ version 6.1 from Xcode 7.0 - 64-bit gfortran GNU Fortran (GCC) 4.9.2 - (osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3 + Visual Studio 2017 w/ Intel Fortran 19 (cmake) + Visual Studio 2019 w/ Intel Fortran 19 (cmake) - Mac OS X El Capitan 10.11.6 Apple clang/clang++ version 7.3.0 from Xcode 7.3 - 64-bit gfortran GNU Fortran (GCC) 5.2.0 - (osx1011dev/osx1011test) Intel icc/icpc/ifort version 16.0.2 + macOS 10.13.6, Darwin, Apple clang LLVM version 10.0.0 + 17.7.0, x86_64 gfortran GNU Fortran (GCC) 6.3.0 + (bear) Intel icc/icpc/ifort version 19.0.4 - MacOS High Sierra 10.13.6 Apple LLVM version 10.0.0 (clang/clang++-1000.10.44.4) - 64-bit gfortran GNU Fortran (GCC) 8.3.0 + macOS 10.14.6, Darwin Apple clang LLVM version 10.0.1 + 18.7.0, x86_64 gfortran GNU Fortran (GCC) 6.3.0 + (bobcat) Intel icc/icpc/ifort version 19.0.4 Tested Configuration Features Summary @@ -588,7 +447,7 @@ More Tested Platforms ===================== The following configurations are not supported but have been tested for this release. - Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++) + Linux 2.6.32-754.11.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++) #1 SMP x86_64 GNU/Linux compilers: (mayll/platypus) Version 4.4.7 20120313 Version 4.9.3, 5.3.0, 6.2.0 @@ -602,31 +461,45 @@ The following configurations are not supported but have been tested for this rel Linux 3.10.0-327.18.2.el7 GNU C (gcc) and C++ (g++) compilers #1 SMP x86_64 GNU/Linux Version 4.8.5 20150623 (Red Hat 4.8.5-4) - (jelly) with NAG Fortran Compiler Release 6.1(Tozai) + (jelly) with NAG Fortran Compiler Release 6.2(Chiyoda) GCC Version 7.1.0 MPICH 3.2-GCC-4.9.3 MPICH 3.2.1-GCC-7.2.0-2.29 OpenMPI 2.1.5-GCC-7.2.0-2.29 Intel(R) C (icc) and C++ (icpc) compilers Version 17.0.0.098 Build 20160721 - with NAG Fortran Compiler Release 6.1(Tozai) + with NAG Fortran Compiler Release 6.2(Chiyoda) Linux 3.10.0-327.10.1.el7 MPICH 3.2 compiled with GCC 5.3.0 #1 SMP x86_64 GNU/Linux (moohan) - Fedora 29 4.20.10-200.fc29.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux - gcc, g++ (GCC) 8.2.1 20181215 - (Red Hat 8.2.1-6) - GNU Fortran (GCC) 8.2.1 20181215 - (Red Hat 8.2.1-6) - (cmake and autotools) + Linux 2.6.32-573.18.1.el6.ppc64 MPICH mpich 3.1.4 compiled with + #1 SMP ppc64 GNU/Linux IBM XL C/C++ for Linux, V13.1 + (ostrich) and IBM XL Fortran for Linux, V15.1 + + Fedora30 5.3.11-200.fc30.x86_64 + #1 SMP x86_64 GNU/Linux GNU gcc (GCC) 9.2.1 20190827 (Red Hat 9.2.1 20190827) + GNU Fortran (GCC) 9.2.1 20190827 (Red Hat 9.2.1 20190827) + (cmake and autotools) + + Mac OS X 10.11.6, Darwin, Apple clang version 7.3.0 from Xcode 7.3 + 15.6.0, x86-64 gfortran GNU Fortran (GCC) 5.2.0 + (osx1011test) Intel icc/icpc/ifort version 16.0.2 + + macOS 10.12.6, Darwin, Apple clang LLVM version 8.1.0 from Xcode 8.3 + 16.6.0, x86_64 gfortran GNU Fortran (GCC) 7.1.0 + (kite) Intel icc/icpc/ifort version 17.0.2 Windows 7 x64 Visual Studio 2008 Known Problems ============== + CMake files do not behave correctly with paths containing spaces. + Do not use spaces in paths because the required escaping for handling spaces + results in very complex and fragile build files. + ADB - 2019/05/07 At present, metadata cache images may not be generated by parallel applications. Parallel applications can read files with metadata cache @@ -639,6 +512,12 @@ Known Problems t_pflush1/fails on exit The first two tests fail attempting collective writes. + Parallel builds using OpenMPI 3.03 or later and romio fail several tests + with collective writes or compression that will not fail when ompio is used + instead of romio. This can be done by adding "--mca io ompio" to the mpirun + command. For example, in autotools builds RUNPARALLEL can be set to + "mpirun --mca io ompio -n 6" provided ompio is installed. + CPP ptable test fails on VS2017 with Intel compiler, JIRA issue: HDFFV-10628. This test will pass with VS2015 with Intel compiler. |