diff options
| author | Dana Robinson <43805+derobins@users.noreply.github.com> | 2023-03-20 15:57:39 (GMT) |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2023-03-20 15:57:39 (GMT) |
| commit | 3ce739cb63344b808f75a200e5e3cbf076504632 (patch) | |
| tree | 1ffa2c7bd2969dcd6fb62401bb4fb3d8a17015ed /release_docs/HISTORY-1_10.txt | |
| parent | be54deeacce621ab974a98c8277496c54393c35c (diff) | |
| download | hdf5-3ce739cb63344b808f75a200e5e3cbf076504632.zip hdf5-3ce739cb63344b808f75a200e5e3cbf076504632.tar.gz hdf5-3ce739cb63344b808f75a200e5e3cbf076504632.tar.bz2 | |
Bring release_docs scheme from develop (#2612)
Diffstat (limited to 'release_docs/HISTORY-1_10.txt')
| -rw-r--r-- | release_docs/HISTORY-1_10.txt | 6562 |
1 files changed, 0 insertions, 6562 deletions
diff --git a/release_docs/HISTORY-1_10.txt b/release_docs/HISTORY-1_10.txt deleted file mode 100644 index 1fc3c60..0000000 --- a/release_docs/HISTORY-1_10.txt +++ /dev/null @@ -1,6562 +0,0 @@ -HDF5 History -============ - -This file contains development history of the HDF5 1.10 branch - -09. Release Information for hdf5-1.10.7 -08. Release Information for hdf5-1.10.6 -07. Release Information for hdf5-1.10.5 -06. Release Information for hdf5-1.10.4 -05. Release Information for hdf5-1.10.3 -04. Release Information for hdf5-1.10.2 -03. Release Information for hdf5-1.10.1 -02. Release Information for hdf5-1.10.0-patch1 -01. Release Information for hdf5-1.10.0 - -[Search on the string '%%%%' for section breaks of each release.] - -%%%%1.10.7%%%% - -HDF5 version 1.10.7 released on 2020-09-11 -================================================================================ - - -INTRODUCTION - -This document describes the differences between this release and the previous -HDF5 release. It contains information on the platforms tested and known -problems in this release. For more details check the HISTORY*.txt files in the -HDF5 source. - -Note that documentation in the links below will be updated at the time of each -final release. - -Links to HDF5 documentation can be found on The HDF5 web page: - - https://portal.hdfgroup.org/display/HDF5/HDF5 - -The official HDF5 releases can be obtained from: - - https://www.hdfgroup.org/downloads/hdf5/ - - HDF5 binaries provided are fully tested with ZLIB and the free - Open Source SZIP successor Libaec (with BSD license). - The official ZLIB and SZIP/Libaec pages are at: - - ZLIB: http://www.zlib.net/ - http://www.zlib.net/zlib_license.html - SZIP/Libaec: https://gitlab.dkrz.de/k202009/libaec - https://gitlab.dkrz.de/k202009/libaec/-/blob/master/Copyright.txt - -Changes from Release to Release and New Features in the HDF5-1.10.x release series -can be found at: - - https://portal.hdfgroup.org/display/HDF5/HDF5+Application+Developer%27s+Guide - -If you have any questions or comments, please send them to the HDF Help Desk: - - help@hdfgroup.org - - -CONTENTS - -- New Features -- Support for new platforms and languages -- Bug Fixes since HDF5-1.10.6 -- Supported Platforms -- Tested Configuration Features Summary -- More Tested Platforms -- Known Problems -- CMake vs. Autotools installations - - -New Features -============ - - Configuration: - ------------- - - Disable memory sanity checks in the Autotools in release branches - - The library can be configured to use internal memory sanity checking, - which replaces C API calls like malloc(3) and free(3) with our own calls - which add things like heap canaries. These canaries can cause problems - when external filter plugins reallocate canary-marked buffers. - - For this reason, the default will be to not use the memory allocation - sanity check feature in release branches (e.g., hdf5_1_10_7). - Debug builds in development branches (e.g., develop, hdf5_1_10) will - still use them by default. - - This change only affects Autotools debug builds. Non-debug autotools - builds and all CMake builds do not enable this feature by default. - - (DER - 2020/08/19) - - - Add file locking configure and CMake options - - HDF5 1.10.0 introduced a file locking scheme, primarily to help - enforce SWMR setup. Formerly, the only user-level control of the scheme - was via the HDF5_USE_FILE_LOCKING environment variable. - - This change introduces configure-time options that control whether - or not file locking will be used and whether or not the library - ignores errors when locking has been disabled on the file system - (useful on some HPC Lustre installations). - - In both the Autotools and CMake, the settings have the effect of changing - the default property list settings (see the H5Pset/get_file_locking() - entry, below). - - The yes/no/best-effort file locking configure setting has also been - added to the libhdf5.settings file. - - Autotools: - - An --enable-file-locking=(yes|no|best-effort) option has been added. - - yes: Use file locking. - no: Do not use file locking. - best-effort: Use file locking and ignore "disabled" errors. - - CMake: - - Two self-explanatory options have been added: - - HDF5_USE_FILE_LOCKING - HDF5_IGNORE_DISABLED_FILE_LOCKS - - Setting both of these to ON is the equivalent to the Autotools' - best-effort setting. - - NOTE: - The precedence order of the various file locking control mechanisms is: - - 1) HDF5_USE_FILE_LOCKING environment variable (highest) - - 2) H5Pset_file_locking() - - 3) configure/CMake options (which set the property list defaults) - - 4) library defaults (currently best-effort) - - (DER - 2020/07/30, HDFFV-11092) - - - CMake option to link the generated Fortran MOD files into the include - directory. - - The Fortran generation of MOD files by a Fortran compile can produce - different binary files between SHARED and STATIC compiles with different - compilers and/or different platforms. Note that it has been found that - different versions of Fortran compilers will produce incompatible MOD - files. Currently, CMake will locate these MOD files in subfolders of - the include directory and add that path to the Fortran library target - in the CMake config file, which can be used by the CMake find library - process. For other build systems using the binary from a CMake install, - a new CMake configuration can be used to copy the pre-chosen version - of the Fortran MOD files into the install include directory. - - The default will depend on the configuration of - BUILD_STATIC_LIBS and BUILD_SHARED_LIBS: - YES YES Default to SHARED - YES NO Default to STATIC - NO YES Default to SHARED - NO NO Default to SHARED - The defaults can be overridden by setting the config option - HDF5_INSTALL_MOD_FORTRAN to one of NO, SHARED, or STATIC - - (ADB - 2020/07/09, HDFFV-11116) - - - CMake option to use AEC (open source SZip) library instead of SZip - - The open source AEC library is a replacement library for SZip. In - order to use it for hdf5, the libaec CMake source was changed to add - "-fPIC" and exclude test files. A new option USE_LIBAEC is required - to compensate for the different files produced by AEC build. - - Autotools does not build the compression libraries within hdf5 builds, - but will use an installed libaec when configured as before with the - option --with-libsz=<path to libaec directory>. - - (ADB - 2020/04/22, OESS-65) - - - CMake ConfigureChecks.cmake file now uses CHECK_STRUCT_HAS_MEMBER - - Some handcrafted tests in HDFTests.c have been removed and the CMake - CHECK_STRUCT_HAS_MEMBER module has been used. - - (ADB - 2020/03/24, TRILAB-24) - - - Both build systems use same set of warnings flags - - GNU C, C++ and gfortran warnings flags were moved to files in a config - sub-folder named gnu-warnings. Flags that only are available for a specific - version of the compiler are in files named with that version. - Clang C warnings flags were moved to files in a config sub-folder - named clang-warnings. - Intel C, Fortran warnings flags were moved to files in a config sub-folder - named intel-warnings. - - There are flags in named "error-xxx" files with warnings that may - be promoted to errors. Some source files may still need fixes. - - There are also pairs of files named "developer-xxx" and "no-developer-xxx" - that are chosen by the CMake option:HDF5_ENABLE_DEV_WARNINGS or the - configure option:--enable-developer-warnings. - - In addition, CMake no longer applies these warnings for examples. - - (ADB - 2020/03/24, TRILAB-192) - - - Update CMake minimum version to 3.12 - - Updated CMake minimum version to 3.12 and added version checks - for Windows features. - - (ADB - 2020/02/05, TRILABS-142) - - - Fixed CMake include properties for Fortran libraries - - Corrected the library properties for Fortran to use the - correct path for the Fortran module files. - - (ADB - 2020/02/04, HDFFV-11012) - - - Added common warnings files for gnu and intel - - Added warnings files to use one common set of flags - during configure for both autotools and CMake build - systems. The initial implementation only affects a - general set of flags for gnu and intel compilers. - - (ADB - 2020/01/17) - - - Added new options to CMake for control of testing - - Added CMake options (default ON); - HDF5_TEST_SERIAL AND/OR HDF5_TEST_PARALLEL - combined with: - HDF5_TEST_TOOLS - HDF5_TEST_EXAMPLES - HDF5_TEST_SWMR - HDF5_TEST_FORTRAN - HDF5_TEST_CPP - HDF5_TEST_JAVA - - (ADB - 2020/01/15, HDFFV-11001) - - - Added Clang sanitizers to CMake for analyzer support if compiler is clang. - - Added CMake code and files to execute the Clang sanitizers if - HDF5_ENABLE_SANITIZERS is enabled and the USE_SANITIZER option - is set to one of the following: - Address - Memory - MemoryWithOrigins - Undefined - Thread - Leak - 'Address;Undefined' - - (ADB - 2019/12/12, TRILAB-135) - - - Library: - -------- - - Add metadata cache optimization to reduce skip list usage - - On file flush or close, the metadata cache attempts to write out - all dirty entries in increasing address order. To do this, it needs - an address sorted list of metadata entries. Further, since flushing - one metadata cache entry can dirty another, this list must support - efficient insertion and deletion. - - The metadata cache uses a skip list of all dirty entries for this - purpose. Before this release, this skip list was maintained at all - times. However, since profiling indicates that this imposes a - significant cost, we now construct and maintain the skip list only - when needed. Specifically, we enable the skip list and load it with - a list of all dirty entries in the metadata cache just before a flush, - and disable it after the flush. - - (JRM - 2020/08/17, HDFFV-11034) - - - Add BEST_EFFORT value to HDF5_USE_FILE_LOCKING environment variable - - This change adds a BEST_EFFORT to the TRUE/FALSE, 1/0 settings that - were previously accepted. This option turns on file locking but - ignores locking errors when the library detects that file locking - has been disabled on a file system (useful on some HPC Lustre - installations). - - The capitalization of BEST_EFFORT is mandatory. - - See the configure option discussion for HDFFV-11092 (above) for more - information on the file locking feature and how it's controlled. - - (DER - 2020/07/30, HDFFV-11092) - - - - Add H5Pset/get_file_locking() API calls - - This change adds new API calls which can be used to set or get the - file locking parameters. The single API call sets both the "use file - locking" flag and the "ignore disabled file locking" flag. - - When opening a file multiple times without closing, the file MUST be - opened with the same file locking settings. Opening a file with different - file locking settings will fail (this is similar to the behavior of - H5Pset_fclose_degree()). - - See the configure option discussion for HDFFV-11092 (above) for more - information on the file locking feature and how it's controlled. - - (DER - 2020/07/30, HDFFV-11092) - - - Add Mirror VFD - - Use TCP/IP sockets to perform write-only (W/O) file I/O on a remote - machine. Must be used in conjunction with the Splitter VFD. - - (JOS - 2020/03/13, TBD) - - - Add Splitter VFD - - Maintain separate R/W and W/O channels for "concurrent" file writes - to two files using a single HDF5 file handle. - - (JOS - 2020/03/13, TBD) - - - Fixed an assertion failure in the parallel library when collectively - filling chunks. As it is required that chunks be written in - monotonically non-decreasing order of offset in the file, this assertion - was being triggered when the list of chunk file space allocations being - passed to the collective chunk filling routine was not sorted according - to this particular requirement. - - The addition of a sort of the out of order chunks trades a bit of - performance for the elimination of this assertion and of any complaints - from MPI implementations about the file offsets used being out of order. - - (JTH - 2019/10/07) - - Fortran Library: - ---------------- - - Add wrappers for H5Pset/get_file_locking() API calls - - h5pget_file_locking_f() - h5pset_file_locking_f() - - See the configure option discussion for HDFFV-11092 (above) for more - information on the file locking feature and how it's controlled. - - (DER - 2020/07/30, HDFFV-11092) - - - Added new Fortran parameters: - - H5F_LIBVER_ERROR_F - H5F_LIBVER_NBOUNDS_F - H5F_LIBVER_V18_F - H5F_LIBVER_V110_F - - - Added new Fortran API: h5pget_libver_bounds_f - - (MSB - 2020/02/11, HDFFV-11018) - - C++ Library: - ------------ - - Add wrappers for H5Pset/get_file_locking() API calls - - FileAccPropList::setFileLocking() - FileAccPropList::getFileLocking() - - See the configure option discussion for HDFFV-11092 (above) for more - information on the file locking feature and how it's controlled. - - (DER - 2020/07/30, HDFFV-11092) - - Java Library: - ---------------- - - Add wrappers for H5Pset/get_file_locking() API calls - - H5Pset_file_locking() - H5Pget_use_file_locking() - H5Pget_ignore_disabled_file_locking() - - Unlike the C++ and Fortran wrappers, there are separate getters for the - two file locking settings, each of which returns a boolean value. - - See the configure option discussion for HDFFV-11092 (above) for more - information on the file locking feature and how it's controlled. - - (DER - 2020/07/30, HDFFV-11092) - - Tools: - ------ - - h5repack added options to control how external links are handled. - - Currently h5repack preserves external links and cannot copy and merge - data from the external files. Two options, merge and prune, were added to - control how to merge data from an external link into the resulting file. - --merge Follow external soft link recursively and merge data. - --prune Do not follow external soft links and remove link. - --merge --prune Follow external link, merge data and remove dangling link. - - (ADB - 2020/08/05, HDFFV-9984) - - High-Level APIs: - --------------- - - None - - C Packet Table API - ------------------ - - None - - Internal header file - -------------------- - - None - - Documentation - ------------- - - None - - -Support for new platforms, languages and compilers. -======================================= - - None - - -Bug Fixes since HDF5-1.10.6 release -================================== - - Library - ------- - - Fix bug and simplify collective metadata write operation when some ranks - have no entries to contribute. This fixes parallel regression test - failures with IBM SpectrumScale MPI on the Summit system at ORNL. - - (QAK - 2020/09/02) - - - Avoid setting up complex MPI types with 0-length vectors, which some - MPI implementations don't handle well. (In particular, IBM - SpectrumScale MPI on the Summit system at ORNL) - - (QAK - 2020/08/21) - - - Fixed use-of-uninitialized-value error - - Appropriate initialization of local structs was added to remove the - use-of-uninitialized-value errors reported by MemorySanitizer. - - (BMR - 2020/8/13, HDFFV-11101) - - - Creation of dataset with optional filter - - When the combination of type, space, etc doesn't work for filter - and the filter is optional, it was supposed to be skipped but it was - not skipped and the creation failed. - - A fix is applied to allow the creation of a dataset in such - situation, as specified in the user documentation. - - (BMR - 2020/8/13, HDFFV-10933) - - - Explicitly declared dlopen to use RTLD_LOCAL - - dlopen documentation states that if neither RTLD_GLOBAL nor - RTLD_LOCAL are specified, then the default behavior is unspecified. - The default on linux is usually RTLD_LOCAL while macos will default - to RTLD_GLOBAL. - - (ADB - 2020/08/12, HDFFV-11127) - - - Fixed issues CVE-2018-13870 and CVE-2018-13869 - - When a buffer overflow occurred because a name length was corrupted - and became very large, h5dump crashed on memory access violation. - - A check for reading past the end of the buffer was added to multiple - locations to prevent the crashes and h5dump now simply fails with an - error message when this error condition occurs. - - (BMR - 2020/7/31, HDFFV-11120 and HDFFV-11121) - - - H5Sset_extent_none() sets the dataspace class to H5S_NO_CLASS which - causes asserts/errors when passed to other dataspace API calls. - - H5S_NO_CLASS is an internal class value that should not have been - exposed via a public API call. - - In debug builds of the library, this can cause asserts to trip. In - non-debug builds, it will produce normal library errors. - - The new library behavior is for H5Sset_extent_none() to convert - the dataspace into one of type H5S_NULL, which is better handled - by the library and easier for developers to reason about. - - (DER - 2020/07/27, HDFFV-11027) - - - Fixed the segmentation fault when reading attributes with multiple threads - - It was reported that the reading of attributes with variable length string - datatype will crash with segmentation fault particularly when the number - of threads is high (>16 threads). The problem was due to the file pointer - that was set in the variable length string datatype for the attribute. - That file pointer was already closed when the attribute was accessed. - - The problem was fixed by setting the file pointer to the current opened - file pointer when the attribute was accessed. Similar patch up was done - before when reading dataset with variable length string datatype. - - (VC - 2020/07/13, HDFFV-11080) - - - Fixed issue CVE-2018-17438 - - A division by zero was discovered in H5D__select_io() of H5Dselect.c. - https://security-tracker.debian.org/tracker/CVE-2018-17438 - - A check was added to protect against division by zero. When such - situation occurs again, the normal HDF5 error handling will be invoked, - instead of segmentation fault. - - (BMR, DER - 2020/07/09, HDFFV-10587) - - - Fixed CVE-2018-17435 - - The tool h52gif produced a segfault when the size of an attribute message - was corrupted and caused a buffer overflow. - - The problem was fixed by verifying the attribute message's size against the - buffer size before accessing the buffer. h52gif was also fixed to display - the failure instead of silently exiting after the segfault was eliminated. - - (BMR - 2020/6/19, HDFFV-10591) - - - Don't allocate an empty (0-dimensioned) chunked dataset's chunk - index, until the dataset's dimensions are increased. - - (QAK - 2020/05/07) - - Configuration - ------------- - - Stopped addition of szip header and include directory path for - incompatible libsz - - szlib.h is the same for both 32-bit and 64-bit szip, and the header file - and its path were added to the HDF5 binary even though the configure - check of a function in libsz later failed and szip compression was not - enabled. The header file and include path are now added only when the - libsz function passes the configure check. - - (LRK - 2020/08/17, HDFFV-10830) - - - Added -fsanitize=address autotools configure option for Clang compiler - - Clang sanitizer options were also added for Clang compilers with CMake. - - (LRK, 2020/08/05, HDFFV-10836) - - - Updated testh5cc.sh.in for functions versioned in HDF5 1.10. - - testh5cc.sh previously tested that the correct version of a function - versioned in HDF5 1.6 or 1.8 was compiled when one of - H5_NO_DEPRECATED_SYMBOLS or H5_USE_16_API_DEFAULT were defined. This - test was extended for additional testing with H5_USE_18_API_DEFAULT. - - (LRK, 2020/06/22, HDFFV-11000) - - - Fixed CMake include properties for Fortran libraries - - Corrected the library properties for Fortran to use the - correct path for the Fortran module files. - - (ADB - 2020/02/04, HDFFV-11012) - - Performance - ------------- - - None - - Java Library: - ---------------- - - None - - Fortran - -------- - - Corrected INTERFACE INTENT(IN) to INTENT(OUT) for buf_size in h5fget_file_image_f. - - (MSB - 2020/2/18, HDFFV-11029) - - - Fixed configure issue when building HDF5 with NAG Fortran 7.0. - - HDF5 now accounts for the addition of half-precision floating-point - in NAG 7.0 with a KIND=16. - - (MSB - 2020/02/28, HDFFV-11033) - - Tools - ----- - - The tools library was updated by standardizing the error stack process. - - General sequence is: - h5tools_setprogname(PROGRAMNAME); - h5tools_setstatus(EXIT_SUCCESS); - h5tools_init(); - ... process the command-line (check for error-stack enable) ... - h5tools_error_report(); - ... (do work) ... - h5diff_exit(ret); - - (ADB - 2020/07/20, HDFFV-11066) - - - h5diff fixed a command line parsing error. - - h5diff would ignore the argument to -d (delta) if it is smaller than DBL_EPSILON. - The macro H5_DBL_ABS_EQUAL was removed and a direct value comparison was used. - - (ADB - 2020/07/20, HDFFV-10897) - - - h5diff added a command line option to ignore attributes. - - h5diff would ignore all objects with a supplied path if the exclude-path argument is used. - Adding the exclude-attribute argument will only eclude attributes, with the supplied path, - from comparison. - - (ADB - 2020/07/20, HDFFV-5935) - - - h5diff added another level to the verbose argument to print filenames. - - Added verbose level 3 that is level 2 plus the filenames. The levels are: - 0 : Identical to '-v' or '--verbose' - 1 : All level 0 information plus one-line attribute status summary - 2 : All level 1 information plus extended attribute status report - 3 : All level 2 information plus file names - - (ADB - 2020/07/20, HDFFV-10005) - - - h5repack was fixed to repack the reference attributes properly. - The code line that checks if the update of reference inside a compound - datatype is misplaced outside the code block loop that carries out the - check. In consequence, the next attribute that is not the reference - type was repacked again as the reference type and caused the failure of - repacking. The fix is to move the corresponding code line to the correct - code block. - - (KY -2020/02/10, HDFFV-11014) - - High-Level APIs: - ------ - - The H5DSis_scale function was updated to return "not a dimension scale" (0) - instead of failing (-1), when CLASS or DIMENSION_SCALE attributes are - not written according to Dimension Scales Specification. - - (EIP - 2020/08/12, HDFFV-10436) - - Fortran High-Level APIs: - ------ - - None - - Documentation - ------------- - - None - - F90 APIs - -------- - - None - - C++ APIs - -------- - - None - - Testing - ------- - - Stopped java/test/junit.sh.in installing libs for testing under ${prefix} - - Lib files needed are now copied to a subdirectory in the java/test - directory, and on Macs the loader path for libhdf5.xxxs.so is changed - in the temporary copy of libhdf5_java.dylib. - - (LRK, 2020/7/2, HDFFV-11063) - - -Supported Platforms -=================== - - Linux 3.10.0-1127.10.1.el7 gcc (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39) - #1 SMP ppc64 GNU/Linux g++ (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39) - (echidna) GNU Fortran (GCC) 4.8.5 20150623 (Red Hat 4.8.5-39) - - Linux 2.6.32-754.31.1.el6 IBM XL C/C++ V13.1 - #1 SMP ppc64 GNU/Linux IBM XL Fortran V15.1 - (ostrich) - - Linux 3.10.0-327.18.2.el7 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (jelly/kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4) - Version 4.9.3, Version 5.3.0, Version 6.3.0, - Version 7.2.0, Version 8.3.0, Version 9.1.0 - Intel(R) C (icc), C++ (icpc), Fortran (icc) - compilers: - Version 17.0.0.098 Build 20160721 - MPICH 3.3 compiled with GCC 7.2.0 - OpenMPI 4.0.0 compiled with GCC 7.2.0 - - SunOS 5.11 11.4.5.12.5.0 Sun C 5.15 SunOS_sparc 2017/05/30 - 32- and 64-bit Studio 12.6 Fortran 95 8.8 SunOS_sparc 2017/05/30 - (hedgehog) Sun C++ 5.15 SunOS_sparc 2017/05/30 - - Windows 7 x64 Visual Studio 2015 w/ Intel C, Fortran 2018 (cmake) - - Windows 10 x64 Visual Studio 2015 w/ Intel Fortran 18 (cmake) - Visual Studio 2017 w/ Intel Fortran 19 (cmake) - Visual Studio 2019 w/ Intel Fortran 19 (cmake) - Visual Studio 2019 w/ MSMPI 10.1 (cmake) - - macOS Mojave 10.14.6 Apple LLVM version 10.0.1 (clang-1001.0.46.4) - 64-bit gfortran GNU Fortran (GCC) 6.3.0 - (swallow) Intel icc/icpc/ifort version 19.0.4.233 20190416 - -Tested Configuration Features Summary -===================================== - - In the tables below - y = tested - n = not tested in this release - C = Cluster - W = Workstation - x = not working in this release - dna = does not apply - ( ) = footnote appears below second table - <blank> = testing incomplete on this feature or platform - -Platform C F90/ F90 C++ zlib SZIP - parallel F2003 parallel -Solaris2.11 32-bit n y/y n y y y -Solaris2.11 64-bit n y/n n y y y -Windows 7 x64 y y/y y y y y -Windows 10 y y/y n y y y -Windows 10 x64 y y/y n y y y -MacOS Sierra 10.12.6 64-bit n y/y n y y y -MacOS High Sierra 10.13.6 64-bit n y/y n y y y -MacOS Mojave 10.14.6 64-bit n y/y n y y y -CentOS 7.2 Linux 3.10.0 x86_64 PGI n y/y n y y y -CentOS 7.2 Linux 3.10.0 x86_64 GNU y y/y y y y y -CentOS 7.2 Linux 3.10.0 x86_64 Intel n y/y n y y y -Linux 2.6.32-754.31.1.el6.ppc64 XL n y/y n y y y - - -Platform Shared Shared Shared Thread- - C libs F90 libs C++ libs safe -Solaris2.11 32-bit y y y y -Solaris2.11 64-bit y y y y -Windows 7 x64 y y y y -Windows 10 y y y y -Windows 10 x64 y y y y -MacOS Sierra 10.12.6 64-bit y n y y -MacOS High Sierra 10.13.6 64-bit y n y y -MacOS Mojave 10.14.6 64-bit y n y y -CentOS 7.2 Linux 3.10.0 x86_64 PGI y y y n -CentOS 7.2 Linux 3.10.0 x86_64 GNU y y y y -CentOS 7.2 Linux 3.10.0 x86_64 Intel y y y n -Linux 2.6.32-754.31.1.el6.ppc64 XL y y y n - -Compiler versions for each platform are listed in the preceding -"Supported Platforms" table. - - -More Tested Platforms -===================== -The following platforms are not supported but have been tested for this release. - - Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (platypus) Version 4.4.7 20120313 - Version 4.9.3, 5.3.0, 6.2.0 - PGI C, Fortran, C++ for 64-bit target on - x86-64; - Version 19.10-0 - MPICH 3.1.4 compiled with GCC 4.9.3 - - Linux 2.6.32-754.31.1.el6 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) - #1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) - (ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) - - Linux 3.10.0-327.18.2.el7 GNU C (gcc) and C++ (g++) compilers - #1 SMP x86_64 GNU/Linux Version 4.8.5 20150623 (Red Hat 4.8.5-4) - (jelly) with NAG Fortran Compiler Release 6.1(Tozai) - GCC Version 7.1.0 - OpenMPI 2.1.6-GCC-7.2.0-2.29, - 3.1.3-GCC-7.2.0-2.29 - Intel(R) C (icc) and C++ (icpc) compilers - Version 17.0.0.098 Build 20160721 - with NAG Fortran Compiler Release 6.1(Tozai) - - Linux 3.10.0-327.10.1.el7 MPICH 3.1.4 compiled with GCC 4.9.3 - #1 SMP x86_64 GNU/Linux - (moohan) - - Linux-3.10.0-1127.0.0.1chaos openmpi-4.0.0 - #1 SMP x86_64 GNU/Linux clang/3.9.0, 8.0.1 - (quartz) gcc/7.3.0, 8.1.0 - intel/16.0.4 - - Linux-4.14.0-115.10.1.1 spectrum-mpi/rolling-release - #1 SMP ppc64le GNU/Linux clang/coral-2018.08.08 - (lassen) gcc/7.3.1 - xl/2019.02.07 - - Linux-4.12.14-150.52-default cray-mpich/7.7.10 - #1 SMP x86_64 GNU/Linux gcc/7.3.0, 8.2.0 - (cori) intel/19.0.3 - - Linux-4.4.180-94.107-default cray-mpich/7.7.6 - # 1SMP x86_64 GNU/Linux gcc/7.2.0, 8.2.0 - (mutrino) intel/17.0.4, 18.0.2, 19.0.4 - - Fedora 32 5.7.15-200.fc32.x86_64 Clang version 10.0.0 (Fedora 10.0.0-2.fc32) - #1 SMP x86_64 GNU/Linux GNU gcc (GCC) 10.2.1 20200723 (Red Hat 10.2.1-1) - GNU Fortran (GCC) 10.2.1 20200723 (Red Hat 10.2.1) - (cmake and autotools) - - Mac OS X El Capitan 10.11.6 Apple clang version 7.3.0 from Xcode 7.3 - 64-bit gfortran GNU Fortran (GCC) 5.2.0 - (osx1011test) Intel icc/icpc/ifort version 16.0.2 - - macOS Sierra 10.12.6 Apple LLVM version 9.0.0 (clang-900.39.2) - 64-bit gfortran GNU Fortran (GCC) 7.4.0 - (kite) Intel icc/icpc/ifort version 17.0.2 - - macOS High Sierra 10.13.6 Apple LLVM version 10.0.0 (clang-1000.10.44.4) - 64-bit gfortran GNU Fortran (GCC) 6.3.0 - (bear) Intel icc/icpc/ifort version 19.0.4.233 20190416 - - SunOS 5.11 11.3 Sun C 5.15 SunOS_sparc - 32- and 64-bit Sun Fortran 95 8.8 SunOS_sparc - (emu) Sun C++ 5.15 SunOS_sparc - - -Known Problems -============== - CMake files do not behave correctly with paths containing spaces. - Do not use spaces in paths because the required escaping for handling spaces - results in very complex and fragile build files. - ADB - 2019/05/07 - - At present, metadata cache images may not be generated by parallel - applications. Parallel applications can read files with metadata cache - images, but since this is a collective operation, a deadlock is possible - if one or more processes do not participate. - - Two tests fail attempting collective writes with OpenMPI 3.0.0/GCC-7.2.0-2.29: - testphdf5 (ecdsetw, selnone, cchunk1, cchunk3, cchunk4, and actualio) - t_shapesame (sscontig2) - - CPP ptable test fails on both VS2017 and VS2019 with Intel compiler, JIRA - issue: HDFFV-10628. This test will pass with VS2015 with Intel compiler. - - Known problems in previous releases can be found in the HISTORY*.txt files - in the HDF5 source. Please report any new problems found to - help@hdfgroup.org. - - -CMake vs. Autotools installations -================================= -While both build systems produce similar results, there are differences. -Each system produces the same set of folders on linux (only CMake works -on standard Windows); bin, include, lib and share. Autotools places the -COPYING and RELEASE.txt file in the root folder, CMake places them in -the share folder. - -The bin folder contains the tools and the build scripts. Additionally, CMake -creates dynamic versions of the tools with the suffix "-shared". Autotools -installs one set of tools depending on the "--enable-shared" configuration -option. - build scripts - ------------- - Autotools: h5c++, h5cc, h5fc - CMake: h5c++, h5cc, h5hlc++, h5hlcc - -The include folder holds the header files and the fortran mod files. CMake -the share folder. - -The bin folder contains the tools and the build scripts. Additionally, CMake -places the fortran mod files into separate shared and static subfolders, -while Autotools places one set of mod files into the include folder. Because -CMake produces a tools library, the header files for tools will appear in -the include folder. - -The lib folder contains the library files, and CMake adds the pkgconfig -subfolder with the hdf5*.pc files used by the bin/build scripts created by -the CMake build. CMake separates the C interface code from the fortran code by -creating C-stub libraries for each Fortran library. In addition, only CMake -installs the tools library. The names of the szip libraries are different -between the build systems. - -The share folder will have the most differences because CMake builds include -a number of CMake specific files for support of CMake's find_package and support -for the HDF5 Examples CMake project. - - - -%%%%1.10.6%%%% - -HDF5 version 1.10.6 released on 2019-12-23 -================================================================================ - - -INTRODUCTION - -This document describes the differences between this release and the previous -HDF5 release. It contains information on the platforms tested and known -problems in this release. For more details check the HISTORY*.txt files in the -HDF5 source. - -Note that documentation in the links below will be updated at the time of each -final release. - -Links to HDF5 documentation can be found on The HDF5 web page: - - https://portal.hdfgroup.org/display/HDF5/HDF5 - -The official HDF5 releases can be obtained from: - - https://www.hdfgroup.org/downloads/hdf5/ - -Changes from Release to Release and New Features in the HDF5-1.10.x release series -can be found at: - - https://portal.hdfgroup.org/display/HDF5/HDF5+Application+Developer%27s+Guide - -If you have any questions or comments, please send them to the HDF Help Desk: - - help@hdfgroup.org - - -CONTENTS - -- New Features -- Support for new platforms and languages -- Bug Fixes since HDF5-1.10.5 -- Supported Platforms -- Tested Configuration Features Summary -- More Tested Platforms -- Known Problems -- CMake vs. Autotools installations - - -New Features -============ - - Configuration: - ------------- - - Update CMake for VS2019 support - - CMake added support for VS2019 in version 3.15. Changes to the CMake - generator setting required changes to scripts. Also updated version - references in CMake files as necessary. - - (ADB - 2019/11/18, HDFFV-10962) - - - Update CMake options to match new autotools options - - Add configure options (autotools - CMake): - enable-asserts HDF5_ENABLE_ASSERTS - enable-symbols HDF5_ENABLE_SYMBOLS - enable-profiling HDF5_ENABLE_PROFILING - enable-optimization HDF5_ENABLE_OPTIMIZATION - In addition NDEBUG is no longer forced defined and relies on the CMake - process. - - (ADB - 2019/10/07, HDFFV-100901, HDFFV-10637, TRILAB-97) - - - Update CMake tests to use FIXTURES - - CMake test fixtures allow setup/cleanup tests and other dependency - requirements as properties for tests. This is more flexible for - modern CMake code. - - (ADB - 2019/07/23, HDFFV-10529) - - - Windows PDB files are always installed - - There are build configuration or flag settings for Windows that may not - generate PDB files. If those files are not generated then the install - utility will fail because those PDB files are not found. An optional - variable, DISABLE_PDB_FILES, was added to not install PDB files. - - (ADB - 2019/07/17, HDFFV-10424) - - - Add mingw CMake support with a toolchain file - - There have been a number of mingw issues that have been linked under - HDFFV-10845. It has been decided to implement the CMake cross-compiling - technique of toolchain files. We will use a linux platform with the mingw - compiler stack for testing. Only the C language is fully supported, and - the error tests are skipped. The C++ language works for static but shared - builds have a shared library issue with the mingw Standard Exception Handling - library, which is not available on Windows. Fortran has a common cross-compile - problem with the fortran configure tests. - - (ADB - 2019/07/12, HDFFV-10845, HDFFV-10595) - - - Windows PDB files are installed incorrectly - - For static builds, the PDB files for windows should be installed next - to the static libraries in the lib folder. Also the debug versions of - libraries and PDB files are now correctly built using the default - CMAKE_DEBUG_POSTFIX setting. - - (ADB - 2019/07/09, HDFFV-10581) - - - Add option to build only shared libs - - A request was made to prevent building static libraries and only build - shared. A new option was added to CMake, ONLY_SHARED_LIBS, which will - skip building static libraries. Certain utility functions will build with - static libs but are not published. Tests are adjusted to use the correct - libraries depending on SHARED/STATIC settings. - - (ADB - 2019/06/12, HDFFV-10805) - - - Add options to enable or disable building tools and tests - - Configure options --enable-tests and --enable-tools were added for - autotools configure. These options are enabled by default, and can be - disabled with either --disable-tests (or tools) or --enable-tests=no - (or --enable-tools=no). Build time is reduced ~20% when tools are - disabled, 35% when tests are disabled, 45% when both are disabled. - Re-enabling them after the initial build requires running configure - again with the option(s) enabled. - - (LRK - 2019/06/12, HDFFV-9976) - - - Change tools tests to search the error stack - - There are some use cases which can cause the error stack of tools to be - different then the expected output. These tests now use grepTest.cmake; - this was changed to allow the error file to be searched for an expected string. - - (ADB - 2019/04/15, HDFFV-10741) - - - Library: - -------- - - Added S3 and HDFS Virtual File Drivers (VFDs) to HDF5 - - These new VFDs have been introduced in HDF5-1.10.6. Instructions to - enable them when configuring HDF5 on Linux and Mac may be found at - https://portal.hdfgroup.org/display/HDF5/Virtual+File+Drivers+-+S3+and+HDFS. - - Installing on Windows requires CMake 3.13 and the following additional setup. - Install openssl library (with dev files); - from "Shining Light Productions". msi package preferred. - - PATH should have been updated with the installation dir. - set ENV variable OPENSSL_ROOT_DIR to the installation dir. - set ENV variable OPENSSL_CONF to the cfg file, likely %OPENSSL_ROOT_DIR%\bin\openssl.cfg - Install libcurl library (with dev files); - download the latest released version using git: https://github.com/curl/curl.git - - Open a Visual Studio Command prompt - change to the libcurl root folder - run the "buildconf.bat" batch file - change to the winbuild directory - nmake /f Makefile.vc mode=dll MACHINE=x64 - copy libcurl-vc-x64-release-dll-ipv6-sspi-winssl dir to C:\curl (installation dir) - set ENV variable CURL_ROOT to C:\curl (installation dir) - update PATH ENV variable to %CURL_ROOT%\bin (installation bin dir). - the aws credentials file should be in %USERPROFILE%\.aws folder - set the ENV variable "HDF5_ROS3_TEST_BUCKET_URL=https://s3.us-east-2.amazonaws.com/hdf5ros3" - - (ADB - 2019/09/12, HDFFV-10854) - - C++ Library: - ------------ - - Added new wrappers for H5Pset/get_create_intermediate_group() - LinkCreatPropList::setCreateIntermediateGroup() - LinkCreatPropList::getCreateIntermediateGroup() - - (BMR - 2019/04/22, HDFFV-10622) - - - Java Library: - ---------------- - - Fixed a failure in JUnit-TestH5P on 32-bit architectures - - (JTH - 2019/04/30) - - -Support for new platforms, languages and compilers. -======================================= - - CMake added support for VS2019 in version 3.15. Updated scripts. - - - macOS 10.13.6 Darwin 17.7.0 with Apple clang LLVM version 10.0.0 - - - macOS 10.14.6 Darwin 18.7.0 with Apple clang LLVM version 10.0.1 - - -Bug Fixes since HDF5-1.10.5 release -================================== - - Library - ------- - - Improved performance when creating a large number of small datasets by - retrieving default property values from the API context instead of doing - skip list searches. More work is required to achieve parity with HDF5 1.8. - - (CJH - 2019/12/10, HDFFV-10658) - - - Fixed user-created data access properties not existing in the property list - returned by H5Dget_access_plist. Thanks to Steven Varga for submitting a - reproducer and a patch. - - (CJH - 2019/12/9, HDFFV-10934) - - - Inappropriate linking with deprecated MPI C++ libraries - - HDF5 does not define *_SKIP_MPICXX in the public headers, so applications - can inadvertently wind up linking to the deprecated MPI C++ wrappers. - - MPICH_SKIP_MPICXX and OMPI_SKIP_MPICXX have both been defined in H5public.h - so this should no longer be an issue. HDF5 makes no use of the deprecated - MPI C++ wrappers. - - (DER - 2019/09/17, HDFFV-10893) - - - fcntl(2)-based file locking incorrectly passed the lock argument struct - instead of a pointer to the struct, causing errors on systems where - flock(2) is not available. - - File locking is used when files are opened to enforce SWMR semantics. A - lock operation takes place on all file opens unless the - HDF5_USE_FILE_LOCKING environment variable is set to the string "FALSE". - flock(2) is preferentially used, with fcntl(2) locks as a backup if - flock(2) is unavailable on a system (if neither is available, the lock - operation fails). On these systems, the file lock will often fail, which - causes HDF5 to not open the file and report an error. - - This bug only affects POSIX systems. Win32 builds on Windows use a no-op - locking call which always succeeds. Systems which exhibit this bug will - have H5_HAVE_FCNTL defined but not H5_HAVE_FLOCK in the configure output. - - This bug affects HDF5 1.10.0 through 1.10.5. - - fcntl(2)-based file locking now correctly passes the struct pointer. - - (DER - 2019/08/27, HDFFV-10892) - - - Fixed a bug caused by a bad tag value when condensing object header - messages - - There was an assertion failure when moving messages from running a - user test program with library release HDF5 1.10.4. It was because - the tag value (object header's address) was not set up when entering - the library routine H5O__chunk_update_idx(), which eventually - verifies the metadata tag value when protecting the object header. - - The problem was fixed by replacing FUNC_ENTER_PACKAGE in H5O__chunk_update_idx() - with FUNC_ENTER_PACKAGE_TAG(oh->cache_info.addr) to set up the metadata tag. - - (VC - 2019/08/23, HDFFV-10873) - - - Fixed the test failure from test_metadata_read_retry_info() in - test/swmr.c - - The test failure is due to an incorrect number of bins returned for - retry info (info.nbins). The # of bins expected for 101 read attempts - is 3 instead of 2. The routine H5F_set_retries() in src/H5Fint.c - calculates the # of bins by first obtaining the log10 value for - (read attempts - 1). For PGI/19, the log10 value for 100 read attempts - is 1.9999999999999998 instead of 2.00000. When casting the log10 value - to unsigned later on, the decimal part is chopped off causing the test - failure. - - This was fixed by obtaining the rounded integer value (HDceil) for the - log10 value of read attempts first before casting the result to unsigned. - - (VC - 2019/8/14, HDFFV-10813) - - - Fixed an issue when creating a file with non-default file space info - together with library high bound setting to H5F_LIBVER_V18. - - When setting non-default file space info in fcpl via - H5Pset_file_space_strategy() and then creating a file with both high and - low library bounds set to H5F_LIBVER_V18 in fapl, the library succeeds in - creating the file. File creation should fail because the feature of - setting non-default file space info does not exist in library release 1.8 - or earlier. - - This was fixed by setting and checking the proper version in the file - space info message based on the library low and high bounds when creating - and opening the HDF5 file. - - (VC - 2019/6/25, HDFFV-10808) - - - Fixed an issue where copying a version 1.8 dataset between files using - H5Ocopy fails due to an incompatible fill version - - When using the HDF5 1.10.x H5Ocopy() API call to copy a version 1.8 - dataset to a file created with both high and low library bounds set to - H5F_LIBVER_V18, the H5Ocopy() call will fail with the error stack indicating - that the fill value version is out of bounds. - - This was fixed by changing the fill value message version to H5O_FILL_VERSION_3 - (from H5O_FILL_VERSION_2) for H5F_LIBVER_V18. - - (VC - 2019/6/14, HDFFV-10800) - - - Fixed a bug that would cause an error or cause fill values to be - incorrectly read from a chunked dataset using the "single chunk" index if - the data was held in cache and there was no data on disk. - - (NAF - 2019/03/06) - - - Fixed a bug that could cause an error or cause fill values to be - incorrectly read from a dataset that was written to using H5Dwrite_chunk - if the dataset was not closed after writing. - - (NAF - 2019/03/06, HDFFV-10716) - - - Fixed memory leak in scale offset filter - - In a special case where the MinBits is the same as the number of bits in - the datatype's precision, the filter's data buffer was not freed, causing - the memory usage to grow. In general the buffer was freed correctly. The - Minbits are the minimal number of bits to store the data values. Please - see the reference manual for H5Pset_scaleoffset for the details. - - (RL - 2019/3/4, HDFFV-10705) - - - Configuration - ------------- - - Correct option for default API version - - CMake options for default API version are not mutually exclusive. - Change the multiple BOOL options to a single STRING option with the - strings; v16, v18, v110. - - (ADB - 2019/08/12, HDFFV-10879) - - Tools - ----- - - h5repack was fixed to repack datasets with external storage - to other types of storage. - - New test added to repack files and verify the correct data using h5diff. - - (JS - 2019/09/25, HDFFV-10408) - (ADB - 2019/10/02, HDFFV-10918) - - -Supported Platforms -=================== - - Linux 2.6.32-696.20.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23) - #1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23) - (ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-23) - IBM XL C/C++ V13.1 - IBM XL Fortran V15.1 - - Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (jelly/kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4) - Version 4.9.3, Version 5.2.0 - Intel(R) C (icc), C++ (icpc), Fortran (icc) - compilers: - Version 17.0.0.098 Build 20160721 - MPICH 3.1.4 compiled with GCC 4.9.3 - - SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc - (emu) Sun Fortran 95 8.6 SunOS_sparc - Sun C++ 5.12 SunOS_sparc - - Windows 7 Visual Studio 2015 w/ Intel Fortran 18 (cmake) - - Windows 7 x64 Visual Studio 2015 w/ Intel C, Fortran 2018 (cmake) - Visual Studio 2015 w/ MSMPI 8 (cmake) - - Windows 10 Visual Studio 2015 w/ Intel Fortran 18 (cmake) - - Windows 10 x64 Visual Studio 2015 w/ Intel Fortran 18 (cmake) - Visual Studio 2017 w/ Intel Fortran 19 (cmake) - Visual Studio 2019 w/ Intel Fortran 19 (cmake) - - macOS 10.13.6, Darwin, Apple clang LLVM version 10.0.0 - 17.7.0, x86_64 gfortran GNU Fortran (GCC) 6.3.0 - (bear) Intel icc/icpc/ifort version 19.0.4 - - macOS 10.14.6, Darwin Apple clang LLVM version 10.0.1 - 18.7.0, x86_64 gfortran GNU Fortran (GCC) 6.3.0 - (bobcat) Intel icc/icpc/ifort version 19.0.4 - - -Tested Configuration Features Summary -===================================== - - In the tables below - y = tested - n = not tested in this release - C = Cluster - W = Workstation - x = not working in this release - dna = does not apply - ( ) = footnote appears below second table - <blank> = testing incomplete on this feature or platform - -Platform C F90/ F90 C++ zlib SZIP - parallel F2003 parallel -Solaris2.11 32-bit n y/y n y y y -Solaris2.11 64-bit n y/n n y y y -Windows 7 y y/y n y y y -Windows 7 x64 y y/y y y y y -Windows 7 Cygwin n y/n n y y y -Windows 7 x64 Cygwin n y/n n y y y -Windows 10 y y/y n y y y -Windows 10 x64 y y/y n y y y -Mac OS X Yosemite 10.10.5 64-bit n y/y n y y y -Mac OS X El Capitan 10.11.6 64-bit n y/y n y y y -MacOS High Sierra 10.13.6 64-bit n y/y n y y y -CentOS 7.2 Linux 3.10.0 x86_64 PGI n y/y n y y y -CentOS 7.2 Linux 3.10.0 x86_64 GNU y y/y y y y y -CentOS 7.2 Linux 3.10.0 x86_64 Intel n y/y n y y y -Linux 2.6.32-573.18.1.el6.ppc64 n y/y n y y y - - -Platform Shared Shared Shared Thread- - C libs F90 libs C++ libs safe -Solaris2.11 32-bit y y y y -Solaris2.11 64-bit y y y y -Windows 7 y y y y -Windows 7 x64 y y y y -Windows 7 Cygwin n n n y -Windows 7 x64 Cygwin n n n y -Windows 10 y y y y -Windows 10 x64 y y y y -Mac OS X Yosemite 10.10.5 64-bit y y y y -Mac OS X El Capitan 10.11.6 64-bit y y y y -MacOS High Sierra 10.13.6 64-bit y y y y -CentOS 7.2 Linux 3.10.0 x86_64 PGI y y y n -CentOS 7.2 Linux 3.10.0 x86_64 GNU y y y y -CentOS 7.2 Linux 3.10.0 x86_64 Intel y y y n -Linux 2.6.32-573.18.1.el6.ppc64 y y y n - -Compiler versions for each platform are listed in the preceding -"Supported Platforms" table. - - -More Tested Platforms -===================== -The following configurations are not supported but have been tested for this release. - - Linux 2.6.32-754.11.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (mayll/platypus) Version 4.4.7 20120313 - Version 4.9.3, 5.3.0, 6.2.0 - PGI C, Fortran, C++ for 64-bit target on - x86-64; - Version 17.10-0 - Intel(R) C (icc), C++ (icpc), Fortran (icc) - compilers: - Version 17.0.4.196 Build 20170411 - MPICH 3.1.4 compiled with GCC 4.9.3 - - Linux 3.10.0-327.18.2.el7 GNU C (gcc) and C++ (g++) compilers - #1 SMP x86_64 GNU/Linux Version 4.8.5 20150623 (Red Hat 4.8.5-4) - (jelly) with NAG Fortran Compiler Release 6.2(Chiyoda) - GCC Version 7.1.0 - MPICH 3.2-GCC-4.9.3 - MPICH 3.2.1-GCC-7.2.0-2.29 - OpenMPI 2.1.5-GCC-7.2.0-2.29 - Intel(R) C (icc) and C++ (icpc) compilers - Version 17.0.0.098 Build 20160721 - with NAG Fortran Compiler Release 6.2(Chiyoda) - - Linux 3.10.0-327.10.1.el7 MPICH 3.2 compiled with GCC 5.3.0 - #1 SMP x86_64 GNU/Linux - (moohan) - - Linux 2.6.32-573.18.1.el6.ppc64 MPICH mpich 3.1.4 compiled with - #1 SMP ppc64 GNU/Linux IBM XL C/C++ for Linux, V13.1 - (ostrich) and IBM XL Fortran for Linux, V15.1 - - Fedora30 5.3.11-200.fc30.x86_64 - #1 SMP x86_64 GNU/Linux GNU gcc (GCC) 9.2.1 20190827 (Red Hat 9.2.1 20190827) - GNU Fortran (GCC) 9.2.1 20190827 (Red Hat 9.2.1 20190827) - (cmake and autotools) - - Mac OS X 10.11.6, Darwin, Apple clang version 7.3.0 from Xcode 7.3 - 15.6.0, x86-64 gfortran GNU Fortran (GCC) 5.2.0 - (osx1011test) Intel icc/icpc/ifort version 16.0.2 - - macOS 10.12.6, Darwin, Apple clang LLVM version 8.1.0 from Xcode 8.3 - 16.6.0, x86_64 gfortran GNU Fortran (GCC) 7.1.0 - (kite) Intel icc/icpc/ifort version 17.0.2 - - Windows 7 x64 Visual Studio 2008 - - -Known Problems -============== - CMake files do not behave correctly with paths containing spaces. - Do not use spaces in paths because the required escaping for handling spaces - results in very complex and fragile build files. - ADB - 2019/05/07 - - At present, metadata cache images may not be generated by parallel - applications. Parallel applications can read files with metadata cache - images, but since this is a collective operation, a deadlock is possible - if one or more processes do not participate. - - Three tests fail with OpenMPI 3.0.0/GCC-7.2.0-2.29: - testphdf5 (ecdsetw, selnone, cchunk1, cchunk3, cchunk4, and actualio) - t_shapesame (sscontig2) - t_pflush1/fails on exit - The first two tests fail attempting collective writes. - - Parallel builds using OpenMPI 3.03 or later and romio fail several tests - with collective writes or compression that will not fail when ompio is used - instead of romio. This can be done by adding "--mca io ompio" to the mpirun - command. For example, in autotools builds RUNPARALLEL can be set to - "mpirun --mca io ompio -n 6" provided ompio is installed. - - CPP ptable test fails on VS2017 with Intel compiler, JIRA issue: HDFFV-10628. - This test will pass with VS2015 with Intel compiler. - - Older MPI libraries such as OpenMPI 2.0.1 and MPICH 2.1.5 were tested - while attempting to resolve the Jira issue: HDFFV-10540. - The known problems of reading or writing > 2GBs when using MPI-2 was - partially resolved with the MPICH library. The proposed support recognizes - IO operations > 2GB and if the datatype is not a derived type, the library - breaks the IO into chunks which can be input or output with the existing - MPI 2 limitations, i.e. size reporting and function API size/count - arguments are restricted to be 32 bit integers. For derived types larger - than 2GB, MPICH 2.1.5 fails while attempting to read or write data. - OpenMPI in contrast, implements MPI-3 APIs even in the older releases - and thus does not suffer from the 32 bit size limitation described here. - OpenMPI releases prior to v3.1.3 appear to have other datatype issues however, - e.g. within a single parallel test (testphdf5) the subtests (cdsetr, eidsetr) - report data verification errors before eventually aborting. - The most recent versions of OpenMPI (v3.1.3 or newer) have evidently - resolved these issues and parallel HDF5 testing does not currently report - errors though occasional hangs have been observed. - - Known problems in previous releases can be found in the HISTORY*.txt files - in the HDF5 source. Please report any new problems found to - help@hdfgroup.org. - - -CMake vs. Autotools installations -================================= -While both build systems produce similar results, there are differences. -Each system produces the same set of folders on linux (only CMake works -on standard Windows); bin, include, lib and share. Autotools places the -COPYING and RELEASE.txt file in the root folder, CMake places them in -the share folder. - -The bin folder contains the tools and the build scripts. Additionally, CMake -creates dynamic versions of the tools with the suffix "-shared". Autotools -installs one set of tools depending on the "--enable-shared" configuration -option. - build scripts - ------------- - Autotools: h5c++, h5cc, h5fc - CMake: h5c++, h5cc, h5hlc++, h5hlcc - -The include folder holds the header files and the fortran mod files. CMake -places the fortran mod files into separate shared and static subfolders, -while Autotools places one set of mod files into the include folder. Because -CMake produces a tools library, the header files for tools will appear in -the include folder. - -The lib folder contains the library files, and CMake adds the pkgconfig -subfolder with the hdf5*.pc files used by the bin/build scripts created by -the CMake build. CMake separates the C interface code from the fortran code by -creating C-stub libraries for each Fortran library. In addition, only CMake -installs the tools library. The names of the szip libraries are different -between the build systems. - -The share folder will have the most differences because CMake builds include -a number of CMake specific files for support of CMake's find_package and support -for the HDF5 Examples CMake project. - -%%%%1.10.5%%%% - -HDF5 version 1.10.5 released on 2019-02-25 -================================================================================ - - -INTRODUCTION - -This document describes the differences between this release and the previous -HDF5 release. It contains information on the platforms tested and known -problems in this release. For more details check the HISTORY*.txt files in the -HDF5 source. - -Note that documentation in the links below will be updated at the time of each -final release. - -Links to HDF5 documentation can be found on The HDF5 web page: - - https://portal.hdfgroup.org/display/HDF5/HDF5 - -The official HDF5 releases can be obtained from: - - https://www.hdfgroup.org/downloads/hdf5/ - -Changes from Release to Release and New Features in the HDF5-1.10.x release series -can be found at: - - https://portal.hdfgroup.org/display/HDF5/HDF5+Application+Developer%27s+Guide - -If you have any questions or comments, please send them to the HDF Help Desk: - - help@hdfgroup.org - - -CONTENTS - -- New Features -- Support for new platforms and languages -- Bug Fixes since HDF5-1.10.4 -- Supported Platforms -- Tested Configuration Features Summary -- More Tested Platforms -- Known Problems -- CMake vs. Autotools installations - - -New Features -============ - - Configuration: - ------------- - - Cross compile on mutrino and other Cray XC40 systems. - - Added support for CMake options to use CrayLinuxEnvironment, craype-mic-knl - module for building with craype-haswell module for configuration, and - batch scripts in bin/batch for use with sbatch or bsub to run tests in - batch jobs on compute nodes. An instruction file README_HPC describing - the use of these options was added in release_docs. - - (LRK - 2019/02/18, TRILABS-34) - - - Rework CMake command files to fix MPI testing. - - Added setup fixture to remove any test generated files and added DEPENDS - to test properties to execute tests in order expected. - - (ADB - 2019/02/14, TRILABS-111) - - - Disable SZIP or ZLIB options if TGZ files are not available. - - Changed the TGZ option for SZip and ZLib to disable the options - if the source tar.gz files are not found. - - (ADB - 2019/02/05, HDFFV-10697) - - - Added a new option to enable/disable using pread/pwrite instead of - read/write in the sec2, log, and core VFDs. - - This option is enabled by default when pread/pwrite are detected. - - Autotools: --enable-preadwrite - CMake: HDF5_ENABLE_PREADWRITE - - (DER - 2019/02/03, HDFFV-10696) - - - Rework CMake versioning for OSX platforms. - - Changed the current_version and compatibility_version flags from optional - with HDF5_BUILD_WITH_INSTALL_NAME to always setting the flags for OSX. - - (ADB - 2019/01/22, HDFFV-10685) - - - Rework CMake command files to eliminate developer CMP005 warning - - Use variables without quotes in if () statements. - - (ADB - 2019/01/18, TILABS-105) - - - Rework CMake configure files to eliminate developer CMP0075 warning - - Renamed varname to HDF5_REQUIRED_LIBRARIES as the contents were not - required for configuration. Also moved check includes calls to top of - files. - - (ADB - 2019/01/03, HDFFV-10546) - - - Keep stderr and stdout separate in tests - - Changed test handling of output capture. Tests now keep the stderr - output separate from the stdout output. It is up to the test to decide - which output to check against a reference. Also added the option - to grep for a string in either output. - - (ADB - 2018/12/12, HDFFV-10632) - - - Incorrectly installed private header files were removed from - CMake installs. - - The CMake build files incorrectly flagged the following header files - as public and installed them. They are private and will no longer be - installed. - - HDF5 library private package files (H5Xpkg.h) - H5Edefin.h - H5Einit.h - H5Eterm.h - H5LTparse.h - h5diff.h - h5tools_dump.h - h5tools.h - h5tools_ref.h - h5tools_str.h - h5tools_utils.h - h5trav.h - - (DER - 2018/10/26, HDFFV-10614, 10609) - - - Autotools installs now install H5FDwindows.h - - This is simply to align the installed header files between the - autotools and CMake. H5FDwindows.h has no functionality on - non-Windows systems. - - (DER - 2018/10/26, HDFFV-10614) - - - Library: - -------- - - The sec2, log, and core VFDs can now use pread/pwrite instead of - read/write. - - pread and pwrite do not change the file offset, a feature that was - requested by a user working with a multi-threaded application. - - The option to configure this feature is described above. - - (DER - 2019/02/03, HDFFV-10696) - - - Add ability to minimize dataset object headers. - - Creation of many, very small datasets resulted in extensive file bloat - due to extra space in the dataset object headers -- this space is - allocated by default to allow for the insertion of a small number of - attributes within the object header and not require a continuation - block, an unnecessary provision in the target use case. - - Inform the library to expect no attributes on created datasets, and to - allocate the least space possible for the object headers. - NOTE: A continuation block is created if attributes are added to a - 'minimized' dataset, which can reduce performance. - NOTE: Some extra space is allocated for attributes essential to the - correct behavior of the object header (store creation times, e.g.). This - does not violate the design principle, as the space is calculated and - allocated as needed at the time of dataset object header creation -- - unused space is not generated. - New API calls: - H5Fget_dset_no_attrs_hint - H5Fset_dset_no_attrs_hint - H5Pget_dset_no_attrs_hint - H5Pset_dset_no_attrs_hint - - (JOS - 2019/01/04, TRILAB-45) - - - Added new chunk query functions - - The following public functions were added to discover information about - the chunks in an HDF5 file. - herr_t H5Dget_num_chunks(dset_id, fspace_id, *nchunks) - herr_t H5Dget_chunk_info_by_coord(dset_id, *coord, *filter_mask, *addr, *size) - herr_t H5Dget_chunk_info(dset_id, fspace_id, index, *coord, *filter_mask, *addr, *size) - - (BMR - 2018/11/07, HDFFV-10615) - - - Several empty public header files where removed from the distribution - - The following files were empty placeholders. They are for internal - packages that are unlikely to ever have public functionality and have - thus been removed. - - H5Bpublic.h - H5B2public.h - H5FSpublic.h - H5HFpublic.h - H5HGpublic.h - H5HLpublic.h - - They were only installed in CMake builds. - - (DER - 2018/10/26, HDFFV-10614) - - - Parallel Library: - ----------------- - - Changed the default behavior in parallel when reading the same dataset in its entirety - (i.e. H5S_ALL dataset selection) which is being read by all the processes collectively. - The dataset must be contiguous, less than 2GB, and of an atomic datatype. - The new behavior is the HDF5 library will use an MPI_Bcast to pass the data read from - the disk by the root process to the remain processes in the MPI communicator associated - with the HDF5 file. - - (MSB - 2019/01/02, HDFFV-10652) - - - All MPI-1 API calls have been replaced with MPI-2 equivalents. - - This was done to better support OpenMPI, as default builds no longer - include MPI-1 support (as of OpenMPI 4.0). - - (DER - 2018/12/30, HDFFV-10566) - - Fortran Library: - ---------------- - - Added wrappers for dataset object header minimization calls. - (see the note for TRILAB-45, above) - - New API calls: - - h5fget_dset_no_attrs_hint_f - h5fset_dset_no_attrs_hint_f - h5pget_dset_no_attrs_hint_f - h5pset_dset_no_attrs_hint_f - - (DER - 2019/01/09, TRILAB-45) - - - Added new Fortran derived type, c_h5o_info_t, which is interoperable with - C's h5o_info_t. This is needed for callback functions which - pass C's h5o_info_t data type definition. - - (MSB, 2019/01/08, HDFFV-10443) - - - Added new Fortran API, H5gmtime, which converts (C) 'time_t' structure - to Fortran DATE AND TIME storage format. - - (MSB, 2019/01/08, HDFFV-10443) - - - Added new Fortran 'fields' optional parameter to: h5ovisit_f, h5oget_info_by_name_f, - h5oget_info, h5oget_info_by_idx and h5ovisit_by_name_f. - - (MSB, 2019/01/08, HDFFV-10443) - - C++ Library: - ------------ - - Added new function to the C++ interface - - Added wrapper for H5Ovisit2: - H5Object::visit() - - (BMR - 2019/02/14, HDFFV-10532) - - - Java Library: - ---------------- - - Rewrote the JNI error handling to be much cleaner - - (JTH - 2019/02/12) - - - Add new functions to java interface - - Added wrappers for: - H5Fset_libver_bounds - H5Fget_dset_no_attrs_hint/H5Fset_dset_no_attrs_hint - H5Pget_dset_no_attrs_hint/H5Pset_dset_no_attrs_hint - - (ADB - 2019/01/07, HDFFV-10664) - - - Fix java unit tests when Time is a natural number - - Time substitution in java/test/junit.sh.in doesn't - handle the case when Time is a natural number. Fixed - the regular expression. - - (ADB - 2019/01/07, HDFFV-10674) - - - Duplicate the data read/write functions of Datasets for Attributes. - - Region references could not be displayed for attributes as they could - for datasets. Datasets had overloaded read and write functions for different - datatypes that were not available for attributes. After adding similar - functions, attribute region references work normally. - - (ADB - 2018/12/12, HDFVIEW-4) - - - Tools: - ------ - - The h5repart -family-to-sec2 argument was changed to -family-to-single - - In order to better support other single-file VFDs which could work with - h5repart, the -family-to-sec2 argument was renamed to -family-to-single. - This is just a name change and the functionality of the argument has not - changed. - - The -family-to-sec2 argument has been kept for backwards-compatibility. - This argument should be considered deprecated. - - (DER - 2018/11/14, HDFFV-10633) - - -Bug Fixes since HDF5-1.10.4 release -================================== - - Library - ------- - - Fix hangs with collective metadata reads during chunked dataset I/O - - In the parallel library, it was discovered that when a particular - sequence of operations following a pattern of: - - "write to chunked dataset" -> "flush file" -> "read from dataset" - - occurred with collective metadata reads enabled, hangs could be - observed due to certain MPI ranks not participating in the collective - metadata reads. - - To fix the issue, collective metadata reads are now disabled during - chunked dataset raw data I/O. - - (JTH - 2019/02/11, HDFFV-10563, HDFFV-10688) - - - Performance issue when closing an object - - The slow down is due to the search of the "tag_list" to find - out the "corked" status of an object and "uncork" it if so. - - Improve performance by skipping the search of the "tag_list" - if there are no "corked" objects when closing an object. - - (VC - 2019/02/06) - - - Uninitialized bytes from a type conversion buffer could be written - to disk in H5Dwrite calls where type conversion takes place - and the type conversion buffer was created by the HDF5 library. - - When H5Dwrite is called and datatype conversion must be performed, - the library will create a temporary buffer for type conversion if - one is not provided by the user via H5Pset_buffer. This internal - buffer is allocated via malloc and contains uninitialized data. In - some datatype conversions (float to long double, possibly others), - some of this uninitialized data could be written to disk. - - This was flagged by valgrind in the dtransform test and does not - appear to be a common occurrence (it is flagged in one test out - of the entire HDF5 test suite). - - Switching to calloc fixed the problem. - - (DER - 2019/02/03, HDFFV-10694) - - - There was missing protection against division by zero reported to - The HDF Group as issue #CVE-2018-17434. - - Protection against division by zero was added to address the issue - #CVE-2018-17434. - - (BMR - 2019/01/29, HDFFV-10586) - - - The issue CVE-2018-17437 was reported to The HDF Group - - Although CVE-2018-17437 reported a memory leak, the actual issue - was invalid read. It was found that the attribute name length - in an attribute message was corrupted, which caused the buffer - pointer to be advanced too far and later caused an invalid read. - - A check was added to detect when the attribute name or its length - was corrupted and report the potential of data corruption. - - (BMR - 2019/01/29, HDFFV-10588) - - - H5Ewalk did not stop when it was supposed to - - H5Ewalk was supposed to stop when the callback function stopped - even though the errors in the stack were not all visited, but it - did not. This problem is now fixed. - - (BMR - 2019/01/29, HDFFV-10684) - - - Revert H5Oget_info* and H5Ovisit* functions - - In 1.10.3 new H5Oget_info*2 and H5Ovisit*2 functions were - added for performance. Inadvertently, the original functions; - H5Oget_info, - H5Oget_info_by_name, - H5Oget_info_by_idx, - H5Ovisit, - H5Ovisit_by_name - were versioned to H5Oget_info*1 and H5Ovisit*1. This - broke the API compatibility for a maintenance release. The - original functions have been restored. - - (ADB - 2019/01/24, HDFFV-10686) - - - Fixed a potential invalid memory access and failure that could occur when - decoding an unknown object header message (from a future version of the - library). - - (NAF - 2019/01/07) - - - Deleting attributes in dense storage - - The library aborts with "infinite loop closing library" after - attributes in dense storage are created and then deleted. - - When deleting the attribute nodes from the name index v2 B-tree, - if an attribute is found in the intermediate B-tree nodes, - which may be merged/redistributed in the process, we need to - free the dynamically allocated spaces for the intermediate - decoded attribute. - - (VC - 2018/12/26, HDFFV-10659) - - - There was missing protection against division by zero reported to - The HDF Group as issue #CVE-2018-17233. - - Protection against division by zero was added to address the issue - #CVE-2018-17233. In addition, several similar occurrences in the same - file were fixed as well. - - (BMR - 2018/12/23, HDFFV-10577) - - - Fixed an issue where the parallel filters tests would fail - if zlib was not available on the system. Until support can - be added in the tests for filters beyond gzip/zlib, the tests - will be skipped if zlib is not available. - - (JTH - 2018/12/05) - - - A bug was discovered in the parallel library where an application - would eventually consume all of the available MPI communicators - when continually writing to a compressed dataset in parallel. This - was due to internal copies of an HDF5 File Access Property List, - which each contained a copy of the MPI communicator, not being - closed at the end of each write operation. This problem was - exacerbated by larger numbers of processors. - - (JTH - 2018/12/05, HDFFV-10629) - - - Fortran - -------- - - Fixed issue with Fortran not returning h5o_info_t field values - meta_size%attr%index_size and meta_size%attr%heap_size. - - (MSB, 2019/01/08, HDFFV-10443) - - - Added symbolic links libhdf5_hl_fortran.so to libhdf5hl_fortran.so and - libhdf5_hl_fortran.a to libhdf5hl_fortran.a in hdf5/lib directory for - autotools installs. These were added to match the name of the files - installed by cmake and the general pattern of hl lib files. We will - change the names of the installed lib files to the matching name in - the next major release. - - (LRK - 2019/01/04, HDFFV-10596) - - - Made Fortran specific subroutines PRIVATE in generic procedures. - - Affected generic procedures were functions in H5A, H5D, H5P, H5R and H5T. - - (MSB, 2018/12/04, HDFFV-10511) - - - Testing - ------- - - Fixed a test failure in testpar/t_dset.c caused by - the test trying to use the parallel filters feature - on MPI-2 implementations. - - (JTH, 2019/2/7) - - -Supported Platforms -=================== - - Linux 2.6.32-696.16.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) - #1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) - (ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) - IBM XL C/C++ V13.1 - IBM XL Fortran V15.1 - - Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4) - Version 4.9.3, Version 5.2.0 - Intel(R) C (icc), C++ (icpc), Fortran (icc) - compilers: - Version 17.0.0.098 Build 20160721 - MPICH 3.1.4 compiled with GCC 4.9.3 - - SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc - (emu) Sun Fortran 95 8.6 SunOS_sparc - Sun C++ 5.12 SunOS_sparc - - Windows 7 Visual Studio 2015 w/ Intel Fortran 16 (cmake) - - Windows 7 x64 Visual Studio 2013 - Visual Studio 2015 w/ Intel Fortran 16 (cmake) - Visual Studio 2015 w/ Intel C, Fortran 2018 (cmake) - Visual Studio 2015 w/ MSMPI 8 (cmake) - - Windows 10 Visual Studio 2015 w/ Intel Fortran 18 (cmake) - - Windows 10 x64 Visual Studio 2015 w/ Intel Fortran 18 (cmake) - Visual Studio 2017 w/ Intel Fortran 18 (cmake) - - Mac OS X Yosemite 10.10.5 Apple clang/clang++ version 6.1 from Xcode 7.0 - 64-bit gfortran GNU Fortran (GCC) 4.9.2 - (osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3 - - Mac OS X El Capitan 10.11.6 Apple clang/clang++ version 7.3.0 from Xcode 7.3 - 64-bit gfortran GNU Fortran (GCC) 5.2.0 - (osx1011dev/osx1011test) Intel icc/icpc/ifort version 16.0.2 - - MacOS High Sierra 10.13.6 Apple LLVM version 10.0.0 (clang/clang++-1000.10.44.4) - 64-bit gfortran GNU Fortran (GCC) 8.3.0 - - -Tested Configuration Features Summary -===================================== - - In the tables below - y = tested - n = not tested in this release - C = Cluster - W = Workstation - x = not working in this release - dna = does not apply - ( ) = footnote appears below second table - <blank> = testing incomplete on this feature or platform - -Platform C F90/ F90 C++ zlib SZIP - parallel F2003 parallel -Solaris2.11 32-bit n y/y n y y y -Solaris2.11 64-bit n y/n n y y y -Windows 7 y y/y n y y y -Windows 7 x64 y y/y y y y y -Windows 7 Cygwin n y/n n y y y -Windows 7 x64 Cygwin n y/n n y y y -Windows 10 y y/y n y y y -Windows 10 x64 y y/y n y y y -Mac OS X Yosemite 10.10.5 64-bit n y/y n y y y -Mac OS X El Capitan 10.11.6 64-bit n y/y n y y y -MacOS High Sierra 10.13.6 64-bit n y/y n y y y -CentOS 7.2 Linux 3.10.0 x86_64 PGI n y/y n y y y -CentOS 7.2 Linux 3.10.0 x86_64 GNU y y/y y y y y -CentOS 7.2 Linux 3.10.0 x86_64 Intel n y/y n y y y -Linux 2.6.32-573.18.1.el6.ppc64 n y/y n y y y - - -Platform Shared Shared Shared Thread- - C libs F90 libs C++ libs safe -Solaris2.11 32-bit y y y y -Solaris2.11 64-bit y y y y -Windows 7 y y y y -Windows 7 x64 y y y y -Windows 7 Cygwin n n n y -Windows 7 x64 Cygwin n n n y -Windows 10 y y y y -Windows 10 x64 y y y y -Mac OS X Yosemite 10.10.5 64-bit y y y y -Mac OS X El Capitan 10.11.6 64-bit y y y y -MacOS High Sierra 10.13.6 64-bit y y y y -CentOS 7.2 Linux 3.10.0 x86_64 PGI y y y n -CentOS 7.2 Linux 3.10.0 x86_64 GNU y y y y -CentOS 7.2 Linux 3.10.0 x86_64 Intel y y y n -Linux 2.6.32-573.18.1.el6.ppc64 y y y n - -Compiler versions for each platform are listed in the preceding -"Supported Platforms" table. - - -More Tested Platforms -===================== -The following configurations are not supported but have been tested for this release. - - Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (mayll/platypus) Version 4.4.7 20120313 - Version 4.9.3, 5.3.0, 6.2.0 - PGI C, Fortran, C++ for 64-bit target on - x86-64; - Version 17.10-0 - Intel(R) C (icc), C++ (icpc), Fortran (icc) - compilers: - Version 17.0.4.196 Build 20170411 - MPICH 3.1.4 compiled with GCC 4.9.3 - - Linux 3.10.0-327.18.2.el7 GNU C (gcc) and C++ (g++) compilers - #1 SMP x86_64 GNU/Linux Version 4.8.5 20150623 (Red Hat 4.8.5-4) - (jelly) with NAG Fortran Compiler Release 6.1(Tozai) - GCC Version 7.1.0 - MPICH 3.2-GCC-4.9.3 - MPICH 3.2.1-GCC-7.2.0-2.29 - OpenMPI 2.1.5-GCC-7.2.0-2.29 - Intel(R) C (icc) and C++ (icpc) compilers - Version 17.0.0.098 Build 20160721 - with NAG Fortran Compiler Release 6.1(Tozai) - - Linux 3.10.0-327.10.1.el7 MPICH 3.2 compiled with GCC 5.3.0 - #1 SMP x86_64 GNU/Linux - (moohan) - - Fedora 29 4.20.10-200.fc29.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux - gcc, g++ (GCC) 8.2.1 20181215 - (Red Hat 8.2.1-6) - GNU Fortran (GCC) 8.2.1 20181215 - (Red Hat 8.2.1-6) - (cmake and autotools) - - Windows 7 x64 Visual Studio 2008 - - -Known Problems -============== - - At present, metadata cache images may not be generated by parallel - applications. Parallel applications can read files with metadata cache - images, but since this is a collective operation, a deadlock is possible - if one or more processes do not participate. - - Three tests fail with OpenMPI 3.0.0/GCC-7.2.0-2.29: - testphdf5 (ecdsetw, selnone, cchunk1, cchunk3, cchunk4, and actualio) - t_shapesame (sscontig2) - t_pflush1/fails on exit - The first two tests fail attempting collective writes. - - CPP ptable test fails on VS2017 with Intel compiler, JIRA issue: HDFFV-10628. - This test will pass with VS2015 with Intel compiler. - - Older MPI libraries such as OpenMPI 2.0.1 and MPICH 2.1.5 were tested - while attempting to resolve the Jira issue: HDFFV-10540. - The known problems of reading or writing > 2GBs when using MPI-2 was - partially resolved with the MPICH library. The proposed support recognizes - IO operations > 2GB and if the datatype is not a derived type, the library - breaks the IO into chunks which can be input or output with the existing - MPI 2 limitations, i.e. size reporting and function API size/count - arguments are restricted to be 32 bit integers. For derived types larger - than 2GB, MPICH 2.1.5 fails while attempting to read or write data. - OpenMPI in contrast, implements MPI-3 APIs even in the older releases - and thus does not suffer from the 32 bit size limitation described here. - OpenMPI releases prior to v3.1.3 appear to have other datatype issues however, - e.g. within a single parallel test (testphdf5) the subtests (cdsetr, eidsetr) - report data verification errors before eventually aborting. - The most recent versions of OpenMPI (v3.1.3 or newer) have evidently - resolved these issues and parallel HDF5 testing does not currently report - errors though occasional hangs have been observed. - - Known problems in previous releases can be found in the HISTORY*.txt files - in the HDF5 source. Please report any new problems found to - help@hdfgroup.org. - - -CMake vs. Autotools installations -================================= -While both build systems produce similar results, there are differences. -Each system produces the same set of folders on linux (only CMake works -on standard Windows); bin, include, lib and share. Autotools places the -COPYING and RELEASE.txt file in the root folder, CMake places them in -the share folder. - -The bin folder contains the tools and the build scripts. Additionally, CMake -creates dynamic versions of the tools with the suffix "-shared". Autotools -installs one set of tools depending on the "--enable-shared" configuration -option. - build scripts - ------------- - Autotools: h5c++, h5cc, h5fc - CMake: h5c++, h5cc, h5hlc++, h5hlcc - -The include folder holds the header files and the fortran mod files. CMake -places the fortran mod files into separate shared and static subfolders, -while Autotools places one set of mod files into the include folder. Because -CMake produces a tools library, the header files for tools will appear in -the include folder. - -The lib folder contains the library files, and CMake adds the pkgconfig -subfolder with the hdf5*.pc files used by the bin/build scripts created by -the CMake build. CMake separates the C interface code from the fortran code by -creating C-stub libraries for each Fortran library. In addition, only CMake -installs the tools library. The names of the szip libraries are different -between the build systems. - -The share folder will have the most differences because CMake builds include -a number of CMake specific files for support of CMake's find_package and support -for the HDF5 Examples CMake project. - -%%%%1.10.4%%%% - -HDF5 version 1.10.4 released on 2018-10-05 -================================================================================ - - -INTRODUCTION - -This document describes the differences between this release and the previous -HDF5 release. It contains information on the platforms tested and known -problems in this release. For more details check the HISTORY*.txt files in the -HDF5 source. - -Note that documentation in the links below will be updated at the time of each -final release. - -Links to HDF5 documentation can be found on The HDF5 web page: - - https://portal.hdfgroup.org/display/HDF5/HDF5 - -The official HDF5 releases can be obtained from: - - https://www.hdfgroup.org/downloads/hdf5/ - -Changes from Release to Release and New Features in the HDF5-1.10.x release series -can be found at: - - https://portal.hdfgroup.org/display/HDF5/HDF5+Application+Developer%27s+Guide - -If you have any questions or comments, please send them to the HDF Help Desk: - - help@hdfgroup.org - - -CONTENTS - -- Bug Fixes since HDF5-1.10.3 -- Supported Platforms -- Tested Configuration Features Summary -- More Tested Platforms -- Known Problems -- CMake vs. Autotools installations - - -New Features -============ - - Configuration: - ------------- - - Add toolchain and cross-compile support - - Added info on using a toolchain file to INSTALL_CMAKE.txt. A - toolchain file is also used in cross-compiling, which requires - CMAKE_CROSSCOMPILING_EMULATOR to be set. To help with cross-compiling - the fortran configure process, the HDF5UseFortran.cmake file macros - were improved. Fixed a Fortran configure file issue that incorrectly - used #cmakedefine instead of #define. - - (ADB - 2018/10/04, HDFFV-10594) - - - Add warning flags for Intel compilers - - Identified Intel compiler specific warnings flags that should be used - instead of GNU flags. - - (ADB - 2018/10/04, TRILABS-21) - - - Add default rpath to targets - - Default rpaths should be set in shared executables and - libraries to allow the use of loading dependent libraries - without requiring LD_LIBRARY_PATH to be set. The default - path should be relative using @rpath on osx and $ORIGIN - on linux. Windows is not affected. - - (ADB - 2018/09/26, HDFFV-10594) - - Library: - -------- - - Allow pre-generated H5Tinit.c and H5make_libsettings.c to be used. - - Rather than always running H5detect and generating H5Tinit.c and - H5make_libsettings.c, supply a location for those files. - - (ADB - 2018/09/18, HDFFV-10332) - - -Bug Fixes since HDF5-1.10.3 release -================================== - - Library - ------- - - Allow H5detect and H5make_libsettings to take a file as an argument. - - Rather than only writing to stdout, add a command argument to name - the file that H5detect and H5make_libsettings will use for output. - Without an argument, stdout is still used, so backwards compatibility - is maintained. - - (ADB - 2018/09/05, HDFFV-9059) - - - A bug was discovered in the parallel library where an application - would hang if a collective read/write of a chunked dataset occurred - when collective metadata reads were enabled and some of the ranks - had no selection in the dataset's dataspace. The ranks which had no - selection in the dataset's dataspace called H5D__chunk_addrmap() to - retrieve the lowest chunk address in the dataset. This is because we - require reads/writes to be performed in strictly non-decreasing order - of chunk address in the file. - - When the chunk index used was a version 1 or 2 B-tree, these - non-participating ranks would issue a collective MPI_Bcast() call - that the participating ranks would not issue, causing the hang. Since - the non-participating ranks are not actually reading/writing anything, - the H5D__chunk_addrmap() call can be safely removed and the address used - for the read/write can be set to an arbitrary number (0 was chosen). - - (JTH - 2018/08/25, HDFFV-10501) - - Java Library: - ---------------- - - JNI native library dependencies - - The build for the hdf5_java native library used the wrong - hdf5 target library for CMake builds. Correcting the hdf5_java - library to build with the shared hdf5 library required testing - paths to change also. - - (ADB - 2018/08/31, HDFFV-10568) - - - Java iterator callbacks - - Change global callback object to a small stack structure in order - to fix a runtime crash. This crash was discovered when iterating - through a file with nested group members. The global variable - visit_callback is overwritten when recursion starts. When recursion - completes, visit_callback will be pointing to the wrong callback method. - - (ADB - 2018/08/15, HDFFV-10536) - - - Java HDFLibraryException class - - Change parent class from Exception to RuntimeException. - - (ADB - 2018/07/30, HDFFV-10534) - - - JNI Read and Write - - Refactored variable-length functions, H5DreadVL and H5AreadVL, - to correct dataset and attribute reads. New write functions, - H5DwriteVL and H5AwriteVL, are under construction. - - (ADB - 2018/06/02, HDFFV-10519) - - -Supported Platforms -=================== - - Linux 2.6.32-696.16.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) - #1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) - (ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) - IBM XL C/C++ V13.1 - IBM XL Fortran V15.1 - - Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4) - Version 4.9.3, Version 5.2.0 - Intel(R) C (icc), C++ (icpc), Fortran (icc) - compilers: - Version 17.0.0.098 Build 20160721 - MPICH 3.1.4 compiled with GCC 4.9.3 - - SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc - (emu) Sun Fortran 95 8.6 SunOS_sparc - Sun C++ 5.12 SunOS_sparc - - Windows 7 Visual Studio 2015 w/ Intel Fortran 16 (cmake) - - Windows 7 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake) - Visual Studio 2013 w/ Intel Fortran 15 (cmake) - Visual Studio 2015 w/ Intel Fortran 16 (cmake) - Visual Studio 2015 w/ Intel C, Fortran 2017 (cmake) - Visual Studio 2015 w/ MSMPI 8 (cmake) - - Windows 10 Visual Studio 2015 w/ Intel Fortran 18 (cmake) - - Windows 10 x64 Visual Studio 2015 w/ Intel Fortran 18 (cmake) - Visual Studio 2017 w/ Intel Fortran 18 (cmake) - - Mac OS X Yosemite 10.10.5 Apple clang/clang++ version 6.1 from Xcode 7.0 - 64-bit gfortran GNU Fortran (GCC) 4.9.2 - (osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3 - - Mac OS X El Capitan 10.11.6 Apple clang/clang++ version 7.3.0 from Xcode 7.3 - 64-bit gfortran GNU Fortran (GCC) 5.2.0 - (osx1011dev/osx1011test) Intel icc/icpc/ifort version 16.0.2 - - Mac OS Sierra 10.12.6 Apple LLVM version 8.1.0 (clang/clang++-802.0.42) - 64-bit gfortran GNU Fortran (GCC) 7.1.0 - (kite) Intel icc/icpc/ifort version 17.0.2 - - -Tested Configuration Features Summary -===================================== - - In the tables below - y = tested - n = not tested in this release - C = Cluster - W = Workstation - x = not working in this release - dna = does not apply - ( ) = footnote appears below second table - <blank> = testing incomplete on this feature or platform - -Platform C F90/ F90 C++ zlib SZIP - parallel F2003 parallel -Solaris2.11 32-bit n y/y n y y y -Solaris2.11 64-bit n y/n n y y y -Windows 7 y y/y n y y y -Windows 7 x64 y y/y y y y y -Windows 7 Cygwin n y/n n y y y -Windows 7 x64 Cygwin n y/n n y y y -Windows 10 y y/y n y y y -Windows 10 x64 y y/y n y y y -Mac OS X Mavericks 10.9.5 64-bit n y/y n y y y -Mac OS X Yosemite 10.10.5 64-bit n y/y n y y y -Mac OS X El Capitan 10.11.6 64-bit n y/y n y y y -Mac OS Sierra 10.12.6 64-bit n y/y n y y y -CentOS 7.2 Linux 3.10.0 x86_64 PGI n y/y n y y y -CentOS 7.2 Linux 3.10.0 x86_64 GNU y y/y y y y y -CentOS 7.2 Linux 3.10.0 x86_64 Intel n y/y n y y y -Linux 2.6.32-573.18.1.el6.ppc64 n y/y n y y y - - -Platform Shared Shared Shared Thread- - C libs F90 libs C++ libs safe -Solaris2.11 32-bit y y y y -Solaris2.11 64-bit y y y y -Windows 7 y y y y -Windows 7 x64 y y y y -Windows 7 Cygwin n n n y -Windows 7 x64 Cygwin n n n y -Windows 10 y y y y -Windows 10 x64 y y y y -Mac OS X Mavericks 10.9.5 64-bit y n y y -Mac OS X Yosemite 10.10.5 64-bit y n y y -Mac OS X El Capitan 10.11.6 64-bit y n y y -Mac OS Sierra 10.12.6 64-bit y n y y -CentOS 7.2 Linux 3.10.0 x86_64 PGI y y y n -CentOS 7.2 Linux 3.10.0 x86_64 GNU y y y y -CentOS 7.2 Linux 3.10.0 x86_64 Intel y y y n -Linux 2.6.32-573.18.1.el6.ppc64 y y y n - -Compiler versions for each platform are listed in the preceding -"Supported Platforms" table. - - -More Tested Platforms -===================== -The following platforms are not supported but have been tested for this release. - - Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (mayll/platypus) Version 4.4.7 20120313 - Version 4.9.3, 5.3.0, 6.2.0 - PGI C, Fortran, C++ for 64-bit target on - x86-64; - Version 17.10-0 - Intel(R) C (icc), C++ (icpc), Fortran (icc) - compilers: - Version 17.0.4.196 Build 20170411 - MPICH 3.1.4 compiled with GCC 4.9.3 - - Linux 3.10.0-327.18.2.el7 GNU C (gcc) and C++ (g++) compilers - #1 SMP x86_64 GNU/Linux Version 4.8.5 20150623 (Red Hat 4.8.5-4) - (jelly) with NAG Fortran Compiler Release 6.1(Tozai) - GCC Version 7.1.0 - OpenMPI 3.0.0-GCC-7.2.0-2.29, - 3.1.0-GCC-7.2.0-2.29 - Intel(R) C (icc) and C++ (icpc) compilers - Version 17.0.0.098 Build 20160721 - with NAG Fortran Compiler Release 6.1(Tozai) - - Linux 3.10.0-327.10.1.el7 MPICH 3.2 compiled with GCC 5.3.0 - #1 SMP x86_64 GNU/Linux - (moohan) - - Linux 2.6.32-573.18.1.el6.ppc64 MPICH mpich 3.1.4 compiled with - #1 SMP ppc64 GNU/Linux IBM XL C/C++ for Linux, V13.1 - (ostrich) and IBM XL Fortran for Linux, V15.1 - - Debian 8.4 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1 x86_64 GNU/Linux - gcc, g++ (Debian 4.9.2-10) 4.9.2 - GNU Fortran (Debian 4.9.2-10) 4.9.2 - (cmake and autotools) - - Fedora 24 4.7.2-201.fc24.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux - gcc, g++ (GCC) 6.1.1 20160621 - (Red Hat 6.1.1-3) - GNU Fortran (GCC) 6.1.1 20160621 - (Red Hat 6.1.1-3) - (cmake and autotools) - - Ubuntu 16.04.1 4.4.0-38-generic #57-Ubuntu SMP x86_64 GNU/Linux - gcc, g++ (Ubuntu 5.4.0-6ubuntu1~16.04.2) - 5.4.0 20160609 - GNU Fortran (Ubuntu 5.4.0-6ubuntu1~16.04.2) - 5.4.0 20160609 - (cmake and autotools) - - -Known Problems -============== - - At present, metadata cache images may not be generated by parallel - applications. Parallel applications can read files with metadata cache - images, but since this is a collective operation, a deadlock is possible - if one or more processes do not participate. - - Three tests fail with OpenMPI 3.0.0/GCC-7.2.0-2.29: - testphdf5 (ecdsetw, selnone, cchunk1, cchunk3, cchunk4, and actualio) - t_shapesame (sscontig2) - t_pflush1/fails on exit - The first two tests fail attempting collective writes. - - Known problems in previous releases can be found in the HISTORY*.txt files - in the HDF5 source. Please report any new problems found to - help@hdfgroup.org. - - -CMake vs. Autotools installations -================================= -While both build systems produce similar results, there are differences. -Each system produces the same set of folders on linux (only CMake works -on standard Windows); bin, include, lib and share. Autotools places the -COPYING and RELEASE.txt file in the root folder, CMake places them in -the share folder. - -The bin folder contains the tools and the build scripts. Additionally, CMake -creates dynamic versions of the tools with the suffix "-shared". Autotools -installs one set of tools depending on the "--enable-shared" configuration -option. - build scripts - ------------- - Autotools: h5c++, h5cc, h5fc - CMake: h5c++, h5cc, h5hlc++, h5hlcc - -The include folder holds the header files and the fortran mod files. CMake -places the fortran mod files into separate shared and static subfolders, -while Autotools places one set of mod files into the include folder. Because -CMake produces a tools library, the header files for tools will appear in -the include folder. - -The lib folder contains the library files, and CMake adds the pkgconfig -subfolder with the hdf5*.pc files used by the bin/build scripts created by -the CMake build. CMake separates the C interface code from the fortran code by -creating C-stub libraries for each Fortran library. In addition, only CMake -installs the tools library. The names of the szip libraries are different -between the build systems. - -The share folder will have the most differences because CMake builds include -a number of CMake specific files for support of CMake's find_package and support -for the HDF5 Examples CMake project. - -%%%%1.10.3%%%% - -HDF5 version 1.10.3 released on 2018-08-21 -================================================================================ - - -INTRODUCTION - -This document describes the differences between this release and the previous -HDF5 release. It contains information on the platforms tested and known -problems in this release. For more details check the HISTORY*.txt files in the -HDF5 source. - -Note that documentation in the links below will be updated at the time of each -final release. - -Links to HDF5 documentation can be found on The HDF5 web page: - - https://portal.hdfgroup.org/display/HDF5/HDF5 - -The official HDF5 releases can be obtained from: - - https://www.hdfgroup.org/downloads/hdf5/ - -Changes from Release to Release and New Features in the HDF5-1.10.x release series -can be found at: - - https://portal.hdfgroup.org/display/HDF5/HDF5+Application+Developer%27s+Guide - -If you have any questions or comments, please send them to the HDF Help Desk: - - help@hdfgroup.org - - -CONTENTS - -- New Features -- Bug Fixes since HDF5-1.10.2 -- Supported Platforms -- Tested Configuration Features Summary -- More Tested Platforms -- Known Problems -- CMake vs. Autotools installations - - -New Features -============ - - Library - ------- - - Moved the H5DOread/write_chunk() API calls to H5Dread/write_chunk() - - The functionality of the direct chunk I/O calls in the high-level - library has been moved to the H5D package in the main library. This - will allow using those functions without building the high-level - library. The parameters and functionality of the H5D calls are - identical to the H5DO calls. - - The original H5DO high-level API calls have been retained, though - they are now just wrappers for the H5D calls. They are marked as - deprecated and are only available when the library is built with - deprecated functions. New code should use the H5D calls for this - reason. - - As a part of this work, the following symbols from H5Dpublic.h are no - longer used: - - H5D_XFER_DIRECT_CHUNK_WRITE_FLAG_NAME - H5D_XFER_DIRECT_CHUNK_WRITE_FILTERS_NAME - H5D_XFER_DIRECT_CHUNK_WRITE_OFFSET_NAME - H5D_XFER_DIRECT_CHUNK_WRITE_DATASIZE_NAME - H5D_XFER_DIRECT_CHUNK_READ_FLAG_NAME - H5D_XFER_DIRECT_CHUNK_READ_OFFSET_NAME - H5D_XFER_DIRECT_CHUNK_READ_FILTERS_NAME - - And properties with these names are no longer stored in the dataset - transfer property lists. The symbols are still defined in H5Dpublic.h, - but only when the library is built with deprecated symbols. - - (DER - 2018/05/04) - - Configuration: - ------------- - - Add missing USE_110_API_DEFAULT option. - - Option USE_110_API_DEFAULT sets the default version of - versioned APIs. The bin/makevers perl script did not set - the maxidx variable correctly when the 1.10 branch was - created. This caused the versioning process to always use - the latest version of any API. - - (ADB - 2018/08/17, HDFFV-10552) - - - Added configuration checks for the following MPI functions: - - MPI_Mprobe - Used for the Parallel Compression feature - MPI_Imrecv - Used for the Parallel Compression feature - - MPI_Get_elements_x - Used for the "big Parallel I/O" feature - MPI_Type_size_x - Used for the "big Parallel I/O" feature - - (JTH - 2018/08/02, HDFFV-10512) - - - Added section to the libhdf5.settings file to indicate - the status of the Parallel Compression and "big Parallel I/O" - features. - - (JTH - 2018/08/02, HDFFV-10512) - - - Add option to execute swmr shell scripts from CMake. - - Option TEST_SHELL_SCRIPTS redirects processing into a - separate ShellTests.cmake file for UNIX types. The tests - execute the shell scripts if a SH program is found. - - (ADB - 2018/07/16) - - - C++ Library: - ------------ - - New wrappers - - Added the following items: - - + Class DSetAccPropList for the dataset access property list. - - + Wrapper for H5Dget_access_plist to class DataSet - // Gets the access property list of this dataset. - DSetAccPropList getAccessPlist() const; - - + Wrappers for H5Pset_chunk_cache and H5Pget_chunk_cache to class DSetAccPropList - // Sets the raw data chunk cache parameters. - void setChunkCache(size_t rdcc_nslots, size_t rdcc_nbytes, double rdcc_w0) - - // Retrieves the raw data chunk cache parameters. - void getChunkCache(size_t &rdcc_nslots, size_t &rdcc_nbytes, double &rdcc_w0) - - + New operator!= to class DataType (HDFFV-10472) - // Determines whether two datatypes are not the same. - bool operator!=(const DataType& compared_type) - - + Wrappers for H5Oget_info2, H5Oget_info_by_name2, and H5Oget_info_by_idx2 - (HDFFV-10458) - - // Retrieves information about an HDF5 object. - void getObjinfo(H5O_info_t& objinfo, unsigned fields = H5O_INFO_BASIC) const; - - // Retrieves information about an HDF5 object, given its name. - void getObjinfo(const char* name, H5O_info_t& objinfo, - unsigned fields = H5O_INFO_BASIC, - const LinkAccPropList& lapl = LinkAccPropList::DEFAULT) const; - void getObjinfo(const H5std_string& name, H5O_info_t& objinfo, - unsigned fields = H5O_INFO_BASIC, - const LinkAccPropList& lapl = LinkAccPropList::DEFAULT) const; - - // Retrieves information about an HDF5 object, given its index. - void getObjinfo(const char* grp_name, H5_index_t idx_type, - H5_iter_order_t order, hsize_t idx, H5O_info_t& objinfo, - unsigned fields = H5O_INFO_BASIC, - const LinkAccPropList& lapl = LinkAccPropList::DEFAULT) const; - void getObjinfo(const H5std_string& grp_name, H5_index_t idx_type, - H5_iter_order_t order, hsize_t idx, H5O_info_t& objinfo, - unsigned fields = H5O_INFO_BASIC, - const LinkAccPropList& lapl = LinkAccPropList::DEFAULT) const; - - (BMR - 2018/07/22, HDFFV-10150, HDFFV-10458, HDFFV-1047) - - - Java Library: - ---------------- - - Java HDFLibraryException class - - Change parent class from Exception to RuntimeException. - - (ADB - 2018/07/30, HDFFV-10534) - - - JNI Read and Write - - Refactored variable-length functions, H5DreadVL and H5AreadVL, - to correct dataset and attribute reads. New write functions, - H5DwriteVL and H5AwriteVL, are under construction. - - (ADB - 2018/06/02, HDFFV-10519) - - -Bug Fixes since HDF5-1.10.2 release -================================== - - Library - ------- - - Performance issue with H5Oget_info - - H5Oget_info family of routines retrieves information for an object such - as object type, access time, number of attributes, and storage space etc. - Retrieving all such information regardless is an overkill and causes - performance issue when doing so for many objects. - - Add an additional parameter "fields" to the the H5Oget_info family of routines - indicating the type of information to be retrieved. The same is done to - the H5Ovisit family of routines which recursively visits an object - returning object information in a callback function. Both sets of routines - are versioned and the corresponding compatibility macros are added. - - The version 2 names of the two sets of routines are: - (1) H5Oget_info2, H5Oget_info_by_idx2, H5Oget_info_by_name2 - (2) H5Ovisit2, H5Ovisit_by_name2 - - (VC - 2018/08/15, HDFFV-10180) - - - Test failure due to metadata size in test/vds.c - - The size of metadata from test_api_get_ex_dcpl() in test/vds.c is not as expected - because the latest format should be used when encoding the layout for VDS. - - Set the latest format in a temporary fapl and pass the setting to the routines that - encode the dataset selection for VDS. - - (VC - 2018/08/14 HDFFV-10469) - - - Java HDF5LibraryException class - - The error minor and major values would be lost after the - constructor executed. - - Created two local class variables to hold the values obtained during - execution of the constructor. Refactored the class functions to retrieve - the class values rather then calling the native functions. - The native functions were renamed and called only during execution - of the constructor. - Added error checking to calling class constructors in JNI classes. - - (ADB - 2018/08/06, HDFFV-10544) - - - Added checks of the defined MPI_VERSION to guard against usage of - MPI-3 functions in the Parallel Compression and "big Parallel I/O" - features when HDF5 is built with MPI-2. Previously, the configure - step would pass but the build itself would fail when it could not - locate the MPI-3 functions used. - - As a result of these new checks, HDF5 can again be built with MPI-2, - but the Parallel Compression feature will be disabled as it relies - on the MPI-3 functions used. - - (JTH - 2018/08/02, HDFFV-10512) - - - User's patches: CVEs - - The following patches have been applied: - - CVE-2018-11202 - NULL pointer dereference was discovered in - H5S_hyper_make_spans in H5Shyper.c (HDFFV-10476) - https://security-tracker.debian.org/tracker/CVE-2018-11202 - https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2018-11202 - - CVE-2018-11203 - A division by zero was discovered in - H5D__btree_decode_key in H5Dbtree.c (HDFFV-10477) - https://security-tracker.debian.org/tracker/CVE-2018-11203 - https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2018-11203 - - CVE-2018-11204 - A NULL pointer dereference was discovered in - H5O__chunk_deserialize in H5Ocache.c (HDFFV-10478) - https://security-tracker.debian.org/tracker/CVE-2018-11204 - https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2018-11204 - - CVE-2018-11206 - An out of bound read was discovered in - H5O_fill_new_decode and H5O_fill_old_decode in H5Ofill.c - (HDFFV-10480) - https://security-tracker.debian.org/tracker/CVE-2018-11206 - https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2018-11206 - - CVE-2018-11207 - A division by zero was discovered in - H5D__chunk_init in H5Dchunk.c (HDFFV-10481) - https://security-tracker.debian.org/tracker/CVE-2018-11207 - https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2018-11207 - - (BMR - 2018/7/22, PR#s: 1134 and 1139, - HDFFV-10476, HDFFV-10477, HDFFV-10478, HDFFV-10480, HDFFV-10481) - - - H5Adelete - - H5Adelete failed when deleting the last "large" attribute that - is stored densely via fractal heap/v2 b-tree. - - After removing the attribute, update the ainfo message. If the - number of attributes goes to zero, remove the message. - - (VC - 2018/07/20, HDFFV-9277) - - - A bug was discovered in the parallel library which caused partial - parallel reads of filtered datasets to return incorrect data. The - library used the incorrect dataspace for each chunk read, causing - the selection used in each chunk to be wrong. - - The bug was not caught during testing because all of the current - tests which do parallel reads of filtered data read all of the data - using an H5S_ALL selection. Several tests were added which exercise - partial parallel reads. - - (JTH - 2018/07/16, HDFFV-10467) - - - A bug was discovered in the parallel library which caused parallel - writes of filtered datasets to trigger an assertion failure in the - file free space manager. - - This occurred when the filter used caused chunks to repeatedly shrink - and grow over the course of several dataset writes. The previous chunk - information, such as the size of the chunk and the offset in the file, - was being cached and not updated after each write, causing the next write - to the chunk to retrieve the incorrect cached information and run into - issues when reallocating space in the file for the chunk. - - (JTH - 2018/07/16, HDFFV-10509) - - - A bug was discovered in the parallel library which caused the - H5D__mpio_array_gatherv() function to allocate too much memory. - - When the function is called with the 'allgather' parameter set - to a non-true value, the function will receive data from all MPI - ranks and gather it to the single rank specified by the 'root' - parameter. However, the bug in the function caused memory for - the received data to be allocated on all MPI ranks, not just the - singular rank specified as the receiver. In some circumstances, - this would cause an application to fail due to the large amounts - of memory being allocated. - - (JTH - 2018/07/16, HDFFV-10467) - - - Error checks in h5stat and when decoding messages - - h5stat exited with seg fault/core dumped when - errors are encountered in the internal library. - - Add error checks and --enable-error-stack option to h5stat. - Add range checks when decoding messages: old fill value, old - layout and refcount. - - (VC - 2018/07/11, HDFFV-10333) - - - If an HDF5 file contains a malformed compound datatype with a - suitably large offset, the type conversion code can run off - the end of the type conversion buffer, causing a segmentation - fault. - - This issue was reported to The HDF Group as issue #CVE-2017-17507. - - NOTE: The HDF5 C library cannot produce such a file. This condition - should only occur in a corrupt (or deliberately altered) file - or a file created by third-party software. - - THE HDF GROUP WILL NOT FIX THIS BUG AT THIS TIME - - Fixing this problem would involve updating the publicly visible - H5T_conv_t function pointer typedef and versioning the API calls - which use it. We normally only modify the public API during - major releases, so this bug will not be fixed at this time. - - (DER - 2018/02/26, HDFFV-10356) - - - Configuration - ------------- - - Applied patches to address Cywin build issues - - There were three issues for Cygwin builds: - - Shared libs were not built. - - The -std=c99 flag caused a SIG_SETMASK undeclared error. - - Undefined errors when buildbing test shared libraries. - - Patches to address these issues were received and incorporated in this version. - - (LRK - 2018/07/18, HDFFV-10475) - - - The --enable-debug/production configure flags are listed as 'deprecated' - when they should really be listed as 'removed'. - - In the autotools overhaul several years ago, we removed these flags and - implemented a new --enable-build-mode= flag. This was done because we - changed the semantics of the modes and didn't want users to silently - be exposed to them. The newer system is also more flexible and us to - add other modes (like 'clean'). - - The --enable-debug/production flags are now listed as removed. - - (DER - 2018/05/31, HDFFV-10505) - - - Moved the location of gcc attribute. - - The gcc attribute(no_sanitize), named as the macro HDF_NO_UBSAN, - was located after the function name. Builds with GCC 7 did not - indicate any problem, but GCC 8 issued errors. Moved the - attribute before the function name, as required. - - (ADB - 2018/05/22, HDFFV-10473) - - - Reworked java test suite into individual JUnit tests. - - Testing the whole suite of java unit tests in a single JUnit run - made it difficult to determine actual failures when tests would fail. - Running each file set of tests individually, allows individual failures - to be diagnosed easier. A side benefit is that tests for optional components - of the library can be disabled if not configured. - - (ADB - 2018/05/16, HDFFV-9739) - - - Converted CMake global commands ADD_DEFINITIONS and INCLUDE_DIRECTORIES - to use target_* type commands. This change modernizes the CMake usage - in the HDF5 library. - - In addition, there is the intention to convert to generator expressions, - where possible. The exception is Fortran FLAGS on Windows Visual Studio. - The HDF macros TARGET_C_PROPERTIES and TARGET_FORTRAN_PROPERTIES have - been removed with this change in usage. - - The additional language (C++ and Fortran) checks have also been localized - to only be checked when that language is enabled. - - (ADB - 2018/05/08) - - - Performance - ------------- - - Revamped internal use of DXPLs, improving performance - - (QAK - 2018/05/20) - - - Fortran - -------- - - Fixed issue with h5fget_obj_count_f and using a file id of H5F_OBJ_ALL_F not - returning the correct count. - - (MSB - 2018/5/15, HDFFV-10405) - - - C++ APIs - -------- - - Adding default arguments to existing functions - - Added the following items: - + Two more property list arguments are added to H5Location::createDataSet: - const DSetAccPropList& dapl = DSetAccPropList::DEFAULT - const LinkCreatPropList& lcpl = LinkCreatPropList::DEFAULT - - + One more property list argument is added to H5Location::openDataSet: - const DSetAccPropList& dapl = DSetAccPropList::DEFAULT - - (BMR - 2018/07/21, PR# 1146) - - - Improvement C++ documentation - - Replaced the table in main page of the C++ documentation from mht to htm format - for portability. - - (BMR - 2018/07/17, PR# 1141) - - -Supported Platforms -=================== - - Linux 2.6.32-696.16.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) - #1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) - (ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) - IBM XL C/C++ V13.1 - IBM XL Fortran V15.1 - - Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4) - Version 4.9.3, Version 5.2.0 - Intel(R) C (icc), C++ (icpc), Fortran (icc) - compilers: - Version 17.0.0.098 Build 20160721 - MPICH 3.1.4 compiled with GCC 4.9.3 - - SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc - (emu) Sun Fortran 95 8.6 SunOS_sparc - Sun C++ 5.12 SunOS_sparc - - Windows 7 Visual Studio 2015 w/ Intel Fortran 16 (cmake) - - Windows 7 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake) - Visual Studio 2013 w/ Intel Fortran 15 (cmake) - Visual Studio 2015 w/ Intel Fortran 16 (cmake) - Visual Studio 2015 w/ Intel C, Fortran 2017 (cmake) - Visual Studio 2015 w/ MSMPI 8 (cmake) - - Windows 10 Visual Studio 2015 w/ Intel Fortran 18 (cmake) - - Windows 10 x64 Visual Studio 2015 w/ Intel Fortran 18 (cmake) - Visual Studio 2017 w/ Intel Fortran 18 (cmake) - - Mac OS X Yosemite 10.10.5 Apple clang/clang++ version 6.1 from Xcode 7.0 - 64-bit gfortran GNU Fortran (GCC) 4.9.2 - (osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3 - - Mac OS X El Capitan 10.11.6 Apple clang/clang++ version 7.3.0 from Xcode 7.3 - 64-bit gfortran GNU Fortran (GCC) 5.2.0 - (osx1011dev/osx1011test) Intel icc/icpc/ifort version 16.0.2 - - Mac OS Sierra 10.12.6 Apple LLVM version 8.1.0 (clang/clang++-802.0.42) - 64-bit gfortran GNU Fortran (GCC) 7.1.0 - (swallow/kite) Intel icc/icpc/ifort version 17.0.2 - -Tested Configuration Features Summary -===================================== - - In the tables below - y = tested - n = not tested in this release - C = Cluster - W = Workstation - x = not working in this release - dna = does not apply - ( ) = footnote appears below second table - <blank> = testing incomplete on this feature or platform - -Platform C F90/ F90 C++ zlib SZIP - parallel F2003 parallel -Solaris2.11 32-bit n y/y n y y y -Solaris2.11 64-bit n y/n n y y y -Windows 7 y y/y n y y y -Windows 7 x64 y y/y y y y y -Windows 7 Cygwin n y/n n y y y -Windows 7 x64 Cygwin n y/n n y y y -Windows 10 y y/y n y y y -Windows 10 x64 y y/y n y y y -Mac OS X Mavericks 10.9.5 64-bit n y/y n y y y -Mac OS X Yosemite 10.10.5 64-bit n y/y n y y y -Mac OS X El Capitan 10.11.6 64-bit n y/y n y y y -Mac OS Sierra 10.12.6 64-bit n y/y n y y y -CentOS 7.2 Linux 2.6.32 x86_64 PGI n y/y n y y y -CentOS 7.2 Linux 2.6.32 x86_64 GNU y y/y y y y y -CentOS 7.2 Linux 2.6.32 x86_64 Intel n y/y n y y y -Linux 2.6.32-573.18.1.el6.ppc64 n y/y n y y y - - -Platform Shared Shared Shared Thread- - C libs F90 libs C++ libs safe -Solaris2.11 32-bit y y y y -Solaris2.11 64-bit y y y y -Windows 7 y y y y -Windows 7 x64 y y y y -Windows 7 Cygwin n n n y -Windows 7 x64 Cygwin n n n y -Windows 10 y y y y -Windows 10 x64 y y y y -Mac OS X Mavericks 10.9.5 64-bit y n y y -Mac OS X Yosemite 10.10.5 64-bit y n y y -Mac OS X El Capitan 10.11.6 64-bit y n y y -Mac OS Sierra 10.12.6 64-bit y n y y -CentOS 7.2 Linux 2.6.32 x86_64 PGI y y y n -CentOS 7.2 Linux 2.6.32 x86_64 GNU y y y y -CentOS 7.2 Linux 2.6.32 x86_64 Intel y y y n -Linux 2.6.32-573.18.1.el6.ppc64 y y y n - -Compiler versions for each platform are listed in the preceding -"Supported Platforms" table. - - -More Tested Platforms -===================== -The following platforms are not supported but have been tested for this release. - - Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (mayll/platypus) Version 4.4.7 20120313 - Version 4.9.3, 5.3.0, 6.2.0 - PGI C, Fortran, C++ for 64-bit target on - x86-64; - Version 17.10-0 - Intel(R) C (icc), C++ (icpc), Fortran (icc) - compilers: - Version 17.0.4.196 Build 20170411 - MPICH 3.1.4 compiled with GCC 4.9.3 - - Linux 3.10.0-327.18.2.el7 GNU C (gcc) and C++ (g++) compilers - #1 SMP x86_64 GNU/Linux Version 4.8.5 20150623 (Red Hat 4.8.5-4) - (jelly) with NAG Fortran Compiler Release 6.1(Tozai) - GCC Version 7.1.0 - OpenMPI 3.0.0-GCC-7.2.0-2.29, - 3.1.0-GCC-7.2.0-2.29 - Intel(R) C (icc) and C++ (icpc) compilers - Version 17.0.0.098 Build 20160721 - with NAG Fortran Compiler Release 6.1(Tozai) - - Linux 3.10.0-327.10.1.el7 MPICH 3.2 compiled with GCC 5.3.0 - #1 SMP x86_64 GNU/Linux - (moohan) - - Linux 2.6.32-573.18.1.el6.ppc64 MPICH mpich 3.1.4 compiled with - #1 SMP ppc64 GNU/Linux IBM XL C/C++ for Linux, V13.1 - (ostrich) and IBM XL Fortran for Linux, V15.1 - - Debian 8.4 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1 x86_64 GNU/Linux - gcc, g++ (Debian 4.9.2-10) 4.9.2 - GNU Fortran (Debian 4.9.2-10) 4.9.2 - (cmake and autotools) - - Fedora 24 4.7.2-201.fc24.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux - gcc, g++ (GCC) 6.1.1 20160621 - (Red Hat 6.1.1-3) - GNU Fortran (GCC) 6.1.1 20160621 - (Red Hat 6.1.1-3) - (cmake and autotools) - - Ubuntu 16.04.1 4.4.0-38-generic #57-Ubuntu SMP x86_64 GNU/Linux - gcc, g++ (Ubuntu 5.4.0-6ubuntu1~16.04.2) - 5.4.0 20160609 - GNU Fortran (Ubuntu 5.4.0-6ubuntu1~16.04.2) - 5.4.0 20160609 - (cmake and autotools) - - -Known Problems -============== - - At present, metadata cache images may not be generated by parallel - applications. Parallel applications can read files with metadata cache - images, but since this is a collective operation, a deadlock is possible - if one or more processes do not participate. - - Three tests fail with OpenMPI 3.0.0/GCC-7.2.0-2.29: - testphdf5 (ecdsetw, selnone, cchunk1, cchunk3, cchunk4, and actualio) - t_shapesame (sscontig2) - t_pflush1/fails on exit - The first two tests fail attempting collective writes. - - Known problems in previous releases can be found in the HISTORY*.txt files - in the HDF5 source. Please report any new problems found to - help@hdfgroup.org. - - -CMake vs. Autotools installations -================================= -While both build systems produce similar results, there are differences. -Each system produces the same set of folders on linux (only CMake works -on standard Windows); bin, include, lib and share. Autotools places the -COPYING and RELEASE.txt file in the root folder, CMake places them in -the share folder. - -The bin folder contains the tools and the build scripts. Additionally, CMake -creates dynamic versions of the tools with the suffix "-shared". Autotools -installs one set of tools depending on the "--enable-shared" configuration -option. - build scripts - ------------- - Autotools: h5c++, h5cc, h5fc - CMake: h5c++, h5cc, h5hlc++, h5hlcc - -The include folder holds the header files and the fortran mod files. CMake -places the fortran mod files into separate shared and static subfolders, -while Autotools places one set of mod files into the include folder. Because -CMake produces a tools library, the header files for tools will appear in -the include folder. - -The lib folder contains the library files, and CMake adds the pkgconfig -subfolder with the hdf5*.pc files used by the bin/build scripts created by -the CMake build. CMake separates the C interface code from the fortran code by -creating C-stub libraries for each Fortran library. In addition, only CMake -installs the tools library. The names of the szip libraries are different -between the build systems. - -The share folder will have the most differences because CMake builds include -a number of CMake specific files for support of CMake's find_package and support -for the HDF5 Examples CMake project. - - -%%%%1.10.2%%%% - -HDF5 version 1.10.2 released on 2018-03-29 -================================================================================ - - -INTRODUCTION - -This document describes the differences between this release and the previous -HDF5 release. It contains information on the platforms tested and known -problems in this release. For more details check the HISTORY*.txt files in the -HDF5 source. - -Note that documentation in the links below will be updated at the time of each -final release. - -Links to HDF5 documentation can be found on The HDF5 web page: - - https://portal.hdfgroup.org/display/HDF5/HDF5 - -The official HDF5 releases can be obtained from: - - https://www.hdfgroup.org/downloads/hdf5/ - -Changes from Release to Release and New Features in the HDF5-1.10.x release series -can be found at: - - https://portal.hdfgroup.org/display/HDF5/HDF5+Application+Developer%27s+Guide - -If you have any questions or comments, please send them to the HDF Help Desk: - - help@hdfgroup.org - - -CONTENTS - -- New Features -- Support for new platforms and languages -- Bug Fixes since HDF5-1.10.1 -- Supported Platforms -- Tested Configuration Features Summary -- More Tested Platforms -- Known Problems - - -New Features -============ - - Configuration and Build Systems: - -------------------------------- - - CMake builds - -------------- - - - Changed minimum CMake required version to 3.10. - - This change removed the need to support a copy of the FindMPI.cmake module, - which has been removed, along with its subfolder in the config/cmake_ext_mod - location. - - (ADB - 2018/03/09) - - - Added pkg-config file generation - - Added pkg-config file generation for the C, C++, HL, and HL C++ libraries. - In addition, builds on Linux will create h5cc, h5c++, h5hlcc, and h5hlc++ scripts in the bin - directory that use the pkg-config files. The scripts can be used to build HDF5 C and C++ - applications (i.e, similar to the compiler scripts produced by the Autotools builds). - - (ADB - 2018/03/08, HDFFV-4359) - - - Refactored use of CMAKE_BUILD_TYPE for new variable, which understands - the type of generator in use. - - Added new configuration macros to use new HDF_BUILD_TYPE variable. This - variable is set correctly for the type of generator being used for the build. - - (ADB - 2018/01/08, HDFFV-10385, HDFFV-10296) - - - Autotools builds - ------------------ - - - Removed version-specific gcc/gfortran flags for version 4.0 (inclusive) - and earlier. - - The config/gnu-flags file, which is sourced as a part of the configure - process, adds version-specific flags for use when building HDF5. Most of - these flags control warnings and do not affect the final product. - - Flags for older versions of the compiler were consolidated into the - common flags section. Moving these flags simplifies maintenance of - the file. - - The upshot of this is that building with ancient versions of gcc - (<= 4.0) will possibly no longer work without hand-hacking the file - to remove the flags not understood by that version of the compiler. - Nothing should change when building with gcc >= 4.1. - - (DER - 2017/05/31, HDFFV-9937) - - - -fno-omit-frame-pointer was added when building with debugging symbols - enabled. - - Debugging symbols can be enabled independently of the overall build - mode in both the autotools and CMake. This allows (limited) debugging - of optimized code. Since many debuggers rely on the frame pointer, - we've disabled this optimization when debugging symbols are requested - (e.g.: via building with --enable-symbols). - - (DER - 2017/05/31, HDFFV-10226) - - - Library: - -------- - - Added an enumerated value to H5F_libver_t for H5Pset_libver_bounds(). - - Currently, the library defines two values for H5F_libver_t and supports - only two pairs of (low, high) combinations as derived from these values. - Thus the bounds setting via H5Pset_libver_bounds() is rather restricted. - - Added an enumerated value (H5F_LIBVER_V18) to H5F_libver_t and - H5Pset_libver_bounds() now supports five pairs of (low, high) combinations - as derived from these values. This addition provides the user more - flexibility in setting bounds for object creation. - - (VC - 2018/03/14) - - - Added prefix option to VDS files. - - Currently, VDS source files must be in the active directory to be - found by the virtual file. Adding the option of a prefix to be set - on the virtual file, using a data access property list (DAPL), - allows the source files to locate at an absolute or relative path - to the virtual file. - Private utility functions in H5D and H5L packages merged into single - function in H5F package. - - New public APIs: - herr_t H5Pset_virtual_prefix(hid_t dapl_id, const char* prefix); - ssize_t H5Pget_virtual_prefix(hid_t dapl_id, char* prefix /*out*/, size_t size); - The prefix can also be set with an environment variable, HDF5_VDS_PREFIX. - - (ADB - 2017/12/12, HDFFV-9724, HDFFV-10361) - - - H5FDdriver_query() API call added to the C library. - - This new library call allows the user to query a virtual file driver - (VFD) for the feature flags it supports (listed in H5FDpublic.h). - This can be useful to determine if a VFD supports SWMR, for example. - - Note that some VFDs have feature flags that may only be present - after a file has been created or opened (e.g.: the core VFD will - have the H5FD_FEAT_POSIX_COMPAT_HANDLE flag set if the backing - store is switched on). Since the new API call queries a generic VFD - unassociated with a file, these flags will never be returned. - - (DER - 2017/05/31, HDFFV-10215) - - - H5FD_FEAT_DEFAULT_VFD_COMPATIBLE VFD feature flag added to the C library. - - This new feature flag indicates that the VFD is compatible with the - default VFD. VFDs that set this flag create single files that follow - the canonical HDF5 file format. - - (DER - 2017/05/31, HDFFV-10214) - - - The H5I_REFERENCE value in the H5I_type_t enum (defined in H5Ipublic.h) - has been marked as deprecated. - - This ID type value is not used in the C library. i.e.: There are no - hid_t values that are of ID type H5I_REFERENCE. - - This enum value will be removed in a future major version of the library. - The code will remain unchanged in the HDF5 1.10.x releases and branches. - - (DER - 2017/04/05, HDFFV-10252) - - - Parallel Library: - ----------------- - - Enabled compression for parallel applications. - - With this release parallel applications can create and write compressed - datasets (or the datasets with the filters such as Fletcher32 applied). - - (EIP - 2018/03/29) - - - Addressed slow file close on some Lustre file systems. - - Slow file close has been reported on some Lustre file systems. - While the ultimate cause is not understood fully, the proximate - cause appears to be long delays in MPI_File_set_size() calls at - file close and flush. - - To minimize this problem pending a definitive diagnosis and fix, - PHDF5 has been modified to avoid MPI_File_set_size() calls when - possible. This is done by comparing the library's EOA (End of - Allocation) with the file systems EOF, and skipping the - MPI_File_set_size() call if the two match. - - (JRM - 2018/03/29) - - - Optimized parallel open/location of the HDF5 super-block. - - Previous releases of PHDF5 required all parallel ranks to - search for the HDF5 superblock signature when opening the - file. As this is accomplished more or less as a synchronous - operation, a large number of processes can experience a - slowdown in the file open due to filesystem contention. - - As a first step in improving the startup/file-open performance, - we allow MPI rank 0 of the associated MPI communicator to locate - the base offset of the super-block and then broadcast that result - to the remaining ranks in the parallel group. Note that this - approach is utilized ONLY during file opens which employ the MPIO - file driver in HDF5 by previously having called H5Pset_fapl_mpio(). - - HDF5 parallel file operations which do not employ multiple ranks - e.g. specifying MPI_COMM_SELF (whose MPI_Comm_size == 1) - as opposed to MPI_COMM_WORLD, will not be affected by this - optimization. Conversely, parallel file operations on subgroups - of MPI_COMM_WORLD are allowed to be run in parallel with each - subgroup operating as an independent collection of processes. - - (RAW - 2017/10/10, HDFFV-10294) - - - Added large (>2GB) MPI-IO transfers. - - Previous releases of PHDF5 would fail when attempting to - read or write greater than 2GB of data in a single IO operation. - This issue stems principally from an MPI API whose definitions - utilize 32 bit integers to describe the number of data elements - and datatype that MPI should use to effect a data transfer. - Historically, HDF5 has invoked MPI-IO with the number of - elements in a contiguous buffer represented as the length - of that buffer in bytes. - - Resolving the issue and thus enabling larger MPI-IO transfers - is accomplished first, by detecting when a user IO request would - exceed the 2GB limit as described above. Once a transfer request - is identified as requiring special handling, PHDF5 now creates a - derived datatype consisting of a vector of fixed sized blocks - which is in turn wrapped within a single MPI_Type_struct to - contain the vector and any remaining data. The newly created - datatype is then used in place of MPI_BYTE and can be used to - fulfill the original user request without encountering API - errors. - - (RAW - 2017/09/10, HDFFV-8839) - - - C++ Library: - ------------ - - The following C++ API wrappers have been added to the C++ Library: - + H5Lcreate_soft: - // Creates a soft link from link_name to target_name. - void link(const char *target_name, const char *link_name,...) - void link(const H5std_string& target_name,...) - - + H5Lcreate_hard: - // Creates a hard link from new_name to curr_name. - void link(const char *curr_name, const Group& new_loc,...) - void link(const H5std_string& curr_name, const Group& new_loc,...) - - // Creates a hard link from new_name to curr_name in same location. - void link(const char *curr_name, const hid_t same_loc,...) - void link(const H5std_string& curr_name, const hid_t same_loc,...) - - Note: previous version of H5Location::link will be deprecated. - - + H5Lcopy: - // Copy an object from a group of file to another. - void copyLink(const char *src_name, const Group& dst,...) - void copyLink(const H5std_string& src_name, const Group& dst,...) - - // Copy an object from a group of file to the same location. - void copyLink(const char *src_name, const char *dst_name,...) - void copyLink(const H5std_string& src_name,...) - - + H5Lmove: - // Rename an object in a group or file to a new location. - void moveLink(const char* src_name, const Group& dst,...) - void moveLink(const H5std_string& src_name, const Group& dst,...) - - // Rename an object in a group or file to the same location. - void moveLink(const char* src_name, const char* dst_name,...) - void moveLink(const H5std_string& src_name,...) - - Note: previous version H5Location::move will be deprecated. - - + H5Ldelete: - // Removes the specified link from this location. - void unlink(const char *link_name, - const LinkAccPropList& lapl = LinkAccPropList::DEFAULT) - void unlink(const H5std_string& link_name, - const LinkAccPropList& lapl = LinkAccPropList::DEFAULT) - - Note: additional parameter is added to previous H5Location::unlink. - - + H5Tencode and H5Tdecode: - // Creates a binary object description of this datatype. - void DataType::encode() - C API H5Tencode() - - // Returns the decoded type from the binary object description. - DataType::decode() - C API H5Tdecode() - ArrayType::decode() - C API H5Tdecode() - CompType::decode() - C API H5Tdecode() - DataType::decode() - C API H5Tdecode() - EnumType::decode() - C API H5Tdecode() - FloatType::decode() - C API H5Tdecode() - IntType::decode() - C API H5Tdecode() - StrType::decode() - C API H5Tdecode() - VarLenType::decode() - C API H5Tdecode() - - + H5Lget_info: - // Returns the information of the named link. - H5L_info_t getLinkInfo(const H5std_string& link_name,...) - - (BMR - 2018/03/11, HDFFV-10149) - - - Added class LinkCreatPropList for link create property list. - - (BMR - 2018/03/11, HDFFV-10149) - - - Added overloaded functions H5Location::createGroup to take a link - creation property list. - Group createGroup(const char* name, const LinkCreatPropList& lcpl) - Group createGroup(const H5std_string& name, const LinkCreatPropList& lcpl) - - (BMR - 2018/03/11, HDFFV-10149) - - - A document is added to the HDF5 C++ API Reference Manual to show the - mapping from a C API to C++ wrappers. It can be found from the main - page of the C++ API Reference Manual. - - (BMR - 2017/10/17, HDFFV-10151) - - - Java Library: - ---------------- - - Wrapper added for enabling the error stack. - - H5error_off would disable the error stack reporting. In order - to re-enable the reporting, the error stack info needs to be - saved so that H5error_on can revert state. - - (ADB - 2018/03/13, HDFFV-10412) - - - Wrappers were added for the following C APIs: - H5Pset_evict_on_close - H5Pget_evict_on_close - H5Pset_chunk_opts - H5Pget_chunk_opts - H5Pset_efile_prefix - H5Pget_efile_prefix - H5Pset_virtual_prefix - H5Pget_virtual_prefix - - (ADB - 2017/12/20) - - - The H5I_REFERENCE value in the H5I_type_t enum (defined in H5Ipublic.h) - has been marked as deprecated. - - JNI code which refers to this value will be removed in a future - major version of the library. The code will remain unchanged in the - 1.10.x releases and branches. - - See the C library section, above, for further information. - - (HDFFV-10252, DER, 2017/04/05) - - - Tools: - ------ - - h5diff has a new option to display error stack. - - Updated h5diff with the --enable-error-stack argument, which - enables the display of the hdf5 error stack. This completes the - improvement to the main tools: h5copy, h5diff, h5dump, h5ls and - h5repack. - - (ADB - 2017/08/30, HDFFV-9774) - - -Support for new platforms, languages and compilers. -======================================= - - None - -Bug Fixes since HDF5-1.10.1 release -================================== - - Library - ------- - - The data read after a direct chunk write to a chunked dataset with - one chunk was incorrect. - - The problem was due to the passing of a null dataset pointer to - the insert callback for the chunk index in the routine - H5D__chunk_direct_write() in H5Dchunk.c - The dataset was a single-chunked dataset which will use the - single chunk index when latest format was enabled on file creation. - The single chunk index was the only index that used this pointer - in the insert callback. - - Passed the dataset pointer to the insert callback for the chunk - index in H5D__chunk_direct_write(). - - (VC - 2018/03/20, HDFFV-10425) - - - Added public routine H5DOread_chunk to the high-level C library. - - The patch for H5DOwrite_chunk() to write an entire chunk to the file - directly was contributed by GE Healthcare and integrated by The HDF Group - developers. - - (VC - 2017/05/19, HDFFV-9934) - - - Freeing of object header after failed checksum verification. - - It was discovered that the object header (in H5Ocache.c) was not released properly - when the checksum verification failed and a re-load of the object - header was needed. - - Freed the object header that failed the chksum verification only - after the new object header is reloaded, deserialized and set up. - - (VC - 2018/03/14, HDFFV-10209) - - - Updated H5Pset_evict_on_close in H5Pfapl.c - - Changed the minor error number from H5E_CANTSET to H5E_UNSUPPORTED for - parallel library. - - (ADB - 2018/03/06, HDFFV-10414) - - - Fixed the problems with the utility function that could not handle lowercase - Windows drive letters. - - Added call to upper function for drive letter. - - (ADB - 2017/12/18, HDFFV-10307) - - - Fixed H5Sencode() bug when the number of elements selected was > 2^32. - - H5Sencode() incorrectly encodes dataspace selection with number of - elements exceeding 2^32. When decoding such selection via H5Sdecode(), - the number of elements in the decoded dataspace is not the same as - what is encoded. This problem exists for H5S_SEL_HYPER and - H5S_SEL_POINTS encoding. - - The cause of the problem is due to the fact that the library uses 32 bits to - encode counts and block offsets for the selection. - The solution is to use the original 32 bit encodings if possible, - but use a different way to encode selection if more that 32 bits is needed. - See details in the RFC: H5Sencode/H5Sdecode Format Change i - https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5doc/browse/RFCs/HDF5_Library/H5SencodeFormatChange. - - (VC - 2017/11/28, HDFFV-9947) - - - Fixed filter plugin handling in H5PL.c and H5Z.c to not require i availability of - dependent libraries (e.g., szip or zlib). - - It was discovered that the dynamic loading process used by - filter plugins had issues with library dependencies. - - CMake build process changed to use LINK INTERFACE keywords, which - allowed HDF5 C library to make dependent libraries private. The - filter plugin libraries no longer require dependent libraries - (such as szip or zlib) to be available. - - (ADB - 2017/11/16, HDFFV-10328) - - - Fixed rare object header corruption bug. - - In certain cases, such as when converting large attributes to dense - storage, an error could occur which would either fail an assertion or - cause file corruption. Fixed and added test. - - (NAF - 2017/11/14, HDFFV-10274) - - - Updated H5Zfilter_avail in H5Z.c. - - The public function checked for plugins, while the private - function did not. - - Modified H5Zfilter_avail and private function, H5Z_filter_avail. - Moved check for plugin from public to private function. Updated - H5P__set_filter due to change in H5Z_filter_avail. Updated tests. - - (ADB - 2017/10/10, HDFFV-10297, HDFFV-10319) - - - h5dump produced SEGFAULT when dumping corrypted file. - - The behavior was due to the error in the internal function H5HL_offset_into(). - - (1) Fixed H5HL_offset_into() to return error when offset exceeds heap data - block size. - (2) Fixed other places in the library that call this routine to detect - error routine. - - (VC - 2017/08/30, HDFFV-10216) - - - Fixes for paged aggregation feature. - - Skip test in test/fheap.c when: - (1) multi/split drivers and - (2) persisting free-space or using paged aggregation strategy - - (VC, 2017/07/10) - - Changes made based on RFC review comments: - (1) Added maximum value for file space page size - (2) Dropped check for page end metadata threshold - (3) Removed "can_shrink" and "shrink" callbacks for small section class - - (VC - 2017/06/09) - - - Fixed for infinite loop in H5VM_power2up(). - - The function H5VM_power2up() returns the next power of 2 - for n. When n exceeds 2^63, it overflows and becomes 0 causing - the infinite looping. - - The fix ensures that the function checks for n >= 2^63 - and returns 0. - - (VC - 2017/07/10, HDFFV-10217) - - - Fixed for H5Ocopy doesn't work with open identifiers. - - Changes made so that raw data for dataset objects are copied from - cached info when possible instead of flushing objects to file and - read them back in again. - - (VC - 2017/07/05, HDFFV-7853) - - - An uninitialized struct could cause a memory access error when using - variable-length or reference types in a compressed, chunked dataset. - - A struct containing a callback function pointer and a pointer to some - associated data was used before initialization. This could cause a - memory access error and system crash. This could only occur under - unusual conditions when using variable-lenth and reference types in - a compressed, chunked dataset. - - On recent versions of Visual Studio, when built in debug mode, the - debug heap will complain and cause a crash if the code in question - is executed (this will cause the objcopy test to fail). - - (DER - 2017/11/21, HDFFV-10330) - - - Fixed collective metadata writes on file close. - - It was discovered that metadata was being written twice as part of - the parallel file close behavior, once independently and once - collectively. - - A fix for this error was included as part of the parallel compression - feature but remained undocumented here. - - (RAW - 2017/12/01, HDFFV-10272) - - - If an HDF5 file contains a filter pipeline message with a 'number of - filters' field that exceeds the maximum number of allowed filters, - the error handling code will attempt to dereference a NULL pointer. - - This issue was reported to The HDF Group as issue #CVE-2017-17505. - https://security-tracker.debian.org/tracker/CVE-2017-17505 - https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2017-17505 - - NOTE: The HDF5 C library cannot produce such a file. This condition - should only occur in a corrupt (or deliberately altered) file - or a file created by third-party software. - - This problem arose because the error handling code assumed that - the 'number of filters' field implied that a dynamic array of that - size had already been created and that the cleanup code should - iterate over that array and clean up each element's resources. If - an error occurred before the array has been allocated, this will - not be true. - - This has been changed so that the number of filters is set to - zero on errors. Additionally, the filter array traversal in the - error handling code now requires that the filter array not be NULL. - - (DER - 2018/02/06, HDFFV-10354) - - - If an HDF5 file contains a filter pipeline message which contains - a 'number of filters' field that exceeds the actual number of - filters in the message, the HDF5 C library will read off the end of - the read buffer. - - This issue was reported to The HDF Group as issue #CVE-2017-17506. - https://security-tracker.debian.org/tracker/CVE-2017-17506 - https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2017-17506 - - NOTE: The HDF5 C library cannot produce such a file. This condition - should only occur in a corrupt (or deliberately altered) file - or a file created by third-party software. - - The problem was fixed by passing the buffer size with the buffer - and ensuring that the pointer cannot be incremented off the end - of the buffer. A mismatch between the number of filters declared - and the actual number of filters will now invoke normal HDF5 - error handling. - - (DER - 2018/02/26, HDFFV-10355) - - - If an HDF5 file contains a malformed compound datatype with a - suitably large offset, the type conversion code can run off - the end of the type conversion buffer, causing a segmentation - fault. - - This issue was reported to The HDF Group as issue #CVE-2017-17507. - https://security-tracker.debian.org/tracker/CVE-2017-17506 - https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2017-17506 - - NOTE: The HDF5 C library cannot produce such a file. This condition - should only occur in a corrupt (or deliberately altered) file - or a file created by third-party software. - - THE HDF GROUP WILL NOT FIX THIS BUG AT THIS TIME - - Fixing this problem would involve updating the publicly visible - H5T_conv_t function pointer typedef and versioning the API calls - which use it. We normally only modify the public API during - major releases, so this bug will not be fixed at this time. - - (DER - 2018/02/26, HDFFV-10356) - - - If an HDF5 file contains a malformed compound type which contains - a member of size zero, a division by zero error will occur while - processing the type. - - This issue was reported to The HDF Group as issue #CVE-2017-17508. - https://security-tracker.debian.org/tracker/CVE-2017-17508 - https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2017-17508 - - NOTE: The HDF5 C library cannot produce such a file. This condition - should only occur in a corrupt (or deliberately altered) file - or a file created by third-party software. - - Checking for zero before dividing fixes the problem. Instead of the - division by zero, the normal HDF5 error handling is invoked. - - (DER - 2018/02/26, HDFFV-10357) - - - If an HDF5 file contains a malformed symbol table node that declares - it contains more symbols than it actually contains, the library - can run off the end of the metadata cache buffer while processing - the symbol table node. - - This issue was reported to The HDF Group as issue #CVE-2017-17509. - https://security-tracker.debian.org/tracker/CVE-2017-17509 - https://cve.mitre.org/cgi-bin/cvename.cgi?name=3DCVE-2017-17509 - - NOTE: The HDF5 C library cannot produce such a file. This condition - should only occur in a corrupt (or deliberately altered) file - or a file created by third-party software. - - Performing bounds checks on the buffer while processing fixes the - problem. Instead of the segmentation fault, the normal HDF5 error - handling is invoked. - - (DER - 2018/03/12, HDFFV-10358) - - - Fixed permissions passed to open(2) on file create. - - On Windows, the POSIX permissions passed to open(2) when creating files - were only incidentally correct. They are now set to the correct value of - (_S_IREAD | _S_IWRITE). - - On other platforms, the permissions were set to a mix of 666, 644, and - 000. They are now set uniformly to 666. - - (DER - 2017/04/28, HDFFV-9877) - - - The H5FD_FEAT_POSIX_COMPAT_HANDLE flag is no longer used to determine - if a virtual file driver (VFD) is compatible with SWMR. - - Use of this VFD feature flag was not in line with the documentation in - the public H5FDpublic.h file. In particular, it was being used as a - proxy for determining if SWMR I/O is allowed. This is unnecessary as we - already have a feature flag for this (H5FD_SUPPORTS_SWMR_IO). - - (DER - 2017/05/31, HDFFV-10214) - - - Configuration - ------------- - - CMake changes - - - Updated CMake commands configuration. - - A number of improvements were made to the CMake commands. Most - changes simplify usage or eliminate unused constructs. Also, - some changes support better cross-platform support. - - (ADB - 2018/02/01, HDFFV-10398) - - - Corrected usage of CMAKE_BUILD_TYPE variable. - - The use of the CMAKE_BUILD_TYPE is incorrect for multi-config - generators (Visual Studio and XCode) and is optional for single - config generators. Created a new macro to check - GLOBAL PROPERTY -> GENERATOR_IS_MULTI_CONFIG - Created two new HDF variable, HDF_BUILD_TYPE and HDF_CFG_BUILD_TYPE. - Defaults for these variables is "Release". - - (ADB - 2018/01/10, HDFFV-10385) - - - Added replacement of fortran flags if using static CRT. - - Added TARGET_STATIC_CRT_FLAGS call to HDFUseFortran.cmake file in - config/cmake_ext_mod folder. - - (ADB - 2018/01/08, HDFFV-10334) - - - - The hdf5 library used shared szip and zlib, which needlessly required - applications to link with the same szip and zlib libraries. - - Changed the target_link_libraries commands to use the static libs. - Removed improper link duplication of szip and zlib. - Adjusted the link dependencies and the link interface values of - the target_link_libraries commands. - - (ADB - 2017/11/14, HDFFV-10329) - - - CMake MPI - - CMake implementation for MPI was problematic and would create incorrect - MPI library references in the hdf5 libraries. - - Reworked the CMake MPI code to properly create CMake targets. Also merged - the latest CMake FindMPI.cmake changes to the local copy. This is necessary - until HDF changes the CMake minimum to 3.9 or greater. - - (ADB - 2017/11/02, HDFFV-10321) - - - Corrected FORTRAN_HAVE_C_LONG_DOUBLE processing in the autotools. - - A bug in the autotools Fortran processing code always set the - FORTRAN_HAVE_C_LONG_DOUBLE variable to be true regardless of - whether or not a C long double type was present. - - This would cause compilation failures on platforms where a C - long double type was not available and the Fortran wrappers - were being built. - - (DER - 2017/07/05, HDFFV-10247) - - - The deprecated --enable-production and --enable-debug configure options - failed to emit errors when passed an empty string - (e.g.: --enable-debug=""). - - Due to the way we checked for these options being set, it was possible - to avoid the error message and continue configuration if an empty string - was passed to the option. - - Any use of --enable-production or --enable-debug will now halt the - configuration step and emit a helpful error message - (use --enable-build-mode=debug|production instead). - - (DER - 2017/07/05, HDFFV-10248) - - - CMake - - Too many commands for POST_BUILD step caused command line to be - too big on windows. - - Changed foreach of copy command to use a custom command with the - use of the HDFTEST_COPY_FILE macro. - - (ADB - 2017/07/12, HDFFV-10254) - - - CMake test execution environment - - The parallel HDF5 test: 't_pread' assumed the use of autotools - and the directory structure associated with that testing approach. - Modified the test code to check whether the 'h5jam' utility can be - found in the same directory as the test executable (which is - preferred directory structure utilized by cmake) and if found - will invoke the tool directly rather than utilizing a relative path. - - (RAW - 2017/11/03, HDFFV-10318) - - - Fortran compilation fails for xlf and CMake builds. - - Fixed CMake shared library build for H5match_types and modules - - (MSB - 2017/12/19, HDFFV-10363) - - - Shared libraries fail test on OSX with Fortran enabled with CMake. - - Fixed by removing the F77 use of EQUIVALENCE and COMMON, replaced - using MODULES. Updated CMake. - - (MSB - 2017/12/07, HDFFV-10223) - - - The bin/trace script now emits an error code on problems and autogen.sh - will fail if bin/trace fails. - - The bin/trace script adds tracing functionality to public HDF5 API calls. - It is only of interest to developers who modify the HDF5 source code. - Previously, bin/trace just wrote an error message to stdout when it - encountered problems, so autogen.sh processing did not halt and a broken - version of the library could be built. The script will now return an - error code when it encounters problems, and autogen.sh will fail. - - This only affects users who run autogen.sh to rebuild the Autotools files, - which is not necessary to build HDF5 from source in official releases of the - library. CMake users are unaffected as bin/trace is not run via CMake - at this time. - - (DER - 2017/04/25, HDFFV-10178) - - - FC_BASENAME was changed from gfortran40 to gfortran in a few places. - - In the autotools, FC_BASENAME was set to gfortran40 in a few locations - (config/gnu-fflags and config/freebsd). This was probably a historical - artifact and did not seem to affect many users. - - The value is now correctly set to gfortran. - - (DER - 2017/05/26, HDFFV-10249) - - - The ar flags were changed to -cr (was: -cru) - - The autotools set the flags for ar to -cru by default. The -u flag, - which allows selective replacement of only the members which have - changed, raises warnings on some platforms, so the flags are now set to - -cr via AR_FLAGS in configure.ac. This causes the static library to - always be completely recreated from the object files on each build. - - (DER - 2017/11/15, HDFFV-10428) - - - Fortran - -------- - - Fixed compilation errors when using Intel 18 Fortran compilers - (MSB - 2017/11/3, HDFFV-10322) - - Tools - ----- - - h5clear - - An enhancement to the tool in setting a file's stored EOA. - - It was discovered that a crashed file's stored EOA in the superblock - was smaller than the actual file's EOF. When the file was reopened - and closed, the library truncated the file to the stored EOA. - - Added an option to the tool in setting the file's stored EOA in the - superblock to the maximum of (EOA, EOF) + increment. - An option was also added to print the file's EOA and EOF. - - (VC - 2018/03/14, HDFFV-10360) - - - h5repack - - h5repack changes the chunk parameters when a change of layout is not - specified and a filter is applied. - - HDFFV-10297, HDFFV-10319 reworked code for h5repack and h5diff code - in the tools library. The check for an existing layout was incorrectly - placed into an if block and not executed. The check was moved into - the normal path of the function. - - (ADB - 2018/02/21, HDFFV-10412) - - - h5dump - - The tools library will hide the error stack during file open. - - While this is preferable almost always, there are reasons to enable - display of the error stack when a tool will not open a file. Adding an - optional argument to the --enable-error-stack will provide this use case. - As an optional argument it will not affect the operation of the - --enable-error-stack. h5dump is the only tool to implement this change. - - (ADB - 2018/02/15, HDFFV-10384) - - - h5dump - - h5dump would output an indented blank line in the filters section. - - h5dump overused the h5tools_simple_prefix function, which is a - function intended to account for the data index (x,y,z) option. - Removed the function call for header information. - - (ADB - 2018/01/25, HDFFV-10396) - - - h5repack - - h5repack incorrectly searched internal object table for name. - - h5repack would search the table of objects for a name, if the - name did not match it tried to determine if the name without a - leading slash would match. The logic was flawed! The table - stored names(paths) without a leading slash and did a strstr - of the table path to the name. - The assumption was that if there was a difference of one then - it was a match, however "pressure" would match "/pressure" as - well as "/pressure1", "/pressure2", etc. Changed logic to remove - any leading slash and then do a full compare of the name. - - (ADB - 2018/01/18, HDFFV-10393) - - - h5repack - - h5repack failed to handle command line parameters for customer filters. - - User defined filter parameter conversions would fail when integers were - represented on the command line with character string - larger then 9 characters. Increased local variable array for storing - the current command line parameter to prevent buffer overflows. - - (ADB - 2018/01/17, HDFFV-10392) - - - h5diff - - h5diff seg faulted if comparing VL strings against fixed strings. - - Reworked solution for HDFFV-8625 and HDFFV-8639. Implemented the check - for string objects of same type in the diff_can_type function by - adding an if(tclass1 == H5T_STRING) block. This "if block" moves the - same check that was added for attributes to this function, which is - used by all object types. This function handles complex type structures. - Also added a new test file in h5diffgenttest for testing this issue - and removed the temporary files used in the test scripts. - - (ADB - 2018/01/04, HDFFV-8745) - - - h5repack - - h5repack failed to copy a dataset with existing filter. - - Reworked code for h5repack and h5diff code in the tools library. Added - improved error handling, cleanup of resources and checks of calls. - Modified H5Zfilter_avail and private function, H5Z_filter_avail. - Moved check for plugin from public to private function. Updated - H5P__set_filter due to change in H5Z_filter_avail. Updated tests. - Note, h5repack output display has changed to clarify the individual - steps of the repack process. The output indicates if an operation - applies to all objects. Lines with notation and no information - have been removed. - - (ADB - 2017/10/10, HDFFV-10297, HDFFV-10319) - - - h5repack - - h5repack always set the User Defined filter flag to H5Z_FLAG_MANDATORY. - - Added another parameter to the 'UD=' option to set the flag by default - to '0' or H5Z_FLAG_MANDATORY, the other choice is '1' or H5Z_FLAG_OPTIONAL. - - (ADB - 2017/08/31, HDFFV-10269) - - - h5ls - - h5ls generated error on stack when it encountered a H5S_NULL - dataspace. - - Adding checks for H5S_NULL before calling H5Sis_simple (located - in the h5tools_dump_mem function) fixed the issue. - - (ADB - 2017/08/17, HDFFV-10188) - - - h5repack - - Added tests to h5repack.sh.in to verify options added for paged - aggregation work as expected. - - (VC - 2017/08/03) - - - h5dump - - h5dump segfaulted on output of XML file. - - Function that escape'd strings used the full buffer length - instead of just the length of the replacement string in a - strncpy call. Using the correct length fixed the issue. - - (ADB - 2017/08/01, HDFFV-10256) - - - h5diff - - h5diff segfaulted on compare of a NULL variable length string. - - Improved h5diff compare of strings by adding a check for - NULL strings and setting the lengths to zero. - - (ADB - 2017/07/25, HDFFV-10246) - - - h5import - - h5import crashed trying to import data from a subset of a dataset. - - Improved h5import by adding the SUBSET keyword. h5import understands - to use the Count times the Block as the size of the dimensions. - Added INPUT_B_ORDER keyword to old-style configuration files. - The import from h5dump function expects the binary files to use native - types (FILE '-b' option) in the binary file. - - (ADB - 2017/06/15, HDFFV-10219) - - - h5repack - - h5repack did not maintain the creation order flag of the root - group. - - Improved h5repack by reading the creation order and applying the - flag to the new root group. Also added arguments to set the - order and index direction, which applies to the traversing of the - original file, on the command line. - - (ADB - 2017/05/26, HDFFV-8611) - - - h5diff - - h5diff failed to account for strpad type and null terminators - of char strings. Also, h5diff failed to account for string length - differences and would give a different result depending on file - order in the command line. - - Improved h5diff compare of strings and arrays by adding a check for - string lengths and if the strpad was null filled. - - (ADB - 2017/05/18, HDFFV-9055, HDFFV-10128) - - High-Level APIs: - ------ - - H5DOwrite_chunk() problems when overwriting an existing chunk with - no filters enabled. - - When overwriting chunks and no filters were being used, the library would - fail (when asserts are enabled, e.g. debug builds) or incorrectly - insert additional chunks instead of overwriting (when asserts are not - enabled, e.g. production builds). - - This has been fixed and a test was added to the hl/test_dset_opt test. - - (DER - 2017/05/11, HDFFV-10187) - - C++ APIs - -------- - - Removal of memory leaks. - - A private function was inadvertently called, causing memory leaks. This - is now fixed. - - (BMR - 2018/03/12 - User's reported in email) - - Testing - ------- - - Memory for three variables in testphdf5's coll_write_test was malloced - but not freed, leaking memory when running the test. - - The variables' memory is now freed. - - (LRK - 2018/03/12, HDFFV-10397) - - - Refactored the testpar/t_bigio.c test to include ALARM macros - - Changed the test to include the ALARM_ON and ALARM_OFF macros which - are intended to prevent nightly test hangs that have been observed - with this particular parallel test example. The code was also modified to - simplify status reporting (only from MPI rank 0) and additional - status checking added. - - (RAW - 2017/11/08, HDFFV-10301) - - -Supported Platforms -=================== - - Linux 2.6.32-696.16.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) - #1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) - (ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) - IBM XL C/C++ V13.1 - IBM XL Fortran V15.1 - - Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4) - Version 4.9.3, Version 5.2.0, - Intel(R) C (icc), C++ (icpc), Fortran (icc) - compilers: - Version 17.0.0.098 Build 20160721 - MPICH 3.1.4 compiled with GCC 4.9.3 - - SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc - (emu) Sun Fortran 95 8.6 SunOS_sparc - Sun C++ 5.12 SunOS_sparc - - Windows 7 Visual Studio 2012 w/ Intel Fortran 15 (cmake) - Visual Studio 2013 w/ Intel Fortran 15 (cmake) - Visual Studio 2015 w/ Intel Fortran 16 (cmake) - - Windows 7 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake) - Visual Studio 2013 w/ Intel Fortran 15 (cmake) - Visual Studio 2015 w/ Intel Fortran 16 (cmake) - Visual Studio 2015 w/ Intel C, Fortran 2017 (cmake) - Visual Studio 2015 w/ MSMPI 8 (cmake) - Cygwin(CYGWIN_NT-6.1 2.8.0(0.309/5/3) - gcc and gfortran compilers (GCC 5.4.0) - (cmake and autotools) - - Windows 10 Visual Studio 2015 w/ Intel Fortran 16 (cmake) - Cygwin(CYGWIN_NT-6.1 2.8.0(0.309/5/3) - gcc and gfortran compilers (GCC 5.4.0) - (cmake and autotools) - - Windows 10 x64 Visual Studio 2015 w/ Intel Fortran 16 (cmake) - - Mac OS X Yosemite 10.10.5 Apple clang/clang++ version 6.1 from Xcode 7.0 - 64-bit gfortran GNU Fortran (GCC) 4.9.2 - (osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3 - - Mac OS X El Capitan 10.11.6 Apple clang/clang++ version 7.3.0 from Xcode 7.3 - 64-bit gfortran GNU Fortran (GCC) 5.2.0 - (osx1011dev/osx1011test) Intel icc/icpc/ifort version 16.0.2 - - Mac OS Sierra 10.12.6 Apple LLVM version 8.1.0 (clang/clang++-802.0.42) - 64-bit gfortran GNU Fortran (GCC) 7.1.0 - (swallow/kite) Intel icc/icpc/ifort version 17.0.2 - - -Tested Configuration Features Summary -===================================== - - In the tables below - y = tested - n = not tested in this release - C = Cluster - W = Workstation - x = not working in this release - dna = does not apply - ( ) = footnote appears below second table - <blank> = testing incomplete on this feature or platform - -Platform C F90/ F90 C++ zlib SZIP - parallel F2003 parallel -Solaris2.11 32-bit n y/y n y y y -Solaris2.11 64-bit n y/n n y y y -Windows 7 y y/y n y y y -Windows 7 x64 y y/y y y y y -Windows 7 Cygwin n y/n n y y y -Windows 7 x64 Cygwin n y/n n y y y -Windows 10 y y/y n y y y -Windows 10 x64 y y/y n y y y -Mac OS X Mountain Lion 10.8.5 64-bit n y/y n y y y -Mac OS X Mavericks 10.9.5 64-bit n y/y n y y y -Mac OS X Yosemite 10.10.5 64-bit n y/y n y y y -Mac OS X El Capitan 10.11.6 64-bit n y/y n y y y -CentOS 7.2 Linux 2.6.32 x86_64 PGI n y/y n y y y -CentOS 7.2 Linux 2.6.32 x86_64 GNU y y/y y y y y -CentOS 7.2 Linux 2.6.32 x86_64 Intel n y/y n y y y -Linux 2.6.32-573.18.1.el6.ppc64 n y/y n y y y - - -Platform Shared Shared Shared Thread- - C libs F90 libs C++ libs safe -Solaris2.11 32-bit y y y y -Solaris2.11 64-bit y y y y -Windows 7 y y y y -Windows 7 x64 y y y y -Windows 7 Cygwin n n n y -Windows 7 x64 Cygwin n n n y -Windows 10 y y y y -Windows 10 x64 y y y y -Mac OS X Mountain Lion 10.8.5 64-bit y n y y -Mac OS X Mavericks 10.9.5 64-bit y n y y -Mac OS X Yosemite 10.10.5 64-bit y n y y -Mac OS X El Capitan 10.11.6 64-bit y n y y -CentOS 7.2 Linux 2.6.32 x86_64 PGI y y y n -CentOS 7.2 Linux 2.6.32 x86_64 GNU y y y y -CentOS 7.2 Linux 2.6.32 x86_64 Intel y y y n -Linux 2.6.32-573.18.1.el6.ppc64 y y y n - -Compiler versions for each platform are listed in the preceding -"Supported Platforms" table. - - -More Tested Platforms -===================== -The following platforms are not supported but have been tested for this release. - - Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (mayll/platypus) Version 4.4.7 20120313 - Version 4.9.3, 5.3.0, 6.2.0 - PGI C, Fortran, C++ for 64-bit target on - x86-64; - Version 17.10-0 - Intel(R) C (icc), C++ (icpc), Fortran (icc) - compilers: - Version 17.0.4.196 Build 20170411 - MPICH 3.1.4 compiled with GCC 4.9.3 - - Linux 3.10.0-327.18.2.el7 GNU C (gcc) and C++ (g++) compilers - #1 SMP x86_64 GNU/Linux Version 4.8.5 20150623 (Red Hat 4.8.5-4) - (jelly) with NAG Fortran Compiler Release 6.1(Tozai) - GCC Version 7.1.0 - OpenMPI 3.0.0-GCC-7.2.0-2.29 - Intel(R) C (icc) and C++ (icpc) compilers - Version 17.0.0.098 Build 20160721 - with NAG Fortran Compiler Release 6.1(Tozai) - - Linux 3.10.0-327.10.1.el7 MPICH 3.2 compiled with GCC 5.3.0 - #1 SMP x86_64 GNU/Linux - (moohan) - - Linux 2.6.32-573.18.1.el6.ppc64 MPICH mpich 3.1.4 compiled with - #1 SMP ppc64 GNU/Linux IBM XL C/C++ for Linux, V13.1 - (ostrich) and IBM XL Fortran for Linux, V15.1 - - Debian 8.4 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1 x86_64 GNU/Linux - gcc, g++ (Debian 4.9.2-10) 4.9.2 - GNU Fortran (Debian 4.9.2-10) 4.9.2 - (cmake and autotools) - - Fedora 24 4.7.2-201.fc24.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux - gcc, g++ (GCC) 6.1.1 20160621 - (Red Hat 6.1.1-3) - GNU Fortran (GCC) 6.1.1 20160621 - (Red Hat 6.1.1-3) - (cmake and autotools) - - Ubuntu 16.04.1 4.4.0-38-generic #57-Ubuntu SMP x86_64 GNU/Linux - gcc, g++ (Ubuntu 5.4.0-6ubuntu1~16.04.2) - 5.4.0 20160609 - GNU Fortran (Ubuntu 5.4.0-6ubuntu1~16.04.2) - 5.4.0 20160609 - (cmake and autotools) - - -Known Problems -============== - - At present, metadata cache images may not be generated by parallel - applications. Parallel applications can read files with metadata cache - images, but since this is a collective operation, a deadlock is possible - if one or more processes do not participate. - - Three tests fail with OpenMPI 3.0.0/GCC-7.2.0-2.29: - testphdf5 (ecdsetw, selnone, cchunk1, cchunk3, cchunk4, and actualio) - t_shapesame (sscontig2) - t_pflush1/fails on exit - The first two tests fail attempting collective writes. - - Known problems in previous releases can be found in the HISTORY*.txt files - in the HDF5 source. Please report any new problems found to - help@hdfgroup.org. - - -%%%%1.10.1%%%% - -HDF5 version 1.10.1 released on 2017-04-27 -================================================================================ - -INTRODUCTION - -This document describes the differences between HDF5-1.10.0-patch1 and -HDF5 1.10.1, and contains information on the platforms tested and known -problems in HDF5-1.10.1. For more details check the HISTORY*.txt files -in the HDF5 source. - -Links to HDF5 1.10.1 source code, documentation, and additional materials can -be found on The HDF5 web page at: - - https://support.hdfgroup.org/HDF5/ - -The HDF5 1.10.1 release can be obtained from: - - https://support.hdfgroup.org/HDF5/release/obtain5.html - -User documentation for the snapshot can be accessed directly at this location: - - https://support.hdfgroup.org/HDF5/doc/ - -New features in the HDF5-1.10.x release series, including brief general -descriptions of some new and modified APIs, are described in the "New Features -in HDF5 Release 1.10" document: - - https://support.hdfgroup.org/HDF5/docNewFeatures/index.html - -All new and modified APIs are listed in detail in the "HDF5 Software Changes -from Release to Release" document, in the section "Release 10.1 (current -release) versus Release 1.10.0 - - https://support.hdfgroup.org/HDF5/doc/ADGuide/Changes.html - -If you have any questions or comments, please send them to the HDF Help Desk: - - help@hdfgroup.org - - -CONTENTS - -- Major New Features Introduced in HDF5 1.10.1 -- Other New Features and Enhancements -- Support for New Platforms, Languages, and Compilers -- Bug Fixes since HDF5-1.10.0-patch1 -- Supported Platforms -- Tested Configuration Features Summary -- More Tested Platforms -- Known Problems - - -Major New Features Introduced in HDF5 1.10.1 -============================================ - -For links to the RFCs and documentation in this section please view -https://support.hdfgroup.org/HDF5/docNewFeatures in a web browser. - -________________________________________ -Metadata Cache Image -________________________________________ - - HDF5 metadata is typically small, and scattered throughout the HDF5 file. - This can affect performance, particularly on large HPC systems. The - Metadata Cache Image feature can improve performance by writing the - metadata cache in a single block on file close, and then populating the - cache with the contents of this block on file open, thus avoiding the many - small I/O operations that would otherwise be required on file open and - close. See the RFC for complete details regarding this feature. Also, - see the Fine Tuning the Metadata Cache documentation. - - At present, metadata cache images may not be generated by parallel - applications. Parallel applications can read files with metadata cache - images, but since this is a collective operation, a deadlock is possible - if one or more processes do not participate. - -________________________________________ -Metadata Cache Evict on Close -________________________________________ - - The HDF5 library's metadata cache is fairly conservative about holding on - to HDF5 object metadata (object headers, chunk index structures, etc.), - which can cause the cache size to grow, resulting in memory pressure on - an application or system. The "evict on close" property will cause all - metadata for an object to be evicted from the cache as long as metadata - is not referenced from any other open object. See the Fine Tuning the - Metadata Cache documentation for information on the APIs. - - At present, evict on close is disabled in parallel builds. - -________________________________________ -Paged Aggregation -________________________________________ - - The current HDF5 file space allocation accumulates small pieces of metadata - and raw data in aggregator blocks which are not page aligned and vary - widely in sizes. The paged aggregation feature was implemented to provide - efficient paged access of these small pieces of metadata and raw data. - See the RFC for details. Also, see the File Space Management documentation. - -________________________________________ -Page Buffering -________________________________________ - - Small and random I/O accesses on parallel file systems result in poor - performance for applications. Page buffering in conjunction with paged - aggregation can improve performance by giving an application control of - minimizing HDF5 I/O requests to a specific granularity and alignment. - See the RFC for details. Also, see the Page Buffering documentation. - - At present, page buffering is disabled in parallel builds. - - - -Other New Features and Enhancements -=================================== - - Library - ------- - - Added a mechanism for disabling the SWMR file locking scheme. - - The file locking calls used in HDF5 1.10.0 (including patch1) - will fail when the underlying file system does not support file - locking or where locks have been disabled. To disable all file - locking operations, an environment variable named - HDF5_USE_FILE_LOCKING can be set to the five-character string - 'FALSE'. This does not fundamentally change HDF5 library - operation (aside from initial file open/create, SWMR is lock-free), - but users will have to be more careful about opening files - to avoid problematic access patterns (i.e.: multiple writers) - that the file locking was designed to prevent. - - Additionally, the error message that is emitted when file lock - operations set errno to ENOSYS (typical when file locking has been - disabled) has been updated to describe the problem and potential - resolution better. - - (DER, 2016/10/26, HDFFV-9918) - - - The return type of H5Pget_driver_info() has been changed from void * - to const void *. - - The pointer returned by this function points to internal library - memory and should not be freed by the user. - - (DER, 2016/11/04, HDFFV-10017) - - - The direct I/O VFD has been removed from the list of VFDs that - support SWMR. - - This configuration was never officially tested and several SWMR - tests fail when this VFD is set. - - (DER, 2016/11/03, HDFFV-10169) - - Configuration: - -------------- - - The minimum version of CMake required to build HDF5 is now 3.2.2. - - (ADB, 2017/01/10) - - - An --enable/disable-developer-warnings option has been added to - configure. - - This disables warnings that do not indicate poor code quality such - as -Winline and gcc's -Wsuggest-attribute. Developer warnings are - disabled by default. - - (DER, 2017/01/10) - - - A bin/restore.sh script was added that reverts autogen.sh processing. - - (DER, 2016/11/08) - - - CMake: Added NAMESPACE hdf5:: to package configuration files to allow - projects using installed HDF5 binaries built with CMake to link with - them without specifying the HDF5 library location via IMPORTED_LOCATION. - - (ABD, 2016/10/17, HDFFV-10003) - - - CMake: Changed the CTEST_BUILD_CONFIGURATION option to - CTEST_CONFIGURATION_TYPE as recommended by the CMake documentation. - - (ABD, 2016/10/17, HDFFV-9971) - - - Fortran Library: - ---------------- - - - The HDF5 Fortran library can now be compiled with the NAG compiler. - - (MSB, 2017/2/10, HDFFV-9973) - - - C++ Library: - ------------ - - - The following C++ API wrappers have been added to the C++ Library: - - // Sets/Gets the strategy and the threshold value that the library - // will employ in managing file space. - FileCreatPropList::setFileSpaceStrategy - H5Pset_file_space_strategy - FileCreatPropList::getFileSpaceStrategy - H5Pget_file_space_strategy - - // Sets/Gets the file space page size for paged aggregation. - FileCreatPropList::setFileSpacePagesize - H5Pset_file_space_page_size - FileCreatPropList::getFileSpacePagesize - H5Pget_file_space_page_size - - // Checks if the given ID is valid. - IdComponent::isValid - H5Iis_valid - - // Sets/Gets the number of soft or user-defined links that can be - // traversed before a failure occurs. - LinkAccPropList::setNumLinks - H5Pset_nlinks - LinkAccPropList::getNumLinks - H5Pget_nlinks - - // Returns a copy of the creation property list of a datatype. - DataType::getCreatePlist - H5Tget_create_plist - - // Opens/Closes an object within a group or a file, regardless of object - // type - Group::getObjId - H5Oopen - Group::closeObjId - H5Oclose - - // Maps elements of a virtual dataset to elements of the source dataset. - DSetCreatPropList::setVirtual - H5Pset_virtual - - // Gets general information about this file. - H5File::getFileInfo - H5Fget_info2 - - // Returns the number of members in a type. - IdComponent::getNumMembers - H5Inmembers - - // Determines if an element type exists. - IdComponent::typeExists - H5Itype_exists - - // Determines if an object exists. - H5Location::exists - H5Lexists. - - // Returns the header version of an HDF5 object. - H5Object::objVersion - H5Oget_info for version - - (BMR, 2017/03/20, HDFFV-10004, HDFFV-10139, HDFFV-10145) - - - New exception: ObjHeaderIException for H5O interface. - - (BMR, 2017/03/15, HDFFV-10145) - - - New class LinkAccPropList for link access property list, to be used by - wrappers of H5Lexists. - - (BMR, 2017/01/04, HDFFV-10145) - - - New constructors to open datatypes in ArrayType, CompType, DataType, - EnumType, FloatType, IntType, StrType, and VarLenType. - - (BMR, 2016/12/26, HDFFV-10056) - - - New member functions: - - DSetCreatPropList::setNbit() to setup N-bit compression for a dataset. - - ArrayType::getArrayNDims() const - ArrayType::getArrayDims() const - both to replace the non-const versions. - - (BMR, 2016/04/25, HDFFV-8623, HDFFV-9725) - - - Tools: - ------ - - The following options have been added to h5clear: - -s: clear the status_flags field in the file's superblock - -m: Remove the metadata cache image from the file - - (QAK, 2017/03/22, PR#361) - - - High-Level APIs: - --------------- - - Added New Fortran 2003 API for h5tbmake_table_f. - - (MSB, 2017/02/10, HDFFV-8486) - - - -Support for New Platforms, Languages, and Compilers -=================================================== - - - Added NAG compiler - - - -Bug Fixes since HDF5-1.10.0-patch1 release -================================== - - Library - ------- - - Outdated data structure was used in H5D_CHUNK_DEBUG blocks, causing - compilation errors when H5D_CHUNK_DEBUG was defined. This is fixed. - - (BMR, 2017/04/04, HDFFV-8089) - - - SWMR implementation in the HDF5 1.10.0 and 1.10.0-patch1 releases has a - broken metadata flush dependency that manifested itself with the following - error at the end of the HDF5 error stack: - - H5Dint.c line 846 in H5D__swmr_setup(): dataspace chunk index must be 0 - for SWMR access, chunkno = 1 - major: Dataset - minor: Bad value - - It was also reported at https://github.com/areaDetector/ADCore/issues/203 - - The flush dependency is fixed in this release. - - - Changed the plugins dlopen option from RTLD_NOW to RTLD_LAZY - - (ABD, 2016/12/12, PR#201) - - - A number of issues were fixed when reading/writing from/to corrupted - files to ensure that the library fails gracefully in these cases: - - * Writing to a corrupted file that has an object message which is - incorrectly marked as shareable on disk results in a buffer overflow / - invalid write instead of a clean error message. - - * Decoding data from a corrupted file with a dataset encoded with the - H5Z_NBIT decoding can result in a code execution vulnerability under - the context of the application using the HDF5 library. - - * When decoding an array datatype from a corrupted file, the HDF5 library - fails to return an error in production if the number of dimensions - decoded is greater than the maximum rank. - - * When decoding an "old style" array datatype from a corrupted file, the - HDF5 library fails to return an error in production if the number of - dimensions decoded is greater than the maximum rank. - - (NAF, 2016/10/06, HDFFV-9950, HDFFV-9951, HDFFV-9992, HDFFV-9993) - - - Fixed an error that would occur when copying an object with an attribute - which is a compound datatype consisting of a variable length string. - - (VC, 2016/08/24, HDFFV-7991) - - - H5DOappend will no longer fail if a dataset has no append callback - registered. - - (VC, 2016/08/14, HDFFV-9960) - - - Fixed an issue where H5Pset_alignment could result in misaligned blocks - with some input combinations, causing an assertion failure in debug mode. - - (NAF, 2016/08/11, HDFFV-9948) - - - Fixed a problem where a plugin compiled into a DLL in the default plugin - directory could not be found by the HDF5 library at runtime on Windows - when the HDF5_PLUGIN_PATH environment variable was not set. - - (ABD, 2016/08/01, HDFFV-9706) - - - Fixed an error that would occur when calling H5Adelete on an attribute - which is attached to an externally linked object in the target file and - whose datatype is a committed datatype in the main file. - - (VC, 2016/07/06, HDFFV-9940) - - - (a) Throw an error instead of assertion when v1 btree level hits the 1 - byte limit. - (b) Modifications to better handle error recovery when conversion by - h5format_convert fails. - - (VC, 2016/05/29, HDFFV-9434) - - - Fixed a memory leak where an array used by the library to track SWMR - read retries was unfreed. - - The leaked memory was small (on the order of a few tens of ints) and - allocated per-file. The memory was allocated (and lost) only when a - file was opened for SWMR access. - - (DER, 2016/04/27, HDFFV-9786) - - - Fixed a memory leak that could occur when opening a file for the first - time (including creating) and the call fails. - - This occurred when the file-driver-specific info was not cleaned up. - The amount of memory leaked varied with the file driver, but would - normally be less than 1 kB. - - (DER, 2016/12/06, HDFFV-10168) - - - Fixed a failure in collective metadata writes. - - This failure only appeared when collective metadata writes - were enabled (via H5Pset_coll_metadata_write()). - - (JRM, 2017/04/10, HDFFV-10055) - - - Parallel Library - ---------------- - - Fixed a bug that could occur when allocating a chunked dataset in parallel - with an alignment set and an alignment threshold greater than the chunk - size but less than or equal to the raw data aggregator size. - - (NAF, 2016/08/11, HDFFV-9969) - - - Configuration - ------------- - - Configuration will check for the strtoll and strtoull functions - before using alternatives - - (ABD, 2017/03/17, PR#340) - - - CMake uses a Windows pdb directory variable if available and - will generate both static and shared pdb files. - - (ABD, 2017/02/06, HDFFV-9875) - - - CMake now builds shared versions of tools. - - (ABD, 2017/02/01, HDFFV-10123) - - - Makefiles and test scripts have been updated to correctly remove files - created when running "make check" and to avoid removing any files under - source control. In-source builds followed by "make clean" and "make - distclean" should result in the original source files. - (LRK, 2017/01/17, HDFFV-10099) - - - The tools directory has been divided into two separate source and test - directories. This resolves a build dependency and, as a result, - 'make check' will no longer fail in the tools directory if 'make' was - not executed first. - - (ABD, 2016/10/27, HDFFV-9719) - - - CMake: Fixed a timeout error that would occasionally occur when running - the virtual file driver tests simultaneously due to test directory - and file name collisions. - - (ABD, 2016/09/19, HDFFV-9431) - - - CMake: Fixed a command length overflow error by converting custom - commands inside CMakeTest.cmake files into regular dependencies and - targets. - - (ABD, 2016/07/12, HDFFV-9939) - - - Fixed a problem preventing HDF5 to be built on 32-bit CYGWIN by - condensing cygwin configuration files into a single file and - removing outdated compiler settings. - - (ABD, 2016/07/12, HDFFV-9946) - - - Fortran - -------- - - Changed H5S_ALL_F from INTEGER to INTEGER(HID_T) - - (MSB, 2016/10/14, HDFFV-9987) - - - Tools - ----- - - h5diff now correctly ignores strpad in comparing strings. - - (ABD, 2017/03/03, HDFFV-10128) - - - h5repack now correctly parses the command line filter options. - - (ABD, 2017/01/24, HDFFV-10046) - - - h5diff now correctly returns an error when it cannot read data due - to an unavailable filter plugin. - - (ADB 2017/01/18, HDFFV-9994 ) - - - Fixed an error in the compiler wrapper scripts (h5cc, h5fc, et al.) - in which they would erroneously drop the file argument specified via - the -o flag when the -o flag was specified before the -c flag on the - command line, resulting in a failure to compile. - - (LRK, 2016/11/04, HDFFV-9938, HDFFV-9530) - - - h5repack User Defined (UD) filter parameters were not parsed correctly. - - The UD filter parameters were not being parsed correctly. Reworked coding - section to parse the correct values and verify number of parameters. - - (ABD, 2016/10/19, HDFFV-9996, HDFFV-9974, HDFFV-9515, HDFFV-9039) - - - h5repack allows the --enable-error-stack option on the command line. - - (ADB, 2016/08/08, HDFFV-9775) - - - C++ APIs - -------- - - The member function H5Location::getNumObjs() is moved to - class Group because the objects are in a group or a file only, - and H5Object::getNumAttrs to H5Location to get the number of - attributes at a given location. - - (BMR, 2017/03/17, PR#466) - - - Due to the change in the C API, the overloaded functions of - PropList::setProperty now need const for some arguments. They are - planned for deprecation and are replaced by new versions with proper - consts. - - (BMR, 2017/03/17, PR#344) - - - The high-level API Packet Table (PT) did not write data correctly when - the datatype is a compound type that has string type as one of the - members. This problem started in 1.8.15, after the fix of HDFFV-9042 - was applied, which caused the Packet Table to use native type to access - the data. It should be up to the application to specify whether the - buffer to be read into memory is in the machine's native architecture. - Thus, the PT is fixed to not use native type but to make a copy of the - user's provided datatype during creation or the packet table's datatype - during opening. If an application wishes to use native type to read the - data, then the application will request that. However, the Packet Table - doesn't provide a way to specify memory datatype in this release. This - feature will be available in future releases. - - (BMR, 2016/10/27, HDFFV-9758) - - - The obsolete macros H5_NO_NAMESPACE and H5_NO_STD have been removed from - the HDF5 C++ API library. - - (BMR, 2016/10/23, HDFFV-9532) - - - The problem where a user-defined function cannot access both, attribute - and dataset, using only one argument is now fixed. - - (BMR, 2016/10/11, HDFFV-9920) - - - In-memory array information, ArrayType::rank and - ArrayType::dimensions, were removed. This is an implementation - detail and should not affect applications. - - (BMR, 2016/04/25, HDFFV-9725) - - - Testing - ------- - - Fixed a problem that caused tests using SWMR to occasionally fail when - running "make check" using parallel make. - - (LRK, 2016/03/22, PR#338, PR#346, PR#358) - - -Supported Platforms -=================== - - Linux 2.6.32-573.18.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4) - #1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-4) - (ostrich) GNU Fortran (GCC) 4.4.7 20120313 - (Red Hat 4.4.7-4) - IBM XL C/C++ V13.1 - IBM XL Fortran V15.1 - - Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4) - Version 4.9.3, Version 5.2.0 - Intel(R) C (icc), C++ (icpc), Fortran (icc) - compilers: - Version 15.0.3.187 Build 20150407 - MPICH 3.1.4 compiled with GCC 4.9.3 - - SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc - (emu) Sun Fortran 95 8.6 SunOS_sparc - Sun C++ 5.12 SunOS_sparc - - Windows 7 Visual Studio 2012 w/ Intel Fortran 15 (cmake) - Visual Studio 2013 w/ Intel Fortran 15 (cmake) - Visual Studio 2015 w/ Intel Fortran 16 (cmake) - - Windows 7 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake) - Visual Studio 2013 w/ Intel Fortran 15 (cmake) - Visual Studio 2015 w/ Intel Fortran 16 (cmake) - Visual Studio 2015 w/ MSMPI 8 (cmake) - Cygwin(CYGWIN_NT-6.1 2.8.0(0.309/5/3) - gcc and gfortran compilers (GCC 5.4.0) - (cmake and autotools) - - Windows 10 Visual Studio 2015 w/ Intel Fortran 16 (cmake) - Cygwin(CYGWIN_NT-6.1 2.8.0(0.309/5/3) - gcc and gfortran compilers (GCC 5.4.0) - (cmake and autotools) - - Windows 10 x64 Visual Studio 2015 w/ Intel Fortran 16 (cmake) - - Mac OS X Mt. Lion 10.8.5 Apple clang/clang++ version 5.1 from Xcode 5.1 - 64-bit gfortran GNU Fortran (GCC) 4.8.2 - (swallow/kite) Intel icc/icpc/ifort version 15.0.3 - - Mac OS X Mavericks 10.9.5 Apple clang/clang++ version 6.0 from Xcode 6.2 - 64-bit gfortran GNU Fortran (GCC) 4.9.2 - (wren/quail) Intel icc/icpc/ifort version 15.0.3 - - Mac OS X Yosemite 10.10.5 Apple clang/clang++ version 6.1 from Xcode 7.0 - 64-bit gfortran GNU Fortran (GCC) 4.9.2 - (osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3 - - Mac OS X El Capitan 10.11.6 Apple clang/clang++ version 7.3 from Xcode 7.3 - 64-bit gfortran GNU Fortran (GCC) 5.2.0 - (osx1010dev/osx1010test) Intel icc/icpc/ifort version 16.0.2 - - -Tested Configuration Features Summary -===================================== - - In the tables below - y = tested - n = not tested in this release - C = Cluster - W = Workstation - x = not working in this release - dna = does not apply - ( ) = footnote appears below second table - <blank> = testing incomplete on this feature or platform - -Platform C F90/ F90 C++ zlib SZIP - parallel F2003 parallel -Solaris2.11 32-bit n y/y n y y y -Solaris2.11 64-bit n y/n n y y y -Windows 7 y y/y n y y y -Windows 7 x64 y y/y y y y y -Windows 7 Cygwin n y/n n y y y -Windows 7 x64 Cygwin n y/n n y y y -Windows 10 y y/y n y y y -Windows 10 x64 y y/y n y y y -Mac OS X Mountain Lion 10.8.5 64-bit n y/y n y y y -Mac OS X Mavericks 10.9.5 64-bit n y/y n y y y -Mac OS X Yosemite 10.10.5 64-bit n y/y n y y y -Mac OS X El Capitan 10.11.6 64-bit n y/y n y y y -CentOS 7.2 Linux 2.6.32 x86_64 PGI n y/y n y y y -CentOS 7.2 Linux 2.6.32 x86_64 GNU y y/y y y y y -CentOS 7.2 Linux 2.6.32 x86_64 Intel n y/y n y y y -Linux 2.6.32-573.18.1.el6.ppc64 n y/y n y y y - - -Platform Shared Shared Shared Thread- - C libs F90 libs C++ libs safe -Solaris2.11 32-bit y y y y -Solaris2.11 64-bit y y y y -Windows 7 y y y y -Windows 7 x64 y y y y -Windows 7 Cygwin n n n y -Windows 7 x64 Cygwin n n n y -Windows 10 y y y y -Windows 10 x64 y y y y -Mac OS X Mountain Lion 10.8.5 64-bit y n y y -Mac OS X Mavericks 10.9.5 64-bit y n y y -Mac OS X Yosemite 10.10.5 64-bit y n y y -Mac OS X El Capitan 10.11.6 64-bit y n y y -CentOS 7.2 Linux 2.6.32 x86_64 PGI y y y n -CentOS 7.2 Linux 2.6.32 x86_64 GNU y y y y -CentOS 7.2 Linux 2.6.32 x86_64 Intel y y y n -Linux 2.6.32-573.18.1.el6.ppc64 y y y n - -Compiler versions for each platform are listed in the preceding -"Supported Platforms" table. - - -More Tested Platforms -===================== - -The following platforms are not supported but have been tested for this release. - - Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (mayll/platypus) Version 4.4.7 20120313 - Version 4.8.4 - PGI C, Fortran, C++ for 64-bit target on - x86-64; - Version 16.10-0 - Intel(R) C (icc), C++ (icpc), Fortran (icc) - compilers: - Version 15.0.3.187 (Build 20150407) - MPICH 3.1.4 compiled with GCC 4.9.3 - - Linux 3.10.0-327.18.2.el7 GNU C (gcc) and C++ (g++) compilers - #1 SMP x86_64 GNU/Linux Version 4.8.5 20150623 (Red Hat 4.8.5-4) - (jelly) with NAG Fortran Compiler Release 6.1(Tozai) - Intel(R) C (icc) and C++ (icpc) compilers - Version 15.0.3.187 (Build 20150407) - with NAG Fortran Compiler Release 6.1(Tozai) - - Linux 2.6.32-573.18.1.el6.ppc64 MPICH mpich 3.1.4 compiled with - #1 SMP ppc64 GNU/Linux IBM XL C/C++ for Linux, V13.1 - (ostrich) and IBM XL Fortran for Linux, V15.1 - - Debian 8.4 3.16.0-4-amd64 #1 SMP Debian 3.16.36-1 x86_64 GNU/Linux - gcc, g++ (Debian 4.9.2-10) 4.9.2 - GNU Fortran (Debian 4.9.2-10) 4.9.2 - (cmake and autotools) - - Fedora 24 4.7.2-201.fc24.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux - gcc, g++ (GCC) 6.1.1 20160621 - (Red Hat 6.1.1-3) - GNU Fortran (GCC) 6.1.1 20160621 - (Red Hat 6.1.1-3) - (cmake and autotools) - - Ubuntu 16.04.1 4.4.0-38-generic #57-Ubuntu SMP x86_64 GNU/Linux - gcc, g++ (Ubuntu 5.4.0-6ubuntu1~16.04.2) - 5.4.0 20160609 - GNU Fortran (Ubuntu 5.4.0-6ubuntu1~16.04.2) - 5.4.0 20160609 - (cmake and autotools) - - -Known Problems -============== - - At present, metadata cache images may not be generated by parallel - applications. Parallel applications can read files with metadata cache - images, but since this is a collective operation, a deadlock is possible - if one or more processes do not participate. - - Known problems in previous releases can be found in the HISTORY*.txt files - in the HDF5 source. Please report any new problems found to - help@hdfgroup.org. - - -%%%%1.10.0-patch1%%%% - - -HDF5 version 1.10.0-patch1 released on 2016-05-23 -================================================================================ - -INTRODUCTION - -This document describes the differences between HDF5-1.8 series and -HDF5 1.10.0 releases, and contains information on the platforms -tested. - -Links to HDF5 1.10.0 source code can be found on The HDF Group's -development FTP server at the following location: - - https://www.hdfgroup.org/HDF5/release/obtain5110.html - -User documentation can be accessed directly at this location: - - https://www.hdfgroup.org/HDF5/docNewFeatures/ - -For more information, see the HDF5 home page: - - https://www.hdfgroup.org/HDF5/ - -If you have any questions or comments, please send them to the HDF -Help Desk: - - help@hdfgroup.org - - - -CONTENTS - -- New Features -- Issues Addressed in this Release -- Supported Platforms -- Tested Configuration Features Summary -- More Tested Platforms -- Known Problems and Limitations - - - -New Features -============ -This release supports the following features: - - Configuration - ------------- - - API Compatibility with HDF5 1.8 Flag Was Added - - The 1.10 version of the HDF5 Library can be configured to operate - identically to the 1.8 library with the --with-default-api-version=v18 - configure flag. This allows existing code to be compiled with the 1.10 - library without requiring immediate changes to the application source - code. For additional configuration options and other details, see - "API Compatibility Macros in HDF5" at - https://www.hdfgroup.org/HDF5/doc/RM/APICompatMacros.html. - - - Autotools Configuration Has Been Extensively Reworked - - The autotools configuration options have been updated to allow more - fine-grained control of the build options and to correct some bugs. - See configure --help for comprehensive information on each option. - - Specific changes: - - * --enable-debug and --enable-production are no longer accepted. - Use --enable-build-mode=(debug | production) instead. These set - appropriate defaults for symbols, optimizations, and other - configuration options. These defaults can be overridden by the - user. - - * Extra debug output messages are no longer enabled with - --enable-debug=<package list>. Use --enable-internal-debug=<pkg list> - instead. - - * A new --enable-symbols option allows symbols to be generated - independently of the build mode. --disable-symbols can be used - to strip symbols from the binary. - - * A new --enable-asserts option sets/unsets NDEBUG. This is - independent of the build mode. This also enables some extra - low-overhead debug checks in the library. - - * A new --enable-profiling option sets profiling flags. This is - independent of the build mode. - - * A new --enable-optimization option sets the optimization level. - This is independent of the build mode. - - * Many of these options can take a flags string that will be used - to build the library. This can be useful for specifying custom - optimization flags such as -Os and -Ofast. - - * gnu C++ and Fortran use configure sub-files that update the - build flags and turn on warnings. The increase in warnings when - building these wrapper libraries is due to these flag changes - and not to a decrease in code quality. - - * The option to clear file buffers has been removed. Any buffer that - will eventually be written to disk will now always be memset - to zero. This prevents the previous contents of the buffer from - being written to the disk if the buffer contents are not - completely overwritten, which has security implications. - - - LFS Changes - - The way the autotools handle large file support (LFS) has been - overhauled in this release. - - * We assume ftello and fseeko exist - - * We no longer explicitly use the *64 I/O functions. Instead, we - rely on a mapping provided by _FILE_OFFSET_BITS or its equivalent. - - * _LARGEFILE(64)_SOURCE is no longer exported via AM_CPPFLAGS. - - - - Parallel Library - ----------------- - - Collective Metadata I/O - - Calls for HDF5 metadata can result in many small reads and writes. - On metadata reads, collective metadata I/O can improve performance - by allowing the library to perform optimizations when reading the - metadata by having one rank read the data and broadcasting it to - all other ranks. - - Collective metadata I/O improves metadata write performance through - the construction of an MPI derived datatype that is then written - collectively in a single call. For more information, see - https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesCollectiveMetadataIoDocs.html. - - - - Library - -------- - - Concurrent Access to HDF5 Files - Single Writer/ Multiple Reader (SWMR) - - The Single Writer/ Multiple Reader or SWMR feature enables users to - read data concurrently while writing it. Communications between the - processes and file locking are not required. The processes can run - on the same or on different platforms as long as they share a common - file system that is POSIX compliant. For more information, see the - Single-Writer/Multiple-Reader (SWMR) documentation at - https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesSwmrDocs.html. - - - Virtual Dataset (VDS) - - The VDS feature enables data to be accessed across HDF5 files - using standard HDF5 objects such as groups and datasets without - rewriting or rearranging the data. An HDF5 virtual dataset (VDS) - is an HDF5 dataset that is composed of source HDF5 datasets in - a predefined mapping. VDS can be used with the SWMR feature. For - documentation, check - https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesVirtualDatasetDocs.html. - - - Persistent Free File Space Tracking - - Usage patterns when working with an HDF5 file sometimes result in - wasted space within the file. This can also impair access times - when working with the resulting files. The new file space management - feature provides strategies for managing space in a file to improve - performance in both of these areas. For more information, see - https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesFileSpaceMgmtDocs.html. - - - Version 3 Metadata Cache - - The version 3 metadata cache moves management of metadata I/O from - the clients to the metadata cache proper. This change is essential for - SWMR and other features that have yet to be released. - - - - C++ Library - ------------ - - New Member Function Added to H5::ArrayType - - The assignment operator ArrayType::operator= was added because - ArrayType has pointer data members. - - (BMR - 2016/03/07, HDFFV-9562) - - - - Tools - ------ - - h5watch - - The h5watch tool allows users to output new records appended to - a dataset under SWMR access as it grows. The functionality is - similar to the Unix user command "tail" with the follow option, - which outputs appended data as the file grows. For more - information, see - https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesSwmrDocs.html#Tools. - - - h5format_convert - - The h5format_convert tool allows users to convert the indexing - type of a chunked dataset made with a 1.10.x version of the HDF5 - Library when the latest file format is used to the 1.8.x version 1 B-tree indexing - type. For example, datasets created using SWMR access, can be - converted to be accessed by the HDF5 1.18 library and tools. The - tool does not rewrite raw data, but it does rewrite HDF5 metadata. - - - - High-Level APIs - ---------------- - - H5DOappend - - The function appends data to a dataset along a specified dimension. - - - C Packet Table API - ------------------ - - Replacement of a Public Function with H5PTcreate - - The existing function H5PTcreate_fl limits applications so they - can use the deflate compression only. The public function - H5PTcreate has been added to replace H5PTcreate_fl. H5PTcreate - takes a property list identifier to provide flexibility on - creation properties. - - (BMR - 2016/03/04, HDFFV-8623) - - - New Public Functions: H5PTget_dataset and H5PTget_type - - Two accessor functions have been added. H5PTget_dataset returns - the identifier of the dataset associated with the packet table, - and H5PTget_type returns the identifier of the datatype used by - the packet table. - - (BMR, 2016/03/04, HDFFV-8623) - - - Regarding #ifdef VLPT_REMOVED - - The #ifdef VLPT_REMOVED blocks have been removed from the packet - table (PT) library source except for the following functions: - + H5PTis_varlen() has been made available again - + H5PTfree_vlen_readbuff() is now H5PTfree_vlen_buff() - - (BMR - 2016/03/04, HDFFV-442) - - C++ Packet Table API - -------------------- - - New Constructor Added to FL_PacketTable - - An overloaded constructor has been added to FL_PacketTable and - takes a property list identifier to provide flexibility on - creation properties. - - (BMR - 2016/03/08, HDFFV-8623) - - - New Public Functions - - Two accessor wrappers are added to class PacketTable. - PacketTable::GetDataset() returns the identifier of the dataset - associated with the packet table, and PacketTable::GetDatatype() - returns the identifier of the datatype that the packet table uses. - - (BMR - 2016/03/04, HDFFV-8623) - - - Member Functions with "char*" as an Argument - - Overloaded functions were added to provide the "const char*" - argument; the existing version will be deprecated in future - releases. - - (BMR - 2016/03/04, HDFFV-8623) - - - Regarding #ifdef VLPT_REMOVED - - The #ifdef VLPT_REMOVED blocks have been removed from the packet - table library source code except for the following functions: - + VL_PacketTable::IsVariableLength() was moved to PacketTable - + VL_PacketTable::FreeReadBuff() is now PacketTable::FreeBuff() - - (BMR - 2016/03/04, HDFFV-442) - - - - Java Wrapper Library - -------------------- - - The Java HDF5 JNI library has been integrated into the HDF5 repository. - The configure option is "--enable-java", and the CMake option is - HDF5_BUILD_JAVA:BOOL=ON. The package hierarchy has changed from the - HDF5 1.8 JNI, which was "ncsa.hdf.hdflib.hdf5", to HDF5 1.10, - "hdf.hdflib.hdf5". - - A number of new APIs were added including some for VDS and SWMR. - - - - Other Important Changes - ----------------------- - - The hid_t type was changed from 32-bit to a 64-bit value. - - - -Issues Addressed in this Release Since 1.10.0 -============================================= - - - h5diff would return from a compare attributes abnormally if one of the datatypes - was a vlen. This resulted in a memory leak as well as an incorrect report of - attribute comparison. - - Fixed. - (ADB - 2016/04/26, HDFFV-9784) - - - The JUnit-interface test may fail on Solaris platforms. The result of - a test for verifying the content of the error stack to stdout is - in a different order on Solaris then other platforms. - - This test is skipped on Solaris - (ADB - 2016/04/21, HDFFV-9734) - - - When building HDF5 with Java using CMake and specifying Debug for CMAKE_BUILD_TYPE, - there was a missing command argument for the tests of the examples. - - Fixed. - (ADB - 2016/04/21, HDFFV-9743) - - - Changed h5diff to print a warning when a dataset is virtual, enabling - the data to be compared. In addition h5repack failed to copy the data - of a virtual dataset to the new file. Function H5D__get_space_status changed - to correctly determine the H5D_space_status_t allocation value. - - CMake added the Fixed Array indexing tests that were only in the autotools - test scripts. - - Fixed and tests added for vds issues. - (ADB,NAF - 2016/04/21, HDFFV-9756) - - - CMake added the h5format_convert tool and tests that were only in the autotools - build and test scripts. The autotools test script was reworked to allow CMake - to execute the test suite in parallel. - - Also, h5clear tool and tests were added to the misc folder. - - Fixed. - (ADB - 2016/04/21, HDFFV-9766) - - - CMake added the h5watch tool and argument tests that were only in the autotools - build and test scripts. The POSIX only tests were not added to CMake. - - CMake HL tools files were refactored to move the CMake test scripts into each tool folder. - - Fixed. - (ADB - 2016/04/21, HDFFV-9770) - - - Configure fails to detect valid real KINDs on FreeBSD 9.3 (i386) with Fortran enabled. - - Fixed. Added the exponential option to SELECTED_REAL_KIND to distinguish - KINDs of same precision - (MSB - 2016/05/14,HDFFV-9912) - - - - Corrected the f90 H5AWRITE_F integer interface's buf to be INTENT(IN). - (MSB - 2016/05/14) - - - Configure fails in sed command on FreeBSD 9.3 (i386) with Fortran enabled. - - Fixed. - (MSB - 2016/05/14,HDFFV-9912) - - - Compile time error in H5f90global.F90 with IBM XL Fortran 14.1.0.13 on BG/Q with Fortran - enabled. - - Fixed. - (MSB - 2016/05/16,HDFFV-9917) - - - A cmake build with Fortran enabled does not install module h5fortkit - - Fixed. - (MSB - 2016/05/23,HDFFV-9923) - - -Issues Addressed in this Release Since alpha1 -============================================= - - - H5Pget_virtual_printf_gap, H5Pget_virtual_view, H5Pget_efile_prefix - - The correct access property list settings from the - H5Pget_virtual_printf_gap, H5Pget_virtual_view, and - H5Pget_efile_prefix function calls could not be retrieved - using H5Dget_access_plist(). - - Fixed. - - (DER and NAF - 2016/03/14, HDFFV-9716) - - - h5dump - - When h5dump was provided with the name of a non-existing file or - when optional arguments were the last option on the command line, - h5dump would segfault. - - Fixed. - - (ADB 2016/02/28 HDFFV-9639, HDFFV-9684) - - - No Error Message for Corrupt Metadata - - The HDF5 Library did not propagate an error when it encountered - corrupt metadata in an HDF5 file. The issue was fixed for a - specific file provided by a user. If you still see the problem, - please contact help@hdfgroup.org - - Fixed. - - (MC - 2016/02/18, HDFFV-9670) - - - Problem Reading Chunked Datasets with a String Datatype Larger - Than the Chunk Size in Bytes - - When the latest file format was used and when a chunked dataset - was created with a datatype with the size bigger than a chunk - size, the data could not be read back. The issue was reported - for chunked datasets with a string datatype and was confirmed - for other datatypes with the sizes bigger than the chunk size in - bytes. - - Fixed. - - (JM - 2016/02/13, HDFFV-9672) - - - Control over the Location of External Files - - Users were unable to specify the locations of external files. - - Two APIs - H5Pget_efile_prefix and H5Pset_efile_prefix - were - added so that users could specify the locations of external files. - - (DER - 2016/02/04, HDFFV-8740) - - - -Issues Addressed in this Release Since alpha0 -============================================= - - h5format_convert - - The h5format_convert tool did not downgrade the version of the - superblock. - - Fixed. The tool now will downgrade the version of the superblock. - - (EIP 2016/01/11) - - - Crashes with multiple threads: invalid pointers - - It was reported that alpha0 crashed when used with multiple - threads. The issue exists in the HDF5 Library versions 1.8 and - 1.9. The problem is related to a shared file pointer used in some - miscellaneous data structures. The thread-safe library exposed - paths in the library where a file pointer became invalid. - - The alpha1 release contains the fixes for the specific use case - as described in HDFFV-9643. We will keep working on identifying - and fixing other paths in the library with similar problems. - - (EIP - 2016/01/15, HDFFV-9643) - - - -Supported Platforms -=================== -The following platforms are supported and have been tested for this release. -They are built with the configure process unless specified otherwise. - - AIX 6.1 xlc/xlc_r 10.1.0.5 - (NASA G-ADA) xlC/xlC_r 10.1.0.5 - xlf90/xlf90_r 12.1.0.6 - - Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (mayll/platypus) Version 4.4.7 20120313 (Red Hat 4.4.7-16) - Version 4.9.3, Version 5.2.0 - PGI C, Fortran, C++ for 64-bit target on - x86-64; - Version 15.7-0 - Intel(R) C (icc), C++ (icpc), Fortran (icc) - compilers: - Version 15.0.3.187 Build 20150407 - MPICH 3.1.4 compiled with GCC 4.9.3 - - Linux 2.6.32-573.18.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11) - #1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11) - (ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11) - IBM XL C/C++ V13.1 - IBM XL Fortran V15.1 - - Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4) - Version 4.9.3, Version 5.2.0 - Intel(R) C (icc), C++ (icpc), Fortran (icc) - compilers: - Version 15.0.3.187 Build 20150407 - MPICH 3.1.4 compiled with GCC 4.9.3 - - SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc - (emu) Sun Fortran 95 8.6 SunOS_sparc - Sun C++ 5.12 SunOS_sparc - - Windows 7 Visual Studio 2012 w/ Intel Fortran 15 (cmake) - Visual Studio 2013 w/ Intel Fortran 15 (cmake) - Visual Studio 2015 w/ Intel Fortran 16 (cmake) - Cygwin(CYGWIN_NT-6.1 2.2.1(0.289/5/3) gcc(4.9.3) compiler and gfortran) - (cmake and autotools) - - Windows 7 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake) - Visual Studio 2013 w/ Intel Fortran 15 (cmake) - Visual Studio 2015 w/ Intel Fortran 16 (cmake) - - Windows 8.1 Visual Studio 2012 w/ Intel Fortran 15 (cmake) - Visual Studio 2013 w/ Intel Fortran 15 (cmake) - - Windows 8.1 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake) - Visual Studio 2013 w/ Intel Fortran 15 (cmake) - - Mac OS X Mt. Lion 10.8.5 Apple clang/clang++ version 5.1 from Xcode 5.1 - 64-bit gfortran GNU Fortran (GCC) 4.8.2 - (swallow/kite) Intel icc/icpc/ifort version 15.0.3 - - Mac OS X Mavericks 10.9.5 Apple clang/clang++ version 6.0 from Xcode 6.2 - 64-bit gfortran GNU Fortran (GCC) 4.9.2 - (wren/quail) Intel icc/icpc/ifort version 15.0.3 - - Mac OS X Yosemite 10.10.5 Apple clang/clang++ version 6.0 from Xcode 7.0 - 64-bit gfortran GNU Fortran (GCC) 4.9.2 - (osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3 - - Mac OS X El Capitan 10.11.4 Apple clang/clang++ version 7.3.0 from Xcode 7.3 - 64-bit gfortran GNU Fortran (GCC) 5.2.0 - (osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3 - - - -Tested Configuration Features Summary -===================================== - - In the tables below - y = tested - n = not tested in this release - C = Cluster - W = Workstation - x = not working in this release - dna = does not apply - ( ) = footnote appears below second table - <blank> = testing incomplete on this feature or platform - -Platform C F90/ F90 C++ zlib SZIP - parallel F2003 parallel -SunOS 5.11 32-bit n y/y n y y y -SunOS 5.11 64-bit n y/y n y y y -Windows 7 y y/y n y y y -Windows 7 x64 y y/y n y y y -Windows 7 Cygwin n y/y n y y n -Windows 8.1 n y/y n y y y -Windows 8.1 x64 n y/y n y y y -Mac OS X Mountain Lion 10.8.5 64-bit n y/y n y y y -Mac OS X Mavericks 10.9.5 64-bit n y/y n y y y -Mac OS X Yosemeti 10.10.5 64-bit n y/y n y y y -AIX 6.1 32- and 64-bit n y/n n y y y -CentOS 6.7 Linux 2.6.32 x86_64 GNU y y/y y y y y -CentOS 6.7 Linux 2.6.32 x86_64 Intel n y/y n y y y -CentOS 6.7 Linux 2.6.32 x86_64 PGI n y/y n y y y -CentOS 7.1 Linux 3.10.0 x86_64 GNU y y/y y y y y -CentOS 7.1 Linux 3.10.0 x86_64 Intel n y/y n y y y -Linux 2.6.32-431.11.2.el6.ppc64 n y/n n y y y - -Platform Shared Shared Shared Thread- - C libs F90 libs C++ libs safe -SunOS 5.11 32-bit y y y y -SunOS 5.11 64-bit y y y y -Windows 7 y y y y -Windows 7 x64 y y y y -Windows 7 Cygwin n n n y -Windows 8.1 y y y y -Windows 8.1 x64 y y y y -Mac OS X Mountain Lion 10.8.5 64-bit y n y y -Mac OS X Mavericks 10.9.5 64-bit y n y y -Mac OS X Yosemeti 10.10.5 64-bit y n y y -AIX 6.1 32- and 64-bit y n n y -CentOS 6.7 Linux 2.6.32 x86_64 GNU y y y y -CentOS 6.7 Linux 2.6.32 x86_64 Intel y y y y -CentOS 6.7 Linux 2.6.32 x86_64 PGI y y y y -CentOS 7.1 Linux 3.10.0 x86_64 GNU y y y y -CentOS 7.1 Linux 3.10.0 x86_64 Intel y y y y -Linux 2.6.32-431.11.2.el6.ppc64 y y y y - -Compiler versions for each platform are listed in the preceding -"Supported Platforms" table. - - - -More Tested Platforms -===================== -The following platforms are not supported but have been tested for this release. - - Linux 2.6.18-431.11.2.el6 g95 (GCC 4.0.3 (g95 0.94!) - #1 SMP x86_64 GNU/Linux - (platypus) - - Windows 7 Visual Studio 2008 (cmake) - - Windows 7 x64 Visual Studio 2008 (cmake) - - Windows 7 x64 Visual Studio 2010 (cmake) with SWMR using GPFS - - Windows 10 Visual Studio 2013 w/ Intel Fortran 15 (cmake) - - Windows 10 x64 Visual Studio 2013 w/ Intel Fortran 15 (cmake) - - Debian7.5.0 3.2.0-4-amd64 #1 SMP Debian 3.2.51-1 x86_64 GNU/Linux - gcc (Debian 4.7.2-5) 4.7.2 - GNU Fortran (Debian 4.7.2-5) 4.7.2 - (cmake and autotools) - - Fedora20 3.15.3-200.fc20.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux - gcc (GCC) 4.8.3 20140624 (Red Hat 4.8.3-1) - GNU Fortran (GCC) 4.8.3 20140624 (Red Hat 4.8.3-1) - (cmake and autotools) - - SUSE 13.1 3.11.10-17-desktop #1 SMP PREEMPT x86_64 x86_64 x86_64 GNU/Linux - gcc (SUSE Linux) 4.8.1 - GNU Fortran (SUSE Linux) 4.8.1 - (cmake and autotools) - - Ubuntu 14.04 3.13.0-35-generic #62-Ubuntu SMP x86_64 GNU/Linux - gcc (Ubuntu/Linaro 4.9.1-0ubuntu1) 4.9.1 - GNU Fortran (Ubuntu/Linaro 4.9.1-0ubuntu1) 4.9.1 - (cmake and autotools) - - hopper.nersc.gov PrgEnv-gnu/5.2.40 - gcc (GCC) 4.9.2 20141030 (Cray Inc.) - GNU Fortran (GCC) 4.9.2 20141030 (Cray Inc.) - g++ (GCC) 4.9.2 20141030 (Cray Inc.) - - - -Known Problems and Limitations -============================== -This section contains the list of known problems and limitations introduced -in this release of HDF5. - -Note: this list is not exhaustive of all known issues discovered in HDF5 -software to date. For a list of significant problems and known workarounds -identified in past releases, please refer to: - -https://www.hdfgroup.org/HDF5/release/known_problems/ - -The HDF Group also maintains a JIRA issue-tracking database which is used to -capture all known issues which are too numerous to reasonably list in this -document. The HDF Group is taking steps to make our JIRA issue database -open to the public, and this section will refer to that database in a future -release. In the meantime, please contact help@hdfgroup.org if you come across -an issue not listed here or at the link above, and we will provide any -information about known workarounds that we have or add it to our list of -known issues if it is a new issue. - - - The flush/refresh test occasionally fails on OS X platforms. This is - being investigated but no fix or workaround is available at this time. - (DER - 2016/03/22, HDFFV-9731) - - - The VDS/SWMR test will fail with a segmentation fault if the library - is built with --enable-using-memchecker. The is due to a VDS shutdown - procedure freeing a shared resource too early when the memory - checker changes are built. This problem does not arise when the - memory checker changes are not used since the internal library free - lists behave differently. The memory checker configure option should - normally only be used under special circumstances so this should not - affect most users. Users should be aware that the --enable-using-memchecker - + VDS combination may cause a segfault, however, so Valgrind et al. may - have to be used with an HDF5 library built without the feature if this - proves to be a problem. - (DER - 2016/03/21, HDFFV-9732) - - - SWMR feature limitations - The SWMR feature will only work if an HDF5 file under SWMR access resides - on a file system that obeys POSIX write() ordering semantics. Because of - this, SWMR will not work on network file systems such as NFS or SMB/Windows - file shares since those systems do not guarantee write odering. SWMR - regression tests are likely to fail if run on a network file system. SWMR - is currently not tested on Windows though it can be tested manually - (some of the SWMR test programs are built by CMake), and there are no - obvious reasons for it to not work on NTFS or GPFS. - (EIP - 2016/03/20, HDFFV-9733) - - - VDS feature limitation - Currently, the path to a VDS source file is interpreted as relative to the - directory where the executable program runs and not to the HDF5 file with - the VDS dataset unless a full path to the source file is specified during - the mapping. - (EIP - 2016/03/20, HDFFV-9724) - - - The H5Lexists API changed behavior in HDF5-1.10 when used with a file handle - and root group name ("/"): - - H5Lexists(fileid, "/") - - In HDF5-1.8 it returns false (0) and in HDF5-1.10 it returns true (1). - The documentation will be updated with information regarding this change. - (LRK - 2016/03/30, HDFFV-8746) - - -%%%%1.10.0%%%% - -HDF5 version 1.10.0 released on 2016-03-30 -================================================================================ - - - -INTRODUCTION - -This document describes the differences between HDF5-1.8 series and -HDF5 1.10.0 releases, and contains information on the platforms -tested. - -Links to HDF5 1.10.0 source code can be found on The HDF Group's -development FTP server at the following location: - - https://www.hdfgroup.org/HDF5/release/obtain5110.html - -User documentation can be accessed directly at this location: - - https://www.hdfgroup.org/HDF5/docNewFeatures/ - -For more information, see the HDF5 home page: - - https://www.hdfgroup.org/HDF5/ - -If you have any questions or comments, please send them to the HDF -Help Desk: - - help@hdfgroup.org - - - -CONTENTS - -- New Features -- Issues Addressed in this Release -- Supported Platforms -- Tested Configuration Features Summary -- More Tested Platforms -- Known Problems and Limitations - - - -New Features -============ -This release supports the following features: - - Configuration - ------------- - - API Compatibility with HDF5 1.8 Flag Was Added - - The 1.10 version of the HDF5 Library can be configured to operate - identically to the 1.8 library with the --with-default-api-version=v18 - configure flag. This allows existing code to be compiled with the 1.10 - library without requiring immediate changes to the application source - code. For additional configuration options and other details, see - "API Compatibility Macros in HDF5" at - https://www.hdfgroup.org/HDF5/doc/RM/APICompatMacros.html. - - - Autotools Configuration Has Been Extensively Reworked - - The autotools configuration options have been updated to allow more - fine-grained control of the build options and to correct some bugs. - See configure --help for comprehensive information on each option. - - Specific changes: - - * --enable-debug and --enable-production are no longer accepted. - Use --enable-build-mode=(debug | production) instead. These set - appropriate defaults for symbols, optimizations, and other - configuration options. These defaults can be overridden by the - user. - - * Extra debug output messages are no longer enabled with - --enable-debug=<package list>. Use --enable-internal-debug=<pkg list> - instead. - - * A new --enable-symbols option allows symbols to be generated - independently of the build mode. --disable-symbols can be used - to strip symbols from the binary. - - * A new --enable-asserts option sets/unsets NDEBUG. This is - independent of the build mode. This also enables some extra - low-overhead debug checks in the library. - - * A new --enable-profiling option sets profiling flags. This is - independent of the build mode. - - * A new --enable-optimization option sets the optimization level. - This is independent of the build mode. - - * Many of these options can take a flags string that will be used - to build the library. This can be useful for specifying custom - optimization flags such as -Os and -Ofast. - - * gnu C++ and Fortran use configure sub-files that update the - build flags and turn on warnings. The increase in warnings when - building these wrapper libraries is due to these flag changes - and not to a decrease in code quality. - - * The option to clear file buffers has been removed. Any buffer that - will eventually be written to disk will now always be memset - to zero. This prevents the previous contents of the buffer from - being written to the disk if the buffer contents are not - completely overwritten, which has security implications. - - - LFS Changes - - The way the autotools handle large file support (LFS) has been - overhauled in this release. - - * We assume ftello and fseeko exist - - * We no longer explicitly use the *64 I/O functions. Instead, we - rely on a mapping provided by _FILE_OFFSET_BITS or its equivalent. - - * _LARGEFILE(64)_SOURCE is no longer exported via AM_CPPFLAGS. - - - - Parallel Library - ----------------- - - Collective Metadata I/O - - Calls for HDF5 metadata can result in many small reads and writes. - On metadata reads, collective metadata I/O can improve performance - by allowing the library to perform optimizations when reading the - metadata by having one rank read the data and broadcasting it to - all other ranks. - - Collective metadata I/O improves metadata write performance through - the construction of an MPI derived datatype that is then written - collectively in a single call. For more information, see - https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesCollectiveMetadataIoDocs.html. - - - - Library - -------- - - Concurrent Access to HDF5 Files - Single Writer/ Multiple Reader (SWMR) - - The Single Writer/ Multiple Reader or SWMR feature enables users to - read data concurrently while writing it. Communications between the - processes and file locking are not required. The processes can run - on the same or on different platforms as long as they share a common - file system that is POSIX compliant. For more information, see the - Single-Writer/Multiple-Reader (SWMR) documentation at - https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesSwmrDocs.html. - - - Virtual Dataset (VDS) - - The VDS feature enables data to be accessed across HDF5 files - using standard HDF5 objects such as groups and datasets without - rewriting or rearranging the data. An HDF5 virtual dataset (VDS) - is an HDF5 dataset that is composed of source HDF5 datasets in - a predefined mapping. VDS can be used with the SWMR feature. For - documentation, check - https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesVirtualDatasetDocs.html. - - - Persistent Free File Space Tracking - - Usage patterns when working with an HDF5 file sometimes result in - wasted space within the file. This can also impair access times - when working with the resulting files. The new file space management - feature provides strategies for managing space in a file to improve - performance in both of these areas. For more information, see - https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesFileSpaceMgmtDocs.html. - - - Version 3 Metadata Cache - - The version 3 metadata cache moves management of metadata I/O from - the clients to the metadata cache proper. This change is essential for - SWMR and other features that have yet to be released. - - - - C++ Library - ------------ - - New Member Function Added to H5::ArrayType - - The assignment operator ArrayType::operator= was added because - ArrayType has pointer data members. - - (BMR - 2016/03/07, HDFFV-9562) - - - - Tools - ------ - - h5watch - - The h5watch tool allows users to output new records appended to - a dataset under SWMR access as it grows. The functionality is - similar to the Unix user command "tail" with the follow option, - which outputs appended data as the file grows. For more - information, see - https://www.hdfgroup.org/HDF5/docNewFeatures/NewFeaturesSwmrDocs.html#Tools. - - - h5format_convert - - The h5format_convert tool allows users to convert the indexing - type of a chunked dataset made with a 1.10.x version of the HDF5 - Library when the latest file format is used to the 1.8.x version 1 B-tree indexing - type. For example, datasets created using SWMR access, can be - converted to be accessed by the HDF5 1.18 library and tools. The - tool does not rewrite raw data, but it does rewrite HDF5 metadata. - - - - High-Level APIs - ---------------- - - H5DOappend - - The function appends data to a dataset along a specified dimension. - - - C Packet Table API - ------------------ - - Replacement of a Public Function with H5PTcreate - - The existing function H5PTcreate_fl limits applications so they - can use the deflate compression only. The public function - H5PTcreate has been added to replace H5PTcreate_fl. H5PTcreate - takes a property list identifier to provide flexibility on - creation properties. - - (BMR - 2016/03/04, HDFFV-8623) - - - New Public Functions: H5PTget_dataset and H5PTget_type - - Two accessor functions have been added. H5PTget_dataset returns - the identifier of the dataset associated with the packet table, - and H5PTget_type returns the identifier of the datatype used by - the packet table. - - (BMR, 2016/03/04, HDFFV-8623) - - - Regarding #ifdef VLPT_REMOVED - - The #ifdef VLPT_REMOVED blocks have been removed from the packet - table (PT) library source except for the following functions: - + H5PTis_varlen() has been made available again - + H5PTfree_vlen_readbuff() is now H5PTfree_vlen_buff() - - (BMR - 2016/03/04, HDFFV-442) - - C++ Packet Table API - -------------------- - - New Constructor Added to FL_PacketTable - - An overloaded constructor has been added to FL_PacketTable and - takes a property list identifier to provide flexibility on - creation properties. - - (BMR - 2016/03/08, HDFFV-8623) - - - New Public Functions - - Two accessor wrappers are added to class PacketTable. - PacketTable::GetDataset() returns the identifier of the dataset - associated with the packet table, and PacketTable::GetDatatype() - returns the identifier of the datatype that the packet table uses. - - (BMR - 2016/03/04, HDFFV-8623) - - - Member Functions with "char*" as an Argument - - Overloaded functions were added to provide the "const char*" - argument; the existing version will be deprecated in future - releases. - - (BMR - 2016/03/04, HDFFV-8623) - - - Regarding #ifdef VLPT_REMOVED - - The #ifdef VLPT_REMOVED blocks have been removed from the packet - table library source code except for the following functions: - + VL_PacketTable::IsVariableLength() was moved to PacketTable - + VL_PacketTable::FreeReadBuff() is now PacketTable::FreeBuff() - - (BMR - 2016/03/04, HDFFV-442) - - - - Java Wrapper Library - -------------------- - - The Java HDF5 JNI library has been integrated into the HDF5 repository. - The configure option is "--enable-java", and the CMake option is - HDF5_BUILD_JAVA:BOOL=ON. The package hierarchy has changed from the - HDF5 1.8 JNI, which was "ncsa.hdf.hdflib.hdf5", to HDF5 1.10, - "hdf.hdflib.hdf5". - - A number of new APIs were added including some for VDS and SWMR. - - - - Other Important Changes - ----------------------- - - The hid_t type was changed from 32-bit to a 64-bit value. - - - -Issues Addressed in this Release Since alpha1 -============================================= - - - H5Pget_virtual_printf_gap, H5Pget_virtual_view, H5Pget_efile_prefix - - The correct access property list settings from the - H5Pget_virtual_printf_gap, H5Pget_virtual_view, and - H5Pget_efile_prefix function calls could not be retrieved - using H5Dget_access_plist(). - - Fixed. - - (DER and NAF - 2016/03/14, HDFFV-9716) - - - h5dump - - When h5dump was provided with the name of a non-existing file or - when optional arguments were the last option on the command line, - h5dump would segfault. - - Fixed. - - (ADB 2016/02/28 HDFFV-9639, HDFFV-9684) - - - No Error Message for Corrupt Metadata - - The HDF5 Library did not propagate an error when it encountered - corrupt metadata in an HDF5 file. The issue was fixed for a - specific file provided by a user. If you still see the problem, - please contact help@hdfgroup.org - - Fixed. - - (MC - 2016/02/18, HDFFV-9670) - - - Problem Reading Chunked Datasets with a String Datatype Larger - Than the Chunk Size in Bytes - - When the latest file format was used and when a chunked dataset - was created with a datatype with the size bigger than a chunk - size, the data could not be read back. The issue was reported - for chunked datasets with a string datatype and was confirmed - for other datatypes with the sizes bigger than the chunk size in - bytes. - - Fixed. - - (JM - 2016/02/13, HDFFV-9672) - - - Control over the Location of External Files - - Users were unable to specify the locations of external files. - - Two APIs - H5Pget_efile_prefix and H5Pset_efile_prefix - were - added so that users could specify the locations of external files. - - (DER - 2016/02/04, HDFFV-8740) - - - -Issues Addressed in this Release Since alpha0 -============================================= - - h5format_convert - - The h5format_convert tool did not downgrade the version of the - superblock. - - Fixed. The tool now will downgrade the version of the superblock. - - (EIP 2016/01/11) - - - Crashes with multiple threads: invalid pointers - - It was reported that alpha0 crashed when used with multiple - threads. The issue exists in the HDF5 Library versions 1.8 and - 1.9. The problem is related to a shared file pointer used in some - miscellaneous data structures. The thread-safe library exposed - paths in the library where a file pointer became invalid. - - The alpha1 release contains the fixes for the specific use case - as described in HDFFV-9643. We will keep working on identifying - and fixing other paths in the library with similar problems. - - (EIP - 2016/01/15, HDFFV-9643) - - - -Supported Platforms -=================== -The following platforms are supported and have been tested for this release. -They are built with the configure process unless specified otherwise. - - AIX 6.1 xlc/xlc_r 10.1.0.5 - (NASA G-ADA) xlC/xlC_r 10.1.0.5 - xlf90/xlf90_r 12.1.0.6 - - Linux 2.6.32-573.18.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (mayll/platypus) Version 4.4.7 20120313 (Red Hat 4.4.7-16) - Version 4.9.3, Version 5.2.0 - PGI C, Fortran, C++ for 64-bit target on - x86-64; - Version 15.7-0 - Intel(R) C (icc), C++ (icpc), Fortran (icc) - compilers: - Version 15.0.3.187 Build 20150407 - MPICH 3.1.4 compiled with GCC 4.9.3 - - Linux 2.6.32-504.8.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11) - #1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11) - (ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-11) - IBM XL C/C++ V13.1 - IBM XL Fortran V15.1 - - Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++) - #1 SMP x86_64 GNU/Linux compilers: - (kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4) - Version 4.9.3, Version 5.2.0 - Intel(R) C (icc), C++ (icpc), Fortran (icc) - compilers: - Version 15.0.3.187 Build 20150407 - MPICH 3.1.4 compiled with GCC 4.9.3 - - SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc - (emu) Sun Fortran 95 8.6 SunOS_sparc - Sun C++ 5.12 SunOS_sparc - - Windows 7 Visual Studio 2012 w/ Intel Fortran 15 (cmake) - Visual Studio 2013 w/ Intel Fortran 15 (cmake) - Visual Studio 2015 w/ Intel Fortran 16 (cmake) - Cygwin(CYGWIN_NT-6.1 2.2.1(0.289/5/3) gcc(4.9.3) compiler and gfortran) - (cmake and autotools) - - Windows 7 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake) - Visual Studio 2013 w/ Intel Fortran 15 (cmake) - Visual Studio 2015 w/ Intel Fortran 16 (cmake) - - Windows 8.1 Visual Studio 2012 w/ Intel Fortran 15 (cmake) - Visual Studio 2013 w/ Intel Fortran 15 (cmake) - - Windows 8.1 x64 Visual Studio 2012 w/ Intel Fortran 15 (cmake) - Visual Studio 2013 w/ Intel Fortran 15 (cmake) - - Mac OS X Mt. Lion 10.8.5 Apple clang/clang++ version 5.1 from Xcode 5.1 - 64-bit gfortran GNU Fortran (GCC) 4.8.2 - (swallow/kite) Intel icc/icpc/ifort version 15.0.3 - - Mac OS X Mavericks 10.9.5 Apple clang/clang++ version 6.0 from Xcode 6.2.0 - 64-bit gfortran GNU Fortran (GCC) 4.9.2 - (wren/quail) Intel icc/icpc/ifort version 15.0.3 - - Mac OS X Yosemite 10.10.5 Apple clang/clang++ version 6.0 from Xcode 7.0.0 - 64-bit gfortran GNU Fortran (GCC) 4.9.2 - (osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3 - - Mac OS X El Capitan 10.11.3 Apple clang/clang++ version 7.0.2 from Xcode 7.0.2 - 64-bit gfortran GNU Fortran (GCC) 5.2.0 - (osx1010dev/osx1010test) Intel icc/icpc/ifort version 15.0.3 - - - -Tested Configuration Features Summary -===================================== - - In the tables below - y = tested - n = not tested in this release - C = Cluster - W = Workstation - x = not working in this release - dna = does not apply - ( ) = footnote appears below second table - <blank> = testing incomplete on this feature or platform - -Platform C F90/ F90 C++ zlib SZIP - parallel F2003 parallel -SunOS 5.11 32-bit n y/y n y y y -SunOS 5.11 64-bit n y/y n y y y -Windows 7 y y/y n y y y -Windows 7 x64 y y/y n y y y -Windows 7 Cygwin n y/y n y y n -Windows 8.1 n y/y n y y y -Windows 8.1 x64 n y/y n y y y -Mac OS X Mountain Lion 10.8.5 64-bit n y/y n y y y -Mac OS X Mavericks 10.9.5 64-bit n y/y n y y y -Mac OS X Yosemeti 10.10.5 64-bit n y/y n y y y -AIX 6.1 32- and 64-bit n y/n n y y y -CentOS 6.7 Linux 2.6.32 x86_64 GNU y y/y y y y y -CentOS 6.7 Linux 2.6.32 x86_64 Intel n y/y n y y y -CentOS 6.7 Linux 2.6.32 x86_64 PGI n y/y n y y y -CentOS 7.1 Linux 3.10.0 x86_64 GNU y y/y y y y y -CentOS 7.1 Linux 3.10.0 x86_64 Intel n y/y n y y y -Linux 2.6.32-431.11.2.el6.ppc64 n y/n n y y y - -Platform Shared Shared Shared Thread- - C libs F90 libs C++ libs safe -SunOS 5.11 32-bit y y y y -SunOS 5.11 64-bit y y y y -Windows 7 y y y y -Windows 7 x64 y y y y -Windows 7 Cygwin n n n y -Windows 8.1 y y y y -Windows 8.1 x64 y y y y -Mac OS X Mountain Lion 10.8.5 64-bit y n y y -Mac OS X Mavericks 10.9.5 64-bit y n y y -Mac OS X Yosemeti 10.10.5 64-bit y n y y -AIX 6.1 32- and 64-bit y n n y -CentOS 6.7 Linux 2.6.32 x86_64 GNU y y y y -CentOS 6.7 Linux 2.6.32 x86_64 Intel y y y y -CentOS 6.7 Linux 2.6.32 x86_64 PGI y y y y -CentOS 7.1 Linux 3.10.0 x86_64 GNU y y y y -CentOS 7.1 Linux 3.10.0 x86_64 Intel y y y y -Linux 2.6.32-431.11.2.el6.ppc64 y y y y - -Compiler versions for each platform are listed in the preceding -"Supported Platforms" table. - - - -More Tested Platforms -===================== -The following platforms are not supported but have been tested for this release. - - Linux 2.6.18-431.11.2.el6 g95 (GCC 4.0.3 (g95 0.94!) - #1 SMP x86_64 GNU/Linux - (platypus) - - Windows 7 Visual Studio 2008 (cmake) - - Windows 7 x64 Visual Studio 2008 (cmake) - - Windows 7 x64 Visual Studio 2010 (cmake) with SWMR using GPFS - - Windows 10 Visual Studio 2013 w/ Intel Fortran 15 (cmake) - - Windows 10 x64 Visual Studio 2013 w/ Intel Fortran 15 (cmake) - - Debian7.5.0 3.2.0-4-amd64 #1 SMP Debian 3.2.51-1 x86_64 GNU/Linux - gcc (Debian 4.7.2-5) 4.7.2 - GNU Fortran (Debian 4.7.2-5) 4.7.2 - (cmake and autotools) - - Fedora20 3.15.3-200.fc20.x86_64 #1 SMP x86_64 x86_64 x86_64 GNU/Linux - gcc (GCC) 4.8.3 20140624 (Red Hat 4.8.3-1) - GNU Fortran (GCC) 4.8.3 20140624 (Red Hat 4.8.3-1) - (cmake and autotools) - - SUSE 13.1 3.11.10-17-desktop #1 SMP PREEMPT x86_64 x86_64 x86_64 GNU/Linux - gcc (SUSE Linux) 4.8.1 - GNU Fortran (SUSE Linux) 4.8.1 - (cmake and autotools) - - Ubuntu 14.04 3.13.0-35-generic #62-Ubuntu SMP x86_64 GNU/Linux - gcc (Ubuntu/Linaro 4.9.1-0ubuntu1) 4.9.1 - GNU Fortran (Ubuntu/Linaro 4.9.1-0ubuntu1) 4.9.1 - (cmake and autotools) - - hopper.nersc.gov PrgEnv-gnu/5.2.40 - gcc (GCC) 4.9.2 20141030 (Cray Inc.) - GNU Fortran (GCC) 4.9.2 20141030 (Cray Inc.) - g++ (GCC) 4.9.2 20141030 (Cray Inc.) - - - -Known Problems and Limitations -============================== -This section contains the list of known problems and limitations introduced -in this release of HDF5. - -Note: this list is not exhaustive of all known issues discovered in HDF5 -software to date. For a list of significant problems and known workarounds -identified in past releases, please refer to: - -https://www.hdfgroup.org/HDF5/release/known_problems/ - -The HDF Group also maintains a JIRA issue-tracking database which is used to -capture all known issues which are too numerous to reasonably list in this -document. The HDF Group is taking steps to make our JIRA issue database -open to the public, and this section will refer to that database in a future -release. In the meantime, please contact help@hdfgroup.org if you come across -an issue not listed here or at the link above, and we will provide any -information about known workarounds that we have or add it to our list of -known issues if it is a new issue. - - - The JUnit-interface test may fail on Solaris platforms. The result of - a test for verifying the content of the error stack to stdout is - in a different order on Solaris then other platforms. Use make -i option - to test beyond the java/test folder. - (ADB - 2016/03/22, HDFFV-9734) - - - The flush/refresh test occasionally fails on OS X platforms. This is - being investigated but no fix or workaround is available at this time. - (DER - 2016/03/22, HDFFV-9731) - - - The VDS/SWMR test will fail with a segmentation fault if the library - is built with --enable-using-memchecker. The is due to a VDS shutdown - procedure freeing a shared resource too early when the memory - checker changes are built. This problem does not arise when the - memory checker changes are not used since the internal library free - lists behave differently. The memory checker configure option should - normally only be used under special circumstances so this should not - affect most users. Users should be aware that the --enable-using-memchecker - + VDS combination may cause a segfault, however, so Valgrind et al. may - have to be used with an HDF5 library built without the feature if this - proves to be a problem. - (DER - 2016/03/21, HDFFV-9732) - - - SWMR feature limitations - The SWMR feature will only work if an HDF5 file under SWMR access resides - on a file system that obeys POSIX write() ordering semantics. Because of - this, SWMR will not work on network file systems such as NFS or SMB/Windows - file shares since those systems do not guarantee write odering. SWMR - regression tests are likely to fail if run on a network file system. SWMR - is currently not tested on Windows though it can be tested manually - (some of the SWMR test programs are built by CMake), and there are no - obvious reasons for it to not work on NTFS or GPFS. - (EIP - 2016/03/20, HDFFV-9733) - - - VDS feature limitation - Currently, the path to a VDS source file is interpreted as relative to the - directory where the executable program runs and not to the HDF5 file with - the VDS dataset unless a full path to the source file is specified during - the mapping. - (EIP - 2016/03/20, HDFFV-9724) - - - When building HDF5 with Java using CMake and specifying Debug for CMAKE_BUILD_TYPE, - there is a missing command argument for the tests of the examples. - - This error can be avoided by not building Java with Debug, HDF5_BUILD_JAVA:BOOL=OFF, - or not building Examples, HDF5_BUILD_EXAMPLES:BOOL=OFF. - (LRK - 2016/03/30, HDFFV-9743) - - - The H5Lexists API changed behavior in HDF5-1.10 when used with a file handle - and root group name ("/"): - - H5Lexists(fileid, "/") - - In HDF5-1.8 it returns false (0) and in HDF5-1.10 it returns true (1). - The documentation will be updated with information regarding this change. - (LRK - 2016/03/30, HDFFV-8746) - |