diff options
Diffstat (limited to 'release_docs/HISTORY-1_12.txt')
| -rw-r--r-- | release_docs/HISTORY-1_12.txt | 628 |
1 files changed, 628 insertions, 0 deletions
diff --git a/release_docs/HISTORY-1_12.txt b/release_docs/HISTORY-1_12.txt new file mode 100644 index 0000000..6d0c1e2 --- /dev/null +++ b/release_docs/HISTORY-1_12.txt @@ -0,0 +1,628 @@ +HDF5 History +============ + +This file contains development history of the HDF5 1.12 branch + +01. Release Information for hdf5-1.12.0 + +[Search on the string '%%%%' for section breaks of each release.] + +%%%%1.12.0%%%% + +HDF5 version 1.12.0 released on 2020-02-28 +================================================================================ + + +INTRODUCTION + +This document describes the new features introduced in the HDF5 1.12.0 release. +It contains information on the platforms tested and known problems in this +release. For more details check the HISTORY*.txt files in the HDF5 source. + +Note that documentation in the links below will be updated at the time of the +release. + +Links to HDF5 documentation can be found on The HDF5 web page: + + https://portal.hdfgroup.org/display/HDF5/HDF5 + +The official HDF5 releases can be obtained from: + + https://www.hdfgroup.org/downloads/hdf5/ + +More information about the new features can be found at: + + https://portal.hdfgroup.org/display/HDF5/New+Features+in+HDF5+Release+1.12 + +If you have any questions or comments, please send them to the HDF Help Desk: + + help@hdfgroup.org + + +CONTENTS + +- New Features +- Support for new platforms and languages +- Bug Fixes since HDF5-1.12.0-alpha1 +- Major Bug Fixes since HDF5-1.10.0 +- Supported Platforms +- Tested Configuration Features Summary +- More Tested Platforms +- Known Problems +- CMake vs. Autotools installations + + +New Features +============ + + Configuration: + ------------- + - Added test script for file size compare + + If CMake minimum version is at least 3.14, the fileCompareTest.cmake + script will compare file sizes. + + (ADB - 2020/02/24, HDFFV-11036) + + - Update CMake minimum version to 3.12 + + Updated CMake minimum version to 3.12 and added version checks + for Windows features. + + (ADB - 2020/02/05, TRILABS-142) + + - Fixed CMake include properties for Fortran libraries + + Corrected the library properties for Fortran to use the + correct path for the Fortran module files. + + (ADB - 2020/02/04, HDFFV-11012) + + - Added common warnings files for gnu and intel + + Added warnings files to use one common set of flags + during configure for both autotools and CMake build + systems. The initial implementation only affects a + general set of flags for gnu and intel compilers. + + (ADB - 2020/01/17) + + - Added new options to CMake for control of testing + + Added CMake options (default ON); + HDF5_TEST_SERIAL AND/OR HDF5_TEST_PARALLEL + combined with: + HDF5_TEST_TOOLS + HDF5_TEST_EXAMPLES + HDF5_TEST_SWMR + HDF5_TEST_FORTRAN + HDF5_TEST_CPP + HDF5_TEST_JAVA + + (ADB - 2020/01/15, HDFFV-11001) + + - Added Clang sanitizers to CMake for analyzer support if compiler is clang. + + Added CMake code and files to execute the Clang sanitizers if + HDF5_ENABLE_SANITIZERS is enabled and the USE_SANITIZER option + is set to one of the following: + Address + Memory + MemoryWithOrigins + Undefined + Thread + Leak + 'Address;Undefined' + + (ADB - 2019/12/12, TRILAB-135) + + - Update CMake for VS2019 support + + CMake added support for VS2019 in version 3.15. Changes to the CMake + generator setting required changes to scripts. Also updated version + references in CMake files as necessary. + + (ADB - 2019/11/18, HDFFV-10962) + + + Library: + -------- + - Refactored public exposure of haddr_t type in favor of "object tokens" + + To better accommodate HDF5 VOL connectors where "object addresses in a file" + may not make much sense, the following changes were made to the library: + + * Introduced new H5O_token_t "object token" type, which represents a + unique and permanent identifier for referencing an HDF5 object within + a container; these "object tokens" are meant to replace object addresses. + Along with the new type, a new H5Oopen_by_token API call was introduced + to open an object by a token, similar to how object addresses were + previously used with H5Oopen_by_addr. + + * Introduced new H5Lget_info2, H5Lget_info_by_idx2, H5Literate2, H5Literate_by_name2, + H5Lvisit2 and H5Lvisit_by_name2 API calls, along with their associated H5L_info2_t + struct and H5L_iterate2_t callback function, which work with the newly-introduced + object tokens, instead of object addresses. The original functions have been + renamed to version 1 functions and are deprecated in favor of the new version 2 + functions. The H5L_info_t and H5L_iterate_t types have been renamed to version 1 + types and are now deprecated in favor of their version 2 counterparts. For each of + the functions and types, compatibility macros take place of the original symbols. + + * Introduced new H5Oget_info3, H5Oget_info_by_name3, H5Oget_info_by_idx3, + H5Ovisit3 and H5Ovisit_by_name3 API calls, along with their associated H5O_info2_t + struct and H5O_iterate2_t callback function, which work with the newly-introduced + object tokens, instead of object addresses. The version 2 functions are now + deprecated in favor of the version 3 functions. The H5O_info_t and H5O_iterate_t + types have been renamed to version 1 types and are now deprecated in favor of their + version 2 counterparts. For each, compatibility macros take place of the original + symbols. + + * Introduced new H5Oget_native_info, H5Oget_native_info_by_name and + H5Oget_native_info_by_idx API calls, along with their associated H5O_native_info_t + struct, which are used to retrieve the native HDF5 file format-specific information + about an object. This information (such as object header info and B-tree/heap info) + has been removed from the new H5O_info2_t struct so that the more generic + H5Oget_info(_by_name/_by_idx)3 routines will not try to retrieve it for non-native + VOL connectors. + + * Added new H5Otoken_cmp, H5Otoken_to_str and H5Otoken_from_str routines to compare + two object tokens, convert an object token into a nicely-readable string format and + to convert an object token string back into a real object token, respectively. + + (DER, QAK, JTH - 2020/01/16) + + - Virtual Object Layer (VOL) + + In this major HDF5 release we introduce HDF5 Virtual Object Layer (VOL). + VOL is an abstraction layer within the HDF5 library that enables different + methods for accessing data and objects that conform to the HDF5 data model. + The VOL layer intercepts all HDF5 API calls that potentially modify data + on disk and forwards those calls to a plugin "object driver". The data on + disk can be a different format than the HDF5 format. For more information + about VOL we refer the reader to the following documents (under review): + + VOL HDF5 APIs + https://portal.hdfgroup.org/display/HDF5/Virtual+Object++Layer + + VOL Documentation + https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5doc/browse/RFCs/HDF5/VOL + + Repository with VOL plugins + https://bitbucket.hdfgroup.org/projects/HDF5VOL + + - Enhancements to HDF5 References + + HDF5 references were extended to support attributes, and object and dataset + selections that reside in another HDF5 file. For more information including + a list of new APIs, see + + https://portal.hdfgroup.org/display/HDF5/Update+to+References + + - Add new public function H5Sselect_adjust. + + This function shifts a dataspace selection by a specified logical offset + within the dataspace extent. This can be useful for VOL developers to + implement chunked datasets. + + (NAF - 2019/11/18) + + - Add new public function H5Sselect_project_intersection. + + This function computes the intersection between two dataspace selections + and projects that intersection into a third selection. This can be useful + for VOL developers to implement chunked or virtual datasets. + + (NAF - 2019/11/13, ID-148) + + - Add new public function H5VLget_file_type. + + This function returns a datatype equivalent to the supplied datatype but + with the location set to be in the file. This datatype can then be used + with H5Tconvert to convert data between file and in-memory representation. + This funcition is intended for use only by VOL connector developers. + + (NAF - 2019/11/08, ID-127) + + - New S3 and HDFS Virtual File Drivers (VFDs) + + This release has two new VFDs. The S3 VFD allows accessing HDF5 files on + AWS S3 buckets. HDFS VFD allows accessing HDF5 files stored on Apache HDFS. + See https://portal.hdfgroup.org/display/HDF5/Virtual+File+Drivers+-+S3+and+HDFS + for information on enabling those drivers and using those APIs. + + Below are specific instructions for enabling S3 VFD on Windows: + + Fix windows requirements and java tests. Windows requires CMake 3.13. + - Install openssl library (with dev files); + from "Shining Light Productions". msi package preferred. + - PATH should have been updated with the installation dir. + - set ENV variable OPENSSL_ROOT_DIR to the installation dir. + - set ENV variable OPENSSL_CONF to the cfg file, likely %OPENSSL_ROOT_DIR%\bin\openssl.cfg + - Install libcurl library (with dev files); + - download the latest released version using git: https://github.com/curl/curl.git + - Open a Visual Studio Command prompt + - change to the libcurl root folder + - run the "buildconf.bat" batch file + - change to the winbuild directory + - nmake /f Makefile.vc mode=dll MACHINE=x64 + - copy libcurl-vc-x64-release-dll-ipv6-sspi-winssl dir to C:\curl (installation dir) + - set ENV variable CURL_ROOT to C:\curl (installation dir) + - update PATH ENV variable to %CURL_ROOT%\bin (installation bin dir). + - the aws credentials file should be in %USERPROFILE%\.aws folder + - set the ENV variable HDF5_ROS3_TEST_BUCKET_URL to the s3 url for the + s3 bucket containing the HDF5 files to be accessed. + + FORTRAN Library: + ---------------- + - Added new Fortran parameters: + + H5F_LIBVER_ERROR_F + H5F_LIBVER_NBOUNDS_F + H5F_LIBVER_V18_F + H5F_LIBVER_V110_F + H5F_LIBVER_V112_F + + - Added new Fortran API: h5pget_libver_bounds_f + + (MSB - 2020/02/11, HDFFV-11018) + + Java Library: + ---------------- + - Added ability to test java library with VOLs. + + Created new CMake script that combines the java and vol test scripts. + + (ADB - 2020/02/03, HDFFV-10996) + + - Tests fail for non-English locale. + + In the JUnit tests with a non-English locale, only the part before + the decimal comma is replaced by XXXX and this leads to a comparison + error. Changed the regex for the Time substitution. + + (ADB - 2020/01/09, HDFFV-10995) + + + Tools: + ------ + - h5diff was updated to use the new reference APIs. + + h5diff uses the new reference APIs to compare references. + Attribute references can also be compared. + + (ADB - 2019/12/19, HDFFV-10980) + + - h5dump and h5ls were updated to use the new reference APIs. + + The tools library now use the new reference APIs to inspect a + file. Also the DDL spec was updated to reflect the format + changes produced with the new APIs. The export API and support + functions in the JNI were updated to match. + + + Other improvements and changes: + + - Hyperslab selection code was reworked to improve performance, getting more + than 10x speedup in some cases. + + - The HDF5 Library was enhanced to open files with Unicode names on Windows. + + - Deprecated H5Dvlen_reclaim() and replaced it with H5Treclaim(). + This routine is meant to be used when resources are internally allocated + when reading data, i.e. when using either vlen or new reference types. + This is applicable to both attribute and dataset reads. + + - h5repack was fixed to repack datasets with external storage + to other types of storage. + + +Support for new platforms, languages and compilers. +======================================= + - Added spectrum-mpi with clang, gcc and xl compilers on Linux 3.10.0 + - Added OpenMPI 3.1 and 4.0 with clang, gcc and Intel compilers on Linux 3.10.0 + - Added cray-mpich/PrgEnv with gcc and Intel compilers on Linux 4.14.180 + - Added spectrum mpi with clang, gcc and xl compilers on Linux 4.14.0 + + +Bug Fixes since HDF5-1.12.0-alpha1 release +========================================== + Library + ------- + - Improved performance when creating a large number of small datasets by + retrieving default property values from the API context instead of doing + skip list searches. + + (CJH - 2019/12/10, HDFFV-10658) + + - Fixed user-created data access properties not existing in the property list + returned by H5Dget_access_plist. Thanks to Steven Varga for submitting a + reproducer and a patch. + + (CJH - 2019/12/09, HDFFV-10934) + + - Fixed an assertion failure in the parallel library when collectively + filling chunks. As it is required that chunks be written in + monotonically non-decreasing order of offset in the file, this assertion + was being triggered when the list of chunk file space allocations being + passed to the collective chunk filling routine was not sorted according + to this particular requirement. + + The addition of a sort of the out of order chunks trades a bit of + performance for the elimination of this assertion and of any complaints + from MPI implementations about the file offsets used being out of order. + + (JTH - 2019/10/07, HDFFV-10792) + + FORTRAN library: + ---------------- + + - Corrected INTERFACE INTENT(IN) to INTENT(OUT) for buf_size in h5fget_file_image_f. + + (MSB - 2020/2/18, HDFFV-11029) + + Java Library: + ---------------- + - Added ability to test java library with VOLs. + + Created new CMake script that combines the java and vol test scripts. + + (ADB - 2020/02/03, HDFFV-10996) + + - Tests fail for non-English locale. + + In the JUnit tests with a non-English locale, only the part before + the decimal comma is replaced by XXXX and this leads to a comparison + error. Changed the regex for the Time substitution. + + (ADB - 2020/01/09, HDFFV-10995) + + Tools: + ------ + - h5repack was fixed to repack the reference attributes properly. + The code line that checks if the update of reference inside a compound + datatype is misplaced outside the code block loop that carries out the + check. In consequence, the next attribute that is not the reference + type was repacked again as the reference type and caused the failure of + repacking. The fix is to move the corresponding code line to the correct + code block. + + (KY -2020/02/10, HDFFV-11014) + + - h5diff was updated to use the new reference APIs. + + h5diff uses the new reference APIs to compare references. + Attribute references can also be compared. + + (ADB - 2019/12/19, HDFFV-10980) + + - h5dump and h5ls were updated to use the new reference APIs. + + The tools library now use the new reference APIs to inspect a + file. Also the DDL spec was updated to reflect the format + changes produced with the new APIs. The export API and support + functions in the JNI were updated to match. + + (ADB - 2019/12/06, HDFFV-10876 and HDFFV-10877) + + +Major Bug Fixes since HDF5-1.10.0 release +========================================= + + - For major bug fixes please see HISTORY-1_10_0-1_12_0.txt file + + +Supported Platforms +=================== + + Linux 2.6.32-696.16.1.el6.ppc64 gcc (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) + #1 SMP ppc64 GNU/Linux g++ (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) + (ostrich) GNU Fortran (GCC) 4.4.7 20120313 (Red Hat 4.4.7-18) + IBM XL C/C++ V13.1 + IBM XL Fortran V15.1 + + Linux 3.10.0-327.10.1.el7 GNU C (gcc), Fortran (gfortran), C++ (g++) + #1 SMP x86_64 GNU/Linux compilers: + (kituo/moohan) Version 4.8.5 20150623 (Red Hat 4.8.5-4) + Version 4.9.3, 5.2.0, 7.1.0 + Intel(R) C (icc), C++ (icpc), Fortran (icc) + compilers: + Version 17.0.0.098 Build 20160721 + MPICH 3.1.4 + + Linux-3.10.0- spectrum-mpi/rolling-release with cmake>3.10 and + 862.14.4.1chaos.ch6.ppc64le clang/3.9,8.0 + #1 SMP ppc64le GNU/Linux gcc/7.3 + (ray) xl/2016,2019 + + Linux 3.10.0- openmpi/3.1,4.0 with cmake>3.10 and + 957.12.2.1chaos.ch6.x86_64 clang 5.0 + #1 SMP x86_64 GNU/Linux gcc/7.3,8.2 + (serrano) intel/17.0,18.0/19.0 + + Linux 3.10.0- openmpi/3.1/4.0 with cmake>3.10 and + 1062.1.1.1chaos.ch6.x86_64 clang/3.9,5.0,8.0 + #1 SMP x86_64 GNU/Linux gcc/7.3,8.1,8.2 + (chama,quartz) intel/16.0,18.0,19.0 + + Linux 4.4.180-94.100-default cray-mpich/7.7.6 with PrgEnv-*/6.0.5, cmake>3.10 and + #1 SMP x86_64 GNU/Linux gcc/7.2.0,8.2.0 + (mutrino) intel/17.0,18.0 + + Linux 4.14.0- spectrum-mpi/rolling-release with cmake>3.10 and + 49.18.1.bl6.ppc64le clang/6.0,8.0 + #1 SMP ppc64le GNU/Linux gcc/7.3 + (lassen) xl/2019 + + SunOS 5.11 32- and 64-bit Sun C 5.12 SunOS_sparc + (emu) Sun Fortran 95 8.6 SunOS_sparc + Sun C++ 5.12 SunOS_sparc + + Windows 7 x64 Visual Studio 2015 w/ Intel C, Fortran 2018 (cmake) + Visual Studio 2015 w/ MSMPI 10 (cmake) + + Windows 10 x64 Visual Studio 2015 w/ Intel Fortran 18 (cmake) + Visual Studio 2017 w/ Intel Fortran 19 (cmake) + Visual Studio 2019 w/ Intel Fortran 19 (cmake) + + macOS 10.13.6 High Sierra Apple LLVM version 10.0.0 (clang/clang++-1000.10.44.4) + 64-bit gfortran GNU Fortran (GCC) 6.3.0 + (bear) Intel icc/icpc/ifort version 19.0.4 + + macOS 10.14.6 Mohave Apple LLVM version 10.0.1 (clang/clang++-1001.0.46.4) + 64-bit gfortran GNU Fortran (GCC) 6.3.0 + (bobcat) Intel icc/icpc/ifort version 19.0.4 + + +Tested Configuration Features Summary +===================================== + + In the tables below + y = tested + n = not tested in this release + C = Cluster + W = Workstation + x = not working in this release + dna = does not apply + ( ) = footnote appears below second table + <blank> = testing incomplete on this feature or platform + +Platform C F90/ F90 C++ zlib SZIP + parallel F2003 parallel +SunOS 5.11 32-bit n y/y n y y y +SunOS 5.11 64-bit n y/n n y y y +Windows 7 y y/y n y y y +Windows 7 x64 y y/y y y y y +Windows 7 Cygwin n y/n n y y y +Windows 7 x64 Cygwin n y/n n y y y +Windows 10 y y/y n y y y +Windows 10 x64 y y/y n y y y +macOS 10.13.6 64-bit n y/y n y y ? +macOS 10.14.6 64-bit n y/y n y y ? +CentOS 6.7 Linux 2.6.18 x86_64 GNU n y/y n y y y +CentOS 6.7 Linux 2.6.18 x86_64 Intel n y/y n y y y +CentOS 6.7 Linux 2.6.32 x86_64 PGI n y/y n y y y +CentOS 7.2 Linux 2.6.32 x86_64 GNU y y/y y y y y +CentOS 7.2 Linux 2.6.32 x86_64 Intel n y/y n y y y +Linux 2.6.32-573.18.1.el6.ppc64 n y/n n y y y + + +Platform Shared Shared Shared Thread- + C libs F90 libs C++ libs safe +SunOS 5.11 32-bit y y y y +SunOS 5.11 64-bit y y y y +Windows 7 y y y y +Windows 7 x64 y y y y +Windows 7 Cygwin n n n y +Windows 7 x64 Cygwin n n n y +Windows 10 y y y y +Windows 10 x64 y y y y +macOS 10.13.6 64-bit y n y y +macOS 10.14.6 64-bit y n y y +CentOS 6.7 Linux 2.6.18 x86_64 GNU y y y y +CentOS 6.7 Linux 2.6.18 x86_64 Intel y y y n +CentOS 6.7 Linux 2.6.32 x86_64 PGI y y y n +CentOS 7.2 Linux 2.6.32 x86_64 GNU y y y n +CentOS 7.2 Linux 2.6.32 x86_64 Intel y y y n +Linux 2.6.32-573.18.1.el6.ppc64 y y y n + +Compiler versions for each platform are listed in the preceding +"Supported Platforms" table. + + +More Tested Platforms +===================== +The following platforms are not supported but have been tested for this release. + + Linux 2.6.32-573.22.1.el6 GNU C (gcc), Fortran (gfortran), C++ (g++) + #1 SMP x86_64 GNU/Linux compilers: + (mayll/platypus) Version 4.4.7 20120313 + Version 4.9.3, 5.3.0, 6.2.0 + PGI C, Fortran, C++ for 64-bit target on + x86-64; + Version 17.10-0 + Intel(R) C (icc), C++ (icpc), Fortran (icc) + compilers: + Version 17.0.4.196 Build 20170411 + MPICH 3.1.4 compiled with GCC 4.9.3 + + Linux 3.10.0-327.18.2.el7 GNU C (gcc) and C++ (g++) compilers + #1 SMP x86_64 GNU/Linux Version 4.8.5 20150623 (Red Hat 4.8.5-4) + (jelly) with NAG Fortran Compiler Release 6.1(Tozai) + GCC Version 7.1.0 + OpenMPI 3.0.0-GCC-7.2.0-2.29 + Intel(R) C (icc) and C++ (icpc) compilers + Version 17.0.0.098 Build 20160721 + with NAG Fortran Compiler Release 6.1(Tozai) + PGI C (pgcc), C++ (pgc++), Fortran (pgf90) + compilers: + Version 18.4, 19.4 + MPICH 3.3 + OpenMPI 2.1.5, 3.1.3, 4.0.0 + + Fedora30 5.3.11-200.fc30.x86_64 + #1 SMP x86_64 GNU/Linux GNU gcc (GCC) 9.2.1 20190827 (Red Hat 9.2.1 20190827) + GNU Fortran (GCC) 9.2.1 20190827 (Red Hat 9.2.1 20190827) + (cmake and autotools) + + Mac OS X El Capitan 10.11.6 Apple LLVM version 7.3.0 (clang/clang++-703.0.29) + 64-bit gfortran GNU Fortran (GCC) 5.2.0 + (osx1011dev/osx1011test) Intel icc/icpc/ifort version 16.0.2 + + macOS 10.12.6 Sierra Apple LLVM version 9.0.0 (clang/clang++-900.0.39.2) + 64-bit gfortran GNU Fortran (GCC) 7.4.0 + (kite) Intel icc/icpc/ifort version 17.0.8 + + +Known Problems +============== + CMake files do not behave correctly with paths containing spaces. + Do not use spaces in paths because the required escaping for handling spaces + results in very complex and fragile build files. + ADB - 2019/05/07 + + At present, metadata cache images may not be generated by parallel + applications. Parallel applications can read files with metadata cache + images, but since this is a collective operation, a deadlock is possible + if one or more processes do not participate. + + Known problems in previous releases can be found in the HISTORY*.txt files + in the HDF5 source, and in the HDF5 Jira database, available at + https://jira.hdfgroup.org/. Please register at https://www.hdfgroup.org to + create a free account for accessing the Jira database. Please report any + new problems found to help@hdfgroup.org. + + +CMake vs. Autotools installations +================================= +While both build systems produce similar results, there are differences. +Each system produces the same set of folders on linux (only CMake works +on standard Windows); bin, include, lib and share. Autotools places the +COPYING and RELEASE.txt file in the root folder, CMake places them in +the share folder. + +The bin folder contains the tools and the build scripts. Additionally, CMake +creates dynamic versions of the tools with the suffix "-shared". Autotools +installs one set of tools depending on the "--enable-shared" configuration +option. + build scripts + ------------- + Autotools: h5c++, h5cc, h5fc + CMake: h5c++, h5cc, h5hlc++, h5hlcc + +The include folder holds the header files and the fortran mod files. CMake +places the fortran mod files into separate shared and static subfolders, +while Autotools places one set of mod files into the include folder. Because +CMake produces a tools library, the header files for tools will appear in +the include folder. + +The lib folder contains the library files, and CMake adds the pkgconfig +subfolder with the hdf5*.pc files used by the bin/build scripts created by +the CMake build. CMake separates the C interface code from the fortran code by +creating C-stub libraries for each Fortran library. In addition, only CMake +installs the tools library. The names of the szip libraries are different +between the build systems. + +The share folder will have the most differences because CMake builds include +a number of CMake specific files for support of CMake's find_package and support +for the HDF5 Examples CMake project. + |