VFD SWMR: Normalization with develop (#1078)

Brings many changes from develop, particularly VOL changes for async

5 files changed, 160 insertions, 16 deletions

diff --git a/release_docs/INSTALL_CMake.txt b/release_docs/INSTALL_CMake.txt
index 5d11957..91eb593 100644
--- a/release_docs/INSTALL_CMake.txt
+++ b/release_docs/INSTALL_CMake.txt
@@ -740,6 +740,7 @@ HDF5_BUILD_FORTRAN  "Build FORTRAN support"          OFF
 HDF5_BUILD_JAVA     "Build JAVA support"             OFF
 HDF5_BUILD_HL_LIB   "Build HIGH Level HDF5 Library"  ON
 HDF5_BUILD_TOOLS    "Build HDF5 Tools"               ON
+HDF5_BUILD_HL_TOOLS "Build HIGH Level HDF5 Tools"    ON
 
 ---------------- HDF5 Advanced Options ---------------------
 ONLY_SHARED_LIBS               "Only Build Shared Libraries"                                  OFF
diff --git a/release_docs/README_HDF5_CMake b/release_docs/README_HDF5_CMake
index cf0ab6f..c575320 100644
--- a/release_docs/README_HDF5_CMake
+++ b/release_docs/README_HDF5_CMake
@@ -1,15 +1,16 @@
 This tar file contains
 
-    build-unix.sh       script to build HDF5 with CMake on unix machines
-    build-unix-hpc.sh   script to build HDF5 with CMake on unix machines and run
-                        tests with batch scripts (sbatch).
+    build-unix.sh         script to build HDF5 with CMake on unix machines
+    build-unix-hpc.sh     script to build HDF5 with CMake on unix machines and run
+                          tests with batch scripts (sbatch).
     CTestScript.cmake
-    HDF5config.cmake    CMake scripts for building HDF5
+    HDF5config.cmake      CMake scripts for building HDF5
     HDF5options.cmake
-    hdf5-1.13.0         HDF5 1.13.0 source
-    LIBAEC.tar.gz       source for building SZIP replacement
-    ZLib.tar.gz         source for building ZLIB
-    hdf5_plugins.tar.gz source for building compression plugins
+    hdf5-1.13.0           HDF5 1.13.0 source
+    LIBAEC.tar.gz         source for building SZIP replacement
+    ZLib.tar.gz           source for building ZLIB
+    hdf5_plugins.tar.gz   source for building compression plugins
+    HDF5Examples-1.14.4-Source.tar.gz  source for building examples
 
 For more information about building HDF5 with CMake, see USING_HDF5_CMake.txt in
 hdf5-1.13.0/release_docs, or
diff --git a/release_docs/RELEASE.txt b/release_docs/RELEASE.txt
index 1e69f54..ed12c5e 100644
--- a/release_docs/RELEASE.txt
+++ b/release_docs/RELEASE.txt
@@ -49,6 +49,14 @@ New Features
 
     Configuration:
     -------------
+    - Added new option to control the build of High-Level tools
+      that default ON/enabled.
+
+      Add configure options (autotools - CMake):
+            enable-hltools       HDF5_BUILD_HL_TOOLS
+
+      (ADB - 2021/09/16, HDFFV-11266)
+
     - Adds C++ Autotools configuration file for Intel
 
       * Checks for icpc as the compiler
@@ -463,6 +471,25 @@ New Features
 
     Library:
     --------
+    - Change how the release part of version, in major.minor.release is checked
+      for compatibility
+
+      The HDF5 library uses a function, H5check_version, to check that
+      the version defined in the header files, which is used to compile an
+      application is compatible with the version codified in the library, which
+      the application loads at runtime. This previously required an exact match
+      or the library would print a warning, dump the build settings and then
+      abort or continue. An environment variable controlled the logic.
+
+      Now the function first checks that the library release version, in
+      major.minor.release, is not older than the version in the headers.
+      Secondly, if the release version is different, it checks if either
+      the library version or the header version is in the exception list, in
+      which case the release part of version, in major.minor.release, must
+      be exact. An environment variable still controls the logic.
+
+        (ADB - 2021/07/27)
+
     - gcc warning suppression macros were moved out of H5public.h
 
       The HDF5 library uses a set of macros to suppress warnings on gcc.
@@ -726,6 +753,11 @@ New Features
 
     Fortran Library:
     ----------------
+    - H5Fget_name_f fixed to handle correctly trailing whitespaces and
+      newly allocated buffers.
+
+      (MSB - 2021/08/30, github-826,972)
+
     - Add wrappers for H5Pset/get_file_locking() API calls
 
         h5pget_file_locking_f()
@@ -801,6 +833,26 @@ New Features
 
     Tools:
     ------
+    - Refactored the perform tools and removed depends on test library.
+
+        Moved the perf and h5perf tools from tools/test/perform to
+        tools/src/h5perf so that they can be installed. This required
+        that the test library dependency be removed by copying the
+        needed functions from h5test.c.
+        The standalone scripts and other perform tools remain in the
+        tools/test/perform folder.
+
+        (ADB - 2021/08/10)
+
+    - Removed partial long exceptions
+
+        Some of the tools accepted shortened versions of the long options
+        (ex: --datas instead of --dataset). These were implemented inconsistently,
+        are difficult to maintian, and occasionally block useful long option
+        names. These partial long options have been removed from all the tools.
+
+        (DER - 2021/08/03)
+
     - h5repack added help text for user-defined filters.
 
         Added help text line that states the valid values of the filter flag
@@ -863,7 +915,13 @@ New Features
 
     High-Level APIs:
     ----------------
-    -
+    - added set/get for unsigned long long attributes
+
+        the attribute writing high-level API has been expanded to include
+        public set/get functions for ULL attributes, analogously to the
+        existing set/get for other types.
+
+        (AF - 2021/09/08)
 
     C Packet Table API:
     -------------------
@@ -885,6 +943,35 @@ Bug Fixes since HDF5-1.12.0 release
 ===================================
     Library
     -------
+    - Detection of simple data transform function "x"
+
+        In the case of the simple data transform function "x" the (parallel)
+        library recognizes this is the same as not applying this data transform
+        function. This improves the I/O performance. In the case of the parallel
+        library, it also avoids breaking to independent I/O, which makes it
+        possible to apply a filter when writing or reading data to or from
+        teh HDF5 file.
+
+        (JWSB - 2021/09/13)
+
+    - Fixed an invalid read and memory leak when parsing corrupt file space
+      info messages
+
+        When the corrupt file from CVE-2020-10810 was parsed by the library,
+        the code that imports the version 0 file space info object header
+        message to the version 1 struct could read past the buffer read from
+        the disk, causing an invalid memory read. Not catching this error would
+        cause downstream errors that eventually resulted in a previously
+        allocated buffer to be unfreed when the library shut down. In builds
+        where the free lists are in use, this could result in an infinite loop
+        and SIGABRT when the library shuts down.
+
+        We now track the buffer size and raise an error on attempts to read
+        past the end of it.
+
+        (DER - 2021/08/12, HDFFV-11053)
+
+
     - Fixed CVE-2018-14460
 
         The tool h5repack produced a segfault when the rank in dataspace
@@ -1060,6 +1147,38 @@ Bug Fixes since HDF5-1.12.0 release
 
     Configuration
     -------------
+    - Corrected pkg-config compile script
+
+      It was discovered that the position of the "$@" argument for the command
+      in the compile script may fail on some platforms and configurations. The
+      position of the "$@"command argument was moved before the pkg-config sub command.
+
+      (ADB - 2021/08/30)
+
+    - Fixed CMake C++ compiler flags
+
+      A recent refactoring of the C++ configure files accidently removed the
+      file that executed the enable_language command for C++ needed by the
+      HDFCXXCompilerFlags.cmake file. Also updated the intel warnings files,
+      including adding support for windows platforms.
+
+      (ADB - 2021/08/10)
+
+    - Better support for libaec (open-source Szip library) in CMake
+
+      Implemented better support for libaec 1.0.5 (or later) library. This version
+      of libaec contains improvements for better integration with HDF5. Furthermore,
+      the variable USE_LIBAEC_STATIC has been introduced to allow to make use of
+      static version of libaec library. Use libaec_DIR or libaec_ROOT to set
+      the location in which libaec can be found.
+
+      Be aware, the Szip library of libaec 1.0.4 depends on another library within
+      libaec library. This dependency is not specified in the current CMake
+      configuration which means that one can not use the static Szip library of
+      libaec 1.0.4 when building HDF5. This has been resolved in libaec 1.0.5.
+
+      (JWSB - 2021/06/22)
+
     - Refactor CMake configure for Fortran
 
       The Fortran configure tests for KINDs reused a single output file that was
@@ -1421,6 +1540,10 @@ The following platforms are not supported but have been tested for this release.
 
 Known Problems
 ==============
+    Setting a variable-length dataset fill value will leak the memory allocated
+    for the p field of the hvl_t struct. A fix is in progress for this.
+    HDFFV-10840
+
     CMake files do not behave correctly with paths containing spaces.
     Do not use spaces in paths because the required escaping for handling spaces
     results in very complex and fragile build files.
diff --git a/release_docs/USING_CMake_Examples.txt b/release_docs/USING_CMake_Examples.txt
index bd089a6..a12a952 100644
--- a/release_docs/USING_CMake_Examples.txt
+++ b/release_docs/USING_CMake_Examples.txt
@@ -83,6 +83,7 @@ III. Defaults in the HDF5_Examples_options.cmake file
 ####         HDF_BUILD_CXX:BOOL=OFF                                                       ###
 ####         HDF_BUILD_FORTRAN:BOOL=OFF                                                   ###
 ####         HDF_BUILD_JAVA:BOOL=OFF                                                      ###
+####         HDF_BUILD_FILTERS:BOOL=OFF                                                   ###
 ####         BUILD_TESTING:BOOL=OFF                                                       ###
 ####         HDF_ENABLE_PARALLEL:BOOL=OFF                                                 ###
 ####         HDF_ENABLE_THREADSAFE:BOOL=OFF                                               ###
diff --git a/release_docs/USING_HDF5_CMake.txt b/release_docs/USING_HDF5_CMake.txt
index f2d7754..792c719 100644
--- a/release_docs/USING_HDF5_CMake.txt
+++ b/release_docs/USING_HDF5_CMake.txt
@@ -41,13 +41,30 @@ I. Preconditions
 
    2. You have installed the HDF5 library built with CMake, by executing
       the HDF Install Utility (the *.msi file in the binary package for
-      Windows). If you are using a Windows platform, you can obtain a
-      pre-built Windows binary from The HDF Group's website at
+      Windows). You can obtain pre-built binaries from The HDF Group's website at
       www.hdfgroup.org.
 
    3. Set the environment variable HDF5_DIR to the installed location of
-      the config files for HDF5. On Windows:
+      the config files for HDF5.
+         On Windows:
             HDF5_DIR=C:/Program Files/HDF_Group/HDF5/1.13.x/cmake
+         On unix:
+            HDF5_DIR=<install root folder>/HDF_Group/HDF5/1.13.x/cmake
+
+      If you are using shared libraries, you may need to add to the path
+      environment variable. Set the path environment variable to the
+      installed location of the library files for HDF5.
+         On Windows (*.dll):
+            PATH=%PATH%;C:/Program Files/HDF_Group/HDF5/1.13.x/bin
+         On unix (*.so):
+            LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<install root folder>/HDF_Group/HDF5/1.13.x/lib
+
+      If you are using filter plugin libraries, you will need to set the
+      HDF5_PLUGIN_PATH environment variable.
+         On Windows:
+            HDF5_PLUGIN_PATH=C:/Program Files/HDF_Group/HDF5/1.13.x/lib/plugin
+         On unix:
+            HDF5_PLUGIN_PATH=<install root folder>/HDF_Group/HDF5/1.13.x/lib/plugin
 
       (Note there are no quote characters used on Windows and all platforms
       use forward slashes)
@@ -99,12 +116,13 @@ These steps are described in more detail below.
             * MinGW Makefiles
             * NMake Makefiles
             * Unix Makefiles
-            * Visual Studio 12 2013
-            * Visual Studio 12 2013 Win64
             * Visual Studio 14 2015
             * Visual Studio 14 2015 Win64
             * Visual Studio 15 2017
             * Visual Studio 15 2017 Win64
+            * Visual Studio 16 2019
+            * ... in addition VS2019 will need to set the "-A" option,
+            * ... [Win32, x64, ARM, ARM64]
 
         <options> is:
             * BUILD_TESTING:BOOL=ON
@@ -114,7 +132,7 @@ These steps are described in more detail below.
 
       2.1  Visual CMake users, click the Configure button. If this is the first time you are
            running cmake-gui in this directory, you will be prompted for the
-           generator you wish to use (for example on Windows, Visual Studio 12 2013).
+           generator you wish to use (for example on Windows, Visual Studio 14 2015 Win64).
            CMake will read in the CMakeLists.txt files from the source directory and
            display options for the HDF5 project. After the first configure you
            can adjust the cache settings and/or specify locations of other programs.
@@ -132,7 +150,7 @@ These steps are described in more detail below.
 
       2.2  Alternative command line example on Windows in c:\MyHDFstuff\hdf5\build directory:
 
-           cmake -G "Visual Studio 12 2013" -DBUILD_TESTING:BOOL=ON ..
+           cmake -G "Visual Studio 14 2015 Win64" -DBUILD_TESTING:BOOL=ON ..
 
    3. Build HDF5 Applications