Sync 1.14 branch with develop (#3887)

* Preserve MPI-I/O file hints when fapl is closed (#3755)

* Fix for issue #3025: Save the MPI info in the file struct so H5Fget_access_plist() can retrieve it from there.

* Add compression tests for subfiling (#3769)

* Fix typo in comment (#3775)

* Fixed a file handle leak in the core VFD (#3779)

When opening a file with the core VFD and a file image, if the file
already exists, the file check would leak the POSIX file handle.

Fixes GitHub issue #635

* Fix a format string warning in the C++ examples (#3776)

* Cancel running GitHub workflows on push to same PR (#3772)

* Cancel running GitHub workflows on push to same PR

* Remove github.sha from workflow concurrency groups

* Print some messages in parallel tests on MPI rank 0 only (#3785)

Avoids overly verbose output from all processes emitting progress, etc. info.

* Avoid attempted use of NULL pointer in parallel compression code (#3786)

The parallel compression test code tests for the case where all MPI ranks have no selection in a dataset when writing to it. Add an early exit to the code to avoid attempting to use a NULL pointer due to there being no work to do.

* Don't install h5tools_test_utils test program on system (#3793)

* Add Doxygen to H5FDsplitter.h (#3794)

* H5FD_CURR_SPLITTER_VFD_CONFIG_VERSION
* H5FD_SPLITTER_PATH_MAX
* H5FD_SPLITTER_MAGIC
* H5FD_splitter_vfd_config_t
* H5Pset_fapl_splitter()
* H5Pget_fapl_splitter()

* Update Doxygen initializers & identifiers in VFDs (#3795)

* Add Doxygen for all H5FD_<VFD> initializers
* Add Doxygen for all H5FD_<VFD>_VALUE values
* Mark H5FD_<vfd>_init() calls private in Doxygen

* Fix memory corruption in 'MPI I/O FAPL preserve' test (#3806)

* Fix usage of h5_clean_files in t_pflush2.c (#3807)

* Fix parallel driver check in h5_fixname_real (#3808)

* Fix a couple usages of MPI_Info_get (#3809)

* Remove H5system.c warning on Windows oneAPI. (#3812)

* Add processing of NVHPC flags in linux-gnulibc1 file (#3804)

* Disable testing as tests are failing the same as in CMake

* Use the current toolchain for examples as default (#3810)

* Fix misc. warnings from GCC when compiling with -fsanitize=undefined (#3787)

* Set NVHPC maximum optimization level to -O1 for now (#3800)

* Set NVHPC maximum optimization level to -O1 for now

Compiling HDF5 with NVHPC 23.5 - 23.9 results in test failures in
4 different test files that need to be resolved. Since those tests
pass with an optimization level of -O1 (and -O0) and it is currently
unclear whether the test failures are due to issues in HDF5 or issues
in the 'nvc' compiler, set the maximum optimization level for NVHPC
to -O1 until the test failures are resolved.

* Disable nvhpc Java testing in CMake and amend known issues

* Re-enable testing of Autotools nvhpc

* Update some doxygen links to local refs (#3814)

* Rework MPI Info FAPL preserve PR to use VFD 'ctl' operations (#3782)

* Removed the use of C wrappers from H5P APIs. (#3824)

* fix seg fault on frontier/cray

* fix seg fault on frontier/cray

* fix seg fault on frontier/cray

* removed the use of h5pclose_c

* removed the use of h5pclose_c

* Fortran Wrappers H5VLnative_addr_to_token_f and H5VLnative_token_to_address_f (#3801)

* Added H5VLnative_addr_to_token_f and H5VLnative_token_to_address_f

* Added H5VLnative_addr_to_token_f and H5VLnative_token_to_address_f tests

* Create test for H5Pget_dxpl_mpio (#3825)

* Create test and add to testphdf5

* Renamed h5fuse.sh to h5fuse (#3834)

* provide an alternative to mapfile for older bash

* Disable FP exceptions in H5T init code (#3837)

The H5T floating-point datatype initialization code can raise exceptions when handling signaling NaNs. This change disables FE_INVALID exceptions during initialization.

Also removes the -ieee=full change for NAG Fortran as that shouldn't be necessary anymore.

Fixes #3831

* Add intel oneapi windows build to CI CMake (#3836)

* Remove printf format warning on Windows oneAPI. (#3838)

* Correct ENV variables (#3841)

* Remove Autotools sed hack (#3848)

configure.ac contains a sed line that cleans up incorrect library
flags which was added to paper over some bugs in earlier versions
of the Autotools. These issues are not a problem with the current
versions of the Autootols.

The sed line causes problems on MacOS, so it has been removed.

Fixes #3843

* Make filter unregister callbacks safe for VOL connectors (#3629)

* Make filter callbacks use top-level API functions

When using VOL connectors, H5I_iterate may not provide
valid object pointers to its callback. This change keeps
existing functionality in H5Zunregister() without using
potentially unsafe pointers.

* Filter callbacks use internal API

* Skip MPI work on non-native VOL

* Add extra space in comments for consistency (#3852)

* Add extra space in comments for consistency

* uncomment tfloatsattrs test

* Update Actions badges to link to relevant workflow (#3850)

* Add CMake long double cross-compile defaults (#3683)

HDF5 performs a couple of checks at build time to see if long double
values can be converted correctly (IBM's Power architecture uses a
special format for long doubles). These checks were performed using
TRY_RUN, which is a problem when cross-compiling.

These checks now use default values appropriate for most non-Power
systems when cross-compiling. The cache values can be pre-set if
necessary, which will preempt both the TRY_RUN and the default.

Affected values:
    H5_LDOUBLE_TO_LONG_SPECIAL      (default no)
    H5_LONG_TO_LDOUBLE_SPECIAL      (default no)
    H5_LDOUBLE_TO_LLONG_ACCURATE    (default yes)
    H5_LLONG_TO_LDOUBLE_CORRECT     (default yes)
    H5_DISABLE_SOME_LDOUBLE_CONV    (default no)

Fixes GitHub #3585

* Updates for building and testing VOL connectors

* Fix issue with HDF5_VOL_ALLOW_EXTERNAL CMake variable

* Initialize parallel testing with MPI_THREAD_MULTIPLE when testing API

* Add CMake variable to allow specifying a VOL connector's package name

* Remove call to MPI_Init in serial API tests

While previously necessary, it now interferes with VOL connectors that
may need to be initialized with MPI_THREAD_MULTIPLE

* Fixes for CI and presets (#3853)

* Change dest for doxygen (#3856)

* Implement selection vector I/O with collective chunk filling (#3826)

* Changes for ECP-344: Implement selection vector I/O with collective chunk filling.
Also fix a bug in H5FD__mpio_write_vector() to account for fixed size optimization
when computing max address.

* Fixes based on PR review comments:
For H5Dchunk.c: fix H5MM_xfree()
For H5FDmpio.c:
1) Revert the fix to H5FD__mpio_write_vector()
2) Apply the patch from Neil on the proper length of s_sizes reported by H5FD__mpio_vector_build_types()

* Put back the logic of dividing up the work among all the mpi ranks similar to the
original H5D__chunk_collective_fill() routine.

* Add a test to verify the fix for the illegal reference problem in H5FD__mpio_write_vector().

* Do not publish compression headers or docs (#3865)

* Fix typo: look -> loop (#3866)

* Moved the README to markdown and expanded its overview of the files, file generation, and other Fortran wrapper development practices as mentioned in the HDF5 architectural document. I added a new figure and included the SVG file and the original xfig file it was generated from. (#3862)

* Add HDF5_DISABLE_TESTS_REGEX option to skip tests (#3859)

* Fix typo in error message for `MPI_Type_dup`. (#3867)

* Complete the `if command line option` sentence. (#3868)

* Fix h5dump segmentation fault when --vfd-value option is used (#3873)

* Updated URL in funding.yml (#3882)

Using new shortened URL, might look better.

* Remove unused variable from unmerged changes

* Update src/H5Tinit_float.c

1 files changed, 113 insertions, 26 deletions

diff --git a/CMakeVOL.cmake b/CMakeVOL.cmake
index 79afa59..8b2207a 100644
--- a/CMakeVOL.cmake
+++ b/CMakeVOL.cmake
@@ -26,14 +26,51 @@ function (get_generated_cmake_targets out_var dir)
   set (${out_var} "${dir_targets}" PARENT_SCOPE)
 endfunction ()
 
-# For now, only support building of external VOL connectors with FetchContent
-option (HDF5_VOL_ALLOW_EXTERNAL "Allow building of external HDF5 VOL connectors with FetchContent" "NO")
-mark_as_advanced (HDF5_VOL_ALLOW_EXTERNAL)
-if (HDF5_VOL_ALLOW_EXTERNAL)
-  if (HDF5_VOL_ALLOW_EXTERNAL MATCHES "NO" OR (NOT HDF5_VOL_ALLOW_EXTERNAL MATCHES "GIT" AND NOT HDF5_VOL_ALLOW_EXTERNAL MATCHES "LOCAL_DIR"))
-    message (FATAL_ERROR "HDF5_VOL_ALLOW_EXTERNAL must be set to 'GIT' or 'LOCAL_DIR' to allow building of external HDF5 VOL connectors")
-  endif()
+# Function to apply connector-specify workarounds to build
+# code once a connector has been populated through FetchContent
+function (apply_connector_workarounds connector_name source_dir)
+  # For the cache VOL, remove the call to find_package(ASYNC).
+  # Eventually, the FetchContent OVERRIDE_FIND_PACKAGE should be
+  # able to fulfill this dependency when building the cache VOL,
+  # but for now we have to hack around this until the async and
+  # cache VOLs create CMake .config files
+  if ("${connector_name}" MATCHES "vol-cache")
+    # Remove find_package(ASYNC) call from connector's CMake code
+    file (READ "${source_dir}/CMakeLists.txt" vol_cmake_contents)
+    string (REGEX REPLACE "[ \t]*find_package[ \t]*\\([ \t]*ASYNC[^\r\n\\)]*\\)[ \t]*[\r\n]+" "" vol_cmake_contents "${vol_cmake_contents}")
+    file (WRITE "${source_dir}/CMakeLists.txt" "${vol_cmake_contents}")
+
+    # Remove setting of HDF5_VOL_CONNECTOR and HDF5_PLUGIN_PATH
+    # in connector's external tests CMake code
+    file (STRINGS "${source_dir}/tests/CMakeLists.txt" file_lines)
+    file (WRITE "${source_dir}/tests/CMakeLists.txt" "")
+    foreach (line IN LISTS file_lines)
+      set (stripped_line "${line}")
+      string (REGEX MATCH "^[ \t]*set_tests_properties\\([ \t]*[\r\n]?" match_string "${line}")
+      if (NOT "${match_string}" STREQUAL "")
+        string (REGEX REPLACE "^[ \t]*set_tests_properties\\([ \t]*[\r\n]?" "" stripped_line "${line}")
+      endif ()
+      string (REGEX MATCH "^[ \t]*.\\{test\\}[ \t]*[\r\n]?" match_string "${line}")
+      if (NOT "${match_string}" STREQUAL "")
+        string (REGEX REPLACE "^[ \t]*.\\{[A-Za-z]*\\}[ \t]*[\r\n]?" "" stripped_line "${line}")
+      endif ()
+      string (REGEX MATCH "^[ \t]*PROPERTIES[ \t]*[\r\n]?" match_string "${line}")
+      if (NOT "${match_string}" STREQUAL "")
+        string (REGEX REPLACE "^[ \t]*PROPERTIES[ \t]*[\r\n]?" "" stripped_line "${line}")
+      endif ()
+      string (REGEX MATCH "^[ \t]*ENVIRONMENT[ \t]*.*[\r\n]?" match_string "${line}")
+      if (NOT "${match_string}" STREQUAL "")
+        string (REGEX REPLACE "^[ \t]*ENVIRONMENT[ \t]*.*[\r\n]?" "" stripped_line "${line}")
+      endif ()
+      file (APPEND "${source_dir}/tests/CMakeLists.txt" "${stripped_line}\n")
+    endforeach ()
+  endif ()
+endfunction ()
 
+set (HDF5_VOL_ALLOW_EXTERNAL "NO" CACHE STRING "Allow building of external HDF5 VOL connectors with FetchContent")
+set_property (CACHE HDF5_VOL_ALLOW_EXTERNAL PROPERTY STRINGS NO GIT LOCAL_DIR)
+mark_as_advanced (HDF5_VOL_ALLOW_EXTERNAL)
+if (HDF5_VOL_ALLOW_EXTERNAL MATCHES "GIT" OR HDF5_VOL_ALLOW_EXTERNAL MATCHES "LOCAL_DIR")
   # For compatibility, set some variables that projects would
   # typically look for after calling find_package(HDF5)
   set (HDF5_FOUND 1)
@@ -103,6 +140,13 @@ if (HDF5_VOL_ALLOW_EXTERNAL)
         mark_as_advanced ("HDF5_VOL_${hdf5_vol_name_upper}_BRANCH")
       endif()
 
+      set ("HDF5_VOL_${hdf5_vol_name_upper}_CMAKE_PACKAGE_NAME"
+        "${hdf5_vol_name_lower}"
+        CACHE
+        STRING
+        "CMake package name used by find_package(...) calls for VOL connector '${hdf5_vol_name}'"
+      )
+
       set ("HDF5_VOL_${hdf5_vol_name_upper}_NAME" "" CACHE STRING "Name of VOL connector to set for the HDF5_VOL_CONNECTOR environment variable")
       option ("HDF5_VOL_${hdf5_vol_name_upper}_TEST_PARALLEL" "Whether to test VOL connector '${hdf5_vol_name}' against the parallel API tests" OFF)
 
@@ -124,22 +168,40 @@ if (HDF5_VOL_ALLOW_EXTERNAL)
           message (FATAL_ERROR "HDF5_VOL_PATH${vol_idx_fixed} must be an absolute path to a valid directory")
       endif ()
 
+      # Set internal convenience variables for FetchContent dependency name
+      set (hdf5_vol_depname "${HDF5_VOL_${hdf5_vol_name_upper}_CMAKE_PACKAGE_NAME}")
+      string (TOLOWER "${hdf5_vol_depname}" hdf5_vol_depname_lower)
+
       if (HDF5_VOL_ALLOW_EXTERNAL MATCHES "GIT")
-        FetchContent_Declare (HDF5_VOL_${hdf5_vol_name_lower}
-        GIT_REPOSITORY "${HDF5_VOL_SOURCE}"
-        GIT_TAG "${HDF5_VOL_${hdf5_vol_name_upper}_BRANCH}"
-        )
+        if (${CMAKE_VERSION} VERSION_GREATER_EQUAL "3.24")
+          FetchContent_Declare (${hdf5_vol_depname}
+            GIT_REPOSITORY "${HDF5_VOL_SOURCE}"
+            GIT_TAG "${HDF5_VOL_${hdf5_vol_name_upper}_BRANCH}"
+            OVERRIDE_FIND_PACKAGE
+          )
+        else ()
+          FetchContent_Declare (${hdf5_vol_depname}
+            GIT_REPOSITORY "${HDF5_VOL_SOURCE}"
+            GIT_TAG "${HDF5_VOL_${hdf5_vol_name_upper}_BRANCH}"
+            FIND_PACKAGE_ARGS NAMES ${hdf5_vol_name_lower}
+          )
+        endif ()
       elseif(HDF5_VOL_ALLOW_EXTERNAL MATCHES "LOCAL_DIR")
-        FetchContent_Declare (HDF5_VOL_${hdf5_vol_name_lower}
-        SOURCE_DIR "${HDF5_VOL_SOURCE}"
+        FetchContent_Declare (${hdf5_vol_depname}
+          SOURCE_DIR "${HDF5_VOL_SOURCE}"
         )
       endif()
 
-      FetchContent_GetProperties(HDF5_VOL_${hdf5_vol_name_lower})
-      if (NOT hdf5_vol_${hdf5_vol_name_lower}_POPULATED)
-        FetchContent_Populate(HDF5_VOL_${hdf5_vol_name_lower})
+      FetchContent_GetProperties(${hdf5_vol_depname})
+      if (NOT ${hdf5_vol_depname}_POPULATED)
+        FetchContent_Populate(${hdf5_vol_depname})
+
+        # Now that content has been populated, set other internal
+        # convenience variables for FetchContent dependency
+        set (hdf5_vol_depname_source_dir "${${hdf5_vol_depname_lower}_SOURCE_DIR}")
+        set (hdf5_vol_depname_binary_dir "${${hdf5_vol_depname_lower}_BINARY_DIR}")
 
-        if (NOT EXISTS "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/CMakeLists.txt")
+        if (NOT EXISTS "${hdf5_vol_depname_source_dir}/CMakeLists.txt")
           if (HDF5_VOL_ALLOW_EXTERNAL MATCHES "GIT")
             message (SEND_ERROR "The git repository branch '${HDF5_VOL_${hdf5_vol_name_upper}_BRANCH}' for VOL connector '${hdf5_vol_name}' does not appear to contain a CMakeLists.txt file")
           elseif (HDF5_VOL_ALLOW_EXTERNAL MATCHES "LOCAL_DIR")
@@ -150,21 +212,24 @@ if (HDF5_VOL_ALLOW_EXTERNAL)
         # If there are any calls to find_package(HDF5) in the connector's
         # CMakeLists.txt files, remove those since any found HDF5 targets
         # will conflict with targets being generated by this build of HDF5
-        if (EXISTS "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/CMakeLists.txt")
-          file (READ "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/CMakeLists.txt" vol_cmake_contents)
+        if (EXISTS "${hdf5_vol_depname_source_dir}/CMakeLists.txt")
+          file (READ "${hdf5_vol_depname_source_dir}/CMakeLists.txt" vol_cmake_contents)
           string (REGEX REPLACE "[ \t]*find_package[ \t]*\\([ \t]*HDF5[^\r\n\\)]*\\)[ \t]*[\r\n]+" "" vol_cmake_contents "${vol_cmake_contents}")
-          file (WRITE "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/CMakeLists.txt" "${vol_cmake_contents}")
+          file (WRITE "${hdf5_vol_depname_source_dir}/CMakeLists.txt" "${vol_cmake_contents}")
         endif ()
-        if (EXISTS "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/src/CMakeLists.txt")
-          file (READ "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/src/CMakeLists.txt" vol_cmake_contents)
+        if (EXISTS "${hdf5_vol_depname_source_dir}/src/CMakeLists.txt")
+          file (READ "${hdf5_vol_depname_source_dir}/src/CMakeLists.txt" vol_cmake_contents)
           string (REGEX REPLACE "[ \t]*find_package[ \t]*\\([ \t]*HDF5[^\r\n\\)]*\\)[ \t]*[\r\n]+" "" vol_cmake_contents "${vol_cmake_contents}")
-          file (WRITE "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/src/CMakeLists.txt" "${vol_cmake_contents}")
+          file (WRITE "${hdf5_vol_depname_source_dir}/src/CMakeLists.txt" "${vol_cmake_contents}")
         endif ()
 
-        add_subdirectory (${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR} ${hdf5_vol_${hdf5_vol_name_lower}_BINARY_DIR})
+        # Apply any connector-specific workarounds
+        apply_connector_workarounds ("${hdf5_vol_name_lower}" "${hdf5_vol_depname_source_dir}")
+
+        add_subdirectory (${hdf5_vol_depname_source_dir} ${hdf5_vol_depname_binary_dir})
 
         # Get list of targets generated by build of connector
-        get_generated_cmake_targets (connector_targets ${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR})
+        get_generated_cmake_targets (connector_targets ${hdf5_vol_depname_source_dir})
 
         # Create a custom target for the connector to encompass all its
         # targets and other custom properties set by us for later use
@@ -217,8 +282,30 @@ if (HDF5_VOL_ALLOW_EXTERNAL)
             HDF5_VOL_TEST_PARALLEL ${HDF5_VOL_${hdf5_vol_name_upper}_TEST_PARALLEL}
         )
 
-        # Add this connector's target to the list of external connector targets
+        # Add this VOL connector's target to the list of external connector targets
         list (APPEND HDF5_EXTERNAL_VOL_TARGETS "HDF5_VOL_${hdf5_vol_name_lower}")
+
+        # Get the list of library targets from this VOL connector
+        unset (connector_lib_targets)
+        foreach (connector_target ${connector_targets})
+          get_target_property (target_type ${connector_target} TYPE)
+          if (target_type STREQUAL "SHARED_LIBRARY" OR target_type STREQUAL "STATIC_LIBRARY")
+            list (APPEND connector_lib_targets "${connector_target}")
+          endif ()
+        endforeach ()
+
+        # Add all of the previous VOL connector's library targets as
+        # dependencies for the current VOL connector to ensure that
+        # VOL connectors get built serially in case there are dependencies
+        if (DEFINED last_vol_lib_targets)
+          foreach (connector_target ${connector_targets})
+            add_dependencies (${connector_target} ${last_vol_lib_targets})
+          endforeach ()
+        endif ()
+
+        # Use this connector's library targets as dependencies
+        # for the next connector that is built
+        set (last_vol_lib_targets "${connector_lib_targets}")
       endif ()
     endif ()
   endforeach ()