diff options
author | jhendersonHDF <jhenderson@hdfgroup.org> | 2023-12-07 18:13:21 (GMT) |
---|---|---|
committer | GitHub <noreply@github.com> | 2023-12-07 18:13:21 (GMT) |
commit | fb84c696a476c39db2c35e7d86b818138e089358 (patch) | |
tree | 56c0fec578e2e0d2eaee455cf3ae133dca541989 /CMakeVOL.cmake | |
parent | dc8b455ac00e899b53127de0d270230f4d4e40a1 (diff) | |
download | hdf5-fb84c696a476c39db2c35e7d86b818138e089358.zip hdf5-fb84c696a476c39db2c35e7d86b818138e089358.tar.gz hdf5-fb84c696a476c39db2c35e7d86b818138e089358.tar.bz2 |
Sync 1.14 branch with develop (#3887)
* Preserve MPI-I/O file hints when fapl is closed (#3755)
* Fix for issue #3025: Save the MPI info in the file struct so H5Fget_access_plist() can retrieve it from there.
* Add compression tests for subfiling (#3769)
* Fix typo in comment (#3775)
* Fixed a file handle leak in the core VFD (#3779)
When opening a file with the core VFD and a file image, if the file
already exists, the file check would leak the POSIX file handle.
Fixes GitHub issue #635
* Fix a format string warning in the C++ examples (#3776)
* Cancel running GitHub workflows on push to same PR (#3772)
* Cancel running GitHub workflows on push to same PR
* Remove github.sha from workflow concurrency groups
* Print some messages in parallel tests on MPI rank 0 only (#3785)
Avoids overly verbose output from all processes emitting progress, etc. info.
* Avoid attempted use of NULL pointer in parallel compression code (#3786)
The parallel compression test code tests for the case where all MPI ranks have no selection in a dataset when writing to it. Add an early exit to the code to avoid attempting to use a NULL pointer due to there being no work to do.
* Don't install h5tools_test_utils test program on system (#3793)
* Add Doxygen to H5FDsplitter.h (#3794)
* H5FD_CURR_SPLITTER_VFD_CONFIG_VERSION
* H5FD_SPLITTER_PATH_MAX
* H5FD_SPLITTER_MAGIC
* H5FD_splitter_vfd_config_t
* H5Pset_fapl_splitter()
* H5Pget_fapl_splitter()
* Update Doxygen initializers & identifiers in VFDs (#3795)
* Add Doxygen for all H5FD_<VFD> initializers
* Add Doxygen for all H5FD_<VFD>_VALUE values
* Mark H5FD_<vfd>_init() calls private in Doxygen
* Fix memory corruption in 'MPI I/O FAPL preserve' test (#3806)
* Fix usage of h5_clean_files in t_pflush2.c (#3807)
* Fix parallel driver check in h5_fixname_real (#3808)
* Fix a couple usages of MPI_Info_get (#3809)
* Remove H5system.c warning on Windows oneAPI. (#3812)
* Add processing of NVHPC flags in linux-gnulibc1 file (#3804)
* Disable testing as tests are failing the same as in CMake
* Use the current toolchain for examples as default (#3810)
* Fix misc. warnings from GCC when compiling with -fsanitize=undefined (#3787)
* Set NVHPC maximum optimization level to -O1 for now (#3800)
* Set NVHPC maximum optimization level to -O1 for now
Compiling HDF5 with NVHPC 23.5 - 23.9 results in test failures in
4 different test files that need to be resolved. Since those tests
pass with an optimization level of -O1 (and -O0) and it is currently
unclear whether the test failures are due to issues in HDF5 or issues
in the 'nvc' compiler, set the maximum optimization level for NVHPC
to -O1 until the test failures are resolved.
* Disable nvhpc Java testing in CMake and amend known issues
* Re-enable testing of Autotools nvhpc
* Update some doxygen links to local refs (#3814)
* Rework MPI Info FAPL preserve PR to use VFD 'ctl' operations (#3782)
* Removed the use of C wrappers from H5P APIs. (#3824)
* fix seg fault on frontier/cray
* fix seg fault on frontier/cray
* fix seg fault on frontier/cray
* removed the use of h5pclose_c
* removed the use of h5pclose_c
* Fortran Wrappers H5VLnative_addr_to_token_f and H5VLnative_token_to_address_f (#3801)
* Added H5VLnative_addr_to_token_f and H5VLnative_token_to_address_f
* Added H5VLnative_addr_to_token_f and H5VLnative_token_to_address_f tests
* Create test for H5Pget_dxpl_mpio (#3825)
* Create test and add to testphdf5
* Renamed h5fuse.sh to h5fuse (#3834)
* provide an alternative to mapfile for older bash
* Disable FP exceptions in H5T init code (#3837)
The H5T floating-point datatype initialization code can raise exceptions when handling signaling NaNs. This change disables FE_INVALID exceptions during initialization.
Also removes the -ieee=full change for NAG Fortran as that shouldn't be necessary anymore.
Fixes #3831
* Add intel oneapi windows build to CI CMake (#3836)
* Remove printf format warning on Windows oneAPI. (#3838)
* Correct ENV variables (#3841)
* Remove Autotools sed hack (#3848)
configure.ac contains a sed line that cleans up incorrect library
flags which was added to paper over some bugs in earlier versions
of the Autotools. These issues are not a problem with the current
versions of the Autootols.
The sed line causes problems on MacOS, so it has been removed.
Fixes #3843
* Make filter unregister callbacks safe for VOL connectors (#3629)
* Make filter callbacks use top-level API functions
When using VOL connectors, H5I_iterate may not provide
valid object pointers to its callback. This change keeps
existing functionality in H5Zunregister() without using
potentially unsafe pointers.
* Filter callbacks use internal API
* Skip MPI work on non-native VOL
* Add extra space in comments for consistency (#3852)
* Add extra space in comments for consistency
* uncomment tfloatsattrs test
* Update Actions badges to link to relevant workflow (#3850)
* Add CMake long double cross-compile defaults (#3683)
HDF5 performs a couple of checks at build time to see if long double
values can be converted correctly (IBM's Power architecture uses a
special format for long doubles). These checks were performed using
TRY_RUN, which is a problem when cross-compiling.
These checks now use default values appropriate for most non-Power
systems when cross-compiling. The cache values can be pre-set if
necessary, which will preempt both the TRY_RUN and the default.
Affected values:
H5_LDOUBLE_TO_LONG_SPECIAL (default no)
H5_LONG_TO_LDOUBLE_SPECIAL (default no)
H5_LDOUBLE_TO_LLONG_ACCURATE (default yes)
H5_LLONG_TO_LDOUBLE_CORRECT (default yes)
H5_DISABLE_SOME_LDOUBLE_CONV (default no)
Fixes GitHub #3585
* Updates for building and testing VOL connectors
* Fix issue with HDF5_VOL_ALLOW_EXTERNAL CMake variable
* Initialize parallel testing with MPI_THREAD_MULTIPLE when testing API
* Add CMake variable to allow specifying a VOL connector's package name
* Remove call to MPI_Init in serial API tests
While previously necessary, it now interferes with VOL connectors that
may need to be initialized with MPI_THREAD_MULTIPLE
* Fixes for CI and presets (#3853)
* Change dest for doxygen (#3856)
* Implement selection vector I/O with collective chunk filling (#3826)
* Changes for ECP-344: Implement selection vector I/O with collective chunk filling.
Also fix a bug in H5FD__mpio_write_vector() to account for fixed size optimization
when computing max address.
* Fixes based on PR review comments:
For H5Dchunk.c: fix H5MM_xfree()
For H5FDmpio.c:
1) Revert the fix to H5FD__mpio_write_vector()
2) Apply the patch from Neil on the proper length of s_sizes reported by H5FD__mpio_vector_build_types()
* Put back the logic of dividing up the work among all the mpi ranks similar to the
original H5D__chunk_collective_fill() routine.
* Add a test to verify the fix for the illegal reference problem in H5FD__mpio_write_vector().
* Do not publish compression headers or docs (#3865)
* Fix typo: look -> loop (#3866)
* Moved the README to markdown and expanded its overview of the files, file generation, and other Fortran wrapper development practices as mentioned in the HDF5 architectural document. I added a new figure and included the SVG file and the original xfig file it was generated from. (#3862)
* Add HDF5_DISABLE_TESTS_REGEX option to skip tests (#3859)
* Fix typo in error message for `MPI_Type_dup`. (#3867)
* Complete the `if command line option` sentence. (#3868)
* Fix h5dump segmentation fault when --vfd-value option is used (#3873)
* Updated URL in funding.yml (#3882)
Using new shortened URL, might look better.
* Remove unused variable from unmerged changes
* Update src/H5Tinit_float.c
Diffstat (limited to 'CMakeVOL.cmake')
-rw-r--r-- | CMakeVOL.cmake | 139 |
1 files changed, 113 insertions, 26 deletions
diff --git a/CMakeVOL.cmake b/CMakeVOL.cmake index 79afa59..8b2207a 100644 --- a/CMakeVOL.cmake +++ b/CMakeVOL.cmake @@ -26,14 +26,51 @@ function (get_generated_cmake_targets out_var dir) set (${out_var} "${dir_targets}" PARENT_SCOPE) endfunction () -# For now, only support building of external VOL connectors with FetchContent -option (HDF5_VOL_ALLOW_EXTERNAL "Allow building of external HDF5 VOL connectors with FetchContent" "NO") -mark_as_advanced (HDF5_VOL_ALLOW_EXTERNAL) -if (HDF5_VOL_ALLOW_EXTERNAL) - if (HDF5_VOL_ALLOW_EXTERNAL MATCHES "NO" OR (NOT HDF5_VOL_ALLOW_EXTERNAL MATCHES "GIT" AND NOT HDF5_VOL_ALLOW_EXTERNAL MATCHES "LOCAL_DIR")) - message (FATAL_ERROR "HDF5_VOL_ALLOW_EXTERNAL must be set to 'GIT' or 'LOCAL_DIR' to allow building of external HDF5 VOL connectors") - endif() +# Function to apply connector-specify workarounds to build +# code once a connector has been populated through FetchContent +function (apply_connector_workarounds connector_name source_dir) + # For the cache VOL, remove the call to find_package(ASYNC). + # Eventually, the FetchContent OVERRIDE_FIND_PACKAGE should be + # able to fulfill this dependency when building the cache VOL, + # but for now we have to hack around this until the async and + # cache VOLs create CMake .config files + if ("${connector_name}" MATCHES "vol-cache") + # Remove find_package(ASYNC) call from connector's CMake code + file (READ "${source_dir}/CMakeLists.txt" vol_cmake_contents) + string (REGEX REPLACE "[ \t]*find_package[ \t]*\\([ \t]*ASYNC[^\r\n\\)]*\\)[ \t]*[\r\n]+" "" vol_cmake_contents "${vol_cmake_contents}") + file (WRITE "${source_dir}/CMakeLists.txt" "${vol_cmake_contents}") + + # Remove setting of HDF5_VOL_CONNECTOR and HDF5_PLUGIN_PATH + # in connector's external tests CMake code + file (STRINGS "${source_dir}/tests/CMakeLists.txt" file_lines) + file (WRITE "${source_dir}/tests/CMakeLists.txt" "") + foreach (line IN LISTS file_lines) + set (stripped_line "${line}") + string (REGEX MATCH "^[ \t]*set_tests_properties\\([ \t]*[\r\n]?" match_string "${line}") + if (NOT "${match_string}" STREQUAL "") + string (REGEX REPLACE "^[ \t]*set_tests_properties\\([ \t]*[\r\n]?" "" stripped_line "${line}") + endif () + string (REGEX MATCH "^[ \t]*.\\{test\\}[ \t]*[\r\n]?" match_string "${line}") + if (NOT "${match_string}" STREQUAL "") + string (REGEX REPLACE "^[ \t]*.\\{[A-Za-z]*\\}[ \t]*[\r\n]?" "" stripped_line "${line}") + endif () + string (REGEX MATCH "^[ \t]*PROPERTIES[ \t]*[\r\n]?" match_string "${line}") + if (NOT "${match_string}" STREQUAL "") + string (REGEX REPLACE "^[ \t]*PROPERTIES[ \t]*[\r\n]?" "" stripped_line "${line}") + endif () + string (REGEX MATCH "^[ \t]*ENVIRONMENT[ \t]*.*[\r\n]?" match_string "${line}") + if (NOT "${match_string}" STREQUAL "") + string (REGEX REPLACE "^[ \t]*ENVIRONMENT[ \t]*.*[\r\n]?" "" stripped_line "${line}") + endif () + file (APPEND "${source_dir}/tests/CMakeLists.txt" "${stripped_line}\n") + endforeach () + endif () +endfunction () +set (HDF5_VOL_ALLOW_EXTERNAL "NO" CACHE STRING "Allow building of external HDF5 VOL connectors with FetchContent") +set_property (CACHE HDF5_VOL_ALLOW_EXTERNAL PROPERTY STRINGS NO GIT LOCAL_DIR) +mark_as_advanced (HDF5_VOL_ALLOW_EXTERNAL) +if (HDF5_VOL_ALLOW_EXTERNAL MATCHES "GIT" OR HDF5_VOL_ALLOW_EXTERNAL MATCHES "LOCAL_DIR") # For compatibility, set some variables that projects would # typically look for after calling find_package(HDF5) set (HDF5_FOUND 1) @@ -103,6 +140,13 @@ if (HDF5_VOL_ALLOW_EXTERNAL) mark_as_advanced ("HDF5_VOL_${hdf5_vol_name_upper}_BRANCH") endif() + set ("HDF5_VOL_${hdf5_vol_name_upper}_CMAKE_PACKAGE_NAME" + "${hdf5_vol_name_lower}" + CACHE + STRING + "CMake package name used by find_package(...) calls for VOL connector '${hdf5_vol_name}'" + ) + set ("HDF5_VOL_${hdf5_vol_name_upper}_NAME" "" CACHE STRING "Name of VOL connector to set for the HDF5_VOL_CONNECTOR environment variable") option ("HDF5_VOL_${hdf5_vol_name_upper}_TEST_PARALLEL" "Whether to test VOL connector '${hdf5_vol_name}' against the parallel API tests" OFF) @@ -124,22 +168,40 @@ if (HDF5_VOL_ALLOW_EXTERNAL) message (FATAL_ERROR "HDF5_VOL_PATH${vol_idx_fixed} must be an absolute path to a valid directory") endif () + # Set internal convenience variables for FetchContent dependency name + set (hdf5_vol_depname "${HDF5_VOL_${hdf5_vol_name_upper}_CMAKE_PACKAGE_NAME}") + string (TOLOWER "${hdf5_vol_depname}" hdf5_vol_depname_lower) + if (HDF5_VOL_ALLOW_EXTERNAL MATCHES "GIT") - FetchContent_Declare (HDF5_VOL_${hdf5_vol_name_lower} - GIT_REPOSITORY "${HDF5_VOL_SOURCE}" - GIT_TAG "${HDF5_VOL_${hdf5_vol_name_upper}_BRANCH}" - ) + if (${CMAKE_VERSION} VERSION_GREATER_EQUAL "3.24") + FetchContent_Declare (${hdf5_vol_depname} + GIT_REPOSITORY "${HDF5_VOL_SOURCE}" + GIT_TAG "${HDF5_VOL_${hdf5_vol_name_upper}_BRANCH}" + OVERRIDE_FIND_PACKAGE + ) + else () + FetchContent_Declare (${hdf5_vol_depname} + GIT_REPOSITORY "${HDF5_VOL_SOURCE}" + GIT_TAG "${HDF5_VOL_${hdf5_vol_name_upper}_BRANCH}" + FIND_PACKAGE_ARGS NAMES ${hdf5_vol_name_lower} + ) + endif () elseif(HDF5_VOL_ALLOW_EXTERNAL MATCHES "LOCAL_DIR") - FetchContent_Declare (HDF5_VOL_${hdf5_vol_name_lower} - SOURCE_DIR "${HDF5_VOL_SOURCE}" + FetchContent_Declare (${hdf5_vol_depname} + SOURCE_DIR "${HDF5_VOL_SOURCE}" ) endif() - FetchContent_GetProperties(HDF5_VOL_${hdf5_vol_name_lower}) - if (NOT hdf5_vol_${hdf5_vol_name_lower}_POPULATED) - FetchContent_Populate(HDF5_VOL_${hdf5_vol_name_lower}) + FetchContent_GetProperties(${hdf5_vol_depname}) + if (NOT ${hdf5_vol_depname}_POPULATED) + FetchContent_Populate(${hdf5_vol_depname}) + + # Now that content has been populated, set other internal + # convenience variables for FetchContent dependency + set (hdf5_vol_depname_source_dir "${${hdf5_vol_depname_lower}_SOURCE_DIR}") + set (hdf5_vol_depname_binary_dir "${${hdf5_vol_depname_lower}_BINARY_DIR}") - if (NOT EXISTS "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/CMakeLists.txt") + if (NOT EXISTS "${hdf5_vol_depname_source_dir}/CMakeLists.txt") if (HDF5_VOL_ALLOW_EXTERNAL MATCHES "GIT") message (SEND_ERROR "The git repository branch '${HDF5_VOL_${hdf5_vol_name_upper}_BRANCH}' for VOL connector '${hdf5_vol_name}' does not appear to contain a CMakeLists.txt file") elseif (HDF5_VOL_ALLOW_EXTERNAL MATCHES "LOCAL_DIR") @@ -150,21 +212,24 @@ if (HDF5_VOL_ALLOW_EXTERNAL) # If there are any calls to find_package(HDF5) in the connector's # CMakeLists.txt files, remove those since any found HDF5 targets # will conflict with targets being generated by this build of HDF5 - if (EXISTS "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/CMakeLists.txt") - file (READ "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/CMakeLists.txt" vol_cmake_contents) + if (EXISTS "${hdf5_vol_depname_source_dir}/CMakeLists.txt") + file (READ "${hdf5_vol_depname_source_dir}/CMakeLists.txt" vol_cmake_contents) string (REGEX REPLACE "[ \t]*find_package[ \t]*\\([ \t]*HDF5[^\r\n\\)]*\\)[ \t]*[\r\n]+" "" vol_cmake_contents "${vol_cmake_contents}") - file (WRITE "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/CMakeLists.txt" "${vol_cmake_contents}") + file (WRITE "${hdf5_vol_depname_source_dir}/CMakeLists.txt" "${vol_cmake_contents}") endif () - if (EXISTS "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/src/CMakeLists.txt") - file (READ "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/src/CMakeLists.txt" vol_cmake_contents) + if (EXISTS "${hdf5_vol_depname_source_dir}/src/CMakeLists.txt") + file (READ "${hdf5_vol_depname_source_dir}/src/CMakeLists.txt" vol_cmake_contents) string (REGEX REPLACE "[ \t]*find_package[ \t]*\\([ \t]*HDF5[^\r\n\\)]*\\)[ \t]*[\r\n]+" "" vol_cmake_contents "${vol_cmake_contents}") - file (WRITE "${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}/src/CMakeLists.txt" "${vol_cmake_contents}") + file (WRITE "${hdf5_vol_depname_source_dir}/src/CMakeLists.txt" "${vol_cmake_contents}") endif () - add_subdirectory (${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR} ${hdf5_vol_${hdf5_vol_name_lower}_BINARY_DIR}) + # Apply any connector-specific workarounds + apply_connector_workarounds ("${hdf5_vol_name_lower}" "${hdf5_vol_depname_source_dir}") + + add_subdirectory (${hdf5_vol_depname_source_dir} ${hdf5_vol_depname_binary_dir}) # Get list of targets generated by build of connector - get_generated_cmake_targets (connector_targets ${hdf5_vol_${hdf5_vol_name_lower}_SOURCE_DIR}) + get_generated_cmake_targets (connector_targets ${hdf5_vol_depname_source_dir}) # Create a custom target for the connector to encompass all its # targets and other custom properties set by us for later use @@ -217,8 +282,30 @@ if (HDF5_VOL_ALLOW_EXTERNAL) HDF5_VOL_TEST_PARALLEL ${HDF5_VOL_${hdf5_vol_name_upper}_TEST_PARALLEL} ) - # Add this connector's target to the list of external connector targets + # Add this VOL connector's target to the list of external connector targets list (APPEND HDF5_EXTERNAL_VOL_TARGETS "HDF5_VOL_${hdf5_vol_name_lower}") + + # Get the list of library targets from this VOL connector + unset (connector_lib_targets) + foreach (connector_target ${connector_targets}) + get_target_property (target_type ${connector_target} TYPE) + if (target_type STREQUAL "SHARED_LIBRARY" OR target_type STREQUAL "STATIC_LIBRARY") + list (APPEND connector_lib_targets "${connector_target}") + endif () + endforeach () + + # Add all of the previous VOL connector's library targets as + # dependencies for the current VOL connector to ensure that + # VOL connectors get built serially in case there are dependencies + if (DEFINED last_vol_lib_targets) + foreach (connector_target ${connector_targets}) + add_dependencies (${connector_target} ${last_vol_lib_targets}) + endforeach () + endif () + + # Use this connector's library targets as dependencies + # for the next connector that is built + set (last_vol_lib_targets "${connector_lib_targets}") endif () endif () endforeach () |