diff options
author | Brad King <brad.king@kitware.com> | 2020-09-10 14:56:32 (GMT) |
---|---|---|
committer | Brad King <brad.king@kitware.com> | 2020-09-14 14:48:16 (GMT) |
commit | b4fc4da903ad5c69cfbf693839ee5020c3f2ede6 (patch) | |
tree | d473e70256a9f5ec5dd34149bf3c1463f6c51de3 /Modules/ExternalProject.cmake | |
parent | f5791e24c63617901ae033c14d99c0e02d402b05 (diff) | |
download | CMake-b4fc4da903ad5c69cfbf693839ee5020c3f2ede6.zip CMake-b4fc4da903ad5c69cfbf693839ee5020c3f2ede6.tar.gz CMake-b4fc4da903ad5c69cfbf693839ee5020c3f2ede6.tar.bz2 |
ExternalProject: Add policy CMP0114 to refine step target dependencies
`ExternalProject_Add_StepTargets` and `INDEPENDENT_STEP_TARGETS` have
some limitations and lack some sanity checks. They can cause confusing
build systems to be generated. The basic problems are:
* The notion of step independence is attached to the step target
rather than the step itself.
* The custom commands implementing the steps are duplicated in the
step targets and the primary targets. This can cause races.
It is also incompatible with the Xcode "new build system".
Fix this by introducing policy CMP0114 to change the way step target
dependencies are handled. Define independence from external
dependencies as a property of each individual step regardless of whether
there is a target for it. Add dependencies among the primary target and
the step targets such that each custom command only appears in one
target. When some steps are disconnected from the primary target, add
step targets for the steps commonly depended upon so that there is a
place to hold their custom commands uniquely.
Fixes: #18663
Diffstat (limited to 'Modules/ExternalProject.cmake')
-rw-r--r-- | Modules/ExternalProject.cmake | 278 |
1 files changed, 235 insertions, 43 deletions
diff --git a/Modules/ExternalProject.cmake b/Modules/ExternalProject.cmake index 5ab9784..b795784 100644 --- a/Modules/ExternalProject.cmake +++ b/Modules/ExternalProject.cmake @@ -398,6 +398,9 @@ External Project Definition project also provides a cache variable or some other convenient method for setting the directory property). + This may cause a step target to be created automatically for the + ``download`` step. See policy :policy:`CMP0114`. + ``PATCH_COMMAND <cmd>...`` Specifies a custom command to patch the sources after an update. By default, no patch command is defined. Note that it can be quite difficult @@ -578,6 +581,8 @@ External Project Definition If enabled, the main build's default ALL target will not depend on the test step. This can be a useful way of ensuring the test step is defined but only gets invoked when manually requested. + This may cause a step target to be created automatically for either + the ``install`` or ``build`` step. See policy :policy:`CMP0114`. **Output Logging Options:** Each of the following ``LOG_...`` options can be used to wrap the relevant @@ -664,19 +669,21 @@ External Project Definition steps need to be triggered manually or if they need to be used as dependencies of other targets. If this option is not specified, the default value is taken from the ``EP_STEP_TARGETS`` directory property. - See :command:`ExternalProject_Add_Step` below for further discussion of - the effects of this option. + See :command:`ExternalProject_Add_StepTargets` below for further + discussion of the effects of this option. ``INDEPENDENT_STEP_TARGETS <step-target>...`` - Generate custom targets for the specified steps and prevent these targets + Deprecated. This is allowed only if policy :policy:`CMP0114` is not set + to ``NEW``. + Generates custom targets for the specified steps and prevent these targets from having the usual dependencies applied to them. If this option is not specified, the default value is taken from the ``EP_INDEPENDENT_STEP_TARGETS`` directory property. This option is mostly useful for allowing individual steps to be driven independently, such as for a CDash setup where each step should be initiated and reported individually rather than as one whole build. See - :command:`ExternalProject_Add_Step` below for further discussion of the - effects of this option. + :command:`ExternalProject_Add_StepTargets` below for further discussion + of the effects of this option. **Miscellaneous Options:** ``LIST_SEPARATOR <sep>`` @@ -772,6 +779,21 @@ control needed to implement such step-level capabilities. ``DEPENDS <file>...`` Files on which this custom step depends. + ``INDEPENDENT <bool>`` + Specifies whether this step is independent of the external dependencies + specified by the :command:`ExternalProject_Add`'s ``DEPENDS`` option. + The default is ``FALSE``. Steps marked as independent may depend only + on other steps marked independent. See policy :policy:`CMP0114`. + + Note that this use of the term "independent" refers only to independence + from external targets specified by the ``DEPENDS`` option and is + orthogonal to a step's dependencies on other steps. + + If a step target is created for an independent step by the + :command:`ExternalProject_Add` ``STEP_TARGETS`` option or by the + :command:`ExternalProject_Add_StepTargets` function, it will not depend + on the external targets, but may depend on targets for other steps. + ``BYPRODUCTS <file>...`` Files that will be generated by this custom step but which might or might not have their modification time updated by subsequent builds. This list of @@ -785,6 +807,8 @@ control needed to implement such step-level capabilities. ``EXCLUDE_FROM_MAIN <bool>`` When enabled, this option specifies that the external project's main target does not depend on the custom step. + This may cause step targets to be created automatically for the steps on + which this step depends. See policy :policy:`CMP0114`. ``WORKING_DIRECTORY <dir>`` Specifies the working directory to set before running the custom step's @@ -815,7 +839,7 @@ control needed to implement such step-level capabilities. .. code-block:: cmake - ExternalProject_Add_StepTargets(<name> [NO_DEPENDS] <step1> [<step2>...]) + ExternalProject_Add_StepTargets(<name> <step1> [<step2>...]) Creating a target for a step allows it to be used as a dependency of another target or to be triggered manually. Having targets for specific steps also @@ -827,37 +851,53 @@ control needed to implement such step-level capabilities. through the step dependency chain, then all the previous steps will also run to ensure everything is up to date. - If the ``NO_DEPENDS`` option is specified, the step target will not depend on - the dependencies of the external project (i.e. on any dependencies of the - ``<name>`` custom target created by :command:`ExternalProject_Add`). This is - usually safe for the ``download``, ``update`` and ``patch`` steps, since they - do not typically require that the dependencies are updated and built. Using - ``NO_DEPENDS`` for any of the other pre-defined steps, however, may break - parallel builds. Only use ``NO_DEPENDS`` where it is certain that the named - steps genuinely do not have dependencies. For custom steps, consider whether - or not the custom commands require the dependencies to be configured, built - and installed. - Internally, :command:`ExternalProject_Add` calls :command:`ExternalProject_Add_Step` to create each step. If any - ``STEP_TARGETS`` or ``INDEPENDENT_STEP_TARGETS`` were specified, then - ``ExternalProject_Add_StepTargets()`` will also be called after - :command:`ExternalProject_Add_Step`. ``INDEPENDENT_STEP_TARGETS`` have the - ``NO_DEPENDS`` option set, whereas ``STEP_TARGETS`` do not. Other than that, - the two options result in ``ExternalProject_Add_StepTargets()`` being called - in the same way. Even if a step is not mentioned in either of those two - options, ``ExternalProject_Add_StepTargets()`` can still be called later to - manually define a target for the step. - - The ``STEP_TARGETS`` and ``INDEPENDENT_STEP_TARGETS`` options for - :command:`ExternalProject_Add` are generally the easiest way to ensure - targets are created for specific steps of interest. For custom steps, - ``ExternalProject_Add_StepTargets()`` must be called explicitly if a target - should also be created for that custom step. An alternative to these two - options is to populate the ``EP_STEP_TARGETS`` and - ``EP_INDEPENDENT_STEP_TARGETS`` directory properties. These act as defaults - for the step target options and can save having to repeatedly specify the - same set of step targets when multiple external projects are being defined. + ``STEP_TARGETS`` were specified, then ``ExternalProject_Add_StepTargets()`` + will also be called after :command:`ExternalProject_Add_Step`. Even if a + step is not mentioned in the ``STEP_TARGETS`` option, + ``ExternalProject_Add_StepTargets()`` can still be called later to manually + define a target for the step. + + The ``STEP_TARGETS`` option for :command:`ExternalProject_Add` is generally + the easiest way to ensure targets are created for specific steps of interest. + For custom steps, ``ExternalProject_Add_StepTargets()`` must be called + explicitly if a target should also be created for that custom step. + An alternative to these two options is to populate the ``EP_STEP_TARGETS`` + directory property. It acts as a default for the step target options and + can save having to repeatedly specify the same set of step targets when + multiple external projects are being defined. + + If :policy:`CMP0114` is set to ``NEW``, step targets are fully responsible + for holding the custom commands implementing their steps. The primary target + created by ``ExternalProject_Add`` depends on the step targets, and the + step targets depend on each other. The target-level dependencies match + the file-level dependencies used by the custom commands for each step. + The targets for steps created with :command:`ExternalProject_Add_Step`'s + ``INDEPENDENT`` option do not depend on the external targets specified + by :command:`ExternalProject_Add`'s ``DEPENDS`` option. The predefined + steps ``mkdir``, ``download``, ``update``, and ``patch`` are independent. + + If :policy:`CMP0114` is not ``NEW``, the following deprecated behavior + is available: + + * A deprecated ``NO_DEPENDS`` option may be specified immediately after the + ``<name>`` and before the first step. + If the ``NO_DEPENDS`` option is specified, the step target will not depend on + the dependencies of the external project (i.e. on any dependencies of the + ``<name>`` custom target created by :command:`ExternalProject_Add`). This is + usually safe for the ``download``, ``update`` and ``patch`` steps, since they + do not typically require that the dependencies are updated and built. Using + ``NO_DEPENDS`` for any of the other pre-defined steps, however, may break + parallel builds. Only use ``NO_DEPENDS`` where it is certain that the named + steps genuinely do not have dependencies. For custom steps, consider whether + or not the custom commands require the dependencies to be configured, built + and installed. + + * The ``INDEPENDENT_STEP_TARGETS`` option for :command:`ExternalProject_Add`, + or the ``EP_INDEPENDENT_STEP_TARGETS`` directory property, tells the + function to call ``ExternalProject_Add_StepTargets()`` internally + using the ``NO_DEPENDS`` option for the specified steps. .. command:: ExternalProject_Add_StepDependencies @@ -1991,19 +2031,43 @@ endfunction() function(_ep_step_add_target name step no_deps) + if(TARGET ${name}-${step}) + return() + endif() + get_property(cmp0114 TARGET ${name} PROPERTY _EP_CMP0114) _ep_get_step_stampfile(${name} ${step} stamp_file) + cmake_policy(PUSH) + if(cmp0114 STREQUAL "NEW") + # To implement CMP0114 NEW behavior with Makefile generators, + # we need CMP0113 NEW behavior. + cmake_policy(SET CMP0113 NEW) + endif() add_custom_target(${name}-${step} DEPENDS ${stamp_file}) + cmake_policy(POP) set_property(TARGET ${name}-${step} PROPERTY _EP_IS_EXTERNAL_PROJECT_STEP 1) set_property(TARGET ${name}-${step} PROPERTY LABELS ${name}) set_property(TARGET ${name}-${step} PROPERTY FOLDER "ExternalProjectTargets/${name}") - if(no_deps AND "${step}" MATCHES "^(configure|build|install|test)$") - message(AUTHOR_WARNING "Using NO_DEPENDS for \"${step}\" step might break parallel builds") + if(cmp0114 STREQUAL "NEW") + # Add target-level dependencies for the step. + get_property(exclude_from_main TARGET ${name} PROPERTY _EP_${step}_EXCLUDE_FROM_MAIN) + if(NOT exclude_from_main) + add_dependencies(${name} ${name}-${step}) + endif() + _ep_step_add_target_dependencies(${name} ${step} ${step}) + _ep_step_add_target_dependents(${name} ${step} ${step}) + + get_property(independent TARGET ${name} PROPERTY _EP_${step}_INDEPENDENT) + else() + if(no_deps AND "${step}" MATCHES "^(configure|build|install|test)$") + message(AUTHOR_WARNING "Using NO_DEPENDS for \"${step}\" step might break parallel builds") + endif() + set(independent ${no_deps}) endif() # Depend on other external projects (target-level). - if(NOT no_deps) + if(NOT independent) get_property(deps TARGET ${name} PROPERTY _EP_DEPENDS) foreach(arg IN LISTS deps) add_dependencies(${name}-${step} ${arg}) @@ -2012,7 +2076,43 @@ function(_ep_step_add_target name step no_deps) endfunction() +function(_ep_step_add_target_dependencies name step node) + get_property(dependees TARGET ${name} PROPERTY _EP_${node}_INTERNAL_DEPENDEES) + list(REMOVE_DUPLICATES dependees) + foreach(dependee IN LISTS dependees) + get_property(exclude_from_main TARGET ${name} PROPERTY _EP_${step}_EXCLUDE_FROM_MAIN) + get_property(dependee_dependers TARGET ${name} PROPERTY _EP_${dependee}_INTERNAL_DEPENDERS) + if(exclude_from_main OR dependee_dependers MATCHES ";") + # The step on which our step target depends itself has + # dependents in multiple targes. It needs a step target too + # so that there is a unique place for its custom command. + _ep_step_add_target("${name}" "${dependee}" "FALSE") + endif() + + if(TARGET ${name}-${dependee}) + add_dependencies(${name}-${step} ${name}-${dependee}) + else() + _ep_step_add_target_dependencies(${name} ${step} ${dependee}) + endif() + endforeach() +endfunction() + + +function(_ep_step_add_target_dependents name step node) + get_property(dependers TARGET ${name} PROPERTY _EP_${node}_INTERNAL_DEPENDERS) + list(REMOVE_DUPLICATES dependers) + foreach(depender IN LISTS dependers) + if(TARGET ${name}-${depender}) + add_dependencies(${name}-${depender} ${name}-${step}) + else() + _ep_step_add_target_dependents(${name} ${step} ${depender}) + endif() + endforeach() +endfunction() + + function(ExternalProject_Add_StepTargets name) + get_property(cmp0114 TARGET ${name} PROPERTY _EP_CMP0114) set(steps ${ARGN}) if(ARGC GREATER 1 AND "${ARGV1}" STREQUAL "NO_DEPENDS") set(no_deps 1) @@ -2020,6 +2120,28 @@ function(ExternalProject_Add_StepTargets name) else() set(no_deps 0) endif() + if(cmp0114 STREQUAL "NEW") + if(no_deps) + message(FATAL_ERROR + "The 'NO_DEPENDS' option is no longer allowed. " + "It has been superseded by the per-step 'INDEPENDENT' option. " + "See policy CMP0114." + ) + endif() + elseif(cmp0114 STREQUAL "") + cmake_policy(GET_WARNING CMP0114 _cmp0114_warning) + string(APPEND _cmp0114_warning "\n" + "ExternalProject target '${name}' would depend on the targets for " + "step(s) '${steps}' under policy CMP0114, but this is being left out " + "for compatibility since the policy is not set." + ) + if(no_deps) + string(APPEND _cmp0114_warning + " Also, the NO_DEPENDS option is deprecated in favor of policy CMP0114." + ) + endif() + message(AUTHOR_WARNING "${_cmp0114_warning}") + endif() foreach(step ${steps}) _ep_step_add_target("${name}" "${step}" "${no_deps}") endforeach() @@ -2027,12 +2149,19 @@ endfunction() function(ExternalProject_Add_Step name step) + get_property(cmp0114 TARGET ${name} PROPERTY _EP_CMP0114) _ep_get_complete_stampfile(${name} complete_stamp_file) _ep_get_step_stampfile(${name} ${step} stamp_file) _ep_parse_arguments(ExternalProject_Add_Step ${name} _EP_${step}_ "${ARGN}") + get_property(independent TARGET ${name} PROPERTY _EP_${step}_INDEPENDENT) + if(independent STREQUAL "") + set(independent FALSE) + set_property(TARGET ${name} PROPERTY _EP_${step}_INDEPENDENT "${independent}") + endif() + get_property(exclude_from_main TARGET ${name} PROPERTY _EP_${step}_EXCLUDE_FROM_MAIN) if(NOT exclude_from_main) add_custom_command(APPEND @@ -2043,12 +2172,21 @@ function(ExternalProject_Add_Step name step) # Steps depending on this step. get_property(dependers TARGET ${name} PROPERTY _EP_${step}_DEPENDERS) + set_property(TARGET ${name} APPEND PROPERTY _EP_${step}_INTERNAL_DEPENDERS ${dependers}) foreach(depender IN LISTS dependers) + set_property(TARGET ${name} APPEND PROPERTY _EP_${depender}_INTERNAL_DEPENDEES ${step}) _ep_get_step_stampfile(${name} ${depender} depender_stamp_file) add_custom_command(APPEND OUTPUT ${depender_stamp_file} DEPENDS ${stamp_file} ) + if(cmp0114 STREQUAL "NEW" AND NOT independent) + get_property(dep_independent TARGET ${name} PROPERTY _EP_${depender}_INDEPENDENT) + if(dep_independent) + message(FATAL_ERROR "ExternalProject '${name}' step '${depender}' is marked INDEPENDENT " + "but depends on step '${step}' that is not marked INDEPENDENT.") + endif() + endif() endforeach() # Dependencies on files. @@ -2059,9 +2197,18 @@ function(ExternalProject_Add_Step name step) # Dependencies on steps. get_property(dependees TARGET ${name} PROPERTY _EP_${step}_DEPENDEES) + set_property(TARGET ${name} APPEND PROPERTY _EP_${step}_INTERNAL_DEPENDEES ${dependees}) foreach(dependee IN LISTS dependees) + set_property(TARGET ${name} APPEND PROPERTY _EP_${dependee}_INTERNAL_DEPENDERS ${step}) _ep_get_step_stampfile(${name} ${dependee} dependee_stamp_file) list(APPEND depends ${dependee_stamp_file}) + if(cmp0114 STREQUAL "NEW" AND independent) + get_property(dep_independent TARGET ${name} PROPERTY _EP_${dependee}_INDEPENDENT) + if(NOT dep_independent) + message(FATAL_ERROR "ExternalProject '${name}' step '${step}' is marked INDEPENDENT " + "but depends on step '${dependee}' that is not marked INDEPENDENT.") + endif() + endif() endforeach() # The command to run. @@ -2165,12 +2312,37 @@ function(ExternalProject_Add_Step name step) if(NOT independent_step_targets) get_property(independent_step_targets DIRECTORY PROPERTY EP_INDEPENDENT_STEP_TARGETS) endif() - foreach(st ${independent_step_targets}) - if("${st}" STREQUAL "${step}") - _ep_step_add_target("${name}" "${step}" "TRUE") - break() + if(cmp0114 STREQUAL "NEW") + if(independent_step_targets) + message(FATAL_ERROR + "ExternalProject '${name}' option 'INDEPENDENT_STEP_TARGETS' is set to\n" + " ${independent_step_targets}\n" + "but the option is no longer allowed. " + "It has been superseded by the per-step 'INDEPENDENT' option. " + "See policy CMP0114." + ) endif() - endforeach() + else() + if(independent_step_targets AND cmp0114 STREQUAL "") + get_property(warned TARGET ${name} PROPERTY _EP_CMP0114_WARNED_INDEPENDENT_STEP_TARGETS) + if(NOT warned) + set_property(TARGET ${name} PROPERTY _EP_CMP0114_WARNED_INDEPENDENT_STEP_TARGETS 1) + cmake_policy(GET_WARNING CMP0114 _cmp0114_warning) + string(APPEND _cmp0114_warning "\n" + "ExternalProject '${name}' option INDEPENDENT_STEP_TARGETS is set to\n" + " ${independent_step_targets}\n" + "but the option is deprecated in favor of policy CMP0114." + ) + message(AUTHOR_WARNING "${_cmp0114_warning}") + endif() + endif() + foreach(st ${independent_step_targets}) + if("${st}" STREQUAL "${step}") + _ep_step_add_target("${name}" "${step}" "TRUE") + break() + endif() + endforeach() + endif() endfunction() @@ -2233,6 +2405,7 @@ function(_ep_add_mkdir_command name) _ep_get_configuration_subdir_suffix(cfgdir) ExternalProject_Add_Step(${name} mkdir + INDEPENDENT TRUE COMMENT "Creating directories for '${name}'" COMMAND ${CMAKE_COMMAND} -E make_directory ${source_dir} COMMAND ${CMAKE_COMMAND} -E make_directory ${binary_dir} @@ -2608,6 +2781,7 @@ function(_ep_add_download_command name) endforeach() cmake_language(EVAL CODE " ExternalProject_Add_Step(\${name} download + INDEPENDENT TRUE COMMENT \${comment} COMMAND ${__cmdQuoted} WORKING_DIRECTORY \${work_dir} @@ -2771,6 +2945,7 @@ Update to Mercurial >= 2.1.1. endforeach() cmake_language(EVAL CODE " ExternalProject_Add_Step(${name} update + INDEPENDENT TRUE COMMENT \${comment} COMMAND ${__cmdQuoted} ALWAYS \${always} @@ -2817,6 +2992,7 @@ function(_ep_add_patch_command name) endforeach() cmake_language(EVAL CODE " ExternalProject_Add_Step(${name} patch + INDEPENDENT TRUE COMMAND ${__cmdQuoted} WORKING_DIRECTORY \${work_dir} DEPENDEES \${patch_dep} @@ -2981,6 +3157,7 @@ function(_ep_add_configure_command name) endforeach() cmake_language(EVAL CODE " ExternalProject_Add_Step(${name} configure + INDEPENDENT FALSE COMMAND ${__cmdQuoted} WORKING_DIRECTORY \${binary_dir} DEPENDEES patch @@ -3032,6 +3209,7 @@ function(_ep_add_build_command name) endforeach() cmake_language(EVAL CODE " ExternalProject_Add_Step(${name} build + INDEPENDENT FALSE COMMAND ${__cmdQuoted} BYPRODUCTS \${build_byproducts} WORKING_DIRECTORY \${binary_dir} @@ -3075,6 +3253,7 @@ function(_ep_add_install_command name) endforeach() cmake_language(EVAL CODE " ExternalProject_Add_Step(${name} install + INDEPENDENT FALSE COMMAND ${__cmdQuoted} WORKING_DIRECTORY \${binary_dir} DEPENDEES build @@ -3142,6 +3321,7 @@ function(_ep_add_test_command name) endforeach() cmake_language(EVAL CODE " ExternalProject_Add_Step(${name} test + INDEPENDENT FALSE COMMAND ${__cmdQuoted} WORKING_DIRECTORY \${binary_dir} ${dependees_args} @@ -3159,6 +3339,9 @@ function(ExternalProject_Add name) cmake_policy(GET CMP0097 _EP_CMP0097 PARENT_SCOPE # undocumented, do not use outside of CMake ) + cmake_policy(GET CMP0114 cmp0114 + PARENT_SCOPE # undocumented, do not use outside of CMake + ) _ep_get_configuration_subdir_suffix(cfgdir) @@ -3166,14 +3349,23 @@ function(ExternalProject_Add name) set(cmf_dir ${CMAKE_CURRENT_BINARY_DIR}/CMakeFiles) _ep_get_complete_stampfile(${name} complete_stamp_file) + cmake_policy(PUSH) + if(cmp0114 STREQUAL "NEW") + # To implement CMP0114 NEW behavior with Makefile generators, + # we need CMP0113 NEW behavior. + cmake_policy(SET CMP0113 NEW) + endif() # The "ALL" option to add_custom_target just tells it to not set the # EXCLUDE_FROM_ALL target property. Later, if the EXCLUDE_FROM_ALL # argument was passed, we explicitly set it for the target. add_custom_target(${name} ALL DEPENDS ${complete_stamp_file}) + cmake_policy(POP) set_property(TARGET ${name} PROPERTY _EP_IS_EXTERNAL_PROJECT 1) set_property(TARGET ${name} PROPERTY LABELS ${name}) set_property(TARGET ${name} PROPERTY FOLDER "ExternalProjectTargets/${name}") + set_property(TARGET ${name} PROPERTY _EP_CMP0114 "${cmp0114}") + _ep_parse_arguments(ExternalProject_Add ${name} _EP_ "${ARGN}") _ep_set_directories(${name}) _ep_get_step_stampfile(${name} "done" done_stamp_file) |