From aa3fffa4bb5772fa83cf82189ecdd6a8c88e61b4 Mon Sep 17 00:00:00 2001 From: Brad King Date: Fri, 31 May 2024 14:49:26 -0400 Subject: README: Drop outdated link on running CMake --- README.rst | 7 ++----- 1 file changed, 2 insertions(+), 5 deletions(-) diff --git a/README.rst b/README.rst index e793d27..1ae77ef 100644 --- a/README.rst +++ b/README.rst @@ -51,11 +51,8 @@ Building CMake with CMake ------------------------- You can build CMake as any other project with a CMake-based build system: -run the installed CMake on the sources of this CMake with your preferred -options and generators. Then build it and install it. -For instructions how to do this, see documentation on `Running CMake`_. - -.. _`Running CMake`: https://cmake.org/runningcmake +run an already-installed CMake on this source tree with your preferred +generator and options. Then build it and install it. To build the documentation, install `Sphinx`_ and configure CMake with ``-DSPHINX_HTML=ON`` and/or ``-DSPHINX_MAN=ON`` to enable the "html" or -- cgit v0.12 From d9ab1fb5da2a49de7e1a69cbd0775a219019e6be Mon Sep 17 00:00:00 2001 From: Brad King Date: Fri, 31 May 2024 15:14:50 -0400 Subject: Help/dev: Fix formatting in CMake Documentation Guide --- Help/dev/documentation.rst | 18 +++++++++--------- 1 file changed, 9 insertions(+), 9 deletions(-) diff --git a/Help/dev/documentation.rst b/Help/dev/documentation.rst index 00413e1..8270bb1 100644 --- a/Help/dev/documentation.rst +++ b/Help/dev/documentation.rst @@ -310,17 +310,17 @@ are suppressed inside of square- or angle-brackets. This behavior can be controlled using the ``:break:`` option; note, however, that there is no way to *force* a line break. The default value is 'smart'. Allowable values are: - ``all`` - Allow line breaks at any whitespace. +``all`` + Allow line breaks at any whitespace. - ``smart`` (default) - Allow line breaks at whitespace, except between matched square- or - angle-brackets. For example, if a signature contains the text - ``... [OUTPUT_VARIABLE ]``, a line break would be allowed - after ``...`` but not between ``OUTPUT_VARIABLE`` and ````. +``smart`` (default) + Allow line breaks at whitespace, except between matched square- or + angle-brackets. For example, if a signature contains the text + ``... [OUTPUT_VARIABLE ]``, a line break would be allowed + after ``...`` but not between ``OUTPUT_VARIABLE`` and ````. - ``verbatim`` - Allow line breaks only where the source document contains a newline. +``verbatim`` + Allow line breaks only where the source document contains a newline. The directive treats its content as the documentation of the signature(s). Indent the signature documentation accordingly. -- cgit v0.12 From 7cc124d4b55aea0cd6f2439669f30c019d611d92 Mon Sep 17 00:00:00 2001 From: Brad King Date: Fri, 31 May 2024 14:33:22 -0400 Subject: Help/dev: Move integration testing to a more-specific document Avoid breaking existing links to `Help/dev/testing.rst` by keeping the document with minimal text to direct readers to the new one. --- CONTRIBUTING.rst | 4 ++-- Help/dev/README.rst | 5 +++-- Help/dev/integration-testing.rst | 43 ++++++++++++++++++++++++++++++++++++++++ Help/dev/review.rst | 4 ++-- Help/dev/testing.rst | 43 ++++------------------------------------ 5 files changed, 54 insertions(+), 45 deletions(-) create mode 100644 Help/dev/integration-testing.rst diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 81f26f5..302c38a 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -61,10 +61,10 @@ drive testing and submit results to the `CMake CDash Page`_. Anyone is welcome to provide testing machines in order to help keep support for their platforms working. -See documentation on `CMake Testing Process`_ for more information. +See documentation on `CMake Integration Testing`_ for more information. .. _`CMake CDash Page`: https://open.cdash.org/index.php?project=CMake -.. _`CMake Testing Process`: Help/dev/testing.rst +.. _`CMake Integration Testing`: Help/dev/integration-testing.rst License ======= diff --git a/Help/dev/README.rst b/Help/dev/README.rst index 9c3878b..abb4ac9 100644 --- a/Help/dev/README.rst +++ b/Help/dev/README.rst @@ -23,12 +23,13 @@ branches and tags. Upstream development processes are covered by the following documents: * The `CMake Review Process`_ manages integration of changes. -* The `CMake Testing Process`_ drives integration testing. +* The `CMake Integration Testing`_ infrastructure tests changes + before and after merging. .. _`Kitware's GitLab Instance`: https://gitlab.kitware.com .. _`CMake Repository`: https://gitlab.kitware.com/cmake/cmake .. _`CMake Review Process`: review.rst -.. _`CMake Testing Process`: testing.rst +.. _`CMake Integration Testing`: integration-testing.rst Developer Documentation ======================= diff --git a/Help/dev/integration-testing.rst b/Help/dev/integration-testing.rst new file mode 100644 index 0000000..1b666d6 --- /dev/null +++ b/Help/dev/integration-testing.rst @@ -0,0 +1,43 @@ +CMake Integration Testing +************************* + +The following documents how to run integration testing builds. +See documentation on `CMake Development`_ for more information. + +.. _`CMake Development`: README.rst + +CMake Dashboard Scripts +======================= + +The *integration testing* step of the `CMake Review Process`_ uses a set of +testing machines that follow an integration branch on their own schedule to +drive testing and submit results to the `CMake CDash Page`_. Anyone is +welcome to provide testing machines in order to help keep support for their +platforms working. + +The `CMake Dashboard Scripts Repository`_ provides CTest scripts to drive +nightly, continuous, and experimental testing of CMake. Use the following +commands to set up a new integration testing client: + +.. code-block:: console + + $ mkdir -p ~/Dashboards + $ cd ~/Dashboards + $ git clone https://gitlab.kitware.com/cmake/dashboard-scripts.git CMakeScripts + $ cd CMakeScripts + +The `cmake_common.cmake`_ script contains comments at the top with +instructions to set up a testing client. As it instructs, create a +CTest script with local settings and include ``cmake_common.cmake``. + +.. _`CMake Review Process`: review.rst +.. _`CMake CDash Page`: https://open.cdash.org/index.php?project=CMake +.. _`CMake Dashboard Scripts Repository`: https://gitlab.kitware.com/cmake/dashboard-scripts +.. _`cmake_common.cmake`: https://gitlab.kitware.com/cmake/dashboard-scripts/-/blob/master/cmake_common.cmake + +Nightly Start Time +------------------ + +The ``cmake_common.cmake`` script expects its includer to be run from a +nightly scheduled task (cron job). Schedule such tasks for sometime after +``1:00am UTC``, the time at which our nightly testing branches fast-forward. diff --git a/Help/dev/review.rst b/Help/dev/review.rst index 34796a1..40bf3e5 100644 --- a/Help/dev/review.rst +++ b/Help/dev/review.rst @@ -348,7 +348,7 @@ Integration Testing The above `topic testing`_ tests the MR topic independent of other merge requests and on only a few key platforms and configurations. -The `CMake Testing Process`_ also has a large number of machines +`CMake Integration Testing`_ also uses a large number of machines provided by Kitware and generous volunteers that cover nearly all supported platforms, generators, and configurations. In order to avoid overwhelming these resources, they do not test every MR @@ -403,7 +403,7 @@ until one of the following occurs: Once a MR has been removed from the topic stage a new ``Do: stage`` command is needed to stage it again. -.. _`CMake Testing Process`: testing.rst +.. _`CMake Integration Testing`: integration-testing.rst Resolve ======= diff --git a/Help/dev/testing.rst b/Help/dev/testing.rst index 279e4b2..2179f60 100644 --- a/Help/dev/testing.rst +++ b/Help/dev/testing.rst @@ -1,43 +1,8 @@ -CMake Testing Process -********************* +CMake Testing Guide +******************* -The following documents the process for running integration testing builds. +See `CMake Integration Testing`_ for running integration testing builds. See documentation on `CMake Development`_ for more information. +.. _`CMake Integration Testing`: integration-testing.rst .. _`CMake Development`: README.rst - -CMake Dashboard Scripts -======================= - -The *integration testing* step of the `CMake Review Process`_ uses a set of -testing machines that follow an integration branch on their own schedule to -drive testing and submit results to the `CMake CDash Page`_. Anyone is -welcome to provide testing machines in order to help keep support for their -platforms working. - -The `CMake Dashboard Scripts Repository`_ provides CTest scripts to drive -nightly, continuous, and experimental testing of CMake. Use the following -commands to set up a new integration testing client: - -.. code-block:: console - - $ mkdir -p ~/Dashboards - $ cd ~/Dashboards - $ git clone https://gitlab.kitware.com/cmake/dashboard-scripts.git CMakeScripts - $ cd CMakeScripts - -The `cmake_common.cmake`_ script contains comments at the top with -instructions to set up a testing client. As it instructs, create a -CTest script with local settings and include ``cmake_common.cmake``. - -.. _`CMake Review Process`: review.rst -.. _`CMake CDash Page`: https://open.cdash.org/index.php?project=CMake -.. _`CMake Dashboard Scripts Repository`: https://gitlab.kitware.com/cmake/dashboard-scripts -.. _`cmake_common.cmake`: https://gitlab.kitware.com/cmake/dashboard-scripts/-/blob/master/cmake_common.cmake - -Nightly Start Time ------------------- - -The ``cmake_common.cmake`` script expects its includer to be run from a -nightly scheduled task (cron job). Schedule such tasks for sometime after -``1:00am UTC``, the time at which our nightly testing branches fast-forward. -- cgit v0.12 From 8a688e556e0068f75f232576ccca7cec74e668a3 Mon Sep 17 00:00:00 2001 From: Brad King Date: Fri, 31 May 2024 14:54:59 -0400 Subject: Help/dev: Add a CMake Testing Guide Document how developers working on CMake itself may run the test suite locally. Fixes: #26001 --- CONTRIBUTING.rst | 4 +- Help/dev/README.rst | 2 + Help/dev/integration-testing.rst | 3 ++ Help/dev/source.rst | 2 + Help/dev/testing.rst | 107 ++++++++++++++++++++++++++++++++++++++- README.rst | 4 ++ Tests/README.rst | 3 +- Tests/RunCMake/README.rst | 4 +- 8 files changed, 124 insertions(+), 5 deletions(-) diff --git a/CONTRIBUTING.rst b/CONTRIBUTING.rst index 302c38a..88f39f1 100644 --- a/CONTRIBUTING.rst +++ b/CONTRIBUTING.rst @@ -26,7 +26,8 @@ To contribute patches: #. Fork the upstream `CMake Repository`_ into a personal account. #. Run `Utilities/SetupForDevelopment.sh`_ for local git configuration. #. See `Building CMake`_ for building CMake locally. -#. See the `CMake Source Code Guide`_ for coding guidelines. +#. See the `CMake Source Code Guide`_ for coding guidelines + and the `CMake Testing Guide`_ for testing instructions. #. Create a topic branch named suitably for your work. Base all new work on the upstream ``master`` branch. Base work on the upstream ``release`` branch only if it fixes a @@ -49,6 +50,7 @@ The merge request will enter the `CMake Review Process`_ for consideration. .. _`Utilities/SetupForDevelopment.sh`: Utilities/SetupForDevelopment.sh .. _`Building CMake`: README.rst#building-cmake .. _`CMake Source Code Guide`: Help/dev/source.rst +.. _`CMake Testing Guide`: Help/dev/testing.rst .. _`commit messages`: Help/dev/review.rst#commit-messages .. _`CMake Review Process`: Help/dev/review.rst diff --git a/Help/dev/README.rst b/Help/dev/README.rst index abb4ac9..2a2d32d 100644 --- a/Help/dev/README.rst +++ b/Help/dev/README.rst @@ -38,10 +38,12 @@ CMake developer documentation is provided by the following documents: * The `CMake Source Code Guide`_. * The `CMake Documentation Guide`_. +* The `CMake Testing Guide`_. * The `CMake Experimental Features Guide`_. .. _`CMake Source Code Guide`: source.rst .. _`CMake Documentation Guide`: documentation.rst +.. _`CMake Testing Guide`: testing.rst .. _`CMake Experimental Features Guide`: experimental.rst Maintainer Documentation diff --git a/Help/dev/integration-testing.rst b/Help/dev/integration-testing.rst index 1b666d6..434cf23 100644 --- a/Help/dev/integration-testing.rst +++ b/Help/dev/integration-testing.rst @@ -4,7 +4,10 @@ CMake Integration Testing The following documents how to run integration testing builds. See documentation on `CMake Development`_ for more information. +See the `CMake Testing Guide`_ for running the test suite locally. + .. _`CMake Development`: README.rst +.. _`CMake Testing Guide`: testing.rst CMake Dashboard Scripts ======================= diff --git a/Help/dev/source.rst b/Help/dev/source.rst index fa7d620..c3d7ac3 100644 --- a/Help/dev/source.rst +++ b/Help/dev/source.rst @@ -310,6 +310,7 @@ The CMake source tree is organized as follows. * ``Tests/``: The test suite. See `Tests/README.rst`_. + To run the tests, see the `CMake Testing Guide`_. * ``Utilities/``: Scripts, third-party source code. @@ -331,5 +332,6 @@ The CMake source tree is organized as follows. See `Utilities/Release/README.rst`_. .. _`CMake Documentation Guide`: documentation.rst +.. _`CMake Testing Guide`: testing.rst .. _`Tests/README.rst`: ../../Tests/README.rst .. _`Utilities/Release/README.rst`: ../../Utilities/Release/README.rst diff --git a/Help/dev/testing.rst b/Help/dev/testing.rst index 2179f60..f90cef7 100644 --- a/Help/dev/testing.rst +++ b/Help/dev/testing.rst @@ -1,8 +1,111 @@ CMake Testing Guide ******************* -See `CMake Integration Testing`_ for running integration testing builds. +The following is a guide to the CMake test suite for developers. See documentation on `CMake Development`_ for more information. -.. _`CMake Integration Testing`: integration-testing.rst +See `CMake Integration Testing`_ for running integration testing builds. + +See `Tests/README.rst`_ for the test suite layout in the source tree. + .. _`CMake Development`: README.rst +.. _`CMake Integration Testing`: integration-testing.rst +.. _`Tests/README.rst`: ../../Tests/README.rst + +Running Tests in the Build Tree +=============================== + +After `Building CMake`_, one may run the test suite in the build tree +using `ctest(1)`_: + +* With a single-configuration CMake generator, such as ``Ninja`` + or ``Unix Makefiles``, one may simply run ``ctest``: + + .. code-block:: console + + $ ctest + +* With a multi-configuration CMake generator, such as + ``Ninja Multi-Config``, ``Visual Studio``, or ``Xcode``, + one must tell ``ctest`` which configuration to test + by passing the ``-C `` option: + + .. code-block:: console + + $ ctest -C Debug + +Some useful `ctest(1)`_ options include: + +``-N`` + List test names without running them. + +``-V`` + Show verbose output from each test. + +``-j `` + Run to run up to ``N`` tests concurrently. + +``-R `` + Select tests for which the regular expression matches a substring + of their name. + +Cleaning Test Build Trees +------------------------- + +Many CMake tests create their own test project build trees underneath +the ``Tests/`` directory at the top of the CMake build tree. These +build trees are left behind after testing completes in order to +facilitate manual investigation of results. Many of the tests do *not* +clean their build trees if they are run again, with the exception of +tests using the `RunCMake`_ infrastructure. + +In order to clear test build trees, drive the ``test_clean`` custom target +in the CMake build tree: + +.. code-block:: console + + $ cmake --build . --target test_clean + +This removes the ``Tests/`` subdirectories created by individual tests +so they will use a fresh directory next time they run. + +.. _`Building CMake`: ../../README.rst#building-cmake +.. _`ctest(1)`: https://cmake.org/cmake/help/latest/manual/ctest.1.html +.. _`RunCMake`: ../../Tests/RunCMake/README.rst + +Running Tests with a Different Generator +======================================== + +After `Building CMake`_ with one CMake generator, one may configure the +test suite using a different generator in a separate build tree, without +building CMake itself again, by defining ``CMake_TEST_EXTERNAL_CMAKE`` +to be the absolute path to the ``bin`` directory containing the ``cmake``, +``ctest``, and ``cpack`` executables. + +For example, after building CMake with the ``Ninja`` generator: + +.. code-block:: console + + $ cmake -B build-ninja -G Ninja -DCMAKE_BUILD_TYPE=Debug + $ cmake --build build-ninja + +one may configure a second build tree to drive tests with the +``Ninja Multi-Config`` generator: + +.. code-block:: console + + $ cmake -B build-nmc-tests -G "Ninja Multi-Config" \ + -DCMake_TEST_EXTERNAL_CMAKE="$PWD/build-ninja/bin" + $ cmake --build build-nmc-tests --config Release + +The second build tree does not build CMake itself, but does configure +the test suite and build test binaries. One may then run tests normally: + +.. code-block:: console + + $ cd build-nmc-tests + $ ctest -C Release + +Note that the configuration with which one drives tests in the second +build tree is independent of the configuration with which CMake was +built in the first. diff --git a/README.rst b/README.rst index 1ae77ef..8a471db 100644 --- a/README.rst +++ b/README.rst @@ -59,7 +59,11 @@ To build the documentation, install `Sphinx`_ and configure CMake with "man" builder. Add ``-DSPHINX_EXECUTABLE=/path/to/sphinx-build`` if the tool is not found automatically. +To run the test suite, run ``ctest`` in the CMake build directory after +building. See the `CMake Testing Guide`_ for details. + .. _`Sphinx`: https://sphinx-doc.org +.. _`CMake Testing Guide`: Help/dev/testing.rst Building CMake from Scratch --------------------------- diff --git a/Tests/README.rst b/Tests/README.rst index 2810fec..4bbbce1 100644 --- a/Tests/README.rst +++ b/Tests/README.rst @@ -2,8 +2,9 @@ CMake Tests Directory ********************* This directory contains the CMake test suite. -See also the `CMake Source Code Guide`_. +See also the `CMake Testing Guide`_ and the `CMake Source Code Guide`_. +.. _`CMake Testing Guide`: ../Help/dev/testing.rst .. _`CMake Source Code Guide`: ../Help/dev/source.rst Many tests exist as immediate subdirectories, but some tests diff --git a/Tests/RunCMake/README.rst b/Tests/RunCMake/README.rst index ea6db54..f4bcc57 100644 --- a/Tests/RunCMake/README.rst +++ b/Tests/RunCMake/README.rst @@ -6,9 +6,11 @@ precisely checking their return code and stdout/stderr content. The RunCMake infrastructure is useful for testing error cases and diagnostic output. -See also `../README.rst`_ and the `CMake Source Code Guide`_. +See also `../README.rst`_, the `CMake Testing Guide`_, +and the `CMake Source Code Guide`_. .. _`../README.rst`: ../README.rst +.. _`CMake Testing Guide`: ../../Help/dev/testing.rst .. _`CMake Source Code Guide`: ../../Help/dev/source.rst .. _`CMakeLists.txt`: CMakeLists.txt -- cgit v0.12