summaryrefslogtreecommitdiffstats
path: root/Help/prop_test
diff options
context:
space:
mode:
authorscivision <scivision@users.noreply.github.com>2023-11-01 15:47:34 (GMT)
committerscivision <scivision@users.noreply.github.com>2023-11-07 02:46:17 (GMT)
commit7b4983536a634a7888466088a52319d7efe7a5c3 (patch)
tree7e09cb9853c305fb9127c7497229a14ce9f319ae /Help/prop_test
parent51d985da3e12f06ff9349970093d38d497ed3da0 (diff)
downloadCMake-7b4983536a634a7888466088a52319d7efe7a5c3.zip
CMake-7b4983536a634a7888466088a52319d7efe7a5c3.tar.gz
CMake-7b4983536a634a7888466088a52319d7efe7a5c3.tar.bz2
Help:test prop WILL_FAIL clarify and example
This makes the WILL_FAIL behavior explicit. Users could be confused because some other build systems treat equivalent test properties as allowing a failure, but still pass on a passing test, and may catch SIGABRT unlike CMake. We show an example of how to wrap a test so that sigabrt, segfault, etc. are also handled by WILL_FAIL.
Diffstat (limited to 'Help/prop_test')
-rw-r--r--Help/prop_test/WILL_FAIL.rst35
1 files changed, 31 insertions, 4 deletions
diff --git a/Help/prop_test/WILL_FAIL.rst b/Help/prop_test/WILL_FAIL.rst
index 4926f40..006b1c3 100644
--- a/Help/prop_test/WILL_FAIL.rst
+++ b/Help/prop_test/WILL_FAIL.rst
@@ -1,8 +1,35 @@
WILL_FAIL
---------
-If set to true, this will invert the pass/fail flag of the test.
+If ``true``, inverts the pass / fail test criteria. Tests for which
+``WILL_FAIL`` is ``true`` fail with return code 0 and pass with non-zero
+return code. Tests that exceed the timeout specified by :prop_test:`TIMEOUT`
+still fail regardless of ``WILL_FAIL``.
-This property can be used for tests that are expected to fail and return a
-non-zero return code. Note that system-level test failures such as segmentation
-faults or heap errors will still fail the test even if ``WILL_FALL`` is true.
+Caveat: system-level test failures including segmentation faults,
+signal abort, or heap errors may fail the test even if ``WILL_FAIL`` is true.
+
+Example of a test that would ordinarily pass, but fails because ``WILL_FAIL``
+is ``true``:
+
+.. code-block:: cmake
+
+ add_test(NAME failed COMMAND ${CMAKE_COMMAND} -E true)
+ set_property(TEST failed PROPERTY WILL_FAIL true)
+
+To run a test that may have a system-level failure, but still pass if
+``WILL_FAIL`` is set, use a CMake command to wrap the executable run like:
+
+.. code-block:: cmake
+
+ add_executable(main main.c)
+
+ add_test(NAME sigabrt COMMAND ${CMAKE_COMMAND} -E env $<TARGET_FILE:main>)
+
+ set_property(TEST sigabrt PROPERTY WILL_FAIL TRUE)
+
+.. code-block:: c
+
+ #include <signal.h>
+
+ int main(void){ raise(SIGABRT); return 0; }