From 7b4983536a634a7888466088a52319d7efe7a5c3 Mon Sep 17 00:00:00 2001 From: scivision Date: Wed, 1 Nov 2023 11:47:34 -0400 Subject: Help:test prop WILL_FAIL clarify and example This makes the WILL_FAIL behavior explicit. Users could be confused because some other build systems treat equivalent test properties as allowing a failure, but still pass on a passing test, and may catch SIGABRT unlike CMake. We show an example of how to wrap a test so that sigabrt, segfault, etc. are also handled by WILL_FAIL. --- Help/prop_test/WILL_FAIL.rst | 35 +++++++++++++++++++++++++++++++---- 1 file changed, 31 insertions(+), 4 deletions(-) diff --git a/Help/prop_test/WILL_FAIL.rst b/Help/prop_test/WILL_FAIL.rst index 4926f40..006b1c3 100644 --- a/Help/prop_test/WILL_FAIL.rst +++ b/Help/prop_test/WILL_FAIL.rst @@ -1,8 +1,35 @@ WILL_FAIL --------- -If set to true, this will invert the pass/fail flag of the test. +If ``true``, inverts the pass / fail test criteria. Tests for which +``WILL_FAIL`` is ``true`` fail with return code 0 and pass with non-zero +return code. Tests that exceed the timeout specified by :prop_test:`TIMEOUT` +still fail regardless of ``WILL_FAIL``. -This property can be used for tests that are expected to fail and return a -non-zero return code. Note that system-level test failures such as segmentation -faults or heap errors will still fail the test even if ``WILL_FALL`` is true. +Caveat: system-level test failures including segmentation faults, +signal abort, or heap errors may fail the test even if ``WILL_FAIL`` is true. + +Example of a test that would ordinarily pass, but fails because ``WILL_FAIL`` +is ``true``: + +.. code-block:: cmake + + add_test(NAME failed COMMAND ${CMAKE_COMMAND} -E true) + set_property(TEST failed PROPERTY WILL_FAIL true) + +To run a test that may have a system-level failure, but still pass if +``WILL_FAIL`` is set, use a CMake command to wrap the executable run like: + +.. code-block:: cmake + + add_executable(main main.c) + + add_test(NAME sigabrt COMMAND ${CMAKE_COMMAND} -E env $) + + set_property(TEST sigabrt PROPERTY WILL_FAIL TRUE) + +.. code-block:: c + + #include + + int main(void){ raise(SIGABRT); return 0; } -- cgit v0.12