summaryrefslogtreecommitdiffstats
path: root/Tests/RunCMake/GoogleTest/GoogleTest.cmake
Commit message (Collapse)AuthorAgeFilesLines
* GoogleTest: Reports skipped test with gtest_add_testsDavidPerretSN2022-11-071-0/+4
| | | | | | Skipped GTests were reported as success when tests where added with gtest_add_tests. This fixes this behaviour. Fixes: #24130
* GoogleTest: Add TEST_FILTER arg to gtest_discover_testsAshish Sadanandan2021-08-051-0/+18
| | | | | | | | | The `TEST_FILTER` argument can be used to filter tests during the discovery phase. It combines `--gtest_filter=<expr>` with the `--gtest_list_tests` argument when invoking the test excutable for listing defined tests. Fixes: #17493
* Tests: Suppress failures on macOS arm64 due to separate Xcode signing phaseBrad King2021-02-221-0/+6
| | | | | | | | | | | | | | | | Some tests fail because Xcode runs `POST_BUILD` commands before signing the binaries they run. Tell the linker to perform ad-hoc codesign even though Xcode normally tells it not to. Other tests fail because `install_name_tool` does not revise ad-hoc signatures without the codesign `linker-signed` flag. Add that flag ourselves where needed by our tests. For now these changes help our test suite pass so we can use it to cover everything else. Both of these cases may need further investigation to update CMake to help projects in general. Issue: #21845, #21854
* Tests: Clarify internal project name in RunCMake.GoogleTest casesBrad King2021-02-221-1/+1
|
* Tests: Don't ask for things not required for GoogleTestCraig Scott2020-06-241-2/+1
| | | | The test cases only need C or C++, but not both. The CTest module is also not needed because we are not running a dashboard script.
* GoogleTest: Add testcase for skipped testsAlexander Stein2020-04-101-0/+6
| | | | | This simply runs a mocked test case which uses the prefix for signaling a skipped test. CTest's output is checked for a skipped test result.
* GoogleTest: Parameterize tests to check PRE_TEST/POST_BUILD discovery modeRyan Thornton2020-03-271-8/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Now, the unit tests are ran twice -- once with POST_BUILD (i.e. default mode) and again with PRE_TEST (i.e. new discovery mode). Both modes of setting gtest discovery mode are also tested: 1. Using the global override (i.e. CMAKE_GTEST_DISCOVER_TESTS_DISCOVERY_MODE) 2. Explicitly passing DISCOVERY_MODE in calls to gtest_discover_tests (in GoogleTestDiscoveryTimeout.cmake) The goal is to show that the new PRE_TEST discovery mode does not break existing behavior (i.e. should not break POST_BUILD mode) and should also pass the same tests in the same way. A few non trivial implementation details worth noting: 1. Refactoring discovery_timeout_test into own project Originally, I tried doing: ``` run_GoogleTest(POST_BUILD) run_GoogleTest(PRE_TEST) ``` Without changing the internal structure of run_GoogleTest. But since discovery_timeout_test is part of the same project as the other tests, and CTest include files always get evaluated and that's where test discovery occurs, this means every other test now notices the timeout problem when running in PRE_TEST mode. As a result, keeping the existing test structure meant that each existing test (and any new test) would need to have its own PRE_TEST / POST_BUILD variant for stderr and stdout in order to handle the case where discovery_timeout_test timed out. This exponential increase in test output files introduced unnecessary complexity and made it more cumbersome to work on test cases. Why should an unrelated test case care about discovery_timeout_test? So, to fix that issue, the tests were broken apart into two main groups: 1. run_GoogleTest_discovery_timeout (the test dealing with discovery_timeout_test) 2. run_GoogleTest (everything else) This isolates the PRE_TEST / POST_BUILD timeout variants to a single test case. And the other test cases remain unchanged -- further driving home the point that DISCOVERY_MODE shouldn't change existing behavior. 2. Different number of PRE_TEST / POST_BUILD file variants On the PRE_TEST path, different build systems / compilers (i.e. MSBuild and ninja/gcc) produces different build output when building discovery_timeout_test, but we don't actually care what it is, just as long as it builds successfully. This the fundamental difference in behavior between POST_BUILD (which would have failed) and PRE_TEST (which doesn't) and is the reason why we don't need a GoogleTest-discovery-build-result.txt or GoogleTest-discovery-build-stdout.txt 3. Fix flaky discovery timeout test The test expects to see: > Output: > timeout > case. But sometimes, the test would only produce: > Output: > timout In certain environments, specifically when built with OpenWatcom 1.4, and while the build server was under heavy load (i.e. running many tests in parallel), std::endl behaves inconsistently and doesn't completely flush std::cout when the program is terminated due to timeout. This results in inconsistent test failures because the actual output doesn't fully match what's expected. At first we tried adding an additional: std::cout << std::flush That didn't work. But using C-style printf() and fflush() appears to do the trick: > This time I managed to get on the machine while it was still busy doing other nightly builds > and could reproduce the problem reliably. With that I was finally able to find a fix. > It turns out my earlier hypothesis that C++ stream flushing was not working on the old compiler was correct, > but even .flush() is not enough. > I changed it to use C-style printf() and fflush() and now the test passes on that build. > -- Brad King <brad.king@kitware.com> Co-authored-by: Ryan Thornton <ThorntonRyan@JohnDeere.com> Co-authored-by: Kevin Puetz <PuetzKevinA@JohnDeere.com>
* GoogleTest: Rename TIMEOUT parameter to avoid clashCraig Scott2018-03-151-2/+32
| | | | | | | | | | | | | | | In gtest_discover_tests(), the TIMEOUT keyword was making it impossible to set the TIMEOUT test property via the PROPERTIES keyword. This would be a frequent case, but it doesn't complain and instead silently does something different to what would normally be expected. The TIMEOUT keyword has been renamed to DISCOVERY_TIMEOUT, thereby removing the clash. This is a breaking change. 3.10.1 and 3.10.2 were the only versions that supported the TIMEOUT keyword and uses of it were likely not working as intended. Fixes: #17801
* GoogleTest: Add timeout to discoveryMatthew Woehlke2017-12-061-0/+6
| | | | | | | | | | Add a TIMEOUT option to gtest_discover_tests. This provides a work-around in case a test goes out to lunch, rather than causing the build to simply hang. (Although this is still a problem with the user's project, hanging the build is not cool, especially in the case of automatically running CI builds. It is much preferred that the build should actively fail in this case, and it is trivially easy for us to implement that.)
* GoogleTest: Fix multiple discovery on same targetMatthew Woehlke2017-11-211-1/+9
| | | | | | | | | | | | | According to the documentation, tests can be discovered for a target multiple times by using a different prefix and/or suffix to ensure name uniqueness. However, while this worked for gtest_add_tests, it did not work with gtest_discover_tests because the generated file that sets up the tests was named based only on the target name, and so subsequent discovery from the same target would clobber earlier discovery. Fix this by introducing a counter that records how many times discovery has been used on a target, and use this to generate unique names of the generated test list files.
* Add dynamic test discovery for for Google TestMatthew Woehlke2017-07-271-0/+15
Add a new gtest_discover_tests function to GoogleTest.cmake, implementing dynamic test discovery (i.e. tests are discovered by actually running the test executable and asking for the list of available tests, which is used to dynamically declare the tests) rather than the source-parsing approach used by gtest_add_tests. Compared to the source-parsing approach, this has the advantage of being robust against users declaring tests in unusual ways, and much better support for advanced features such as parameterized tests. A unit test, modeled after the TEST_INCLUDE_DIR[S] test, is also included. Note that the unit test does not actually require that Google Test is available. The new functionality does not actually depend on Google Test as such; it only requires that the test executable lists tests in the expected format when invoked with --gtest_list_tests, which the unit test can fake readily.