| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
|
|
|
|
| |
Fix a regression from commit 073dd1bd81 (GoogleTest: Change format for
typed tests, 2022-02-07, v3.23.0-rc1~4^2) in the suite name detection.
Co-authored-by: Evgeniy Shcherbina <ixsci@pm.me>
Fixes: #24563
|
| |
|
|
|
|
|
|
|
|
|
|
| |
Due to add_command() being a macro it introduced excessive and
nonobvious escaping in different parts of the script. Because of
one of such places the resulting script would have an erroneous
${TEST_LIST} if the user data (in test parameters) had a semicolon.
To eliminate this non-obvious escaping, add_command() was converted
to function. Updated the escaping accordingly.
Fixes: #23059
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Before it would output a typed test as follows:
Suit/Type.Case
And now it would be:
Suit.Case<Type>
In case of NO_PRETTY_TYPES it would simply use the type number
instead of its text representation:
Suit.Case<0>
The change is introduced to make sure CTest outputs tests in a
similar fashion which is "*Suit.Case*" and angle brackets "<>"
emphasize that we are dealing with a typed (template) kind.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
When there were many cases (two digits or more) the "prettier" would
fail to recognize the pretty part leaving the test name unprocessed.
The fix made sure the processing would work correctly, irrespective
of the case number.
Before the fix, for the following input:
TypedSuite/1. # TypeParam = int
case
TypedSuite/10. # TypeParam = char
case
The output would be:
TypedSuite/int.case
TypedSuite/10. # TypeParam = char.case
Now the output will be:
TypedSuite/int.case
TypedSuite/char.case
|
| |
|
|
|
|
|
|
| |
Before the fix the gtest_discover_tests() function would strip the
user data in test parameters (everything to the right of GetParam())
of spaces. Now the parameters aren't altered in any way.
Fixes #23058
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
If a test name contains a square bracket (due to parameters) then it
breaks gtest_discovery_test() function in some not-so-predictable
way. That happens due to the special meaning these brackets have in
the CMake language and they can't be escaped universally.
So the following treatment has been implemented:
* Every occurrence of ('[' | ']') in a test name gets replaced with
the corresponding placeholder ("__osb_*" | "__csb_*") before the
Google Test output processing and gets replaced back before adding
a new test to CTest, keeping the original test name intact in the
CTest output.
The placeholders are chosen that way to minimize the chance of
clashing with something in the user tests but even if the default
ones would clash with something then they are enhanced to not clash
with anything (hence "_*" at the placeholder's end).
* The GTest output gets searched for the default test name guards
("[=[" | "]=]") and if they are found a new one gets generated until
the one is found which can safely encompass any test name. The
search is quite simple: find the least amount of '=' which would
allow escaping any test.
* The resulting ${TEST_LIST} variable will contain every test but
tests with square brackets as there is no way to make sure such
tests won't break the list altogether.
Fixes: #23039
|
| | |
|
| |
|
|
| |
Fixes: #22920
|
| |
|
|
|
|
|
|
|
| |
The `TEST_FILTER` argument can be used to filter tests during the
discovery phase. It combines `--gtest_filter=<expr>` with the
`--gtest_list_tests` argument when invoking the test excutable for
listing defined tests.
Fixes: #17493
|
| |
|
|
|
|
|
| |
Test names that start in `DISABLED` not followed by an underscore are
not disabled.
Fixes: #21543
|
| |
|
|
|
|
| |
Refactoring in commit 889a7146ff (GoogleTestAddTests: Refactor into
callable method, 2020-03-16, v3.18.0-rc1~450^2~3) accidentally
parsed `TEST_EXECUTOR` as a single-value argument instead of a list.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The Google test framework allows to write the results into an XML file
since commit e9ab39eb1d (GoogleTest: Add XML_OUTPUT_DIR parameter,
2020-03-06, v3.18.0-rc1~538^2~2). This file is passed on the command
line: `--gtest_output=xml:FILE_NAME`.
The module allows to specify a directory to save those files with
**TEST_XML_OUTPUT_PARAM**.
If the option is set, the filename will be set to
`${prefix}${pretty_suite}.${pretty_test}${suffix}.xml`.
The pretty names contain parameters for the tests, if
value-parameterized tests are used. These parameters may not be safe to
use in file names.
There are two possible options:
1. sanitize the file name
2. omit the values and use the internal numbering of gtest
This commit chose option 2. The testname needs to be a valid C++
identifier and should therefore be reasonable for a filename.
Note that the generated names contain slashes. This will lead to
subdirectories, but works on both Linux and Windows.
Fixes: #20877
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
The function gtest_discover_tests calls the passed test executable with
the parameter --gtest_list_tests and parses the output to find all
tests.
In case of value-parameterized tests ([1]), the test values are included
in the output. While test names are alphanumeric, the values can contain
arbitrary content.
First, the output is separated into lines with `foreach`. Included
semi-colons breaks this and need to get escaped.
Afterwards, the testname is passed on to the `add_command` helper. This
helper was converted into a macro in commit dac201442d (GoogleTest:
Optimize gtest_discover_tests, 2020-02-18). As a macro, its arguments
are re-evaluated. Therefore we need to escape `\`, `;` and to prevent
unwanted variable expansion `$`.
Fixes: #20661
[1] <https://github.com/google/googletest/blob/0eea2e9/googletest/docs/advanced.md#value-parameterized-tests>
|
| |
|
|
|
|
|
|
|
|
| |
Skipped tests are currently reported as successful. Using
SKIP_REGULAR_EXPRESSION on googletest's output prefix, skipped tests can
be detected and accounted accordingly.
Using SKIP_RETURN_CODE is not possible, googletests exit code is not
affected by skipped tests.
Fixes: #19669
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Move test discovery logic into new gtest_discover_tests_impl method
and make GoogleTestAddTests aware of whether it is being launched in
CMake's script mode.
When launched in script mode, gtest_discover_tests_impl is called
passing arguments obtained from the definitions passed into the call to cmake.
(i.e. cmake -P GoogleTestAddTests -D <arg1> -D <arg2> ...)
This preserves the existing behavior assumed by GoogleTest.cmake.
Unit tests are unchanged and still pass.
Looking ahead, it also allows GoogleTestAddTests to be included in generated files
and call gtest_discover_tests_impl to perform test discovery at test runtime
with the new PRE_TEST discovery mode introduced later in this branch.
My original approach attempted to call execute_process(cmake -P ...) in
the generated file, the same way POST_BUILD is doing, but I ran into
difficulties serializing the command arguments correctly.
By exposing a way to call gtest_discover_tests_impl directly from our generated file,
we remove a layer of shell quoting / parsing that our arguments have to survive,
which simplifies the act of producing a generated file that behaves the
same as its POST_BUILD counterpart.
|
| |
|
|
|
|
|
|
|
|
|
|
| |
When executing googltests in parallel using 'ctest -j n' and using
'--gtest_output=xml' there is a race condition upon file creation.
See googletest issue https://github.com/google/googletest/issues/2506.
As all testcases (potentially) can be run in parallel each testcase has
to create it's own XML JUnit file. EXTRA_ARGS is not suitable because it
is identical per testsuite.
So instead a new (opitional) parameter has been introduced to specify
the storage location for each test of the testsuite.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Fixes regression introduced dac201442d (GoogleTest: Optimize gtest_discover_tests, 2020-02-18).
The generated CTest include files has the form:
if(EXISTS "foo_tests.cmake")
include("foo_tests.cmake")
else()
add_test(foo_NOT_BUILT foo_test_NOT_BUILT)
endif()
Starting in dac201442d, an empty discovery_timeout_test[1]_tests.cmake was written
as soon as GoogleTestAddTests was processed.
This meant, that even if test discovery would fail (due to a crash or timeout in the executable),
we would always produce an empty CTest file.
So instead of reporting:
Unable to find executable: foo_NOT_BUILT
Errors while running CTest
We instead get:
No tests were found!!!
To fix the problem, we WRITE the file on the first call to flush_script,
thus creating the file once we know we have valid output
and the call to gtest_discover_tests hasn't failed.
After creating the file, we then set the mode to APPEND
and append to the file for every subsequent call.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Prior to this, `gtest_discover_tests` could take multiple minutes if
many tests are present. This behavior was caused by a repeated addition
to the variable `script` in the `add_command` function using:
set(script "${script}${NAME}(${_args})\n" PARENT_SCOPE)
This takes very long for large variables.
This commit flushes the contents of the variable to ${CTEST_FILE} after
a certain size of the variable is reached.
In addition:
- cmake_minimum_required(VERSION ${CMAKE_VERSION}) is set to allow usage
of new policies. In particular, CMP0053 speeds up variable expansion.
- No longer appends strings using set(), but instead uses string(APPEND).
- An additional buffer for the tests variable is set.
|
| | |
|
| |
|
|
|
|
|
|
|
|
| |
Add a TIMEOUT option to gtest_discover_tests. This provides a
work-around in case a test goes out to lunch, rather than causing the
build to simply hang. (Although this is still a problem with the user's
project, hanging the build is not cool, especially in the case of
automatically running CI builds. It is much preferred that the build
should actively fail in this case, and it is trivially easy for us to
implement that.)
|
| |
|
|
|
| |
Adjust the formatting of error messages produced when test discovery
goes wrong, in order to avoid spurious line breaking.
|
|
|
Add a new gtest_discover_tests function to GoogleTest.cmake,
implementing dynamic test discovery (i.e. tests are discovered by
actually running the test executable and asking for the list of
available tests, which is used to dynamically declare the tests) rather
than the source-parsing approach used by gtest_add_tests. Compared to
the source-parsing approach, this has the advantage of being robust
against users declaring tests in unusual ways, and much better support
for advanced features such as parameterized tests.
A unit test, modeled after the TEST_INCLUDE_DIR[S] test, is also
included. Note that the unit test does not actually require that Google
Test is available. The new functionality does not actually depend on
Google Test as such; it only requires that the test executable lists
tests in the expected format when invoked with --gtest_list_tests, which
the unit test can fake readily.
|
