summaryrefslogtreecommitdiffstats
path: root/testing/framework/TestCmd.py
Commit message (Collapse)AuthorAgeFilesLines
* Use f-strings in framework and framework testsMats Wichmann2022-11-241-31/+31
| | | | | | | | | This is a mostly tool-based conversion (a couple added by hand), and other changes were not made, to try to keep the diff manageable. Adds a GitHub Action to run framework tests if framework changes. Signed-off-by: Mats Wichmann <mats@linux.com>
* [framewqork] use subprocess timeoutsMats Wichmann2022-08-061-36/+62
| | | | | | | | | | | | Since Python 3.3, the subprocess module has its own timeout implementation, so remove the test framework's custom one. Required a little rejigger since the subprocess timeout is done on the communicate() call, not set up before the test is started. Noted that the framework intends to support two levels: one for the testing class instance, and one for an individual start call, which should override the one in the instance. Signed-off-by: Mats Wichmann <mats@linux.com>
* framework psutil usage: let the kill proceedMats Wichmann2022-08-041-2/+2
| | | | | | | Instead of skipping the whole kill/check loop if psutil module is not found, instead just skip the check part - should be okay to issue the kill. Signed-off-by: Mats Wichmann <mats@linux.com>
* Let test framework still run if no psutilMats Wichmann2022-08-041-28/+37
| | | | | | | | | | | Some odd environments - the one I'm running into it is inside an msys2 POSIX shell on Windows, using msys Python - don't have a psutil module to install. Let the test framework continue to work even if this is the case. fixes #4199 Signed-off-by: Mats Wichmann <mats@linux.com>
* Test framework improvementsMats Wichmann2022-08-031-1/+1
| | | | | | | | | Replaces one custom function with one from Python stdlib (textwrap.dedent) Fixes one test case that did not check for None Changes "fill line with a character" implementations to use Python string formatting, which has this capability. Signed-off-by: Mats Wichmann <mats@linux.com>
* fix issue when deleting cleaning up deamonDaniel Moody2022-06-141-1/+1
|
* Fixed some typos. Added skip_test() if psutil is not present for new test. ↵William Deegan2022-06-141-0/+6
| | | | Added note to CHANGES.txt/RELEASE.txt that psutil is required for the new test for this function
* Merge branch 'master' into ninja_exit_daemonWilliam Deegan2022-06-141-0/+6
|\
| * Added test for test.run()'s arguments set to a dict which will expand into ↵William Deegan2022-06-131-0/+6
| | | | | | | | KEY=VAL for each item in that dict. Also changed to unittest.main() form TestSuite()
* | install psutil for testing and fix sider complaintsDaniel Moody2022-06-071-1/+1
| |
* | Added new alias 'shutdown-ninja-scons-daemon' to allow ninja to shutdown the ↵Daniel Moody2022-06-071-0/+32
|/ | | | daemon
* test framework: fix exception on timeoutMats Wichmann2022-04-051-5/+9
| | | | | | | | | | If the framework wait_for() method actually times out, it tries to return stdout and stderr by calling the framework methods of those names. The stdout() method was protected against the message not having been captured (as is the case on timeout). Updated the stderr() method to use the same technique. Signed-off-by: Mats Wichmann <mats@linux.com>
* Fix tests to not hang on Windows with bad .py assocMats Wichmann2021-11-261-3/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | For systems where the association for .py files is not to an actual Python interpreter, those few cases where we need to run a Python script directly as a program don't work. This could be because the association was never set up, or because some other program (e.g. Visual Studio Code) has taken it over. In some cases may appear to "hang" because the alternate program is waiting for user interaction runtest.py now has a mechanism to check (thanks to Brett Cannon for providing this incantation). It isn't super precise (looks for the substring "py" in the queried association), but should work out. It sets an environment variable which the test framework can read and as a result set a flag which individual tests can read. Two tests in scons-time which had previously been set to skip-if-win32 now look at this flag instead. Three tests in sconsign now also look at this flag. This allows a clean run on my dev box with VS Code having taken over the .py association. Various things can break if the environment used to fire off Windows processes doesn't contain %UserProfile%. Added this to the short list of passthrough env vars. Apparently an environment without this value is now considered invalid (it blew up the erroneously launched VS Code, but we've apparently been lucky it hasn't blown up more things - believe there was also a report of a problem with the Visual Studio setup scripts). A little extra cleanup: - a couple of Py2-isms were cleaned out (Script/Main.py and in the test framework) - The paths to look for site-scons were rewritten (part of this was another Py2-ism), and the system path changed a bit - the old path is still checked, and the manpage updated to reflect this. - runtest.py dropped the unused whereis functions. - the three sconsign tests now use f-string formatting, mostly as an experiment to see how easy it is to convert. Fixes #4053 Signed-off-by: Mats Wichmann <mats@linux.com>
* Fix ninja tool rules for macos/ar for static libs to skip response files ↵William Deegan2021-04-131-0/+1
| | | | for now. Also fix build_libraries to have proper shlib suffix
* Test harness add a from_fw to skip callsMats Wichmann2021-04-131-27/+54
| | | | | | | | | | | | | | | | | | Main functional change is a new kwarg to skip_test() to allow calls from inside the framework to skip an additional line of traceback in the skip output - i.e. don't just skip the entry for skip_test, but also the function in the fw that called it. Other functional change is for the try block in skip_if_not_msvc() (which is one of the internal callers of skip_test() mentioned for the other change) to catch Exception, thus avoiding system-existing exceptions that were caught by the existing bare except, which caused the skip to not actually skip. The remainder of the patch is docstring reformatting, some minor code reformats, top-of-file license blocks, etc. Signed-off-by: Mats Wichmann <mats@linux.com>
* One more "drop py2"ism from test frameworkMats Wichmann2020-10-131-3/+3
| | | | Signed-off-by: Mats Wichmann <mats@linux.com>
* Run autoflake on codeMats Wichmann2020-09-211-1/+0
| | | | | | | | | | Eliminate unneeded imports, and a few unneeded statements - usually "pass" where it is not syntactically needed. A couple of import try blocks were eliminated or changed when they're "cannot happen" due to current floor Python version. Signed-off-by: Mats Wichmann <mats@linux.com>
* Fix testing subdir usageMats Wichmann2020-06-191-51/+55
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Various tests called subdir several times with the same directory, which is pointless. In previous code, this emitted a message though it did not fail. Stop doing that. The dir_fixture() method did some convoluted things to make sure the directories to write to exist, change this around to simplify. The subdir() method already combines the testdir, so dir_fixture doesn't need to. Also handle more cleanly the case of the target directory being an absolute path, which seems to have been intended, but would not work. Also clean up file_fixture() along the same principles. Change the subdir() method to not need to be given an ordered list of directories; instead call os.makedirs on each so intermediate steps are made automatically. Along the way, the print about making a directory that already existed vanishes. Tests which did os.path.join on directory pieces when calling file_fixture no longer do so, since file_fixture (and dir_fixture) do joining on an arg which is a list of paths like many other scons functions. The testing doc was updated to fix some wording and reflect the above changes. Signed-off-by: Mats Wichmann <mats@linux.com>
* Cleanups in tests and in frameworkMats Wichmann2020-06-141-186/+178
| | | | | | | | | | | | * Some Py2/Py3 compat blocks were simplified to Py3 only * some sequences of define-cleaup-function + atexit.register were changed to use the decorator (this works if func needs to take no arguments) * Lightly update a bunch of docstrings in TestCmd, and reformat a few places, towards the style we're generally using now. * call_python() in TestCmdTests was modernized to use subprocess.run Signed-off-by: Mats Wichmann <mats@linux.com>
* classes no longer explicitly inherit from objectMats Wichmann2020-05-241-2/+2
| | | | | | In Python3 this is the default. Signed-off-by: Mats Wichmann <mats@linux.com>
* [PR #3571] fix error and improve docsMats Wichmann2020-02-271-1/+1
| | | | | | | | | | | | sider spotted a cut-n-paste error introduced in the PR (srcdir set instead of srcfile) framework doc now uses directory consistently (over folder, which is a desktop concept rather than a filesystem concept) tweaked wording a bit more Signed-off-by: Mats Wichmann <mats@linux.com>
* Fix some test fixture-handling nits.Mats Wichmann2020-02-261-58/+53
| | | | | | | | | | | | | | | | | | | | | | | | dir-fixture() did not concatenate a source list into a path as the testing doc says it should. fix-fixture() did not either, and the doc didn't say so; now it does (docstring and external testing doc). There was a way to slip through both dir_fixtures and file_fixture with the path invalid - if you have fixture dirs defined, and the dir/file is not found in them, you'll come out of the loop set to the last fixture dir and not try the current dir. This doesn't break anything, just leads to a somewhat misleading message if the fixture really isn't found - the traceback indicates the fixture was not found in whatever the last passed-in fixture dir was. Now it looks like it was not found in the source testing dir. The message can still be improved to be more descriptive. A couple of minor Py2 removals. Testing doc didn't mention FIXTURE_DIRS, so this was added. A bunch of other doc fiddling. Signed-off-by: Mats Wichmann <mats@linux.com>
* rm-py2: Remove "from __future__" from more placesMats Wichmann2020-02-201-20/+23
| | | | | | | | | | | | | | | A couple of minor reformats along the way, most prominently, in tests, if being edited anyway, make sure the docstring most tests have is actually the docstring (sometimes the __revision__ line came before, which makes the string not be the docstring). Snuck in some minor framework changes that were orphaned when another draft PR was not needed: this almost all docstring changes, the functional part is using casefold instead of lower in a match func - a slightly better approach which is now possible that Py2 compatibility is not needed. Signed-off-by: Mats Wichmann <mats@linux.com>
* more unicode cleanup post py27William Deegan2020-02-181-10/+3
|
* [ci skip] Add logic to know when running in pypyWilliam Deegan2020-01-021-0/+1
|
* syntax fixups suggested by PyCharmMats Wichmann2019-12-231-1/+1
| | | | | | | | | | | | | | Drop unneeded parens. Drop trailing semicolons. Triple double-quote docstrings. Regexes drop unneeded escapes. Spaces around parens, braces: remove/add. Some one-tuples get their missing closing comma. A couple of sets use set init syntax {foo} instead of set([iter]) now. And a fiddle in Node to reduce lookup time on md5 signature functions (came about because of a line-too-long issue, initially) Signed-off-by: Mats Wichmann <mats@linux.com>
* checker fixes: None, trailing ws, list initMats Wichmann2019-12-211-1/+1
| | | | | | | | | | | checker-suggested fixes: Mostly, fix remaining instances of comparing none without "is" Some trailing whitespace on lines A couple of instances of list init followed immediately by several appends, turned into a single list init Some double comparisons turned into a single expression Signed-off-by: Mats Wichmann <mats@linux.com>
* Add test case with Latin-1 encoded Latex log file. Required fix in the test ↵maiphi2019-11-011-1/+1
| | | | | | | | | framework. In order to make the test work, it was necessary to handle the encoding issue also in the test framework. Otherwise, though the Latex builder can handle the case, the test framework chokes on it.
* test harness: fix TestCmd testsMats Wichmann2019-06-071-38/+106
| | | | | | | | | | | | | | | | | | | | | | This is the companion to PR #3382. The testing/harness/TestCmdTests.py unit tests do not currently pass. Note they are not run by the CI system, or in fact by doing runtests -a, since they're in a directory that is not searched. After these changes, there are no fails. This is a test-only change. The method simple_diff, modeled on difflib functions, is converted to a generator to match difflib, and now has a doctest as well. This means calls to it which aren't going to iterate needs to convert the return with list(), but that just makes usage more consistent, since the calls to difflib.context_diff and difflib.unified_diff already had to do so. Also, the methods that used Google-style docbook markup are changed to REsT-style markup. Our current doc producer, epydoc, does not understand the Google style, and we shouldn't mix styles; can convert them all in bulk later if we switch to Sphinx as the production tool. Signed-off-by: Mats Wichmann <mats@linux.com>
* PY38: Resolve duplicate creation of 'work' subdir which yielded warnings. ↵William Deegan2019-05-301-1/+1
| | | | Also improve exception handling message in test.subdir()
* Merge pull request #3359 from bdbaddog/fortran_issue_3135William Deegan2019-04-271-0/+6
|\ | | | | Fix Issue #3135 - Type Bound procedures in Fortran submodules
| * Remove debug logicWilliam Deegan2019-04-251-1/+0
| |
| * Add initializing fixture_dirs from shell variable FIXTURE_DIRSWilliam Deegan2019-04-251-0/+7
| |
* | [WIP] [PY 3.8] fix more warningsMats Wichmann2019-03-301-1/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Several locations with simple usage of deprecated "imp" module changed to use "importlib". These match with work in #3159, but this is not a complete implementation of #3159. More regex patterns are changed to be raw strings. Some strings which did not seem appropriate to change to raw strings (e.g. contain embedded tabs, which Python should honor) had backslashes escaped to avoid accidental Python interpretation. Example: '\t<Import Project="$(VCTargetsPath)\\Microsoft.Cpp.targets" />\n' Python 3.8 was Warning \M was an unknown escape. More open().write(), open().read() style usage changed to use context managers so the file is closed. WIP part: even with Python 3.7, the tests which call sconsign.py fail; oddly they do not fail without the patch to compat.py. sconsign.py does an import using imp module (which is what generates the errors) so needs to be updated anyway. It does not quite fit the "simple usage" pattern - can't do a simple relative import since sconsign is normally located elsewhere in the tree than the main scons code body. With this version of the patch, 700 tests now pass with 3.8, and Warning messages reduced to 2800 (current master has 200 pass, 9000 warns) Signed-off-by: Mats Wichmann <mats@linux.com>
* | [PR #333] close files to avoid scons-time racesMats Wichmann2019-03-291-7/+6
|/ | | | | | | | | | | | | | With runtest now honoring the -j 2 option given to it in CI setup on Windows, there were some problems where scons-time tests could try to remove a test directory while some files in it were still open (these locations were complained about by Python 3.8 also). Switch test framework to using mkdtemp also, and to not use tempfile.template (usage of that and mktemp are long deprecated) Signed-off-by: Mats Wichmann <mats@linux.com>
* Fix is/is not syntaxMats Wichmann2019-02-111-1/+1
| | | | | | | | In a few places, "is" and "is not" are used to compare with a string or integer literal. Python 3.8 flags these with a SyntaxWarning. Changed to == and != Signed-off-by: Mats Wichmann <mats@linux.com>
* Allow TestCmd.diff() to accept lists as well as stringsWilliam Deegan2018-12-061-1/+7
|
* For PR #3231, update some docstrings per reviewMats Wichmann2018-11-051-5/+12
| | | | Signed-off-by: Mats Wichmann <mats@linux.com>
* Fix some problems found if no MS compiler at allMats Wichmann2018-10-301-1/+4
| | | | | | | | A few tests blew up with exceptions (AttributeError, IndexError) if no compiler is installed on Windows - from where they are it could possibly happen on other platforms as well. Signed-off-by: Mats Wichmann <mats@linux.com>
* Fixes for Java tests on win32William Deegan2018-10-021-1/+2
|
* Fix py3 ResourceWarning in TestCmdMats Wichmann2018-07-241-2/+4
| | | | | | | | | | class TestCmd method read() uses a shortcut to return data from a file, "return open(...).read()". Python 3 warns this is a resource leak because the file has no chance to be closed with the open being in the return statement. Split into two lines and use a context manager (with statement). Signed-off-by: Mats Wichmann <mats@linux.com>
* pr-3052: moved some 64 bit logic to TestCmd and used TestCmd system checksDaniel Moody2018-05-251-1/+1
|
* move test files from QMTest to testing/framework. QMtest hasn't been used in ↵William Deegan2018-04-301-0/+1907
quite some time