summaryrefslogtreecommitdiffstats
path: root/Lib/test/libregrtest/cmdline.py
Commit message (Collapse)AuthorAgeFilesLines
* [3.12] gh-108834: Sync libregrtest with the main branch (#108966)Victor Stinner2023-09-081-4/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * gh-108834: regrtest reruns failed tests in subprocesses (#108839) When using --rerun option, regrtest now re-runs failed tests in verbose mode in fresh worker processes to have more deterministic behavior. So it can write its final report even if a test killed a worker progress. Add --fail-rerun option to regrtest: exit with non-zero exit code if a test failed pass passed when re-run in verbose mode (in a fresh process). That's now more useful since tests can pass when re-run in a fresh worker progress, whereas they failed when run after other tests when tests are run sequentially. Rename --verbose2 option (-w) to --rerun. Keep --verbose2 as a deprecated alias. Changes: * Fix and enhance statistics in regrtest summary. Add "(filtered)" when --match and/or --ignore options are used. * Add RunTests class. * Add TestResult.get_rerun_match_tests() method * Rewrite code to serialize/deserialize worker arguments as JSON using a new WorkerJob class. * Fix stats when a test is run with --forever --rerun. * If failed test names cannot be parsed, log a warning and don't filter tests. * test_regrtest.test_rerun_success() now uses a marker file, since the test is re-run in a separated process. * Add tests on normalize_test_name() function. * Add test_success() and test_skip() tests to test_regrtest. (cherry picked from commit 31c2945f143c6b80c837fcf09a5cfb85fea9ea4c) * gh-108834: regrtest --fail-rerun exits with code 5 (#108896) When the --fail-rerun option is used and a test fails and then pass, regrtest now uses exit code 5 ("rerun) instead of 2 ("bad test"). (cherry picked from commit 1170d5a292b46f754cd29c245a040f1602f70301) * gh-108416: Mark slow but not CPU bound test methods with requires_resource('walltime') (GH-108480) (cherry picked from commit 1e0d62793a84001e92f1c80b511d3a212b435acc) * Manually sync Lib/test/libregrtest/ from main --------- Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* gh-94052: Don't re-run failed tests with --python option (#94054)Christian Heimes2022-06-211-2/+6
|
* gh-84461: Add ability for multiprocessed libregrtest to use a different ↵Ethan Smith2022-05-021-0/+4
| | | | Python executable (GH-91930)
* bpo-45745: Remove regrtest --findleaks options (GH-29514)Victor Stinner2021-11-121-7/+0
| | | | | Remove the --findleaks command line option of regrtest: use the --fail-env-changed option instead. Since Python 3.7, it was a deprecated alias to the --fail-env-changed option.
* bpo-45410: regrtest -W leaves stdout/err FD unchanged (GH-28915)Victor Stinner2021-10-131-1/+0
| | | | | | | | | | | | support.print_warning() now stores the original value of sys.__stderr__ and uses it to log warnings. libregrtest uses the same stream to log unraisable exceptions and uncaught threading exceptions. Partially revert commit dbe213de7ef28712bbfdb9d94a33abb9c33ef0c2: libregrtest no longer replaces sys.__stdout__, sys.__stderr__, and stdout and stderr file descriptors. Remove also a few unused imports in libregrtest.
* bpo-44708: Only re-run test methods that match names of previously failing ↵Łukasz Langa2021-07-221-7/+34
| | | | | | | | | | | test methods (GH-27287) * Move to a static argparse.Namespace subclass * Roughly annotate runtest.py * Refactor libregrtest to use lossless test result objects * Only re-run test methods that match names of previously failing test methods * Adopt tests to cover test method name matching Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* bpo-40275: Use new test.support helper submodules in tests (GH-21219)Hai Shi2020-06-301-1/+2
|
* bpo-37957: Allow regrtest to receive a file with test (and subtests) to ↵Pablo Galindo2019-11-191-1/+15
| | | | | | | ignore (GH-16989) When building Python in some uncommon platforms there are some known tests that will fail. Right now, the test suite has the ability to ignore entire tests using the -x option and to receive a filter file using the --matchfile filter. The problem with the --matchfile option is that it receives a file with patterns to accept and when you want to ignore a couple of tests and subtests, is too cumbersome to lists ALL tests that are not the ones that you want to accept and he problem with -x is that is not easy to ignore just a subtests that fail and the whole test needs to be ignored. For these reasons, add a new option to allow to ignore a list of test and subtests for these situations.
* bpo-36044: Reduce number of unit tests run for PGO build (GH-14702)Neil Schemenauer2019-07-221-1/+5
| | | | | | | | | | | | Reduce the number of unit tests run for the PGO generation task. This speeds up the task by a factor of about 15x. Running the full unit test suite is slow. This change may result in a slightly less optimized build since not as many code branches will be executed. If you are willing to wait for the much slower build, the old behavior can be restored using './configure [..] PROFILE_TASK="-m test --pgo-extended"'. We make no guarantees as to which PGO task set produces a faster build. Users who care should run their own relevant benchmarks as results can depend on the environment, workload, and compiler tool chain.
* bpo-37359: Add --cleanup option to python3 -m test (GH-14332)Victor Stinner2019-06-241-1/+3
| | | | | * regrtest: Add --cleanup option to remove "test_python_*" directories of previous failed test jobs. * Add "make cleantest" to run "python3 -m test --cleanup".
* bpo-36719: regrtest -jN no longer stops on crash (GH-13231)Victor Stinner2019-05-131-1/+4
| | | | | | | "python3 -m test -jN ..." now continues the execution of next tests when a worker process crash (CHILD_ERROR state). Previously, the test suite stopped immediately. Use --failfast to stop at the first error. Moreover, --forever now also implies --failfast.
* bpo-36719: regrtest always detect uncollectable objects (GH-12951)Victor Stinner2019-04-261-5/+7
| | | | | | | regrtest now always detects uncollectable objects. Previously, the check was only enabled by --findleaks. The check now also works with -jN/--multiprocess N. --findleaks becomes a deprecated alias to --fail-env-changed.
* Fix typo in test module usage message (GH-11374)sth2018-12-311-1/+1
| | | | | A minor typo in the output of `python -m test -h`. A space was missing in between two words. howmuch -> how much
* Add --tempdir option for test run (GH-10322)Steve Dower2018-11-171-3/+3
|
* bpo-34582: Adds JUnit XML output for regression tests (GH-9210)Steve Dower2018-09-181-0/+4
|
* bpo-34605, libregrtest: Rename --slaveargs to --worker-args (GH-9099)Victor Stinner2018-09-071-1/+1
| | | Rename also run_tests_slave() to run_tests_worker().
* Replace KB unit with KiB (#4293)Victor Stinner2017-11-081-1/+1
| | | | | | | | | | | kB (*kilo* byte) unit means 1000 bytes, whereas KiB ("kibibyte") means 1024 bytes. KB was misused: replace kB or KB with KiB when appropriate. Same change for MB and GB which become MiB and GiB. Change the output of Tools/iobench/iobench.py. Round also the size of the documentation from 5.5 MB to 5 MiB.
* bpo-30822: Exclude tzdata from regrtest --all (#2775)Victor Stinner2017-07-201-3/+12
| | | | | | | | | | | | | When running the test suite using --use=all / -u all, exclude tzdata since it makes test_datetime too slow (15-20 min on some buildbots) which then times out on some buildbots. -u tzdata must now be enabled explicitly, -u tzdata or -u all,tzdata, to run all test_datetime tests. Fix also regrtest command line parser to allow passing -u extralargefile to run test_zipfile64. Travis CI: remove -tzdata. Replace -u all,-tzdata,-cpu with -u all,-cpu since tzdata is now excluded from -u all.
* bpo-30764: regrtest: add --fail-env-changed option (#2402)Victor Stinner2017-06-261-0/+3
| | | | | | | | | | | | * bpo-30764: regrtest: change exit code on failure * Exit code 2 if failed tests ("bad") * Exit code 3 if interrupted * bpo-30764: regrtest: add --fail-env-changed option If the option is set, mark a test as failed if it alters the environment, for example if it creates a file without removing it.
* bpo-30523: regrtest: Add --list-cases option (#2238)mlouielu2017-06-161-0/+3
| | | | | | | | | | | * bpo-30523: regrtest: Add --list-cases option * bpo-30523: Enhance --list-cases * Add get_abs_module() function, use it in list_cases() * list_cases() now logs skipped tests into stderr * Remove unused doctest
* bpo-30540: regrtest: add --matchfile option (#1909)Victor Stinner2017-06-091-1/+19
| | | | | | | | | | | | | | | * Add a new option taking a filename to get a list of test names to filter tests. * support.match_tests becomes a list. * Modify run_unittest() to accept to match the whole test identifier, not just a part of a test identifier. For example, the following command only runs test_default_timeout() of the BarrierTests class of test_threading: $ ./python -m test -v test_threading -m test.test_threading.BarrierTests.test_default_timeout Remove also some empty lines from test_regrtest.py to make flake8 tool happy.
* bpo-27103: regrtest disables -W if -R is used (#1651)Victor Stinner2017-05-181-0/+5
| | | Workaround for a regrtest bug.
* bpo-30263: regrtest: log system load (#1452)Victor Stinner2017-05-041-0/+2
| | | | | | | | * bpo-30263: regrtest: log system load * regrtest: log the number of CPUs Log the number of CPUs in the header. --verbose now imply --header.
* Issue #28950: Disallow -j0 combined with -T/-l in regrtest.Xiang Zhang2016-12-191-2/+2
|
* Merge 3.6: Issue #28409: regrtest: fix the parser of command line arguments.Victor Stinner2016-10-171-4/+7
|
* Merge 3.5 (regrtest)Victor Stinner2016-09-101-1/+1
|
* regrtest: rename --slow option to --slowestVictor Stinner2016-08-171-1/+1
| | | | Thanks to optparse, --slow syntax still works ;-)
* Closes issue #24773: Implement PEP 495 (Local Time Disambiguation).Alexander Belopolsky2016-07-221-1/+3
|
* regrtest doesn't ignore -j1 anymoreVictor Stinner2016-05-201-2/+0
| | | | | | | | * regrtest now uses subprocesses when the -j1 command line option is used: each test file runs in a fresh child process. Before, the -j1 option was ignored. * Tools/buildbot/test.bat script now uses -j1 by default to run each test file in fresh child process.
* Issue #25523: Merge a-to-an corrections from 3.5.Serhiy Storchaka2015-11-021-1/+1
|
* Fix missing import in libregrtest.Steve Dower2015-10-081-0/+1
|
* Issue #23919: Prevents assert dialogs appearing in the test suite.Steve Dower2015-10-081-0/+4
|
* Merge from 3.5 for issue #25188.Brett Cannon2015-10-021-1/+5
|
* Issue #22806: Add ``python -m test --list-tests`` command to list tests.Victor Stinner2015-10-021-7/+4
|
* Issue #25220, libregrtest: Call setup_python(ns) in the slavesVictor Stinner2015-09-291-2/+0
| | | | | | | | | | Slaves (child processes running tests for regrtest -jN) now inherit --memlimit/-M, --threshold/-t and --nowindows/-n options. * -M, -t and -n are now supported with -jN * Factorize code to run tests. * run_test_in_subprocess() now pass the whole "ns" namespace to the child process.
* Issue #25220: Move most regrtest.py code to libregrtestVictor Stinner2015-09-261-1/+1
|
* Issue #25220: Create Lib/test/libregrtest/Victor Stinner2015-09-261-0/+340
| | | | | | | Start to split regrtest.py into smaller parts with the creation of Lib/test/libregrtest/cmdline.py: code to handle the command line, especially parsing command line arguments. This part of the code is tested by test_regrtest.
* Issue #25220: Backed out changeset eaf9a99b6bb8Victor Stinner2015-09-231-340/+0
|
* Issue #25220: Create Lib/test/libregrtest/Victor Stinner2015-09-231-0/+340
Start to split regrtest.py into smaller parts with the creation of Lib/test/libregrtest/cmdline.py.
generated by cgit v0.12 at 2025-09-19 01:32:57 (GMT)