summaryrefslogtreecommitdiffstats
path: root/Lib/test/libregrtest
Commit message (Collapse)AuthorAgeFilesLines
* bpo-34582: Adds JUnit XML output for regression tests (GH-9210)Miss Islington (bot)2018-09-184-20/+68
| | | | | (cherry picked from commit d0f49d2f5085ca68e3dc8725f1fb1c9674bfb5ed) Co-authored-by: Steve Dower <steve.dower@microsoft.com>
* bpo-33873: Backport regrtest from master to 3.7 (GH-7935) (GH-7937)Victor Stinner2018-06-264-11/+37
| | | | | | | | | | | | | | | | | | | | | | | * bpo-33718: regrtest: use format_duration() to display failed tests (GH-7686) * Enhance also format_duration(): work on integers and rounds towards +infinity (math.ceil). * Write unit tests on format_duration() (cherry picked from commit 4ffe9c2b251f6e027b26250b7a2618e78d4edd22) * bpo-33873: regrtest: Add warning on -R 1:3 (GH-7736) regrtest: Add warning when using less than 3 warmup runs like -R 1:3. (cherry picked from commit cac4fef8860e66a9da67d09762f5b614b9471a12) * bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters (GH-7735) Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs. Add check for invalid --huntrleaks/-R parameters. (cherry picked from commit 58ed7307ea0b5c5aa052291ebc3030f314f938d8) (cherry picked from commit d1f9481b7a2d31c40fca1347ef99d819eb656ce7)
* bpo-33718: regrtest: use "xxx then yyy" result if re-run (GH-7521)Miss Islington (bot)2018-06-091-1/+7
| | | | | | | | | If tests are re-run, use "xxx then yyy" result format (ex: "FAILURE then SUCCESS") to show that some failing tests have been re-run. Add also test_regrtest.test_rerun_fail() test. (cherry picked from commit c45fc7673e23f911639d10d3771ffef7be870c7a) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-33718: Update regrtest from master (GH-7325)Victor Stinner2018-06-017-70/+87
| | | | * Add support.environment_altered: unused yet * VSTS: don't run tests with --fail-env-changed
* bpo-31009: Move fd_count() to test.support (#7308)Victor Stinner2018-06-011-25/+1
| | | | | | * Move fd_count() from test.libregrtest.refleak to test.support * Fix support.fd_count() on Windows: Call CrtSetReportMode() to not kill the process on invalid file descriptor if Python is compiled in debug mode.
* regrtest: repeat summary after re-run (GH-7159) (GH-7245)Victor Stinner2018-05-301-6/+19
| | | | | | Using -w, when failing tests are re-run in verbose mode, display again the tests results at the end. (cherry picked from commit c6c05d0e69cd5a7d0205019c29a1236b7bf3f5b9)
* test.regrtest: flush stdout when display progress (GH-7105)Miss Islington (bot)2018-05-251-2/+2
| | | | | | runtest_mp.py: call print() with flush=True. (cherry picked from commit 4f0bc7f7ab6ec23594b0efe11d082f78ae42abed) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-32872: Avoid regrtest compatibility issue with namespace packages. ↵Miss Islington (bot)2018-03-281-1/+1
| | | | | | | (GH-6276) (GH-6278) (cherry picked from commit e52ac045972a4f75d7f52e4ee0d6de128259134d) Co-authored-by: Ned Deily <nad@python.org>
* [3.6] bpo-31324: Optimize support._match_test() (#4523)Victor Stinner2017-11-232-3/+3
| | | | | | | | | | | | | | | | | | | * bpo-31324: Optimize support._match_test() (#4421) * Rename support._match_test() to support.match_test(): make it public * Remove support.match_tests global variable. It is replaced with a new support.set_match_tests() function, so match_test() doesn't have to check each time if patterns were modified. * Rewrite match_test(): use different code paths depending on the kind of patterns for best performances. Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com> (cherry picked from commit 803ddd8ce22f0de3ab42fb98a225a704c000ef06) * bpo-31324: Fix test.support.set_match_tests(None) (#4505) (cherry picked from commit bb11c3c967afaf263e00844d4ab461b7fafd6d36)
* [3.6] bpo-25588: Fix regrtest when run inside IDLE (GH-3962) (#3987)Miss Islington (bot)2017-10-131-12/+31
| | | | | | | | When regrtest in run inside IDLE, sys.stdout and sys.stderr are not TextIOWrapper objects and have no file descriptor associated: sys.stderr.fileno() raises io.UnsupportedOperation. Disable faulthandler and don't replace sys.stdout in that case. (cherry picked from commit ccef823939d4ef602f2d8d13d0bfec29eda597a5)
* bpo-31217: Fix regrtest -R for small integer (#3260) (#3261)Victor Stinner2017-09-011-3/+11
| | | | | | | | | Use a pool of integer objects toprevent false alarm when checking for memory block leaks. Fill the pool with values in -1000..1000 which are the most common (reference, memory block, file descriptor) differences. Co-Authored-By: Antoine Pitrou <pitrou@free.fr> (cherry picked from commit 6c2feabc5dac2f3049b15134669e9ad5af573193)
* [3.6] bpo-30871: Add test.pythoninfo (#3174)Victor Stinner2017-08-211-3/+0
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * bpo-30871: Add test.pythoninfo (#3075) * Add Lib/test/pythoninfo.py: script collecting various informations about Python to help debugging test failures. * regrtest: remove sys.hash_info and sys.flags from header. * Travis CI, Appveyor: run pythoninfo before tests (cherry picked from commit b907abc88589f7bea52c5afe172ececc6edcda70) * bpo-30871: pythoninfo: add expat and _decimal (#3121) * bpo-30871: pythoninfo: add expat and _decimal * Remove _decimal.__version__ The string is hardcoded, not really interesting. (cherry picked from commit f6ebd838f00b4c211c72d85ee49749e910cd3afe) * bpo-30871: Add "make pythoninfo" (#3120) (cherry picked from commit a3a01a2fceab2188b282ab9911f79c99a4c32273) * bpo-30871: pythoninfo: more sys, os, time data (#3130) * bpo-30871: pythoninfo: more sys, os, time data PythonInfo now converts types other than intger to string by default. * fix typo (cherry picked from commit ad7eaed54382b346784e51a6f0122ce81e8842b5) * bpo-31231: Fix pythoninfo in Travis config (#3134) bpo-31231, bpo-30871: Replace "./python -m test.pythoninfo" with "make pythoninfo", since macOS uses ./python.exe. (cherry picked from commit 92b1f90143286385c0ff5be98d3721b90580a912)
* [3.6] bpo-31160: Backport reap_children() fixes from master to 3.6 (#3060)Victor Stinner2017-08-101-0/+5
| | | | | | | | | | | | | | | | | | | | | | | | | * bpo-31160: Fix test_builtin for zombie process (#3043) PtyTests.run_child() now calls os.waitpid() to read the exit status of the child process to avoid creating zombie process and leaking processes in the background. (cherry picked from commit 4baca1b0f7325032598cd38e7ceffc79b616d255) * bpo-31160: regrtest now reaps child processes (#3044) Add a post_test_cleanup() function which currently only calls support.reap_children(). (cherry picked from commit e3510d74aacc477c30f42f2b941d69689bbc478e) * bpo-31160: test_builtin: don't check waitpid() status (#3050) (cherry picked from commit 3ca9f50f96cfa5c1b7aa56639042531b57f07fbb) * bpo-31160: test_tempfile: Fix reap_children() warning (#3056) TestRandomNameSequence.test_process_awareness() now calls os.waitpid() to avoid leaking a zombie process. (cherry picked from commit 6c8c2943d996b59a48d331f61f22cbe72933910e)
* bpo-30822: Exclude tzdata from regrtest --all (#2775) (#2777)Victor Stinner2017-07-202-4/+13
| | | | | | | | | | | | | | When running the test suite using --use=all / -u all, exclude tzdata since it makes test_datetime too slow (15-20 min on some buildbots) which then times out on some buildbots. -u tzdata must now be enabled explicitly, -u tzdata or -u all,tzdata, to run all test_datetime tests. Fix also regrtest command line parser to allow passing -u extralargefile to run test_zipfile64. Travis CI: remove -tzdata. Replace -u all,-tzdata,-cpu with -u all,-cpu since tzdata is now excluded from -u all. (cherry picked from commit 5b392bbaeb9d9b1db961ecfc7315d8c8662c27f6)
* bpo-30764: Fix regrtest --fail-env-changed --forever (#2536) (#2539)Victor Stinner2017-07-031-1/+3
| | | | --forever now stops if a fail changes the environment. (cherry picked from commit 5e87592fd12e0b7c41edc11d4885ed7298d5063b)
* [3.6] bpo-29512, bpo-30776: Backport regrtest enhancements from master to ↵Victor Stinner2017-06-301-10/+3
| | | | | | | | | | | | | | | | | 3.6 (#2513) * bpo-29512: Add test.bisect, bisect failing tests (#2452) Add a new "python3 -m test.bisect" tool to bisect failing tests. It can be used to find which test method(s) leak references, leak files, etc. (cherry picked from commit 84d9d14a1fa395fbd21262ba195490be25a7b3bc) * bpo-30776: regrtest: reduce memleak false positive (#2484) Only report a leak if each run leaks at least one memory block. (cherry picked from commit beeca6e1e5fd01531b1db7059498b13d07dca525)
* [3.6] bpo-30523, bpo-30764, bpo-30776: Sync regrtest from master (#2441)Victor Stinner2017-06-273-5/+39
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * bpo-30523: regrtest --list-cases --match (#2401) * regrtest --list-cases now supports --match and --match-file options. Example: ./python -m test --list-cases -m FileTests test_os * --list-cases now also sets support.verbose to False to prevent messages to stdout when loading test modules. * Add support._match_test() private function. (cherry picked from commit ace56d583664f855d89d1219ece7c21c2fddcf30) * bpo-30764: regrtest: add --fail-env-changed option (#2402) * bpo-30764: regrtest: change exit code on failure * Exit code 2 if failed tests ("bad") * Exit code 3 if interrupted * bpo-30764: regrtest: add --fail-env-changed option If the option is set, mark a test as failed if it alters the environment, for example if it creates a file without removing it. (cherry picked from commit 63f54c68936d648c70ca411661e4208329edcf26) * bpo-30776: reduce regrtest -R false positives (#2422) * Change the regrtest --huntrleaks checker to decide if a test file leaks or not. Require that each run leaks at least 1 reference. * Warmup runs are now completely ignored: ignored in the checker test and not used anymore to compute the sum. * Add an unit test for a reference leak. Example of reference differences previously considered a failure (leak) and now considered as success (success, no leak): [3, 0, 0] [0, 1, 0] [8, -8, 1] (cherry picked from commit 48b5c422ffb03affb00c184b9a99e5537be92732)
* Synchronize libregrtest from master to 3.6 (#2244)Victor Stinner2017-06-165-52/+119
| | | | | | | * bpo-30523: regrtest: Add --list-cases option (#2238) * bpo-30284: Fix regrtest for out of tree build (#1481) * bpo-30540: regrtest: add --matchfile option (#1909) * bpo-30258: regrtest: Fix run_tests_multiprocess() (#1479) * bpo-30263: regrtest: log system load (#1452)
* bpo-27103: regrtest disables -W if -R is used (#1651) (#1656)Victor Stinner2017-05-181-0/+5
| | | | Workaround for a regrtest bug. (cherry picked from commit fcdd9b6b7e73427ce5aa63cf095312f603c4edce)
* regrtest: always show before/after of modified env (#1192) (#1406)Victor Stinner2017-05-031-3/+2
| | | | | | | | | Buildbots don't run tests with -vv and so only log "xxx was modified by test_xxx" which is not enough to debug such random issue. In many cases, I'm unable to reproduce the warning and so unable to fix it. Always logging the value before and value after should help to debug such warning on buildbots. (cherry picked from commit ec4b17239d899550be4ee6104b61751bb3c70382)
* bpo-29638: Fix spurious refleaks after typing is imported (#469) (#483)Ivan Levkivskyi2017-03-051-3/+8
|
* Issue #28950: Disallow -j0 combined with -T/-l in regrtest.Xiang Zhang2016-12-191-2/+2
|
* Issue #23839: Various caches now are cleared before running every test file.Serhiy Storchaka2016-11-112-26/+86
|
* Issue #28649: Clear the typing module caches when search for reference leaks.Serhiy Storchaka2016-11-091-0/+8
|
* Merge 3.6: Issue #28409: regrtest: fix the parser of command line arguments.Victor Stinner2016-10-171-4/+7
|
* Merge 3.5 (issue #27829)Victor Stinner2016-09-231-1/+1
|
* Merge 3.5 (regrtest)Victor Stinner2016-09-101-1/+1
|
* Issue 27948: Allow backslashes in the literal string portion of f-strings, ↵Eric V. Smith2016-09-101-1/+1
| | | | but not in the expressions. Also, require expressions to begin and end with literal curly braces.
* regrtest: log FS and locale encodingsVictor Stinner2016-09-091-1/+5
|
* Adds test.support.PGO and skips tests that are not useful for PGO.Steve Dower2016-09-071-0/+2
|
* Closes issue 27921: Disallow backslashes anywhere in f-strings. This is a ↵Eric V. Smith2016-09-031-1/+1
| | | | temporary restriction. In 3.6 beta 2, the plan is to again allow backslashes in the string parts of f-strings, but disallow them in the expression parts.
* Issue #27829: libregrtest.save_env: flush stderrVictor Stinner2016-08-221-6/+4
| | | | | | Use flush=True to try to get a warning which is missing in buildbots. Use also f-string to make the code shorter.
* Cleanup libregrtestVictor Stinner2016-08-222-37/+20
| | | | | * main.py: remove unused import * runtest: simplify runtest_inner() parameters, reuse ns parameter
* Issue #27787: Merge regrtest fixup from 3.5Martin Panter2016-08-201-0/+1
|
* regrtest: replace "Result:" with "Tests result:"Victor Stinner2016-08-191-1/+1
|
* regrtest: add a summary of the summary, "Result: xxx"Victor Stinner2016-08-171-0/+8
| | | | | | | It's sometimes hard to check quickly if tests succeeded, failed or something bad happened. I added a final "Result: xxx" line which summarizes all outputs into a single line, written at the end (it should always be the last line of the output).
* regrtest: set interrupted to True if re-run is interruptedVictor Stinner2016-08-171-1/+2
|
* regrtest: add newlines in output for readabilityVictor Stinner2016-08-171-0/+3
|
* regrtest: nicer output for durationsVictor Stinner2016-08-171-12/+19
| | | | Use milliseconds and minutes units, not only seconds.
* regrtest: rename --slow option to --slowestVictor Stinner2016-08-171-1/+1
| | | | Thanks to optparse, --slow syntax still works ;-)
* Closes issue #24773: Implement PEP 495 (Local Time Disambiguation).Alexander Belopolsky2016-07-221-1/+3
|
* regrtest: display test result (passed, failed, ...)Victor Stinner2016-05-203-8/+26
| | | | | * in multiprocessing mode: always display the result * sequential mode: only display the result if the test did not pass
* regrtest doesn't ignore -j1 anymoreVictor Stinner2016-05-201-2/+0
| | | | | | | | * regrtest now uses subprocesses when the -j1 command line option is used: each test file runs in a fresh child process. Before, the -j1 option was ignored. * Tools/buildbot/test.bat script now uses -j1 by default to run each test file in fresh child process.
* Issue #26295: When using "python3 -m test --testdir=TESTDIR", regrtest doesn'tVictor Stinner2016-03-302-4/+9
| | | | | | add "test." prefix to test module names. regrtest also prepends testdir to sys.path.
* Merge 3.5 (regrtest)Victor Stinner2016-03-291-0/+1
|
* fix typo in commentVictor Stinner2016-03-291-1/+1
| | | | Thanks Arfrever for the report :)
* regrtest: round final timing towards +infVictor Stinner2016-03-271-3/+8
|
* Backed out changeset 245a16f33c4bVictor Stinner2016-03-251-279/+202
| | | | Serhiy asked me to review it.
* Rework libregrtest.save_envVictor Stinner2016-03-251-202/+279
| | | | | | | | | | | * Replace get/restore methods with a Resource class and Resource subclasses * Create ModuleAttr, ModuleAttrList and ModuleAttrDict helper classes * Use __subclasses__() to get resource classes instead of using an hardcoded list (2 shutil resources were missinged in the list!) * Don't define MultiprocessingProcessDangling resource if the multiprocessing module is missing * Nicer diff for dictionaries. Useful for the big os.environ dict * Reorder code to group resources
* Cleanup regrtest "main()" functionVictor Stinner2016-03-242-23/+27
| | | | | | | * Rename libregrtest.main_in_temp_cwd() to libregrtest.main() * Add regrtest.main_in_temp_cwd() alias to libregrtest.main() * Move old main_in_temp_cwd() code into libregrtest.Regrtest.main() * Update multiple scripts to call libregrtest.main()
generated by cgit v0.12 at 2026-01-10 02:27:45 (GMT)