summaryrefslogtreecommitdiffstats
path: root/Lib/test/libregrtest
Commit message (Collapse)AuthorAgeFilesLines
* Update libregrtest from master (GH-19517)Victor Stinner2020-04-146-44/+91
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * bpo-36670: regrtest bug fixes (GH-16537) * Fix TestWorkerProcess.__repr__(): start_time is only valid if _popen is not None. * Fix _kill(): don't set _killed to True if _popen is None. * _run_process(): only set _killed to False after calling run_test_in_subprocess(). (cherry picked from commit 2ea71a07d0a720707094ee55f78fd232c40724bc) * [3.8] Update libregrtest from master (GH-19516) * bpo-37531: regrtest now catchs ProcessLookupError (GH-16827) Fix a warning on a race condition on TestWorkerProcess.kill(): ignore silently ProcessLookupError rather than logging an useless warning. (cherry picked from commit a661392f8fb5ac4fc095aa1845d1eb7a25c4e9be) * bpo-38502: regrtest uses process groups if available (GH-16829) test.regrtest now uses process groups in the multiprocessing mode (-jN command line option) if process groups are available: if os.setsid() and os.killpg() functions are available. (cherry picked from commit ecb035cd14c11521276343397151929a94018a22) * bpo-37957: Allow regrtest to receive a file with test (and subtests) to ignore (GH-16989) When building Python in some uncommon platforms there are some known tests that will fail. Right now, the test suite has the ability to ignore entire tests using the -x option and to receive a filter file using the --matchfile filter. The problem with the --matchfile option is that it receives a file with patterns to accept and when you want to ignore a couple of tests and subtests, is too cumbersome to lists ALL tests that are not the ones that you want to accept and he problem with -x is that is not easy to ignore just a subtests that fail and the whole test needs to be ignored. For these reasons, add a new option to allow to ignore a list of test and subtests for these situations. (cherry picked from commit e0cd8aa70a3ce19c3d3712568940aa0cbd9aa97b) * regrtest: log timeout at startup (GH-19514) Reduce also worker timeout. (cherry picked from commit 4cf65a630a8d45bad3fe5cdc4c2632ec64e7ba27) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com> (cherry picked from commit 67b8a1f0f0f78ec38b8626fa9f5b2f5a55c17e15) * bpo-36842: Fix reference leak in tests by running out-of-proc (GH-13556) (cherry picked from commit 9ddc416e9f6635376312c3615193f19480ac772a) * Backport libregrtest changes from master Co-authored-by: Steve Dower <steve.dower@python.org>
* bpo-37531: regrtest ignores output on timeout (GH-16659)Miss Islington (bot)2019-10-091-27/+37
| | | | | | | | | | | | | | bpo-37531, bpo-38207: On timeout, regrtest no longer attempts to call `popen.communicate() again: it can hang until all child processes using stdout and stderr pipes completes. Kill the worker process and ignores its output. Reenable test_regrtest.test_multiprocessing_timeout(). bpo-37531: Change also the faulthandler timeout of the main process from 1 minute to 5 minutes, for Python slowest buildbots. (cherry picked from commit 0ec618af98ac250a91ee9c91f8569e6df6772758) Co-authored-by: Victor Stinner <vstinner@python.org>
* [3.8] bpo-36670, regrtest: Fix WindowsLoadTracker() for partial line ↵Miss Islington (bot)2019-10-034-68/+126
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (GH-16550) (GH-16560) * bpo-36670, regrtest: Fix WindowsLoadTracker() for partial line (GH-16550) WindowsLoadTracker.read_output() now uses a short buffer for incomplete line. (cherry picked from commit 3e04cd268ee9a57f95dc78d8974b21a6fac3f666) * bpo-36670: Enhance regrtest WindowsLoadTracker (GH-16553) The last line is now passed to the parser even if it does not end with a newline, but only if it's a valid value. (cherry picked from commit c65119d5bfded03f80a9805889391b66fa7bf551) * bpo-36670: Enhance regrtest (GH-16556) * Add log() method: add timestamp and load average prefixes to main messages. * WindowsLoadTracker: * LOAD_FACTOR_1 is now computed using SAMPLING_INTERVAL * Initialize the load to the arithmetic mean of the first 5 values of the Processor Queue Length value (so over 5 seconds), rather than 0.0. * Handle BrokenPipeError and when typeperf exit. * format_duration(1.5) now returns '1.5 sec', rather than '1 sec 500 ms' (cherry picked from commit 098e25672f1c3578855d5ded4f5147795c9ed956) (cherry picked from commit de3195c937d5fca0d79cc93dbafa76c0f89ca5b8) Co-authored-by: Victor Stinner <vstinner@python.org>
* bpo-36670: Multiple regrtest bugfixes (GH-16511)Miss Islington (bot)2019-10-013-133/+175
| | | | | | | | | | | | | | | | | | | * Windows: Fix counter name in WindowsLoadTracker. Counter names are localized: use the registry to get the counter name. Original change written by Lorenz Mende. * Regrtest.main() now ensures that the Windows load tracker is also killed if an exception is raised * TestWorkerProcess now ensures that worker processes are no longer running before exiting: kill also worker processes when an exception is raised. * Enhance regrtest messages and warnings: include test name, duration, add a worker identifier, etc. * Rename MultiprocessRunner to TestWorkerProcess * Use print_warning() to display warnings. Co-Authored-By: Lorenz Mende <Lorenz.mende@gmail.com> (cherry picked from commit 982bfa4da07b2e5749a0f4e68f99e972bcc3a549) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-37531: sync regrtest with master branch (GH-16285) (GH-16289)Victor Stinner2019-09-192-17/+26
| | | (cherry picked from commit fb7746d5d10ec4a34198da672018ba15f5667079)
* bpo-37531: regrtest main process uses shorter timeout (GH-16220) (GH-16223)Miss Islington (bot)2019-09-171-6/+6
| | | | | | | | | When using multiprocesss (-jN), the main process now uses a timeout of 60 seconds instead of the double of the --timeout value. The buildbot server stops a job which does not produce any output in 1200 seconds. (cherry picked from commit 46b0b81220a23bc4aee5ba3ba67e8cf1b5df7960) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* [3.7] bpo-37531: Enhance regrtest multiprocess timeout (GH-15345) (GH-15874)Victor Stinner2019-09-102-19/+63
| | | | | | | | | | | bpo-37531: Enhance regrtest multiprocess timeout (GH-15345) * Write a message when killing a worker process * Put a timeout on the second popen.communicate() call (after killing the process) * Put a timeout on popen.wait() call * Catch popen.kill() and popen.wait() exceptions (cherry picked from commit de2d9eed8bc628533e1628b843cc4c7a5010f6e5)
* [3.7] bpo-37531: Fix regrtest timeout for subprocesses (GH-15072) (GH-15280)Victor Stinner2019-08-144-15/+51
| | | | | | Co-Authored-By: Joannah Nanjekye <joannah.nanjekye@ibm.com> (cherry picked from commit b0c8369c603633f445ccbb5ca7a8742145ff9eec) Backport also minor win_utils.py fixes from master.
* Revert "Remove unused imports in tests (GH-14518) (GH-14522)" (GH-14555)Victor Stinner2019-07-021-0/+1
| | | This reverts commit e34b5f4d6483187969d5149c801d056b72ef2ddb.
* bpo-34602: Avoid failures setting macOS stack resource limit (GH-14546)Miss Islington (bot)2019-07-021-16/+0
| | | | | | | | | | | | | | | | | | | | Under some conditions the earlier fix for bpo-18075, "Infinite recursion tests triggering a segfault on Mac OS X", now causes failures on macOS when attempting to change stack limit with resource.setrlimit resource.RLIMIT_STACK, like regrtest does when running the test suite. The reverted change had specified a non-default stack size when linking the python executable on macOS. As of macOS 10.14.4, the previous code causes a hard failure when running tests, although similar failures had been seen under some conditions under some earlier systems. Reverting the change to the interpreter stack size at link time helped for release builds but caused some tests to fail when built --with-pydebug. Try the opposite approach: continue to build the interpreter with an increased stack size on macOS and remove the failing setrlimit call in regrtest initialization. This will definitely avoid the resource.RLIMIT_STACK error and should have no, or fewer, side effects. (cherry picked from commit 5bbbc733e6cc0804f19b071944af8d4719e26ae6) Co-authored-by: Ned Deily <nad@python.org>
* Remove unused imports in tests (GH-14518) (GH-14522)Victor Stinner2019-07-011-1/+0
| | | (cherry picked from commit 8f4ef3b019ce380022018587571b0f970e668de3)
* bpo-37359: Fix regrtest --cleanup (GH-14336)Miss Islington (bot)2019-06-241-1/+1
| | | | | (cherry picked from commit 9bbf4d7083a819cbcee2a6cd3df2802d4c50f734) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-37359: Add --cleanup option to python3 -m test (GH-14332) (GH-14335)Victor Stinner2019-06-242-14/+41
| | | | | | | * regrtest: Add --cleanup option to remove "test_python_*" directories of previous failed test jobs. * Add "make cleantest" to run "python3 -m test --cleanup". (cherry picked from commit 47fbc4e45b35b3111e2d947a66490a43ac21d363)
* bpo-36915: regrtest always remove tempdir of worker processes (GH-13312)Miss Islington (bot)2019-05-142-44/+66
| | | | | | | | | | | | When using multiprocessing (-jN option), worker processes now create their temporary directory inside the temporary directory of the main process. So the main process is able to remove temporary directories of worker processes even if they crash or when they are killed by regrtest on KeyboardInterrupt (CTRL+c). Rework also how multiprocessing arguments are parsed in main.py. (cherry picked from commit 3c93153f7db5dd9b06f229e61978fd9199b3c097) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-36719: Fix regrtest MultiprocessThread (GH-13301) (GH-13303)Miss Islington (bot)2019-05-141-4/+55
| | | | | | | MultiprocessThread.kill() now closes stdout and stderr to prevent popen.communicate() to hang. (cherry picked from commit c923c3449f825021b13521b2380e67ba35a36f55) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-36719: regrtest -jN no longer stops on crash (GH-13231)Miss Islington (bot)2019-05-134-18/+46
| | | | | | | | | | "python3 -m test -jN ..." now continues the execution of next tests when a worker process crash (CHILD_ERROR state). Previously, the test suite stopped immediately. Use --failfast to stop at the first error. Moreover, --forever now also implies --failfast. (cherry picked from commit b0917df329ba14b7bc6fa782c1b61e7a2163af0b) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* [3.7] bpo-36719: sync regrtest with master branch (GH-12967)Victor Stinner2019-04-268-363/+584
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Clean up code which checked presence of os.{stat,lstat,chmod} (GH-11643) (cherry picked from commit 8377cd4fcd0d51d86834c9b0518d29aac3b49e18) * bpo-36725: regrtest: add TestResult type (GH-12960) * Add TestResult and MultiprocessResult types to ensure that results always have the same fields. * runtest() now handles KeyboardInterrupt * accumulate_result() and format_test_result() now takes a TestResult * cleanup_test_droppings() is now called by runtest() and mark the test as ENV_CHANGED if the test leaks support.TESTFN file. * runtest() now includes code "around" the test in the test timing * Add print_warning() in test.libregrtest.utils to standardize how libregrtest logs warnings to ease parsing the test output. * support.unload() is now called with abstest rather than test_name * Rename 'test' variable/parameter to 'test_name' * dash_R(): remove unused the_module parameter * Remove unused imports (cherry picked from commit 4d29983185bc12ca685a1eb3873bacb8a7b67416) * bpo-36725: Refactor regrtest multiprocessing code (GH-12961) Rewrite run_tests_multiprocess() function as a new MultiprocessRunner class with multiple methods to better report errors and stop immediately when needed. Changes: * Worker processes are now killed immediately if tests are interrupted or if a test does crash (CHILD_ERROR): worker processes are killed. * Rewrite how errors in a worker thread are reported to the main thread. No longer ignore BaseException or parsing errors silently. * Remove 'finished' variable: use worker.is_alive() instead * Always compute omitted tests. Add Regrtest.get_executed() method. (cherry picked from commit 3cde440f20a9db75fb2c4e65e8e4d04a53216a2d) * bpo-36719: regrtest always detect uncollectable objects (GH-12951) regrtest now always detects uncollectable objects. Previously, the check was only enabled by --findleaks. The check now also works with -jN/--multiprocess N. --findleaks becomes a deprecated alias to --fail-env-changed. (cherry picked from commit 75120d2205af086140e5e4e2dc620eb19cdf9078) * bpo-34060: Report system load when running test suite for Windows (GH-8357) While Windows exposes the system processor queue length, the raw value used for load calculations on Unix systems, it does not provide an API to access the averaged value. Hence to calculate the load we must track and average it ourselves. We can't use multiprocessing or a thread to read it in the background while the tests run since using those would conflict with test_multiprocessing and test_xxsubprocess. Thus, we use Window's asynchronous IO API to run the tracker in the background with it sampling at the correct rate. When we wish to access the load we check to see if there's new data on the stream, if there is, we update our load values. (cherry picked from commit e16467af0bfcc9f399df251495ff2d2ad20a1669) * bpo-36719: Fix regrtest re-run (GH-12964) Properly handle a test which fail but then pass. Add test_rerun_success() unit test. (cherry picked from commit 837acc1957d86ca950433f5064fd06d09b57d23b) * bpo-36719: regrtest closes explicitly WindowsLoadTracker (GH-12965) Regrtest.finalize() now closes explicitly the WindowsLoadTracker instance. (cherry picked from commit 00db7c73af4f60df61e9df87cde7401c3ed9df69)
* [3.7] bpo-36560: regrtest: don't collect the GC twice (GH-12747) (GH-12749)Victor Stinner2019-04-093-33/+44
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * bpo-36560: Fix reference leak hunting in regrtest (GH-12744) Fix reference leak hunting in regrtest: compute also deltas (of reference count, allocated memory blocks, file descriptor count) during warmup, to ensure that everything is initialized before starting to hunt reference leaks. Other changes: * Replace gc.collect() with support.gc_collect() * Move calls to read memory statistics from dash_R_cleanup() to dash_R() * Pass regrtest 'ns' to dash_R() * dash_R() is now more quiet with --quiet option (don't display progress). * Precompute the full range for "for it in range(repcount):" to ensure that the iteration doesn't allocate anything new. * dash_R() now is responsible to call warm_caches(). (cherry picked from commit 5aaac94eeb44697e92b0951385cd557bc27e0f6a) * bpo-36560: regrtest: don't collect the GC twice (GH-12747) dash_R() function of libregrtest doesn't call support.gc_collect() directly anymore: it's already called by dash_R_cleanup(). Call dash_R_cleanup() before starting the loop. (cherry picked from commit bb4447897a5f141eecf42987a1191a3330c5d7ed)
* bpo-36565: Fix libregrtest for Python without builtin _abc (GH-12733) (GH-12734)Miss Islington (bot)2019-04-081-2/+6
| | | | | | | Fix reference hunting (``python3 -m test -R 3:3``) when Python has no built-in abc module: fix _get_dump() reimplementation of libregrtest. (cherry picked from commit 79b5d29041bd85ea3baa050b3fa2481344ea35c9) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-35683: Improve Azure Pipelines steps (GH-11493)Steve Dower2019-01-221-0/+4
|
* Fix typo in test module usage message (GH-11374)Miss Islington (bot)2018-12-311-1/+1
| | | | | | | | A minor typo in the output of `python -m test -h`. A space was missing in between two words. howmuch -> how much (cherry picked from commit 7108aab49c2b6d6d81766c354f9a809278e03d4c) Co-authored-by: sth <sth.dev@tejp.de>
* bpo-35513, unittest: TextTestRunner uses time.perf_counter() (GH-11180)Miss Islington (bot)2018-12-171-2/+2
| | | | | | | | | | TextTestRunner of unittest.runner now uses time.perf_counter() rather than time.time() to measure the execution time of a test: time.time() can go backwards, whereas time.perf_counter() is monotonic. Similar change made in libregrtest, pprint and random. (cherry picked from commit 8db5b54463118e5eb60cb3582e3108623f01f5df) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* [3.7] bpo-34279: Synchronize regrtest with master (GH-10800)Victor Stinner2018-11-294-12/+28
| | | | | | | | | | | | | * bpo-34605, libregrtest: Rename --slaveargs to --worker-args (GH-9099) Rename also run_tests_slave() to run_tests_worker(). (cherry picked from commit 012f5b968a738b15ae9b40c499a1c0778b0615a9) * bpo-34279, regrtest: Issue a warning if no tests have been executed (GH-10150) (cherry picked from commit 9724348b43a9005a449ba532ccd3c6726f031097) * test_regrtest: remove unused threading import
* Add --tempdir option for test run (GH-10322)Miss Islington (bot)2018-11-172-10/+8
| | | | | (cherry picked from commit 38df97a03c5102e717a110ab69bff8e5c9ebfd08) Co-authored-by: Steve Dower <steve.dower@microsoft.com>
* bpo-34582: Adds JUnit XML output for regression tests (GH-9210)Miss Islington (bot)2018-09-184-20/+68
| | | | | (cherry picked from commit d0f49d2f5085ca68e3dc8725f1fb1c9674bfb5ed) Co-authored-by: Steve Dower <steve.dower@microsoft.com>
* bpo-33873: Backport regrtest from master to 3.7 (GH-7935)Victor Stinner2018-06-264-11/+37
| | | | | | | | | | | | | | | | | | | | | | * bpo-33718: regrtest: use format_duration() to display failed tests (GH-7686) * Enhance also format_duration(): work on integers and rounds towards +infinity (math.ceil). * Write unit tests on format_duration() (cherry picked from commit 4ffe9c2b251f6e027b26250b7a2618e78d4edd22) * bpo-33873: regrtest: Add warning on -R 1:3 (GH-7736) regrtest: Add warning when using less than 3 warmup runs like -R 1:3. (cherry picked from commit cac4fef8860e66a9da67d09762f5b614b9471a12) * bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` parameters (GH-7735) Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs. Add check for invalid --huntrleaks/-R parameters. (cherry picked from commit 58ed7307ea0b5c5aa052291ebc3030f314f938d8)
* bpo-33718: regrtest: use "xxx then yyy" result if re-run (GH-7521)Miss Islington (bot)2018-06-091-1/+7
| | | | | | | | | If tests are re-run, use "xxx then yyy" result format (ex: "FAILURE then SUCCESS") to show that some failing tests have been re-run. Add also test_regrtest.test_rerun_fail() test. (cherry picked from commit c45fc7673e23f911639d10d3771ffef7be870c7a) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-33718: regrtest: update from master (GH-7305)Victor Stinner2018-06-013-61/+87
|
* test.regrtest: flush stdout when display progress (GH-7105)Miss Islington (bot)2018-05-251-2/+2
| | | | | | runtest_mp.py: call print() with flush=True. (cherry picked from commit 4f0bc7f7ab6ec23594b0efe11d082f78ae42abed) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-32872: Avoid regrtest compatibility issue with namespace packages. ↵Miss Islington (bot)2018-03-281-1/+1
| | | | | | | (GH-6276) (#6277) (cherry picked from commit e52ac045972a4f75d7f52e4ee0d6de128259134d) Co-authored-by: Ned Deily <nad@python.org>
* bpo-31333: Re-implement ABCMeta in C (GH-5733)Ivan Levkivskyi2018-02-181-5/+12
| | | | This adds C versions of methods used by ABCMeta that improve performance of various ABC operations.
* bpo-32226: PEP 560: improve typing module (#4906)Ivan Levkivskyi2018-01-201-5/+0
| | | | This PR re-designs the internal typing API using the new PEP 560 features. However, there are only few minor changes in the public API.
* bpo-31324: Optimize support._match_test() (#4421)Victor Stinner2017-11-212-3/+3
| | | | | | | | | | | * Rename support._match_test() to support.match_test(): make it public * Remove support.match_tests global variable. It is replaced with a new support.set_match_tests() function, so match_test() doesn't have to check each time if patterns were modified. * Rewrite match_test(): use different code paths depending on the kind of patterns for best performances. Co-Authored-By: Serhiy Storchaka <storchaka@gmail.com>
* Replace KB unit with KiB (#4293)Victor Stinner2017-11-081-1/+1
| | | | | | | | | | | kB (*kilo* byte) unit means 1000 bytes, whereas KiB ("kibibyte") means 1024 bytes. KB was misused: replace kB or KB with KiB when appropriate. Same change for MB and GB which become MiB and GiB. Change the output of Tools/iobench/iobench.py. Round also the size of the documentation from 5.5 MB to 5 MiB.
* bpo-25588: Fix regrtest when run inside IDLE (#3962)Victor Stinner2017-10-131-12/+31
| | | | | | | When regrtest in run inside IDLE, sys.stdout and sys.stderr are not TextIOWrapper objects and have no file descriptor associated: sys.stderr.fileno() raises io.UnsupportedOperation. Disable faulthandler and don't replace sys.stdout in that case.
* bpo-31370: Remove support for threads-less builds (#3385)Antoine Pitrou2017-09-072-13/+2
| | | | | | * Remove Setup.config * Always define WITH_THREAD for compatibility.
* bpo-31217: Fix regrtest -R for small integer (#3260)Victor Stinner2017-09-011-3/+11
| | | | | | | | Use a pool of integer objects toprevent false alarm when checking for memory block leaks. Fill the pool with values in -1000..1000 which are the most common (reference, memory block, file descriptor) differences. Co-Authored-By: Antoine Pitrou <pitrou@free.fr>
* bpo-30871: Add test.pythoninfo (#3075)Victor Stinner2017-08-171-3/+0
| | | | | | * Add Lib/test/pythoninfo.py: script collecting various informations about Python to help debugging test failures. * regrtest: remove sys.hash_info and sys.flags from header. * Travis CI, Appveyor: run pythoninfo before tests
* regrtest: count also env changed as failures in progress (#3061)Victor Stinner2017-08-101-2/+3
|
* bpo-31160: regrtest now reaps child processes (#3044)Victor Stinner2017-08-091-0/+5
| | | | Add a post_test_cleanup() function which currently only calls support.reap_children().
* bpo-26732: fix too many fds in processes started with the "forkserver" ↵Antoine Pitrou2017-07-221-31/+1
| | | | | | | | | | method (#2813) * bpo-26732: fix too many fds in processes started with the "forkserver" method A child process would inherit as many fds as the number of still-running children. * Add blurb and test comment
* bpo-30822: Exclude tzdata from regrtest --all (#2775)Victor Stinner2017-07-202-4/+13
| | | | | | | | | | | | | When running the test suite using --use=all / -u all, exclude tzdata since it makes test_datetime too slow (15-20 min on some buildbots) which then times out on some buildbots. -u tzdata must now be enabled explicitly, -u tzdata or -u all,tzdata, to run all test_datetime tests. Fix also regrtest command line parser to allow passing -u extralargefile to run test_zipfile64. Travis CI: remove -tzdata. Replace -u all,-tzdata,-cpu with -u all,-cpu since tzdata is now excluded from -u all.
* bpo-30764: Fix regrtest --fail-env-changed --forever (#2536)Victor Stinner2017-07-031-1/+3
| | | --forever now stops if a fail changes the environment.
* threading_cleanup() failure marks test as ENV_CHANGED (#2500)Victor Stinner2017-06-302-1/+10
| | | | | | | If threading_cleanup() fails to cleanup threads, set a a new support.environment_altered flag to true, flag uses by save_env which is used by regrtest to check if a test altered the environment. At the end, the test file fails with ENV_CHANGED instead of SUCCESS, to report that it altered the environment.
* bpo-30776: regrtest: reduce memleak false positive (#2484)Victor Stinner2017-06-291-10/+3
| | | Only report a leak if each run leaks at least one memory block.
* bpo-30776: reduce regrtest -R false positives (#2422)Victor Stinner2017-06-271-3/+22
| | | | | | | | | | | | | | * Change the regrtest --huntrleaks checker to decide if a test file leaks or not. Require that each run leaks at least 1 reference. * Warmup runs are now completely ignored: ignored in the checker test and not used anymore to compute the sum. * Add an unit test for a reference leak. Example of reference differences previously considered a failure (leak) and now considered as success (success, no leak): [3, 0, 0] [0, 1, 0] [8, -8, 1]
* bpo-30764: regrtest: add --fail-env-changed option (#2402)Victor Stinner2017-06-262-1/+12
| | | | | | | | | | | | * bpo-30764: regrtest: change exit code on failure * Exit code 2 if failed tests ("bad") * Exit code 3 if interrupted * bpo-30764: regrtest: add --fail-env-changed option If the option is set, mark a test as failed if it alters the environment, for example if it creates a file without removing it.
* bpo-30523: regrtest --list-cases --match (#2401)Victor Stinner2017-06-261-1/+5
| | | | | | | * regrtest --list-cases now supports --match and --match-file options. Example: ./python -m test --list-cases -m FileTests test_os * --list-cases now also sets support.verbose to False to prevent messages to stdout when loading test modules. * Add support._match_test() private function.
* bpo-30523: regrtest: Add --list-cases option (#2238)mlouielu2017-06-163-8/+44
| | | | | | | | | | | * bpo-30523: regrtest: Add --list-cases option * bpo-30523: Enhance --list-cases * Add get_abs_module() function, use it in list_cases() * list_cases() now logs skipped tests into stderr * Remove unused doctest
* bpo-30284: Fix regrtest for out of tree build (#1481)Victor Stinner2017-06-141-1/+7
| | | | | | | | Use a build/ directory in the build directory, not in the source directory, since the source directory may be read-only and must not be modified. Fallback on the source directory if the build directory is not available (missing "abs_builddir" sysconfig variable).
generated by cgit v0.12 at 2026-01-07 00:28:40 (GMT)