summaryrefslogtreecommitdiffstats
path: root/Lib/test/libregrtest
Commit message (Collapse)AuthorAgeFilesLines
* [3.8] bpo-41043: Escape literal part of the path for glob(). (GH-20994). ↵Serhiy Storchaka2020-07-021-1/+1
| | | | | (GH-21277) (cherry picked from commit 935586845815f5b4c7814794413f6a812d4bd45f)
* bpo-40826: Fix test_repl.test_close_stdin() on Windows (GH-20779) (GH-20785) ↵Victor Stinner2020-06-101-26/+1
| | | | | | | | | | | | | | | | (GH-20787) test_repl.test_close_stdin() now calls support.suppress_msvcrt_asserts() to fix the test on Windows. * Move suppress_msvcrt_asserts() from test.libregrtest.setup to test.support. Make its verbose parameter optional: verbose=False by default. * SuppressCrashReport now uses SetErrorMode() of the msvcrt module, rather than using ctypes. * Remove also an unused variable (deadline) in wait_process(). (cherry picked from commit f6e58aefde2e57e4cb11ea7743955da53a3f1e80) (cherry picked from commit 4a4f660cfde8b683634c53e6214a6baa51de43b1)
* bpo-39983: Add test.support.print_warning() (GH-19683) (GH-19687)Victor Stinner2020-04-232-2/+3
| | | | | | | | | Log "Warning -- ..." test warnings into sys.__stderr__ rather than sys.stderr, to ensure to display them even if sys.stderr is captured. test.libregrtest.utils.print_warning() now calls test.support.print_warning(). (cherry picked from commit d663d34685e18588748569468c672763f4c73b3e)
* [3.8] Update libregrtest from master (GH-19516)Victor Stinner2020-04-144-11/+56
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * bpo-37531: regrtest now catchs ProcessLookupError (GH-16827) Fix a warning on a race condition on TestWorkerProcess.kill(): ignore silently ProcessLookupError rather than logging an useless warning. (cherry picked from commit a661392f8fb5ac4fc095aa1845d1eb7a25c4e9be) * bpo-38502: regrtest uses process groups if available (GH-16829) test.regrtest now uses process groups in the multiprocessing mode (-jN command line option) if process groups are available: if os.setsid() and os.killpg() functions are available. (cherry picked from commit ecb035cd14c11521276343397151929a94018a22) * bpo-37957: Allow regrtest to receive a file with test (and subtests) to ignore (GH-16989) When building Python in some uncommon platforms there are some known tests that will fail. Right now, the test suite has the ability to ignore entire tests using the -x option and to receive a filter file using the --matchfile filter. The problem with the --matchfile option is that it receives a file with patterns to accept and when you want to ignore a couple of tests and subtests, is too cumbersome to lists ALL tests that are not the ones that you want to accept and he problem with -x is that is not easy to ignore just a subtests that fail and the whole test needs to be ignored. For these reasons, add a new option to allow to ignore a list of test and subtests for these situations. (cherry picked from commit e0cd8aa70a3ce19c3d3712568940aa0cbd9aa97b) * regrtest: log timeout at startup (GH-19514) Reduce also worker timeout. (cherry picked from commit 4cf65a630a8d45bad3fe5cdc4c2632ec64e7ba27) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-37531: regrtest ignores output on timeout (GH-16659)Miss Islington (bot)2019-10-091-27/+37
| | | | | | | | | | | | | | bpo-37531, bpo-38207: On timeout, regrtest no longer attempts to call `popen.communicate() again: it can hang until all child processes using stdout and stderr pipes completes. Kill the worker process and ignores its output. Reenable test_regrtest.test_multiprocessing_timeout(). bpo-37531: Change also the faulthandler timeout of the main process from 1 minute to 5 minutes, for Python slowest buildbots. (cherry picked from commit 0ec618af98ac250a91ee9c91f8569e6df6772758) Co-authored-by: Victor Stinner <vstinner@python.org>
* [3.8] bpo-36670, regrtest: Fix WindowsLoadTracker() for partial line ↵Victor Stinner2019-10-034-68/+126
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (GH-16550) (GH-16560) * bpo-36670, regrtest: Fix WindowsLoadTracker() for partial line (GH-16550) WindowsLoadTracker.read_output() now uses a short buffer for incomplete line. (cherry picked from commit 3e04cd268ee9a57f95dc78d8974b21a6fac3f666) * bpo-36670: Enhance regrtest WindowsLoadTracker (GH-16553) The last line is now passed to the parser even if it does not end with a newline, but only if it's a valid value. (cherry picked from commit c65119d5bfded03f80a9805889391b66fa7bf551) * bpo-36670: Enhance regrtest (GH-16556) * Add log() method: add timestamp and load average prefixes to main messages. * WindowsLoadTracker: * LOAD_FACTOR_1 is now computed using SAMPLING_INTERVAL * Initialize the load to the arithmetic mean of the first 5 values of the Processor Queue Length value (so over 5 seconds), rather than 0.0. * Handle BrokenPipeError and when typeperf exit. * format_duration(1.5) now returns '1.5 sec', rather than '1 sec 500 ms' (cherry picked from commit 098e25672f1c3578855d5ded4f5147795c9ed956)
* bpo-36670: regrtest bug fixes (GH-16537)Miss Islington (bot)2019-10-021-10/+12
| | | | | | | | | | * Fix TestWorkerProcess.__repr__(): start_time is only valid if _popen is not None. * Fix _kill(): don't set _killed to True if _popen is None. * _run_process(): only set _killed to False after calling run_test_in_subprocess(). (cherry picked from commit 2ea71a07d0a720707094ee55f78fd232c40724bc) Co-authored-by: Victor Stinner <vstinner@python.org>
* bpo-36670: Multiple regrtest bugfixes (GH-16511)Miss Islington (bot)2019-10-013-133/+175
| | | | | | | | | | | | | | | | | | | * Windows: Fix counter name in WindowsLoadTracker. Counter names are localized: use the registry to get the counter name. Original change written by Lorenz Mende. * Regrtest.main() now ensures that the Windows load tracker is also killed if an exception is raised * TestWorkerProcess now ensures that worker processes are no longer running before exiting: kill also worker processes when an exception is raised. * Enhance regrtest messages and warnings: include test name, duration, add a worker identifier, etc. * Rename MultiprocessRunner to TestWorkerProcess * Use print_warning() to display warnings. Co-Authored-By: Lorenz Mende <Lorenz.mende@gmail.com> (cherry picked from commit 982bfa4da07b2e5749a0f4e68f99e972bcc3a549) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-37531: sync regrtest with master branch (GH-16285)Victor Stinner2019-09-192-17/+26
|
* bpo-37531: regrtest main process uses shorter timeout (GH-16220) (GH-16224)Miss Islington (bot)2019-09-171-6/+6
| | | | | | | | | When using multiprocesss (-jN), the main process now uses a timeout of 60 seconds instead of the double of the --timeout value. The buildbot server stops a job which does not produce any output in 1200 seconds. (cherry picked from commit 46b0b81220a23bc4aee5ba3ba67e8cf1b5df7960) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* [3.8] bpo-37936: Systematically distinguish rooted vs. unrooted in ↵Zachary Ware2019-09-111-3/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | .gitignore (GH-15823) (GH-15900) A root cause of bpo-37936 is that it's easy to write a .gitignore rule that's intended to apply to a specific file (e.g., the `pyconfig.h` generated by `./configure`) but actually applies to all similarly-named files in the tree (e.g., `PC/pyconfig.h`.) Specifically, any rule with no non-trailing slashes is applied in an "unrooted" way, to files anywhere in the tree. This means that if we write the rules in the most obvious-looking way, then * for specific files we want to ignore that happen to be in subdirectories (like `Modules/config.c`), the rule will work as intended, staying "rooted" to the top of the tree; but * when a specific file we want to ignore happens to be at the root of the repo (like `platform`), then the obvious rule (`platform`) will apply much more broadly than intended: if someone tries to add a file or directory named `platform` somewhere else in the tree, it will unexpectedly get ignored. That's surprising behavior that can make the .gitignore file's behavior feel finicky and unpredictable. To avoid it, we can simply always give a rule "rooted" behavior when that's what's intended, by systematically using leading slashes. Further, to help make the pattern obvious when looking at the file and minimize any need for thinking about the syntax when adding new rules: separate the rules into one group for each type, with brief comments identifying them. For most of these rules it's clear whether they're meant to be rooted or unrooted, but in a handful of cases I've only guessed. In that case the safer default (the choice that won't hide information) is the narrower, rooted meaning, with a leading slash. If for some of these the unrooted meaning is desired after all, it'll be easy to move them to the unrooted section at the top. (cherry picked from commit 455122a0094c8cfdf7e062eccc5e5b5885c75e8b) Co-authored-by: Greg Price <gnprice@gmail.com>
* bpo-37531: Enhance regrtest multiprocess timeout (GH-15345) (GH-15871)Victor Stinner2019-09-101-18/+63
| | | | | | | | | * Write a message when killing a worker process * Put a timeout on the second popen.communicate() call (after killing the process) * Put a timeout on popen.wait() call * Catch popen.kill() and popen.wait() exceptions (cherry picked from commit de2d9eed8bc628533e1628b843cc4c7a5010f6e5)
* [3.8] bpo-37531: Fix regrtest timeout for subprocesses (GH-15072) (GH-15279)Victor Stinner2019-08-145-18/+51
| | | | | | | | | | | | * bpo-37531: Fix regrtest timeout for subprocesses (GH-15072) Co-Authored-By: Joannah Nanjekye <joannah.nanjekye@ibm.com> (cherry picked from commit b0c8369c603633f445ccbb5ca7a8742145ff9eec) * bpo-36511: Fix failures in Windows ARM32 buildbot (GH-15181) (cherry picked from commit ed70a344b5fbddea85726ebc1964ee0cfdef9c40) Backport also minor fixes from master (fix typo, remove importlib import).
* bpo-37707: Exclude expensive unit tests from PGO task (GH-15009) (#15024)Miss Islington (bot)2019-07-301-0/+1
| | | | | | | | Mark some individual tests to skip when --pgo is used. The tests marked increase the PGO task time significantly and likely don't help improve optimization of the final executable. (cherry picked from commit 52a48e62c6a94577152f9301bbe5f3bc806cfcf1) Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
* bpo-36044: Avoid warnings in Windows PGO build and add lzma, bz2 and sqlite ↵Miss Islington (bot)2019-07-291-0/+3
| | | | | | | | | | coverage (GH-14985) https://bugs.python.org/issue36044 Automerge-Triggered-By: @zooba (cherry picked from commit e1b900247227dad49d8231f1d028872412230ab4) Co-authored-by: Steve Dower <steve.dower@python.org>
* Only setup PGO tests when --pgo is enabled. (GH-14927)Miss Islington (bot)2019-07-241-2/+3
| | | | | (cherry picked from commit f0807ab24cbd13163bff7e0de3a97fe83584b80d) Co-authored-by: Gregory P. Smith <greg@krypto.org>
* bpo-36044: Reduce number of unit tests run for PGO build (GH-14702)Miss Islington (bot)2019-07-223-1/+61
| | | | | | | | | | | | | | Reduce the number of unit tests run for the PGO generation task. This speeds up the task by a factor of about 15x. Running the full unit test suite is slow. This change may result in a slightly less optimized build since not as many code branches will be executed. If you are willing to wait for the much slower build, the old behavior can be restored using './configure [..] PROFILE_TASK="-m test --pgo-extended"'. We make no guarantees as to which PGO task set produces a faster build. Users who care should run their own relevant benchmarks as results can depend on the environment, workload, and compiler tool chain. (cherry picked from commit 4e16a4a3112161a5c6981c0588142d4a4673a934) Co-authored-by: Neil Schemenauer <nas-github@arctrix.com>
* bpo-34602: Avoid failures setting macOS stack resource limit (GH-14546)Miss Islington (bot)2019-07-021-16/+0
| | | | | | | | | | | | | | | | | | | | Under some conditions the earlier fix for bpo-18075, "Infinite recursion tests triggering a segfault on Mac OS X", now causes failures on macOS when attempting to change stack limit with resource.setrlimit resource.RLIMIT_STACK, like regrtest does when running the test suite. The reverted change had specified a non-default stack size when linking the python executable on macOS. As of macOS 10.14.4, the previous code causes a hard failure when running tests, although similar failures had been seen under some conditions under some earlier systems. Reverting the change to the interpreter stack size at link time helped for release builds but caused some tests to fail when built --with-pydebug. Try the opposite approach: continue to build the interpreter with an increased stack size on macOS and remove the failing setrlimit call in regrtest initialization. This will definitely avoid the resource.RLIMIT_STACK error and should have no, or fewer, side effects. (cherry picked from commit 5bbbc733e6cc0804f19b071944af8d4719e26ae6) Co-authored-by: Ned Deily <nad@python.org>
* Remove unused imports in tests (GH-14518) (GH-14520)Victor Stinner2019-07-011-1/+0
| | | (cherry picked from commit 8f4ef3b019ce380022018587571b0f970e668de3)
* bpo-37359: Fix regrtest --cleanup (GH-14336)Miss Islington (bot)2019-06-241-1/+1
| | | | | (cherry picked from commit 9bbf4d7083a819cbcee2a6cd3df2802d4c50f734) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-37359: Add --cleanup option to python3 -m test (GH-14332) (GH-14334)Victor Stinner2019-06-242-14/+41
| | | | | | | * regrtest: Add --cleanup option to remove "test_python_*" directories of previous failed test jobs. * Add "make cleantest" to run "python3 -m test --cleanup". (cherry picked from commit 47fbc4e45b35b3111e2d947a66490a43ac21d363)
* bpo-36842: Fix reference leak in tests by running out-of-proc (GH-13556)Steve Dower2019-05-291-21/+26
|
* bpo-36842: Implement PEP 578 (GH-12613)Steve Dower2019-05-231-0/+6
| | | Adds sys.audit, sys.addaudithook, io.open_code, and associated C APIs.
* bpo-36915: regrtest always remove tempdir of worker processes (GH-13312)Victor Stinner2019-05-142-44/+66
| | | | | | | | | When using multiprocessing (-jN option), worker processes now create their temporary directory inside the temporary directory of the main process. So the main process is able to remove temporary directories of worker processes even if they crash or when they are killed by regrtest on KeyboardInterrupt (CTRL+c). Rework also how multiprocessing arguments are parsed in main.py.
* bpo-36719: Fix regrtest MultiprocessThread (GH-13301)Victor Stinner2019-05-141-4/+55
| | | | MultiprocessThread.kill() now closes stdout and stderr to prevent popen.communicate() to hang.
* bpo-36719: regrtest -jN no longer stops on crash (GH-13231)Victor Stinner2019-05-134-18/+46
| | | | | | | "python3 -m test -jN ..." now continues the execution of next tests when a worker process crash (CHILD_ERROR state). Previously, the test suite stopped immediately. Use --failfast to stop at the first error. Moreover, --forever now also implies --failfast.
* bpo-36719: regrtest closes explicitly WindowsLoadTracker (GH-12965)Victor Stinner2019-04-262-14/+29
| | | | Regrtest.finalize() now closes explicitly the WindowsLoadTracker instance.
* bpo-36719: Fix regrtest re-run (GH-12964)Victor Stinner2019-04-262-13/+13
| | | | | Properly handle a test which fail but then pass. Add test_rerun_success() unit test.
* bpo-36719: regrtest always detect uncollectable objects (GH-12951)Victor Stinner2019-04-263-63/+27
| | | | | | | regrtest now always detects uncollectable objects. Previously, the check was only enabled by --findleaks. The check now also works with -jN/--multiprocess N. --findleaks becomes a deprecated alias to --fail-env-changed.
* bpo-36725: Refactor regrtest multiprocessing code (GH-12961)Victor Stinner2019-04-263-150/+200
| | | | | | | | | | | | | | | | Rewrite run_tests_multiprocess() function as a new MultiprocessRunner class with multiple methods to better report errors and stop immediately when needed. Changes: * Worker processes are now killed immediately if tests are interrupted or if a test does crash (CHILD_ERROR): worker processes are killed. * Rewrite how errors in a worker thread are reported to the main thread. No longer ignore BaseException or parsing errors silently. * Remove 'finished' variable: use worker.is_alive() instead * Always compute omitted tests. Add Regrtest.get_executed() method.
* bpo-36725: regrtest: add TestResult type (GH-12960)Victor Stinner2019-04-267-195/+271
| | | | | | | | | | | | | | | * Add TestResult and MultiprocessResult types to ensure that results always have the same fields. * runtest() now handles KeyboardInterrupt * accumulate_result() and format_test_result() now takes a TestResult * cleanup_test_droppings() is now called by runtest() and mark the test as ENV_CHANGED if the test leaks support.TESTFN file. * runtest() now includes code "around" the test in the test timing * Add print_warning() in test.libregrtest.utils to standardize how libregrtest logs warnings to ease parsing the test output. * support.unload() is now called with abstest rather than test_name * Rename 'test' variable/parameter to 'test_name' * dash_R(): remove unused the_module parameter * Remove unused imports
* bpo-36638: Fix WindowsLoadTracker exception on some Windows versions (GH-12849)Paul Monson2019-04-181-2/+7
|
* bpo-36560: regrtest: don't collect the GC twice (GH-12747)Victor Stinner2019-04-091-2/+4
| | | | | | dash_R() function of libregrtest doesn't call support.gc_collect() directly anymore: it's already called by dash_R_cleanup(). Call dash_R_cleanup() before starting the loop.
* bpo-36560: Fix reference leak hunting in regrtest (GH-12744)Victor Stinner2019-04-093-33/+42
| | | | | | | | | | | | | | | | | | Fix reference leak hunting in regrtest: compute also deltas (of reference count, allocated memory blocks, file descriptor count) during warmup, to ensure that everything is initialized before starting to hunt reference leaks. Other changes: * Replace gc.collect() with support.gc_collect() * Move calls to read memory statistics from dash_R_cleanup() to dash_R() * Pass regrtest 'ns' to dash_R() * dash_R() is now more quiet with --quiet option (don't display progress). * Precompute the full range for "for it in range(repcount):" to ensure that the iteration doesn't allocate anything new. * dash_R() now is responsible to call warm_caches().
* bpo-34060: Report system load when running test suite for Windows (GH-8357)Ammar Askar2019-04-092-2/+115
| | | | | | | | | | | | | | | While Windows exposes the system processor queue length, the raw value used for load calculations on Unix systems, it does not provide an API to access the averaged value. Hence to calculate the load we must track and average it ourselves. We can't use multiprocessing or a thread to read it in the background while the tests run since using those would conflict with test_multiprocessing and test_xxsubprocess. Thus, we use Window's asynchronous IO API to run the tracker in the background with it sampling at the correct rate. When we wish to access the load we check to see if there's new data on the stream, if there is, we update our load values.
* bpo-36565: Fix libregrtest for Python without builtin _abc (GH-12733)Victor Stinner2019-04-081-2/+6
| | | | Fix reference hunting (``python3 -m test -R 3:3``) when Python has no built-in abc module: fix _get_dump() reimplementation of libregrtest.
* Clean up code which checked presence of os.{stat,lstat,chmod} (#11643)Anthony Sottile2019-02-251-4/+2
|
* bpo-35683: Improve Azure Pipelines steps (GH-11493)Steve Dower2019-01-221-0/+4
|
* Fix typo in test module usage message (GH-11374)sth2018-12-311-1/+1
| | | | | A minor typo in the output of `python -m test -h`. A space was missing in between two words. howmuch -> how much
* bpo-35513, unittest: TextTestRunner uses time.perf_counter() (GH-11180)Victor Stinner2018-12-171-2/+2
| | | | | | | TextTestRunner of unittest.runner now uses time.perf_counter() rather than time.time() to measure the execution time of a test: time.time() can go backwards, whereas time.perf_counter() is monotonic. Similar change made in libregrtest, pprint and random.
* bpo-34279, regrtest: Issue a warning if no tests have been executed (GH-10150)Pablo Galindo2018-11-292-1/+17
|
* Add --tempdir option for test run (GH-10322)Steve Dower2018-11-172-10/+8
|
* bpo-34582: Adds JUnit XML output for regression tests (GH-9210)Steve Dower2018-09-184-20/+68
|
* bpo-34605, libregrtest: Rename --slaveargs to --worker-args (GH-9099)Victor Stinner2018-09-073-11/+11
| | | Rename also run_tests_slave() to run_tests_worker().
* bpo-33873: Fix bug in `runtest.py` and add checks for invalid `-R` ↵Pablo Galindo2018-06-262-1/+11
| | | | | | | | parameters (GH-7735) Fix bug in `Lib/test/libregrtest/runtest.py` that makes running tests an extra time than the specified number of runs. Add check for invalid --huntrleaks/-R parameters.
* bpo-33873: regrtest: Add warning on -R 1:3 (GH-7736)Pablo Galindo2018-06-161-0/+7
| | | regrtest: Add warning when using less than 3 warmup runs like -R 1:3.
* bpo-33718: regrtest: use format_duration() to display failed tests (GH-7686)Victor Stinner2018-06-142-10/+19
| | | | | * Enhance also format_duration(): work on integers and rounds towards +infinity (math.ceil). * Write unit tests on format_duration()
* bpo-33718: regrtest: use "xxx then yyy" result if re-run (GH-7521)Victor Stinner2018-06-081-1/+7
| | | | | | If tests are re-run, use "xxx then yyy" result format (ex: "FAILURE then SUCCESS") to show that some failing tests have been re-run. Add also test_regrtest.test_rerun_fail() test.
* bpo-33562: Check the global asyncio event loop policy isn't set after any ↵Brett Cannon2018-06-021-0/+7
| | | | tests (GH-7328)
* bpo-33718: regrtest: enhance "running:" formatting (GH-7292)Victor Stinner2018-06-013-43/+51
| | | | | * "running:" progress: Format number of seconds as hours and minutes * format_duration(): count also minutes as hours * Create Lib/test/libregrtest/utils.py
generated by cgit v0.12 at 2025-04-05 13:04:41 (GMT)