summaryrefslogtreecommitdiffstats
path: root/Lib/test/libregrtest
Commit message (Collapse)AuthorAgeFilesLines
* gh-111863: Rename `Py_NOGIL` to `Py_GIL_DISABLED` (#111864)Hugo van Kemenade2023-11-201-1/+1
| | | Rename Py_NOGIL to Py_GIL_DISABLED
* gh-110722: Make `-m test -T -j` use sys.monitoring (GH-111710)Łukasz Langa2023-11-107-22/+68
| | | | | | | | | | | Now all results from worker processes are aggregated and displayed together as a summary at the end of a regrtest run. The traditional trace is left in place for use with sequential in-process test runs but now raises a warning that those numbers are not precise. `-T -j` requires `--with-pydebug` as it relies on `-Xpresite=`.
* gh-111881: Import doctest lazily in libregrtest (#111884)Victor Stinner2023-11-091-5/+8
| | | | | In most cases, doctest is not needed. So don't always import it at startup. The change reduces the number of modules already imported when a test is run.
* gh-110367: Make regrtest --verbose3 compatible with --huntrleaks -jN (#111577)Victor Stinner2023-11-011-2/+8
| | | | | "./python -m test -j1 -R 3:3 --verbose3" now works as expected, since run_single_test() does not replace sys.stdout with StringIO in this case.
* gh-111165: Add missed "support." prefix for "verbose" (GH-111327)Serhiy Storchaka2023-10-251-1/+1
|
* gh-111165: Move test running code from test.support to libregrtest (GH-111166)Serhiy Storchaka2023-10-257-11/+338
| | | | Remove no longer used functions run_unittest() and run_doctest() from the test.support module.
* gh-110918: regrtest: allow to intermix --match and --ignore options (GH-110919)Serhiy Storchaka2023-10-218-42/+36
| | | | | | | Test case matching patterns specified by options --match, --ignore, --matchfile and --ignorefile are now tested in the order of specification, and the last match determines whether the test case be run or ignored.
* gh-110932: Fix regrtest for SOURCE_DATE_EPOCH (#111143)Victor Stinner2023-10-212-9/+14
| | | | If the SOURCE_DATE_EPOCH environment variable is defined, use its value as the random seed.
* regrtest: Prepend 'use' options in --{fast,slow}-ci (GH-110363)Zachary Ware2023-10-151-4/+6
| | | This allows individual resources to be disabled without having to explicitly re-enable all others.
* regrtest: --fast-ci/slow-ci no longer imply --rerun-fail (#110849)Victor Stinner2023-10-141-1/+0
|
* gh-108303: Move all inspect test files to `test_inspect/` (#109607)Nikita Sobolev2023-10-101-0/+1
|
* Add support.MS_WINDOWS constant (#110446)Victor Stinner2023-10-064-7/+6
|
* gh-110167: Increase support.LOOPBACK_TIMEOUT to 10 seconds (#110413)Victor Stinner2023-10-051-0/+2
| | | | | | Increase support.LOOPBACK_TIMEOUT from 5 to 10 seconds. Also increase the timeout depending on the --timeout option. For example, for a test timeout of 40 minutes (ARM Raspbian 3.x), use LOOPBACK_TIMEOUT of 20 seconds instead of 5 seconds before.
* gh-110367: Enhance regrtest -jN --verbose3 (#110368)Victor Stinner2023-10-051-2/+13
| | | | | | When using worker processes (-jN) with --verbose3 option, regrtest can now display the worker output even if a worker process does crash. Previously, sys.stdout and sys.stderr were replaced and so the worker output was lost on a crash.
* gh-109276: regrtest: shorter list of resources (#110326)Victor Stinner2023-10-042-17/+43
|
* gh-110171: `libregrtest` always sets `random.seed` (#110172)Nikita Sobolev2023-10-043-14/+15
|
* gh-110276: Run `test_str`, not `test_unicode`, as part of the PGO build ↵Alex Waygood2023-10-031-1/+1
| | | | | (#110277) `test_unicode` does not exist
* gh-110164: regrtest disables random if SOURCE_DATE_EPOCH (#110168)Victor Stinner2023-10-011-2/+7
| | | | If the SOURCE_DATE_EPOCH environment variable is defined, regrtest now disables randomization of tests.
* gh-109649: Use os.process_cpu_count() (#110165)Victor Stinner2023-10-012-1/+4
| | | | | | | | | | | | | | | | | | | | Replace os.cpu_count() with os.process_cpu_count() in modules: * compileall * concurrent.futures * multiprocessing Replace os.cpu_count() with os.process_cpu_count() in programs: * _decimal deccheck.py test * freeze.py * multissltests.py * python -m test (regrtest) * wasm_build.py Other changes: * test.pythoninfo logs os.process_cpu_count(). * regrtest gets os.process_cpu_count() / os.cpu_count() in headers.
* gh-110152: regrtest handles cross compilation and HOSTRUNNER (#110156)Victor Stinner2023-09-302-18/+119
| | | | | | | | * _add_python_opts() now handles cross compilation and HOSTRUNNER. * display_header() now tells if Python is cross-compiled, display HOSTRUNNER, and get the host platform. * Remove Tools/scripts/run_tests.py script. * Remove "make hostrunnertest": use "make buildbottest" or "make test" instead.
* gh-109276: regrtest: add WORKER_FAILED state (#110148)Victor Stinner2023-09-304-28/+73
| | | | | | | | | Rename WORKER_ERROR to WORKER_BUG. Add WORKER_FAILED state: it does not stop the manager, whereas WORKER_BUG does. Change also TestResults.display_result() order: display failed tests at the end, the important important information. WorkerThread now tries to get the signal name for negative exit code.
* gh-109566: regrtest --fast-ci no longer enables --nowindows (#110121)Victor Stinner2023-09-291-5/+1
| | | | The --nowindows option is deprecated and does nothing but logs a warning.
* gh-109566: regrtest _add_python_opts() handles KeyboardInterrupt (#110062)Victor Stinner2023-09-294-14/+33
| | | | | | | | | | In the subprocess code path, wait until the child process completes with a timeout of EXIT_TIMEOUT seconds. Fix create_worker_process() regression: use start_new_session=True if USE_PROCESS_GROUP is true. WorkerThread.wait_stopped() uses a timeout of 60 seconds, instead of 30 seconds.
* gh-109972: Split test_gdb.py into test_gdb package (#109977)Victor Stinner2023-09-281-0/+1
| | | | | | | | | | | | | | Split test_gdb.py file into a test_gdb package made of multiple tests, so tests can now be run in parallel. * Create Lib/test/test_gdb/ directory. * Split test_gdb.py into multiple files in Lib/test/test_gdb/ directory. * Move Lib/test/gdb_sample.py to Lib/test/test_gdb/ directory. Update get_sample_script(): use __file__ to locate gdb_sample.py. * Move gdb_has_frame_select() and HAS_PYUP_PYDOWN to test_misc.py. * Explicitly skip test_gdb on Windows. Previously, test_gdb was skipped even if gdb was available because of gdb_has_frame_select().
* gh-109566: regrtest doesn't enable --rerun if --python is used (#109969)Victor Stinner2023-09-271-1/+2
| | | | regrtest: --fast-ci and --slow-ci options no longer enable --rerun if the --python option is used.
* gh-109615: Fix support test_copy_python_src_ignore() (#109958)Victor Stinner2023-09-271-5/+7
| | | | | | | | | | Fix the test when run on an installed Python: use "abs_srcdir" of sysconfig, and skip the test if the Python source code cannot be found. * Tools/patchcheck/patchcheck.py, Tools/freeze/test/freeze.py and Lib/test/libregrtest/utils.py now first try to get "abs_srcdir" from sysconfig, before getting "srcdir" from sysconfig. * test.pythoninfo logs sysconfig "abs_srcdir".
* gh-109566: Fix regrtest Python options for WASM/WASI (#109954)Victor Stinner2023-09-272-4/+11
| | | | | WASM and WASI buildbots use multiple PYTHON environment variables such as PYTHONPATH and _PYTHON_HOSTRUNNER. Don't use -E if the --python=COMMAND option is used.
* gh-109615: Fix test_tools.test_freeze SRCDIR (#109935)Victor Stinner2023-09-271-1/+1
| | | | | | | | | Fix copy_source_tree() function of test_tools.test_freeze: * Don't copy SRC_DIR/build/ anymore. This directory is modified by other tests running in parallel. * Add test.support.copy_python_src_ignore(). * Use sysconfig to get the source directory. * Use sysconfig.get_config_var() to get CONFIG_ARGS variable.
* gh-109566: Fix regrtest code adding Python options (#109926)Victor Stinner2023-09-262-26/+46
| | | | | | * On Windows, use subprocess.run() instead of os.execv(). * Only add needed options * Rename reexec parameter to _add_python_opts. * Rename --no-reexec option to --dont-add-python-opts.
* gh-109276, gh-109508: Fix libregrtest stdout (#109903)Victor Stinner2023-09-263-53/+26
| | | | | | | | | Remove replace_stdout(): call sys.stdout.reconfigure() instead of set the error handler to backslashreplace. display_header() logs an empty line and flush stdout. Remove encoding workaround in display_header() since stdout error handler is now set to backslashreplace earlier.
* gh-109566: regrtest reexecutes the process (#109909)Victor Stinner2023-09-262-4/+42
| | | | | | | | | | | | | When --fast-ci or --slow-ci option is used, regrtest now replaces the current process with a new process to add "-u -W default -bb -E" options to Python. Changes: * PCbuild/rt.bat and Tools/scripts/run_tests.py no longer need to add "-u -W default -bb -E" options to Python: it's now done by regrtest. * Fix Tools/scripts/run_tests.py: flush stdout before replacing the process. Previously, buffered messages were lost.
* gh-109566, regrtest: Add --fast-ci and --slow-ci options (#109570)Victor Stinner2023-09-264-5/+65
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add --fast-ci and --slow-ci options to libregrtest: * --fast-ci uses a default timeout of 10 minutes and "-u all,-cpu" (skip slowest tests). * --slow-ci uses a default timeout of 20 minues and "-u all" (run all tests). * regrtest header now lists test resources. * Makefile changes: * "make test", "make hostrunnertest" and "make coverage-report" now use --fast-ci option and TESTTIMEOUT variable. * "make buildbottest" now uses "--slow-ci". Remove options which became redundant with "--slow-ci". * "make testall" and "make testuniversal" now use --slow-ci option and TESTTIMEOUT variable. * "make testall" now uses "find -exec rm ..." instead of "find ... -print|xargs rm ...", same as "make clean". * GitHub Actions workflow: * Ubuntu and Address Sanitizer jobs now use "make test". Remove options which became redundant with "--fast-ci". * Windows jobs now use --fast-ci option. * Use -j0 to detect the number of CPUs. * Set Makefile TESTTIMEOUT default to an empty string, since --slow-ci and --fast-ci use different default timeout. It's now accepted to pass "--timeout=" to regrtest: treated as not timeout. * Tools/scripts/run_tests.py now uses --fast-ci option. * Tools/buildbot/test.bat now uses --slow-ci option. Remove --timeout=1200 option, redundant with --slow-ci.
* gh-109739: regrtest disables load tracker if refleak (#109871)Victor Stinner2023-09-261-3/+13
| | | | regrtest: Fix reference leak check on Windows. Disable the load tracker on Windows in the reference leak check mode (-R option).
* gh-109276: regrtest re-runs "env changed" tests (#109831)Victor Stinner2023-09-251-5/+7
| | | | When a test fails with "env changed" and --rerun option is used, the test is now re-run in verbose mode in a fresh process.
* gh-109276: Enhance libregrtest results (#109828)Victor Stinner2023-09-251-38/+29
| | | | | | * Factorize code listing "bad / env changed / ..." tests. * Add TestResults.is_all_good() method. * Move "All 400 tests OK." to the end * Move "Test suite interrupted by signal SIGINT." to the end.
* gh-103053: Skip test_freeze_simple_script() on PGO build (#109591)Victor Stinner2023-09-201-10/+2
| | | | | | | | | | Skip test_freeze_simple_script() of test_tools.test_freeze if Python is built with "./configure --enable-optimizations", which means with Profile Guided Optimization (PGO): it just makes the test too slow. The freeze tool is tested by many other CIs with other (faster) compiler flags. test.pythoninfo now gets also get_build_info() of test.libregrtests.utils.
* gh-109508: Fix libregrtest formatting of getcwd() (#109537)Victor Stinner2023-09-181-1/+10
|
* gh-109413: Improve mypy config for libregrtest (#109518)Alex Waygood2023-09-181-23/+9
| | | Improve the mypy config file for libregrtest
* gh-109413: Fix some trivial mypy nitpicks in libregrtest (#109454)Alex Waygood2023-09-154-13/+11
|
* gh-108303: Move `test_future` into its own test_future_stmt subdir (#109368)Nikita Sobolev2023-09-151-0/+1
|
* gh-109425: regrtest decodes worker stdout with backslashreplace (#109428)Victor Stinner2023-09-141-1/+6
| | | | | libregrtest now decodes stdout of test worker processes with the "backslashreplace" error handler to log corrupted stdout, instead of failing with an error and not logging the stdout.
* gh-109413: libregrtest: Add and improve type annotations (#109405)Alex Waygood2023-09-1413-33/+83
|
* gh-109402: Fix regrtest findtests() (#109403)Victor Stinner2023-09-141-4/+9
| | | | Check for the full module name in SPLITTESTDIRS, not the relative module name.
* gh-109276: libregrtest: fix work dir on WASI (#109356)Victor Stinner2023-09-132-11/+21
| | | | | | | | On WASI platform, get_temp_dir() should behave differently since the parent process is a WASI process and uses a different get_temp_dir() path. Fix also WorkerThread._runtest(): don't read JSON file if the worker process exit code is non-zero.
* gh-109276: libregrtest: WASM use stdout for JSON (#109355)Victor Stinner2023-09-133-32/+35
| | | | | | | | | | On Emscripten and WASI, or if --python command line is used, libregrtest now writes JSON into stdout, instead of using a name file. * Add JsonFileType.STDOUT. * Remove JsonFileType.FILENAME. * test.pythoninfo logs environment variables related to cross-compilation and running Python on Emscripten/WASI.
* gh-109276: libregrtest: WASM use filename for JSON (#109340)Victor Stinner2023-09-125-133/+241
| | | | | | | | | | | | | | | | On Emscripten and WASI platforms, or if --python command line option is used, libregrtest now uses a filename for the JSON file. Emscripten and WASI buildbot workers run the main test process with a different Python (Linux) which spawns Emscripten/WASI processes using the command specified in --python command line option. Passing a file descriptor from the parent process to the child process doesn't work in this case. * Add JsonFile and JsonFileType classes * Add RunTests.json_file_use_filename() method. * Add a test in test_regrtest on the --python command line option. * test_regrtest: add parallel=False parameter. * Split long RunWorkers._runtest() function into sub-functions.
* gh-109276: libregrtest: fix worker working dir (#109313)Victor Stinner2023-09-124-16/+24
| | | | | | | | | | | | | | Fix Emscripten and WASI: start the test worker process in the Python source code directory, where 'python.js' and 'python.wasm' can be found. Then worker_process() changes to a temporary directory created to run tests. * create_worker_process() uses os_helper.SAVEDCWD as cwd. * worker_process() uses get_temp_dir() as the parent directory for get_work_dir(). * Don't use plural but singual for "test" in "Run 1 test ..." message. * Remove unused imports. * Add WORK_DIR_PREFIX and WORKER_WORK_DIR_PREFIX constants.
* gh-109276: libregrtest: limit number workers (#109288)Victor Stinner2023-09-124-5/+40
| | | | | | | | | Don't spawn more threads than the number of jobs: these worker threads would never get anything to do. * Add the number of tests in "Run ... tests in ..." message. * Add RunTests.get_jobs() method. * Add plural() function. * count() uses f-string.
* gh-109276: libregrtest calls random.seed() before each test (#109279)Victor Stinner2023-09-123-23/+38
| | | | | | | | | | | | | libregrtest now calls random.seed() before running each test file when -r/--randomize command line option is used. Moreover, it's also called in worker processes. It should help to make tests more deterministic. Previously, it was only called once in the main process before running all test files and it was not called in worker processes. * Convert some f-strings to regular strings in test_regrtest when f-string is not needed. * Remove unused all_methods variable from test_regrtest. * Add RunTests members are now mandatory.
* gh-109276: libregrtest only checks saved_test_environment() once (#109278)Victor Stinner2023-09-122-15/+12
| | | There is no need to check for environment changes twice.
generated by cgit v0.12 at 2025-02-16 12:20:32 (GMT)