summaryrefslogtreecommitdiffstats
path: root/Lib/test/_test_multiprocessing.py
Commit message (Collapse)AuthorAgeFilesLines
* gh-126417: validate ABC methods on multiprocessing proxy types (#126454)Stephen Morton2024-11-111-0/+22
| | | | | Checks that appropriate dunder __ methods exist on the dict and list proxy types. Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
* gh-117378: Only run the new multiprocessing SysPath test when appropriate ↵Gregory P. Smith2024-11-101-29/+30
| | | | | | | | | (GH-126635) The first version had it running two forkserver and one spawn tests underneath each of the _fork, _forkserver, and _spawn test suites that build off the generic one. This adds to the existing complexity of the multiprocessing test suite by offering BaseTestCase classes another attribute to control which suites they are invoked under. Practicality vs purity here. :/ Net result: we don't over-run the new test and their internal logic is simplified.
* gh-117378: Fix multiprocessing forkserver preload sys.path inheritance. ↵Gregory P. Smith2024-11-091-0/+78
| | | | | | | | | | | | | | | | | | | | | | | | | | | (GH-126538) gh-117378: Fix multiprocessing forkserver preload sys.path inheritance. `sys.path` was not properly being sent from the parent process when launching the multiprocessing forkserver process to preload imports. This bug has been there since the forkserver start method was introduced in Python 3.4. It was always _supposed_ to inherit `sys.path` the same way the spawn method does. Observable behavior change: A `''` value in `sys.path` will now be replaced in the forkserver's `sys.path` with an absolute pathname `os.path.abspath(os.getcwd())` saved at the time that `multiprocessing` was imported in the parent process as it already was when using the spawn start method. **This will only be observable during forkserver preload imports**. The code invoked before calling things in another process already correctly sets `sys.path`. Which is likely why this went unnoticed for so long as a mere performance issue in some configurations. A workaround for the bug on impacted Pythons is to set PYTHONPATH in the environment before multiprocessing's forkserver process was started. Not perfect as that is then inherited by other children, etc, but likely good enough for many people's purposes. Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* gh-125679: multiprocessing Lock and RLock - fix invalid representation ↵Duprat2024-11-071-0/+122
| | | | string on MacOSX. (#125680)
* gh-126417: Register multiprocessing proxy types to an appropriate ↵Stephen Morton2024-11-051-0/+9
| | | | collections.abc class (#126419)
* gh-125620: Skip check_resource_tracker_death on NetBSD due to long wait for ↵Furkan Onder2024-10-181-0/+2
| | | | | SIGKILL process termination (GH-125621) * Skip test_resource_tracker_sigkill on NetBSD
* gh-84559: Change the multiprocessing start method default to `forkserver` ↵Gregory P. Smith2024-09-261-5/+19
| | | | | | | | (GH-101556) Change the default multiprocessing start method away from fork to forkserver or spawn on the remaining platforms where it was fork. See the issue for context. This makes the default far more thread safe (other than for people spawning threads at import time... - don't do that!). Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
* gh-120012: clarify the behaviour of `multiprocessing.Queue.empty` on closed ↵Bénédikt Tran2024-06-131-0/+26
| | | | | | | | queues. (GH-120102) * improve doc for `multiprocessing.Queue.empty` * add tests for checking emptiness of queues Co-authored-by: Gregory P. Smith <greg@krypto.org>
* gh-103134: Update multiprocessing.managers.ListProxy and DictProxy (GH-103133)Roy Hyunjin Han2024-05-201-3/+43
|
* gh-119064: Use os_helper.FakePath instead of pathlib.Path in tests (GH-119065)Serhiy Storchaka2024-05-161-3/+3
|
* gh-83856: Honor atexit for all multiprocessing start methods (GH-114279)Tian Gao2024-05-031-0/+23
| | | | | Use atexit for all multiprocessing start methods to cleanup. See the GH-114279 PR discussion and related issue for details as to why.
* gh-118413: Fix test_release_task_refs on free-threaded build (#118494)Sam Gross2024-05-021-2/+1
| | | | | | The `time.sleep()` call should happen before the GC to give the worker threads time to clean-up their remaining references to objs. Additionally, use `support.gc_collect()` instead of `gc.collect()` just in case the extra GC calls matter.
* gh-118413: Temporarily skip `test_release_task_refs` in free-threaded builds ↵Sam Gross2024-05-011-0/+1
| | | | (#118491)
* gh-116622: Test updates for Android (#117299)Malcolm Smith2024-04-051-1/+1
| | | | | | | - re-enable test_fcntl_64_bit on Linux aarch64, but disable it on all Android ABIs - use support.setswitchinterval in all relevant tests - skip test_fma_zero_result on Android x86_64 - accept EACCES when calling os.get_terminal_size on Android
* gh-88118: Fix some test_multiprocessing flakiness. (#116434)Gregory P. Smith2024-03-061-5/+10
| | | | | | | | | | | | | | | | | | Fix some test_multiprocessing flakiness. Potentially introduced by https://github.com/python/cpython/pull/25845 not joining that thread likely leads to recently observed "environment changed" logically passing but overall failing tests seen on some buildbots similar to: ``` 1 test altered the execution environment (env changed): test.test_multiprocessing_fork.test_processes 2 re-run tests: test.test_multiprocessing_fork.test_processes test.test_multiprocessing_forkserver.test_processes ```
* bpo-43952: Fix multiprocessing Listener authkey bug (GH-25845)Miguel Brito2024-02-271-0/+19
| | | | Listener.accept() no longer hangs when authkey is an empty bytes object.
* gh-115886: Handle embedded null characters in shared memory name (GH-115887)Serhiy Storchaka2024-02-251-1/+16
| | | | shm_open() and shm_unlink() now check for embedded null characters in the name and raise an error instead of silently truncating it.
* gh-104090: Add exit code to multiprocessing ResourceTracker (GH-115410)Petr Viktorin2024-02-211-1/+34
| | | | | | | | | | | | | This builds on https://github.com/python/cpython/pull/106807, which adds a return code to ResourceTracker, to make future debugging easier. Testing this “in situ” proved difficult, since the global ResourceTracker is involved in test infrastructure. So, the tests here create a new instance and feed it fake data. --------- Co-authored-by: Yonatan Bitton <yonatan.bitton@perception-point.io> Co-authored-by: Yonatan Bitton <bityob@gmail.com> Co-authored-by: Antoine Pitrou <antoine@python.org>
* gh-89240: Enable multiprocessing on Windows to use large process pools ↵Steve Dower2024-02-131-0/+18
| | | | | | | (GH-107873) We add _winapi.BatchedWaitForMultipleObjects to wait for larger numbers of handles. This is an internal module, hence undocumented, and should be used with caution. Check the docstring for info before using BatchedWaitForMultipleObjects.
* gh-113205: test_multiprocessing.test_terminate: Give tasks a chance to start ↵Serhiy Storchaka2024-01-241-0/+1
| | | | (GH-114249)
* gh-113205: test_multiprocessing.test_terminate: Test the API on threadpools ↵Petr Viktorin2024-01-181-3/+8
| | | | | | | | | | | | | | | | (#114186) gh-113205: test_multiprocessing.test_terminate: Test the API works on threadpools Threads can't be forced to terminate (without potentially corrupting too much state), so the expected behaviour of `ThreadPool.terminate` is to wait for the currently executing tasks to finish. The entire test was skipped in GH-110848 (0e9c364f4ac18a2237bdbac702b96bcf8ef9cb09). Instead of skipping it entirely, we should ensure the API eventually succeeds: use a shorter timeout. For the record: on my machine, when the test is un-skipped, the task manages to start in about 1.5% cases.
* gh-113421: Fix multiprocessing logger for "%(filename)s" (GH-113423)Xu Song2023-12-241-0/+23
|
* gh-82300: Add track parameter to multiprocessing.shared_memory (#110778)pan3242023-12-051-0/+53
| | | | | | | | | | Add a track parameter to shared memory to allow resource tracking via the side-launched resource tracker process to be disabled on platforms that use it (POSIX). This allows people who do not want automated cleanup at process exit because they are using the shared memory with processes not participating in Python's resource tracking to use the shared_memory API. Co-authored-by: Łukasz Langa <lukasz@langa.pl> Co-authored-by: Guido van Rossum <gvanrossum@gmail.com> Co-authored-by: Antoine Pitrou <pitrou@free.fr> Co-authored-by: Gregory P. Smith <greg@krypto.org>
* GH-110829: Ensure Thread.join() joins the OS thread (#110848)Antoine Pitrou2023-11-041-0/+3
| | | | | | | Joining a thread now ensures the underlying OS thread has exited. This is required for safer fork() in multi-threaded processes. --------- Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
* gh-111284: Make multiprocessing tests with threads faster and more reliable ↵Serhiy Storchaka2023-10-301-9/+21
| | | | (GH-111285)
* gh-110662: multiprocessing test_async_timeout() increase timeout (#110663)Victor Stinner2023-10-111-1/+1
| | | | Increase timeout from 1 second to 30 seconds, if not longer. The important part is that apply_async() takes longer than TIMEOUT2.
* gh-110666: Fix multiprocessing test_terminate() elapsed (#110667)Victor Stinner2023-10-111-20/+15
| | | | | | multiprocessing test_terminate() and test_wait_socket_slow() no longer test the CI performance: no longer check maximum elapsed time. Add CLOCK_RES constant: tolerate a difference of 100 ms.
* gh-110656: Fix logging test_post_fork_child_no_deadlock() if ASAN (#110657)Victor Stinner2023-10-111-2/+2
| | | | | Skip test_post_fork_child_no_deadlock() if Python is built with ASAN. Add support.HAVE_ASAN_FORK_BUG.
* gh-109840: Fix multiprocessing test_waitfor_timeout() (#110428)Victor Stinner2023-10-051-3/+3
| | | | Don't measure the CI performance: don't fail if cond.wait_for() takes longer than 1 second on a slow CI.
* gh-110036: multiprocessing Popen.terminate() catches PermissionError (#110037)Victor Stinner2023-09-291-2/+3
| | | | | | On Windows, multiprocessing Popen.terminate() now catchs PermissionError and get the process exit code. If the process is still running, raise again the PermissionError. Otherwise, the process terminated as expected: store its exit code.
* gh-89363: Skip threading test_is_alive_after_fork() if ASAN (#109835)Victor Stinner2023-09-251-1/+1
| | | | Skip test_is_alive_after_fork() of test_threading if Python is built with Address Sanitizer (ASAN).
* gh-109706: Fix multiprocessing test_nested_startmethod() (#109707)Victor Stinner2023-09-221-1/+3
| | | Don't check order, queue items can be written in any order.
* gh-108416: Mark slow but not CPU bound test methods with ↵Serhiy Storchaka2023-09-051-0/+3
| | | | requires_resource('walltime') (GH-108480)
* gh-108416: Mark slow test methods with @requires_resource('cpu') (GH-108421)Serhiy Storchaka2023-09-021-0/+2
| | | | Only mark tests which spend significant system or user time, by itself or in subprocesses.
* gh-108520: Fix bad fork detection in nested multiprocessing use case (#108568)albanD2023-08-301-0/+26
| | | | | | | | gh-107275 introduced a regression where a SemLock would fail being passed along nested child processes, as the `is_fork_ctx` attribute would be left missing after the first deserialization. --------- Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Antoine Pitrou <pitrou@free.fr>
* gh-108388: Split test_multiprocessing_spawn (#108396)Victor Stinner2023-08-241-1/+9
| | | | | | Split test_multiprocessing_fork, test_multiprocessing_forkserver and test_multiprocessing_spawn into test packages. Each package is made of 4 sub-tests: processes, threads, manager and misc. It allows running more tests in parallel and so reduce the total test duration.
* gh-77377: Ensure multiprocessing SemLock is valid for spawn-based Process ↵albanD2023-08-231-0/+22
| | | | | | | | | | | | before serializing it (#107275) Ensure multiprocessing SemLock is valid for spawn Process before serializing it. Creating a multiprocessing SemLock with a fork context, and then trying to pass it to a spawn-created Process, would segfault if not detected early. --------- Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Antoine Pitrou <pitrou@free.fr>
* gh-107963: Fix set_forkserver_preload to check the type of given list (#107965)Dong-hee Na2023-08-151-0/+8
| | | | gh-107963: Fix set_forkserver_preload to check the type of given list
* gh-106558: break ref cycles through exceptions in multiprocessing manager ↵Andrew Geng2023-08-111-0/+38
| | | | (#106559)
* gh-90876: Restore the ability to import multiprocessing when ↵Gregory P. Smith2023-07-061-6/+76
| | | | | | | | | `sys.executable` is `None` (#106464) Prevent `multiprocessing.spawn` from failing to *import* in environments where `sys.executable` is `None`. This regressed in 3.11 with the addition of support for path-like objects in multiprocessing. Adds a test decorator to have tests only run when part of test_multiprocessing_spawn to `_test_multiprocessing.py` so we can start to avoid re-running the same not-global-state specific test in all 3 modes when there is no need.
* gh-105407: Remove unused imports in tests (#105408)Victor Stinner2023-06-061-1/+1
|
* gh-61460: Stronger HMAC in multiprocessing (#20380)Christian Heimes2023-05-201-10/+47
| | | | | | | | | | | | | | | | | | | | | | bpo-17258: `multiprocessing` now supports stronger HMAC algorithms for inter-process connection authentication rather than only HMAC-MD5. Signed-off-by: Christian Heimes <christian@python.org> gpshead: I Reworked to be more robust while keeping the idea. The protocol modification idea remains, but we now take advantage of the message length as an indicator of legacy vs modern protocol version. No more regular expression usage. We now default to HMAC-SHA256, but do so in a way that will be compatible when communicating with older clients or older servers. No protocol transition period is needed. More integration tests to verify these claims remain true are required. I'm unaware of anyone depending on multiprocessing connections between different Python versions. --------- Signed-off-by: Christian Heimes <christian@python.org> Co-authored-by: Gregory P. Smith [Google] <greg@krypto.org>
* gh-84559: Remove the new multiprocessing warning, too disruptive. (#101551)Gregory P. Smith2023-02-031-3/+2
| | | | This reverts the core of #100618 while leaving relevant documentation improvements and minor refactorings in place.
* GH-84559: Deprecate fork being the multiprocessing default. (#100618)Gregory P. Smith2023-02-021-8/+13
| | | | | | | | This starts the process. Users who don't specify their own start method and use the default on platforms where it is 'fork' will see a DeprecationWarning upon multiprocessing.Pool() construction or upon multiprocessing.Process.start() or concurrent.futures.ProcessPool use. See the related issue and documentation within this change for details.
* gh-60203: Always pass True/False as boolean arguments in tests (GH-99983)Serhiy Storchaka2022-12-041-1/+1
| | | Unless we explicitly test non-bool values.
* gh-92886: Fix tests that fail when running with optimizations (`-O`) in ↵Jack Hindmarch2022-10-191-23/+28
| | | | `_test_multiprocessing.py` (GH-93233)
* gh-96819: multiprocessing.resource_tracker: check if length of pipe write <= ↵Koki Saito2022-10-031-0/+8
| | | | | 512 (#96890) Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
* gh-97706: multiprocessing tests: Delete unused variable `rand` (#97707)Koki Saito2022-10-031-2/+1
|
* gh-95212: make multiprocessing test case parallel-safe (GH-95213)Christian Heimes2022-07-251-1/+1
|
* GH-94736: mark SemLock test as linux only (GH-94750)Kumar Aditya2022-07-111-0/+1
| | | | | | See https://buildbot.python.org/all/#/builders/172/builds/2522 The PR skips the test on non-linux platforms. Automerge-Triggered-By: GH:pablogsal
generated by cgit v0.12 at 2025-04-29 12:57:27 (GMT)