summaryrefslogtreecommitdiffstats
path: root/Lib/multiprocessing
Commit message (Collapse)AuthorAgeFilesLines
* bpo-40882: Fix a memory leak in SharedMemory on Windows (GH-20684)Luke Garland2022-12-021-1/+4
| | | | | | | | In multiprocessing.shared_memory.SharedMemory(), the temporary view returned by MapViewOfFile() should be unmapped when it is no longer needed. (cherry picked from commit 85c128e34daec7625b74746e127afa25888ccde1) Co-authored-by: Zackery Spytz <zspytz@gmail.com>
* [3.10] gh-97514: Don't use Linux abstract sockets for multiprocessing ↵Miss Islington (bot)2022-10-201-5/+0
| | | | | | | | | | | | | | | | | | | | | | (GH-98501) (GH-98503) Linux abstract sockets are insecure as they lack any form of filesystem permissions so their use allows anyone on the system to inject code into the process. This removes the default preference for abstract sockets in multiprocessing introduced in Python 3.9+ via https://github.com/python/cpython/pull/18866 while fixing https://github.com/python/cpython/issues/84031. Explicit use of an abstract socket by a user now generates a RuntimeWarning. If we choose to keep this warning, it should be backported to the 3.7 and 3.8 branches. (cherry picked from commit 49f61068f49747164988ffc5a442d2a63874fc17) Co-authored-by: Gregory P. Smith <greg@krypto.org> Automerge-Triggered-By: GH:gpshead
* gh-96819: multiprocessing.resource_tracker: check if length of pipe write <= ↵Miss Islington (bot)2022-10-031-2/+2
| | | | | | | | 512 (GH-96890) Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com> (cherry picked from commit 19ca114645bd8796cf4094e152b1fa9944da473d) Co-authored-by: Koki Saito <49419225+saito828koki@users.noreply.github.com>
* GH-83658: make multiprocessing.Pool raise an exception if maxtasksperchild ↵Miss Islington (bot)2022-06-171-0/+3
| | | | | | | | | | is not None or a positive int (GH-93364) (GH-93924) Closes GH-83658. (cherry picked from commit e37a158725dec561f234b81864363d55f05c7b4e) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com> Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
* gh-91577: SharedMemory move imports out of methods (GH-91579)Miss Islington (bot)2022-06-161-4/+3
| | | | | | | | | | SharedMemory.unlink() uses the unregister() function from resource_tracker. Previously it was imported in the method, but this can fail if the method is called during interpreter shutdown, for example when unlink is part of a __del__() method. Moving the import to the top of the file, means that the unregister() method is available during interpreter shutdown. The register call in SharedMemory.__init__() can also use this imported resource_tracker. (cherry picked from commit 9a458befdd68625d088f4fea7df135a57d147deb) Co-authored-by: samtygier <samtygier@yahoo.co.uk>
* gh-90549: Fix leak of global named resources using multiprocessing spawn ↵Miss Islington (bot)2022-06-102-2/+22
| | | | | | | | | (GH-30617) Co-authored-by: XD Trol <milestonejxd@gmail.com> Co-authored-by: Antoine Pitrou <pitrou@free.fr> (cherry picked from commit 30610d28374f5a9698d456cebf3ae496ac01af51) Co-authored-by: Leo Trol <milestone.jxd@gmail.com>
* bpo-47029: Fix BrokenPipeError in multiprocessing.Queue at garbage ↵Miss Islington (bot)2022-05-041-12/+11
| | | | | | | collection and explicit close (GH-31913) (cherry picked from commit dfb1b9da8a4becaeaed3d9cffcaac41bcaf746f4) Co-authored-by: Géry Ogam <gery.ogam@gmail.com>
* Fix missing `f` prefix on f-strings (GH-91910)Miss Islington (bot)2022-04-271-1/+1
| | | | | (cherry picked from commit f882d33778ee2625ab32d90e28edb6878fb8af93) Co-authored-by: Alexander Shadchin <alexandr.shadchin@gmail.com>
* gh-91231: multiprocessing BaseManager waits 1.0 second (#91701)Victor Stinner2022-04-191-1/+1
| | | | | Shutting down a multiprocessing BaseManager now waits for 1 second until the process completes, rather than 0.1 second, after the process is terminated.
* bpo-38840: Incorrect __all__ in multiprocessing.managers (GH-18034)Miss Islington (bot)2021-08-091-3/+4
| | | | | | | | | | This was causing test___all__ to fail on platforms lacking a shared memory implementation. Co-Authored-By: Xavier de Gaye <xdegaye@gmail.com> Co-authored-by: Łukasz Langa <lukasz@langa.pl> (cherry picked from commit d0978761118856e8ca8ea7b162a6585b8da83df9) Co-authored-by: Zackery Spytz <zspytz@gmail.com>
* bpo-30256: Add manager_owned keyword arg to AutoProxy (GH-16341) (#26987)Miss Islington (bot)2021-07-021-2/+2
| | | | | | | Co-authored-by: Jordan Speicher <jordan@jspeicher.com> (cherry picked from commit 85b920498b42c69185540ecc2f5c4907fd38d877) Co-authored-by: finefoot <33361833+finefoot@users.noreply.github.com>
* bpo-43867: multiprocessing Server catchs SystemExit (GH-25441)Victor Stinner2021-04-161-6/+14
| | | | | The multiprocessing Server class now explicitly catchs SystemExit and closes the client connection in this case. It happens when the Server.serve_client() method reachs the end of file (EOF).
* bpo-43651: Fix test_compileall with PEP 597 (GH-25128)Inada Naoki2021-04-021-1/+1
|
* bpo-40692: Run more test_concurrent_futures tests (GH-20239)Asheesh Laroia2021-02-081-1/+9
| | | | | | | | | | | | | | | | | | | | | | | | | | | In the case of multiprocessing.synchronize() being missing, the test_concurrent_futures test suite now skips only the tests that require multiprocessing.synchronize(). Validate that multiprocessing.synchronize exists as part of _check_system_limits(), allowing ProcessPoolExecutor to raise NotImplementedError during __init__, rather than crashing with ImportError during __init__ when creating a lock imported from multiprocessing.synchronize. Use _check_system_limits() to disable tests of ProcessPoolExecutor on systems without multiprocessing.synchronize. Running the test suite without multiprocessing.synchronize reveals that Lib/compileall.py crashes when it uses a ProcessPoolExecutor. Therefore, change Lib/compileall.py to call _check_system_limits() before creating the ProcessPoolExecutor. Note that both Lib/compileall.py and Lib/test/test_compileall.py were attempting to sanity-check ProcessPoolExecutor by expecting ImportError. In multiprocessing.resource_tracker, sem_unlink() is also absent on platforms where POSIX semaphores aren't available. Avoid using sem_unlink() if it, too, does not exist. Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-41344: Raise ValueError when creating shared memory of size 0 (GH-21556)Vinay Sharma2020-08-301-0/+2
|
* Fix typo in message from assert statement (GH-21283)Allen2020-08-171-1/+1
| | | The error message was missing space between the action "acquire" and "_wait_semaphore" which is an attribute for instances of Condition.
* bpo-39244: multiprocessing return default start method first on macOS (GH-18625)idomic2020-05-261-3/+4
|
* bpo-40443: Remove unused imports in stdlib (GH-19815)Victor Stinner2020-05-011-1/+0
|
* bpo-30966: Add multiprocessing.SimpleQueue.close() (GH-19735)Victor Stinner2020-04-271-0/+4
| | | | | | Add a new close() method to multiprocessing.SimpleQueue to explicitly close the queue. Automerge-Triggered-By: @pitrou
* bpo-40330: Fix utf-8 size check in ShareableList (GH-19606)Antoine Pitrou2020-04-201-4/+6
| | | | | The item size must be checked after encoding to bytes, not before. Automerge-Triggered-By: @pitrou
* bpo-38891: avoid quadratic item access performance of ShareableList (GH-18996)Thomas Krennwallner2020-04-191-30/+45
| | | | | | | | Avoid linear runtime of ShareableList.__getitem__ and ShareableList.__setitem__ by storing running allocated bytes in ShareableList._allocated_bytes instead of the number of bytes for a particular stored item. Co-authored-by: Antoine Pitrou <antoine@python.org>
* Fix two typos in multiprocessing (GH-19571)Galden2020-04-181-2/+2
|
* bpo-40221: Update multiprocessing to use _at_fork_reinit (GH-19511)Dong-hee Na2020-04-142-9/+10
|
* bpo-40221: Update multiprocessing to use _at_fork_reinit (GH-19477)Dong-hee Na2020-04-141-4/+4
|
* bpo-39481: PEP 585 for a variety of modules (GH-19423)Batuhan Taşkaya2020-04-104-0/+12
| | | | | | | | | | - concurrent.futures - ctypes - http.cookies - multiprocessing - queue - tempfile - unittest.case - urllib.parse
* bpo-40094: Add os.waitstatus_to_exitcode() (GH-19201)Victor Stinner2020-04-012-13/+3
| | | | | | | | | | | | | | Add os.waitstatus_to_exitcode() function to convert a wait status to an exitcode. Suggest waitstatus_to_exitcode() usage in the documentation when appropriate. Use waitstatus_to_exitcode() in: * multiprocessing, os, subprocess and _bootsubprocess modules; * test.support.wait_process(); * setup.py: run_command(); * and many tests.
* bpo-39360: Ensure all workers exit when finalizing a multiprocessing Pool ↵Batuhan Taşkaya2020-03-151-2/+5
| | | | | | | | | (GH-19009) When the pull is not used via the context manager or terminate() is called, there is a system in multiprocessing.util that handles finalization of all pools via an atexit handler (the Finalize) class. This class registers the _terminate_pool handler in the registry of finalizers of the module, and that registry is called on interpreter exit via _exit_function. The problem is that the "happy" path with the context manager or manual call to finalize() does some extra steps that _terminate_pool does not. The step that is not executed when the atexit() handler calls _terminate_pool is pinging the _change_notifier queue to unblock the maintenance threads. This commit moves the notification to the _terminate_pool function so is called from both code paths. Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-39850: Add support for abstract sockets in multiprocessing (GH-18866)Pablo Galindo2020-03-094-5/+40
|
* bpo-35727: Use exit code 0 on sys.exit() in multiprocessing.Process. (GH-11538)Christopher Hunt2020-02-211-5/+5
|
* bpo-39439: Fix multiprocessing spawn path in a venv on Windows (GH-18158)Adam Meily2020-01-281-1/+1
|
* bpo-38546: multiprocessing tests stop the resource tracker (GH-17641)Victor Stinner2019-12-172-0/+38
| | | | | | | | | | Multiprocessing and concurrent.futures tests now stop the resource tracker process when tests complete. Add ResourceTracker._stop() method to multiprocessing.resource_tracker. Add _cleanup_tests() helper function to multiprocessing.util: share code between multiprocessing and concurrent.futures tests.
* Remove binding of captured exceptions when not used to reduce the chances of ↵Pablo Galindo2019-11-192-3/+3
| | | | | | | creating cycles (GH-17246) Capturing exceptions into names can lead to reference cycles though the __traceback__ attribute of the exceptions in some obscure cases that have been reported previously and fixed individually. As these variables are not used anyway, we can remove the binding to reduce the chances of creating reference cycles. See for example GH-13135
* bpo-38707: Fix for multiprocessing.Process MainThread.native_id (GH-17088)Jake Tesler2019-11-191-0/+2
| | | | | | | | | | | | | | | | | | | This PR implements a fix for `multiprocessing.Process` objects; the error occurs when Processes are created using either `fork` or `forkserver` as the `start_method`. In these instances, the `MainThread` of the newly created `Process` object retains all attributes from its parent's `MainThread` object, including the `native_id` attribute. The resulting behavior is such that the new process' `MainThread` captures an incorrect/outdated `native_id` (the parent's instead of its own). This change forces the Process object to update its `native_id` attribute during the bootstrap process. cc @vstinner https://bugs.python.org/issue38707 Automerge-Triggered-By: @pitrou
* bpo-38417: Add umask support to subprocess (GH-16726)Gregory P. Smith2019-10-121-1/+1
| | | | On POSIX systems, allow the umask to be set in the child process before we exec.
* bpo-38092: Reduce overhead when using multiprocessing in a Windows virtual ↵Steve Dower2019-09-132-2/+2
| | | | | environment (GH-16098) https://bugs.python.org/issue38092
* bpo-36046: Add user and group parameters to subprocess (GH-11950)Patrick McLean2019-09-121-1/+1
| | | | | | | | | | | | | | * subprocess: Add user, group and extra_groups paremeters to subprocess.Popen This adds a `user` parameter to the Popen constructor that will call setreuid() in the child before calling exec(). This allows processes running as root to safely drop privileges before running the subprocess without having to use a preexec_fn. This also adds a `group` parameter that will call setregid() in the child process before calling exec(). Finally an `extra_groups` parameter was added that will call setgroups() to set the supplimental groups.
* Fix typos mostly in comments, docs and test names (GH-15209)Min ho Kim2019-08-301-1/+1
|
* bpo-37421: multiprocessing tests now stop ForkServer (GH-14601)Victor Stinner2019-07-051-0/+19
| | | | | multiprocessing tests now stop the ForkServer instance if it's running: close the "alive" file descriptor to ask the server to stop and then remove its UNIX address.
* bpo-37421: Fix multiprocessing get_temp_dir() finalizer (GH-14572)Victor Stinner2019-07-041-1/+13
| | | | | | Fix multiprocessing.util.get_temp_dir() finalizer: clear also the 'tempdir' configuration of the current process, so next call to get_temp_dir() will create a new temporary directory, rather than reusing the removed temporary directory.
* bpo-37369: Fix initialization of sys members when launched via an app ↵Steve Dower2019-06-291-2/+1
| | | | | | | container (GH-14428) sys._base_executable is now always defined on all platforms, and can be overridden through configuration. Also adds test.support.PythonSymlink to encapsulate platform-specific logic for symlinking sys.executable
* [3.9] bpo-37116: Use PEP 570 syntax for positional-only parameters. (GH-12620)Serhiy Storchaka2019-06-051-41/+3
| | | Turn deprecation warnings added in 3.8 into TypeError.
* bpo-37116: Use PEP 570 syntax for positional-only parameters. (GH-13700)Serhiy Storchaka2019-06-013-9/+6
|
* bpo-33725: multiprocessing uses spawn by default on macOS (GH-13603)Victor Stinner2019-05-281-1/+6
| | | | On macOS, the multiprocessing module now uses the "spawn" start method by default.
* bpo-36888: Add multiprocessing.parent_process() (GH-13247)Thomas Moreau2019-05-208-20/+85
|
* bpo-36867: Create the resource_tracker before launching SharedMemoryManagers ↵Pierre Glaser2019-05-131-0/+9
| | | | (GH-13276)
* bpo-36894: Fix regression in test_multiprocessing_spawn (no tests run on ↵Antoine Pitrou2019-05-131-4/+9
| | | | Windows) (GH-13290)
* bpo-36867: Make semaphore_tracker track other system resources (GH-13222)Pierre Glaser2019-05-106-70/+92
| | | The multiprocessing.resource_tracker replaces the multiprocessing.semaphore_tracker module. Other than semaphores, resource_tracker also tracks shared_memory segments. Patch by Pierre Glaser.
* bpo-36368: Ignore SIGINT in SharedMemoryManager servers. (GH-12483)Pierre Glaser2019-05-101-0/+4
| | | | Fix a bug crashing SharedMemoryManager instances in interactive sessions after a Ctrl-C (KeyboardInterrupt) was sent.
* bpo-36542: Allow to overwrite the signature for Python functions. (GH-12705)Serhiy Storchaka2019-05-061-0/+2
|
* bpo-36766: Typos in docs and code comments (GH-13116)penguindustin2019-05-061-1/+1
|
generated by cgit v0.12 at 2025-04-27 01:26:22 (GMT)