summaryrefslogtreecommitdiffstats
path: root/Lib/multiprocessing
Commit message (Collapse)AuthorAgeFilesLines
* bpo-37116: Use PEP 570 syntax for positional-only parameters. (GH-13700)Serhiy Storchaka2019-06-013-9/+6
|
* bpo-33725: multiprocessing uses spawn by default on macOS (GH-13603)Victor Stinner2019-05-281-1/+6
| | | | On macOS, the multiprocessing module now uses the "spawn" start method by default.
* bpo-36888: Add multiprocessing.parent_process() (GH-13247)Thomas Moreau2019-05-208-20/+85
|
* bpo-36867: Create the resource_tracker before launching SharedMemoryManagers ↵Pierre Glaser2019-05-131-0/+9
| | | | (GH-13276)
* bpo-36894: Fix regression in test_multiprocessing_spawn (no tests run on ↵Antoine Pitrou2019-05-131-4/+9
| | | | Windows) (GH-13290)
* bpo-36867: Make semaphore_tracker track other system resources (GH-13222)Pierre Glaser2019-05-106-70/+92
| | | The multiprocessing.resource_tracker replaces the multiprocessing.semaphore_tracker module. Other than semaphores, resource_tracker also tracks shared_memory segments. Patch by Pierre Glaser.
* bpo-36368: Ignore SIGINT in SharedMemoryManager servers. (GH-12483)Pierre Glaser2019-05-101-0/+4
| | | | Fix a bug crashing SharedMemoryManager instances in interactive sessions after a Ctrl-C (KeyboardInterrupt) was sent.
* bpo-36542: Allow to overwrite the signature for Python functions. (GH-12705)Serhiy Storchaka2019-05-061-0/+2
|
* bpo-36766: Typos in docs and code comments (GH-13116)penguindustin2019-05-061-1/+1
|
* bpo-36668: FIX reuse semaphore tracker for child processes (#5172)Thomas Moreau2019-04-241-10/+26
| | | Fix the multiprocessing.semaphore_tracker so it is reused by child processes.
* bpo-36492: Deprecate passing some arguments as keyword arguments. (GH-12637)Serhiy Storchaka2019-04-011-4/+43
| | | | | | | | | | | | | | | | | | | | | | Deprecated passing the following arguments as keyword arguments: - "func" in functools.partialmethod(), weakref.finalize(), profile.Profile.runcall(), cProfile.Profile.runcall(), bdb.Bdb.runcall(), trace.Trace.runfunc() and curses.wrapper(). - "function" in unittest.addModuleCleanup() and unittest.TestCase.addCleanup(). - "fn" in the submit() method of concurrent.futures.ThreadPoolExecutor and concurrent.futures.ProcessPoolExecutor. - "callback" in contextlib.ExitStack.callback(), contextlib.AsyncExitStack.callback() and contextlib.AsyncExitStack.push_async_callback(). - "c" and "typeid" in the create() method of multiprocessing.managers.Server and multiprocessing.managers.SharedMemoryServer. - "obj" in weakref.finalize(). Also allowed to pass arbitrary keyword arguments (even "self" and "func") if the above arguments are passed as positional argument.
* bpo-35493: Use Process.sentinel instead of sleeping for polling worker ↵Pablo Galindo2019-03-161-11/+77
| | | | | | | | | | | | | | | | | | | | status in multiprocessing.Pool (#11488) * bpo-35493: Use Process.sentinel instead of sleeping for polling worker status in multiprocessing.Pool * Use self-pipe pattern to avoid polling for changes * Refactor some variable names and add comments * Restore timeout and poll * Use reader object only on wait() * Recompute worker sentinels every time * Remove timeout and use change notifier * Refactor some methods to be overloaded by the ThreadPool, document the cache class and fix typos
* bpo-36102: Prepend slash to all POSIX shared memory block names (#12036)Davin Potts2019-02-251-4/+10
|
* bpo-35813: Tests and docs for shared_memory (#11816)Davin Potts2019-02-242-319/+392
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Added tests for shared_memory submodule. * Added tests for ShareableList. * Fix bug in allocationn size during creation of empty ShareableList illuminated by existing test run on Linux. * Initial set of docs for shared_memory module. * Added docs for ShareableList, added doctree entry for shared_memory submodule, name refactoring for greater clarity. * Added examples to SharedMemoryManager docs, for ease of documentation switched away from exclusively registered functions to some explicit methods on SharedMemoryManager. * Wording tweaks to docs. * Fix test failures on Windows. * Added tests around SharedMemoryManager. * Documentation tweaks. * Fix inappropriate test on Windows. * Further documentation tweaks. * Fix bare exception. * Removed __copyright__. * Fixed typo in doc, removed comment. * Updated SharedMemoryManager preliminary tests to reflect change of not supporting all registered functions on SyncManager. * Added Sphinx doctest run controls. * CloseHandle should be in a finally block in case MapViewOfFile fails. * Missed opportunity to use with statement. * Switch to self.addCleanup to spare long try/finally blocks and save one indentation, change to use decorator to skip test instead. * Simplify the posixshmem extension module. Provide shm_open() and shm_unlink() functions. Move other functionality into the shared_memory.py module. * Added to doc around size parameter of SharedMemory. * Changed PosixSharedMemory.size to use os.fstat. * Change SharedMemory.buf to a read-only property as well as NamedSharedMemory.size. * Marked as provisional per PEP411 in docstring. * Changed SharedMemoryTracker to be private. * Removed registered Proxy Objects from SharedMemoryManager. * Removed shareable_wrap(). * Removed shareable_wrap() and dangling references to it. * For consistency added __reduce__ to key classes. * Fix for potential race condition on Windows for O_CREX. * Remove unused imports. * Update access to kernel32 on Windows per feedback from eryksun. * Moved kernel32 calls to _winapi. * Removed ShareableList.copy as redundant. * Changes to _winapi use from eryksun feedback. * Adopt simpler SharedMemory API, collapsing PosixSharedMemory and WindowsNamedSharedMemory into one. * Fix missing docstring on class, add test for ignoring size when attaching. * Moved SharedMemoryManager to managers module, tweak to fragile test. * Tweak to exception in OpenFileMapping suggested by eryksun. * Mark a few dangling bits as private as suggested by Giampaolo.
* bpo-35918: Remove broken has_key method and add test (#11819)Rémi Lapeyre2019-02-121-1/+1
|
* bpo-35378: Fix multiprocessing.Pool references (GH-11627)Pablo Galindo2019-02-111-38/+74
| | | | | | | | | Changes in this commit: 1. Use a _strong_ reference between the Pool and associated iterators 2. Rework PR #8450 to eliminate a cycle in the Pool. There is no test in this commit because any test that automatically tests this behaviour needs to eliminate the pool before joining the pool to check that the pool object is garbaged collected/does not hang. But doing this will potentially leak threads and processes (see https://bugs.python.org/issue35413).
* bpo-35872 and bpo-35873: Clears __PYVENV_LAUNCHER__ variable (GH-11745)Steve Dower2019-02-042-9/+20
| | | | After reading __PYVENV_LAUNCHER__ we now set sys._base_executable value for later use. Make the same changes for macOS to avoid extra platform checks.
* bpo-35813: Added shared_memory submodule of multiprocessing. (#11664)Davin Potts2019-02-021-0/+573
| | | Added shared_memory submodule to multiprocessing in time for first alpha with cross-platform tests soon to follow.
* bpo-35797: Fix default executable used by the multiprocessing module (GH-11676)Steve Dower2019-01-251-1/+8
|
* bpo-35424: emit ResourceWarning at multiprocessing.Pool destruction (GH-10974)Victor Stinner2018-12-201-5/+18
| | | | multiprocessing.Pool destructor now emits ResourceWarning if the pool is still running.
* bpo-35491, multiprocessing: replace "RUN" with RUN (GH-11178)Victor Stinner2018-12-161-1/+1
|
* bpo-35491: Enhance multiprocessing.BaseProcess.__repr__() (GH-11138)Victor Stinner2018-12-141-11/+16
| | | | | | | | | | | | | | | | * Add the pid and parent pid to multiprocessing.BaseProcess.__repr__(). * Add negative sign (ex: "-SIGTERM") to exitcode (process killed by a signal) * Only call _popen.poll() once. Example: <ForkProcess(ForkPoolWorker-1, started daemon)> becomes: <ForkProcess name='ForkPoolWorker-1' pid=12449 parent=12448 started daemon> Example: <ForkProcess(ForkPoolWorker-1, stopped[SIGTERM] daemon)> becomes: <ForkProcess name='ForkPoolWorker-1' pid=12960 parent=12959 stopped exitcode=-SIGTERM daemon>
* Add multiprocessing.Pool.__repr__() (GH-11137)Victor Stinner2018-12-141-5/+11
| | | | | * Add multiprocessing.Pool.__repr__() to ease debug * RUN, CLOSE and TERMINATE constants values are now strings rather than integer to ease debug
* bpo-35477: multiprocessing.Pool.__enter__() fails if called twice (GH-11134)Victor Stinner2018-12-131-8/+9
| | | | multiprocessing.Pool.__enter__() now fails if the pool is not running: "with pool:" fails if used more than once.
* Revert "bpo-34172: multiprocessing.Pool leaks resources after being deleted ↵Victor Stinner2018-12-061-50/+24
| | | | | (GH-8450)" (GH-10971) This reverts commit 97bfe8d3ebb0a54c8798f57555cb4152f9b2e1d0.
* bpo-35202: Remove unused imports in Lib directory. (GH-10446)Srinivas Thatiparthy (శ్రీనివాస్ తాటిపర్తి)2018-11-101-1/+0
|
* bpo-17560: Too small type for struct.pack/unpack in ↵Alexander Buchkovsky2018-11-061-10/+20
| | | | | mutliprocessing.Connection (GH-10305) Allow sending more than 2 GB at once on a multiprocessing connection on non-Windows systems.
* bpo-19675: Terminate processes if construction of a pool is failing. (GH-5614)Julien Palard2018-11-041-2/+10
|
* bpo-22872: multiprocessing.Queue's put() and get() now raise ValueError if ↵Zackery Spytz2018-10-131-2/+6
| | | | | | | the queue is closed. (GH-9010) Previously, put() and get() would raise AssertionError and OSError, respectively.
* bpo-34172: multiprocessing.Pool leaks resources after being deleted (GH-8450)tzickel2018-10-021-24/+50
| | | Fix a reference issue inside multiprocessing.Pool that caused the pool to remain alive if it was deleted without being closed or terminated explicitly.
* bpo-34521: Fix tests in test_multiprocessing_spawn to use correctly ↵Pablo Galindo2018-09-281-1/+1
| | | | | | | | | | | | | | | | CMSG_SPACE (GH-9613) After some failures in AMD64 FreeBSD CURRENT Debug 3.x buildbots regarding tests in test_multiprocessing_spawn and after examining similar failures in test_socket, some errors in the calculation of ancillary data buffers were found in multiprocessing.reduction. CMSG_LEN() can often be used as the buffer size for recvmsg() to receive a single item of ancillary data, but RFC 3542 requires portable applications to use CMSG_SPACE() and thus include space for padding, even when the item will be the last in the buffer. The failures we experience are due to the usage of CMSG_LEN() instead of CMSG_SPACE().
* bpo-34610: Fixed iterator of multiprocessing.managers.DictProxy. (GH-9113)Serhiy Storchaka2018-09-171-1/+4
|
* bpo-33613, test_semaphore_tracker_sigint: fix race condition (#7850)Pablo Galindo2018-09-041-6/+28
| | | | | | | | | | | | Fail `test_semaphore_tracker_sigint` if no warnings are expected and one is received. Fix race condition when the child receives SIGINT before it can register signal handlers for it. The race condition occurs when the parent calls `_semaphore_tracker.ensure_running()` (which in turn spawns the semaphore_tracker using `_posixsubprocess.fork_exec`), the child registers the signal handlers and the parent tries to kill the child. What seem to happen is that in some slow systems, the parent sends the signal to kill the child before the child protects against the signal.
* Simplify __all__ in multiprocessing (GH-6856)Derek B. Kim2018-07-112-7/+4
|
* bpo-34054: multiprocessing uses time.monotonic() (GH-8118)Victor Stinner2018-07-064-13/+12
| | | | | The multiprocessing module now uses the monotonic clock time.monotonic() instead of the system clock time.time() to implement timeouts.
* bpo-33929: Fix regression in spawn_main() (#7962)Victor Stinner2018-06-271-2/+6
| | | OpenProcess() creates a new handle that must be closed later.
* bpo-33929: multiprocessing: fix handle leak on race condition (GH-7921)Victor Stinner2018-06-273-6/+29
| | | | | | | | | Fix a race condition in Popen of multiprocessing.popen_spawn_win32. The child process now duplicates the read end of pipe instead of "stealing" it. Previously, the read end of pipe was "stolen" by the child process, but it leaked a handle if the child process had been terminated before it could steal the handle from the parent process.
* bpo-33332: Add signal.valid_signals() (GH-6581)Antoine Pitrou2018-05-041-1/+1
|
* bpo-32759: Free unused arenas in multiprocessing.heap (GH-5827)Antoine Pitrou2018-04-091-23/+91
| | | | | Large shared arrays allocated using multiprocessing would remain allocated until the process ends.
* bpo-33078 - Fix queue size on pickling error (GH-6119)Thomas Moreau2018-03-211-2/+8
|
* bpo-31804: Fix multiprocessing.Process with broken standard streams (#6079)Antoine Pitrou2018-03-113-10/+16
| | | | | In some conditions the standard streams will be None or closed in the child process (for example if using "pythonw" instead of "python" on Windows). Avoid failing with a non-0 exit code in those conditions. Report and initial patch by poxthegreat.
* bpo-32576: use queue.SimpleQueue in critical places (#5216)Antoine Pitrou2018-01-181-8/+11
| | | | Where a queue may be invoked from a weakref callback, we need to use the reentrant SimpleQueue.
* bpo-31699 Deadlocks in `concurrent.futures.ProcessPoolExecutor` with ↵Thomas Moreau2018-01-051-5/+16
| | | | | | pickling error (#3895) Fix deadlocks in :class:`concurrent.futures.ProcessPoolExecutor` when task arguments or results cause pickling or unpickling errors. This should make sure that calls to the :class:`ProcessPoolExecutor` API always eventually return.
* bpo-32157: Removed explicit quotes around %r and {!r}. (#4582)Serhiy Storchaka2017-11-281-2/+2
|
* bpo-31310: multiprocessing's semaphore tracker should be launched again if ↵Antoine Pitrou2017-11-031-3/+17
| | | | | | | | | | | crashed (#3247) * bpo-31310: multiprocessing's semaphore tracker should be launched again if crashed * Avoid mucking with process state in test. Add a warning if the semaphore process died, as semaphores may then be leaked. * Add NEWS entry
* bpo-31308: If multiprocessing's forkserver dies, launch it again when ↵Antoine Pitrou2017-11-031-5/+16
| | | | | | | | | | | | | | | | | | | | | | necessary (#3246) * bpo-31308: If multiprocessing's forkserver dies, launch it again when necessary. * Fix test on Windows * Add NEWS entry * Adopt a different approach: ignore SIGINT and SIGTERM, as in semaphore tracker. * Fix comment * Make sure the test doesn't muck with process state * Also test previously-started processes * Update 2017-08-30-17-59-36.bpo-31308.KbexyC.rst * Avoid masking SIGTERM in forkserver. It's not necessary and causes a race condition in test_many_processes.
* bpo-28326: Fix multiprocessing.Process when stdout and/or stderr is closed ↵Antoine Pitrou2017-10-221-2/+8
| | | | or None. (#4073)
* [Trivial] Remove now redundant assert (#3245)Antoine Pitrou2017-08-301-1/+0
|
* bpo-5001: More-informative multiprocessing error messages (#3079)Allen W. Smith, Ph.D2017-08-2914-39/+119
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Make error message more informative Replace assertions in error-reporting code with more-informative version that doesn't cause confusion over where and what the error is. * Additional clarification + get travis to check * Change from SystemError to TypeError As suggested in PR comment by @pitrou, changing from SystemError; TypeError appears appropriate. * NEWS file installation; ACKS addition (will do my best to justify it by additional work) * Making current AssertionErrors in multiprocessing more informative * Blurb added re multiprocessing managers.py, queues.py cleanup * Further multiprocessing cleanup - went through pool.py * Fix two asserts in multiprocessing/util.py * Most asserts in multiprocessing more informative * Didn't save right version * Further work on multiprocessing error messages * Correct typo * Correct typo v2 * Blasted colon... serves me right for trying to work on two things at once * Simplify NEWS entry * Update 2017-08-18-17-16-38.bpo-5001.gwnthq.rst * Update 2017-08-18-17-16-38.bpo-5001.gwnthq.rst OK, never mind. * Corrected (thanks to pitrou) error messages for notify * Remove extraneous backslash in docstring.
* bpo-18966: non-daemonic threads created by a multiprocessing.Process should ↵Antoine Pitrou2017-08-161-0/+2
| | | | | | | | be joined on exit (#3111) * bpo-18966: non-daemonic threads created by a multiprocessing.Process should be joined on exit * Add NEWS blurb
generated by cgit v0.12 at 2024-09-30 21:41:22 (GMT)