summaryrefslogtreecommitdiffstats
path: root/Lib/test/test_concurrent_futures.py
Commit message (Collapse)AuthorAgeFilesLines
* [3.10] gh-90622: Do not spawn ProcessPool workers on demand via fork method. ↵Miss Islington (bot)2022-05-081-1/+11
| | | | | | | | | | | | | | | | (GH-91598) (GH-92497) (#92499) Do not spawn ProcessPool workers on demand when they spawn via fork. This avoids potential deadlocks in the child processes due to forking from a multithreaded process.. (cherry picked from commit ebb37fc3fdcb03db4e206db017eeef7aaffbae84) Co-authored-by: Gregory P. Smith <greg@krypto.org> (cherry picked from commit b795376a628ae7cc354addbb926d724ebe364fec) Co-authored-by: Gregory P. Smith <greg@krypto.org> Co-authored-by: Gregory P. Smith <greg@krypto.org>
* [3.9] [3.10] gh-91607: Fix several test_concurrent_futures tests to actually ↵Gregory P. Smith2022-04-171-54/+42
| | | | | | | | | | | | | | | | | | | | test what they claim (GH-91600) (GH-91612) (#91617) * Fix test_concurrent_futures to actually test what it says. Many ProcessPoolExecutor based tests were ignoring the mp_context and using the default instead. This meant we lacked proper test coverage of all of them. Also removes the old _prime_executor() worker delay seeding code as it appears to have no point and causes 20-30 seconds extra latency on this already long test. It also interfered with some of the refactoring to fix the above to not needlessly create their own executor when setUp has already created an appropriate one. * Don't import the name from multiprocessing directly to avoid confusion. (cherry picked from commit 7fa3a5a2197896066e3fe53ee325ac6ab54c3414) (cherry picked from commit 9a458934f7be37c59525350f1f9ecbdcd69903c1)
* [3.10] bpo-46633: Skip tests on ASAN and/or MSAN builds (GH-31632) ↵Victor Stinner2022-03-021-0/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | (GH-31634) (GH-31644) * Refactor sanitiser skip tests into test.support (GH-30889) * Refactor sanitizer skip tests into test.support (cherry picked from commit b1cb8430504931f7854eac5d32cba74770078a4e) * Add skips to crashing tests under sanitizers instead of manually skipping them (GH-30897) (cherry picked from commit a27505345e34d462139f5f8b6b5e7c9a59955150) * bpo-46633: Skip tests on ASAN and/or MSAN builds (GH-31632) Skip tests on ASAN and/or MSAN builds: * multiprocessing tests * test___all__ * test_concurrent_futures * test_decimal * test_peg_generator * test_tools (cherry picked from commit 9204bb72a2da5885facc747e63d2bd2d654606fe) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com> (cherry picked from commit 93264452d952d9ba604bacf2947c2df5dd477931)
* bpo-20369: concurrent.futures.wait() now deduplicates futures given a… ↵Miss Islington (bot)2022-01-041-0/+8
| | | | | | | | | | | | (GH-30168) * bpo-20369: concurrent.futures.wait() now deduplicates futures given as arg. * 📜🤖 Added by blurb_it. Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> (cherry picked from commit 7d7817cf0f826e566d8370a0e974bbfed6611d91) Co-authored-by: Kumar Aditya <59607654+kumaraditya303@users.noreply.github.com>
* [3.9] bpo-45021: Fix a hang in forked children (GH-28007) (GH-28481)Miss Islington (bot)2021-09-201-0/+14
| | | | | | | | | _global_shutdown_lock should be reinitialized in forked children (cherry picked from commit 0bfa1106acfcddc03590e1f5d6789dbad3affe70) Co-authored-by: nullptr <3621629+0x0L@users.noreply.github.com> Automerge-Triggered-By: GH:gpshead
* [3.9] bpo-45229: Remove test_main in many tests (GH-28405) (GH-28456)Serhiy Storchaka2021-09-201-9/+3
| | | | | | | | | | | Instead of explicitly enumerate test classes for run_unittest() use the unittest ability to discover tests. This also makes these tests discoverable and runnable with unittest. load_tests() can be used for dynamic generating tests and adding doctests. setUpModule(), tearDownModule() and addModuleCleanup() can be used for running code before and after all module tests.. (cherry picked from commit 40348acc180580371d25f75f46b27048e35f2435)
* [3.9] bpo-25130: Add calls of gc.collect() in tests to support PyPy ↵Serhiy Storchaka2021-08-291-0/+6
| | | | | | (GH-28005). (GH-28028) (cherry picked from commit 2a8127cafe1d196f858a3ecabf5f1df3eebf9a12)
* bpo-38377: Add support.skip_if_broken_multiprocessing_synchronize() ↵Victor Stinner2020-06-181-1/+1
| | | | | | | | | | (GH-20944) (GH-20962) On Linux, skip tests using multiprocessing if the current user cannot create a file in /dev/shm/ directory. Add the skip_if_broken_multiprocessing_synchronize() function to the test.support module. (cherry picked from commit ddbeb2f3e02a510c5784ffd74c5e09e8c70b5881)
* bpo-17258: Add requires_hashdigest to multiprocessing tests (GH-20412)Miss Islington (bot)2020-06-041-0/+2
| | | | | | | | Skip some :mod:`multiprocessing` tests when MD5 hash digest is blocked. Signed-off-by: Christian Heimes <christian@python.org> (cherry picked from commit b022e5cffbd3ff51ae361cf80f2a3b660be8b1ee) Co-authored-by: Christian Heimes <christian@python.org>
* bpo-39995: Split test_concurrent_futures.test_crash() into sub-tests (GH-19739)Victor Stinner2020-04-271-63/+85
| | | | | | | | | Now only test_error_during_result_unpickle_in_result_handler() captures and ignores sys.stderr in the test process. Tools like test.bisect_cmd don't support subTest() but only work with the granularity of one method. Remove unused ExecutorDeadlockTest._sleep_id() method.
* bpo-39207: Spawn workers on demand in ProcessPoolExecutor (GH-19453)Kyle Stanley2020-04-191-4/+40
| | | | | Roughly based on https://github.com/python/cpython/commit/904e34d4e6b6007986dcc585d5c553ee8ae06f95, but with a few substantial differences. /cc @pitrou @brianquinlan
* bpo-39678: refactor queue manager thread (GH-18551)Thomas Moreau2020-03-011-8/+8
|
* bpo-39104: Fix hanging ProcessPoolExecutor on shutdown nowait with pickling ↵Thomas Moreau2020-02-161-2/+72
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | failure (GH-17670) As reported initially by @rad-pat in #6084, the following script causes a deadlock. ``` from concurrent.futures import ProcessPoolExecutor class ObjectWithPickleError(): """Triggers a RuntimeError when sending job to the workers""" def __reduce__(self): raise RuntimeError() if __name__ == "__main__": e = ProcessPoolExecutor() f = e.submit(id, ObjectWithPickleError()) e.shutdown(wait=False) f.result() # Deadlock on get ``` This is caused by the fact that the main process is closing communication channels that might be necessary to the `queue_management_thread` later. To avoid this, this PR let the `queue_management_thread` manage all the closing. https://bugs.python.org/issue39104 Automerge-Triggered-By: @pitrou
* bpo-39349: Add *cancel_futures* to Executor.shutdown() (GH-18057)Kyle Stanley2020-02-021-0/+39
|
* bpo-39205: Tests that highlight a hang on ProcessPoolExecutor shutdown (#18221)Brian Quinlan2020-01-281-0/+20
|
* bpo-38546: Fix concurrent.futures test_ressources_gced_in_workers() (GH-17652)Victor Stinner2019-12-181-3/+11
| | | | | Fix test_ressources_gced_in_workers() of test_concurrent_futures: explicitly stop the manager to prevent leaking a child process running in the background after the test completes.
* bpo-38546: multiprocessing tests stop the resource tracker (GH-17641)Victor Stinner2019-12-171-11/+1
| | | | | | | | | | Multiprocessing and concurrent.futures tests now stop the resource tracker process when tests complete. Add ResourceTracker._stop() method to multiprocessing.resource_tracker. Add _cleanup_tests() helper function to multiprocessing.util: share code between multiprocessing and concurrent.futures tests.
* bpo-38614: Use test.support.SHORT_TIMEOUT constant (GH-17566)Victor Stinner2019-12-111-20/+21
| | | | | | | | | | | | | | | Replace hardcoded timeout constants in tests with SHORT_TIMEOUT of test.support, so it's easier to ajdust this timeout for all tests at once. SHORT_TIMEOUT is 30 seconds by default, but it can be longer depending on --timeout command line option. The change makes almost all timeouts longer, except test_reap_children() of test_support which is made 2x shorter: SHORT_TIMEOUT should be enough. If this test starts to fail, LONG_TIMEOUT should be used instead. Uniformize also "from test import support" import in some test files.
* bpo-37421: test_concurrent_futures stops ForkServer (GH-14643)Victor Stinner2019-07-081-0/+3
| | | | test_concurrent_futures now explicitly stops the ForkServer instance if it's running.
* bpo-37421: test_concurrent_futures cleans up multiprocessing (GH-14563)Victor Stinner2019-07-031-7/+22
| | | | | | | test_concurrent_futures now cleans up multiprocessing to remove immediately temporary directories created by multiprocessing.util.get_temp_dir(). The test now uses setUpModule() and tearDownModule().
* [3.9] bpo-37116: Use PEP 570 syntax for positional-only parameters. (GH-12620)Serhiy Storchaka2019-06-051-3/+2
| | | Turn deprecation warnings added in 3.8 into TypeError.
* bpo-35279: reduce default max_workers of ThreadPoolExecutor (GH-13618)Inada Naoki2019-05-281-2/+2
|
* bpo-24882: Let ThreadPoolExecutor reuse idle threads before creating new ↵Sean2019-05-221-3/+29
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | thread (#6375) * Fixes issue 24882 * Add news file entry for change. * Change test_concurrent_futures.ThreadPoolShutdownTest Adjust the shutdown test so that, after submitting three jobs to the executor, the test checks for less than three threads, instead of looking for exactly three threads. If idle threads are being recycled properly, then we should have less than three threads. * Switched idle count to semaphor, Updated tests As suggested by reviewer tomMoral, swapped lock-protected counter with a semaphore to track the number of unused threads. Adjusted test_threads_terminate to wait for completiton of the previous future before submitting a new one (and checking the number of threads used). Also added a new test to confirm the thread pool can be saturated. * Updates tests as requested by pitrou. * Correct minor whitespace error. * Make test_saturation faster
* bpo-33110: Catch errors raised when running add_done_callback on already ↵Sam Martin2019-05-221-0/+16
| | | | | | completed futures (GH-13141) Wrap the callback call within the `add_done_callback` function within concurrent.futures, in order to behave in an identical manner to callbacks added to a running future are triggered once it has completed.
* bpo-26903: Limit ProcessPoolExecutor to 61 workers on Windows (GH-13132)Brian Quinlan2019-05-081-0/+7
| | | | Co-Authored-By: brianquinlan <brian@sweetapp.com>
* bpo-36492: Deprecate passing some arguments as keyword arguments. (GH-12637)Serhiy Storchaka2019-04-011-0/+10
| | | | | | | | | | | | | | | | | | | | | | Deprecated passing the following arguments as keyword arguments: - "func" in functools.partialmethod(), weakref.finalize(), profile.Profile.runcall(), cProfile.Profile.runcall(), bdb.Bdb.runcall(), trace.Trace.runfunc() and curses.wrapper(). - "function" in unittest.addModuleCleanup() and unittest.TestCase.addCleanup(). - "fn" in the submit() method of concurrent.futures.ThreadPoolExecutor and concurrent.futures.ProcessPoolExecutor. - "callback" in contextlib.ExitStack.callback(), contextlib.AsyncExitStack.callback() and contextlib.AsyncExitStack.push_async_callback(). - "c" and "typeid" in the create() method of multiprocessing.managers.Server and multiprocessing.managers.SharedMemoryServer. - "obj" in weakref.finalize(). Also allowed to pass arbitrary keyword arguments (even "self" and "func") if the above arguments are passed as positional argument.
* bpo-33716, test_concurrent_futures: increase timeout (GH-7828)Pablo Galindo2018-06-211-5/+5
| | | | | Increase the timeout from 1 min to 5 min. Replace also time.time() with time.monotonic() for timeouts.
* bpo-33238: Add InvalidStateError to concurrent.futures. (GH-7056)jhaydaman2018-05-301-0/+28
| | | | | | | Future.set_result and Future.set_exception now raise InvalidStateError if the futures are not pending or running. This mirrors the behavior of asyncio.Future, and prevents AssertionErrors in asyncio.wrap_future when set_result is called multiple times.
* bpo-33097: Fix submit accepting callable after executor shutdown by ↵Mark Nemec2018-04-101-0/+28
| | | | | | interpreter exit (GH-6144) Executors in concurrent.futures accepted tasks after executor was shutdown by interpreter exit. Tasks were left in PENDING state forever. This fix changes submit to instead raise a RuntimeError.
* bpo-32746: Fix multiple typos (GH-5144)Leo Arias2018-02-041-6/+6
| | | Fix typos found by codespell in docs, docstrings, and comments.
* bpo-31699 Deadlocks in `concurrent.futures.ProcessPoolExecutor` with ↵Thomas Moreau2018-01-051-2/+170
| | | | | | pickling error (#3895) Fix deadlocks in :class:`concurrent.futures.ProcessPoolExecutor` when task arguments or results cause pickling or unpickling errors. This should make sure that calls to the :class:`ProcessPoolExecutor` API always eventually return.
* Silence error output in test_concurrent_futures (bpo-21423) (#4347)Antoine Pitrou2017-11-091-22/+47
| | | | * Silence error output in test_concurrent_futures (bpo-21423)
* bpo-21423: Add an initializer argument to {Process,Thread}PoolExecutor (#4241)Antoine Pitrou2017-11-041-65/+119
| | | | | | * bpo-21423: Add an initializer argument to {Process,Thread}PoolExecutor * Fix docstring
* bpo-31540: Allow passing multiprocessing context to ProcessPoolExecutor (#3682)Thomas Moreau2017-10-031-9/+120
|
* bpo-31641: Allow arbitrary iterables in `concurrent.futures.as_completed()` ↵Łukasz Langa2017-09-291-1/+5
| | | | | | | (#3830) This was possible before. GH-1560 introduced a regression after 3.6.2 got released where only sequences were accepted now. This commit addresses this problem.
* bpo-31234: Join threads in tests (#3572)Victor Stinner2017-09-141-0/+3
| | | | Call thread.join() on threads to prevent the "dangling threads" warning.
* bpo-31249: Fix test_concurrent_futures dangling thread (#3521)Victor Stinner2017-09-131-0/+3
| | | | ProcessPoolShutdownTest.test_del_shutdown() now closes the call queue and joins its thread, to prevent leaking a dangling thread.
* bpo-31370: Remove support for threads-less builds (#3385)Antoine Pitrou2017-09-071-4/+0
| | | | | | * Remove Setup.config * Always define WITH_THREAD for compatibility.
* Fix a c.f.as_completed() refleak previously introduced in bpo-27144 (#3270)Antoine Pitrou2017-09-031-1/+1
|
* bpo-27144: concurrent.futures as_complete and map iterators do not keep ↵Grzegorz Grzywacz2017-09-011-0/+48
| | | | | | | | | | reference to returned object (#1560) * bpo-27144: concurrent.futures as_complie and map iterators do not keep reference to returned object * Some nits. Improve wordings in docstrings and comments, and avoid relying on sys.getrefcount() in tests.
* bpo-31249: test_concurrent_futures checks dangling threads (#3167)Victor Stinner2017-08-211-12/+20
| | | | | Add a BaseTestCase class to test_concurrent_futures to check for dangling threads and processes on all tests, not only tests using ExecutorMixin.
* bpo-30845: Enhance test_concurrent_futures cleanup (#2564)Victor Stinner2017-07-041-0/+7
| | | | | | | | | | | * bpo-30845: reap_children() now logs warnings * bpo-30845: Enhance test_concurrent_futures cleanup In setUp() and tearDown() methods of test_concurrent_futures tests, make sure that tests don't leak threads nor processes. Clear explicitly the reference to the executor to make it that it's destroyed (to prevent "dangling threads" warning).
* bpo-29212: Fix the ugly repr() ThreadPoolExecutor thread name. (#2315)Gregory P. Smith2017-06-221-4/+3
| | | | | | | bpo-29212: Fix the ugly ThreadPoolExecutor thread name. Fixes the newly introduced ugly default thread name for concurrent.futures thread.ThreadPoolExecutor threads. They'll now resemble the old <=3.5 threading default Thread-x names by being named ThreadPoolExecutor-y_n.
* Issue #27895: Spelling fixes (Contributed by Ville Skyttä).Raymond Hettinger2016-08-301-1/+1
|
* Issue #27664: Add to concurrent.futures.thread.ThreadPoolExecutor()Gregory P. Smith2016-08-071-0/+24
| | | | the ability to specify a thread name prefix.
* Issue #27076: Doc, comment and tests spelling fixesMartin Panter2016-05-261-2/+2
| | | | Most fixes to Doc/ and Lib/ directories by Ville Skyttä.
* Issue #9517: Move script_helper to the support package.Berker Peksag2015-05-061-1/+1
| | | | Patch by Christie Wilson.
* Issue #21817: When an exception is raised in a task submitted to a ↵Antoine Pitrou2015-01-171-0/+26
| | | | | | ProcessPoolExecutor, the remote traceback is now displayed in the parent process. Patch by Claudiu Popa.
* Issue #11271: concurrent.futures.Executor.map() now takes a *chunksize*Antoine Pitrou2014-10-041-0/+16
| | | | | argument to allow batching of tasks in child processes and improve performance of ProcessPoolExecutor. Patch by Dan O'Reilly.
* Closes #21527: Add default number of workers to ThreadPoolExecutor. (Claudiu ↵Guido van Rossum2014-09-021-0/+6
| | | | Popa.)
generated by cgit v0.12 at 2025-09-21 17:21:46 (GMT)