diff options
| author | Antoine Pitrou <solipsis@pitrou.net> | 2014-10-04 18:20:10 (GMT) |
|---|---|---|
| committer | Antoine Pitrou <solipsis@pitrou.net> | 2014-10-04 18:20:10 (GMT) |
| commit | 4aae276eca9a9213ad45158d30ae2f15843dd463 (patch) | |
| tree | e3ec365b19f7eb88212595c3ea47f3362bfe75e3 /Doc/library/concurrent.futures.rst | |
| parent | e4f47088af0040d73449d0cb0fe1e6d863f3ad07 (diff) | |
| download | cpython-4aae276eca9a9213ad45158d30ae2f15843dd463.zip cpython-4aae276eca9a9213ad45158d30ae2f15843dd463.tar.gz cpython-4aae276eca9a9213ad45158d30ae2f15843dd463.tar.bz2 | |
Issue #11271: concurrent.futures.Executor.map() now takes a *chunksize*
argument to allow batching of tasks in child processes and improve
performance of ProcessPoolExecutor. Patch by Dan O'Reilly.
Diffstat (limited to 'Doc/library/concurrent.futures.rst')
| -rw-r--r-- | Doc/library/concurrent.futures.rst | 13 |
1 files changed, 11 insertions, 2 deletions
diff --git a/Doc/library/concurrent.futures.rst b/Doc/library/concurrent.futures.rst index e487817..2bebd4b 100644 --- a/Doc/library/concurrent.futures.rst +++ b/Doc/library/concurrent.futures.rst @@ -38,7 +38,7 @@ Executor Objects future = executor.submit(pow, 323, 1235) print(future.result()) - .. method:: map(func, *iterables, timeout=None) + .. method:: map(func, *iterables, timeout=None, chunksize=1) Equivalent to :func:`map(func, *iterables) <map>` except *func* is executed asynchronously and several calls to *func* may be made concurrently. The @@ -48,7 +48,16 @@ Executor Objects *timeout* can be an int or a float. If *timeout* is not specified or ``None``, there is no limit to the wait time. If a call raises an exception, then that exception will be raised when its value is - retrieved from the iterator. + retrieved from the iterator. When using :class:`ProcessPoolExecutor`, this + method chops *iterables* into a number of chunks which it submits to the + pool as separate tasks. The (approximate) size of these chunks can be + specified by setting *chunksize* to a positive integer. For very long + iterables, using a large value for *chunksize* can significantly improve + performance compared to the default size of 1. With :class:`ThreadPoolExecutor`, + *chunksize* has no effect. + + .. versionchanged:: 3.5 + Added the *chunksize* argument. .. method:: shutdown(wait=True) |