Initial implementation of PEP 3148

1 files changed, 366 insertions, 0 deletions

diff --git a/Doc/library/concurrent.futures.rst b/Doc/library/concurrent.futures.rst
new file mode 100644
index 0000000..abb3dc5
--- /dev/null
+++ b/Doc/library/concurrent.futures.rst
@@ -0,0 +1,366 @@
+:mod:`concurrent.futures` --- Concurrent computation
+====================================================
+
+.. module:: concurrent.futures
+   :synopsis: Execute computations concurrently using threads or processes.
+
+The :mod:`concurrent.futures` module provides a high-level interface for
+asynchronously executing callables.
+
+The asynchronous execution can be be performed by threads using
+:class:`ThreadPoolExecutor` or seperate processes using
+:class:`ProcessPoolExecutor`. Both implement the same interface, which is
+defined by the abstract :class:`Executor` class.
+
+Executor Objects
+^^^^^^^^^^^^^^^^
+
+:class:`Executor` is an abstract class that provides methods to execute calls
+asynchronously. It should not be used directly, but through its two
+subclasses: :class:`ThreadPoolExecutor` and :class:`ProcessPoolExecutor`.
+
+.. class:: Executor()
+
+   An abstract class that provides methods to execute calls asynchronously. It
+   should not be used directly, but through its two subclasses:
+   :class:`ThreadPoolExecutor` and :class:`ProcessPoolExecutor`.
+
+    .. method:: submit(fn, *args, **kwargs)
+
+       Schedules the callable to be executed as *fn*(*\*args*, *\*\*kwargs*) and
+       returns a :class:`Future` representing the execution of the callable.
+
+       ::
+
+        with ThreadPoolExecutor(max_workers=1) as executor:
+            future = executor.submit(pow, 323, 1235)
+            print(future.result())
+
+    .. method:: map(func, *iterables, timeout=None)
+
+       Equivalent to `map(*func*, *\*iterables*)` but func is executed
+       asynchronously and several calls to *func* may be made concurrently. The
+       returned iterator raises a :exc:`TimeoutError` if :meth:`__next__()` is
+       called and the result isn't available after *timeout* seconds from the
+       original call to :meth:`Executor.map()`. *timeout* can be an int or
+       float. If *timeout* is not specified or ``None`` then there is no limit
+       to the wait time. If a call raises an exception then that exception will
+       be raised when its value is retrieved from the iterator.
+
+    .. method:: shutdown(wait=True)
+
+       Signal the executor that it should free any resources that it is using
+       when the currently pending futures are done executing. Calls to
+       :meth:`Executor.submit` and :meth:`Executor.map` made after shutdown will
+       raise :exc:`RuntimeError`.
+
+       If *wait* is `True` then this method will not return until all the
+       pending futures are done executing and the resources associated with the
+       executor have been freed. If *wait* is `False` then this method will
+       return immediately and the resources associated with the executor will
+       be freed when all pending futures are done executing. Regardless of the
+       value of *wait*, the entire Python program will not exit until all
+       pending futures are done executing.
+
+       You can avoid having to call this method explicitly if you use the `with`
+       statement, which will shutdown the `Executor` (waiting as if
+       `Executor.shutdown` were called with *wait* set to `True`):
+
+       ::
+
+        import shutil
+        with ThreadPoolExecutor(max_workers=4) as e:
+            e.submit(shutil.copy, 'src1.txt', 'dest1.txt')
+            e.submit(shutil.copy, 'src2.txt', 'dest2.txt')
+            e.submit(shutil.copy, 'src3.txt', 'dest3.txt')
+            e.submit(shutil.copy, 'src3.txt', 'dest4.txt')
+
+ThreadPoolExecutor
+^^^^^^^^^^^^^^^^^^
+
+The :class:`ThreadPoolExecutor` class is an :class:`Executor` subclass that uses
+a pool of threads to execute calls asynchronously.
+
+Deadlock can occur when the callable associated with a :class:`Future` waits on
+the results of another :class:`Future`. For example:
+
+::
+
+    import time
+    def wait_on_b():
+        time.sleep(5)
+        print(b.result()) # b will never complete because it is waiting on a.
+        return 5
+
+    def wait_on_a():
+        time.sleep(5)
+        print(a.result()) # a will never complete because it is waiting on b.
+        return 6
+
+
+    executor = ThreadPoolExecutor(max_workers=2)
+    a = executor.submit(wait_on_b)
+    b = executor.submit(wait_on_a)
+
+And:
+
+::
+
+    def wait_on_future():
+        f = executor.submit(pow, 5, 2)
+        # This will never complete because there is only one worker thread and
+        # it is executing this function.
+        print(f.result())
+
+    executor = ThreadPoolExecutor(max_workers=1)
+    executor.submit(wait_on_future)
+
+
+.. class:: ThreadPoolExecutor(max_workers)
+
+   An :class:`Executor` subclass that uses a pool of at most *max_workers*
+   threads to execute calls asynchronously.
+
+   Deadlock can occur when the callable associated with a :class:`Future` waits
+   on the results of another :class:`Future`.
+
+.. _threadpoolexecutor-example:
+
+ThreadPoolExecutor Example
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+::
+
+    import concurrent.futures
+    import urllib.request
+
+    URLS = ['http://www.foxnews.com/',
+            'http://www.cnn.com/',
+            'http://europe.wsj.com/',
+            'http://www.bbc.co.uk/',
+            'http://some-made-up-domain.com/']
+
+    def load_url(url, timeout):
+        return urllib.request.urlopen(url, timeout=timeout).read()
+
+    with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
+        future_to_url = dict((executor.submit(load_url, url, 60), url)
+                             for url in URLS)
+
+        for future in concurrent.futures.as_completed(future_to_url):
+            url = future_to_url[future]
+            if future.exception() is not None:
+                print('%r generated an exception: %s' % (url,
+                                                         future.exception()))
+            else:
+                print('%r page is %d bytes' % (url, len(future.result())))
+
+
+ProcessPoolExecutor
+^^^^^^^^^^^^^^^^^^^
+
+The :class:`ProcessPoolExecutor` class is an :class:`Executor` subclass that
+uses a pool of processes to execute calls asynchronously.
+:class:`ProcessPoolExecutor` uses the :mod:`multiprocessing` module, which
+allows it to side-step the :term:`Global Interpreter Lock` but also means that
+only picklable objects can be executed and returned.
+
+Calling :class:`Executor` or :class:`Future` methods from a callable submitted
+to a :class:`ProcessPoolExecutor` will result in deadlock.
+
+.. class:: ProcessPoolExecutor(max_workers=None)
+
+   An :class:`Executor` subclass that executes calls asynchronously using a
+   pool of at most *max_workers* processes. If *max_workers* is ``None`` or
+   not given then as many worker processes will be created as the machine has
+   processors.
+
+.. _processpoolexecutor-example:
+
+ProcessPoolExecutor Example
+^^^^^^^^^^^^^^^^^^^^^^^^^^^
+::
+
+    import concurrent.futures
+    import math
+
+    PRIMES = [
+        112272535095293,
+        112582705942171,
+        112272535095293,
+        115280095190773,
+        115797848077099,
+        1099726899285419]
+
+    def is_prime(n):
+        if n % 2 == 0:
+            return False
+
+        sqrt_n = int(math.floor(math.sqrt(n)))
+        for i in range(3, sqrt_n + 1, 2):
+            if n % i == 0:
+                return False
+        return True
+
+    def main():
+        with concurrent.futures.ProcessPoolExecutor() as executor:
+            for number, prime in zip(PRIMES, executor.map(is_prime, PRIMES)):
+                print('%d is prime: %s' % (number, prime))
+
+    if __name__ == '__main__':
+        main()
+
+Future Objects
+^^^^^^^^^^^^^^
+
+The :class:`Future` class encapulates the asynchronous execution of a callable.
+:class:`Future` instances are created by :meth:`Executor.submit`.
+
+.. class:: Future()
+
+   Encapulates the asynchronous execution of a callable. :class:`Future`
+   instances are created by :meth:`Executor.submit` and should not be created
+   directly except for testing.
+
+    .. method:: cancel()
+
+       Attempt to cancel the call. If the call is currently being executed then
+       it cannot be cancelled and the method will return `False`, otherwise the
+       call will be cancelled and the method will return `True`.
+
+    .. method:: cancelled()
+
+       Return `True` if the call was successfully cancelled.
+
+    .. method:: running()
+
+       Return `True` if the call is currently being executed and cannot be
+       cancelled.
+
+    .. method:: done()
+
+       Return `True` if the call was successfully cancelled or finished running.
+
+    .. method:: result(timeout=None)
+
+       Return the value returned by the call. If the call hasn't yet completed
+       then this method will wait up to *timeout* seconds. If the call hasn't
+       completed in *timeout* seconds then a :exc:`TimeoutError` will be
+       raised. *timeout* can be an int or float.If *timeout* is not specified
+       or ``None`` then there is no limit to the wait time.
+
+       If the future is cancelled before completing then :exc:`CancelledError`
+       will be raised.
+
+       If the call raised then this method will raise the same exception.
+
+    .. method:: exception(timeout=None)
+
+       Return the exception raised by the call. If the call hasn't yet completed
+       then this method will wait up to *timeout* seconds. If the call hasn't
+       completed in *timeout* seconds then a :exc:`TimeoutError` will be raised.
+       *timeout* can be an int or float. If *timeout* is not specified or
+       ``None`` then there is no limit to the wait time.
+
+       If the future is cancelled before completing then :exc:`CancelledError`
+       will be raised.
+
+       If the call completed without raising then ``None`` is returned.
+
+    .. method:: add_done_callback(fn)
+
+       Attaches the callable *fn* to the future. *fn* will be called, with the
+       future as its only argument, when the future is cancelled or finishes
+       running.
+
+       Added callables are called in the order that they were added and are
+       always called in a thread belonging to the process that added them. If
+       the callable raises an :exc:`Exception` then it will be logged and
+       ignored. If the callable raises another :exc:`BaseException` then the
+       behavior is not defined.
+
+       If the future has already completed or been cancelled then *fn* will be
+       called immediately.
+
+   The following :class:`Future` methods are meant for use in unit tests and
+   :class:`Executor` implementations.
+
+    .. method:: set_running_or_notify_cancel()
+
+       This method should only be called by :class:`Executor` implementations
+       before executing the work associated with the :class:`Future` and by
+       unit tests.
+
+       If the method returns `False` then the :class:`Future` was cancelled i.e.
+       :meth:`Future.cancel` was called and returned `True`. Any threads waiting
+       on the :class:`Future` completing (i.e. through :func:`as_completed` or
+       :func:`wait`) will be woken up.
+
+       If the method returns `True` then the :class:`Future` was not cancelled
+       and has been put in the running state i.e. calls to
+       :meth:`Future.running` will return `True`.
+
+       This method can only be called once and cannot be called after
+       :meth:`Future.set_result` or :meth:`Future.set_exception` have been
+       called.
+
+    .. method:: set_result(result)
+
+       Sets the result of the work associated with the :class:`Future` to
+       *result*.
+
+       This method should only be used by :class:`Executor` implementations and
+       unit tests.
+
+    .. method:: set_exception(exception)
+
+       Sets the result of the work associated with the :class:`Future` to the
+       :class:`Exception` *exception*.
+
+       This method should only be used by :class:`Executor` implementations and
+       unit tests.
+
+
+Module Functions
+^^^^^^^^^^^^^^^^
+
+.. function:: wait(fs, timeout=None, return_when=ALL_COMPLETED)
+
+   Wait for the :class:`Future` instances (possibly created by different
+   :class:`Executor` instances) given by *fs*  to complete. Returns a named
+   2-tuple of sets. The first set, named "done", contains the futures that
+   completed (finished or were cancelled) before the wait completed. The second
+   set, named "not_done", contains uncompleted futures.
+
+   *timeout* can be used to control the maximum number of seconds to wait before
+   returning. *timeout* can be an int or float. If *timeout* is not specified or
+   ``None`` then there is no limit to the wait time.
+
+   *return_when* indicates when this function should return. It must be one of
+   the following constants:
+
+      +-----------------------------+----------------------------------------+
+      | Constant                    | Description                            |
+      +=============================+========================================+
+      | :const:`FIRST_COMPLETED`    | The function will return when any      |
+      |                             | future finishes or is cancelled.       |
+      +-----------------------------+----------------------------------------+
+      | :const:`FIRST_EXCEPTION`    | The function will return when any      |
+      |                             | future finishes by raising an          |
+      |                             | exception. If no future raises an      |
+      |                             | exception then it is equivalent to     |
+      |                             | `ALL_COMPLETED`.                       |
+      +-----------------------------+----------------------------------------+
+      | :const:`ALL_COMPLETED`      | The function will return when all      |
+      |                             | futures finish or are cancelled.       |
+      +-----------------------------+----------------------------------------+
+
+.. function:: as_completed(fs, timeout=None)
+
+   Returns an iterator over the :class:`Future` instances  (possibly created
+   by different :class:`Executor` instances) given by *fs* that yields futures
+   as they complete (finished or were cancelled). Any futures that completed
+   before :func:`as_completed()` was called will be yielded first. The returned
+   iterator raises a :exc:`TimeoutError` if :meth:`__next__()` is called and
+   the result isn't available after *timeout* seconds from the original call
+   to :func:`as_completed()`. *timeout* can be an int or float. If *timeout*
+   is not specified or ``None`` then there is no limit to the wait time.