diff options
Diffstat (limited to 'Doc/library/multiprocessing.rst')
| -rw-r--r-- | Doc/library/multiprocessing.rst | 76 |
1 files changed, 65 insertions, 11 deletions
diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index 8535aed..6a72759 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -98,8 +98,8 @@ necessary, see :ref:`multiprocessing-programming`. -Start methods -~~~~~~~~~~~~~ +Contexts and start methods +~~~~~~~~~~~~~~~~~~~~~~~~~~ Depending on the platform, :mod:`multiprocessing` supports three ways to start a process. These *start methods* are @@ -132,7 +132,7 @@ to start a process. These *start methods* are unnecessary resources are inherited. Available on Unix platforms which support passing file descriptors - over unix pipes. + over Unix pipes. Before Python 3.4 *fork* was the only option available on Unix. Also, prior to Python 3.4, child processes would inherit all the parents @@ -153,18 +153,46 @@ example:: import multiprocessing as mp - def foo(): - print('hello') + def foo(q): + q.put('hello') if __name__ == '__main__': mp.set_start_method('spawn') - p = mp.Process(target=foo) + q = mp.Queue() + p = mp.Process(target=foo, args=(q,)) p.start() + print(q.get()) p.join() :func:`set_start_method` should not be used more than once in the program. +Alternatively, you can use :func:`get_context` to obtain a context +object. Context objects have the same API as the multiprocessing +module, and allow one to use multiple start methods in the same +program. :: + + import multiprocessing as mp + + def foo(q): + q.put('hello') + + if __name__ == '__main__': + ctx = mp.get_context('spawn') + q = ctx.Queue() + p = ctx.Process(target=foo, args=(q,)) + p.start() + print(q.get()) + p.join() + +Note that objects related to one context may not be compatible with +processes for a different context. In particular, locks created using +the *fork* context cannot be passed to a processes started using the +*spawn* or *forkserver* start methods. + +A library which wants to use a particular start method should probably +use :func:`get_context` to avoid interfering with the choice of the +library user. Exchanging objects between processes @@ -859,11 +887,30 @@ Miscellaneous .. versionadded:: 3.4 -.. function:: get_start_method() +.. function:: get_context(method=None) + + Return a context object which has the same attributes as the + :mod:`multiprocessing` module. - Return the current start method. This can be ``'fork'``, - ``'spawn'`` or ``'forkserver'``. ``'fork'`` is the default on - Unix, while ``'spawn'`` is the default on Windows. + If *method* is *None* then the default context is returned. + Otherwise *method* should be ``'fork'``, ``'spawn'``, + ``'forkserver'``. :exc:`ValueError` is raised if the specified + start method is not available. + + .. versionadded:: 3.4 + +.. function:: get_start_method(allow_none=False) + + Return the name of start method used for starting processes. + + If the start method has not been fixed and *allow_none* is false, + then the start method is fixed to the default and the name is + returned. If the start method has not been fixed and *allow_none* + is true then *None* is returned. + + The return value can be ``'fork'``, ``'spawn'``, ``'forkserver'`` + or *None*. ``'fork'`` is the default on Unix, while ``'spawn'`` is + the default on Windows. .. versionadded:: 3.4 @@ -1785,7 +1832,7 @@ Process Pools One can create a pool of processes which will carry out tasks submitted to it with the :class:`Pool` class. -.. class:: Pool([processes[, initializer[, initargs[, maxtasksperchild]]]]) +.. class:: Pool([processes[, initializer[, initargs[, maxtasksperchild [, context]]]]]) A process pool object which controls a pool of worker processes to which jobs can be submitted. It supports asynchronous results with timeouts and @@ -1805,6 +1852,13 @@ with the :class:`Pool` class. unused resources to be freed. The default *maxtasksperchild* is None, which means worker processes will live as long as the pool. + .. versionadded:: 3.4 + *context* can be used to specify the context used for starting + the worker processes. Usually a pool is created using the + function :func:`multiprocessing.Pool` or the :meth:`Pool` method + of a context object. In both cases *context* is set + appropriately. + .. note:: Worker processes within a :class:`Pool` typically live for the complete |