diff options
author | R David Murray <rdmurray@bitdance.com> | 2014-03-10 18:27:21 (GMT) |
---|---|---|
committer | R David Murray <rdmurray@bitdance.com> | 2014-03-10 18:27:21 (GMT) |
commit | d840b8d6139ed0784d7fea1293a6b7d88a1d8a4f (patch) | |
tree | b440a8afe089af1d2e159114299bacdbbb7ee489 /Doc | |
parent | f2e677cec97b4e1198bde5fe65cd8a9610cd322b (diff) | |
download | cpython-d840b8d6139ed0784d7fea1293a6b7d88a1d8a4f.zip cpython-d840b8d6139ed0784d7fea1293a6b7d88a1d8a4f.tar.gz cpython-d840b8d6139ed0784d7fea1293a6b7d88a1d8a4f.tar.bz2 |
whatsnew: multiprocessing start methods and context (#8713 and #18999)
Also tweaked the docs a bit to use our standard style for
versionadded/changed. (I'm guessing there are other places
in the multiprocessing docs where similar tweaks should be made.)
Diffstat (limited to 'Doc')
-rw-r--r-- | Doc/library/multiprocessing.rst | 35 | ||||
-rw-r--r-- | Doc/whatsnew/3.4.rst | 24 |
2 files changed, 40 insertions, 19 deletions
diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index f757b31..16aabd5 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -136,9 +136,11 @@ to start a process. These *start methods* are Available on Unix platforms which support passing file descriptors over Unix pipes. -Before Python 3.4 *fork* was the only option available on Unix. Also, -prior to Python 3.4, child processes would inherit all the parents -inheritable handles on Windows. +.. versionchanged:: 3.4 + *span* added on all unix platforms, and *forkserver* added for + some unix platforms. + Child processes no longer inherit all of the parents inheritable + handles on Windows. On Unix using the *spawn* or *forkserver* start methods will also start a *semaphore tracker* process which tracks the unlinked named @@ -1853,25 +1855,30 @@ with the :class:`Pool` class. callbacks and has a parallel map implementation. *processes* is the number of worker processes to use. If *processes* is - ``None`` then the number returned by :func:`os.cpu_count` is used. If - *initializer* is not ``None`` then each worker process will call + ``None`` then the number returned by :func:`os.cpu_count` is used. + + If *initializer* is not ``None`` then each worker process will call ``initializer(*initargs)`` when it starts. + *maxtasksperchild* is the number of tasks a worker process can complete + before it will exit and be replaced with a fresh worker process, to enable + unused resources to be freed. The default *maxtasksperchild* is None, which + means worker processes will live as long as the pool. + + *context* can be used to specify the context used for starting + the worker processes. Usually a pool is created using the + function :func:`multiprocessing.Pool` or the :meth:`Pool` method + of a context object. In both cases *context* is set + appropriately. + Note that the methods of the pool object should only be called by the process which created the pool. .. versionadded:: 3.2 - *maxtasksperchild* is the number of tasks a worker process can complete - before it will exit and be replaced with a fresh worker process, to enable - unused resources to be freed. The default *maxtasksperchild* is None, which - means worker processes will live as long as the pool. + *maxtasksperchild* .. versionadded:: 3.4 - *context* can be used to specify the context used for starting - the worker processes. Usually a pool is created using the - function :func:`multiprocessing.Pool` or the :meth:`Pool` method - of a context object. In both cases *context* is set - appropriately. + *context* .. note:: diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst index 38c0250..8cf9165 100644 --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -1063,11 +1063,25 @@ On Unix, two new :ref:`start methods <multiprocessing-start-methods>` (``spawn`` and ``forkserver``) have been added for starting processes using :mod:`multiprocessing`. These make the mixing of processes with threads more robust, and the ``spawn`` method matches the semantics that multiprocessing has -always used on Windows. (Contributed by Richard Oudkerk in :issue:`8713`). - -Also, except when using the old *fork* start method, child processes -will no longer inherit unneeded handles/file descriptors from their parents -(part of :issue:`8713`). +always used on Windows. New function +:func:`~multiprocessing.get_all_start_methods` reports all start methods +available on the platform, :func:`~multiprocessing.get_start_method` reports +the current start method, and :func:`~multiprocessing.set_start_method` sets +the start method. (Contributed by Richard Oudkerk in :issue:`8713`). + +:mod:`multiprocessing` also now has the concept of a ``context``, which +determines how child processes are created. New function +:func:`~multiprocessing.get_context` returns a context that uses a specified +start method. It has the same API as the :mod:`multiprocessing` module itself, +so you can use it to create :class:`~multiprocessing.pool.Pool`\ s and other +objects that will operate within that context. This allows a framework and an +application or different parts of the same application to use multiprocessing +without interfering with each other. (Contributed by Richard Oudkerk in +:issue:`18999`.) + +Except when using the old *fork* start method, child processes no longer +inherit unneeded handles/file descriptors from their parents (part of +:issue:`8713`). :mod:`multiprocessing` now relies on :mod:`runpy` (which implements the ``-m`` switch) to initialise ``__main__`` appropriately in child processes |