diff options
author | Antoine Pitrou <solipsis@pitrou.net> | 2015-01-11 14:06:39 (GMT) |
---|---|---|
committer | Antoine Pitrou <solipsis@pitrou.net> | 2015-01-11 14:06:39 (GMT) |
commit | 2cae11e87eac4a086b3188112f08ee9ed6269556 (patch) | |
tree | 7716b1d630a17edfa1715dbf71d05f44d267e324 /Doc | |
parent | cc8617b93a34f8dfcff9e686ee91d8fb1802bc2b (diff) | |
parent | 73dd030c8b71a7080648554652912982054b1177 (diff) | |
download | cpython-2cae11e87eac4a086b3188112f08ee9ed6269556.zip cpython-2cae11e87eac4a086b3188112f08ee9ed6269556.tar.gz cpython-2cae11e87eac4a086b3188112f08ee9ed6269556.tar.bz2 |
Issue #22952: improve multiprocessing doc introduction and defer notes until appropriate.
Patch by Davin Potts.
Diffstat (limited to 'Doc')
-rw-r--r-- | Doc/library/multiprocessing.rst | 86 |
1 files changed, 54 insertions, 32 deletions
diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index b919bbe..4b829af 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -16,41 +16,27 @@ to this, the :mod:`multiprocessing` module allows the programmer to fully leverage multiple processors on a given machine. It runs on both Unix and Windows. -.. note:: +The :mod:`multiprocessing` module also introduces APIs which do not have +analogs in the :mod:`threading` module. A prime example of this is the +:class:`~multiprocessing.pool.Pool` object which offers a convenient means of +parallelizing the execution of a function across multiple input values, +distributing the input data across processes (data parallelism). The following +example demonstrates the common practice of defining such functions in a module +so that child processes can successfully import that module. This basic example +of data parallelism using :class:`~multiprocessing.pool.Pool`, :: - Some of this package's functionality requires a functioning shared semaphore - implementation on the host operating system. Without one, the - :mod:`multiprocessing.synchronize` module will be disabled, and attempts to - import it will result in an :exc:`ImportError`. See - :issue:`3770` for additional information. + from multiprocessing import Pool -.. note:: + def f(x): + return x*x + + if __name__ == '__main__': + with Pool(5) as p: + print(p.map(f, [1, 2, 3])) - Functionality within this package requires that the ``__main__`` module be - importable by the children. This is covered in :ref:`multiprocessing-programming` - however it is worth pointing out here. This means that some examples, such - as the :class:`multiprocessing.pool.Pool` examples will not work in the - interactive interpreter. For example:: - - >>> from multiprocessing import Pool - >>> p = Pool(5) - >>> def f(x): - ... return x*x - ... - >>> p.map(f, [1,2,3]) - Process PoolWorker-1: - Process PoolWorker-2: - Process PoolWorker-3: - Traceback (most recent call last): - Traceback (most recent call last): - Traceback (most recent call last): - AttributeError: 'module' object has no attribute 'f' - AttributeError: 'module' object has no attribute 'f' - AttributeError: 'module' object has no attribute 'f' - - (If you try this it will actually output three full tracebacks - interleaved in a semi-random fashion, and then you may have to - stop the master process somehow.) +will print to standard output :: + + [1, 4, 9] The :class:`Process` class @@ -276,6 +262,14 @@ that only one process prints to standard output at a time:: Without using the lock output from the different processes is liable to get all mixed up. +.. note:: + + Some of this package's functionality requires a functioning shared semaphore + implementation on the host operating system. Without one, the + :mod:`multiprocessing.synchronize` module will be disabled, and attempts to + import it will result in an :exc:`ImportError`. See + :issue:`3770` for additional information. + Sharing state between processes ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -406,6 +400,34 @@ For example:: Note that the methods of a pool should only ever be used by the process which created it. +.. note:: + + Functionality within this package requires that the ``__main__`` module be + importable by the children. This is covered in :ref:`multiprocessing-programming` + however it is worth pointing out here. This means that some examples, such + as the :class:`multiprocessing.pool.Pool` examples will not work in the + interactive interpreter. For example:: + + >>> from multiprocessing import Pool + >>> p = Pool(5) + >>> def f(x): + ... return x*x + ... + >>> p.map(f, [1,2,3]) + Process PoolWorker-1: + Process PoolWorker-2: + Process PoolWorker-3: + Traceback (most recent call last): + Traceback (most recent call last): + Traceback (most recent call last): + AttributeError: 'module' object has no attribute 'f' + AttributeError: 'module' object has no attribute 'f' + AttributeError: 'module' object has no attribute 'f' + + (If you try this it will actually output three full tracebacks + interleaved in a semi-random fashion, and then you may have to + stop the master process somehow.) + Reference --------- |