summaryrefslogtreecommitdiffstats
path: root/Doc
diff options
context:
space:
mode:
authorAntoine Pitrou <solipsis@pitrou.net>2015-01-11 14:05:29 (GMT)
committerAntoine Pitrou <solipsis@pitrou.net>2015-01-11 14:05:29 (GMT)
commit73dd030c8b71a7080648554652912982054b1177 (patch)
treefddba10dd91672d57933d5b9e5dedfbbae44d5b8 /Doc
parentdfe0b2326b9700f5f5f5af566488dfe913348ab9 (diff)
downloadcpython-73dd030c8b71a7080648554652912982054b1177.zip
cpython-73dd030c8b71a7080648554652912982054b1177.tar.gz
cpython-73dd030c8b71a7080648554652912982054b1177.tar.bz2
Issue #22952: improve multiprocessing doc introduction and defer notes until appropriate.
Patch by Davin Potts.
Diffstat (limited to 'Doc')
-rw-r--r--Doc/library/multiprocessing.rst86
1 files changed, 54 insertions, 32 deletions
diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
index dc0370a..ecac5b0 100644
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -16,41 +16,27 @@ to this, the :mod:`multiprocessing` module allows the programmer to fully
leverage multiple processors on a given machine. It runs on both Unix and
Windows.
-.. note::
+The :mod:`multiprocessing` module also introduces APIs which do not have
+analogs in the :mod:`threading` module. A prime example of this is the
+:class:`~multiprocessing.pool.Pool` object which offers a convenient means of
+parallelizing the execution of a function across multiple input values,
+distributing the input data across processes (data parallelism). The following
+example demonstrates the common practice of defining such functions in a module
+so that child processes can successfully import that module. This basic example
+of data parallelism using :class:`~multiprocessing.pool.Pool`, ::
- Some of this package's functionality requires a functioning shared semaphore
- implementation on the host operating system. Without one, the
- :mod:`multiprocessing.synchronize` module will be disabled, and attempts to
- import it will result in an :exc:`ImportError`. See
- :issue:`3770` for additional information.
+ from multiprocessing import Pool
-.. note::
+ def f(x):
+ return x*x
+
+ if __name__ == '__main__':
+ with Pool(5) as p:
+ print(p.map(f, [1, 2, 3]))
- Functionality within this package requires that the ``__main__`` module be
- importable by the children. This is covered in :ref:`multiprocessing-programming`
- however it is worth pointing out here. This means that some examples, such
- as the :class:`multiprocessing.pool.Pool` examples will not work in the
- interactive interpreter. For example::
-
- >>> from multiprocessing import Pool
- >>> p = Pool(5)
- >>> def f(x):
- ... return x*x
- ...
- >>> p.map(f, [1,2,3])
- Process PoolWorker-1:
- Process PoolWorker-2:
- Process PoolWorker-3:
- Traceback (most recent call last):
- Traceback (most recent call last):
- Traceback (most recent call last):
- AttributeError: 'module' object has no attribute 'f'
- AttributeError: 'module' object has no attribute 'f'
- AttributeError: 'module' object has no attribute 'f'
-
- (If you try this it will actually output three full tracebacks
- interleaved in a semi-random fashion, and then you may have to
- stop the master process somehow.)
+will print to standard output ::
+
+ [1, 4, 9]
The :class:`Process` class
@@ -276,6 +262,14 @@ that only one process prints to standard output at a time::
Without using the lock output from the different processes is liable to get all
mixed up.
+.. note::
+
+ Some of this package's functionality requires a functioning shared semaphore
+ implementation on the host operating system. Without one, the
+ :mod:`multiprocessing.synchronize` module will be disabled, and attempts to
+ import it will result in an :exc:`ImportError`. See
+ :issue:`3770` for additional information.
+
Sharing state between processes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -406,6 +400,34 @@ For example::
Note that the methods of a pool should only ever be used by the
process which created it.
+.. note::
+
+ Functionality within this package requires that the ``__main__`` module be
+ importable by the children. This is covered in :ref:`multiprocessing-programming`
+ however it is worth pointing out here. This means that some examples, such
+ as the :class:`multiprocessing.pool.Pool` examples will not work in the
+ interactive interpreter. For example::
+
+ >>> from multiprocessing import Pool
+ >>> p = Pool(5)
+ >>> def f(x):
+ ... return x*x
+ ...
+ >>> p.map(f, [1,2,3])
+ Process PoolWorker-1:
+ Process PoolWorker-2:
+ Process PoolWorker-3:
+ Traceback (most recent call last):
+ Traceback (most recent call last):
+ Traceback (most recent call last):
+ AttributeError: 'module' object has no attribute 'f'
+ AttributeError: 'module' object has no attribute 'f'
+ AttributeError: 'module' object has no attribute 'f'
+
+ (If you try this it will actually output three full tracebacks
+ interleaved in a semi-random fashion, and then you may have to
+ stop the master process somehow.)
+
Reference
---------