diff options
author | Jesse Noller <jnoller@gmail.com> | 2008-11-28 18:37:06 (GMT) |
---|---|---|
committer | Jesse Noller <jnoller@gmail.com> | 2008-11-28 18:37:06 (GMT) |
commit | 6e158600812c89c3f119929dd87b056e0a858667 (patch) | |
tree | 4d56cdc9c8407f25716f674798feccff02cc5e2f | |
parent | 268be91d0137febefd7d1db47f9ccab078b82dac (diff) | |
download | cpython-6e158600812c89c3f119929dd87b056e0a858667.zip cpython-6e158600812c89c3f119929dd87b056e0a858667.tar.gz cpython-6e158600812c89c3f119929dd87b056e0a858667.tar.bz2 |
Merge r67419 - mp doc fixes
-rw-r--r-- | Doc/includes/mp_distributing.py | 1 | ||||
-rw-r--r-- | Doc/library/multiprocessing.rst | 128 |
2 files changed, 108 insertions, 21 deletions
diff --git a/Doc/includes/mp_distributing.py b/Doc/includes/mp_distributing.py index c8f5068..3e9ffd6 100644 --- a/Doc/includes/mp_distributing.py +++ b/Doc/includes/mp_distributing.py @@ -37,7 +37,6 @@ def get_logger(): _logger = logging.getLogger('distributing')
_logger.propogate = 0
-util.fix_up_logger(_logger)
_formatter = logging.Formatter(util.DEFAULT_LOGGING_FORMAT)
_handler = logging.StreamHandler()
_handler.setFormatter(_formatter)
diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index 0874a80..16bb8f9 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -26,6 +26,29 @@ Windows. import it will result in an :exc:`ImportError`. See :issue:`3770` for additional information. +.. note:: + + Functionality within this package requires that the ``__main__`` method be + importable by the children. This is covered in :ref:`multiprocessing-programming` + however it is worth pointing out here. This means that some examples, such + as the :class:`multiprocessing.Pool` examples will not work in the + interactive interpreter. For example:: + + >>> from multiprocessing import Pool + >>> p = Pool(5) + >>> def f(x): + ... return x*x + ... + >>> p.map(f, [1,2,3]) + Process PoolWorker-1: + Process PoolWorker-2: + Traceback (most recent call last): + Traceback (most recent call last): + AttributeError: 'module' object has no attribute 'f' + AttributeError: 'module' object has no attribute 'f' + AttributeError: 'module' object has no attribute 'f' + + The :class:`Process` class ~~~~~~~~~~~~~~~~~~~~~~~~~~ @@ -34,17 +57,36 @@ object and then calling its :meth:`~Process.start` method. :class:`Process` follows the API of :class:`threading.Thread`. A trivial example of a multiprocess program is :: - from multiprocessing import Process + from multiprocessing import Process - def f(name): - print 'hello', name + def f(name): + print 'hello', name - if __name__ == '__main__': - p = Process(target=f, args=('bob',)) - p.start() - p.join() - -Here the function ``f`` is run in a child process. + if __name__ == '__main__': + p = Process(target=f, args=('bob',)) + p.start() + p.join() + +To show the individual process IDs involved, here is an expanded example:: + + from multiprocessing import Process + import os + + def info(title): + print title + print 'module name:', __name__ + print 'parent process:', os.getppid() + print 'process id:', os.getpid() + + def f(name): + info('function f') + print 'hello', name + + if __name__ == '__main__': + info('main line') + p = Process(target=f, args=('bob',)) + p.start() + p.join() For an explanation of why (on Windows) the ``if __name__ == '__main__'`` part is necessary, see :ref:`multiprocessing-programming`. @@ -234,7 +276,7 @@ For example:: if __name__ == '__main__': pool = Pool(processes=4) # start 4 worker processes - result = pool.applyAsync(f, [10]) # evaluate "f(10)" asynchronously + result = pool.apply_async(f, [10]) # evaluate "f(10)" asynchronously print result.get(timeout=1) # prints "100" unless your computer is *very* slow print pool.map(f, range(10)) # prints "[0, 1, 4,..., 81]" @@ -307,7 +349,7 @@ The :mod:`multiprocessing` package mostly replicates the API of the semantics. Multiple processes may be given the same name. The initial name is set by the constructor. - .. method:: is_alive() + .. method:: is_alive Return whether the process is alive. @@ -816,6 +858,9 @@ object -- see :ref:`multiprocessing-managers`. acceptable. If *block* is ``True`` and *timeout* is not ``None`` then it specifies a timeout in seconds. If *block* is ``False`` then *timeout* is ignored. + + Note that on OS/X ``sem_timedwait`` is unsupported, so timeout arguments + for these will be ignored. .. note:: @@ -1089,6 +1134,27 @@ their parent process exits. The manager classes are defined in the A class method which creates a manager object referring to a pre-existing server process which is using the given address and authentication key. + .. method:: get_server() + + Returns a :class:`Server` object which represents the actual server under + the control of the Manager. The :class:`Server` object supports the + :meth:`serve_forever` method:: + + >>> from multiprocessing.managers import BaseManager + >>> m = BaseManager(address=('', 50000), authkey='abc')) + >>> server = m.get_server() + >>> s.serve_forever() + + :class:`Server` additionally have an :attr:`address` attribute. + + .. method:: connect() + + Connect a local manager object to a remote manager process:: + + >>> from multiprocessing.managers import BaseManager + >>> m = BaseManager(address='127.0.0.1', authkey='abc)) + >>> m.connect() + .. method:: shutdown() Stop the process used by the manager. This is only available if @@ -1267,19 +1333,20 @@ remote clients can access:: >>> queue = Queue.Queue() >>> class QueueManager(BaseManager): pass ... - >>> QueueManager.register('getQueue', callable=lambda:queue) + >>> QueueManager.register('get_queue', callable=lambda:queue) >>> m = QueueManager(address=('', 50000), authkey='abracadabra') - >>> m.serveForever() + >>> s = m.get_server() + >>> s.serveForever() One client can access the server as follows:: >>> from multiprocessing.managers import BaseManager >>> class QueueManager(BaseManager): pass ... - >>> QueueManager.register('getQueue') - >>> m = QueueManager.from_address(address=('foo.bar.org', 50000), - >>> authkey='abracadabra') - >>> queue = m.getQueue() + >>> QueueManager.register('get_queue') + >>> m = QueueManager(address=('foo.bar.org', 50000), authkey='abracadabra') + >>> m.connect() + >>> queue = m.get_queue() >>> queue.put('hello') Another client can also use it:: @@ -1293,6 +1360,27 @@ Another client can also use it:: >>> queue.get() 'hello' +Local processes can also access that queue, using the code from above on the +client to access it remotely:: + + >>> from multiprocessing import Process, Queue + >>> from multiprocessing.managers import BaseManager + >>> class Worker(Process): + ... def __init__(self, q): + ... self.q = q + ... super(Worker, self).__init__() + ... def run(self): + ... self.q.put('local hello') + ... + >>> queue = Queue() + >>> w = Worker(queue) + >>> w.start() + >>> class QueueManager(BaseManager): pass + ... + >>> QueueManager.register('get_queue', callable=lambda: queue) + >>> m = QueueManager(address=('', 50000), authkey='abracadabra') + >>> s = m.get_server() + >>> s.serve_forever() Proxy Objects ~~~~~~~~~~~~~ @@ -1507,7 +1595,7 @@ with the :class:`Pool` class. The class of the result returned by :meth:`Pool.apply_async` and :meth:`Pool.map_async`. - .. method:: get([timeout) + .. method:: get([timeout]) Return the result when it arrives. If *timeout* is not ``None`` and the result does not arrive within *timeout* seconds then @@ -1537,7 +1625,7 @@ The following example demonstrates the use of a pool:: if __name__ == '__main__': pool = Pool(processes=4) # start 4 worker processes - result = pool.applyAsync(f, (10,)) # evaluate "f(10)" asynchronously + result = pool.apply_async(f, (10,)) # evaluate "f(10)" asynchronously print result.get(timeout=1) # prints "100" unless your computer is *very* slow print pool.map(f, range(10)) # prints "[0, 1, 4,..., 81]" @@ -1548,7 +1636,7 @@ The following example demonstrates the use of a pool:: print it.next(timeout=1) # prints "4" unless your computer is *very* slow import time - result = pool.applyAsync(time.sleep, (10,)) + result = pool.apply_async(time.sleep, (10,)) print result.get(timeout=1) # raises TimeoutError |