diff options
| -rw-r--r-- | Doc/library/asyncio-eventloop.rst | 5 | ||||
| -rw-r--r-- | Doc/library/asyncio-protocol.rst | 2 | ||||
| -rw-r--r-- | Doc/library/asyncio-queue.rst | 169 | ||||
| -rw-r--r-- | Doc/library/asyncio-stream.rst | 4 | ||||
| -rw-r--r-- | Doc/library/asyncio-subprocess.rst | 5 | ||||
| -rw-r--r-- | Doc/library/asyncio-sync.rst | 172 | ||||
| -rw-r--r-- | Doc/library/asyncio-task.rst | 4 | ||||
| -rw-r--r-- | Doc/library/asyncio.rst | 6 |
8 files changed, 201 insertions, 166 deletions
diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index f2c8945..d27eb4b 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -22,6 +22,8 @@ It provides multiple facilities, amongst which: Base class of event loops. + This class is :ref:`not thread safe <asyncio-multithreading>`. + Run an event loop ----------------- @@ -104,6 +106,9 @@ keywords to your callback, use :func:`functools.partial`. For example, Like :meth:`call_soon`, but thread safe. + See the :ref:`concurrency and multithreading <asyncio-multithreading>` + section of the documentation. + .. _asyncio-delayed-calls: diff --git a/Doc/library/asyncio-protocol.rst b/Doc/library/asyncio-protocol.rst index b6fcc48..2e671e8 100644 --- a/Doc/library/asyncio-protocol.rst +++ b/Doc/library/asyncio-protocol.rst @@ -23,6 +23,8 @@ then call the transport's methods for various purposes. subprocess pipes. The methods available on a transport depend on the transport's kind. +The transport classes are :ref:`not thread safe <asyncio-multithreading>`. + BaseTransport ------------- diff --git a/Doc/library/asyncio-queue.rst b/Doc/library/asyncio-queue.rst new file mode 100644 index 0000000..c82e08b --- /dev/null +++ b/Doc/library/asyncio-queue.rst @@ -0,0 +1,169 @@ +.. currentmodule:: asyncio + +Queues +====== + +Queues: + +* :class:`Queue` +* :class:`PriorityQueue` +* :class:`LifoQueue` +* :class:`JoinableQueue` + +asyncio queue API was designed to be close to classes of the :mod:`queue` +module (:class:`~queue.Queue`, :class:`~queue.PriorityQueue`, +:class:`~queue.LifoQueue`), but it has no *timeout* parameter. The +:func:`asyncio.wait_for` function can be used to cancel a task after a timeout. + +Queue +----- + +.. class:: Queue(maxsize=0, \*, loop=None) + + A queue, useful for coordinating producer and consumer coroutines. + + If *maxsize* is less than or equal to zero, the queue size is infinite. If + it is an integer greater than ``0``, then ``yield from put()`` will block + when the queue reaches *maxsize*, until an item is removed by :meth:`get`. + + Unlike the standard library :mod:`queue`, you can reliably know this Queue's + size with :meth:`qsize`, since your single-threaded asyncio application won't + be interrupted between calling :meth:`qsize` and doing an operation on the + Queue. + + This class is :ref:`not thread safe <asyncio-multithreading>`. + + .. versionchanged:: 3.4.3 + New :meth:`join` and :meth:`task_done` methods. + + .. method:: empty() + + Return ``True`` if the queue is empty, ``False`` otherwise. + + .. method:: full() + + Return ``True`` if there are :attr:`maxsize` items in the queue. + + .. note:: + + If the Queue was initialized with ``maxsize=0`` (the default), then + :meth:`full()` is never ``True``. + + .. coroutinemethod:: get() + + Remove and return an item from the queue. If queue is empty, wait until + an item is available. + + This method is a :ref:`coroutine <coroutine>`. + + .. seealso:: + + The :meth:`empty` method. + + .. method:: get_nowait() + + Remove and return an item from the queue. + + Return an item if one is immediately available, else raise + :exc:`QueueEmpty`. + + .. coroutinemethod:: join() + + Block until all items in the queue have been gotten and processed. + + The count of unfinished tasks goes up whenever an item is added to the + queue. The count goes down whenever a consumer thread calls + :meth:`task_done` to indicate that the item was retrieved and all work on + it is complete. When the count of unfinished tasks drops to zero, + :meth:`join` unblocks. + + This method is a :ref:`coroutine <coroutine>`. + + .. versionadded:: 3.4.3 + + .. coroutinemethod:: put(item) + + Put an item into the queue. If the queue is full, wait until a free slot + is available before adding item. + + This method is a :ref:`coroutine <coroutine>`. + + .. seealso:: + + The :meth:`full` method. + + .. method:: put_nowait(item) + + Put an item into the queue without blocking. + + If no free slot is immediately available, raise :exc:`QueueFull`. + + .. method:: qsize() + + Number of items in the queue. + + .. method:: task_done() + + Indicate that a formerly enqueued task is complete. + + Used by queue consumers. For each :meth:`~Queue.get` used to fetch a task, a + subsequent call to :meth:`task_done` tells the queue that the processing + on the task is complete. + + If a :meth:`join` is currently blocking, it will resume when all items + have been processed (meaning that a :meth:`task_done` call was received + for every item that had been :meth:`~Queue.put` into the queue). + + Raises :exc:`ValueError` if called more times than there were items + placed in the queue. + + .. versionadded:: 3.4.3 + + .. attribute:: maxsize + + Number of items allowed in the queue. + + +PriorityQueue +------------- + +.. class:: PriorityQueue + + A subclass of :class:`Queue`; retrieves entries in priority order (lowest + first). + + Entries are typically tuples of the form: (priority number, data). + + +LifoQueue +--------- + +.. class:: LifoQueue + + A subclass of :class:`Queue` that retrieves most recently added entries + first. + + +JoinableQueue +^^^^^^^^^^^^^ + +.. class:: JoinableQueue + + Deprecated alias for :class:`Queue`. + + .. deprecated:: 3.4.3 + + +Exceptions +^^^^^^^^^^ + +.. exception:: QueueEmpty + + Exception raised when the :meth:`~Queue.get_nowait` method is called on a + :class:`Queue` object which is empty. + + +.. exception:: QueueFull + + Exception raised when the :meth:`~Queue.put_nowait` method is called on a + :class:`Queue` object which is full. diff --git a/Doc/library/asyncio-stream.rst b/Doc/library/asyncio-stream.rst index 22b7341..41b24ac 100644 --- a/Doc/library/asyncio-stream.rst +++ b/Doc/library/asyncio-stream.rst @@ -85,6 +85,8 @@ StreamReader .. class:: StreamReader(limit=None, loop=None) + This class is :ref:`not thread safe <asyncio-multithreading>`. + .. method:: exception() Get the exception. @@ -155,6 +157,8 @@ StreamWriter wait for flow control. It also adds a transport attribute which references the :class:`Transport` directly. + This class is :ref:`not thread safe <asyncio-multithreading>`. + .. attribute:: transport Transport. diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index 1334f5b..1b82030 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -193,6 +193,9 @@ Process :meth:`~subprocess.Popen.wait` method of the :class:`~subprocess.Popen` class is implemented as a busy loop. + This class is :ref:`not thread safe <asyncio-multithreading>`. See also the + :ref:`Subprocess and threads <asyncio-subprocess-threads>` section. + .. coroutinemethod:: wait() Wait for child process to terminate. Set and return :attr:`returncode` @@ -310,6 +313,8 @@ are limits: subprocesses from other threads. Call the :func:`get_child_watcher` function in the main thread to instantiate the child watcher. +The :class:`asyncio.subprocess.Process` class is not thread safe. + .. seealso:: The :ref:`Concurrency and multithreading in asyncio diff --git a/Doc/library/asyncio-sync.rst b/Doc/library/asyncio-sync.rst index e3d82b0..f53c7d0 100644 --- a/Doc/library/asyncio-sync.rst +++ b/Doc/library/asyncio-sync.rst @@ -9,22 +9,16 @@ Locks: * :class:`Lock` * :class:`Event` * :class:`Condition` -* :class:`Semaphore` -* :class:`BoundedSemaphore` -Queues: +Semaphores: -* :class:`Queue` -* :class:`PriorityQueue` -* :class:`LifoQueue` -* :class:`JoinableQueue` +* :class:`Semaphore` +* :class:`BoundedSemaphore` -asyncio locks and queues API were designed to be close to classes of the -:mod:`threading` module (:class:`~threading.Lock`, :class:`~threading.Event`, +asyncio lock API was designed to be close to classes of the :mod:`threading` +module (:class:`~threading.Lock`, :class:`~threading.Event`, :class:`~threading.Condition`, :class:`~threading.Semaphore`, -:class:`~threading.BoundedSemaphore`) and the :mod:`queue` module -(:class:`~queue.Queue`, :class:`~queue.PriorityQueue`, -:class:`~queue.LifoQueue`), but they have no *timeout* parameter. The +:class:`~threading.BoundedSemaphore`), but it has no *timeout* parameter. The :func:`asyncio.wait_for` function can be used to cancel a task after a timeout. Locks @@ -290,157 +284,3 @@ BoundedSemaphore This raises :exc:`ValueError` in :meth:`~Semaphore.release` if it would increase the value above the initial value. - -Queues ------- - -Queue -^^^^^ - -.. class:: Queue(maxsize=0, \*, loop=None) - - A queue, useful for coordinating producer and consumer coroutines. - - If *maxsize* is less than or equal to zero, the queue size is infinite. If - it is an integer greater than ``0``, then ``yield from put()`` will block - when the queue reaches *maxsize*, until an item is removed by :meth:`get`. - - Unlike the standard library :mod:`queue`, you can reliably know this Queue's - size with :meth:`qsize`, since your single-threaded asyncio application won't - be interrupted between calling :meth:`qsize` and doing an operation on the - Queue. - - .. versionchanged:: 3.4.3 - New :meth:`join` and :meth:`task_done` methods. - - .. method:: empty() - - Return ``True`` if the queue is empty, ``False`` otherwise. - - .. method:: full() - - Return ``True`` if there are :attr:`maxsize` items in the queue. - - .. note:: - - If the Queue was initialized with ``maxsize=0`` (the default), then - :meth:`full()` is never ``True``. - - .. coroutinemethod:: get() - - Remove and return an item from the queue. If queue is empty, wait until - an item is available. - - This method is a :ref:`coroutine <coroutine>`. - - .. seealso:: - - The :meth:`empty` method. - - .. method:: get_nowait() - - Remove and return an item from the queue. - - Return an item if one is immediately available, else raise - :exc:`QueueEmpty`. - - .. coroutinemethod:: join() - - Block until all items in the queue have been gotten and processed. - - The count of unfinished tasks goes up whenever an item is added to the - queue. The count goes down whenever a consumer thread calls - :meth:`task_done` to indicate that the item was retrieved and all work on - it is complete. When the count of unfinished tasks drops to zero, - :meth:`join` unblocks. - - This method is a :ref:`coroutine <coroutine>`. - - .. versionadded:: 3.4.3 - - .. coroutinemethod:: put(item) - - Put an item into the queue. If the queue is full, wait until a free slot - is available before adding item. - - This method is a :ref:`coroutine <coroutine>`. - - .. seealso:: - - The :meth:`full` method. - - .. method:: put_nowait(item) - - Put an item into the queue without blocking. - - If no free slot is immediately available, raise :exc:`QueueFull`. - - .. method:: qsize() - - Number of items in the queue. - - .. method:: task_done() - - Indicate that a formerly enqueued task is complete. - - Used by queue consumers. For each :meth:`~Queue.get` used to fetch a task, a - subsequent call to :meth:`task_done` tells the queue that the processing - on the task is complete. - - If a :meth:`join` is currently blocking, it will resume when all items - have been processed (meaning that a :meth:`task_done` call was received - for every item that had been :meth:`~Queue.put` into the queue). - - Raises :exc:`ValueError` if called more times than there were items - placed in the queue. - - .. versionadded:: 3.4.3 - - .. attribute:: maxsize - - Number of items allowed in the queue. - - -PriorityQueue -^^^^^^^^^^^^^ - -.. class:: PriorityQueue - - A subclass of :class:`Queue`; retrieves entries in priority order (lowest - first). - - Entries are typically tuples of the form: (priority number, data). - - -LifoQueue -^^^^^^^^^ - -.. class:: LifoQueue - - A subclass of :class:`Queue` that retrieves most recently added entries - first. - - -JoinableQueue -^^^^^^^^^^^^^ - -.. class:: JoinableQueue - - Deprecated alias for :class:`Queue`. - - .. deprecated:: 3.4.3 - - -Exceptions -^^^^^^^^^^ - -.. exception:: QueueEmpty - - Exception raised when the :meth:`~Queue.get_nowait` method is called on a - :class:`Queue` object which is empty. - - -.. exception:: QueueFull - - Exception raised when the :meth:`~Queue.put_nowait` method is called on a - :class:`Queue` object which is full. diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index edc05c3..158a0d8 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -209,6 +209,8 @@ Future :func:`~concurrent.futures.as_completed` functions in the :mod:`concurrent.futures` package. + This class is :ref:`not thread safe <asyncio-multithreading>`. + .. method:: cancel() Cancel the future and schedule callbacks. @@ -375,6 +377,8 @@ Task Don't directly create :class:`Task` instances: use the :func:`async` function or the :meth:`BaseEventLoop.create_task` method. + This class is :ref:`not thread safe <asyncio-multithreading>`. + .. classmethod:: all_tasks(loop=None) Return a set of all tasks for an event loop. diff --git a/Doc/library/asyncio.rst b/Doc/library/asyncio.rst index 6900198..9b4d65e 100644 --- a/Doc/library/asyncio.rst +++ b/Doc/library/asyncio.rst @@ -46,6 +46,11 @@ Here is a more detailed list of the package contents: you absolutely, positively have to use a library that makes blocking I/O calls. +Asynchronous programming is more complex than classical "sequential" +programming: see the :ref:`Develop with asyncio <asyncio-dev>` page which lists +common traps and explains how to avoid them. :ref:`Enable the debug mode +<asyncio-debug-mode>` during development to detect common issues. + Table of contents: .. toctree:: @@ -58,6 +63,7 @@ Table of contents: asyncio-stream.rst asyncio-subprocess.rst asyncio-sync.rst + asyncio-queue.rst asyncio-dev.rst .. seealso:: |