summaryrefslogtreecommitdiffstats
path: root/Doc
diff options
context:
space:
mode:
authorYury Selivanov <yselivanov@sprymix.com>2015-06-22 16:19:30 (GMT)
committerYury Selivanov <yselivanov@sprymix.com>2015-06-22 16:19:30 (GMT)
commit5376ba9630e45ad177150ae68c9712640330a2fc (patch)
tree68eacabe0721f40098654fe5f2e0b0e3391d95c1 /Doc
parentcd881b850c95cdb410620f3acc6ebf37e5467192 (diff)
downloadcpython-5376ba9630e45ad177150ae68c9712640330a2fc.zip
cpython-5376ba9630e45ad177150ae68c9712640330a2fc.tar.gz
cpython-5376ba9630e45ad177150ae68c9712640330a2fc.tar.bz2
Issue #24400: Introduce a distinct type for 'async def' coroutines.
Summary of changes: 1. Coroutines now have a distinct, separate from generators type at the C level: PyGen_Type, and a new typedef PyCoroObject. PyCoroObject shares the initial segment of struct layout with PyGenObject, making it possible to reuse existing generators machinery. The new type is exposed as 'types.CoroutineType'. As a consequence of having a new type, CO_GENERATOR flag is no longer applied to coroutines. 2. Having a separate type for coroutines made it possible to add an __await__ method to the type. Although it is not used by the interpreter (see details on that below), it makes coroutines naturally (without using __instancecheck__) conform to collections.abc.Coroutine and collections.abc.Awaitable ABCs. [The __instancecheck__ is still used for generator-based coroutines, as we don't want to add __await__ for generators.] 3. Add new opcode: GET_YIELD_FROM_ITER. The opcode is needed to allow passing native coroutines to the YIELD_FROM opcode. Before this change, 'yield from o' expression was compiled to: (o) GET_ITER LOAD_CONST YIELD_FROM Now, we use GET_YIELD_FROM_ITER instead of GET_ITER. The reason for adding a new opcode is that GET_ITER is used in some contexts (such as 'for .. in' loops) where passing a coroutine object is invalid. 4. Add two new introspection functions to the inspec module: getcoroutinestate(c) and getcoroutinelocals(c). 5. inspect.iscoroutine(o) is updated to test if 'o' is a native coroutine object. Before this commit it used abc.Coroutine, and it was requested to update inspect.isgenerator(o) to use abc.Generator; it was decided, however, that inspect functions should really be tailored for checking for native types. 6. sys.set_coroutine_wrapper(w) API is updated to work with only native coroutines. Since types.coroutine decorator supports any type of callables now, it would be confusing that it does not work for all types of coroutines. 7. Exceptions logic in generators C implementation was updated to raise clearer messages for coroutines: Before: TypeError("generator raised StopIteration") After: TypeError("coroutine raised StopIteration")
Diffstat (limited to 'Doc')
-rw-r--r--Doc/c-api/concrete.rst1
-rw-r--r--Doc/c-api/coro.rst34
-rw-r--r--Doc/c-api/gen.rst20
-rw-r--r--Doc/data/refcounts.dat6
-rw-r--r--Doc/glossary.rst25
-rw-r--r--Doc/library/dis.rst8
-rw-r--r--Doc/library/inspect.rst57
-rw-r--r--Doc/library/sys.rst5
-rw-r--r--Doc/library/types.rst8
-rw-r--r--Doc/reference/compound_stmts.rst21
-rw-r--r--Doc/whatsnew/3.5.rst11
11 files changed, 156 insertions, 40 deletions
diff --git a/Doc/c-api/concrete.rst b/Doc/c-api/concrete.rst
index 2d56386..47dab81 100644
--- a/Doc/c-api/concrete.rst
+++ b/Doc/c-api/concrete.rst
@@ -112,5 +112,6 @@ Other Objects
weakref.rst
capsule.rst
gen.rst
+ coro.rst
datetime.rst
diff --git a/Doc/c-api/coro.rst b/Doc/c-api/coro.rst
new file mode 100644
index 0000000..2fe50b5
--- /dev/null
+++ b/Doc/c-api/coro.rst
@@ -0,0 +1,34 @@
+.. highlightlang:: c
+
+.. _coro-objects:
+
+Coroutine Objects
+-----------------
+
+.. versionadded:: 3.5
+
+Coroutine objects are what functions declared with an ``async`` keyword
+return.
+
+
+.. c:type:: PyCoroObject
+
+ The C structure used for coroutine objects.
+
+
+.. c:var:: PyTypeObject PyCoro_Type
+
+ The type object corresponding to coroutine objects.
+
+
+.. c:function:: int PyCoro_CheckExact(PyObject *ob)
+
+ Return true if *ob*'s type is *PyCoro_Type*; *ob* must not be *NULL*.
+
+
+.. c:function:: PyObject* PyCoro_New(PyFrameObject *frame, PyObject *name, PyObject *qualname)
+
+ Create and return a new coroutine object based on the *frame* object,
+ with ``__name__`` and ``__qualname__`` set to *name* and *qualname*.
+ A reference to *frame* is stolen by this function. The *frame* argument
+ must not be *NULL*.
diff --git a/Doc/c-api/gen.rst b/Doc/c-api/gen.rst
index 33cd27a..3ab073b 100644
--- a/Doc/c-api/gen.rst
+++ b/Doc/c-api/gen.rst
@@ -7,7 +7,7 @@ Generator Objects
Generator objects are what Python uses to implement generator iterators. They
are normally created by iterating over a function that yields values, rather
-than explicitly calling :c:func:`PyGen_New`.
+than explicitly calling :c:func:`PyGen_New` or :c:func:`PyGen_NewWithQualName`.
.. c:type:: PyGenObject
@@ -20,19 +20,25 @@ than explicitly calling :c:func:`PyGen_New`.
The type object corresponding to generator objects
-.. c:function:: int PyGen_Check(ob)
+.. c:function:: int PyGen_Check(PyObject *ob)
Return true if *ob* is a generator object; *ob* must not be *NULL*.
-.. c:function:: int PyGen_CheckExact(ob)
+.. c:function:: int PyGen_CheckExact(PyObject *ob)
- Return true if *ob*'s type is *PyGen_Type* is a generator object; *ob* must not
- be *NULL*.
+ Return true if *ob*'s type is *PyGen_Type*; *ob* must not be *NULL*.
.. c:function:: PyObject* PyGen_New(PyFrameObject *frame)
- Create and return a new generator object based on the *frame* object. A
- reference to *frame* is stolen by this function. The parameter must not be
+ Create and return a new generator object based on the *frame* object.
+ A reference to *frame* is stolen by this function. The argument must not be
*NULL*.
+
+.. c:function:: PyObject* PyGen_NewWithQualName(PyFrameObject *frame, PyObject *name, PyObject *qualname)
+
+ Create and return a new generator object based on the *frame* object,
+ with ``__name__`` and ``__qualname__`` set to *name* and *qualname*.
+ A reference to *frame* is stolen by this function. The *frame* argument
+ must not be *NULL*.
diff --git a/Doc/data/refcounts.dat b/Doc/data/refcounts.dat
index d1c24e5..e388195 100644
--- a/Doc/data/refcounts.dat
+++ b/Doc/data/refcounts.dat
@@ -491,6 +491,12 @@ PyFunction_SetDefaults:PyObject*:defaults:+1:
PyGen_New:PyObject*::+1:
PyGen_New:PyFrameObject*:frame:0:
+PyGen_NewWithQualName:PyObject*::+1:
+PyGen_NewWithQualName:PyFrameObject*:frame:0:
+
+PyCoro_New:PyObject*::+1:
+PyCoro_New:PyFrameObject*:frame:0:
+
Py_InitModule:PyObject*::0:
Py_InitModule:const char*:name::
Py_InitModule:PyMethodDef[]:methods::
diff --git a/Doc/glossary.rst b/Doc/glossary.rst
index 3b7975b..bdbb272 100644
--- a/Doc/glossary.rst
+++ b/Doc/glossary.rst
@@ -333,14 +333,23 @@ Glossary
.. index:: single: generator
generator
- A function which returns an iterator. It looks like a normal function
- except that it contains :keyword:`yield` statements for producing a series
- of values usable in a for-loop or that can be retrieved one at a time with
- the :func:`next` function. Each :keyword:`yield` temporarily suspends
- processing, remembering the location execution state (including local
- variables and pending try-statements). When the generator resumes, it
- picks-up where it left-off (in contrast to functions which start fresh on
- every invocation).
+ A function which returns a :term:`generator iterator`. It looks like a
+ normal function except that it contains :keyword:`yield` expressions
+ for producing a series of values usable in a for-loop or that can be
+ retrieved one at a time with the :func:`next` function.
+
+ Usually refers to a generator function, but may refer to a
+ *generator iterator* in some contexts. In cases where the intended
+ meaning isn't clear, using the full terms avoids ambiguity.
+
+ generator iterator
+ An object created by a :term:`generator` function.
+
+ Each :keyword:`yield` temporarily suspends processing, remembering the
+ location execution state (including local variables and pending
+ try-statements). When the *generator iterator* resumes, it picks-up where
+ it left-off (in contrast to functions which start fresh on every
+ invocation).
.. index:: single: generator expression
diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst
index 2e938ab..836c4c1 100644
--- a/Doc/library/dis.rst
+++ b/Doc/library/dis.rst
@@ -346,6 +346,14 @@ result back on the stack.
Implements ``TOS = iter(TOS)``.
+.. opcode:: GET_YIELD_FROM_ITER
+
+ If ``TOS`` is a :term:`generator iterator` or :term:`coroutine` object
+ it is left as is. Otherwise, implements ``TOS = iter(TOS)``.
+
+ .. versionadded:: 3.5
+
+
**Binary operations**
Binary operations remove the top of the stack (TOS) and the second top-most
diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst
index 98b7cb7..d4ebffd 100644
--- a/Doc/library/inspect.rst
+++ b/Doc/library/inspect.rst
@@ -178,6 +178,16 @@ attributes:
+-----------+-----------------+---------------------------+
| | gi_code | code |
+-----------+-----------------+---------------------------+
+| coroutine | __name__ | name |
++-----------+-----------------+---------------------------+
+| | __qualname__ | qualified name |
++-----------+-----------------+---------------------------+
+| | cr_frame | frame |
++-----------+-----------------+---------------------------+
+| | cr_running | is the coroutine running? |
++-----------+-----------------+---------------------------+
+| | cr_code | code |
++-----------+-----------------+---------------------------+
| builtin | __doc__ | documentation string |
+-----------+-----------------+---------------------------+
| | __name__ | original name of this |
@@ -279,29 +289,16 @@ attributes:
.. function:: iscoroutinefunction(object)
- Return true if the object is a :term:`coroutine function`.
-
- Coroutine functions are defined with an ``async def`` syntax,
- or are generators decorated with :func:`types.coroutine`
- or :func:`asyncio.coroutine`.
-
- The function will return false for plain Python generator
- functions.
+ Return true if the object is a :term:`coroutine function`
+ (a function defined with an :keyword:`async def` syntax).
.. versionadded:: 3.5
.. function:: iscoroutine(object)
- Return true if the object is a :term:`coroutine`.
-
- Coroutines are results of calls of coroutine functions or
- generator functions decorated with :func:`types.coroutine`
- or :func:`asyncio.coroutine`.
-
- The function will return false for plain python generators.
-
- See also :class:`collections.abc.Coroutine`.
+ Return true if the object is a :term:`coroutine` created by an
+ :keyword:`async def` function.
.. versionadded:: 3.5
@@ -1116,8 +1113,8 @@ code execution::
pass
-Current State of a Generator
-----------------------------
+Current State of Generators and Coroutines
+------------------------------------------
When implementing coroutine schedulers and for other advanced uses of
generators, it is useful to determine whether a generator is currently
@@ -1137,6 +1134,21 @@ generator to be determined easily.
.. versionadded:: 3.2
+.. function:: getcoroutinestate(coroutine)
+
+ Get current state of a coroutine object. The function is intended to be
+ used with coroutine objects created by :keyword:`async def` functions, but
+ will accept any coroutine-like object that has ``cr_running`` and
+ ``cr_frame`` attributes.
+
+ Possible states are:
+ * CORO_CREATED: Waiting to start execution.
+ * CORO_RUNNING: Currently being executed by the interpreter.
+ * CORO_SUSPENDED: Currently suspended at an await expression.
+ * CORO_CLOSED: Execution has completed.
+
+ .. versionadded:: 3.5
+
The current internal state of the generator can also be queried. This is
mostly useful for testing purposes, to ensure that internal state is being
updated as expected:
@@ -1161,6 +1173,13 @@ updated as expected:
.. versionadded:: 3.3
+.. function:: getcoroutinelocals(coroutine)
+
+ This function is analogous to :func:`~inspect.getgeneratorlocals`, but
+ works for coroutine objects created by :keyword:`async def` functions.
+
+ .. versionadded:: 3.5
+
.. _inspect-module-cli:
diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst
index 9d62835..144c986 100644
--- a/Doc/library/sys.rst
+++ b/Doc/library/sys.rst
@@ -1075,7 +1075,10 @@ always available.
.. function:: set_coroutine_wrapper(wrapper)
- Allows to intercept creation of :term:`coroutine` objects.
+ Allows intercepting creation of :term:`coroutine` objects (only ones that
+ are created by an :keyword:`async def` function; generators decorated with
+ :func:`types.coroutine` or :func:`asyncio.coroutine` will not be
+ intercepted).
*wrapper* must be either:
diff --git a/Doc/library/types.rst b/Doc/library/types.rst
index 3fb0c9c..d7b14e7 100644
--- a/Doc/library/types.rst
+++ b/Doc/library/types.rst
@@ -90,6 +90,14 @@ Standard names are defined for the following types:
generator function.
+.. data:: CoroutineType
+
+ The type of :term:`coroutine` objects, produced by calling a
+ function defined with an :keyword:`async def` statement.
+
+ .. versionadded:: 3.5
+
+
.. data:: CodeType
.. index:: builtin: compile
diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst
index 58b6d71..c73e886 100644
--- a/Doc/reference/compound_stmts.rst
+++ b/Doc/reference/compound_stmts.rst
@@ -666,6 +666,15 @@ can be used to create instance variables with different implementation details.
Coroutines
==========
+.. index::
+ statement: async def
+ statement: async for
+ statement: async with
+ keyword: async
+ keyword: await
+
+.. versionadded:: 3.5
+
.. _`async def`:
Coroutine function definition
@@ -683,7 +692,11 @@ even if they do not contain ``await`` or ``async`` keywords.
It is a :exc:`SyntaxError` to use :keyword:`yield` expressions in coroutines.
-.. versionadded:: 3.5
+An example of a coroutine function::
+
+ async def func(param1, param2):
+ do_stuff()
+ await some_coroutine()
.. _`async for`:
@@ -725,7 +738,8 @@ Is semantically equivalent to::
See also :meth:`__aiter__` and :meth:`__anext__` for details.
-.. versionadded:: 3.5
+It is a :exc:`SyntaxError` to use ``async for`` statement outside of an
+:keyword:`async def` function.
.. _`async with`:
@@ -762,7 +776,8 @@ Is semantically equivalent to::
See also :meth:`__aenter__` and :meth:`__aexit__` for details.
-.. versionadded:: 3.5
+It is a :exc:`SyntaxError` to use ``async with`` statement outside of an
+:keyword:`async def` function.
.. seealso::
diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst
index 6b9fd02..b0d5efe 100644
--- a/Doc/whatsnew/3.5.rst
+++ b/Doc/whatsnew/3.5.rst
@@ -531,6 +531,9 @@ inspect
and :func:`~inspect.isawaitable` functions. (Contributed by Yury Selivanov
in :issue:`24017`.)
+* New :func:`~inspect.getcoroutinelocals` and :func:`~inspect.getcoroutinestate`
+ functions. (Contributed by Yury Selivanov in :issue:`24400`.)
+
ipaddress
---------
@@ -734,6 +737,9 @@ types
* New :func:`~types.coroutine` function. (Contributed by Yury Selivanov
in :issue:`24017`.)
+* New :class:`~types.CoroutineType`. (Contributed by Yury Selivanov
+ in :issue:`24400`.)
+
urllib
------
@@ -1101,6 +1107,7 @@ Changes in the C API
the :attr:`__module__` attribute. Would be an AttributeError in future.
(:issue:`20204`)
-* As part of PEP 492 implementation, ``tp_reserved`` slot of
+* As part of :pep:`492` implementation, ``tp_reserved`` slot of
:c:type:`PyTypeObject` was replaced with a
- :c:member:`PyTypeObject.tp_as_async` slot.
+ :c:member:`PyTypeObject.tp_as_async` slot. Refer to :ref:`coro-objects` for
+ new types, structures and functions.