gh-103921: Document PEP 695 (#104642)

Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>

5 files changed, 422 insertions, 12 deletions

diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst
index 9d1e5b6..6d30ecc 100644
--- a/Doc/reference/compound_stmts.rst
+++ b/Doc/reference/compound_stmts.rst
@@ -1206,7 +1206,7 @@ A function definition defines a user-defined function object (see section
 :ref:`types`):
 
 .. productionlist:: python-grammar
-   funcdef: [`decorators`] "def" `funcname` "(" [`parameter_list`] ")"
+   funcdef: [`decorators`] "def" `funcname` [`type_params`] "(" [`parameter_list`] ")"
           : ["->" `expression`] ":" `suite`
    decorators: `decorator`+
    decorator: "@" `assignment_expression` NEWLINE
@@ -1256,6 +1256,15 @@ except that the original function is not temporarily bound to the name ``func``.
    :token:`~python-grammar:assignment_expression`. Previously, the grammar was
    much more restrictive; see :pep:`614` for details.
 
+A list of :ref:`type parameters <type-params>` may be given in square brackets
+between the function's name and the opening parenthesis for its parameter list.
+This indicates to static type checkers that the function is generic. At runtime,
+the type parameters can be retrieved from the function's ``__type_params__``
+attribute. See :ref:`generic-functions` for more.
+
+.. versionchanged:: 3.12
+   Type parameter lists are new in Python 3.12.
+
 .. index::
    triple: default; parameter; value
    single: argument; function definition
@@ -1378,7 +1387,7 @@ Class definitions
 A class definition defines a class object (see section :ref:`types`):
 
 .. productionlist:: python-grammar
-   classdef: [`decorators`] "class" `classname` [`inheritance`] ":" `suite`
+   classdef: [`decorators`] "class" `classname` [`type_params`] [`inheritance`] ":" `suite`
    inheritance: "(" [`argument_list`] ")"
    classname: `identifier`
 
@@ -1434,6 +1443,15 @@ decorators.  The result is then bound to the class name.
    :token:`~python-grammar:assignment_expression`. Previously, the grammar was
    much more restrictive; see :pep:`614` for details.
 
+A list of :ref:`type parameters <type-params>` may be given in square brackets
+immediately after the class's name.
+This indicates to static type checkers that the class is generic. At runtime,
+the type parameters can be retrieved from the class's ``__type_params__``
+attribute. See :ref:`generic-classes` for more.
+
+.. versionchanged:: 3.12
+   Type parameter lists are new in Python 3.12.
+
 **Programmer's note:** Variables defined in the class definition are class
 attributes; they are shared by instances.  Instance attributes can be set in a
 method with ``self.name = value``.  Both class and instance attributes are
@@ -1589,6 +1607,228 @@ body of a coroutine function.
       The proposal that made coroutines a proper standalone concept in Python,
       and added supporting syntax.
 
+.. _type-params:
+
+Type parameter lists
+====================
+
+.. versionadded:: 3.12
+
+.. index::
+   single: type parameters
+
+.. productionlist:: python-grammar
+   type_params: "[" `type_param` ("," `type_param`)* "]"
+   type_param: `typevar` | `typevartuple` | `paramspec`
+   typevar: `identifier` (":" `expression`)?
+   typevartuple: "*" `identifier`
+   paramspec: "**" `identifier`
+
+:ref:`Functions <def>` (including :ref:`coroutines <async def>`),
+:ref:`classes <class>` and :ref:`type aliases <type>` may
+contain a type parameter list::
+
+   def max[T](args: list[T]) -> T:
+       ...
+
+   async def amax[T](args: list[T]) -> T:
+       ...
+
+   class Bag[T]:
+       def __iter__(self) -> Iterator[T]:
+           ...
+
+       def add(self, arg: T) -> None:
+           ...
+
+   type ListOrSet[T] = list[T] | set[T]
+
+Semantically, this indicates that the function, class, or type alias is
+generic over a type variable. This information is primarily used by static
+type checkers, and at runtime, generic objects behave much like their
+non-generic counterparts.
+
+Type parameters are declared in square brackets (``[]``) immediately
+after the name of the function, class, or type alias. The type parameters
+are accessible within the scope of the generic object, but not elsewhere.
+Thus, after a declaration ``def func[T](): pass``, the name ``T`` is not available in
+the module scope. Below, the semantics of generic objects are described
+with more precision. The scope of type parameters is modeled with a special
+function (technically, an :ref:`annotation scope <annotation-scopes>`) that
+wraps the creation of the generic object.
+
+Generic functions, classes, and type aliases have a :attr:`!__type_params__`
+attribute listing their type parameters.
+
+Type parameters come in three kinds:
+
+* :data:`typing.TypeVar`, introduced by a plain name (e.g., ``T``). Semantically, this
+  represents a single type to a type checker.
+* :data:`typing.TypeVarTuple`, introduced by a name prefixed with a single
+  asterisk (e.g., ``*Ts``). Semantically, this stands for a tuple of any
+  number of types.
+* :data:`typing.ParamSpec`, introduced by a name prefixed with two asterisks
+  (e.g., ``**P``). Semantically, this stands for the parameters of a callable.
+
+:data:`typing.TypeVar` declarations can define *bounds* and *constraints* with
+a colon (``:``) followed by an expression. A single expression after the colon
+indicates a bound (e.g. ``T: int``). Semantically, this means
+that the :data:`!typing.TypeVar` can only represent types that are a subtype of
+this bound. A parenthesized tuple of expressions after the colon indicates a
+set of constraints (e.g. ``T: (str, bytes)``). Each member of the tuple should be a
+type (again, this is not enforced at runtime). Constrained type variables can only
+take on one of the types in the list of constraints.
+
+For :data:`!typing.TypeVar`\ s declared using the type parameter list syntax,
+the bound and constraints are not evaluated when the generic object is created,
+but only when the value is explicitly accessed through the attributes ``__bound__``
+and ``__constraints__``. To accomplish this, the bounds or constraints are
+evaluated in a separate :ref:`annotation scope <annotation-scopes>`.
+
+:data:`typing.TypeVarTuple`\ s and :data:`typing.ParamSpec`\ s cannot have bounds
+or constraints.
+
+The following example indicates the full set of allowed type parameter declarations::
+
+   def overly_generic[
+      SimpleTypeVar,
+      TypeVarWithBound: int,
+      TypeVarWithConstraints: (str, bytes),
+      *SimpleTypeVarTuple,
+      **SimpleParamSpec,
+   ](
+      a: SimpleTypeVar,
+      b: TypeVarWithBound,
+      c: Callable[SimpleParamSpec, TypeVarWithConstraints],
+      *d: SimpleTypeVarTuple,
+   ): ...
+
+.. _generic-functions:
+
+Generic functions
+-----------------
+
+Generic functions are declared as follows::
+
+   def func[T](arg: T): ...
+
+This syntax is equivalent to::
+
+   annotation-def TYPE_PARAMS_OF_func():
+       T = typing.TypeVar("T")
+       def func(arg: T): ...
+       func.__type_params__ = (T,)
+       return func
+   func = TYPE_PARAMS_OF_func()
+
+Here ``annotation-def`` indicates an :ref:`annotation scope <annotation-scopes>`,
+which is not actually bound to any name at runtime. (One
+other liberty is taken in the translation: the syntax does not go through
+attribute access on the :mod:`typing` module, but creates an instance of
+:data:`typing.TypeVar` directly.)
+
+The annotations of generic functions are evaluated within the annotation scope
+used for declaring the type parameters, but the function's defaults and
+decorators are not.
+
+The following example illustrates the scoping rules for these cases,
+as well as for additional flavors of type parameters::
+
+   @decorator
+   def func[T: int, *Ts, **P](*args: *Ts, arg: Callable[P, T] = some_default):
+       ...
+
+Except for the :ref:`lazy evaluation <lazy-evaluation>` of the
+:class:`~typing.TypeVar` bound, this is equivalent to::
+
+   DEFAULT_OF_arg = some_default
+
+   annotation-def TYPE_PARAMS_OF_func():
+
+       annotation-def BOUND_OF_T():
+           return int
+       # In reality, BOUND_OF_T() is evaluated only on demand.
+       T = typing.TypeVar("T", bound=BOUND_OF_T())
+
+       Ts = typing.TypeVarTuple("Ts")
+       P = typing.ParamSpec("P")
+
+       def func(*args: *Ts, arg: Callable[P, T] = DEFAULT_OF_arg):
+           ...
+
+       func.__type_params__ = (T, Ts, P)
+       return func
+   func = decorator(TYPE_PARAMS_OF_func())
+
+The capitalized names like ``DEFAULT_OF_arg`` are not actually
+bound at runtime.
+
+.. _generic-classes:
+
+Generic classes
+---------------
+
+Generic classes are declared as follows::
+
+   class Bag[T]: ...
+
+This syntax is equivalent to::
+
+   annotation-def TYPE_PARAMS_OF_Bag():
+       T = typing.TypeVar("T")
+       class Bag(typing.Generic[T]):
+           __type_params__ = (T,)
+           ...
+       return Bag
+   Bag = TYPE_PARAMS_OF_Bag()
+
+Here again ``annotation-def`` (not a real keyword) indicates an
+:ref:`annotation scope <annotation-scopes>`, and the name
+``TYPE_PARAMS_OF_Bag`` is not actually bound at runtime.
+
+Generic classes implicitly inherit from :data:`typing.Generic`.
+The base classes and keyword arguments of generic classes are
+evaluated within the type scope for the type parameters,
+and decorators are evaluated outside that scope. This is illustrated
+by this example::
+
+   @decorator
+   class Bag(Base[T], arg=T): ...
+
+This is equivalent to::
+
+   annotation-def TYPE_PARAMS_OF_Bag():
+       T = typing.TypeVar("T")
+       class Bag(Base[T], typing.Generic[T], arg=T):
+           __type_params__ = (T,)
+           ...
+       return Bag
+   Bag = decorator(TYPE_PARAMS_OF_Bag())
+
+.. _generic-type-aliases:
+
+Generic type aliases
+--------------------
+
+The :keyword:`type` statement can also be used to create a generic type alias::
+
+   type ListOrSet[T] = list[T] | set[T]
+
+Except for the :ref:`lazy evaluation <lazy-evaluation>` of the value,
+this is equivalent to::
+
+   annotation-def TYPE_PARAMS_OF_ListOrSet():
+       T = typing.TypeVar("T")
+
+       annotation-def VALUE_OF_ListOrSet():
+           return list[T] | set[T]
+       # In reality, the value is lazily evaluated
+       return typing.TypeAliasType("ListOrSet", VALUE_OF_ListOrSet(), type_params=(T,))
+   ListOrSet = TYPE_PARAMS_OF_ListOrSet()
+
+Here, ``annotation-def`` (not a real keyword) indicates an
+:ref:`annotation scope <annotation-scopes>`. The capitalized names
+like ``TYPE_PARAMS_OF_ListOrSet`` are not actually bound at runtime.
 
 .. rubric:: Footnotes
 
diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst
index c0734e4..e8f9775 100644
--- a/Doc/reference/datamodel.rst
+++ b/Doc/reference/datamodel.rst
@@ -499,6 +499,7 @@ Callable types
          single: __globals__ (function attribute)
          single: __annotations__ (function attribute)
          single: __kwdefaults__ (function attribute)
+         single: __type_params__ (function attribute)
          pair: global; namespace
 
       +-------------------------+-------------------------------+-----------+
@@ -561,6 +562,12 @@ Callable types
       | :attr:`__kwdefaults__`  | A dict containing defaults    | Writable  |
       |                         | for keyword-only parameters.  |           |
       +-------------------------+-------------------------------+-----------+
+      | :attr:`__type_params__` | A tuple containing the        | Writable  |
+      |                         | :ref:`type parameters         |           |
+      |                         | <type-params>` of a           |           |
+      |                         | :ref:`generic function        |           |
+      |                         | <generic-functions>`.         |           |
+      +-------------------------+-------------------------------+-----------+
 
       Most of the attributes labelled "Writable" check the type of the assigned value.
 
@@ -837,6 +844,7 @@ Custom classes
       single: __bases__ (class attribute)
       single: __doc__ (class attribute)
       single: __annotations__ (class attribute)
+      single: __type_params__ (class attribute)
 
    Special attributes:
 
@@ -863,6 +871,10 @@ Custom classes
          working with :attr:`__annotations__`, please see
          :ref:`annotations-howto`.
 
+      :attr:`__type_params__`
+         A tuple containing the :ref:`type parameters <type-params>` of
+         a :ref:`generic class <generic-classes>`.
+
 Class instances
    .. index::
       pair: object; class instance
diff --git a/Doc/reference/executionmodel.rst b/Doc/reference/executionmodel.rst
index 8917243..cea3a56 100644
--- a/Doc/reference/executionmodel.rst
+++ b/Doc/reference/executionmodel.rst
@@ -71,6 +71,8 @@ The following constructs bind names:
   + in a capture pattern in structural pattern matching
 
 * :keyword:`import` statements.
+* :keyword:`type` statements.
+* :ref:`type parameter lists <type-params>`.
 
 The :keyword:`!import` statement of the form ``from ... import *`` binds all
 names defined in the imported module, except those beginning with an underscore.
@@ -149,7 +151,8 @@ a global statement, the free variable is treated as a global.
 The :keyword:`nonlocal` statement causes corresponding names to refer
 to previously bound variables in the nearest enclosing function scope.
 :exc:`SyntaxError` is raised at compile time if the given name does not
-exist in any enclosing function scope.
+exist in any enclosing function scope. :ref:`Type parameters <type-params>`
+cannot be rebound with the :keyword:`!nonlocal` statement.
 
 .. index:: pair: module; __main__
 
@@ -163,14 +166,119 @@ These references follow the normal rules for name resolution with an exception
 that unbound local variables are looked up in the global namespace.
 The namespace of the class definition becomes the attribute dictionary of
 the class. The scope of names defined in a class block is limited to the
-class block; it does not extend to the code blocks of methods -- this includes
-comprehensions and generator expressions since they are implemented using a
-function scope.  This means that the following will fail::
+class block; it does not extend to the code blocks of methods. This includes
+comprehensions and generator expressions, but it does not include
+:ref:`annotation scopes <annotation-scopes>`,
+which have access to their enclosing class scopes.
+This means that the following will fail::
 
    class A:
        a = 42
        b = list(a + i for i in range(10))
 
+However, the following will succeed::
+
+   class A:
+       type Alias = Nested
+       class Nested: pass
+
+   print(A.Alias.__value__)  # <type 'A.Nested'>
+
+.. _annotation-scopes:
+
+Annotation scopes
+-----------------
+
+:ref:`Type parameter lists <type-params>` and :keyword:`type` statements
+introduce *annotation scopes*, which behave mostly like function scopes,
+but with some exceptions discussed below. :term:`Annotations <annotation>`
+currently do not use annotation scopes, but they are expected to use
+annotation scopes in Python 3.13 when :pep:`649` is implemented.
+
+Annotation scopes are used in the following contexts:
+
+* Type parameter lists for :ref:`generic type aliases <generic-type-aliases>`.
+* Type parameter lists for :ref:`generic functions <generic-functions>`.
+  A generic function's annotations are
+  executed within the annotation scope, but its defaults and decorators are not.
+* Type parameter lists for :ref:`generic classes <generic-classes>`.
+  A generic class's base classes and
+  keyword arguments are executed within the annotation scope, but its decorators are not.
+* The bounds and constraints for type variables
+  (:ref:`lazily evaluated <lazy-evaluation>`).
+* The value of type aliases (:ref:`lazily evaluated <lazy-evaluation>`).
+
+Annotation scopes differ from function scopes in the following ways:
+
+* Annotation scopes have access to their enclosing class namespace.
+  If an annotation scope is immediately within a class scope, or within another
+  annotation scope that is immediately within a class scope, the code in the
+  annotation scope can use names defined in the class scope as if it were
+  executed directly within the class body. This contrasts with regular
+  functions defined within classes, which cannot access names defined in the class scope.
+* Expressions in annotation scopes cannot contain :keyword:`yield`, ``yield from``,
+  :keyword:`await`, or :token:`:= <python-grammar:assignment_expression>`
+  expressions. (These expressions are allowed in other scopes contained within the
+  annotation scope.)
+* Names defined in annotation scopes cannot be rebound with :keyword:`nonlocal`
+  statements in inner scopes. This includes only type parameters, as no other
+  syntactic elements that can appear within annotation scopes can introduce new names.
+* While annotation scopes have an internal name, that name is not reflected in the
+  :term:`__qualname__ <qualified name>` of objects defined within the scope.
+  Instead, the :attr:`!__qualname__`
+  of such objects is as if the object were defined in the enclosing scope.
+
+.. versionadded:: 3.12
+   Annotation scopes were introduced in Python 3.12 as part of :pep:`695`.
+
+.. _lazy-evaluation:
+
+Lazy evaluation
+---------------
+
+The values of type aliases created through the :keyword:`type` statement are
+*lazily evaluated*. The same applies to the bounds and constraints of type
+variables created through the :ref:`type parameter syntax <type-params>`.
+This means that they are not evaluated when the type alias or type variable is
+created. Instead, they are only evaluated when doing so is necessary to resolve
+an attribute access.
+
+Example:
+
+.. doctest::
+
+   >>> type Alias = 1/0
+   >>> Alias.__value__
+   Traceback (most recent call last):
+     ...
+   ZeroDivisionError: division by zero
+   >>> def func[T: 1/0](): pass
+   >>> T = func.__type_params__[0]
+   >>> T.__bound__
+   Traceback (most recent call last):
+     ...
+   ZeroDivisionError: division by zero
+
+Here the exception is raised only when the ``__value__`` attribute
+of the type alias or the ``__bound__`` attribute of the type variable
+is accessed.
+
+This behavior is primarily useful for references to types that have not
+yet been defined when the type alias or type variable is created. For example,
+lazy evaluation enables creation of mutually recursive type aliases::
+
+   from typing import Literal
+
+   type SimpleExpr = int | Parenthesized
+   type Parenthesized = tuple[Literal["("], Expr, Literal[")"]]
+   type Expr = SimpleExpr | tuple[SimpleExpr, Literal["+", "-"], Expr]
+
+Lazily evaluated values are evaluated in :ref:`annotation scope <annotation-scopes>`,
+which means that names that appear inside the lazily evaluated value are looked up
+as if they were used in the immediately enclosing scope.
+
+.. versionadded:: 3.12
+
 .. _restrict_exec:
 
 Builtins and restricted execution
diff --git a/Doc/reference/lexical_analysis.rst b/Doc/reference/lexical_analysis.rst
index 757f887..47062f8 100644
--- a/Doc/reference/lexical_analysis.rst
+++ b/Doc/reference/lexical_analysis.rst
@@ -361,15 +361,19 @@ Soft Keywords
 .. versionadded:: 3.10
 
 Some identifiers are only reserved under specific contexts. These are known as
-*soft keywords*.  The identifiers ``match``, ``case`` and ``_`` can
-syntactically act as keywords in contexts related to the pattern matching
-statement, but this distinction is done at the parser level, not when
-tokenizing.
+*soft keywords*.  The identifiers ``match``, ``case``, ``type`` and ``_`` can
+syntactically act as keywords in certain contexts,
+but this distinction is done at the parser level, not when tokenizing.
 
-As soft keywords, their use with pattern matching is possible while still
-preserving compatibility with existing code that uses ``match``, ``case`` and ``_`` as
+As soft keywords, their use in the grammar is possible while still
+preserving compatibility with existing code that uses these names as
 identifier names.
 
+``match``, ``case``, and ``_`` are used in the :keyword:`match` statement.
+``type`` is used in the :keyword:`type` statement.
+
+.. versionchanged:: 3.12
+   ``type`` is now a soft keyword.
 
 .. index::
    single: _, identifiers
diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst
index f7a8b44..662a4b6 100644
--- a/Doc/reference/simple_stmts.rst
+++ b/Doc/reference/simple_stmts.rst
@@ -28,6 +28,7 @@ simple statements is:
               : | `future_stmt`
               : | `global_stmt`
               : | `nonlocal_stmt`
+              : | `type_stmt`
 
 
 .. _exprstmts:
@@ -1012,3 +1013,48 @@ pre-existing bindings in the local scope.
 
    :pep:`3104` - Access to Names in Outer Scopes
       The specification for the :keyword:`nonlocal` statement.
+
+.. _type:
+
+The :keyword:`!type` statement
+==============================
+
+.. index:: pair: statement; type
+
+.. productionlist:: python-grammar
+   type_stmt: 'type' `identifier` [`type_params`] "=" `expression`
+
+The :keyword:`!type` statement declares a type alias, which is an instance
+of :class:`typing.TypeAliasType`.
+
+For example, the following statement creates a type alias::
+
+   type Point = tuple[float, float]
+
+This code is roughly equivalent to::
+
+   annotation-def VALUE_OF_Point():
+       return tuple[float, float]
+   Point = typing.TypeAliasType("Point", VALUE_OF_Point())
+
+``annotation-def`` indicates an :ref:`annotation scope <annotation-scopes>`, which behaves
+mostly like a function, but with several small differences.
+
+The value of the
+type alias is evaluated in the annotation scope. It is not evaluated when the
+type alias is created, but only when the value is accessed through the type alias's
+:attr:`!__value__` attribute (see :ref:`lazy-evaluation`).
+This allows the type alias to refer to names that are not yet defined.
+
+Type aliases may be made generic by adding a :ref:`type parameter list <type-params>`
+after the name. See :ref:`generic-type-aliases` for more.
+
+:keyword:`!type` is a :ref:`soft keyword <soft-keywords>`.
+
+.. versionadded:: 3.12
+
+.. seealso::
+
+   :pep:`695` - Type Parameter Syntax
+      Introduced the :keyword:`!type` statement and syntax for
+      generic classes and functions.