diff options
| -rw-r--r-- | Doc/library/stdtypes.rst | 5 | ||||
| -rw-r--r-- | Doc/library/typing.rst | 204 | ||||
| -rw-r--r-- | Doc/whatsnew/3.10.rst | 22 | ||||
| -rw-r--r-- | Lib/typing.py | 15 |
4 files changed, 238 insertions, 8 deletions
diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 2869378..2331849 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -4959,6 +4959,11 @@ All parameterized generics implement special read-only attributes. (~T,) + .. note:: + A ``GenericAlias`` object with :class:`typing.ParamSpec` parameters may not + have correct ``__parameters__`` after substitution because + :class:`typing.ParamSpec` is intended primarily for static type checking. + .. seealso:: * :pep:`585` -- "Type Hinting Generics In Standard Collections" diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index b07bb89..d74f8bc 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -18,7 +18,8 @@ -------------- This module provides runtime support for type hints as specified by -:pep:`484`, :pep:`526`, :pep:`544`, :pep:`586`, :pep:`589`, :pep:`591`, and :pep:`613`. +:pep:`484`, :pep:`526`, :pep:`544`, :pep:`586`, :pep:`589`, :pep:`591`, +:pep:`612` and :pep:`613`. The most fundamental support consists of the types :data:`Any`, :data:`Union`, :data:`Tuple`, :data:`Callable`, :class:`TypeVar`, and :class:`Generic`. For full specification please see :pep:`484`. For @@ -171,6 +172,22 @@ It is possible to declare the return type of a callable without specifying the call signature by substituting a literal ellipsis for the list of arguments in the type hint: ``Callable[..., ReturnType]``. +Callables which take other callables as arguments may indicate that their +parameter types are dependent on each other using :class:`ParamSpec`. +Additionally, if that callable adds or removes arguments from other +callables, the :data:`Concatenate` operator may be used. They +take the form ``Callable[ParamSpecVariable, ReturnType]`` and +``Callable[Concatenate[Arg1Type, Arg2Type, ..., ParamSpecVariable], ReturnType]`` +respectively. + +.. versionchanged:: 3.10 + ``Callable`` now supports :class:`ParamSpec` and :data:`Concatenate`. + See :pep:`612` for more information. + +.. seealso:: + The documentation for :class:`ParamSpec` and :class:`Concatenate` provide + examples of usage in ``Callable``. + .. _generics: Generics @@ -316,6 +333,43 @@ User defined generic type aliases are also supported. Examples:: .. versionchanged:: 3.7 :class:`Generic` no longer has a custom metaclass. +User-defined generics for parameter expressions are also supported via parameter +specification variables in the form ``Generic[P]``. The behavior is consistent +with type variables' described above as parameter specification variables are +treated by the typing module as a specialized type variable. The one exception +to this is that a list of types can be used to substitute a :class:`ParamSpec`:: + + >>> from typing import Generic, ParamSpec, TypeVar + + >>> T = TypeVar('T') + >>> P = ParamSpec('P') + + >>> class Z(Generic[T, P]): ... + ... + >>> Z[int, [dict, float]] + __main__.Z[int, (<class 'dict'>, <class 'float'>)] + + +Furthermore, a generic with only one parameter specification variable will accept +parameter lists in the forms ``X[[Type1, Type2, ...]]`` and also +``X[Type1, Type2, ...]`` for aesthetic reasons. Internally, the latter is converted +to the former and are thus equivalent:: + + >>> class X(Generic[P]): ... + ... + >>> X[int, str] + __main__.X[(<class 'int'>, <class 'str'>)] + >>> X[[int, str]] + __main__.X[(<class 'int'>, <class 'str'>)] + +Do note that generics with :class:`ParamSpec` may not have correct +``__parameters__`` after substitution in some cases because they +are intended primarily for static type checking. + +.. versionchanged:: 3.10 + :class:`Generic` can now be parameterized over parameter expressions. + See :class:`ParamSpec` and :pep:`612` for more details. + A user-defined generic class can have ABCs as base classes without a metaclass conflict. Generic metaclasses are not supported. The outcome of parameterizing generics is cached, and most types in the typing module are hashable and @@ -602,10 +656,80 @@ These can be used as types in annotations using ``[]``, each having a unique syn ``Callable[..., Any]``, and in turn to :class:`collections.abc.Callable`. + Callables which take other callables as arguments may indicate that their + parameter types are dependent on each other using :class:`ParamSpec`. + Additionally, if that callable adds or removes arguments from other + callables, the :data:`Concatenate` operator may be used. They + take the form ``Callable[ParamSpecVariable, ReturnType]`` and + ``Callable[Concatenate[Arg1Type, Arg2Type, ..., ParamSpecVariable], ReturnType]`` + respectively. + .. deprecated:: 3.9 :class:`collections.abc.Callable` now supports ``[]``. See :pep:`585` and :ref:`types-genericalias`. + .. versionchanged:: 3.10 + ``Callable`` now supports :class:`ParamSpec` and :data:`Concatenate`. + See :pep:`612` for more information. + + .. seealso:: + The documentation for :class:`ParamSpec` and :class:`Concatenate` provide + examples of usage with ``Callable``. + +.. data:: Concatenate + + Used with :data:`Callable` and :class:`ParamSpec` to type annotate a higher + order callable which adds, removes, or transforms parameters of another + callable. Usage is in the form + ``Concatenate[Arg1Type, Arg2Type, ..., ParamSpecVariable]``. ``Concatenate`` + is currently only valid when used as the first argument to a :data:`Callable`. + The last parameter to ``Concatenate`` must be a :class:`ParamSpec`. + + For example, to annotate a decorator ``with_lock`` which provides a + :class:`threading.Lock` to the decorated function, ``Concatenate`` can be + used to indicate that ``with_lock`` expects a callable which takes in a + ``Lock`` as the first argument, and returns a callable with a different type + signature. In this case, the :class:`ParamSpec` indicates that the returned + callable's parameter types are dependent on the parameter types of the + callable being passed in:: + + from collections.abc import Callable + from threading import Lock + from typing import Any, Concatenate, ParamSpec + + P = ParamSpec('P') + R = ParamSpec('R') + + # Use this lock to ensure that only one thread is executing a function + # at any time. + my_lock = Lock() + + def with_lock(f: Callable[Concatenate[Lock, P], R]) -> Callable[P, R]: + '''A type-safe decorator which provides a lock.''' + global my_lock + def inner(*args: P.args, **kwargs: P.kwargs) -> T: + # Provide the lock as the first argument. + return f(my_lock, *args, **kwargs) + return inner + + @with_lock + def sum_threadsafe(lock: Lock, numbers: list[float]) -> float: + '''Add a list of numbers together in a thread-safe manner.''' + with lock: + return sum(numbers) + + # We don't need to pass in the lock ourselves thanks to the decorator. + sum_threadsafe([1.1, 2.2, 3.3]) + +.. versionadded:: 3.10 + +.. seealso:: + + * :pep:`612` -- Parameter Specification Variables (the PEP which introduced + ``ParamSpec`` and ``Concatenate``). + * :class:`ParamSpec` and :class:`Callable`. + + .. class:: Type(Generic[CT_co]) A variable annotated with ``C`` may accept a value of type ``C``. In @@ -876,6 +1000,84 @@ These are not used in annotations. They are building blocks for creating generic for the type variable must be a subclass of the boundary type, see :pep:`484`. +.. class:: ParamSpec(name, *, bound=None, covariant=False, contravariant=False) + + Parameter specification variable. A specialized version of + :class:`type variables <TypeVar>`. + + Usage:: + + P = ParamSpec('P') + + Parameter specification variables exist primarily for the benefit of static + type checkers. They are used to forward the parameter types of one + callable to another callable -- a pattern commonly found in higher order + functions and decorators. They are only valid when used in ``Concatenate``, + or as the first argument to ``Callable``, or as parameters for user-defined + Generics. See :class:`Generic` for more information on generic types. + + For example, to add basic logging to a function, one can create a decorator + ``add_logging`` to log function calls. The parameter specification variable + tells the type checker that the callable passed into the decorator and the + new callable returned by it have inter-dependent type parameters:: + + from collections.abc import Callable + from typing import TypeVar, ParamSpec + import logging + + T = TypeVar('T') + P = ParamSpec('P') + + def add_logging(f: Callable[P, T]) -> Callable[P, T]: + '''A type-safe decorator to add logging to a function.''' + def inner(*args: P.args, **kwargs: P.kwargs) -> T: + logging.info(f'{f.__name__} was called') + return f(*args, **kwargs) + return inner + + @add_logging + def add_two(x: float, y: float) -> float: + '''Add two numbers together.''' + return x + y + + Without ``ParamSpec``, the simplest way to annotate this previously was to + use a :class:`TypeVar` with bound ``Callable[..., Any]``. However this + causes two problems: + + 1. The type checker can't type check the ``inner`` function because + ``*args`` and ``**kwargs`` have to be typed :data:`Any`. + 2. :func:`~cast` may be required in the body of the ``add_logging`` + decorator when returning the ``inner`` function, or the static type + checker must be told to ignore the ``return inner``. + + .. attribute:: args + .. attribute:: kwargs + + Since ``ParamSpec`` captures both positional and keyword parameters, + ``P.args`` and ``P.kwargs`` can be used to split a ``ParamSpec`` into its + components. ``P.args`` represents the tuple of positional parameters in a + given call and should only be used to annotate ``*args``. ``P.kwargs`` + represents the mapping of keyword parameters to their values in a given call, + and should be only be used to annotate ``**kwargs`` or ``**kwds``. Both + attributes require the annotated parameter to be in scope. + + Parameter specification variables created with ``covariant=True`` or + ``contravariant=True`` can be used to declare covariant or contravariant + generic types. The ``bound`` argument is also accepted, similar to + :class:`TypeVar`. However the actual semantics of these keywords are yet to + be decided. + + .. versionadded:: 3.10 + + .. note:: + Only parameter specification variables defined in global scope can + be pickled. + + .. seealso:: + * :pep:`612` -- Parameter Specification Variables (the PEP which introduced + ``ParamSpec`` and ``Concatenate``). + * :class:`Callable` and :class:`Concatenate`. + .. data:: AnyStr ``AnyStr`` is a type variable defined as diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index aa547ff..be529d7 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -144,6 +144,28 @@ See :pep:`604` for more details. (Contributed by Maggie Moss and Philippe Prados in :issue:`41428`.) +PEP 612: Parameter Specification Variables +------------------------------------------ + +Two new options to improve the information provided to static type checkers for +:pep:`484`\ 's ``Callable`` have been added to the :mod:`typing` module. + +The first is the parameter specification variable. They are used to forward the +parameter types of one callable to another callable -- a pattern commonly +found in higher order functions and decorators. Examples of usage can be found +in :class:`typing.ParamSpec`. Previously, there was no easy way to type annotate +dependency of parameter types in such a precise manner. + +The second option is the new ``Concatenate`` operator. It's used in conjunction +with parameter specification variables to type annotate a higher order callable +which adds or removes parameters of another callable. Examples of usage can +be found in :class:`typing.Concatenate`. + +See :class:`typing.Callable`, :class:`typing.ParamSpec`, +:class:`typing.Concatenate` and :pep:`612` for more details. + +(Contributed by Ken Jin in :issue:`41559`.) + Other Language Changes ====================== diff --git a/Lib/typing.py b/Lib/typing.py index b140b0e..88d0d62 100644 --- a/Lib/typing.py +++ b/Lib/typing.py @@ -544,8 +544,9 @@ def TypeAlias(self, parameters): @_SpecialForm def Concatenate(self, parameters): - """Used in conjunction with ParamSpec and Callable to represent a higher - order function which adds, removes or transforms parameters of a Callable. + """Used in conjunction with ``ParamSpec`` and ``Callable`` to represent a + higher order function which adds, removes or transforms parameters of a + callable. For example:: @@ -735,11 +736,11 @@ class ParamSpec(_Final, _Immutable, _TypeVarLike, _root=True): Parameter specification variables exist primarily for the benefit of static type checkers. They are used to forward the parameter types of one - Callable to another Callable, a pattern commonly found in higher order - functions and decorators. They are only valid when used in Concatenate, or - as the first argument to Callable, or as parameters for user-defined Generics. - See class Generic for more information on generic types. An example for - annotating a decorator:: + callable to another callable, a pattern commonly found in higher order + functions and decorators. They are only valid when used in ``Concatenate``, + or s the first argument to ``Callable``, or as parameters for user-defined + Generics. See class Generic for more information on generic types. An + example for annotating a decorator:: T = TypeVar('T') P = ParamSpec('P') |