From 31b3a70edb1216bdc8fab3b2eafd8ddb00487f41 Mon Sep 17 00:00:00 2001 From: Alex Waygood Date: Thu, 18 Nov 2021 15:54:25 +0000 Subject: [doc] bpo-45680: Disambiguate ``__getitem__`` and ``__class_getitem__`` in the data model (GH-29389) The documentation explaining Python's data model does not adequately explain the differences between ``__getitem__`` and ``__class_getitem__``, nor does it explain when each is called. There is an attempt at explaining ``__class_getitem__`` in the documentation for ``GenericAlias`` objects, but this does not give sufficient clarity into how the method works. Moreover, it is the wrong place for that information to be found; the explanation of ``__class_getitem__`` should be in the documentation explaining the data model. This PR has been split off from GH-29335. --- Doc/library/typing.rst | 1 + Doc/reference/datamodel.rst | 164 +++++++++++++++++++++++++++++++++++++++----- 2 files changed, 147 insertions(+), 18 deletions(-) diff --git a/Doc/library/typing.rst b/Doc/library/typing.rst index c37e97e..735d477 100644 --- a/Doc/library/typing.rst +++ b/Doc/library/typing.rst @@ -256,6 +256,7 @@ called :class:`TypeVar`. def first(l: Sequence[T]) -> T: # Generic function return l[0] +.. _user-defined-generics: User-defined generic types ========================== diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index a6eee22..1ecfa81 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -2215,22 +2215,142 @@ case the instance is itself a class. Emulating generic types ----------------------- -One can implement the generic class syntax as specified by :pep:`484` -(for example ``List[int]``) by defining a special method: +When using :term:`type annotations`, it is often useful to +*parameterize* a :term:`generic type` using Python's square-brackets notation. +For example, the annotation ``list[int]`` might be used to signify a +:class:`list` in which all the elements are of type :class:`int`. + +.. seealso:: + + :pep:`484` - Type Hints + Introducing Python's framework for type annotations + + :ref:`Generic Alias Types` + Documentation for objects representing parameterized generic classes + + :ref:`Generics`, :ref:`user-defined generics` and :class:`typing.Generic` + Documentation on how to implement generic classes that can be + parameterized at runtime and understood by static type-checkers. + +A class can *generally* only be parameterized if it defines the special +class method ``__class_getitem__()``. .. classmethod:: object.__class_getitem__(cls, key) Return an object representing the specialization of a generic class by type arguments found in *key*. -This method is looked up on the class object itself, and when defined in -the class body, this method is implicitly a class method. Note, this -mechanism is primarily reserved for use with static type hints, other usage -is discouraged. + When defined on a class, ``__class_getitem__()`` is automatically a class + method. As such, there is no need for it to be decorated with + :func:`@classmethod` when it is defined. -.. seealso:: - :pep:`560` - Core support for typing module and generic types +The purpose of *__class_getitem__* +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +The purpose of :meth:`~object.__class_getitem__` is to allow runtime +parameterization of standard-library generic classes in order to more easily +apply :term:`type hints` to these classes. + +To implement custom generic classes that can be parameterized at runtime and +understood by static type-checkers, users should either inherit from a standard +library class that already implements :meth:`~object.__class_getitem__`, or +inherit from :class:`typing.Generic`, which has its own implementation of +``__class_getitem__()``. + +Custom implementations of :meth:`~object.__class_getitem__` on classes defined +outside of the standard library may not be understood by third-party +type-checkers such as mypy. Using ``__class_getitem__()`` on any class for +purposes other than type hinting is discouraged. + + +.. _classgetitem-versus-getitem: + + +*__class_getitem__* versus *__getitem__* +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Usually, the :ref:`subscription` of an object using square +brackets will call the :meth:`~object.__getitem__` instance method defined on +the object's class. However, if the object being subscribed is itself a class, +the class method :meth:`~object.__class_getitem__` may be called instead. +``__class_getitem__()`` should return a :ref:`GenericAlias` +object if it is properly defined. + +Presented with the :term:`expression` ``obj[x]``, the Python interpreter +follows something like the following process to decide whether +:meth:`~object.__getitem__` or :meth:`~object.__class_getitem__` should be +called:: + + from inspect import isclass + + def subscribe(obj, x): + """Return the result of the expression `obj[x]`""" + + class_of_obj = type(obj) + + # If the class of obj defines __getitem__, + # call class_of_obj.__getitem__(obj, x) + if hasattr(class_of_obj, '__getitem__'): + return class_of_obj.__getitem__(obj, x) + + # Else, if obj is a class and defines __class_getitem__, + # call obj.__class_getitem__(x) + elif isclass(obj) and hasattr(obj, '__class_getitem__'): + return obj.__class_getitem__(x) + + # Else, raise an exception + else: + raise TypeError( + f"'{class_of_obj.__name__}' object is not subscriptable" + ) + +In Python, all classes are themselves instances of other classes. The class of +a class is known as that class's :term:`metaclass`, and most classes have the +:class:`type` class as their metaclass. :class:`type` does not define +:meth:`~object.__getitem__`, meaning that expressions such as ``list[int]``, +``dict[str, float]`` and ``tuple[str, bytes]`` all result in +:meth:`~object.__class_getitem__` being called:: + + >>> # list has class "type" as its metaclass, like most classes: + >>> type(list) + + >>> type(dict) == type(list) == type(tuple) == type(str) == type(bytes) + True + >>> # "list[int]" calls "list.__class_getitem__(int)" + >>> list[int] + list[int] + >>> # list.__class_getitem__ returns a GenericAlias object: + >>> type(list[int]) + + +However, if a class has a custom metaclass that defines +:meth:`~object.__getitem__`, subscribing the class may result in different +behaviour. An example of this can be found in the :mod:`enum` module:: + + >>> from enum import Enum + >>> class Menu(Enum): + ... """A breakfast menu""" + ... SPAM = 'spam' + ... BACON = 'bacon' + ... + >>> # Enum classes have a custom metaclass: + >>> type(Menu) + + >>> # EnumMeta defines __getitem__, + >>> # so __class_getitem__ is not called, + >>> # and the result is not a GenericAlias object: + >>> Menu['SPAM'] + + >>> type(Menu['SPAM']) + + + +.. seealso:: + :pep:`560` - Core Support for typing module and generic types + Introducing :meth:`~object.__class_getitem__`, and outlining when a + :ref:`subscription` results in ``__class_getitem__()`` + being called instead of :meth:`~object.__getitem__` .. _callable-types: @@ -2330,19 +2450,27 @@ through the object's keys; for sequences, it should iterate through the values. .. method:: object.__getitem__(self, key) - Called to implement evaluation of ``self[key]``. For sequence types, the - accepted keys should be integers and slice objects. Note that the special - interpretation of negative indexes (if the class wishes to emulate a sequence - type) is up to the :meth:`__getitem__` method. If *key* is of an inappropriate - type, :exc:`TypeError` may be raised; if of a value outside the set of indexes - for the sequence (after any special interpretation of negative values), - :exc:`IndexError` should be raised. For mapping types, if *key* is missing (not - in the container), :exc:`KeyError` should be raised. + Called to implement evaluation of ``self[key]``. For :term:`sequence` types, + the accepted keys should be integers and slice objects. Note that the + special interpretation of negative indexes (if the class wishes to emulate a + :term:`sequence` type) is up to the :meth:`__getitem__` method. If *key* is + of an inappropriate type, :exc:`TypeError` may be raised; if of a value + outside the set of indexes for the sequence (after any special + interpretation of negative values), :exc:`IndexError` should be raised. For + :term:`mapping` types, if *key* is missing (not in the container), + :exc:`KeyError` should be raised. + + .. note:: + + :keyword:`for` loops expect that an :exc:`IndexError` will be raised for + illegal indexes to allow proper detection of the end of the sequence. .. note:: - :keyword:`for` loops expect that an :exc:`IndexError` will be raised for illegal - indexes to allow proper detection of the end of the sequence. + When :ref:`subscripting` a *class*, the special + class method :meth:`~object.__class_getitem__` may be called instead of + ``__getitem__()``. See :ref:`classgetitem-versus-getitem` for more + details. .. method:: object.__setitem__(self, key, value) -- cgit v0.12