diff options
| -rw-r--r-- | Doc/glossary.rst | 6 | ||||
| -rw-r--r-- | Doc/howto/annotations.rst | 226 | ||||
| -rw-r--r-- | Doc/howto/index.rst | 1 | ||||
| -rw-r--r-- | Doc/library/inspect.rst | 7 | ||||
| -rw-r--r-- | Doc/reference/datamodel.rst | 70 | ||||
| -rw-r--r-- | Doc/whatsnew/3.10.rst | 8 | ||||
| -rw-r--r-- | Misc/NEWS.d/next/Documentation/2021-04-30-04-27-02.bpo-43987.1DftVa.rst | 1 |
7 files changed, 298 insertions, 21 deletions
diff --git a/Doc/glossary.rst b/Doc/glossary.rst index 0661c82..29c68ed 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -57,6 +57,8 @@ Glossary See :term:`variable annotation`, :term:`function annotation`, :pep:`484` and :pep:`526`, which describe this functionality. + Also see :ref:`annotations-howto` + for best practices on working with annotations. argument A value passed to a :term:`function` (or :term:`method`) when calling the @@ -455,6 +457,8 @@ Glossary See :term:`variable annotation` and :pep:`484`, which describe this functionality. + Also see :ref:`annotations-howto` + for best practices on working with annotations. __future__ A pseudo-module which programmers can use to enable new language features @@ -1211,6 +1215,8 @@ Glossary See :term:`function annotation`, :pep:`484` and :pep:`526`, which describe this functionality. + Also see :ref:`annotations-howto` + for best practices on working with annotations. virtual environment A cooperatively isolated runtime environment that allows Python users diff --git a/Doc/howto/annotations.rst b/Doc/howto/annotations.rst new file mode 100644 index 0000000..3e61103 --- /dev/null +++ b/Doc/howto/annotations.rst @@ -0,0 +1,226 @@ +.. _annotations-howto: + +************************** +Annotations Best Practices +************************** + +:author: Larry Hastings + +.. topic:: Abstract + + This document is designed to encapsulate the best practices + for working with annotations dicts. If you write Python code + that examines ``__annotations__`` on Python objects, we + encourage you to follow the guidelines described below. + + The document is organized into four sections: + best practices for accessing the annotations of an object + in Python versions 3.10 and newer, + best practices for accessing the annotations of an object + in Python versions 3.9 and older, + other best practices + for ``__annotations__`` that apply to any Python version, + and + quirks of ``__annotations__``. + + Note that this document is specifically about working with + ``__annotations__``, not uses *for* annotations. + If you're looking for information on how to use "type hints" + in your code, please see the :mod:`typing` module. + + +Accessing The Annotations Dict Of An Object In Python 3.10 And Newer +==================================================================== + + Python 3.10 adds a new function to the standard library: + :func:`inspect.get_annotations`. In Python versions 3.10 + and newer, calling this function is the best practice for + accessing the annotations dict of any object that supports + annotations. This function can also "un-stringize" + stringized annotations for you. + + If for some reason :func:`inspect.get_annotations` isn't + viable for your use case, you may access the + ``__annotations__`` data member manually. Best practice + for this changed in Python 3.10 as well: as of Python 3.10, + ``o.__annotations__`` is guaranteed to *always* work + on Python functions, classes, and modules. If you're + certain the object you're examining is one of these three + *specific* objects, you may simply use ``o.__annotations__`` + to get at the object's annotations dict. + + However, other types of callables--for example, + callables created by :func:`functools.partial`--may + not have an ``__annotations__`` attribute defined. When + accessing the ``__annotations__`` of a possibly unknown + object, best practice in Python versions 3.10 and + newer is to call :func:`getattr` with three arguments, + for example ``getattr(o, '__annotations__', None)``. + + +Accessing The Annotations Dict Of An Object In Python 3.9 And Older +=================================================================== + + In Python 3.9 and older, accessing the annotations dict + of an object is much more complicated than in newer versions. + The problem is a design flaw in these older versions of Python, + specifically to do with class annotations. + + Best practice for accessing the annotations dict of other + objects--functions, other callables, and modules--is the same + as best practice for 3.10, assuming you aren't calling + :func:`inspect.get_annotations`: you should use three-argument + :func:`getattr` to access the object's ``__annotations__`` + attribute. + + Unfortunately, this isn't best practice for classes. The problem + is that, since ``__annotations__`` is optional on classes, and + because classes can inherit attributes from their base classes, + accessing the ``__annotations__`` attribute of a class may + inadvertently return the annotations dict of a *base class.* + As an example:: + + class Base: + a: int = 3 + b: str = 'abc' + + class Derived(Base): + pass + + print(Derived.__annotations__) + + This will print the annotations dict from ``Base``, not + ``Derived``. + + Your code will have to have a separate code path if the object + you're examining is a class (``isinstance(o, type)``). + In that case, best practice relies on an implementation detail + of Python 3.9 and before: if a class has annotations defined, + they are stored in the class's ``__dict__`` dictionary. Since + the class may or may not have annotations defined, best practice + is to call the ``get`` method on the class dict. + + To put it all together, here is some sample code that safely + accesses the ``__annotations__`` attribute on an arbitrary + object in Python 3.9 and before:: + + if isinstance(o, type): + ann = o.__dict__.get('__annotations__', None) + else: + ann = getattr(o, '__annotations__', None) + + After running this code, ``ann`` should be either a + dictionary or ``None``. You're encouraged to double-check + the type of ``ann`` using :func:`isinstance` before further + examination. + + Note that some exotic or malformed type objects may not have + a ``__dict__`` attribute, so for extra safety you may also wish + to use :func:`getattr` to access ``__dict__``. + + +Manually Un-Stringizing Stringized Annotations +============================================== + + In situations where some annotations may be "stringized", + and you wish to evaluate those strings to produce the + Python values they represent, it really is best to + call :func:`inspect.get_annotations` to do this work + for you. + + If you're using Python 3.9 or older, or if for some reason + you can't use :func:`inspect.get_annotations`, you'll need + to duplicate its logic. You're encouraged to examine the + implementation of :func:`inspect.get_annotations` in the + current Python version and follow a similar approach. + + In a nutshell, if you wish to evaluate a stringized annotation + on an arbitrary object ``o``: + + * If ``o`` is a module, use ``o.__dict__`` as the + ``globals`` when calling :func:`eval`. + * If ``o`` is a class, use ``sys.modules[o.__module__].__dict__`` + as the ``globals``, and ``dict(vars(o))`` as the ``locals``, + when calling :func:`eval`. + * If ``o`` is a wrapped callable using :func:`functools.update_wrapper`, + :func:`functools.wraps`, or :func:`functools.partial`, iteratively + unwrap it by accessing either ``o.__wrapped__`` or ``o.func`` as + appropriate, until you have found the root unwrapped function. + * If ``o`` is a callable (but not a class), use + ``o.__globals__`` as the globals when calling :func:`eval`. + + However, not all string values used as annotations can + be successfully turned into Python values by :func:`eval`. + String values could theoretically contain any valid string, + and in practice there are valid use cases for type hints that + require annotating with string values that specifically + *can't* be evaluated. For example: + + * :pep:`604` union types using `|`, before support for this + was added to Python 3.10. + * Definitions that aren't needed at runtime, only imported + when :const:`typing.TYPE_CHECKING` is true. + + If :func:`eval` attempts to evaluate such values, it will + fail and raise an exception. So, when designing a library + API that works with annotations, it's recommended to only + attempt to evaluate string values when explicitly requested + to by the caller. + + +Best Practices For ``__annotations__`` In Any Python Version +============================================================ + + * You should avoid assigning to the ``__annotations__`` member + of objects directly. Let Python manage setting ``__annotations__``. + + * If you do assign directly to the ``__annotations__`` member + of an object, you should always set it to a ``dict`` object. + + * If you directly access the ``__annotations__`` member + of an object, you should ensure that it's a + dictionary before attempting to examine its contents. + + * You should avoid modifying ``__annotations__`` dicts. + + * You should avoid deleting the ``__annotations__`` attribute + of an object. + + +``__annotations__`` Quirks +========================== + + In all versions of Python 3, function + objects lazy-create an annotations dict if no annotations + are defined on that object. You can delete the ``__annotations__`` + attribute using ``del fn.__annotations__``, but if you then + access ``fn.__annotations__`` the object will create a new empty dict + that it will store and return as its annotations. Deleting the + annotations on a function before it has lazily created its annotations + dict will throw an ``AttributeError``; using ``del fn.__annotations__`` + twice in a row is guaranteed to always throw an ``AttributeError``. + + Everything in the above paragraph also applies to class and module + objects in Python 3.10 and newer. + + In all versions of Python 3, you can set ``__annotations__`` + on a function object to ``None``. However, subsequently + accessing the annotations on that object using ``fn.__annotations__`` + will lazy-create an empty dictionary as per the first paragraph of + this section. This is *not* true of modules and classes, in any Python + version; those objects permit setting ``__annotations__`` to any + Python value, and will retain whatever value is set. + + If Python stringizes your annotations for you + (using ``from __future__ import annotations``), and you + specify a string as an annotation, the string will + itself be quoted. In effect the annotation is quoted + *twice.* For example:: + + from __future__ import annotations + def foo(a: "str"): pass + + print(foo.__annotations__) + + This prints ``{'a': "'str'"}``. This shouldn't really be considered + a "quirk"; it's mentioned here simply because it might be surprising. diff --git a/Doc/howto/index.rst b/Doc/howto/index.rst index e0dacd2..eae8f14 100644 --- a/Doc/howto/index.rst +++ b/Doc/howto/index.rst @@ -30,4 +30,5 @@ Currently, the HOWTOs are: ipaddress.rst clinic.rst instrumentation.rst + annotations.rst diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst index 56c2f767..b9e8be1 100644 --- a/Doc/library/inspect.rst +++ b/Doc/library/inspect.rst @@ -1149,6 +1149,9 @@ Classes and functions with the result of calling :func:`eval()` on those values: * If eval_str is true, :func:`eval()` is called on values of type ``str``. + (Note that ``get_annotations`` doesn't catch exceptions; if :func:`eval()` + raises an exception, it will unwind the stack past the ``get_annotations`` + call.) * If eval_str is false (the default), values of type ``str`` are unchanged. ``globals`` and ``locals`` are passed in to :func:`eval()`; see the documentation @@ -1164,6 +1167,10 @@ Classes and functions although if ``obj`` is a wrapped function (using ``functools.update_wrapper()``) it is first unwrapped. + Calling ``get_annotations`` is best practice for accessing the + annotations dict of any object. See :ref:`annotations-howto` for + more information on annotations best practices. + .. versionadded:: 3.10 diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index 3a812eb..eefdc3d 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -553,7 +553,10 @@ Callable types | | the dict are the parameter | | | | names, and ``'return'`` for | | | | the return annotation, if | | - | | provided. | | + | | provided. For more | | + | | information on working with | | + | | this attribute, see | | + | | :ref:`annotations-howto`. | | +-------------------------+-------------------------------+-----------+ | :attr:`__kwdefaults__` | A dict containing defaults | Writable | | | for keyword-only parameters. | | @@ -748,16 +751,29 @@ Modules single: __annotations__ (module attribute) pair: module; namespace - Predefined (writable) attributes: :attr:`__name__` is the module's name; - :attr:`__doc__` is the module's documentation string, or ``None`` if - unavailable; :attr:`__annotations__` (optional) is a dictionary containing - :term:`variable annotations <variable annotation>` collected during module - body execution; :attr:`__file__` is the pathname of the file from which the - module was loaded, if it was loaded from a file. The :attr:`__file__` - attribute may be missing for certain types of modules, such as C modules - that are statically linked into the interpreter; for extension modules - loaded dynamically from a shared library, it is the pathname of the shared - library file. + Predefined (writable) attributes: + + :attr:`__name__` + The module's name. + + :attr:`__doc__` + The module's documentation string, or ``None`` if + unavailable. + + :attr:`__file__` + The pathname of the file from which the + module was loaded, if it was loaded from a file. + The :attr:`__file__` + attribute may be missing for certain types of modules, such as C modules + that are statically linked into the interpreter. For extension modules + loaded dynamically from a shared library, it's the pathname of the shared + library file. + + :attr:`__annotations__` + A dictionary containing + :term:`variable annotations <variable annotation>` collected during + module body execution. For best practices on working + with :attr:`__annotations__`, please see :ref:`annotations-howto`. .. index:: single: __dict__ (module attribute) @@ -821,14 +837,30 @@ Custom classes single: __doc__ (class attribute) single: __annotations__ (class attribute) - Special attributes: :attr:`~definition.__name__` is the class name; :attr:`__module__` is - the module name in which the class was defined; :attr:`~object.__dict__` is the - dictionary containing the class's namespace; :attr:`~class.__bases__` is a - tuple containing the base classes, in the order of their occurrence in the - base class list; :attr:`__doc__` is the class's documentation string, - or ``None`` if undefined; :attr:`__annotations__` (optional) is a dictionary - containing :term:`variable annotations <variable annotation>` collected during - class body execution. + Special attributes: + + :attr:`~definition.__name__` + The class name. + + :attr:`__module__` + The name of the module in which the class was defined. + + :attr:`~object.__dict__` + The dictionary containing the class's namespace. + + :attr:`~class.__bases__` + A tuple containing the base classes, in the order of + their occurrence in the base class list. + + :attr:`__doc__` + The class's documentation string, or ``None`` if undefined. + + :attr:`__annotations__` + A dictionary containing + :term:`variable annotations <variable annotation>` + collected during class body execution. For best practices on + working with :attr:`__annotations__`, please see + :ref:`annotations-howto`. Class instances .. index:: diff --git a/Doc/whatsnew/3.10.rst b/Doc/whatsnew/3.10.rst index a59e2e5..679522b 100644 --- a/Doc/whatsnew/3.10.rst +++ b/Doc/whatsnew/3.10.rst @@ -807,7 +807,9 @@ Other Language Changes * Class and module objects now lazy-create empty annotations dicts on demand. The annotations dicts are stored in the object’s ``__dict__`` for - backwards compatibility. + backwards compatibility. This improves the best practices for working + with ``__annotations__``; for more information, please see + :ref:`annotations-howto`. (Contributed by Larry Hastings in :issue:`43901`.) New Modules @@ -996,7 +998,9 @@ defined on an object. It works around the quirks of accessing the annotations on various types of objects, and makes very few assumptions about the object it examines. :func:`inspect.get_annotations` can also correctly un-stringize stringized annotations. :func:`inspect.get_annotations` is now considered -best practice for accessing the annotations dict defined on any Python object. +best practice for accessing the annotations dict defined on any Python object; +for more information on best practices for working with annotations, please see +:ref:`annotations-howto`. Relatedly, :func:`inspect.signature`, :func:`inspect.Signature.from_callable`, and ``inspect.Signature.from_function`` now call :func:`inspect.get_annotations` to retrieve annotations. This means diff --git a/Misc/NEWS.d/next/Documentation/2021-04-30-04-27-02.bpo-43987.1DftVa.rst b/Misc/NEWS.d/next/Documentation/2021-04-30-04-27-02.bpo-43987.1DftVa.rst new file mode 100644 index 0000000..158259e --- /dev/null +++ b/Misc/NEWS.d/next/Documentation/2021-04-30-04-27-02.bpo-43987.1DftVa.rst @@ -0,0 +1 @@ +Add "Annotations Best Practices" document as a new HOWTO. |