String annotations [PEP 563] (#4390)

* Document `from __future__ import annotations`
* Provide plumbing and tests for `from __future__ import annotations`
* Implement unparsing the AST back to string form

This is required for PEP 563 and as such only implements a part of the
unparsing process that covers expressions.

5 files changed, 94 insertions, 19 deletions

diff --git a/Doc/glossary.rst b/Doc/glossary.rst
index f0fcb62..dcfe086 100644
--- a/Doc/glossary.rst
+++ b/Doc/glossary.rst
@@ -372,9 +372,11 @@ Glossary
       may be accessed via the :attr:`__annotations__` special attribute of a
       function object.
 
-      Python itself does not assign any particular meaning to function
-      annotations. They are intended to be interpreted by third-party libraries
-      or tools.  See :pep:`3107`, which describes some of their potential uses.
+      See also the :term:`variable annotation` glossary entry.
+
+      Annotations are meant to provide a standard way for programmers to
+      document types of functions they design.  See :pep:`484`, which
+      describes this functionality.
 
    __future__
       A pseudo-module which programmers can use to enable new language features
@@ -1021,10 +1023,11 @@ Glossary
       attribute of a class or module object and can be accessed using
       :func:`typing.get_type_hints`.
 
-      Python itself does not assign any particular meaning to variable
-      annotations. They are intended to be interpreted by third-party libraries
-      or type checking tools. See :pep:`526`, :pep:`484` which describe
-      some of their potential uses.
+      See also the :term:`function annotation` glossary entry.
+
+      Annotations are meant to provide a standard way for programmers to
+      document types of functions they design.  See :pep:`484` and :pep:`526`
+      which describe this functionality.
 
    virtual environment
       A cooperatively isolated runtime environment that allows Python users
diff --git a/Doc/library/__future__.rst b/Doc/library/__future__.rst
index 73d8b6b..e3d749e 100644
--- a/Doc/library/__future__.rst
+++ b/Doc/library/__future__.rst
@@ -90,6 +90,11 @@ language using this mechanism:
 | generator_stop   | 3.5.0b1     | 3.7          | :pep:`479`:                                 |
 |                  |             |              | *StopIteration handling inside generators*  |
 +------------------+-------------+--------------+---------------------------------------------+
+| annotations      | 3.7.0b1     | 4.0          | :pep:`563`:                                 |
+|                  |             |              | *Postponed evaluation of annotations*       |
++------------------+-------------+--------------+---------------------------------------------+
+
+.. XXX Adding a new entry?  Remember to update simple_stmts.rst, too.
 
 
 .. seealso::
diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst
index dca9362..d7792f1 100644
--- a/Doc/reference/compound_stmts.rst
+++ b/Doc/reference/compound_stmts.rst
@@ -559,12 +559,14 @@ Parameters may have annotations of the form "``: expression``" following the
 parameter name.  Any parameter may have an annotation even those of the form
 ``*identifier`` or ``**identifier``.  Functions may have "return" annotation of
 the form "``-> expression``" after the parameter list.  These annotations can be
-any valid Python expression and are evaluated when the function definition is
-executed.  Annotations may be evaluated in a different order than they appear in
-the source code.  The presence of annotations does not change the semantics of a
-function.  The annotation values are available as values of a dictionary keyed
-by the parameters' names in the :attr:`__annotations__` attribute of the
-function object.
+any valid Python expression.  The presence of annotations does not change the
+semantics of a function.  The annotation values are available as values of
+a dictionary keyed by the parameters' names in the :attr:`__annotations__`
+attribute of the function object.  If the ``annotations`` import from
+:mod:`__future__` is used, annotations are preserved as strings at runtime which
+enables postponed evaluation.  Otherwise, they are evaluated when the function
+definition is executed.  In this case annotations may be evaluated in
+a different order than they appear in the source code.
 
 .. index:: pair: lambda; expression
 
@@ -587,6 +589,17 @@ access the local variables of the function containing the def.  See section
    :pep:`3107` - Function Annotations
       The original specification for function annotations.
 
+   :pep:`484` - Type Hints
+      Definition of a standard meaning for annotations: type hints.
+
+   :pep:`526` - Syntax for Variable Annotations
+      Ability to type hint variable declarations, including class
+      variables and instance variables
+
+   :pep:`563` - Postponed Evaluation of Annotations
+      Support for forward references within annotations by preserving
+      annotations in a string form at runtime instead of eager evaluation.
+
 
 .. _class:
 
diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst
index 8d17383..ef9a5f0 100644
--- a/Doc/reference/simple_stmts.rst
+++ b/Doc/reference/simple_stmts.rst
@@ -853,12 +853,15 @@ can appear before a future statement are:
 * blank lines, and
 * other future statements.
 
-.. XXX change this if future is cleaned out
-
-The features recognized by Python 3.0 are ``absolute_import``, ``division``,
-``generators``, ``unicode_literals``, ``print_function``, ``nested_scopes`` and
-``with_statement``.  They are all redundant because they are always enabled, and
-only kept for backwards compatibility.
+The only feature in Python 3.7 that requires using the future statement is
+``annotations``.
+
+All historical features enabled by the future statement are still recognized
+by Python 3.  The list includes ``absolute_import``, ``division``,
+``generators``, ``generator_stop``, ``unicode_literals``,
+``print_function``, ``nested_scopes`` and ``with_statement``.  They are
+all redundant because they are always enabled, and only kept for
+backwards compatibility.
 
 A future statement is recognized and treated specially at compile time: Changes
 to the semantics of core constructs are often implemented by generating
diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst
index 0e1714e..a350919 100644
--- a/Doc/whatsnew/3.7.rst
+++ b/Doc/whatsnew/3.7.rst
@@ -179,6 +179,57 @@ a normal ``__getattr__`` method, except that it will be defined on module
        PEP written and implemented by Ivan Levkivskyi
 
 
+PEP 563: Postponed evaluation of annotations
+--------------------------------------------
+
+The advent of type hints in Python uncovered two glaring usability issues
+with the functionality of annotations added in :pep:`3107` and refined
+further in :pep:`526`:
+
+* annotations could only use names which were already available in the
+  current scope, in other words they didn't support forward references
+  of any kind; and
+
+* annotating source code had adverse effects on startup time of Python
+  programs.
+
+Both of these issues are fixed by postponing the evaluation of
+annotations.  Instead of compiling code which executes expressions in
+annotations at their definition time, the compiler stores the annotation
+in a string form equivalent to the AST of the expression in question.
+If needed, annotations can be resolved at runtime using
+``typing.get_type_hints()``.  In the common case where this is not
+required, the annotations are cheaper to store (since short strings
+are interned by the interpreter) and make startup time faster.
+
+Usability-wise, annotations now support forward references, making the
+following syntax valid::
+
+    class C:
+        @classmethod
+        def from_string(cls, source: str) -> C:
+            ...
+
+        def validate_b(self, obj: B) -> bool:
+            ...
+
+    class B:
+        ...
+
+Since this change breaks compatibility, the new behavior can be enabled
+on a per-module basis in Python 3.7 using a ``__future__`` import, like
+this::
+
+    from __future__ import annotations
+
+It will become the default in Python 4.0.
+
+.. seealso::
+
+    :pep:`563` -- Postponed evaluation of annotations
+       PEP written and implemented by Łukasz Langa.
+
+
 PEP 564: Add new time functions with nanosecond resolution
 ----------------------------------------------------------