Issue #1602: Windows console doesn't input or print Unicode (PEP 528)

Closes #17602: Adds a readline implementation for the Windows console

3 files changed, 60 insertions, 16 deletions

diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
index 3e2fb72..db04b10 100644
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -1055,30 +1055,38 @@ are always available.  They are listed here in alphabetical order.
    (where :func:`open` is declared), :mod:`os`, :mod:`os.path`, :mod:`tempfile`,
    and :mod:`shutil`.
 
-   .. versionchanged:: 3.3
-      The *opener* parameter was added.
-      The ``'x'`` mode was added.
-      :exc:`IOError` used to be raised, it is now an alias of :exc:`OSError`.
-      :exc:`FileExistsError` is now raised if the file opened in exclusive
-      creation mode (``'x'``) already exists.
+   .. versionchanged::
+      3.3
 
-   .. versionchanged:: 3.4
-      The file is now non-inheritable.
+         * The *opener* parameter was added.
+         * The ``'x'`` mode was added.
+         * :exc:`IOError` used to be raised, it is now an alias of :exc:`OSError`.
+         * :exc:`FileExistsError` is now raised if the file opened in exclusive
+         * creation mode (``'x'``) already exists.
+
+   .. versionchanged::
+      3.4
+
+         * The file is now non-inheritable.
 
    .. deprecated-removed:: 3.4 4.0
 
       The ``'U'`` mode.
 
-   .. versionchanged:: 3.5
-      If the system call is interrupted and the signal handler does not raise an
-      exception, the function now retries the system call instead of raising an
-      :exc:`InterruptedError` exception (see :pep:`475` for the rationale).
+   .. versionchanged::
+      3.5
 
-   .. versionchanged:: 3.5
-      The ``'namereplace'`` error handler was added.
+         * If the system call is interrupted and the signal handler does not raise an
+           exception, the function now retries the system call instead of raising an
+           :exc:`InterruptedError` exception (see :pep:`475` for the rationale).
+         * The ``'namereplace'`` error handler was added.
 
-   .. versionchanged:: 3.6
-      Support added to accept objects implementing :class:`os.PathLike`.
+   .. versionchanged::
+      3.6
+
+         * Support added to accept objects implementing :class:`os.PathLike`.
+         * On Windows, opening a console buffer may return a subclass of
+           :class:`io.RawIOBase` other than :class:`io.FileIO`.
 
 .. function:: ord(c)
 
diff --git a/Doc/using/cmdline.rst b/Doc/using/cmdline.rst
index 2a83bd1..75cb8ea 100644
--- a/Doc/using/cmdline.rst
+++ b/Doc/using/cmdline.rst
@@ -559,6 +559,10 @@ conflict.
    .. versionchanged:: 3.4
       The ``encodingname`` part is now optional.
 
+   .. versionchanged:: 3.6
+      On Windows, the encoding specified by this variable is ignored for interactive
+      console buffers unless :envvar:`PYTHONLEGACYWINDOWSIOENCODING` is also specified.
+      Files and pipes redirected through the standard streams are not affected.
 
 .. envvar:: PYTHONNOUSERSITE
 
@@ -686,6 +690,19 @@ conflict.
    .. versionadded:: 3.6
       See :pep:`529` for more details.
 
+.. envvar:: PYTHONLEGACYWINDOWSIOENCODING
+
+   If set to a non-empty string, does not use the new console reader and
+   writer. This means that Unicode characters will be encoded according to
+   the active console code page, rather than using utf-8.
+
+   This variable is ignored if the standard streams are redirected (to files
+   or pipes) rather than referring to console buffers.
+
+   Availability: Windows
+
+   .. versionadded:: 3.6
+
 Debug-mode variables
 ~~~~~~~~~~~~~~~~~~~~
 
diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst
index 14d0579..fa3886c 100644
--- a/Doc/whatsnew/3.6.rst
+++ b/Doc/whatsnew/3.6.rst
@@ -78,6 +78,8 @@ Windows improvements:
 
 * PEP 529: :ref:`Change Windows filesystem encoding to UTF-8 <pep-529>`
 
+* PEP 528: :ref:`Change Windows console encoding to UTF-8 <pep-528>`
+
 * The ``py.exe`` launcher, when used interactively, no longer prefers
   Python 2 over Python 3 when the user doesn't specify a version (via
   command line arguments or a config file).  Handling of shebang lines
@@ -267,6 +269,23 @@ Also see :pep:`487` and the updated class customization documentation at
 
 (Contributed by Martin Teichmann in :issue:`27366`)
 
+.. _pep-528:
+
+PEP 528: Change Windows console encoding to UTF-8
+-------------------------------------------------
+
+The default console on Windows will now accept all Unicode characters and
+provide correctly read str objects to Python code. ``sys.stdin``,
+``sys.stdout`` and ``sys.stderr`` now default to utf-8 encoding.
+
+This change only applies when using an interactive console, and not when
+redirecting files or pipes. To revert to the previous behaviour for interactive
+console use, set :envvar:`PYTHONLEGACYWINDOWSIOENCODING`.
+
+.. seealso::
+
+   :pep:`528` -- Change Windows console encoding to UTF-8
+      PEP written and implemented by Steve Dower.
 
 PYTHONMALLOC environment variable
 =================================