diff options
| author | Elvis Pranskevichus <elvis@magic.io> | 2018-05-20 03:15:06 (GMT) |
|---|---|---|
| committer | Yury Selivanov <yury@magic.io> | 2018-05-20 03:15:06 (GMT) |
| commit | 63536bd286097e770909052052a21804a5e09b66 (patch) | |
| tree | fe233adb6cb63bac4c0bdc9811fd52cab12fc9f4 | |
| parent | 6b0d09b8f06e6967fa3d41425cecf6499a353a6d (diff) | |
| download | cpython-63536bd286097e770909052052a21804a5e09b66.zip cpython-63536bd286097e770909052052a21804a5e09b66.tar.gz cpython-63536bd286097e770909052052a21804a5e09b66.tar.bz2 | |
bpo-32996: The bulk of What's New in Python 3.7 (GH-6978)
| -rw-r--r-- | Doc/library/asyncio-task.rst | 2 | ||||
| -rw-r--r-- | Doc/library/ensurepip.rst | 3 | ||||
| -rw-r--r-- | Doc/reference/compound_stmts.rst | 2 | ||||
| -rw-r--r-- | Doc/whatsnew/3.7.rst | 2136 |
4 files changed, 1549 insertions, 594 deletions
diff --git a/Doc/library/asyncio-task.rst b/Doc/library/asyncio-task.rst index db0e04a..2488a90 100644 --- a/Doc/library/asyncio-task.rst +++ b/Doc/library/asyncio-task.rst @@ -92,7 +92,7 @@ Coroutines (and tasks) can only run when the event loop is running. used in a callback-style code, wrap its result with :func:`ensure_future`. -.. function:: asyncio.run(coro, \*, debug=False) +.. function:: run(coro, \*, debug=False) This function runs the passed coroutine, taking care of managing the asyncio event loop and finalizing asynchronous diff --git a/Doc/library/ensurepip.rst b/Doc/library/ensurepip.rst index ed22180..c797f63 100644 --- a/Doc/library/ensurepip.rst +++ b/Doc/library/ensurepip.rst @@ -78,9 +78,6 @@ options: Providing both of the script selection options will trigger an exception. -.. versionchanged:: 3.7.0 - The exit status is non-zero if the command fails. - Module API ---------- diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 153e85b..5076e5d 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -684,6 +684,8 @@ can be used to create instance variables with different implementation details. :pep:`3129` - Class Decorators +.. _async: + Coroutines ========== diff --git a/Doc/whatsnew/3.7.rst b/Doc/whatsnew/3.7.rst index 24a5b3d..09a6eac 100644 --- a/Doc/whatsnew/3.7.rst +++ b/Doc/whatsnew/3.7.rst @@ -2,6 +2,8 @@ What's New In Python 3.7 **************************** +:Editor: Elvis Pranskevichus <elvis@magic.io> + .. Rules for maintenance: * Anyone can add text to this document. Do not spend very much time @@ -39,34 +41,141 @@ module. (Contributed by P.Y. Developer in :issue:`12345`.) - This saves the maintainer the effort of going through the Mercurial log + This saves the maintainer the effort of going through the Git log when researching a change. This article explains the new features in Python 3.7, compared to 3.6. For full details, see the :ref:`changelog <changelog>`. -.. note:: - - Prerelease users should be aware that this document is currently in draft - form. It will be updated substantially as Python 3.7 moves towards release, - so it's worth checking back even after reading earlier versions. - -Summary -- Release highlights +Summary -- Release Highlights ============================= .. This section singles out the most important changes in Python 3.7. Brevity is key. +New syntax features: + +* :ref:`PEP 563 <whatsnew37-pep563>`, postponed evaluation of type annotations. + +Backwards incompatible syntax changes: + +* :keyword:`async` and :keyword:`await` are now reserved keywords. + +New library modules: + +* :mod:`contextvars`: :ref:`PEP 567 -- Context Variables <whatsnew37-pep567>` +* :mod:`dataclasses`: :ref:`PEP 557 -- Data Classes <whatsnew37-pep557>` +* :ref:`whatsnew37_importlib_resources` + +New built-in features: + +* :ref:`PEP 553 <whatsnew37-pep553>`, the new :func:`breakpoint` function. + +Python data model improvements: + +* :ref:`PEP 562 <whatsnew37-pep562>`, customization of access to + module attributes. + +* :ref:`PEP 560 <whatsnew37-pep560>`, core support for typing module and + generic types. + +* the insertion-order preservation nature of :ref:`dict <typesmapping>` + objects `has been declared`_ to be an official + part of the Python language spec. + +.. _has been declared: https://mail.python.org/pipermail/python-dev/2017-December/151283.html + +Significant improvements in the standard library: + +* The :mod:`asyncio` module has received new features, significant + :ref:`usability and performance improvements <whatsnew37_asyncio>`. -.. PEP-sized items next. +* The :mod:`time` module gained support for + :ref:`functions with nanosecond resolution <whatsnew37-pep564>`. +CPython implementation improvements: + +* :ref:`PEP 552 <whatsnew37-pep552>`, deterministic .pycs +* :ref:`PEP 538 <whatsnew37-pep538>`, legacy C locale coercion +* :ref:`PEP 540 <whatsnew37-pep540>`, forced UTF-8 runtime mode +* :ref:`the new development runtime mode <whatsnew37-devmode>` +* :ref:`PEP 565 <whatsnew37-pep565>`, improved :exc:`DeprecationWarning` + handling + +C API improvements: + +* :ref:`PEP 539 <whatsnew37-pep539>`, new C API for thread-local storage + +Documentation improvements: + +* :ref:`PEP 545 <whatsnew37-pep545>`, Python documentation translations +* New documentation translations: `Japanese <https://docs.python.org/ja/>`_, + `French <https://docs.python.org/fr/>`_, and + `Korean <https://docs.python.org/ko/>`_. + +This release features notable performance improvements in many areas. +The :ref:`whatsnew37-perf` section lists them in detail. + +For a list of changes that may affect compatibility with previous Python +releases please refer to the :ref:`porting-to-python-37` section. New Features ============ +.. _whatsnew37-pep563: + +PEP 563: Postponed Evaluation of Annotations +-------------------------------------------- + +The advent of type hints in Python uncovered two glaring usability issues +with the functionality of annotations added in :pep:`3107` and refined +further in :pep:`526`: + +* annotations could only use names which were already available in the + current scope, in other words they didn't support forward references + of any kind; and + +* annotating source code had adverse effects on startup time of Python + programs. + +Both of these issues are fixed by postponing the evaluation of +annotations. Instead of compiling code which executes expressions in +annotations at their definition time, the compiler stores the annotation +in a string form equivalent to the AST of the expression in question. +If needed, annotations can be resolved at runtime using +:func:`typing.get_type_hints`. In the common case where this is not +required, the annotations are cheaper to store (since short strings +are interned by the interpreter) and make startup time faster. + +Usability-wise, annotations now support forward references, making the +following syntax valid:: + + class C: + @classmethod + def from_string(cls, source: str) -> C: + ... + + def validate_b(self, obj: B) -> bool: + ... + + class B: + ... + +Since this change breaks compatibility, the new behavior needs to be enabled +on a per-module basis in Python 3.7 using a :mod:`__future__` import:: + + from __future__ import annotations + +It will become the default in Python 4.0. + +.. seealso:: + + :pep:`563` -- Postponed evaluation of annotations + PEP written and implemented by Łukasz Langa. + .. _whatsnew37-pep538: @@ -106,19 +215,46 @@ legacy C locale remains active when the core interpreter is initialized. PEP written and implemented by Nick Coghlan. +.. _whatsnew37-pep540: + +PEP 540: Forced UTF-8 Runtime Mode +----------------------------------- + +The new :option:`-X` ``utf8`` command line option and :envvar:`PYTHONUTF8` +environment variable can be used to enable the CPython *UTF-8 mode*. + +When in UTF-8 mode, CPython ignores the locale settings, and uses the +UTF-8 encoding by default. The error handlers for :data:`sys.stdin` and +:data:`sys.stdout` streams are set to ``surrogateescape``. + +The forced UTF-8 mode can be used to change the text handling behavior in +an embedded Python interpreter without changing the locale settings of +an embedding application. + +The UTF-8 mode is enabled by default when the locale is "C". See +:ref:`whatsnew37-pep538` for details. + +.. seealso:: + + :pep:`540` -- Add a new UTF-8 mode + PEP written and implemented by Victor Stinner + + .. _whatsnew37-pep553: -PEP 553: Built-in breakpoint() ------------------------------- +PEP 553: Built-in ``breakpoint()`` +---------------------------------- + +Python 3.7 includes the new built-in :func:`breakpoint` function as +an easy and consistent way to enter the Python debugger. -:pep:`553` describes a new built-in called :func:`breakpoint` which makes it -easy and consistent to enter the Python debugger. Built-in ``breakpoint()`` -calls :func:`sys.breakpointhook`. By default, this latter imports :mod:`pdb` and -then calls ``pdb.set_trace()``, but by binding ``sys.breakpointhook()`` to the -function of your choosing, ``breakpoint()`` can enter any debugger. Or, the -environment variable :envvar:`PYTHONBREAKPOINT` can be set to the callable of -your debugger of choice. Set ``PYTHONBREAKPOINT=0`` to completely disable -built-in ``breakpoint()``. +Built-in ``breakpoint()`` calls :func:`sys.breakpointhook`. By default, the +latter imports :mod:`pdb` and then calls ``pdb.set_trace()``, but by binding +``sys.breakpointhook()`` to the function of your choosing, ``breakpoint()`` can +enter any debugger. Additionally, the environment variable +:envvar:`PYTHONBREAKPOINT` can be set to the callable of your debugger of +choice. Set ``PYTHONBREAKPOINT=0`` to completely disable built-in +``breakpoint()``. .. seealso:: @@ -128,8 +264,8 @@ built-in ``breakpoint()``. .. _whatsnew37-pep539: -PEP 539: A New C-API for Thread-Local Storage in CPython --------------------------------------------------------- +PEP 539: New C API for Thread-Local Storage +------------------------------------------- While Python provides a C API for thread-local storage support; the existing :ref:`Thread Local Storage (TLS) API <thread-local-storage-api>` has used @@ -158,17 +294,16 @@ effort will be made to add such support. PEP written by Erik M. Bray; implementation by Masayuki Yamamoto. -PEP 562: Customization of access to module attributes +.. _whatsnew37-pep562: + +PEP 562: Customization of Access to Module Attributes ----------------------------------------------------- -It is sometimes convenient to customize or otherwise have control over access -to module attributes. A typical example is managing deprecation warnings. -Typical workarounds are assigning :attr:`~instance.__class__` of a module -object to a custom subclass of :class:`types.ModuleType` or replacing the -:data:`sys.modules` item with a custom wrapper instance. This procedure is -now simplified by recognizing ``__getattr__`` defined directly in a module -that would act like a normal :meth:`__getattr__` method, except that it will -be defined on module *instances*. +Python 3.7 allows defining :meth:`__getattr__` on modules and will call +it whenever a module attribute is otherwise not found. + +A typical example of where this may be useful is module attribute deprecation +and lazy loading. .. seealso:: @@ -176,62 +311,13 @@ be defined on module *instances*. PEP written and implemented by Ivan Levkivskyi -PEP 563: Postponed evaluation of annotations --------------------------------------------- - -The advent of type hints in Python uncovered two glaring usability issues -with the functionality of annotations added in :pep:`3107` and refined -further in :pep:`526`: - -* annotations could only use names which were already available in the - current scope, in other words they didn't support forward references - of any kind; and - -* annotating source code had adverse effects on startup time of Python - programs. - -Both of these issues are fixed by postponing the evaluation of -annotations. Instead of compiling code which executes expressions in -annotations at their definition time, the compiler stores the annotation -in a string form equivalent to the AST of the expression in question. -If needed, annotations can be resolved at runtime using -``typing.get_type_hints()``. In the common case where this is not -required, the annotations are cheaper to store (since short strings -are interned by the interpreter) and make startup time faster. - -Usability-wise, annotations now support forward references, making the -following syntax valid:: - - class C: - @classmethod - def from_string(cls, source: str) -> C: - ... - - def validate_b(self, obj: B) -> bool: - ... - - class B: - ... +.. _whatsnew37-pep564: -Since this change breaks compatibility, the new behavior can be enabled -on a per-module basis in Python 3.7 using a :mod:`__future__` import, like -this:: +PEP 564: New Time Functions With Nanosecond Resolution +------------------------------------------------------ - from __future__ import annotations - -It will become the default in Python 4.0. - -.. seealso:: - - :pep:`563` -- Postponed evaluation of annotations - PEP written and implemented by Łukasz Langa. - - -PEP 564: Add new time functions with nanosecond resolution ----------------------------------------------------------- - -Add six new "nanosecond" variants of existing functions to the :mod:`time` -module: +:pep:`564` adds six new "nanosecond" variants of existing functions +to the :mod:`time` module: * :func:`time.clock_gettime_ns` * :func:`time.clock_settime_ns` @@ -240,12 +326,12 @@ module: * :func:`time.process_time_ns` * :func:`time.time_ns` -While similar to the existing functions without the ``_ns`` suffix, they -provide nanosecond resolution: they return a number of nanoseconds as a Python -``int``. +The new functions are similar in function to the existing functions +without the ``_ns`` suffix. They differ by returning nanoseconds as +integers instead of fractional seconds. -The ``time.time_ns()`` resolution is 3 times better than the ``time.time()`` -resolution on Linux and Windows. +On Linux and Windows the resolution of :func:`time.time_ns` is 3 times +better than that of :func:`time.time`. .. seealso:: @@ -291,73 +377,10 @@ by breaking changes in the APIs they used. PEP written and implemented by Nick Coghlan -PEP 540: Add a new UTF-8 mode ------------------------------ - -Add a new UTF-8 mode to ignore the locale, use the UTF-8 encoding, and change -:data:`sys.stdin` and :data:`sys.stdout` error handlers to ``surrogateescape``. -This mode is enabled by default in the POSIX locale, but otherwise disabled by -default. - -The new :option:`-X` ``utf8`` command line option and :envvar:`PYTHONUTF8` -environment variable are added to control the UTF-8 mode. - -.. seealso:: +.. _whatsnew37-pep560: - :pep:`540` -- Add a new UTF-8 mode - PEP written and implemented by Victor Stinner - - -.. _whatsnew37-pep557: - -PEP 557: Data Classes ---------------------- - -Adds a new module :mod:`dataclasses`. It provides a class decorator -:func:`~dataclasses.dataclass` which inspects the class's variable annotations (see -:pep:`526`) and using them, adds methods such as ``__init__``, -``__repr__``, and ``__eq__`` to the class. It is similar to -:class:`typing.NamedTuple`, but also works on classes with mutable -instances, among other features. - -For example:: - - @dataclass - class Point: - x: float - y: float - z: float = 0.0 - - p = Point(1.5, 2.5) - print(p) # produces "Point(x=1.5, y=2.5, z=0.0)" - -.. seealso:: - - :pep:`557` -- Data Classes - PEP written and implemented by Eric V. Smith - - -PEP 567: Context Variables --------------------------- - -Adds a new module :mod:`contextvars`, that provides APIs to manage, -store, and access non-local state. - -Context variables are natively supported in :mod:`asyncio` and are -ready to be used without any extra configuration. - -The :mod:`decimal` module was updated to use *contextvars* to store -the current decimal context. This allows decimal operations to work -with the correct context in async/await code. - -.. seealso:: - - :pep:`567` -- Context Variables - PEP written and implemented by Yury Selivanov - - -PEP 560: Core support for typing module and generic types ---------------------------------------------------------- +PEP 560: Core Support for ``typing`` module and Generic Types +------------------------------------------------------------- Initially :pep:`484` was designed in such way that it would not introduce *any* changes to the core CPython interpreter. Now type hints and the :mod:`typing` @@ -375,17 +398,23 @@ fixed. PEP written and implemented by Ivan Levkivskyi -New Development Mode: -X dev ----------------------------- +.. _whatsnew37-devmode: -Add a new "development mode": :option:`-X` ``dev`` command line option and -:envvar:`PYTHONDEVMODE` environment variable to enable CPython's "development -mode", introducing additional runtime checks which are too expensive to be -enabled by default. See :option:`-X` ``dev`` documentation for the effects of -the development mode. +Development Runtime Mode: -X dev +-------------------------------- + +The new :option:`-X` ``dev`` command line option or the new +:envvar:`PYTHONDEVMODE` environment variable can be used to enable +CPython's *development mode*. When in development mode, CPython performs +additional runtime checks which are too expensive to be enabled by default. +See :option:`-X` ``dev`` documentation for the full description of the effects +of this mode. -Hash-based pycs ---------------- + +.. _whatsnew37-pep552: + +PEP 552: Hash-based .pyc Files +------------------------------ Python has traditionally checked the up-to-dateness of bytecode cache files (i.e., ``.pyc`` files) by comparing the source metadata (last-modified timestamp @@ -412,6 +441,27 @@ keeping ``.pyc`` files up-to-date. See :ref:`pyc-invalidation` for more information. +.. _whatsnew37-pep545: + +PEP 545: Python Documentation Translations +------------------------------------------ + +:pep:`545` describes the process of creating and maintaining Python +documentation translations. + +Three new translations have been added: + +- Japanese: https://docs.python.org/ja/ +- French: https://docs.python.org/fr/ +- Korean: https://docs.python.org/ko/ + +.. seealso:: + + :pep:`545` -- Python Documentation Translations + PEP written and implemented by Julien Palard, Inada Naoki, and + Victor Stinner. + + Other Language Changes ====================== @@ -422,6 +472,11 @@ Other Language Changes * :meth:`bytes.fromhex` and :meth:`bytearray.fromhex` now ignore all ASCII whitespace, not only spaces. (Contributed by Robert Xiao in :issue:`28927`.) +* :class:`str`, :class:`bytes`, and :class:`bytearray` gained support for + the new :meth:`isascii() <str.isascii>` method, which can be used to + test if a string or bytes contain only the ASCII characters. + (Contributed by INADA Naoki in :issue:`32677`.) + * :exc:`ImportError` now displays module name and module ``__file__`` path when ``from ... import ...`` fails. (Contributed by Matthias Bussonnier in :issue:`29546`.) @@ -446,22 +501,83 @@ Other Language Changes time when an import occurs) (Contributed by Nick Coghlan in :issue:`33053`.) +* The new :option:`-X` ``importtime`` option or the + :envvar:`PYTHONPROFILEIMPORTTIME` environment variable can be used to show + the timing of each module import. + (Contributed by Victor Stinner in :issue:`31415`.) + New Modules =========== +.. _whatsnew37-pep567: + +contextvars +----------- + +The new :mod:`contextvars` module and a set of new C APIs introduce +support for *context variables*. Context variables are conceptually +similar to thread-local variables. Unlike TLS, context variables +support asynchronous code correctly. + +The :mod:`asyncio` and :mod:`decimal` modules have been updated to use +and support context variables out of the box. Particularly the active +decimal context is now stored in a context variable, which allows +decimal operations to work with the correct context in asynchronous code. + +.. seealso:: + + :pep:`567` -- Context Variables + PEP written and implemented by Yury Selivanov + + +.. _whatsnew37-pep557: + +dataclasses +----------- + +The new :func:`~dataclasses.dataclass` decorator provides a way to declare +*data classes*. A data class describes its attributes using class variable +annotations. Its constructor and other magic methods, such as +:meth:`~object.__repr__`, :meth:`~object.__eq__`, and +:meth:`~object.__hash__` are generated automatically. + +Example:: + + @dataclass + class Point: + x: float + y: float + z: float = 0.0 + + p = Point(1.5, 2.5) + print(p) # produces "Point(x=1.5, y=2.5, z=0.0)" + +.. seealso:: + + :pep:`557` -- Data Classes + PEP written and implemented by Eric V. Smith + + +.. _whatsnew37_importlib_resources: + importlib.resources ------------------- -This module provides several new APIs and one new ABC for access to, opening, -and reading *resources* inside packages. Resources are roughly akin to files -inside of packages, but they needn't be actual files on the physical file -system. Module loaders can provide a :meth:`get_resource_reader()` function -which returns a :class:`importlib.abc.ResourceReader` instance to support this +The new :mod:`importlib.resources` module provides several new APIs and one +new ABC for access to, opening, and reading *resources* inside packages. +Resources are roughly similar to files inside packages, but they needn't +be actual files on the physical file system. Module loaders can provide a +:meth:`get_resource_reader()` function which returns +a :class:`importlib.abc.ResourceReader` instance to support this new API. Built-in file path loaders and zip file loaders both support this. -(see the PyPI package -`importlib_resources <http://importlib-resources.readthedocs.io/en/latest/>`_ -as a compatible back port for older Python versions). + +Contributed by Barry Warsaw and Brett Cannon in :issue:`32248`. + +.. seealso:: + + `importlib_resources <http://importlib-resources.readthedocs.io/en/latest/>`_ + -- a PyPI backport for earlier Python versions. Improved Modules @@ -471,10 +587,143 @@ Improved Modules argparse -------- -The :meth:`~argparse.ArgumentParser.parse_intermixed_args` supports letting -the user intermix options and positional arguments on the command line, -as is possible in many unix commands. It supports most but not all -argparse features. (Contributed by paul.j3 in :issue:`14191`.) +The new :meth:`ArgumentParser.parse_intermixed_args() +<argparse.ArgumentParser.parse_intermixed_args>` +method allows intermixing options and positional arguments. +(Contributed by paul.j3 in :issue:`14191`.) + + +.. _whatsnew37_asyncio: + +asyncio +------- + +The :mod:`asyncio` module has received many new features, usability and +:ref:`performance improvements <whatsnew37-asyncio-perf>`. Notable changes +include: + +* The new :term:`provisional <provisional api>` :func:`asyncio.run` function can + be used to run a coroutine from synchronous code by automatically creating and + destroying the event loop. + (Contributed by Yury Selivanov in :issue:`32314`.) + +* The new :func:`asyncio.create_task` function has been added as a shortcut + to ``asyncio.get_event_loop().create_task()``. + (Contributed by Andrew Svetlov in :issue:`32311`.) + +* The new :meth:`loop.start_tls() <asyncio.AbstractEventLoop.start_tls>` + method can be used to upgrade an existing connection to TLS. + (Contributed by Yury Selivanov in :issue:`23749`.) + +* The new :meth:`loop.sock_recv_into() <asyncio.AbstractEventLoop.sock_recv_into>` + method allows reading data from a socket directly into a provided buffer making + it possible to reduce data copies. + (Contributed by Antoine Pitrou in :issue:`31819`.) + +* The new :func:`asyncio.current_task` function returns the currently running + :class:`~asyncio.Task` instance, and the new :func:`asyncio.all_tasks` + function returns a set of all existing ``Task`` instances in a given loop. + The :meth:`Task.current_task() <asyncio.Task.current_task>` and + :meth:`Task.all_tasks() <asyncio.Task.all_tasks>` methods have been deprecated. + (Contributed by Andrew Svetlov in :issue:`32250`.) + +* The new *provisional* :class:`~asyncio.BufferedProtocol` class allows + implementing streaming protocols with manual control over the receive buffer. + (Contributed by Yury Selivanov in :issue:`32251`.) + +* The new :func:`asyncio.get_running_loop` function returns the currently + running loop, and raises a :exc:`RuntimeError` if no loop is running. + This is in contrast with :func:`asyncio.get_event_loop`, which will *create* + a new event loop if none is running. + (Contributed by Yury Selivanov in :issue:`32269`.) + +* The new :meth:`StreamWriter.wait_closed() <asyncio.StreamWriter.wait_closed>` + coroutine method allows waiting until the stream writer is closed. The new + :meth:`StreamWriter.is_closing() <asyncio.StreamWriter.is_closing>` method + can be used to determine if the writer is closing. + (Contributed by Andrew Svetlov in :issue:`32391`.) + +* The new :meth:`loop.sock_sendfile() <asyncio.AbstractEventLoop.sock_sendfile>` + coroutine method allows sending files using :mod:`os.sendfile` when possible. + (Contributed by Andrew Svetlov in :issue:`32410`.) + +* The new :meth:`Task.get_loop() <asyncio.Task.get_loop>` and + :meth:`Future.get_loop() <asyncio.Future.get_loop>` methods + return the instance of the loop on which a task or a future were created. + :meth:`Server.get_loop() <asyncio.Server.get_loop>` allows doing the same for + :class:`asyncio.Server` objects. + (Contributed by Yury Selivanov in :issue:`32415` and + Srinivas Reddy Thatiparthy in :issue:`32418`.) + +* It is now possible to control how instances of :class:`asyncio.Server` begin + serving. Previously, the server would start serving immediately when created. + The new *start_serving* keyword argument to + :meth:`loop.create_server() <asyncio.AbstractEventLoop.create_server>` and + :meth:`loop.create_unix_server() <asyncio.AbstractEventLoop.create_unix_server>`, + as well as :meth:`Server.start_serving() <asyncio.Server.start_serving>`, and + :meth:`Server.serve_forever() <asyncio.Server.serve_forever>` + can be used to decouple server instantiation and serving. The new + :meth:`Server.is_serving() <asyncio.Server.is_serving>` method returns ``True`` + if the server is serving. :class:`~asyncio.Server` objects are now + asynchronous context managers:: + + srv = await loop.create_server(...) + + async with srv: + # some code + + # At this point, srv is closed and no longer accepts new connections. + + (Contributed by Yury Selivanov in :issue:`32662`.) + +* Callback objects returned by + :func:`loop.call_later() <asyncio.AbstractEventLoop.call_later>` + gained the new :meth:`when() <asyncio.TimerHandle.when>` method which + returns an absolute scheduled callback timestamp. + (Contributed by Andrew Svetlov in :issue:`32741`.) + +* The :meth:`loop.create_datagram_endpoint() \ + <asyncio.AbstractEventLoop.create_datagram_endpoint>` method + gained support for Unix sockets. + (Contributed by Quentin Dawans in :issue:`31245`.) + +* The :meth:`loop.create_connection() <asyncio.AbstractEventLoop.create_connection>`, + :meth:`loop.create_server() <asyncio.AbstractEventLoop.create_server>`, + :meth:`loop.create_unix_server() <asyncio.AbstractEventLoop.create_unix_server>`, and + :meth:`loop.create_accepted_socket() <asyncio.BaseEventLoop.connect_accepted_socket>` + now accept the *ssl_handshake_timeout* keyword argument. + (Contributed by Neil Aspinall in :issue:`29970`.) + +* The new :meth:`Handle.cancelled() <asyncio.Handle.cancelled>` method returns + ``True`` if the callback was cancelled. + (Contributed by Marat Sharafutdinov in :issue:`31943`.) + +* The asyncio source has been converted to use the + :keyword:`async`/:keyword:`await` syntax. + (Contributed by Andrew Svetlov in :issue:`32193`.) + +* The new :meth:`ReadTransport.is_reading() <asyncio.ReadTransport.is_reading>` + method can be used to determine the reading state of the transport. + Additionally, calls to + :meth:`ReadTransport.resume_reading() <asyncio.ReadTransport.resume_reading>` + and :meth:`ReadTransport.pause_reading() <asyncio.ReadTransport.pause_reading>` + are now idempotent. + (Contributed by Yury Selivanov in :issue:`32356`.) + +* Loop methods which accept socket paths now support passing + :term:`path-like objects <path-like object>`. + (Contributed by Yury Selivanov in :issue:`32066`.) + +* In :mod:`asyncio` TCP sockets on Linux are now created with ``TCP_NODELAY`` + flag set by default. + (Contributed by Yury Selivanov and Victor Stinner in :issue:`27456`.) + +* Exceptions occurring in cancelled tasks are no longer logged. + (Contributed by Yury Selivanov in :issue:`30508`.) + +Several ``asyncio`` APIs have been +:ref:`deprecated <whatsnew37-asyncio-deprecated>`. + binascii -------- @@ -483,54 +732,116 @@ The :func:`~binascii.b2a_uu` function now accepts an optional *backtick* keyword argument. When it's true, zeros are represented by ``'`'`` instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.) + calendar -------- -The class :class:`~calendar.HTMLCalendar` has new class attributes which ease -the customisation of the CSS classes in the produced HTML calendar. +The :class:`~calendar.HTMLCalendar` class has new class attributes which ease +the customization of CSS classes in the produced HTML calendar. (Contributed by Oz Tiram in :issue:`30095`.) + +collections +----------- + +``collections.namedtuple()`` now supports default values. +(Contributed by Raymond Hettinger in :issue:`32320`.) + + +compileall +---------- + +:func:`compileall.compile_dir` learned the new *invalidation_mode* parameter, +which can be used to enable +:ref:`hash-based .pyc invalidation <whatsnew37-pep552>`. The invalidation +mode can also be specified on the command line using the new +``--invalidation-mode`` argument. +(Contributed by Benjamin Peterson in :issue:`31650`.) + + +concurrent.futures +------------------ + +:class:`ProcessPoolExecutor <concurrent.futures.ProcessPoolExecutor>` and +:class:`ThreadPoolExecutor <concurrent.futures.ThreadPoolExecutor>` now +support the new *initializer* and *initargs* constructor arguments. +(Contributed by Antoine Pitrou in :issue:`21423`.) + +The :class:`ProcessPoolExecutor <concurrent.futures.ProcessPoolExecutor>` +can now take the multiprocessing context via the new *mp_context* argument. +(Contributed by Thomas Moreau in :issue:`31540`.) + + contextlib ---------- -:func:`~contextlib.asynccontextmanager` and -:class:`~contextlib.AbstractAsyncContextManager` have been added. (Contributed -by Jelle Zijlstra in :issue:`29679` and :issue:`30241`.) +The new :func:`~contextlib.nullcontext` is a simpler and faster no-op +context manager than :class:`~contextlib.ExitStack`. +(Contributed by Jesse-Bakker in :issue:`10049`.) + +The new :func:`~contextlib.asynccontextmanager`, +:class:`~contextlib.AbstractAsyncContextManager`, and +:class:`~contextlib.AsyncExitStack` have been added to +complement their synchronous counterparts. (Contributed +by Jelle Zijlstra in :issue:`29679` and :issue:`30241`, +and by Alexander Mohr and Ilya Kulakov in :issue:`29302`.) -:class:`contextlib.AsyncExitStack` has been added. (Contributed by -Alexander Mohr and Ilya Kulakov in :issue:`29302`.) cProfile -------- -:mod:`cProfile` command line now accepts ``-m module_name`` as an alternative -to script path. (Contributed by Sanyam Khurana in :issue:`21862`.) +The :mod:`cProfile` command line now accepts ``-m module_name`` as an +alternative to script path. (Contributed by Sanyam Khurana in :issue:`21862`.) + crypt ----- -Added support for the Blowfish method. +The :mod:`crypt` module now supports the Blowfish hashing method. (Contributed by Serhiy Storchaka in :issue:`31664`.) -The :func:`~crypt.mksalt` function now allows to specify the number of rounds +The :func:`~crypt.mksalt` function now allows specifying the number of rounds for hashing. (Contributed by Serhiy Storchaka in :issue:`31702`.) + datetime -------- -Added the :meth:`datetime.fromisoformat <datetime.datetime.fromisoformat>` -method, which constructs a :class:`~datetime.datetime` object from a string +The new :meth:`datetime.fromisoformat() <datetime.datetime.fromisoformat>` +method constructs a :class:`~datetime.datetime` object from a string in one of the formats output by -:meth:`datetime.isoformat <datetime.datetime.isoformat>`. +:meth:`datetime.isoformat() <datetime.datetime.isoformat>`. (Contributed by Paul Ganssle in :issue:`15873`.) +The :class:`tzinfo <datetime.tzinfo>` class now supports sub-minute offsets. +(Contributed by Alexander Belopolsky in :issue:`5288`.) + + +dbm +--- + +:mod:`dbm.dumb` now supports reading read-only files and no longer writes the +index file when it is not changed. + + +decimal +------- + +The :mod:`decimal` module now uses :ref:`context variables <whatsnew37-pep567>` +to store the decimal context. +(Contributed by Yury Selivanov in :issue:`32630`.) + + dis --- -The :func:`~dis.dis` function now is able to +The :func:`~dis.dis` function is now able to disassemble nested code objects (the code of comprehensions, generator expressions and nested functions, and the code used for building nested -classes). (Contributed by Serhiy Storchaka in :issue:`11822`.) +classes). The maximum depth of disassembly recursion is controlled by +the new *depth* parameter. +(Contributed by Serhiy Storchaka in :issue:`11822`.) + distutils --------- @@ -539,97 +850,244 @@ distutils therefore included in source distributions. (Contributed by Ryan Gonzalez in :issue:`11913`.) -:class:`distutils.core.setup` now warns if the ``classifiers``, ``keywords`` -and ``platforms`` fields are not specified as a list or a string. -(Contributed by Berker Peksag in :issue:`19610`.) -The ``upload`` command no longer tries to change CR end-of-line characters -to CRLF. This fixes a corruption issue with sdists that ended with a byte -equivalent to CR. -(Contributed by Bo Bayles in :issue:`32304`.) +enum +---- + +The :class:`Enum <enum.Enum>` learned the new ``_ignore_`` class property, +which allows listing the names of properties which should not become +enum members. +(Contributed by Ethan Furman in :issue:`31801`.) + +In Python 3.8, attempting to check for non-Enum objects in :class:`Enum` +classes will raise a :exc:`TypeError` (e.g. ``1 in Color``); similarly, +attempting to check for non-Flag objects in a :class:`Flag` member will +raise :exc:`TypeError` (e.g. ``1 in Perm.RW``); currently, both operations +return :const:`False` instead and are deprecated. +(Contributed by Ethan Furman in :issue:`33217`.) + + +functools +--------- + +:func:`functools.singledispatch` now supports registering implementations +using type annotations. +(Contributed by Łukasz Langa in :issue:`32227`.) + + +gc +-- + +The new :func:`gc.freeze` function allows freezing all objects tracked +by the garbage collector and excluding them from future collections. +This can be used before a POSIX ``fork()`` call to make the GC copy-on-write +friendly or to speed up collection. The new :func:`gc.unfreeze` functions +reverses this operation. Additionally, :func:`gc.get_freeze_count` can +be used to obtain the number of frozen objects. +(Contributed by Li Zekun in :issue:`31558`.) + + +hmac +---- + +The :mod:`hmac` module now has an optimized one-shot :func:`~hmac.digest` +function, which is up to three times faster than :func:`~hmac.HMAC`. +(Contributed by Christian Heimes in :issue:`32433`.) + http.client ----------- -Add configurable *blocksize* to :class:`~http.client.HTTPConnection` and -:class:`~http.client.HTTPSConnection` for improved upload throughput. +:class:`~http.client.HTTPConnection` and :class:`~http.client.HTTPSConnection` +now support the new *blocksize* argument for improved upload throughput. (Contributed by Nir Soffer in :issue:`31945`.) + http.server ----------- -:class:`~http.server.SimpleHTTPRequestHandler` supports the HTTP -``If-Modified-Since`` header. The server returns the 304 response status if the -target file was not modified after the time specified in the header. +:class:`~http.server.SimpleHTTPRequestHandler` now supports the HTTP +``If-Modified-Since`` header. The server returns the 304 response status if +the target file was not modified after the time specified in the header. (Contributed by Pierre Quentel in :issue:`29654`.) -Add the parameter *directory* to the :class:`~http.server.SimpleHTTPRequestHandler` -and the ``--directory`` to the command line of the module :mod:`http.server`. +:class:`~http.server.SimpleHTTPRequestHandler` accepts the new *directory* +argument, in addition to the new ``--directory`` command line argument. With this parameter, the server serves the specified directory, by default it uses the current working directory. (Contributed by Stéphane Wirtel and Julien Palard in :issue:`28707`.) -hmac ----- +The new :class:`ThreadedHTTPServer <http.server.ThreadedHTTPServer>` class +uses threads to handle requests using :class:`~socketserver.ThreadingMixin`. +It is used when ``http.server`` is run with ``-m``. +(Contributed by Julien Palard in :issue:`31639`.) -The :mod:`hmac` module now has an optimized one-shot :func:`~hmac.digest` -function, which is up to three times faster than :func:`~hmac.HMAC`. -(Contributed by Christian Heimes in :issue:`32433`.) importlib --------- The :class:`importlib.abc.ResourceReader` ABC was introduced to -support the loading of resource from packages. +support the loading of resources from packages. See also +:ref:`whatsnew37_importlib_resources`. +(Contributed by Barry Warsaw, Brett Cannon in :issue:`32248`.) + +:func:`importlib.reload` now raises :exc:`ModuleNotFoundError` if the module +lacks a spec. +(Contributed by Garvit Khatri in :issue:`29851`.) + +:func:`importlib.find_spec` now raises ``ModuleNotFoundError`` instead of +:exc:`AttributeError` if the specified parent module is not a package (i.e. +lacks a ``__path__`` attribute). +(Contributed by Milan Oberkirch in :issue:`30436`.) + +The new :func:`importlib.source_hash` can be used to compute the hash of +the passed source. A :ref:`hash-based .pyc file <whatsnew37-pep552>` +embeds the value returned by this function. + + +io +-- + +The new :meth:`TextIOWrapper.reconfigure() <io.TextIOWrapper.reconfigure>` +method can be used to reconfigure the text stream with the new settings. +(Contributed by Antoine Pitrou in :issue:`30526` and +INADA Naoki in :issue:`15216`.) + + +ipaddress +--------- + +The new ``subnet_of()`` and ``supernet_of()`` methods of +:class:`ipaddress.IPv6Network` and :class:`ipaddress.IPv4Network` can +be used for network containment tests. +(Contributed by Michel Albert and Cheryl Sabella in :issue:`20825`.) + + +itertools +--------- + +:func:`itertools.islice` now accepts +:meth:`integer-like objects <object.__index__>` as start, stop, +and slice arguments. +(Contributed by Will Roberts in :issue:`30537`.) + locale ------ -Added another argument *monetary* in :func:`~locale.format_string` of :mod:`locale`. -If *monetary* is true, the conversion uses monetary thousands separator and -grouping strings. (Contributed by Garvit in :issue:`10379`.) +The new *monetary* argument to :func:`locale.format_string` can be used +to make the conversion use monetary thousands separators and +grouping strings. (Contributed by Garvit in :issue:`10379`.) + +The :func:`locale.getpreferredencoding` function now always returns ``'UTF-8'`` +on Android or when in the :ref:`forced UTF-8 mode <whatsnew37-pep540>`. + + +logging +------- + +:class:`~logging.Logger` instances can now be pickled. +(Contributed by Vinay Sajip in :issue:`30520`.) + +The new :meth:`StreamHandler.setStream() <logging.StreamHandler.setStream>` +method can be used to replace the logger stream after handler creation. +(Contributed by Vinay Sajip in :issue:`30522`.) + +It is now possible to specify keyword arguments to handler constructors in +configuration passed to :func:`logging.config.fileConfig`. +(Contributed by Preston Landers in :issue:`31080`.) -The :func:`~locale.getpreferredencoding` function now always returns ``'UTF-8'`` -on Android or in the UTF-8 mode (:option:`-X` ``utf8`` option), the locale and -the *do_setlocale* argument are ignored. math ---- -New :func:`~math.remainder` function, implementing the IEEE 754-style remainder -operation. (Contributed by Mark Dickinson in :issue:`29962`.) +The new :func:`math.remainder` function implements the IEEE 754-style remainder +operation. (Contributed by Mark Dickinson in :issue:`29962`.) + + +mimetypes +--------- + +The MIME type of .bmp has been changed from ``'image/x-ms-bmp'`` to +``'image/bmp'``. +(Contributed by Nitish Chandra in :issue:`22589`.) + + +msilib +------ + +The new :meth:`Database.Close() <msilib.Database.Close>` method can be used +to close the :abbr:`MSI` database. +(Contributed by Berker Peksag in :issue:`20486`.) + + +multiprocessing +--------------- + +The new :meth:`Process.close() <multiprocessing.Process.close>` method +explicitly closes the process object and releases all resources associated +with it. :exc:`ValueError` is raised if the underlying process is still +running. +(Contributed by Antoine Pitrou in :issue:`30596`.) + +The new :meth:`Process.kill() <multiprocessing.Process.kill>` method can +be used to terminate the process using the :data:`SIGKILL` signal on Unix. +(Contributed by Vitor Pereira in :issue:`30794`.) + +Non-daemonic threads created by :class:`~multiprocessing.Process` are now +joined on process exit. +(Contributed by Antoine Pitrou in :issue:`18966`.) + os -- -Added support for :class:`bytes` paths in :func:`~os.fwalk`. (Contributed by -Serhiy Storchaka in :issue:`28682`.) +:func:`os.fwalk` now accepts the *path* argument as :class:`bytes`. +(Contributed by Serhiy Storchaka in :issue:`28682`.) -Added support for :ref:`file descriptors <path_fd>` in :func:`~os.scandir` -on Unix. (Contributed by Serhiy Storchaka in :issue:`25996`.) +:func:`os.scandir` gained support for :ref:`file descriptors <path_fd>`. +(Contributed by Serhiy Storchaka in :issue:`25996`.) -New function :func:`~os.register_at_fork` allows registering Python callbacks -to be executed on a process fork. (Contributed by Antoine Pitrou in -:issue:`16500`.) +The new :func:`~os.register_at_fork` function allows registering Python +callbacks to be executed at process fork. +(Contributed by Antoine Pitrou in :issue:`16500`.) -Exposed the system calls *preadv*, *preadv2*, *pwritev* and *pwritev2* through -the new functions :func:`~os.preadv` and :func:`~os.pwritev`. (Contributed by -Pablo Galindo in :issue:`31368`.) +Exposed the *preadv*, *preadv2*, *pwritev* and *pwritev2* system calls through +the new :func:`~os.preadv` and :func:`~os.pwritev` functions. +(Contributed by Pablo Galindo in :issue:`31368`.) + +The mode argument of :func:`os.makedirs` no longer affects the file +permission bits of newly-created intermediate-level directories. +(Contributed by Serhiy Storchaka in :issue:`19930`.) + +:func:`os.dup2` now returns the new file descriptor. Previously, ``None`` +was always returned. +(Contributed by Benjamin Peterson in :issue:`32441`.) + +The structure returned by :func:`os.stat` now contains the +:attr:`~os.stat_result.st_fstype` attribute on Solaris and its derivatives. +(Contributed by Jesús Cea Avión in :issue:`32659`.) + + +pathlib +------- + +The new :meth:`Path.is_mount() <pathlib.Path.is_mount>` method is now available +on POSIX systems and can be used to determine whether a path is a mount point. +(Contributed by Cooper Ry Lees in :issue:`30897`.) -Exposed the system call *posix_spawn* through the new function -:func:`~os.posix_spawn`. (Contributed by Pablo Galindo, Serhiy Storchaka and -Gregory P. Smith in :issue:`20104`.) pdb --- -:func:`~pdb.set_trace` now takes an optional *header* keyword-only -argument. If given, this is printed to the console just before debugging +:func:`pdb.set_trace` now takes an optional *header* keyword-only +argument. If given, it is printed to the console just before debugging begins. (Contributed by Barry Warsaw in :issue:`31389`.) :mod:`pdb` command line now accepts ``-m module_name`` as an alternative to script file. (Contributed by Mario Corchero in :issue:`32206`.) + py_compile ---------- @@ -641,6 +1099,22 @@ This allows for guaranteeing files when they are created eagerly. (Contributed by Bernhard M. Wiedemann in :issue:`29708`.) + +pydoc +----- + +The pydoc server can now bind to an arbitrary hostname specified by the +new ``-n`` command-line argument. +(Contributed by Feanil Patel in :issue:`31128`.) + + +queue +----- + +The new :class:`~queue.SimpleQueue` class is an unbounded :abbr:`FIFO` queue. +(Contributed by Antoine Pitrou in :issue:`14976`.) + + re -- @@ -652,70 +1126,128 @@ can be set within the scope of a group. ``'^$'`` or ``(?=-)`` that matches an empty string. (Contributed by Serhiy Storchaka in :issue:`25054`.) +Regular expressions compiled with the :const:`re.LOCALE` flag no longer +depend on the locale at compile time. Locale settings are applied only +when the compiled regular expression is used. +(Contributed by Serhiy Storchaka in :issue:`30215`.) + +:exc:`FutureWarning` is now emitted if a regular expression contains +character set constructs that will change semantically in the future, +such as nested sets and set operations. +(Contributed by Serhiy Storchaka in :issue:`30349`.) + +Compiled regular expression and match objects can now be copied +using :func:`copy.copy` and :func:`copy.deepcopy`. +(Contributed by Serhiy Storchaka in :issue:`10076`.) + + +signal +------ + +The new *warn_on_full_buffer* argument to the :func:`signal.set_wakeup_fd` +function makes it possible to specify whether Python prints a warning on +stderr when the wakeup buffer overflows. +(Contributed by Nathaniel J. Smith in :issue:`30050`.) + + +socket +------ + +The new :func:`socket.getblocking() <socket.socket.getblocking>` method +returns ``True`` if the socket is in blocking mode and ``False`` otherwise. +(Contributed by Yury Selivanov in :issue:`32373`.) + +The new :func:`socket.close` function closes the passed socket file descriptor. +This function should be used instead of :func:`os.close` for better +compatibility across platforms. +(Contributed by Christian Heimes in :issue:`32454`.) + +The :mod:`socket` module now exposes the :data:`socket.TCP_CONGESTION` +(Linux 2.6.13), :data:`socket.TCP_USER_TIMEOUT` (Linux 2.6.37), and +:data:`socket.TCP_NOTSENT_LOWAT` (Linux 3.12) constants. +(Contributed by Omar Sandoval in :issue:`26273` and +Nathaniel J. Smith in :issue:`29728`.) + +Support for :data:`socket.AF_VSOCK` sockets has been added to allow +communication between virtual machines and their hosts. +(Contributed by Cathy Avery in :issue:`27584`.) + +Sockets now auto-detect family, type and protocol from file descriptor +by default. +(Contributed by Christian Heimes in :issue:`28134`.) + sqlite3 ------- -:class:`sqlite3.Connection` now exposes a :class:`~sqlite3.Connection.backup` -method, if the underlying SQLite library is at version 3.6.11 or higher. +:class:`sqlite3.Connection` now exposes the :meth:`~sqlite3.Connection.backup` +method when the underlying SQLite library is at version 3.6.11 or higher. (Contributed by Lele Gaifax in :issue:`27645`.) +The *database* argument of :func:`sqlite3.connect` now accepts any +:term:`path-like object`, instead of just a string. +(Contributed by Anders Lorentsen in :issue:`31843`.) + ssl --- The :mod:`ssl` module now uses OpenSSL's builtin API instead of -:func:`~ssl.match_hostname` to check host name or IP address. Values -are validated during TLS handshake. Any cert validation error including -a failing host name match now raises :exc:`~ssl.SSLCertVerificationError` and -aborts the handshake with a proper TLS Alert message. The new exception -contains additional information. Host name validation can be customized -with :attr:`~ssl.SSLContext.host_flags`. +:func:`~ssl.match_hostname` to check a host name or an IP address. Values +are validated during TLS handshake. Any certificate validation error +including failing the host name check now raises +:exc:`~ssl.SSLCertVerificationError` and aborts the handshake with a proper +TLS Alert message. The new exception contains additional information. +Host name validation can be customized with +:attr:`SSLContext.host_flags <ssl.SSLContext.host_flags>`. (Contributed by Christian Heimes in :issue:`31399`.) .. note:: - The improved host name check requires an OpenSSL 1.0.2 or 1.1 compatible - libssl. OpenSSL 0.9.8 and 1.0.1 are no longer supported. LibreSSL is - temporarily not supported until it gains the necessary OpenSSL 1.0.2 APIs. + The improved host name check requires a *libssl* implementation compatible + with OpenSSL 1.0.2 or 1.1. Consequently, OpenSSL 0.9.8 and 1.0.1 are no + longer supported and LibreSSL is temporarily not supported until it gains + the necessary OpenSSL 1.0.2 APIs. -The ssl module no longer sends IP addresses in SNI TLS extension. +The ``ssl`` module no longer sends IP addresses in SNI TLS extension. (Contributed by Christian Heimes in :issue:`32185`.) :func:`~ssl.match_hostname` no longer supports partial wildcards like -``www*.example.org``. :attr:`~ssl.SSLContext.host_flags` has partial -wildcard matching disabled by default. +``www*.example.org``. :attr:`SSLContext.host_flags <ssl.SSLContext.host_flags>` +has partial wildcard matching disabled by default. (Contributed by Mandeep Singh in :issue:`23033` and Christian Heimes in :issue:`31399`.) -The default cipher suite selection of the ssl module now uses a blacklist -approach rather than a hard-coded whitelist. Python no longer re-enables -ciphers that have been blocked by OpenSSL security update. Default cipher -suite selection can be configured on compile time. +The default cipher suite selection of the ``ssl`` module now uses a blacklist +approach rather than a hard-coded whitelist. Python no longer re-enables +ciphers that have been blocked by OpenSSL security updates. Default cipher +suite selection can be configured at compile time. (Contributed by Christian Heimes in :issue:`31429`.) -Added support for validating server certificates containing -internationalized domain names (IDNs). As part of this change, the -:attr:`ssl.SSLSocket.server_hostname` attribute now stores the -expected hostname in A-label form (``"xn--pythn-mua.org"``), rather -than the U-label form (``"pythön.org"``). (Contributed by +Validation of server certificates containing internationalized domain names +(IDNs) is now supported. As part of this change, the +:attr:`SSLSocket.server_hostname <ssl.SSLSocket.server_hostname>` attribute +now stores the expected hostname in A-label form (``"xn--pythn-mua.org"``), +rather than the U-label form (``"pythön.org"``). (Contributed by Nathaniel J. Smith and Christian Heimes in :issue:`28414`.) -The ssl module has preliminary and experimental support for TLS 1.3 and -OpenSSL 1.1.1. (Contributed by Christian Heimes in :issue:`32947`, +The ``ssl`` module has preliminary and experimental support for TLS 1.3 and +OpenSSL 1.1.1. (Contributed by Christian Heimes in :issue:`32947`, :issue:`20995`, :issue:`29136`, and :issue:`30622`) :class:`~ssl.SSLSocket` and :class:`~ssl.SSLObject` no longer have a public -constructor. Direct instantiation was never a documented and supported -feature. Instances must be created with :class:`~ssl.SSLContext` methods +constructor. Direct instantiation was never a documented and supported +feature. Instances must be created with :class:`~ssl.SSLContext` methods :meth:`~ssl.SSLContext.wrap_socket` and :meth:`~ssl.SSLContext.wrap_bio`. (Contributed by Christian Heimes in :issue:`32951`) OpenSSL 1.1 APIs for setting the minimum and maximum TLS protocol version are -available as :attr:`~ssl.SSLContext.minimum_version` and -:attr:`~ssl.SSLContext.maximum_version`. Supported protocols are indicated -by new flags like :data:`~ssl.HAS_TLSv1_1`. +available as :attr:`SSLContext.minimum_version <ssl.SSLContext.minimum_version>` +and :attr:`SSLContext.maximum_version <ssl.SSLContext.maximum_version>`. +Supported protocols are indicated by serveral new flags, such as +:data:`~ssl.HAS_TLSv1_1`. (Contributed by Christian Heimes in :issue:`32609`.) + string ------ @@ -723,32 +1255,61 @@ string expression pattern for braced placeholders and non-braced placeholders separately. (Contributed by Barry Warsaw in :issue:`1198569`.) + subprocess ---------- +The :func:`subprocess.run` function accepts the new *capture_output* +keyword argument. When true, stdout and stderr will be captured. +This is equivalent to passing :data:`subprocess.PIPE` as *stdout* and +*stderr* arguments. +(Contributed by Bo Bayles in :issue:`32102`.) + +The ``subprocess.run`` function and the :class:`subprocess.Popen` constructor +now accept the *text* keyword argument as an alias +to *universal_newlines*. +(Contributed by Andrew Clegg in :issue:`31756`.) + On Windows the default for *close_fds* was changed from ``False`` to ``True`` when redirecting the standard handles. It's now possible to set -*close_fds* to ``True`` when redirecting the standard handles. See -:class:`subprocess.Popen`. +*close_fds* to true when redirecting the standard handles. See +:class:`subprocess.Popen`. This means that *close_fds* now defaults to +``True`` on all supported platforms. +(Contributed by Segev Finer in :issue:`19764`.) + +The subprocess module is now more graceful when handling +:exc:`KeyboardInterrupt` during :func:`subprocess.call`, +:func:`subprocess.run`, or in a :class:`~subprocess.Popen` +context manager. It now waits a short amount of time for the child +to exit, before continuing the handling of the ``KeyboardInterrupt`` +exception. +(Contributed by Gregory P. Smith in :issue:`25942`.) -This means that *close_fds* now defaults to ``True`` on all supported -platforms. (Contributed by Segev Finer in :issue:`19764`.) sys --- -Added :attr:`sys.flags.dev_mode` flag for the new development mode. +The new :func:`sys.breakpointhook` hook function is called by the +built-in :func:`breakpoint`. +(Contributed by Barry Warsaw in :issue:`31353`.) -tkinter -------- +On Android, the new :func:`sys.getandroidapilevel` returns the build-time +Android API version. +(Contributed by Victor Stinner in :issue:`28740`.) + +The new :func:`sys.get_coroutine_origin_tracking_depth` function returns +the current coroutine origin tracking depth, as set by +the new :func:`sys.set_coroutine_origin_tracking_depth`. :mod:`asyncio` +has been converted to use this new API instead of +the deprecated :func:`sys.set_coroutine_wrapper`. +(Contributed by Nathaniel J. Smith in :issue:`32591`.) -Added :class:`tkinter.ttk.Spinbox`. -(Contributed by Alan Moore in :issue:`32585`.) time ---- -The :pep:`564` added six new functions with nanosecond resolution: +:pep:`564` adds six new functions with nanosecond resolution to the +:mod:`time` module: * :func:`time.clock_gettime_ns` * :func:`time.clock_settime_ns` @@ -757,21 +1318,58 @@ The :pep:`564` added six new functions with nanosecond resolution: * :func:`time.process_time_ns` * :func:`time.time_ns` -Add new clock identifiers: +New clock identifiers have been added: * :data:`time.CLOCK_BOOTTIME` (Linux): Identical to :data:`time.CLOCK_MONOTONIC`, except it also includes any time that the system is suspended. * :data:`time.CLOCK_PROF` (FreeBSD, NetBSD and OpenBSD): High-resolution - per-process timer from the CPU. + per-process CPU timer. * :data:`time.CLOCK_UPTIME` (FreeBSD, OpenBSD): Time whose absolute value is the time the system has been running and not suspended, providing accurate - uptime measurement, both absolute and interval. + uptime measurement. -Added functions :func:`time.thread_time` and :func:`time.thread_time_ns` -to get per-thread CPU time measurements. +The new :func:`time.thread_time` and :func:`time.thread_time_ns` functions +can be used to get per-thread CPU time measurements. (Contributed by Antoine Pitrou in :issue:`32025`.) +The new :func:`time.pthread_getcpuclockid` function returns the clock ID +of the thread-specific CPU-time clock. + + +tkinter +------- + +The new :class:`tkinter.ttk.Spinbox` class is now available. +(Contributed by Alan Moore in :issue:`32585`.) + + +tracemalloc +----------- + +:class:`tracemalloc.Traceback` behaves more like regular tracebacks, +sorting the frames from oldest to most recent. +:meth:`Traceback.format() <tracemalloc.Traceback.format>` +now accepts negative *limit*, truncating the result to the +``abs(limit)`` oldest frames. To get the old behaviour, use +the new *most_recent_first* argument to ``Traceback.format()``. +(Contributed by Jesse Bakker in :issue:`32121`.) + + +types +----- + +The new :class:`~types.WrapperDescriptorType`, +:class:`~types.MethodWrapperType`, :class:`~types.MethodDescriptorType`, +and :class:`~types.ClassMethodDescriptorType` classes are now available. +(Contributed by Manuel Krebber and Guido van Rossum in :issue:`29377`, +and Serhiy Storchaka in :issue:`32265`.) + +The new :func:`types.resolve_bases` function resolves MRO entries +dynamically as specified by :pep:`560`. +(Contributed by Ivan Levkivskyi in :issue:`32717`.) + + unicodedata ----------- @@ -779,15 +1377,18 @@ The internal :mod:`unicodedata` database has been upgraded to use `Unicode 10 <http://www.unicode.org/versions/Unicode10.0.0/>`_. (Contributed by Benjamin Peterson.) + unittest -------- -Added new command-line option ``-k`` to filter tests to run with a substring or -Unix shell-like pattern. For example, ``python -m unittest -k foo`` runs the -tests ``foo_tests.SomeTest.test_something``, ``bar_tests.SomeTest.test_foo``, +The new ``-k`` command-line option allows filtering tests by a name +substring or a Unix shell-like pattern. +For example, ``python -m unittest -k foo`` runs +``foo_tests.SomeTest.test_something``, ``bar_tests.SomeTest.test_foo``, but not ``bar_tests.FooTest.test_something``. (Contributed by Jonas Haag in :issue:`32071`.) + unittest.mock ------------- @@ -795,47 +1396,75 @@ The :const:`~unittest.mock.sentinel` attributes now preserve their identity when they are :mod:`copied <copy>` or :mod:`pickled <pickle>`. (Contributed by Serhiy Storchaka in :issue:`20804`.) -New function :func:`~unittest.mock.seal` will disable the creation of mock -children by preventing to get or set any new attribute on the sealed mock. -The sealing process is performed recursively. (Contributed by Mario Corchero -in :issue:`30541`.) +The new :func:`~unittest.mock.seal` function allows sealing +:class:`~unittest.mock.Mock` instances, which will disallow further creation +of attribute mocks. The seal is applied recursively to all attributes that +are themselves mocks. +(Contributed by Mario Corchero in :issue:`30541`.) + urllib.parse ------------ :func:`urllib.parse.quote` has been updated from :rfc:`2396` to :rfc:`3986`, -adding ``~`` to the set of characters that is never quoted by default. +adding ``~`` to the set of characters that are never quoted by default. (Contributed by Christian Theune and Ratnadeep Debnath in :issue:`16285`.) + uu -- -Function :func:`~uu.encode` now accepts an optional *backtick* +The :func:`uu.encode` function now accepts an optional *backtick* keyword argument. When it's true, zeros are represented by ``'`'`` instead of spaces. (Contributed by Xiang Zhang in :issue:`30103`.) + +uuid +---- + +The new :attr:`UUID.is_safe <uuid.UUID.is_safe>` attribute relays information +from the platform about whether generated UUIDs are generated with a +multiprocessing-safe method. +(Contributed by Barry Warsaw in :issue:`22807`.) + +:func:`uuid.getnode` now prefers universally administered +MAC addresses over locally administered MAC addresses. +This makes a better guarantee for global uniqueness of UUIDs returned +from :func:`uuid.uuid1`. If only locally administered MAC addresses are +available, the first such one found is returned. +(Contributed by Barry Warsaw in :issue:`32107`.) + + warnings -------- The initialization of the default warnings filters has changed as follows: * warnings enabled via command line options (including those for :option:`-b` - and the new CPython-specific ``-X dev`` option) are always passed to the - warnings machinery via the ``sys.warnoptions`` attribute. + and the new CPython-specific :option:`-X` ``dev`` option) are always passed + to the warnings machinery via the :data:`sys.warnoptions` attribute. + * warnings filters enabled via the command line or the environment now have the - following precedence order: + following order of precedence: * the ``BytesWarning`` filter for :option:`-b` (or ``-bb``) - * any filters specified with :option:`-W` - * any filters specified with :envvar:`PYTHONWARNINGS` + * any filters specified with the :option:`-W` option + * any filters specified with the :envvar:`PYTHONWARNINGS` environment + variable * any other CPython specific filters (e.g. the ``default`` filter added for the new ``-X dev`` mode) * any implicit filters defined directly by the warnings machinery + * in CPython debug builds, all warnings are now displayed by default (the implicit filter list is empty) (Contributed by Nick Coghlan and Victor Stinner in :issue:`20361`, -:issue:`32043`, and :issue:`32230`) +:issue:`32043`, and :issue:`32230`.) + +Deprecation warnings are once again shown by default in single-file scripts and +at the interactive prompt. See :ref:`whatsnew37-pep565` for details. +(Contributed by Nick Coghlan in :issue:`31975`.) + xml.etree --------- @@ -845,13 +1474,15 @@ methods can now compare text of the current node with ``[. = "text"]``, not only text in children. Predicates also allow adding spaces for better readability. (Contributed by Stefan Behnel in :issue:`31648`.) + xmlrpc.server ------------- -:meth:`register_function` of :class:`~xmlrpc.server.SimpleXMLRPCDispatcher` and -its subclasses can be used as a decorator. (Contributed by Xiang Zhang in +:meth:`SimpleXMLRPCDispatcher.register_function <xmlrpc.server.SimpleXMLRPCDispatcher>` +can now be used as a decorator. (Contributed by Xiang Zhang in :issue:`7769`.) + zipapp ------ @@ -865,144 +1496,267 @@ argument to generate a compressed archive. A command line option (Contributed by Zhiming Wang in :issue:`31638`.) +zipfile +------- + +:class:`~zipfile.ZipFile` now accepts the new *compresslevel* parameter to +control the compression level. +(Contributed by Bo Bayles in :issue:`21417`.) + +Subdirectories in archives created by ``ZipFile`` are now stored in +alphabetical order. +(Contributed by Bernhard M. Wiedemann in :issue:`30693`.) + + +C API Changes +============= + +A new API for thread-local storage has been implemented. See +:ref:`whatsnew37-pep539` for an overview and +:ref:`thread-specific-storage-api` for a complete reference. +(Contributed by Masayuki Yamamoto in :issue:`25658`.) + +The new :c:func:`PyImport_GetModule` function returns the previously +imported module with the given name. +(Contributed by Eric Snow in :issue:`28411`.) + +The new :c:macro:`Py_RETURN_RICHCOMPARE` macro eases writing rich +comparison functions. +(Contributed by Petr Victorin in :issue:`23699`.) + +The new :c:macro:`Py_UNREACHABLE` macro can be used to mark unreachable +code paths. +(Contributed by Barry Warsaw in :issue:`31338`.) + +The :mod:`tracemalloc` now exposes a C API through the new +:c:func:`PyTraceMalloc_Track` and :c:func:`PyTraceMalloc_Untrack` +functions. +(Contributed by Victor Stinner in :issue:`30054`.) + +The new :c:func:`import__find__load__start` and +:c:func:`import__find__load__done` static markers can be used to trace +module imports. +(Contributed by Christian Heimes in :issue:`31574`.) + +The fields :c:member:`name` and :c:member:`doc` of structures +:c:type:`PyMemberDef`, :c:type:`PyGetSetDef`, +:c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`, +and :c:type:`wrapperbase` are now of type ``const char *`` rather of +``char *``. (Contributed by Serhiy Storchaka in :issue:`28761`.) + +The result of :c:func:`PyUnicode_AsUTF8AndSize` and :c:func:`PyUnicode_AsUTF8` +is now of type ``const char *`` rather of ``char *``. (Contributed by Serhiy +Storchaka in :issue:`28769`.) + +The result of :c:func:`PyMapping_Keys`, :c:func:`PyMapping_Values` and +:c:func:`PyMapping_Items` is now always a list, rather than a list or a +tuple. (Contributed by Oren Milman in :issue:`28280`.) + +Added functions :c:func:`PySlice_Unpack` and :c:func:`PySlice_AdjustIndices`. +(Contributed by Serhiy Storchaka in :issue:`27867`.) + +:c:func:`PyOS_AfterFork` is deprecated in favour of the new functions +:c:func:`PyOS_BeforeFork`, :c:func:`PyOS_AfterFork_Parent` and +:c:func:`PyOS_AfterFork_Child`. (Contributed by Antoine Pitrou in +:issue:`16500`.) + +The ``PyExc_RecursionErrorInst`` singleton that was part of the public API +has been removed as its members being never cleared may cause a segfault +during finalization of the interpreter. Contributed by Xavier de Gaye in +:issue:`22898` and :issue:`30697`. + +Added C API support for timezones with timezone constructors +:c:func:`PyTimeZone_FromOffset` and :c:func:`PyTimeZone_FromOffsetAndName`, +and access to the UTC singleton with :c:data:`PyDateTime_TimeZone_UTC`. +Contributed by Paul Ganssle in :issue:`10381`. + +The type of results of :c:func:`PyThread_start_new_thread` and +:c:func:`PyThread_get_thread_ident`, and the *id* parameter of +:c:func:`PyThreadState_SetAsyncExc` changed from :c:type:`long` to +:c:type:`unsigned long`. +(Contributed by Serhiy Storchaka in :issue:`6532`.) + +:c:func:`PyUnicode_AsWideCharString` now raises a :exc:`ValueError` if the +second argument is *NULL* and the :c:type:`wchar_t*` string contains null +characters. (Contributed by Serhiy Storchaka in :issue:`30708`.) + +Changes to the startup sequence and the management of dynamic memory +allocators mean that the long documented requirement to call +:c:func:`Py_Initialize` before calling most C API functions is now +relied on more heavily, and failing to abide by it may lead to segfaults in +embedding applications. See the :ref:`porting-to-python-37` section in this +document and the :ref:`pre-init-safe` section in the C API documentation +for more details. + +The new :c:func:`PyInterpreterState_GetID` returns the unique ID for a +given interpreter. +(Contributed by Eric Snow in :issue:`29102`.) + +:c:func:`Py_DecodeLocale`, :c:func:`Py_EncodeLocale` now use the UTF-8 +encoding when the :ref:`UTF-8 mode <whatsnew37-pep540>` is enabled. +(Contributed by Victor Stinner in :issue:`29240`.) + +:c:func:`PyUnicode_DecodeLocaleAndSize` and :c:func:`PyUnicode_EncodeLocale` +now use the current locale encoding for ``surrogateescape`` error handler. +(Contributed by Victor Stinner in :issue:`29240`.) + +The *start* and *end* parameters of :c:func:`PyUnicode_FindChar` are +now adjusted to behave like string slices. +(Contributed by Xiang Zhang in :issue:`28822`.) + + +Build Changes +============= + +Support for building ``--without-threads`` has been removed. The +:mod:`threading` module is now always available. +(Contributed by Antoine Pitrou in :issue:`31370`.). + +A full copy of libffi is no longer bundled for use when building the +:mod:`_ctypes <ctypes>` module on non-OSX UNIX platforms. An installed copy +of libffi is now required when building ``_ctypes`` on such platforms. +(Contributed by Zachary Ware in :issue:`27979`.) + +The Windows build process no longer depends on Subversion to pull in external +sources, a Python script is used to download zipfiles from GitHub instead. +If Python 3.6 is not found on the system (via ``py -3.6``), NuGet is used to +download a copy of 32-bit Python for this purpose. (Contributed by Zachary +Ware in :issue:`30450`.) + +The :mod:`ssl` module requires OpenSSL 1.0.2 or 1.1 compatible libssl. +OpenSSL 1.0.1 has reached end of lifetime on 2016-12-31 and is no longer +supported. LibreSSL is temporarily not supported as well. LibreSSL releases +up to version 2.6.4 are missing required OpenSSL 1.0.2 APIs. + + +.. _whatsnew37-perf: + Optimizations ============= -* Added two new opcodes: :opcode:`LOAD_METHOD` and :opcode:`CALL_METHOD` to avoid - instantiation of bound method objects for method calls, which results - in method calls being faster up to 20%. (Contributed by Yury Selivanov and - INADA Naoki in :issue:`26110`.) - -* Searching some unlucky Unicode characters (like Ukrainian capital "Є") - in a string was up to 25 times slower than searching other characters. - Now it is slower only by 3 times in the worst case. - (Contributed by Serhiy Storchaka in :issue:`24821`.) - -* Fast implementation from standard C library is now used for functions - :func:`~math.erf` and :func:`~math.erfc` in the :mod:`math` module. - (Contributed by Serhiy Storchaka in :issue:`26121`.) - -* The :func:`os.fwalk` function has been sped up by 2 times. This was done - using the :func:`os.scandir` function. - (Contributed by Serhiy Storchaka in :issue:`25996`.) - -* The :func:`shutil.rmtree` function has been sped up to 20--40%. - This was done using the :func:`os.scandir` function. - (Contributed by Serhiy Storchaka in :issue:`28564`.) - -* Optimized case-insensitive matching and searching of :mod:`regular - expressions <re>`. Searching some patterns can now be up to 20 times faster. - (Contributed by Serhiy Storchaka in :issue:`30285`.) - -* :func:`re.compile` now converts ``flags`` parameter to int object if - it is ``RegexFlag``. It is now as fast as Python 3.5, and faster than - Python 3.6 by about 10% depending on the pattern. - (Contributed by INADA Naoki in :issue:`31671`.) - -* :meth:`~selectors.BaseSelector.modify` methods of classes - :class:`selectors.EpollSelector`, :class:`selectors.PollSelector` - and :class:`selectors.DevpollSelector` may be around 10% faster under - heavy loads. (Contributed by Giampaolo Rodola' in :issue:`30014`) - -* Constant folding is moved from peephole optimizer to new AST optimizer. - (Contributed by Eugene Toder and INADA Naoki in :issue:`29469`) - -* Most functions and methods in :mod:`abc` have been rewritten in C. - This makes creation of abstract base classes, and calling :func:`isinstance` - and :func:`issubclass` on them 1.5x faster. This also reduces Python - start-up time by up to 10%. (Contributed by Ivan Levkivskyi and INADA Naoki - in :issue:`31333`) - -* Significant speed improvements to alternate constructors for - :class:`datetime.date` and :class:`datetime.datetime` by using fast-path - constructors when not constructing subclasses. (Contributed by Paul Ganssle - in :issue:`32403`) - - -Build and C API Changes -======================= - -* :mod:`py_compile` and :mod:`compileall` now support the - :envvar:`SOURCE_DATE_EPOCH` environment variable by unconditionally - building ``.pyc`` files for hash verification instead of potentially - timestamp-based ``.pyc`` files. See the notes for the `py_compile`_ - improvement notes for more details. - -* A full copy of libffi is no longer bundled for use when building the - :mod:`_ctypes <ctypes>` module on non-OSX UNIX platforms. An installed copy - of libffi is now required when building ``_ctypes`` on such platforms. - (Contributed by Zachary Ware in :issue:`27979`.) - -* The fields :c:member:`name` and :c:member:`doc` of structures - :c:type:`PyMemberDef`, :c:type:`PyGetSetDef`, - :c:type:`PyStructSequence_Field`, :c:type:`PyStructSequence_Desc`, - and :c:type:`wrapperbase` are now of type ``const char *`` rather of - ``char *``. (Contributed by Serhiy Storchaka in :issue:`28761`.) - -* The result of :c:func:`PyUnicode_AsUTF8AndSize` and :c:func:`PyUnicode_AsUTF8` - is now of type ``const char *`` rather of ``char *``. (Contributed by Serhiy - Storchaka in :issue:`28769`.) - -* The result of :c:func:`PyMapping_Keys`, :c:func:`PyMapping_Values` and - :c:func:`PyMapping_Items` is now always a list, rather than a list or a - tuple. (Contributed by Oren Milman in :issue:`28280`.) - -* Added functions :c:func:`PySlice_Unpack` and :c:func:`PySlice_AdjustIndices`. - (Contributed by Serhiy Storchaka in :issue:`27867`.) - -* :c:func:`PyOS_AfterFork` is deprecated in favour of the new functions - :c:func:`PyOS_BeforeFork`, :c:func:`PyOS_AfterFork_Parent` and - :c:func:`PyOS_AfterFork_Child`. (Contributed by Antoine Pitrou in - :issue:`16500`.) - -* The Windows build process no longer depends on Subversion to pull in external - sources, a Python script is used to download zipfiles from GitHub instead. - If Python 3.6 is not found on the system (via ``py -3.6``), NuGet is used to - download a copy of 32-bit Python for this purpose. (Contributed by Zachary - Ware in :issue:`30450`.) - -* The ``PyExc_RecursionErrorInst`` singleton that was part of the public API - has been removed as its members being never cleared may cause a segfault - during finalization of the interpreter. Contributed by Xavier de Gaye in - :issue:`22898` and :issue:`30697`. - -* Support for building ``--without-threads`` is removed. - (Contributed by Antoine Pitrou in :issue:`31370`.). - -* Added C API support for timezones with timezone constructors - :c:func:`PyTimeZone_FromOffset` and :c:func:`PyTimeZone_FromOffsetAndName`, - and access to the UTC singleton with :c:data:`PyDateTime_TimeZone_UTC`. - Contributed by Paul Ganssle in :issue:`10381`. - -- The type of results of :c:func:`PyThread_start_new_thread` and - :c:func:`PyThread_get_thread_ident`, and the *id* parameter of - :c:func:`PyThreadState_SetAsyncExc` changed from :c:type:`long` to - :c:type:`unsigned long`. - (Contributed by Serhiy Storchaka in :issue:`6532`.) - -- :c:func:`PyUnicode_AsWideCharString` now raises a :exc:`ValueError` if the - second argument is *NULL* and the :c:type:`wchar_t*` string contains null - characters. (Contributed by Serhiy Storchaka in :issue:`30708`.) - -- Changes to the startup sequence and the management of dynamic memory - allocators mean that the long documented requirement to call - :c:func:`Py_Initialize` before calling most C API functions is now - relied on more heavily, and failing to abide by it may lead to segfaults in - embedding applications. See the :ref:`porting-to-python-37` section in this - document and the :ref:`pre-init-safe` section in the C API documentation - for more details. +The overhead of calling many methods of various standard library classes +implemented in C has been significantly reduced by porting more code +to use the ``METH_FASTCALL`` convention. +(Contributed by Victor Stinner in :issue:`29300`, :issue:`29507`, +:issue:`29452`, and :issue:`29286`.) + +Various optimizations have reduced Python startup time by 10% on Linux and +up to 30% on macOS. +(Contributed by Victor Stinner, INADA Naoki in :issue:`29585`, and +Ivan Levkivskyi in :issue:`31333`.) + +Method calls are now up to 20% faster due to the bytecode changes which +avoid creating bound method instances. +(Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.) + +.. _whatsnew37-asyncio-perf: + +The :mod:`asyncio` module received a number of notable optimizations for +commonly used functions: + +* The :func:`asyncio.get_event_loop` function has been reimplemented in C to + make it up to 15 times faster. + (Contributed by Yury Selivanov in :issue:`32296`.) + +* :class:`asyncio.Future` callback management has been optimized. + (Contributed by Yury Selivanov in :issue:`32348`.) + +* :func:`asyncio.gather` is now up to 15% faster. + (Contributed by Yury Selivanov in :issue:`32355`.) + +* :func:`asyncio.sleep` is now up to 2 times faster when the *delay* + argument is zero or negative. + (Contributed by Andrew Svetlov in :issue:`32351`.) + +* The performance overhead of asyncio debug mode has been reduced. + (Contributed by Antoine Pitrou in :issue:`31970`.) + +As a result of :ref:`PEP 560 work <whatsnew37-pep560>`, the import time +of :mod:`typing` has been reduced by a factor of 7, and many typing operations +are now faster. +(Contributed by Ivan Levkivskyi in :issue:`32226`.) + +:func:`sorted` and :meth:`list.sort` have been optimized for common cases +to be up to 40-75% faster. +(Contributed by Elliot Gorokhovsky in :issue:`28685`.) + +:meth:`dict.copy` is now up to 5.5 times faster. +(Contributed by Yury Selivanov in :issue:`31179`.) + +:func:`hasattr` and :func:`getattr` are now about 4 times faster when +*name* is not found and *obj* does not override :meth:`object.__getattr__` +or :meth:`object.__getattribute__`. +(Contributed by INADA Naoki in :issue:`32544`.) + +Searching for certain Unicode characters (like Ukrainian capital "Є") +in a string was up to 25 times slower than searching for other characters. +It is now only 3 times slower in the worst case. +(Contributed by Serhiy Storchaka in :issue:`24821`.) + +The :func:`collections.namedtuple` factory has been reimplemented to +make the creation of named tuples 4 to 6 times faster. +(Contributed by Jelle Zijlstra with further improvements by INADA Naoki, +Serhiy Storchaka, and Raymond Hettinger in :issue:`28638`.) + +:meth:`date.fromordinal` and :meth:`date.fromtimestamp` are now up to +30% faster in the common case. +(Contributed by Paul Ganssle in :issue:`32403`.) + +The :func:`os.fwalk` function is now up to 2 times faster thanks to +the use of :func:`os.scandir`. +(Contributed by Serhiy Storchaka in :issue:`25996`.) + +The speed of the :func:`shutil.rmtree` function has been improved by +20--40% thanks to the use of the :func:`os.scandir` function. +(Contributed by Serhiy Storchaka in :issue:`28564`.) + +Optimized case-insensitive matching and searching of :mod:`regular +expressions <re>`. Searching some patterns can now be up to 20 times faster. +(Contributed by Serhiy Storchaka in :issue:`30285`.) + +:func:`re.compile` now converts ``flags`` parameter to int object if +it is ``RegexFlag``. It is now as fast as Python 3.5, and faster than +Python 3.6 by about 10% depending on the pattern. +(Contributed by INADA Naoki in :issue:`31671`.) + +The :meth:`~selectors.BaseSelector.modify` methods of classes +:class:`selectors.EpollSelector`, :class:`selectors.PollSelector` +and :class:`selectors.DevpollSelector` may be around 10% faster under +heavy loads. (Contributed by Giampaolo Rodola' in :issue:`30014`) + +Constant folding has been moved from the peephole optimizer to the new AST +optimizer, which is able perform optimizations more consistently. +(Contributed by Eugene Toder and INADA Naoki in :issue:`29469` and +:issue:`11549`.) + +Most functions and methods in :mod:`abc` have been rewritten in C. +This makes creation of abstract base classes, and calling :func:`isinstance` +and :func:`issubclass` on them 1.5x faster. This also reduces Python +start-up time by up to 10%. (Contributed by Ivan Levkivskyi and INADA Naoki +in :issue:`31333`) + +Significant speed improvements to alternate constructors for +:class:`datetime.date` and :class:`datetime.datetime` by using fast-path +constructors when not constructing subclasses. (Contributed by Paul Ganssle +in :issue:`32403`) + +The speed of comparison of :class:`array.array` instances has been +improved considerably in certain cases. It is now from 10x to 70x faster +when comparing arrays holding values of the same integer type. +(Contributed by Adrian Wielgosik in :issue:`24700`.) + +The :func:`math.erf` and :func:`math.erfc` functions now use the (faster) +C library implementation on most platforms. +(Contributed by Serhiy Storchaka in :issue:`26121`.) Other CPython Implementation Changes ==================================== -* Trace hooks may now opt out of receiving ``line`` events from the interpreter - by setting the new ``f_trace_lines`` attribute to :const:`False` on the frame - being traced. (Contributed by Nick Coghlan in :issue:`31344`.) - -* Trace hooks may now opt in to receiving ``opcode`` events from the interpreter - by setting the new ``f_trace_opcodes`` attribute to :const:`True` on the frame - being traced. (Contributed by Nick Coghlan in :issue:`31344`.) +* Trace hooks may now opt out of receiving the ``line`` and opt into receiving + the ``opcode`` events from the interpreter by setting the corresponding new + ``f_trace_lines`` and ``f_trace_opcodes`` attributes on the + frame being traced. (Contributed by Nick Coghlan in :issue:`31344`.) * Fixed some consistency problems with namespace package module attributes. Namespace module objects now have an ``__file__`` that is set to ``None`` @@ -1012,90 +1766,213 @@ Other CPython Implementation Changes ``__loader__`` (previously, the former was set to ``None``). See :issue:`32303`. +* The :func:`locals` dictionary now displays in the lexical order that + variables were defined. Previously, the order was undefined. + (Contributed by Raymond Hettinger in :issue:`32690`.) -Deprecated -========== +* The :mod:`distutils` ``upload`` command no longer tries to change CR + end-of-line characters to CRLF. This fixes a corruption issue with sdists + that ended with a byte equivalent to CR. + (Contributed by Bo Bayles in :issue:`32304`.) -* In Python 3.8, the abstract base classes in :mod:`collections.abc` will no - longer be exposed in the regular :mod:`collections` module. This will help - create a clearer distinction between the concrete classes and the abstract - base classes. -* Yield expressions (both ``yield`` and ``yield from`` clauses) are now deprecated - in comprehensions and generator expressions (aside from the iterable expression - in the leftmost :keyword:`for` clause). This ensures that comprehensions - always immediately return a container of the appropriate type (rather than - potentially returning a :term:`generator iterator` object), while generator - expressions won't attempt to interleave their implicit output with the output - from any explicit yield expressions. +Deprecated Python Behavior +========================== - In Python 3.7, such expressions emit :exc:`DeprecationWarning` when compiled, - in Python 3.8+ they will emit :exc:`SyntaxError`. (Contributed by Serhiy - Storchaka in :issue:`10544`.) +Yield expressions (both ``yield`` and ``yield from`` clauses) are now deprecated +in comprehensions and generator expressions (aside from the iterable expression +in the leftmost :keyword:`for` clause). This ensures that comprehensions +always immediately return a container of the appropriate type (rather than +potentially returning a :term:`generator iterator` object), while generator +expressions won't attempt to interleave their implicit output with the output +from any explicit yield expressions. In Python 3.7, such expressions emit +:exc:`DeprecationWarning` when compiled, in Python 3.8 this will be a +:exc:`SyntaxError`. +(Contributed by Serhiy Storchaka in :issue:`10544`.) -- Function :c:func:`PySlice_GetIndicesEx` is deprecated and replaced with - a macro if ``Py_LIMITED_API`` is not set or set to the value between - ``0x03050400`` and ``0x03060000`` (not including) or ``0x03060100`` or - higher. (Contributed by Serhiy Storchaka in :issue:`27867`.) +Returning a subclass of :class:`complex` from :meth:`object.__complex__` is +deprecated and will be an error in future Python versions. This makes +``__complex__()`` consistent with :meth:`object.__int__` and +:meth:`object.__float__`. +(Contributed by Serhiy Storchaka in :issue:`28894`.) -- Deprecated :meth:`format` from :mod:`locale`, use the :meth:`format_string` - instead. (Contributed by Garvit in :issue:`10379`.) -- Methods - :meth:`MetaPathFinder.find_module() <importlib.abc.MetaPathFinder.find_module>` - (replaced by - :meth:`MetaPathFinder.find_spec() <importlib.abc.MetaPathFinder.find_spec>`) - and - :meth:`PathEntryFinder.find_loader() <importlib.abc.PathEntryFinder.find_loader>` - (replaced by - :meth:`PathEntryFinder.find_spec() <importlib.abc.PathEntryFinder.find_spec>`) - both deprecated in Python 3.4 now emit :exc:`DeprecationWarning`. - (Contributed by Matthias Bussonnier in :issue:`29576`) -- Using non-integer value for selecting a plural form in :mod:`gettext` is - now deprecated. It never correctly worked. (Contributed by Serhiy Storchaka - in :issue:`28692`.) +Deprecated Python modules, functions and methods +================================================ -- The :mod:`macpath` is now deprecated and will be removed in Python 3.8. +aifc +---- -- The :class:`importlib.abc.ResourceLoader` ABC has been deprecated in - favour of :class:`importlib.abc.ResourceReader`. +:func:`aifc.openfp` has been deprecated and will be removed in Python 3.9. +Use :func:`aifc.open` instead. +(Contributed by Brian Curtin in :issue:`31985`.) -- Deprecated :func:`sys.set_coroutine_wrapper` and - :func:`sys.get_coroutine_wrapper`. -- :func:`ssl.wrap_socket` is deprecated. Use - :meth:`ssl.SSLContext.wrap_socket` instead. - (Contributed by Christian Heimes in :issue:`28124`.) +.. _whatsnew37-asyncio-deprecated: +asyncio +------- + +Support for directly ``await``-ing instances of :class:`asyncio.Lock` and +other asyncio synchronization primitives has been deprecated. An +asynchronous context manager must be used in order to acquire and release +the synchronization resource. See :ref:`async-with-locks` for more +information. +(Contributed by Andrew Svetlov in :issue:`32253`.) + +The :meth:`asyncio.Task.current_task` and :meth:`asyncio.Task.all_tasks` +methods have been deprecated. +(Contributed by Andrew Svetlov in :issue:`32250`.) + + +collections +----------- + +In Python 3.8, the abstract base classes in :mod:`collections.abc` will no +longer be exposed in the regular :mod:`collections` module. This will help +create a clearer distinction between the concrete classes and the abstract +base classes. +(Contributed by Serhiy Storchaka in :issue:`25988`.) -Windows Only ------------- -- The python launcher, (py.exe), can accept 32 & 64 bit specifiers **without** - having to specify a minor version as well. So ``py -3-32`` and ``py -3-64`` - become valid as well as ``py -3.7-32``, also the -*m*-64 and -*m.n*-64 forms - are now accepted to force 64 bit python even if 32 bit would have otherwise - been used. If the specified version is not available py.exe will error exit. - (Contributed by Steve Barnes in :issue:`30291`.) +dbm +--- + +:mod:`dbm.dumb` now supports reading read-only files and no longer writes the +index file when it is not changed. A deprecation warning is now emitted +if the index file is missing and recreated in the ``'r'`` and ``'w'`` +modes (this will be an error in future Python releases). +(Contributed by Serhiy Storchaka in :issue:`28847`.) + + +enum +---- + +In Python 3.8, attempting to check for non-Enum objects in :class:`Enum` +classes will raise a :exc:`TypeError` (e.g. ``1 in Color``); similarly, +attempting to check for non-Flag objects in a :class:`Flag` member will +raise :exc:`TypeError` (e.g. ``1 in Perm.RW``); currently, both operations +return :const:`False` instead. +(Contributed by Ethan Furman in :issue:`33217`.) + + +gettext +------- -- The launcher can be run as ``py -0`` to produce a list of the installed pythons, - *with default marked with an asterisk*. Running ``py -0p`` will include the paths. - If py is run with a version specifier that cannot be matched it will also print - the *short form* list of available specifiers. - (Contributed by Steve Barnes in :issue:`30362`.) +Using non-integer value for selecting a plural form in :mod:`gettext` is +now deprecated. It never correctly worked. (Contributed by Serhiy Storchaka +in :issue:`28692`.) -Removed -======= +importlib +--------- + +Methods +:meth:`MetaPathFinder.find_module() <importlib.abc.MetaPathFinder.find_module>` +(replaced by +:meth:`MetaPathFinder.find_spec() <importlib.abc.MetaPathFinder.find_spec>`) +and +:meth:`PathEntryFinder.find_loader() <importlib.abc.PathEntryFinder.find_loader>` +(replaced by +:meth:`PathEntryFinder.find_spec() <importlib.abc.PathEntryFinder.find_spec>`) +both deprecated in Python 3.4 now emit :exc:`DeprecationWarning`. +(Contributed by Matthias Bussonnier in :issue:`29576`) + +The :class:`importlib.abc.ResourceLoader` ABC has been deprecated in +favour of :class:`importlib.abc.ResourceReader`. + + +locale +------ + +:func:`locale.format` has been deprecated, use :meth:`locale.format_string` +instead. (Contributed by Garvit in :issue:`10379`.) + + +macpath +------- + +The :mod:`macpath` is now deprecated and will be removed in Python 3.8. +(Contributed by Chi Hsuan Yen in :issue:`9850`.) + + +threading +--------- + +:mod:`dummy_threading` and :mod:`_dummy_thread` have been deprecated. It is +no longer possible to build Python with threading disabled. +Use :mod:`threading` instead. +(Contributed by Antoine Pitrou in :issue:`31370`.) + + +socket +------ + +The silent argument value trunctation in :func:`socket.htons` and +:func:`socket.ntohs` has been deprecated. In future versions of Python, +if the passed argument is larger than 16 bits, an exception will be raised. +(Contributed by Oren Milman in :issue:`28332`.) + + +ssl +--- + +:func:`ssl.wrap_socket` is deprecated. Use +:meth:`ssl.SSLContext.wrap_socket` instead. +(Contributed by Christian Heimes in :issue:`28124`.) + + +sunau +----- + +:func:`sunau.openfp` has been deprecated and will be removed in Python 3.9. +Use :func:`sunau.open` instead. +(Contributed by Brian Curtin in :issue:`31985`.) + + +sys +--- + +Deprecated :func:`sys.set_coroutine_wrapper` and +:func:`sys.get_coroutine_wrapper`. + +The undocumented ``sys.callstats()`` function has been deprecated and +will be removed in a future Python version. +(Contributed by Victor Stinner in :issue:`28799`.) + + +wave +---- + +:func:`wave.openfp` has been deprecated and will be removed in Python 3.9. +Use :func:`wave.open` instead. +(Contributed by Brian Curtin in :issue:`31985`.) + + +Deprecated functions and types of the C API +=========================================== + +Function :c:func:`PySlice_GetIndicesEx` is deprecated and replaced with +a macro if ``Py_LIMITED_API`` is not set or set to a value in the range +between ``0x03050400`` and ``0x03060000`` (not inclusive), or is ``0x03060100`` +or higher. (Contributed by Serhiy Storchaka in :issue:`27867`.) + +:c:func:`PyOS_AfterFork` has been deprecated. Use :c:func:`PyOS_BeforeFork`, +:c:func:`PyOS_AfterFork_Parent` or :c:func:`PyOS_AfterFork_Child()` instead. +(Contributed by Antoine Pitrou in :issue:`16500`.) + Platform Support Removals -------------------------- +========================= + +FreeBSD 9 and older are no longer officially supported. -* FreeBSD 9 and older are no longer supported. API and Feature Removals ------------------------- +======================== + +The following features and APIs have been removed from Python 3.7: * The ``os.stat_float_times()`` function has been removed. It was introduced in Python 2.3 for backward compatibility with Python 2.2, and was deprecated @@ -1129,15 +2006,51 @@ API and Feature Removals can use attribute access to access items of these dictionaries. * The ``asyncio.windows_utils.socketpair()`` function has been - removed: use directly :func:`socket.socketpair` which is available on all - platforms since Python 3.5 (before, it wasn't available on Windows). + removed. Use the :func:`socket.socketpair` function instead, + it is available on all platforms since Python 3.5. ``asyncio.windows_utils.socketpair`` was just an alias to ``socket.socketpair`` on Python 3.5 and newer. -* :mod:`asyncio`: The module doesn't export :mod:`selectors` and +* :mod:`asyncio` no longer exports the :mod:`selectors` and :mod:`_overlapped` modules as ``asyncio.selectors`` and ``asyncio._overlapped``. Replace ``from asyncio import selectors`` with - ``import selectors`` for example. + ``import selectors``. + +* Direct instantiation of :class:`ssl.SSLSocket` and :class:`ssl.SSLObject` + objects is now prohibited. The constructors were never documented, tested, + or designed as public constructors. Users were supposed to use + :func:`ssl.wrap_socket` or :class:`ssl.SSLContext`. + (Contributed by Christian Heimes in :issue:`32951`.) + +* The unused :mod:`distutils` ``install_misc`` command has been removed. + (Contributed by Eric N. Vander Weele in :issue:`29218`.) + + +Module Removals +=============== + +The ``fpectl`` module has been removed. It was never enabled by +default, never worked correctly on x86-64, and it changed the Python +ABI in ways that caused unexpected breakage of C extensions. +(Contributed by Nathaniel J. Smith in :issue:`29137`.) + + +Windows-only Changes +==================== + +The python launcher, (py.exe), can accept 32 & 64 bit specifiers **without** +having to specify a minor version as well. So ``py -3-32`` and ``py -3-64`` +become valid as well as ``py -3.7-32``, also the -*m*-64 and -*m.n*-64 forms +are now accepted to force 64 bit python even if 32 bit would have otherwise +been used. If the specified version is not available py.exe will error exit. +(Contributed by Steve Barnes in :issue:`30291`.) + +The launcher can be run as ``py -0`` to produce a list of the installed pythons, +*with default marked with an asterisk*. Running ``py -0p`` will include the paths. +If py is run with a version specifier that cannot be matched it will also print +the *short form* list of available specifiers. +(Contributed by Steve Barnes in :issue:`30362`.) + .. _porting-to-python-37: @@ -1148,15 +2061,22 @@ This section lists previously described changes and other bugfixes that may require changes to your code. -Changes in Python behavior +Changes in Python Behavior -------------------------- +* :keyword:`async` and :keyword:`await` names are now reserved keywords. + Code using these names as identifiers will now raise a :exc:`SyntaxError`. + (Contributed by Jelle Zijlstra in :issue:`30406`.) + * :pep:`479` is enabled for all code in Python 3.7, meaning that :exc:`StopIteration` exceptions raised directly or indirectly in coroutines and generators are transformed into :exc:`RuntimeError` exceptions. (Contributed by Yury Selivanov in :issue:`32670`.) +* :meth:`object.__aiter__` methods can no longer be declared as + asynchronous. (Contributed by Yury Selivanov in :issue:`31709`.) + * Due to an oversight, earlier Python versions erroneously accepted the following syntax:: @@ -1183,22 +2103,25 @@ Changes in Python behavior Changes in the Python API ------------------------- -* :meth:`socketserver.ThreadingMixIn.server_close` now waits until all - non-daemon threads complete. Use daemonic threads by setting - :data:`socketserver.ThreadingMixIn.daemon_threads` to ``True`` to not - wait until threads complete. - (Contributed by Victor Stinner in :issue:`31233`.) +* ``Module``, ``FunctionDef``, ``AsyncFunctionDef``, and + ``ClassDef`` AST nodes now have the new ``docstring`` attribute. + The first statement in their body is not considered as a docstring + anymore. ``co_firstlineno`` and ``co_lnotab`` of code object for class + and module are affected by this change. (Contributed by INADA Naoki and + Eugene Toder in :issue:`29463`.) -* :meth:`socketserver.ForkingMixIn.server_close` now waits until all - child processes complete. (Contributed by Victor Stinner in :issue:`31151`.) +* :meth:`~socketserver.BaseServer.server_close` in + :class:`socketserver.ThreadingMixIn` and :class:`socketserver.ForkingMixIn` + now waits until all non-daemon threads complete. + (Contributed by Victor Stinner in :issue:`31233` and :issue:`31151`.) -* The :func:`locale.localeconv` function now sets temporarily the ``LC_CTYPE`` - locale to the ``LC_NUMERIC`` locale in some cases. +* The :func:`locale.localeconv` function now temporarily sets the ``LC_CTYPE`` + locale to the value of ``LC_NUMERIC`` in some cases. (Contributed by Victor Stinner in :issue:`31900`.) -* :meth:`pkgutil.walk_packages` now raises :exc:`ValueError` if *path* is a string. - Previously an empty list was returned. (Contributed by Sanyam Khurana in - :issue:`24744`.) +* :meth:`pkgutil.walk_packages` now raises a :exc:`ValueError` if *path* is + a string. Previously an empty list was returned. + (Contributed by Sanyam Khurana in :issue:`24744`.) * A format string argument for :meth:`string.Formatter.format` is now :ref:`positional-only <positional-only_parameter>`. @@ -1213,13 +2136,6 @@ Changes in the Python API Use the :meth:`~http.cookies.Morsel.set` method for setting them. (Contributed by Serhiy Storchaka in :issue:`29192`.) -* ``Module``, ``FunctionDef``, ``AsyncFunctionDef``, and - ``ClassDef`` AST nodes now have a new ``docstring`` field. - The first statement in their body is not considered as a docstring - anymore. ``co_firstlineno`` and ``co_lnotab`` of code object for class - and module are affected by this change. (Contributed by INADA Naoki and - Eugene Toder in :issue:`29463`.) - * The *mode* argument of :func:`os.makedirs` no longer affects the file permission bits of newly-created intermediate-level directories. To set their file permission bits you can set the umask before invoking @@ -1229,34 +2145,35 @@ Changes in the Python API * The :attr:`struct.Struct.format` type is now :class:`str` instead of :class:`bytes`. (Contributed by Victor Stinner in :issue:`21071`.) -* :func:`~cgi.parse_multipart` returns now the same results as +* :func:`~cgi.parse_multipart` now accepts the *encoding* and *errors* + arguments and returns the same results as :class:`~FieldStorage`: for non-file fields, the value associated to a key is a list of strings, not bytes. (Contributed by Pierre Quentel in :issue:`29979`.) -* Due to internal changes in :mod:`socket` you won't be able to - :func:`socket.fromshare` a socket :func:`~socket.socket.share`-ed in older - Python versions. +* Due to internal changes in :mod:`socket`, calling :func:`socket.fromshare` + on a socket created by :func:`socket.share <socket.socket.share>` in older + Python versions is not supported. -* ``repr`` for :exc:`BaseException` has changed not to include trailing comma - in the output. Mind that most exceptions are affected by this change. +* ``repr`` for :exc:`BaseException` has changed to not include the trailing + comma. Most exceptions are affected by this change. (Contributed by Serhiy Storchaka in :issue:`30399`.) -* ``repr`` for :class:`datetime.timedelta` has changed to include keyword arguments - in the output. (Contributed by Utkarsh Upadhyay in :issue:`30302`.) +* ``repr`` for :class:`datetime.timedelta` has changed to include the keyword + arguments in the output. (Contributed by Utkarsh Upadhyay in :issue:`30302`.) * Because :func:`shutil.rmtree` is now implemented using the :func:`os.scandir` function, the user specified handler *onerror* is now called with the first argument ``os.scandir`` instead of ``os.listdir`` when listing the direcory is failed. -* Support of nested sets and set operations in regular expressions as in +* Support for nested sets and set operations in regular expressions as in `Unicode Technical Standard #18`_ might be added in the future. This would - change the syntax, so to facilitate this change a :exc:`FutureWarning` will - be raised in ambiguous cases for the time being. + change the syntax. To facilitate this future change a :exc:`FutureWarning` + will be raised in ambiguous cases for the time being. That include sets starting with a literal ``'['`` or containing literal character sequences ``'--'``, ``'&&'``, ``'~~'``, and ``'||'``. To - avoid a warning escape them with a backslash. + avoid a warning, escape them with a backslash. (Contributed by Serhiy Storchaka in :issue:`30349`.) .. _Unicode Technical Standard #18: https://unicode.org/reports/tr18/ @@ -1308,85 +2225,124 @@ Changes in the Python API previous behaviour, or use :attr:`STARTUPINFO.lpAttributeList <subprocess.STARTUPINFO.lpAttributeList>`. +* :meth:`importlib.machinery.PathFinder.invalidate_caches` -- which implicitly + affects :func:`importlib.invalidate_caches` -- now deletes entries + in :data:`sys.path_importer_cache` which are set to ``None``. + (Contributed by Brett Cannon in :issue:`33169`.) + +* In :mod:`asyncio`, + :meth:`loop.sock_recv() <asyncio.AbstractEventLoop.sock_recv>`, + :meth:`loop.sock_sendall() <asyncio.AbstractEventLoop.sock_sendall>`, + :meth:`loop.sock_accept() <asyncio.AbstractEventLoop.sock_accept>`, + :meth:`loop.getaddrinfo() <asyncio.AbstractEventLoop.getaddrinfo>`, + :meth:`loop.getnameinfo() <asyncio.AbstractEventLoop.getnameinfo>` + have been changed to be proper coroutine methods to match their + documentation. Previously, these methods returned :class:`asyncio.Future` + instances. + (Contributed by Yury Selivanov in :issue:`32327`.) + +* :attr:`asyncio.Server.sockets` now returns a copy of the internal list + of server sockets, instead of returning it directly. + (Contributed by Yury Selivanov in :issue:`32662`.) + +* :attr:`Struct.format <struct.Struct.format>` is now a :class:`str` instance + instead of a :class:`bytes` instance. + (Contributed by Victor Stinner in :issue:`21071`.) + +* :mod:`argparse` subparsers are now required by default. This matches the + behaviour in Python 2. To add an optional subparser, pass + ``required=False`` to + :meth:`ArgumentParser.add_subparsers() <argparse.ArgumentParser.add_subparsers>`. + (Contributed by Anthony Sottile in :issue:`26510`.) + +* :meth:`ast.literal_eval()` is now stricter. Addition and subtraction of + arbitrary numbers are no longer allowed. + (Contributed by Serhiy Storchaka in :issue:`31778`.) + +* :meth:`Calendar.itermonthdates <calendar.Calendar.itermonthdates>` + will now consistently raise an exception when a date falls outside of the + ``0001-01-01`` through ``9999-12-31`` range. To support applications that + cannot tolerate such exceptions, the new + :meth:`Calendar.itermonthdays3 <calendar.Calendar.itermonthdays3>` and + :meth:`Calendar.itermonthdays4 <calendar.Calendar.itermonthdays4>` can be used. + The new methods return tuples and are not restricted by the range supported by + :class:`datetime.date`. + (Contributed by Alexander Belopolsky in :issue:`28292`.) + +* :class:`collections.ChainMap` now preserves the order of the underlying + mappings. (Contributed by Raymond Hettinger in :issue:`32792`.) + +* The ``submit()`` method of :class:`concurrent.futures.ThreadPoolExecutor` + and :class:`concurrent.futures.ProcessPoolExecutor` now raises + a :exc:`RuntimeError` if called during interpreter shutdown. + (Contributed by Mark Nemec in :issue:`33097`.) + +* The :class:`configparser.ConfigParser` constructor now uses ``read_dict()`` + to process the default values, making its behavior consistent with the + rest of the parser. Non-string keys and values in the defaults + dictionary are now being implicitly converted to strings. + (Contributed by James Tocknell in :issue:`23835`.) + Changes in the C API -------------------- -* The function :c:func:`PySlice_GetIndicesEx` is considered not safe for - resizable sequences. If the slice indices are not instances of :class:`int`, - but objects that implement the :meth:`!__index__` method, the sequence can be - resized after passing its length to :c:func:`!PySlice_GetIndicesEx`. This - can lead to returning indices out of the length of the sequence. For - avoiding possible problems use new functions :c:func:`PySlice_Unpack` and - :c:func:`PySlice_AdjustIndices`. - (Contributed by Serhiy Storchaka in :issue:`27867`.) +The function :c:func:`PySlice_GetIndicesEx` is considered unsafe for +resizable sequences. If the slice indices are not instances of :class:`int`, +but objects that implement the :meth:`!__index__` method, the sequence can be +resized after passing its length to :c:func:`!PySlice_GetIndicesEx`. This +can lead to returning indices out of the length of the sequence. For +avoiding possible problems use new functions :c:func:`PySlice_Unpack` and +:c:func:`PySlice_AdjustIndices`. +(Contributed by Serhiy Storchaka in :issue:`27867`.) CPython bytecode changes ------------------------ -* Added two new opcodes: :opcode:`LOAD_METHOD` and :opcode:`CALL_METHOD`. - (Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.) - -* Removed the :opcode:`STORE_ANNOTATION` opcode. - (Contributed by Mark Shannon in :issue:`32550`.) +There are two new opcodes: :opcode:`LOAD_METHOD` and :opcode:`CALL_METHOD`. +(Contributed by Yury Selivanov and INADA Naoki in :issue:`26110`.) +The :opcode:`STORE_ANNOTATION` opcode has been removed. +(Contributed by Mark Shannon in :issue:`32550`.) -Other CPython implementation changes ------------------------------------- - -* In preparation for potential future changes to the public CPython runtime - initialization API (see :pep:`432` for an initial, but somewhat outdated, - draft), CPython's internal startup - and configuration management logic has been significantly refactored. While - these updates are intended to be entirely transparent to both embedding - applications and users of the regular CPython CLI, they're being mentioned - here as the refactoring changes the internal order of various operations - during interpreter startup, and hence may uncover previously latent defects, - either in embedding applications, or in CPython itself. - (Initially contributed by Nick Coghlan and Eric Snow as part of - :issue:`22257`, and further updated by Nick, Eric, and Victor Stinner in a - number of other issues). Some known details affected: - - * :c:func:`PySys_AddWarnOptionUnicode` is not currently usable by embedding - applications due to the requirement to create a Unicode object prior to - calling `Py_Initialize`. Use :c:func:`PySys_AddWarnOption` instead. - * warnings filters added by an embedding application with - :c:func:`PySys_AddWarnOption` should now more consistently take precedence - over the default filters set by the interpreter - -* Due to changes in the way the default warnings filters are configured, - setting :c:data:`Py_BytesWarningFlag` to a value greater than one is no longer - sufficient to both emit :exc:`BytesWarning` messages and have them converted - to exceptions. Instead, the flag must be set (to cause the warnings to be - emitted in the first place), and an explicit ``error::BytesWarning`` - warnings filter added to convert them to exceptions. - -* CPython' :mod:`ssl` module requires OpenSSL 1.0.2 or 1.1 compatible libssl. - OpenSSL 1.0.1 has reached end of lifetime on 2016-12-31 and is no longer - supported. LibreSSL is temporarily not supported as well. LibreSSL releases - up to version 2.6.4 are missing required OpenSSL 1.0.2 APIs. - - -Documentation -============= -.. _whatsnew37-pep545: - -PEP 545: Python Documentation Translations ------------------------------------------- - -:pep:`545` describes the process to translate Python documentation, -and three translations have been added: +Windows-only Changes +-------------------- -- Japanese: https://docs.python.org/ja/ and associated GitHub - repository: https://github.com/python/python-docs-ja +The file used to override :data:`sys.path` is now called +``<python-executable>._pth`` instead of ``'sys.path'``. +See :ref:`finding_modules` for more information. +(Contributed by Steve Dower in :issue:`28137`.) -- French: https://docs.python.org/fr/ and associated GitHub - repository: https://github.com/python/python-docs-fr -- Korean: https://docs.python.org/ko/ and associated GitHub - repository: https://github.com/python/python-docs-ko +Other CPython implementation changes +------------------------------------ -(Contributed by Julien Palard, Inada Naoki, and Victor Stinner in -:issue:`26546`.) +In preparation for potential future changes to the public CPython runtime +initialization API (see :pep:`432` for an initial, but somewhat outdated, +draft), CPython's internal startup +and configuration management logic has been significantly refactored. While +these updates are intended to be entirely transparent to both embedding +applications and users of the regular CPython CLI, they're being mentioned +here as the refactoring changes the internal order of various operations +during interpreter startup, and hence may uncover previously latent defects, +either in embedding applications, or in CPython itself. +(Initially contributed by Nick Coghlan and Eric Snow as part of +:issue:`22257`, and further updated by Nick, Eric, and Victor Stinner in a +number of other issues). Some known details affected: + +* :c:func:`PySys_AddWarnOptionUnicode` is not currently usable by embedding + applications due to the requirement to create a Unicode object prior to + calling `Py_Initialize`. Use :c:func:`PySys_AddWarnOption` instead. + +* warnings filters added by an embedding application with + :c:func:`PySys_AddWarnOption` should now more consistently take precedence + over the default filters set by the interpreter + +Due to changes in the way the default warnings filters are configured, +setting :c:data:`Py_BytesWarningFlag` to a value greater than one is no longer +sufficient to both emit :exc:`BytesWarning` messages and have them converted +to exceptions. Instead, the flag must be set (to cause the warnings to be +emitted in the first place), and an explicit ``error::BytesWarning`` +warnings filter added to convert them to exceptions. |