diff options
Diffstat (limited to 'Doc/whatsnew')
| -rw-r--r-- | Doc/whatsnew/2.1.rst | 2 | ||||
| -rw-r--r-- | Doc/whatsnew/3.2.rst | 4 | ||||
| -rw-r--r-- | Doc/whatsnew/3.3.rst | 2 | ||||
| -rw-r--r-- | Doc/whatsnew/3.4.rst | 10 | ||||
| -rw-r--r-- | Doc/whatsnew/3.5.rst | 6 | ||||
| -rw-r--r-- | Doc/whatsnew/3.6.rst | 1479 | ||||
| -rw-r--r-- | Doc/whatsnew/index.rst | 1 |
7 files changed, 1492 insertions, 12 deletions
diff --git a/Doc/whatsnew/2.1.rst b/Doc/whatsnew/2.1.rst index be72bfe..d97ab67 100644 --- a/Doc/whatsnew/2.1.rst +++ b/Doc/whatsnew/2.1.rst @@ -367,7 +367,7 @@ dictionary:: This version works for simple things such as integers, but it has a side effect; the ``_cache`` dictionary holds a reference to the return values, so they'll -never be deallocated until the Python process exits and cleans up This isn't +never be deallocated until the Python process exits and cleans up. This isn't very noticeable for integers, but if :func:`f` returns an object, or a data structure that takes up a lot of memory, this can be a problem. diff --git a/Doc/whatsnew/3.2.rst b/Doc/whatsnew/3.2.rst index 9acc734..b900e77 100644 --- a/Doc/whatsnew/3.2.rst +++ b/Doc/whatsnew/3.2.rst @@ -1253,9 +1253,9 @@ definition:: @contextmanager def track_entry_and_exit(name): - logging.info('Entering: {}'.format(name)) + logging.info('Entering: %s', name) yield - logging.info('Exiting: {}'.format(name)) + logging.info('Exiting: %s', name) Formerly, this would have only been usable as a context manager:: diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index 0c9f2bb..93b297c 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -108,7 +108,7 @@ packages. Their concept and implementation are inspired by the popular with the interpreter core. This PEP adds the :mod:`venv` module for programmatic access, and the -:ref:`pyvenv <scripts-pyvenv>` script for command-line access and +``pyvenv`` script for command-line access and administration. The Python interpreter checks for a ``pyvenv.cfg``, file whose existence signals the base of a virtual environment's directory tree. diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst index 1e5c9d1..72398f9 100644 --- a/Doc/whatsnew/3.4.rst +++ b/Doc/whatsnew/3.4.rst @@ -197,7 +197,7 @@ will also be installed. On other platforms, the system wide unversioned ``pip`` command typically refers to the separately installed Python 2 version. -The :ref:`pyvenv <scripts-pyvenv>` command line utility and the :mod:`venv` +The ``pyvenv`` command line utility and the :mod:`venv` module make use of the :mod:`ensurepip` module to make ``pip`` readily available in virtual environments. When using the command line utility, ``pip`` is installed by default, while when using the :mod:`venv` module @@ -824,7 +824,7 @@ currently in the new module, which is being added as part of email's new :term:`provisional API`. These classes provide a number of new methods that make extracting content from and inserting content into email messages much easier. For details, see the :mod:`~email.contentmanager` documentation and -the :ref:`email-contentmanager-api-examples`. These API additions complete the +the :ref:`email-examples`. These API additions complete the bulk of the work that was planned as part of the email6 project. The currently provisional API is scheduled to become final in Python 3.5 (possibly with a few minor additions in the area of error handling). (Contributed by R. David @@ -1989,11 +1989,11 @@ Other Improvements Stinner using his :pep:`445`-based ``pyfailmalloc`` tool (:issue:`18408`, :issue:`18520`). -* The :ref:`pyvenv <scripts-pyvenv>` command now accepts a ``--copies`` option +* The ``pyvenv`` command now accepts a ``--copies`` option to use copies rather than symlinks even on systems where symlinks are the default. (Contributed by Vinay Sajip in :issue:`18807`.) -* The :ref:`pyvenv <scripts-pyvenv>` command also accepts a ``--without-pip`` +* The ``pyvenv`` command also accepts a ``--without-pip`` option to suppress the otherwise-automatic bootstrapping of pip into the virtual environment. (Contributed by Nick Coghlan in :issue:`19552` as part of the :pep:`453` implementation.) @@ -2459,7 +2459,7 @@ Changes in the Python API stream in :mod:`~io.TextIOWrapper` to use its *newline* argument (:issue:`15204`). -* If you use :ref:`pyvenv <scripts-pyvenv>` in a script and desire that pip +* If you use ``pyvenv`` in a script and desire that pip *not* be installed, you must add ``--without-pip`` to your command invocation. diff --git a/Doc/whatsnew/3.5.rst b/Doc/whatsnew/3.5.rst index 2ef4e91..246d9c5 100644 --- a/Doc/whatsnew/3.5.rst +++ b/Doc/whatsnew/3.5.rst @@ -46,7 +46,8 @@ This article explains the new features in Python 3.5, compared to 3.4. Python 3.5 was released on September 13, 2015. See the -:ref:`changelog <changelog>` for a full list of changes. +`changelog <https://docs.python.org/3.5/whatsnew/changelog.html>`_ for a full +list of changes. .. seealso:: @@ -2175,8 +2176,7 @@ New ``calloc`` functions were added: * :c:func:`PyMem_RawCalloc`, * :c:func:`PyMem_Calloc`, -* :c:func:`PyObject_Calloc`, -* :c:func:`_PyObject_GC_Calloc`. +* :c:func:`PyObject_Calloc`. (Contributed by Victor Stinner in :issue:`21233`.) diff --git a/Doc/whatsnew/3.6.rst b/Doc/whatsnew/3.6.rst new file mode 100644 index 0000000..e88d618 --- /dev/null +++ b/Doc/whatsnew/3.6.rst @@ -0,0 +1,1479 @@ +**************************** + What's New In Python 3.6 +**************************** + +:Release: |release| +:Date: |today| + +.. Rules for maintenance: + + * Anyone can add text to this document. Do not spend very much time + on the wording of your changes, because your text will probably + get rewritten to some degree. + + * The maintainer will go through Misc/NEWS periodically and add + changes; it's therefore more important to add your changes to + Misc/NEWS than to this file. + + * This is not a complete list of every single change; completeness + is the purpose of Misc/NEWS. Some changes I consider too small + or esoteric to include. If such a change is added to the text, + I'll just remove it. (This is another reason you shouldn't spend + too much time on writing your addition.) + + * If you want to draw your new text to the attention of the + maintainer, add 'XXX' to the beginning of the paragraph or + section. + + * It's OK to just add a fragmentary note about a change. For + example: "XXX Describe the transmogrify() function added to the + socket module." The maintainer will research the change and + write the necessary text. + + * You can comment out your additions if you like, but it's not + necessary (especially when a final release is some months away). + + * Credit the author of a patch or bugfix. Just the name is + sufficient; the e-mail address isn't necessary. + + * It's helpful to add the bug/patch number as a comment: + + XXX Describe the transmogrify() function added to the socket + module. + (Contributed by P.Y. Developer in :issue:`12345`.) + + This saves the maintainer the effort of going through the Mercurial log + when researching a change. + +This article explains the new features in Python 3.6, compared to 3.5. + +For full details, see the :ref:`changelog <changelog>`. + +.. note:: + + Prerelease users should be aware that this document is currently in draft + form. It will be updated substantially as Python 3.6 moves towards release, + so it's worth checking back even after reading earlier versions. + + +Summary -- Release highlights +============================= + +.. This section singles out the most important changes in Python 3.6. + Brevity is key. + +New syntax features: + +* A ``global`` or ``nonlocal`` statement must now textually appear + before the first use of the affected name in the same scope. + Previously this was a SyntaxWarning. + +* PEP 498: :ref:`Formatted string literals <whatsnew-fstrings>` + +* PEP 515: Underscores in Numeric Literals + +* PEP 526: :ref:`Syntax for Variable Annotations <variable-annotations>` + +* PEP 525: Asynchronous Generators + +* PEP 530: Asynchronous Comprehensions + +Standard library improvements: + +Security improvements: + +* On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool + is initialized to increase the security. See the :pep:`524` for the + rationale. + +* :mod:`hashlib` and :mod:`ssl` now support OpenSSL 1.1.0. + +* The default settings and feature set of the :mod:`ssl` have been improved. + +* The :mod:`hashlib` module has got support for BLAKE2, SHA-3 and SHAKE hash + algorithms and :func:`~hashlib.scrypt` key derivation function. + +Windows improvements: + +* PEP 529: :ref:`Change Windows filesystem encoding to UTF-8 <pep-529>` + +* PEP 528: :ref:`Change Windows console encoding to UTF-8 <pep-528>` + +* The ``py.exe`` launcher, when used interactively, no longer prefers + Python 2 over Python 3 when the user doesn't specify a version (via + command line arguments or a config file). Handling of shebang lines + remains unchanged - "python" refers to Python 2 in that case. + +* ``python.exe`` and ``pythonw.exe`` have been marked as long-path aware, + which means that when the 260 character path limit may no longer apply. + See :ref:`removing the MAX_PATH limitation <max-path>` for details. + +* A ``._pth`` file can be added to force isolated mode and fully specify + all search paths to avoid registry and environment lookup. See + :ref:`the documentation <finding_modules>` for more information. + +* A ``python36.zip`` file now works as a landmark to infer + :envvar:`PYTHONHOME`. See :ref:`the documentation <finding_modules>` for + more information. + +.. PEP-sized items next. + +.. _pep-4XX: + +.. PEP 4XX: Virtual Environments +.. ============================= + + +.. (Implemented by Foo Bar.) + +.. .. seealso:: + + :pep:`4XX` - Python Virtual Environments + PEP written by Carl Meyer + +New built-in features: + +* PEP 520: :ref:`Preserving Class Attribute Definition Order<whatsnew-deforder>` + +* PEP 468: :ref:`Preserving Keyword Argument Order<whatsnew-kwargs>` + +A complete list of PEP's implemented in Python 3.6: + +* :pep:`468`, :ref:`Preserving Keyword Argument Order<whatsnew-kwargs>` +* :pep:`487`, :ref:`Simpler customization of class creation<whatsnew-pep487>` +* :pep:`495`, Local Time Disambiguation +* :pep:`498`, :ref:`Formatted string literals <whatsnew-fstrings>` +* :pep:`506`, Adding A Secrets Module To The Standard Library +* :pep:`509`, :ref:`Add a private version to dict<whatsnew-pep509>` +* :pep:`515`, :ref:`Underscores in Numeric Literals<pep-515>` +* :pep:`519`, :ref:`Adding a file system path protocol<pep-519>` +* :pep:`520`, :ref:`Preserving Class Attribute Definition Order<whatsnew-deforder>` +* :pep:`523`, :ref:`Adding a frame evaluation API to CPython<pep-523>` +* :pep:`524`, Make os.urandom() blocking on Linux (during system startup) +* :pep:`525`, Asynchronous Generators (provisional) +* :pep:`526`, :ref:`Syntax for Variable Annotations (provisional)<variable-annotations>` +* :pep:`528`, :ref:`Change Windows console encoding to UTF-8 (provisional)<pep-528>` +* :pep:`529`, :ref:`Change Windows filesystem encoding to UTF-8 (provisional)<pep-529>` +* :pep:`530`, Asynchronous Comprehensions + + +New Features +============ + +.. _pep-515: + +PEP 515: Underscores in Numeric Literals +---------------------------------------- + +Prior to PEP 515, there was no support for writing long numeric +literals with some form of separator to improve readability. For +instance, how big is ``1000000000000000``? With :pep:`515`, though, +you can use underscores to separate digits as desired to make numeric +literals easier to read: ``1_000_000_000_000_000``. Underscores can be +used with other numeric literals beyond integers, e.g. +``0x_FF_FF_FF_FF``. + +Single underscores are allowed between digits and after any base +specifier. More than a single underscore in a row, leading, or +trailing underscores are not allowed. + +.. seealso:: + + :pep:`515` -- Underscores in Numeric Literals + PEP written by Georg Brandl and Serhiy Storchaka. + + +.. _pep-523: + +PEP 523: Adding a frame evaluation API to CPython +------------------------------------------------- + +While Python provides extensive support to customize how code +executes, one place it has not done so is in the evaluation of frame +objects. If you wanted some way to intercept frame evaluation in +Python there really wasn't any way without directly manipulating +function pointers for defined functions. + +:pep:`523` changes this by providing an API to make frame +evaluation pluggable at the C level. This will allow for tools such +as debuggers and JITs to intercept frame evaluation before the +execution of Python code begins. This enables the use of alternative +evaluation implementations for Python code, tracking frame +evaluation, etc. + +This API is not part of the limited C API and is marked as private to +signal that usage of this API is expected to be limited and only +applicable to very select, low-level use-cases. Semantics of the +API will change with Python as necessary. + +.. seealso:: + + :pep:`523` -- Adding a frame evaluation API to CPython + PEP written by Brett Cannon and Dino Viehland. + + +.. _pep-519: + +PEP 519: Adding a file system path protocol +------------------------------------------- + +File system paths have historically been represented as :class:`str` +or :class:`bytes` objects. This has led to people who write code which +operate on file system paths to assume that such objects are only one +of those two types (an :class:`int` representing a file descriptor +does not count as that is not a file path). Unfortunately that +assumption prevents alternative object representations of file system +paths like :mod:`pathlib` from working with pre-existing code, +including Python's standard library. + +To fix this situation, a new interface represented by +:class:`os.PathLike` has been defined. By implementing the +:meth:`~os.PathLike.__fspath__` method, an object signals that it +represents a path. An object can then provide a low-level +representation of a file system path as a :class:`str` or +:class:`bytes` object. This means an object is considered +:term:`path-like <path-like object>` if it implements +:class:`os.PathLike` or is a :class:`str` or :class:`bytes` object +which represents a file system path. Code can use :func:`os.fspath`, +:func:`os.fsdecode`, or :func:`os.fsencode` to explicitly get a +:class:`str` and/or :class:`bytes` representation of a path-like +object. + +The built-in :func:`open` function has been updated to accept +:class:`os.PathLike` objects as have all relevant functions in the +:mod:`os` and :mod:`os.path` modules. :c:func:`PyUnicode_FSConverter` +and :c:func:`PyUnicode_FSConverter` have been changed to accept +path-like objects. The :class:`os.DirEntry` class +and relevant classes in :mod:`pathlib` have also been updated to +implement :class:`os.PathLike`. + +The hope in is that updating the fundamental functions for operating +on file system paths will lead to third-party code to implicitly +support all :term:`path-like objects <path-like object>` without any +code changes or at least very minimal ones (e.g. calling +:func:`os.fspath` at the beginning of code before operating on a +path-like object). + +Here are some examples of how the new interface allows for +:class:`pathlib.Path` to be used more easily and transparently with +pre-existing code:: + + >>> import pathlib + >>> with open(pathlib.Path("README")) as f: + ... contents = f.read() + ... + >>> import os.path + >>> os.path.splitext(pathlib.Path("some_file.txt")) + ('some_file', '.txt') + >>> os.path.join("/a/b", pathlib.Path("c")) + '/a/b/c' + >>> import os + >>> os.fspath(pathlib.Path("some_file.txt")) + 'some_file.txt' + +(Implemented by Brett Cannon, Ethan Furman, Dusty Phillips, and Jelle Zijlstra.) + +.. seealso:: + + :pep:`519` -- Adding a file system path protocol + PEP written by Brett Cannon and Koos Zevenhoven. + + +.. _whatsnew-fstrings: + +PEP 498: Formatted string literals +---------------------------------- + +Formatted string literals are a new kind of string literal, prefixed +with ``'f'``. They are similar to the format strings accepted by +:meth:`str.format`. They contain replacement fields surrounded by +curly braces. The replacement fields are expressions, which are +evaluated at run time, and then formatted using the :func:`format` protocol:: + + >>> name = "Fred" + >>> f"He said his name is {name}." + 'He said his name is Fred.' + +See :pep:`498` and the main documentation at :ref:`f-strings`. + + +.. _variable-annotations: + +PEP 526: Syntax for variable annotations +---------------------------------------- + +:pep:`484` introduced standard for type annotations of function parameters, +a.k.a. type hints. This PEP adds syntax to Python for annotating the +types of variables including class variables and instance variables:: + + primes: List[int] = [] + + captain: str # Note: no initial value! + + class Starship: + stats: Dict[str, int] = {} + +Just as for function annotations, the Python interpreter does not attach any +particular meaning to variable annotations and only stores them in a special +attribute ``__annotations__`` of a class or module. +In contrast to variable declarations in statically typed languages, +the goal of annotation syntax is to provide an easy way to specify structured +type metadata for third party tools and libraries via the abstract syntax tree +and the ``__annotations__`` attribute. + +.. seealso:: + + :pep:`526` -- Syntax for variable annotations. + PEP written by Ryan Gonzalez, Philip House, Ivan Levkivskyi, Lisa Roach, + and Guido van Rossum. Implemented by Ivan Levkivskyi. + + Tools that use or will use the new syntax: + `mypy <http://github.com/python/mypy>`_, + `pytype <http://github.com/google/pytype>`_, PyCharm, etc. + + +.. _pep-529: + +PEP 529: Change Windows filesystem encoding to UTF-8 +---------------------------------------------------- + +Representing filesystem paths is best performed with str (Unicode) rather than +bytes. However, there are some situations where using bytes is sufficient and +correct. + +Prior to Python 3.6, data loss could result when using bytes paths on Windows. +With this change, using bytes to represent paths is now supported on Windows, +provided those bytes are encoded with the encoding returned by +:func:`sys.getfilesystemencoding()`, which now defaults to ``'utf-8'``. + +Applications that do not use str to represent paths should use +:func:`os.fsencode()` and :func:`os.fsdecode()` to ensure their bytes are +correctly encoded. To revert to the previous behaviour, set +:envvar:`PYTHONLEGACYWINDOWSFSENCODING` or call +:func:`sys._enablelegacywindowsfsencoding`. + +See :pep:`529` for more information and discussion of code modifications that +may be required. + +.. note:: + + This change is considered experimental for 3.6.0 beta releases. The default + encoding may change before the final release. + +.. _whatsnew-pep487: + +PEP 487: Simpler customization of class creation +------------------------------------------------ + +Upon subclassing a class, the ``__init_subclass__`` classmethod (if defined) is +called on the base class. This makes it straightforward to write classes that +customize initialization of future subclasses without introducing the +complexity of a full custom metaclass. + +The descriptor protocol has also been expanded to include a new optional method, +``__set_name__``. Whenever a new class is defined, the new method will be called +on all descriptors included in the definition, providing them with a reference +to the class being defined and the name given to the descriptor within the +class namespace. + +Also see :pep:`487` and the updated class customization documentation at +:ref:`class-customization` and :ref:`descriptors`. + +(Contributed by Martin Teichmann in :issue:`27366`) + +.. _pep-528: + +PEP 528: Change Windows console encoding to UTF-8 +------------------------------------------------- + +The default console on Windows will now accept all Unicode characters and +provide correctly read str objects to Python code. ``sys.stdin``, +``sys.stdout`` and ``sys.stderr`` now default to utf-8 encoding. + +This change only applies when using an interactive console, and not when +redirecting files or pipes. To revert to the previous behaviour for interactive +console use, set :envvar:`PYTHONLEGACYWINDOWSIOENCODING`. + +.. seealso:: + + :pep:`528` -- Change Windows console encoding to UTF-8 + PEP written and implemented by Steve Dower. + +PYTHONMALLOC environment variable +--------------------------------- + +The new :envvar:`PYTHONMALLOC` environment variable allows setting the Python +memory allocators and/or install debug hooks. + +It is now possible to install debug hooks on Python memory allocators on Python +compiled in release mode using ``PYTHONMALLOC=debug``. Effects of debug hooks: + +* Newly allocated memory is filled with the byte ``0xCB`` +* Freed memory is filled with the byte ``0xDB`` +* Detect violations of Python memory allocator API. For example, + :c:func:`PyObject_Free` called on a memory block allocated by + :c:func:`PyMem_Malloc`. +* Detect write before the start of the buffer (buffer underflow) +* Detect write after the end of the buffer (buffer overflow) +* Check that the :term:`GIL <global interpreter lock>` is held when allocator + functions of :c:data:`PYMEM_DOMAIN_OBJ` (ex: :c:func:`PyObject_Malloc`) and + :c:data:`PYMEM_DOMAIN_MEM` (ex: :c:func:`PyMem_Malloc`) domains are called. + +Checking if the GIL is held is also a new feature of Python 3.6. + +See the :c:func:`PyMem_SetupDebugHooks` function for debug hooks on Python +memory allocators. + +It is now also possible to force the usage of the :c:func:`malloc` allocator of +the C library for all Python memory allocations using ``PYTHONMALLOC=malloc``. +It helps to use external memory debuggers like Valgrind on a Python compiled in +release mode. + +On error, the debug hooks on Python memory allocators now use the +:mod:`tracemalloc` module to get the traceback where a memory block was +allocated. + +Example of fatal error on buffer overflow using +``python3.6 -X tracemalloc=5`` (store 5 frames in traces):: + + Debug memory block at address p=0x7fbcd41666f8: API 'o' + 4 bytes originally requested + The 7 pad bytes at p-7 are FORBIDDENBYTE, as expected. + The 8 pad bytes at tail=0x7fbcd41666fc are not all FORBIDDENBYTE (0xfb): + at tail+0: 0x02 *** OUCH + at tail+1: 0xfb + at tail+2: 0xfb + at tail+3: 0xfb + at tail+4: 0xfb + at tail+5: 0xfb + at tail+6: 0xfb + at tail+7: 0xfb + The block was made by call #1233329 to debug malloc/realloc. + Data at p: 1a 2b 30 00 + + Memory block allocated at (most recent call first): + File "test/test_bytes.py", line 323 + File "unittest/case.py", line 600 + File "unittest/case.py", line 648 + File "unittest/suite.py", line 122 + File "unittest/suite.py", line 84 + + Fatal Python error: bad trailing pad byte + + Current thread 0x00007fbcdbd32700 (most recent call first): + File "test/test_bytes.py", line 323 in test_hex + File "unittest/case.py", line 600 in run + File "unittest/case.py", line 648 in __call__ + File "unittest/suite.py", line 122 in run + File "unittest/suite.py", line 84 in __call__ + File "unittest/suite.py", line 122 in run + File "unittest/suite.py", line 84 in __call__ + ... + +(Contributed by Victor Stinner in :issue:`26516` and :issue:`26564`.) + + +DTrace and SystemTap probing support +------------------------------------ + +Python can now be built ``--with-dtrace`` which enables static markers +for the following events in the interpreter: + +* function call/return + +* garbage collection started/finished + +* line of code executed. + +This can be used to instrument running interpreters in production, +without the need to recompile specific debug builds or providing +application-specific profiling/debugging code. + +More details in :ref:`instrumentation`. + +The current implementation is tested on Linux and macOS. Additional +markers may be added in the future. + +(Contributed by Łukasz Langa in :issue:`21590`, based on patches by +Jesús Cea Avión, David Malcolm, and Nikhil Benesch.) + + +.. _whatsnew-deforder: + +PEP 520: Preserving Class Attribute Definition Order +---------------------------------------------------- + +Attributes in a class definition body have a natural ordering: the same +order in which the names appear in the source. This order is now +preserved in the new class's ``__dict__`` attribute. + +Also, the effective default class *execution* namespace (returned from +``type.__prepare__()``) is now an insertion-order-preserving mapping. + +.. seealso:: + + :pep:`520` -- Preserving Class Attribute Definition Order + PEP written and implemented by Eric Snow. + + +.. _whatsnew-kwargs: + +PEP 468: Preserving Keyword Argument Order +------------------------------------------ + +``**kwargs`` in a function signature is now guaranteed to be an +insertion-order-preserving mapping. + +.. seealso:: + + :pep:`468` -- Preserving Keyword Argument Order + PEP written and implemented by Eric Snow. + +.. _whatsnew-pep509: + +PEP 509: Add a private version to dict +-------------------------------------- + +Add a new private version to the builtin ``dict`` type, incremented at +each dictionary creation and at each dictionary change, to implement +fast guards on namespaces. + +(Contributed by Victor Stinner in :issue:`26058`.) + + +Other Language Changes +====================== + +Some smaller changes made to the core Python language are: + +* :func:`dict` now uses a "compact" representation `pioneered by PyPy + <https://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html>`_. + The memory usage of the new :func:`dict` is between 20% and 25% smaller + compared to Python 3.5. + :pep:`468` (Preserving the order of ``**kwargs`` in a function.) is + implemented by this. The order-preserving aspect of this new + implementation is considered an implementation detail and should + not be relied upon (this may change in the future, but it is desired + to have this new dict implementation in the language for a few + releases before changing the language spec to mandate + order-preserving semantics for all current and future Python + implementations; this also helps preserve backwards-compatibility + with older versions of the language where random iteration order is + still in effect, e.g. Python 3.5). + (Contributed by INADA Naoki in :issue:`27350`. Idea + `originally suggested by Raymond Hettinger + <https://mail.python.org/pipermail/python-dev/2012-December/123028.html>`_.) + +* Long sequences of repeated traceback lines are now abbreviated as + ``"[Previous line repeated {count} more times]"`` (see + :ref:`py36-traceback` for an example). + (Contributed by Emanuel Barry in :issue:`26823`.) + +* Import now raises the new exception :exc:`ModuleNotFoundError` + (subclass of :exc:`ImportError`) when it cannot find a module. Code + that current checks for ImportError (in try-except) will still work. + + +New Modules +=========== + +* None yet. + + +Improved Modules +================ + +On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool +is initialized to increase the security. See the :pep:`524` for the rationale. + + +asyncio +------- + +Since the :mod:`asyncio` module is :term:`provisional <provisional api>`, +all changes introduced in Python 3.6 have also been backported to Python +3.5.x. + +Notable changes in the :mod:`asyncio` module since Python 3.5.0: + +* The :func:`~asyncio.ensure_future` function and all functions that + use it, such as :meth:`loop.run_until_complete() <asyncio.BaseEventLoop.run_until_complete>`, + now accept all kinds of :term:`awaitable objects <awaitable>`. + (Contributed by Yury Selivanov.) + +* New :func:`~asyncio.run_coroutine_threadsafe` function to submit + coroutines to event loops from other threads. + (Contributed by Vincent Michel.) + +* New :meth:`Transport.is_closing() <asyncio.BaseTransport.is_closing>` + method to check if the transport is closing or closed. + (Contributed by Yury Selivanov.) + +* The :meth:`loop.create_server() <asyncio.BaseEventLoop.create_server>` + method can now accept a list of hosts. + (Contributed by Yann Sionneau.) + +* New :meth:`loop.create_future() <asyncio.BaseEventLoop.create_future>` + method to create Future objects. This allows alternative event + loop implementations, such as + `uvloop <https://github.com/MagicStack/uvloop>`_, to provide a faster + :class:`asyncio.Future` implementation. + (Contributed by Yury Selivanov.) + +* New :meth:`loop.get_exception_handler() <asyncio.BaseEventLoop.get_exception_handler>` + method to get the current exception handler. + (Contributed by Yury Selivanov.) + +* New :meth:`StreamReader.readuntil() <asyncio.StreamReader.readuntil>` + method to read data from the stream until a separator bytes + sequence appears. + (Contributed by Mark Korenberg.) + +* The :meth:`loop.getaddrinfo() <asyncio.BaseEventLoop.getaddrinfo>` + method is optimized to avoid calling the system ``getaddrinfo`` + function if the address is already resolved. + (Contributed by A. Jesse Jiryu Davis.) + + +contextlib +---------- + +The :class:`contextlib.AbstractContextManager` class has been added to +provide an abstract base class for context managers. It provides a +sensible default implementation for `__enter__()` which returns +``self`` and leaves `__exit__()` an abstract method. A matching +class has been added to the :mod:`typing` module as +:class:`typing.ContextManager`. +(Contributed by Brett Cannon in :issue:`25609`.) + + +venv +---- + +:mod:`venv` accepts a new parameter ``--prompt``. This parameter provides an +alternative prefix for the virtual environment. (Proposed by Łukasz.Balcerzak +and ported to 3.6 by Stéphane Wirtel in :issue:`22829`.) + + +datetime +-------- + +The :meth:`datetime.strftime() <datetime.datetime.strftime>` and +:meth:`date.strftime() <datetime.date.strftime>` methods now support ISO 8601 date +directives ``%G``, ``%u`` and ``%V``. +(Contributed by Ashley Anderson in :issue:`12006`.) + + +distutils.command.sdist +----------------------- + +The ``default_format`` attribute has been removed from +:class:`distutils.command.sdist.sdist` and the ``formats`` +attribute defaults to ``['gztar']``. Although not anticipated, +Any code relying on the presence of ``default_format`` may +need to be adapted. See :issue:`27819` for more details. + + +email +----- + +The new email API, enabled via the *policy* keyword to various constructors, is +no longer provisional. The :mod:`email` documentation has been reorganized and +rewritten to focus on the new API, while retaining the old documentation for +the legacy API. (Contributed by R. David Murray in :issue:`24277`.) + +The :mod:`email.mime` classes now all accept an optional *policy* keyword. +(Contributed by Berker Peksag in :issue:`27331`.) + +The :class:`~email.generator.DecodedGenerator` now supports the *policy* +keyword. + +There is a new :mod:`~email.policy` attribute, +:attr:`~email.policy.Policy.message_factory`, that controls what class is used +by default when the parser creates new message objects. For the +:attr:`email.policy.compat32` policy this is :class:`~email.message.Message`, +for the new policies it is :class:`~email.message.EmailMessage`. +(Contributed by R. David Murray in :issue:`20476`.) + + +encodings +--------- + +On Windows, added the ``'oem'`` encoding to use ``CP_OEMCP`` and the ``'ansi'`` +alias for the existing ``'mbcs'`` encoding, which uses the ``CP_ACP`` code page. + + +faulthandler +------------ + +On Windows, the :mod:`faulthandler` module now installs a handler for Windows +exceptions: see :func:`faulthandler.enable`. (Contributed by Victor Stinner in +:issue:`23848`.) + + +hashlib +------- + +:mod:`hashlib` supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2. +It has been tested with 0.9.8zc, 0.9.8zh and 1.0.1t as well as LibreSSL 2.3 +and 2.4. +(Contributed by Christian Heimes in :issue:`26470`.) + +BLAKE2 hash functions were added to the module. :func:`~hashlib.blake2b` +and :func:`~hashlib.blake2s` are always available and support the full +feature set of BLAKE2. +(Contributed by Christian Heimes in :issue:`26798` based on code by +Dmitry Chestnykh and Samuel Neves. Documentation written by Dmitry Chestnykh.) + +The SHA-3 hash functions :func:`~hashlib.sha3_224`, :func:`~hashlib.sha3_256`, +:func:`~hashlib.sha3_384`, :func:`~hashlib.sha3_512`, and SHAKE hash functions +:func:`~hashlib.shake_128` and :func:`~hashlib.shake_256` were added. +(Contributed by Christian Heimes in :issue:`16113`. Keccak Code Package +by Guido Bertoni, Joan Daemen, Michaël Peeters, Gilles Van Assche, and +Ronny Van Keer.) + +The password-based key derivation function :func:`~hashlib.scrypt` is now +available with OpenSSL 1.1.0 and newer. +(Contributed by Christian Heimes in :issue:`27928`.) + +http.client +----------- + +:meth:`HTTPConnection.request() <http.client.HTTPConnection.request>` and +:meth:`~http.client.HTTPConnection.endheaders` both now support +chunked encoding request bodies. +(Contributed by Demian Brecht and Rolf Krahl in :issue:`12319`.) + + +idlelib and IDLE +---------------- + +The idlelib package is being modernized and refactored to make IDLE look and work better and to make the code easier to understand, test, and improve. Part of making IDLE look better, especially on Linux and Mac, is using ttk widgets, mostly in the dialogs. As a result, IDLE no longer runs with tcl/tk 8.4. It now requires tcl/tk 8.5 or 8.6. We recommend running the latest release of either. + +'Modernizing' includes renaming and consolidation of idlelib modules. The renaming of files with partial uppercase names is similar to the renaming of, for instance, Tkinter and TkFont to tkinter and tkinter.font in 3.0. As a result, imports of idlelib files that worked in 3.5 will usually not work in 3.6. At least a module name change will be needed (see idlelib/README.txt), sometimes more. (Name changes contributed by Al Swiegart and Terry Reedy in :issue:`24225`. Most idlelib patches since have been and will be part of the process.) + +In compensation, the eventual result with be that some idlelib classes will be easier to use, with better APIs and docstrings explaining them. Additional useful information will be added to idlelib when available. + + +importlib +--------- + +:class:`importlib.util.LazyLoader` now calls +:meth:`~importlib.abc.Loader.create_module` on the wrapped loader, removing the +restriction that :class:`importlib.machinery.BuiltinImporter` and +:class:`importlib.machinery.ExtensionFileLoader` couldn't be used with +:class:`importlib.util.LazyLoader`. + +:func:`importlib.util.cache_from_source`, +:func:`importlib.util.source_from_cache`, and +:func:`importlib.util.spec_from_file_location` now accept a +:term:`path-like object`. + + +json +---- + +:func:`json.load` and :func:`json.loads` now support binary input. Encoded +JSON should be represented using either UTF-8, UTF-16, or UTF-32. +(Contributed by Serhiy Storchaka in :issue:`17909`.) + + +os +-- + +A new :meth:`~os.scandir.close` method allows explicitly closing a +:func:`~os.scandir` iterator. The :func:`~os.scandir` iterator now +supports the :term:`context manager` protocol. If a :func:`scandir` +iterator is neither exhausted nor explicitly closed a :exc:`ResourceWarning` +will be emitted in its destructor. +(Contributed by Serhiy Storchaka in :issue:`25994`.) + +The Linux ``getrandom()`` syscall (get random bytes) is now exposed as the new +:func:`os.getrandom` function. +(Contributed by Victor Stinner, part of the :pep:`524`) + +See the summary for :ref:`PEP 519 <pep-519>` for details on how the +:mod:`os` and :mod:`os.path` modules now support +:term:`path-like objects <path-like object>`. + + +pickle +------ + +Objects that need calling ``__new__`` with keyword arguments can now be pickled +using :ref:`pickle protocols <pickle-protocols>` older than protocol version 4. +Protocol version 4 already supports this case. (Contributed by Serhiy +Storchaka in :issue:`24164`.) + + +re +-- + +Added support of modifier spans in regular expressions. Examples: +``'(?i:p)ython'`` matches ``'python'`` and ``'Python'``, but not ``'PYTHON'``; +``'(?i)g(?-i:v)r'`` matches ``'GvR'`` and ``'gvr'``, but not ``'GVR'``. +(Contributed by Serhiy Storchaka in :issue:`433028`.) + +Match object groups can be accessed by ``__getitem__``, which is +equivalent to ``group()``. So ``mo['name']`` is now equivalent to +``mo.group('name')``. (Contributed by Eric Smith in :issue:`24454`.) + + +readline +-------- + +Added :func:`~readline.set_auto_history` to enable or disable +automatic addition of input to the history list. (Contributed by +Tyler Crompton in :issue:`26870`.) + + +rlcompleter +----------- + +Private and special attribute names now are omitted unless the prefix starts +with underscores. A space or a colon is added after some completed keywords. +(Contributed by Serhiy Storchaka in :issue:`25011` and :issue:`25209`.) + +Names of most attributes listed by :func:`dir` are now completed. +Previously, names of properties and slots which were not yet created on +an instance were excluded. (Contributed by Martin Panter in :issue:`25590`.) + + +site +---- + +When specifying paths to add to :attr:`sys.path` in a `.pth` file, +you may now specify file paths on top of directories (e.g. zip files). +(Contributed by Wolfgang Langner in :issue:`26587`). + + +sqlite3 +------- + +:attr:`sqlite3.Cursor.lastrowid` now supports the ``REPLACE`` statement. +(Contributed by Alex LordThorsen in :issue:`16864`.) + + +socket +------ + +The :func:`~socket.socket.ioctl` function now supports the :data:`~socket.SIO_LOOPBACK_FAST_PATH` +control code. +(Contributed by Daniel Stokes in :issue:`26536`.) + +The :meth:`~socket.socket.getsockopt` constants ``SO_DOMAIN``, +``SO_PROTOCOL``, ``SO_PEERSEC``, and ``SO_PASSSEC`` are now supported. +(Contributed by Christian Heimes in :issue:`26907`.) + +The socket module now supports the address family +:data:`~socket.AF_ALG` to interface with Linux Kernel crypto API. ``ALG_*``, +``SOL_ALG`` and :meth:`~socket.socket.sendmsg_afalg` were added. +(Contributed by Christian Heimes in :issue:`27744` with support from +Victor Stinner.) + +socketserver +------------ + +Servers based on the :mod:`socketserver` module, including those +defined in :mod:`http.server`, :mod:`xmlrpc.server` and +:mod:`wsgiref.simple_server`, now support the :term:`context manager` +protocol. +(Contributed by Aviv Palivoda in :issue:`26404`.) + +The :attr:`~socketserver.StreamRequestHandler.wfile` attribute of +:class:`~socketserver.StreamRequestHandler` classes now implements +the :class:`io.BufferedIOBase` writable interface. In particular, +calling :meth:`~io.BufferedIOBase.write` is now guaranteed to send the +data in full. (Contributed by Martin Panter in :issue:`26721`.) + +ssl +--- + +:mod:`ssl` supports OpenSSL 1.1.0. The minimum recommend version is 1.0.2. +It has been tested with 0.9.8zc, 0.9.8zh and 1.0.1t as well as LibreSSL 2.3 +and 2.4. +(Contributed by Christian Heimes in :issue:`26470`.) + +3DES has been removed from the default cipher suites and ChaCha20 Poly1305 +cipher suites are now in the right position. +(Contributed by Christian Heimes in :issue:`27850` and :issue:`27766`.) + +:class:`~ssl.SSLContext` has better default configuration for options +and ciphers. +(Contributed by Christian Heimes in :issue:`28043`.) + +SSL session can be copied from one client-side connection to another +with :class:`~ssl.SSLSession`. TLS session resumption can speed up +the initial handshake, reduce latency and improve performance +(Contributed by Christian Heimes in :issue:`19500` based on a draft by +Alex Warhawk.) + +All constants and flags have been converted to :class:`~enum.IntEnum` and +:class:`~enum.IntFlags`. +(Contributed by Christian Heimes in :issue:`28025`.) + +Server and client-side specific TLS protocols for :class:`~ssl.SSLContext` +were added. +(Contributed by Christian Heimes in :issue:`28085`.) + +General resource ids (``GEN_RID``) in subject alternative name extensions +no longer case a SystemError. +(Contributed by Christian Heimes in :issue:`27691`.) + +subprocess +---------- + +:class:`subprocess.Popen` destructor now emits a :exc:`ResourceWarning` warning +if the child process is still running. Use the context manager protocol (``with +proc: ...``) or call explicitly the :meth:`~subprocess.Popen.wait` method to +read the exit status of the child process (Contributed by Victor Stinner in +:issue:`26741`). + +The :class:`subprocess.Popen` constructor and all functions that pass arguments +through to it now accept *encoding* and *errors* arguments. Specifying either +of these will enable text mode for the *stdin*, *stdout* and *stderr* streams. + +telnetlib +--------- + +:class:`~telnetlib.Telnet` is now a context manager (contributed by +Stéphane Wirtel in :issue:`25485`). + + +tkinter +------- + +Added methods :meth:`~tkinter.Variable.trace_add`, +:meth:`~tkinter.Variable.trace_remove` and :meth:`~tkinter.Variable.trace_info` +in the :class:`tkinter.Variable` class. They replace old methods +:meth:`~tkinter.Variable.trace_variable`, :meth:`~tkinter.Variable.trace`, +:meth:`~tkinter.Variable.trace_vdelete` and +:meth:`~tkinter.Variable.trace_vinfo` that use obsolete Tcl commands and might +not work in future versions of Tcl. +(Contributed by Serhiy Storchaka in :issue:`22115`). + + +.. _py36-traceback: + +traceback +--------- + +Both the traceback module and the interpreter's builtin exception display now +abbreviate long sequences of repeated lines in tracebacks as shown in the +following example:: + + >>> def f(): f() + ... + >>> f() + Traceback (most recent call last): + File "<stdin>", line 1, in <module> + File "<stdin>", line 1, in f + File "<stdin>", line 1, in f + File "<stdin>", line 1, in f + [Previous line repeated 995 more times] + RecursionError: maximum recursion depth exceeded + +(Contributed by Emanuel Barry in :issue:`26823`.) + + +typing +------ + +The :class:`typing.ContextManager` class has been added for +representing :class:`contextlib.AbstractContextManager`. +(Contributed by Brett Cannon in :issue:`25609`.) + + +unicodedata +----------- + +The internal database has been upgraded to use Unicode 9.0.0. (Contributed by +Benjamin Peterson.) + + +unittest.mock +------------- + +The :class:`~unittest.mock.Mock` class has the following improvements: + +* Two new methods, :meth:`Mock.assert_called() + <unittest.mock.Mock.assert_called>` and :meth:`Mock.assert_called_once() + <unittest.mock.Mock.assert_called_once>` to check if the mock object + was called. + (Contributed by Amit Saha in :issue:`26323`.) + + +urllib.request +-------------- + +If a HTTP request has a file or iterable body (other than a +bytes object) but no Content-Length header, rather than +throwing an error, :class:`~urllib.request.AbstractHTTPHandler` now +falls back to use chunked transfer encoding. +(Contributed by Demian Brecht and Rolf Krahl in :issue:`12319`.) + + +urllib.robotparser +------------------ + +:class:`~urllib.robotparser.RobotFileParser` now supports the ``Crawl-delay`` and +``Request-rate`` extensions. +(Contributed by Nikolay Bogoychev in :issue:`16099`.) + + +warnings +-------- + +A new optional *source* parameter has been added to the +:func:`warnings.warn_explicit` function: the destroyed object which emitted a +:exc:`ResourceWarning`. A *source* attribute has also been added to +:class:`warnings.WarningMessage` (contributed by Victor Stinner in +:issue:`26568` and :issue:`26567`). + +When a :exc:`ResourceWarning` warning is logged, the :mod:`tracemalloc` is now +used to try to retrieve the traceback where the detroyed object was allocated. + +Example with the script ``example.py``:: + + import warnings + + def func(): + return open(__file__) + + f = func() + f = None + +Output of the command ``python3.6 -Wd -X tracemalloc=5 example.py``:: + + example.py:7: ResourceWarning: unclosed file <_io.TextIOWrapper name='example.py' mode='r' encoding='UTF-8'> + f = None + Object allocated at (most recent call first): + File "example.py", lineno 4 + return open(__file__) + File "example.py", lineno 6 + f = func() + +The "Object allocated at" traceback is new and only displayed if +:mod:`tracemalloc` is tracing Python memory allocations and if the +:mod:`warnings` was already imported. + + +winreg +------ + +Added the 64-bit integer type :data:`REG_QWORD <winreg.REG_QWORD>`. +(Contributed by Clement Rouault in :issue:`23026`.) + + +winsound +-------- + +Allowed keyword arguments to be passed to :func:`Beep <winsound.Beep>`, +:func:`MessageBeep <winsound.MessageBeep>`, and :func:`PlaySound +<winsound.PlaySound>` (:issue:`27982`). + + +xmlrpc.client +------------- + +The module now supports unmarshalling additional data types used by +Apache XML-RPC implementation for numerics and ``None``. +(Contributed by Serhiy Storchaka in :issue:`26885`.) + + +zipfile +------- + +A new :meth:`ZipInfo.from_file() <zipfile.ZipInfo.from_file>` class method +allows making a :class:`~zipfile.ZipInfo` instance from a filesystem file. +A new :meth:`ZipInfo.is_dir() <zipfile.ZipInfo.is_dir>` method can be used +to check if the :class:`~zipfile.ZipInfo` instance represents a directory. +(Contributed by Thomas Kluyver in :issue:`26039`.) + +The :meth:`ZipFile.open() <zipfile.ZipFile.open>` method can now be used to +write data into a ZIP file, as well as for extracting data. +(Contributed by Thomas Kluyver in :issue:`26039`.) + + +zlib +---- + +The :func:`~zlib.compress` function now accepts keyword arguments. +(Contributed by Aviv Palivoda in :issue:`26243`.) + + +fileinput +--------- + +:func:`~fileinput.hook_encoded` now supports the *errors* argument. +(Contributed by Joseph Hackman in :issue:`25788`.) + + +Optimizations +============= + +* The ASCII decoder is now up to 60 times as fast for error handlers + ``surrogateescape``, ``ignore`` and ``replace`` (Contributed + by Victor Stinner in :issue:`24870`). + +* The ASCII and the Latin1 encoders are now up to 3 times as fast for the + error handler ``surrogateescape`` (Contributed by Victor Stinner in :issue:`25227`). + +* The UTF-8 encoder is now up to 75 times as fast for error handlers + ``ignore``, ``replace``, ``surrogateescape``, ``surrogatepass`` (Contributed + by Victor Stinner in :issue:`25267`). + +* The UTF-8 decoder is now up to 15 times as fast for error handlers + ``ignore``, ``replace`` and ``surrogateescape`` (Contributed + by Victor Stinner in :issue:`25301`). + +* ``bytes % args`` is now up to 2 times faster. (Contributed by Victor Stinner + in :issue:`25349`). + +* ``bytearray % args`` is now between 2.5 and 5 times faster. (Contributed by + Victor Stinner in :issue:`25399`). + +* Optimize :meth:`bytes.fromhex` and :meth:`bytearray.fromhex`: they are now + between 2x and 3.5x faster. (Contributed by Victor Stinner in :issue:`25401`). + +* Optimize ``bytes.replace(b'', b'.')`` and ``bytearray.replace(b'', b'.')``: + up to 80% faster. (Contributed by Josh Snider in :issue:`26574`). + +* Allocator functions of the :c:func:`PyMem_Malloc` domain + (:c:data:`PYMEM_DOMAIN_MEM`) now use the :ref:`pymalloc memory allocator + <pymalloc>` instead of :c:func:`malloc` function of the C library. The + pymalloc allocator is optimized for objects smaller or equal to 512 bytes + with a short lifetime, and use :c:func:`malloc` for larger memory blocks. + (Contributed by Victor Stinner in :issue:`26249`). + +* :func:`pickle.load` and :func:`pickle.loads` are now up to 10% faster when + deserializing many small objects (Contributed by Victor Stinner in + :issue:`27056`). + +- Passing :term:`keyword arguments <keyword argument>` to a function has an + overhead in comparison with passing :term:`positional arguments + <positional argument>`. Now in extension functions implemented with using + Argument Clinic this overhead is significantly decreased. + (Contributed by Serhiy Storchaka in :issue:`27574`). + +* Optimized :func:`~glob.glob` and :func:`~glob.iglob` functions in the + :mod:`glob` module; they are now about 3--6 times faster. + (Contributed by Serhiy Storchaka in :issue:`25596`). + +* Optimized globbing in :mod:`pathlib` by using :func:`os.scandir`; + it is now about 1.5--4 times faster. + (Contributed by Serhiy Storchaka in :issue:`26032`). + +Build and C API Changes +======================= + +* Python now requires some C99 support in the toolchain to build. For more + information, see :pep:`7`. + +* Cross-compiling CPython with the Android NDK and the Android API level set to + 21 (Android 5.0 Lollilop) or greater, runs successfully. While Android is not + yet a supported platform, the Python test suite runs on the Android emulator + with only about 16 tests failures. See the Android meta-issue :issue:`26865`. + +* The ``--with-optimizations`` configure flag has been added. Turning it on + will activate LTO and PGO build support (when available). + (Original patch by Alecsandru Patrascu of Intel in :issue:`26539`.) + +* New :c:func:`Py_FinalizeEx` API which indicates if flushing buffered data + failed (:issue:`5319`). + +* :c:func:`PyArg_ParseTupleAndKeywords` now supports :ref:`positional-only + parameters <positional-only_parameter>`. Positional-only parameters are + defined by empty names. + (Contributed by Serhiy Storchaka in :issue:`26282`). + +* ``PyTraceback_Print`` method now abbreviates long sequences of repeated lines + as ``"[Previous line repeated {count} more times]"``. + (Contributed by Emanuel Barry in :issue:`26823`.) + + +Deprecated +========== + +Deprecated Build Options +------------------------ + +The ``--with-system-ffi`` configure flag is now on by default on non-OSX UNIX +platforms. It may be disabled by using ``--without-system-ffi``, but using the +flag is deprecated and will not be accepted in Python 3.7. OSX is unaffected +by this change. Note that many OS distributors already use the +``--with-system-ffi`` flag when building their system Python. + + +New Keywords +------------ + +``async`` and ``await`` are not recommended to be used as variable, class, +function or module names. Introduced by :pep:`492` in Python 3.5, they will +become proper keywords in Python 3.7. + + +Deprecated Python modules, functions and methods +------------------------------------------------ + +* :meth:`importlib.machinery.SourceFileLoader.load_module` and + :meth:`importlib.machinery.SourcelessFileLoader.load_module` are now + deprecated. They were the only remaining implementations of + :meth:`importlib.abc.Loader.load_module` in :mod:`importlib` that had not + been deprecated in previous versions of Python in favour of + :meth:`importlib.abc.Loader.exec_module`. + +* The :mod:`tkinter.tix` module is now deprecated. :mod:`tkinter` users should + use :mod:`tkinter.ttk` instead. + + +Deprecated functions and types of the C API +------------------------------------------- + +* Undocumented functions :c:func:`PyUnicode_AsEncodedObject`, + :c:func:`PyUnicode_AsDecodedObject`, :c:func:`PyUnicode_AsEncodedUnicode` + and :c:func:`PyUnicode_AsDecodedUnicode` are deprecated now. + Use :ref:`generic codec based API <codec-registry>` instead. + + +Deprecated features +------------------- + +* The ``pyvenv`` script has been deprecated in favour of ``python3 -m venv``. + This prevents confusion as to what Python interpreter ``pyvenv`` is + connected to and thus what Python interpreter will be used by the virtual + environment. (Contributed by Brett Cannon in :issue:`25154`.) + +* When performing a relative import, falling back on ``__name__`` and + ``__path__`` from the calling module when ``__spec__`` or + ``__package__`` are not defined now raises an :exc:`ImportWarning`. + (Contributed by Rose Ames in :issue:`25791`.) + +* Unlike to other :mod:`dbm` implementations, the :mod:`dbm.dumb` module + creates database in ``'r'`` and ``'w'`` modes if it doesn't exist and + allows modifying database in ``'r'`` mode. This behavior is now deprecated + and will be removed in 3.8. + (Contributed by Serhiy Storchaka in :issue:`21708`.) + +* Undocumented support of general :term:`bytes-like objects <bytes-like object>` + as paths in :mod:`os` functions, :func:`compile` and similar functions is + now deprecated. + (Contributed by Serhiy Storchaka in :issue:`25791` and :issue:`26754`.) + +* The undocumented ``extra_path`` argument to a distutils Distribution + is now considered + deprecated, will raise a warning during install if set. Support for this + parameter will be dropped in a future Python release and likely earlier + through third party tools. See :issue:`27919` for details. + +* A backslash-character pair that is not a valid escape sequence now generates + a DeprecationWarning. Although this will eventually become a SyntaxError, + that will not be for several Python releases. (Contributed by Emanuel Barry + in :issue:`27364`.) + +* Inline flags ``(?letters)`` now should be used only at the start of the + regular expression. Inline flags in the middle of the regular expression + affects global flags in Python :mod:`re` module. This is an exception to + other regular expression engines that either apply flags to only part of + the regular expression or treat them as an error. To avoid distinguishing + inline flags in the middle of the regular expression now emit a deprecation + warning. It will be an error in future Python releases. + (Contributed by Serhiy Storchaka in :issue:`22493`.) + +* SSL-related arguments like ``certfile``, ``keyfile`` and ``check_hostname`` + in :mod:`ftplib`, :mod:`http.client`, :mod:`imaplib`, :mod:`poplib`, + and :mod:`smtplib` have been deprecated in favor of ``context``. + (Contributed by Christian Heimes in :issue:`28022`.) + +* A couple of protocols and functions of the :mod:`ssl` module are now + deprecated. Some features will no longer be available in future versions + of OpenSSL. Other features are deprecated in favor of a different API. + (Contributed by Christian Heimes in :issue:`28022` and :issue:`26470`.) + + +Deprecated Python behavior +-------------------------- + +* Raising the :exc:`StopIteration` exception inside a generator will now generate a + :exc:`DeprecationWarning`, and will trigger a :exc:`RuntimeError` in Python 3.7. + See :ref:`whatsnew-pep-479` for details. + + +Removed +======= + +API and Feature Removals +------------------------ + +* ``inspect.getmoduleinfo()`` was removed (was deprecated since CPython 3.3). + :func:`inspect.getmodulename` should be used for obtaining the module + name for a given path. + +* ``traceback.Ignore`` class and ``traceback.usage``, ``traceback.modname``, + ``traceback.fullmodname``, ``traceback.find_lines_from_code``, + ``traceback.find_lines``, ``traceback.find_strings``, + ``traceback.find_executable_lines`` methods were removed from the + :mod:`traceback` module. They were undocumented methods deprecated since + Python 3.2 and equivalent functionality is available from private methods. + +* The ``tk_menuBar()`` and ``tk_bindForTraversal()`` dummy methods in + :mod:`tkinter` widget classes were removed (corresponding Tk commands + were obsolete since Tk 4.0). + +* The :meth:`~zipfile.ZipFile.open` method of the :class:`zipfile.ZipFile` + class no longer supports the ``'U'`` mode (was deprecated since Python 3.4). + Use :class:`io.TextIOWrapper` for reading compressed text files in + :term:`universal newlines` mode. + +* The undocumented ``IN``, ``CDROM``, ``DLFCN``, ``TYPES``, ``CDIO``, and + ``STROPTS`` modules have been removed. They had been available in the + platform specific ``Lib/plat-*/`` directories, but were chronically out of + date, inconsistently available across platforms, and unmaintained. The + script that created these modules is still available in the source + distribution at :source:`Tools/scripts/h2py.py`. + + +Porting to Python 3.6 +===================== + +This section lists previously described changes and other bugfixes +that may require changes to your code. + +Changes in 'python' Command Behavior +------------------------------------ + +* The output of a special Python build with defined ``COUNT_ALLOCS``, + ``SHOW_ALLOC_COUNT`` or ``SHOW_TRACK_COUNT`` macros is now off by + default. It can be re-enabled using the ``-X showalloccount`` option. + It now outputs to ``stderr`` instead of ``stdout``. + (Contributed by Serhiy Storchaka in :issue:`23034`.) + + +Changes in the Python API +------------------------- + +* :mod:`sqlite3` no longer implicitly commit an open transaction before DDL + statements. + +* On Linux, :func:`os.urandom` now blocks until the system urandom entropy pool + is initialized to increase the security. + +* When :meth:`importlib.abc.Loader.exec_module` is defined, + :meth:`importlib.abc.Loader.create_module` must also be defined. + +* :c:func:`PyErr_SetImportError` now sets :exc:`TypeError` when its **msg** + argument is not set. Previously only ``NULL`` was returned. + +* The format of the ``co_lnotab`` attribute of code objects changed to support + negative line number delta. By default, Python does not emit bytecode with + negative line number delta. Functions using ``frame.f_lineno``, + ``PyFrame_GetLineNumber()`` or ``PyCode_Addr2Line()`` are not affected. + Functions decoding directly ``co_lnotab`` should be updated to use a signed + 8-bit integer type for the line number delta, but it's only required to + support applications using negative line number delta. See + ``Objects/lnotab_notes.txt`` for the ``co_lnotab`` format and how to decode + it, and see the :pep:`511` for the rationale. + +* The functions in the :mod:`compileall` module now return booleans instead + of ``1`` or ``0`` to represent success or failure, respectively. Thanks to + booleans being a subclass of integers, this should only be an issue if you + were doing identity checks for ``1`` or ``0``. See :issue:`25768`. + +* Reading the :attr:`~urllib.parse.SplitResult.port` attribute of + :func:`urllib.parse.urlsplit` and :func:`~urllib.parse.urlparse` results + now raises :exc:`ValueError` for out-of-range values, rather than + returning :const:`None`. See :issue:`20059`. + +* The :mod:`imp` module now raises a :exc:`DeprecationWarning` instead of + :exc:`PendingDeprecationWarning`. + +* The following modules have had missing APIs added to their :attr:`__all__` + attributes to match the documented APIs: + :mod:`calendar`, :mod:`cgi`, :mod:`csv`, + :mod:`~xml.etree.ElementTree`, :mod:`enum`, + :mod:`fileinput`, :mod:`ftplib`, :mod:`logging`, :mod:`mailbox`, + :mod:`mimetypes`, :mod:`optparse`, :mod:`plistlib`, :mod:`smtpd`, + :mod:`subprocess`, :mod:`tarfile`, :mod:`threading` and + :mod:`wave`. This means they will export new symbols when ``import *`` + is used. See :issue:`23883`. + +* When performing a relative import, if ``__package__`` does not compare equal + to ``__spec__.parent`` then :exc:`ImportWarning` is raised. + (Contributed by Brett Cannon in :issue:`25791`.) + +* When a relative import is performed and no parent package is known, then + :exc:`ImportError` will be raised. Previously, :exc:`SystemError` could be + raised. (Contributed by Brett Cannon in :issue:`18018`.) + +* Servers based on the :mod:`socketserver` module, including those + defined in :mod:`http.server`, :mod:`xmlrpc.server` and + :mod:`wsgiref.simple_server`, now only catch exceptions derived + from :exc:`Exception`. Therefore if a request handler raises + an exception like :exc:`SystemExit` or :exc:`KeyboardInterrupt`, + :meth:`~socketserver.BaseServer.handle_error` is no longer called, and + the exception will stop a single-threaded server. (Contributed by + Martin Panter in :issue:`23430`.) + +* :func:`spwd.getspnam` now raises a :exc:`PermissionError` instead of + :exc:`KeyError` if the user doesn't have privileges. + +* The :meth:`socket.socket.close` method now raises an exception if + an error (e.g. EBADF) was reported by the underlying system call. + See :issue:`26685`. + +* The *decode_data* argument for :class:`smtpd.SMTPChannel` and + :class:`smtpd.SMTPServer` constructors is now ``False`` by default. + This means that the argument passed to + :meth:`~smtpd.SMTPServer.process_message` is now a bytes object by + default, and ``process_message()`` will be passed keyword arguments. + Code that has already been updated in accordance with the deprecation + warning generated by 3.5 will not be affected. + +* All optional parameters of the :func:`~json.dump`, :func:`~json.dumps`, + :func:`~json.load` and :func:`~json.loads` functions and + :class:`~json.JSONEncoder` and :class:`~json.JSONDecoder` class + constructors in the :mod:`json` module are now :ref:`keyword-only + <keyword-only_parameter>`. + (Contributed by Serhiy Storchaka in :issue:`18726`.) + +* As part of :pep:`487`, the handling of keyword arguments passed to + :class:`type` (other than the metaclass hint, ``metaclass``) is now + consistently delegated to :meth:`object.__init_subclass__`. This means that + :meth:`type.__new__` and :meth:`type.__init__` both now accept arbitrary + keyword arguments, but :meth:`object.__init_subclass__` (which is called from + :meth:`type.__new__`) will reject them by default. Custom metaclasses + accepting additional keyword arguments will need to adjust their calls to + :meth:`type.__new__` (whether direct or via :class:`super`) accordingly. + +* In :class:`distutils.command.sdist.sdist`, the ``default_format`` + attribute has been removed and is no longer honored. Instead, the + gzipped tarfile format is the default on all platforms and no + platform-specific selection is made. + In environments where distributions are + built on Windows and zip distributions are required, configure + the project with a ``setup.cfg`` file containing the following:: + + [sdist] + formats=zip + + This behavior has also been backported to earlier Python versions + by Setuptools 26.0.0. + +* In the :mod:`urllib.request` module and the + :meth:`http.client.HTTPConnection.request` method, if no Content-Length + header field has been specified and the request body is a file object, + it is now sent with HTTP 1.1 chunked encoding. If a file object has to + be sent to a HTTP 1.0 server, the Content-Length value now has to be + specified by the caller. See :issue:`12319`. + +Changes in the C API +-------------------- + +* :c:func:`PyMem_Malloc` allocator family now uses the :ref:`pymalloc allocator + <pymalloc>` rather than system :c:func:`malloc`. Applications calling + :c:func:`PyMem_Malloc` without holding the GIL can now crash. Set the + :envvar:`PYTHONMALLOC` environment variable to ``debug`` to validate the + usage of memory allocators in your application. See :issue:`26249`. + +* :c:func:`Py_Exit` (and the main interpreter) now override the exit status + with 120 if flushing buffered data failed. See :issue:`5319`. diff --git a/Doc/whatsnew/index.rst b/Doc/whatsnew/index.rst index edb5502..7c92524 100644 --- a/Doc/whatsnew/index.rst +++ b/Doc/whatsnew/index.rst @@ -11,6 +11,7 @@ anyone wishing to stay up-to-date after a new release. .. toctree:: :maxdepth: 2 + 3.6.rst 3.5.rst 3.4.rst 3.3.rst |