diff options
Diffstat (limited to 'Doc/library')
| -rw-r--r-- | Doc/library/asyncio-dev.rst | 2 | ||||
| -rw-r--r-- | Doc/library/asyncio-eventloop.rst | 2 | ||||
| -rw-r--r-- | Doc/library/development.rst | 4 | ||||
| -rw-r--r-- | Doc/library/devmode.rst | 214 | ||||
| -rw-r--r-- | Doc/library/exceptions.rst | 16 | ||||
| -rw-r--r-- | Doc/library/faulthandler.rst | 3 | ||||
| -rw-r--r-- | Doc/library/stdtypes.rst | 8 | ||||
| -rw-r--r-- | Doc/library/sys.rst | 15 |
8 files changed, 246 insertions, 18 deletions
diff --git a/Doc/library/asyncio-dev.rst b/Doc/library/asyncio-dev.rst index 101e781..ff51c4f 100644 --- a/Doc/library/asyncio-dev.rst +++ b/Doc/library/asyncio-dev.rst @@ -25,7 +25,7 @@ There are several ways to enable asyncio debug mode: * Setting the :envvar:`PYTHONASYNCIODEBUG` environment variable to ``1``. -* Using the :option:`-X` ``dev`` Python command line option. +* Using the :ref:`Python Development Mode <devmode>`. * Passing ``debug=True`` to :func:`asyncio.run`. diff --git a/Doc/library/asyncio-eventloop.rst b/Doc/library/asyncio-eventloop.rst index 25a3692..0029d94 100644 --- a/Doc/library/asyncio-eventloop.rst +++ b/Doc/library/asyncio-eventloop.rst @@ -1200,7 +1200,7 @@ Enabling debug mode .. versionchanged:: 3.7 - The new ``-X dev`` command line option can now also be used + The new :ref:`Python Development Mode <devmode>` can now also be used to enable the debug mode. .. seealso:: diff --git a/Doc/library/development.rst b/Doc/library/development.rst index ab34e1f..9edce75 100644 --- a/Doc/library/development.rst +++ b/Doc/library/development.rst @@ -18,12 +18,10 @@ The list of modules described in this chapter is: typing.rst pydoc.rst + devmode.rst doctest.rst unittest.rst unittest.mock.rst unittest.mock-examples.rst 2to3.rst test.rst - -See also the Python development mode: the :option:`-X` ``dev`` option and -:envvar:`PYTHONDEVMODE` environment variable. diff --git a/Doc/library/devmode.rst b/Doc/library/devmode.rst new file mode 100644 index 0000000..d5a40cd --- /dev/null +++ b/Doc/library/devmode.rst @@ -0,0 +1,214 @@ +.. _devmode: + +Python Development Mode +======================= + +.. versionadded:: 3.7 + +The Python Development Mode introduces additional runtime checks that are too +expensive to be enabled by default. It should not be more verbose than the +default if the code is correct; new warnings are only emitted when an issue is +detected. + +It can be enabled using the :option:`-X dev <-X>` command line option or by +setting the :envvar:`PYTHONDEVMODE` environment variable to ``1``. + +Effects of the Python Development Mode +====================================== + +Enabling the Python Development Mode is similar to the following command, but +with additional effects described below:: + + PYTHONMALLOC=debug PYTHONASYNCIODEBUG=1 python3 -W default -X faulthandler + +Effects of the Python Development Mode: + +* Add ``default`` :ref:`warning filter <describing-warning-filters>`. The + following warnings are shown: + + * :exc:`DeprecationWarning` + * :exc:`ImportWarning` + * :exc:`PendingDeprecationWarning` + * :exc:`ResourceWarning` + + Normally, the above warnings are filtered by the default :ref:`warning + filters <describing-warning-filters>`. + + It behaves as if the :option:`-W default <-W>` command line option is used. + + Use the :option:`-W error <-W>` command line option or set the + :envvar:`PYTHONWARNINGS` environment variable to ``error`` to treat warnings + as errors. + +* Install debug hooks on memory allocators to check for: + + * Buffer underflow + * Buffer overflow + * Memory allocator API violation + * Unsafe usage of the GIL + + See the :c:func:`PyMem_SetupDebugHooks` C function. + + It behaves as if the :envvar:`PYTHONMALLOC` environment variable is set to + ``debug``. + + To enable the Python Development Mode without installing debug hooks on + memory allocators, set the :envvar:`PYTHONMALLOC` environment variable to + ``default``. + +* Call :func:`faulthandler.enable` at Python startup to install handlers for + the :const:`SIGSEGV`, :const:`SIGFPE`, :const:`SIGABRT`, :const:`SIGBUS` and + :const:`SIGILL` signals to dump the Python traceback on a crash. + + It behaves as if the :option:`-X faulthandler <-X>` command line option is + used or if the :envvar:`PYTHONFAULTHANDLER` environment variable is set to + ``1``. + +* Enable :ref:`asyncio debug mode <asyncio-debug-mode>`. For example, + :mod:`asyncio` checks for coroutines that were not awaited and logs them. + + It behaves as if the :envvar:`PYTHONASYNCIODEBUG` environment variable is set + to ``1``. + +* Check the *encoding* and *errors* arguments for string encoding and decoding + operations. Examples: :func:`open`, :meth:`str.encode` and + :meth:`bytes.decode`. + + By default, for best performance, the *errors* argument is only checked at + the first encoding/decoding error and the *encoding* argument is sometimes + ignored for empty strings. + +* The :class:`io.IOBase` destructor logs ``close()`` exceptions. +* Set the :attr:`~sys.flags.dev_mode` attribute of :attr:`sys.flags` to + ``True``. + +The Python Development Mode does not enable the :mod:`tracemalloc` module by +default, because the overhead cost (to performance and memory) would be too +large. Enabling the :mod:`tracemalloc` module provides additional information +on the origin of some errors. For example, :exc:`ResourceWarning` logs the +traceback where the resource was allocated, and a buffer overflow error logs +the traceback where the memory block was allocated. + +The Python Development Mode does not prevent the :option:`-O` command line +option from removing :keyword:`assert` statements nor from setting +:const:`__debug__` to ``False``. + +.. versionchanged:: 3.8 + The :class:`io.IOBase` destructor now logs ``close()`` exceptions. + +.. versionchanged:: 3.9 + The *encoding* and *errors* arguments are now checked for string encoding + and decoding operations. + + +ResourceWarning Example +======================= + +Example of a script counting the number of lines of the text file specified in +the command line:: + + import sys + + def main(): + fp = open(sys.argv[1]) + nlines = len(fp.readlines()) + print(nlines) + # The file is closed implicitly + + if __name__ == "__main__": + main() + +The script does not close the file explicitly. By default, Python does not emit +any warning. Example using README.txt, which has 269 lines: + +.. code-block:: shell-session + + $ python3 script.py README.txt + 269 + +Enabling the Python Development Mode displays a :exc:`ResourceWarning` warning: + +.. code-block:: shell-session + + $ python3 -X dev script.py README.txt + 269 + script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'> + main() + ResourceWarning: Enable tracemalloc to get the object allocation traceback + +In addition, enabling :mod:`tracemalloc` shows the line where the file was +opened: + +.. code-block:: shell-session + + $ python3 -X dev -X tracemalloc=5 script.py README.rst + 269 + script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='README.rst' mode='r' encoding='UTF-8'> + main() + Object allocated at (most recent call last): + File "script.py", lineno 10 + main() + File "script.py", lineno 4 + fp = open(sys.argv[1]) + +The fix is to close explicitly the file. Example using a context manager:: + + def main(): + # Close the file explicitly when exiting the with block + with open(sys.argv[1]) as fp: + nlines = len(fp.readlines()) + print(nlines) + +Not closing a resource explicitly can leave a resource open for way longer than +expected; it can cause severe issues upon exiting Python. It is bad in +CPython, but it is even worse in PyPy. Closing resources explicitly makes an +application more deterministic and more reliable. + + +Bad file descriptor error example +================================= + +Script displaying the first line of itself:: + + import os + + def main(): + fp = open(__file__) + firstline = fp.readline() + print(firstline.rstrip()) + os.close(fp.fileno()) + # The file is closed implicitly + + main() + +By default, Python does not emit any warning: + +.. code-block:: shell-session + + $ python3 script.py + import os + +The Python Development Mode shows a :exc:`ResourceWarning` and logs a "Bad file +descriptor" error when finalizing the file object: + +.. code-block:: shell-session + + $ python3 script.py + import os + script.py:10: ResourceWarning: unclosed file <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'> + main() + ResourceWarning: Enable tracemalloc to get the object allocation traceback + Exception ignored in: <_io.TextIOWrapper name='script.py' mode='r' encoding='UTF-8'> + Traceback (most recent call last): + File "script.py", line 10, in <module> + main() + OSError: [Errno 9] Bad file descriptor + +``os.close(fp.fileno())`` closes the file descriptor. When the file object +finalizer tries to close the file descriptor again, it fails with the ``Bad +file descriptor`` error. A file descriptor must be closed only once. In the +worst case scenario, closing it twice can lead to a crash (see :issue:`18748` +for an example). + +The fix is to remove the ``os.close(fp.fileno())`` line, or open the file with +``closefd=False``. diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst index 52a505e..df2cda9 100644 --- a/Doc/library/exceptions.rst +++ b/Doc/library/exceptions.rst @@ -688,6 +688,10 @@ The following exceptions are used as warning categories; see the Base class for warnings about deprecated features when those warnings are intended for other Python developers. + Ignored by the default warning filters, except in the ``__main__`` module + (:pep:`565`). Enabling the :ref:`Python Development Mode <devmode>` shows + this warning. + .. exception:: PendingDeprecationWarning @@ -699,6 +703,9 @@ The following exceptions are used as warning categories; see the upcoming deprecation is unusual, and :exc:`DeprecationWarning` is preferred for already active deprecations. + Ignored by the default warning filters. Enabling the :ref:`Python + Development Mode <devmode>` shows this warning. + .. exception:: SyntaxWarning @@ -720,6 +727,9 @@ The following exceptions are used as warning categories; see the Base class for warnings about probable mistakes in module imports. + Ignored by the default warning filters. Enabling the :ref:`Python + Development Mode <devmode>` shows this warning. + .. exception:: UnicodeWarning @@ -733,8 +743,10 @@ The following exceptions are used as warning categories; see the .. exception:: ResourceWarning - Base class for warnings related to resource usage. Ignored by the default - warning filters. + Base class for warnings related to resource usage. + + Ignored by the default warning filters. Enabling the :ref:`Python + Development Mode <devmode>` shows this warning. .. versionadded:: 3.2 diff --git a/Doc/library/faulthandler.rst b/Doc/library/faulthandler.rst index b588dfa..59274c1 100644 --- a/Doc/library/faulthandler.rst +++ b/Doc/library/faulthandler.rst @@ -40,6 +40,9 @@ alternatively be passed to :func:`faulthandler.enable`. The module is implemented in C, so tracebacks can be dumped on a crash or when Python is deadlocked. +The :ref:`Python Development Mode <devmode>` calls :func:`faulthandler.enable` +at Python startup. + Dumping the traceback --------------------- diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst index 3e25faa..fd3401f 100644 --- a/Doc/library/stdtypes.rst +++ b/Doc/library/stdtypes.rst @@ -1559,8 +1559,8 @@ expression support in the :mod:`re` module). list of possible encodings, see section :ref:`standard-encodings`. By default, the *errors* argument is not checked for best performances, but - only used at the first encoding error. Enable the development mode - (:option:`-X` ``dev`` option), or use a debug build, to check *errors*. + only used at the first encoding error. Enable the :ref:`Python Development + Mode <devmode>`, or use a debug build to check *errors*. .. versionchanged:: 3.1 Support for keyword arguments added. @@ -2596,8 +2596,8 @@ arbitrary binary data. list of possible encodings, see section :ref:`standard-encodings`. By default, the *errors* argument is not checked for best performances, but - only used at the first decoding error. Enable the development mode - (:option:`-X` ``dev`` option), or use a debug build, to check *errors*. + only used at the first decoding error. Enable the :ref:`Python Development + Mode <devmode>`, or use a debug build to check *errors*. .. note:: diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 351a8e4..d28b356 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -428,9 +428,9 @@ always available. The :term:`named tuple` *flags* exposes the status of command line flags. The attributes are read only. - ============================= ============================= + ============================= ================================================================ attribute flag - ============================= ============================= + ============================= ================================================================ :const:`debug` :option:`-d` :const:`inspect` :option:`-i` :const:`interactive` :option:`-i` @@ -444,9 +444,9 @@ always available. :const:`bytes_warning` :option:`-b` :const:`quiet` :option:`-q` :const:`hash_randomization` :option:`-R` - :const:`dev_mode` :option:`-X` ``dev`` - :const:`utf8_mode` :option:`-X` ``utf8`` - ============================= ============================= + :const:`dev_mode` :option:`-X dev <-X>` (:ref:`Python Development Mode <devmode>`) + :const:`utf8_mode` :option:`-X utf8 <-X>` + ============================= ================================================================ .. versionchanged:: 3.2 Added ``quiet`` attribute for the new :option:`-q` flag. @@ -461,8 +461,9 @@ always available. Added ``isolated`` attribute for :option:`-I` ``isolated`` flag. .. versionchanged:: 3.7 - Added ``dev_mode`` attribute for the new :option:`-X` ``dev`` flag - and ``utf8_mode`` attribute for the new :option:`-X` ``utf8`` flag. + Added the ``dev_mode`` attribute for the new :ref:`Python Development + Mode <devmode>` and the ``utf8_mode`` attribute for the new :option:`-X` + ``utf8`` flag. .. data:: float_info |