Add a subprocess.run() function than returns a CalledProcess instance for a

more consistent API than the existing call* functions.
(enhancement from issue 23342)

1 files changed, 170 insertions, 119 deletions

diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst
index 49a82bd..4fef6ea 100644
--- a/Doc/library/subprocess.rst
+++ b/Doc/library/subprocess.rst
@@ -25,160 +25,99 @@ modules and functions can be found in the following sections.
 Using the :mod:`subprocess` Module
 ----------------------------------
 
-The recommended approach to invoking subprocesses is to use the following
-convenience functions for all use cases they can handle. For more advanced
+The recommended approach to invoking subprocesses is to use the :func:`run`
+function for all use cases it can handle. For more advanced
 use cases, the underlying :class:`Popen` interface can be used directly.
 
+The :func:`run` function was added in Python 3.5; if you need to retain
+compatibility with older versions, see the :ref:`call-function-trio` section.
 
-.. function:: call(args, *, stdin=None, stdout=None, stderr=None, shell=False, timeout=None)
+
+.. function:: run(args, *, stdin=None, input=None, stdout=None, stderr=None,\
+                  shell=False, timeout=None, check=False)
 
    Run the command described by *args*.  Wait for command to complete, then
-   return the :attr:`returncode` attribute.
+   return a :class:`CompletedProcess` instance.
 
    The arguments shown above are merely the most common ones, described below
    in :ref:`frequently-used-arguments` (hence the use of keyword-only notation
    in the abbreviated signature). The full function signature is largely the
-   same as that of the :class:`Popen` constructor - this function passes all
-   supplied arguments other than *timeout* directly through to that interface.
+   same as that of the :class:`Popen` constructor - apart from *timeout*,
+   *input* and *check*, all the arguments to this function are passed through to
+   that interface.
 
-   The *timeout* argument is passed to :meth:`Popen.wait`. If the timeout
-   expires, the child process will be killed and then waited for again.  The
+   This does not capture stdout or stderr by default. To do so, pass
+   :data:`PIPE` for the *stdout* and/or *stderr* arguments.
+
+   The *timeout* argument is passed to :meth:`Popen.communicate`. If the timeout
+   expires, the child process will be killed and waited for.  The
    :exc:`TimeoutExpired` exception will be re-raised after the child process
    has terminated.
 
-   Examples::
-
-      >>> subprocess.call(["ls", "-l"])
-      0
-
-      >>> subprocess.call("exit 1", shell=True)
-      1
-
-   .. note::
-
-      Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this
-      function.  The child process will block if it generates enough
-      output to a pipe to fill up the OS pipe buffer as the pipes are
-      not being read from.
-
-   .. versionchanged:: 3.3
-      *timeout* was added.
-
-
-.. function:: check_call(args, *, stdin=None, stdout=None, stderr=None, shell=False, timeout=None)
-
-   Run command with arguments.  Wait for command to complete. If the return
-   code was zero then return, otherwise raise :exc:`CalledProcessError`. The
-   :exc:`CalledProcessError` object will have the return code in the
-   :attr:`~CalledProcessError.returncode` attribute.
-
-   The arguments shown above are merely the most common ones, described below
-   in :ref:`frequently-used-arguments` (hence the use of keyword-only notation
-   in the abbreviated signature). The full function signature is largely the
-   same as that of the :class:`Popen` constructor - this function passes all
-   supplied arguments other than *timeout* directly through to that interface.
+   The *input* argument is passed to :meth:`Popen.communicate` and thus to the
+   subprocess's stdin.  If used it must be a byte sequence, or a string if
+   ``universal_newlines=True``.  When used, the internal :class:`Popen` object
+   is automatically created with ``stdin=PIPE``, and the *stdin* argument may
+   not be used as well.
 
-   The *timeout* argument is passed to :meth:`Popen.wait`. If the timeout
-   expires, the child process will be killed and then waited for again.  The
-   :exc:`TimeoutExpired` exception will be re-raised after the child process
-   has terminated.
+   If *check* is True, and the process exits with a non-zero exit code, a
+   :exc:`CalledProcessError` exception will be raised. Attributes of that
+   exception hold the arguments, the exit code, and stdout and stderr if they
+   were captured.
 
    Examples::
 
-      >>> subprocess.check_call(["ls", "-l"])
-      0
+      >>> subprocess.run(["ls", "-l"])  # doesn't capture output
+      CompletedProcess(args=['ls', '-l'], returncode=0)
 
-      >>> subprocess.check_call("exit 1", shell=True)
+      >>> subprocess.run("exit 1", shell=True, check=True)
       Traceback (most recent call last):
-         ...
+        ...
       subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1
 
-   .. note::
+      >>> subprocess.run(["ls", "-l", "/dev/null"], stdout=subprocess.PIPE)
+      CompletedProcess(args=['ls', '-l', '/dev/null'], returncode=0,
+      stdout=b'crw-rw-rw- 1 root root 1, 3 Jan 23 16:23 /dev/null\n')
 
-      Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this
-      function.  The child process will block if it generates enough
-      output to a pipe to fill up the OS pipe buffer as the pipes are
-      not being read from.
+   .. versionadded:: 3.5
 
-   .. versionchanged:: 3.3
-      *timeout* was added.
+.. class:: CompletedProcess
 
+   The return value from :func:`run`, representing a process that has finished.
 
-.. function:: check_output(args, *, input=None, stdin=None, stderr=None, shell=False, universal_newlines=False, timeout=None)
+   .. attribute:: args
 
-   Run command with arguments and return its output.
+      The arguments used to launch the process. This may be a list or a string.
 
-   If the return code was non-zero it raises a :exc:`CalledProcessError`. The
-   :exc:`CalledProcessError` object will have the return code in the
-   :attr:`~CalledProcessError.returncode` attribute and any output in the
-   :attr:`~CalledProcessError.output` attribute.
+   .. attribute:: returncode
 
-   The arguments shown above are merely the most common ones, described below
-   in :ref:`frequently-used-arguments` (hence the use of keyword-only notation
-   in the abbreviated signature). The full function signature is largely the
-   same as that of the :class:`Popen` constructor - this functions passes all
-   supplied arguments other than *input* and *timeout* directly through to
-   that interface.  In addition, *stdout* is not permitted as an argument, as
-   it is used internally to collect the output from the subprocess.
+      Exit status of the child process. Typically, an exit status of 0 indicates
+      that it ran successfully.
 
-   The *timeout* argument is passed to :meth:`Popen.wait`. If the timeout
-   expires, the child process will be killed and then waited for again.  The
-   :exc:`TimeoutExpired` exception will be re-raised after the child process
-   has terminated.
+      A negative value ``-N`` indicates that the child was terminated by signal
+      ``N`` (POSIX only).
 
-   The *input* argument is passed to :meth:`Popen.communicate` and thus to the
-   subprocess's stdin.  If used it must be a byte sequence, or a string if
-   ``universal_newlines=True``.  When used, the internal :class:`Popen` object
-   is automatically created with ``stdin=PIPE``, and the *stdin* argument may
-   not be used as well.
+   .. attribute:: stdout
 
-   Examples::
+      Captured stdout from the child process. A bytes sequence, or a string if
+      :func:`run` was called with ``universal_newlines=True``. None if stdout
+      was not captured.
 
-      >>> subprocess.check_output(["echo", "Hello World!"])
-      b'Hello World!\n'
+      If you ran the process with ``stderr=subprocess.STDOUT``, stdout and
+      stderr will be combined in this attribute, and :attr:`stderr` will be
+      None.
 
-      >>> subprocess.check_output(["echo", "Hello World!"], universal_newlines=True)
-      'Hello World!\n'
+   .. attribute:: stderr
 
-      >>> subprocess.check_output(["sed", "-e", "s/foo/bar/"],
-      ...                         input=b"when in the course of fooman events\n")
-      b'when in the course of barman events\n'
+      Captured stderr from the child process. A bytes sequence, or a string if
+      :func:`run` was called with ``universal_newlines=True``. None if stderr
+      was not captured.
 
-      >>> subprocess.check_output("exit 1", shell=True)
-      Traceback (most recent call last):
-         ...
-      subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1
+   .. method:: check_returncode()
 
-   By default, this function will return the data as encoded bytes. The actual
-   encoding of the output data may depend on the command being invoked, so the
-   decoding to text will often need to be handled at the application level.
+      If :attr:`returncode` is non-zero, raise a :exc:`CalledProcessError`.
 
-   This behaviour may be overridden by setting *universal_newlines* to
-   ``True`` as described below in :ref:`frequently-used-arguments`.
-
-   To also capture standard error in the result, use
-   ``stderr=subprocess.STDOUT``::
-
-      >>> subprocess.check_output(
-      ...     "ls non_existent_file; exit 0",
-      ...     stderr=subprocess.STDOUT,
-      ...     shell=True)
-      'ls: non_existent_file: No such file or directory\n'
-
-   .. note::
-
-      Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this
-      function.  The child process will block if it generates enough
-      output to a pipe to fill up the OS pipe buffer as the pipes are
-      not being read from.
-
-   .. versionadded:: 3.1
-
-   .. versionchanged:: 3.3
-      *timeout* was added.
-
-   .. versionchanged:: 3.4
-      *input* was added.
+   .. versionadded:: 3.5
 
 .. data:: DEVNULL
 
@@ -225,11 +164,22 @@ use cases, the underlying :class:`Popen` interface can be used directly.
 
     .. attribute:: output
 
-        Output of the child process if this exception is raised by
+        Output of the child process if it was captured by :func:`run` or
         :func:`check_output`.  Otherwise, ``None``.
 
+    .. attribute:: stdout
+
+        Alias for output, for symmetry with :attr:`stderr`.
+
+    .. attribute:: stderr
+
+        Stderr output of the child process if it was captured by :func:`run`.
+        Otherwise, ``None``.
+
     .. versionadded:: 3.3
 
+    .. versionchanged:: 3.5
+        *stdout* and *stderr* attributes added
 
 .. exception:: CalledProcessError
 
@@ -246,9 +196,20 @@ use cases, the underlying :class:`Popen` interface can be used directly.
 
     .. attribute:: output
 
-        Output of the child process if this exception is raised by
+        Output of the child process if it was captured by :func:`run` or
         :func:`check_output`.  Otherwise, ``None``.
 
+    .. attribute:: stdout
+
+        Alias for output, for symmetry with :attr:`stderr`.
+
+    .. attribute:: stderr
+
+        Stderr output of the child process if it was captured by :func:`run`.
+        Otherwise, ``None``.
+
+    .. versionchanged:: 3.5
+        *stdout* and *stderr* attributes added
 
 
 .. _frequently-used-arguments:
@@ -852,6 +813,96 @@ The :mod:`subprocess` module exposes the following constants.
 
    This flag is ignored if :data:`CREATE_NEW_CONSOLE` is specified.
 
+.. _call-function-trio:
+
+Older high-level API
+--------------------
+
+Prior to Python 3.5, these three functions comprised the high level API to
+subprocess. You can now use :func:`run` in many cases, but lots of existing code
+calls these functions.
+
+.. function:: call(args, *, stdin=None, stdout=None, stderr=None, shell=False, timeout=None)
+
+   Run the command described by *args*.  Wait for command to complete, then
+   return the :attr:`returncode` attribute.
+
+   This is equivalent to::
+
+       run(...).returncode
+
+   (except that the *input* and *check* parameters are not supported)
+
+   .. note::
+
+      Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this
+      function.  The child process will block if it generates enough
+      output to a pipe to fill up the OS pipe buffer as the pipes are
+      not being read from.
+
+   .. versionchanged:: 3.3
+      *timeout* was added.
+
+.. function:: check_call(args, *, stdin=None, stdout=None, stderr=None, shell=False, timeout=None)
+
+   Run command with arguments.  Wait for command to complete. If the return
+   code was zero then return, otherwise raise :exc:`CalledProcessError`. The
+   :exc:`CalledProcessError` object will have the return code in the
+   :attr:`~CalledProcessError.returncode` attribute.
+
+   This is equivalent to::
+
+       run(..., check=True)
+
+   (except that the *input* parameter is not supported)
+
+   .. note::
+
+      Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this
+      function.  The child process will block if it generates enough
+      output to a pipe to fill up the OS pipe buffer as the pipes are
+      not being read from.
+
+   .. versionchanged:: 3.3
+      *timeout* was added.
+
+
+.. function:: check_output(args, *, input=None, stdin=None, stderr=None, shell=False, universal_newlines=False, timeout=None)
+
+   Run command with arguments and return its output.
+
+   If the return code was non-zero it raises a :exc:`CalledProcessError`. The
+   :exc:`CalledProcessError` object will have the return code in the
+   :attr:`~CalledProcessError.returncode` attribute and any output in the
+   :attr:`~CalledProcessError.output` attribute.
+
+   This is equivalent to::
+
+       run(..., check=True, stdout=PIPE).stdout
+
+   By default, this function will return the data as encoded bytes. The actual
+   encoding of the output data may depend on the command being invoked, so the
+   decoding to text will often need to be handled at the application level.
+
+   This behaviour may be overridden by setting *universal_newlines* to
+   ``True`` as described above in :ref:`frequently-used-arguments`.
+
+   To also capture standard error in the result, use
+   ``stderr=subprocess.STDOUT``::
+
+      >>> subprocess.check_output(
+      ...     "ls non_existent_file; exit 0",
+      ...     stderr=subprocess.STDOUT,
+      ...     shell=True)
+      'ls: non_existent_file: No such file or directory\n'
+
+   .. versionadded:: 3.1
+
+   .. versionchanged:: 3.3
+      *timeout* was added.
+
+   .. versionchanged:: 3.4
+      *input* was added.
 
 .. _subprocess-replacements: