merge from 3.4 - clean up the subprocess docs warning-soup and

s/Unix/POSIX/.

1 files changed, 62 insertions, 93 deletions

diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst
index 7efd4e7..e542780 100644
--- a/Doc/library/subprocess.rst
+++ b/Doc/library/subprocess.rst
@@ -54,18 +54,12 @@ use cases, the underlying :class:`Popen` interface can be used directly.
       >>> subprocess.call("exit 1", shell=True)
       1
 
-   .. warning::
-
-      Invoking the system shell with ``shell=True`` can be a security hazard
-      if combined with untrusted input. See the warning under
-      :ref:`frequently-used-arguments` for details.
-
    .. note::
 
-      Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function. As
-      the pipes are not being read in the current process, the child
-      process may block if it generates enough output to a pipe to fill up
-      the OS pipe buffer.
+      Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this
+      function.  The child process will block if it generates enough
+      output to a pipe to fill up the OS pipe buffer as the pipes are
+      not being read from.
 
    .. versionchanged:: 3.3
       *timeout* was added.
@@ -99,18 +93,12 @@ use cases, the underlying :class:`Popen` interface can be used directly.
          ...
       subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1
 
-   .. warning::
-
-      Invoking the system shell with ``shell=True`` can be a security hazard
-      if combined with untrusted input. See the warning under
-      :ref:`frequently-used-arguments` for details.
-
    .. note::
 
-      Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function. As
-      the pipes are not being read in the current process, the child
-      process may block if it generates enough output to a pipe to fill up
-      the OS pipe buffer.
+      Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this
+      function.  The child process will block if it generates enough
+      output to a pipe to fill up the OS pipe buffer as the pipes are
+      not being read from.
 
    .. versionchanged:: 3.3
       *timeout* was added.
@@ -177,17 +165,12 @@ use cases, the underlying :class:`Popen` interface can be used directly.
       ...     shell=True)
       'ls: non_existent_file: No such file or directory\n'
 
-   .. warning::
-
-      Invoking the system shell with ``shell=True`` can be a security hazard
-      if combined with untrusted input. See the warning under
-      :ref:`frequently-used-arguments` for details.
-
    .. note::
 
-      Do not use ``stderr=PIPE`` with this function. As the pipe is not being
-      read in the current process, the child process may block if it
-      generates enough output to the pipe to fill up the OS pipe buffer.
+      Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this
+      function.  The child process will block if it generates enough
+      output to a pipe to fill up the OS pipe buffer as the pipes are
+      not being read from.
 
    .. versionadded:: 3.1
 
@@ -210,7 +193,7 @@ use cases, the underlying :class:`Popen` interface can be used directly.
 
    Special value that can be used as the *stdin*, *stdout* or *stderr* argument
    to :class:`Popen` and indicates that a pipe to the standard stream should be
-   opened.
+   opened.  Most useful with :meth:`Popen.communicate`.
 
 
 .. data:: STDOUT
@@ -336,28 +319,9 @@ default values. The arguments that are most commonly needed are:
       instead of ``locale.getpreferredencoding()``.  See the
       :class:`io.TextIOWrapper` class for more information on this change.
 
-   .. warning::
-
-      Executing shell commands that incorporate unsanitized input from an
-      untrusted source makes a program vulnerable to `shell injection
-      <http://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_,
-      a serious security flaw which can result in arbitrary command execution.
-      For this reason, the use of ``shell=True`` is **strongly discouraged**
-      in cases where the command string is constructed from external input::
-
-         >>> from subprocess import call
-         >>> filename = input("What file would you like to display?\n")
-         What file would you like to display?
-         non_existent; rm -rf / #
-         >>> call("cat " + filename, shell=True) # Uh-oh. This will end badly...
-
-      ``shell=False`` disables all shell based features, but does not suffer
-      from this vulnerability; see the Note in the :class:`Popen` constructor
-      documentation for helpful hints in getting ``shell=False`` to work.
+   .. note::
 
-      When using ``shell=True``, :func:`shlex.quote` can be used to properly
-      escape whitespace and shell metacharacters in strings that are going to
-      be used to construct shell commands.
+      Read the `Security Considerations`_ section before using ``shell=True``.
 
 These options, along with all of the other options, are described in more
 detail in the :class:`Popen` constructor documentation.
@@ -378,7 +342,7 @@ functions.
                  startupinfo=None, creationflags=0, restore_signals=True, \
                  start_new_session=False, pass_fds=())
 
-   Execute a child program in a new process.  On Unix, the class uses
+   Execute a child program in a new process.  On POSIX, the class uses
    :meth:`os.execvp`-like behavior to execute the child program.  On Windows,
    the class uses the Windows ``CreateProcess()`` function.  The arguments to
    :class:`Popen` are as follows.
@@ -390,7 +354,7 @@ functions.
    arguments for additional differences from the default behavior.  Unless
    otherwise stated, it is recommended to pass *args* as a sequence.
 
-   On Unix, if *args* is a string, the string is interpreted as the name or
+   On POSIX, if *args* is a string, the string is interpreted as the name or
    path of the program to execute.  However, this can only be done if not
    passing arguments to the program.
 
@@ -421,7 +385,7 @@ functions.
    the shell as the program to execute.  If *shell* is *True*, it is
    recommended to pass *args* as a string rather than as a sequence.
 
-   On Unix with ``shell=True``, the shell defaults to :file:`/bin/sh`.  If
+   On POSIX with ``shell=True``, the shell defaults to :file:`/bin/sh`.  If
    *args* is a string, the string specifies the command
    to execute through the shell.  This means that the string must be
    formatted exactly as it would be when typed at the shell prompt.  This
@@ -438,11 +402,9 @@ functions.
    into the shell (e.g. :command:`dir` or :command:`copy`).  You do not need
    ``shell=True`` to run a batch file or console-based executable.
 
-   .. warning::
+   .. note::
 
-      Passing ``shell=True`` can be a security hazard if combined with
-      untrusted input.  See the warning under :ref:`frequently-used-arguments`
-      for details.
+      Read the `Security Considerations`_ section before using ``shell=True``.
 
    *bufsize* will be supplied as the corresponding argument to the :func:`open`
    function when creating the stdin/stdout/stderr pipe file objects: :const:`0`
@@ -463,9 +425,9 @@ functions.
    program to execute specified by *args*.  However, the original *args* is
    still passed to the program.  Most programs treat the program specified
    by *args* as the command name, which can then be different from the program
-   actually executed.  On Unix, the *args* name
+   actually executed.  On POSIX, the *args* name
    becomes the display name for the executable in utilities such as
-   :program:`ps`.  If ``shell=True``, on Unix the *executable* argument
+   :program:`ps`.  If ``shell=True``, on POSIX the *executable* argument
    specifies a replacement shell for the default :file:`/bin/sh`.
 
    *stdin*, *stdout* and *stderr* specify the executed program's standard input,
@@ -481,7 +443,7 @@ functions.
 
    If *preexec_fn* is set to a callable object, this object will be called in the
    child process just before the child is executed.
-   (Unix only)
+   (POSIX only)
 
    .. warning::
 
@@ -499,8 +461,8 @@ functions.
       common use of *preexec_fn* to call os.setsid() in the child.
 
    If *close_fds* is true, all file descriptors except :const:`0`, :const:`1` and
-   :const:`2` will be closed before the child process is executed. (Unix only).
-   The default varies by platform:  Always true on Unix.  On Windows it is
+   :const:`2` will be closed before the child process is executed. (POSIX only).
+   The default varies by platform:  Always true on POSIX.  On Windows it is
    true when *stdin*/*stdout*/*stderr* are :const:`None`, false otherwise.
    On Windows, if *close_fds* is true then no handles will be inherited by the
    child process.  Note that on Windows, you cannot set *close_fds* to true and
@@ -512,7 +474,7 @@ functions.
 
    *pass_fds* is an optional sequence of file descriptors to keep open
    between the parent and child.  Providing any *pass_fds* forces
-   *close_fds* to be :const:`True`.  (Unix only)
+   *close_fds* to be :const:`True`.  (POSIX only)
 
    .. versionadded:: 3.2
       The *pass_fds* parameter was added.
@@ -525,13 +487,13 @@ functions.
    If *restore_signals* is true (the default) all signals that Python has set to
    SIG_IGN are restored to SIG_DFL in the child process before the exec.
    Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals.
-   (Unix only)
+   (POSIX only)
 
    .. versionchanged:: 3.2
       *restore_signals* was added.
 
    If *start_new_session* is true the setsid() system call will be made in the
-   child process prior to the execution of the subprocess.  (Unix only)
+   child process prior to the execution of the subprocess.  (POSIX only)
 
    .. versionchanged:: 3.2
       *start_new_session* was added.
@@ -598,14 +560,21 @@ Exceptions defined in this module all inherit from :exc:`SubprocessError`.
       The :exc:`SubprocessError` base class was added.
 
 
-Security
-^^^^^^^^
+Security Considerations
+-----------------------
+
+Unlike some other popen functions, this implementation will never
+implicitly call a system shell.  This means that all characters,
+including shell metacharacters, can safely be passed to child processes.
+If the shell is invoked explicitly, via ``shell=True``, it is the application's
+responsibility to ensure that all whitespace and metacharacters are
+quoted appropriately to avoid
+`shell injection <http://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_
+vulnerabilities.
 
-Unlike some other popen functions, this implementation will never call a
-system shell implicitly.  This means that all characters, including shell
-metacharacters, can safely be passed to child processes. Obviously, if the
-shell is invoked explicitly, then it is the application's responsibility to
-ensure that all whitespace and metacharacters are quoted appropriately.
+When using ``shell=True``, the :func:`shlex.quote` function can be
+used to properly escape whitespace and shell metacharacters in strings
+that are going to be used to construct shell commands.
 
 
 Popen Objects
@@ -631,25 +600,25 @@ Instances of the :class:`Popen` class have the following methods:
 
    .. note::
 
+      This will deadlock when using ``stdout=PIPE`` or ``stderr=PIPE``
+      and the child process generates enough output to a pipe such that
+      it blocks waiting for the OS pipe buffer to accept more data.
+      Use :meth:`Popen.communicate` when using pipes to avoid that.
+
+   .. note::
+
       The function is implemented using a busy loop (non-blocking call and
       short sleeps). Use the :mod:`asyncio` module for an asynchronous wait:
       see :class:`asyncio.create_subprocess_exec`.
 
-   .. warning::
-
-      This will deadlock when using ``stdout=PIPE`` and/or
-      ``stderr=PIPE`` and the child process generates enough output to
-      a pipe such that it blocks waiting for the OS pipe buffer to
-      accept more data.  Use :meth:`communicate` to avoid that.
-
    .. versionchanged:: 3.3
       *timeout* was added.
 
    .. deprecated:: 3.4
 
-      Do not use the undocumented *endtime* parameter.  It is was
-      unintentionally exposed in 3.3 but was intended to be private
-      for internal use.  Use *timeout* instead.
+      Do not use the *endtime* parameter.  It is was unintentionally
+      exposed in 3.3 but was left undocumented as it was intended to be
+      private for internal use.  Use *timeout* instead.
 
 .. method:: Popen.communicate(input=None, timeout=None)
 
@@ -717,13 +686,6 @@ Instances of the :class:`Popen` class have the following methods:
 
 The following attributes are also available:
 
-.. warning::
-
-   Use :meth:`~Popen.communicate` rather than :attr:`.stdin.write <Popen.stdin>`,
-   :attr:`.stdout.read <Popen.stdout>` or :attr:`.stderr.read <Popen.stderr>` to avoid
-   deadlocks due to any of the other OS pipe buffers filling up and blocking the
-   child process.
-
 .. attribute:: Popen.args
 
    The *args* argument as it was passed to :class:`Popen` -- a
@@ -757,6 +719,13 @@ The following attributes are also available:
    ``True``, the stream is a text stream, otherwise it is a byte stream. If the
    *stderr* argument was not :data:`PIPE`, this attribute is ``None``.
 
+.. warning::
+
+   Use :meth:`~Popen.communicate` rather than :attr:`.stdin.write <Popen.stdin>`,
+   :attr:`.stdout.read <Popen.stdout>` or :attr:`.stderr.read <Popen.stderr>` to avoid
+   deadlocks due to any of the other OS pipe buffers filling up and blocking the
+   child process.
+
 
 .. attribute:: Popen.pid
 
@@ -773,7 +742,7 @@ The following attributes are also available:
    hasn't terminated yet.
 
    A negative value ``-N`` indicates that the child was terminated by signal
-   ``N`` (Unix only).
+   ``N`` (POSIX only).
 
 
 Windows Popen Helpers
@@ -1098,7 +1067,7 @@ handling consistency are valid for these functions.
       >>> subprocess.getstatusoutput('/bin/junk')
       (256, 'sh: /bin/junk: not found')
 
-   Availability: Unix & Windows
+   Availability: POSIX & Windows
 
    .. versionchanged:: 3.3.4
       Windows support added
@@ -1114,7 +1083,7 @@ handling consistency are valid for these functions.
       >>> subprocess.getoutput('ls /bin/ls')
       '/bin/ls'
 
-   Availability: Unix & Windows
+   Availability: POSIX & Windows
 
    .. versionchanged:: 3.3.4
       Windows support added