diff options
Diffstat (limited to 'Doc/library/subprocess.rst')
| -rw-r--r-- | Doc/library/subprocess.rst | 1134 |
1 files changed, 295 insertions, 839 deletions
diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index f2e5463..53a0161 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -1,15 +1,14 @@ + :mod:`subprocess` --- Subprocess management =========================================== .. module:: subprocess :synopsis: Subprocess management. - .. moduleauthor:: Peter Åstrand <astrand@lysator.liu.se> .. sectionauthor:: Peter Åstrand <astrand@lysator.liu.se> -**Source code:** :source:`Lib/subprocess.py` --------------- +.. versionadded:: 2.4 The :mod:`subprocess` module allows you to spawn new processes, connect to their input/output/error pipes, and obtain their return codes. This module intends to @@ -17,209 +16,173 @@ replace several older modules and functions:: os.system os.spawn* + os.popen* + popen2.* + commands.* -Information about how the :mod:`subprocess` module can be used to replace these -modules and functions can be found in the following sections. +Information about how this module can be used to replace the older +functions can be found in the subprocess-replacements_ section. .. seealso:: + POSIX users (Linux, BSD, etc.) are strongly encouraged to install + and use the much more recent subprocess32_ module instead of the + version included with python 2.7. It is a drop in replacement with + better behavior in many situations. + :pep:`324` -- PEP proposing the subprocess module +.. _subprocess32: https://pypi.org/project/subprocess32/ Using the :mod:`subprocess` Module ---------------------------------- -The recommended approach to invoking subprocesses is to use the :func:`run` -function for all use cases it can handle. For more advanced use cases, the -underlying :class:`Popen` interface can be used directly. +The recommended way to launch subprocesses is to use the following +convenience functions. For more advanced use cases when these do not +meet your needs, use the underlying :class:`Popen` interface. -The :func:`run` function was added in Python 3.5; if you need to retain -compatibility with older versions, see the :ref:`call-function-trio` section. - -.. function:: run(args, *, stdin=None, input=None, stdout=None, stderr=None,\ - capture_output=False, shell=False, cwd=None, timeout=None, \ - check=False, encoding=None, errors=None, text=None, env=None, \ - universal_newlines=None) +.. function:: call(args, *, stdin=None, stdout=None, stderr=None, shell=False) Run the command described by *args*. Wait for command to complete, then - return a :class:`CompletedProcess` instance. + return the :attr:`returncode` attribute. The arguments shown above are merely the most common ones, described below - in :ref:`frequently-used-arguments` (hence the use of keyword-only notation - in the abbreviated signature). The full function signature is largely the - same as that of the :class:`Popen` constructor - most of the arguments to - this function are passed through to that interface. (*timeout*, *input*, - *check*, and *capture_output* are not.) - - If *capture_output* is true, stdout and stderr will be captured. - When used, the internal :class:`Popen` object is automatically created with - ``stdout=PIPE`` and ``stderr=PIPE``. The *stdout* and *stderr* arguments may - not be supplied at the same time as *capture_output*. If you wish to capture - and combine both streams into one, use ``stdout=PIPE`` and ``stderr=STDOUT`` - instead of *capture_output*. - - The *timeout* argument is passed to :meth:`Popen.communicate`. If the timeout - expires, the child process will be killed and waited for. The - :exc:`TimeoutExpired` exception will be re-raised after the child process - has terminated. - - The *input* argument is passed to :meth:`Popen.communicate` and thus to the - subprocess's stdin. If used it must be a byte sequence, or a string if - *encoding* or *errors* is specified or *text* is true. When - used, the internal :class:`Popen` object is automatically created with - ``stdin=PIPE``, and the *stdin* argument may not be used as well. - - If *check* is true, and the process exits with a non-zero exit code, a - :exc:`CalledProcessError` exception will be raised. Attributes of that - exception hold the arguments, the exit code, and stdout and stderr if they - were captured. - - If *encoding* or *errors* are specified, or *text* is true, - file objects for stdin, stdout and stderr are opened in text mode using the - specified *encoding* and *errors* or the :class:`io.TextIOWrapper` default. - The *universal_newlines* argument is equivalent to *text* and is provided - for backwards compatibility. By default, file objects are opened in binary mode. - - If *env* is not ``None``, it must be a mapping that defines the environment - variables for the new process; these are used instead of the default - behavior of inheriting the current process' environment. It is passed directly - to :class:`Popen`. + in :ref:`frequently-used-arguments` (hence the slightly odd notation in + the abbreviated signature). The full function signature is the same as + that of the :class:`Popen` constructor - this functions passes all + supplied arguments directly through to that interface. Examples:: - >>> subprocess.run(["ls", "-l"]) # doesn't capture output - CompletedProcess(args=['ls', '-l'], returncode=0) - - >>> subprocess.run("exit 1", shell=True, check=True) - Traceback (most recent call last): - ... - subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1 - - >>> subprocess.run(["ls", "-l", "/dev/null"], capture_output=True) - CompletedProcess(args=['ls', '-l', '/dev/null'], returncode=0, - stdout=b'crw-rw-rw- 1 root root 1, 3 Jan 23 16:23 /dev/null\n', stderr=b'') + >>> subprocess.call(["ls", "-l"]) + 0 - .. versionadded:: 3.5 + >>> subprocess.call("exit 1", shell=True) + 1 - .. versionchanged:: 3.6 - - Added *encoding* and *errors* parameters - - .. versionchanged:: 3.7 - - Added the *text* parameter, as a more understandable alias of *universal_newlines*. - Added the *capture_output* parameter. - -.. class:: CompletedProcess - - The return value from :func:`run`, representing a process that has finished. - - .. attribute:: args - - The arguments used to launch the process. This may be a list or a string. - - .. attribute:: returncode + .. warning:: - Exit status of the child process. Typically, an exit status of 0 indicates - that it ran successfully. + Using ``shell=True`` can be a security hazard. See the warning + under :ref:`frequently-used-arguments` for details. - A negative value ``-N`` indicates that the child was terminated by signal - ``N`` (POSIX only). + .. note:: - .. attribute:: stdout + Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function + as that can deadlock based on the child process output volume. + Use :class:`Popen` with the :meth:`communicate` method when you + need pipes. - Captured stdout from the child process. A bytes sequence, or a string if - :func:`run` was called with an encoding, errors, or text=True. - ``None`` if stdout was not captured. - If you ran the process with ``stderr=subprocess.STDOUT``, stdout and - stderr will be combined in this attribute, and :attr:`stderr` will be - ``None``. +.. function:: check_call(args, *, stdin=None, stdout=None, stderr=None, shell=False) - .. attribute:: stderr + Run command with arguments. Wait for command to complete. If the return + code was zero then return, otherwise raise :exc:`CalledProcessError`. The + :exc:`CalledProcessError` object will have the return code in the + :attr:`~CalledProcessError.returncode` attribute. - Captured stderr from the child process. A bytes sequence, or a string if - :func:`run` was called with an encoding, errors, or text=True. - ``None`` if stderr was not captured. + The arguments shown above are merely the most common ones, described below + in :ref:`frequently-used-arguments` (hence the slightly odd notation in + the abbreviated signature). The full function signature is the same as + that of the :class:`Popen` constructor - this functions passes all + supplied arguments directly through to that interface. - .. method:: check_returncode() + Examples:: - If :attr:`returncode` is non-zero, raise a :exc:`CalledProcessError`. + >>> subprocess.check_call(["ls", "-l"]) + 0 - .. versionadded:: 3.5 + >>> subprocess.check_call("exit 1", shell=True) + Traceback (most recent call last): + ... + subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1 -.. data:: DEVNULL + .. versionadded:: 2.5 - Special value that can be used as the *stdin*, *stdout* or *stderr* argument - to :class:`Popen` and indicates that the special file :data:`os.devnull` - will be used. + .. warning:: - .. versionadded:: 3.3 + Using ``shell=True`` can be a security hazard. See the warning + under :ref:`frequently-used-arguments` for details. + .. note:: -.. data:: PIPE + Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function + as that can deadlock based on the child process output volume. + Use :class:`Popen` with the :meth:`communicate` method when you + need pipes. - Special value that can be used as the *stdin*, *stdout* or *stderr* argument - to :class:`Popen` and indicates that a pipe to the standard stream should be - opened. Most useful with :meth:`Popen.communicate`. +.. function:: check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False) -.. data:: STDOUT - - Special value that can be used as the *stderr* argument to :class:`Popen` and - indicates that standard error should go into the same handle as standard - output. + Run command with arguments and return its output as a byte string. + If the return code was non-zero it raises a :exc:`CalledProcessError`. The + :exc:`CalledProcessError` object will have the return code in the + :attr:`~CalledProcessError.returncode` attribute and any output in the + :attr:`~CalledProcessError.output` attribute. -.. exception:: SubprocessError + The arguments shown above are merely the most common ones, described below + in :ref:`frequently-used-arguments` (hence the slightly odd notation in + the abbreviated signature). The full function signature is largely the + same as that of the :class:`Popen` constructor, except that *stdout* is + not permitted as it is used internally. All other supplied arguments are + passed directly through to the :class:`Popen` constructor. - Base class for all other exceptions from this module. + Examples:: - .. versionadded:: 3.3 + >>> subprocess.check_output(["echo", "Hello World!"]) + 'Hello World!\n' + >>> subprocess.check_output("exit 1", shell=True) + Traceback (most recent call last): + ... + subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1 -.. exception:: TimeoutExpired + To also capture standard error in the result, use + ``stderr=subprocess.STDOUT``:: - Subclass of :exc:`SubprocessError`, raised when a timeout expires - while waiting for a child process. + >>> subprocess.check_output( + ... "ls non_existent_file; exit 0", + ... stderr=subprocess.STDOUT, + ... shell=True) + 'ls: non_existent_file: No such file or directory\n' - .. attribute:: cmd + .. versionadded:: 2.7 - Command that was used to spawn the child process. + .. warning:: - .. attribute:: timeout + Using ``shell=True`` can be a security hazard. See the warning + under :ref:`frequently-used-arguments` for details. - Timeout in seconds. + .. note:: - .. attribute:: output + Do not use ``stderr=PIPE`` with this function as that can deadlock + based on the child process error volume. Use :class:`Popen` with + the :meth:`communicate` method when you need a stderr pipe. - Output of the child process if it was captured by :func:`run` or - :func:`check_output`. Otherwise, ``None``. - .. attribute:: stdout +.. data:: PIPE - Alias for output, for symmetry with :attr:`stderr`. + Special value that can be used as the *stdin*, *stdout* or *stderr* argument + to :class:`Popen` and indicates that a pipe to the standard stream should be + opened. - .. attribute:: stderr - Stderr output of the child process if it was captured by :func:`run`. - Otherwise, ``None``. +.. data:: STDOUT - .. versionadded:: 3.3 + Special value that can be used as the *stderr* argument to :class:`Popen` and + indicates that standard error should go into the same handle as standard + output. - .. versionchanged:: 3.5 - *stdout* and *stderr* attributes added .. exception:: CalledProcessError - Subclass of :exc:`SubprocessError`, raised when a process run by - :func:`check_call` or :func:`check_output` returns a non-zero exit status. + Exception raised when a process run by :func:`check_call` or + :func:`check_output` returns a non-zero exit status. .. attribute:: returncode - Exit status of the child process. If the process exited due to a - signal, this will be the negative signal number. + Exit status of the child process. .. attribute:: cmd @@ -227,20 +190,9 @@ compatibility with older versions, see the :ref:`call-function-trio` section. .. attribute:: output - Output of the child process if it was captured by :func:`run` or + Output of the child process if this exception is raised by :func:`check_output`. Otherwise, ``None``. - .. attribute:: stdout - - Alias for output, for symmetry with :attr:`stderr`. - - .. attribute:: stderr - - Stderr output of the child process if it was captured by :func:`run`. - Otherwise, ``None``. - - .. versionchanged:: 3.5 - *stdout* and *stderr* attributes added .. _frequently-used-arguments: @@ -263,44 +215,20 @@ default values. The arguments that are most commonly needed are: *stdin*, *stdout* and *stderr* specify the executed program's standard input, standard output and standard error file handles, respectively. Valid values - are :data:`PIPE`, :data:`DEVNULL`, an existing file descriptor (a positive - integer), an existing file object, and ``None``. :data:`PIPE` indicates - that a new pipe to the child should be created. :data:`DEVNULL` indicates - that the special file :data:`os.devnull` will be used. With the default - settings of ``None``, no redirection will occur; the child's file handles - will be inherited from the parent. Additionally, *stderr* can be - :data:`STDOUT`, which indicates that the stderr data from the child - process should be captured into the same file handle as for *stdout*. + are :data:`PIPE`, an existing file descriptor (a positive integer), an + existing file object, and ``None``. :data:`PIPE` indicates that a new pipe + to the child should be created. With the default settings of ``None``, no + redirection will occur; the child's file handles will be inherited from the + parent. Additionally, *stderr* can be :data:`STDOUT`, which indicates that + the stderr data from the child process should be captured into the same file + handle as for stdout. .. index:: single: universal newlines; subprocess module - If *encoding* or *errors* are specified, or *text* (also known as - *universal_newlines*) is true, - the file objects *stdin*, *stdout* and *stderr* will be opened in text - mode using the *encoding* and *errors* specified in the call or the - defaults for :class:`io.TextIOWrapper`. - - For *stdin*, line ending characters ``'\n'`` in the input will be converted - to the default line separator :data:`os.linesep`. For *stdout* and *stderr*, - all line endings in the output will be converted to ``'\n'``. For more - information see the documentation of the :class:`io.TextIOWrapper` class - when the *newline* argument to its constructor is ``None``. - - If text mode is not used, *stdin*, *stdout* and *stderr* will be opened as - binary streams. No encoding or line ending conversion is performed. - - .. versionadded:: 3.6 - Added *encoding* and *errors* parameters. - - .. versionadded:: 3.7 - Added the *text* parameter as an alias for *universal_newlines*. - - .. note:: - - The newlines attribute of the file objects :attr:`Popen.stdin`, - :attr:`Popen.stdout` and :attr:`Popen.stderr` are not updated by - the :meth:`Popen.communicate` method. + When *stdout* or *stderr* are pipes and *universal_newlines* is + ``True`` then all line endings will be converted to ``'\n'`` as described + for the :term:`universal newlines` ``'U'`` mode argument to :func:`open`. If *shell* is ``True``, the specified command will be executed through the shell. This can be useful if you are using Python primarily for the @@ -312,15 +240,28 @@ default values. The arguments that are most commonly needed are: :mod:`fnmatch`, :func:`os.walk`, :func:`os.path.expandvars`, :func:`os.path.expanduser`, and :mod:`shutil`). - .. versionchanged:: 3.3 - When *universal_newlines* is ``True``, the class uses the encoding - :func:`locale.getpreferredencoding(False) <locale.getpreferredencoding>` - instead of ``locale.getpreferredencoding()``. See the - :class:`io.TextIOWrapper` class for more information on this change. + .. warning:: - .. note:: + Executing shell commands that incorporate unsanitized input from an + untrusted source makes a program vulnerable to `shell injection + <http://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_, + a serious security flaw which can result in arbitrary command execution. + For this reason, the use of ``shell=True`` is **strongly discouraged** + in cases where the command string is constructed from external input:: + + >>> from subprocess import call + >>> filename = input("What file would you like to display?\n") + What file would you like to display? + non_existent; rm -rf / # + >>> call("cat " + filename, shell=True) # Uh-oh. This will end badly... - Read the `Security Considerations`_ section before using ``shell=True``. + ``shell=False`` disables all shell based features, but does not suffer + from this vulnerability; see the Note in the :class:`Popen` constructor + documentation for helpful hints in getting ``shell=False`` to work. + + When using ``shell=True``, :func:`pipes.quote` can be used to properly + escape whitespace and shell metacharacters in strings that are going to + be used to construct shell commands. These options, along with all of the other options, are described in more detail in the :class:`Popen` constructor documentation. @@ -335,28 +276,24 @@ are able to handle the less common cases not covered by the convenience functions. -.. class:: Popen(args, bufsize=-1, executable=None, stdin=None, stdout=None, \ - stderr=None, preexec_fn=None, close_fds=True, shell=False, \ - cwd=None, env=None, universal_newlines=None, \ - startupinfo=None, creationflags=0, restore_signals=True, \ - start_new_session=False, pass_fds=(), \*, group=None, \ - extra_groups=None, user=None, umask=-1, \ - encoding=None, errors=None, text=None) +.. class:: Popen(args, bufsize=0, executable=None, stdin=None, stdout=None, \ + stderr=None, preexec_fn=None, close_fds=False, shell=False, \ + cwd=None, env=None, universal_newlines=False, \ + startupinfo=None, creationflags=0) - Execute a child program in a new process. On POSIX, the class uses + Execute a child program in a new process. On Unix, the class uses :meth:`os.execvp`-like behavior to execute the child program. On Windows, the class uses the Windows ``CreateProcess()`` function. The arguments to :class:`Popen` are as follows. - *args* should be a sequence of program arguments or else a single string - or :term:`path-like object`. + *args* should be a sequence of program arguments or else a single string. By default, the program to execute is the first item in *args* if *args* is a sequence. If *args* is a string, the interpretation is platform-dependent and described below. See the *shell* and *executable* arguments for additional differences from the default behavior. Unless otherwise stated, it is recommended to pass *args* as a sequence. - On POSIX, if *args* is a string, the string is interpreted as the name or + On Unix, if *args* is a string, the string is interpreted as the name or path of the program to execute. However, this can only be done if not passing arguments to the program. @@ -366,10 +303,10 @@ functions. tokenization for *args*, especially in complex cases:: >>> import shlex, subprocess - >>> command_line = input() + >>> command_line = raw_input() /bin/vikings -input eggs.txt -output "spam spam.txt" -cmd "echo '$MONEY'" >>> args = shlex.split(command_line) - >>> print(args) + >>> print args ['/bin/vikings', '-input', 'eggs.txt', '-output', 'spam spam.txt', '-cmd', "echo '$MONEY'"] >>> p = subprocess.Popen(args) # Success! @@ -383,20 +320,11 @@ functions. manner described in :ref:`converting-argument-sequence`. This is because the underlying ``CreateProcess()`` operates on strings. - .. versionchanged:: 3.6 - *args* parameter accepts a :term:`path-like object` if *shell* is - ``False`` and a sequence containing path-like objects on POSIX. - - .. versionchanged:: 3.8 - *args* parameter accepts a :term:`path-like object` if *shell* is - ``False`` and a sequence containing bytes and path-like objects - on Windows. - The *shell* argument (which defaults to ``False``) specifies whether to use the shell as the program to execute. If *shell* is ``True``, it is recommended to pass *args* as a string rather than as a sequence. - On POSIX with ``shell=True``, the shell defaults to :file:`/bin/sh`. If + On Unix with ``shell=True``, the shell defaults to :file:`/bin/sh`. If *args* is a string, the string specifies the command to execute through the shell. This means that the string must be formatted exactly as it would be when typed at the shell prompt. This @@ -413,246 +341,97 @@ functions. into the shell (e.g. :command:`dir` or :command:`copy`). You do not need ``shell=True`` to run a batch file or console-based executable. - .. note:: + .. warning:: - Read the `Security Considerations`_ section before using ``shell=True``. + Passing ``shell=True`` can be a security hazard if combined with + untrusted input. See the warning under :ref:`frequently-used-arguments` + for details. - *bufsize* will be supplied as the corresponding argument to the - :func:`open` function when creating the stdin/stdout/stderr pipe - file objects: + *bufsize*, if given, has the same meaning as the corresponding argument to the + built-in open() function: :const:`0` means unbuffered, :const:`1` means line + buffered, any other positive value means use a buffer of (approximately) that + size. A negative *bufsize* means to use the system default, which usually means + fully buffered. The default value for *bufsize* is :const:`0` (unbuffered). - - :const:`0` means unbuffered (read and write are one - system call and can return short) - - :const:`1` means line buffered - (only usable if ``universal_newlines=True`` i.e., in a text mode) - - any other positive value means use a buffer of approximately that - size - - negative bufsize (the default) means the system default of - io.DEFAULT_BUFFER_SIZE will be used. + .. note:: - .. versionchanged:: 3.3.1 - *bufsize* now defaults to -1 to enable buffering by default to match the - behavior that most code expects. In versions prior to Python 3.2.4 and - 3.3.1 it incorrectly defaulted to :const:`0` which was unbuffered - and allowed short reads. This was unintentional and did not match the - behavior of Python 2 as most code expected. + If you experience performance issues, it is recommended that you try to + enable buffering by setting *bufsize* to either -1 or a large enough + positive value (such as 4096). The *executable* argument specifies a replacement program to execute. It is very seldom needed. When ``shell=False``, *executable* replaces the program to execute specified by *args*. However, the original *args* is still passed to the program. Most programs treat the program specified by *args* as the command name, which can then be different from the program - actually executed. On POSIX, the *args* name + actually executed. On Unix, the *args* name becomes the display name for the executable in utilities such as - :program:`ps`. If ``shell=True``, on POSIX the *executable* argument + :program:`ps`. If ``shell=True``, on Unix the *executable* argument specifies a replacement shell for the default :file:`/bin/sh`. - .. versionchanged:: 3.6 - *executable* parameter accepts a :term:`path-like object` on POSIX. - - .. versionchanged:: 3.8 - *executable* parameter accepts a bytes and :term:`path-like object` - on Windows. - *stdin*, *stdout* and *stderr* specify the executed program's standard input, standard output and standard error file handles, respectively. Valid values - are :data:`PIPE`, :data:`DEVNULL`, an existing file descriptor (a positive - integer), an existing :term:`file object`, and ``None``. :data:`PIPE` - indicates that a new pipe to the child should be created. :data:`DEVNULL` - indicates that the special file :data:`os.devnull` will be used. With the - default settings of ``None``, no redirection will occur; the child's file - handles will be inherited from the parent. Additionally, *stderr* can be - :data:`STDOUT`, which indicates that the stderr data from the applications - should be captured into the same file handle as for stdout. + are :data:`PIPE`, an existing file descriptor (a positive integer), an + existing file object, and ``None``. :data:`PIPE` indicates that a new pipe + to the child should be created. With the default settings of ``None``, no + redirection will occur; the child's file handles will be inherited from the + parent. Additionally, *stderr* can be :data:`STDOUT`, which indicates that + the stderr data from the child process should be captured into the same file + handle as for stdout. If *preexec_fn* is set to a callable object, this object will be called in the - child process just before the child is executed. - (POSIX only) - - .. warning:: - - The *preexec_fn* parameter is not safe to use in the presence of threads - in your application. The child process could deadlock before exec is - called. - If you must use it, keep it trivial! Minimize the number of libraries - you call into. - - .. note:: - - If you need to modify the environment for the child use the *env* - parameter rather than doing it in a *preexec_fn*. - The *start_new_session* parameter can take the place of a previously - common use of *preexec_fn* to call os.setsid() in the child. - - .. versionchanged:: 3.8 - - The *preexec_fn* parameter is no longer supported in subinterpreters. - The use of the parameter in a subinterpreter raises - :exc:`RuntimeError`. The new restriction may affect applications that - are deployed in mod_wsgi, uWSGI, and other embedded environments. + child process just before the child is executed. (Unix only) If *close_fds* is true, all file descriptors except :const:`0`, :const:`1` and - :const:`2` will be closed before the child process is executed. Otherwise - when *close_fds* is false, file descriptors obey their inheritable flag - as described in :ref:`fd_inheritance`. - - On Windows, if *close_fds* is true then no handles will be inherited by the - child process unless explicitly passed in the ``handle_list`` element of - :attr:`STARTUPINFO.lpAttributeList`, or by standard handle redirection. - - .. versionchanged:: 3.2 - The default for *close_fds* was changed from :const:`False` to - what is described above. - - .. versionchanged:: 3.7 - On Windows the default for *close_fds* was changed from :const:`False` to - :const:`True` when redirecting the standard handles. It's now possible to - set *close_fds* to :const:`True` when redirecting the standard handles. - - *pass_fds* is an optional sequence of file descriptors to keep open - between the parent and child. Providing any *pass_fds* forces - *close_fds* to be :const:`True`. (POSIX only) - - .. versionchanged:: 3.2 - The *pass_fds* parameter was added. - - If *cwd* is not ``None``, the function changes the working directory to - *cwd* before executing the child. *cwd* can be a string, bytes or - :term:`path-like <path-like object>` object. In particular, the function - looks for *executable* (or for the first item in *args*) relative to *cwd* - if the executable path is a relative path. - - .. versionchanged:: 3.6 - *cwd* parameter accepts a :term:`path-like object` on POSIX. - - .. versionchanged:: 3.7 - *cwd* parameter accepts a :term:`path-like object` on Windows. - - .. versionchanged:: 3.8 - *cwd* parameter accepts a bytes object on Windows. - - If *restore_signals* is true (the default) all signals that Python has set to - SIG_IGN are restored to SIG_DFL in the child process before the exec. - Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals. - (POSIX only) - - .. versionchanged:: 3.2 - *restore_signals* was added. - - If *start_new_session* is true the setsid() system call will be made in the - child process prior to the execution of the subprocess. (POSIX only) - - .. versionchanged:: 3.2 - *start_new_session* was added. + :const:`2` will be closed before the child process is executed. (Unix only). + Or, on Windows, if *close_fds* is true then no handles will be inherited by the + child process. Note that on Windows, you cannot set *close_fds* to true and + also redirect the standard handles by setting *stdin*, *stdout* or *stderr*. - If *group* is not ``None``, the setregid() system call will be made in the - child process prior to the execution of the subprocess. If the provided - value is a string, it will be looked up via :func:`grp.getgrnam()` and - the value in ``gr_gid`` will be used. If the value is an integer, it - will be passed verbatim. (POSIX only) - - .. availability:: POSIX - .. versionadded:: 3.9 - - If *extra_groups* is not ``None``, the setgroups() system call will be - made in the child process prior to the execution of the subprocess. - Strings provided in *extra_groups* will be looked up via - :func:`grp.getgrnam()` and the values in ``gr_gid`` will be used. - Integer values will be passed verbatim. (POSIX only) - - .. availability:: POSIX - .. versionadded:: 3.9 - - If *user* is not ``None``, the setreuid() system call will be made in the - child process prior to the execution of the subprocess. If the provided - value is a string, it will be looked up via :func:`pwd.getpwnam()` and - the value in ``pw_uid`` will be used. If the value is an integer, it will - be passed verbatim. (POSIX only) - - .. availability:: POSIX - .. versionadded:: 3.9 - - If *umask* is not negative, the umask() system call will be made in the - child process prior to the execution of the subprocess. - - .. availability:: POSIX - .. versionadded:: 3.9 + If *cwd* is not ``None``, the child's current directory will be changed to *cwd* + before it is executed. Note that this directory is not considered when + searching the executable, so you can't specify the program's path relative to + *cwd*. If *env* is not ``None``, it must be a mapping that defines the environment - variables for the new process; these are used instead of the default - behavior of inheriting the current process' environment. + variables for the new process; these are used instead of inheriting the current + process' environment, which is the default behavior. .. note:: - If specified, *env* must provide any variables required for the program to - execute. On Windows, in order to run a `side-by-side assembly`_ the - specified *env* **must** include a valid :envvar:`SystemRoot`. + If specified, *env* must provide any variables required + for the program to execute. On Windows, in order to run a + `side-by-side assembly`_ the specified *env* **must** include a valid + :envvar:`SystemRoot`. .. _side-by-side assembly: https://en.wikipedia.org/wiki/Side-by-Side_Assembly - If *encoding* or *errors* are specified, or *text* is true, the file objects - *stdin*, *stdout* and *stderr* are opened in text mode with the specified - encoding and *errors*, as described above in :ref:`frequently-used-arguments`. - The *universal_newlines* argument is equivalent to *text* and is provided - for backwards compatibility. By default, file objects are opened in binary mode. + If *universal_newlines* is ``True``, the file objects *stdout* and *stderr* + are opened as text files in :term:`universal newlines` mode. Lines may be + terminated by any of ``'\n'``, the Unix end-of-line convention, ``'\r'``, + the old Macintosh convention or ``'\r\n'``, the Windows convention. All of + these external representations are seen as ``'\n'`` by the Python program. - .. versionadded:: 3.6 - *encoding* and *errors* were added. + .. note:: - .. versionadded:: 3.7 - *text* was added as a more readable alias for *universal_newlines*. + This feature is only available if Python is built with universal newline + support (the default). Also, the newlines attribute of the file objects + :attr:`stdout`, :attr:`stdin` and :attr:`stderr` are not updated by the + communicate() method. If given, *startupinfo* will be a :class:`STARTUPINFO` object, which is passed to the underlying ``CreateProcess`` function. - *creationflags*, if given, can be one or more of the following flags: - - * :data:`CREATE_NEW_CONSOLE` - * :data:`CREATE_NEW_PROCESS_GROUP` - * :data:`ABOVE_NORMAL_PRIORITY_CLASS` - * :data:`BELOW_NORMAL_PRIORITY_CLASS` - * :data:`HIGH_PRIORITY_CLASS` - * :data:`IDLE_PRIORITY_CLASS` - * :data:`NORMAL_PRIORITY_CLASS` - * :data:`REALTIME_PRIORITY_CLASS` - * :data:`CREATE_NO_WINDOW` - * :data:`DETACHED_PROCESS` - * :data:`CREATE_DEFAULT_ERROR_MODE` - * :data:`CREATE_BREAKAWAY_FROM_JOB` - - Popen objects are supported as context managers via the :keyword:`with` statement: - on exit, standard file descriptors are closed, and the process is waited for. - :: - - with Popen(["ifconfig"], stdout=PIPE) as proc: - log.write(proc.stdout.read()) - - .. audit-event:: subprocess.Popen executable,args,cwd,env subprocess.Popen - - Popen and the other functions in this module that use it raise an - :ref:`auditing event <auditing>` ``subprocess.Popen`` with arguments - ``executable``, ``args``, ``cwd``, and ``env``. The value for ``args`` - may be a single string or a list of strings, depending on platform. - - .. versionchanged:: 3.2 - Added context manager support. - - .. versionchanged:: 3.6 - Popen destructor now emits a :exc:`ResourceWarning` warning if the child - process is still running. - - .. versionchanged:: 3.8 - Popen can use :func:`os.posix_spawn` in some cases for better - performance. On Windows Subsystem for Linux and QEMU User Emulation, - Popen constructor using :func:`os.posix_spawn` no longer raise an - exception on errors like missing program, but the child process fails - with a non-zero :attr:`~Popen.returncode`. + *creationflags*, if given, can be :data:`CREATE_NEW_CONSOLE` or + :data:`CREATE_NEW_PROCESS_GROUP`. (Windows only) Exceptions ^^^^^^^^^^ Exceptions raised in the child process, before the new program has started to -execute, will be re-raised in the parent. +execute, will be re-raised in the parent. Additionally, the exception object +will have one extra attribute called :attr:`child_traceback`, which is a string +containing traceback information from the child's point of view. The most common exception raised is :exc:`OSError`. This occurs, for example, when trying to execute a non-existent file. Applications should prepare for @@ -665,31 +444,15 @@ arguments. :exc:`CalledProcessError` if the called process returns a non-zero return code. -All of the functions and methods that accept a *timeout* parameter, such as -:func:`call` and :meth:`Popen.communicate` will raise :exc:`TimeoutExpired` if -the timeout expires before the process exits. - -Exceptions defined in this module all inherit from :exc:`SubprocessError`. - .. versionadded:: 3.3 - The :exc:`SubprocessError` base class was added. +Security +^^^^^^^^ - -Security Considerations ------------------------ - -Unlike some other popen functions, this implementation will never -implicitly call a system shell. This means that all characters, -including shell metacharacters, can safely be passed to child processes. -If the shell is invoked explicitly, via ``shell=True``, it is the application's -responsibility to ensure that all whitespace and metacharacters are -quoted appropriately to avoid -`shell injection <https://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_ -vulnerabilities. - -When using ``shell=True``, the :func:`shlex.quote` function can be -used to properly escape whitespace and shell metacharacters in strings -that are going to be used to construct shell commands. +Unlike some other popen functions, this implementation will never call a +system shell implicitly. This means that all characters, including shell +metacharacters, can safely be passed to child processes. Obviously, if the +shell is invoked explicitly, then it is the application's responsibility to +ensure that all whitespace and metacharacters are quoted appropriately. Popen Objects @@ -701,74 +464,41 @@ Instances of the :class:`Popen` class have the following methods: .. method:: Popen.poll() Check if child process has terminated. Set and return - :attr:`~Popen.returncode` attribute. Otherwise, returns ``None``. + :attr:`~Popen.returncode` attribute. -.. method:: Popen.wait(timeout=None) +.. method:: Popen.wait() Wait for child process to terminate. Set and return :attr:`~Popen.returncode` attribute. - If the process does not terminate after *timeout* seconds, raise a - :exc:`TimeoutExpired` exception. It is safe to catch this exception and - retry the wait. - - .. note:: - - This will deadlock when using ``stdout=PIPE`` or ``stderr=PIPE`` - and the child process generates enough output to a pipe such that - it blocks waiting for the OS pipe buffer to accept more data. - Use :meth:`Popen.communicate` when using pipes to avoid that. - - .. note:: + .. warning:: - The function is implemented using a busy loop (non-blocking call and - short sleeps). Use the :mod:`asyncio` module for an asynchronous wait: - see :class:`asyncio.create_subprocess_exec`. + This will deadlock when using ``stdout=PIPE`` and/or + ``stderr=PIPE`` and the child process generates enough output to + a pipe such that it blocks waiting for the OS pipe buffer to + accept more data. Use :meth:`communicate` to avoid that. - .. versionchanged:: 3.3 - *timeout* was added. -.. method:: Popen.communicate(input=None, timeout=None) +.. method:: Popen.communicate(input=None) Interact with process: Send data to stdin. Read data from stdout and stderr, - until end-of-file is reached. Wait for process to terminate. The optional - *input* argument should be data to be sent to the child process, or - ``None``, if no data should be sent to the child. If streams were opened in - text mode, *input* must be a string. Otherwise, it must be bytes. + until end-of-file is reached. Wait for process to terminate. The optional + *input* argument should be a string to be sent to the child process, or + ``None``, if no data should be sent to the child. - :meth:`communicate` returns a tuple ``(stdout_data, stderr_data)``. - The data will be strings if streams were opened in text mode; otherwise, - bytes. + :meth:`communicate` returns a tuple ``(stdoutdata, stderrdata)``. Note that if you want to send data to the process's stdin, you need to create the Popen object with ``stdin=PIPE``. Similarly, to get anything other than ``None`` in the result tuple, you need to give ``stdout=PIPE`` and/or ``stderr=PIPE`` too. - If the process does not terminate after *timeout* seconds, a - :exc:`TimeoutExpired` exception will be raised. Catching this exception and - retrying communication will not lose any output. - - The child process is not killed if the timeout expires, so in order to - cleanup properly a well-behaved application should kill the child process and - finish communication:: - - proc = subprocess.Popen(...) - try: - outs, errs = proc.communicate(timeout=15) - except TimeoutExpired: - proc.kill() - outs, errs = proc.communicate() - .. note:: The data read is buffered in memory, so do not use this method if the data size is large or unlimited. - .. versionchanged:: 3.3 - *timeout* was added. - .. method:: Popen.send_signal(signal) @@ -780,6 +510,8 @@ Instances of the :class:`Popen` class have the following methods: CTRL_BREAK_EVENT can be sent to processes started with a *creationflags* parameter which includes `CREATE_NEW_PROCESS_GROUP`. + .. versionadded:: 2.6 + .. method:: Popen.terminate() @@ -787,56 +519,44 @@ Instances of the :class:`Popen` class have the following methods: child. On Windows the Win32 API function :c:func:`TerminateProcess` is called to stop the child. + .. versionadded:: 2.6 + .. method:: Popen.kill() Kills the child. On Posix OSs the function sends SIGKILL to the child. On Windows :meth:`kill` is an alias for :meth:`terminate`. + .. versionadded:: 2.6 + The following attributes are also available: -.. attribute:: Popen.args +.. warning:: - The *args* argument as it was passed to :class:`Popen` -- a - sequence of program arguments or else a single string. + Use :meth:`~Popen.communicate` rather than :attr:`.stdin.write <Popen.stdin>`, + :attr:`.stdout.read <Popen.stdout>` or :attr:`.stderr.read <Popen.stderr>` to avoid + deadlocks due to any of the other OS pipe buffers filling up and blocking the + child process. - .. versionadded:: 3.3 .. attribute:: Popen.stdin - If the *stdin* argument was :data:`PIPE`, this attribute is a writeable - stream object as returned by :func:`open`. If the *encoding* or *errors* - arguments were specified or the *universal_newlines* argument was ``True``, - the stream is a text stream, otherwise it is a byte stream. If the *stdin* - argument was not :data:`PIPE`, this attribute is ``None``. + If the *stdin* argument was :data:`PIPE`, this attribute is a file object + that provides input to the child process. Otherwise, it is ``None``. .. attribute:: Popen.stdout - If the *stdout* argument was :data:`PIPE`, this attribute is a readable - stream object as returned by :func:`open`. Reading from the stream provides - output from the child process. If the *encoding* or *errors* arguments were - specified or the *universal_newlines* argument was ``True``, the stream is a - text stream, otherwise it is a byte stream. If the *stdout* argument was not - :data:`PIPE`, this attribute is ``None``. + If the *stdout* argument was :data:`PIPE`, this attribute is a file object + that provides output from the child process. Otherwise, it is ``None``. .. attribute:: Popen.stderr - If the *stderr* argument was :data:`PIPE`, this attribute is a readable - stream object as returned by :func:`open`. Reading from the stream provides - error output from the child process. If the *encoding* or *errors* arguments - were specified or the *universal_newlines* argument was ``True``, the stream - is a text stream, otherwise it is a byte stream. If the *stderr* argument was - not :data:`PIPE`, this attribute is ``None``. - -.. warning:: - - Use :meth:`~Popen.communicate` rather than :attr:`.stdin.write <Popen.stdin>`, - :attr:`.stdout.read <Popen.stdout>` or :attr:`.stderr.read <Popen.stderr>` to avoid - deadlocks due to any of the other OS pipe buffers filling up and blocking the - child process. + If the *stderr* argument was :data:`PIPE`, this attribute is a file object + that provides error output from the child process. Otherwise, it is + ``None``. .. attribute:: Popen.pid @@ -854,7 +574,7 @@ The following attributes are also available: hasn't terminated yet. A negative value ``-N`` indicates that the child was terminated by signal - ``N`` (POSIX only). + ``N`` (Unix only). Windows Popen Helpers @@ -863,16 +583,11 @@ Windows Popen Helpers The :class:`STARTUPINFO` class and following constants are only available on Windows. -.. class:: STARTUPINFO(*, dwFlags=0, hStdInput=None, hStdOutput=None, \ - hStdError=None, wShowWindow=0, lpAttributeList=None) +.. class:: STARTUPINFO() Partial support of the Windows `STARTUPINFO <https://msdn.microsoft.com/en-us/library/ms686331(v=vs.85).aspx>`__ - structure is used for :class:`Popen` creation. The following attributes can - be set by passing them as keyword-only arguments. - - .. versionchanged:: 3.7 - Keyword-only argument support was added. + structure is used for :class:`Popen` creation. .. attribute:: dwFlags @@ -914,36 +629,9 @@ on Windows. :data:`SW_HIDE` is provided for this attribute. It is used when :class:`Popen` is called with ``shell=True``. - .. attribute:: lpAttributeList - - A dictionary of additional attributes for process creation as given in - ``STARTUPINFOEX``, see - `UpdateProcThreadAttribute <https://msdn.microsoft.com/en-us/library/windows/desktop/ms686880(v=vs.85).aspx>`__. - - Supported attributes: - **handle_list** - Sequence of handles that will be inherited. *close_fds* must be true if - non-empty. - - The handles must be temporarily made inheritable by - :func:`os.set_handle_inheritable` when passed to the :class:`Popen` - constructor, else :class:`OSError` will be raised with Windows error - ``ERROR_INVALID_PARAMETER`` (87). - - .. warning:: - - In a multithreaded process, use caution to avoid leaking handles - that are marked inheritable when combining this feature with - concurrent calls to other process creation functions that inherit - all handles such as :func:`os.system`. This also applies to - standard handle redirection, which temporarily creates inheritable - handles. - - .. versionadded:: 3.7 - -Windows Constants -^^^^^^^^^^^^^^^^^ +Constants +^^^^^^^^^ The :mod:`subprocess` module exposes the following constants. @@ -982,6 +670,8 @@ The :mod:`subprocess` module exposes the following constants. The new process has a new console, instead of inheriting its parent's console (the default). + This flag is always set when :class:`Popen` is created with ``shell=True``. + .. data:: CREATE_NEW_PROCESS_GROUP A :class:`Popen` ``creationflags`` parameter to specify that a new process @@ -990,200 +680,6 @@ The :mod:`subprocess` module exposes the following constants. This flag is ignored if :data:`CREATE_NEW_CONSOLE` is specified. -.. data:: ABOVE_NORMAL_PRIORITY_CLASS - - A :class:`Popen` ``creationflags`` parameter to specify that a new process - will have an above average priority. - - .. versionadded:: 3.7 - -.. data:: BELOW_NORMAL_PRIORITY_CLASS - - A :class:`Popen` ``creationflags`` parameter to specify that a new process - will have a below average priority. - - .. versionadded:: 3.7 - -.. data:: HIGH_PRIORITY_CLASS - - A :class:`Popen` ``creationflags`` parameter to specify that a new process - will have a high priority. - - .. versionadded:: 3.7 - -.. data:: IDLE_PRIORITY_CLASS - - A :class:`Popen` ``creationflags`` parameter to specify that a new process - will have an idle (lowest) priority. - - .. versionadded:: 3.7 - -.. data:: NORMAL_PRIORITY_CLASS - - A :class:`Popen` ``creationflags`` parameter to specify that a new process - will have an normal priority. (default) - - .. versionadded:: 3.7 - -.. data:: REALTIME_PRIORITY_CLASS - - A :class:`Popen` ``creationflags`` parameter to specify that a new process - will have realtime priority. - You should almost never use REALTIME_PRIORITY_CLASS, because this interrupts - system threads that manage mouse input, keyboard input, and background disk - flushing. This class can be appropriate for applications that "talk" directly - to hardware or that perform brief tasks that should have limited interruptions. - - .. versionadded:: 3.7 - -.. data:: CREATE_NO_WINDOW - - A :class:`Popen` ``creationflags`` parameter to specify that a new process - will not create a window. - - .. versionadded:: 3.7 - -.. data:: DETACHED_PROCESS - - A :class:`Popen` ``creationflags`` parameter to specify that a new process - will not inherit its parent's console. - This value cannot be used with CREATE_NEW_CONSOLE. - - .. versionadded:: 3.7 - -.. data:: CREATE_DEFAULT_ERROR_MODE - - A :class:`Popen` ``creationflags`` parameter to specify that a new process - does not inherit the error mode of the calling process. Instead, the new - process gets the default error mode. - This feature is particularly useful for multithreaded shell applications - that run with hard errors disabled. - - .. versionadded:: 3.7 - -.. data:: CREATE_BREAKAWAY_FROM_JOB - - A :class:`Popen` ``creationflags`` parameter to specify that a new process - is not associated with the job. - - .. versionadded:: 3.7 - -.. _call-function-trio: - -Older high-level API --------------------- - -Prior to Python 3.5, these three functions comprised the high level API to -subprocess. You can now use :func:`run` in many cases, but lots of existing code -calls these functions. - -.. function:: call(args, *, stdin=None, stdout=None, stderr=None, shell=False, cwd=None, timeout=None) - - Run the command described by *args*. Wait for command to complete, then - return the :attr:`~Popen.returncode` attribute. - - Code needing to capture stdout or stderr should use :func:`run` instead:: - - run(...).returncode - - To suppress stdout or stderr, supply a value of :data:`DEVNULL`. - - The arguments shown above are merely some common ones. - The full function signature is the - same as that of the :class:`Popen` constructor - this function passes all - supplied arguments other than *timeout* directly through to that interface. - - .. note:: - - Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this - function. The child process will block if it generates enough - output to a pipe to fill up the OS pipe buffer as the pipes are - not being read from. - - .. versionchanged:: 3.3 - *timeout* was added. - -.. function:: check_call(args, *, stdin=None, stdout=None, stderr=None, shell=False, cwd=None, timeout=None) - - Run command with arguments. Wait for command to complete. If the return - code was zero then return, otherwise raise :exc:`CalledProcessError`. The - :exc:`CalledProcessError` object will have the return code in the - :attr:`~CalledProcessError.returncode` attribute. - - Code needing to capture stdout or stderr should use :func:`run` instead:: - - run(..., check=True) - - To suppress stdout or stderr, supply a value of :data:`DEVNULL`. - - The arguments shown above are merely some common ones. - The full function signature is the - same as that of the :class:`Popen` constructor - this function passes all - supplied arguments other than *timeout* directly through to that interface. - - .. note:: - - Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this - function. The child process will block if it generates enough - output to a pipe to fill up the OS pipe buffer as the pipes are - not being read from. - - .. versionchanged:: 3.3 - *timeout* was added. - - -.. function:: check_output(args, *, stdin=None, stderr=None, shell=False, \ - cwd=None, encoding=None, errors=None, \ - universal_newlines=None, timeout=None, text=None) - - Run command with arguments and return its output. - - If the return code was non-zero it raises a :exc:`CalledProcessError`. The - :exc:`CalledProcessError` object will have the return code in the - :attr:`~CalledProcessError.returncode` attribute and any output in the - :attr:`~CalledProcessError.output` attribute. - - This is equivalent to:: - - run(..., check=True, stdout=PIPE).stdout - - The arguments shown above are merely some common ones. - The full function signature is largely the same as that of :func:`run` - - most arguments are passed directly through to that interface. - However, explicitly passing ``input=None`` to inherit the parent's - standard input file handle is not supported. - - By default, this function will return the data as encoded bytes. The actual - encoding of the output data may depend on the command being invoked, so the - decoding to text will often need to be handled at the application level. - - This behaviour may be overridden by setting *text*, *encoding*, *errors*, - or *universal_newlines* to ``True`` as described in - :ref:`frequently-used-arguments` and :func:`run`. - - To also capture standard error in the result, use - ``stderr=subprocess.STDOUT``:: - - >>> subprocess.check_output( - ... "ls non_existent_file; exit 0", - ... stderr=subprocess.STDOUT, - ... shell=True) - 'ls: non_existent_file: No such file or directory\n' - - .. versionadded:: 3.1 - - .. versionchanged:: 3.3 - *timeout* was added. - - .. versionchanged:: 3.4 - Support for the *input* keyword argument was added. - - .. versionchanged:: 3.6 - *encoding* and *errors* were added. See :func:`run` for details. - - .. versionadded:: 3.7 - *text* was added as a more readable alias for *universal_newlines*. - .. _subprocess-replacements: @@ -1207,12 +703,12 @@ In the following examples, we assume that the relevant functions have already been imported from the :mod:`subprocess` module. -Replacing :program:`/bin/sh` shell command substitution -^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Replacing /bin/sh shell backquote +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ .. code-block:: bash - output=$(mycmd myarg) + output=`mycmd myarg` becomes:: @@ -1223,7 +719,7 @@ Replacing shell pipeline .. code-block:: bash - output=$(dmesg | grep hda) + output=`dmesg | grep hda` becomes:: @@ -1232,15 +728,15 @@ becomes:: p1.stdout.close() # Allow p1 to receive a SIGPIPE if p2 exits. output = p2.communicate()[0] -The ``p1.stdout.close()`` call after starting the p2 is important in order for -p1 to receive a SIGPIPE if p2 exits before p1. +The p1.stdout.close() call after starting the p2 is important in order for p1 +to receive a SIGPIPE if p2 exits before p1. Alternatively, for trusted input, the shell's own pipeline support may still be used directly: .. code-block:: bash - output=$(dmesg | grep hda) + output=`dmesg | grep hda` becomes:: @@ -1252,9 +748,9 @@ Replacing :func:`os.system` :: - sts = os.system("mycmd" + " myarg") + status = os.system("mycmd" + " myarg") # becomes - sts = call("mycmd" + " myarg", shell=True) + status = subprocess.call("mycmd" + " myarg", shell=True) Notes: @@ -1265,11 +761,11 @@ A more realistic example would look like this:: try: retcode = call("mycmd" + " myarg", shell=True) if retcode < 0: - print("Child was terminated by signal", -retcode, file=sys.stderr) + print >>sys.stderr, "Child was terminated by signal", -retcode else: - print("Child returned", retcode, file=sys.stderr) + print >>sys.stderr, "Child returned", retcode except OSError as e: - print("Execution failed:", e, file=sys.stderr) + print >>sys.stderr, "Execution failed:", e Replacing the :func:`os.spawn <os.spawnl>` family @@ -1300,15 +796,26 @@ Environment example:: Popen(["/bin/mycmd", "myarg"], env={"PATH": "/usr/bin"}) - Replacing :func:`os.popen`, :func:`os.popen2`, :func:`os.popen3` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ :: - (child_stdin, child_stdout) = os.popen2(cmd, mode, bufsize) + pipe = os.popen("cmd", 'r', bufsize) + ==> + pipe = Popen("cmd", shell=True, bufsize=bufsize, stdout=PIPE).stdout + +:: + + pipe = os.popen("cmd", 'w', bufsize) ==> - p = Popen(cmd, shell=True, bufsize=bufsize, + pipe = Popen("cmd", shell=True, bufsize=bufsize, stdin=PIPE).stdin + +:: + + (child_stdin, child_stdout) = os.popen2("cmd", mode, bufsize) + ==> + p = Popen("cmd", shell=True, bufsize=bufsize, stdin=PIPE, stdout=PIPE, close_fds=True) (child_stdin, child_stdout) = (p.stdin, p.stdout) @@ -1316,9 +823,9 @@ Replacing :func:`os.popen`, :func:`os.popen2`, :func:`os.popen3` (child_stdin, child_stdout, - child_stderr) = os.popen3(cmd, mode, bufsize) + child_stderr) = os.popen3("cmd", mode, bufsize) ==> - p = Popen(cmd, shell=True, bufsize=bufsize, + p = Popen("cmd", shell=True, bufsize=bufsize, stdin=PIPE, stdout=PIPE, stderr=PIPE, close_fds=True) (child_stdin, child_stdout, @@ -1326,35 +833,42 @@ Replacing :func:`os.popen`, :func:`os.popen2`, :func:`os.popen3` :: - (child_stdin, child_stdout_and_stderr) = os.popen4(cmd, mode, bufsize) + (child_stdin, child_stdout_and_stderr) = os.popen4("cmd", mode, + bufsize) ==> - p = Popen(cmd, shell=True, bufsize=bufsize, + p = Popen("cmd", shell=True, bufsize=bufsize, stdin=PIPE, stdout=PIPE, stderr=STDOUT, close_fds=True) (child_stdin, child_stdout_and_stderr) = (p.stdin, p.stdout) +On Unix, os.popen2, os.popen3 and os.popen4 also accept a sequence as +the command to execute, in which case arguments will be passed +directly to the program without shell intervention. This usage can be +replaced as follows:: + + (child_stdin, child_stdout) = os.popen2(["/bin/ls", "-l"], mode, + bufsize) + ==> + p = Popen(["/bin/ls", "-l"], bufsize=bufsize, stdin=PIPE, stdout=PIPE) + (child_stdin, child_stdout) = (p.stdin, p.stdout) + Return code handling translates as follows:: - pipe = os.popen(cmd, 'w') + pipe = os.popen("cmd", 'w') ... rc = pipe.close() if rc is not None and rc >> 8: - print("There were some errors") + print "There were some errors" ==> - process = Popen(cmd, stdin=PIPE) + process = Popen("cmd", shell=True, stdin=PIPE) ... process.stdin.close() if process.wait() != 0: - print("There were some errors") + print "There were some errors" Replacing functions from the :mod:`popen2` module ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. note:: - - If the cmd argument to popen2 functions is a string, the command is executed - through /bin/sh. If it is a list, the command is directly executed. - :: (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode) @@ -1363,9 +877,12 @@ Replacing functions from the :mod:`popen2` module stdin=PIPE, stdout=PIPE, close_fds=True) (child_stdout, child_stdin) = (p.stdout, p.stdin) -:: +On Unix, popen2 also accepts a sequence as the command to execute, in +which case arguments will be passed directly to the program without +shell intervention. This usage can be replaced as follows:: - (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, mode) + (child_stdout, child_stdin) = popen2.popen2(["mycmd", "myarg"], bufsize, + mode) ==> p = Popen(["mycmd", "myarg"], bufsize=bufsize, stdin=PIPE, stdout=PIPE, close_fds=True) @@ -1376,68 +893,12 @@ Replacing functions from the :mod:`popen2` module * :class:`Popen` raises an exception if the execution fails. -* The *capturestderr* argument is replaced with the *stderr* argument. +* the *capturestderr* argument is replaced with the *stderr* argument. * ``stdin=PIPE`` and ``stdout=PIPE`` must be specified. * popen2 closes all file descriptors by default, but you have to specify - ``close_fds=True`` with :class:`Popen` to guarantee this behavior on - all platforms or past Python versions. - - -Legacy Shell Invocation Functions ---------------------------------- - -This module also provides the following legacy functions from the 2.x -``commands`` module. These operations implicitly invoke the system shell and -none of the guarantees described above regarding security and exception -handling consistency are valid for these functions. - -.. function:: getstatusoutput(cmd) - - Return ``(exitcode, output)`` of executing *cmd* in a shell. - - Execute the string *cmd* in a shell with :meth:`Popen.check_output` and - return a 2-tuple ``(exitcode, output)``. The locale encoding is used; - see the notes on :ref:`frequently-used-arguments` for more details. - - A trailing newline is stripped from the output. - The exit code for the command can be interpreted as the return code - of subprocess. Example:: - - >>> subprocess.getstatusoutput('ls /bin/ls') - (0, '/bin/ls') - >>> subprocess.getstatusoutput('cat /bin/junk') - (1, 'cat: /bin/junk: No such file or directory') - >>> subprocess.getstatusoutput('/bin/junk') - (127, 'sh: /bin/junk: not found') - >>> subprocess.getstatusoutput('/bin/kill $$') - (-15, '') - - .. availability:: POSIX & Windows. - - .. versionchanged:: 3.3.4 - Windows support was added. - - The function now returns (exitcode, output) instead of (status, output) - as it did in Python 3.3.3 and earlier. exitcode has the same value as - :attr:`~Popen.returncode`. - - -.. function:: getoutput(cmd) - - Return output (stdout and stderr) of executing *cmd* in a shell. - - Like :func:`getstatusoutput`, except the exit code is ignored and the return - value is a string containing the command's output. Example:: - - >>> subprocess.getoutput('ls /bin/ls') - '/bin/ls' - - .. availability:: POSIX & Windows. - - .. versionchanged:: 3.3.4 - Windows support added + ``close_fds=True`` with :class:`Popen`. Notes @@ -1472,8 +933,3 @@ runtime): backslash escapes the next double quotation mark as described in rule 3. - -.. seealso:: - - :mod:`shlex` - Module which provides function to parse and escape command lines. |