diff options
Diffstat (limited to 'Doc')
| -rw-r--r-- | Doc/library/asyncio-subprocess.rst | 256 | ||||
| -rw-r--r-- | Doc/library/subprocess.rst | 2 |
2 files changed, 180 insertions, 78 deletions
diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index 4713106..1cbfcf2 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -27,23 +27,34 @@ Example to use it on Windows:: Create a subprocess: high-level API using Process ------------------------------------------------- -.. function:: create_subprocess_shell(cmd, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds) +.. function:: create_subprocess_exec(\*args, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds) + + Create a subprocess. - Run the shell command *cmd*. See :meth:`BaseEventLoop.subprocess_shell` for - parameters. Return a :class:`~asyncio.subprocess.Process` instance. + The *limit* parameter sets the buffer limit passed to the + :class:`StreamReader`. See :meth:`BaseEventLoop.subprocess_exec` for other + parameters. - The optional *limit* parameter sets the buffer limit passed to the - :class:`StreamReader`. + Return a :class:`~asyncio.subprocess.Process` instance. This function is a :ref:`coroutine <coroutine>`. -.. function:: create_subprocess_exec(\*args, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds) +.. function:: create_subprocess_shell(cmd, stdin=None, stdout=None, stderr=None, loop=None, limit=None, \*\*kwds) + + Run the shell command *cmd*. - Create a subprocess. See :meth:`BaseEventLoop.subprocess_exec` for - parameters. Return a :class:`~asyncio.subprocess.Process` instance. + The *limit* parameter sets the buffer limit passed to the + :class:`StreamReader`. See :meth:`BaseEventLoop.subprocess_shell` for other + parameters. - The optional *limit* parameter sets the buffer limit passed to the - :class:`StreamReader`. + Return a :class:`~asyncio.subprocess.Process` instance. + + It is the application's responsibility to ensure that all whitespace and + metacharacters are quoted appropriately to avoid `shell injection + <http://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_ + vulnerabilities. The :func:`shlex.quote` function can be used to properly + escape whitespace and shell metacharacters in strings that are going to be + used to construct shell commands. This function is a :ref:`coroutine <coroutine>`. @@ -121,9 +132,14 @@ Run subprocesses asynchronously using the :mod:`subprocess` module. Returns a pair of ``(transport, protocol)``, where *transport* is an instance of :class:`BaseSubprocessTransport`. - This method is a :ref:`coroutine <coroutine>`. + It is the application's responsibility to ensure that all whitespace and + metacharacters are quoted appropriately to avoid `shell injection + <http://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_ + vulnerabilities. The :func:`shlex.quote` function can be used to properly + escape whitespace and shell metacharacters in strings that are going to be + used to construct shell commands. - See the constructor of the :class:`subprocess.Popen` class for parameters. + This method is a :ref:`coroutine <coroutine>`. .. seealso:: @@ -159,35 +175,37 @@ Process .. class:: asyncio.subprocess.Process - .. attribute:: pid + A subprocess created by the :func:`create_subprocess_exec` or the + :func:`create_subprocess_shell` function. - The identifier of the process. + The API of the :class:`~asyncio.subprocess.Process` class was designed to be + closed the API of the :class:`subprocess.Popen` class, but they are some + differences: - Note that if you set the *shell* argument to ``True``, this is the - process identifier of the spawned shell. + * There is no explicit :meth:`~subprocess.Popen.poll` method + * The :meth:`~subprocess.Popen.communicate` and + :meth:`~subprocess.Popen.wait` methods don't take a *timeout* parameter: + use the :func:`wait_for` function + * The *universal_newlines* parameter is not supported (only bytes strings + are supported) + * The :meth:`~asyncio.subprocess.Process.wait` method of + the :class:`~asyncio.subprocess.Process` class is asynchronous whereas the + :meth:`~subprocess.Popen.wait` method of the :class:`~subprocess.Popen` + class is implemented as a busy loop. - .. attribute:: returncode - - Return code of the process when it exited. A ``None`` value indicates - that the process has not terminated yet. + .. method:: wait() - A negative value ``-N`` indicates that the child was terminated by signal - ``N`` (Unix only). - - .. attribute:: stdin - - Standard input stream (write), ``None`` if the process was created with - ``stdin=None``. - - .. attribute:: stdout + Wait for child process to terminate. Set and return :attr:`returncode` + attribute. - Standard output stream (read), ``None`` if the process was created with - ``stdout=None``. + This method is a :ref:`coroutine <coroutine>`. - .. attribute:: stderr + .. note:: - Standard error stream (read), ``None`` if the process was created with - ``stderr=None``. + This will deadlock when using ``stdout=PIPE`` or ``stderr=PIPE`` and + the child process generates enough output to a pipe such that it + blocks waiting for the OS pipe buffer to accept more data. Use the + :meth:`communicate` method when using pipes to avoid that. .. method:: communicate(input=None) @@ -197,33 +215,28 @@ Process process, or ``None``, if no data should be sent to the child. The type of *input* must be bytes. + :meth:`communicate` returns a tuple ``(stdout_data, stderr_data)``. + If a :exc:`BrokenPipeError` or :exc:`ConnectionResetError` exception is raised when writing *input* into stdin, the exception is ignored. It occurs when the process exits before all data are written into stdin. - :meth:`communicate` returns a tuple ``(stdoutdata, stderrdata)``. - Note that if you want to send data to the process's stdin, you need to create the Process object with ``stdin=PIPE``. Similarly, to get anything other than ``None`` in the result tuple, you need to give ``stdout=PIPE`` and/or ``stderr=PIPE`` too. + This method is a :ref:`coroutine <coroutine>`. + .. note:: The data read is buffered in memory, so do not use this method if the data size is large or unlimited. - This method is a :ref:`coroutine <coroutine>`. - .. versionchanged:: 3.4.2 The method now ignores :exc:`BrokenPipeError` and :exc:`ConnectionResetError`. - .. method:: kill() - - Kills the child. On Posix OSs the function sends :py:data:`SIGKILL` to - the child. On Windows :meth:`kill` is an alias for :meth:`terminate`. - .. method:: send_signal(signal) Sends the signal *signal* to the child process. @@ -241,53 +254,142 @@ Process to the child. On Windows the Win32 API function :c:func:`TerminateProcess` is called to stop the child. - .. method:: wait(): + .. method:: kill() - Wait for child process to terminate. Set and return :attr:`returncode` - attribute. + Kills the child. On Posix OSs the function sends :py:data:`SIGKILL` to + the child. On Windows :meth:`kill` is an alias for :meth:`terminate`. - This method is a :ref:`coroutine <coroutine>`. + .. attribute:: stdin + Standard input stream (:class:`StreamWriter`), ``None`` if the process + was created with ``stdin=None``. -Example -------- + .. attribute:: stdout + + Standard output stream (:class:`StreamReader`), ``None`` if the process + was created with ``stdout=None``. + + .. attribute:: stderr + + Standard error stream (:class:`StreamReader`), ``None`` if the process + was created with ``stderr=None``. + + .. warning:: + + Use the :meth:`communicate` method rather than :attr:`.stdin.write + <stdin>`, :attr:`.stdout.read <stdout>` or :attr:`.stderr.read <stderr>` + to avoid deadlocks due to streams pausing reading or writing and blocking + the child process. + + .. attribute:: pid + + The identifier of the process. + + Note that for processes created by the :func:`create_subprocess_shell` + function, this attribute is the process identifier of the spawned shell. + + .. attribute:: returncode + + Return code of the process when it exited. A ``None`` value indicates + that the process has not terminated yet. + + A negative value ``-N`` indicates that the child was terminated by signal + ``N`` (Unix only). + + +Subprocess examples +=================== -Implement a function similar to :func:`subprocess.getstatusoutput`, except that -it does not use a shell. Get the output of the "python -m platform" command and -display the output:: +Subprocess using transport and protocol +--------------------------------------- + +Example of a subprocess protocol using to get the output of a subprocess and to +wait for the subprocess exit. The subprocess is created by the +:meth:`BaseEventLoop.subprocess_exec` method:: import asyncio - import os import sys - from asyncio import subprocess - @asyncio.coroutine - def getstatusoutput(*args): - proc = yield from asyncio.create_subprocess_exec( - *args, - stdout=subprocess.PIPE, - stderr=subprocess.STDOUT) - try: - stdout, _ = yield from proc.communicate() - except: - proc.kill() - yield from proc.wait() - raise - exitcode = yield from proc.wait() - return (exitcode, stdout) + class DateProtocol(asyncio.SubprocessProtocol): + def __init__(self, exit_future): + self.exit_future = exit_future + self.output = bytearray() - if os.name == 'nt': + def pipe_data_received(self, fd, data): + self.output.extend(data) + + def process_exited(self): + self.exit_future.set_result(True) + + @asyncio.coroutine + def get_date(loop): + code = 'import datetime; print(datetime.datetime.now())' + exit_future = asyncio.Future(loop=loop) + + # Create the subprocess controlled by the protocol DateProtocol, + # redirect the standard output into a pipe + create = loop.subprocess_exec(lambda: DateProtocol(exit_future), + sys.executable, '-c', code, + stdin=None, stderr=None) + transport, protocol = yield from create + + # Wait for the subprocess exit using the process_exited() method + # of the protocol + yield from exit_future + + # Close the stdout pipe + transport.close() + + # Read the output which was collected by the pipe_data_received() + # method of the protocol + data = bytes(protocol.output) + return data.decode('ascii').rstrip() + + if sys.platform == "win32": loop = asyncio.ProactorEventLoop() asyncio.set_event_loop(loop) else: loop = asyncio.get_event_loop() - coro = getstatusoutput(sys.executable, '-m', 'platform') - exitcode, stdout = loop.run_until_complete(coro) - if not exitcode: - stdout = stdout.decode('ascii').rstrip() - print("Platform: %s" % stdout) + + date = loop.run_until_complete(get_date(loop)) + print("Current date: %s" % date) + loop.close() + + +Subprocess using streams +------------------------ + +Example using the :class:`~asyncio.subprocess.Process` class to control the +subprocess and the :class:`StreamReader` class to read from the standard +output. The subprocess is created by the :func:`create_subprocess_exec` +function:: + + import asyncio.subprocess + import sys + + @asyncio.coroutine + def get_date(): + code = 'import datetime; print(datetime.datetime.now())' + + # Create the subprocess, redirect the standard output into a pipe + create = asyncio.create_subprocess_exec(sys.executable, '-c', code, + stdout=asyncio.subprocess.PIPE) + proc = yield from create + + # Read one line of output + data = yield from proc.stdout.readline() + line = data.decode('ascii').rstrip() + + # Wait for the subprocess exit + yield from proc.wait() + return line + + if sys.platform == "win32": + loop = asyncio.ProactorEventLoop() + asyncio.set_event_loop(loop) else: - print("Python failed with exit code %s:" % exitcode, flush=True) - sys.stdout.buffer.write(stdout) - sys.stdout.buffer.flush() + loop = asyncio.get_event_loop() + + date = loop.run_until_complete(get_date()) + print("Current date: %s" % date) loop.close() diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index b2238f0..36cbf3c 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -634,7 +634,7 @@ Instances of the :class:`Popen` class have the following methods: ``None``, if no data should be sent to the child. The type of *input* must be bytes or, if *universal_newlines* was ``True``, a string. - :meth:`communicate` returns a tuple ``(stdoutdata, stderrdata)``. + :meth:`communicate` returns a tuple ``(stdout_data, stderr_data)``. Note that if you want to send data to the process's stdin, you need to create the Popen object with ``stdin=PIPE``. Similarly, to get anything other than |