diff options
author | Larry Hastings <larry@hastings.org> | 2014-05-18 04:05:10 (GMT) |
---|---|---|
committer | Larry Hastings <larry@hastings.org> | 2014-05-18 04:05:10 (GMT) |
commit | 3a260d228b32b04a88d947b887bf81759e8e5f10 (patch) | |
tree | e4a34e9cbbf877cf021ffba743ddbf54e17526b7 /Doc | |
parent | 2110603344316d927e6d639275c12f5da78601d5 (diff) | |
parent | b1a1ec3151155a1ae65831793b4a5b7a87d9d09f (diff) | |
download | cpython-3a260d228b32b04a88d947b887bf81759e8e5f10.zip cpython-3a260d228b32b04a88d947b887bf81759e8e5f10.tar.gz cpython-3a260d228b32b04a88d947b887bf81759e8e5f10.tar.bz2 |
Merge.
Diffstat (limited to 'Doc')
-rw-r--r-- | Doc/c-api/import.rst | 8 | ||||
-rw-r--r-- | Doc/library/asyncio-subprocess.rst | 6 | ||||
-rw-r--r-- | Doc/library/codecs.rst | 10 | ||||
-rw-r--r-- | Doc/library/email.generator.rst | 10 | ||||
-rw-r--r-- | Doc/library/email.message.rst | 2 | ||||
-rw-r--r-- | Doc/library/email.parser.rst | 41 | ||||
-rw-r--r-- | Doc/library/imp.rst | 12 | ||||
-rw-r--r-- | Doc/library/importlib.rst | 12 | ||||
-rw-r--r-- | Doc/library/ssl.rst | 23 | ||||
-rw-r--r-- | Doc/library/subprocess.rst | 157 | ||||
-rw-r--r-- | Doc/library/sys.rst | 27 | ||||
-rw-r--r-- | Doc/library/venv.rst | 4 | ||||
-rw-r--r-- | Doc/reference/compound_stmts.rst | 30 | ||||
-rw-r--r-- | Doc/reference/datamodel.rst | 2 |
14 files changed, 185 insertions, 159 deletions
diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst index 0dcac2d..3641fc6 100644 --- a/Doc/c-api/import.rst +++ b/Doc/c-api/import.rst @@ -132,8 +132,14 @@ Importing Modules such modules have no way to know that the module object is an unknown (and probably damaged with respect to the module author's intents) state. + The module's :attr:`__spec__` and :attr:`__loader__` will be set, if + not set already, with the appropriate values. The spec's loader will + be set to the module's ``__loader__`` (if set) and to an instance of + :class:`SourceFileLoader` otherwise. + The module's :attr:`__file__` attribute will be set to the code object's - :c:member:`co_filename`. + :c:member:`co_filename`. If applicable, :attr:`__cached__` will also + be set. This function will reload the module if it was already imported. See :c:func:`PyImport_ReloadModule` for the intended way to reload a module. diff --git a/Doc/library/asyncio-subprocess.rst b/Doc/library/asyncio-subprocess.rst index b387543..dc48eb5 100644 --- a/Doc/library/asyncio-subprocess.rst +++ b/Doc/library/asyncio-subprocess.rst @@ -8,7 +8,7 @@ Operating system support On Windows, the default event loop uses :class:`selectors.SelectSelector` which only supports sockets. The :class:`ProactorEventLoop` should be used to -support subprocesses. +support subprocesses. However, the latter does not support SSL. On Mac OS X older than 10.9 (Mavericks), :class:`selectors.KqueueSelector` does not support character devices like PTY, whereas it is used by the @@ -262,9 +262,7 @@ display the output:: stdout = stdout.decode('ascii').rstrip() print("Platform: %s" % stdout) else: - print("Python failed with exit code %s:" % exitcode) - sys.stdout.flush() - sys.stdout.buffer.flush() + print("Python failed with exit code %s:" % exitcode, flush=True) sys.stdout.buffer.write(stdout) sys.stdout.buffer.flush() loop.close() diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst index 3729dac..fb3af3b 100644 --- a/Doc/library/codecs.rst +++ b/Doc/library/codecs.rst @@ -22,9 +22,10 @@ manages the codec and error handling lookup process. It defines the following functions: -.. function:: encode(obj, encoding='utf-8', errors='strict') +.. function:: encode(obj, [encoding[, errors]]) - Encodes *obj* using the codec registered for *encoding*. + Encodes *obj* using the codec registered for *encoding*. The default + encoding is ``utf-8``. *Errors* may be given to set the desired error handling scheme. The default error handler is ``strict`` meaning that encoding errors raise @@ -32,9 +33,10 @@ It defines the following functions: :exc:`UnicodeEncodeError`). Refer to :ref:`codec-base-classes` for more information on codec error handling. -.. function:: decode(obj, encoding='utf-8', errors='strict') +.. function:: decode(obj, [encoding[, errors]]) - Decodes *obj* using the codec registered for *encoding*. + Decodes *obj* using the codec registered for *encoding*. The default + encoding is ``utf-8``. *Errors* may be given to set the desired error handling scheme. The default error handler is ``strict`` meaning that decoding errors raise diff --git a/Doc/library/email.generator.rst b/Doc/library/email.generator.rst index c172acb..48d41e1 100644 --- a/Doc/library/email.generator.rst +++ b/Doc/library/email.generator.rst @@ -112,7 +112,7 @@ formatted string representation of a message object. For more detail, see :mod:`email.message`. .. class:: BytesGenerator(outfp, mangle_from_=True, maxheaderlen=78, *, \ - policy=policy.default) + policy=None) The constructor for the :class:`BytesGenerator` class takes a binary :term:`file-like object` called *outfp* for an argument. *outfp* must @@ -134,9 +134,11 @@ formatted string representation of a message object. For more detail, see wrapping. The default is 78, as recommended (but not required) by :rfc:`2822`. + The *policy* keyword specifies a :mod:`~email.policy` object that controls a - number of aspects of the generator's operation. The default policy - maintains backward compatibility. + number of aspects of the generator's operation. If no *policy* is specified, + then the *policy* attached to the message object passed to :attr:`flatten` + is used. .. versionchanged:: 3.3 Added the *policy* keyword. @@ -174,7 +176,7 @@ formatted string representation of a message object. For more detail, see Optional *linesep* specifies the line separator character used to terminate lines in the output. If specified it overrides the value - specified by the ``Generator``\ 's ``policy``. + specified by the ``Generator``\ or *msg*\ 's ``policy``. .. method:: clone(fp) diff --git a/Doc/library/email.message.rst b/Doc/library/email.message.rst index 58f708c..8573e85 100644 --- a/Doc/library/email.message.rst +++ b/Doc/library/email.message.rst @@ -34,7 +34,7 @@ Here are the methods of the :class:`Message` class: .. class:: Message(policy=compat32) If *policy* is specified (it must be an instance of a :mod:`~email.policy` - class) use the rules it specifies to udpate and serialize the representation + class) use the rules it specifies to update and serialize the representation of the message. If *policy* is not set, use the :class:`compat32 <email.policy.Compat32>` policy, which maintains backward compatibility with the Python 3.2 version of the email package. For more information see the diff --git a/Doc/library/email.parser.rst b/Doc/library/email.parser.rst index ee6af3f..9f137cd 100644 --- a/Doc/library/email.parser.rst +++ b/Doc/library/email.parser.rst @@ -60,15 +60,18 @@ list of defects that it can find. Here is the API for the :class:`FeedParser`: -.. class:: FeedParser(_factory=email.message.Message, *, policy=policy.default) +.. class:: FeedParser(_factory=email.message.Message, *, policy=policy.compat32) Create a :class:`FeedParser` instance. Optional *_factory* is a no-argument callable that will be called whenever a new message object is needed. It defaults to the :class:`email.message.Message` class. - The *policy* keyword specifies a :mod:`~email.policy` object that controls a - number of aspects of the parser's operation. The default policy maintains - backward compatibility. + If *policy* is specified (it must be an instance of a :mod:`~email.policy` + class) use the rules it specifies to update the representation of the + message. If *policy* is not set, use the :class:`compat32 + <email.policy.Compat32>` policy, which maintains backward compatibility with + the Python 3.2 version of the email package. For more information see the + :mod:`~email.policy` documentation. .. versionchanged:: 3.3 Added the *policy* keyword. @@ -113,7 +116,7 @@ have the same API as the :class:`Parser` and :class:`BytesParser` classes. The BytesHeaderParser class. -.. class:: Parser(_class=email.message.Message, *, policy=policy.default) +.. class:: Parser(_class=email.message.Message, *, policy=policy.compat32) The constructor for the :class:`Parser` class takes an optional argument *_class*. This must be a callable factory (such as a function or a class), and @@ -121,9 +124,12 @@ have the same API as the :class:`Parser` and :class:`BytesParser` classes. :class:`~email.message.Message` (see :mod:`email.message`). The factory will be called without arguments. - The *policy* keyword specifies a :mod:`~email.policy` object that controls a - number of aspects of the parser's operation. The default policy maintains - backward compatibility. + If *policy* is specified (it must be an instance of a :mod:`~email.policy` + class) use the rules it specifies to update the representation of the + message. If *policy* is not set, use the :class:`compat32 + <email.policy.Compat32>` policy, which maintains backward compatibility with + the Python 3.2 version of the email package. For more information see the + :mod:`~email.policy` documentation. .. versionchanged:: 3.3 Removed the *strict* argument that was deprecated in 2.4. Added the @@ -159,15 +165,18 @@ have the same API as the :class:`Parser` and :class:`BytesParser` classes. Optional *headersonly* is as with the :meth:`parse` method. -.. class:: BytesParser(_class=email.message.Message, *, policy=policy.default) +.. class:: BytesParser(_class=email.message.Message, *, policy=policy.compat32) This class is exactly parallel to :class:`Parser`, but handles bytes input. The *_class* and *strict* arguments are interpreted in the same way as for the :class:`Parser` constructor. - The *policy* keyword specifies a :mod:`~email.policy` object that - controls a number of aspects of the parser's operation. The default - policy maintains backward compatibility. + If *policy* is specified (it must be an instance of a :mod:`~email.policy` + class) use the rules it specifies to update the representation of the + message. If *policy* is not set, use the :class:`compat32 + <email.policy.Compat32>` policy, which maintains backward compatibility with + the Python 3.2 version of the email package. For more information see the + :mod:`~email.policy` documentation. .. versionchanged:: 3.3 Removed the *strict* argument. Added the *policy* keyword. @@ -209,7 +218,7 @@ in the top-level :mod:`email` package namespace. .. currentmodule:: email .. function:: message_from_string(s, _class=email.message.Message, *, \ - policy=policy.default) + policy=policy.compat32) Return a message object structure from a string. This is exactly equivalent to ``Parser().parsestr(s)``. *_class* and *policy* are interpreted as @@ -219,7 +228,7 @@ in the top-level :mod:`email` package namespace. Removed the *strict* argument. Added the *policy* keyword. .. function:: message_from_bytes(s, _class=email.message.Message, *, \ - policy=policy.default) + policy=policy.compat32) Return a message object structure from a byte string. This is exactly equivalent to ``BytesParser().parsebytes(s)``. Optional *_class* and @@ -231,7 +240,7 @@ in the top-level :mod:`email` package namespace. Removed the *strict* argument. Added the *policy* keyword. .. function:: message_from_file(fp, _class=email.message.Message, *, \ - policy=policy.default) + policy=policy.compat32) Return a message object structure tree from an open :term:`file object`. This is exactly equivalent to ``Parser().parse(fp)``. *_class* @@ -242,7 +251,7 @@ in the top-level :mod:`email` package namespace. Removed the *strict* argument. Added the *policy* keyword. .. function:: message_from_binary_file(fp, _class=email.message.Message, *, \ - policy=policy.default) + policy=policy.compat32) Return a message object structure tree from an open binary :term:`file object`. This is exactly equivalent to ``BytesParser().parse(fp)``. diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst index ed4820e..c2dbdc5 100644 --- a/Doc/library/imp.rst +++ b/Doc/library/imp.rst @@ -79,7 +79,9 @@ This module provides an interface to the mechanisms used to implement the When *P* itself has a dotted name, apply this recipe recursively. .. deprecated:: 3.3 - Use :func:`importlib.find_loader` instead. + Use :func:`importlib.util.find_spec` instead unless Python 3.3 + compatibility is required, in which case use + :func:`importlib.find_loader`. .. function:: load_module(name, file, pathname, description) @@ -104,9 +106,11 @@ This module provides an interface to the mechanisms used to implement the .. deprecated:: 3.3 If previously used in conjunction with :func:`imp.find_module` then - call ``load_module()`` on the returned loader. If you wish to load a - module from a specific file, then use one of the file-based loaders found - in :mod:`importlib.machinery`. + consider using :func:`importlib.import_module`, otherwise use the loader + returned by the replacement you chose for :func:`imp.find_module`. If you + called :func:`imp.load_module` and related functions directly then use the + classes in :mod:`importlib.machinery`, e.g. + ``importlib.machinery.SourceFileLoader(name, path).load_module()``. .. function:: new_module(name) diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst index afdae9e..09a5d71 100644 --- a/Doc/library/importlib.rst +++ b/Doc/library/importlib.rst @@ -887,6 +887,11 @@ find and load modules. Concrete implementation of :meth:`importlib.abc.SourceLoader.set_data`. + .. method:: load_module(name=None) + + Concrete implementation of :meth:`importlib.abc.Loader.load_module` where + specifying the name of the module to load is optional. + .. class:: SourcelessFileLoader(fullname, path) @@ -921,6 +926,11 @@ find and load modules. Returns ``None`` as bytecode files have no source when this loader is used. + .. method:: load_module(name=None) + + Concrete implementation of :meth:`importlib.abc.Loader.load_module` where + specifying the name of the module to load is optional. + .. class:: ExtensionFileLoader(fullname, path) @@ -940,7 +950,7 @@ find and load modules. Path to the extension module. - .. method:: load_module(fullname) + .. method:: load_module(name=None) Loads the extension module if and only if *fullname* is the same as :attr:`name` or is ``None``. diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 94a0c81..b473c45 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -1588,8 +1588,19 @@ the sockets in non-blocking mode and use an event loop). Notes on non-blocking sockets ----------------------------- -When working with non-blocking sockets, there are several things you need -to be aware of: +SSL sockets behave slightly different than regular sockets in +non-blocking mode. When working with non-blocking sockets, there are +thus several things you need to be aware of: + +- Most :class:`SSLSocket` methods will raise either + :exc:`SSLWantWriteError` or :exc:`SSLWantReadError` instead of + :exc:`BlockingIOError` if an I/O operation would + block. :exc:`SSLWantReadError` will be raised if a read operation on + the underlying socket is necessary, and :exc:`SSLWantWriteError` for + a write operation on the underlying socket. Note that attempts to + *write* to an SSL socket may require *reading* from the underlying + socket first, and attempts to *read* from the SSL socket may require + a prior *write* to the underlying socket. - Calling :func:`~select.select` tells you that the OS-level socket can be read from (or written to), but it does not imply that there is sufficient @@ -1598,8 +1609,14 @@ to be aware of: and :meth:`SSLSocket.send` failures, and retry after another call to :func:`~select.select`. +- Conversely, since the SSL layer has its own framing, a SSL socket may + still have data available for reading without :func:`~select.select` + being aware of it. Therefore, you should first call + :meth:`SSLSocket.recv` to drain any potentially available data, and then + only block on a :func:`~select.select` call if still necessary. + (of course, similar provisions apply when using other primitives such as - :func:`~select.poll`) + :func:`~select.poll`, or those in the :mod:`selectors` module) - The SSL handshake itself will be non-blocking: the :meth:`SSLSocket.do_handshake` method has to be retried until it returns diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index 4b18396..854993c 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -54,18 +54,12 @@ use cases, the underlying :class:`Popen` interface can be used directly. >>> subprocess.call("exit 1", shell=True) 1 - .. warning:: - - Invoking the system shell with ``shell=True`` can be a security hazard - if combined with untrusted input. See the warning under - :ref:`frequently-used-arguments` for details. - .. note:: - Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function. As - the pipes are not being read in the current process, the child - process may block if it generates enough output to a pipe to fill up - the OS pipe buffer. + Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this + function. The child process will block if it generates enough + output to a pipe to fill up the OS pipe buffer as the pipes are + not being read from. .. versionchanged:: 3.3 *timeout* was added. @@ -99,18 +93,12 @@ use cases, the underlying :class:`Popen` interface can be used directly. ... subprocess.CalledProcessError: Command 'exit 1' returned non-zero exit status 1 - .. warning:: - - Invoking the system shell with ``shell=True`` can be a security hazard - if combined with untrusted input. See the warning under - :ref:`frequently-used-arguments` for details. - .. note:: - Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this function. As - the pipes are not being read in the current process, the child - process may block if it generates enough output to a pipe to fill up - the OS pipe buffer. + Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this + function. The child process will block if it generates enough + output to a pipe to fill up the OS pipe buffer as the pipes are + not being read from. .. versionchanged:: 3.3 *timeout* was added. @@ -177,17 +165,12 @@ use cases, the underlying :class:`Popen` interface can be used directly. ... shell=True) 'ls: non_existent_file: No such file or directory\n' - .. warning:: - - Invoking the system shell with ``shell=True`` can be a security hazard - if combined with untrusted input. See the warning under - :ref:`frequently-used-arguments` for details. - .. note:: - Do not use ``stderr=PIPE`` with this function. As the pipe is not being - read in the current process, the child process may block if it - generates enough output to the pipe to fill up the OS pipe buffer. + Do not use ``stdout=PIPE`` or ``stderr=PIPE`` with this + function. The child process will block if it generates enough + output to a pipe to fill up the OS pipe buffer as the pipes are + not being read from. .. versionadded:: 3.1 @@ -210,7 +193,7 @@ use cases, the underlying :class:`Popen` interface can be used directly. Special value that can be used as the *stdin*, *stdout* or *stderr* argument to :class:`Popen` and indicates that a pipe to the standard stream should be - opened. + opened. Most useful with :meth:`Popen.communicate`. .. data:: STDOUT @@ -336,28 +319,9 @@ default values. The arguments that are most commonly needed are: instead of ``locale.getpreferredencoding()``. See the :class:`io.TextIOWrapper` class for more information on this change. - .. warning:: - - Executing shell commands that incorporate unsanitized input from an - untrusted source makes a program vulnerable to `shell injection - <http://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_, - a serious security flaw which can result in arbitrary command execution. - For this reason, the use of ``shell=True`` is **strongly discouraged** - in cases where the command string is constructed from external input:: - - >>> from subprocess import call - >>> filename = input("What file would you like to display?\n") - What file would you like to display? - non_existent; rm -rf / # - >>> call("cat " + filename, shell=True) # Uh-oh. This will end badly... - - ``shell=False`` disables all shell based features, but does not suffer - from this vulnerability; see the Note in the :class:`Popen` constructor - documentation for helpful hints in getting ``shell=False`` to work. + .. note:: - When using ``shell=True``, :func:`shlex.quote` can be used to properly - escape whitespace and shell metacharacters in strings that are going to - be used to construct shell commands. + Read the `Security Considerations`_ section before using ``shell=True``. These options, along with all of the other options, are described in more detail in the :class:`Popen` constructor documentation. @@ -378,7 +342,7 @@ functions. startupinfo=None, creationflags=0, restore_signals=True, \ start_new_session=False, pass_fds=()) - Execute a child program in a new process. On Unix, the class uses + Execute a child program in a new process. On POSIX, the class uses :meth:`os.execvp`-like behavior to execute the child program. On Windows, the class uses the Windows ``CreateProcess()`` function. The arguments to :class:`Popen` are as follows. @@ -390,7 +354,7 @@ functions. arguments for additional differences from the default behavior. Unless otherwise stated, it is recommended to pass *args* as a sequence. - On Unix, if *args* is a string, the string is interpreted as the name or + On POSIX, if *args* is a string, the string is interpreted as the name or path of the program to execute. However, this can only be done if not passing arguments to the program. @@ -421,7 +385,7 @@ functions. the shell as the program to execute. If *shell* is *True*, it is recommended to pass *args* as a string rather than as a sequence. - On Unix with ``shell=True``, the shell defaults to :file:`/bin/sh`. If + On POSIX with ``shell=True``, the shell defaults to :file:`/bin/sh`. If *args* is a string, the string specifies the command to execute through the shell. This means that the string must be formatted exactly as it would be when typed at the shell prompt. This @@ -438,11 +402,9 @@ functions. into the shell (e.g. :command:`dir` or :command:`copy`). You do not need ``shell=True`` to run a batch file or console-based executable. - .. warning:: + .. note:: - Passing ``shell=True`` can be a security hazard if combined with - untrusted input. See the warning under :ref:`frequently-used-arguments` - for details. + Read the `Security Considerations`_ section before using ``shell=True``. *bufsize* will be supplied as the corresponding argument to the :func:`open` function when creating the stdin/stdout/stderr pipe file objects: :const:`0` @@ -463,9 +425,9 @@ functions. program to execute specified by *args*. However, the original *args* is still passed to the program. Most programs treat the program specified by *args* as the command name, which can then be different from the program - actually executed. On Unix, the *args* name + actually executed. On POSIX, the *args* name becomes the display name for the executable in utilities such as - :program:`ps`. If ``shell=True``, on Unix the *executable* argument + :program:`ps`. If ``shell=True``, on POSIX the *executable* argument specifies a replacement shell for the default :file:`/bin/sh`. *stdin*, *stdout* and *stderr* specify the executed program's standard input, @@ -481,7 +443,7 @@ functions. If *preexec_fn* is set to a callable object, this object will be called in the child process just before the child is executed. - (Unix only) + (POSIX only) .. warning:: @@ -499,8 +461,8 @@ functions. common use of *preexec_fn* to call os.setsid() in the child. If *close_fds* is true, all file descriptors except :const:`0`, :const:`1` and - :const:`2` will be closed before the child process is executed. (Unix only). - The default varies by platform: Always true on Unix. On Windows it is + :const:`2` will be closed before the child process is executed. (POSIX only). + The default varies by platform: Always true on POSIX. On Windows it is true when *stdin*/*stdout*/*stderr* are :const:`None`, false otherwise. On Windows, if *close_fds* is true then no handles will be inherited by the child process. Note that on Windows, you cannot set *close_fds* to true and @@ -512,7 +474,7 @@ functions. *pass_fds* is an optional sequence of file descriptors to keep open between the parent and child. Providing any *pass_fds* forces - *close_fds* to be :const:`True`. (Unix only) + *close_fds* to be :const:`True`. (POSIX only) .. versionadded:: 3.2 The *pass_fds* parameter was added. @@ -525,13 +487,13 @@ functions. If *restore_signals* is true (the default) all signals that Python has set to SIG_IGN are restored to SIG_DFL in the child process before the exec. Currently this includes the SIGPIPE, SIGXFZ and SIGXFSZ signals. - (Unix only) + (POSIX only) .. versionchanged:: 3.2 *restore_signals* was added. If *start_new_session* is true the setsid() system call will be made in the - child process prior to the execution of the subprocess. (Unix only) + child process prior to the execution of the subprocess. (POSIX only) .. versionchanged:: 3.2 *start_new_session* was added. @@ -598,14 +560,21 @@ Exceptions defined in this module all inherit from :exc:`SubprocessError`. The :exc:`SubprocessError` base class was added. -Security -^^^^^^^^ +Security Considerations +----------------------- + +Unlike some other popen functions, this implementation will never +implicitly call a system shell. This means that all characters, +including shell metacharacters, can safely be passed to child processes. +If the shell is invoked explicitly, via ``shell=True``, it is the application's +responsibility to ensure that all whitespace and metacharacters are +quoted appropriately to avoid +`shell injection <http://en.wikipedia.org/wiki/Shell_injection#Shell_injection>`_ +vulnerabilities. -Unlike some other popen functions, this implementation will never call a -system shell implicitly. This means that all characters, including shell -metacharacters, can safely be passed to child processes. Obviously, if the -shell is invoked explicitly, then it is the application's responsibility to -ensure that all whitespace and metacharacters are quoted appropriately. +When using ``shell=True``, the :func:`shlex.quote` function can be +used to properly escape whitespace and shell metacharacters in strings +that are going to be used to construct shell commands. Popen Objects @@ -631,25 +600,25 @@ Instances of the :class:`Popen` class have the following methods: .. note:: + This will deadlock when using ``stdout=PIPE`` or ``stderr=PIPE`` + and the child process generates enough output to a pipe such that + it blocks waiting for the OS pipe buffer to accept more data. + Use :meth:`Popen.communicate` when using pipes to avoid that. + + .. note:: + The function is implemented using a busy loop (non-blocking call and short sleeps). Use the :mod:`asyncio` module for an asynchronous wait: see :class:`asyncio.create_subprocess_exec`. - .. warning:: - - This will deadlock when using ``stdout=PIPE`` and/or - ``stderr=PIPE`` and the child process generates enough output to - a pipe such that it blocks waiting for the OS pipe buffer to - accept more data. Use :meth:`communicate` to avoid that. - .. versionchanged:: 3.3 *timeout* was added. .. deprecated:: 3.4 - Do not use the undocumented *endtime* parameter. It is was - unintentionally exposed in 3.3 but was intended to be private - for internal use. Use *timeout* instead. + Do not use the *endtime* parameter. It is was unintentionally + exposed in 3.3 but was left undocumented as it was intended to be + private for internal use. Use *timeout* instead. .. method:: Popen.communicate(input=None, timeout=None) @@ -716,13 +685,6 @@ Instances of the :class:`Popen` class have the following methods: The following attributes are also available: -.. warning:: - - Use :meth:`~Popen.communicate` rather than :attr:`.stdin.write <Popen.stdin>`, - :attr:`.stdout.read <Popen.stdout>` or :attr:`.stderr.read <Popen.stderr>` to avoid - deadlocks due to any of the other OS pipe buffers filling up and blocking the - child process. - .. attribute:: Popen.args The *args* argument as it was passed to :class:`Popen` -- a @@ -756,6 +718,13 @@ The following attributes are also available: ``True``, the stream is a text stream, otherwise it is a byte stream. If the *stderr* argument was not :data:`PIPE`, this attribute is ``None``. +.. warning:: + + Use :meth:`~Popen.communicate` rather than :attr:`.stdin.write <Popen.stdin>`, + :attr:`.stdout.read <Popen.stdout>` or :attr:`.stderr.read <Popen.stderr>` to avoid + deadlocks due to any of the other OS pipe buffers filling up and blocking the + child process. + .. attribute:: Popen.pid @@ -772,7 +741,7 @@ The following attributes are also available: hasn't terminated yet. A negative value ``-N`` indicates that the child was terminated by signal - ``N`` (Unix only). + ``N`` (POSIX only). Windows Popen Helpers @@ -1044,7 +1013,7 @@ Replacing functions from the :mod:`popen2` module (child_stdout, child_stdin) = popen2.popen2("somestring", bufsize, mode) ==> - p = Popen(["somestring"], shell=True, bufsize=bufsize, + p = Popen("somestring", shell=True, bufsize=bufsize, stdin=PIPE, stdout=PIPE, close_fds=True) (child_stdout, child_stdin) = (p.stdout, p.stdin) @@ -1097,7 +1066,7 @@ handling consistency are valid for these functions. >>> subprocess.getstatusoutput('/bin/junk') (256, 'sh: /bin/junk: not found') - Availability: Unix & Windows + Availability: POSIX & Windows .. versionchanged:: 3.3.4 Windows support added @@ -1113,7 +1082,7 @@ handling consistency are valid for these functions. >>> subprocess.getoutput('ls /bin/ls') '/bin/ls' - Availability: Unix & Windows + Availability: POSIX & Windows .. versionchanged:: 3.3.4 Windows support added diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 2328a10..bee309e 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -1066,8 +1066,9 @@ always available. statements and for the prompts of :func:`input`; * The interpreter's own prompts and its error messages go to ``stderr``. - By default, these streams are regular text streams as returned by the - :func:`open` function. Their parameters are chosen as follows: + These streams are regular :term:`text files <text file>` like those + returned by the :func:`open` function. Their parameters are chosen as + follows: * The character encoding is platform-dependent. Under Windows, if the stream is interactive (that is, if its :meth:`isatty` method returns ``True``), the @@ -1075,26 +1076,22 @@ always available. platforms, the locale encoding is used (see :meth:`locale.getpreferredencoding`). Under all platforms though, you can override this value by setting the - :envvar:`PYTHONIOENCODING` environment variable. + :envvar:`PYTHONIOENCODING` environment variable before starting Python. * When interactive, standard streams are line-buffered. Otherwise, they are block-buffered like regular text files. You can override this value with the :option:`-u` command-line option. - To write or read binary data from/to the standard streams, use the - underlying binary :data:`~io.TextIOBase.buffer`. For example, to write - bytes to :data:`stdout`, use ``sys.stdout.buffer.write(b'abc')``. Using - :meth:`io.TextIOBase.detach`, streams can be made binary by default. This - function sets :data:`stdin` and :data:`stdout` to binary:: + .. note:: - def make_streams_binary(): - sys.stdin = sys.stdin.detach() - sys.stdout = sys.stdout.detach() + To write or read binary data from/to the standard streams, use the + underlying binary :data:`~io.TextIOBase.buffer` object. For example, to + write bytes to :data:`stdout`, use ``sys.stdout.buffer.write(b'abc')``. - Note that the streams may be replaced with objects (like :class:`io.StringIO`) - that do not support the :attr:`~io.BufferedIOBase.buffer` attribute or the - :meth:`~io.BufferedIOBase.detach` method and can raise :exc:`AttributeError` - or :exc:`io.UnsupportedOperation`. + However, if you are writing a library (and do not control in which + context its code will be executed), be aware that the standard streams + may be replaced with file-like objects like :class:`io.StringIO` which + do not support the :attr:`~io.BufferedIOBase.buffer` attribute. .. data:: __stdin__ diff --git a/Doc/library/venv.rst b/Doc/library/venv.rst index e93f48e..e9ede8b 100644 --- a/Doc/library/venv.rst +++ b/Doc/library/venv.rst @@ -40,11 +40,11 @@ Creating virtual environments A venv is a directory tree which contains Python executable files and other files which indicate that it is a venv. - Common installation tools such as ``Distribute`` and ``pip`` work as + Common installation tools such as ``Setuptools`` and ``pip`` work as expected with venvs - i.e. when a venv is active, they install Python packages into the venv without needing to be told to do so explicitly. Of course, you need to install them into the venv first: this could be - done by running ``distribute_setup.py`` with the venv activated, + done by running ``ez_setup.py`` with the venv activated, followed by running ``easy_install pip``. Alternatively, you could download the source tarballs and run ``python setup.py install`` after unpacking, with the venv activated. diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 4e03711..5e093cc 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -313,14 +313,14 @@ exception, the saved exception is set as the context of the new exception. If the :keyword:`finally` clause executes a :keyword:`return` or :keyword:`break` statement, the saved exception is discarded:: - def f(): - try: - 1/0 - finally: - return 42 - - >>> f() - 42 + >>> def f(): + ... try: + ... 1/0 + ... finally: + ... return 42 + ... + >>> f() + 42 The exception information is not available to the program during execution of the :keyword:`finally` clause. @@ -337,6 +337,20 @@ statement, the :keyword:`finally` clause is also executed 'on the way out.' A reason is a problem with the current implementation --- this restriction may be lifted in the future). +The return value of a function is determined by the last :keyword:`return` +statement executed. Since the :keyword:`finally` clause always executes, a +:keyword:`return` statement executed in the :keyword:`finally` clause will +always be the last one executed:: + + >>> def foo(): + ... try: + ... return 'try' + ... finally: + ... return 'finally' + ... + >>> foo() + 'finally' + Additional information on exceptions can be found in section :ref:`exceptions`, and information on using the :keyword:`raise` statement to generate exceptions may be found in section :ref:`raise`. diff --git a/Doc/reference/datamodel.rst b/Doc/reference/datamodel.rst index ccaa4f7..78dfd79 100644 --- a/Doc/reference/datamodel.rst +++ b/Doc/reference/datamodel.rst @@ -323,8 +323,6 @@ Sequences object: mutable sequence object: mutable pair: assignment; statement - single: delete - statement: del single: subscription single: slicing |