diff options
30 files changed, 587 insertions, 254 deletions
diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst index 535bf12..9a78834 100644 --- a/Doc/c-api/init.rst +++ b/Doc/c-api/init.rst @@ -489,7 +489,7 @@ the fork, and releasing them afterwards. In addition, it resets any :ref:`lock-objects` in the child. When extending or embedding Python, there is no way to inform Python of additional (non-Python) locks that need to be acquired before or reset after a fork. OS facilities such as -:c:func:`posix_atfork` would need to be used to accomplish the same thing. +:c:func:`pthread_atfork` would need to be used to accomplish the same thing. Additionally, when extending or embedding Python, calling :c:func:`fork` directly rather than through :func:`os.fork` (and returning to or calling into Python) may result in a deadlock by one of Python's internal locks diff --git a/Doc/glossary.rst b/Doc/glossary.rst index ec8af62..4e425c3 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -580,6 +580,14 @@ Glossary an :term:`expression` or a one of several constructs with a keyword, such as :keyword:`if`, :keyword:`while` or :keyword:`for`. + struct sequence + A tuple with named elements. Struct sequences expose an interface similiar + to :term:`named tuple` in that elements can either be accessed either by + index or as an attribute. However, they do not have any of the named tuple + methods like :meth:`~collections.somenamedtuple._make` or + :meth:`~collections.somenamedtuple._asdict`. Examples of struct sequences + include :data:`sys.float_info` and the return value of :func:`os.stat`. + triple-quoted string A string which is bound by three instances of either a quotation mark (") or an apostrophe ('). While they don't provide any functionality diff --git a/Doc/howto/pyporting.rst b/Doc/howto/pyporting.rst index 9b8bc2d..124ef33 100644 --- a/Doc/howto/pyporting.rst +++ b/Doc/howto/pyporting.rst @@ -247,8 +247,8 @@ If you are supporting `Python 2.5`_ and newer there are still some features of Python that you can utilize. -``from __future__ import absolute_imports`` -''''''''''''''''''''''''''''''''''''''''''' +``from __future__ import absolute_import`` +'''''''''''''''''''''''''''''''''''''''''' Implicit relative imports (e.g., importing ``spam.bacon`` from within ``spam.eggs`` with the statement ``import bacon``) does not work in Python 3. diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst index c8b0a61..a2bceae 100644 --- a/Doc/library/argparse.rst +++ b/Doc/library/argparse.rst @@ -79,7 +79,7 @@ The first step in using the :mod:`argparse` is creating an >>> parser = argparse.ArgumentParser(description='Process some integers.') The :class:`ArgumentParser` object will hold all the information necessary to -parse the command line into python data types. +parse the command line into Python data types. Adding arguments @@ -97,7 +97,7 @@ used when :meth:`~ArgumentParser.parse_args` is called. For example:: ... const=sum, default=max, ... help='sum the integers (default: find the max)') -Later, calling :meth:`parse_args` will return an object with +Later, calling :meth:`~ArgumentParser.parse_args` will return an object with two attributes, ``integers`` and ``accumulate``. The ``integers`` attribute will be a list of one or more ints, and the ``accumulate`` attribute will be either the :func:`sum` function, if ``--sum`` was specified at the command line, @@ -295,10 +295,10 @@ arguments they contain. For example:: Namespace(f='bar') Arguments read from a file must by default be one per line (but see also -:meth:`convert_arg_line_to_args`) and are treated as if they were in the same -place as the original file referencing argument on the command line. So in the -example above, the expression ``['-f', 'foo', '@args.txt']`` is considered -equivalent to the expression ``['-f', 'foo', '-f', 'bar']``. +:meth:`~ArgumentParser.convert_arg_line_to_args`) and are treated as if they +were in the same place as the original file referencing argument on the command +line. So in the example above, the expression ``['-f', 'foo', '@args.txt']`` +is considered equivalent to the expression ``['-f', 'foo', '-f', 'bar']``. The ``fromfile_prefix_chars=`` argument defaults to ``None``, meaning that arguments will never be treated as file references. @@ -308,11 +308,12 @@ argument_default ^^^^^^^^^^^^^^^^ Generally, argument defaults are specified either by passing a default to -:meth:`add_argument` or by calling the :meth:`set_defaults` methods with a -specific set of name-value pairs. Sometimes however, it may be useful to -specify a single parser-wide default for arguments. This can be accomplished by -passing the ``argument_default=`` keyword argument to :class:`ArgumentParser`. -For example, to globally suppress attribute creation on :meth:`parse_args` +:meth:`~ArgumentParser.add_argument` or by calling the +:meth:`~ArgumentParser.set_defaults` methods with a specific set of name-value +pairs. Sometimes however, it may be useful to specify a single parser-wide +default for arguments. This can be accomplished by passing the +``argument_default=`` keyword argument to :class:`ArgumentParser`. For example, +to globally suppress attribute creation on :meth:`~ArgumentParser.parse_args` calls, we supply ``argument_default=SUPPRESS``:: >>> parser = argparse.ArgumentParser(argument_default=argparse.SUPPRESS) @@ -361,7 +362,13 @@ formatter_class ^^^^^^^^^^^^^^^ :class:`ArgumentParser` objects allow the help formatting to be customized by -specifying an alternate formatting class. +specifying an alternate formatting class. Currently, there are four such +classes: + +.. class:: RawDescriptionHelpFormatter + RawTextHelpFormatter + ArgumentDefaultsHelpFormatter + MetavarTypeHelpFormatter :class:`RawDescriptionHelpFormatter` and :class:`RawTextHelpFormatter` give more control over how textual descriptions are displayed. @@ -598,7 +605,7 @@ The add_argument() method has its own more detailed description below, but in short they are: * `name or flags`_ - Either a name or a list of option strings, e.g. ``foo`` - or ``-f, --foo`` + or ``-f, --foo``. * action_ - The basic type of action to be taken when this argument is encountered at the command line. @@ -630,11 +637,12 @@ The following sections describe how each of these are used. name or flags ^^^^^^^^^^^^^ -The :meth:`add_argument` method must know whether an optional argument, like -``-f`` or ``--foo``, or a positional argument, like a list of filenames, is -expected. The first arguments passed to :meth:`add_argument` must therefore be -either a series of flags, or a simple argument name. For example, an optional -argument could be created like:: +The :meth:`~ArgumentParser.add_argument` method must know whether an optional +argument, like ``-f`` or ``--foo``, or a positional argument, like a list of +filenames, is expected. The first arguments passed to +:meth:`~ArgumentParser.add_argument` must therefore be either a series of +flags, or a simple argument name. For example, an optional argument could +be created like:: >>> parser.add_argument('-f', '--foo') @@ -642,8 +650,9 @@ while a positional argument could be created like:: >>> parser.add_argument('bar') -When :meth:`parse_args` is called, optional arguments will be identified by the -``-`` prefix, and the remaining arguments will be assumed to be positional:: +When :meth:`~ArgumentParser.parse_args` is called, optional arguments will be +identified by the ``-`` prefix, and the remaining arguments will be assumed to +be positional:: >>> parser = argparse.ArgumentParser(prog='PROG') >>> parser.add_argument('-f', '--foo') @@ -663,11 +672,11 @@ action :class:`ArgumentParser` objects associate command-line args with actions. These actions can do just about anything with the command-line args associated with them, though most actions simply add an attribute to the object returned by -:meth:`parse_args`. The ``action`` keyword argument specifies how the -command-line args should be handled. The supported actions are: +:meth:`~ArgumentParser.parse_args`. The ``action`` keyword argument specifies +how the command-line args should be handled. The supported actions are: * ``'store'`` - This just stores the argument's value. This is the default - action. For example:: + action. For example:: >>> parser = argparse.ArgumentParser() >>> parser.add_argument('--foo') @@ -675,9 +684,9 @@ command-line args should be handled. The supported actions are: Namespace(foo='1') * ``'store_const'`` - This stores the value specified by the const_ keyword - argument. (Note that the const_ keyword argument defaults to the rather - unhelpful ``None``.) The ``'store_const'`` action is most commonly used with - optional arguments that specify some sort of flag. For example:: + argument. (Note that the const_ keyword argument defaults to the rather + unhelpful ``None``.) The ``'store_const'`` action is most commonly used with + optional arguments that specify some sort of flag. For example:: >>> parser = argparse.ArgumentParser() >>> parser.add_argument('--foo', action='store_const', const=42) @@ -716,8 +725,8 @@ command-line args should be handled. The supported actions are: Namespace(types=[<type 'str'>, <type 'int'>]) * ``'version'`` - This expects a ``version=`` keyword argument in the - :meth:`add_argument` call, and prints version information and exits when - invoked. + :meth:`~ArgumentParser.add_argument` call, and prints version information + and exits when invoked. >>> import argparse >>> parser = argparse.ArgumentParser(prog='PROG') @@ -733,11 +742,12 @@ the Action API. The easiest way to do this is to extend * ``parser`` - The ArgumentParser object which contains this action. * ``namespace`` - The namespace object that will be returned by - :meth:`parse_args`. Most actions add an attribute to this object. + :meth:`~ArgumentParser.parse_args`. Most actions add an attribute to this + object. * ``values`` - The associated command-line args, with any type-conversions applied. (Type-conversions are specified with the type_ keyword argument to - :meth:`add_argument`. + :meth:`~ArgumentParser.add_argument`. * ``option_string`` - The option string that was used to invoke this action. The ``option_string`` argument is optional, and will be absent if the action @@ -765,7 +775,7 @@ nargs ArgumentParser objects usually associate a single command-line argument with a single action to be taken. The ``nargs`` keyword argument associates a -different number of command-line arguments with a single action.. The supported +different number of command-line arguments with a single action. The supported values are: * N (an integer). N args from the command line will be gathered together into a @@ -844,21 +854,20 @@ will be consumed and a single item (not a list) will be produced. const ^^^^^ -The ``const`` argument of :meth:`add_argument` is used to hold constant values -that are not read from the command line but are required for the various -ArgumentParser actions. The two most common uses of it are: +The ``const`` argument of :meth:`~ArgumentParser.add_argument` is used to hold +constant values that are not read from the command line but are required for +the various :class:`ArgumentParser` actions. The two most common uses of it are: -* When :meth:`add_argument` is called with ``action='store_const'`` or - ``action='append_const'``. These actions add the ``const`` value to one of - the attributes of the object returned by :meth:`parse_args`. See the action_ - description for examples. +* When :meth:`~ArgumentParser.add_argument` is called with + ``action='store_const'`` or ``action='append_const'``. These actions add the + ``const`` value to one of the attributes of the object returned by :meth:`~ArgumentParser.parse_args`. See the action_ description for examples. -* When :meth:`add_argument` is called with option strings (like ``-f`` or - ``--foo``) and ``nargs='?'``. This creates an optional argument that can be - followed by zero or one command-line args. When parsing the command line, if - the option string is encountered with no command-line arg following it, the - value of ``const`` will be assumed instead. See the nargs_ description for - examples. +* When :meth:`~ArgumentParser.add_argument` is called with option strings + (like ``-f`` or ``--foo``) and ``nargs='?'``. This creates an optional + argument that can be followed by zero or one command-line args. + When parsing the command line, if the option string is encountered with no + command-line arg following it, the value of ``const`` will be assumed instead. + See the nargs_ description for examples. The ``const`` keyword argument defaults to ``None``. @@ -867,10 +876,11 @@ default ^^^^^^^ All optional arguments and some positional arguments may be omitted at the -command line. The ``default`` keyword argument of :meth:`add_argument`, whose -value defaults to ``None``, specifies what value should be used if the -command-line arg is not present. For optional arguments, the ``default`` value -is used when the option string was not present at the command line:: +command line. The ``default`` keyword argument of +:meth:`~ArgumentParser.add_argument`, whose value defaults to ``None``, +specifies what value should be used if the command-line arg is not present. +For optional arguments, the ``default`` value is used when the option string +was not present at the command line:: >>> parser = argparse.ArgumentParser() >>> parser.add_argument('--foo', default=42) @@ -904,12 +914,12 @@ command-line argument was not present.:: type ^^^^ -By default, ArgumentParser objects read command-line args in as simple strings. -However, quite often the command-line string should instead be interpreted as -another type, like a :class:`float` or :class:`int`. The ``type`` keyword -argument of :meth:`add_argument` allows any necessary type-checking and -type-conversions to be performed. Common built-in types and functions can be -used directly as the value of the ``type`` argument:: +By default, :class:`ArgumentParser` objects read command-line args in as simple +strings. However, quite often the command-line string should instead be +interpreted as another type, like a :class:`float` or :class:`int`. The +``type`` keyword argument of :meth:`~ArgumentParser.add_argument` allows any +necessary type-checking and type-conversions to be performed. Common built-in +types and functions can be used directly as the value of the ``type`` argument:: >>> parser = argparse.ArgumentParser() >>> parser.add_argument('foo', type=int) @@ -965,9 +975,9 @@ choices Some command-line args should be selected from a restricted set of values. These can be handled by passing a container object as the ``choices`` keyword -argument to :meth:`add_argument`. When the command line is parsed, arg values -will be checked, and an error message will be displayed if the arg was not one -of the acceptable values:: +argument to :meth:`~ArgumentParser.add_argument`. When the command line is +parsed, arg values will be checked, and an error message will be displayed if +the arg was not one of the acceptable values:: >>> parser = argparse.ArgumentParser(prog='PROG') >>> parser.add_argument('foo', choices='abc') @@ -997,10 +1007,10 @@ etc. are all supported. required ^^^^^^^^ -In general, the argparse module assumes that flags like ``-f`` and ``--bar`` +In general, the :mod:`argparse` module assumes that flags like ``-f`` and ``--bar`` indicate *optional* arguments, which can always be omitted at the command line. To make an option *required*, ``True`` can be specified for the ``required=`` -keyword argument to :meth:`add_argument`:: +keyword argument to :meth:`~ArgumentParser.add_argument`:: >>> parser = argparse.ArgumentParser() >>> parser.add_argument('--foo', required=True) @@ -1010,8 +1020,9 @@ keyword argument to :meth:`add_argument`:: usage: argparse.py [-h] [--foo FOO] argparse.py: error: option --foo is required -As the example shows, if an option is marked as ``required``, :meth:`parse_args` -will report an error if that option is not present at the command line. +As the example shows, if an option is marked as ``required``, +:meth:`~ArgumentParser.parse_args` will report an error if that option is not +present at the command line. .. note:: @@ -1045,7 +1056,7 @@ argument:: The ``help`` strings can include various format specifiers to avoid repetition of things like the program name or the argument default_. The available specifiers include the program name, ``%(prog)s`` and most keyword arguments to -:meth:`add_argument`, e.g. ``%(default)s``, ``%(type)s``, etc.:: +:meth:`~ArgumentParser.add_argument`, e.g. ``%(default)s``, ``%(type)s``, etc.:: >>> parser = argparse.ArgumentParser(prog='frobble') >>> parser.add_argument('bar', nargs='?', type=int, default=42, @@ -1105,8 +1116,8 @@ An alternative name can be specified with ``metavar``:: --foo YYY Note that ``metavar`` only changes the *displayed* name - the name of the -attribute on the :meth:`parse_args` object is still determined by the dest_ -value. +attribute on the :meth:`~ArgumentParser.parse_args` object is still determined +by the dest_ value. Different values of ``nargs`` may cause the metavar to be used multiple times. Providing a tuple to ``metavar`` specifies a different display for each of the @@ -1128,10 +1139,11 @@ dest ^^^^ Most :class:`ArgumentParser` actions add some value as an attribute of the -object returned by :meth:`parse_args`. The name of this attribute is determined -by the ``dest`` keyword argument of :meth:`add_argument`. For positional -argument actions, ``dest`` is normally supplied as the first argument to -:meth:`add_argument`:: +object returned by :meth:`~ArgumentParser.parse_args`. The name of this +attribute is determined by the ``dest`` keyword argument of +:meth:`~ArgumentParser.add_argument`. For positional argument actions, +``dest`` is normally supplied as the first argument to +:meth:`~ArgumentParser.add_argument`:: >>> parser = argparse.ArgumentParser() >>> parser.add_argument('bar') @@ -1182,9 +1194,9 @@ The parse_args() method Option value syntax ^^^^^^^^^^^^^^^^^^^ -The :meth:`parse_args` method supports several ways of specifying the value of -an option (if it takes one). In the simplest case, the option and its value are -passed as two separate arguments:: +The :meth:`~ArgumentParser.parse_args` method supports several ways of +specifying the value of an option (if it takes one). In the simplest case, the +option and its value are passed as two separate arguments:: >>> parser = argparse.ArgumentParser(prog='PROG') >>> parser.add_argument('-x') @@ -1221,10 +1233,10 @@ as long as only the last option (or none of them) requires a value:: Invalid arguments ^^^^^^^^^^^^^^^^^ -While parsing the command line, ``parse_args`` checks for a variety of errors, -including ambiguous options, invalid types, invalid options, wrong number of -positional arguments, etc. When it encounters such an error, it exits and -prints the error along with a usage message:: +While parsing the command line, :meth:`~ArgumentParser.parse_args` checks for a +variety of errors, including ambiguous options, invalid types, invalid options, +wrong number of positional arguments, etc. When it encounters such an error, +it exits and prints the error along with a usage message:: >>> parser = argparse.ArgumentParser(prog='PROG') >>> parser.add_argument('--foo', type=int) @@ -1249,13 +1261,13 @@ prints the error along with a usage message:: Arguments containing ``"-"`` ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -The ``parse_args`` method attempts to give errors whenever the user has clearly -made a mistake, but some situations are inherently ambiguous. For example, the -command-line arg ``'-1'`` could either be an attempt to specify an option or an -attempt to provide a positional argument. The ``parse_args`` method is cautious -here: positional arguments may only begin with ``'-'`` if they look like -negative numbers and there are no options in the parser that look like negative -numbers:: +The :meth:`~ArgumentParser.parse_args` method attempts to give errors whenever +the user has clearly made a mistake, but some situations are inherently +ambiguous. For example, the command-line arg ``'-1'`` could either be an +attempt to specify an option or an attempt to provide a positional argument. +The :meth:`~ArgumentParser.parse_args` method is cautious here: positional +arguments may only begin with ``'-'`` if they look like negative numbers and +there are no options in the parser that look like negative numbers:: >>> parser = argparse.ArgumentParser(prog='PROG') >>> parser.add_argument('-x') @@ -1289,7 +1301,8 @@ numbers:: If you have positional arguments that must begin with ``'-'`` and don't look like negative numbers, you can insert the pseudo-argument ``'--'`` which tells -``parse_args`` that everything after that is a positional argument:: +:meth:`~ArgumentParser.parse_args` that everything after that is a positional +argument:: >>> parser.parse_args(['--', '-f']) Namespace(foo='-f', one=None) @@ -1298,8 +1311,8 @@ like negative numbers, you can insert the pseudo-argument ``'--'`` which tells Argument abbreviations ^^^^^^^^^^^^^^^^^^^^^^ -The :meth:`parse_args` method allows long options to be abbreviated if the -abbreviation is unambiguous:: +The :meth:`~ArgumentParser.parse_args` method allows long options to be +abbreviated if the abbreviation is unambiguous:: >>> parser = argparse.ArgumentParser(prog='PROG') >>> parser.add_argument('-bacon') @@ -1320,7 +1333,8 @@ Beyond ``sys.argv`` Sometimes it may be useful to have an ArgumentParser parse args other than those of :data:`sys.argv`. This can be accomplished by passing a list of strings to -``parse_args``. This is useful for testing at the interactive prompt:: +:meth:`~ArgumentParser.parse_args`. This is useful for testing at the +interactive prompt:: >>> parser = argparse.ArgumentParser() >>> parser.add_argument( @@ -1338,11 +1352,11 @@ of :data:`sys.argv`. This can be accomplished by passing a list of strings to The Namespace object ^^^^^^^^^^^^^^^^^^^^ -By default, :meth:`parse_args` will return a new object of type :class:`Namespace` -where the necessary attributes have been set. This class is deliberately simple, -just an :class:`object` subclass with a readable string representation. If you -prefer to have dict-like view of the attributes, you can use the standard Python -idiom via :func:`vars`:: +By default, :meth:`~ArgumentParser.parse_args` will return a new object of type +:class:`Namespace` where the necessary attributes have been set. This class is +deliberately simple, just an :class:`object` subclass with a readable string +representation. If you prefer to have dict-like view of the attributes, you +can use the standard Python idiom via :func:`vars`:: >>> parser = argparse.ArgumentParser() >>> parser.add_argument('--foo') @@ -1381,9 +1395,9 @@ Sub-commands :class:`ArgumentParser` supports the creation of such sub-commands with the :meth:`add_subparsers` method. The :meth:`add_subparsers` method is normally called with no arguments and returns an special action object. This object - has a single method, ``add_parser``, which takes a command name and any - :class:`ArgumentParser` constructor arguments, and returns an - :class:`ArgumentParser` object that can be modified as usual. + has a single method, :meth:`~ArgumentParser.add_parser`, which takes a + command name and any :class:`ArgumentParser` constructor arguments, and + returns an :class:`ArgumentParser` object that can be modified as usual. Some example usage:: @@ -1417,7 +1431,7 @@ Sub-commands for that particular parser will be printed. The help message will not include parent parser or sibling parser messages. (A help message for each subparser command, however, can be given by supplying the ``help=`` argument - to ``add_parser`` as above.) + to :meth:`add_parser` as above.) :: @@ -1617,9 +1631,9 @@ Mutual exclusion .. method:: add_mutually_exclusive_group(required=False) - Create a mutually exclusive group. argparse will make sure that only one of - the arguments in the mutually exclusive group was present on the command - line:: + Create a mutually exclusive group. :mod:`argparse` will make sure that only + one of the arguments in the mutually exclusive group was present on the + command line:: >>> parser = argparse.ArgumentParser(prog='PROG') >>> group = parser.add_mutually_exclusive_group() @@ -1646,7 +1660,8 @@ Mutual exclusion PROG: error: one of the arguments --foo --bar is required Note that currently mutually exclusive argument groups do not support the - *title* and *description* arguments of :meth:`add_argument_group`. + *title* and *description* arguments of + :meth:`~ArgumentParser.add_argument_group`. Parser defaults @@ -1656,7 +1671,7 @@ Parser defaults Most of the time, the attributes of the object returned by :meth:`parse_args` will be fully determined by inspecting the command-line args and the argument - actions. :meth:`ArgumentParser.set_defaults` allows some additional + actions. :meth:`set_defaults` allows some additional attributes that are determined without any inspection of the command line to be added:: @@ -1693,9 +1708,9 @@ Parser defaults Printing help ^^^^^^^^^^^^^ -In most typical applications, :meth:`parse_args` will take care of formatting -and printing any usage or error messages. However, several formatting methods -are available: +In most typical applications, :meth:`~ArgumentParser.parse_args` will take +care of formatting and printing any usage or error messages. However, several +formatting methods are available: .. method:: ArgumentParser.print_usage(file=None) @@ -1730,7 +1745,7 @@ Partial parsing Sometimes a script may only parse a few of the command-line arguments, passing the remaining arguments on to another script or program. In these cases, the -:meth:`parse_known_args` method can be useful. It works much like +:meth:`~ArgumentParser.parse_known_args` method can be useful. It works much like :meth:`~ArgumentParser.parse_args` except that it does not produce an error when extra arguments are present. Instead, it returns a two item tuple containing the populated namespace and the list of remaining argument strings. @@ -1786,17 +1801,17 @@ Exiting methods Upgrading optparse code ----------------------- -Originally, the argparse module had attempted to maintain compatibility with -optparse. However, optparse was difficult to extend transparently, particularly -with the changes required to support the new ``nargs=`` specifiers and better -usage messages. When most everything in optparse had either been copy-pasted -over or monkey-patched, it no longer seemed practical to try to maintain the -backwards compatibility. +Originally, the :mod:`argparse` module had attempted to maintain compatibility +with :mod:`optparse`. However, :mod:`optparse` was difficult to extend +transparently, particularly with the changes required to support the new +``nargs=`` specifiers and better usage messages. When most everything in +:mod:`optparse` had either been copy-pasted over or monkey-patched, it no +longer seemed practical to try to maintain the backwards compatibility. -A partial upgrade path from optparse to argparse: +A partial upgrade path from :mod:`optparse` to :mod:`argparse`: -* Replace all ``add_option()`` calls with :meth:`ArgumentParser.add_argument` - calls. +* Replace all :meth:`optparse.OptionParser.add_option` calls with + :meth:`ArgumentParser.add_argument` calls. * Replace ``options, args = parser.parse_args()`` with ``args = parser.parse_args()`` and add additional :meth:`ArgumentParser.add_argument` @@ -1813,7 +1828,7 @@ A partial upgrade path from optparse to argparse: :exc:`ArgumentError`. * Replace strings with implicit arguments such as ``%default`` or ``%prog`` with - the standard python syntax to use dictionaries to format strings, that is, + the standard Python syntax to use dictionaries to format strings, that is, ``%(default)s`` and ``%(prog)s``. * Replace the OptionParser constructor ``version`` argument with a call to diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst index 07378d1..9cdad6e 100644 --- a/Doc/library/itertools.rst +++ b/Doc/library/itertools.rst @@ -104,9 +104,13 @@ loops that truncate the stream. total = func(total, element) yield total - Uses for the *func* argument include :func:`min` for a running minimum, - :func:`max` for a running maximum, and :func:`operator.mul` for a running - product:: + There are a number of uses for the *func* argument. It can be set to + :func:`min` for a running minimum, :func:`max` for a running maximum, or + :func:`operator.mul` for a running product. Amortization tables can be + built by accumulating interest and applying payments. First-order + `recurrence relations <http://en.wikipedia.org/wiki/Recurrence_relation>`_ + can be modeled by supplying the initial value in the iterable and using only + the accumulated total in *func* argument:: >>> data = [3, 4, 6, 2, 1, 9, 0, 7, 5, 8] >>> list(accumulate(data, operator.mul)) # running product @@ -119,6 +123,17 @@ loops that truncate the stream. >>> list(accumulate(cashflows, lambda bal, pmt: bal*1.05 + pmt)) [1000, 960.0, 918.0, 873.9000000000001, 827.5950000000001] + # Chaotic recurrence relation http://en.wikipedia.org/wiki/Logistic_map + >>> logistic_map = lambda x, _: r * x * (1 - x) + >>> r = 3.8 + >>> x0 = 0.4 + >>> inputs = repeat(x0, 36) # only the initial value is used + >>> [format(x, '.2f') for x in accumulate(inputs, logistic_map)] + ['0.40', '0.91', '0.30', '0.81', '0.60', '0.92', '0.29', '0.79', '0.63', + '0.88' ,'0.39', '0.90', '0.33', '0.84', '0.52', '0.95', '0.18', '0.57', + '0.93', '0.25', '0.71', '0.79', '0.63', '0.88', '0.39', '0.91', '0.32', + '0.83', '0.54', '0.95', '0.20', '0.60', '0.91', '0.30', '0.80', '0.60'] + .. versionadded:: 3.2 .. versionchanged:: 3.3 diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst index 3faf60f..ab5668c 100644 --- a/Doc/library/subprocess.rst +++ b/Doc/library/subprocess.rst @@ -255,7 +255,7 @@ This module defines one class called :class:`Popen`: Convenience Functions ^^^^^^^^^^^^^^^^^^^^^ -This module also defines four shortcut functions: +This module also defines the following shortcut functions: .. function:: call(*popenargs, timeout=None, **kwargs) diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst index 5ecc46f..4b1f02e 100644 --- a/Doc/library/sys.rst +++ b/Doc/library/sys.rst @@ -224,8 +224,8 @@ always available. .. data:: flags - The struct sequence *flags* exposes the status of command line flags. The - attributes are read only. + The :term:`struct sequence` *flags* exposes the status of command line + flags. The attributes are read only. ============================= ============================= attribute flag @@ -249,12 +249,12 @@ always available. .. data:: float_info - A structseq holding information about the float type. It contains low level - information about the precision and internal representation. The values - correspond to the various floating-point constants defined in the standard - header file :file:`float.h` for the 'C' programming language; see section - 5.2.4.2.2 of the 1999 ISO/IEC C standard [C99]_, 'Characteristics of - floating types', for details. + A :term:`struct sequence` holding information about the float type. It + contains low level information about the precision and internal + representation. The values correspond to the various floating-point + constants defined in the standard header file :file:`float.h` for the 'C' + programming language; see section 5.2.4.2.2 of the 1999 ISO/IEC C standard + [C99]_, 'Characteristics of floating types', for details. +---------------------+----------------+--------------------------------------------------+ | attribute | float.h macro | explanation | @@ -500,8 +500,9 @@ always available. .. data:: hash_info - A structseq giving parameters of the numeric hash implementation. For - more details about hashing of numeric types, see :ref:`numeric-hash`. + A :term:`struct sequence` giving parameters of the numeric hash + implementation. For more details about hashing of numeric types, see + :ref:`numeric-hash`. +---------------------+--------------------------------------------------+ | attribute | explanation | @@ -542,8 +543,8 @@ always available. .. data:: int_info - A struct sequence that holds information about Python's - internal representation of integers. The attributes are read only. + A :term:`struct sequence` that holds information about Python's internal + representation of integers. The attributes are read only. +-------------------------+----------------------------------------------+ | attribute | explanation | diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst index df47045..dd2226d 100644 --- a/Doc/library/threading.rst +++ b/Doc/library/threading.rst @@ -175,6 +175,30 @@ This module defines the following functions and objects: Availability: Windows, systems with POSIX threads. +.. function:: _info() + + Return a dictionary with informations about the thread implementation. + The ``'name'`` key gives the name of the thread implementation (string): + + * ``'nt'``: Windows threads + * ``'os2'``: OS/2 threads + * ``'pthread'``: POSIX threads + * ``'solaris'``: Solaris threads + + POSIX threads have two more keys: + + * ``'lock_implementation'`` (string): name of the lock + implementation + + * ``'semaphore'``: a lock uses a semaphore + * ``'mutex+cond'``: a lock uses a mutex and a condition variable + + * ``'pthread_version'`` (string, optional): name and version of the pthread + library + + .. versionadded:: 3.3 + + This module also defines the following constant: .. data:: TIMEOUT_MAX diff --git a/Doc/whatsnew/3.3.rst b/Doc/whatsnew/3.3.rst index 602b2b7..d3c9a97 100644 --- a/Doc/whatsnew/3.3.rst +++ b/Doc/whatsnew/3.3.rst @@ -112,6 +112,14 @@ connection when done:: (Contributed by Giampaolo Rodolà in :issue:`9795`) +threading +--------- + +* The :mod:`threading` module has a new :func:`~threading._info` function which + provides informations about the thread implementation. + + (:issue:`11223`) + Optimizations ============= diff --git a/Include/pythread.h b/Include/pythread.h index 9806c61..9a35e5d 100644 --- a/Include/pythread.h +++ b/Include/pythread.h @@ -32,7 +32,7 @@ PyAPI_FUNC(int) PyThread_acquire_lock(PyThread_type_lock, int); on a lock (see PyThread_acquire_lock_timed() below). PY_TIMEOUT_MAX is the highest usable value (in microseconds) of that type, and depends on the system threading API. - + NOTE: this isn't the same value as `_thread.TIMEOUT_MAX`. The _thread module exposes a higher-level API, with timeouts expressed in seconds and floating-point numbers allowed. @@ -74,6 +74,8 @@ PyAPI_FUNC(void) PyThread_release_lock(PyThread_type_lock); PyAPI_FUNC(size_t) PyThread_get_stacksize(void); PyAPI_FUNC(int) PyThread_set_stacksize(size_t); +PyAPI_FUNC(PyObject*) _PyThread_Info(void); + /* Thread Local Storage (TLS) API */ PyAPI_FUNC(int) PyThread_create_key(void); PyAPI_FUNC(void) PyThread_delete_key(int); diff --git a/Lib/_dummy_thread.py b/Lib/_dummy_thread.py index 13b1f26..f2465a9 100644 --- a/Lib/_dummy_thread.py +++ b/Lib/_dummy_thread.py @@ -149,3 +149,6 @@ def interrupt_main(): else: global _interrupt _interrupt = True + +def info(): + return {'name': 'dummy'} diff --git a/Lib/collections/__init__.py b/Lib/collections/__init__.py index a6bcaea..c0c2180 100644 --- a/Lib/collections/__init__.py +++ b/Lib/collections/__init__.py @@ -152,17 +152,6 @@ class OrderedDict(dict): link.next = first root.next = first.prev = link - def __reduce__(self): - 'Return state information for pickling' - items = [[k, self[k]] for k in self] - tmp = self.__map, self.__root, self.__hardroot - del self.__map, self.__root, self.__hardroot - inst_dict = vars(self).copy() - self.__map, self.__root, self.__hardroot = tmp - if inst_dict: - return (self.__class__, (items,), inst_dict) - return self.__class__, (items,) - def __sizeof__(self): sizeof = _sys.getsizeof n = len(self) + 1 # number of links including root @@ -203,6 +192,16 @@ class OrderedDict(dict): return '%s()' % (self.__class__.__name__,) return '%s(%r)' % (self.__class__.__name__, list(self.items())) + def __reduce__(self): + 'Return state information for pickling' + items = [[k, self[k]] for k in self] + inst_dict = vars(self).copy() + for k in vars(OrderedDict()): + inst_dict.pop(k, None) + if inst_dict: + return (self.__class__, (items,), inst_dict) + return self.__class__, (items,) + def copy(self): 'od.copy() -> a shallow copy of od' return self.__class__(self) diff --git a/Lib/test/string_tests.py b/Lib/test/string_tests.py index 1cc36d8..fca38c3 100644 --- a/Lib/test/string_tests.py +++ b/Lib/test/string_tests.py @@ -1183,6 +1183,63 @@ class MixinStrUnicodeUserStringTest: self.checkraises(ValueError, S, 'rpartition', '') self.checkraises(TypeError, S, 'rpartition', None) + def test_none_arguments(self): + # issue 11828 + s = 'hello' + self.checkequal(2, s, 'find', 'l', None) + self.checkequal(3, s, 'find', 'l', -2, None) + self.checkequal(2, s, 'find', 'l', None, -2) + self.checkequal(0, s, 'find', 'h', None, None) + + self.checkequal(3, s, 'rfind', 'l', None) + self.checkequal(3, s, 'rfind', 'l', -2, None) + self.checkequal(2, s, 'rfind', 'l', None, -2) + self.checkequal(0, s, 'rfind', 'h', None, None) + + self.checkequal(2, s, 'index', 'l', None) + self.checkequal(3, s, 'index', 'l', -2, None) + self.checkequal(2, s, 'index', 'l', None, -2) + self.checkequal(0, s, 'index', 'h', None, None) + + self.checkequal(3, s, 'rindex', 'l', None) + self.checkequal(3, s, 'rindex', 'l', -2, None) + self.checkequal(2, s, 'rindex', 'l', None, -2) + self.checkequal(0, s, 'rindex', 'h', None, None) + + self.checkequal(2, s, 'count', 'l', None) + self.checkequal(1, s, 'count', 'l', -2, None) + self.checkequal(1, s, 'count', 'l', None, -2) + self.checkequal(0, s, 'count', 'x', None, None) + + self.checkequal(True, s, 'endswith', 'o', None) + self.checkequal(True, s, 'endswith', 'lo', -2, None) + self.checkequal(True, s, 'endswith', 'l', None, -2) + self.checkequal(False, s, 'endswith', 'x', None, None) + + self.checkequal(True, s, 'startswith', 'h', None) + self.checkequal(True, s, 'startswith', 'l', -2, None) + self.checkequal(True, s, 'startswith', 'h', None, -2) + self.checkequal(False, s, 'startswith', 'x', None, None) + + def test_find_etc_raise_correct_error_messages(self): + # issue 11828 + s = 'hello' + x = 'x' + self.assertRaisesRegex(TypeError, r'^find\(', s.find, + x, None, None, None) + self.assertRaisesRegex(TypeError, r'^rfind\(', s.rfind, + x, None, None, None) + self.assertRaisesRegex(TypeError, r'^index\(', s.index, + x, None, None, None) + self.assertRaisesRegex(TypeError, r'^rindex\(', s.rindex, + x, None, None, None) + self.assertRaisesRegex(TypeError, r'^count\(', s.count, + x, None, None, None) + self.assertRaisesRegex(TypeError, r'^startswith\(', s.startswith, + x, None, None, None) + self.assertRaisesRegex(TypeError, r'^endswith\(', s.endswith, + x, None, None, None) + class MixinStrUnicodeTest: # Additional tests that only work with str and unicode. diff --git a/Lib/test/test_bytes.py b/Lib/test/test_bytes.py index 0b70c3a..5c6238b 100644 --- a/Lib/test/test_bytes.py +++ b/Lib/test/test_bytes.py @@ -478,6 +478,68 @@ class BaseBytesTest(unittest.TestCase): self.assertRaises(ValueError, self.type2test.maketrans, b'abc', b'xyzq') self.assertRaises(TypeError, self.type2test.maketrans, 'abc', 'def') + def test_none_arguments(self): + # issue 11828 + b = self.type2test(b'hello') + l = self.type2test(b'l') + h = self.type2test(b'h') + x = self.type2test(b'x') + o = self.type2test(b'o') + + self.assertEqual(2, b.find(l, None)) + self.assertEqual(3, b.find(l, -2, None)) + self.assertEqual(2, b.find(l, None, -2)) + self.assertEqual(0, b.find(h, None, None)) + + self.assertEqual(3, b.rfind(l, None)) + self.assertEqual(3, b.rfind(l, -2, None)) + self.assertEqual(2, b.rfind(l, None, -2)) + self.assertEqual(0, b.rfind(h, None, None)) + + self.assertEqual(2, b.index(l, None)) + self.assertEqual(3, b.index(l, -2, None)) + self.assertEqual(2, b.index(l, None, -2)) + self.assertEqual(0, b.index(h, None, None)) + + self.assertEqual(3, b.rindex(l, None)) + self.assertEqual(3, b.rindex(l, -2, None)) + self.assertEqual(2, b.rindex(l, None, -2)) + self.assertEqual(0, b.rindex(h, None, None)) + + self.assertEqual(2, b.count(l, None)) + self.assertEqual(1, b.count(l, -2, None)) + self.assertEqual(1, b.count(l, None, -2)) + self.assertEqual(0, b.count(x, None, None)) + + self.assertEqual(True, b.endswith(o, None)) + self.assertEqual(True, b.endswith(o, -2, None)) + self.assertEqual(True, b.endswith(l, None, -2)) + self.assertEqual(False, b.endswith(x, None, None)) + + self.assertEqual(True, b.startswith(h, None)) + self.assertEqual(True, b.startswith(l, -2, None)) + self.assertEqual(True, b.startswith(h, None, -2)) + self.assertEqual(False, b.startswith(x, None, None)) + + def test_find_etc_raise_correct_error_messages(self): + # issue 11828 + b = self.type2test(b'hello') + x = self.type2test(b'x') + self.assertRaisesRegex(TypeError, r'\bfind\b', b.find, + x, None, None, None) + self.assertRaisesRegex(TypeError, r'\brfind\b', b.rfind, + x, None, None, None) + self.assertRaisesRegex(TypeError, r'\bindex\b', b.index, + x, None, None, None) + self.assertRaisesRegex(TypeError, r'\brindex\b', b.rindex, + x, None, None, None) + self.assertRaisesRegex(TypeError, r'\bcount\b', b.count, + x, None, None, None) + self.assertRaisesRegex(TypeError, r'\bstartswith\b', b.startswith, + x, None, None, None) + self.assertRaisesRegex(TypeError, r'\bendswith\b', b.endswith, + x, None, None, None) + class BytesTest(BaseBytesTest): type2test = bytes diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py index b2e3327..48add6d 100644 --- a/Lib/test/test_logging.py +++ b/Lib/test/test_logging.py @@ -2255,7 +2255,7 @@ class RecordingHandler(logging.NullHandler): class ShutdownTest(BaseTest): - """Tets suite for the shutdown method.""" + """Test suite for the shutdown method.""" def setUp(self): super(ShutdownTest, self).setUp() @@ -2342,7 +2342,7 @@ class ShutdownTest(BaseTest): class ModuleLevelMiscTest(BaseTest): - """Tets suite for some module level methods.""" + """Test suite for some module level methods.""" def test_disable(self): old_disable = logging.root.manager.disable @@ -2356,7 +2356,7 @@ class ModuleLevelMiscTest(BaseTest): def _test_log(self, method, level=None): called = [] patch(self, logging, 'basicConfig', - lambda *a, **kw: called.append(a, kw)) + lambda *a, **kw: called.append((a, kw))) recording = RecordingHandler() logging.root.addHandler(recording) @@ -2410,19 +2410,30 @@ class ModuleLevelMiscTest(BaseTest): class BasicConfigTest(unittest.TestCase): - """Tets suite for logging.basicConfig.""" + """Test suite for logging.basicConfig.""" def setUp(self): super(BasicConfigTest, self).setUp() - handlers = logging.root.handlers - self.addCleanup(lambda: setattr(logging.root, 'handlers', handlers)) + self.handlers = logging.root.handlers + self.saved_handlers = logging._handlers.copy() + self.saved_handler_list = logging._handlerList[:] + self.original_logging_level = logging.root.level + self.addCleanup(self.cleanup) logging.root.handlers = [] def tearDown(self): - logging.shutdown() + for h in logging.root.handlers[:]: + logging.root.removeHandler(h) + h.close() super(BasicConfigTest, self).tearDown() - @unittest.skipIf(True, "test disabled, issue #11557") + def cleanup(self): + setattr(logging.root, 'handlers', self.handlers) + logging._handlers.clear() + logging._handlers.update(self.saved_handlers) + logging._handlerList[:] = self.saved_handler_list + logging.root.level = self.original_logging_level + def test_no_kwargs(self): logging.basicConfig() @@ -2441,7 +2452,7 @@ class BasicConfigTest(unittest.TestCase): self.assertIsInstance(formatter._style, logging.PercentStyle) # level is not explicitely set - self.assertEqual(logging.root.level, logging.WARNING) + self.assertEqual(logging.root.level, self.original_logging_level) def test_filename(self): logging.basicConfig(filename='test.log') diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py index 35aa7fa..5432412 100644 --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -27,12 +27,15 @@ except ImportError: # and unmaintained) linuxthreads threading library. There's an issue # when combining linuxthreads with a failed execv call: see # http://bugs.python.org/issue4970. -if (hasattr(os, "confstr_names") and - "CS_GNU_LIBPTHREAD_VERSION" in os.confstr_names): - libpthread = os.confstr("CS_GNU_LIBPTHREAD_VERSION") - USING_LINUXTHREADS= libpthread.startswith("linuxthreads") -else: - USING_LINUXTHREADS= False +USING_LINUXTHREADS = False +if threading: + info = threading._info() + try: + pthread_version = info['pthread_version'] + except KeyError: + pass + else: + USING_LINUXTHREADS = pthread_version.startswith("linuxthreads") # Tests creating TESTFN class FileTests(unittest.TestCase): diff --git a/Lib/test/test_threading.py b/Lib/test/test_threading.py index c107652..fd63d39 100644 --- a/Lib/test/test_threading.py +++ b/Lib/test/test_threading.py @@ -718,6 +718,17 @@ class BoundedSemaphoreTests(lock_tests.BoundedSemaphoreTests): class BarrierTests(lock_tests.BarrierTests): barriertype = staticmethod(threading.Barrier) + +class MiscTests(unittest.TestCase): + def test_info(self): + info = threading._info() + self.assertIn(info['name'], + 'nt os2 pthread solaris'.split()) + if info['name'] == 'pthread': + self.assertIn(info['lock_implementation'], + ('semaphore', 'mutex+cond')) + + def test_main(): test.support.run_unittest(LockTests, PyRLockTests, CRLockTests, EventTests, ConditionAsRLockTests, ConditionTests, @@ -725,7 +736,7 @@ def test_main(): ThreadTests, ThreadJoinOnShutdown, ThreadingExceptionTests, - BarrierTests + BarrierTests, MiscTests, ) if __name__ == "__main__": diff --git a/Lib/test/test_threadsignals.py b/Lib/test/test_threadsignals.py index 46e405a..b0bc607 100644 --- a/Lib/test/test_threadsignals.py +++ b/Lib/test/test_threadsignals.py @@ -14,6 +14,9 @@ if sys.platform[:3] in ('win', 'os2') or sys.platform=='riscos': process_pid = os.getpid() signalled_all=thread.allocate_lock() +info = thread.info() +USING_PTHREAD_COND = (info['name'] == 'pthread' + and info['lock_implementation'] == 'mutex+cond') def registerSignals(for_usr1, for_usr2, for_alrm): usr1 = signal.signal(signal.SIGUSR1, for_usr1) @@ -70,6 +73,8 @@ class ThreadSignals(unittest.TestCase): def alarm_interrupt(self, sig, frame): raise KeyboardInterrupt + @unittest.skipIf(USING_PTHREAD_COND, + 'POSIX condition variables cannot be interrupted') def test_lock_acquire_interruption(self): # Mimic receiving a SIGINT (KeyboardInterrupt) with SIGALRM while stuck # in a deadlock. @@ -91,6 +96,8 @@ class ThreadSignals(unittest.TestCase): finally: signal.signal(signal.SIGALRM, oldalrm) + @unittest.skipIf(USING_PTHREAD_COND, + 'POSIX condition variables cannot be interrupted') def test_rlock_acquire_interruption(self): # Mimic receiving a SIGINT (KeyboardInterrupt) with SIGALRM while stuck # in a deadlock. diff --git a/Lib/threading.py b/Lib/threading.py index cb09afa..eb3cb62 100644 --- a/Lib/threading.py +++ b/Lib/threading.py @@ -19,7 +19,7 @@ from collections import deque __all__ = ['active_count', 'Condition', 'current_thread', 'enumerate', 'Event', 'Lock', 'RLock', 'Semaphore', 'BoundedSemaphore', 'Thread', 'Barrier', - 'Timer', 'setprofile', 'settrace', 'local', 'stack_size'] + 'Timer', 'setprofile', 'settrace', 'local', 'stack_size', '_info'] # Rename some stuff so "from threading import *" is safe _start_new_thread = _thread.start_new_thread @@ -31,6 +31,7 @@ try: except AttributeError: _CRLock = None TIMEOUT_MAX = _thread.TIMEOUT_MAX +_info = _thread.info del _thread @@ -67,6 +67,7 @@ Donald Beaudry David Beazley Robin Becker Neal Becker +Torsten Becker Bill Bedford Stefan Behnel Reimer Behrends @@ -88,6 +88,9 @@ Core and Builtins - Issue #11272: On Windows, input() strips '\r' (and not only '\n'), and sys.stdin uses universal newline (replace '\r\n' by '\n'). +- issue #11828: startswith and endswith don't accept None as slice index. + Patch by Torsten Becker. + - Issue #10830: Fix PyUnicode_FromFormatV("%c") for non-BMP characters on narrow build. @@ -110,6 +113,9 @@ Core and Builtins Library ------- +- Issue #11223: Add threading._info() function providing informations about + the thread implementation. + - Issue #11731: simplify/enhance email parser/generator API by introducing policy objects. @@ -121,6 +127,9 @@ Library - Issue #11852: Add missing imports and update tests. +- Issue #11875: collections.OrderedDict's __reduce__ was temporarily + mutating the object instead of just working on a copy. + - Issue #11467: Fix urlparse behavior when handling urls which contains scheme specific part only digits. Patch by Santoso Wijaya. @@ -460,9 +469,9 @@ Build - Issue #11268: Prevent Mac OS X Installer failure if Documentation package had previously been installed. - + - Issue #11495: OSF support is eliminated. It was deprecated in Python 3.2. - + IDLE ---- @@ -484,6 +493,12 @@ Extensions Tests ----- +- Issue #11223: Skip test_lock_acquire_interruption() and + test_rlock_acquire_interruption() of test_threadsignals if a thread lock is + implemented using a POSIX mutex and a POSIX condition variable. A POSIX + condition variable cannot be interrupted by a signal (e.g. on Linux, the + futex system call is restarted). + - Issue #11790: Fix sporadic failures in test_multiprocessing.WithProcessesTestCondition. - Fix possible "file already exists" error when running the tests in parallel. diff --git a/Modules/_collectionsmodule.c b/Modules/_collectionsmodule.c index 5545d1e..8743408 100644 --- a/Modules/_collectionsmodule.c +++ b/Modules/_collectionsmodule.c @@ -1552,12 +1552,8 @@ _count_elements(PyObject *self, PyObject *args) if (PyDict_CheckExact(mapping)) { while (1) { key = PyIter_Next(it); - if (key == NULL) { - if (PyErr_Occurred() && PyErr_ExceptionMatches(PyExc_StopIteration)) - PyErr_Clear(); - else - break; - } + if (key == NULL) + break; oldval = PyDict_GetItem(mapping, key); if (oldval == NULL) { if (PyDict_SetItem(mapping, key, one) == -1) @@ -1575,12 +1571,8 @@ _count_elements(PyObject *self, PyObject *args) } else { while (1) { key = PyIter_Next(it); - if (key == NULL) { - if (PyErr_Occurred() && PyErr_ExceptionMatches(PyExc_StopIteration)) - PyErr_Clear(); - else - break; - } + if (key == NULL) + break; oldval = PyObject_GetItem(mapping, key); if (oldval == NULL) { if (!PyErr_Occurred() || !PyErr_ExceptionMatches(PyExc_KeyError)) diff --git a/Modules/_threadmodule.c b/Modules/_threadmodule.c index ef17b28..914d671 100644 --- a/Modules/_threadmodule.c +++ b/Modules/_threadmodule.c @@ -1221,13 +1221,22 @@ requiring allocation in multiples of the system memory page size\n\ (4kB pages are common; using multiples of 4096 for the stack size is\n\ the suggested approach in the absence of more specific information)."); +static PyObject * +thread_info(PyObject *self) +{ + return _PyThread_Info(); +} + +PyDoc_STRVAR(thread_info_doc, +"info() -> dict\n\ +\n\ +Informations about the thread implementation."); + static PyMethodDef thread_methods[] = { {"start_new_thread", (PyCFunction)thread_PyThread_start_new_thread, - METH_VARARGS, - start_new_doc}, + METH_VARARGS, start_new_doc}, {"start_new", (PyCFunction)thread_PyThread_start_new_thread, - METH_VARARGS, - start_new_doc}, + METH_VARARGS, start_new_doc}, {"allocate_lock", (PyCFunction)thread_PyThread_allocate_lock, METH_NOARGS, allocate_doc}, {"allocate", (PyCFunction)thread_PyThread_allocate_lock, @@ -1243,8 +1252,9 @@ static PyMethodDef thread_methods[] = { {"_count", (PyCFunction)thread__count, METH_NOARGS, _count_doc}, {"stack_size", (PyCFunction)thread_stack_size, - METH_VARARGS, - stack_size_doc}, + METH_VARARGS, stack_size_doc}, + {"info", (PyCFunction)thread_info, + METH_NOARGS, thread_info_doc}, {NULL, NULL} /* sentinel */ }; @@ -1310,7 +1320,7 @@ PyInit__thread(void) d = PyModule_GetDict(m); ThreadError = PyExc_RuntimeError; Py_INCREF(ThreadError); - + PyDict_SetItemString(d, "error", ThreadError); Locktype.tp_doc = lock_doc; Py_INCREF(&Locktype); diff --git a/Modules/faulthandler.c b/Modules/faulthandler.c index abedd5b..48c4391 100644 --- a/Modules/faulthandler.c +++ b/Modules/faulthandler.c @@ -418,16 +418,12 @@ faulthandler_thread(void *unused) const char* errmsg; PyThreadState *current; int ok; -#ifdef HAVE_PTHREAD_H +#if defined(HAVE_PTHREAD_SIGMASK) && !defined(HAVE_BROKEN_PTHREAD_SIGMASK) sigset_t set; /* we don't want to receive any signal */ sigfillset(&set); -#if defined(HAVE_PTHREAD_SIGMASK) && !defined(HAVE_BROKEN_PTHREAD_SIGMASK) pthread_sigmask(SIG_SETMASK, &set, NULL); -#else - sigprocmask(SIG_SETMASK, &set, NULL); -#endif #endif do { diff --git a/Objects/bytearrayobject.c b/Objects/bytearrayobject.c index e98c658..970a6b5 100644 --- a/Objects/bytearrayobject.c +++ b/Objects/bytearrayobject.c @@ -1081,8 +1081,8 @@ bytearray_find_internal(PyByteArrayObject *self, PyObject *args, int dir) Py_ssize_t start=0, end=PY_SSIZE_T_MAX; Py_ssize_t res; - if (!PyArg_ParseTuple(args, "O|O&O&:find/rfind/index/rindex", &subobj, - _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) + if (!stringlib_parse_args_finds("find/rfind/index/rindex", + args, &subobj, &start, &end)) return -2; if (_getbuffer(subobj, &subbuf) < 0) return -2; @@ -1132,8 +1132,7 @@ bytearray_count(PyByteArrayObject *self, PyObject *args) Py_buffer vsub; PyObject *count_obj; - if (!PyArg_ParseTuple(args, "O|O&O&:count", &sub_obj, - _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) + if (!stringlib_parse_args_finds("count", args, &sub_obj, &start, &end)) return NULL; if (_getbuffer(sub_obj, &vsub) < 0) @@ -1315,8 +1314,7 @@ bytearray_startswith(PyByteArrayObject *self, PyObject *args) PyObject *subobj; int result; - if (!PyArg_ParseTuple(args, "O|O&O&:startswith", &subobj, - _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) + if (!stringlib_parse_args_finds("startswith", args, &subobj, &start, &end)) return NULL; if (PyTuple_Check(subobj)) { Py_ssize_t i; @@ -1355,8 +1353,7 @@ bytearray_endswith(PyByteArrayObject *self, PyObject *args) PyObject *subobj; int result; - if (!PyArg_ParseTuple(args, "O|O&O&:endswith", &subobj, - _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) + if (!stringlib_parse_args_finds("endswith", args, &subobj, &start, &end)) return NULL; if (PyTuple_Check(subobj)) { Py_ssize_t i; diff --git a/Objects/bytesobject.c b/Objects/bytesobject.c index 2d7f16d..7561ae5 100644 --- a/Objects/bytesobject.c +++ b/Objects/bytesobject.c @@ -1240,19 +1240,9 @@ bytes_find_internal(PyBytesObject *self, PyObject *args, int dir) const char *sub; Py_ssize_t sub_len; Py_ssize_t start=0, end=PY_SSIZE_T_MAX; - PyObject *obj_start=Py_None, *obj_end=Py_None; - if (!PyArg_ParseTuple(args, "O|OO:find/rfind/index/rindex", &subobj, - &obj_start, &obj_end)) - return -2; - /* To support None in "start" and "end" arguments, meaning - the same as if they were not passed. - */ - if (obj_start != Py_None) - if (!_PyEval_SliceIndex(obj_start, &start)) - return -2; - if (obj_end != Py_None) - if (!_PyEval_SliceIndex(obj_end, &end)) + if (!stringlib_parse_args_finds("find/rfind/index/rindex", + args, &subobj, &start, &end)) return -2; if (PyBytes_Check(subobj)) { @@ -1499,8 +1489,7 @@ bytes_count(PyBytesObject *self, PyObject *args) Py_ssize_t sub_len; Py_ssize_t start = 0, end = PY_SSIZE_T_MAX; - if (!PyArg_ParseTuple(args, "O|O&O&:count", &sub_obj, - _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) + if (!stringlib_parse_args_finds("count", args, &sub_obj, &start, &end)) return NULL; if (PyBytes_Check(sub_obj)) { @@ -2218,8 +2207,7 @@ bytes_startswith(PyBytesObject *self, PyObject *args) PyObject *subobj; int result; - if (!PyArg_ParseTuple(args, "O|O&O&:startswith", &subobj, - _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) + if (!stringlib_parse_args_finds("startswith", args, &subobj, &start, &end)) return NULL; if (PyTuple_Check(subobj)) { Py_ssize_t i; @@ -2259,8 +2247,7 @@ bytes_endswith(PyBytesObject *self, PyObject *args) PyObject *subobj; int result; - if (!PyArg_ParseTuple(args, "O|O&O&:endswith", &subobj, - _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) + if (!stringlib_parse_args_finds("endswith", args, &subobj, &start, &end)) return NULL; if (PyTuple_Check(subobj)) { Py_ssize_t i; diff --git a/Objects/stringlib/find.h b/Objects/stringlib/find.h index f915296c..ce615dc 100644 --- a/Objects/stringlib/find.h +++ b/Objects/stringlib/find.h @@ -93,32 +93,33 @@ stringlib_contains_obj(PyObject* str, PyObject* sub) #endif /* STRINGLIB_WANT_CONTAINS_OBJ */ -#if STRINGLIB_IS_UNICODE - /* This function is a helper for the "find" family (find, rfind, index, -rindex) of unicodeobject.c file, because they all have the same -behaviour for the arguments. +rindex) and for count, startswith and endswith, because they all have +the same behaviour for the arguments. It does not touch the variables received until it knows everything is ok. - -Note that we receive a pointer to the pointer of the substring object, -so when we create that object in this function we don't DECREF it, -because it continues living in the caller functions (those functions, -after finishing using the substring, must DECREF it). */ +#define FORMAT_BUFFER_SIZE 50 + Py_LOCAL_INLINE(int) -_ParseTupleFinds (PyObject *args, PyObject **substring, - Py_ssize_t *start, Py_ssize_t *end) { - PyObject *tmp_substring; +stringlib_parse_args_finds(const char * function_name, PyObject *args, + PyObject **subobj, + Py_ssize_t *start, Py_ssize_t *end) +{ + PyObject *tmp_subobj; Py_ssize_t tmp_start = 0; Py_ssize_t tmp_end = PY_SSIZE_T_MAX; PyObject *obj_start=Py_None, *obj_end=Py_None; + char format[FORMAT_BUFFER_SIZE] = "O|OO:"; + size_t len = strlen(format); - if (!PyArg_ParseTuple(args, "O|OO:find", &tmp_substring, - &obj_start, &obj_end)) + strncpy(format + len, function_name, FORMAT_BUFFER_SIZE - len - 1); + format[FORMAT_BUFFER_SIZE - 1] = '\0'; + + if (!PyArg_ParseTuple(args, format, &tmp_subobj, &obj_start, &obj_end)) return 0; /* To support None in "start" and "end" arguments, meaning @@ -131,16 +132,44 @@ _ParseTupleFinds (PyObject *args, PyObject **substring, if (!_PyEval_SliceIndex(obj_end, &tmp_end)) return 0; - tmp_substring = PyUnicode_FromObject(tmp_substring); - if (!tmp_substring) - return 0; - *start = tmp_start; *end = tmp_end; - *substring = tmp_substring; + *subobj = tmp_subobj; return 1; } +#undef FORMAT_BUFFER_SIZE + +#if STRINGLIB_IS_UNICODE + +/* +Wraps stringlib_parse_args_finds() and additionally ensures that the +first argument is a unicode object. + +Note that we receive a pointer to the pointer of the substring object, +so when we create that object in this function we don't DECREF it, +because it continues living in the caller functions (those functions, +after finishing using the substring, must DECREF it). +*/ + +Py_LOCAL_INLINE(int) +stringlib_parse_args_finds_unicode(const char * function_name, PyObject *args, + PyUnicodeObject **substring, + Py_ssize_t *start, Py_ssize_t *end) +{ + PyObject *tmp_substring; + + if(stringlib_parse_args_finds(function_name, args, &tmp_substring, + start, end)) { + tmp_substring = PyUnicode_FromObject(tmp_substring); + if (!tmp_substring) + return 0; + *substring = (PyUnicodeObject *)tmp_substring; + return 1; + } + return 0; +} + #endif /* STRINGLIB_IS_UNICODE */ #endif /* STRINGLIB_FIND_H */ diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index db9f706..efe6879 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -7515,13 +7515,8 @@ unicode_count(PyUnicodeObject *self, PyObject *args) Py_ssize_t end = PY_SSIZE_T_MAX; PyObject *result; - if (!PyArg_ParseTuple(args, "O|O&O&:count", &substring, - _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) - return NULL; - - substring = (PyUnicodeObject *)PyUnicode_FromObject( - (PyObject *)substring); - if (substring == NULL) + if (!stringlib_parse_args_finds_unicode("count", args, &substring, + &start, &end)) return NULL; ADJUST_INDICES(start, end, self->length); @@ -7658,12 +7653,13 @@ Return -1 on failure."); static PyObject * unicode_find(PyUnicodeObject *self, PyObject *args) { - PyObject *substring; + PyUnicodeObject *substring; Py_ssize_t start; Py_ssize_t end; Py_ssize_t result; - if (!_ParseTupleFinds(args, &substring, &start, &end)) + if (!stringlib_parse_args_finds_unicode("find", args, &substring, + &start, &end)) return NULL; result = stringlib_find_slice( @@ -7720,11 +7716,12 @@ static PyObject * unicode_index(PyUnicodeObject *self, PyObject *args) { Py_ssize_t result; - PyObject *substring; + PyUnicodeObject *substring; Py_ssize_t start; Py_ssize_t end; - if (!_ParseTupleFinds(args, &substring, &start, &end)) + if (!stringlib_parse_args_finds_unicode("index", args, &substring, + &start, &end)) return NULL; result = stringlib_find_slice( @@ -8583,12 +8580,13 @@ Return -1 on failure."); static PyObject * unicode_rfind(PyUnicodeObject *self, PyObject *args) { - PyObject *substring; + PyUnicodeObject *substring; Py_ssize_t start; Py_ssize_t end; Py_ssize_t result; - if (!_ParseTupleFinds(args, &substring, &start, &end)) + if (!stringlib_parse_args_finds_unicode("rfind", args, &substring, + &start, &end)) return NULL; result = stringlib_rfind_slice( @@ -8610,12 +8608,13 @@ Like S.rfind() but raise ValueError when the substring is not found."); static PyObject * unicode_rindex(PyUnicodeObject *self, PyObject *args) { - PyObject *substring; + PyUnicodeObject *substring; Py_ssize_t start; Py_ssize_t end; Py_ssize_t result; - if (!_ParseTupleFinds(args, &substring, &start, &end)) + if (!stringlib_parse_args_finds_unicode("rindex", args, &substring, + &start, &end)) return NULL; result = stringlib_rfind_slice( @@ -9083,8 +9082,7 @@ unicode_startswith(PyUnicodeObject *self, Py_ssize_t end = PY_SSIZE_T_MAX; int result; - if (!PyArg_ParseTuple(args, "O|O&O&:startswith", &subobj, - _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) + if (!stringlib_parse_args_finds("startswith", args, &subobj, &start, &end)) return NULL; if (PyTuple_Check(subobj)) { Py_ssize_t i; @@ -9129,8 +9127,7 @@ unicode_endswith(PyUnicodeObject *self, Py_ssize_t end = PY_SSIZE_T_MAX; int result; - if (!PyArg_ParseTuple(args, "O|O&O&:endswith", &subobj, - _PyEval_SliceIndex, &start, _PyEval_SliceIndex, &end)) + if (!stringlib_parse_args_finds("endswith", args, &subobj, &start, &end)) return NULL; if (PyTuple_Check(subobj)) { Py_ssize_t i; diff --git a/Python/import.c b/Python/import.c index 4159a8e..5360d57 100644 --- a/Python/import.c +++ b/Python/import.c @@ -1198,6 +1198,7 @@ parse_source_module(PyObject *pathname, FILE *fp) /* Helper to open a bytecode file for writing in exclusive mode */ +#ifndef MS_WINDOWS static FILE * open_exclusive(char *filename, mode_t mode) { @@ -1228,6 +1229,7 @@ open_exclusive(char *filename, mode_t mode) return fopen(filename, "wb"); #endif } +#endif /* Write a compiled module to a file, placing the time of last @@ -1250,7 +1252,12 @@ write_compiled_module(PyCodeObject *co, PyObject *cpathname, S_IWUSR | S_IWGRP | S_IWOTH); PyObject *dirbytes; #endif - PyObject *cpathbytes, *dirname; +#ifdef MS_WINDOWS + int fd; +#else + PyObject *cpathbytes; +#endif + PyObject *dirname; Py_UNICODE *dirsep; int res, ok; @@ -1294,6 +1301,16 @@ write_compiled_module(PyCodeObject *co, PyObject *cpathname, } Py_DECREF(dirname); +#ifdef MS_WINDOWS + (void)DeleteFileW(PyUnicode_AS_UNICODE(cpathname)); + fd = _wopen(PyUnicode_AS_UNICODE(cpathname), + O_EXCL | O_CREAT | O_WRONLY | O_TRUNC | O_BINARY, + mode); + if (0 <= fd) + fp = fdopen(fd, "wb"); + else + fp = NULL; +#else cpathbytes = PyUnicode_EncodeFSDefault(cpathname); if (cpathbytes == NULL) { PyErr_Clear(); @@ -1301,11 +1318,14 @@ write_compiled_module(PyCodeObject *co, PyObject *cpathname, } fp = open_exclusive(PyBytes_AS_STRING(cpathbytes), mode); +#endif if (fp == NULL) { if (Py_VerboseFlag) PySys_FormatStderr( "# can't create %R\n", cpathname); +#ifndef MS_WINDOWS Py_DECREF(cpathbytes); +#endif return; } PyMarshal_WriteLongToFile(pyc_magic, fp, Py_MARSHAL_VERSION); @@ -1321,11 +1341,13 @@ write_compiled_module(PyCodeObject *co, PyObject *cpathname, (void)DeleteFileW(PyUnicode_AS_UNICODE(cpathname)); #else (void) unlink(PyBytes_AS_STRING(cpathbytes)); -#endif Py_DECREF(cpathbytes); +#endif return; } +#ifndef MS_WINDOWS Py_DECREF(cpathbytes); +#endif /* Now write the true mtime */ fseek(fp, 4L, 0); assert(mtime < LONG_MAX); diff --git a/Python/thread.c b/Python/thread.c index d224046..1f15a22 100644 --- a/Python/thread.c +++ b/Python/thread.c @@ -100,6 +100,7 @@ static size_t _pythread_stacksize = 0; #endif #ifdef SOLARIS_THREADS +#define PYTHREAD_NAME "solaris" #include "thread_solaris.h" #endif @@ -115,6 +116,7 @@ static size_t _pythread_stacksize = 0; #endif #ifdef _POSIX_THREADS +#define PYTHREAD_NAME "pthread" #include "thread_pthread.h" #endif @@ -124,14 +126,17 @@ static size_t _pythread_stacksize = 0; #endif #ifdef NT_THREADS +#define PYTHREAD_NAME "nt" #include "thread_nt.h" #endif #ifdef OS2_THREADS +#define PYTHREAD_NAME "os2" #include "thread_os2.h" #endif #ifdef PLAN9_THREADS +#define PYTHREAD_NAME "plan9" #include "thread_plan9.h" #endif @@ -409,3 +414,58 @@ PyThread_ReInitTLS(void) } #endif /* Py_HAVE_NATIVE_TLS */ + +PyObject* +_PyThread_Info(void) +{ + PyObject *info, *value; + int ret; +#if (defined(_POSIX_THREADS) && defined(HAVE_CONFSTR) \ + && defined(_CS_GNU_LIBPTHREAD_VERSION)) + char buffer[255]; + int len; +#endif + + info = PyDict_New(); + if (info == NULL) + return NULL; + + value = PyUnicode_FromString(PYTHREAD_NAME); + ret = PyDict_SetItemString(info, "name", value); + Py_DECREF(value); + if (ret) + goto error; + +#ifdef _POSIX_THREADS +#ifdef USE_SEMAPHORES + value = PyUnicode_FromString("semaphore"); +#else + value = PyUnicode_FromString("mutex+cond"); +#endif + if (value == NULL) + return NULL; + ret = PyDict_SetItemString(info, "lock_implementation", value); + Py_DECREF(value); + if (ret) + goto error; + +#if defined(HAVE_CONFSTR) && defined(_CS_GNU_LIBPTHREAD_VERSION) + len = confstr(_CS_GNU_LIBPTHREAD_VERSION, buffer, sizeof(buffer)); + if (0 < len && len < sizeof(buffer)) { + value = PyUnicode_DecodeFSDefaultAndSize(buffer, len-1); + if (value == NULL) + goto error; + ret = PyDict_SetItemString(info, "pthread_version", value); + Py_DECREF(value); + if (ret) + goto error; + } +#endif +#endif + + return info; + +error: + Py_DECREF(info); + return NULL; +} |