diff options
| author | Benjamin Peterson <benjamin@python.org> | 2013-01-04 04:34:40 (GMT) |
|---|---|---|
| committer | Benjamin Peterson <benjamin@python.org> | 2013-01-04 04:34:40 (GMT) |
| commit | d73f369f4c52e921ba225043c0a820856369ef44 (patch) | |
| tree | 4bc702b04312d48bc99c52c7621e15ef99f3c0ac | |
| parent | 0a2c4f55c4d3288dce74513ef135fae167646f02 (diff) | |
| parent | c44057dfbdb4e4f651e7cc4761aa63b7e8d128c3 (diff) | |
| download | cpython-d73f369f4c52e921ba225043c0a820856369ef44.zip cpython-d73f369f4c52e921ba225043c0a820856369ef44.tar.gz cpython-d73f369f4c52e921ba225043c0a820856369ef44.tar.bz2 | |
merge heads
65 files changed, 1101 insertions, 253 deletions
diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst index 2d5d2b1..aac8e81 100644 --- a/Doc/faq/programming.rst +++ b/Doc/faq/programming.rst @@ -313,6 +313,10 @@ calling another function by using ``*`` and ``**``:: g(x, *args, **kwargs) +.. index:: + single: argument; difference from parameter + single: parameter; difference from argument + .. _faq-argument-vs-parameter: What is the difference between arguments and parameters? diff --git a/Doc/glossary.rst b/Doc/glossary.rst index ddecc09..55c3470 100644 --- a/Doc/glossary.rst +++ b/Doc/glossary.rst @@ -240,8 +240,9 @@ Glossary function A series of statements which returns some value to a caller. It can also - be passed zero or more arguments which may be used in the execution of - the body. See also :term:`argument` and :term:`method`. + be passed zero or more :term:`arguments <argument>` which may be used in + the execution of the body. See also :term:`parameter`, :term:`method`, + and the :ref:`function` section. __future__ A pseudo-module which programmers can use to enable new language features @@ -355,17 +356,17 @@ Glossary slowly. See also :term:`interactive`. iterable - An object capable of returning its members one at a - time. Examples of iterables include all sequence types (such as - :class:`list`, :class:`str`, and :class:`tuple`) and some non-sequence - types like :class:`dict` and :class:`file` and objects of any classes you - define with an :meth:`__iter__` or :meth:`__getitem__` method. Iterables - can be used in a :keyword:`for` loop and in many other places where a - sequence is needed (:func:`zip`, :func:`map`, ...). When an iterable - object is passed as an argument to the built-in function :func:`iter`, it - returns an iterator for the object. This iterator is good for one pass - over the set of values. When using iterables, it is usually not necessary - to call :func:`iter` or deal with iterator objects yourself. The ``for`` + An object capable of returning its members one at a time. Examples of + iterables include all sequence types (such as :class:`list`, :class:`str`, + and :class:`tuple`) and some non-sequence types like :class:`dict`, + :term:`file objects <file object>`, and objects of any classes you define + with an :meth:`__iter__` or :meth:`__getitem__` method. Iterables can be + used in a :keyword:`for` loop and in many other places where a sequence is + needed (:func:`zip`, :func:`map`, ...). When an iterable object is passed + as an argument to the built-in function :func:`iter`, it returns an + iterator for the object. This iterator is good for one pass over the set + of values. When using iterables, it is usually not necessary to call + :func:`iter` or deal with iterator objects yourself. The ``for`` statement does that automatically for you, creating a temporary unnamed variable to hold the iterator for the duration of the loop. See also :term:`iterator`, :term:`sequence`, and :term:`generator`. diff --git a/Doc/howto/regex.rst b/Doc/howto/regex.rst index 3beca38..9adfa85 100644 --- a/Doc/howto/regex.rst +++ b/Doc/howto/regex.rst @@ -354,9 +354,9 @@ for a complete listing. +------------------+-----------------------------------------------+ :meth:`match` and :meth:`search` return ``None`` if no match can be found. If -they're successful, a ``MatchObject`` instance is returned, containing -information about the match: where it starts and ends, the substring it matched, -and more. +they're successful, a :ref:`match object <match-objects>` instance is returned, +containing information about the match: where it starts and ends, the substring +it matched, and more. You can learn about this by interactively experimenting with the :mod:`re` module. If you have :mod:`tkinter` available, you may also want to look at @@ -386,16 +386,16 @@ interpreter to print no output. You can explicitly print the result of None Now, let's try it on a string that it should match, such as ``tempo``. In this -case, :meth:`match` will return a :class:`MatchObject`, so you should store the -result in a variable for later use. :: +case, :meth:`match` will return a :ref:`match object <match-objects>`, so you +should store the result in a variable for later use. :: >>> m = p.match('tempo') >>> m #doctest: +ELLIPSIS <_sre.SRE_Match object at 0x...> -Now you can query the :class:`MatchObject` for information about the matching -string. :class:`MatchObject` instances also have several methods and -attributes; the most important ones are: +Now you can query the :ref:`match object <match-objects>` for information +about the matching string. :ref:`match object <match-objects>` instances +also have several methods and attributes; the most important ones are: +------------------+--------------------------------------------+ | Method/Attribute | Purpose | @@ -436,8 +436,9 @@ case. :: >>> m.span() (4, 11) -In actual programs, the most common style is to store the :class:`MatchObject` -in a variable, and then check if it was ``None``. This usually looks like:: +In actual programs, the most common style is to store the +:ref:`match object <match-objects>` in a variable, and then check if it was +``None``. This usually looks like:: p = re.compile( ... ) m = p.match( 'string goes here' ) @@ -454,8 +455,8 @@ Two pattern methods return all of the matches for a pattern. ['12', '11', '10'] :meth:`findall` has to create the entire list before it can be returned as the -result. The :meth:`finditer` method returns a sequence of :class:`MatchObject` -instances as an :term:`iterator`:: +result. The :meth:`finditer` method returns a sequence of +:ref:`match object <match-objects>` instances as an :term:`iterator`:: >>> iterator = p.finditer('12 drummers drumming, 11 ... 10 ...') >>> iterator #doctest: +ELLIPSIS @@ -476,7 +477,7 @@ You don't have to create a pattern object and call its methods; the :func:`search`, :func:`findall`, :func:`sub`, and so forth. These functions take the same arguments as the corresponding pattern method, with the RE string added as the first argument, and still return either ``None`` or a -:class:`MatchObject` instance. :: +:ref:`match object <match-objects>` instance. :: >>> print(re.match(r'From\s+', 'Fromage amk')) None @@ -786,9 +787,9 @@ Groups indicated with ``'('``, ``')'`` also capture the starting and ending index of the text that they match; this can be retrieved by passing an argument to :meth:`group`, :meth:`start`, :meth:`end`, and :meth:`span`. Groups are numbered starting with 0. Group 0 is always present; it's the whole RE, so -:class:`MatchObject` methods all have group 0 as their default argument. Later -we'll see how to express groups that don't capture the span of text that they -match. :: +:ref:`match object <match-objects>` methods all have group 0 as their default +argument. Later we'll see how to express groups that don't capture the span +of text that they match. :: >>> p = re.compile('(a)b') >>> m = p.match('ab') @@ -908,10 +909,10 @@ numbers, groups can be referenced by a name. The syntax for a named group is one of the Python-specific extensions: ``(?P<name>...)``. *name* is, obviously, the name of the group. Named groups also behave exactly like capturing groups, and additionally associate a name -with a group. The :class:`MatchObject` methods that deal with capturing groups -all accept either integers that refer to the group by number or strings that -contain the desired group's name. Named groups are still given numbers, so you -can retrieve information about a group in two ways:: +with a group. The :ref:`match object <match-objects>` methods that deal with +capturing groups all accept either integers that refer to the group by number +or strings that contain the desired group's name. Named groups are still +given numbers, so you can retrieve information about a group in two ways:: >>> p = re.compile(r'(?P<word>\b\w+\b)') >>> m = p.search( '(((( Lots of punctuation )))' ) @@ -1175,11 +1176,11 @@ three variations of the replacement string. :: *replacement* can also be a function, which gives you even more control. If *replacement* is a function, the function is called for every non-overlapping -occurrence of *pattern*. On each call, the function is passed a -:class:`MatchObject` argument for the match and can use this information to -compute the desired replacement string and return it. +occurrence of *pattern*. On each call, the function is passed a +:ref:`match object <match-objects>` argument for the match and can use this +information to compute the desired replacement string and return it. -In the following example, the replacement function translates decimals into +In the following example, the replacement function translates decimals into hexadecimal:: >>> def hexrepl(match): diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst index 5273e9b..3c2b862 100644 --- a/Doc/library/argparse.rst +++ b/Doc/library/argparse.rst @@ -1440,7 +1440,7 @@ Sub-commands different functions which require different kinds of command-line arguments. :class:`ArgumentParser` supports the creation of such sub-commands with the :meth:`add_subparsers` method. The :meth:`add_subparsers` method is normally - called with no arguments and returns an special action object. This object + called with no arguments and returns a special action object. This object has a single method, :meth:`~ArgumentParser.add_parser`, which takes a command name and any :class:`ArgumentParser` constructor arguments, and returns an :class:`ArgumentParser` object that can be modified as usual. diff --git a/Doc/library/audioop.rst b/Doc/library/audioop.rst index 51391ef..c947608 100644 --- a/Doc/library/audioop.rst +++ b/Doc/library/audioop.rst @@ -7,7 +7,7 @@ The :mod:`audioop` module contains some useful operations on sound fragments. It operates on sound fragments consisting of signed integer samples 8, 16 or 32 -bits wide, stored in Python strings. All scalar items are integers, unless +bits wide, stored in bytes objects. All scalar items are integers, unless specified otherwise. .. index:: @@ -126,7 +126,7 @@ The module defines the following variables and functions: .. function:: lin2alaw(fragment, width) Convert samples in the audio fragment to a-LAW encoding and return this as a - Python string. a-LAW is an audio encoding format whereby you get a dynamic + bytes object. a-LAW is an audio encoding format whereby you get a dynamic range of about 13 bits using only 8 bit samples. It is used by the Sun audio hardware, among others. @@ -151,7 +151,7 @@ The module defines the following variables and functions: .. function:: lin2ulaw(fragment, width) Convert samples in the audio fragment to u-LAW encoding and return this as a - Python string. u-LAW is an audio encoding format whereby you get a dynamic + bytes object. u-LAW is an audio encoding format whereby you get a dynamic range of about 14 bits using only 8 bit samples. It is used by the Sun audio hardware, among others. diff --git a/Doc/library/configparser.rst b/Doc/library/configparser.rst index 958375b..0b8212c 100644 --- a/Doc/library/configparser.rst +++ b/Doc/library/configparser.rst @@ -389,7 +389,13 @@ However, there are a few differences that should be taken into account: the default value to be visible again. Trying to delete a default value causes a ``KeyError``. -* Trying to delete the ``DEFAULTSECT`` raises ``ValueError``. +* ``DEFAULTSECT`` cannot be removed from the parser: + + * trying to delete it raises ``ValueError``, + + * ``parser.clear()`` leaves it intact, + + * ``parser.popitem()`` never returns it. * ``parser.get(section, option, **kwargs)`` - the second argument is **not** a fallback value. Note however that the section-level ``get()`` methods are diff --git a/Doc/library/ftplib.rst b/Doc/library/ftplib.rst index a669b1b..3274f19 100644 --- a/Doc/library/ftplib.rst +++ b/Doc/library/ftplib.rst @@ -377,10 +377,10 @@ followed by ``lines`` for the text version or ``binary`` for the binary version. .. method:: FTP.close() Close the connection unilaterally. This should not be applied to an already - closed connection such as after a successful call to :meth:`quit`. After this - call the :class:`FTP` instance should not be used any more (after a call to - :meth:`close` or :meth:`quit` you cannot reopen the connection by issuing - another :meth:`login` method). + closed connection such as after a successful call to :meth:`~FTP.quit`. + After this call the :class:`FTP` instance should not be used any more (after + a call to :meth:`close` or :meth:`~FTP.quit` you cannot reopen the + connection by issuing another :meth:`login` method). FTP_TLS Objects diff --git a/Doc/library/pickle.rst b/Doc/library/pickle.rst index a1f9af2..e250dcf 100644 --- a/Doc/library/pickle.rst +++ b/Doc/library/pickle.rst @@ -367,8 +367,9 @@ The following types can be pickled: * classes that are defined at the top level of a module -* instances of such classes whose :attr:`__dict__` or :meth:`__setstate__` is - picklable (see section :ref:`pickle-inst` for details) +* instances of such classes whose :attr:`__dict__` or the result of calling + :meth:`__getstate__` is picklable (see section :ref:`pickle-inst` for + details). Attempts to pickle unpicklable objects will raise the :exc:`PicklingError` exception; when this happens, an unspecified number of bytes may have already @@ -379,8 +380,8 @@ raised in this case. You can carefully raise this limit with Note that functions (built-in and user-defined) are pickled by "fully qualified" name reference, not by value. This means that only the function name is -pickled, along with the name of the module the function is defined in. Neither the -function's code, nor any of its function attributes are pickled. Thus the +pickled, along with the name of the module the function is defined in. Neither +the function's code, nor any of its function attributes are pickled. Thus the defining module must be importable in the unpickling environment, and the module must contain the named object, otherwise an exception will be raised. [#]_ diff --git a/Doc/library/poplib.rst b/Doc/library/poplib.rst index 01f680e..b4080d6 100644 --- a/Doc/library/poplib.rst +++ b/Doc/library/poplib.rst @@ -105,7 +105,7 @@ An :class:`POP3` instance has the following methods: .. method:: POP3.pass_(password) Send password, response includes message count and mailbox size. Note: the - mailbox on the server is locked until :meth:`quit` is called. + mailbox on the server is locked until :meth:`~poplib.quit` is called. .. method:: POP3.apop(user, secret) diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst index 3101ab7..4a539fc 100644 --- a/Doc/library/smtplib.rst +++ b/Doc/library/smtplib.rst @@ -32,7 +32,8 @@ Protocol) and :rfc:`1869` (SMTP Service Extensions). setting will be used). For normal use, you should only require the initialization/connect, - :meth:`sendmail`, and :meth:`quit` methods. An example is included below. + :meth:`sendmail`, and :meth:`~smtplib.quit` methods. + An example is included below. .. class:: SMTP_SSL(host='', port=0, local_hostname=None, keyfile=None, certfile=None[, timeout]) diff --git a/Doc/library/socketserver.rst b/Doc/library/socketserver.rst index 5287f17..4f22347 100644 --- a/Doc/library/socketserver.rst +++ b/Doc/library/socketserver.rst @@ -299,8 +299,8 @@ request. .. method:: RequestHandler.finish() Called after the :meth:`handle` method to perform any clean-up actions - required. The default implementation does nothing. If :meth:`setup` or - :meth:`handle` raise an exception, this function will not be called. + required. The default implementation does nothing. If :meth:`setup` + raises an exception, this function will not be called. .. method:: RequestHandler.handle() diff --git a/Doc/reference/compound_stmts.rst b/Doc/reference/compound_stmts.rst index 6889176..d0d0646 100644 --- a/Doc/reference/compound_stmts.rst +++ b/Doc/reference/compound_stmts.rst @@ -417,6 +417,9 @@ is equivalent to :: statement. +.. index:: + single: parameter; function definition + .. _function: .. _def: @@ -478,11 +481,14 @@ is equivalent to :: def func(): pass func = f1(arg)(f2(func)) -.. index:: triple: default; parameter; value +.. index:: + triple: default; parameter; value + single: argument; function definition -When one or more parameters have the form *parameter* ``=`` *expression*, the -function is said to have "default parameter values." For a parameter with a -default value, the corresponding argument may be omitted from a call, in which +When one or more :term:`parameters <parameter>` have the form *parameter* ``=`` +*expression*, the function is said to have "default parameter values." For a +parameter with a default value, the corresponding :term:`argument` may be +omitted from a call, in which case the parameter's default value is substituted. If a parameter has a default value, all following parameters up until the "``*``" must also have a default value --- this is a syntactic restriction that is not expressed by the grammar. diff --git a/Doc/reference/expressions.rst b/Doc/reference/expressions.rst index 2c6acb6..68de3c8 100644 --- a/Doc/reference/expressions.rst +++ b/Doc/reference/expressions.rst @@ -600,17 +600,18 @@ upper bound and stride, respectively, substituting ``None`` for missing expressions. +.. index:: + object: callable + single: call + single: argument; call semantics + .. _calls: Calls ----- -.. index:: single: call - -.. index:: object: callable - -A call calls a callable object (e.g., a function) with a possibly empty series -of arguments: +A call calls a callable object (e.g., a :term:`function`) with a possibly empty +series of :term:`arguments <argument>`: .. productionlist:: call: `primary` "(" [`argument_list` [","] | `comprehension`] ")" @@ -628,11 +629,14 @@ of arguments: A trailing comma may be present after the positional and keyword arguments but does not affect the semantics. +.. index:: + single: parameter; call semantics + The primary must evaluate to a callable object (user-defined functions, built-in functions, methods of built-in objects, class objects, methods of class instances, and all objects having a :meth:`__call__` method are callable). All argument expressions are evaluated before the call is attempted. Please refer -to section :ref:`function` for the syntax of formal parameter lists. +to section :ref:`function` for the syntax of formal :term:`parameter` lists. .. XXX update with kwonly args PEP @@ -1266,8 +1270,8 @@ their suffixes:: .. _operator-summary: -Summary -======= +Operator precedence +=================== .. index:: pair: operator; precedence @@ -1291,9 +1295,9 @@ groups from right to left). +-----------------------------------------------+-------------------------------------+ | :keyword:`and` | Boolean AND | +-----------------------------------------------+-------------------------------------+ -| :keyword:`not` *x* | Boolean NOT | +| :keyword:`not` ``x`` | Boolean NOT | +-----------------------------------------------+-------------------------------------+ -| :keyword:`in`, :keyword:`not` :keyword:`in`, | Comparisons, including membership | +| :keyword:`in`, :keyword:`not in`, | Comparisons, including membership | | :keyword:`is`, :keyword:`is not`, ``<``, | tests and identity tests, | | ``<=``, ``>``, ``>=``, ``!=``, ``==`` | | +-----------------------------------------------+-------------------------------------+ @@ -1319,7 +1323,7 @@ groups from right to left). +-----------------------------------------------+-------------------------------------+ | ``(expressions...)``, | Binding or tuple display, | | ``[expressions...]``, | list display, | -| ``{key:datum...}``, | dictionary display, | +| ``{key: value...}``, | dictionary display, | | ``{expressions...}`` | set display | +-----------------------------------------------+-------------------------------------+ diff --git a/Doc/using/windows.rst b/Doc/using/windows.rst index 742a290..c85afd2 100644 --- a/Doc/using/windows.rst +++ b/Doc/using/windows.rst @@ -82,6 +82,8 @@ In order to run Python flawlessly, you might have to change certain environment settings in Windows. +.. _setting-envvars: + Excursus: Setting environment variables --------------------------------------- diff --git a/Lib/aifc.py b/Lib/aifc.py index 775f39c..841f5ae 100644 --- a/Lib/aifc.py +++ b/Lib/aifc.py @@ -692,7 +692,9 @@ class Aifc_write: self._patchheader() def close(self): - if self._file: + if self._file is None: + return + try: self._ensure_header_written(0) if self._datawritten & 1: # quick pad to even size @@ -703,10 +705,12 @@ class Aifc_write: self._datalength != self._datawritten or \ self._marklength: self._patchheader() + finally: # Prevent ref cycles self._convert = None - self._file.close() + f = self._file self._file = None + f.close() # # Internal methods. diff --git a/Lib/configparser.py b/Lib/configparser.py index d148b88..e5536a0 100644 --- a/Lib/configparser.py +++ b/Lib/configparser.py @@ -99,10 +99,9 @@ ConfigParser -- responsible for parsing a list of yes, on for True). Returns False or True. items(section=_UNSET, raw=False, vars=None) - If section is given, return a list of tuples with (section_name, - section_proxy) for each section, including DEFAULTSECT. Otherwise, - return a list of tuples with (name, value) for each option - in the section. + If section is given, return a list of tuples with (name, value) for + each option in the section. Otherwise, return a list of tuples with + (section_name, section_proxy) for each section, including DEFAULTSECT. remove_section(section) Remove the given file section and all its options. @@ -852,6 +851,19 @@ class RawConfigParser(MutableMapping): value_getter = lambda option: d[option] return [(option, value_getter(option)) for option in d.keys()] + def popitem(self): + """Remove a section from the parser and return it as + a (section_name, section_proxy) tuple. If no section is present, raise + KeyError. + + The section DEFAULT is never returned because it cannot be removed. + """ + for key in self.sections(): + value = self[key] + del self[key] + return key, value + raise KeyError + def optionxform(self, optionstr): return optionstr.lower() @@ -947,7 +959,8 @@ class RawConfigParser(MutableMapping): # XXX this is not atomic if read_dict fails at any point. Then again, # no update method in configparser is atomic in this implementation. - self.remove_section(key) + if key in self._sections: + self._sections[key].clear() self.read_dict({key: value}) def __delitem__(self, key): diff --git a/Lib/curses/__init__.py b/Lib/curses/__init__.py index a3e9def..8f09b24 100644 --- a/Lib/curses/__init__.py +++ b/Lib/curses/__init__.py @@ -5,7 +5,7 @@ the package, and perhaps a particular module inside it. import curses from curses import textpad - curses.initwin() + curses.initscr() ... """ diff --git a/Lib/glob.py b/Lib/glob.py index 58888d6..f16e8e1 100644 --- a/Lib/glob.py +++ b/Lib/glob.py @@ -58,8 +58,8 @@ def glob1(dirname, pattern): names = os.listdir(dirname) except os.error: return [] - if pattern[0] != '.': - names = [x for x in names if x[0] != '.'] + if not _ishidden(pattern): + names = [x for x in names if not _ishidden(x)] return fnmatch.filter(names, pattern) def glob0(dirname, basename): @@ -83,3 +83,6 @@ def has_magic(s): else: match = magic_check.search(s) return match is not None + +def _ishidden(path): + return path[0] in ('.', b'.'[0]) diff --git a/Lib/idlelib/EditorWindow.py b/Lib/idlelib/EditorWindow.py index ab62f91..16f63c5 100644 --- a/Lib/idlelib/EditorWindow.py +++ b/Lib/idlelib/EditorWindow.py @@ -170,13 +170,15 @@ class EditorWindow(object): 'recent-files.lst') self.text_frame = text_frame = Frame(top) self.vbar = vbar = Scrollbar(text_frame, name='vbar') - self.width = idleConf.GetOption('main','EditorWindow','width') + self.width = idleConf.GetOption('main', 'EditorWindow', + 'width', type='int') text_options = { 'name': 'text', 'padx': 5, 'wrap': 'none', 'width': self.width, - 'height': idleConf.GetOption('main', 'EditorWindow', 'height')} + 'height': idleConf.GetOption('main', 'EditorWindow', + 'height', type='int')} if TkVersion >= 8.5: # Starting with tk 8.5 we have to set the new tabstyle option # to 'wordprocessor' to achieve the same display of tabs as in @@ -253,7 +255,8 @@ class EditorWindow(object): if idleConf.GetOption('main', 'EditorWindow', 'font-bold', type='bool'): fontWeight='bold' text.config(font=(idleConf.GetOption('main', 'EditorWindow', 'font'), - idleConf.GetOption('main', 'EditorWindow', 'font-size'), + idleConf.GetOption('main', 'EditorWindow', + 'font-size', type='int'), fontWeight)) text_frame.pack(side=LEFT, fill=BOTH, expand=1) text.pack(side=TOP, fill=BOTH, expand=1) @@ -268,7 +271,8 @@ class EditorWindow(object): # Although use-spaces=0 can be configured manually in config-main.def, # configuration of tabs v. spaces is not supported in the configuration # dialog. IDLE promotes the preferred Python indentation: use spaces! - usespaces = idleConf.GetOption('main', 'Indent', 'use-spaces', type='bool') + usespaces = idleConf.GetOption('main', 'Indent', + 'use-spaces', type='bool') self.usetabs = not usespaces # tabwidth is the display width of a literal tab character. @@ -382,9 +386,11 @@ class EditorWindow(object): self.text.tag_remove("sel", "1.0", "end") else: if not self.text.index("sel.first"): - self.text.mark_set("my_anchor", "insert") # there was no previous selection + # there was no previous selection + self.text.mark_set("my_anchor", "insert") else: - if self.text.compare(self.text.index("sel.first"), "<", self.text.index("insert")): + if self.text.compare(self.text.index("sel.first"), "<", + self.text.index("insert")): self.text.mark_set("my_anchor", "sel.first") # extend back else: self.text.mark_set("my_anchor", "sel.last") # extend forward @@ -766,7 +772,8 @@ class EditorWindow(object): if idleConf.GetOption('main','EditorWindow','font-bold',type='bool'): fontWeight='bold' self.text.config(font=(idleConf.GetOption('main','EditorWindow','font'), - idleConf.GetOption('main','EditorWindow','font-size'), + idleConf.GetOption('main','EditorWindow','font-size', + type='int'), fontWeight)) def RemoveKeybindings(self): @@ -1611,7 +1618,7 @@ class IndentSearcher(object): tokens = _tokenize.generate_tokens(self.readline) for token in tokens: self.tokeneater(*token) - except _tokenize.TokenError: + except (_tokenize.TokenError, SyntaxError): # since we cut off the tokenizer early, we can trigger # spurious errors pass diff --git a/Lib/idlelib/FormatParagraph.py b/Lib/idlelib/FormatParagraph.py index 6a5f9b5..e3ca7b9 100644 --- a/Lib/idlelib/FormatParagraph.py +++ b/Lib/idlelib/FormatParagraph.py @@ -32,7 +32,8 @@ class FormatParagraph: self.editwin = None def format_paragraph_event(self, event): - maxformatwidth = int(idleConf.GetOption('main','FormatParagraph','paragraph')) + maxformatwidth = int(idleConf.GetOption('main', 'FormatParagraph', + 'paragraph', type='int')) text = self.editwin.text first, last = self.editwin.get_selection_indices() if first and last: @@ -46,7 +47,8 @@ class FormatParagraph: lines = data.split("\n") lines = map(lambda st, l=len(comment_header): st[l:], lines) data = "\n".join(lines) - # Reformat to maxformatwidth chars or a 20 char width, whichever is greater. + # Reformat to maxformatwidth chars or a 20 char width, + # whichever is greater. format_width = max(maxformatwidth - len(comment_header), 20) newdata = reformat_paragraph(data, format_width) # re-split and re-insert the comment header. diff --git a/Lib/idlelib/HyperParser.py b/Lib/idlelib/HyperParser.py index 38a19f2..4414de7 100644 --- a/Lib/idlelib/HyperParser.py +++ b/Lib/idlelib/HyperParser.py @@ -232,6 +232,11 @@ class HyperParser: pass else: # We can't continue after other types of brackets + if rawtext[pos] in "'\"": + # Scan a string prefix + while pos > 0 and rawtext[pos - 1] in "rRbB": + pos -= 1 + last_identifier_pos = pos break else: diff --git a/Lib/idlelib/configDialog.py b/Lib/idlelib/configDialog.py index 434114e..1f4a3a5 100644 --- a/Lib/idlelib/configDialog.py +++ b/Lib/idlelib/configDialog.py @@ -925,7 +925,7 @@ class ConfigDialog(Toplevel): for font in fonts: self.listFontName.insert(END,font) configuredFont=idleConf.GetOption('main','EditorWindow','font', - default='courier') + default='courier') lc_configuredFont = configuredFont.lower() self.fontName.set(lc_configuredFont) lc_fonts = [s.lower() for s in fonts] @@ -935,13 +935,13 @@ class ConfigDialog(Toplevel): self.listFontName.select_set(currentFontIndex) self.listFontName.select_anchor(currentFontIndex) ##font size dropdown - fontSize=idleConf.GetOption('main','EditorWindow','font-size', - default='10') + fontSize=idleConf.GetOption('main', 'EditorWindow', 'font-size', + type='int', default='10') self.optMenuFontSize.SetMenu(('7','8','9','10','11','12','13','14', - '16','18','20','22'),fontSize ) + '16','18','20','22'), fontSize ) ##fontWeight self.fontBold.set(idleConf.GetOption('main','EditorWindow', - 'font-bold',default=0,type='bool')) + 'font-bold',default=0,type='bool')) ##font sample self.SetFontSample() @@ -1022,10 +1022,13 @@ class ConfigDialog(Toplevel): self.autoSave.set(idleConf.GetOption('main', 'General', 'autosave', default=0, type='bool')) #initial window size - self.winWidth.set(idleConf.GetOption('main','EditorWindow','width')) - self.winHeight.set(idleConf.GetOption('main','EditorWindow','height')) + self.winWidth.set(idleConf.GetOption('main','EditorWindow','width', + type='int')) + self.winHeight.set(idleConf.GetOption('main','EditorWindow','height', + type='int')) #initial paragraph reformat size - self.paraWidth.set(idleConf.GetOption('main','FormatParagraph','paragraph')) + self.paraWidth.set(idleConf.GetOption('main','FormatParagraph','paragraph', + type='int')) # default source encoding self.encoding.set(idleConf.GetOption('main', 'EditorWindow', 'encoding', default='none')) diff --git a/Lib/idlelib/configHandler.py b/Lib/idlelib/configHandler.py index da92726..e22ceb0 100644 --- a/Lib/idlelib/configHandler.py +++ b/Lib/idlelib/configHandler.py @@ -237,24 +237,39 @@ class IdleConf: printed to stderr. """ - if self.userCfg[configType].has_option(section,option): - return self.userCfg[configType].Get(section, option, - type=type, raw=raw) - elif self.defaultCfg[configType].has_option(section,option): - return self.defaultCfg[configType].Get(section, option, - type=type, raw=raw) - else: #returning default, print warning - if warn_on_default: - warning = ('\n Warning: configHandler.py - IdleConf.GetOption -\n' - ' problem retrieving configuration option %r\n' - ' from section %r.\n' - ' returning default value: %r\n' % - (option, section, default)) - try: - sys.stderr.write(warning) - except IOError: - pass - return default + try: + if self.userCfg[configType].has_option(section,option): + return self.userCfg[configType].Get(section, option, + type=type, raw=raw) + except ValueError: + warning = ('\n Warning: configHandler.py - IdleConf.GetOption -\n' + ' invalid %r value for configuration option %r\n' + ' from section %r: %r\n' % + (type, option, section, + self.userCfg[configType].Get(section, option, + raw=raw))) + try: + sys.stderr.write(warning) + except IOError: + pass + try: + if self.defaultCfg[configType].has_option(section,option): + return self.defaultCfg[configType].Get(section, option, + type=type, raw=raw) + except ValueError: + pass + #returning default, print warning + if warn_on_default: + warning = ('\n Warning: configHandler.py - IdleConf.GetOption -\n' + ' problem retrieving configuration option %r\n' + ' from section %r.\n' + ' returning default value: %r\n' % + (option, section, default)) + try: + sys.stderr.write(warning) + except IOError: + pass + return default def SetOption(self, configType, section, option, value): """In user's config file, set section's option to value. diff --git a/Lib/multiprocessing/connection.py b/Lib/multiprocessing/connection.py index 4fa6f70..f73faf6 100644 --- a/Lib/multiprocessing/connection.py +++ b/Lib/multiprocessing/connection.py @@ -41,6 +41,7 @@ import errno import time import tempfile import itertools +import select import _multiprocessing from multiprocessing import current_process, AuthenticationError @@ -213,6 +214,27 @@ if sys.platform != 'win32': return c1, c2 else: + if hasattr(select, 'poll'): + def _poll(fds, timeout): + if timeout is not None: + timeout = int(timeout) * 1000 # timeout is in milliseconds + fd_map = {} + pollster = select.poll() + for fd in fds: + pollster.register(fd, select.POLLIN) + if hasattr(fd, 'fileno'): + fd_map[fd.fileno()] = fd + else: + fd_map[fd] = fd + ls = [] + for fd, event in pollster.poll(timeout): + if event & select.POLLNVAL: + raise ValueError('invalid file descriptor %i' % fd) + ls.append(fd_map[fd]) + return ls + else: + def _poll(fds, timeout): + return select.select(fds, [], [], timeout)[0] from _multiprocessing import win32 diff --git a/Lib/socketserver.py b/Lib/socketserver.py index adf9f38..8f80a7d 100644 --- a/Lib/socketserver.py +++ b/Lib/socketserver.py @@ -700,7 +700,12 @@ class StreamRequestHandler(BaseRequestHandler): def finish(self): if not self.wfile.closed: - self.wfile.flush() + try: + self.wfile.flush() + except socket.error: + # An final socket error may have occurred here, such as + # the local error ECONNABORTED. + pass self.wfile.close() self.rfile.close() diff --git a/Lib/subprocess.py b/Lib/subprocess.py index a322829..f32f081e 100644 --- a/Lib/subprocess.py +++ b/Lib/subprocess.py @@ -1411,7 +1411,7 @@ class Popen(object): def _internal_poll(self, _deadstate=None, _waitpid=os.waitpid, - _WNOHANG=os.WNOHANG, _os_error=os.error): + _WNOHANG=os.WNOHANG, _os_error=os.error, _ECHILD=errno.ECHILD): """Check if child process has terminated. Returns returncode attribute. @@ -1427,7 +1427,7 @@ class Popen(object): except _os_error as e: if _deadstate is not None: self.returncode = _deadstate - elif e.errno == errno.ECHILD: + elif e.errno == _ECHILD: # This happens if SIGCLD is set to be ignored or # waiting for child processes has otherwise been # disabled for our process. This child is dead, we diff --git a/Lib/test/regrtest.py b/Lib/test/regrtest.py index e098522..84beb8d 100755 --- a/Lib/test/regrtest.py +++ b/Lib/test/regrtest.py @@ -33,7 +33,7 @@ Verbosity Selecting tests --r/--random -- randomize test execution order (see below) +-r/--randomize -- randomize test execution order (see below) --randseed -- pass a random seed to reproduce a previous random run -f/--fromfile -- read names of tests to run from a file (see below) -x/--exclude -- arguments are tests to *exclude* @@ -274,11 +274,11 @@ def main(tests=None, testdir=None, verbose=0, quiet=False, try: opts, args = getopt.getopt(sys.argv[1:], 'hvqxsoS:rf:lu:t:TD:NLR:FdwWM:nj:Gm:', ['help', 'verbose', 'verbose2', 'verbose3', 'quiet', - 'exclude', 'single', 'slow', 'random', 'fromfile', 'findleaks', + 'exclude', 'single', 'slow', 'randomize', 'fromfile=', 'findleaks', 'use=', 'threshold=', 'coverdir=', 'nocoverdir', 'runleaks', 'huntrleaks=', 'memlimit=', 'randseed=', 'multiprocess=', 'coverage', 'slaveargs=', 'forever', 'debug', - 'start=', 'nowindows', 'header', 'failfast', 'match']) + 'start=', 'nowindows', 'header', 'failfast', 'match=']) except getopt.error as msg: usage(msg) diff --git a/Lib/test/support.py b/Lib/test/support.py index e1ec9e2..2130fd6 100644 --- a/Lib/test/support.py +++ b/Lib/test/support.py @@ -523,6 +523,49 @@ else: # module name. TESTFN = "{}_{}_tmp".format(TESTFN, os.getpid()) +# FS_NONASCII: non-ASCII character encodable by os.fsencode(), +# or None if there is no such character. +FS_NONASCII = None +for character in ( + # First try printable and common characters to have a readable filename. + # For each character, the encoding list are just example of encodings able + # to encode the character (the list is not exhaustive). + + # U+00E6 (Latin Small Letter Ae): cp1252, iso-8859-1 + '\u00E6', + # U+0130 (Latin Capital Letter I With Dot Above): cp1254, iso8859_3 + '\u0130', + # U+0141 (Latin Capital Letter L With Stroke): cp1250, cp1257 + '\u0141', + # U+03C6 (Greek Small Letter Phi): cp1253 + '\u03C6', + # U+041A (Cyrillic Capital Letter Ka): cp1251 + '\u041A', + # U+05D0 (Hebrew Letter Alef): Encodable to cp424 + '\u05D0', + # U+060C (Arabic Comma): cp864, cp1006, iso8859_6, mac_arabic + '\u060C', + # U+062A (Arabic Letter Teh): cp720 + '\u062A', + # U+0E01 (Thai Character Ko Kai): cp874 + '\u0E01', + + # Then try more "special" characters. "special" because they may be + # interpreted or displayed differently depending on the exact locale + # encoding and the font. + + # U+00A0 (No-Break Space) + '\u00A0', + # U+20AC (Euro Sign) + '\u20AC', +): + try: + os.fsdecode(os.fsencode(character)) + except UnicodeError: + pass + else: + FS_NONASCII = character + break # TESTFN_UNICODE is a non-ascii filename TESTFN_UNICODE = TESTFN + "-\xe0\xf2\u0258\u0141\u011f" @@ -567,6 +610,41 @@ elif sys.platform != 'darwin': # the byte 0xff. Skip some unicode filename tests. pass +# TESTFN_UNDECODABLE is a filename (bytes type) that should *not* be able to be +# decoded from the filesystem encoding (in strict mode). It can be None if we +# cannot generate such filename (ex: the latin1 encoding can decode any byte +# sequence). On UNIX, TESTFN_UNDECODABLE can be decoded by os.fsdecode() thanks +# to the surrogateescape error handler (PEP 383), but not from the filesystem +# encoding in strict mode. +TESTFN_UNDECODABLE = None +for name in ( + # b'\xff' is not decodable by os.fsdecode() with code page 932. Windows + # accepts it to create a file or a directory, or don't accept to enter to + # such directory (when the bytes name is used). So test b'\xe7' first: it is + # not decodable from cp932. + b'\xe7w\xf0', + # undecodable from ASCII, UTF-8 + b'\xff', + # undecodable from iso8859-3, iso8859-6, iso8859-7, cp424, iso8859-8, cp856 + # and cp857 + b'\xae\xd5' + # undecodable from UTF-8 (UNIX and Mac OS X) + b'\xed\xb2\x80', b'\xed\xb4\x80', + # undecodable from shift_jis, cp869, cp874, cp932, cp1250, cp1251, cp1252, + # cp1253, cp1254, cp1255, cp1257, cp1258 + b'\x81\x98', +): + try: + name.decode(TESTFN_ENCODING) + except UnicodeDecodeError: + TESTFN_UNDECODABLE = os.fsencode(TESTFN) + name + break + +if FS_NONASCII: + TESTFN_NONASCII = TESTFN + '-' + FS_NONASCII +else: + TESTFN_NONASCII = None + # Save the initial cwd SAVEDCWD = os.getcwd() diff --git a/Lib/test/test_aifc.py b/Lib/test/test_aifc.py index ee4ad6b..0b19af6 100644 --- a/Lib/test/test_aifc.py +++ b/Lib/test/test_aifc.py @@ -112,6 +112,13 @@ class AIFCTest(unittest.TestCase): self.assertEqual(testfile.closed, False) f.close() self.assertEqual(testfile.closed, True) + testfile = open(TESTFN, 'wb') + fout = aifc.open(testfile, 'wb') + self.assertFalse(testfile.closed) + with self.assertRaises(aifc.Error): + fout.close() + self.assertTrue(testfile.closed) + fout.close() # do nothing def test_write_header_comptype_sampwidth(self): for comptype in (b'ULAW', b'ulaw', b'ALAW', b'alaw', b'G722'): @@ -291,11 +298,13 @@ class AIFCLowLevelTest(unittest.TestCase): def test_write_header_raises(self): fout = aifc.open(io.BytesIO(), 'wb') self.assertRaises(aifc.Error, fout.close) + fout = aifc.open(io.BytesIO(), 'wb') fout.setnchannels(1) self.assertRaises(aifc.Error, fout.close) + fout = aifc.open(io.BytesIO(), 'wb') + fout.setnchannels(1) fout.setsampwidth(1) self.assertRaises(aifc.Error, fout.close) - fout.initfp(None) def test_write_header_comptype_raises(self): for comptype in (b'ULAW', b'ulaw', b'ALAW', b'alaw', b'G722'): diff --git a/Lib/test/test_bz2.py b/Lib/test/test_bz2.py index 5b48e2b..977d10b 100644 --- a/Lib/test/test_bz2.py +++ b/Lib/test/test_bz2.py @@ -24,6 +24,7 @@ class BaseTest(unittest.TestCase): TEXT = b'root:x:0:0:root:/root:/bin/bash\nbin:x:1:1:bin:/bin:\ndaemon:x:2:2:daemon:/sbin:\nadm:x:3:4:adm:/var/adm:\nlp:x:4:7:lp:/var/spool/lpd:\nsync:x:5:0:sync:/sbin:/bin/sync\nshutdown:x:6:0:shutdown:/sbin:/sbin/shutdown\nhalt:x:7:0:halt:/sbin:/sbin/halt\nmail:x:8:12:mail:/var/spool/mail:\nnews:x:9:13:news:/var/spool/news:\nuucp:x:10:14:uucp:/var/spool/uucp:\noperator:x:11:0:operator:/root:\ngames:x:12:100:games:/usr/games:\ngopher:x:13:30:gopher:/usr/lib/gopher-data:\nftp:x:14:50:FTP User:/var/ftp:/bin/bash\nnobody:x:65534:65534:Nobody:/home:\npostfix:x:100:101:postfix:/var/spool/postfix:\nniemeyer:x:500:500::/home/niemeyer:/bin/bash\npostgres:x:101:102:PostgreSQL Server:/var/lib/pgsql:/bin/bash\nmysql:x:102:103:MySQL server:/var/lib/mysql:/bin/bash\nwww:x:103:104::/var/www:/bin/false\n' DATA = b'BZh91AY&SY.\xc8N\x18\x00\x01>_\x80\x00\x10@\x02\xff\xf0\x01\x07n\x00?\xe7\xff\xe00\x01\x99\xaa\x00\xc0\x03F\x86\x8c#&\x83F\x9a\x03\x06\xa6\xd0\xa6\x93M\x0fQ\xa7\xa8\x06\x804hh\x12$\x11\xa4i4\xf14S\xd2<Q\xb5\x0fH\xd3\xd4\xdd\xd5\x87\xbb\xf8\x94\r\x8f\xafI\x12\xe1\xc9\xf8/E\x00pu\x89\x12]\xc9\xbbDL\nQ\x0e\t1\x12\xdf\xa0\xc0\x97\xac2O9\x89\x13\x94\x0e\x1c7\x0ed\x95I\x0c\xaaJ\xa4\x18L\x10\x05#\x9c\xaf\xba\xbc/\x97\x8a#C\xc8\xe1\x8cW\xf9\xe2\xd0\xd6M\xa7\x8bXa<e\x84t\xcbL\xb3\xa7\xd9\xcd\xd1\xcb\x84.\xaf\xb3\xab\xab\xad`n}\xa0lh\tE,\x8eZ\x15\x17VH>\x88\xe5\xcd9gd6\x0b\n\xe9\x9b\xd5\x8a\x99\xf7\x08.K\x8ev\xfb\xf7xw\xbb\xdf\xa1\x92\xf1\xdd|/";\xa2\xba\x9f\xd5\xb1#A\xb6\xf6\xb3o\xc9\xc5y\\\xebO\xe7\x85\x9a\xbc\xb6f8\x952\xd5\xd7"%\x89>V,\xf7\xa6z\xe2\x9f\xa3\xdf\x11\x11"\xd6E)I\xa9\x13^\xca\xf3r\xd0\x03U\x922\xf26\xec\xb6\xed\x8b\xc3U\x13\x9d\xc5\x170\xa4\xfa^\x92\xacDF\x8a\x97\xd6\x19\xfe\xdd\xb8\xbd\x1a\x9a\x19\xa3\x80ankR\x8b\xe5\xd83]\xa9\xc6\x08\x82f\xf6\xb9"6l$\xb8j@\xc0\x8a\xb0l1..\xbak\x83ls\x15\xbc\xf4\xc1\x13\xbe\xf8E\xb8\x9d\r\xa8\x9dk\x84\xd3n\xfa\xacQ\x07\xb1%y\xaav\xb4\x08\xe0z\x1b\x16\xf5\x04\xe9\xcc\xb9\x08z\x1en7.G\xfc]\xc9\x14\xe1B@\xbb!8`' DATA_CRLF = b'BZh91AY&SY\xaez\xbbN\x00\x01H\xdf\x80\x00\x12@\x02\xff\xf0\x01\x07n\x00?\xe7\xff\xe0@\x01\xbc\xc6`\x86*\x8d=M\xa9\x9a\x86\xd0L@\x0fI\xa6!\xa1\x13\xc8\x88jdi\x8d@\x03@\x1a\x1a\x0c\x0c\x83 \x00\xc4h2\x19\x01\x82D\x84e\t\xe8\x99\x89\x19\x1ah\x00\r\x1a\x11\xaf\x9b\x0fG\xf5(\x1b\x1f?\t\x12\xcf\xb5\xfc\x95E\x00ps\x89\x12^\xa4\xdd\xa2&\x05(\x87\x04\x98\x89u\xe40%\xb6\x19\'\x8c\xc4\x89\xca\x07\x0e\x1b!\x91UIFU%C\x994!DI\xd2\xfa\xf0\xf1N8W\xde\x13A\xf5\x9cr%?\x9f3;I45A\xd1\x8bT\xb1<l\xba\xcb_\xc00xY\x17r\x17\x88\x08\x08@\xa0\ry@\x10\x04$)`\xf2\xce\x89z\xb0s\xec\x9b.iW\x9d\x81\xb5-+t\x9f\x1a\'\x97dB\xf5x\xb5\xbe.[.\xd7\x0e\x81\xe7\x08\x1cN`\x88\x10\xca\x87\xc3!"\x80\x92R\xa1/\xd1\xc0\xe6mf\xac\xbd\x99\xcca\xb3\x8780>\xa4\xc7\x8d\x1a\\"\xad\xa1\xabyBg\x15\xb9l\x88\x88\x91k"\x94\xa4\xd4\x89\xae*\xa6\x0b\x10\x0c\xd6\xd4m\xe86\xec\xb5j\x8a\x86j\';\xca.\x01I\xf2\xaaJ\xe8\x88\x8cU+t3\xfb\x0c\n\xa33\x13r2\r\x16\xe0\xb3(\xbf\x1d\x83r\xe7M\xf0D\x1365\xd8\x88\xd3\xa4\x92\xcb2\x06\x04\\\xc1\xb0\xea//\xbek&\xd8\xe6+t\xe5\xa1\x13\xada\x16\xder5"w]\xa2i\xb7[\x97R \xe2IT\xcd;Z\x04dk4\xad\x8a\t\xd3\x81z\x10\xf1:^`\xab\x1f\xc5\xdc\x91N\x14$+\x9e\xae\xd3\x80' + EMPTY_DATA = b'BZh9\x17rE8P\x90\x00\x00\x00\x00' with open(findfile("testbz2_bigmem.bz2"), "rb") as f: DATA_BIGMEM = f.read() @@ -303,6 +304,12 @@ class BZ2CompressorTest(BaseTest): data += bz2c.flush() self.assertEqual(self.decompress(data), self.TEXT) + def testCompressEmptyString(self): + bz2c = BZ2Compressor() + data = bz2c.compress(b'') + data += bz2c.flush() + self.assertEqual(data, self.EMPTY_DATA) + def testCompressChunks10(self): # "Test BZ2Compressor.compress()/flush() with chunks of 10 bytes" bz2c = BZ2Compressor() @@ -383,6 +390,10 @@ class FuncTest(BaseTest): data = bz2.compress(self.TEXT) self.assertEqual(self.decompress(data), self.TEXT) + def testCompressEmptyString(self): + text = bz2.compress(b'') + self.assertEqual(text, self.EMPTY_DATA) + def testDecompress(self): # "Test decompress() function" text = bz2.decompress(self.DATA) @@ -393,6 +404,10 @@ class FuncTest(BaseTest): text = bz2.decompress(b"") self.assertEqual(text, b"") + def testDecompressToEmptyString(self): + text = bz2.decompress(self.EMPTY_DATA) + self.assertEqual(text, b'') + def testDecompressIncomplete(self): # "Test decompress() function with incomplete data" self.assertRaises(ValueError, bz2.decompress, self.DATA[:-10]) diff --git a/Lib/test/test_cfgparser.py b/Lib/test/test_cfgparser.py index a6e9050..cec9b44 100644 --- a/Lib/test/test_cfgparser.py +++ b/Lib/test/test_cfgparser.py @@ -770,6 +770,59 @@ boolean {0[0]} NO with self.assertRaises(configparser.NoSectionError): cf.items("no such section") + def test_popitem(self): + cf = self.fromstring(""" + [section1] + name1 {0[0]} value1 + [section2] + name2 {0[0]} value2 + [section3] + name3 {0[0]} value3 + """.format(self.delimiters), defaults={"default": "<default>"}) + self.assertEqual(cf.popitem()[0], 'section1') + self.assertEqual(cf.popitem()[0], 'section2') + self.assertEqual(cf.popitem()[0], 'section3') + with self.assertRaises(KeyError): + cf.popitem() + + def test_clear(self): + cf = self.newconfig({"foo": "Bar"}) + self.assertEqual( + cf.get(self.default_section, "Foo"), "Bar", + "could not locate option, expecting case-insensitive option names") + cf['zing'] = {'option1': 'value1', 'option2': 'value2'} + self.assertEqual(cf.sections(), ['zing']) + self.assertEqual(set(cf['zing'].keys()), {'option1', 'option2', 'foo'}) + cf.clear() + self.assertEqual(set(cf.sections()), set()) + self.assertEqual(set(cf[self.default_section].keys()), {'foo'}) + + def test_setitem(self): + cf = self.fromstring(""" + [section1] + name1 {0[0]} value1 + [section2] + name2 {0[0]} value2 + [section3] + name3 {0[0]} value3 + """.format(self.delimiters), defaults={"nameD": "valueD"}) + self.assertEqual(set(cf['section1'].keys()), {'name1', 'named'}) + self.assertEqual(set(cf['section2'].keys()), {'name2', 'named'}) + self.assertEqual(set(cf['section3'].keys()), {'name3', 'named'}) + self.assertEqual(cf['section1']['name1'], 'value1') + self.assertEqual(cf['section2']['name2'], 'value2') + self.assertEqual(cf['section3']['name3'], 'value3') + self.assertEqual(cf.sections(), ['section1', 'section2', 'section3']) + cf['section2'] = {'name22': 'value22'} + self.assertEqual(set(cf['section2'].keys()), {'name22', 'named'}) + self.assertEqual(cf['section2']['name22'], 'value22') + self.assertNotIn('name2', cf['section2']) + self.assertEqual(cf.sections(), ['section1', 'section2', 'section3']) + cf['section3'] = {} + self.assertEqual(set(cf['section3'].keys()), {'named'}) + self.assertNotIn('name3', cf['section3']) + self.assertEqual(cf.sections(), ['section1', 'section2', 'section3']) + class StrictTestCase(BasicTestCase): config_class = configparser.RawConfigParser diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py index c10bd76..67375cd 100644 --- a/Lib/test/test_cmd_line.py +++ b/Lib/test/test_cmd_line.py @@ -100,11 +100,11 @@ class CmdLineTest(unittest.TestCase): # All good if execution is successful assert_python_ok('-c', 'pass') - @unittest.skipIf(sys.getfilesystemencoding() == 'ascii', - 'need a filesystem encoding different than ASCII') + @unittest.skipUnless(test.support.FS_NONASCII, 'need support.FS_NONASCII') def test_non_ascii(self): # Test handling of non-ascii data - command = "assert(ord('\xe9') == 0xe9)" + command = ("assert(ord(%r) == %s)" + % (test.support.FS_NONASCII, ord(test.support.FS_NONASCII))) assert_python_ok('-c', command) # On Windows, pass bytes to subprocess doesn't test how Python decodes the diff --git a/Lib/test/test_cmd_line_script.py b/Lib/test/test_cmd_line_script.py index 6b59d96..70f7d1e 100644 --- a/Lib/test/test_cmd_line_script.py +++ b/Lib/test/test_cmd_line_script.py @@ -294,6 +294,30 @@ class CmdLineTest(unittest.TestCase): print(out) self.assertEqual(rc, 1) + def test_non_ascii(self): + # Mac OS X denies the creation of a file with an invalid UTF-8 name. + # Windows allows to create a name with an arbitrary bytes name, but + # Python cannot a undecodable bytes argument to a subprocess. + if (support.TESTFN_UNDECODABLE + and sys.platform not in ('win32', 'darwin')): + name = os.fsdecode(support.TESTFN_UNDECODABLE) + elif support.TESTFN_NONASCII: + name = support.TESTFN_NONASCII + else: + self.skipTest("need support.TESTFN_NONASCII") + + # Issue #16218 + source = 'print(ascii(__file__))\n' + script_name = _make_test_script(os.curdir, name, source) + self.addCleanup(support.unlink, script_name) + rc, stdout, stderr = assert_python_ok(script_name) + self.assertEqual( + ascii(script_name), + stdout.rstrip().decode('ascii'), + 'stdout=%r stderr=%r' % (stdout, stderr)) + self.assertEqual(0, rc) + + def test_main(): support.run_unittest(CmdLineTest) support.reap_children() diff --git a/Lib/test/test_genericpath.py b/Lib/test/test_genericpath.py index 50638a1..6020923 100644 --- a/Lib/test/test_genericpath.py +++ b/Lib/test/test_genericpath.py @@ -292,11 +292,20 @@ class CommonTest(GenericTest): for path in ('', 'fuu', 'f\xf9\xf9', '/fuu', 'U:\\'): self.assertIsInstance(abspath(path), str) - @unittest.skipIf(sys.platform == 'darwin', - "Mac OS X denies the creation of a directory with an invalid utf8 name") def test_nonascii_abspath(self): - # Test non-ASCII, non-UTF8 bytes in the path. - with support.temp_cwd(b'\xe7w\xf0'): + if (support.TESTFN_UNDECODABLE + # Mac OS X denies the creation of a directory with an invalid + # UTF-8 name. Windows allows to create a directory with an + # arbitrary bytes name, but fails to enter this directory + # (when the bytes name is used). + and sys.platform not in ('win32', 'darwin')): + name = support.TESTFN_UNDECODABLE + elif support.TESTFN_NONASCII: + name = support.TESTFN_NONASCII + else: + self.skipTest("need support.TESTFN_NONASCII") + + with support.temp_cwd(name): self.test_abspath() diff --git a/Lib/test/test_glob.py b/Lib/test/test_glob.py index 2dba8de..1cdc24a 100644 --- a/Lib/test/test_glob.py +++ b/Lib/test/test_glob.py @@ -1,9 +1,10 @@ -import unittest -from test.support import run_unittest, TESTFN, skip_unless_symlink, can_symlink import glob import os import shutil import sys +import unittest + +from test.support import run_unittest, TESTFN, skip_unless_symlink, can_symlink class GlobTests(unittest.TestCase): @@ -31,7 +32,8 @@ class GlobTests(unittest.TestCase): self.mktemp('a', 'bcd', 'efg', 'ha') if can_symlink(): os.symlink(self.norm('broken'), self.norm('sym1')) - os.symlink(self.norm('broken'), self.norm('sym2')) + os.symlink('broken', self.norm('sym2')) + os.symlink(os.path.join('a', 'bcd'), self.norm('sym3')) def tearDown(self): shutil.rmtree(self.tempdir) @@ -44,10 +46,16 @@ class GlobTests(unittest.TestCase): p = os.path.join(self.tempdir, pattern) res = glob.glob(p) self.assertEqual(list(glob.iglob(p)), res) + bres = [os.fsencode(x) for x in res] + self.assertEqual(glob.glob(os.fsencode(p)), bres) + self.assertEqual(list(glob.iglob(os.fsencode(p))), bres) return res def assertSequencesEqual_noorder(self, l1, l2): + l1 = list(l1) + l2 = list(l2) self.assertEqual(set(l1), set(l2)) + self.assertEqual(sorted(l1), sorted(l2)) def test_glob_literal(self): eq = self.assertSequencesEqual_noorder @@ -56,15 +64,15 @@ class GlobTests(unittest.TestCase): eq(self.glob('aab'), [self.norm('aab')]) eq(self.glob('zymurgy'), []) - # test return types are unicode, but only if os.listdir - # returns unicode filenames - uniset = set([str]) - tmp = os.listdir('.') - if set(type(x) for x in tmp) == uniset: - u1 = glob.glob('*') - u2 = glob.glob('./*') - self.assertEqual(set(type(r) for r in u1), uniset) - self.assertEqual(set(type(r) for r in u2), uniset) + res = glob.glob('*') + self.assertEqual({type(r) for r in res}, {str}) + res = glob.glob(os.path.join(os.curdir, '*')) + self.assertEqual({type(r) for r in res}, {str}) + + res = glob.glob(b'*') + self.assertEqual({type(r) for r in res}, {bytes}) + res = glob.glob(os.path.join(os.fsencode(os.curdir), b'*')) + self.assertEqual({type(r) for r in res}, {bytes}) def test_glob_one_directory(self): eq = self.assertSequencesEqual_noorder @@ -93,20 +101,20 @@ class GlobTests(unittest.TestCase): eq(self.glob('*', '*a'), []) eq(self.glob('a', '*', '*', '*a'), [self.norm('a', 'bcd', 'efg', 'ha')]) - eq(self.glob('?a?', '*F'), map(self.norm, [os.path.join('aaa', 'zzzF'), - os.path.join('aab', 'F')])) + eq(self.glob('?a?', '*F'), [self.norm('aaa', 'zzzF'), + self.norm('aab', 'F')]) def test_glob_directory_with_trailing_slash(self): # Patterns ending with a slash shouldn't match non-dirs - res = glob.glob(os.path.join(self.tempdir, 'Z*Z') + os.sep) + res = glob.glob(self.norm('Z*Z') + os.sep) self.assertEqual(res, []) - res = glob.glob(os.path.join(self.tempdir, 'ZZZ') + os.sep) + res = glob.glob(self.norm('ZZZ') + os.sep) self.assertEqual(res, []) - # When there is wildcard pattern which ends with os.sep, glob() + # When there is a wildcard pattern which ends with os.sep, glob() # doesn't blow up. - res = glob.glob(os.path.join(self.tempdir, 'aa*') + os.sep) + res = glob.glob(self.norm('aa*') + os.sep) self.assertEqual(len(res), 2) - # either of these results are reasonable + # either of these results is reasonable self.assertIn(set(res), [ {self.norm('aaa'), self.norm('aab')}, {self.norm('aaa') + os.sep, self.norm('aab') + os.sep}, @@ -115,22 +123,37 @@ class GlobTests(unittest.TestCase): def test_glob_bytes_directory_with_trailing_slash(self): # Same as test_glob_directory_with_trailing_slash, but with a # bytes argument. - res = glob.glob(os.fsencode(os.path.join(self.tempdir, 'Z*Z') + os.sep)) + res = glob.glob(os.fsencode(self.norm('Z*Z') + os.sep)) self.assertEqual(res, []) - res = glob.glob(os.fsencode(os.path.join(self.tempdir, 'ZZZ') + os.sep)) + res = glob.glob(os.fsencode(self.norm('ZZZ') + os.sep)) self.assertEqual(res, []) - res = glob.glob(os.fsencode(os.path.join(self.tempdir, 'aa*') + os.sep)) + res = glob.glob(os.fsencode(self.norm('aa*') + os.sep)) self.assertEqual(len(res), 2) - # either of these results are reasonable - self.assertIn({os.fsdecode(x) for x in res}, [ - {self.norm('aaa'), self.norm('aab')}, - {self.norm('aaa') + os.sep, self.norm('aab') + os.sep}, + # either of these results is reasonable + self.assertIn(set(res), [ + {os.fsencode(self.norm('aaa')), + os.fsencode(self.norm('aab'))}, + {os.fsencode(self.norm('aaa') + os.sep), + os.fsencode(self.norm('aab') + os.sep)}, ]) @skip_unless_symlink + def test_glob_symlinks(self): + eq = self.assertSequencesEqual_noorder + eq(self.glob('sym3'), [self.norm('sym3')]) + eq(self.glob('sym3', '*'), [self.norm('sym3', 'EF'), + self.norm('sym3', 'efg')]) + self.assertIn(self.glob('sym3' + os.sep), + [[self.norm('sym3')], [self.norm('sym3') + os.sep]]) + eq(self.glob('*', '*F'), + [self.norm('aaa', 'zzzF'), + self.norm('aab', 'F'), self.norm('sym3', 'EF')]) + + @skip_unless_symlink def test_glob_broken_symlinks(self): eq = self.assertSequencesEqual_noorder - eq(self.glob('sym*'), [self.norm('sym1'), self.norm('sym2')]) + eq(self.glob('sym*'), [self.norm('sym1'), self.norm('sym2'), + self.norm('sym3')]) eq(self.glob('sym1'), [self.norm('sym1')]) eq(self.glob('sym2'), [self.norm('sym2')]) diff --git a/Lib/test/test_int.py b/Lib/test/test_int.py index 437e323..671b20a 100644 --- a/Lib/test/test_int.py +++ b/Lib/test/test_int.py @@ -1,6 +1,7 @@ import sys import unittest +from test import support from test.support import run_unittest L = [ @@ -100,10 +101,6 @@ class IntTestCases(unittest.TestCase): self.assertRaises(ValueError, int, "0b", 2) self.assertRaises(ValueError, int, "0b", 0) - # Bug #3236: Return small longs from PyLong_FromString - self.assertTrue(int("10") is 10) - self.assertTrue(int("-1") is -1) - # SF bug 1334662: int(string, base) wrong answers # Various representations of 2**32 evaluated to 0 # rather than 2**32 in previous versions @@ -221,6 +218,22 @@ class IntTestCases(unittest.TestCase): self.assertEqual(int('2br45qc', 35), 4294967297) self.assertEqual(int('1z141z5', 36), 4294967297) + @support.cpython_only + def test_small_ints(self): + # Bug #3236: Return small longs from PyLong_FromString + self.assertIs(int('10'), 10) + self.assertIs(int('-1'), -1) + self.assertIs(int(b'10'), 10) + self.assertIs(int(b'-1'), -1) + + def test_keyword_args(self): + # Test invoking int() using keyword arguments. + self.assertEqual(int(x=1.2), 1) + self.assertEqual(int('100', base=2), 4) + self.assertEqual(int(x='100', base=2), 4) + self.assertRaises(TypeError, int, base=10) + self.assertRaises(TypeError, int, base=0) + def test_intconversion(self): # Test __int__() class ClassicMissingMethods: diff --git a/Lib/test/test_multiprocessing.py b/Lib/test/test_multiprocessing.py index ab6d36a..fa4865b 100644 --- a/Lib/test/test_multiprocessing.py +++ b/Lib/test/test_multiprocessing.py @@ -1574,6 +1574,7 @@ class _TestConnection(BaseTestCase): self.assertTimingAlmostEqual(poll.elapsed, TIMEOUT1) conn.send(None) + time.sleep(.1) self.assertEqual(poll(TIMEOUT1), True) self.assertTimingAlmostEqual(poll.elapsed, 0) diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py index 612c62a..720e78b 100644 --- a/Lib/test/test_os.py +++ b/Lib/test/test_os.py @@ -634,6 +634,50 @@ class MakedirTests(unittest.TestCase): os.removedirs(path) + +class RemoveDirsTests(unittest.TestCase): + def setUp(self): + os.makedirs(support.TESTFN) + + def tearDown(self): + support.rmtree(support.TESTFN) + + def test_remove_all(self): + dira = os.path.join(support.TESTFN, 'dira') + os.mkdir(dira) + dirb = os.path.join(dira, 'dirb') + os.mkdir(dirb) + os.removedirs(dirb) + self.assertFalse(os.path.exists(dirb)) + self.assertFalse(os.path.exists(dira)) + self.assertFalse(os.path.exists(support.TESTFN)) + + def test_remove_partial(self): + dira = os.path.join(support.TESTFN, 'dira') + os.mkdir(dira) + dirb = os.path.join(dira, 'dirb') + os.mkdir(dirb) + with open(os.path.join(dira, 'file.txt'), 'w') as f: + f.write('text') + os.removedirs(dirb) + self.assertFalse(os.path.exists(dirb)) + self.assertTrue(os.path.exists(dira)) + self.assertTrue(os.path.exists(support.TESTFN)) + + def test_remove_nothing(self): + dira = os.path.join(support.TESTFN, 'dira') + os.mkdir(dira) + dirb = os.path.join(dira, 'dirb') + os.mkdir(dirb) + with open(os.path.join(dirb, 'file.txt'), 'w') as f: + f.write('text') + with self.assertRaises(OSError): + os.removedirs(dirb) + self.assertTrue(os.path.exists(dirb)) + self.assertTrue(os.path.exists(dira)) + self.assertTrue(os.path.exists(support.TESTFN)) + + class DevNullTests(unittest.TestCase): def test_devnull(self): with open(os.devnull, 'wb') as f: @@ -642,6 +686,7 @@ class DevNullTests(unittest.TestCase): with open(os.devnull, 'rb') as f: self.assertEqual(f.read(), b'') + class URandomTests(unittest.TestCase): def test_urandom_length(self): self.assertEqual(len(os.urandom(0)), 0) @@ -968,6 +1013,8 @@ if sys.platform != 'win32': def setUp(self): if support.TESTFN_UNENCODABLE: self.dir = support.TESTFN_UNENCODABLE + elif support.TESTFN_NONASCII: + self.dir = support.TESTFN_NONASCII else: self.dir = support.TESTFN self.bdir = os.fsencode(self.dir) @@ -982,6 +1029,8 @@ if sys.platform != 'win32': add_filename(support.TESTFN_UNICODE) if support.TESTFN_UNENCODABLE: add_filename(support.TESTFN_UNENCODABLE) + if support.TESTFN_NONASCII: + add_filename(support.TESTFN_NONASCII) if not bytesfn: self.skipTest("couldn't create any non-ascii filename") @@ -1012,6 +1061,15 @@ if sys.platform != 'win32': f = open(os.path.join(self.dir, fn), 'rb') f.close() + @unittest.skipUnless(hasattr(os, 'statvfs'), + "need os.statvfs()") + def test_statvfs(self): + # issue #9645 + for fn in self.unicodefn: + # should not fail with file not found error + fullname = os.path.join(self.dir, fn) + os.statvfs(fullname) + def test_stat(self): for fn in self.unicodefn: os.stat(os.path.join(self.dir, fn)) @@ -1310,6 +1368,7 @@ def test_main(): PidTests, LoginTests, LinkTests, + RemoveDirsTests, ) if __name__ == "__main__": diff --git a/Lib/test/test_ssl.py b/Lib/test/test_ssl.py index 2f0b3e6..4f254a9 100644 --- a/Lib/test/test_ssl.py +++ b/Lib/test/test_ssl.py @@ -558,6 +558,17 @@ class NetworkedTests(unittest.TestCase): finally: s.close() + def test_connect_ex_error(self): + with support.transient_internet("svn.python.org"): + s = ssl.wrap_socket(socket.socket(socket.AF_INET), + cert_reqs=ssl.CERT_REQUIRED, + ca_certs=SVN_PYTHON_ORG_ROOT_CERT) + try: + self.assertEqual(errno.ECONNREFUSED, + s.connect_ex(("svn.python.org", 444))) + finally: + s.close() + def test_connect_with_context(self): with support.transient_internet("svn.python.org"): # Same as test_connect, but with a separately created context diff --git a/Lib/test/test_subprocess.py b/Lib/test/test_subprocess.py index f7a7b11..75eb852 100644 --- a/Lib/test/test_subprocess.py +++ b/Lib/test/test_subprocess.py @@ -2023,8 +2023,7 @@ class ContextManagerTests(BaseTestCase): stderr=subprocess.PIPE) as proc: pass - if c.exception.errno != errno.ENOENT: # ignore "no such file" - raise c.exception + self.assertEqual(c.exception.errno, errno.ENOENT) def test_main(): diff --git a/Lib/test/test_tarfile.py b/Lib/test/test_tarfile.py index f32eb8c..a53181e 100644 --- a/Lib/test/test_tarfile.py +++ b/Lib/test/test_tarfile.py @@ -328,31 +328,21 @@ class MiscReadTest(CommonReadTest): @support.skip_unless_symlink def test_extract_hardlink(self): # Test hardlink extraction (e.g. bug #857297). - tar = tarfile.open(tarname, errorlevel=1, encoding="iso8859-1") - - try: + with tarfile.open(tarname, errorlevel=1, encoding="iso8859-1") as tar: tar.extract("ustar/regtype", TEMPDIR) - try: - tar.extract("ustar/lnktype", TEMPDIR) - except EnvironmentError as e: - if e.errno == errno.ENOENT: - self.fail("hardlink not extracted properly") + self.addCleanup(os.remove, os.path.join(TEMPDIR, "ustar/regtype")) + tar.extract("ustar/lnktype", TEMPDIR) + self.addCleanup(os.remove, os.path.join(TEMPDIR, "ustar/lnktype")) with open(os.path.join(TEMPDIR, "ustar/lnktype"), "rb") as f: data = f.read() self.assertEqual(md5sum(data), md5_regtype) - try: - tar.extract("ustar/symtype", TEMPDIR) - except EnvironmentError as e: - if e.errno == errno.ENOENT: - self.fail("symlink not extracted properly") - + tar.extract("ustar/symtype", TEMPDIR) + self.addCleanup(os.remove, os.path.join(TEMPDIR, "ustar/symtype")) with open(os.path.join(TEMPDIR, "ustar/symtype"), "rb") as f: data = f.read() self.assertEqual(md5sum(data), md5_regtype) - finally: - tar.close() def test_extractall(self): # Test if extractall() correctly restores directory permissions diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py index d8c302b..5eab30a 100644 --- a/Lib/test/test_urllib2.py +++ b/Lib/test/test_urllib2.py @@ -1438,16 +1438,32 @@ class RequestTests(unittest.TestCase): req = Request(url) self.assertEqual(req.get_full_url(), url) -def test_HTTPError_interface(): - """ - Issue 13211 reveals that HTTPError didn't implement the URLError - interface even though HTTPError is a subclass of URLError. - - >>> err = urllib.error.HTTPError(msg='something bad happened', url=None, code=None, hdrs=None, fp=None) - >>> assert hasattr(err, 'reason') - >>> err.reason - 'something bad happened' - """ + def test_HTTPError_interface(self): + """ + Issue 13211 reveals that HTTPError didn't implement the URLError + interface even though HTTPError is a subclass of URLError. + + >>> err = urllib.error.HTTPError(msg='something bad happened', url=None, code=None, hdrs=None, fp=None) + >>> assert hasattr(err, 'reason') + >>> err.reason + 'something bad happened' + """ + + def test_HTTPError_interface_call(self): + """ + Issue 15701 - HTTPError interface has info method available from URLError + """ + err = urllib.request.HTTPError(msg="something bad happened", url=None, + code=None, hdrs='Content-Length:42', fp=None) + self.assertTrue(hasattr(err, 'reason')) + assert hasattr(err, 'reason') + assert hasattr(err, 'info') + assert callable(err.info) + try: + err.info() + except AttributeError: + self.fail('err.info call failed.') + self.assertEqual(err.info(), "Content-Length:42") def test_main(verbose=None): from test import test_urllib2 diff --git a/Lib/test/test_urllib2_localnet.py b/Lib/test/test_urllib2_localnet.py index 9e1ce5b..f07471d 100644 --- a/Lib/test/test_urllib2_localnet.py +++ b/Lib/test/test_urllib2_localnet.py @@ -352,6 +352,8 @@ class TestUrlopen(unittest.TestCase): def setUp(self): super(TestUrlopen, self).setUp() + # Ignore proxies for localhost tests. + os.environ['NO_PROXY'] = '*' self.server = None def tearDown(self): diff --git a/Lib/test/test_urlparse.py b/Lib/test/test_urlparse.py index e9adaef..378a427 100755 --- a/Lib/test/test_urlparse.py +++ b/Lib/test/test_urlparse.py @@ -818,6 +818,35 @@ class UrlParseTestCase(unittest.TestCase): p2 = urllib.parse.urlsplit('tel:+31641044153') self.assertEqual(p2.scheme, 'tel') self.assertEqual(p2.path, '+31641044153') + # assert the behavior for urlparse + p1 = urllib.parse.urlparse('tel:+31-641044153') + self.assertEqual(p1.scheme, 'tel') + self.assertEqual(p1.path, '+31-641044153') + p2 = urllib.parse.urlparse('tel:+31641044153') + self.assertEqual(p2.scheme, 'tel') + self.assertEqual(p2.path, '+31641044153') + + def test_telurl_params(self): + p1 = urllib.parse.urlparse('tel:123-4;phone-context=+1-650-516') + self.assertEqual(p1.scheme, 'tel') + self.assertEqual(p1.path, '123-4') + self.assertEqual(p1.params, 'phone-context=+1-650-516') + + p1 = urllib.parse.urlparse('tel:+1-201-555-0123') + self.assertEqual(p1.scheme, 'tel') + self.assertEqual(p1.path, '+1-201-555-0123') + self.assertEqual(p1.params, '') + + p1 = urllib.parse.urlparse('tel:7042;phone-context=example.com') + self.assertEqual(p1.scheme, 'tel') + self.assertEqual(p1.path, '7042') + self.assertEqual(p1.params, 'phone-context=example.com') + + p1 = urllib.parse.urlparse('tel:863-1234;phone-context=+1-914-555') + self.assertEqual(p1.scheme, 'tel') + self.assertEqual(p1.path, '863-1234') + self.assertEqual(p1.params, 'phone-context=+1-914-555') + def test_main(): support.run_unittest(UrlParseTestCase) diff --git a/Lib/test/test_winreg.py b/Lib/test/test_winreg.py index c7b50dd..a164d2f 100644 --- a/Lib/test/test_winreg.py +++ b/Lib/test/test_winreg.py @@ -323,6 +323,35 @@ class LocalWinregTests(BaseWinregTests): finally: DeleteKey(HKEY_CURRENT_USER, test_key_name) + def test_setvalueex_value_range(self): + # Test for Issue #14420, accept proper ranges for SetValueEx. + # Py2Reg, which gets called by SetValueEx, was using PyLong_AsLong, + # thus raising OverflowError. The implementation now uses + # PyLong_AsUnsignedLong to match DWORD's size. + try: + with CreateKey(HKEY_CURRENT_USER, test_key_name) as ck: + self.assertNotEqual(ck.handle, 0) + SetValueEx(ck, "test_name", None, REG_DWORD, 0x80000000) + finally: + DeleteKey(HKEY_CURRENT_USER, test_key_name) + + def test_queryvalueex_return_value(self): + # Test for Issue #16759, return unsigned int from QueryValueEx. + # Reg2Py, which gets called by QueryValueEx, was returning a value + # generated by PyLong_FromLong. The implmentation now uses + # PyLong_FromUnsignedLong to match DWORD's size. + try: + with CreateKey(HKEY_CURRENT_USER, test_key_name) as ck: + self.assertNotEqual(ck.handle, 0) + test_val = 0x80000000 + SetValueEx(ck, "test_name", None, REG_DWORD, test_val) + ret_val, ret_type = QueryValueEx(ck, "test_name") + self.assertEqual(ret_type, REG_DWORD) + self.assertEqual(ret_val, test_val) + finally: + DeleteKey(HKEY_CURRENT_USER, test_key_name) + + @unittest.skipUnless(REMOTE_NAME, "Skipping remote registry tests") class RemoteWinregTests(BaseWinregTests): diff --git a/Lib/tkinter/__init__.py b/Lib/tkinter/__init__.py index f2cb60f..3ba4bb0 100644 --- a/Lib/tkinter/__init__.py +++ b/Lib/tkinter/__init__.py @@ -380,7 +380,7 @@ class Misc: background, highlightColor, selectForeground, disabledForeground, insertBackground, troughColor.""" self.tk.call(('tk_setPalette',) - + _flatten(args) + _flatten(kw.items())) + + _flatten(args) + _flatten(list(kw.items()))) def tk_menuBar(self, *args): """Do not use. Needed in Tk 3.6 and earlier.""" pass # obsolete since Tk 4.0 diff --git a/Lib/tkinter/test/test_tkinter/test_misc.py b/Lib/tkinter/test/test_tkinter/test_misc.py new file mode 100644 index 0000000..d325b31 --- /dev/null +++ b/Lib/tkinter/test/test_tkinter/test_misc.py @@ -0,0 +1,45 @@ +import unittest +import tkinter +from tkinter import ttk +from test import support + +support.requires('gui') + +class MiscTest(unittest.TestCase): + + def setUp(self): + self.root = ttk.setup_master() + + def test_tk_setPalette(self): + root = self.root + root.tk_setPalette('black') + self.assertEqual(root['background'], 'black') + root.tk_setPalette('white') + self.assertEqual(root['background'], 'white') + self.assertRaisesRegex(tkinter.TclError, + '^unknown color name "spam"$', + root.tk_setPalette, 'spam') + + root.tk_setPalette(background='black') + self.assertEqual(root['background'], 'black') + root.tk_setPalette(background='blue', highlightColor='yellow') + self.assertEqual(root['background'], 'blue') + self.assertEqual(root['highlightcolor'], 'yellow') + root.tk_setPalette(background='yellow', highlightColor='blue') + self.assertEqual(root['background'], 'yellow') + self.assertEqual(root['highlightcolor'], 'blue') + self.assertRaisesRegex(tkinter.TclError, + '^unknown color name "spam"$', + root.tk_setPalette, background='spam') + self.assertRaisesRegex(tkinter.TclError, + '^must specify a background color$', + root.tk_setPalette, spam='white') + self.assertRaisesRegex(tkinter.TclError, + '^must specify a background color$', + root.tk_setPalette, highlightColor='blue') + + +tests_gui = (MiscTest, ) + +if __name__ == "__main__": + support.run_unittest(*tests_gui) diff --git a/Lib/urllib/error.py b/Lib/urllib/error.py index 40add41..eb14c1d 100644 --- a/Lib/urllib/error.py +++ b/Lib/urllib/error.py @@ -58,6 +58,10 @@ class HTTPError(URLError, urllib.response.addinfourl): def reason(self): return self.msg + def info(self): + return self.hdrs + + # exception raised when downloaded size does not match content-length class ContentTooShortError(URLError): def __init__(self, message, content): diff --git a/Lib/urllib/parse.py b/Lib/urllib/parse.py index 4811759..dc75f8f 100644 --- a/Lib/urllib/parse.py +++ b/Lib/urllib/parse.py @@ -46,7 +46,7 @@ uses_netloc = ['ftp', 'http', 'gopher', 'nntp', 'telnet', 'svn', 'svn+ssh', 'sftp', 'nfs', 'git', 'git+ssh'] uses_params = ['ftp', 'hdl', 'prospero', 'http', 'imap', 'https', 'shttp', 'rtsp', 'rtspu', 'sip', 'sips', - 'mms', '', 'sftp'] + 'mms', '', 'sftp', 'tel'] # These are not actually used anymore, but should stay for backwards # compatibility. (They are undocumented, but have a public-looking name.) diff --git a/Mac/IDLE/IDLE.app/Contents/Info.plist b/Mac/IDLE/IDLE.app/Contents/Info.plist index 35af012..8bffb9a 100644 --- a/Mac/IDLE/IDLE.app/Contents/Info.plist +++ b/Mac/IDLE/IDLE.app/Contents/Info.plist @@ -36,7 +36,7 @@ <key>CFBundleExecutable</key> <string>IDLE</string> <key>CFBundleGetInfoString</key> - <string>%version%, © 2001-2012 Python Software Foundation</string> + <string>%version%, © 2001-2013 Python Software Foundation</string> <key>CFBundleIconFile</key> <string>IDLE.icns</string> <key>CFBundleIdentifier</key> diff --git a/Mac/PythonLauncher/Info.plist.in b/Mac/PythonLauncher/Info.plist.in index 913a2e0..c216766 100644 --- a/Mac/PythonLauncher/Info.plist.in +++ b/Mac/PythonLauncher/Info.plist.in @@ -40,7 +40,7 @@ <key>CFBundleExecutable</key> <string>PythonLauncher</string> <key>CFBundleGetInfoString</key> - <string>%VERSION%, © 2001-2012 Python Software Foundation</string> + <string>%VERSION%, © 2001-2013 Python Software Foundation</string> <key>CFBundleIconFile</key> <string>PythonLauncher.icns</string> <key>CFBundleIdentifier</key> diff --git a/Mac/Resources/app/Info.plist.in b/Mac/Resources/app/Info.plist.in index 1bbca4f..a91b1ee 100644 --- a/Mac/Resources/app/Info.plist.in +++ b/Mac/Resources/app/Info.plist.in @@ -20,7 +20,7 @@ <key>CFBundleExecutable</key> <string>Python</string> <key>CFBundleGetInfoString</key> - <string>%version%, (c) 2004-2012 Python Software Foundation.</string> + <string>%version%, (c) 2004-2013 Python Software Foundation.</string> <key>CFBundleHelpBookFolder</key> <array> <string>Documentation</string> @@ -37,7 +37,7 @@ <key>CFBundleInfoDictionaryVersion</key> <string>6.0</string> <key>CFBundleLongVersionString</key> - <string>%version%, (c) 2004-2012 Python Software Foundation.</string> + <string>%version%, (c) 2004-2013 Python Software Foundation.</string> <key>CFBundleName</key> <string>Python</string> <key>CFBundlePackageType</key> @@ -55,6 +55,6 @@ <key>NSAppleScriptEnabled</key> <true/> <key>NSHumanReadableCopyright</key> - <string>(c) 2012 Python Software Foundation.</string> + <string>(c) 2013 Python Software Foundation.</string> </dict> </plist> diff --git a/Mac/Resources/framework/Info.plist.in b/Mac/Resources/framework/Info.plist.in index bab46fa..6c3bdc1 100644 --- a/Mac/Resources/framework/Info.plist.in +++ b/Mac/Resources/framework/Info.plist.in @@ -17,9 +17,9 @@ <key>CFBundlePackageType</key> <string>FMWK</string> <key>CFBundleShortVersionString</key> - <string>%VERSION%, (c) 2004-2012 Python Software Foundation.</string> + <string>%VERSION%, (c) 2004-2013 Python Software Foundation.</string> <key>CFBundleLongVersionString</key> - <string>%VERSION%, (c) 2004-2012 Python Software Foundation.</string> + <string>%VERSION%, (c) 2004-2013 Python Software Foundation.</string> <key>CFBundleSignature</key> <string>????</string> <key>CFBundleVersion</key> @@ -543,6 +543,7 @@ Peter van Kampen Rafe Kaplan Jacob Kaplan-Moss Jan Kaliszewski +Anton Kasyanov Lou Kates Hiroaki Kawai Sebastien Keim @@ -800,6 +801,7 @@ Michael Otteneder R. M. Oudkerk Russel Owen Joonas Paalasmaa +Martin Packman Shriphani Palakodety Ondrej Palkovsky Mike Pall @@ -946,6 +948,7 @@ Michael Scharf Andreas Schawo Neil Schemenauer David Scherer +Wolfgang Scherer Hynek Schlawack Bob Schmertz Gregor Schmid @@ -1077,6 +1080,7 @@ Erik Tollerud Matias Torchinsky Sandro Tosi Richard Townsend +Nathan Trapuzzano Laurence Tratt John Tromp Jason Trowbridge @@ -10,6 +10,23 @@ What's New in Python 3.2.4 Core and Builtins ----------------- +- Issue #16367: Fix FileIO.readall() on Windows for files larger than 2 GB. + +- Issue #16455: On FreeBSD and Solaris, if the locale is C, the + ASCII/surrogateescape codec is now used, instead of the locale encoding, to + decode the command line arguments. This change fixes inconsistencies with + os.fsencode() and os.fsdecode() because these operating systems announces an + ASCII locale encoding, whereas the ISO-8859-1 encoding is used in practice. + +- Issue #16761: Calling int() with base argument only now raises TypeError. + +- Issue #16759: Support the full DWORD (unsigned long) range in Reg2Py + when retreiving a REG_DWORD value. This corrects functions like + winreg.QueryValueEx that may have been returning truncated values. + +- Issue #14420: Support the full DWORD (unsigned long) range in Py2Reg + when passed a REG_DWORD value. Fixes OverflowError in winreg.SetValueEx. + - Issue #16602: When a weakref's target was part of a long deallocation chain, the object could remain reachable through its weakref even though its refcount had dropped to zero. @@ -179,6 +196,51 @@ Core and Builtins Library ------- +- Issue #16828: Fix error incorrectly raised by bz2.compress(''). Patch by + Martin Packman. + +- Issue #16541: tk_setPalette() now works with keyword arguments. + +- Issue #16820: In configparser, `parser.popitem()` no longer raises ValueError. + This makes `parser.clean()` work correctly. + +- Issue #16820: In configparser, ``parser['section'] = {}`` now preserves + section order within the parser. This makes `parser.update()` preserve section + order as well. + +- Issue #9644: Fix the encoding used by os.statvfs(): use the filesystem + encoding with the surrogateescape error handler, instead of UTF-8 in strict + mode. + +- Issue #16819: IDLE method completion now correctly works for bytes literals. + +- Issue #9586: Redefine SEM_FAILED on MacOSX to keep compiler happy. + +- Issue 10527: make multiprocessing use poll() instead of select() if available. + +- Issue #16485: Now file descriptors are closed if file header patching failed + on closing an aifc file. + +- Issue #16504: IDLE now catches SyntaxErrors raised by tokenizer. Patch by + Roger Serwy. + +- Issue #16618: Make glob.glob match consistently across strings and bytes + regarding leading dots. Patch by Serhiy Storchaka. + +- Issue #16702: test_urllib2_localnet tests now correctly ignores proxies for + localhost tests. + +- Issue #16511: Use default IDLE width and height if config param is not valid. + Patch Serhiy Storchaka. + +- Issue #16713: Parsing of 'tel' urls using urlparse separates params from + path. + +- Issue #16443: Add docstrings to regular expression match objects. + Patch by Anton Kasyanov. + +- Issue #15701: Fix HTTPError info method call to return the headers information. + - Issue #16646: ftplib.FTP.makeport() might lose socket error details. (patch by Serhiy Storchaka) @@ -696,6 +758,9 @@ Extension Modules Tests ----- +- Issue #15324: Fix regrtest parsing of --fromfile, --match, and --randomize + options. + - Issue #16664: Add regression tests for glob's behaviour concerning entries starting with a ".". Patch by Sebastian Kreft. diff --git a/Modules/_ctypes/callproc.c b/Modules/_ctypes/callproc.c index 3465995..ef14e91 100644 --- a/Modules/_ctypes/callproc.c +++ b/Modules/_ctypes/callproc.c @@ -825,11 +825,11 @@ static int _call_function_pointer(int flags, space[0] = errno; errno = temp; } - Py_XDECREF(error_object); #ifdef WITH_THREAD if ((flags & FUNCFLAG_PYTHONAPI) == 0) Py_BLOCK_THREADS #endif + Py_XDECREF(error_object); #ifdef MS_WIN32 #ifndef DONT_USE_SEH if (dwExceptionCode) { diff --git a/Modules/_io/fileio.c b/Modules/_io/fileio.c index 605ad51..8ea7c58 100644 --- a/Modules/_io/fileio.c +++ b/Modules/_io/fileio.c @@ -558,7 +558,7 @@ fileio_readall(fileio *self) { PyObject *result; Py_ssize_t total = 0; - int n; + Py_ssize_t n; if (self->fd < 0) return err_closed(); @@ -591,9 +591,18 @@ fileio_readall(fileio *self) } Py_BEGIN_ALLOW_THREADS errno = 0; + n = newsize - total; +#if defined(MS_WIN64) || defined(MS_WINDOWS) + if (n > INT_MAX) + n = INT_MAX; + n = read(self->fd, + PyBytes_AS_STRING(result) + total, + (int)n); +#else n = read(self->fd, PyBytes_AS_STRING(result) + total, - newsize - total); + n); +#endif Py_END_ALLOW_THREADS if (n == 0) break; diff --git a/Modules/_multiprocessing/semaphore.c b/Modules/_multiprocessing/semaphore.c index 1f37d6a..1ee224d 100644 --- a/Modules/_multiprocessing/semaphore.c +++ b/Modules/_multiprocessing/semaphore.c @@ -197,6 +197,13 @@ semlock_release(SemLockObject *self, PyObject *args) #define SEM_GETVALUE(sem, pval) sem_getvalue(sem, pval) #define SEM_UNLINK(name) sem_unlink(name) +/* OS X 10.4 defines SEM_FAILED as -1 instead of (sem_t *)-1; this gives + compiler warnings, and (potentially) undefined behaviour. */ +#ifdef __APPLE__ +# undef SEM_FAILED +# define SEM_FAILED ((sem_t *)-1) +#endif + #ifndef HAVE_SEM_UNLINK # define sem_unlink(name) 0 #endif diff --git a/Modules/_sre.c b/Modules/_sre.c index d74bf44..88bbf6a 100644 --- a/Modules/_sre.c +++ b/Modules/_sre.c @@ -2559,35 +2559,35 @@ pattern_deepcopy(PatternObject* self, PyObject* memo) } PyDoc_STRVAR(pattern_match_doc, -"match(string[, pos[, endpos]]) --> match object or None.\n\ +"match(string[, pos[, endpos]]) -> match object or None.\n\n\ Matches zero or more characters at the beginning of the string"); PyDoc_STRVAR(pattern_search_doc, -"search(string[, pos[, endpos]]) --> match object or None.\n\ +"search(string[, pos[, endpos]]) -> match object or None.\n\n\ Scan through string looking for a match, and return a corresponding\n\ - MatchObject instance. Return None if no position in the string matches."); + match object instance. Return None if no position in the string matches."); PyDoc_STRVAR(pattern_split_doc, -"split(string[, maxsplit = 0]) --> list.\n\ +"split(string[, maxsplit = 0]) -> list.\n\n\ Split string by the occurrences of pattern."); PyDoc_STRVAR(pattern_findall_doc, -"findall(string[, pos[, endpos]]) --> list.\n\ +"findall(string[, pos[, endpos]]) -> list.\n\n\ Return a list of all non-overlapping matches of pattern in string."); PyDoc_STRVAR(pattern_finditer_doc, -"finditer(string[, pos[, endpos]]) --> iterator.\n\ +"finditer(string[, pos[, endpos]]) -> iterator.\n\n\ Return an iterator over all non-overlapping matches for the \n\ RE pattern in string. For each match, the iterator returns a\n\ match object."); PyDoc_STRVAR(pattern_sub_doc, -"sub(repl, string[, count = 0]) --> newstring\n\ +"sub(repl, string[, count = 0]) -> newstring.\n\n\ Return the string obtained by replacing the leftmost non-overlapping\n\ occurrences of pattern in string by the replacement repl."); PyDoc_STRVAR(pattern_subn_doc, -"subn(repl, string[, count = 0]) --> (newstring, number of subs)\n\ +"subn(repl, string[, count = 0]) -> (newstring, number of subs)\n\n\ Return the tuple (new_string, number_of_subs_made) found by replacing\n\ the leftmost non-overlapping occurrences of pattern with the\n\ replacement repl."); @@ -3579,14 +3579,54 @@ match_deepcopy(MatchObject* self, PyObject* memo) #endif } +PyDoc_STRVAR(match_doc, +"The result of re.match() and re.search().\n\ +Match objects always have a boolean value of True."); + +PyDoc_STRVAR(match_group_doc, +"group([group1, ...]) -> str or tuple.\n\n\ + Return subgroup(s) of the match by indices or names.\n\ + For 0 returns the entire match."); + +PyDoc_STRVAR(match_start_doc, +"start([group=0]) -> int.\n\n\ + Return index of the start of the substring matched by group."); + +PyDoc_STRVAR(match_end_doc, +"end([group=0]) -> int.\n\n\ + Return index of the end of the substring matched by group."); + +PyDoc_STRVAR(match_span_doc, +"span([group]) -> tuple.\n\n\ + For MatchObject m, return the 2-tuple (m.start(group), m.end(group))."); + +PyDoc_STRVAR(match_groups_doc, +"groups([default=None]) -> tuple.\n\n\ + Return a tuple containing all the subgroups of the match, from 1.\n\ + The default argument is used for groups\n\ + that did not participate in the match"); + +PyDoc_STRVAR(match_groupdict_doc, +"groupdict([default=None]) -> dict.\n\n\ + Return a dictionary containing all the named subgroups of the match,\n\ + keyed by the subgroup name. The default argument is used for groups\n\ + that did not participate in the match"); + +PyDoc_STRVAR(match_expand_doc, +"expand(template) -> str.\n\n\ + Return the string obtained by doing backslash substitution\n\ + on the string template, as done by the sub() method."); + static PyMethodDef match_methods[] = { - {"group", (PyCFunction) match_group, METH_VARARGS}, - {"start", (PyCFunction) match_start, METH_VARARGS}, - {"end", (PyCFunction) match_end, METH_VARARGS}, - {"span", (PyCFunction) match_span, METH_VARARGS}, - {"groups", (PyCFunction) match_groups, METH_VARARGS|METH_KEYWORDS}, - {"groupdict", (PyCFunction) match_groupdict, METH_VARARGS|METH_KEYWORDS}, - {"expand", (PyCFunction) match_expand, METH_O}, + {"group", (PyCFunction) match_group, METH_VARARGS, match_group_doc}, + {"start", (PyCFunction) match_start, METH_VARARGS, match_start_doc}, + {"end", (PyCFunction) match_end, METH_VARARGS, match_end_doc}, + {"span", (PyCFunction) match_span, METH_VARARGS, match_span_doc}, + {"groups", (PyCFunction) match_groups, METH_VARARGS|METH_KEYWORDS, + match_groups_doc}, + {"groupdict", (PyCFunction) match_groupdict, METH_VARARGS|METH_KEYWORDS, + match_groupdict_doc}, + {"expand", (PyCFunction) match_expand, METH_O, match_expand_doc}, {"__copy__", (PyCFunction) match_copy, METH_NOARGS}, {"__deepcopy__", (PyCFunction) match_deepcopy, METH_O}, {NULL, NULL} @@ -3665,7 +3705,7 @@ static PyTypeObject Match_Type = { 0, /* tp_setattro */ 0, /* tp_as_buffer */ Py_TPFLAGS_DEFAULT, /* tp_flags */ - 0, /* tp_doc */ + match_doc, /* tp_doc */ 0, /* tp_traverse */ 0, /* tp_clear */ 0, /* tp_richcompare */ diff --git a/Modules/bz2module.c b/Modules/bz2module.c index 4795965..2e99df9 100644 --- a/Modules/bz2module.c +++ b/Modules/bz2module.c @@ -1979,7 +1979,7 @@ bz2_compress(PyObject *self, PyObject *args, PyObject *kwargs) return NULL; } - action = BZ_RUN; + action = input_left > 0 ? BZ_RUN : BZ_FINISH; for (;;) { char *saved_next_out; diff --git a/Modules/posixmodule.c b/Modules/posixmodule.c index 0d2919b..e7e5305 100644 --- a/Modules/posixmodule.c +++ b/Modules/posixmodule.c @@ -6463,18 +6463,22 @@ Perform a statvfs system call on the given path."); static PyObject * posix_statvfs(PyObject *self, PyObject *args) { + PyObject *opath, *result = NULL; char *path; int res; struct statvfs st; - if (!PyArg_ParseTuple(args, "s:statvfs", &path)) + if (!PyArg_ParseTuple(args, "O&:statvfs", PyUnicode_FSConverter, &opath)) return NULL; + path = PyBytes_AS_STRING(opath); Py_BEGIN_ALLOW_THREADS res = statvfs(path, &st); Py_END_ALLOW_THREADS if (res != 0) - return posix_error_with_filename(path); + return posix_error_with_allocated_filename(opath); - return _pystatvfs_fromstructstatvfs(st); + result = _pystatvfs_fromstructstatvfs(st); + Py_DECREF(opath); + return result; } #endif /* HAVE_STATVFS */ diff --git a/Objects/longobject.c b/Objects/longobject.c index 3a675c4..51da329 100644 --- a/Objects/longobject.c +++ b/Objects/longobject.c @@ -4130,8 +4130,14 @@ long_new(PyTypeObject *type, PyObject *args, PyObject *kwds) if (!PyArg_ParseTupleAndKeywords(args, kwds, "|OO:int", kwlist, &x, &obase)) return NULL; - if (x == NULL) + if (x == NULL) { + if (obase != NULL) { + PyErr_SetString(PyExc_TypeError, + "int() missing string argument"); + return NULL; + } return PyLong_FromLong(0L); + } if (obase == NULL) return PyNumber_Long(x); @@ -4140,7 +4146,7 @@ long_new(PyTypeObject *type, PyObject *args, PyObject *kwds) return NULL; if (overflow || (base != 0 && base < 2) || base > 36) { PyErr_SetString(PyExc_ValueError, - "int() arg 2 must be >= 2 and <= 36"); + "int() base must be >= 2 and <= 36"); return NULL; } diff --git a/Objects/unicodeobject.c b/Objects/unicodeobject.c index bb45b20..f8c738b 100644 --- a/Objects/unicodeobject.c +++ b/Objects/unicodeobject.c @@ -1434,8 +1434,8 @@ PyObject *PyUnicode_FromEncodedObject(register PyObject *obj, /* Convert encoding to lower case and replace '_' with '-' in order to catch e.g. UTF_8. Return 0 on error (encoding is longer than lower_len-1), 1 on success. */ -static int -normalize_encoding(const char *encoding, +int +_Py_normalize_encoding(const char *encoding, char *lower, size_t lower_len) { @@ -1477,7 +1477,7 @@ PyObject *PyUnicode_Decode(const char *s, encoding = PyUnicode_GetDefaultEncoding(); /* Shortcuts for common default encodings */ - if (normalize_encoding(encoding, lower, sizeof(lower))) { + if (_Py_normalize_encoding(encoding, lower, sizeof(lower))) { if (strcmp(lower, "utf-8") == 0) return PyUnicode_DecodeUTF8(s, size, errors); else if ((strcmp(lower, "latin-1") == 0) || @@ -1695,7 +1695,7 @@ PyObject *PyUnicode_AsEncodedString(PyObject *unicode, encoding = PyUnicode_GetDefaultEncoding(); /* Shortcuts for common default encodings */ - if (normalize_encoding(encoding, lower, sizeof(lower))) { + if (_Py_normalize_encoding(encoding, lower, sizeof(lower))) { if (strcmp(lower, "utf-8") == 0) return PyUnicode_EncodeUTF8(PyUnicode_AS_UNICODE(unicode), PyUnicode_GET_SIZE(unicode), diff --git a/PC/winreg.c b/PC/winreg.c index 454ea53..e4b9307 100644 --- a/PC/winreg.c +++ b/PC/winreg.c @@ -785,7 +785,7 @@ Py2Reg(PyObject *value, DWORD typ, BYTE **retDataBuf, DWORD *retDataSize) memcpy(*retDataBuf, &zero, sizeof(DWORD)); } else { - DWORD d = PyLong_AsLong(value); + DWORD d = PyLong_AsUnsignedLong(value); memcpy(*retDataBuf, &d, sizeof(DWORD)); } break; @@ -900,9 +900,9 @@ Reg2Py(BYTE *retDataBuf, DWORD retDataSize, DWORD typ) switch (typ) { case REG_DWORD: if (retDataSize == 0) - obData = PyLong_FromLong(0); + obData = PyLong_FromUnsignedLong(0); else - obData = PyLong_FromLong(*(int *)retDataBuf); + obData = PyLong_FromUnsignedLong(*(int *)retDataBuf); break; case REG_SZ: case REG_EXPAND_SZ: diff --git a/Python/fileutils.c b/Python/fileutils.c index 94ab8e4..53e8a47 100644 --- a/Python/fileutils.c +++ b/Python/fileutils.c @@ -3,11 +3,191 @@ # include <windows.h> #endif +#ifdef HAVE_LANGINFO_H +#include <locale.h> +#include <langinfo.h> +#endif + #ifdef __APPLE__ extern wchar_t* _Py_DecodeUTF8_surrogateescape(const char *s, Py_ssize_t size); #endif -#ifdef HAVE_STAT +#if !defined(__APPLE__) && !defined(MS_WINDOWS) +extern int _Py_normalize_encoding(const char *, char *, size_t); + +/* Workaround FreeBSD and OpenIndiana locale encoding issue with the C locale. + On these operating systems, nl_langinfo(CODESET) announces an alias of the + ASCII encoding, whereas mbstowcs() and wcstombs() functions use the + ISO-8859-1 encoding. The problem is that os.fsencode() and os.fsdecode() use + locale.getpreferredencoding() codec. For example, if command line arguments + are decoded by mbstowcs() and encoded back by os.fsencode(), we get a + UnicodeEncodeError instead of retrieving the original byte string. + + The workaround is enabled if setlocale(LC_CTYPE, NULL) returns "C", + nl_langinfo(CODESET) announces "ascii" (or an alias to ASCII), and at least + one byte in range 0x80-0xff can be decoded from the locale encoding. The + workaround is also enabled on error, for example if getting the locale + failed. + + Values of locale_is_ascii: + + 1: the workaround is used: _Py_wchar2char() uses + encode_ascii_surrogateescape() and _Py_char2wchar() uses + decode_ascii_surrogateescape() + 0: the workaround is not used: _Py_wchar2char() uses wcstombs() and + _Py_char2wchar() uses mbstowcs() + -1: unknown, need to call check_force_ascii() to get the value +*/ +static int force_ascii = -1; + +static int +check_force_ascii(void) +{ + char *loc; +#if defined(HAVE_LANGINFO_H) && defined(CODESET) + char *codeset, **alias; + char encoding[100]; + int is_ascii; + unsigned int i; + char* ascii_aliases[] = { + "ascii", + "646", + "ansi-x3.4-1968", + "ansi-x3-4-1968", + "ansi-x3.4-1986", + "cp367", + "csascii", + "ibm367", + "iso646-us", + "iso-646.irv-1991", + "iso-ir-6", + "us", + "us-ascii", + NULL + }; +#endif + + loc = setlocale(LC_CTYPE, NULL); + if (loc == NULL) + goto error; + if (strcmp(loc, "C") != 0) { + /* the LC_CTYPE locale is different than C */ + return 0; + } + +#if defined(HAVE_LANGINFO_H) && defined(CODESET) + codeset = nl_langinfo(CODESET); + if (!codeset || codeset[0] == '\0') { + /* CODESET is not set or empty */ + goto error; + } + if (!_Py_normalize_encoding(codeset, encoding, sizeof(encoding))) + goto error; + + is_ascii = 0; + for (alias=ascii_aliases; *alias != NULL; alias++) { + if (strcmp(encoding, *alias) == 0) { + is_ascii = 1; + break; + } + } + if (!is_ascii) { + /* nl_langinfo(CODESET) is not "ascii" or an alias of ASCII */ + return 0; + } + + for (i=0x80; i<0xff; i++) { + unsigned char ch; + wchar_t wch; + size_t res; + + ch = (unsigned char)i; + res = mbstowcs(&wch, (char*)&ch, 1); + if (res != (size_t)-1) { + /* decoding a non-ASCII character from the locale encoding succeed: + the locale encoding is not ASCII, force ASCII */ + return 1; + } + } + /* None of the bytes in the range 0x80-0xff can be decoded from the locale + encoding: the locale encoding is really ASCII */ + return 0; +#else + /* nl_langinfo(CODESET) is not available: always force ASCII */ + return 1; +#endif + +error: + /* if an error occured, force the ASCII encoding */ + return 1; +} + +static char* +encode_ascii_surrogateescape(const wchar_t *text, size_t *error_pos) +{ + char *result = NULL, *out; + size_t len, i; + wchar_t ch; + + if (error_pos != NULL) + *error_pos = (size_t)-1; + + len = wcslen(text); + + result = PyMem_Malloc(len + 1); /* +1 for NUL byte */ + if (result == NULL) + return NULL; + + out = result; + for (i=0; i<len; i++) { + ch = text[i]; + + if (ch <= 0x7f) { + /* ASCII character */ + *out++ = (char)ch; + } + else if (0xdc80 <= ch && ch <= 0xdcff) { + /* UTF-8b surrogate */ + *out++ = (char)(ch - 0xdc00); + } + else { + if (error_pos != NULL) + *error_pos = i; + PyMem_Free(result); + return NULL; + } + } + *out = '\0'; + return result; +} +#endif /* !defined(__APPLE__) && !defined(MS_WINDOWS) */ + +#if !defined(__APPLE__) && (!defined(MS_WINDOWS) || !defined(HAVE_MBRTOWC)) +static wchar_t* +decode_ascii_surrogateescape(const char *arg, size_t *size) +{ + wchar_t *res; + unsigned char *in; + wchar_t *out; + + res = PyMem_Malloc((strlen(arg)+1)*sizeof(wchar_t)); + if (!res) + return NULL; + + in = (unsigned char*)arg; + out = res; + while(*in) + if(*in < 128) + *out++ = *in++; + else + *out++ = 0xdc00 + *in++; + *out = 0; + if (size != NULL) + *size = out - res; + return res; +} +#endif + /* Decode a byte string from the locale encoding with the surrogateescape error handler (undecodable bytes are decoded as characters @@ -39,20 +219,35 @@ _Py_char2wchar(const char* arg, size_t *size) return wstr; #else wchar_t *res; + size_t argsize; + size_t count; + unsigned char *in; + wchar_t *out; +#ifdef HAVE_MBRTOWC + mbstate_t mbs; +#endif + +#ifndef MS_WINDOWS + if (force_ascii == -1) + force_ascii = check_force_ascii(); + + if (force_ascii) { + /* force ASCII encoding to workaround mbstowcs() issue */ + res = decode_ascii_surrogateescape(arg, size); + if (res == NULL) + goto oom; + return res; + } +#endif + #ifdef HAVE_BROKEN_MBSTOWCS /* Some platforms have a broken implementation of * mbstowcs which does not count the characters that * would result from conversion. Use an upper bound. */ - size_t argsize = strlen(arg); + argsize = strlen(arg); #else - size_t argsize = mbstowcs(NULL, arg, 0); -#endif - size_t count; - unsigned char *in; - wchar_t *out; -#ifdef HAVE_MBRTOWC - mbstate_t mbs; + argsize = mbstowcs(NULL, arg, 0); #endif if (argsize != (size_t)-1) { res = (wchar_t *)PyMem_Malloc((argsize+1)*sizeof(wchar_t)); @@ -122,23 +317,16 @@ _Py_char2wchar(const char* arg, size_t *size) argsize -= converted; out++; } + if (size != NULL) + *size = out - res; #else /* HAVE_MBRTOWC */ /* Cannot use C locale for escaping; manually escape as if charset is ASCII (i.e. escape all bytes > 128. This will still roundtrip correctly in the locale's charset, which must be an ASCII superset. */ - res = PyMem_Malloc((strlen(arg)+1)*sizeof(wchar_t)); - if (!res) goto oom; - in = (unsigned char*)arg; - out = res; - while(*in) - if(*in < 128) - *out++ = *in++; - else - *out++ = 0xdc00 + *in++; - *out = 0; + res = decode_ascii_surrogateescape(arg, size); + if (res == NULL) + goto oom; #endif /* HAVE_MBRTOWC */ - if (size != NULL) - *size = out - res; return res; oom: fprintf(stderr, "out of memory\n"); @@ -198,6 +386,14 @@ _Py_wchar2char(const wchar_t *text, size_t *error_pos) size_t i, size, converted; wchar_t c, buf[2]; +#ifndef MS_WINDOWS + if (force_ascii == -1) + force_ascii = check_force_ascii(); + + if (force_ascii) + return encode_ascii_surrogateescape(text, error_pos); +#endif + /* The function works in two steps: 1. compute the length of the output buffer in bytes (size) 2. outputs the bytes */ @@ -238,7 +434,7 @@ _Py_wchar2char(const wchar_t *text, size_t *error_pos) } } if (result != NULL) { - *bytes = 0; + *bytes = '\0'; break; } @@ -282,6 +478,8 @@ _Py_wstat(const wchar_t* path, struct stat *buf) } #endif +#ifdef HAVE_STAT + /* Call _wstat() on Windows, or encode the path to the filesystem encoding and call stat() otherwise. Only fill st_mode attribute on Windows. @@ -310,6 +508,8 @@ _Py_stat(PyObject *path, struct stat *statbuf) #endif } +#endif + /* Open a file. Use _wfopen() on Windows, encode the path to the locale encoding and use fopen() otherwise. */ @@ -478,4 +678,3 @@ _Py_wgetcwd(wchar_t *buf, size_t size) #endif } -#endif |