summaryrefslogtreecommitdiffstats
path: root/Parser/tokenizer.c
Commit message (Collapse)AuthorAgeFilesLines
* Issue 24226: Fix parsing of many sequential one-line 'def' statements.Yury Selivanov2015-05-181-8/+17
|
* PEP 0492 -- Coroutines with async and await syntax. Issue #24017.Yury Selivanov2015-05-121-0/+68
|
* merge 3.4 (#24022)Benjamin Peterson2015-04-211-4/+3
|\
| * do not call into python api if an exception is set (#24022)Benjamin Peterson2015-04-211-4/+3
| |
* | merge 3.4 (#21642)Benjamin Peterson2014-06-071-5/+14
|\ \ | |/
| * allow the keyword else immediately after (no space) an integer (closes #21642)Benjamin Peterson2014-06-071-5/+14
| |
* | PEP 465: a dedicated infix operator for matrix multiplication (closes #21176)Benjamin Peterson2014-04-101-1/+7
|/
* Merge with 3.3Martin v. Löwis2014-02-281-2/+12
|\
| * Issue #20731: Properly position in source code files even if theyMartin v. Löwis2014-02-281-2/+12
| | | | | | | | are opened in text mode. Patch by Serhiy Storchaka.
* | Do not reset the line number because we already set file position to correctSerhiy Storchaka2014-01-091-8/+0
|\ \ | |/ | | | | | | | | value. (fixes error in patch for issue #18960)
| * Do not reset the line number because we already set file position to correctSerhiy Storchaka2014-01-091-8/+0
| | | | | | | | | | | | value. (fixes error in patch for issue #18960)
* | Issue #18960: Fix bugs with Python source code encoding in the second line.Serhiy Storchaka2014-01-091-4/+22
|\ \ | |/ | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * The first line of Python script could be executed twice when the source encoding (not equal to 'utf-8') was specified on the second line. * Now the source encoding declaration on the second line isn't effective if the first line contains anything except a comment. * As a consequence, 'python -x' works now again with files with the source encoding declarations specified on the second file, and can be used again to make Python batch files on Windows. * The tokenize module now ignore the source encoding declaration on the second line if the first line contains anything except a comment. * IDLE now ignores the source encoding declaration on the second line if the first line contains anything except a comment. * 2to3 and the findnocoding.py script now ignore the source encoding declaration on the second line if the first line contains anything except a comment.
| * Issue #18960: Fix bugs with Python source code encoding in the second line.Serhiy Storchaka2014-01-091-4/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * The first line of Python script could be executed twice when the source encoding (not equal to 'utf-8') was specified on the second line. * Now the source encoding declaration on the second line isn't effective if the first line contains anything except a comment. * As a consequence, 'python -x' works now again with files with the source encoding declarations specified on the second file, and can be used again to make Python batch files on Windows. * The tokenize module now ignore the source encoding declaration on the second line if the first line contains anything except a comment. * IDLE now ignores the source encoding declaration on the second line if the first line contains anything except a comment. * 2to3 and the findnocoding.py script now ignore the source encoding declaration on the second line if the first line contains anything except a comment.
* | Issue #1772673: The type of `char*` arguments now changed to `const char*`.Serhiy Storchaka2013-10-191-3/+4
| |
* | Issue #18571: Implementation of the PEP 446: file descriptors and file handlesVictor Stinner2013-08-271-0/+5
| | | | | | | | | | are now created non-inheritable; add functions os.get/set_inheritable(), os.get/set_handle_inheritable() and socket.socket.get/set_inheritable().
* | Issue #18722: Remove uses of the "register" keyword in C code.Antoine Pitrou2013-08-131-6/+6
| |
* | merge 3.3Benjamin Peterson2013-07-161-1/+2
|\ \ | |/
| * move declaration to top of blockBenjamin Peterson2013-07-161-1/+2
| |
* | merge 3.3 (closes #18470)Benjamin Peterson2013-07-161-36/+46
|\ \ | |/
| * check the return value of new_string() (closes #18470)Benjamin Peterson2013-07-161-36/+46
| |
* | Issue #18038: SyntaxError raised during compilation sources with illegalSerhiy Storchaka2013-06-091-7/+7
|\ \ | |/ | | | | encoding now always contains an encoding name.
| * Issue #18038: SyntaxError raised during compilation sources with illegalSerhiy Storchaka2013-06-091-7/+7
| | | | | | | | encoding now always contains an encoding name.
* | Issue #9566: Fix compiler warning on Windows 64-bitVictor Stinner2013-06-041-1/+2
| |
* | make _PyParser_TokenNames constBenjamin Peterson2012-10-241-1/+1
|/
* Issue #15096: Drop support for the ur string prefixChristian Heimes2012-06-201-2/+3
|
* Basic support for PEP 414 without docs or tests.Armin Ronacher2012-03-041-3/+7
|
* Issue #13748: Raw bytes literals can now be written with the `rb` prefix as ↵Antoine Pitrou2012-01-121-7/+9
| | | | well as `br`.
* Rename _Py_identifier to _Py_IDENTIFIER.Martin v. Löwis2011-10-141-2/+2
|
* Use identifier API for PyObject_GetAttrString.Martin v. Löwis2011-10-101-1/+2
|
* Add API for static strings, primarily good for identifiers.Martin v. Löwis2011-10-091-1/+2
| | | | Thanks to Konrad Schöbel and Jasper Schulz for helping with the mass-editing.
* Implement PEP 393.Martin v. Löwis2011-09-281-2/+4
|
* Revert bb62908896fe, but keep the testJesus Cea2011-04-251-13/+6
|
* Correctly merging #9319 into 3.3?Jesus Cea2011-04-251-6/+13
|\
| * Issue #9319: Fix a crash on parsing a Python source code without encodingVictor Stinner2011-04-221-1/+4
| | | | | | | | | | cookie and not valid in UTF-8: use "<file>" as the filename instead of reading from NULL.
* | Issue #9319: Include the filename in "Non-UTF8 code ..." syntax error.Victor Stinner2011-04-041-14/+27
| |
* | Issue #10785: Store the filename as Unicode in the Python parser.Victor Stinner2011-04-041-13/+22
|/
* Issue #10841: don't translate newlines for pgenVictor Stinner2011-01-071-1/+1
|
* Issue #10841: set binary mode on files; the parser translates newlinesVictor Stinner2011-01-071-0/+7
| | | | | | On Windows, set the binary mode on stdin, stdout, stderr and all io.FileIO objects (to not translate newlines, \r\n <=> \n). The Python parser translates newlines (\r\n => \n).
* Remove arbitrary string length limitsVictor Stinner2011-01-051-1/+1
| | | | | PyUnicode_FromFormat() and PyErr_Format() allocates a buffer of the needed size, it is no more a fixed-buffer of 500 bytes.
* Issue #10778: decoding_fgets() decodes the filename from the filesystemVictor Stinner2010-12-271-6/+11
| | | | encoding instead of UTF-8.
* Issue #10095: fp_setreadl() doesn't reopen the file, reuse instead the fileVictor Stinner2010-10-141-6/+9
| | | | descriptor.
* Recorded merge of revisions 81029 via svnmerge fromAntoine Pitrou2010-05-091-1384/+1384
| | | | | | | | | | svn+ssh://pythondev@svn.python.org/python/trunk ........ r81029 | antoine.pitrou | 2010-05-09 16:46:46 +0200 (dim., 09 mai 2010) | 3 lines Untabify C files. Will watch buildbots. ........
* Use PyErr_Format() in decoding_fgets()Victor Stinner2010-04-281-3/+1
| | | | Avoid a buffer of 500 bytes allocated on the stack.
* Merged revisions 79725 via svnmerge fromBenjamin Peterson2010-04-031-11/+1
| | | | | | | | | | | | svn+ssh://pythondev@svn.python.org/python/trunk ........ r79725 | benjamin.peterson | 2010-04-03 18:03:35 -0500 (Sat, 03 Apr 2010) | 4 lines use our own locale independent ctype macros requires building pyctype.o into pgen ........
* Merged revisions 79723 via svnmerge fromBenjamin Peterson2010-04-031-1/+11
| | | | | | | | | | svn+ssh://pythondev@svn.python.org/python/trunk ........ r79723 | benjamin.peterson | 2010-04-03 17:48:51 -0500 (Sat, 03 Apr 2010) | 1 line ensure that the locale does not affect the tokenization of identifiers ........
* Merged revisions 78826 via svnmerge fromVictor Stinner2010-03-121-5/+12
| | | | | | | | | | | | | svn+ssh://pythondev@svn.python.org/python/trunk ........ r78826 | victor.stinner | 2010-03-10 23:30:19 +0100 (mer., 10 mars 2010) | 5 lines Issue #3137: Don't ignore errors at startup, especially a keyboard interrupt (SIGINT). If an error occurs while importing the site module, the error is printed and Python exits. Initialize the GIL before importing the site module. ........
* Merged revisions 78603 via svnmerge fromVictor Stinner2010-03-031-23/+28
| | | | | | | | | | | | | svn+ssh://pythondev@svn.python.org/python/trunk ........ r78603 | victor.stinner | 2010-03-03 00:20:02 +0100 (mer., 03 mars 2010) | 5 lines Issue #7820: The parser tokenizer restores all bytes in the right if the BOM check fails. Fix an assertion in pydebug mode. ........
* Merged revisions 76052,76522,76591,76689,76697,76733 via svnmerge fromBenjamin Peterson2009-12-131-12/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | svn+ssh://pythondev@svn.python.org/python/trunk ........ r76052 | gregory.p.smith | 2009-11-01 20:02:38 -0600 (Sun, 01 Nov 2009) | 5 lines see issue1006238, this merges in the following patch to ease cross compiling the printf %zd check. http://sources.gentoo.org/viewcvs.py/gentoo-x86/dev-lang/python/files/python-2.5-cross-printf.patch?rev=1.1&view=markup ........ r76522 | barry.warsaw | 2009-11-25 12:38:32 -0600 (Wed, 25 Nov 2009) | 2 lines Add mktime_tz to __all__. It's documented as being available in email.utils. ........ r76591 | benjamin.peterson | 2009-11-29 16:26:26 -0600 (Sun, 29 Nov 2009) | 4 lines now that deepcopy can handle instance methods, this hack can be removed #7409 Thanks Robert Collins ........ r76689 | benjamin.peterson | 2009-12-06 11:37:48 -0600 (Sun, 06 Dec 2009) | 1 line rewrite translate_newlines for clarity ........ r76697 | benjamin.peterson | 2009-12-06 15:24:30 -0600 (Sun, 06 Dec 2009) | 2 lines fix test_parser from tokenizer tweak ........ r76733 | benjamin.peterson | 2009-12-09 21:37:59 -0600 (Wed, 09 Dec 2009) | 1 line substitute PyDict_Check() for PyObject_IsInstance ........
* Merged revisions ↵Benjamin Peterson2009-11-131-11/+23
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 75149,75260-75263,75265-75267,75292,75300,75376,75405,75429-75433,75437,75445,75501,75551,75572,75589-75591,75657,75742,75868,75952-75957,76057,76105,76139,76143,76162,76223 via svnmerge from svn+ssh://pythondev@svn.python.org/python/trunk ........ r75149 | gregory.p.smith | 2009-09-29 16:56:31 -0500 (Tue, 29 Sep 2009) | 3 lines Mention issue6972 in extractall docs about overwriting things outside of the supplied path. ........ r75260 | andrew.kuchling | 2009-10-05 16:24:20 -0500 (Mon, 05 Oct 2009) | 1 line Wording fix ........ r75261 | andrew.kuchling | 2009-10-05 16:24:35 -0500 (Mon, 05 Oct 2009) | 1 line Fix narkup ........ r75262 | andrew.kuchling | 2009-10-05 16:25:03 -0500 (Mon, 05 Oct 2009) | 1 line Document 'skip' parameter to constructor ........ r75263 | andrew.kuchling | 2009-10-05 16:25:35 -0500 (Mon, 05 Oct 2009) | 1 line Note side benefit of socket.create_connection() ........ r75265 | andrew.kuchling | 2009-10-05 17:31:11 -0500 (Mon, 05 Oct 2009) | 1 line Reword sentence ........ r75266 | andrew.kuchling | 2009-10-05 17:32:48 -0500 (Mon, 05 Oct 2009) | 1 line Use standard comma punctuation; reword some sentences in the docs ........ r75267 | andrew.kuchling | 2009-10-05 17:42:56 -0500 (Mon, 05 Oct 2009) | 1 line Backport r73983: Document the thousands separator. ........ r75292 | benjamin.peterson | 2009-10-08 22:11:36 -0500 (Thu, 08 Oct 2009) | 1 line death to old CVS keyword ........ r75300 | benjamin.peterson | 2009-10-09 16:48:14 -0500 (Fri, 09 Oct 2009) | 1 line fix some coding style ........ r75376 | benjamin.peterson | 2009-10-11 20:26:07 -0500 (Sun, 11 Oct 2009) | 1 line platform we don't care about ........ r75405 | neil.schemenauer | 2009-10-14 12:17:14 -0500 (Wed, 14 Oct 2009) | 4 lines Issue #1754094: Improve the stack depth calculation in the compiler. There should be no other effect than a small decrease in memory use. Patch by Christopher Tur Lesniewski-Laas. ........ r75429 | benjamin.peterson | 2009-10-14 20:47:28 -0500 (Wed, 14 Oct 2009) | 1 line pep8ify if blocks ........ r75430 | benjamin.peterson | 2009-10-14 20:49:37 -0500 (Wed, 14 Oct 2009) | 1 line use floor division and add a test that exercises the tabsize codepath ........ r75431 | benjamin.peterson | 2009-10-14 20:56:25 -0500 (Wed, 14 Oct 2009) | 1 line change test to what I intended ........ r75432 | benjamin.peterson | 2009-10-14 22:05:39 -0500 (Wed, 14 Oct 2009) | 1 line some cleanups ........ r75433 | benjamin.peterson | 2009-10-14 22:06:55 -0500 (Wed, 14 Oct 2009) | 1 line make inspect.isabstract() always return a boolean; add a test for it, too #7069 ........ r75437 | benjamin.peterson | 2009-10-15 10:44:46 -0500 (Thu, 15 Oct 2009) | 1 line only clear a module's __dict__ if the module is the only one with a reference to it #7140 ........ r75445 | vinay.sajip | 2009-10-16 09:06:44 -0500 (Fri, 16 Oct 2009) | 1 line Issue #7120: logging: Removed import of multiprocessing which is causing crash in GAE. ........ r75501 | antoine.pitrou | 2009-10-18 13:37:11 -0500 (Sun, 18 Oct 2009) | 3 lines Add a comment about unreachable code, and fix a typo ........ r75551 | benjamin.peterson | 2009-10-19 22:14:10 -0500 (Mon, 19 Oct 2009) | 1 line use property api ........ r75572 | benjamin.peterson | 2009-10-20 16:55:17 -0500 (Tue, 20 Oct 2009) | 1 line clarify buffer arg #7178 ........ r75589 | benjamin.peterson | 2009-10-21 21:26:47 -0500 (Wed, 21 Oct 2009) | 1 line whitespace ........ r75590 | benjamin.peterson | 2009-10-21 21:36:47 -0500 (Wed, 21 Oct 2009) | 1 line rewrite to be nice to other implementations ........ r75591 | benjamin.peterson | 2009-10-21 21:50:38 -0500 (Wed, 21 Oct 2009) | 4 lines rewrite for style, clarify, and comments Also, use the hasattr() like scheme of allowing BaseException exceptions through. ........ r75657 | antoine.pitrou | 2009-10-24 07:41:27 -0500 (Sat, 24 Oct 2009) | 3 lines Fix compilation error in debug mode. ........ r75742 | benjamin.peterson | 2009-10-26 17:51:16 -0500 (Mon, 26 Oct 2009) | 1 line use 'is' instead of id() ........ r75868 | benjamin.peterson | 2009-10-27 15:59:18 -0500 (Tue, 27 Oct 2009) | 1 line test expect base classes ........ r75952 | georg.brandl | 2009-10-29 15:38:32 -0500 (Thu, 29 Oct 2009) | 1 line Use the correct function name in docstring. ........ r75953 | georg.brandl | 2009-10-29 15:39:50 -0500 (Thu, 29 Oct 2009) | 1 line Remove mention of the old -X command line switch. ........ r75954 | georg.brandl | 2009-10-29 15:53:00 -0500 (Thu, 29 Oct 2009) | 1 line Use constants instead of magic integers for test result. Do not re-run with --verbose3 for environment changing tests. ........ r75955 | georg.brandl | 2009-10-29 15:54:03 -0500 (Thu, 29 Oct 2009) | 1 line Use a single style for all the docstrings in the math module. ........ r75956 | georg.brandl | 2009-10-29 16:16:34 -0500 (Thu, 29 Oct 2009) | 1 line I do not think the "railroad" program mentioned is still available. ........ r75957 | georg.brandl | 2009-10-29 16:44:56 -0500 (Thu, 29 Oct 2009) | 1 line Fix constant name. ........ r76057 | benjamin.peterson | 2009-11-02 09:06:45 -0600 (Mon, 02 Nov 2009) | 1 line prevent a rather unlikely segfault ........ r76105 | georg.brandl | 2009-11-04 01:38:12 -0600 (Wed, 04 Nov 2009) | 1 line #7259: show correct equivalent for operator.i* operations in docstring; fix minor issues in operator docs. ........ r76139 | benjamin.peterson | 2009-11-06 19:04:38 -0600 (Fri, 06 Nov 2009) | 1 line spelling ........ r76143 | georg.brandl | 2009-11-07 02:26:07 -0600 (Sat, 07 Nov 2009) | 1 line #7271: fix typo. ........ r76162 | benjamin.peterson | 2009-11-08 22:10:53 -0600 (Sun, 08 Nov 2009) | 1 line discuss how to use -p ........ r76223 | georg.brandl | 2009-11-12 02:29:46 -0600 (Thu, 12 Nov 2009) | 1 line Give the profile module a module directive. ........
* Merged revisions 76230 via svnmerge fromBenjamin Peterson2009-11-131-18/+74
| | | | | | | | | | svn+ssh://pythondev@svn.python.org/python/trunk ........ r76230 | benjamin.peterson | 2009-11-12 17:39:44 -0600 (Thu, 12 Nov 2009) | 2 lines fix several compile() issues by translating newlines in the tokenizer ........
generated by cgit v0.12 at 2024-11-13 13:41:10 (GMT)