summaryrefslogtreecommitdiffstats
path: root/Parser/tokenizer.c
Commit message (Collapse)AuthorAgeFilesLines
* [Backport r46602 | neal.norwitz]Andrew M. Kuchling2006-10-061-8/+15
| | | | | | | | | | | | Patch #1357836: Prevent an invalid memory read from test_coding in case the done flag is set. In that case, the loop isn't entered. I wonder if rather than setting the done flag in the cases before the loop, if they should just exit early. This code looks like it should be refactored. Backport candidate (also the early break above if decoding_fgets fails)
* [Backport rev39767 by nnorwitz]Andrew M. Kuchling2006-09-291-0/+3
| | | | | | | Free coding spec (cs) if there was an error to prevent mem leak. Maybe backport candidate. [Bugfix seems applicable to 2.4 to me. --amk]
* Backport 41753:Neal Norwitz2006-02-211-0/+6
| | | | | Bug #1378022, UTF-8 files with a leading BOM crashed the interpreter. Also bug #1435487 (dup).
* Backport checkin:Walter Dörwald2005-07-121-27/+45
| | | | | Apply SF patch #1101726: Fix buffer overrun in tokenizer.c when a source file with a PEP 263 encoding declaration results in long decoded line.
* SF #941229: Decode source code with sys.stdin.encoding in interactiveHye-Shik Chang2004-08-041-0/+61
| | | | | | | modes like non-interactive modes. This allows for non-latin-1 users to write unicode strings directly and sets Japanese users free from weird manual escaping <wink> in shift_jis environments. (Reviewed by Martin v. Loewis)
* PEP-0318, @decorator-style. In Guido's words:Anthony Baxter2004-08-021-0/+2
| | | | | "@ seems the syntax that everybody can hate equally" Implementation by Mark Russell, from SF #979728.
* Getting rid of all the code inside #ifdef macintosh too.Jack Jansen2003-11-201-11/+0
|
* Add URL for PEP to the source code encoding warning.Marc-André Lemburg2003-02-171-6/+12
| | | | | | Remove the usage of PyErr_WarnExplicit() since this could cause sensitive information from the source files to appear in e.g. log files.
* patch 680474 that fixes bug 679880: compile/eval/exec refused utf-8 bomJust van Rossum2003-02-091-2/+2
| | | | mark. Added unit test.
* Fix [ 665014 ] files with long lines and an encoding crash.Mark Hammond2003-01-141-1/+2
| | | | | Ensure that the 'size' arg is correctly passed to the encoding reader to prevent buffer overflows.
* Constify filenames and scripts. Fixes #651362.Martin v. Löwis2002-12-111-3/+5
|
* Fix compiler warning on HP-UX.Neal Norwitz2002-11-021-2/+2
| | | | Cast param to isalnum() to int.
* Patch #512981: Update readline input stream on sys.stdin/out change.Martin v. Löwis2002-10-261-2/+2
|
* Removed reliance on gcc/C99 extension.Tim Peters2002-09-031-1/+3
|
* Ignore encoding declarations inside strings. Fixes #603509.Martin v. Löwis2002-09-031-1/+16
|
* Squash a few calls to the hideously expensive PyObject_CallObject(o,a)Guido van Rossum2002-08-161-3/+14
| | | | | | | -- replace then with slightly faster PyObject_Call(o,a,NULL). (The difference is that the latter requires a to be a tuple; the former allows other values and wraps them in a tuple if necessary; it involves two more levels of C function calls to accomplish all that.)
* provide less mysterious error messages when seeing end-of-line inSkip Montanaro2002-08-151-3/+6
| | | | | single-quoted strings or end-of-file in triple-quoted strings. closes patch 586561.
* Use Py_FatalError instead of abort.Martin v. Löwis2002-08-071-2/+3
|
* Fix PEP 263 code --without-unicode. Fixes #591943.Martin v. Löwis2002-08-071-0/+18
|
* Added a cast to shut up a compiler warning.Jack Jansen2002-08-051-1/+1
|
* Add 1 to lineno in deprecation warning. Fixes #590888.Martin v. Löwis2002-08-051-1/+3
|
* Make pgen compile with pydebug. Duplicate normalized names, as it mayMartin v. Löwis2002-08-041-2/+6
| | | | be longer than the old string.
* Group statements properly.Martin v. Löwis2002-08-041-6/+12
|
* Repaired a fatal compiler error in the debug build: it's not clear whatTim Peters2002-08-041-1/+1
| | | | this was trying to assert, but the name it referenced didn't exist.
* Squash compiler wng about signed-vs-unsigned mismatch.Tim Peters2002-08-041-1/+1
|
* Patch #534304: Implement phase 1 of PEP 263.Martin v. Löwis2002-08-041-8/+440
|
* Mass checkin of universal newline support.Jack Jansen2002-04-141-4/+5
| | | | | | | | Highlights: import and friends will understand any of \r, \n and \r\n as end of line. Python file input will do the same if you use mode 'U'. Everything can be disabled by configuring with --without-universal-newlines. See PEP278 for details.
* SF patch #455966: Allow leading 0 in float/imag literals.Tim Peters2001-08-301-3/+22
| | | | Consequences for Jython still unknown (but raised on Jython-Dev).
* SF bug [#455775] float parsing discrepancy.Tim Peters2001-08-271-5/+8
| | | | PyTokenizer_Get: error if exponent contains no digits (3e, 2.0e+, ...).
* Implement PEP 238 in its (almost) full glory.Guido van Rossum2001-08-081-0/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | This introduces: - A new operator // that means floor division (the kind of division where 1/2 is 0). - The "future division" statement ("from __future__ import division) which changes the meaning of the / operator to implement "true division" (where 1/2 is 0.5). - New overloadable operators __truediv__ and __floordiv__. - New slots in the PyNumberMethods struct for true and floor division, new abstract APIs for them, new opcodes, and so on. I emphasize that without the future division statement, the semantics of / will remain unchanged until Python 3.0. Not yet implemented are warnings (default off) when / is used with int or long arguments. This has been on display since 7/31 as SF patch #443474. Flames to /dev/null.
* SF but #417587: compiler warnings compiling 2.1.Tim Peters2001-04-211-3/+0
| | | | Repaired *some* of the SGI compiler warnings Sjoerd Mullender reported.
* REMOVED all CWI, CNRI and BeOpen copyright markings.Guido van Rossum2000-09-011-9/+0
| | | | This should match the situation in the 1.6b1 tree.
* Support for three-token characters (**=, >>=, <<=) which was written byThomas Wouters2000-08-241-0/+94
| | | | | Michael Hudson, and support in general for the augmented assignment syntax. The graminit.c patch is large!
* Mass ANSIfication.Thomas Wouters2000-07-221-25/+12
| | | | | | Work around intrcheck.c's desire to pass 'PyErr_CheckSignals' to 'Py_AddPendingCall' by providing a (static) wrapper function that has the right number of arguments.
* Create two new exceptions: IndentationError and TabError. These areFred Drake2000-07-111-9/+5
| | | | | | | used for indentation related errors. This patch includes Ping's improvements for indentation-related error messages. Closes SourceForge patches #100734 and #100856.
* Nuke all remaining occurrences of Py_PROTO and Py_FPROTO.Tim Peters2000-07-091-4/+4
|
* Change copyright notice - 2nd try.Guido van Rossum2000-06-301-6/+0
|
* Change copyright notice.Guido van Rossum2000-06-301-22/+7
|
* Trent Mick: familiar simple Win64 patchesGuido van Rossum2000-06-281-3/+3
|
* Vladimir Marangozov's long-awaited malloc restructuring.Guido van Rossum2000-05-031-6/+7
| | | | | | | | | | For more comments, read the patches@python.org archives. For documentation read the comments in mymalloc.h and objimpl.h. (This is not exactly what Vladimir posted to the patches list; I've made a few changes, and Vladimir sent me a fix in private email for a problem that only occurs in debug mode. I'm also holding back on his change to main.c, which seems unnecessary to me.)
* Only write message about changed Tab size with -v.Guido van Rossum2000-04-031-1/+2
|
* Fix by Eric Raymond: make the code that looks for various bits ofGuido van Rossum2000-03-311-15/+30
| | | | | tab-setting magic much smarter, more correct, and more easily extensible.
* Marc-Andre Lemburg: add new string token types u"..." and ur"..."Guido van Rossum2000-03-101-0/+10
| | | | (Unicode and raw Unicode).
* One more fprintf bites the dist -- use PySys_WriteStderrGuido van Rossum1998-12-041-1/+1
|
* Replace all calls to fprintf(stderr, ...) with PySys_WriteStderr(...).Guido van Rossum1998-08-251-5/+6
|
* Add checking for inconsistent tab usageGuido van Rossum1998-04-091-3/+43
|
* Swap two statements in the dedent check loop. This makes absolutelyGuido van Rossum1998-02-161-1/+1
| | | | | no difference, but avoids triggering an optimizer bug in the AIX compiler where the loop unrolling does the wrong thing...
* Fixed the bug in searching for triple quotes -- change the 'quote2'Guido van Rossum1998-02-161-2/+2
| | | | | variable from a pointer to an index, so a realloc() of the buffer won't disturb it. Problem found by Vladimir Marangozov.
* tok_nextc() should return unsigned characters, to avoid mistakingGuido van Rossum1998-01-191-2/+10
| | | | '\377' for EOF.
* Another directory quickly renamed.Guido van Rossum1997-04-291-25/+27
|
generated by cgit v0.12 at 2025-12-27 06:49:48 (GMT)