summaryrefslogtreecommitdiffstats
path: root/Lib/tokenize.py
Commit message (Expand)AuthorAgeFilesLines
* bpo-30455: Generate all token related code and docs from Grammar/Tokens. (GH-...Serhiy Storchaka2018-12-221-60/+6
* bpo-33899: Make tokenize module mirror end-of-file is end-of-line behavior (G...Ammar Askar2018-07-061-0/+10
* bpo-12486: Document tokenize.generate_tokens() as public API (#6957)Thomas Kluyver2018-06-051-3/+6
* bpo-33338: [tokenize] Minor code cleanup (#6573)Ɓukasz Langa2018-04-231-11/+8
* bpo-33260: Regenerate token.py after removing ASYNC and AWAIT. (GH-6447)Serhiy Storchaka2018-04-111-1/+1
* bpo-30406: Make async and await proper keywords (#1669)Jelle Zijlstra2017-10-061-61/+1
* bpo-25324: copy tok_name before changing it (#1608)Albert-Jan Nijburg2017-05-311-9/+2
* bpo-30377: Simplify handling of COMMENT and NL in tokenize.py (#1607)Albert-Jan Nijburg2017-05-241-5/+3
* bpo-30296 Remove unnecessary tuples, lists, sets, and dicts (#1489)Jon Dufresne2017-05-181-1/+1
* Add ELLIPSIS and RARROW. Add tests (#666)Jim Fasarakis-Hilliard2017-03-141-1/+3
* Issue #26331: Implement the parsing part of PEP 515.Brett Cannon2016-09-091-8/+9
* Issue #26581: Use the first coding cookie on a line, not the last one.Serhiy Storchaka2016-03-201-1/+1
|\
| * Issue #26581: Use the first coding cookie on a line, not the last one.Serhiy Storchaka2016-03-201-1/+1
* | Issue #25977: Fix typos in Lib/tokenize.pyBerker Peksag2015-12-291-5/+5
|\ \ | |/
| * Issue #25977: Fix typos in Lib/tokenize.pyBerker Peksag2015-12-291-4/+4
* | Issue 25311: Add support for f-strings to tokenize.py. Also added some commen...Eric V. Smith2015-10-261-51/+67
|/
* Issue #24619: Simplify async/await tokenization.Yury Selivanov2015-07-231-16/+23
* Issue #24619: New approach for tokenizing async/await.Yury Selivanov2015-07-221-1/+6
* Issue #20387: Merge test and patch from 3.4.4Jason R. Coombs2015-06-281-0/+17
|\
| * Issue #20387: Restore retention of indentation during untokenize.Dingyuan Wang2015-06-221-0/+17
* | (Merge 3.5) Issue #23840: tokenize.open() now closes the temporary binary fileVictor Stinner2015-05-251-5/+9
|\ \ | |/
| * Issue #23840: tokenize.open() now closes the temporary binary file on error toVictor Stinner2015-05-251-5/+9
* | PEP 0492 -- Coroutines with async and await syntax. Issue #24017.Yury Selivanov2015-05-121-2/+54
* | Issue #23615: Modules bz2, tarfile and tokenize now can be reloaded withSerhiy Storchaka2015-03-111-2/+1
|\ \ | |/
| * Issue #23615: Modules bz2, tarfile and tokenize now can be reloaded withSerhiy Storchaka2015-03-111-2/+1
* | Removed duplicated dict entries.Serhiy Storchaka2015-01-111-1/+0
* | (Merge 3.4) Issue #22599: Enhance tokenize.open() to be able to call it duringVictor Stinner2014-12-051-3/+4
|\ \ | |/
| * Issue #22599: Enhance tokenize.open() to be able to call it during PythonVictor Stinner2014-12-051-3/+4
* | PEP 465: a dedicated infix operator for matrix multiplication (closes #21176)Benjamin Peterson2014-04-101-2/+3
|/
* Merge with 3.3Terry Jan Reedy2014-02-241-1/+1
|\
| * whitespaceTerry Jan Reedy2014-02-241-1/+1
* | Merge with 3.3Terry Jan Reedy2014-02-241-0/+6
|\ \ | |/
| * Issue #9974: When untokenizing, use row info to insert backslash+newline.Terry Jan Reedy2014-02-241-0/+6
* | Merge with 3.3Terry Jan Reedy2014-02-181-13/+11
|\ \ | |/
| * Issue #8478: Untokenizer.compat now processes first token from iterator input.Terry Jan Reedy2014-02-181-13/+11
* | Untokenize, bad assert: Merge with 3.3Terry Jan Reedy2014-02-171-1/+3
|\ \ | |/
| * Untokenize: An logically incorrect assert tested user input validity.Terry Jan Reedy2014-02-171-1/+3
* | Issue #18960: Fix bugs with Python source code encoding in the second line.Serhiy Storchaka2014-01-091-0/+3
|\ \ | |/
| * Issue #18960: Fix bugs with Python source code encoding in the second line.Serhiy Storchaka2014-01-091-0/+3
* | #19620: merge with 3.3.Ezio Melotti2013-11-251-1/+1
|\ \ | |/
| * #19620: Fix typo in docstring (noticed by Christopher Welborn).Ezio Melotti2013-11-251-1/+1
* | Issue #18873: The tokenize module, IDLE, 2to3, and the findnocoding.py scriptSerhiy Storchaka2013-09-161-4/+4
|\ \ | |/
| * Issue #18873: The tokenize module, IDLE, 2to3, and the findnocoding.py scriptSerhiy Storchaka2013-09-161-4/+4
* | Replace IOError with OSError (#16715)Andrew Svetlov2012-12-251-1/+1
|/
* #16152: merge with 3.2.Ezio Melotti2012-11-031-1/+3
|\
| * #16152: fix tokenize to ignore whitespace at the end of the code when no newl...Ezio Melotti2012-11-031-1/+3
* | Merge branchFlorent Xicluna2012-07-071-1/+1
|\ \ | |/
| * Issue #14990: tokenize: correctly fail with SyntaxError on invalid encoding d...Florent Xicluna2012-07-071-1/+1
* | Issue #15096: Drop support for the ur string prefixChristian Heimes2012-06-201-9/+3
* | Issue #15054: Fix incorrect tokenization of 'b' string literals.Meador Inge2012-06-171-1/+1