diff options
| author | Guido van Rossum <guido@python.org> | 2019-03-07 20:38:08 (GMT) |
|---|---|---|
| committer | Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com> | 2019-03-07 20:38:08 (GMT) |
| commit | 495da292255b92dd73758fdd0e4c7d27d82b1e57 (patch) | |
| tree | 1378cf049d2d125593fa970ea1e9a9f77604fab1 /Lib/token.py | |
| parent | bf94cc7b496a379e1f604aa2e4080bb70ca4020e (diff) | |
| download | cpython-495da292255b92dd73758fdd0e4c7d27d82b1e57.zip cpython-495da292255b92dd73758fdd0e4c7d27d82b1e57.tar.gz cpython-495da292255b92dd73758fdd0e4c7d27d82b1e57.tar.bz2 | |
bpo-35975: Support parsing earlier minor versions of Python 3 (GH-12086)
This adds a `feature_version` flag to `ast.parse()` (documented) and `compile()` (hidden) that allow tweaking the parser to support older versions of the grammar. In particular if `feature_version` is 5 or 6, the hacks for the `async` and `await` keyword from PEP 492 are reinstated. (For 7 or higher, these are unconditionally treated as keywords, but they are still special tokens rather than `NAME` tokens that the parser driver recognizes.)
https://bugs.python.org/issue35975
Diffstat (limited to 'Lib/token.py')
| -rw-r--r-- | Lib/token.py | 16 |
1 files changed, 9 insertions, 7 deletions
diff --git a/Lib/token.py b/Lib/token.py index 9bf80a5..493bf04 100644 --- a/Lib/token.py +++ b/Lib/token.py @@ -58,14 +58,16 @@ RARROW = 51 ELLIPSIS = 52 COLONEQUAL = 53 OP = 54 -TYPE_IGNORE = 55 -TYPE_COMMENT = 56 +AWAIT = 55 +ASYNC = 56 +TYPE_IGNORE = 57 +TYPE_COMMENT = 58 # These aren't used by the C tokenizer but are needed for tokenize.py -ERRORTOKEN = 57 -COMMENT = 58 -NL = 59 -ENCODING = 60 -N_TOKENS = 61 +ERRORTOKEN = 59 +COMMENT = 60 +NL = 61 +ENCODING = 62 +N_TOKENS = 63 # Special definitions for cooperation with parser NT_OFFSET = 256 |