diff options
| author | Pablo Galindo <Pablogsal@gmail.com> | 2020-04-21 00:53:04 (GMT) |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2020-04-21 00:53:04 (GMT) |
| commit | 11a7f158ef51b0edcde3c3d9215172354e385877 (patch) | |
| tree | 3bb7e125dce1a522ccbbe1ffbd14204c71bc852e /Parser/tokenizer.c | |
| parent | 6a9e80a93148b13e4d3bceaab5ea1804ab0e64d5 (diff) | |
| download | cpython-11a7f158ef51b0edcde3c3d9215172354e385877.zip cpython-11a7f158ef51b0edcde3c3d9215172354e385877.tar.gz cpython-11a7f158ef51b0edcde3c3d9215172354e385877.tar.bz2 | |
bpo-40335: Correctly handle multi-line strings in tokenize error scenarios (GH-19619)
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
Diffstat (limited to 'Parser/tokenizer.c')
| -rw-r--r-- | Parser/tokenizer.c | 7 |
1 files changed, 4 insertions, 3 deletions
diff --git a/Parser/tokenizer.c b/Parser/tokenizer.c index 97986aa..95dfc53 100644 --- a/Parser/tokenizer.c +++ b/Parser/tokenizer.c @@ -1392,13 +1392,14 @@ tok_get(struct tok_state *tok, const char **p_start, const char **p_end) if (nonascii && !verify_identifier(tok)) { return ERRORTOKEN; } + + *p_start = tok->start; + *p_end = tok->cur; + if (c == '"' || c == '\'') { tok->done = E_BADPREFIX; return ERRORTOKEN; } - *p_start = tok->start; - *p_end = tok->cur; - /* async/await parsing block. */ if (tok->cur - tok->start == 5 && tok->start[0] == 'a') { /* May be an 'async' or 'await' token. For Python 3.7 or |