diff options
| author | Guido van Rossum <guido@python.org> | 1993-11-11 10:31:23 (GMT) |
|---|---|---|
| committer | Guido van Rossum <guido@python.org> | 1993-11-11 10:31:23 (GMT) |
| commit | b31c7f732aea6abf6ce24d3da7fd67b2172acec9 (patch) | |
| tree | e97a352f4308578604255065063766f303ee788d /Lib/tokenize.py | |
| parent | 52f2c05401ab13eab45a91d39089866f55ef3c9b (diff) | |
| download | cpython-b31c7f732aea6abf6ce24d3da7fd67b2172acec9.zip cpython-b31c7f732aea6abf6ce24d3da7fd67b2172acec9.tar.gz cpython-b31c7f732aea6abf6ce24d3da7fd67b2172acec9.tar.bz2 | |
* test_select.py: (some) tests for built-in select module
* test_grammar.py, testall.out: added test for funny things in string literals
* token.py, symbol.py: definitions used with built-in parser module.
* tokenize.py: added double-quote recognition
Diffstat (limited to 'Lib/tokenize.py')
| -rw-r--r-- | Lib/tokenize.py | 4 |
1 files changed, 3 insertions, 1 deletions
diff --git a/Lib/tokenize.py b/Lib/tokenize.py index bd047f8..8f16115 100644 --- a/Lib/tokenize.py +++ b/Lib/tokenize.py @@ -24,7 +24,9 @@ Expfloat = '[0-9]+' + Exponent Floatnumber = Pointfloat + '\|' + Expfloat Number = Floatnumber + '\|' + Intnumber -String = '\'\(\\\\.\|[^\\\n\']\)*\'' +String = '\'\(\\\\.\|[^\\\n\']\)*\'' + '\|' + '"\(\\\\.\|[^\\\n"]\)*"' +# Note: this module *recognizes* double quotes, but for backward +# compatibility, it doesn't *use* them! Operator = '~\|\+\|-\|\*\|/\|%\|\^\|&\||\|<<\|>>\|==\|<=\|<>\|!=\|>=\|=\|<\|>' Bracket = '[][(){}]' |