diff options
author | Raymond Hettinger <rhettinger@users.noreply.github.com> | 2020-04-17 02:54:13 (GMT) |
---|---|---|
committer | GitHub <noreply@github.com> | 2020-04-17 02:54:13 (GMT) |
commit | bf1a81258c0ecc8b52b9dcc53321c066b3ed4a67 (patch) | |
tree | dc8efd742d730ca51fd602cd36c2d6e21901dee4 /Doc/library | |
parent | a75e730075cd25be1143e6183006f3b1d61bb80f (diff) | |
download | cpython-bf1a81258c0ecc8b52b9dcc53321c066b3ed4a67.zip cpython-bf1a81258c0ecc8b52b9dcc53321c066b3ed4a67.tar.gz cpython-bf1a81258c0ecc8b52b9dcc53321c066b3ed4a67.tar.bz2 |
Minor modernization and readability improvement to the tokenizer example (GH-19558)
Diffstat (limited to 'Doc/library')
-rw-r--r-- | Doc/library/re.rst | 8 |
1 files changed, 6 insertions, 2 deletions
diff --git a/Doc/library/re.rst b/Doc/library/re.rst index 7c950bf..9abbd8b 100644 --- a/Doc/library/re.rst +++ b/Doc/library/re.rst @@ -1617,10 +1617,14 @@ The text categories are specified with regular expressions. The technique is to combine those into a single master regular expression and to loop over successive matches:: - import collections + from typing import NamedTuple import re - Token = collections.namedtuple('Token', ['type', 'value', 'line', 'column']) + class Token(NamedTuple): + type: str + value: str + line: int + column: int def tokenize(code): keywords = {'IF', 'THEN', 'ENDIF', 'FOR', 'NEXT', 'GOSUB', 'RETURN'} |