diff options
| author | Skip Montanaro <skip@pobox.com> | 2001-02-28 22:05:41 (GMT) |
|---|---|---|
| committer | Skip Montanaro <skip@pobox.com> | 2001-02-28 22:05:41 (GMT) |
| commit | 58177b9975171060ed29248fc4a984a036f579ff (patch) | |
| tree | 4a1dbb566a8fcd7f7d02c1d636862256d14ae67e /Doc/lib/libtokenize.tex | |
| parent | 0bba7f83f2e04652e074acd918c2966c42f77af1 (diff) | |
| download | cpython-58177b9975171060ed29248fc4a984a036f579ff.zip cpython-58177b9975171060ed29248fc4a984a036f579ff.tar.gz cpython-58177b9975171060ed29248fc4a984a036f579ff.tar.bz2 | |
added description of NL token
Diffstat (limited to 'Doc/lib/libtokenize.tex')
| -rw-r--r-- | Doc/lib/libtokenize.tex | 5 |
1 files changed, 4 insertions, 1 deletions
diff --git a/Doc/lib/libtokenize.tex b/Doc/lib/libtokenize.tex index 82d2e68..3c5e8ef 100644 --- a/Doc/lib/libtokenize.tex +++ b/Doc/lib/libtokenize.tex @@ -36,9 +36,12 @@ The scanner is exposed by a single function: All constants from the \refmodule{token} module are also exported from -\module{tokenize}, as is one additional token type value that might be +\module{tokenize}, as are two additional token type values that might be passed to the \var{tokeneater} function by \function{tokenize()}: \begin{datadesc}{COMMENT} Token value used to indicate a comment. \end{datadesc} +\begin{datadesc}{NL} + Token value used to indicate a newline. +\end{datadesc} |