diff options
author | Ezio Melotti <ezio.melotti@gmail.com> | 2009-12-20 12:24:57 (GMT) |
---|---|---|
committer | Ezio Melotti <ezio.melotti@gmail.com> | 2009-12-20 12:24:57 (GMT) |
commit | a8f6f1e2d63b90549278b1162d264bb279459007 (patch) | |
tree | 875477b66eb7659659be382131beaffa54ab5e32 /Doc | |
parent | 7e5229c3abd6195a7dbca8a0da30dbd203212205 (diff) | |
download | cpython-a8f6f1e2d63b90549278b1162d264bb279459007.zip cpython-a8f6f1e2d63b90549278b1162d264bb279459007.tar.gz cpython-a8f6f1e2d63b90549278b1162d264bb279459007.tar.bz2 |
Fixed capitalization and markup; added imports in the example
Diffstat (limited to 'Doc')
-rw-r--r-- | Doc/library/tokenize.rst | 9 |
1 files changed, 6 insertions, 3 deletions
diff --git a/Doc/library/tokenize.rst b/Doc/library/tokenize.rst index deb7826..d66b07c 100644 --- a/Doc/library/tokenize.rst +++ b/Doc/library/tokenize.rst @@ -94,16 +94,19 @@ function it uses to do this is available: (as a string) and a list of any lines (not decoded from bytes) it has read in. - It detects the encoding from the presence of a utf-8 bom or an encoding - cookie as specified in pep-0263. If both a bom and a cookie are present, + It detects the encoding from the presence of a UTF-8 BOM or an encoding + cookie as specified in :pep:`263`. If both a BOM and a cookie are present, but disagree, a SyntaxError will be raised. - If no encoding is specified, then the default of 'utf-8' will be returned. + If no encoding is specified, then the default of ``'utf-8'`` will be returned. Example of a script re-writer that transforms float literals into Decimal objects:: + from tokenize import tokenize, untokenize, NUMBER, STRING, NAME, OP + from io import BytesIO + def decistmt(s): """Substitute Decimals for floats in a string of statements. |