summaryrefslogtreecommitdiffstats
path: root/Doc
diff options
context:
space:
mode:
authorEzio Melotti <ezio.melotti@gmail.com>2009-12-20 12:24:57 (GMT)
committerEzio Melotti <ezio.melotti@gmail.com>2009-12-20 12:24:57 (GMT)
commita8f6f1e2d63b90549278b1162d264bb279459007 (patch)
tree875477b66eb7659659be382131beaffa54ab5e32 /Doc
parent7e5229c3abd6195a7dbca8a0da30dbd203212205 (diff)
downloadcpython-a8f6f1e2d63b90549278b1162d264bb279459007.zip
cpython-a8f6f1e2d63b90549278b1162d264bb279459007.tar.gz
cpython-a8f6f1e2d63b90549278b1162d264bb279459007.tar.bz2
Fixed capitalization and markup; added imports in the example
Diffstat (limited to 'Doc')
-rw-r--r--Doc/library/tokenize.rst9
1 files changed, 6 insertions, 3 deletions
diff --git a/Doc/library/tokenize.rst b/Doc/library/tokenize.rst
index deb7826..d66b07c 100644
--- a/Doc/library/tokenize.rst
+++ b/Doc/library/tokenize.rst
@@ -94,16 +94,19 @@ function it uses to do this is available:
(as a string) and a list of any lines (not decoded from bytes) it has read
in.
- It detects the encoding from the presence of a utf-8 bom or an encoding
- cookie as specified in pep-0263. If both a bom and a cookie are present,
+ It detects the encoding from the presence of a UTF-8 BOM or an encoding
+ cookie as specified in :pep:`263`. If both a BOM and a cookie are present,
but disagree, a SyntaxError will be raised.
- If no encoding is specified, then the default of 'utf-8' will be returned.
+ If no encoding is specified, then the default of ``'utf-8'`` will be returned.
Example of a script re-writer that transforms float literals into Decimal
objects::
+ from tokenize import tokenize, untokenize, NUMBER, STRING, NAME, OP
+ from io import BytesIO
+
def decistmt(s):
"""Substitute Decimals for floats in a string of statements.