cpython.git - https://github.com/python/cpython.git

	Commit message (Collapse)	Author	Age	Files	Lines
*	Issue #12587: Correct faulty test file and reference in test_tokenize.	Ned Deily	2011-07-19	1	-1/+1
\| \| \| \|	(Patch by Robert Xiao)
*	Ran svneol.py	Martin v. Löwis	2008-06-13	1	-13/+13
\|
*	- Issue #719888: Updated tokenize to use a bytes API. generate_tokens has been	Trent Nelson	2008-03-18	1	-0/+13
	renamed tokenize and now works with bytes rather than strings. A new detect_encoding function has been added for determining source file encoding according to PEP-0263. Token sequences returned by tokenize always start with an ENCODING token which specifies the encoding used to decode the file. This token is used to encode the output of untokenize back to bytes. Credit goes to Michael "I'm-going-to-name-my-first-child-unittest" Foord from Resolver Systems for this work.