Merged revisions 61964-61979 via svnmerge from

svn+ssh://pythondev@svn.python.org/python/trunk

........
  r61964 | benjamin.peterson | 2008-03-27 01:25:33 +0100 (Thu, 27 Mar 2008) | 2 lines

  add commas for introductory clauses
........
  r61965 | christian.heimes | 2008-03-27 02:36:21 +0100 (Thu, 27 Mar 2008) | 1 line

  Hopefully added _fileio module to the Windows build system
........
  r61966 | christian.heimes | 2008-03-27 02:38:47 +0100 (Thu, 27 Mar 2008) | 1 line

  Revert commit accident
........
  r61967 | neal.norwitz | 2008-03-27 04:49:54 +0100 (Thu, 27 Mar 2008) | 3 lines

  Fix bytes so it works on 64-bit platforms.
  (Also remove some #if 0 code that is already handled in _getbytevalue.)
........
  r61968 | neal.norwitz | 2008-03-27 05:40:07 +0100 (Thu, 27 Mar 2008) | 1 line

  Fix memory leaks
........
  r61969 | neal.norwitz | 2008-03-27 05:40:50 +0100 (Thu, 27 Mar 2008) | 3 lines

  Fix warnings about using char as an array subscript.  This is not portable
  since char is signed on some platforms and unsigned on others.
........
  r61970 | neal.norwitz | 2008-03-27 06:02:57 +0100 (Thu, 27 Mar 2008) | 1 line

  Fix test_compiler after adding unicode_literals
........
  r61971 | neal.norwitz | 2008-03-27 06:03:11 +0100 (Thu, 27 Mar 2008) | 1 line

  Fix compiler warnings
........
  r61972 | neal.norwitz | 2008-03-27 07:52:01 +0100 (Thu, 27 Mar 2008) | 1 line

  Pluralss only need one s, not 2 (intss -> ints)
........
  r61973 | christian.heimes | 2008-03-27 10:02:33 +0100 (Thu, 27 Mar 2008) | 1 line

  Quick 'n dirty hack: Increase the magic by 2 to force a rebuild of pyc/pyo files on the build bots
........
  r61974 | eric.smith | 2008-03-27 10:42:35 +0100 (Thu, 27 Mar 2008) | 3 lines

  Added test cases for single quoted strings, both forms of triple quotes,
   and some string concatenations.
  Removed unneeded __future__ print_function import.
........
  r61975 | christian.heimes | 2008-03-27 11:35:52 +0100 (Thu, 27 Mar 2008) | 1 line

  Build bots are working again - removing the hack
........
  r61976 | christian.heimes | 2008-03-27 12:46:37 +0100 (Thu, 27 Mar 2008) | 2 lines

  Fixed tokenize tests
  The tokenize module doesn't understand __future__.unicode_literals yet
........
  r61977 | georg.brandl | 2008-03-27 14:27:31 +0100 (Thu, 27 Mar 2008) | 2 lines

  #2248: return result of QUIT from quit().
........
  r61978 | georg.brandl | 2008-03-27 14:34:59 +0100 (Thu, 27 Mar 2008) | 2 lines

  The bug for which there was a test in outstanding_bugs.py was agreed not to be a bug.
........
  r61979 | amaury.forgeotdarc | 2008-03-28 00:23:54 +0100 (Fri, 28 Mar 2008) | 5 lines

  Issue2495: tokenize.untokenize did not insert space between two consecutive string literals:
  "" "" => """", which is invalid code.

  Will backport
........

1 files changed, 23 insertions, 1 deletions

diff --git a/Lib/test/test_tokenize.py b/Lib/test/test_tokenize.py
index 308158f..371e2b9 100644
--- a/Lib/test/test_tokenize.py
+++ b/Lib/test/test_tokenize.py
@@ -88,7 +88,7 @@ Some error-handling code
 
     >>> roundtrip("try: import somemodule\\n"
     ...           "except ImportError: # comment\\n"
-    ...           "    print 'Can not import' # comment2\\n"
+    ...           "    print('Can not import' # comment2\\n)"
     ...           "else:   print 'Loaded'\\n")
     True
 
@@ -509,6 +509,28 @@ Backslash means line continuation, except for comments
     True
     >>> roundtrip("# Comment \\\\nx = 0")
     True
+
+Two string literals on the same line
+
+    >>> roundtrip("'' ''")
+    True
+
+Test roundtrip on random python modules.
+pass the '-ucompiler' option to process the full directory.
+
+    >>> import random
+    >>> tempdir = os.path.dirname(f) or os.curdir
+    >>> testfiles = glob.glob(os.path.join(tempdir, "test*.py"))
+
+    >>> if not test_support.is_resource_enabled("compiler"):
+    ...     testfiles = random.sample(testfiles, 10)
+    ...
+    >>> for testfile in testfiles:
+    ...     if not roundtrip(open(testfile, 'rb')):
+    ...         print("Roundtrip failed for file %s" % testfile)
+    ...         break
+    ... else: True
+    True
 """
 
 from test import test_support