Merged revisions 61538-61540,61556,61559-61560,61563,61565,61571,61575-61576,61580-61582,61586,61591,61593,61595,61605-61606,61613-61616,61618,61621-61623,61625,61627,61631-61634 via svnmerge from

svn+ssh://pythondev@svn.python.org/python/trunk

........
  r61538 | steven.bethard | 2008-03-18 20:03:50 +0100 (Di, 18 Mär 2008) | 1 line

  cell_compare needs to return -2 instead of NULL.
........
  r61539 | steven.bethard | 2008-03-18 20:04:32 +0100 (Di, 18 Mär 2008) | 1 line

  _have_soundcard() is a bad check for winsound.Beep, since you can have a soundcard but have the beep driver disabled. This revision basically disables the beep tests by wrapping them in a try/except. The Right Way To Do It is to come up with a _have_enabled_beep_driver() and use that.
........
  r61540 | gregory.p.smith | 2008-03-18 20:05:32 +0100 (Di, 18 Mär 2008) | 8 lines

  Fix chown on 64-bit linux.  It needed to take a long (64-bit on 64bit linux) as
  uid and gid input to accept values >=2**31 as valid while still accepting
  negative numbers to pass -1 to chown for "no change".

  Fixes issue1747858.

  This should be backported to release25-maint.
........
  r61556 | steven.bethard | 2008-03-18 20:59:14 +0100 (Di, 18 Mär 2008) | 1 line

  Fix test_atexit so that it still passes when -3 is supplied. (It was catching the warning messages on stdio from using the reload() function.)
........
  r61559 | neal.norwitz | 2008-03-18 21:30:38 +0100 (Di, 18 Mär 2008) | 1 line

  Import the test properly.  This is especially important for py3k.
........
  r61560 | gregory.p.smith | 2008-03-18 21:40:01 +0100 (Di, 18 Mär 2008) | 2 lines

  news entry for the chown fix
........
  r61563 | brett.cannon | 2008-03-18 22:12:42 +0100 (Di, 18 Mär 2008) | 2 lines

  Ignore BIG5HKSCS-2004.TXT which is downloaded as part of a test.
........
  r61565 | steven.bethard | 2008-03-18 22:30:13 +0100 (Di, 18 Mär 2008) | 1 line

  Have regrtest skip test_py3kwarn when the -3 flag is missing.
........
  r61571 | gregory.p.smith | 2008-03-18 23:27:41 +0100 (Di, 18 Mär 2008) | 4 lines

  Add a test to make sure zlib.crc32 and binascii.crc32 return the same thing.
  Fix a buglet in binascii.crc32, the second optional argument could previously
  have a signedness mismatch with the C variable its going into.
........
  r61575 | raymond.hettinger | 2008-03-19 00:22:29 +0100 (Mi, 19 Mär 2008) | 1 line

  Speed-up isinstance() for one easy case.
........
  r61576 | raymond.hettinger | 2008-03-19 00:33:08 +0100 (Mi, 19 Mär 2008) | 1 line

  Issue: 2354: Add 3K warning for the cmp argument to list.sort() and sorted().
........
  r61580 | andrew.kuchling | 2008-03-19 02:05:35 +0100 (Mi, 19 Mär 2008) | 1 line

  Add Jeff Rush
........
  r61581 | gregory.p.smith | 2008-03-19 02:38:35 +0100 (Mi, 19 Mär 2008) | 3 lines

  Mention that crc32 and adler32 are available in a different module (zlib).
  Some people look for them in hashlib.
........
  r61582 | gregory.p.smith | 2008-03-19 02:46:10 +0100 (Mi, 19 Mär 2008) | 3 lines

  Use zlib's crc32 routine instead of binascii when available.  zlib's is faster
  when compiled properly optimized and about the same speed otherwise.
........
  r61586 | david.wolever | 2008-03-19 03:26:57 +0100 (Mi, 19 Mär 2008) | 1 line

  Added my name to ACKS
........
  r61591 | gregory.p.smith | 2008-03-19 04:14:41 +0100 (Mi, 19 Mär 2008) | 5 lines

  Fix the struct module DeprecationWarnings that zipfile was triggering by
  removing all use of signed struct values.

  test_zipfile and test_zipfile64 pass.  no more warnings.
........
  r61593 | raymond.hettinger | 2008-03-19 04:56:59 +0100 (Mi, 19 Mär 2008) | 1 line

  Fix compiler warning.
........
  r61595 | martin.v.loewis | 2008-03-19 05:39:13 +0100 (Mi, 19 Mär 2008) | 2 lines

  Issue #2400: Allow relative imports to "import *".
........
  r61605 | martin.v.loewis | 2008-03-19 07:00:28 +0100 (Mi, 19 Mär 2008) | 2 lines

  Import relimport using a relative import.
........
  r61606 | trent.nelson | 2008-03-19 07:28:24 +0100 (Mi, 19 Mär 2008) | 1 line

  Issue2290: Support x64 Windows builds that live in pcbuild/amd64.  Without it, sysutils._python_build() returns the wrong directory, which causes the test_get_config_h_filename method in Lib/distutils/tests/test_sysconfig.py to fail.
........
  r61613 | trent.nelson | 2008-03-19 08:45:19 +0100 (Mi, 19 Mär 2008) | 3 lines

  Refine the Visual Studio 2008 build solution in order to improve how we deal with external components, as well as fixing outstanding issues with Windows x64 build support.  Introduce two new .vcproj files, _bsddb44.vcproj and sqlite3.vcproj, which replace the previous pre-link event scripts for _bsddb and _sqlite3 respectively.  The new project files inherit from our property files as if they were any other Python module.  This has numerous benefits.  First, the components get built with exactly the same compiler flags and settings as the rest of Python.  Second, it makes it much easier to debug problems in the external components when they're part of the build system.  Third, they'll benefit from profile guided optimisation in the release builds, just like the rest of Python core.

  I've also introduced a slightly new pattern for managing externals in subversion.  New components get checked in as <name>-<version>.x, where <version> matches the exact vendor version string.  After the initial import of the external component, the .x is tagged as .0 (i.e. tcl-8.4.18.x -> tcl-8.4.18.0).  Some components may not need any tweaking, whereas there are others that might (tcl/tk fall into this bucket).  In that case, the relevant modifications are made to the .x branch, which will be subsequently tagged as .1 (and then n+1 going forward) when they build successfully and all tests pass.  Buildbots will be converted to rely on these explicit tags only, which makes it easy for us to switch them over to a new version as and when required.  (Simple change to external(-amd64).bat: if we've bumped tcl to 8.4.18.1, change the .bat to rmdir 8.4.18.0 if it exists and check out a new .1 copy.)
........
  r61614 | trent.nelson | 2008-03-19 08:56:39 +0100 (Mi, 19 Mär 2008) | 1 line

  Remove extraneous apostrophe and semi-colon from AdditionalIncludeDirectories.
........
  r61615 | georg.brandl | 2008-03-19 08:56:40 +0100 (Mi, 19 Mär 2008) | 2 lines

  Remove footnote from versionchanged as it upsets LaTeX.
........
  r61616 | georg.brandl | 2008-03-19 08:57:57 +0100 (Mi, 19 Mär 2008) | 2 lines

  Another one.
........
  r61618 | trent.nelson | 2008-03-19 09:06:03 +0100 (Mi, 19 Mär 2008) | 1 line

  Fix the tcl-8.4.18.1 path and make sure we cd into the right directory when building tcl/tk.
........
  r61621 | trent.nelson | 2008-03-19 10:23:08 +0100 (Mi, 19 Mär 2008) | 1 line

  Lets have another try at getting the Windows buildbots in a consistent state before rebuilding using the new process.
........
  r61622 | eric.smith | 2008-03-19 13:09:55 +0100 (Mi, 19 Mär 2008) | 2 lines

  Use test.test_support.captured_stdout instead of a custom contextmanager.
  Thanks Nick Coghlan.
........
  r61623 | eric.smith | 2008-03-19 13:15:10 +0100 (Mi, 19 Mär 2008) | 1 line

  Trivial typo.
........
  r61625 | thomas.heller | 2008-03-19 17:10:57 +0100 (Mi, 19 Mär 2008) | 2 lines

  Checkout sqlite-source when it is not there.
........
  r61627 | brett.cannon | 2008-03-19 17:50:13 +0100 (Mi, 19 Mär 2008) | 5 lines

  test_nis would fail if test.test_support.verbose was true but NIS was not set
  up on the machine.

  Closes issue2411. Thanks Michael Bishop.
........
  r61631 | brett.cannon | 2008-03-19 18:37:43 +0100 (Mi, 19 Mär 2008) | 2 lines

  Use sys.py3kwarning instead of trying to trigger a Py3k-related warning.
........
  r61632 | raymond.hettinger | 2008-03-19 18:45:19 +0100 (Mi, 19 Mär 2008) | 1 line

  Issue 2354: Fix-up compare warning.  Patch contributed by Jeff Balogh.
........
  r61633 | raymond.hettinger | 2008-03-19 18:58:59 +0100 (Mi, 19 Mär 2008) | 1 line

  The filter() function does support a None argument in Py3.0.
........
  r61634 | raymond.hettinger | 2008-03-19 19:01:58 +0100 (Mi, 19 Mär 2008) | 1 line

  Remove itertools warnings I had added before the 2-to-3 handled the migration.
........

1 files changed, 40 insertions, 28 deletions

diff --git a/Lib/zipfile.py b/Lib/zipfile.py
index 2865c0a..719ed44 100644
--- a/Lib/zipfile.py
+++ b/Lib/zipfile.py
@@ -8,8 +8,10 @@ import binascii, io
 
 try:
     import zlib # We may need its compression method
+    crc32 = zlib.crc32
 except ImportError:
     zlib = None
+    crc32 = binascii.crc32
 
 __all__ = ["BadZipfile", "error", "ZIP_STORED", "ZIP_DEFLATED", "is_zipfile",
            "ZipInfo", "ZipFile", "PyZipFile", "LargeZipFile" ]
@@ -36,13 +38,13 @@ ZIP_DEFLATED = 8
 # Here are some struct module formats for reading headers
 structEndArchive = "<4s4H2LH"     # 9 items, end of archive, 22 bytes
 stringEndArchive = b"PK\005\006"   # magic number for end of archive record
-structCentralDir = "<4s4B4HlLL5HLL"# 19 items, central directory, 46 bytes
+structCentralDir = "<4s4B4HLLL5HLL"# 19 items, central directory, 46 bytes
 stringCentralDir = b"PK\001\002"   # magic number for central directory
-structFileHeader = "<4s2B4HlLL2H"  # 12 items, file header record, 30 bytes
+structFileHeader = "<4s2B4HLLL2H"  # 12 items, file header record, 30 bytes
 stringFileHeader = b"PK\003\004"   # magic number for file header
-structEndArchive64Locator = "<4slql" # 4 items, locate Zip64 header, 20 bytes
+structEndArchive64Locator = "<4sLQL" # 4 items, locate Zip64 header, 20 bytes
 stringEndArchive64Locator = b"PK\x06\x07" # magic token for locator header
-structEndArchive64 = "<4sqhhllqqqq" # 10 items, end of archive (Zip64), 56 bytes
+structEndArchive64 = "<4sQHHLLQQQQ" # 10 items, end of archive (Zip64), 56 bytes
 stringEndArchive64 = b"PK\x06\x06" # magic token for Zip64 header
 
 
@@ -140,7 +142,7 @@ def _EndRecData(fpin):
         endrec = list(endrec)
         endrec.append("")               # Append the archive comment
         endrec.append(filesize - 22)    # Append the record start offset
-        if endrec[-4] == -1 or endrec[-4] == 0xffffffff:
+        if endrec[-4] == 0xffffffff:
             return _EndRecData64(fpin, -22, endrec)
         return endrec
     # Search the last END_BLOCK bytes of the file for the record signature.
@@ -160,7 +162,7 @@ def _EndRecData(fpin):
             # Append the archive comment and start offset
             endrec.append(comment)
             endrec.append(filesize - END_BLOCK + start)
-            if endrec[-4] == -1 or endrec[-4] == 0xffffffff:
+            if endrec[-4] == 0xffffffff:
                 return _EndRecData64(fpin, - END_BLOCK + start, endrec)
             return endrec
     return      # Error, return None
@@ -247,7 +249,7 @@ class ZipInfo (object):
         if file_size > ZIP64_LIMIT or compress_size > ZIP64_LIMIT:
             # File is larger than what fits into a 4 byte integer,
             # fall back to the ZIP64 extension
-            fmt = '<hhqq'
+            fmt = '<HHQQ'
             extra = extra + struct.pack(fmt,
                     1, struct.calcsize(fmt)-4, file_size, compress_size)
             file_size = 0xffffffff # -1
@@ -267,14 +269,14 @@ class ZipInfo (object):
         extra = self.extra
         unpack = struct.unpack
         while extra:
-            tp, ln = unpack('<hh', extra[:4])
+            tp, ln = unpack('<HH', extra[:4])
             if tp == 1:
                 if ln >= 24:
-                    counts = unpack('<qqq', extra[4:28])
+                    counts = unpack('<QQQ', extra[4:28])
                 elif ln == 16:
-                    counts = unpack('<qq', extra[4:20])
+                    counts = unpack('<QQ', extra[4:20])
                 elif ln == 8:
-                    counts = unpack('<q', extra[4:12])
+                    counts = unpack('<Q', extra[4:12])
                 elif ln == 0:
                     counts = ()
                 else:
@@ -283,7 +285,8 @@ class ZipInfo (object):
                 idx = 0
 
                 # ZIP64 extension (large files and/or large archives)
-                if self.file_size == -1 or self.file_size == 0xFFFFFFFF:
+                # XXX Is this correct? won't this exclude 2**32-1 byte files?
+                if self.file_size in (0xffffffffffffffff, 0xffffffff):
                     self.file_size = counts[idx]
                     idx += 1
 
@@ -950,7 +953,7 @@ class ZipFile:
             if not buf:
                 break
             file_size = file_size + len(buf)
-            CRC = binascii.crc32(buf, CRC)
+            CRC = crc32(buf, CRC) & 0xffffffff
             if cmpr:
                 buf = cmpr.compress(buf)
                 compress_size = compress_size + len(buf)
@@ -968,7 +971,7 @@ class ZipFile:
         # Seek backwards and write CRC and file sizes
         position = self.fp.tell()       # Preserve current position in file
         self.fp.seek(zinfo.header_offset + 14, 0)
-        self.fp.write(struct.pack("<lLL", zinfo.CRC, zinfo.compress_size,
+        self.fp.write(struct.pack("<LLL", zinfo.CRC, zinfo.compress_size,
               zinfo.file_size))
         self.fp.seek(position, 0)
         self.filelist.append(zinfo)
@@ -997,7 +1000,7 @@ class ZipFile:
         zinfo.header_offset = self.fp.tell()    # Start of header data
         self._writecheck(zinfo)
         self._didModify = True
-        zinfo.CRC = binascii.crc32(data)       # CRC-32 checksum
+        zinfo.CRC = crc32(data) & 0xffffffff       # CRC-32 checksum
         if zinfo.compress_type == ZIP_DEFLATED:
             co = zlib.compressobj(zlib.Z_DEFAULT_COMPRESSION,
                  zlib.DEFLATED, -15)
@@ -1047,7 +1050,7 @@ class ZipFile:
 
                 if zinfo.header_offset > ZIP64_LIMIT:
                     extra.append(zinfo.header_offset)
-                    header_offset = -1  # struct "l" format:  32 one bits
+                    header_offset = 0xffffffff  # -1 32 bit
                 else:
                     header_offset = zinfo.header_offset
 
@@ -1055,7 +1058,7 @@ class ZipFile:
                 if extra:
                     # Append a ZIP64 field to the extra's
                     extra_data = struct.pack(
-                            '<hh' + 'q'*len(extra),
+                            '<HH' + 'Q'*len(extra),
                             1, 8*len(extra), *extra) + extra_data
 
                     extract_version = max(45, zinfo.extract_version)
@@ -1064,14 +1067,25 @@ class ZipFile:
                     extract_version = zinfo.extract_version
                     create_version = zinfo.create_version
 
-                centdir = struct.pack(structCentralDir,
-                  stringCentralDir, create_version,
-                  zinfo.create_system, extract_version, zinfo.reserved,
-                  zinfo.flag_bits, zinfo.compress_type, dostime, dosdate,
-                  zinfo.CRC, compress_size, file_size,
-                  len(zinfo.filename), len(extra_data), len(zinfo.comment),
-                  0, zinfo.internal_attr, zinfo.external_attr,
-                  header_offset)
+                try:
+                    centdir = struct.pack(structCentralDir,
+                     stringCentralDir, create_version,
+                     zinfo.create_system, extract_version, zinfo.reserved,
+                     zinfo.flag_bits, zinfo.compress_type, dostime, dosdate,
+                     zinfo.CRC, compress_size, file_size,
+                     len(zinfo.filename), len(extra_data), len(zinfo.comment),
+                     0, zinfo.internal_attr, zinfo.external_attr,
+                     header_offset)
+                except DeprecationWarning:
+                    print >>sys.stderr, (structCentralDir,
+                     stringCentralDir, create_version,
+                     zinfo.create_system, extract_version, zinfo.reserved,
+                     zinfo.flag_bits, zinfo.compress_type, dostime, dosdate,
+                     zinfo.CRC, compress_size, file_size,
+                     len(zinfo.filename), len(extra_data), len(zinfo.comment),
+                     0, zinfo.internal_attr, zinfo.external_attr,
+                     header_offset)
+                    raise
                 self.fp.write(centdir)
                 self.fp.write(zinfo.filename.encode("utf-8"))
                 self.fp.write(extra_data)
@@ -1091,10 +1105,8 @@ class ZipFile:
                         stringEndArchive64Locator, 0, pos2, 1)
                 self.fp.write(zip64locrec)
 
-                # XXX Why is `pos3` computed next?  It's never referenced.
-                pos3 = self.fp.tell()
                 endrec = struct.pack(structEndArchive, stringEndArchive,
-                            0, 0, count, count, pos2 - pos1, -1, 0)
+                            0, 0, count, count, pos2 - pos1, 0xffffffff, 0)
                 self.fp.write(endrec)
 
             else: