Merged revisions 53623-53858 via svnmerge from

svn+ssh://pythondev@svn.python.org/python/trunk

........
  r53624 | peter.astrand | 2007-02-02 20:06:36 +0100 (Fri, 02 Feb 2007) | 1 line

  We had several if statements checking the value of a fd. This is unsafe, since valid fds might be zero. We should check for not None instead.
........
  r53635 | kurt.kaiser | 2007-02-05 07:03:18 +0100 (Mon, 05 Feb 2007) | 2 lines

  Add 'raw' support to configHandler. Patch 1650174 Tal Einat.
........
  r53641 | kurt.kaiser | 2007-02-06 00:02:16 +0100 (Tue, 06 Feb 2007) | 5 lines

  1. Calltips now 'handle' tuples in the argument list (display '<tuple>' :)
     Suggested solution by Christos Georgiou, Bug 791968.
  2. Clean up tests, were not failing when they should have been.
  4. Remove some camelcase and an unneeded try/except block.
........
  r53644 | kurt.kaiser | 2007-02-06 04:21:40 +0100 (Tue, 06 Feb 2007) | 2 lines

  Clean up ModifiedInterpreter.runcode() structure
........
  r53646 | peter.astrand | 2007-02-06 16:37:50 +0100 (Tue, 06 Feb 2007) | 1 line

  Applied patch 1124861.3.patch to solve bug #1124861: Automatically create pipes on Windows, if GetStdHandle fails. Will backport.
........
  r53648 | lars.gustaebel | 2007-02-06 19:38:13 +0100 (Tue, 06 Feb 2007) | 4 lines

  Patch #1652681: create nonexistent files in append mode and
  allow appending to empty files.
........
  r53649 | kurt.kaiser | 2007-02-06 20:09:43 +0100 (Tue, 06 Feb 2007) | 4 lines

  Updated patch (CodeContext.061217.patch) to
  [ 1362975 ] CodeContext - Improved text indentation
  Tal Einat 16Dec06
........
  r53650 | kurt.kaiser | 2007-02-06 20:21:19 +0100 (Tue, 06 Feb 2007) | 2 lines

  narrow exception per [ 1540849 ] except too broad
........
  r53653 | kurt.kaiser | 2007-02-07 04:39:41 +0100 (Wed, 07 Feb 2007) | 4 lines

  [ 1621265 ] Auto-completion list placement
  Move AC window below input line unless not enough space, then put it above.
  Patch: Tal Einat
........
  r53654 | kurt.kaiser | 2007-02-07 09:07:13 +0100 (Wed, 07 Feb 2007) | 2 lines

  Handle AttributeError during calltip lookup
........
  r53656 | raymond.hettinger | 2007-02-07 21:08:22 +0100 (Wed, 07 Feb 2007) | 3 lines

  SF #1615701:  make d.update(m) honor __getitem__() and keys() in dict subclasses
........
  r53658 | raymond.hettinger | 2007-02-07 22:04:20 +0100 (Wed, 07 Feb 2007) | 1 line

  SF: 1397711 Set docs conflated immutable and hashable
........
  r53660 | raymond.hettinger | 2007-02-07 22:42:17 +0100 (Wed, 07 Feb 2007) | 1 line

  Check for a common user error with defaultdict().
........
  r53662 | raymond.hettinger | 2007-02-07 23:24:07 +0100 (Wed, 07 Feb 2007) | 1 line

  Bug #1575169: operator.isSequenceType() now returns False for subclasses of dict.
........
  r53664 | raymond.hettinger | 2007-02-08 00:49:03 +0100 (Thu, 08 Feb 2007) | 1 line

  Silence compiler warning
........
  r53666 | raymond.hettinger | 2007-02-08 01:07:32 +0100 (Thu, 08 Feb 2007) | 1 line

  Do not let overflows in enumerate() and count() pass silently.
........
  r53668 | raymond.hettinger | 2007-02-08 01:50:39 +0100 (Thu, 08 Feb 2007) | 1 line

  Bypass set specific optimizations for set and frozenset subclasses.
........
  r53670 | raymond.hettinger | 2007-02-08 02:42:35 +0100 (Thu, 08 Feb 2007) | 1 line

  Fix docstring bug
........
  r53671 | martin.v.loewis | 2007-02-08 10:13:36 +0100 (Thu, 08 Feb 2007) | 3 lines

  Bug #1653736: Complain about keyword arguments to time.isoformat.
  Will backport to 2.5.
........
  r53679 | kurt.kaiser | 2007-02-08 23:58:18 +0100 (Thu, 08 Feb 2007) | 6 lines

  Corrected some bugs in AutoComplete.  Also, Page Up/Down in ACW implemented;
  mouse and cursor selection in ACWindow implemented; double Tab inserts current
  selection and closes ACW (similar to double-click and Return); scroll wheel now
  works in ACW.  Added AutoComplete instructions to IDLE Help.
........
  r53689 | martin.v.loewis | 2007-02-09 13:19:32 +0100 (Fri, 09 Feb 2007) | 3 lines

  Bug #1653736: Properly discard third argument to slot_nb_inplace_power.
  Will backport.
........
  r53691 | martin.v.loewis | 2007-02-09 13:36:48 +0100 (Fri, 09 Feb 2007) | 4 lines

  Bug #1600860: Search for shared python library in LIBDIR, not
  lib/python/config, on "linux" and "gnu" systems.
  Will backport.
........
  r53693 | martin.v.loewis | 2007-02-09 13:58:49 +0100 (Fri, 09 Feb 2007) | 2 lines

  Update broken link. Will backport to 2.5.
........
  r53697 | georg.brandl | 2007-02-09 19:48:41 +0100 (Fri, 09 Feb 2007) | 2 lines

  Bug #1656078: typo in in profile docs.
........
  r53731 | brett.cannon | 2007-02-11 06:36:00 +0100 (Sun, 11 Feb 2007) | 3 lines

  Change a very minor inconsistency (that is purely cosmetic) in the AST
  definition.
........
  r53735 | skip.montanaro | 2007-02-11 19:24:37 +0100 (Sun, 11 Feb 2007) | 1 line

  fix trace.py --ignore-dir
........
  r53741 | brett.cannon | 2007-02-11 20:44:41 +0100 (Sun, 11 Feb 2007) | 3 lines

  Check in changed Python-ast.c from a cosmetic change to Python.asdl (in
  r53731).
........
  r53751 | brett.cannon | 2007-02-12 04:51:02 +0100 (Mon, 12 Feb 2007) | 5 lines

  Modify Parser/asdl_c.py so that the __version__ number for Python/Python-ast.c
  is specified at the top of the file.  Also add a note that Python/Python-ast.c
  needs to be committed separately after a change to the AST grammar to capture
  the revision number of the change (which is what __version__ is set to).
........
  r53752 | lars.gustaebel | 2007-02-12 10:25:53 +0100 (Mon, 12 Feb 2007) | 3 lines

  Bug #1656581: Point out that external file objects are supposed to be
  at position 0.
........
  r53754 | martin.v.loewis | 2007-02-12 13:21:10 +0100 (Mon, 12 Feb 2007) | 3 lines

  Patch 1463026: Support default namespace in XMLGenerator.
  Fixes #847665. Will backport.
........
  r53757 | armin.rigo | 2007-02-12 17:23:24 +0100 (Mon, 12 Feb 2007) | 4 lines

  Fix the line to what is my guess at the original author's meaning.
  (The line has no effect anyway, but is present because it's
  customary call the base class __init__).
........
  r53763 | martin.v.loewis | 2007-02-13 09:34:45 +0100 (Tue, 13 Feb 2007) | 3 lines

  Patch #685268: Consider a package's __path__ in imputil.
  Will backport.
........
  r53765 | martin.v.loewis | 2007-02-13 10:49:38 +0100 (Tue, 13 Feb 2007) | 2 lines

  Patch #698833: Support file decryption in zipfile.
........
  r53766 | martin.v.loewis | 2007-02-13 11:10:39 +0100 (Tue, 13 Feb 2007) | 3 lines

  Patch #1517891: Make 'a' create the file if it doesn't exist.
  Fixes #1514451.
........
  r53767 | martin.v.loewis | 2007-02-13 13:08:24 +0100 (Tue, 13 Feb 2007) | 3 lines

  Bug #1658794: Remove extraneous 'this'.
  Will backport to 2.5.
........
  r53769 | martin.v.loewis | 2007-02-13 13:14:19 +0100 (Tue, 13 Feb 2007) | 3 lines

  Patch #1657276: Make NETLINK_DNRTMSG conditional.
  Will backport.
........
  r53771 | lars.gustaebel | 2007-02-13 17:09:24 +0100 (Tue, 13 Feb 2007) | 4 lines

  Patch #1647484: Renamed GzipFile's filename attribute to name. The
  filename attribute is still accessible as a property that emits a
  DeprecationWarning.
........
  r53772 | lars.gustaebel | 2007-02-13 17:24:00 +0100 (Tue, 13 Feb 2007) | 3 lines

  Strip the '.gz' extension from the filename that is written to the
  gzip header.
........
  r53774 | martin.v.loewis | 2007-02-14 11:07:37 +0100 (Wed, 14 Feb 2007) | 2 lines

  Patch #1432399: Add HCI sockets.
........
  r53775 | martin.v.loewis | 2007-02-14 12:30:07 +0100 (Wed, 14 Feb 2007) | 2 lines

  Update 1432399 to removal of _BT_SOCKADDR_MEMB.
........
  r53776 | martin.v.loewis | 2007-02-14 12:30:56 +0100 (Wed, 14 Feb 2007) | 3 lines

  Ignore directory time stamps when considering
  whether to rerun libffi configure.
........
  r53778 | lars.gustaebel | 2007-02-14 15:45:12 +0100 (Wed, 14 Feb 2007) | 4 lines

  A missing binary mode in AppendTest caused failures in Windows
  Buildbot.
........
  r53782 | martin.v.loewis | 2007-02-15 10:51:35 +0100 (Thu, 15 Feb 2007) | 2 lines

  Patch #1397848: add the reasoning behind no-resize-on-shrinkage.
........
  r53783 | georg.brandl | 2007-02-15 11:37:59 +0100 (Thu, 15 Feb 2007) | 2 lines

  Make functools.wraps() docs a bit clearer.
........
  r53785 | georg.brandl | 2007-02-15 12:29:04 +0100 (Thu, 15 Feb 2007) | 2 lines

  Patch #1494140: Add documentation for the new struct.Struct object.
........
  r53787 | georg.brandl | 2007-02-15 12:29:55 +0100 (Thu, 15 Feb 2007) | 2 lines

  Add missing \versionadded.
........
  r53800 | brett.cannon | 2007-02-15 23:54:39 +0100 (Thu, 15 Feb 2007) | 11 lines

  Update the encoding package's search function to use absolute imports when
  calling __import__.  This helps make the expected search locations for encoding
  modules be more explicit.

  One could use an explicit value for __path__ when making the call to __import__
  to force the exact location searched for encodings.  This would give the most
  strict search path possible if one is worried about malicious code being
  imported.  The unfortunate side-effect of that is that if __path__ was modified
  on 'encodings' on purpose in a safe way it would not be picked up in future
  __import__ calls.
........
  r53801 | brett.cannon | 2007-02-16 20:33:01 +0100 (Fri, 16 Feb 2007) | 2 lines

  Make the __import__ call in encodings.__init__ absolute with a level 0 call.
........
  r53809 | vinay.sajip | 2007-02-16 23:36:24 +0100 (Fri, 16 Feb 2007) | 1 line

  Minor fix for currentframe (SF #1652788).
........
  r53818 | raymond.hettinger | 2007-02-19 03:03:19 +0100 (Mon, 19 Feb 2007) | 3 lines

  Extend work on revision 52962:  Eliminate redundant calls to PyObject_Hash().
........
  r53820 | raymond.hettinger | 2007-02-19 05:08:43 +0100 (Mon, 19 Feb 2007) | 1 line

  Add merge() function to heapq.
........
  r53821 | raymond.hettinger | 2007-02-19 06:28:28 +0100 (Mon, 19 Feb 2007) | 1 line

  Add tie-breaker count to preserve sort stability.
........
  r53822 | raymond.hettinger | 2007-02-19 07:59:32 +0100 (Mon, 19 Feb 2007) | 1 line

  Use C heapreplace() instead of slower _siftup() in pure python.
........
  r53823 | raymond.hettinger | 2007-02-19 08:30:21 +0100 (Mon, 19 Feb 2007) | 1 line

  Add test for merge stability
........
  r53824 | raymond.hettinger | 2007-02-19 10:14:10 +0100 (Mon, 19 Feb 2007) | 1 line

  Provide an example of defaultdict with non-zero constant factory function.
........
  r53825 | lars.gustaebel | 2007-02-19 10:54:47 +0100 (Mon, 19 Feb 2007) | 2 lines

  Moved misplaced news item.
........
  r53826 | martin.v.loewis | 2007-02-19 11:55:19 +0100 (Mon, 19 Feb 2007) | 3 lines

  Patch #1490190: posixmodule now includes os.chflags() and os.lchflags()
  functions on platforms where the underlying system calls are available.
........
  r53827 | raymond.hettinger | 2007-02-19 19:15:04 +0100 (Mon, 19 Feb 2007) | 1 line

  Fixup docstrings for merge().
........
  r53829 | raymond.hettinger | 2007-02-19 21:44:04 +0100 (Mon, 19 Feb 2007) | 1 line

  Fixup set/dict interoperability.
........
  r53837 | raymond.hettinger | 2007-02-21 06:20:38 +0100 (Wed, 21 Feb 2007) | 1 line

  Add itertools.izip_longest().
........
  r53838 | raymond.hettinger | 2007-02-21 18:22:05 +0100 (Wed, 21 Feb 2007) | 1 line

  Remove filler struct item and fix leak.
........

1 files changed, 92 insertions, 2 deletions

diff --git a/Lib/zipfile.py b/Lib/zipfile.py
index c6162ff..dc51168 100644
--- a/Lib/zipfile.py
+++ b/Lib/zipfile.py
@@ -296,6 +296,65 @@ class ZipInfo (object):
             extra = extra[ln+4:]
 
 
+class _ZipDecrypter:
+    """Class to handle decryption of files stored within a ZIP archive.
+
+    ZIP supports a password-based form of encryption. Even though known
+    plaintext attacks have been found against it, it is still useful
+    for low-level securicy.
+
+    Usage:
+        zd = _ZipDecrypter(mypwd)
+        plain_char = zd(cypher_char)
+        plain_text = map(zd, cypher_text)
+    """
+
+    def _GenerateCRCTable():
+        """Generate a CRC-32 table.
+
+        ZIP encryption uses the CRC32 one-byte primitive for scrambling some
+        internal keys. We noticed that a direct implementation is faster than
+        relying on binascii.crc32().
+        """
+        poly = 0xedb88320
+        table = [0] * 256
+        for i in range(256):
+            crc = i
+            for j in range(8):
+                if crc & 1:
+                    crc = ((crc >> 1) & 0x7FFFFFFF) ^ poly
+                else:
+                    crc = ((crc >> 1) & 0x7FFFFFFF)
+            table[i] = crc
+        return table
+    crctable = _GenerateCRCTable()
+
+    def _crc32(self, ch, crc):
+        """Compute the CRC32 primitive on one byte."""
+        return ((crc >> 8) & 0xffffff) ^ self.crctable[(crc ^ ord(ch)) & 0xff]
+
+    def __init__(self, pwd):
+        self.key0 = 305419896
+        self.key1 = 591751049
+        self.key2 = 878082192
+        for p in pwd:
+            self._UpdateKeys(p)
+
+    def _UpdateKeys(self, c):
+        self.key0 = self._crc32(c, self.key0)
+        self.key1 = (self.key1 + (self.key0 & 255)) & 4294967295
+        self.key1 = (self.key1 * 134775813 + 1) & 4294967295
+        self.key2 = self._crc32(chr((self.key1 >> 24) & 255), self.key2)
+
+    def __call__(self, c):
+        """Decrypt a single character."""
+        c = ord(c)
+        k = self.key2 | 2
+        c = c ^ (((k * (k^1)) >> 8) & 255)
+        c = chr(c)
+        self._UpdateKeys(c)
+        return c
+
 class ZipFile:
     """ Class with methods to open, read, write, close, list zip files.
 
@@ -330,13 +389,21 @@ class ZipFile:
         self.filelist = []      # List of ZipInfo instances for archive
         self.compression = compression  # Method of compression
         self.mode = key = mode.replace('b', '')[0]
+        self.pwd = None
 
         # Check if we were passed a file-like object
         if isinstance(file, basestring):
             self._filePassed = 0
             self.filename = file
             modeDict = {'r' : 'rb', 'w': 'wb', 'a' : 'r+b'}
-            self.fp = open(file, modeDict[mode])
+            try:
+                self.fp = open(file, modeDict[mode])
+            except IOError:
+                if mode == 'a':
+                    mode = key = 'w'
+                    self.fp = open(file, modeDict[mode])
+                else:
+                    raise
         else:
             self._filePassed = 1
             self.fp = file
@@ -461,7 +528,11 @@ class ZipFile:
         """Return the instance of ZipInfo given 'name'."""
         return self.NameToInfo[name]
 
-    def read(self, name):
+    def setpassword(self, pwd):
+        """Set default password for encrypted files."""
+        self.pwd = pwd
+
+    def read(self, name, pwd=None):
         """Return file bytes (as a string) for name."""
         if self.mode not in ("r", "a"):
             raise RuntimeError, 'read() requires mode "r" or "a"'
@@ -469,6 +540,13 @@ class ZipFile:
             raise RuntimeError, \
                   "Attempt to read ZIP archive that was already closed"
         zinfo = self.getinfo(name)
+        is_encrypted = zinfo.flag_bits & 0x1
+        if is_encrypted:
+            if not pwd:
+                pwd = self.pwd
+            if not pwd:
+                raise RuntimeError, "File %s is encrypted, " \
+                      "password required for extraction" % name
         filepos = self.fp.tell()
 
         self.fp.seek(zinfo.header_offset, 0)
@@ -489,6 +567,18 @@ class ZipFile:
                           zinfo.orig_filename, fname)
 
         bytes = self.fp.read(zinfo.compress_size)
+        # Go with decryption
+        if is_encrypted:
+            zd = _ZipDecrypter(pwd)
+            # The first 12 bytes in the cypher stream is an encryption header
+            #  used to strengthen the algorithm. The first 11 bytes are
+            #  completely random, while the 12th contains the MSB of the CRC,
+            #  and is used to check the correctness of the password.
+            h = map(zd, bytes[0:12])
+            if ord(h[11]) != ((zinfo.CRC>>24)&255):
+                raise RuntimeError, "Bad password for file %s" % name
+            bytes = "".join(map(zd, bytes[12:]))
+        # Go with decompression
         self.fp.seek(filepos, 0)
         if zinfo.compress_type == ZIP_STORED:
             pass