Merged revisions 68521,68527,68534-68536,68540,68547,68552,68563,68570,68572,68575,68579-68580,68584 via svnmerge from

svn+ssh://pythondev@svn.python.org/python/trunk

........
  r68521 | hirokazu.yamamoto | 2009-01-11 04:28:13 +0100 (So, 11 Jan 2009) | 1 line

  Fixed version number in build_ssl.bat.
........
  r68527 | martin.v.loewis | 2009-01-11 10:43:55 +0100 (So, 11 Jan 2009) | 2 lines

  Issue #4895: Use _strdup on Windows CE.
........
  r68534 | gregory.p.smith | 2009-01-11 18:53:33 +0100 (So, 11 Jan 2009) | 2 lines

  correct email address
........
  r68535 | gregory.p.smith | 2009-01-11 18:57:54 +0100 (So, 11 Jan 2009) | 9 lines

  Update the documentation for binascii and zlib crc32/adler32 functions
  to better describe the signed vs unsigned return value behavior on
  different platforms and versions of python.  Mention the workaround to
  make them all return the same thing by using & 0xffffffff.

  Fixes issue4903.

  Also needs to be merged into release26-maint, release30-maint, & py3k.
........
  r68536 | benjamin.peterson | 2009-01-11 20:48:15 +0100 (So, 11 Jan 2009) | 1 line

  add email addresses
........
  r68540 | martin.v.loewis | 2009-01-12 08:57:11 +0100 (Mo, 12 Jan 2009) | 2 lines

  Issue #4915: Port sysmodule to Windows CE.
........
  r68547 | kristjan.jonsson | 2009-01-12 19:09:27 +0100 (Mo, 12 Jan 2009) | 1 line

  Add tests for invalid format specifiers in strftime, and for handling of invalid file descriptors in the os module.
........
  r68552 | vinay.sajip | 2009-01-12 21:36:18 +0100 (Mo, 12 Jan 2009) | 1 line

  Minor changes/corrections in markup.
........
  r68563 | benjamin.peterson | 2009-01-13 02:49:10 +0100 (Di, 13 Jan 2009) | 1 line

  small logic correction
........
  r68570 | raymond.hettinger | 2009-01-13 10:08:32 +0100 (Di, 13 Jan 2009) | 5 lines

  Issue 4922: Incorrect comments for MutableSet.add() and MutableSet.discard().

  Needs to be backported to 2.6 and forward ported to 3.0 and 3.1.
........
  r68572 | andrew.kuchling | 2009-01-13 14:40:54 +0100 (Di, 13 Jan 2009) | 1 line

  Note that first coord. is left alone
........
  r68575 | thomas.heller | 2009-01-13 18:32:28 +0100 (Di, 13 Jan 2009) | 1 line

  Fix refcount leak in error cases.  Bug found by coverity.
........
  r68579 | benjamin.peterson | 2009-01-13 22:42:23 +0100 (Di, 13 Jan 2009) | 1 line

  make bytearrayobject.o depend on the stringlib #4936
........
  r68580 | benjamin.peterson | 2009-01-13 22:43:11 +0100 (Di, 13 Jan 2009) | 1 line

  add bytearrayobject.h to PYTHON_HEADERS
........
  r68584 | benjamin.peterson | 2009-01-13 23:22:41 +0100 (Di, 13 Jan 2009) | 1 line

  de-spacify
........

15 files changed, 165 insertions, 42 deletions

diff --git a/Doc/library/2to3.rst b/Doc/library/2to3.rst
index 40234ef..375bd10 100644
--- a/Doc/library/2to3.rst
+++ b/Doc/library/2to3.rst
@@ -3,7 +3,7 @@
 2to3 - Automated Python 2 to 3 code translation
 ===============================================
 
-.. sectionauthor:: Benjamin Peterson
+.. sectionauthor:: Benjamin Peterson <benjamin@python.org>
 
 2to3 is a Python program that reads Python 2.x source code and applies a series
 of *fixers* to transform it into valid Python 3.x code.  The standard library
diff --git a/Doc/library/binascii.rst b/Doc/library/binascii.rst
index ffea232..ece0819 100644
--- a/Doc/library/binascii.rst
+++ b/Doc/library/binascii.rst
@@ -113,8 +113,25 @@ The :mod:`binascii` module defines the following functions:
       print binascii.crc32("hello world")
       # Or, in two pieces:
       crc = binascii.crc32("hello")
-      crc = binascii.crc32(" world", crc)
-      print crc
+      crc = binascii.crc32(" world", crc) & 0xffffffff
+      print 'crc32 = 0x%08x' % crc
+
+.. note::
+   To generate the same numeric value across all Python versions and
+   platforms use crc32(data) & 0xffffffff.  If you are only using
+   the checksum in packed binary format this is not necessary as the
+   return value will have the correct 32bit binary representation
+   regardless of sign.
+
+.. versionchanged:: 2.6
+   The return value will always be in the range [-2**31, 2**31-1]
+   regardless of platform.  In the past the value would be signed on
+   some platforms and unsigned on others.  Use & 0xffffffff on the
+   value if you want it to match 3.0 behavior.
+
+.. versionchanged:: 3.0
+   The return value will always be unsigned and in the range [0, 2**32-1]
+   regardless of platform.
 
 
 .. function:: b2a_hex(data)
diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst
index f544cea..73e6e4e 100644
--- a/Doc/library/hashlib.rst
+++ b/Doc/library/hashlib.rst
@@ -4,8 +4,8 @@
 
 .. module:: hashlib
    :synopsis: Secure hash and message digest algorithms.
-.. moduleauthor:: Gregory P. Smith <greg@users.sourceforge.net>
-.. sectionauthor:: Gregory P. Smith <greg@users.sourceforge.net>
+.. moduleauthor:: Gregory P. Smith <greg@krypto.org>
+.. sectionauthor:: Gregory P. Smith <greg@krypto.org>
 
 
 .. versionadded:: 2.5
diff --git a/Doc/library/io.rst b/Doc/library/io.rst
index 86407be..9e870b1 100644
--- a/Doc/library/io.rst
+++ b/Doc/library/io.rst
@@ -6,7 +6,7 @@
 .. moduleauthor:: Guido van Rossum <guido@python.org>
 .. moduleauthor:: Mike Verdone <mike.verdone@gmail.com>
 .. moduleauthor:: Mark Russell <mark.russell@zen.co.uk>
-.. sectionauthor:: Benjamin Peterson
+.. sectionauthor:: Benjamin Peterson <benjamin@python.org>
 .. versionadded:: 2.6
 
 The :mod:`io` module provides the Python interfaces to stream handling.  The
diff --git a/Doc/library/logging.rst b/Doc/library/logging.rst
index cd1cbd2..a3b1326 100644
--- a/Doc/library/logging.rst
+++ b/Doc/library/logging.rst
@@ -526,38 +526,37 @@ provided:
 
 #. :class:`FileHandler` instances send error messages to disk files.
 
-.. currentmodule:: logging.handlers
+#. :class:`handlers.BaseRotatingHandler` is the base class for handlers that
+   rotate log files at a certain point. It is not meant to be  instantiated
+   directly. Instead, use :class:`RotatingFileHandler` or
+   :class:`TimedRotatingFileHandler`.
 
-#. :class:`BaseRotatingHandler` is the base class for handlers that rotate log
-   files at a certain point. It is not meant to be  instantiated directly. Instead,
-   use :class:`RotatingFileHandler` or :class:`TimedRotatingFileHandler`.
-
-#. :class:`RotatingFileHandler` instances send error messages to disk files,
+#. :class:`handlers.RotatingFileHandler` instances send error messages to disk files,
    with support for maximum log file sizes and log file rotation.
 
-#. :class:`TimedRotatingFileHandler` instances send error messages to disk files
+#. :class:`handlers.TimedRotatingFileHandler` instances send error messages to disk files
    rotating the log file at certain timed intervals.
 
-#. :class:`SocketHandler` instances send error messages to TCP/IP sockets.
+#. :class:`handlers.SocketHandler` instances send error messages to TCP/IP sockets.
 
-#. :class:`DatagramHandler` instances send error messages to UDP sockets.
+#. :class:`handlers.DatagramHandler` instances send error messages to UDP sockets.
 
-#. :class:`SMTPHandler` instances send error messages to a designated email
+#. :class:`handlers.SMTPHandler` instances send error messages to a designated email
    address.
 
-#. :class:`SysLogHandler` instances send error messages to a Unix syslog daemon,
+#. :class:`handlers.SysLogHandler` instances send error messages to a Unix syslog daemon,
    possibly on a remote machine.
 
-#. :class:`NTEventLogHandler` instances send error messages to a Windows
+#. :class:`handlers.NTEventLogHandler` instances send error messages to a Windows
    NT/2000/XP event log.
 
-#. :class:`MemoryHandler` instances send error messages to a buffer in memory,
+#. :class:`handlers.MemoryHandler` instances send error messages to a buffer in memory,
    which is flushed whenever specific criteria are met.
 
-#. :class:`HTTPHandler` instances send error messages to an HTTP server using
+#. :class:`handlers.HTTPHandler` instances send error messages to an HTTP server using
    either ``GET`` or ``POST`` semantics.
 
-#. :class:`WatchedFileHandler` instances watch the file they are logging to. If
+#. :class:`handlers.WatchedFileHandler` instances watch the file they are logging to. If
 the file changes, it is closed and reopened using the file name. This handler
 is only useful on Unix-like systems; Windows does not support the underlying
 mechanism used.
diff --git a/Doc/library/turtle.rst b/Doc/library/turtle.rst
index 3155d87..fd84597 100644
--- a/Doc/library/turtle.rst
+++ b/Doc/library/turtle.rst
@@ -325,8 +325,7 @@ Turtle motion
 
    :param y: a number (integer or float)
 
-   Set the turtle's first coordinate to *y*, leave second coordinate
-   unchanged.
+   Set the turtle's second coordinate to *y*, leave first coordinate unchanged.
 
    >>> turtle.position()
    (0.00, 40.00)
diff --git a/Doc/library/zlib.rst b/Doc/library/zlib.rst
index 6ce4e66..ca55a5f 100644
--- a/Doc/library/zlib.rst
+++ b/Doc/library/zlib.rst
@@ -31,22 +31,34 @@ The available exception and functions in this module are:
    Exception raised on compression and decompression errors.
 
 
-.. function:: adler32(string[, value])
+.. function:: adler32(data[, value])
 
-   Computes a Adler-32 checksum of *string*.  (An Adler-32 checksum is almost as
+   Computes a Adler-32 checksum of *data*.  (An Adler-32 checksum is almost as
    reliable as a CRC32 but can be computed much more quickly.)  If *value* is
    present, it is used as the starting value of the checksum; otherwise, a fixed
    default value is used.  This allows computing a running checksum over the
-   concatenation of several input strings.  The algorithm is not cryptographically
+   concatenation of several inputs.  The algorithm is not cryptographically
    strong, and should not be used for authentication or digital signatures.  Since
    the algorithm is designed for use as a checksum algorithm, it is not suitable
    for use as a general hash algorithm.
 
    This function always returns an integer object.
 
-   .. versionchanged:: 2.6
-     For consistent cross-platform behavior we always return a signed integer.
-     ie: Results in the (2**31)...(2**32-1) range will be negative.
+.. note::
+   To generate the same numeric value across all Python versions and
+   platforms use adler32(data) & 0xffffffff.  If you are only using
+   the checksum in packed binary format this is not necessary as the
+   return value will have the correct 32bit binary representation
+   regardless of sign.
+
+.. versionchanged:: 2.6
+   The return value will always be in the range [-2**31, 2**31-1]
+   regardless of platform.  In older versions the value would be
+   signed on some platforms and unsigned on others.
+
+.. versionchanged:: 3.0
+   The return value will always be unsigned and in the range [0, 2**32-1]
+   regardless of platform.
 
 
 .. function:: compress(string[, level])
@@ -66,25 +78,37 @@ The available exception and functions in this module are:
    ``9`` is slowest and produces the most.  The default value is ``6``.
 
 
-.. function:: crc32(string[, value])
+.. function:: crc32(data[, value])
 
    .. index::
       single: Cyclic Redundancy Check
       single: checksum; Cyclic Redundancy Check
 
-   Computes a CRC (Cyclic Redundancy Check)  checksum of *string*. If *value* is
+   Computes a CRC (Cyclic Redundancy Check)  checksum of *data*. If *value* is
    present, it is used as the starting value of the checksum; otherwise, a fixed
    default value is used.  This allows computing a running checksum over the
-   concatenation of several input strings.  The algorithm is not cryptographically
+   concatenation of several inputs.  The algorithm is not cryptographically
    strong, and should not be used for authentication or digital signatures.  Since
    the algorithm is designed for use as a checksum algorithm, it is not suitable
    for use as a general hash algorithm.
 
    This function always returns an integer object.
 
-   .. versionchanged:: 2.6
-     For consistent cross-platform behavior we always return a signed integer.
-     ie: Results in the (2**31)...(2**32-1) range will be negative.
+.. note::
+   To generate the same numeric value across all Python versions and
+   platforms use crc32(data) & 0xffffffff.  If you are only using
+   the checksum in packed binary format this is not necessary as the
+   return value will have the correct 32bit binary representation
+   regardless of sign.
+
+.. versionchanged:: 2.6
+   The return value will always be in the range [-2**31, 2**31-1]
+   regardless of platform.  In older versions the value would be
+   signed on some platforms and unsigned on others.
+
+.. versionchanged:: 3.0
+   The return value will always be unsigned and in the range [0, 2**32-1]
+   regardless of platform.
 
 
 .. function:: decompress(string[, wbits[, bufsize]])
diff --git a/Lib/_abcoll.py b/Lib/_abcoll.py
index a5fee08..38b44a5 100644
--- a/Lib/_abcoll.py
+++ b/Lib/_abcoll.py
@@ -249,12 +249,12 @@ class MutableSet(Set):
 
     @abstractmethod
     def add(self, value):
-        """Return True if it was added, False if already there."""
+        """Add an element."""
         raise NotImplementedError
 
     @abstractmethod
     def discard(self, value):
-        """Return True if it was deleted, False if not there."""
+        """Remove an element.  Do not raise an exception if absent."""
         raise NotImplementedError
 
     def remove(self, value):
diff --git a/Lib/test/test_datetime.py b/Lib/test/test_datetime.py
index ff00d8b..89fa5c8 100644
--- a/Lib/test/test_datetime.py
+++ b/Lib/test/test_datetime.py
@@ -856,6 +856,14 @@ class TestDate(HarmlessMixedComparison, unittest.TestCase):
         # A naive object replaces %z and %Z w/ empty strings.
         self.assertEqual(t.strftime("'%z' '%Z'"), "'' ''")
 
+        #make sure that invalid format specifiers are handled correctly
+        self.assertRaises(ValueError, t.strftime, "%e")
+        self.assertRaises(ValueError, t.strftime, "%")
+        self.assertRaises(ValueError, t.strftime, "%#")
+
+        #check that this standard extension works
+        t.strftime("%f")
+
 
     def test_format(self):
         dt = self.theclass(2007, 9, 10)
diff --git a/Lib/test/test_os.py b/Lib/test/test_os.py
index 583df2b..c89a23f 100644
--- a/Lib/test/test_os.py
+++ b/Lib/test/test_os.py
@@ -533,6 +533,55 @@ class Win32ErrorTests(unittest.TestCase):
     def test_chmod(self):
         self.assertRaises(WindowsError, os.utime, test_support.TESTFN, 0)
 
+class TestInvalidFD(unittest.TestCase):
+    singles = ["fchdir", "fdopen", "close", "dup", "fdatasync", "fstat",
+               "fstatvfs", "fsync", "tcgetpgrp", "ttyname"]
+    def get_single(f):
+        def helper(self):
+            if  getattr(os, f, None):
+                self.assertRaises(OSError, getattr(os, f), 10)
+        return helper
+    for f in singles:
+        locals()["test_"+f] = get_single(f)
+
+    def test_isatty(self):
+        self.assertEqual(os.isatty(10), False)
+
+    def test_closerange(self):
+        self.assertEqual(os.closerange(10, 20), None)
+
+    def test_dup2(self):
+        self.assertRaises(OSError, os.dup2, 10, 20)
+
+    def test_fchmod(self):
+        if hasattr(os, "fchmod"):
+            self.assertRaises(OSError, os.fchmod, 10, 0)
+
+    def test_fchown(self):
+        if hasattr(os, "fchown"):
+            self.assertRaises(OSError, os.fchmod, 10, -1, -1)
+
+    def test_fpathconf(self):
+        if hasattr(os, "fpathconf"):
+            self.assertRaises(OSError, os.fpathconf, 10, "foo")
+
+    def test_ftruncate(self):
+        if hasattr(os, "ftruncate"):
+            self.assertRaises(OSError, os.ftruncate, 10, 0)
+
+    def test_lseek(self):
+        self.assertRaises(OSError, os.lseek, 10, 0, 0)
+
+    def test_read(self):
+        self.assertRaises(OSError, os.read, 10, 1)
+
+    def test_tcsetpgrpt(self):
+        if hasattr(os, "tcsetpgrp"):
+            self.assertRaises(OSError, os.tcsetpgrp, 10, 0)
+
+    def test_write(self):
+        self.assertRaises(OSError, os.write, 10, " ")
+
 if sys.platform != 'win32':
     class Win32ErrorTests(unittest.TestCase):
         pass
@@ -547,7 +596,8 @@ def test_main():
         MakedirTests,
         DevNullTests,
         URandomTests,
-        Win32ErrorTests
+        Win32ErrorTests,
+        TestInvalidFD
     )
 
 if __name__ == "__main__":
diff --git a/Makefile.pre.in b/Makefile.pre.in
index 8c37051..37dd77f 100644
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -565,6 +565,9 @@ STRINGLIB_HEADERS= \
 Objects/unicodeobject.o: $(srcdir)/Objects/unicodeobject.c \
 				$(STRINGLIB_HEADERS)
 
+Objects/bytearrayobject.o: $(srcdir)/Objects/bytearrayobject.c \
+				$(STRINGLIB_HEADERS)
+
 Objects/stringobject.o: $(srcdir)/Objects/stringobject.c \
 				$(STRINGLIB_HEADERS)
 
@@ -585,6 +588,7 @@ PYTHON_HEADERS= \
 		Include/ast.h \
 		Include/bitset.h \
 		Include/boolobject.h \
+		Include/bytearrayobject.h \
 		Include/bytes_methods.h \
 		Include/bytesobject.h \
 		Include/bufferobject.h \
diff --git a/Misc/NEWS b/Misc/NEWS
index 0551492..644098e 100644
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -16,6 +16,8 @@ Core and Builtins
   to str, bytes and bytearray could be optimized away by the compiler, letting
   the interpreter segfault instead of raising an error.
 
+- Issue #4915: Port sysmodule to Windows CE.
+
 - Issue #1180193: When importing a module from a .pyc (or .pyo) file with
   an existing .py counterpart, override the co_filename attributes of all
   code objects if the original filename is obsolete (which can happen if the
@@ -276,6 +278,8 @@ Tools/Demos
 Build
 -----
 
+- Issue #4895: Use _strdup on Windows CE.
+
 - Issue #4472: "configure --enable-shared" now works on OSX
 
 - Issues #4728 and #4060: WORDS_BIGEDIAN is now correct in Universal builds.
diff --git a/Modules/_ctypes/cfield.c b/Modules/_ctypes/cfield.c
index ef85b87..96264e7 100644
--- a/Modules/_ctypes/cfield.c
+++ b/Modules/_ctypes/cfield.c
@@ -1452,11 +1452,14 @@ Z_set(void *ptr, PyObject *value, Py_ssize_t size)
 		size += 1; /* terminating NUL */
 		size *= sizeof(wchar_t);
 		buffer = (wchar_t *)PyMem_Malloc(size);
-		if (!buffer)
+		if (!buffer) {
+			Py_DECREF(value);
 			return PyErr_NoMemory();
+		}
 		memset(buffer, 0, size);
 		keep = PyCObject_FromVoidPtr(buffer, PyMem_Free);
 		if (!keep) {
+			Py_DECREF(value);
 			PyMem_Free(buffer);
 			return NULL;
 		}
diff --git a/PC/pyconfig.h b/PC/pyconfig.h
index 93dbc09..112c83b 100644
--- a/PC/pyconfig.h
+++ b/PC/pyconfig.h
@@ -88,6 +88,12 @@ WIN32 is still required for the locale module.
 #define USE_SOCKET
 #endif
 
+/* CE6 doesn't have strdup() but _strdup(). Assume the same for earlier versions. */
+#if defined(MS_WINCE)
+#  include <stdlib.h>
+#  define strdup _strdup
+#endif
+
 #ifdef MS_WINCE
 /* Python uses GetVersion() to distinguish between
  * Windows NT and 9x/ME where OS Unicode support is concerned.
diff --git a/Python/sysmodule.c b/Python/sysmodule.c
index 4bd0e01..ff7157f 100644
--- a/Python/sysmodule.c
+++ b/Python/sysmodule.c
@@ -1296,8 +1296,13 @@ _PySys_Init(void)
 		PyDict_SetItemString(sysdict, key, v);	\
 	Py_XDECREF(v)
 
+	/* Check that stdin is not a directory
+	Using shell redirection, you can redirect stdin to a directory,
+	crashing the Python interpreter. Catch this common mistake here
+	and output a useful error message. Note that under MS Windows,
+	the shell already prevents that. */
+#if !defined(MS_WINDOWS)
 	{
-		/* XXX: does this work on Win/Win64? (see posix_fstat) */
 		struct stat sb;
 		if (fstat(fileno(stdin), &sb) == 0 &&
 		    S_ISDIR(sb.st_mode)) {
@@ -1307,6 +1312,7 @@ _PySys_Init(void)
 			exit(EXIT_FAILURE);
 		}
 	}
+#endif
 
 	/* Closing the standard FILE* if sys.std* goes aways causes problems
 	 * for embedded Python usages. Closing them when somebody explicitly
@@ -1526,7 +1532,7 @@ PySys_SetArgv(int argc, char **argv)
 {
 #if defined(HAVE_REALPATH)
 	char fullpath[MAXPATHLEN];
-#elif defined(MS_WINDOWS)
+#elif defined(MS_WINDOWS) && !defined(MS_WINCE)
 	char fullpath[MAX_PATH];
 #endif
 	PyObject *av = makeargvobject(argc, argv);
@@ -1571,7 +1577,10 @@ PySys_SetArgv(int argc, char **argv)
 #if SEP == '\\' /* Special case for MS filename syntax */
 		if (argc > 0 && argv0 != NULL && strcmp(argv0, "-c") != 0) {
 			char *q;
-#ifdef MS_WINDOWS
+#if defined(MS_WINDOWS) && !defined(MS_WINCE)
+			/* This code here replaces the first element in argv with the full
+			path that it represents. Under CE, there are no relative paths so
+			the argument must be the full path anyway. */
 			char *ptemp;
 			if (GetFullPathName(argv0,
 					   sizeof(fullpath),