On a sizeof(long)==8 machine, ints in range(2**31, 2**32) were getting

pickled into the signed(!) 4-byte BININT format, so were getting unpickled
again as negative ints.  Repaired that.
Added some minimal docs at the top about what I've learned about the pickle
format codes (little of which was obvious from staring at the code,
although that's partly because all the size-related bugs greatly obscured
the true intent of the code).
Happy side effect:  because save_int() needed to grow a *proper* range
check in order to fix this bug, it can now use the more-efficient BININT1,
BININT2 and BININT formats when the long's value is small enough to fit
in a signed 4-byte int (before this, on a sizeof(long)==8 box it always
used the general INT format for negative ints).
test_cpickle works again on sizeof(long)==8 machines.  test_pickle is
still busted big-time.

1 files changed, 21 insertions, 5 deletions

diff --git a/Modules/cPickle.c b/Modules/cPickle.c
index b87f498..c61035d 100644
--- a/Modules/cPickle.c
+++ b/Modules/cPickle.c
@@ -68,6 +68,20 @@ static char cPickle_module_documentation[] =
 
 #define WRITE_BUF_SIZE 256
 
+/* --------------------------------------------------------------------------
+NOTES on format codes.
+XXX much more is needed here
+
+Integer types
+BININT1		8-bit unsigned integer; followed by 1 byte.
+BININT2         16-bit unsigned integer; followed by 2 bytes, little-endian.
+BININT		32-bit signed integer; followed by 4 bytes, little-endian.
+INT		Integer; natural decimal string conversion, then newline.
+                CAUTION:  INT-reading code can't assume that what follows
+                fits in a Python int, because the size of Python ints varies
+                across platforms.
+LONG		Long (unbounded) integer; repr(i), then newline.
+-------------------------------------------------------------------------- */
 
 #define MARK        '('
 #define STOP        '.'
@@ -904,18 +918,20 @@ save_int(Picklerobject *self, PyObject *args) {
 
     if (!self->bin
 #if SIZEOF_LONG > 4
-        || (l >> 32)
+        || l >  0x7fffffffL
+        || l < -0x80000000L
 #endif
-            ) {
-                /* Save extra-long ints in non-binary mode, so that
-                   we can use python long parsing code to restore,
-                   if necessary. */
+       ) {
+        /* Text-mode pickle, or long too big to fit in the 4-byte
+         * signed BININT format:  store as a string.
+         */
         c_str[0] = INT;
         sprintf(c_str + 1, "%ld\n", l);
         if ((*self->write_func)(self, c_str, strlen(c_str)) < 0)
             return -1;
     }
     else {
+    	/* Binary pickle and l fits in a signed 4-byte int. */
         c_str[1] = (int)( l        & 0xff);
         c_str[2] = (int)((l >> 8)  & 0xff);
         c_str[3] = (int)((l >> 16) & 0xff);