Years in the making.

objimpl.h, pymem.h:  Stop mapping PyMem_{Del, DEL} and PyMem_{Free, FREE}
to PyObject_{Free, FREE} in a release build.  They're aliases for the
system free() now.

_subprocess.c/sp_handle_dealloc():  Since the memory was originally
obtained via PyObject_NEW, it must be released via PyObject_FREE (or
_DEL).

pythonrun.c, tokenizer.c, parsermodule.c:  I lost count of the number of
PyObject vs PyMem mismatches in these -- it's like the specific
function called at each site was picked at random, sometimes even with
memory obtained via PyMem getting released via PyObject.  Changed most
to use PyObject uniformly, since the blobs allocated are predictably
small in most cases, and obmalloc is generally faster than system
mallocs then.

If extension modules in real life prove as sloppy as Python's front
end, we'll have to revert the objimpl.h + pymem.h part of this patch.
Note that no problems will show up in a debug build (all calls still go
thru obmalloc then). Problems will show up only in a release build, most
likely segfaults.

1 files changed, 12 insertions, 1 deletions

diff --git a/Misc/NEWS b/Misc/NEWS
index 1d75424..ae0e971 100644
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -311,7 +311,7 @@ Extension Modules
 
 - Everything under lib-old was removed.  This includes the following modules:
     Para, addpack, cmp, cmpcache, codehack, dircmp, dump, find, fmt, grep,
-    lockfile, newdir, ni, packmail, poly, rand, statcache, tb, tzparse, 
+    lockfile, newdir, ni, packmail, poly, rand, statcache, tb, tzparse,
     util, whatsound, whrandom, zmod
 
 - The following modules were removed:  regsub, reconvert, regex, regex_syntax.
@@ -942,6 +942,17 @@ Build
 C API
 -----
 
+- ``PyMem_{Del, DEL}`` and ``PyMem_{Free, FREE}`` no longer map to
+  ``PyObject_{Free, FREE}``.  They map to the system ``free()`` now.  If memory
+  is obtained via the ``PyObject_`` family, it must be released via the
+  ``PyObject_`` family, and likewise for the ``PyMem_`` family.  This has
+  always been officially true, but when Python's small-object allocator was
+  introduced, an attempt was made to cater to a few extension modules
+  discovered at the time that obtained memory via ``PyObject_New`` but
+  released it via ``PyMem_DEL``.  It's years later, and if such code still
+  exists it will fail now (probably with segfaults, but calling wrong
+  low-level memory management functions can yield many symptoms).
+
 - Added a C API for set and frozenset objects.
 
 - Removed PyRange_New().