Close #16742: Fix misuse of memory allocations in PyOS_Readline()

The GIL must be held to call PyMem_Malloc(), whereas PyOS_Readline() releases
the GIL to read input.

The result of the C callback PyOS_ReadlineFunctionPointer must now be a string
allocated by PyMem_RawMalloc() or PyMem_RawRealloc() (or NULL if an error
occurred), instead of a string allocated by PyMem_Malloc() or PyMem_Realloc().

Fixing this issue was required to setup a hook on PyMem_Malloc(), for example
using the tracemalloc module.

PyOS_Readline() copies the result of PyOS_ReadlineFunctionPointer() into a new
buffer allocated by PyMem_Malloc(). So the public API of PyOS_Readline() does
not change.

1 files changed, 6 insertions, 0 deletions

diff --git a/Doc/whatsnew/3.4.rst b/Doc/whatsnew/3.4.rst
index 636d05d..ed4ea17 100644
--- a/Doc/whatsnew/3.4.rst
+++ b/Doc/whatsnew/3.4.rst
@@ -587,3 +587,9 @@ that may require changes to your code.
   attribute in the chain referring to the innermost function. Introspection
   libraries that assumed the previous behaviour was intentional can use
   :func:`inspect.unwrap` to gain equivalent behaviour.
+
+* (C API) The result of the :c:var:`PyOS_ReadlineFunctionPointer` callback must
+  now be a string allocated by :c:func:`PyMem_RawMalloc` or
+  :c:func:`PyMem_RawRealloc`, or *NULL* if an error occurred, instead of a
+  string allocated by :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`.
+