[3.13] gh-120858: PyDict_Next should not lock the dict (GH-120859) (#120964)

PyDict_Next no longer locks the dictionary in the free-threaded build. Locking
around individual PyDict_Next calls is not sufficient because the function
returns borrowed references and because it allows concurrent modifications
during the iteraiton loop.

The internal locking also interferes with correct external synchronization
because it may suspend outer critical sections created by the caller.
(cherry picked from commit 375b723d5873f948696c7e85a97f4778d9e00ff0)

Co-authored-by: Sam Gross <colesbury@gmail.com>

1 files changed, 19 insertions, 1 deletions

diff --git a/Doc/howto/free-threading-extensions.rst b/Doc/howto/free-threading-extensions.rst
index 080170d..1ba91b0 100644
--- a/Doc/howto/free-threading-extensions.rst
+++ b/Doc/howto/free-threading-extensions.rst
@@ -114,6 +114,24 @@ Containers like :c:struct:`PyListObject`,
 in the free-threaded build.  For example, the :c:func:`PyList_Append` will
 lock the list before appending an item.
 
+.. _PyDict_Next:
+
+``PyDict_Next``
+'''''''''''''''
+
+A notable exception is :c:func:`PyDict_Next`, which does not lock the
+dictionary.  You should use :c:macro:`Py_BEGIN_CRITICAL_SECTION` to protect
+the dictionary while iterating over it if the dictionary may be concurrently
+modified::
+
+    Py_BEGIN_CRITICAL_SECTION(dict);
+    PyObject *key, *value;
+    Py_ssize_t pos = 0;
+    while (PyDict_Next(dict, &pos, &key, &value)) {
+        ...
+    }
+    Py_END_CRITICAL_SECTION();
+
 
 Borrowed References
 ===================
@@ -141,7 +159,7 @@ that return :term:`strong references <strong reference>`.
 +-----------------------------------+-----------------------------------+
 | :c:func:`PyDict_SetDefault`       | :c:func:`PyDict_SetDefaultRef`    |
 +-----------------------------------+-----------------------------------+
-| :c:func:`PyDict_Next`             | no direct replacement             |
+| :c:func:`PyDict_Next`             | none (see :ref:`PyDict_Next`)     |
 +-----------------------------------+-----------------------------------+
 | :c:func:`PyWeakref_GetObject`     | :c:func:`PyWeakref_GetRef`        |
 +-----------------------------------+-----------------------------------+