bpo-39877: Fix PyEval_RestoreThread() for daemon threads (GH-18811)

* exit_thread_if_finalizing() does now access directly _PyRuntime
  variable, rather than using tstate->interp->runtime since tstate
  can be a dangling pointer after Py_Finalize() has been called.
* exit_thread_if_finalizing() is now called *before* calling
  take_gil(). _PyRuntime.finalizing is an atomic variable,
  we don't need to hold the GIL to access it.
* Add ensure_tstate_not_null() function to check that tstate is not
  NULL at runtime. Check tstate earlier. take_gil() does not longer
  check if tstate is NULL.

Cleanup:

* PyEval_RestoreThread() no longer saves/restores errno: it's already
  done inside take_gil().
* PyEval_AcquireLock(), PyEval_AcquireThread(),
  PyEval_RestoreThread() and _PyEval_EvalFrameDefault() now check if
  tstate is valid with the new is_tstate_valid() function which uses
  _PyMem_IsPtrFreed().

1 files changed, 5 insertions, 0 deletions

diff --git a/Misc/NEWS.d/next/Core and Builtins/2020-03-06-18-30-00.bpo-39877.bzd1y0.rst b/Misc/NEWS.d/next/Core and Builtins/2020-03-06-18-30-00.bpo-39877.bzd1y0.rst
new file mode 100644
index 0000000..d545813
--- /dev/null
+++ b/Misc/NEWS.d/next/Core and Builtins/2020-03-06-18-30-00.bpo-39877.bzd1y0.rst
@@ -0,0 +1,5 @@
+Fix :c:func:`PyEval_RestoreThread` random crash at exit with daemon threads.
+It now accesses the ``_PyRuntime`` variable directly instead of using
+``tstate->interp->runtime``, since ``tstate`` can be a dangling pointer after
+:c:func:`Py_Finalize` has been called. Moreover, the daemon thread now exits
+before trying to take the GIL.