Py_Finalize(): disabled the second call of cyclic gc, and added extensive

comments about why both calls to cyclic gc here can cause problems.

I'll backport to 2.3 maint.  Since the calls were introduced in 2.3,
that will be the end of it.

2 files changed, 37 insertions, 2 deletions

diff --git a/Misc/NEWS b/Misc/NEWS
index 8f1c1a1..a7cefff 100644
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -12,6 +12,16 @@ What's New in Python 2.4 alpha 1?
 Core and builtins
 -----------------
 
+- At Python shutdown time (Py_Finalize()), 2.3 called cyclic garbage
+  collection twice, both before and after tearing down modules.  The
+  call after tearing down modules has been disabled, because too much
+  of Python has been torn down then for __del__ methods and weakref
+  callbacks to execute sanely.  The most common symptom was a sequence
+  of uninformative messages on stderr when Python shut down, produced
+  by threads trying to raise exceptions, but unable to report the nature
+  of their problems because too much of the sys module had already been
+  destroyed.
+
 - Removed FutureWarnings related to hex/oct literals and conversions
   and left shifts.  (Thanks to Kalle Svensson for SF patch 849227.)
   This addresses most of the remaining semantic changes promised by
diff --git a/Python/pythonrun.c b/Python/pythonrun.c
index ed54ab3..eedb562 100644
--- a/Python/pythonrun.c
+++ b/Python/pythonrun.c
@@ -329,15 +329,40 @@ Py_Finalize(void)
 	warnings_module = NULL;
 
 	/* Collect garbage.  This may call finalizers; it's nice to call these
-	   before all modules are destroyed. */
+	 * before all modules are destroyed.
+	 * XXX If a __del__ or weakref callback is triggered here, and tries to
+	 * XXX import a module, bad things can happen, because Python no
+	 * XXX longer believes it's initialized.
+	 * XXX     Fatal Python error: Interpreter not initialized (version mismatch?)
+	 * XXX is easy to provoke that way.  I've also seen, e.g.,
+	 * XXX     Exception exceptions.ImportError: 'No module named sha'
+	 * XXX         in <function callback at 0x008F5718> ignored
+	 * XXX but I'm unclear on exactly how that one happens.  In any case,
+	 * XXX I haven't seen a real-life report of either of these.
+         */
 	PyGC_Collect();
 
 	/* Destroy all modules */
 	PyImport_Cleanup();
 
 	/* Collect final garbage.  This disposes of cycles created by
-	   new-style class definitions, for example. */
+	 * new-style class definitions, for example.
+	 * XXX This is disabled because it caused too many problems.  If
+	 * XXX a __del__ or weakref callback triggers here, Python code has
+	 * XXX a hard time running, because even the sys module has been
+	 * XXX cleared out (sys.stdout is gone, sys.excepthook is gone, etc).
+	 * XXX One symptom is a sequence of information-free messages
+	 * XXX coming from threads (if a __del__ or callback is invoked,
+	 * XXX other threads can execute too, and any exception they encounter
+	 * XXX triggers a comedy of errors as subsystem after subsystem
+	 * XXX fails to find what it *expects* to find in sys to help report
+	 * XXX the exception and consequent unexpected failures).  I've also
+	 * XXX seen segfaults then, after adding print statements to the
+	 * XXX Python code getting called.
+	 */
+#if 0
 	PyGC_Collect();
+#endif
 
 	/* Destroy the database used by _PyImport_{Fixup,Find}Extension */
 	_PyImport_Fini();