bpo-39360: Ensure all workers exit when finalizing a multiprocessing Pool (GH-19009)

When the pull is not used via the context manager or terminate() is called, there is a system in multiprocessing.util that handles finalization of all pools via an atexit handler (the Finalize) class. This class registers the _terminate_pool handler in the registry of finalizers of the module, and that registry is called on interpreter exit via _exit_function. The problem is that the "happy" path with the context manager or manual call to finalize() does some extra steps that _terminate_pool does not. The step that is not executed when the atexit() handler calls _terminate_pool is pinging the _change_notifier queue to unblock the maintenance threads.

This commit moves the notification to the _terminate_pool function so is called from both code paths.

Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>

1 files changed, 4 insertions, 0 deletions

diff --git a/Misc/NEWS.d/next/Library/2020-03-15-05-41-05.bpo-39360.cmcU5p.rst b/Misc/NEWS.d/next/Library/2020-03-15-05-41-05.bpo-39360.cmcU5p.rst
new file mode 100644
index 0000000..1488788
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2020-03-15-05-41-05.bpo-39360.cmcU5p.rst
@@ -0,0 +1,4 @@
+Ensure all workers exit when finalizing a :class:`multiprocessing.Pool` implicitly via the module finalization
+handlers of multiprocessing. This fixes a deadlock situation that can be experienced when the Pool is not
+properly finalized via the context manager or a call to ``multiprocessing.Pool.terminate``. Patch by Batuhan Taskaya
+and Pablo Galindo.