bpo-38662: ensurepip invokes pip via runpy (GH-18901)

The ensurepip module now invokes pip via the runpy module.
Hence it is no longer tightly coupled with the internal API of the bundled
pip version, allowing easier updates to a newer pip version both
internally and for distributors.

This way, any changes to the internal pip API won't mean ensurepip needs to be
changed as well. Also, distributors can update their pip wheels independent on
CPython release schedule.

Co-Authored-By: Pradyun Gedam <pradyunsg@gmail.com>
Co-Authored-By: Miro Hrončok <miro@hroncok.cz>

2 files changed, 17 insertions, 3 deletions

diff --git a/Lib/ensurepip/__init__.py b/Lib/ensurepip/__init__.py
index 386ed6c..545fce6 100644
--- a/Lib/ensurepip/__init__.py
+++ b/Lib/ensurepip/__init__.py
@@ -1,6 +1,7 @@
 import os
 import os.path
 import sys
+import runpy
 import tempfile
 from importlib import resources
 
@@ -26,9 +27,18 @@ def _run_pip(args, additional_paths=None):
     if additional_paths is not None:
         sys.path = additional_paths + sys.path
 
-    # Install the bundled software
-    import pip._internal
-    return pip._internal.main(args)
+    # Invoke pip as if it's the main module, and catch the exit.
+    backup_argv = sys.argv[:]
+    sys.argv[1:] = args
+    try:
+        # run_module() alters sys.modules and sys.argv, but restores them at exit
+        runpy.run_module("pip", run_name="__main__", alter_sys=True)
+    except SystemExit as exc:
+        return exc.code
+    finally:
+        sys.argv[:] = backup_argv
+
+    raise SystemError("pip did not exit, this should never happen")
 
 
 def version():
diff --git a/Misc/NEWS.d/next/Library/2020-03-10-15-32-31.bpo-38662.o1DMXj.rst b/Misc/NEWS.d/next/Library/2020-03-10-15-32-31.bpo-38662.o1DMXj.rst
new file mode 100644
index 0000000..241b2a6
--- /dev/null
+++ b/Misc/NEWS.d/next/Library/2020-03-10-15-32-31.bpo-38662.o1DMXj.rst
@@ -0,0 +1,4 @@
+The ``ensurepip`` module now invokes ``pip`` via the ``runpy`` module.
+Hence it is no longer tightly coupled with the internal API of the bundled
+``pip`` version, allowing easier updates to a newer ``pip`` version both
+internally and for distributors.