gh-126925: Modify how iOS test results are gathered (#127592)

Adds a `use_system_log` config item to enable stdout/stderr redirection for
Apple platforms. This log streaming is then used by a new iOS test runner
script, allowing the display of test suite output at runtime. The iOS test
runner script can be used by any Python project, not just the CPython test
suite.

1 files changed, 22 insertions, 32 deletions

diff --git a/iOS/README.rst b/iOS/README.rst
index e33455e..9cea98c 100644
--- a/iOS/README.rst
+++ b/iOS/README.rst
@@ -285,52 +285,42 @@ This will:
 * Install the Python iOS framework into the copy of the testbed project; and
 * Run the test suite on an "iPhone SE (3rd generation)" simulator.
 
-While the test suite is running, Xcode does not display any console output.
-After showing some Xcode build commands, the console output will print ``Testing
-started``, and then appear to stop. It will remain in this state until the test
-suite completes. On a 2022 M1 MacBook Pro, the test suite takes approximately 12
-minutes to run; a couple of extra minutes is required to boot and prepare the
-iOS simulator.
-
 On success, the test suite will exit and report successful completion of the
-test suite. No output of the Python test suite will be displayed.
-
-On failure, the output of the Python test suite *will* be displayed. This will
-show the details of the tests that failed.
+test suite. On a 2022 M1 MacBook Pro, the test suite takes approximately 12
+minutes to run; a couple of extra minutes is required to compile the testbed
+project, and then boot and prepare the iOS simulator.
 
 Debugging test failures
 -----------------------
 
-The easiest way to diagnose a single test failure is to open the testbed project
-in Xcode and run the tests from there using the "Product > Test" menu item.
-
-To test in Xcode, you must ensure the testbed project has a copy of a compiled
-framework. If you've configured your build with the default install location of
-``iOS/Frameworks``, you can copy from that location into the test project. To
-test on an ARM64 simulator, run::
-
-    $ rm -rf iOS/testbed/Python.xcframework/ios-arm64_x86_64-simulator/*
-    $ cp -r iOS/Frameworks/arm64-iphonesimulator/* iOS/testbed/Python.xcframework/ios-arm64_x86_64-simulator
+Running ``make test`` generates a standalone version of the ``iOS/testbed``
+project, and runs the full test suite. It does this using ``iOS/testbed``
+itself - the folder is an executable module that can be used to create and run
+a clone of the testbed project.
 
-To test on an x86-64 simulator, run::
+You can generate your own standalone testbed instance by running::
 
-    $ rm -rf iOS/testbed/Python.xcframework/ios-arm64_x86_64-simulator/*
-    $ cp -r iOS/Frameworks/x86_64-iphonesimulator/* iOS/testbed/Python.xcframework/ios-arm64_x86_64-simulator
+    $ python iOS/testbed clone --framework iOS/Frameworks/arm64-iphonesimulator my-testbed
 
-To test on a physical device::
+This invocation assumes that ``iOS/Frameworks/arm64-iphonesimulator`` is the
+path to the iOS simulator framework for your platform (ARM64 in this case);
+``my-testbed`` is the name of the folder for the new testbed clone.
 
-    $ rm -rf iOS/testbed/Python.xcframework/ios-arm64/*
-    $ cp -r iOS/Frameworks/arm64-iphoneos/* iOS/testbed/Python.xcframework/ios-arm64
+You can then use the ``my-testbed`` folder to run the Python test suite,
+passing in any command line arguments you may require. For example, if you're
+trying to diagnose a failure in the ``os`` module, you might run::
 
-Alternatively, you can configure your build to install directly into the
-testbed project. For a simulator, use::
+    $ python my-testbed run -- test -W test_os
 
-    --enable-framework=$(pwd)/iOS/testbed/Python.xcframework/ios-arm64_x86_64-simulator
+This is the equivalent of running ``python -m test -W test_os`` on a desktop
+Python build. Any arguments after the ``--`` will be passed to testbed as if
+they were arguments to ``python -m`` on a desktop machine.
 
-For a physical device, use::
+You can also open the testbed project in Xcode by running::
 
-    --enable-framework=$(pwd)/iOS/testbed/Python.xcframework/ios-arm64
+    $ open my-testbed/iOSTestbed.xcodeproj
 
+This will allow you to use the full Xcode suite of tools for debugging.
 
 Testing on an iOS device
 ^^^^^^^^^^^^^^^^^^^^^^^^