diff options
author | Victor Stinner <vstinner@python.org> | 2023-10-01 01:14:57 (GMT) |
---|---|---|
committer | GitHub <noreply@github.com> | 2023-10-01 01:14:57 (GMT) |
commit | a46e96076898d126c9f276aef1934195aac34b4e (patch) | |
tree | beaec25a75f381f77b74bf4e164eb813c20ec81f /Doc | |
parent | 53eb9a676f8c59b206dfc536b7590f6563ad65e0 (diff) | |
download | cpython-a46e96076898d126c9f276aef1934195aac34b4e.zip cpython-a46e96076898d126c9f276aef1934195aac34b4e.tar.gz cpython-a46e96076898d126c9f276aef1934195aac34b4e.tar.bz2 |
gh-109649: Use os.process_cpu_count() (#110165)
Replace os.cpu_count() with os.process_cpu_count() in modules:
* compileall
* concurrent.futures
* multiprocessing
Replace os.cpu_count() with os.process_cpu_count() in programs:
* _decimal deccheck.py test
* freeze.py
* multissltests.py
* python -m test (regrtest)
* wasm_build.py
Other changes:
* test.pythoninfo logs os.process_cpu_count().
* regrtest gets os.process_cpu_count() / os.cpu_count() in headers.
Diffstat (limited to 'Doc')
-rw-r--r-- | Doc/library/compileall.rst | 2 | ||||
-rw-r--r-- | Doc/library/concurrent.futures.rst | 10 | ||||
-rw-r--r-- | Doc/library/multiprocessing.rst | 12 | ||||
-rw-r--r-- | Doc/whatsnew/3.13.rst | 7 |
4 files changed, 25 insertions, 6 deletions
diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst index a7455ae..b4723b9 100644 --- a/Doc/library/compileall.rst +++ b/Doc/library/compileall.rst @@ -90,7 +90,7 @@ compile Python sources. .. cmdoption:: -j N Use *N* workers to compile the files within the given directory. - If ``0`` is used, then the result of :func:`os.cpu_count()` + If ``0`` is used, then the result of :func:`os.process_cpu_count()` will be used. .. cmdoption:: --invalidation-mode [timestamp|checked-hash|unchecked-hash] diff --git a/Doc/library/concurrent.futures.rst b/Doc/library/concurrent.futures.rst index 6503d1f..dca5145 100644 --- a/Doc/library/concurrent.futures.rst +++ b/Doc/library/concurrent.futures.rst @@ -188,6 +188,10 @@ And:: ThreadPoolExecutor now reuses idle worker threads before starting *max_workers* worker threads too. + .. versionchanged:: 3.13 + Default value of *max_workers* is changed to + ``min(32, (os.process_cpu_count() or 1) + 4)``. + .. _threadpoolexecutor-example: @@ -243,7 +247,7 @@ to a :class:`ProcessPoolExecutor` will result in deadlock. An :class:`Executor` subclass that executes calls asynchronously using a pool of at most *max_workers* processes. If *max_workers* is ``None`` or not - given, it will default to the number of processors on the machine. + given, it will default to :func:`os.process_cpu_count`. If *max_workers* is less than or equal to ``0``, then a :exc:`ValueError` will be raised. On Windows, *max_workers* must be less than or equal to ``61``. If it is not @@ -301,6 +305,10 @@ to a :class:`ProcessPoolExecutor` will result in deadlock. different start method. See the :func:`os.fork` documentation for further explanation. + .. versionchanged:: 3.13 + *max_workers* uses :func:`os.process_cpu_count` by default, instead of + :func:`os.cpu_count`. + .. _processpoolexecutor-example: ProcessPoolExecutor Example diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst index 2f0f1f8..d19f911 100644 --- a/Doc/library/multiprocessing.rst +++ b/Doc/library/multiprocessing.rst @@ -996,13 +996,13 @@ Miscellaneous This number is not equivalent to the number of CPUs the current process can use. The number of usable CPUs can be obtained with - ``len(os.sched_getaffinity(0))`` + :func:`os.process_cpu_count`. When the number of CPUs cannot be determined a :exc:`NotImplementedError` is raised. .. seealso:: - :func:`os.cpu_count` + :func:`os.cpu_count` and :func:`os.process_cpu_count` .. function:: current_process() @@ -2214,7 +2214,7 @@ with the :class:`Pool` class. callbacks and has a parallel map implementation. *processes* is the number of worker processes to use. If *processes* is - ``None`` then the number returned by :func:`os.cpu_count` is used. + ``None`` then the number returned by :func:`os.process_cpu_count` is used. If *initializer* is not ``None`` then each worker process will call ``initializer(*initargs)`` when it starts. @@ -2249,6 +2249,10 @@ with the :class:`Pool` class. .. versionadded:: 3.4 *context* + .. versionchanged:: 3.13 + *processes* uses :func:`os.process_cpu_count` by default, instead of + :func:`os.cpu_count`. + .. note:: Worker processes within a :class:`Pool` typically live for the complete @@ -2775,7 +2779,7 @@ worker threads rather than worker processes. :meth:`~multiprocessing.pool.Pool.terminate` manually. *processes* is the number of worker threads to use. If *processes* is - ``None`` then the number returned by :func:`os.cpu_count` is used. + ``None`` then the number returned by :func:`os.process_cpu_count` is used. If *initializer* is not ``None`` then each worker process will call ``initializer(*initargs)`` when it starts. diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst index 484443a..a789084 100644 --- a/Doc/whatsnew/3.13.rst +++ b/Doc/whatsnew/3.13.rst @@ -91,6 +91,13 @@ Other Language Changes of the ``optimize`` argument. (Contributed by Irit Katriel in :gh:`108113`). +* :mod:`multiprocessing`, :mod:`concurrent.futures`, :mod:`compileall`: + Replace :func:`os.cpu_count` with :func:`os.process_cpu_count` to select the + default number of worker threads and processes. Get the CPU affinity + if supported. + (Contributed by Victor Stinner in :gh:`109649`.) + + New Modules =========== |