summaryrefslogtreecommitdiffstats
path: root/Doc
diff options
context:
space:
mode:
authorVictor Stinner <vstinner@python.org>2023-10-01 01:14:57 (GMT)
committerGitHub <noreply@github.com>2023-10-01 01:14:57 (GMT)
commita46e96076898d126c9f276aef1934195aac34b4e (patch)
treebeaec25a75f381f77b74bf4e164eb813c20ec81f /Doc
parent53eb9a676f8c59b206dfc536b7590f6563ad65e0 (diff)
downloadcpython-a46e96076898d126c9f276aef1934195aac34b4e.zip
cpython-a46e96076898d126c9f276aef1934195aac34b4e.tar.gz
cpython-a46e96076898d126c9f276aef1934195aac34b4e.tar.bz2
gh-109649: Use os.process_cpu_count() (#110165)
Replace os.cpu_count() with os.process_cpu_count() in modules: * compileall * concurrent.futures * multiprocessing Replace os.cpu_count() with os.process_cpu_count() in programs: * _decimal deccheck.py test * freeze.py * multissltests.py * python -m test (regrtest) * wasm_build.py Other changes: * test.pythoninfo logs os.process_cpu_count(). * regrtest gets os.process_cpu_count() / os.cpu_count() in headers.
Diffstat (limited to 'Doc')
-rw-r--r--Doc/library/compileall.rst2
-rw-r--r--Doc/library/concurrent.futures.rst10
-rw-r--r--Doc/library/multiprocessing.rst12
-rw-r--r--Doc/whatsnew/3.13.rst7
4 files changed, 25 insertions, 6 deletions
diff --git a/Doc/library/compileall.rst b/Doc/library/compileall.rst
index a7455ae..b4723b9 100644
--- a/Doc/library/compileall.rst
+++ b/Doc/library/compileall.rst
@@ -90,7 +90,7 @@ compile Python sources.
.. cmdoption:: -j N
Use *N* workers to compile the files within the given directory.
- If ``0`` is used, then the result of :func:`os.cpu_count()`
+ If ``0`` is used, then the result of :func:`os.process_cpu_count()`
will be used.
.. cmdoption:: --invalidation-mode [timestamp|checked-hash|unchecked-hash]
diff --git a/Doc/library/concurrent.futures.rst b/Doc/library/concurrent.futures.rst
index 6503d1f..dca5145 100644
--- a/Doc/library/concurrent.futures.rst
+++ b/Doc/library/concurrent.futures.rst
@@ -188,6 +188,10 @@ And::
ThreadPoolExecutor now reuses idle worker threads before starting
*max_workers* worker threads too.
+ .. versionchanged:: 3.13
+ Default value of *max_workers* is changed to
+ ``min(32, (os.process_cpu_count() or 1) + 4)``.
+
.. _threadpoolexecutor-example:
@@ -243,7 +247,7 @@ to a :class:`ProcessPoolExecutor` will result in deadlock.
An :class:`Executor` subclass that executes calls asynchronously using a pool
of at most *max_workers* processes. If *max_workers* is ``None`` or not
- given, it will default to the number of processors on the machine.
+ given, it will default to :func:`os.process_cpu_count`.
If *max_workers* is less than or equal to ``0``, then a :exc:`ValueError`
will be raised.
On Windows, *max_workers* must be less than or equal to ``61``. If it is not
@@ -301,6 +305,10 @@ to a :class:`ProcessPoolExecutor` will result in deadlock.
different start method. See the :func:`os.fork` documentation for
further explanation.
+ .. versionchanged:: 3.13
+ *max_workers* uses :func:`os.process_cpu_count` by default, instead of
+ :func:`os.cpu_count`.
+
.. _processpoolexecutor-example:
ProcessPoolExecutor Example
diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
index 2f0f1f8..d19f911 100644
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -996,13 +996,13 @@ Miscellaneous
This number is not equivalent to the number of CPUs the current process can
use. The number of usable CPUs can be obtained with
- ``len(os.sched_getaffinity(0))``
+ :func:`os.process_cpu_count`.
When the number of CPUs cannot be determined a :exc:`NotImplementedError`
is raised.
.. seealso::
- :func:`os.cpu_count`
+ :func:`os.cpu_count` and :func:`os.process_cpu_count`
.. function:: current_process()
@@ -2214,7 +2214,7 @@ with the :class:`Pool` class.
callbacks and has a parallel map implementation.
*processes* is the number of worker processes to use. If *processes* is
- ``None`` then the number returned by :func:`os.cpu_count` is used.
+ ``None`` then the number returned by :func:`os.process_cpu_count` is used.
If *initializer* is not ``None`` then each worker process will call
``initializer(*initargs)`` when it starts.
@@ -2249,6 +2249,10 @@ with the :class:`Pool` class.
.. versionadded:: 3.4
*context*
+ .. versionchanged:: 3.13
+ *processes* uses :func:`os.process_cpu_count` by default, instead of
+ :func:`os.cpu_count`.
+
.. note::
Worker processes within a :class:`Pool` typically live for the complete
@@ -2775,7 +2779,7 @@ worker threads rather than worker processes.
:meth:`~multiprocessing.pool.Pool.terminate` manually.
*processes* is the number of worker threads to use. If *processes* is
- ``None`` then the number returned by :func:`os.cpu_count` is used.
+ ``None`` then the number returned by :func:`os.process_cpu_count` is used.
If *initializer* is not ``None`` then each worker process will call
``initializer(*initargs)`` when it starts.
diff --git a/Doc/whatsnew/3.13.rst b/Doc/whatsnew/3.13.rst
index 484443a..a789084 100644
--- a/Doc/whatsnew/3.13.rst
+++ b/Doc/whatsnew/3.13.rst
@@ -91,6 +91,13 @@ Other Language Changes
of the ``optimize`` argument.
(Contributed by Irit Katriel in :gh:`108113`).
+* :mod:`multiprocessing`, :mod:`concurrent.futures`, :mod:`compileall`:
+ Replace :func:`os.cpu_count` with :func:`os.process_cpu_count` to select the
+ default number of worker threads and processes. Get the CPU affinity
+ if supported.
+ (Contributed by Victor Stinner in :gh:`109649`.)
+
+
New Modules
===========