| Commit message (Collapse) | Author | Age | Files | Lines |
|
|
|
|
| |
* subprocess.Popen now longer uses posix_spawn() if uid, gid or gids are set.
* test_subprocess: add "nobody" and "nfsnobody" group names for test_group().
* test_subprocess: test_user() and test_group() are now also tested with close_fds=False.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* subprocess: Add user, group and extra_groups paremeters to subprocess.Popen
This adds a `user` parameter to the Popen constructor that will call
setreuid() in the child before calling exec(). This allows processes
running as root to safely drop privileges before running the subprocess
without having to use a preexec_fn.
This also adds a `group` parameter that will call setregid() in
the child process before calling exec().
Finally an `extra_groups` parameter was added that will call
setgroups() to set the supplimental groups.
|
|
|
|
|
|
|
| |
Fixes a possible hang when using a timeout on subprocess.run() while
capturing output. If the child process spawned its own children or otherwise
connected its stdout or stderr handles with another process, we could hang
after the timeout was reached and our child was killed when attempting to read
final output from the pipes.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
As noted by @eryksun in [1] and [2], using _cleanup and _active(in
__del__) is not necessary on Windows, since:
> Unlike Unix, a process in Windows doesn't have to be waited on by
> its parent to avoid a zombie. Keeping the handle open will actually
> create a zombie until the next _cleanup() call, which may be never
> if Popen() isn't called again.
This patch simply defines `subprocess._active` as `None`, for which we already
have the proper logic in place in `subprocess.Popen.__del__`, that prevents it
from trying to append the process to the `_active`. This patch also defines
`subprocess._cleanup` as a noop for Windows.
[1] https://bugs.python.org/issue37380#msg346333
[2] https://bugs.python.org/issue36067#msg336262
Signed-off-by: Ruslan Kuprieiev <ruslan@iterative.ai>
|
| |
|
|
|
|
|
|
| |
Fix an unintended ValueError from :func:`subprocess.run` when checking for
conflicting `input` and `stdin` or `capture_output` and `stdout` or `stderr` args
when they were explicitly provided but with `None` values within a passed in
`**kwargs` dict rather than as passed directly by name.
|
| |
|
|
|
|
| |
Classes that define __str__ the same as __repr__ can
just inherit it from object.
|
|
|
|
|
|
|
| |
Close pipes/fds in subprocess by using ExitStack.
"In case of premature failure on X.Close() or os.close(X) the remaining pipes/fds will remain "open". Perhaps it makes sense to use contextlib.ExitStack."
- Rationale: https://github.com/python/cpython/pull/11575#discussion_r250288394
|
| |
|
|
|
|
|
|
| |
* subprocess.Popen can now also use os.posix_spawn() with pipes,
but only if pipe file descriptors are greater than 2.
* Fix Popen._posix_spawn(): set '_child_created' attribute to True.
* Add Popen._close_pipe_fds() helper function to factorize the code.
|
|
|
| |
This reverts commit 07858894689047c77f9c12ddc061d30681368d19.
|
|
|
|
| |
The subprocess module can now use the os.posix_spawnp() function,
if it is available, to locate the program in the PATH.
|
|
|
|
|
|
|
|
|
|
|
|
| |
The subprocess module can now use the os.posix_spawn() function
in some cases for better performance. Currently, it is only used on macOS
and Linux (using glibc 2.24 or newer) if all these conditions are met:
* executable path contains a directory
* close_fds=False
* preexec_fn, pass_fds, cwd, stdin, stdout, stderr
and start_new_session parameters are not set
Co-authored-by: Joannah Nanjekye <nanjekyejoannah@gmail.com>
|
|
|
|
|
|
|
|
| |
The "-I" command line option (run Python in isolated mode) is now
also copied by the multiprocessing and distutils modules when
spawning child processes. Previously, only -E and -s options (enabled
by -I) were copied.
subprocess._args_from_interpreter_flags() now copies the -I flag.
|
|
|
|
|
|
|
|
|
| |
If buffering=1 is specified for open() in binary mode, it is silently
treated as buffering=-1 (i.e., the default buffer size).
Coupled with the fact that line buffering is always supported in Python 2,
such behavior caused several issues (e.g., bpo-10344, bpo-21332).
Warn that line buffering is not supported if open() is called with
binary mode and buffering=1.
|
|
|
|
|
| |
(GH-9163)
8621bb5d93239316f97281826461b85072ff6db7 sets the filename in directly in the FileNotFoundError, so we may revert the earlier fix 5f780400572508a8179de6a6c13b58b7be417ef5.
|
|
|
|
| |
Check for functionality via imports rather than checking sys.platform
specifically for Windows
|
|
|
|
|
|
|
| |
subprocess.Popen now copies the startupinfo argument to leave it
unchanged: it will modify the copy, so that the same STARTUPINFO
object can be used multiple times.
Add subprocess.STARTUPINFO.copy() method.
|
|
|
|
|
|
| |
* Revert "bpo-31961: subprocess now accepts path-like args (GH-4329)"
This reverts commit dd42cb71f2cb02f3a32f016137b12a146bc0d0e2.
|
|
|
| |
Allow os.PathLike args in subprocess APIs.
|
|
|
|
| |
Add "capture_output=True" option to subprocess.run, this is equivalent to
setting stdout=PIPE, stderr=PIPE but is much more readable.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Do not allow receiving a SIGINT to cause the subprocess module to trigger an
immediate SIGKILL of the child process. SIGINT is normally sent to all child
processes by the OS at the same time already as was the established normal
behavior in 2.7 and 3.2. This behavior change was introduced during the fix to https://bugs.python.org/issue12494 and is generally surprising to command line
tool users who expect other tools launched in child processes to get their own
SIGINT and do their own cleanup.
In Python 3.3-3.6 subprocess.call and subprocess.run would immediately
SIGKILL the child process upon receiving a SIGINT (which raises a
KeyboardInterrupt). We now give the child a small amount of time to
exit gracefully before resorting to a SIGKILL.
This is also the case for subprocess.Popen.__exit__ which would
previously block indefinitely waiting for the child to die. This was
hidden from many users by virtue of subprocess.call and subprocess.run
sending the signal immediately.
Behavior change: subprocess.Popen.__exit__ will not block indefinitely
when the exiting exception is a KeyboardInterrupt. This is done for
user friendliness as people expect their ^C to actually happen. This
could cause occasional orphaned Popen objects when not using `call` or
`run` with a child process that hasn't exited.
Refactoring involved: The Popen.wait method deals with the
KeyboardInterrupt second chance, existing platform specific internals
have been renamed to _wait().
Also fixes comment typos.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Windows (#1218)
Even though Python marks any handles it opens as non-inheritable there
is still a race when using `subprocess.Popen` since creating a process
with redirected stdio requires temporarily creating inheritable handles.
By implementing support for `subprocess.Popen(close_fds=True)` we fix
this race.
In order to implement this we use PROC_THREAD_ATTRIBUTE_HANDLE_LIST
which is available since Windows Vista. Which allows to pass an explicit
list of handles to inherit when creating a process.
This commit also adds `STARTUPINFO.lpAttributeList["handle_list"]`
which can be used to control PROC_THREAD_ATTRIBUTE_HANDLE_LIST
directly.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add -X utf8 command line option, PYTHONUTF8 environment variable
and a new sys.flags.utf8_mode flag.
* If the LC_CTYPE locale is "C" at startup: enable automatically the
UTF-8 mode.
* Add _winapi.GetACP(). encodings._alias_mbcs() now calls
_winapi.GetACP() to get the ANSI code page
* locale.getpreferredencoding() now returns 'UTF-8' in the UTF-8
mode. As a side effect, open() now uses the UTF-8 encoding by
default in this mode.
* Py_DecodeLocale() and Py_EncodeLocale() now use the UTF-8 encoding
in the UTF-8 Mode.
* Update subprocess._args_from_interpreter_flags() to handle -X utf8
* Skip some tests relying on the current locale if the UTF-8 mode is
enabled.
* Add test_utf8mode.py.
* _Py_DecodeUTF8_surrogateescape() gets a new optional parameter to
return also the length (number of wide characters).
* pymain_get_global_config() and pymain_set_global_config() now
always copy flag values, rather than only copying if the new value
is greater than the old value.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Rather than supporting dev mode directly in the warnings module, this
instead adjusts the initialisation code to add an extra 'default'
entry to sys.warnoptions when dev mode is enabled.
This ensures that dev mode behaves *exactly* as if `-Wdefault` had
been passed on the command line, including in the way it interacts
with `sys.warnoptions`, and with other command line flags like `-bb`.
Fix also bpo-20361: have -b & -bb options take precedence over any
other warnings options.
Patch written by Nick Coghlan, with minor modifications of Victor Stinner.
|
|
|
|
|
|
|
|
|
|
| |
The developer mode (-X dev) now creates all default warnings filters
to order filters in the correct order to always show ResourceWarning
and make BytesWarning depend on the -b option.
Write a functional test to make sure that ResourceWarning is logged
twice at the same location in the developer mode.
Add a new 'dev_mode' field to _PyCoreConfig.
|
|
|
|
|
|
|
|
| |
Modify subprocess._args_from_interpreter_flags() to handle -X dev
option.
Add also unit tests for test.support.args_from_interpreter_flags()
and test.support.optim_args_from_interpreter_flags().
|
| |
|
| |
|
|
|
|
|
|
|
|
|
| |
Improve human friendliness of the Popen API: Add text=False as a
keyword-only argument to subprocess.Popen along with a Popen
attribute .text_mode and set this based on the
encoding/errors/universal_newlines/text arguments.
The universal_newlines parameter and attribute are maintained for
backwards compatibility.
|
|
|
| |
Explicitly state that communicate() closes stdin after writing input to it.
|
|
|
|
|
| |
To match the documentation updates already made.
Also renames the local variable used within to match
what it actually holds.
|
|
|
|
|
|
| |
* Remove Setup.config
* Always define WITH_THREAD for compatibility.
|
|
|
| |
Avoid concatenating bytes with str in the typically rare subprocess error path (exec failed). Includes a mock based unittest to exercise the codepath.
|
|
|
|
| |
Have the subprocess module set the filename in the FileNotFoundError
exception raised on POSIX systems when the executable or cwd are missing.
|
|
|
|
|
|
|
|
|
| |
* bpo-30121: Fix debug assert in subprocess on Windows
This is caused by closing HANDLEs using os.close which is for CRT file
descriptors and not for HANDLEs.
* bpo-30121: Suppress debug assertion in test_subprocess when ran directly
|
|
|
|
|
|
| |
(#2325)
Prevent passing other invalid environment variables and command arguments.
|
|
|
|
|
| |
On Windows, subprocess.Popen.communicate() now also ignore EINVAL
on stdin.write() if the child process is still running but closed the
pipe.
|
| |
|
| |
|
|
|
| |
#157 added the test, but it's currently (correctly) broken on windows.
|
|
|
|
|
|
|
| |
The Windows-specific subprocess.STARTUPINFO class now accepts
keyword-only arguments to its constructor to set the various
data attributes.
Patch by Subhendu Ghosh.
|
|\
| |
| |
| | |
exited to a stopped instead of terminated state (ex: when under ptrace).
|
| |\
| | |
| | |
| | | |
exited to a stopped instead of terminated state (ex: when under ptrace).
|
| | |
| | |
| | |
| | | |
exited to a stopped instead of terminated state (ex: when under ptrace).
|
|\ \ \
| |/ / |
|
| | |
| | |
| | |
| | |
| | | |
Issue #29174, #26741: subprocess.Popen.__del__() now keeps a strong reference
to warnings.warn() function.
|
| | |
| | |
| | |
| | | |
instead of /bin/sh.
|