summaryrefslogtreecommitdiffstats
path: root/Include/internal
Commit message (Collapse)AuthorAgeFilesLines
* [3.8] gh-95778: CVE-2020-10735: Prevent DoS by very large int() (#96503)Gregory P. Smith2022-09-053-0/+53
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Correctly pre-check for int-to-str conversion Converting a large enough `int` to a decimal string raises `ValueError` as expected. However, the raise comes _after_ the quadratic-time base-conversion algorithm has run to completion. For effective DOS prevention, we need some kind of check before entering the quadratic-time loop. Oops! =) The quick fix: essentially we catch _most_ values that exceed the threshold up front. Those that slip through will still be on the small side (read: sufficiently fast), and will get caught by the existing check so that the limit remains exact. The justification for the current check. The C code check is: ```c max_str_digits / (3 * PyLong_SHIFT) <= (size_a - 11) / 10 ``` In GitHub markdown math-speak, writing $M$ for `max_str_digits`, $L$ for `PyLong_SHIFT` and $s$ for `size_a`, that check is: $$\left\lfloor\frac{M}{3L}\right\rfloor \le \left\lfloor\frac{s - 11}{10}\right\rfloor$$ From this it follows that $$\frac{M}{3L} < \frac{s-1}{10}$$ hence that $$\frac{L(s-1)}{M} > \frac{10}{3} > \log_2(10).$$ So $$2^{L(s-1)} > 10^M.$$ But our input integer $a$ satisfies $|a| \ge 2^{L(s-1)}$, so $|a|$ is larger than $10^M$. This shows that we don't accidentally capture anything _below_ the intended limit in the check. <!-- gh-issue-number: gh-95778 --> * Issue: gh-95778 <!-- /gh-issue-number --> Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org> Co-authored-by: Christian Heimes <christian@python.org> Co-authored-by: Mark Dickinson <dickinsm@gmail.com>
* [3.8] gh-93065: Fix HAMT to iterate correctly over 7-level deep trees ↵Łukasz Langa2022-05-241-1/+13
| | | | | | | | | | | | (GH-93066) (#93148) Also while there, clarify a few things about why we reduce the hash to 32 bits. Co-authored-by: Eli Libman <eli@hyro.ai> Co-authored-by: Yury Selivanov <yury@edgedb.com> Co-authored-by: Łukasz Langa <lukasz@langa.pl> (cherry picked from commit c1f5c903a7e4ed27190488f4e33b00d3c3d952e5)
* bpo-43962: Fix _PyInterpreterState_IDIncref() (GH-25683) (GH-25686)Victor Stinner2021-04-281-1/+1
| | | | | | _PyInterpreterState_IDIncref() now calls _PyInterpreterState_IDInitref() and always increments id_refcount. (cherry picked from commit 32c5a174445ec93747240cd8472012276ed27acf)
* bpo-41686: Always create the SIGINT event on Windows (GH-23344) (GH-23347) ↵Victor Stinner2020-11-171-0/+2
| | | | | | | | | | (GH-23349) bpo-41686, bpo-41713: On Windows, the SIGINT event, _PyOS_SigintEvent(), is now created even if Python is configured to not install signal handlers (PyConfig.install_signal_handlers=0 or Py_InitializeEx(0)). (cherry picked from commit 05a5d697f4f097f37c5c1e2ed0e2338a33c3fb6a)
* [3.9] bpo-40826: Fix GIL usage in PyOS_Readline() (GH-20613) (GH-20616)Victor Stinner2020-06-031-0/+3
| | | | | | | | | | | | | | | | | | | | | | | | | * bpo-40826: Fix GIL usage in PyOS_Readline() (GH-20579) Fix GIL usage in PyOS_Readline(): lock the GIL to set an exception. Pass tstate to my_fgets() and _PyOS_WindowsConsoleReadline(). Cleanup these functions. (cherry picked from commit c353764fd564e401cf47a5d9efab18c72c60014e) * bpo-40826: Add _PyOS_InterruptOccurred(tstate) function (GH-20599) my_fgets() now calls _PyOS_InterruptOccurred(tstate) to check for pending signals, rather calling PyOS_InterruptOccurred(). my_fgets() is called with the GIL released, whereas PyOS_InterruptOccurred() must be called with the GIL held. test_repl: use text=True and avoid SuppressCrashReport in test_multiline_string_parsing(). Fix my_fgets() on Windows: fgets(fp) does crash if fileno(fp) is closed. (cherry picked from commit fa7ab6aa0f9a4f695e5525db5a113cd21fa93787)
* [3.8] bpo-36389: Backport debug enhancements from master (GH-16796)Victor Stinner2019-10-151-4/+22
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * bpo-36389: _PyObject_CheckConsistency() available in release mode (GH-16612) bpo-36389, bpo-38376: The _PyObject_CheckConsistency() function is now also available in release mode. For example, it can be used to debug a crash in the visit_decref() function of the GC. Modify the following functions to also work in release mode: * _PyDict_CheckConsistency() * _PyObject_CheckConsistency() * _PyType_CheckConsistency() * _PyUnicode_CheckConsistency() Other changes: * _PyMem_IsPtrFreed(ptr) now also returns 1 if ptr is NULL (equals to 0). * _PyBytesWriter_CheckConsistency() now returns 1 and is only used with assert(). * Reorder _PyObject_Dump() to write safe fields first, and only attempt to render repr() at the end. (cherry picked from commit 6876257eaabdb30f27ebcbd7d2557278ce2e5705) * bpo-36389: Fix _PyBytesWriter in release mode (GH-16624) Fix _PyBytesWriter API when Python is built in release mode with assertions. (cherry picked from commit 60ec6efd96d95476fe5e38c491491add04f026e5) * bpo-38070: Enhance visit_decref() debug trace (GH-16631) subtract_refs() now pass the parent object to visit_decref() which pass it to _PyObject_ASSERT(). So if the "is freed" assertion fails, the parent is used in debug trace, rather than the freed object. The parent object is more likely to contain useful information. Freed objects cannot be inspected are are displayed as "<object at xxx is freed>" with no other detail. (cherry picked from commit 4d5f94b8cd20f804c7868c5395a15aa6032f874c) * Fix also a typo in PYMEM_DEADBYTE macro comment * bpo-36389: Add newline to _PyObject_AssertFailed() (GH-16629) Add a newline between the verbose object dump and the Py_FatalError() logs for readability. (cherry picked from commit 7775349895088a7ae68cecf0c74cf817f15e2c74)
* bpo-38376: Fix _PyUnicode_CheckConsistency() definition (GH-16623)Victor Stinner2019-10-071-1/+0
| | | Always define _PyUnicode_CheckConsistency() in the CPython C API.
* bpo-38304: PyConfig_InitPythonConfig() cannot fail anymore (GH-16509)Miss Islington (bot)2019-10-011-1/+1
| | | | | | | PyConfig_InitPythonConfig() and PyConfig_InitIsolatedConfig() no longer return PyStatus: they cannot fail anymore. (cherry picked from commit 8462a4936b3a551dc546a6adea04a70b0a07ca67) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-38304: Remove PyConfig.struct_size (GH-16500)Victor Stinner2019-09-301-2/+2
| | | | | For now, we'll rely on the fact that the config structures aren't covered by the stable ABI. We may revisit this in the future if we further explore the idea of offering a stable embedding API.
* bpo-38317: Fix PyConfig.warnoptions priority (GH-16478)Miss Islington (bot)2019-09-292-2/+2
| | | | | | | | | | | | Fix warnings options priority: PyConfig.warnoptions has the highest priority, as stated in the PEP 587. * Document options order in PyConfig.warnoptions documentation. * Make PyWideStringList_INIT macro private: replace "Py" prefix with "_Py". * test_embed: add test_init_warnoptions(). (cherry picked from commit fb4ae152a9930f0e00cae8b2807f534058cf341a) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-38304: Add PyConfig.struct_size (GH-16451) (GH-16453)Victor Stinner2019-09-281-6/+6
| | | | | | | | | | | | | | | | | | | Add a new struct_size field to PyPreConfig and PyConfig structures to allow to modify these structures in the future without breaking the backward compatibility. * Replace private _config_version field with public struct_size field in PyPreConfig and PyConfig. * Public PyPreConfig_InitIsolatedConfig() and PyPreConfig_InitPythonConfig() return type becomes PyStatus, instead of void. * Internal _PyConfig_InitCompatConfig(), _PyPreConfig_InitCompatConfig(), _PyPreConfig_InitFromConfig(), _PyPreConfig_InitFromPreConfig() return type becomes PyStatus, instead of void. * Remove _Py_CONFIG_VERSION * Update the Initialization Configuration documentation. (cherry picked from commit 441b10cf2855955c86565f8d59e72c2efc0f0a57)
* [3.8] bpo-38234: Backport init path config changes from master (GH-16423)Victor Stinner2019-09-261-5/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * bpo-38234: Py_SetPath() uses the program full path (GH-16357) Py_SetPath() now sets sys.executable to the program full path (Py_GetProgramFullPath()), rather than to the program name (Py_GetProgramName()). Fix also memory leaks in pathconfig_set_from_config(). (cherry picked from commit 1ce152a42eaa917d7763bce93f1e1ca72530d7ca) * bpo-38234: Add tests for Python init path config (GH-16358) (cherry picked from commit bb6bf7d342b4503a6227fd209fac934905b6a1aa) * bpo-38234: test_embed: test pyvenv.cfg and pybuilddir.txt (GH-16366) Add test_init_pybuilddir() and test_init_pyvenv_cfg() to test_embed to test pyvenv.cfg and pybuilddir.txt configuration files. Fix sysconfig._generate_posix_vars(): pybuilddir.txt uses UTF-8 encoding, not ASCII. (cherry picked from commit 52ad33abbfb6637d74932617c7013bae0ccf6e32) * bpo-38234: Cleanup getpath.c (GH-16367) * search_for_prefix() directly calls reduce() if found is greater than 0. * Add calculate_pybuilddir() subfunction. * search_for_prefix(): add path string buffer for readability. * Fix some error handling code paths: release resources on error. * calculate_read_pyenv(): rename tmpbuffer to filename. * test.pythoninfo now also logs windows.dll_path (cherry picked from commit 221fd84703c545408bbb4a6e0b58459651331f5c) * bpo-38234: Fix test_embed pathconfig tests (GH-16390) bpo-38234: On macOS and FreeBSD, the temporary directory can be symbolic link. For example, /tmp can be a symbolic link to /var/tmp. Call realpath() to resolve all symbolic links. (cherry picked from commit 00508a7407d7d300b487532e2271534b20e378a7) * bpo-38234: Add test_init_setpath_config() to test_embed (GH-16402) * Add test_embed.test_init_setpath_config(): test Py_SetPath() with PyConfig. * test_init_setpath() and test_init_setpythonhome() no longer call Py_SetProgramName(), but use the default program name. * _PyPathConfig: isolated, site_import and base_executable fields are now only available on Windows. * If executable is set explicitly in the configuration, ignore calculated base_executable: _PyConfig_InitPathConfig() copies executable to base_executable. * Complete path config documentation. (cherry picked from commit 8bf39b606ef7b02c0279a80789f3c4824b0da5e9) * bpo-38234: Complete init config documentation (GH-16404) (cherry picked from commit 88feaecd46a8f427e30ef7ad8cfcddfe392a2402) * bpo-38234: Fix test_embed.test_init_setpath_config() on FreeBSD (GH-16406) Explicitly preinitializes with a Python preconfiguration to avoid Py_SetPath() implicit preinitialization with a compat preconfiguration. Fix also test_init_setpath() and test_init_setpythonhome() on macOS: use self.test_exe as the executable (and base_executable), rather than shutil.which('python3'). (cherry picked from commit 49d99f01e6e51acec5ca57a02e857f0796bc418b) * bpo-38234: Py_Initialize() sets global path configuration (GH-16421) * Py_InitializeFromConfig() now writes PyConfig path configuration to the global path configuration (_Py_path_config). * Add test_embed.test_get_pathconfig(). * Fix typo in _PyWideStringList_Join(). (cherry picked from commit 12f2f177fc483723406d7917194e7f655a20631b)
* bpo-38236: Dump path config at first import error (GH-16300) (GH-16332)Victor Stinner2019-09-232-1/+2
| | | | | | Python now dumps path configuration if it fails to import the Python codecs of the filesystem and stdio encodings. (cherry picked from commit fcdb027234566c4d506d6d753c7d5638490fb088)
* [3.8] bpo-38234: Fix PyConfig_Read() when Py_SetPath() was called (GH-16298) ↵Victor Stinner2019-09-212-6/+6
| | | | | | | | | | | | | | | | | | | | | | | | | (GH-16313) * bpo-38234: Remove _PyPathConfig.dll_path (GH-16307) The DLL path is not computed from any user configuration and cannot be configured by PyConfig. Instead, add a new _Py_dll_path global variable. Remove _PyConfig_SetPathConfig(): replaced with _PyPathConfig_Init(). Py_Initialize() now longer sets the "global path configuration", but only initialize _Py_dll_path. (cherry picked from commit c422167749f92d4170203e996a2c619c818335ea) * bpo-38234: Fix PyConfig_Read() when Py_SetPath() was called (GH-16298) * If Py_SetPath() has been called, _PyConfig_InitPathConfig() now uses its value. * Py_Initialize() now longer copies path configuration from PyConfig to the global path configuration (_Py_path_config). (cherry picked from commit e267793aa4101b2771ed0e66aaff5743d23f59af)
* Fix _PyTraceMalloc_Fini() definition (GH-16259) (GH-16278)Miss Islington (bot)2019-09-191-1/+1
| | | | | | The function return type is void, not int. (cherry picked from commit d299b8b47d9dbea8fec2c2abcb6c32ad32f5f4d1) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* [3.8] bpo-38070: Py_FatalError() logs runtime state (GH-16258)Victor Stinner2019-09-181-3/+8
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * bpo-38070: _Py_DumpTraceback() writes <no Python frame> (GH-16244) When a Python thread has no frame, _Py_DumpTraceback() and _Py_DumpTracebackThreads() now write "<no Python frame>", rather than writing nothing. (cherry picked from commit 8fa3e1740b3f03ea65ddb68411c2238c5f98eec2) * bpo-38070: Enhance _PyObject_Dump() (GH-16243) _PyObject_Dump() now dumps the object address for freed objects and objects with ob_type=NULL. (cherry picked from commit b39afb78768418d9405c4b528c80fa968ccc974d) * bpo-38070: Add _PyRuntimeState.preinitializing (GH-16245) Add _PyRuntimeState.preinitializing field: set to 1 while Py_PreInitialize() is running. _PyRuntimeState: rename also pre_initialized field to preinitialized. (cherry picked from commit d3b904144e86e2442961de6a7dccecbe133d5c6d) * bpo-38070: Py_FatalError() logs runtime state (GH-16246) (cherry picked from commit 1ce16fb0977283ae42a9f8917bbca5f44aa69324)
* bpo-36763: Make Py_BytesMain() public (GH-15532)Miss Islington (bot)2019-08-261-2/+0
| | | | | | | Declare Py_BytesMain() in Include/pylifecycle.h, rather in Include/internal/pycore_pylifecycle.h. (cherry picked from commit 9a943b4ce13fac26873b8100e89c818c5c47ac4b) Co-authored-by: Victor Stinner <vstinner@redhat.com>
* bpo-36763: PyConfig_Read() handles PySys_AddXOption() (GH-15431) (GH-15435)Victor Stinner2019-08-231-0/+2
| | | | | | | | | | | | PyConfig_Read() is now responsible to handle early calls to PySys_AddXOption() and PySys_AddWarnOption(). Options added by PySys_AddXOption() are now handled the same way than PyConfig.xoptions and command line -X options. For example, PySys_AddXOption(L"faulthandler") enables faulthandler as expected. (cherry picked from commit 120b707a6d43452e067daa55a8fdca69f9424abc)
* bpo-37369: Fix initialization of sys members when launched via an app ↵Steve Dower2019-06-291-0/+2
| | | | | | container (GH-14467) sys._base_executable is now always defined on all platforms, and can be overridden through configuration. Also adds test.support.PythonSymlink to encapsulate platform-specific logic for symlinking sys.executable
* Revert "bpo-36818: Add PyInterpreterState.runtime field. (gh-13129)" (GH-13795)Victor Stinner2019-06-043-12/+17
| | | This reverts commit 396e0a8d9dc65453cb9d53500d0a620602656cfe.
* Revert "bpo-33608: Factor out a private, per-interpreter ↵Victor Stinner2019-06-032-18/+7
| | | | | _Py_AddPendingCall(). (gh-13714)" (GH-13780) This reverts commit 6a150bcaeb190d1731b38ab9c7a5d1a352847ddc.
* bpo-26219: per opcode cache for LOAD_GLOBAL (GH-12884)Inada Naoki2019-06-032-0/+30
| | | | | | This patch implements per opcode cache mechanism, and use it in only LOAD_GLOBAL opcode. Based on Yury's opcache3.patch in bpo-26219.
* bpo-33608: Factor out a private, per-interpreter _Py_AddPendingCall(). ↵Eric Snow2019-06-012-7/+18
| | | | (gh-13714)
* bpo-36818: Add PyInterpreterState.runtime field. (gh-13129)Eric Snow2019-06-013-17/+12
| | | https://bugs.python.org/issue36818
* bpo-1230540: Add threading.excepthook() (GH-13515)Victor Stinner2019-05-271-0/+2
| | | | | | | | | | | | | | | | | | | Add a new threading.excepthook() function which handles uncaught Thread.run() exception. It can be overridden to control how uncaught exceptions are handled. threading.ExceptHookArgs is not documented on purpose: it should not be used directly. * threading.excepthook() and threading.ExceptHookArgs. * Add _PyErr_Display(): similar to PyErr_Display(), but accept a 'file' parameter. * Add _thread._excepthook(): C implementation of the exception hook calling _PyErr_Display(). * Add _thread._ExceptHookArgs: structseq type. * Add threading._invoke_excepthook_wrapper() which handles the gory details to ensure that everything remains alive during Python shutdown. * Add unit tests.
* bpo-36763: Implement the PEP 587 (GH-13592)Victor Stinner2019-05-275-199/+204
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | * Add a whole new documentation page: "Python Initialization Configuration" * PyWideStringList_Append() return type is now PyStatus, instead of int * PyInterpreterState_New() now calls PyConfig_Clear() if PyConfig_InitPythonConfig() fails. * Rename files: * Python/coreconfig.c => Python/initconfig.c * Include/cpython/coreconfig.h => Include/cpython/initconfig.h * Include/internal/: pycore_coreconfig.h => pycore_initconfig.h * Rename structures * _PyCoreConfig => PyConfig * _PyPreConfig => PyPreConfig * _PyInitError => PyStatus * _PyWstrList => PyWideStringList * Rename PyConfig fields: * use_module_search_paths => module_search_paths_set * module_search_path_env => pythonpath_env * Rename PyStatus field: _func => func * PyInterpreterState: rename core_config field to config * Rename macros and functions: * _PyCoreConfig_SetArgv() => PyConfig_SetBytesArgv() * _PyCoreConfig_SetWideArgv() => PyConfig_SetArgv() * _PyCoreConfig_DecodeLocale() => PyConfig_SetBytesString() * _PyInitError_Failed() => PyStatus_Exception() * _Py_INIT_ERROR_TYPE_xxx enums => _PyStatus_TYPE_xxx * _Py_UnixMain() => Py_BytesMain() * _Py_ExitInitError() => Py_ExitStatusException() * _Py_PreInitializeFromArgs() => Py_PreInitializeFromBytesArgs() * _Py_PreInitializeFromWideArgs() => Py_PreInitializeFromArgs() * _Py_PreInitialize() => Py_PreInitialize() * _Py_RunMain() => Py_RunMain() * _Py_InitializeFromConfig() => Py_InitializeFromConfig() * _Py_INIT_XXX() => _PyStatus_XXX() * _Py_INIT_FAILED() => _PyStatus_EXCEPTION() * Rename 'err' PyStatus variables to 'status' * Convert RUN_CODE() macro to config_run_code() static inline function * Remove functions: * _Py_InitializeFromArgs() * _Py_InitializeFromWideArgs() * _PyInterpreterState_GetCoreConfig()
* bpo-36710: Add tstate parameter in ceval.c (GH-13547)Victor Stinner2019-05-244-3/+68
| | | | | | | | | | | | | | | | * Fix a possible reference leak in _PyErr_Print() if exception is NULL. * PyErr_BadInternalCall(): replace PyErr_Format() with _PyErr_SetString(). * Add pycore_pyerrors.h header file. * New functions: * _PyErr_Clear() * _PyErr_Fetch() * _PyErr_Print() * _PyErr_Restore() * _PyErr_SetObject() * _PyErr_SetString() * Add 'tstate' parameter to _PyEval_AddPendingCall().
* bpo-36842: Implement PEP 578 (GH-12613)Steve Dower2019-05-231-0/+16
| | | Adds sys.audit, sys.addaudithook, io.open_code, and associated C APIs.
* bpo-36829: PyErr_WriteUnraisable() normalizes exception (GH-13507)Victor Stinner2019-05-221-0/+4
| | | | | | PyErr_WriteUnraisable() now creates a traceback object if there is no current traceback. Moreover, call PyErr_NormalizeException() and PyException_SetTraceback() to normalize the exception value. Ignore silently any error.
* bpo-36763: Add _PyPreConfig._config_init (GH-13481)Victor Stinner2019-05-221-2/+2
| | | | | | | | | | | | | | | | | | | * _PyPreConfig_GetGlobalConfig() and _PyCoreConfig_GetGlobalConfig() now do nothing if the configuration was not initialized with _PyPreConfig_InitCompatConfig() and _PyCoreConfig_InitCompatConfig() * Remove utf8_mode=-2 special case: use utf8_mode=-1 instead. * Fix _PyPreConfig_InitPythonConfig(): * isolated = 0 instead of -1 * use_environment = 1 instead of -1 * Rename _PyConfig_INIT to _PyConfig_INIT_COMPAT * Rename _PyPreConfig_Init() to _PyPreConfig_InitCompatConfig() * Rename _PyCoreConfig_Init() to _PyCoreConfig_InitCompatConfig() * PyInterpreterState_New() now uses _PyCoreConfig_InitPythonConfig() as default configuration, but it's very quickly overriden anyway. * _freeze_importlib.c uses _PyCoreConfig_SetString() to set program_name. * Cleanup preconfig_init_utf8_mode(): cmdline is always non-NULL.
* bpo-36829: Add sys.unraisablehook() (GH-13187)Victor Stinner2019-05-221-0/+4
| | | | | | | | | | | | | | | | | | | Add new sys.unraisablehook() function which can be overridden to control how "unraisable exceptions" are handled. It is called when an exception has occurred but there is no way for Python to handle it. For example, when a destructor raises an exception or during garbage collection (gc.collect()). Changes: * Add an internal UnraisableHookArgs type used to pass arguments to sys.unraisablehook. * Add _PyErr_WriteUnraisableDefaultHook(). * The default hook now ignores exception on writing the traceback. * test_sys now uses unittest.main() to automatically discover tests: remove test_main(). * Add _PyErr_Init(). * Fix PyErr_WriteUnraisable(): hold a strong reference to sys.stderr while using it
* bpo-36763: Fix Python preinitialization (GH-13432)Victor Stinner2019-05-201-2/+8
| | | | | | | | | | | | | | | | | | | | | * Add _PyPreConfig.parse_argv * Add _PyCoreConfig._config_init field and _PyCoreConfigInitEnum enum type * Initialization functions: reject preconfig=NULL and config=NULL * Add config parameter to _PyCoreConfig_DecodeLocaleErr(): pass config->argv to _Py_PreInitializeFromPyArgv(), to parse config command line arguments in preinitialization. * Add config parameter to _PyCoreConfig_SetString(). It now preinitializes Python. * _PyCoreConfig_SetPyArgv() now also preinitializes Python for wide argv * Fix _Py_PreInitializeFromCoreConfig(): don't pass args to _Py_PreInitializeFromPyArgv() if config.parse_argv=0. * Use "char * const *" and "wchar_t * const *" types for 'argv' parameters and _PyArgv.argv. * Add unit test on preinitialization from argv. * _PyPreConfig.allocator type becomes int * Add _PyPreConfig_InitFromPreConfig() and _PyPreConfig_InitFromCoreConfig() helper functions
* bpo-35134: Split traceback.h header (GH-13430)Victor Stinner2019-05-191-0/+92
| | | | Add new Include/cpython/traceback.h and Include/internal/traceback.h header files.
* bpo-36763: Remove _PyCoreConfig.dll_path (GH-13402)Victor Stinner2019-05-181-0/+4
|
* bpo-36763: Use _PyCoreConfig_InitPythonConfig() (GH-13398)Victor Stinner2019-05-181-19/+1
| | | | | | | | | | | | | | | | | _PyPreConfig_InitPythonConfig() and _PyCoreConfig_InitPythonConfig() no longer inherit their values from global configuration variables. Changes: * _PyPreCmdline_Read() now ignores -X dev and PYTHONDEVMODE if dev_mode is already set. * Inline _PyPreConfig_INIT macro into _PyPreConfig_Init() function. * Inline _PyCoreConfig_INIT macro into _PyCoreConfig_Init() function. * Replace _PyCoreConfig_Init() with _PyCoreConfig_InitPythonConfig() in most tests of _testembed.c. * Replace _PyCoreConfig_Init() with _PyCoreConfig_InitIsolatedConfig() in _freeze_importlib.c. * Move some initialization functions from the internal to the private API.
* bpo-36763: _Py_InitializeFromArgs() argc becomes Py_ssize_t (GH-13396)Victor Stinner2019-05-171-4/+4
| | | | | | | * The type of initlization function 'argc' parameters becomes Py_ssize_t, instead of int. * Change _PyPreConfig_Copy() return type to void, instead of int. The function cannot fail anymore. * Fix compilation warnings on Windows.
* bpo-36763: Add _PyInitError functions (GH-13395)Victor Stinner2019-05-171-0/+32
| | | | | | | | | | | | | | | * Add _PyInitError functions: * _PyInitError_Ok() * _PyInitError_Error() * _PyInitError_NoMemory() * _PyInitError_Exit() * _PyInitError_IsError() * _PyInitError_IsExit() * _PyInitError_Failed() * frozenmain.c and _testembed.c now use functions rather than macros. * Move _Py_INIT_xxx() macros to the internal API. * Move _PyWstrList_INIT macro to the internal API.
* bpo-36763: _Py_RunMain() doesn't call Py_Exit() anymore (GH-13390)Victor Stinner2019-05-171-0/+2
| | | | | | | | | Py_Main() and _Py_RunMain() now return the exitcode rather than calling Py_Exit(exitcode) when calling PyErr_Print() if the current exception type is SystemExit. * Add _Py_HandleSystemExit(). * Add pymain_exit_err_print(). * Add pymain_exit_print().
* bpo-36763: Add _PyCoreConfig_InitPythonConfig() (GH-13388)Victor Stinner2019-05-171-1/+6
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Add new functions to get the Python interpreter behavior: * _PyPreConfig_InitPythonConfig() * _PyCoreConfig_InitPythonConfig() Add new functions to get an isolated configuration: * _PyPreConfig_InitIsolatedConfig() * _PyCoreConfig_InitIsolatedConfig() Replace _PyPreConfig_INIT and _PyCoreConfig_INIT with new functions _PyPreConfig_Init() and _PyCoreConfig_Init(). _PyCoreConfig: set configure_c_stdio and parse_argv to 0 by default to behave as Python 3.6 in the default configuration. _PyCoreConfig_Read() no longer sets coerce_c_locale_warn to 1 if it's equal to 0. coerce_c_locale_warn must now be set to -1 (ex: using _PyCoreConfig_InitPythonConfig()) to enable C locale coercion warning. Add unit tests for _PyCoreConfig_InitPythonConfig() and _PyCoreConfig_InitIsolatedConfig(). Changes: * Rename _PyCoreConfig_GetCoreConfig() to _PyPreConfig_GetCoreConfig() * Fix core_read_precmdline(): handle parse_argv=0 * Fix _Py_PreInitializeFromCoreConfig(): pass coreconfig.argv to _Py_PreInitializeFromPyArgv(), except if parse_argv=0
* bpo-36763: Add PyMemAllocatorName (GH-13387)Victor Stinner2019-05-172-2/+10
| | | | | | | | | | | | | | * Add PyMemAllocatorName enum * _PyPreConfig.allocator type becomes PyMemAllocatorName, instead of char* * Remove _PyPreConfig_Clear() * Add _PyMem_GetAllocatorName() * Rename _PyMem_GetAllocatorsName() to _PyMem_GetCurrentAllocatorName() * Remove _PyPreConfig_SetAllocator(): just call _PyMem_SetupAllocators() directly, we don't have do reallocate the configuration with the new allocator anymore! * _PyPreConfig_Write() parameter becomes const, as it should be in the first place!
* bpo-36763: Cleanup precmdline in _PyCoreConfig_Read() (GH-13371)Victor Stinner2019-05-171-1/+1
| | | | precmdline is only needed in _PyCoreConfig_Read(), not in config_read_cmdline(). Reorganize _PyCoreConfig_Read().
* bpo-36900: import.c uses PyInterpreterState.core_config (GH-13278)Victor Stinner2019-05-131-0/+2
| | | | Move _PyImportZip_Init() to the internal C API and add an 'interp' parameter.
* bpo-36728: Remove PyEval_ReInitThreads() from C API (GH-13241)Victor Stinner2019-05-131-0/+2
| | | | | | | | Remove the PyEval_ReInitThreads() function from the Python C API. It should not be called explicitly: use PyOS_AfterFork_Child() instead. Rename PyEval_ReInitThreads() to _PyEval_ReInitThreads() and add a 'runtime' parameter.
* bpo-36710: Add 'ceval' local variable to ceval.c (GH-12934)Victor Stinner2019-05-102-47/+65
| | | | | | | | | | | | | | | | | | | | | | | | | Add "struct _ceval_runtime_state *ceval = &_PyRuntime.ceval;" local variables to function to better highlight the dependency on the global variable _PyRuntime and to point directly to _PyRuntime.ceval field rather than on the larger _PyRuntime. Changes: * Add _PyRuntimeState_GetThreadState(runtime) macro. * Add _PyEval_AddPendingCall(ceval, ...) and _PyThreadState_Swap(gilstate, ...) functions. * _PyThreadState_GET() macro now calls _PyRuntimeState_GetThreadState() using &_PyRuntime. * Add 'ceval' parameter to COMPUTE_EVAL_BREAKER(), SIGNAL_PENDING_SIGNALS(), _PyEval_SignalAsyncExc(), _PyEval_SignalReceived() and _PyEval_FiniThreads() macros and functions. * Add 'tstate' parameter to call_function(), do_call_core() and do_raise(). * Add 'runtime' parameter to _Py_CURRENTLY_FINALIZING(), _Py_FinishPendingCalls() and _PyThreadState_DeleteExcept() macros and functions. * Declare 'runtime', 'tstate', 'ceval' and 'eval_breaker' variables as constant.
* bpo-36737: Use the module state C-API for warnings. (gh-13159)Eric Snow2019-05-102-2/+3
|
* bpo-36763: _PyCoreConfig_SetPyArgv() preinitializes Python (GH-13037)Victor Stinner2019-05-021-2/+7
| | | | | | | | _PyCoreConfig_SetPyArgv() and _PyCoreConfig_SetWideString() now pre-initialize Python if needed to ensure that the locale encoding is properly configured. * Add _Py_PreInitializeFromPyArgv() internal function. * Add 'args' parameter to _Py_PreInitializeFromCoreConfig()
* bpo-36775: _PyCoreConfig only uses wchar_t* (GH-13062)Victor Stinner2019-05-023-5/+12
| | | | | | | | | | | | | | | | | _PyCoreConfig: Change filesystem_encoding, filesystem_errors, stdio_encoding and stdio_errors fields type from char* to wchar_t*. Changes: * PyInterpreterState: replace fscodec_initialized (int) with fs_codec structure. * Add get_error_handler_wide() and unicode_encode_utf8() helper functions. * Add error_handler parameter to unicode_encode_locale() and unicode_decode_locale(). * Remove _PyCoreConfig_SetString(). * Rename _PyCoreConfig_SetWideString() to _PyCoreConfig_SetString(). * Rename _PyCoreConfig_SetWideStringFromString() to _PyCoreConfig_DecodeLocale().
* bpo-36775: Add _PyUnicode_InitEncodings() (GH-13057)Victor Stinner2019-05-021-2/+3
| | | | | | Move get_codec_name() and initfsencoding() from pylifecycle.c to unicodeobject.c. Rename also "init" functions in pylifecycle.c.
* bpo-36763: Add _PyCoreConfig_SetString() (GH-13035)Victor Stinner2019-05-011-2/+10
| | | | | | | | | | | | Add 3 new config methods: * _PyCoreConfig_SetString() * _PyCoreConfig_SetWideString() * _PyCoreConfig_SetWideStringFromString() Changes: * _PyCoreConfig_Copy() returns _PyInitError. * Add CONFIG_GET_ENV_DUP().
* bpo-36763: Add _PyCoreConfig_SetArgv() (GH-13030)Victor Stinner2019-05-011-2/+12
| | | | | | | | | * Add 2 new config methods: * _PyCoreConfig_SetArgv() * _PyCoreConfig_SetWideArgv() * Add also an internal _PyCoreConfig_SetPyArgv() method. * Remove 'args' parameter from _PyCoreConfig_Read().