summaryrefslogtreecommitdiffstats
path: root/Python
Commit message (Collapse)AuthorAgeFilesLines
* bpo-42260: Compute the path config in the main init (GH-23211)Victor Stinner2020-11-103-65/+63
| | | | | | | | | | | | | | | | | | | | | The path configuration is now computed in the "main" initialization. The core initialization no longer computes it. * Add _PyConfig_Read() function to read the configuration without computing the path configuration. * pyinit_core() no longer computes the path configuration: it is now computed by init_interp_main(). * The path configuration output members of PyConfig are now optional: * executable * base_executable * prefix * base_prefix * exec_prefix * base_exec_prefix * _PySys_UpdateConfig() now skips NULL strings in PyConfig. * _testembed: Rename test_set_config() to test_init_set_config() for consistency with other tests.
* bpo-41100: Support macOS 11 and Apple Silicon (GH-22855)Ronald Oussoren2020-11-082-13/+60
| | | | | | | | | | | Co-authored-by: Lawrence D’Anna <lawrence_danna@apple.com> * Add support for macOS 11 and Apple Silicon (aka arm64) As a side effect of this work use the system copy of libffi on macOS, and remove the vendored copy * Support building on recent versions of macOS while deploying to older versions This allows building installers on macOS 11 while still supporting macOS 10.9.
* bpo-42282: Fold constants inside named expressions (GH-23190)Nick Coghlan2020-11-071-11/+43
| | | | | | | * The AST optimiser wasn't descending into named expressions, so any constant subexpressions weren't being folded at compile time * Remove "default:" clauses inside the AST optimiser code to reduce the risk of similar bugs passing unnoticed in future compiler changes
* bpo-42260: PyConfig_Read() only parses argv once (GH-23168)Victor Stinner2020-11-051-4/+7
| | | | | | | | | | | | The PyConfig_Read() function now only parses PyConfig.argv arguments once: PyConfig.parse_argv is set to 2 after arguments are parsed. Since Python arguments are strippped from PyConfig.argv, parsing arguments twice would parse the application options as Python options. * Rework the PyConfig documentation. * Fix _testinternalcapi.set_config() error handling. * SetConfigTests no longer needs parse_argv=0 when restoring the old configuration.
* bpo-42260: Add _PyConfig_FromDict() (GH-23167)Victor Stinner2020-11-052-58/+356
| | | | | | | * Rename config_as_dict() to _PyConfig_AsDict(). * Add 'module_search_paths_set' to _PyConfig_AsDict(). * Add _PyConfig_FromDict(). * Add get_config() and set_config() to _testinternalcapi. * Add config_check_consistency().
* bpo-42266: Handle monkey-patching descriptors in LOAD_ATTR cache (GH-23157)Pablo Galindo2020-11-051-7/+1
|
* bpo-42260: Add _PyInterpreterState_SetConfig() (GH-23158)Victor Stinner2020-11-043-10/+96
| | | | | | | | * Inline _PyInterpreterState_SetConfig(): replace it with _PyConfig_Copy(). * Add _PyErr_SetFromPyStatus() * Add _PyInterpreterState_GetConfigCopy() * Add a new _PyInterpreterState_SetConfig() function. * Add an unit which gets, modifies, and sets the config.
* bpo-42260: Main init modify sys.flags in-place (GH-23150)Victor Stinner2020-11-042-73/+85
| | | | | | | | | | | When Py_Initialize() is called twice, the second call now updates more sys attributes for the configuration, rather than only sys.argv. * Rename _PySys_InitMain() to _PySys_UpdateConfig(). * _PySys_UpdateConfig() now modifies sys.flags in-place, instead of creating a new flags object. * Remove old commented sys.flags flags (unbuffered and skip_first). * Add private _PySys_GetObject() function. * When Py_Initialize(), Py_InitializeFromConfig() and
* bpo-1635741: Fix ref leak in _PyWarnings_Init() error path (GH-23151)Victor Stinner2020-11-042-11/+5
| | | | | | Replace PyModule_AddObject() with PyModule_AddObjectRef() in the _warnings module to fix a reference leak on error. Use also PyModule_AddObjectRef() in importdl.c.
* bpo-1635741: _ast uses PyModule_AddObjectRef() (GH-23146)Victor Stinner2020-11-041-216/+115
| | | | Replace PyModule_AddObject() with PyModule_AddObjectRef() in the _ast module (Python-ast.c).
* bpo-1635741: _contextvars uses PyModule_AddType() (GH-23147)Victor Stinner2020-11-041-3/+3
| | | | | | Replace PyModule_AddObject() with PyModule_AddType() in the _contextvars module (Python-ast.c). Add also the module name to _contextvars types name.
* bpo-1635741: Add PyModule_AddObjectRef() function (GH-23122)Victor Stinner2020-11-041-31/+39
| | | | | Added PyModule_AddObjectRef() function: similar to PyModule_AddObjectRef() but don't steal a reference to the value on success.
* bpo-42236: os.device_encoding() respects UTF-8 Mode (GH-23119)Victor Stinner2020-11-042-17/+14
| | | | On Unix, the os.device_encoding() function now returns 'UTF-8' rather than the device encoding if the Python UTF-8 Mode is enabled.
* bpo-41796: Call _PyAST_Fini() earlier to fix a leak (GH-23131)Victor Stinner2020-11-033-19/+33
| | | | | | | | | | | | | | | Call _PyAST_Fini() on all interpreters, not only on the main interpreter. Also, call it ealier to fix a reference leak. Python types contain a reference to themselves in in their PyTypeObject.tp_mro member. _PyAST_Fini() must called before the last GC collection to destroy AST types. _PyInterpreterState_Clear() now calls _PyAST_Fini(). It now also calls _PyWarnings_Fini() on subinterpeters, not only on the main interpreter. Add an assertion in AST init_types() to ensure that the _ast module is no longer used after _PyAST_Fini() has been called.
* bpo-26789: Fix logging.FileHandler._open() at exit (GH-23053)Victor Stinner2020-11-021-1/+1
| | | | | | | | | | | | | | | | | | | The logging.FileHandler class now keeps a reference to the builtin open() function to be able to open or reopen the file during Python finalization. Fix errors like: Exception ignored in: (...) Traceback (most recent call last): (...) File ".../logging/__init__.py", line 1463, in error File ".../logging/__init__.py", line 1577, in _log File ".../logging/__init__.py", line 1587, in handle File ".../logging/__init__.py", line 1649, in callHandlers File ".../logging/__init__.py", line 948, in handle File ".../logging/__init__.py", line 1182, in emit File ".../logging/__init__.py", line 1171, in _open NameError: name 'open' is not defined
* bpo-41796: Make _ast module state per interpreter (GH-23024)Victor Stinner2020-11-021-112/+133
| | | | | | | | | | | The ast module internal state is now per interpreter. * Rename "astmodulestate" to "struct ast_state" * Add pycore_ast.h internal header: the ast_state structure is now declared in pycore_ast.h. * Add PyInterpreterState.ast (struct ast_state) * Remove get_ast_state() * Rename get_global_ast_state() to get_ast_state() * PyAST_obj2mod() now handles get_ast_state() failures
* bpo-42236: Enhance init and encoding documentation (GH-23109)Victor Stinner2020-11-022-6/+5
| | | | | | | | | | | | | | | | | | | | | Enhance the documentation of the Python startup, filesystem encoding and error handling, locale encoding. Add a new "Python UTF-8 Mode" section. * Add "locale encoding" and "filesystem encoding and error handler" to the glossary * Remove documentation from Include/cpython/initconfig.h: move it to Doc/c-api/init_config.rst. * Doc/c-api/init_config.rst: * Document command line options and environment variables * Document default values. * Add a new "Python UTF-8 Mode" section in Doc/library/os.rst. * Add warnings to Py_DecodeLocale() and Py_EncodeLocale() docs. * Document how Python selects the filesystem encoding and error handler at a single place: PyConfig.filesystem_encoding and PyConfig.filesystem_errors. * PyConfig: move orig_argv member at the right place.
* bpo-41435: Add sys._current_exceptions() function (GH-21689)Julien Danjou2020-11-023-1/+100
| | | | | | This adds a new function named sys._current_exceptions() which is equivalent ot sys._current_frames() except that it returns the exceptions currently handled by other threads. It is equivalent to calling sys.exc_info() for each running thread.
* bpo-42236: Use UTF-8 encoding if nl_langinfo(CODESET) fails (GH-23086)Victor Stinner2020-11-012-35/+11
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | If the nl_langinfo(CODESET) function returns an empty string, Python now uses UTF-8 as the filesystem encoding. In May 2010 (commit b744ba1d14c5487576c95d0311e357b707600b47), I modified Python to log a warning and use UTF-8 as the filesystem encoding (instead of None) if nl_langinfo(CODESET) returns an empty string. In August 2020 (commit 94908bbc1503df830d1d615e7b57744ae1b41079), I modified Python startup to fail with a fatal error and a specific error message if nl_langinfo(CODESET) returns an empty string. The intent was to prevent guessing the encoding and also investigate user configuration where this case happens. In 10 years (2010 to 2020), I saw zero user report about the error message related to nl_langinfo(CODESET) returning an empty string. Today, UTF-8 became the defacto standard and it's safe to make the assumption that the user expects UTF-8. For example, nl_langinfo(CODESET) can return an empty string on macOS if the LC_CTYPE locale is not supported, and UTF-8 is the default encoding on macOS. While this change is likely to not affect anyone in practice, it should make UTF-8 lover happy ;-) Rewrite also the documentation explaining how Python selects the filesystem encoding and error handler.
* bpo-42236: Enhance _locale._get_locale_encoding() (GH-23083)Victor Stinner2020-11-012-49/+72
| | | | | | | | | * Rename _Py_GetLocaleEncoding() to _Py_GetLocaleEncodingObject() * Add _Py_GetLocaleEncoding() which returns a wchar_t* string to share code between _Py_GetLocaleEncodingObject() and config_get_locale_encoding(). * _Py_GetLocaleEncodingObject() now decodes nl_langinfo(CODESET) from the current locale encoding with surrogateescape, rather than using UTF-8.
* bpo-42208: Add _Py_GetLocaleEncoding() (GH-23050)Victor Stinner2020-10-312-62/+106
| | | | | | | | _io.TextIOWrapper no longer calls getpreferredencoding(False) of _bootlocale to get the locale encoding, but calls _Py_GetLocaleEncoding() instead. Add config_get_fs_encoding() sub-function. Reorganize also config_get_locale_encoding() code.
* bpo-42208: Call GC collect earlier in PyInterpreterState_Clear() (GH-23044)Victor Stinner2020-10-302-11/+28
| | | | | | | | The last GC collection is now done before clearing builtins and sys dictionaries. Add also assertions to ensure that gc.collect() is no longer called after _PyGC_Fini(). Pass also the tstate to PyInterpreterState_Clear() to pass the correct tstate to _PyGC_CollectNoFail() and _PyGC_Fini().
* bpo-42208: Move _PyImport_Cleanup() to pylifecycle.c (GH-23040)Victor Stinner2020-10-302-230/+290
| | | | | Move _PyImport_Cleanup() to pylifecycle.c, rename it to finalize_modules(), split it (200 lines) into many smaller sub-functions and cleanup the code.
* bpo-42208: Pass tstate to _PyGC_CollectNoFail() (GH-23038)Victor Stinner2020-10-302-4/+4
| | | | | | | | | | | | Move private _PyGC_CollectNoFail() to the internal C API. Remove the private _PyGC_CollectIfEnabled() which was just an alias to the public PyGC_Collect() function since Python 3.8. Rename functions: * collect() => gc_collect_main() * collect_with_callback() => gc_collect_with_callback() * collect_generations() => gc_collect_generations()
* bpo-42099: Fix reference to ob_type in unionobject.c and ceval (GH-22829)Neil Schemenauer2020-10-271-1/+1
| | | * Use Py_TYPE() rather than o->ob_type.
* bpo-42161: Use _PyLong_GetZero() and _PyLong_GetOne() (GH-22995)Victor Stinner2020-10-272-4/+7
| | | | Use _PyLong_GetZero() and _PyLong_GetOne() in Objects/ and Python/ directories.
* bpo-42157: unicodedata avoids references to UCD_Type (GH-22990)Victor Stinner2020-10-261-4/+2
| | | | | | | | | | * UCD_Check() uses PyModule_Check() * Simplify the internal _PyUnicode_Name_CAPI structure: * Remove size and state members * Remove state and self parameters of getcode() and getname() functions * Remove global_module_state
* bpo-1635741: _PyUnicode_Name_CAPI moves to internal C API (GH-22713)Victor Stinner2020-10-261-7/+10
| | | | | | | | | | The private _PyUnicode_Name_CAPI structure of the PyCapsule API unicodedata.ucnhash_CAPI moves to the internal C API. Moreover, the structure gets a new state member which must be passed to the getcode() and getname() functions. * Move Include/ucnhash.h to Include/internal/pycore_ucnhash.h * unicodedata module is now built with Py_BUILD_CORE_MODULE. * unicodedata: move hashAPI variable into unicodedata_module_state.
* bpo-42152: Use PyDict_Contains and PyDict_SetDefault if appropriate. (GH-22986)Serhiy Storchaka2020-10-263-24/+29
| | | | | | | If PyDict_GetItemWithError is only used to check whether the key is in dict, it is better to use PyDict_Contains instead. And if it is used in combination with PyDict_SetItem, PyDict_SetDefault can replace the combination.
* bpo-42006: Stop using PyDict_GetItem, PyDict_GetItemString and ↵Serhiy Storchaka2020-10-266-39/+74
| | | | | | | | | | | _PyDict_GetItemId. (GH-22648) These functions are considered not safe because they suppress all internal errors and can return wrong result. PyDict_GetItemString and _PyDict_GetItemId can also silence current exception in rare cases. Remove no longer used _PyDict_GetItemId. Add _PyDict_ContainsId and rename _PyDict_Contains into _PyDict_Contains_KnownHash.
* bpo-38324: Fix test__locale.py Windows failures (GH-20529)TIGirardi2020-10-201-2/+13
| | | | Use wide-char _W_* fields of lconv structure on Windows Remove "ps_AF" from test__locale.known_numerics on Windows
* bpo-42093: Add opcode cache for LOAD_ATTR (GH-22803)Pablo Galindo2020-10-201-5/+216
|
* closes bpo-42030: Remove legacy AIX dynload support (GH-22717)Kevin Adler2020-10-161-190/+0
| | | | | | | | Since c19c5a6, AIX builds have defaulted to using dynload_shlib over dynload_aix when dlopen is available. This function has been available since AIX 4.3, which went out of support in 2003, the same year the previously referenced commit was made. It has been nearly 20 years since a version of AIX has been supported which has not used dynload_shlib so there's no reason to keep this legacy code around.
* bpo-41919, test_codecs: Move codecs.register calls to setUp() (GH-22513)Hai Shi2020-10-161-25/+0
| | | | * Move the codecs' (un)register operation to testcases. * Remove _codecs._forget_codec() and _PyCodec_Forget()
* bpo-41894: Fix UnicodeDecodeError while loading native module (GH-22466)Kevin Adler2020-10-153-8/+25
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | When running in a non-UTF-8 locale, if an error occurs while importing a native Python module (say because a dependent share library is missing), the error message string returned may contain non-ASCII code points causing a UnicodeDecodeError. PyUnicode_DecodeFSDefault is used for buffers which may contain filesystem paths. For consistency with os.strerror(), PyUnicode_DecodeLocale is used for buffers which contain system error messages. While the shortname parameter is always encoded in ASCII according to PEP 489, it is left decoded using PyUnicode_FromString to minimize the changes and since it should not affect the decoding (albeit _potentially_ slower). In dynload_hpux, since the error buffer contains a message generated from a static ASCII string and the module filesystem path, PyUnicode_DecodeFSDefault is used instead of PyUnicode_DecodeLocale as is used elsewhere. * bpo-41894: Fix bugs in dynload error msg handling For both dynload_aix and dynload_hpux, properly handle the possibility that decoding strings may return NULL and when such an error happens, properly decrement any previously decoded strings and return early. In addition, in dynload_aix, ensure that we pass the decoded string *object* pathname_ob to PyErr_SetImportError instead of the original pathname buffer. Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* closes bpo-42029: Remove dynload_dl (GH-22687)Kevin Adler2020-10-141-23/+0
| | | | | All references to this dynamic loading method were removed in b9949db, when support for this method was dropped, but the implementation code was not dropped (seemingly in oversight).
* bpo-40422: Move _Py_closerange to fileutils.c (GH-22680)Kyle Evans2020-10-131-0/+76
| | | | | | | This API is relatively lightweight and organizationally, given that it's used by multiple modules, it makes sense to move it to fileutils. Requires making sure that _posixsubprocess is compiled with the appropriate Py_BUIILD_CORE_BUILTIN macro.
* bpo-41993: Fix possible issues in remove_module() (GH-22631)Serhiy Storchaka2020-10-111-10/+13
| | | | | | * PyMapping_HasKey() is not safe because it silences all exceptions and can return incorrect result. * Informative exceptions from PyMapping_DelItem() are overridden with RuntimeError and the original exception raised before calling remove_module() is lost. * There is a race condition between PyMapping_HasKey() and PyMapping_DelItem().
* bpo-42002: Clean up initialization of the sys module. (GH-22642)Serhiy Storchaka2020-10-111-107/+45
| | | Makes the code clearer and make errors handling more correct.
* bpo-38605: bump the magic number for 'annotations' future (#22630)Batuhan Taskaya2020-10-101-111/+111
|
* bpo-41991: Remove _PyObject_HasAttrId (GH-22629)Serhiy Storchaka2020-10-102-4/+21
| | | It can silence arbitrary exceptions.
* bpo-42000: Cleanup the AST related C-code (GH-22641)Batuhan Taskaya2020-10-101-10/+4
| | | | | | | | - Use the proper asdl sequence when creating empty arguments - Remove reduntant casts (thanks to new typed asdl_sequences) - Remove MarshalPrototypeVisitor and some utilities from asdl generator - Fix the header of `Python/ast.c` (kept from pgen times) Automerge-Triggered-By: @pablogsal
* bpo-41756: Add PyIter_Send function (#22443)Vladimir Matveev2020-10-101-14/+7
|
* bpo-38605: Make 'from __future__ import annotations' the default (GH-20434)Batuhan Taskaya2020-10-063-37/+3
| | | | | The hard part was making all the tests pass; there are some subtle issues here, because apparently the future import wasn't tested very thoroughly in previous Python versions. For example, `inspect.signature()` returned type objects normally (except for forward references), but strings with the future import. We changed it to try and return type objects by calling `typing.get_type_hints()`, but fall back on returning strings if that function fails (which it may do if there are future references in the annotations that require passing in a specific namespace to resolve).
* bpo-41936. Remove macros Py_ALLOW_RECURSION/Py_END_ALLOW_RECURSION (GH-22552)Serhiy Storchaka2020-10-052-4/+0
|
* bpo-21955: Change my nickname in BINARY_ADD comment (GH-22481)Victor Stinner2020-10-011-1/+1
|
* bpo-41670: Remove outdated predict macro invocation. (GH-22026)Mark Shannon2020-09-291-2/+0
| | | Remove PREDICTion of POP_BLOCK from FOR_ITER.
* bpo-41842: Add codecs.unregister() function (GH-22360)Hai Shi2020-09-281-0/+25
| | | | Add codecs.unregister() and PyCodec_Unregister() functions to unregister a codec search function.
* bpo-39934: Account for control blocks in 'except' in compiler. (GH-22395)Mark Shannon2020-09-251-8/+11
| | | * Account for control blocks in 'except' in compiler. Fixes #39934.
* bpo-40941: Fix stackdepth compiler warnings (GH-22377)Victor Stinner2020-09-231-4/+4
| | | | Explicitly cast a difference of two pointers to int: PyFrameObject.f_stackdepth is an int.