summaryrefslogtreecommitdiffstats
path: root/Python
Commit message (Collapse)AuthorAgeFilesLines
* [3.12] gh-113993: Make interned strings mortal (GH-120520, GH-121364, ↵Petr Viktorin2024-09-279-29/+96
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | GH-121903, GH-122303) (#123065) This backports several PRs for gh-113993, making interned strings mortal so they can be garbage-collected when no longer needed. * Allow interned strings to be mortal, and fix related issues (GH-120520) * Add an InternalDocs file describing how interning should work and how to use it. * Add internal functions to *explicitly* request what kind of interning is done: - `_PyUnicode_InternMortal` - `_PyUnicode_InternImmortal` - `_PyUnicode_InternStatic` * Switch uses of `PyUnicode_InternInPlace` to those. * Disallow using `_Py_SetImmortal` on strings directly. You should use `_PyUnicode_InternImmortal` instead: - Strings should be interned before immortalization, otherwise you're possibly interning a immortalizing copy. - `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to `SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in backports, as they are now part of public API and version-specific ABI. * Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery. Make sure the statically allocated string singletons are unique. This means these sets are now disjoint: - `_Py_ID` - `_Py_STR` (including the empty string) - one-character latin-1 singletons Now, when you intern a singleton, that exact singleton will be interned. * Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic). * Intern `_Py_STR` singletons at startup. * Beef up the tests. Cover internal details (marked with `@cpython_only`). * Add lots of assertions * Don't immortalize in PyUnicode_InternInPlace; keep immortalizing in other API (GH-121364) * Switch PyUnicode_InternInPlace to _PyUnicode_InternMortal, clarify docs * Document immortality in some functions that take `const char *` This is PyUnicode_InternFromString; PyDict_SetItemString, PyObject_SetAttrString; PyObject_DelAttrString; PyUnicode_InternFromString; and the PyModule_Add convenience functions. Always point out a non-immortalizing alternative. * Don't immortalize user-provided attr names in _ctypes * Immortalize names in code objects to avoid crash (GH-121903) * Intern latin-1 one-byte strings at startup (GH-122303) There are some 3.12-specific changes, mainly to allow statically allocated strings in deepfreeze. (In 3.13, deepfreeze switched to the general `_Py_ID`/`_Py_STR`.) Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* [3.12] gh-124188: Fix PyErr_ProgramTextObject() (GH-124189) (GH-124426)Serhiy Storchaka2024-09-241-35/+56
| | | | | | | | * Detect source file encoding. * Use the "replace" error handler even for UTF-8 (default) encoding. * Remove the BOM. * Fix detection of too long lines if they contain NUL. * Return the head rather than the tail for truncated long lines. (cherry picked from commit e2f710792b0418b8ca1ca3b8cdf39588c7268495)
* [3.12] gh-98442: fix locations of with statement's cleanup instructions ↵Miss Islington (bot)2024-09-151-1/+3
| | | | | | | | | | | (GH-120763) (#120787) gh-98442: fix locations of with statement's cleanup instructions (GH-120763) (cherry picked from commit 55596ae0446e40f47e2a28b8897fe9530c32a19a) gh-98442: fix location of with statement's cleanup instructions Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
* [3.12] gh-123892: Add "_wmi" to sys.stdlib_module_names (GH-123893) (#123897)Miss Islington (bot)2024-09-101-0/+1
| | | | | | gh-123892: Add "_wmi" to sys.stdlib_module_names (GH-123893) (cherry picked from commit fb1b51a58df4315f7ef3171a5abeb74f132b0971) Co-authored-by: Victor Stinner <vstinner@python.org>
* [3.12] gh-93691: fix too broad source locations of with-statement ↵Miss Islington (bot)2024-09-021-3/+2
| | | | | | | | instructions (GH-120125) (#123605) gh-93691: fix too broad source locations of with-statement instructions (GH-120125) (cherry picked from commit eca3f7762c23b22a73a5e0b09520748c88aab4a0) Co-authored-by: Irit Katriel <1055913+iritkatriel@users.noreply.github.com>
* [3.12] gh-123142: fix too wide source location of GET_ITER/GET_AITER ↵Irit Katriel2024-08-281-8/+7
| | | | | (GH-123420). (#123436) (cherry picked from commit 61bef6245c4a32bf430d684ede8603f423d63284)
* [3.12] gh-123083: Fix a potential use-after-free in ``STORE_ATTR_WITH… ↵Donghee Na2024-08-222-203/+205
| | | | | | (#123237) [3.12] gh-123083: Fix a potential use-after-free in ``STORE_ATTR_WITH_HINT`` (gh-123092) (cherry picked from commit 297f2e093ec95800ae2184330b8408c875523467)
* [3.12] gh-123142: Fix too wide source locations in tracebacks of exceptions ↵Irit Katriel2024-08-221-2/+3
| | | | | from broken iterables in comprehensions (GH-123173). (#123210) (cherry picked from commit ec89620e5e147ba028a46dd695ef073a72000b84)
* [3.12] gh-118814: Fix the TypeVar constructor when name is passed by keyword ↵Serhiy Storchaka2024-08-081-1/+1
| | | | | | | | (GH-122664) (GH-122807) Fix _PyArg_UnpackKeywordsWithVararg for the case when argument for positional-or-keyword parameter is passed by keyword. There was only one such case in the stdlib -- the TypeVar constructor. (cherry picked from commit 540fcc62f5da982b79504221cac01bfab8b73ba1)
* [3.12] gh-122334: Fix crash when importing ssl after re-initialization ↵neonene2024-08-021-0/+12
| | | | | | | | | (GH-122481) (#122495) Fix crash when importing ssl after re-initialization The current METH_FASTCALL|METH_KEYWORDS functions in a non-builtin module can cause segfaults after restarting the main interpreter, invoking _PyArg_UnpackKeywords() with an insufficiently cleared _PyArg_Parser struct. This patch fixes the invalidation of the static argument parsers.
* [3.12] gh-122029: Log call events in sys.setprofile when it's a method with ↵Miss Islington (bot)2024-07-231-0/+13
| | | | | | | | | | c function (GH-122072) (GH-122206) gh-122029: Log call events in sys.setprofile when it's a method with c function (GH-122072) Log call events in sys.setprofile when it is a method with a C function. (cherry picked from commit e91ef13861e88c27aed51a24e58d1dcc855a01dc) Co-authored-by: Tian Gao <gaogaotiantian@hotmail.com>
* [3.12] gh-121905: Consistently use "floating-point" instead of "floating ↵Serhiy Storchaka2024-07-191-2/+2
| | | | | point" (GH-121907) (GH-122013) (cherry picked from commit 1a0c7b9ba48a2dffb70bb0c7327abae1d3e87356)
* [3.12] gh-121153: Fix some errors with use of _PyLong_CompactValue() (GH-121154)Serhiy Storchaka2024-07-171-2/+2
| | | | | | | * The result has type Py_ssize_t, not intptr_t. * Type cast between unsigned and signed integer types should be explicit. * Downcasting should be explicit. * Fix integer overflow check in sum(). (cherry picked from commit 1801545)
* [3.12] gh-121657: Display correct error message for yield from outsid… ↵Kirill Podoprigora2024-07-151-1/+1
| | | | | | | | (GH-121769) (cherry picked from commit 178e44de8f023be7a5dc400044ab61983b191f24) Co-authored-by: Gregor <36135323+gege-hoho@users.noreply.github.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* [3.12] gh-121390: tracemalloc: Fix tracebacks memory leak (GH-121391) (#121393)Miss Islington (bot)2024-07-051-1/+1
| | | | | | | | | | | gh-121390: tracemalloc: Fix tracebacks memory leak (GH-121391) The tracemalloc_tracebacks hash table has traceback keys and NULL values, but its destructors do not reflect this -- key_destroy_func is NULL while value_destroy_func is raw_free. Swap these to free the traceback keys instead. (cherry picked from commit db39bc42f90c151b298f97b780e62703adbf1221) Co-authored-by: Josh Brobst <jbrobst@proton.me>
* [3.12] gh-120811: Fix reference leak upon `_PyContext_Exit` failure ↵Miss Islington (bot)2024-06-221-0/+1
| | | | | | | | | (GH-120812) (#120844) gh-120811: Fix reference leak upon `_PyContext_Exit` failure (GH-120812) (cherry picked from commit aed31beca9a54b85a1392631a48da80602210f18) Co-authored-by: Peter <zintensitydev@gmail.com> Co-authored-by: Kumar Aditya <kumaraditya@python.org>
* [3.12] gh-120722: Set position on RETURN_VALUE in lambda (GH-120724) (#120739)Jelle Zijlstra2024-06-191-1/+1
| | | (cherry picked from commit d8f27cb1141fd3575de816438ed80a916c0560ed)
* [3.12] gh-119897: Revert buggy optimization which was removed in 3.13 (#120467)Irit Katriel2024-06-181-13/+5
|
* [3.12] gh-93691: fix too broad source locations of for statement iterators ↵Irit Katriel2024-06-131-0/+7
| | | | | | (GH-120330 (#120405) [3.12] gh-93691: fix too broad source locations of for statement iterators (GH-120330). (cherry picked from commit 97b69db167be28a33688db436551a6c3c3ea4662)
* [3.12] gh-120343: Fix column offsets of multiline tokens in tokenize ↵Miss Islington (bot)2024-06-121-4/+10
| | | | | | | (GH-120391) (#120428) (cherry picked from commit 4b5d3e0e721a952f4ac9d17bee331e6dfe543dcd) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
* [3.12] gh-120343: Do not reset byte_col_offset_diff after multiline tokens ↵Miss Islington (bot)2024-06-111-1/+6
| | | | | | | | (GH-120352) (#120356) (cherry picked from commit 1b62bcee941e54244b3ce6476aef8913604987c9) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
* [3.12] gh-119666: fix multiple class-scope comprehensions referencing ↵Miss Islington (bot)2024-06-101-13/+10
| | | | __class__ (GH-120295) (#120300)
* [3.12] gh-120155: Fix optimize_and_assemble_code_unit() error handling (#120231)Victor Stinner2024-06-071-1/+1
| | | | | | | | | | | | | | | | | | gh-120155: Fix optimize_and_assemble_code_unit() error handling Don't use 'g' before it's being initialized: don't use the 'error' label if consts_dict_keys_inorder() failed. Fix the Coverity issue: Error: UNINIT (CWE-457): Python-3.12.2/Python/compile.c:7670:5: skipped_decl: Jumping over declaration of ""g"". Python-3.12.2/Python/compile.c:7714:5: uninit_use_in_call: Using uninitialized value ""g.g_block_list"" when calling ""_PyCfgBuilder_Fini"". Python-3.12.2/Python/compile.c:7714:5: uninit_use_in_call: Using uninitialized value ""g.g_entryblock"" when calling ""_PyCfgBuilder_Fini"". 7712| Py_XDECREF(consts); 7713| instr_sequence_fini(&optimized_instrs); 7714|-> _PyCfgBuilder_Fini(&g); 7715| return co; 7716| }
* [3.12] gh-119311: Fix name mangling with PEP 695 generic classes (#119464) ↵Jelle Zijlstra2024-06-042-13/+48
| | | | | | | | | | (#119644) * [3.12] gh-119311: Fix name mangling with PEP 695 generic classes (#119464) Fixes #119311. Fixes #119395. (cherry picked from commit a9a74da4a0ca0645f049e67b6434a95e30592c32)
* [3.12] gh-111499: Fix PYTHONMALLOCSTATS at Python exit (#120021) (#120023)Victor Stinner2024-06-041-5/+7
| | | | | | | | gh-111499: Fix PYTHONMALLOCSTATS at Python exit (#120021) Call _PyObject_DebugMallocStats() earlier in Py_FinalizeEx(), before the interpreter is deleted. (cherry picked from commit 5a1205b641df133932ed4c65b9a4ff5724e89963)
* [3.12] gh-119821: Fix refleak in LOAD_FROM_DICT_OR_GLOBALS (#119975)Jelle Zijlstra2024-06-032-291/+295
|
* [3.12] gh-119821: Support non-dict globals in LOAD_FROM_DICT_OR_GLOBALS ↵Jelle Zijlstra2024-06-012-329/+341
| | | | | | | | | | | (#119822) (#119890) The implementation basically copies LOAD_GLOBAL. Possibly it could be deduplicated, but that seems like it may get hairy since the two operations have different operands. This is important to fix in 3.14 for PEP 649, but it's a bug in earlier versions too, and we should backport to 3.13 and 3.12 if possible. (cherry picked from commit 80a4e3899420faaa012c82b4e82cdb6675a6a944)
* [3.12] gh-119585: Fix crash involving `PyGILState_Release()` and ↵Sam Gross2024-05-311-0/+6
| | | | | | | | | | | `PyThreadState_Clear()` (GH-119753) (#119861) Make sure that `gilstate_counter` is not zero in when calling `PyThreadState_Clear()`. A destructor called from `PyThreadState_Clear()` may call back into `PyGILState_Ensure()` and `PyGILState_Release()`. If `gilstate_counter` is zero, it will try to create a new thread state before the current active thread state is destroyed, leading to an assertion failure or crash. (cherry picked from commit bcc1be39cb1d04ad9fc0bd1b9193d3972835a57c)
* [3.12] gh-119704: Fix reference leak in the ``Python/Python-tokenize.c`` ↵Miss Islington (bot)2024-05-291-0/+1
| | | | (GH-119705) (#119708)
* [3.12] gh-119118: Fix performance regression in tokenize module (GH-119615) ↵Miss Islington (bot)2024-05-281-4/+40
| | | | | | | | | | | | | (#119683) - Cache line object to avoid creating a Unicode object for all of the tokens in the same line. - Speed up byte offset to column offset conversion by using the smallest buffer possible to measure the difference. (cherry picked from commit d87b0151062e36e67f9e42e1595fba5bf23a485c) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
* [3.12] gh-119213: Be More Careful About _PyArg_Parser.kwtuple Across ↵Eric Snow2024-05-221-0/+17
| | | | | | | | | | | Interpreters (gh-119331) (gh-119425) _PyArg_Parser holds static global data generated for modules by Argument Clinic. The _PyArg_Parser.kwtuple field is a tuple object, even though it's stored within a static global. In some cases the tuple is statically allocated and thus it's okay that it gets shared by multiple interpreters. However, in other cases the tuple is set lazily, allocated from the heap using the active interprepreter at the point the tuple is needed. This is a problem once that interpreter is destroyed since _PyArg_Parser.kwtuple becomes at dangling pointer, leading to crashes. It isn't a problem if the tuple is allocated under the main interpreter, since its lifetime is bound to the lifetime of the runtime. The solution here is to temporarily switch to the main interpreter. The alternative would be to always statically allocate the tuple. This change also fixes a bug where only the most recent parser was added to the global linked list. (cherry picked from commit 81865002aee8eaaeb3c7e402f86183afa6de77bf)
* [3.12] gh-118513: Fix sibling comprehensions with a name bound in one and ↵Miss Islington (bot)2024-05-031-39/+42
| | | | | | | | | | global in the other (GH-118526) (#118548) gh-118513: Fix sibling comprehensions with a name bound in one and global in the other (GH-118526) (cherry picked from commit c8deb1e4b495bf97ab00c710dfd63f227e1fb645) Co-authored-by: Carl Meyer <carl@oddbird.net> Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> Co-authored-by: Kirill Podoprigora <kirill.bast9@mail.ru>
* [3.12] gh-118272: Clear generator frame's locals when the generator is ↵Irit Katriel2024-05-021-5/+13
| | | | closed (#118451)
* [3.12] gh-116767: fix crash on 'async with' with many context managers ↵Irit Katriel2024-05-011-2/+16
| | | | | | | | | | | | (GH-118348) (#118477) gh-116767: fix crash on 'async with' with many context managers (GH-118348) Account for `add_stopiteration_handler` pushing a block for `async with`. To allow generator functions that previously almost hit the `CO_MAXBLOCKS` limit by nesting non-async blocks, the limit is increased by 1. This increase allows one more block in non-generator functions. (cherry picked from commit c1bf4874c1e9db2beda1d62c8c241229783c789b)
* [3.12] gh-91565: Replace bugs.python.org links with Devguide/GitHub ones ↵Miss Islington (bot)2024-04-151-1/+1
| | | | | | | | | | | | | (GH-91568) (GH-117890) gh-91565: Replace bugs.python.org links with Devguide/GitHub ones (GH-91568) (cherry picked from commit 3de09cadde788065a4f2d45117e789c9353bbd12) Co-authored-by: Steve (Gadget) Barnes <gadgetsteve@hotmail.com> Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com> Co-authored-by: Oleg Iarygin <oleg@arhadthedev.net> Co-authored-by: Petr Viktorin <encukou@gmail.com> Co-authored-by: Ezio Melotti <ezio.melotti@gmail.com>
* [3.12] gh-90300: Remove reference to PYTHON_FROZEN_MODULES in Python CLI ↵Serhiy Storchaka2024-03-191-2/+1
| | | | | help (GH-117035) Fix error introduced in 4be9fa896117bf07dc944a29c98dd18b71dd6c74.
* [3.12] gh-90300: Improve the Python CLI help output (GH-115853) (GH-117022)Serhiy Storchaka2024-03-191-67/+38
| | | | | | | | | | | * document equivalent command-line options for all environment variables * document equivalent environment variables for all command-line options * reduce the size of variable and option descriptions to minimum * remove the ending period in single-sentence descriptions (cherry picked from commit b85572c47dc7a8c65fc366a87a3660fc7a3ed244) Co-authored-by: Éric <merwok@netwok.org> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com>
* [3.12] gh-116735: Use `MISSING` for `CALL` event if argument is absen… ↵Tian Gao2024-03-192-2/+2
| | | | | (#116873) [3.12] gh-116735: Use `MISSING` for `CALL` event if argument is absent (GH-116737)
* [3.12] gh-112536: Add TSAN build on Github Actions (GH-116872)Antoine Pitrou2024-03-181-0/+4
| | | | | (cherry picked from commit 20578a1f68c841a264b72b00591b11ab2fa77b43) Co-authored-by: Donghee Na <donghee.na@python.org>
* [3.12] gh-116626: Emit `CALL` events for all `INSTRUMENTED_CALL_FUNCTION_EX ↵Tian Gao2024-03-142-28/+28
| | | | | (GH-116732) Backport of GH-116627
* [3.12] gh-90300: Fix undocumented envvars in the Python CLI help (GH-116765) ↵Serhiy Storchaka2024-03-141-0/+3
| | | | | (GH-116797) (cherry picked from commit 19ac28bd08fdb16795e6f82ea7bfac73e8f3791b)
* [3.12] gh-90300: Document equivalent -X options for envvars in the Python ↵Serhiy Storchaka2024-03-141-18/+19
| | | | | CLI help (GH-116756) (GH-116786) (cherry picked from commit 8c6db45ce34df7081d7497e638daf3e130303295)
* [3.12] gh-90300: Sort the -X options and some envvars in the Python CLI help ↵Serhiy Storchaka2024-03-131-27/+28
| | | | | (GH-116739) (GH-116766) (cherry picked from commit e54bdeab9ce2958a22ba08c1f1f1431c5e8056bd)
* [3.12] gh-90300: Fix cmdline.rst (GH-116721) (GH-116724)Miss Islington (bot)2024-03-131-2/+2
| | | | | | | * Fix the description of the "-b" option. * Add references to environment variables for "-s" and "-X dev" options. (cherry picked from commit 33662d4e01d73cd4f29a25efc2ef09288129023f) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* [3.12] gh-116447: Fix possible UB in `arraymodule` and `getargs` (GH-116459) ↵Miss Islington (bot)2024-03-081-2/+2
| | | | | | | | (#116496) gh-116447: Fix possible UB in `arraymodule` and `getargs` (GH-116459) (cherry picked from commit fdb2d90a274158aee23b526d972172bf41bd4b7e) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
* [3.12] gh-116326: Handler errors correctly in `getwindowsversion` in ↵Miss Islington (bot)2024-03-051-18/+32
| | | | | | | | `sysmodule` (GH-116339) (#116354) * gh-116326: Handler errors correctly in `getwindowsversion` in `sysmodule` (GH-116339) (cherry picked from commit c91bdf86ef1cf9365b61a46aa2e51e5d1932b00a) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
* [3.12] gh-115320: Refactor `get_hash_info` in `sysmodule.c` not to swallow ↵Miss Islington (bot)2024-03-041-23/+25
| | | | | | | | errors (GH-115321) (#116323) gh-115320: Refactor `get_hash_info` in `sysmodule.c` not to swallow errors (GH-115321) (cherry picked from commit 207030f5527d405940b79c10c1413c1e8ff696c1) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
* [3.12] gh-116034: fix location info on the error of a failed assertion (#116054)Irit Katriel2024-02-291-1/+1
|
* [3.12] gh-90300: Reformat the Python CLI help output (GH-93415) (#115847)Łukasz Langa2024-02-231-95/+95
| | | | | (cherry picked from commit 2e92ffd7fa89e3bd33ee2f31541d3dc53aaa2d12) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* [3.12] gh-96497: Mangle name before symtable lookup in ↵Miss Islington (bot)2024-02-171-4/+10
| | | | | | | 'symtable_extend_namedexpr_scope' (GH-96561) (GH-115603) (cherry picked from commit 664965a1c141e8af5eb465d29099781a6a2fc3f3) Co-authored-by: wookie184 <wookie1840@gmail.com>