summaryrefslogtreecommitdiffstats
path: root/Parser
Commit message (Collapse)AuthorAgeFilesLines
* [3.12] gh-113993: Make interned strings mortal (GH-120520, GH-121364, ↵Petr Viktorin2024-09-272-2/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | GH-121903, GH-122303) (#123065) This backports several PRs for gh-113993, making interned strings mortal so they can be garbage-collected when no longer needed. * Allow interned strings to be mortal, and fix related issues (GH-120520) * Add an InternalDocs file describing how interning should work and how to use it. * Add internal functions to *explicitly* request what kind of interning is done: - `_PyUnicode_InternMortal` - `_PyUnicode_InternImmortal` - `_PyUnicode_InternStatic` * Switch uses of `PyUnicode_InternInPlace` to those. * Disallow using `_Py_SetImmortal` on strings directly. You should use `_PyUnicode_InternImmortal` instead: - Strings should be interned before immortalization, otherwise you're possibly interning a immortalizing copy. - `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to `SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in backports, as they are now part of public API and version-specific ABI. * Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery. Make sure the statically allocated string singletons are unique. This means these sets are now disjoint: - `_Py_ID` - `_Py_STR` (including the empty string) - one-character latin-1 singletons Now, when you intern a singleton, that exact singleton will be interned. * Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic). * Intern `_Py_STR` singletons at startup. * Beef up the tests. Cover internal details (marked with `@cpython_only`). * Add lots of assertions * Don't immortalize in PyUnicode_InternInPlace; keep immortalizing in other API (GH-121364) * Switch PyUnicode_InternInPlace to _PyUnicode_InternMortal, clarify docs * Document immortality in some functions that take `const char *` This is PyUnicode_InternFromString; PyDict_SetItemString, PyObject_SetAttrString; PyObject_DelAttrString; PyUnicode_InternFromString; and the PyModule_Add convenience functions. Always point out a non-immortalizing alternative. * Don't immortalize user-provided attr names in _ctypes * Immortalize names in code objects to avoid crash (GH-121903) * Intern latin-1 one-byte strings at startup (GH-122303) There are some 3.12-specific changes, mainly to allow statically allocated strings in deepfreeze. (In 3.13, deepfreeze switched to the general `_Py_ID`/`_Py_STR`.) Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* [3.12] gh-123321: Fix Parser/myreadline.c to prevent a segfault during a ↵Miss Islington (bot)2024-09-051-3/+10
| | | | | | | | | | | | | | | multi-threaded race (GH-123323) (#123677) * gh-123321: Fix Parser/myreadline.c to prevent a segfault during a multi-threaded race (GH-123323) (cherry picked from commit a4562fedadb73fe1e978dece65c3bcefb4606678) Co-authored-by: Bar Harel <bharel@barharel.com> * Remove @requires_gil_enabled for 3.12 --------- Co-authored-by: Bar Harel <bharel@barharel.com> Co-authored-by: Sam Gross <colesbury@gmail.com>
* [3.12] gh-123229: Fix valgrind warning by initializing the f-string buffers ↵Pablo Galindo Salgado2024-08-231-1/+1
| | | | | | | to 0 in the tokenizer (GH-123263) (#123265) (cherry picked from commit adc5190014efcf7b7a4c5dfc9998faa8345527ed) Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
* [3.12] gh-122300: Preserve AST nodes for format specifiers with single ↵Pablo Galindo Salgado2024-07-271-1/+2
| | | | | elements (GH-122308) (#122363) (cherry picked from commit db2d8b6db1b56c2bd3802b86f9b76da33e8898d7)
* [3.12] gh-122270: Fix typos in the Py_DEBUG macro name (GH-122271) (GH-122276)Miss Islington (bot)2024-07-251-1/+1
| | | | | (cherry picked from commit 6c09b8de5c67406113e8d082e05c9587e35a852a) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* [3.12] gh-122026: Fix identification of mismatched parentheses inside ↵Pablo Galindo Salgado2024-07-201-0/+3
| | | | | | | f-strings (GH-122028) (#122062) (cherry picked from commit 2009e25e26040dca32696e70f91f13665350e7fd) Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
* [3.12] gh-121130: Fix f-string format specifiers with debug expressions ↵Pablo Galindo Salgado2024-07-203-4325/+4123
| | | | (GH-121150) (#122063)
* [3.12] gh-120155: Fix Coverity issue in parse_string() (GH-120997) (#121006)Miss Islington (bot)2024-06-251-0/+5
| | | | | | gh-120155: Fix Coverity issue in parse_string() (GH-120997) (cherry picked from commit 769aea332940f03c3e5b1ad9badd6635c1ac992a) Co-authored-by: Victor Stinner <vstinner@python.org>
* [3.12] gh-119118: Fix performance regression in tokenize module (GH-119615) ↵Miss Islington (bot)2024-05-282-0/+26
| | | | | | | | | | | | | (#119683) - Cache line object to avoid creating a Unicode object for all of the tokens in the same line. - Speed up byte offset to column offset conversion by using the smallest buffer possible to measure the difference. (cherry picked from commit d87b0151062e36e67f9e42e1595fba5bf23a485c) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
* [3.12] gh-109120: Fix syntax error in handlinh of incorrect star ↵Grigoriev Semyon2024-04-041-1173/+1360
| | | | | | | expressions… (#117465) gh-109120: Fix syntax error in handlinh of incorrect star expressions (#117444) (cherry picked from commit c97d3af2391e62ef456ef2365d48ab9b8cdbe27b)
* [3.12] gh-115881: Ensure `ast.parse()` parses conditional context managers ↵Alex Waygood2024-02-261-1/+1
| | | | even with low `feature_version` passed (#115920) (#115959)
* [3.12] gh-115823: Calculate correctly error locations when dealing with ↵Miss Islington (bot)2024-02-261-11/+9
| | | | | | | | implicit encodings (GH-115824) (#115949) gh-115823: Calculate correctly error locations when dealing with implicit encodings (GH-115824) (cherry picked from commit 015b97d19a24a169cc3c0939119e1228791e4253) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.12] GH-112215: Backport C recursion changes (GH-115083)Mark Shannon2024-02-131-3/+2
|
* [3.12] gh-113703: Correctly identify incomplete f-strings in the codeop ↵Pablo Galindo Salgado2024-01-051-3/+7
| | | | | module (GH-113709) (#113733) (cherry picked from commit 3003fbbf00422bce6e327646063e97470afa9091)
* [3.12] gh-113602: Bail out when the parser tries to override existing errors ↵Miss Islington (bot)2024-01-021-0/+4
| | | | | | | | | (GH-113607) (#113652) gh-113602: Bail out when the parser tries to override existing errors (GH-113607) (cherry picked from commit 9ed36d533ab8b256f0a589b5be6d7a2fdcf4aff2) Signed-off-by: Pablo Galindo <pablogsal@gmail.com> Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.12] gh-106905: Use separate structs to track recursion depth in each ↵Serhiy Storchaka2023-12-251-26/+32
| | | | | | | | | PyAST_mod2obj call. (GH-113035) (GH-113472) (cherry picked from commit 48c49739f5502fc7aa82f247ab2e4d7b55bdca62) Co-authored-by: Yilei Yang <yileiyang@google.com> Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
* [3.12] gh-112943: Correctly compute end offsets for multiline tokens in the ↵Pablo Galindo Salgado2023-12-112-5/+12
| | | | | tokenize module (GH-112949) (#112957) (cherry picked from commit a135a6d2c6d503b186695f01efa7eed65611b04e)
* [3.12] gh-112388: Fix an error that was causing the parser to try to ↵Miss Islington (bot)2023-11-271-0/+4
| | | | | | | | | overwrite tokenizer errors (GH-112410) (#112466) gh-112388: Fix an error that was causing the parser to try to overwrite tokenizer errors (GH-112410) (cherry picked from commit 2c8b19174274c183eb652932871f60570123fe99) Signed-off-by: Pablo Galindo <pablogsal@gmail.com> Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.12] gh-112387: Fix error positions for decoded strings with backwards ↵Miss Islington (bot)2023-11-271-0/+4
| | | | | | | | | tokenize errors (GH-112409) (#112468) gh-112387: Fix error positions for decoded strings with backwards tokenize errors (GH-112409) (cherry picked from commit 45d648597b1146431bf3d91041e60d7f040e70bf) Signed-off-by: Pablo Galindo <pablogsal@gmail.com> Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.12] gh-112243: Don't include comments in f-string debug expressions ↵Pablo Galindo Salgado2023-11-201-5/+49
| | | | | (GH-112284) (#112285) (cherry picked from commit d59feb5dbe5395615d06c30a95e6a6a9b7681d4d)
* [3.12] gh-111380: Show SyntaxWarnings only once when parsing if invalid ↵Miss Islington (bot)2023-10-271-0/+5
| | | | | | | | syntax is encouintered (GH-111381) (#111382) gh-111380: Show SyntaxWarnings only once when parsing if invalid syntax is encouintered (GH-111381) (cherry picked from commit 3d2f1f0b830d86f16f42c42b54d3ea4453dac318) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.12] bpo-43950: handle wide unicode characters in tracebacks (GH-28150) ↵Pablo Galindo Salgado2023-10-262-0/+56
| | | | (#111346)
* [3.12] gh-110938: Fix error messages for indented blocks with functio… ↵Jelle Zijlstra2023-10-181-19/+31
| | | | | | | | (#110990) [3.12] gh-110938: Fix error messages for indented blocks with functions and classes with generic type parameters (GH-110973) (cherry picked from commit 24e4ec7766fd471deb5b7e5087f0e7dba8576cfb) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.12] gh-107450: Check for overflow in the tokenizer and fix overflow test ↵Lysandros Nikolaou2023-10-162-6/+9
| | | | | | | | | (GH-110832) (#110931) (cherry picked from commit a1ac5590e0f8fe008e5562d22edab65d0c1c5507) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> Co-authored-by: Filipe Laíns <lains@riseup.net> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* [3.12] gh-110696: Fix incorrect syntax error message for incorrect argument ↵Pablo Galindo Salgado2023-10-121-1133/+1485
| | | | | unpacking (GH-110706) (#110765) (cherry picked from commit 3d180347ae73119bb51500efeeafdcd62bcc6f78)
* [3.12] gh-107450: Raise OverflowError when parser column offset overflows ↵Miss Islington (bot)2023-10-121-0/+6
| | | | | | | (GH-110754) (#110762) (cherry picked from commit fb7843ee895ac7f6eeb58f356b1a320eea081cfc) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
* [3.12] gh-110259: Fix f-strings with multiline expressions and format specs ↵Miss Islington (bot)2023-10-051-6/+18
| | | | | | | | | (GH-110271) (#110396) gh-110259: Fix f-strings with multiline expressions and format specs (GH-110271) (cherry picked from commit cc389ef627b2a486ab89d9a11245bef48224efb1) Signed-off-by: Pablo Galindo <pablogsal@gmail.com> Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.12] gh-109596: Ensure repeated rules in the grammar are not allowed and ↵Pablo Galindo Salgado2023-10-022-1208/+1180
| | | | | fix incorrect soft keywords (GH-109606). (#109752) (cherry picked from commit b28ffaa193efc66f46ab90d383279174a11a11d7)
* [3.12] gh-88943: Improve syntax error for non-ASCII character that follows a ↵Miss Islington (bot)2023-10-021-1/+1
| | | | | | | | | | numerical literal (GH-109081) (#109090) gh-88943: Improve syntax error for non-ASCII character that follows a numerical literal (GH-109081) It now points on the invalid non-ASCII character, not on the valid numerical literal. (cherry picked from commit b2729e93e9d73503b1fda4ea4fecd77c58909091) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* [3.12] gh-109114: Relax the check for invalid lambdas inside f-strings to ↵Pablo Galindo Salgado2023-09-121-1124/+1065
| | | | | | | | avoid false positives (GH-109121). (#109155) (cherry picked from commit 5bda2f637e1cfbca45a83aa6e22db25498064b27) Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
* [3.12] gh-108179: Add error message for parser stack overflows (GH-108256) ↵Miss Islington (bot)2023-08-223-1060/+540
| | | | | | | | (#108263) gh-108179: Add error message for parser stack overflows (GH-108256) (cherry picked from commit 86617518c4ac824e2b6dc20691ba5a08df04f285) Co-authored-by: Dennis Sweeney <36520290+sweeneyde@users.noreply.github.com>
* [3.12] gh-107967: Fix infinite recursion on invalid escape sequence warning ↵Miss Islington (bot)2023-08-151-0/+3
| | | | | | | | (GH-107968) (#107970) gh-107967: Fix infinite recursion on invalid escape sequence warning (GH-107968) (cherry picked from commit d66bc9e8a7a8d6774d912a4b9d151885c4d8de1d) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
* [3.12] GH-107263: Increase C stack limit for most functions, except ↵Miss Islington (bot)2023-08-041-1/+1
| | | | | | | | | | `_PyEval_EvalFrameDefault()` (GH-107535) (#107618) GH-107263: Increase C stack limit for most functions, except `_PyEval_EvalFrameDefault()` (GH-107535) * Set C recursion limit to 1500, set cost of eval loop to 2 frames, and compiler mutliply to 2. (cherry picked from commit fa45958450aa3489607daf9855ca0474a2a20878) Co-authored-by: Mark Shannon <mark@hotpy.org>
* [3.12] gh-106989: Remove tok report warnings (GH-106993) (#107013)Miss Islington (bot)2023-07-222-6/+0
| | | | Co-authored-by: Menelaos Kotoglou <contact@menelaoskotoglou.com>
* [3.12] gh-106396: Special-case empty format spec to gen empty JoinedStr node ↵Miss Islington (bot)2023-07-041-0/+12
| | | | | | | (GH-106401) (#106416) (cherry picked from commit dfe4de203881e8d068e6fc5b8e31075841a86d25) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
* [3.12] gh-106145: Make `end_{lineno,col_offset}` required on `type_param` ↵Miss Islington (bot)2023-07-011-1/+1
| | | | | | | | nodes (GH-106224) (#106295) gh-106145: Make `end_{lineno,col_offset}` required on `type_param` nodes (GH-106224) (cherry picked from commit 46c1097868745eeb47abbc8af8c34e8fcb80ff1d) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
* [3.12] gh-98931: Add custom error messages to invalid import/from with ↵Miss Islington (bot)2023-06-221-596/+719
| | | | | | | multiple targets (GH-105985) (#105991) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
* [3.12] gh-105938: Emit a SyntaxWarning for escaped braces in an f-string ↵Miss Islington (bot)2023-06-202-4/+9
| | | | | | | (GH-105939) (#105941) (cherry picked from commit 6586cee27f32f0354fe4e77c7b8c6e399329b5e2) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
* [3.12] GH-105588: Add missing error checks to some obj2ast_* converters ↵Miss Islington (bot)2023-06-151-0/+1
| | | | | | | | (GH-105838) GH-105588: Add missing error checks to some obj2ast_* converters (GH-105589) (cherry picked from commit a4056c8f9c2d9970d39e3cb6bffb255cd4b8a42c) Co-authored-by: Brandt Bucher <brandtbucher@microsoft.com>
* [3.12] gh-105820: Fix tok_mode expression buffer in file & readline ↵Miss Islington (bot)2023-06-151-3/+8
| | | | | | | tokenizer (GH-105828) (#105832) (cherry picked from commit d382ad49157b3802fc5619f68d96810def517869) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
* [3.12] gh-105800: Issue SyntaxWarning in f-strings for invalid escape ↵Miss Islington (bot)2023-06-151-3/+3
| | | | | | sequences (GH-105801) (#105806) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.12] gh-105718: Fix buffer allocation in tokenizer with readline ↵Miss Islington (bot)2023-06-132-5/+1
| | | | | | (GH-105728) (#105729) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
* [3.12] gh-105549: Tokenize separately NUMBER and NAME tokens and allow ↵Miss Islington (bot)2023-06-091-3/+10
| | | | | | 0-prefixed literals (GH-105555) (#105602) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.12] gh-105259: Ensure we don't show newline characters for trailing ↵Miss Islington (bot)2023-06-062-0/+8
| | | | NEWLINE tokens (GH-105364) (#105367)
* [3.12] gh-105194: Fix format specifier escaped characters in f-strings ↵Miss Islington (bot)2023-06-023-1/+21
| | | | (GH-105231) (#105234)
* [3.12] gh-104799: Default missing lists in AST to the empty list (GH-104834) ↵Miss Islington (bot)2023-06-021-20/+29
| | | | | | | | (#105213) (cherry picked from commit 77d25795862f19c6e3d647b76cfb10d5ce1f149c) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
* [3.12] gh-105069: Add a readline-like callable to the tokenizer to consume ↵Miss Islington (bot)2023-05-312-3/+135
| | | | | | | | input iteratively (GH-105070) (#105119) gh-105069: Add a readline-like callable to the tokenizer to consume input iteratively (GH-105070) (cherry picked from commit 9216e69a87d16d871625721ed5a8aa302511f367) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.12] gh-105042: Disable unmatched parens syntax error in python tokenize ↵Miss Islington (bot)2023-05-311-32/+33
| | | | | | | | (GH-105061) (#105120) gh-105042: Disable unmatched parens syntax error in python tokenize (GH-105061) (cherry picked from commit 70f315c2d6de87b0514ce16cc00a91a5b60a6098) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
* [3.12] gh-104799: Move location of type_params AST fields (GH-104828) (#104974)Miss Islington (bot)2023-05-303-20/+24
| | | | | | | gh-104799: Move location of type_params AST fields (GH-104828) (cherry picked from commit ba73473f4c18ba4cf7ab18d84d94a47d2d37a0c5) Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com> Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com>
* [3.12] gh-105017: Include CRLF lines in strings and column numbers ↵Miss Islington (bot)2023-05-283-16/+30
| | | | | | | | | (GH-105030) (#105041) gh-105017: Include CRLF lines in strings and column numbers (GH-105030) (cherry picked from commit 96fff35325e519cc76ffacf22e57e4c393d4446f) Co-authored-by: Marta Gómez Macías <mgmacias@google.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
generated by cgit v0.12 at 2026-01-05 07:49:47 (GMT)