summaryrefslogtreecommitdiffstats
path: root/Parser
Commit message (Collapse)AuthorAgeFilesLines
* [3.11] gh-102310: Change error range for invalid bytes literals (GH-103663) ↵Miss Islington (bot)2023-04-231-1/+2
| | | | (#103703)
* [3.11] GH-102711: Fix warnings found by clang (GH-102712) (#103075)Miss Islington (bot)2023-03-281-2/+2
| | | | | | | | | | | | | | | | | | | | There are some warnings if build python via clang: Parser/pegen.c:812:31: warning: a function declaration without a prototype is deprecated in all versions of C [-Wstrict-prototypes] _PyPegen_clear_memo_statistics() ^ void Parser/pegen.c:820:29: warning: a function declaration without a prototype is deprecated in all versions of C [-Wstrict-prototypes] _PyPegen_get_memo_statistics() ^ void Fix it to make clang happy. (cherry picked from commit 7703def37e4fa7d25c3d23756de8f527daa4e165) Signed-off-by: Chenxi Mao <chenxi.mao@suse.com> Co-authored-by: Chenxi Mao <chenxi.mao@suse.com>
* [3.11] gh-102416: Do not memoize incorrectly loop rules in the parser ↵Pablo Galindo Salgado2023-03-061-216/+0
| | | | (GH-102467). (#102473)
* [3.11] gh-101046: Fix a potential memory leak in the parser when raising ↵Pablo Galindo Salgado2023-01-161-0/+108
| | | | | MemoryError (GH-101051) (#101085) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* gh-100050: Fix an assertion error when raising unclosed parenthesis errors ↵Miss Islington (bot)2022-12-071-0/+4
| | | | | | | | in the tokenizer (GH-100065) (cherry picked from commit 97e7004cfe48305bcd642c653b406dc7470e196d) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com> Automerge-Triggered-By: GH:pablogsal
* [3.11] gh-99891: Fix infinite recursion in the tokenizer when showing ↵Pablo Galindo Salgado2022-12-012-0/+9
| | | | | | | | warnings (GH-99893) (GH-99896) Automerge-Triggered-By: GH:pablogsal. (cherry picked from commit 417206a05c4545bde96c2bbbea92b53e6cac0d48) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* gh-99581: Fix a buffer overflow in the tokenizer when copying lines that ↵Miss Islington (bot)2022-11-201-1/+6
| | | | | | | fill the available buffer (GH-99605) (cherry picked from commit e13d1d9dda8c27691180bc618bd5e9bf43dfa89f) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.11] gh-99211: Point to except/except* on syntax errors when mixing them ↵Lysandros Nikolaou2022-11-201-671/+713
| | | | | | | (GH-99215) (GH-99622) gh-99211: Point to except/except* on syntax errors when mixing them (GH-99215) (cherry picked from commit 9c4232ae8972a33f84e875cfdd866318a1233e47)
* [3.11] gh-99153: set location on SyntaxError for try with both except and ↵Irit Katriel2022-11-071-3/+3
| | | | except* (GH-99160) (#99168)
* [3.11] gh-96587: Raise `SyntaxError` for PEP654 on older `feature_version` ↵Nikita Sobolev2022-10-051-1/+1
| | | | | | | (GH-96588) (#96591) (cherry picked from commit 2c7d2e8d46164efb6e27a64081d8e949f6876515) Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
* gh-96678: Fix UB of null pointer arithmetic (GH-96782)Miss Islington (bot)2022-09-131-1/+1
| | | | | | Automerge-Triggered-By: GH:pablogsal (cherry picked from commit 81e36f350b75d2ed2668825f7df6e059b57f859c) Co-authored-by: Matthias Görgens <matthias.goergens@gmail.com>
* gh-96268: Fix loading invalid UTF-8 (GH-96270)Miss Islington (bot)2022-09-071-13/+45
| | | | | | | | This makes tokenizer.c:valid_utf8 match stringlib/codecs.h:decode_utf8. It also fixes an off-by-one error introduced in 3.10 for the line number when the tokenizer reports bad UTF8. (cherry picked from commit 8bc356a7dd50cbdb46d10b8c7e457832431f5d9e) Co-authored-by: Michael Droettboom <mdboom@gmail.com>
* gh-96611: Fix error message for invalid UTF-8 in mid-multiline string (GH-96623)Miss Islington (bot)2022-09-061-0/+2
| | | | | (cherry picked from commit 05692c67c51b78a5a5a7bb61d646519025e38015) Co-authored-by: Michael Droettboom <mdboom@gmail.com>
* [3.11] gh-95778: CVE-2020-10735: Prevent DoS by very large int() (#96500)Gregory P. Smith2022-09-021-0/+23
| | | | | | | | | | | | | | | | | | Integer to and from text conversions via CPython's bignum `int` type is not safe against denial of service attacks due to malicious input. Very large input strings with hundred thousands of digits can consume several CPU seconds. This PR comes fresh from a pile of work done in our private PSRT security response team repo. This backports https://github.com/python/cpython/pull/96499 aka 511ca9452033ef95bc7d7fc404b8161068226002 Signed-off-by: Christian Heimes [Red Hat] <christian@python.org> Tons-of-polishing-up-by: Gregory P. Smith [Google] <greg@krypto.org> Reviews via the private PSRT repo via many others (see the NEWS entry in the PR). <!-- gh-issue-number: gh-95778 --> * Issue: gh-95778 <!-- /gh-issue-number --> I wrote up [a one pager for the release managers](https://docs.google.com/document/d/1KjuF_aXlzPUxTK4BMgezGJ2Pn7uevfX7g0_mvgHlL7Y/edit#).
* [3.11] gh-94996: Disallow lambda pos only params with feature_version < (3, ↵Shantanu2022-08-121-2/+2
| | | | | | | | | 8) (GH-95934) (GH-95936) (cherry picked from commit a965db37f27ffb232312bc13d9a509f0d93fcd20) Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com> Automerge-Triggered-By: GH:lysnikolaou
* gh-94996: Disallow parsing pos only params with feature_version < (3, 8) ↵Miss Islington (bot)2022-08-121-2/+2
| | | | | | | (GH-94997) (cherry picked from commit b5e3ea286289fcad12be78480daf3756e350f69f) Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
* gh-95876: Fix format string in pegen error location code (GH-95877)Miss Islington (bot)2022-08-111-1/+1
| | | | | (cherry picked from commit b4c857d0fd74abb1ede6fe083c4fa3ca728b2b83) Co-authored-by: Christian Heimes <christian@python.org>
* gh-95355: Check tokens[0] after allocating memory (GH-95356)Miss Islington (bot)2022-07-281-1/+1
| | | | | | | | GH-95355 Automerge-Triggered-By: GH:pablogsal (cherry picked from commit b946f529efb4a623ac4ad968d8091edb81ebdcdb) Co-authored-by: Honglin Zhu <zhuhonglin.zhl@alibaba-inc.com>
* [3.11] gh-95185: Check recursion depth in the AST constructor (GH-95186) ↵Miss Islington (bot)2022-07-261-1/+36
| | | | | | | | (GH-95208) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com> (cherry picked from commit 00474472944944b346d8409cfded84bb299f601a) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* gh-94949: Disallow parsing parenthesised ctx mgr with old feature_version ↵Miss Islington (bot)2022-07-181-1/+1
| | | | | | | | | | | | | | (GH-94950) * gh-94949: Disallow parsing parenthesised ctx manager with old feature_version * 📜🤖 Added by blurb_it. * Allow it with feature_version=(3, 9) as well Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> (cherry picked from commit 0daba822212cd5d6c63384a27f390f0945330c2b) Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
* gh-94947: Disallow parsing walrus with feature_version < (3, 8) (GH-94948)Miss Islington (bot)2022-07-181-1/+1
| | | | | | | | | | | | * gh-94947: Disallow parsing walrus with feature_version < (3, 8) * oops, commit the parser * 📜🤖 Added by blurb_it. Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> (cherry picked from commit ae0be5a53bb4caee3de4888341addd9c94133f2d) Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
* gh-94869: Fix the location in some expressions for multi-line f-string ast ↵Miss Islington (bot)2022-07-161-1/+4
| | | | | | | nodes (GH-94895) (cherry picked from commit 2e9da8e3522764d09f1d6054a2be567e91a30812) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* gh-94360: Fix a tokenizer crash when reading encoded files with syntax ↵Miss Islington (bot)2022-07-052-5/+13
| | | | | | | | | | | | | | errors from stdin (GH-94386) * gh-94360: Fix a tokenizer crash when reading encoded files with syntax errors from stdin Signed-off-by: Pablo Galindo <pablogsal@gmail.com> * nitty nit Co-authored-by: Łukasz Langa <lukasz@langa.pl> (cherry picked from commit 36fcde61ba48c4e918830691ecf4092e4e3b9b99) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* gh-94192: Fix error for dictionary literals with invalid expression as ↵Miss Islington (bot)2022-06-261-261/+326
| | | | | | | | | | value. (GH-94304) * Fix error for dictionary literals with invalid expression as value. * Remove trailing whitespace (cherry picked from commit 8c237a7a71d52f996f58dc58f6b6ce558d209494) Co-authored-by: wookie184 <wookie1840@gmail.com>
* [3.11] gh-92858: Improve error message for some suites with syntax error ↵Pablo Galindo Salgado2022-06-231-362/+500
| | | | | | | | | before ':' (GH-92894) (#94180) (cherry picked from commit 2fc83ac3afa161578200dbf8d823a20e0801c0c0) Co-authored-by: wookie184 <wookie1840@gmail.com> Co-authored-by: wookie184 <wookie1840@gmail.com>
* gh-93671: Avoid exponential backtracking in deeply nested sequence patterns ↵Miss Islington (bot)2022-06-101-0/+10
| | | | | | | | in match statements (GH-93680) Co-authored-by: Łukasz Langa <lukasz@langa.pl> (cherry picked from commit 53a8b17895e91d08f76a2fb59a555d012cd85ab4) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* gh-93418: Fix an assert when an f-string expression is followed by an '=', ↵Miss Islington (bot)2022-06-021-1/+3
| | | | | | | | | but no closing brace. (gh-93419) (gh-93422) (cherry picked from commit ee70c70aa93d7a41cbe47a0b361b17f9d7ec8acd) Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com> Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
* gh-92597: Ensure that AST nodes without explicit end positions can be ↵Miss Islington (bot)2022-05-311-1/+13
| | | | | | | compiled (GH-93359) (cherry picked from commit 705eaec28f7bee530b1c1635ba385a49a1feaf32) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* gh-90473: Decrease recursion limit and skip tests on WASI (GH-92803)Miss Islington (bot)2022-05-191-1/+5
| | | | | (cherry picked from commit 137fd3d88aa46669f5717734e823f4c594ab2843) Co-authored-by: Christian Heimes <christian@python.org>
* Use static inline function Py_EnterRecursiveCall() (#91988)Victor Stinner2022-05-041-5/+6
| | | | | | | | | | | | | | | | Currently, calling Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() may use a function call or a static inline function call, depending if the internal pycore_ceval.h header file is included or not. Use a different name for the static inline function to ensure that the static inline function is always used in Python internals for best performance. Similar approach than PyThreadState_GET() (function call) and _PyThreadState_GET() (static inline function). * Rename _Py_EnterRecursiveCall() to _Py_EnterRecursiveCallTstate() * Rename _Py_LeaveRecursiveCall() to _Py_LeaveRecursiveCallTstate() * pycore_ceval.h: Rename Py_EnterRecursiveCall() to _Py_EnterRecursiveCall() and Py_LeaveRecursiveCall() and _Py_LeaveRecursiveCall()
* gh-81548: Deprecate octal escape sequences with value larger than 0o377 ↵Serhiy Storchaka2022-04-301-7/+18
| | | | (GH-91668)
* gh-87999: Change warning type for numeric literal followed by keyword (GH-91980)Serhiy Storchaka2022-04-271-4/+6
| | | | | The warning emitted by the Python parser for a numeric literal immediately followed by keyword has been changed from deprecation warning to syntax warning.
* bpo-47212: Improve error messages for un-parenthesized generator expressions ↵Matthieu Dartiailh2022-04-053-4/+5
| | | | (GH-32302)
* bpo-46315: Use fopencookie only on Emscripten 3.x and newer (GH-32266)Christian Heimes2022-04-021-1/+1
|
* bpo-47126: Update to canonical PEP URLs specified by PEP 676 (GH-32124)Hugo van Kemenade2022-03-301-1/+1
|
* bpo-47129: Add more informative messages to f-string syntax errors (32127)Maciej Górski2022-03-281-0/+5
| | | | | | | | | | | | | * Add more informative messages to f-string syntax errors * 📜🤖 Added by blurb_it. * Fix whitespaces * Change error message * Remove the 'else' statement (as sugested in review) Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
* bpo-43224: Implement PEP 646 grammar changes (GH-31018)Matthew Rahtz2022-03-261-2101/+2465
| | | Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
* bpo-47117: Don't crash if we fail to decode characters when the tokenizer ↵Pablo Galindo Salgado2022-03-261-2/+7
| | | | | buffers are uninitialized (GH-32129) Automerge-Triggered-By: GH:pablogsal
* bpo-46315: Use fopencookie() to avoid dup() in ↵Christian Heimes2022-03-221-6/+34
| | | | | _PyTokenizer_FindEncodingFilename (GH-32033) WASI does not have dup() and Emscripten's emulation is slow.
* bpo-46838: Syntax error improvements for function definitions (GH-31590)Pablo Galindo Salgado2022-03-221-867/+3150
|
* bpo-46920: Remove code that has explainers why it was disabled (GH-31813)Oleg Iarygin2022-03-141-24/+0
|
* bpo-46920: Remove disabled debug code added decades ago and likely ↵Oleg Iarygin2022-03-141-11/+0
| | | | unnecessary (GH-31812)
* bpo-46820: Fix a SyntaxError in a numeric literal followed by "not in" ↵Serhiy Storchaka2022-02-221-0/+3
| | | | | | | (GH-31479) Fix parsing a numeric literal immediately (without spaces) followed by "not in" keywords, like in "1not in x". Now the parser only emits a warning, not a syntax error.
* bpo-46762: Fix an assert failure in f-strings where > or < is the last ↵Eric V. Smith2022-02-161-10/+10
| | | | character if the f-string is missing a trailing right brace. (#31365)
* Don't print rejected tokens when using the debug flags in the parser (GH-31258)Pablo Galindo Salgado2022-02-101-1/+0
|
* Allow the parser to avoid nested processing of invalid rules (GH-31252)Pablo Galindo Salgado2022-02-103-1044/+1033
|
* bpo-46707: Avoid potential exponential backtracking in some syntax errors ↵Pablo Galindo Salgado2022-02-101-1015/+1020
| | | | (GH-31241)
* bpo-46541: Replace core use of _Py_IDENTIFIER() with statically initialized ↵Eric Snow2022-02-081-9/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | global objects. (gh-30928) We're no longer using _Py_IDENTIFIER() (or _Py_static_string()) in any core CPython code. It is still used in a number of non-builtin stdlib modules. The replacement is: PyUnicodeObject (not pointer) fields under _PyRuntimeState, statically initialized as part of _PyRuntime. A new _Py_GET_GLOBAL_IDENTIFIER() macro facilitates lookup of the fields (along with _Py_GET_GLOBAL_STRING() for non-identifier strings). https://bugs.python.org/issue46541#msg411799 explains the rationale for this change. The core of the change is in: * (new) Include/internal/pycore_global_strings.h - the declarations for the global strings, along with the macros * Include/internal/pycore_runtime_init.h - added the static initializers for the global strings * Include/internal/pycore_global_objects.h - where the struct in pycore_global_strings.h is hooked into _PyRuntimeState * Tools/scripts/generate_global_objects.py - added generation of the global string declarations and static initializers I've also added a --check flag to generate_global_objects.py (along with make check-global-objects) to check for unused global strings. That check is added to the PR CI config. The remainder of this change updates the core code to use _Py_GET_GLOBAL_IDENTIFIER() instead of _Py_IDENTIFIER() and the related _Py*Id functions (likewise for _Py_GET_GLOBAL_STRING() instead of _Py_static_string()). This includes adding a few functions where there wasn't already an alternative to _Py*Id(), replacing the _Py_Identifier * parameter with PyObject *. The following are not changed (yet): * stop using _Py_IDENTIFIER() in the stdlib modules * (maybe) get rid of _Py_IDENTIFIER(), etc. entirely -- this may not be doable as at least one package on PyPI using this (private) API * (maybe) intern the strings during runtime init https://bugs.python.org/issue46541
* bpo-46521: Fix codeop to use a new partial-input mode of the parser (GH-31010)Pablo Galindo Salgado2022-02-083-11/+31
|
* bpo-14916: use specified tokenizer fd for file input (GH-31006)Paul m. p. P2022-02-011-1/+1
| | | | | | | | | | @pablogsal, sorry i failed to rebase to main, so i recreated https://github.com/python/cpython/pull/22190#issuecomment-1024633392 > PyRun_InteractiveOne\*() functions allow to explicitily set fd instead of stdin. but stdin was hardcoded in readline call. > This patch does not fix target file for prompt unlike original bpo one : prompt fd is unrelated to tokenizer source which could be read only. It is more of a bugfix regarding the docs : actual documentation say "prompt the user" so one would expect prompt to go on stdout not a file for both PyRun_InteractiveOne\*() and PyRun_InteractiveLoop\*(). Automerge-Triggered-By: GH:pablogsal
generated by cgit v0.12 at 2024-11-27 21:43:53 (GMT)