summaryrefslogtreecommitdiffstats
path: root/Parser
Commit message (Collapse)AuthorAgeFilesLines
* [3.10] gh-95778: CVE-2020-10735: Prevent DoS by very large int() (#96501)Gregory P. Smith2022-09-021-0/+23
| | | | | | | | | | | | | | | | | Integer to and from text conversions via CPython's bignum `int` type is not safe against denial of service attacks due to malicious input. Very large input strings with hundred thousands of digits can consume several CPU seconds. This PR comes fresh from a pile of work done in our private PSRT security response team repo. This backports https://github.com/python/cpython/pull/96499 aka 511ca9452033ef95bc7d7fc404b8161068226002 Signed-off-by: Christian Heimes [Red Hat] <christian@python.org> Tons-of-polishing-up-by: Gregory P. Smith [Google] <greg@krypto.org> Reviews via the private PSRT repo via many others (see the NEWS entry in the PR). <!-- gh-issue-number: gh-95778 --> * Issue: gh-95778 <!-- /gh-issue-number --> I wrote up [a one pager for the release managers](https://docs.google.com/document/d/1KjuF_aXlzPUxTK4BMgezGJ2Pn7uevfX7g0_mvgHlL7Y/edit#).
* [3.10] gh-94996: Disallow lambda pos only params with feature_version < (3, ↵Shantanu2022-08-121-2/+2
| | | | | | | | | 8) (GH-95934) (GH-95938) (cherry picked from commit a965db37f27ffb232312bc13d9a509f0d93fcd20) Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com> Automerge-Triggered-By: GH:lysnikolaou
* [3.10] gh-94996: Disallow parsing pos only params with feature_version < (3, ↵Shantanu2022-08-121-2/+2
| | | | | | | | | 8) (GH-95935) (cherry picked from commit https://github.com/python/cpython/commit/b5e3ea286289fcad12be78480daf3756e350f69f) Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com> Automerge-Triggered-By: GH:lysnikolaou
* [3.10] gh-95876: Fix format string in pegen error location code (GH-95877 ↵Christian Heimes2022-08-111-1/+1
| | | | | | | (GH-95901) (cherry picked from commit b4c857d0fd74abb1ede6fe083c4fa3ca728b2b83) Co-authored-by: Christian Heimes <christian@python.org>
* gh-95355: Check tokens[0] after allocating memory (GH-95356)Miss Islington (bot)2022-07-281-1/+1
| | | | | | | | GH-95355 Automerge-Triggered-By: GH:pablogsal (cherry picked from commit b946f529efb4a623ac4ad968d8091edb81ebdcdb) Co-authored-by: Honglin Zhu <zhuhonglin.zhl@alibaba-inc.com>
* [3.10] gh-94949: Disallow parsing parenthesised ctx mgr with old ↵Shantanu2022-07-191-1/+1
| | | | | | | feature_version (GH-94950) (#94990) (cherry picked from commit 0daba822212cd5d6c63384a27f390f0945330c2b) Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
* [3.10] gh-94947: Disallow parsing walrus with feature_version < (3, 8) ↵Shantanu2022-07-181-1/+1
| | | | | | | | | | | | | | (GH-94948) (#94969) * gh-94947: Disallow parsing walrus with feature_version < (3, 8) * oops, commit the parser * 📜🤖 Added by blurb_it. Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>. (cherry picked from commit ae0be5a53bb4caee3de4888341addd9c94133f2d) Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com>
* gh-94869: Fix the location in some expressions for multi-line f-string ast ↵Miss Islington (bot)2022-07-161-1/+4
| | | | | | | | | nodes (GH-94895) (#94911) (cherry picked from commit 2e9da8e3522764d09f1d6054a2be567e91a30812) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com> Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.10] gh-94360: Fix a tokenizer crash when reading encoded files with ↵Pablo Galindo Salgado2022-07-052-5/+13
| | | | | | | | | | | syntax errors from stdin (GH-94386) (GH-94574) Signed-off-by: Pablo Galindo <pablogsal@gmail.com> Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com> Co-authored-by: Łukasz Langa <lukasz@langa.pl> (cherry picked from commit 36fcde61ba48c4e918830691ecf4092e4e3b9b99)
* [3.10] gh-94192: Fix error for dictionary literals with invalid expression ↵wookie1842022-06-271-217/+282
| | | | | as value. (GH-94304) (#94344) Co-authored-by: wookie184 <wookie1840@gmail.com>
* [3.10] gh-92858: Improve error message for some suites with syntax error ↵Pablo Galindo Salgado2022-06-231-293/+431
| | | | | | | | | before ':' (GH-92894). (#94183) (cherry picked from commit 2fc83ac3afa161578200dbf8d823a20e0801c0c0) Co-authored-by: wookie184 <wookie1840@gmail.com> Co-authored-by: wookie184 <wookie1840@gmail.com>
* [3.10] gh-93671: Avoid exponential backtracking in deeply nested sequence ↵Pablo Galindo Salgado2022-06-101-0/+10
| | | | | | | | patterns in match statements (GH-93680) (#93690) Co-authored-by: Łukasz Langa <lukasz@langa.pl>. (cherry picked from commit 53a8b17895e91d08f76a2fb59a555d012cd85ab4) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* gh-93418: Fix an assert when an f-string expression is followed by an '=', ↵Miss Islington (bot)2022-06-021-1/+3
| | | | | | | | | but no closing brace. (gh-93419) (gh-93423) (cherry picked from commit ee70c70aa93d7a41cbe47a0b361b17f9d7ec8acd) Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com> Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
* [3.10] Backport bpo-47212 (GH-32302) to Python 3.10 (GH-32334)Matthieu Dartiailh2022-04-053-4/+5
| | | | | | | | | (cherry picked from commit aa0f056a00c4bcaef83d729e042359ddae903382) # Conflicts: # Grammar/python.gram # Parser/action_helpers.c Automerge-Triggered-By: GH:pablogsal
* [3.10] bpo-47117: Don't crash if we fail to decode characters when the ↵Pablo Galindo Salgado2022-03-261-2/+7
| | | | | | | | tokenizer buffers are uninitialized (GH-32129) (GH-32130) Automerge-Triggered-By: GH:pablogsal. (cherry picked from commit 26cca8067bf5306e372c0e90036d832c5021fd90) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* bpo-46820: Fix a SyntaxError in a numeric literal followed by "not in" ↵Miss Islington (bot)2022-02-221-0/+3
| | | | | | | | | | | (GH-31479) (GH-31493) Fix parsing a numeric literal immediately (without spaces) followed by "not in" keywords, like in "1not in x". Now the parser only emits a warning, not a syntax error. (cherry picked from commit 090e5c4b946b28f50fce445916c5d3ec45c8f45f) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* bpo-46762: Fix an assert failure in f-strings where > or < is the last ↵Miss Islington (bot)2022-02-161-10/+10
| | | | | | | character if the f-string is missing a trailing right brace. (GH-31365) (cherry picked from commit ffd9f8ff84ed53c956b16d027f7d2926ea631051) Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
* [3.10] Allow the parser to avoid nested processing of invalid rules ↵Pablo Galindo Salgado2022-02-103-912/+900
| | | | | | | (GH-31252). (GH-31257) (cherry picked from commit 390459de6db1e68b79c0897cc88c0d562693ec5c) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.10] bpo-46707: Avoid potential exponential backtracking in some syntax ↵Pablo Galindo Salgado2022-02-101-883/+888
| | | | | | | errors (GH-31241). (GH-31242) (cherry picked from commit b71dc71905ab674ccaa4a56230d17a28f61c325c) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.10] bpo-46521: Fix codeop to use a new partial-input mode of the parser ↵Pablo Galindo Salgado2022-02-083-10/+34
| | | | | | | (GH-31010). (GH-31213) (cherry picked from commit 69e10976b2e7682c6d57f4272932ebc19f8e8859) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* bpo-14916: use specified tokenizer fd for file input (GH-31006)Miss Islington (bot)2022-02-031-1/+1
| | | | | | | | | | | | | @pablogsal, sorry i failed to rebase to main, so i recreated https://github.com/python/cpython/pull/22190GH-issuecomment-1024633392 > PyRun_InteractiveOne\*() functions allow to explicitily set fd instead of stdin. but stdin was hardcoded in readline call. > This patch does not fix target file for prompt unlike original bpo one : prompt fd is unrelated to tokenizer source which could be read only. It is more of a bugfix regarding the docs : actual documentation say "prompt the user" so one would expect prompt to go on stdout not a file for both PyRun_InteractiveOne\*() and PyRun_InteractiveLoop\*(). Automerge-Triggered-By: GH:pablogsal (cherry picked from commit 89b13042fcfc95bae21a49806a205ef62f1cdd73) Co-authored-by: Paul m. p. P <mail.peny@free.fr>
* [3.10] bpo-46091: Correctly calculate indentation levels for whitespace ↵Pablo Galindo Salgado2022-01-251-13/+33
| | | | | | | lines with continuation characters (GH-30130). (GH-30898) (cherry picked from commit a0efc0c1960e2c49e0092694d98395555270914c) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* bpo-46503: Prevent an assert from firing when parsing some invalid \N ↵Miss Islington (bot)2022-01-251-2/+14
| | | | | | | | | | | | sequences in f-strings. (GH-30865) (GH-30866) * bpo-46503: Prevent an assert from firing. Also fix one nearby tiny PEP-7 nit. * Added blurb. (cherry picked from commit 0daf72194bd4e31de7f12020685bb39a14d6f45e) Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com> Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
* [3.10] bpo-46240: Correct the error for unclosed parentheses when the ↵Pablo Galindo Salgado2022-01-231-1/+2
| | | | | | | tokenizer is not finished (GH-30378). (GH-30819) (cherry picked from commit 70f415fb8b632247e28d87998642317ca7a652ae) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.10] bpo-46339: Fix crash in the parser when computing error text for ↵Miss Islington (bot)2022-01-202-427/+10
| | | | | | | | | | | | | | multi-line f-strings (GH-30529) (GH-30542) * bpo-46339: Fix crash in the parser when computing error text for multi-line f-strings (GH-30529) Automerge-Triggered-By: GH:pablogsal (cherry picked from commit cedec19be81e6bd153678bfb28c8e217af8bda58) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com> * Fix interactive mode Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* bpo-46237: Fix the line number of tokenizer errors inside f-strings (GH-30463)Miss Islington (bot)2022-01-112-5/+8
| | | | | (cherry picked from commit 6fa8b2ceee38187b0ae96aee12fe4f0a5c8a2ce7) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* bpo-46289: Make conversion of FormattedValue not optional on ASDL (GH-30467)Miss Islington (bot)2022-01-071-1/+1
| | | | | | Automerge-Triggered-By: GH:isidentical (cherry picked from commit d382f7ee0b98e4ab6ade9384268f25c06be462ad) Co-authored-by: Batuhan Taskaya <batuhan@python.org>
* bpo-42918: Improve built-in function compile() in mode 'single' (GH-29934) ↵Miss Islington (bot)2021-12-271-19/+1
| | | | | | | | (GH-30040) Co-authored-by: Alex Waygood <Alex.Waygood@Gmail.com> (cherry picked from commit 28179aac796ed1debdce336c4b8ca18e8475d40d) Co-authored-by: Weipeng Hong <hongweichen8888@sina.com>
* [3.10] bpo-46110: Add a recursion check to avoid stack overflow in the PEG ↵Pablo Galindo Salgado2021-12-202-3069/+4403
| | | | | | | | parser (GH-30177) (GH-30214) Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>. (cherry picked from commit e9898bf153d26059261ffef11f7643ae991e2a4c) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* bpo-46054: Fix parsing error when parsing non-utf8 characters in source ↵Miss Islington (bot)2021-12-121-8/+5
| | | | | | | | | files (GH-30068) (GH-30069) (cherry picked from commit 4325a766f5f603ef6dfb8c4d5798e5e73cb5efd5) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com> Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.10] bpo-46004: Fix error location for loops with invalid targets ↵Pablo Galindo Salgado2021-12-071-1/+2
| | | | | | | (GH-29959). (GH-29961) (cherry picked from commit 1c7a1c3be08ee911d347fffd2716f3911ba751f9) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* bpo-45866: pegen strips directory of "generated from" header (GH-29777) ↵Victor Stinner2021-11-261-1/+1
| | | | | | | | | (GH-29792) "make regen-all" now produces the same output when run from a directory other than the source tree: when building Python out of the source tree. (cherry picked from commit 253b7a0a9fef1d72a4cb87b837885576e68e917c)
* [3.10] bpo-45727: Only trigger the 'did you forgot a comma' error suggestion ↵Pablo Galindo Salgado2021-11-254-4/+7
| | | | | | | if inside parentheses. (GH-29767) Backport of GH-29757 Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
* [3.10] Ensure the str member of the tokenizer is always initialised ↵Pablo Galindo Salgado2021-11-214-3/+428
| | | | | | | (GH-29681). (GH-29683) (cherry picked from commit 4f006a789a35f5d1a7ef142bd1304ce167392457) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* bpo-45494: Fix error location in EOF tokenizer errors (GH-29108)Miss Islington (bot)2021-11-201-2/+7
| | | | | (cherry picked from commit 79ff0d1687e3f823fb121a19f0297ad052871b1b) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.10] bpo-45727: Make the syntax error for missing comma more consistent ↵Pablo Galindo Salgado2021-11-202-328/+263
| | | | | | | (GH-29427) (GH-29647) (cherry picked from commit 546cefcda75d7150b55c8bc1724bea35a1e12890) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.10] bpo-45848: Allow the parser to get error lines from encoded files ↵Łukasz Langa2021-11-201-7/+8
| | | | | | | (GH-29646) (GH-29661) (cherry picked from commit fdcc46d9554094994f78bedf6dc9220e5d5ee668) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* bpo-45820: Fix a segfault when the parser fails without reading any input ↵Miss Islington (bot)2021-11-171-0/+8
| | | | | | | (GH-29580) (cherry picked from commit df4ae55e66e34ea8de6a34f0b104871ddaf35d53) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.10] bpo-45822: Respect PEP 263's coding cookies in the parser even if ↵Pablo Galindo Salgado2021-11-171-1/+1
| | | | | | | flags are not provided (GH-29582) (GH-29586) (cherry picked from commit da20d7401de97b425897d3069f71f77b039eb16f) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.10] bpo-45716: Improve the error message when using True/False/None as ↵Pablo Galindo Salgado2021-11-161-572/+678
| | | | | | | keywords in a call (GH-29413). (GH-29428) (cherry picked from commit e2d65630f36712dbdbf7711520c985c526a5cc25) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* bpo-45738: Fix computation of error location for invalid continuation (GH-29550)Miss Islington (bot)2021-11-142-11/+5
| | | | | | characters in the parser (cherry picked from commit 25835c518aa7446f3680b62c1fb43827e0f190d9) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.10] bpo-44257: fix "assigment_expr" typo + regenerate the grammar, and ↵Pablo Galindo Salgado2021-11-041-60/+60
| | | | | | | | remove unused imports (GH-29393) (GH-29395) Co-authored-by: Wim Glenn <wglenn@jumptrading.com>. (cherry picked from commit 762173c6709f5cf3b59cbbe47d4e6fafbfe7ec2a) Co-authored-by: wim glenn <wim.glenn@gmail.com>
* bpo-45562: Ensure all tokenizer debug messages are printed to stderr (GH-29270)Miss Islington (bot)2021-10-291-1/+1
| | | | | (cherry picked from commit cdc7a5827754bec83970bb052d410d55f85b3fff) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* bpo-45562: Print tokenizer debug messages to stderr (GH-29250) (GH-29252)Miss Islington (bot)2021-10-271-4/+4
| | | | | | | (cherry picked from commit 10bbd41ba8c88bc102df108a4e0444abc7c5ea43) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com> Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* bpo-45574: fix warning about `print_escape` being unused (GH-29172) (#29176)Miss Islington (bot)2021-10-231-0/+2
| | | | | | | | | | | | | | | | It used to be like this: <img width="1232" alt="Снимок экрана 2021-10-22 в 23 07 40" src="https://user-images.githubusercontent.com/4660275/138516608-fef6ec01-a96a-40f4-81ef-52265b0f536b.png"> Quick `grep` tells that it is just used in one place under `Py_DEBUG`: https://github.com/python/cpython/blame/f6e8b80d20159596cf641305bad3a833bedd2f4f/Parser/tokenizer.cGH-L1047-L1051 <img width="752" alt="Снимок экрана 2021-10-22 в 23 08 09" src="https://user-images.githubusercontent.com/4660275/138516684-ea503136-1e92-48a5-95bb-419e190d5866.png"> I am not sure, but it also looks like a private thing, it should not affect other users. Automerge-Triggered-By: GH:pablogsal (cherry picked from commit 4bc5473a42c5eae0928430930b897209492e849d) Co-authored-by: Nikita Sobolev <mail@sobolevn.me> Co-authored-by: Nikita Sobolev <mail@sobolevn.me>
* bpo-45562: Only show debug output from the parser in debug builds (GH-29140) ↵Miss Islington (bot)2021-10-221-0/+2
| | | | | | | | | (#29149) (cherry picked from commit 86dfb55d2e091cf633dbd7aabcd49d96fb1f9d81) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com> Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.10] bpo-45494: Fix parser crash when reporting errors involving invalid ↵Łukasz Langa2021-10-192-122/+130
| | | | | | | | | | | | | | | continuation characters (GH-28993) (GH-29070) There are two errors that this commit fixes: * The parser was not correctly computing the offset and the string source for E_LINECONT errors due to the incorrect usage of strtok(). * The parser was not correctly unwinding the call stack when a tokenizer exception happened in rules involving optionals ('?', [...]) as we always make them return valid results by using the comma operator. We need to check first if we don't have an error before continuing.. (cherry picked from commit a106343f632a99c8ebb0136fa140cf189b4a6a57) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.10] bpo-45461: Fix IncrementalDecoder and StreamReader in the ↵Miss Islington (bot)2021-10-141-1/+1
| | | | | | | | | | | | "unicode-escape" codec (GH-28939) (GH-28943) They support now splitting escape sequences between input chunks. Add the third parameter "final" in codecs.unicode_escape_decode(). It is True by default to match the former behavior. (cherry picked from commit c96d1546b11b4c282a7e21737cb1f5d16349656d) Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
* [3.10] bpo-45408: Don't override previous tokenizer errors in the second ↵Pablo Galindo Salgado2021-10-071-1/+4
| | | | | | | parser pass (GH-28812). (GH-28813) (cherry picked from commit 0219017df7ec41839fd0d56a3076b5f09c58d313) Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
* [3.10] Fix typos in the Objects directory (GH-28766) (GH-28797)Christian Clauss2021-10-071-1/+1
| | | | | (cherry picked from commit 5f401f10400123afa9171548c432ea3fc37c0736) Automerge-Triggered-By: GH:JulienPalard
generated by cgit v0.12 at 2026-01-03 12:31:47 (GMT)