summaryrefslogtreecommitdiffstats
path: root/Parser/pegen
Commit message (Collapse)AuthorAgeFilesLines
* [3.9] bpo-42806: Fix ast locations of f-strings inside parentheses ↵Pablo Galindo2021-01-031-1/+1
| | | | | | | (GH-24067) (GH-24069) (cherry picked from commit bd2728b1e8a99ba8f8c2d481f88aeb99b8b8360f) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* [3.9] bpo-40631: Disallow single parenthesized star target (GH-24027) (GH-24068)Lysandros Nikolaou2021-01-031-540/+787
| | | | | (cherry picked from commit 2ea320dddd553298038bb7d6789e50e199332f66) Automerge-Triggered-By: GH:pablogsal
* [3.9] bpo-42381: Allow walrus in set literals and set comprehensions ↵Pablo Galindo2020-11-181-1120/+955
| | | | | | | | | | | | | | | | (GH-23332) (GH-23333) Currently walruses are not allowerd in set literals and set comprehensions: >>> {y := 4, 4**2, 3**3} File "<stdin>", line 1 {y := 4, 4**2, 3**3} ^ SyntaxError: invalid syntax but they should be allowed as well per PEP 572. (cherry picked from commit b0aba1fcdc3da952698d99aec2334faa79a8b68c) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40998: Address compiler warnings found by ubsan (GH-20929)Miss Islington (bot)2020-11-181-0/+3
| | | | | | | | Signed-off-by: Christian Heimes <christian@python.org> Automerge-Triggered-By: GH:tiran (cherry picked from commit 07f2adedf0940b06d136208ec386d69b7d2d5b43) Co-authored-by: Christian Heimes <christian@python.org>
* bpo-42374: Allow unparenthesized walrus in genexps (GH-23319) (GH-23329)Lysandros Nikolaou2020-11-161-6/+6
| | | | | This fixes a regression that was introduced by the new parser. (cherry picked from commit cb3e5ed0716114393696ec7201e51fe0595eab4f)
* [3.9] bpo-42218: Correctly handle errors in left-recursive rules (GH-23065) ↵Lysandros Nikolaou2020-10-311-0/+18
| | | | | | | | | | | | (GH-23066) Left-recursive rules need to check for errors explicitly, since even if the rule returns NULL, the parsing might continue and lead to long-distance failures. Co-authored-by: Pablo Galindo <Pablogsal@gmail.com> (cherry picked from commit 02cdfc93f82fecdb7eae97a868d4ee222b9875d9) Automerge-Triggered-By: GH:lysnikolaou
* [3.9] bpo-42214: Fix check for NOTEQUAL token in the PEG parser for the ↵Pablo Galindo2020-10-313-4/+3
| | | | | | | barry_as_flufl rule (GH-23048) (GH-23051) (cherry picked from commit 06f8c3328dcd81c84d1ee2b3a57b5381dcb38482) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* [3.9] bpo-42123: Run the parser two times and only enable invalid rules on ↵Lysandros Nikolaou2020-10-283-48/+62
| | | | | | | | | | | | the second run (GH-22111) (GH-23011) * Implement running the parser a second time for the errors messages The first parser run is only responsible for detecting whether there is a `SyntaxError` or not. If there isn't the AST gets returned. Otherwise, the parser is run a second time with all the `invalid_*` rules enabled so that all the customized error messages get produced. (cherry picked from commit bca701403253379409dece03053dbd739c0bd059)
* [3.9] bpo-41659: Disallow curly brace directly after primary (GH-22996) (#23006)Lysandros Nikolaou2020-10-271-167/+234
| | | (cherry picked from commit 15acc4eaba8519d7d5f2acaffde65446b44dcf79)
* bpo-42150: Avoid buffer overflow in the new parser (GH-22978)Miss Skeleton (bot)2020-10-251-1/+2
| | | | | (cherry picked from commit e68c67805e6a4c4ec80bea64be0e8373cc02d322) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* [3.9] bpo-41979: Accept star-unpacking on with-item targets (GH-22611) ↵Batuhan Taskaya2020-10-091-6/+9
| | | | | | | (GH-22612) Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com> Automerge-Triggered-By: @pablogsal
* [3.9] bpo-41697: Correctly handle KeywordOrStarred when parsing arguments in ↵Pablo Galindo2020-09-033-10/+22
| | | | | | | the parser (GH-22077) (GH-22079) (cherry picked from commit 315a61f7a9418d904e0eea14b1f054fac3a90e9f) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* [3.9] bpo-41690: Use a loop to collect args in the parser instead of ↵Pablo Galindo2020-09-023-503/+625
| | | | | | | | | | | | | | | | | | | | | | | recursion (GH-22053) (GH-22067) This program can segfault the parser by stack overflow: ``` import ast code = "f(" + ",".join(['a' for _ in range(100000)]) + ")" print("Ready!") ast.parse(code) ``` the reason is that the rule for arguments has a simple recursion when collecting args: args[expr_ty]: [...] | a=named_expression b=[',' c=args { c }] { [...] }. (cherry picked from commit 4a97b1517a6b5ff22e2984b677a680b07ff0ce11) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* [3.9] Validate the AST produced by the parser in debug mode (GH-21643) ↵Pablo Galindo2020-07-271-0/+9
| | | | | | | | (GH-21646) This will improve the debug experience if something fails in the produced AST. Previously, errors in the produced AST can be felt much later like in the garbage collector or the compiler, making debugging them much more difficult.. (cherry picked from commit 1332226b32da44087a55e1d71990ee6899dfd28a) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* Fix trivial typo in the PEG string parser (GH-21508)Miss Islington (bot)2020-07-161-1/+1
| | | | | (cherry picked from commit 0275e0452a773976827c2b9bd1e598ee08e2d7f5) Co-authored-by: Eric V. Smith <ericvsmith@users.noreply.github.com>
* Fix possibly-unitialized warning in string_parser.c. (GH-21503)Miss Islington (bot)2020-07-161-15/+16
| | | | | | | | | | | | | | | | | | | | | | | GCC says ``` ../cpython/Parser/string_parser.c: In function ‘fstring_find_expr’: ../cpython/Parser/string_parser.c:404:93: warning: ‘cols’ may be used uninitialized in this function [-Wmaybe-uninitialized] 404 | p2->starting_col_offset = p->tok->first_lineno == p->tok->lineno ? t->col_offset + cols : cols; | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~ ../cpython/Parser/string_parser.c:384:16: note: ‘cols’ was declared here 384 | int lines, cols; | ^~~~ ../cpython/Parser/string_parser.c:403:45: warning: ‘lines’ may be used uninitialized in this function [-Wmaybe-uninitialized] 403 | p2->starting_lineno = t->lineno + lines - 1; | ~~~~~~~~~~~~~~~~~~^~~ ../cpython/Parser/string_parser.c:384:9: note: ‘lines’ was declared here 384 | int lines, cols; | ^~~~~ ``` and, indeed, if `PyBytes_AsString` somehow fails, lines & cols will not be initialized. (cherry picked from commit 2ad7e9c011b7606c5c7307176df07419a0e60134) Co-authored-by: Benjamin Peterson <benjamin@python.org>
* bpo-41215: Make assertion in the new parser more strict (GH-21364)Miss Islington (bot)2020-07-061-1/+1
| | | | | (cherry picked from commit 782f44b8fb07ec33cee148b2b6b4cf53024fe0cd) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
* [3.9] bpo-41215: Don't use NULL by default in the PEG parser keyword list ↵Pablo Galindo2020-07-062-9/+12
| | | | | | | | | (GH-21355) (GH-21356) (cherry picked from commit 39e76c0fb07e20acad454deb86a0457b279884a9) Co-authored-by: Pablo Galindo <pablogsal@gmail.com> Automerge-Triggered-By: @lysnikolaou
* [3.9] bpo-35975: Only use cf_feature_version if PyCF_ONLY_AST in cf_flags ↵Guido van Rossum2020-06-281-2/+3
| | | | (#21022)
* [3.9] bpo-41076: Pre-feed the parser with the f-string expression location ↵Pablo Galindo2020-06-282-242/+25
| | | | | | (GH-21054) (GH-21190) This commit changes the parsing of f-string expressions with the new parser. The parser gets pre-fed with the location of the expression itself (not the f-string, which was what we were doing before). This allows us to completely skip the shifting of the AST nodes after the parsing is completed.. (cherry picked from commit 1f0f4abb110b9fbade6175842b6a26ab0b8df6dd)
* [3.9] bpo-40769: Allow extra surrounding parentheses for invalid annotated ↵Pablo Galindo2020-06-271-205/+261
| | | | | assignment rule (GH-20387) (GH-21186) (cherry picked from commit c8f29ad986f8274fc5fbf889bdd2a211878856b9)
* bpo-41084: Adjust message when an f-string expression causes a SyntaxError ↵Miss Islington (bot)2020-06-271-0/+21
| | | | | | | | (GH-21084) Prefix the error message with `fstring: `, when parsing an f-string expression throws a `SyntaxError`. (cherry picked from commit 2e0a920e9eb540654c0bb2298143b00637dc5961) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
* [3.9] bpo-41132: Use pymalloc allocator in the f-string parser (GH-21173) ↵Lysandros Nikolaou2020-06-271-7/+7
| | | | | | | (GH-21183) (cherry picked from commit 6dcbc2422de9e2a7ff89a4689572d84001e230b2) Automerge-Triggered-By: @pablogsal
* [3.9] bpo-41119: Output correct error message for list/tuple followed by ↵Lysandros Nikolaou2020-06-261-320/+284
| | | | | colon (GH-21160) (GH-21172) (cherry picked from commit 4b85e60601489f9ee9dd2909e28d89a31566887c)
* [3.9] bpo-41060: Avoid SEGFAULT when calling GET_INVALID_TARGET in the ↵Lysandros Nikolaou2020-06-212-7/+26
| | | | | | | | | | | | | grammar (GH-21020) (GH-21024) `GET_INVALID_TARGET` might unexpectedly return `NULL`, which if not caught will cause a SEGFAULT. Therefore, this commit introduces a new inline function `RAISE_SYNTAX_ERROR_INVALID_TARGET` that always checks for `GET_INVALID_TARGET` returning NULL and can be used in the grammar, replacing the long C ternary operation used till now. (cherry picked from commit 6c4e0bd974f2895d42b63d9d004587e74b286c88) Automerge-Triggered-By: @pablogsal
* bpo-40958: Avoid 'possible loss of data' warning on Windows (GH-20970)Miss Islington (bot)2020-06-202-2/+2
| | | | | (cherry picked from commit 861efc6e8fe7f030b1e193989b13287b31385939) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
* [3.9] bpo-40334: Produce better error messages on invalid targets (GH-20106) ↵Lysandros Nikolaou2020-06-193-1429/+1738
| | | | | | | | | | | | | | | | | | | (GH-20973) * bpo-40334: Produce better error messages on invalid targets (GH-20106) The following error messages get produced: - `cannot delete ...` for invalid `del` targets - `... is an illegal 'for' target` for invalid targets in for statements - `... is an illegal 'with' target` for invalid targets in with statements Additionally, a few `cut`s were added in various places before the invocation of the `invalid_*` rule, in order to speed things up. Co-authored-by: Pablo Galindo <Pablogsal@gmail.com> (cherry picked from commit 01ece63d42b830df106948db0aefa6c1ba24416a)
* bpo-40958: Avoid buffer overflow in the parser when indexing the current ↵Miss Islington (bot)2020-06-162-13/+11
| | | | | | | line (GH-20875) (GH-20919) (cherry picked from commit 51c5896b6205911d29ac07f167ec7f3cf1cb600d) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* [3.9] Improve readability and style in parser files (GH-20884) (GH-20885)Pablo Galindo2020-06-152-119/+160
| | | | | (cherry picked from commit fb61c42) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* [3.9] bpo-40939: Fix test_keyword for the old parser (GH-20814)Pablo Galindo2020-06-111-7/+7
|
* Raise specialised syntax error for invalid lambda parameters (GH-20776)Miss Islington (bot)2020-06-101-407/+712
| | | | | (cherry picked from commit c6483c989694cfa328dabd45eb191440da54bc68) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40903: Handle multiple '=' in invalid assignment rules in the PEG parser ↵Miss Islington (bot)2020-06-082-234/+429
| | | | | | | | (GH-20697) Automerge-Triggered-By: @pablogsal (cherry picked from commit 9f495908c5bd3645ed1af82d7bae6782720dab77) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40904: Fix segfault in the new parser with f-string containing yield ↵Miss Islington (bot)2020-06-081-0/+3
| | | | | | | statements with no value (GH-20701) (cherry picked from commit 972ab0327675e695373fc6272d5ac24e187579ad) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40880: Fix invalid read in newline_in_string in pegen.c (GH-20666)Miss Islington (bot)2020-06-061-2/+2
| | | | | | | | | | | | | | * bpo-40880: Fix invalid read in newline_in_string in pegen.c * Update Parser/pegen/pegen.c Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> * Add NEWS entry Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> (cherry picked from commit 2e6593db0086004a1ca7f7049218ff9573d473c2) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40883: Fix memory leak in fstring_compile_expr in parse_string.c (GH-20667)Miss Islington (bot)2020-06-061-0/+2
| | | | | (cherry picked from commit a54096e30523534e8eebb8dc1011b4536ed237a8) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* [3.9] bpo-40614: Respect feature version for f-string debug expressions ↵Pablo Galindo2020-05-271-0/+5
| | | | | | | | | (GH-20196) (GH-20464) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com> (cherry picked from commit c116c94) Co-authored-by: Shantanu <hauntsaninja@users.noreply.github.com>
* [3.9] Backport GH-20440: Set p->error_indicator in more places (GH-20457)Lysandros Nikolaou2020-05-271-0/+5
|
* [3.9] Backport GH-20370 and GH-20436: Soft keywords (GH-20458)Lysandros Nikolaou2020-05-272-1/+35
|
* [3.9] bpo-38964: Print correct filename on a SyntaxError in an fstring ↵Lysandros Nikolaou2020-05-261-5/+2
| | | | | | | | | | | (GH-20399) (GH-20404) When a `SyntaxError` in the expression part of a fstring is found, the filename attribute of the `SyntaxError` is always `<fstring>`. With this commit, it gets changed to always have the name of the file the fstring resides in. Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>. (cherry picked from commit f7b1e461567e5e3fa3ba46f589d9edc1b45b2dd0)
* bpo-40750: Do not expand the new parser debug flags if Py_BUILD_CORE is not ↵Miss Islington (bot)2020-05-251-1/+1
| | | | | | | defined (GH-20393) (cherry picked from commit deb4355a37e41edf1199920789fe9572c1fb43c2) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40750: Support -d flag in the new parser (GH-20340)Miss Islington (bot)2020-05-253-23/+4378
| | | | | (cherry picked from commit 800a35c623bbcdb5793c7d7a4974524286311479) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* Use Py_ssize_t for the column number in the PEG support code (GH-20341)Miss Islington (bot)2020-05-241-1/+1
| | | | | (cherry picked from commit b23d7adfdfa66dd8e6f98e968b1ba885692b67d6) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40334: Produce better error messages for non-parenthesized genexps ↵Miss Islington (bot)2020-05-223-1/+73
| | | | | | | | | (GH-20153) The error message, generated for a non-parenthesized generator expression in function calls, was still the generic `invalid syntax`, when the generator expression wasn't appearing as the first argument in the call. With this patch, even on input like `f(a, b, c for c in d, e)`, the correct error message gets produced. (cherry picked from commit ae145833025b0156ee2a28219e3370f3b27b2a36) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
* bpo-40715: Reject dict unpacking on dict comprehensions (GH-20292)Miss Islington (bot)2020-05-211-203/+293
| | | | | | | Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com> (cherry picked from commit b8a65ec1d3d4660d0ee38a9765d98f5cdcabdef5) Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
* [3.9] bpo-40176: Improve error messages for trailing comma on from import ↵Pablo Galindo2020-05-211-150/+207
| | | | | | | (GH-20294) (GH-20302) (cherry picked from commit 72e0aa2) Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
* Regenerate the parser (#20195)Pablo Galindo2020-05-181-0/+3
|
* bpo-40334: Reproduce error message for type comments on bare '*' in the new ↵Lysandros Nikolaou2020-05-183-23/+44
| | | | parser (GH-20151)
* bpo-40661: Fix segfault when parsing invalid input (GH-20165)Lysandros Nikolaou2020-05-181-0/+1590
| | | | | | Fix segfaults when parsing very complex invalid input, like `import äˆ ð£„¯ð¢·žð±‹á”€ð””ð‘©±å®ä±¬ð©¾\n𗶽`. Co-authored-by: Guido van Rossum <guido@python.org> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
* bpo-40334: Improvements to error-handling code in the PEG parser (GH-20003)Lysandros Nikolaou2020-05-171-365/+657
| | | | | | | | The following improvements are implemented in this commit: - `p->error_indicator` is set, in case malloc or realloc fail. - Avoid memory leaks in the case that realloc fails. - Call `PyErr_NoMemory()` instead of `PyErr_Format()`, because it requires no memory. Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40334: Correctly identify invalid target in assignment errors (GH-20076)Pablo Galindo2020-05-153-14/+86
| | | Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
generated by cgit v0.12 at 2026-01-09 22:20:54 (GMT)