summaryrefslogtreecommitdiffstats
path: root/Parser/pegen/parse.c
Commit message (Collapse)AuthorAgeFilesLines
* [3.9] bpo-40631: Disallow single parenthesized star target (GH-24027) (GH-24068)Lysandros Nikolaou2021-01-031-540/+787
| | | | | (cherry picked from commit 2ea320dddd553298038bb7d6789e50e199332f66) Automerge-Triggered-By: GH:pablogsal
* [3.9] bpo-42381: Allow walrus in set literals and set comprehensions ↵Pablo Galindo2020-11-181-1120/+955
| | | | | | | | | | | | | | | | (GH-23332) (GH-23333) Currently walruses are not allowerd in set literals and set comprehensions: >>> {y := 4, 4**2, 3**3} File "<stdin>", line 1 {y := 4, 4**2, 3**3} ^ SyntaxError: invalid syntax but they should be allowed as well per PEP 572. (cherry picked from commit b0aba1fcdc3da952698d99aec2334faa79a8b68c) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-42374: Allow unparenthesized walrus in genexps (GH-23319) (GH-23329)Lysandros Nikolaou2020-11-161-6/+6
| | | | | This fixes a regression that was introduced by the new parser. (cherry picked from commit cb3e5ed0716114393696ec7201e51fe0595eab4f)
* [3.9] bpo-42218: Correctly handle errors in left-recursive rules (GH-23065) ↵Lysandros Nikolaou2020-10-311-0/+18
| | | | | | | | | | | | (GH-23066) Left-recursive rules need to check for errors explicitly, since even if the rule returns NULL, the parsing might continue and lead to long-distance failures. Co-authored-by: Pablo Galindo <Pablogsal@gmail.com> (cherry picked from commit 02cdfc93f82fecdb7eae97a868d4ee222b9875d9) Automerge-Triggered-By: GH:lysnikolaou
* [3.9] bpo-42214: Fix check for NOTEQUAL token in the PEG parser for the ↵Pablo Galindo2020-10-311-1/+1
| | | | | | | barry_as_flufl rule (GH-23048) (GH-23051) (cherry picked from commit 06f8c3328dcd81c84d1ee2b3a57b5381dcb38482) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* [3.9] bpo-42123: Run the parser two times and only enable invalid rules on ↵Lysandros Nikolaou2020-10-281-48/+48
| | | | | | | | | | | | the second run (GH-22111) (GH-23011) * Implement running the parser a second time for the errors messages The first parser run is only responsible for detecting whether there is a `SyntaxError` or not. If there isn't the AST gets returned. Otherwise, the parser is run a second time with all the `invalid_*` rules enabled so that all the customized error messages get produced. (cherry picked from commit bca701403253379409dece03053dbd739c0bd059)
* [3.9] bpo-41659: Disallow curly brace directly after primary (GH-22996) (#23006)Lysandros Nikolaou2020-10-271-167/+234
| | | (cherry picked from commit 15acc4eaba8519d7d5f2acaffde65446b44dcf79)
* [3.9] bpo-41979: Accept star-unpacking on with-item targets (GH-22611) ↵Batuhan Taskaya2020-10-091-6/+9
| | | | | | | (GH-22612) Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com> Automerge-Triggered-By: @pablogsal
* [3.9] bpo-41697: Correctly handle KeywordOrStarred when parsing arguments in ↵Pablo Galindo2020-09-031-1/+10
| | | | | | | the parser (GH-22077) (GH-22079) (cherry picked from commit 315a61f7a9418d904e0eea14b1f054fac3a90e9f) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* [3.9] bpo-41690: Use a loop to collect args in the parser instead of ↵Pablo Galindo2020-09-021-503/+589
| | | | | | | | | | | | | | | | | | | | | | | recursion (GH-22053) (GH-22067) This program can segfault the parser by stack overflow: ``` import ast code = "f(" + ",".join(['a' for _ in range(100000)]) + ")" print("Ready!") ast.parse(code) ``` the reason is that the rule for arguments has a simple recursion when collecting args: args[expr_ty]: [...] | a=named_expression b=[',' c=args { c }] { [...] }. (cherry picked from commit 4a97b1517a6b5ff22e2984b677a680b07ff0ce11) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* [3.9] bpo-41215: Don't use NULL by default in the PEG parser keyword list ↵Pablo Galindo2020-07-061-7/+7
| | | | | | | | | (GH-21355) (GH-21356) (cherry picked from commit 39e76c0fb07e20acad454deb86a0457b279884a9) Co-authored-by: Pablo Galindo <pablogsal@gmail.com> Automerge-Triggered-By: @lysnikolaou
* [3.9] bpo-40769: Allow extra surrounding parentheses for invalid annotated ↵Pablo Galindo2020-06-271-205/+261
| | | | | assignment rule (GH-20387) (GH-21186) (cherry picked from commit c8f29ad986f8274fc5fbf889bdd2a211878856b9)
* [3.9] bpo-41119: Output correct error message for list/tuple followed by ↵Lysandros Nikolaou2020-06-261-320/+284
| | | | | colon (GH-21160) (GH-21172) (cherry picked from commit 4b85e60601489f9ee9dd2909e28d89a31566887c)
* [3.9] bpo-41060: Avoid SEGFAULT when calling GET_INVALID_TARGET in the ↵Lysandros Nikolaou2020-06-211-4/+4
| | | | | | | | | | | | | grammar (GH-21020) (GH-21024) `GET_INVALID_TARGET` might unexpectedly return `NULL`, which if not caught will cause a SEGFAULT. Therefore, this commit introduces a new inline function `RAISE_SYNTAX_ERROR_INVALID_TARGET` that always checks for `GET_INVALID_TARGET` returning NULL and can be used in the grammar, replacing the long C ternary operation used till now. (cherry picked from commit 6c4e0bd974f2895d42b63d9d004587e74b286c88) Automerge-Triggered-By: @pablogsal
* [3.9] bpo-40334: Produce better error messages on invalid targets (GH-20106) ↵Lysandros Nikolaou2020-06-191-1420/+1707
| | | | | | | | | | | | | | | | | | | (GH-20973) * bpo-40334: Produce better error messages on invalid targets (GH-20106) The following error messages get produced: - `cannot delete ...` for invalid `del` targets - `... is an illegal 'for' target` for invalid targets in for statements - `... is an illegal 'with' target` for invalid targets in with statements Additionally, a few `cut`s were added in various places before the invocation of the `invalid_*` rule, in order to speed things up. Co-authored-by: Pablo Galindo <Pablogsal@gmail.com> (cherry picked from commit 01ece63d42b830df106948db0aefa6c1ba24416a)
* [3.9] bpo-40939: Fix test_keyword for the old parser (GH-20814)Pablo Galindo2020-06-111-7/+7
|
* Raise specialised syntax error for invalid lambda parameters (GH-20776)Miss Islington (bot)2020-06-101-407/+712
| | | | | (cherry picked from commit c6483c989694cfa328dabd45eb191440da54bc68) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40903: Handle multiple '=' in invalid assignment rules in the PEG parser ↵Miss Islington (bot)2020-06-081-234/+428
| | | | | | | | (GH-20697) Automerge-Triggered-By: @pablogsal (cherry picked from commit 9f495908c5bd3645ed1af82d7bae6782720dab77) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40750: Do not expand the new parser debug flags if Py_BUILD_CORE is not ↵Miss Islington (bot)2020-05-251-1/+1
| | | | | | | defined (GH-20393) (cherry picked from commit deb4355a37e41edf1199920789fe9572c1fb43c2) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40750: Support -d flag in the new parser (GH-20340)Miss Islington (bot)2020-05-251-23/+4376
| | | | | (cherry picked from commit 800a35c623bbcdb5793c7d7a4974524286311479) Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40334: Produce better error messages for non-parenthesized genexps ↵Miss Islington (bot)2020-05-221-0/+50
| | | | | | | | | (GH-20153) The error message, generated for a non-parenthesized generator expression in function calls, was still the generic `invalid syntax`, when the generator expression wasn't appearing as the first argument in the call. With this patch, even on input like `f(a, b, c for c in d, e)`, the correct error message gets produced. (cherry picked from commit ae145833025b0156ee2a28219e3370f3b27b2a36) Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
* bpo-40715: Reject dict unpacking on dict comprehensions (GH-20292)Miss Islington (bot)2020-05-211-203/+293
| | | | | | | Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com> (cherry picked from commit b8a65ec1d3d4660d0ee38a9765d98f5cdcabdef5) Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
* [3.9] bpo-40176: Improve error messages for trailing comma on from import ↵Pablo Galindo2020-05-211-150/+207
| | | | | | | (GH-20294) (GH-20302) (cherry picked from commit 72e0aa2) Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
* Regenerate the parser (#20195)Pablo Galindo2020-05-181-0/+3
|
* bpo-40334: Reproduce error message for type comments on bare '*' in the new ↵Lysandros Nikolaou2020-05-181-1/+22
| | | | parser (GH-20151)
* bpo-40661: Fix segfault when parsing invalid input (GH-20165)Lysandros Nikolaou2020-05-181-0/+1590
| | | | | | Fix segfaults when parsing very complex invalid input, like `import äˆ ð£„¯ð¢·žð±‹á”€ð””ð‘©±å®ä±¬ð©¾\n𗶽`. Co-authored-by: Guido van Rossum <guido@python.org> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
* bpo-40334: Improvements to error-handling code in the PEG parser (GH-20003)Lysandros Nikolaou2020-05-171-365/+657
| | | | | | | | The following improvements are implemented in this commit: - `p->error_indicator` is set, in case malloc or realloc fail. - Avoid memory leaks in the case that realloc fails. - Call `PyErr_NoMemory()` instead of `PyErr_Format()`, because it requires no memory. Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40334: Correctly identify invalid target in assignment errors (GH-20076)Pablo Galindo2020-05-151-14/+36
| | | Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
* bpo-40618: Disallow invalid targets in augassign and except clauses (GH-20083)Lysandros Nikolaou2020-05-141-36/+33
| | | | | | | | | This commit fixes the new parser to disallow invalid targets in the following scenarios: - Augmented assignments must only accept a single target (Name, Attribute or Subscript), but no tuples or lists. - `except` clauses should only accept a single `Name` as a target. Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40334: Always show the caret on SyntaxErrors (GH-20050)Lysandros Nikolaou2020-05-131-141/+214
| | | | | | | | | | | | | | This commit fixes SyntaxError locations when the caret is not displayed, by doing the following: - `col_number` always gets set to the location of the offending node/expr. When no caret is to be displayed, this gets achieved by setting the object holding the error line to None. - Introduce a new function `_PyPegen_raise_error_known_location`, which can be called, when an arbitrary `lineno`/`col_offset` needs to be passed. This function then gets used in the grammar (through some new macros and inline functions) so that SyntaxError locations of the new parser match that of the old.
* bpo-40334: produce specialized errors for invalid del targets (GH-19911)Shantanu2020-05-111-170/+292
|
* bpo-40334: Avoid collisions between parser variables and grammar variables ↵Pablo Galindo2020-05-101-5637/+5637
| | | | | | | | | | | | | | | (GH-19987) This is for the C generator: - Disallow rule and variable names starting with `_` - Rename most local variable names generated by the parser to start with `_` Exceptions: - Renaming `p` to `_p` will be a separate PR - There are still some names that might clash, e.g. - anything starting with `Py` - C reserved words (`if` etc.) - Macros like `EXTRA` and `CHECK`
* bpo-40555: Check for p->error_indicator in loop rules after the main loop is ↵Pablo Galindo2020-05-081-27/+27
| | | | done (GH-19986)
* bpo-40334: Error message for invalid default args in function call (GH-19973)Lysandros Nikolaou2020-05-071-157/+213
| | | | | When parsing something like `f(g()=2)`, where the name of a default arg is not a NAME, but an arbitrary expression, a specialised error message is emitted.
* bpo-40334: Generate comments in the parser code to improve debugging (GH-19966)Pablo Galindo2020-05-061-1074/+1074
|
* bpo-40334: Allow trailing comma in parenthesised context managers (GH-19964)Pablo Galindo2020-05-061-4/+12
|
* bpo-40334: Add type to the assignment rule in the grammar file (GH-19963)Lysandros Nikolaou2020-05-061-4/+4
|
* bpo-40334: Spacialized error message for invalid args after bare '*' (GH-19865)Lysandros Nikolaou2020-05-041-206/+456
| | | When parsing things like `def f(*): pass` the old parser used to output `SyntaxError: named arguments must follow bare *`, which the new parser wasn't able to do.
* bpo-40493: fix function type comment parsing (GH-19894)Shantanu2020-05-041-0/+66
| | | | | The grammar for func_type_input rejected things like `(*t1) ->t2`. This fixes that. Automerge-Triggered-By: @gvanrossum
* bpo-40334: Refactor lambda_parameters similar to parameters (GH-19830)Guido van Rossum2020-05-011-1045/+1133
|
* bpo-40334: Correct return value of func_type_comment (GH-19833)Pablo Galindo2020-05-011-3/+3
|
* bpo-40334: Improve column offsets for thrown syntax errors by Pegen (GH-19782)Batuhan Taskaya2020-05-011-1/+1
|
* bpo-40334: Simplify type handling in the PEG c_generator (GH-19818)Pablo Galindo2020-05-011-366/+366
|
* bpo-40334: Add support for feature_version in new PEG parser (GH-19827)Lysandros Nikolaou2020-05-011-793/+1283
| | | | | | | | | | | | | | | | | | | `ast.parse` and `compile` support a `feature_version` parameter that tells the parser to parse the input string, as if it were written in an older Python version. The `feature_version` is propagated to the tokenizer, which uses it to handle the three different stages of support for `async` and `await`. Additionally, it disallows the following at parser level: - The '@' operator in < 3.5 - Async functions in < 3.5 - Async comprehensions in < 3.6 - Underscores in numeric literals in < 3.6 - Await expression in < 3.5 - Variable annotations in < 3.6 - Async for-loops in < 3.5 - Async with-statements in < 3.5 - F-strings in < 3.6 Closes we-like-parsers/cpython#124.
* bpo-40334: Support type comments (GH-19780)Guido van Rossum2020-04-301-1399/+2299
| | | | | | | | | | This implements full support for # type: <type> comments, # type: ignore <stuff> comments, and the func_type parsing mode for ast.parse() and compile(). Closes https://github.com/we-like-parsers/cpython/issues/95. (For now, you need to use the master branch of mypy, since another issue unique to 3.9 had to be fixed there, and there's no mypy release yet.) The only thing missing is `feature_version=N`, which is being tracked in https://github.com/we-like-parsers/cpython/issues/124.
* bpo-40334: refactor and cleanup for the PEG generators (GH-19775)Pablo Galindo2020-04-291-23/+23
|
* bpo-40334: Support CO_FUTURE_BARRY_AS_BDFL in the new parser (GH-19721)Pablo Galindo2020-04-271-284/+315
| | | This commit also allows to pass flags to the new parser in all interfaces and fixes a bug in the parser generator that was causing to inline rules with actions, making them disappear.
* bpo-40334: PEP 617 implementation: New PEG parser for CPython (GH-19503)Pablo Galindo2020-04-221-0/+15391
Co-authored-by: Guido van Rossum <guido@python.org> Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
generated by cgit v0.12 at 2026-01-04 11:02:55 (GMT)