summaryrefslogtreecommitdiffstats
path: root/Parser/pegen.c
Commit message (Collapse)AuthorAgeFilesLines
...
* Fix possible refleak involving _PyArena_AddPyObject (GH-25289)Erlend Egeberg Aasland2021-04-081-1/+4
|
* bpo-43244: Rename pycore_ast.h functions to _PyAST_xxx() (GH-25252)Victor Stinner2021-04-071-38/+45
| | | | | | Rename AST functions of pycore_ast.h to use the "_PyAST_" prefix. Remove macros creating aliases without prefix. For example, Module() becomes _PyAST_Module(). Update Grammar/python.gram to use _PyAST_xxx() functions.
* bpo-43244: Remove the pyarena.h header (GH-25007)Victor Stinner2021-03-241-15/+15
| | | | | | | | | | | | | | | | | | Remove the pyarena.h header file with functions: * PyArena_New() * PyArena_Free() * PyArena_Malloc() * PyArena_AddPyObject() These functions were undocumented, excluded from the limited C API, and were only used internally by the compiler. Add pycore_pyarena.h header. Rename functions: * PyArena_New() => _PyArena_New() * PyArena_Free() => _PyArena_Free() * PyArena_Malloc() => _PyArena_Malloc() * PyArena_AddPyObject() => _PyArena_AddPyObject()
* bpo-43244: Remove parser_interface.h header file (GH-25001)Victor Stinner2021-03-241-17/+0
| | | | | | | | | | | | | | | | | | | | | | | | | Remove parser functions using the "struct _mod" type, because the AST C API was removed: * PyParser_ASTFromFile() * PyParser_ASTFromFileObject() * PyParser_ASTFromFilename() * PyParser_ASTFromString() * PyParser_ASTFromStringObject() These functions were undocumented and excluded from the limited C API. Add pycore_parser.h internal header file. Rename functions: * PyParser_ASTFromFileObject() => _PyParser_ASTFromFile() * PyParser_ASTFromStringObject() => _PyParser_ASTFromString() These functions are no longer exported (replace PyAPI_FUNC() with extern). Remove also _PyPegen_run_parser_from_file() function. Update test_peg_generator to use _PyPegen_run_parser_from_file_pointer() instead.
* bpo-43244: Remove ast.h, asdl.h, Python-ast.h headers (GH-24933)Victor Stinner2021-03-231-1/+0
| | | | | | | | | | | | | | | | These functions were undocumented and excluded from the limited C API. Most names defined by these header files were not prefixed by "Py" and so could create names conflicts. For example, Python-ast.h defined a "Yield" macro which was conflict with the "Yield" name used by the Windows <winbase.h> header. Use the Python ast module instead. * Move Include/asdl.h to Include/internal/pycore_asdl.h. * Move Include/Python-ast.h to Include/internal/pycore_ast.h. * Remove ast.h header file. * pycore_symtable.h no longer includes Python-ast.h.
* bpo-43555: Report the column offset for invalid line continuation character ↵Pablo Galindo2021-03-221-4/+3
| | | | (GH-24939)
* bpo-43591: Fix error location in interactive mode for errors at the end of ↵Pablo Galindo2021-03-221-5/+9
| | | | | the line (GH-24973) Co-authored-by: Erlend Egeberg Aasland
* bpo-43244: Remove the PyAST_Validate() function (GH-24911)Victor Stinner2021-03-181-1/+2
| | | | | | | | | | | | Remove the PyAST_Validate() function. It is no longer possible to build a AST object (mod_ty type) with the public C API. The function was already excluded from the limited C API (PEP 384). Rename PyAST_Validate() function to _PyAST_Validate(), move it to the internal C API, and don't export it anymore (replace PyAPI_FUNC with extern). The function was added in bpo-12575 by the commit 832bfe2ebd5ecfa92031cd40c8b41835ba90487f.
* bpo-43244: Fix test_peg_generator for PyAST_Validate() (GH-24912)Victor Stinner2021-03-181-1/+2
| | | | | | | | | test_peg_generator now defines _Py_TEST_PEGEN macro when building C code to not call PyAST_Validate() in Parser/pegen.c. Moreover, it defines Py_BUILD_CORE_MODULE macro to get access to the internal C API. Remove "global_ast_state" from Python-ast.c when it's built by test_peg_generator: always get the AST state from the current interpreter.
* bpo-43410: Fix crash in the parser when producing syntax errors when reading ↵Pablo Galindo2021-03-141-3/+11
| | | | from stdin (GH-24763)
* bpo-42997: Improve error message for missing : before suites (GH-24292)Pablo Galindo2021-02-021-1/+22
| | | | | | | | * Add to the peg generator a new directive ('&&') that allows to expect a token and hard fail the parsing if the token is not found. This allows to quickly emmit syntax errors for missing tokens. * Use the new grammar element to hard-fail if the ':' is missing before suites.
* bpo-42986: Fix parser crash when reporting syntax errors in f-string with ↵Pablo Galindo2021-01-311-1/+1
| | | | newlines (GH-24279)
* bpo-40176: Improve error messages for unclosed string literals (GH-19346)Batuhan Taskaya2021-01-201-6/+0
| | | Automerge-Triggered-By: GH:isidentical
* bpo-42864: Simplify the tokenizer exceptions after generic SyntaxError ↵Pablo Galindo2021-01-201-10/+3
| | | | | (GH-24273) Automerge-Triggered-By: GH:pablogsal
* bpo-42864: Improve error messages regarding unclosed parentheses (GH-24161)Pablo Galindo2021-01-191-3/+70
|
* bpo-42827: Fix crash on SyntaxError in multiline expressions (GH-24140)Lysandros Nikolaou2021-01-141-2/+37
| | | | | | | | | | | | | | | | | | | | | | | When trying to extract the error line for the error message there are two distinct cases: 1. The input comes from a file, which means that we can extract the error line by using `PyErr_ProgramTextObject` and which we already do. 2. The input does not come from a file, at which point we need to get the source code from the tokenizer: * If the tokenizer's current line number is the same with the line of the error, we get the line from `tok->buf` and we're ready. * Else, we can extract the error line from the source code in the following two ways: * If the input comes from a string we have all the input in `tok->str` and we can extract the error line from it. * If the input comes from stdin, i.e. the interactive prompt, we do not have access to the previous line. That's why a new field `tok->stdin_content` is added which holds the whole input for the current (multiline) statement or expression. We can then extract the error line from `tok->stdin_content` like we do in the string case above. Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-42214: Fix check for NOTEQUAL token in the PEG parser for the ↵Pablo Galindo2020-10-301-2/+1
| | | | barry_as_flufl rule (GH-23048)
* bpo-42206: Propagate and raise errors from PyAST_Validate in the parser ↵Batuhan Taskaya2020-10-301-1/+3
| | | | (GH-23035)
* bpo-42123: Run the parser two times and only enable invalid rules on the ↵Lysandros Nikolaou2020-10-261-0/+13
| | | | | | | | | | second run (GH-22111) * Implement running the parser a second time for the errors messages The first parser run is only responsible for detecting whether there is a `SyntaxError` or not. If there isn't the AST gets returned. Otherwise, the parser is run a second time with all the `invalid_*` rules enabled so that all the customized error messages get produced.
* bpo-42150: Avoid buffer overflow in the new parser (GH-22978)Pablo Galindo2020-10-251-1/+2
|
* bpo-42000: Cleanup the AST related C-code (GH-22641)Batuhan Taskaya2020-10-101-1/+1
| | | | | | | | - Use the proper asdl sequence when creating empty arguments - Remove reduntant casts (thanks to new typed asdl_sequences) - Remove MarshalPrototypeVisitor and some utilities from asdl generator - Fix the header of `Python/ast.c` (kept from pgen times) Automerge-Triggered-By: @pablogsal
* bpo-41746: Add type information to asdl_seq objects (GH-22223)Pablo Galindo2020-09-161-87/+100
| | | | | | | | | | | | | * Add new capability to the PEG parser to type variable assignments. For instance: ``` | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a } ``` * Add new sequence types from the asdl definition (automatically generated) * Make `asdl_seq` type a generic aliasing pointer type. * Create a new `asdl_generic_seq` for the generic case using `void*`. * The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed. * New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences. * Changes all possible `asdl_seq` types to use specific versions everywhere.
* bpo-41697: Correctly handle KeywordOrStarred when parsing arguments in the ↵Pablo Galindo2020-09-031-8/+9
| | | | parser (GH-22077)
* bpo-41690: Use a loop to collect args in the parser instead of recursion ↵Pablo Galindo2020-09-021-0/+35
| | | | | | | | | | | | | | | | | | | | | (GH-22053) This program can segfault the parser by stack overflow: ``` import ast code = "f(" + ",".join(['a' for _ in range(100000)]) + ")" print("Ready!") ast.parse(code) ``` the reason is that the rule for arguments has a simple recursion when collecting args: args[expr_ty]: [...] | a=named_expression b=[',' c=args { c }] { [...] }
* Validate the AST produced by the parser in debug mode (GH-21643)Pablo Galindo2020-07-271-0/+9
| | | This will improve the debug experience if something fails in the produced AST. Previously, errors in the produced AST can be felt much later like in the garbage collector or the compiler, making debugging them much more difficult.
* bpo-41215: Make assertion in the new parser more strict (GH-21364)Lysandros Nikolaou2020-07-061-1/+1
|
* bpo-41215: Don't use NULL by default in the PEG parser keyword list (GH-21355)Pablo Galindo2020-07-061-2/+5
| | | Automerge-Triggered-By: @lysnikolaou
* bpo-35975: Only use cf_feature_version if PyCF_ONLY_AST in cf_flags (#21021)Guido van Rossum2020-06-281-2/+3
|
* bpo-41076: Pre-feed the parser with the f-string expression location (GH-21054)Lysandros Nikolaou2020-06-271-0/+3
| | | This commit changes the parsing of f-string expressions with the new parser. The parser gets pre-fed with the location of the expression itself (not the f-string, which was what we were doing before). This allows us to completely skip the shifting of the AST nodes after the parsing is completed.
* bpo-41132: Use pymalloc allocator in the f-string parser (GH-21173)Lysandros Nikolaou2020-06-271-3/+3
|
* bpo-41084: Adjust message when an f-string expression causes a SyntaxError ↵Lysandros Nikolaou2020-06-261-0/+21
| | | | | (GH-21084) Prefix the error message with `fstring: `, when parsing an f-string expression throws a `SyntaxError`.
* bpo-40958: Avoid 'possible loss of data' warning on Windows (GH-20970)Lysandros Nikolaou2020-06-201-1/+1
|
* bpo-40334: Produce better error messages on invalid targets (GH-20106)Lysandros Nikolaou2020-06-181-8/+20
| | | | | | | | | | | | | | The following error messages get produced: - `cannot delete ...` for invalid `del` targets - `... is an illegal 'for' target` for invalid targets in for statements - `... is an illegal 'with' target` for invalid targets in with statements Additionally, a few `cut`s were added in various places before the invocation of the `invalid_*` rule, in order to speed things up. Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40958: Avoid buffer overflow in the parser when indexing the current ↵Pablo Galindo2020-06-161-11/+9
| | | | line (GH-20875)
* Improve readability and style in parser files (GH-20884)Pablo Galindo2020-06-151-21/+30
|
* bpo-40939: Remove the old parser (GH-20768)Pablo Galindo2020-06-111-0/+2163
This commit removes the old parser, the deprecated parser module, the old parser compatibility flags and environment variables and all associated support code and documentation.
generated by cgit v0.12 at 2025-08-25 18:14:34 (GMT)