summaryrefslogtreecommitdiffstats
path: root/Tools/peg_generator
Commit message (Collapse)AuthorAgeFilesLines
* gh-110805: Fix test_peg_generator after the change in the parser for REPL ↵Pablo Galindo Salgado2023-10-131-1/+1
| | | | | tracebacks (#110814) Fix test_peg_generator after the change in the parser for REPL tracebacks
* gh-104169: Fix test_peg_generator after tokenizer refactoring (#110727)Lysandros Nikolaou2023-10-121-1/+10
| | | | * Fix test_peg_generator after tokenizer refactoring * Remove references to tokenizer.c in comments etc.
* gh-109596: Ensure repeated rules in the grammar are not allowed and fix ↵Pablo Galindo Salgado2023-09-221-1/+7
| | | | incorrect soft keywords (#109606)
* gh-108455: peg_generator: enable mypy's `--warn-unreachable` setting and ↵Alex Waygood2023-09-083-9/+13
| | | | `redundant-expr` error code (#109160)
* gh-108455: peg_generator: use `types-setuptools==68.1.0.1` in CI (#108697)Alex Waygood2023-08-312-10/+4
|
* gh-108455: peg_generator: install two stubs packages before running mypy ↵Alex Waygood2023-08-293-6/+13
| | | | (#108637)
* gh-108455: peg generator: Use `strict_optional=True` for `grammar_parser` ↵Nikita Sobolev2023-08-292-4/+1
| | | | (#108629)
* gh-108455: peg_generator: make the mypy config slightly stricter (#108627)Alex Waygood2023-08-294-8/+12
| | | | | | | | | * Enable `--no-implicit-reexport` * Enable the `truthy-bool` error code * Enable the `ignore-without-code` error code * Explicitly note that `--warn-unreachable` cannot yet be enabled
* gh-108455: Simplify the peg_generator mypy config (#108620)Alex Waygood2023-08-291-18/+8
| | | make it easier to see exactly which options from mypy's `--strict` mode can't currently be enabled
* gh-108455: Run `mypy` on `Tools/peg_generator` (#108456)Nikita Sobolev2023-08-284-9/+16
| | | | Co-authored-by: Hugo van Kemenade <hugovk@users.noreply.github.com>
* gh-108179: Add error message for parser stack overflows (#108256)Dennis Sweeney2023-08-221-2/+1
|
* gh-107015: Remove async_hacks from the tokenizer (#107018)Pablo Galindo Salgado2023-07-262-5/+2
|
* gh-105191: Cleanup peg generator; keep only necessary files (#105197)Lysandros Nikolaou2023-06-0120-16832/+60
|
* gh-100238: Use setuptools in peg-generator and reenable tests (#104798)Lysandros Nikolaou2023-05-261-7/+47
|
* gh-104780: Remove 2to3 program and lib2to3 module (#104781)Victor Stinner2023-05-232-3/+1
| | | | | | | | | * Remove the Tools/scripts/2to3 script. * Remove the Lib/test/test_lib2to3/ directory. * Doc/tools/extensions/pyspecific.py: remove the "2to3fixer" object type. * Makefile and PC/layout/main.py no longer compile lib2to3 grammar files. * Update Makefile for 2to3 removal.
* gh-102856: Initial implementation of PEP 701 (#102855)Pablo Galindo Salgado2023-04-191-0/+1
| | | | | | Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> Co-authored-by: Batuhan Taskaya <isidentical@gmail.com> Co-authored-by: Marta Gómez Macías <mgmacias@google.com> Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
* gh-102416: Do not memoize incorrectly loop rules in the parser (#102467)Pablo Galindo Salgado2023-03-061-2/+3
|
* gh-101046: Fix a potential memory leak in the parser when raising ↵Pablo Galindo Salgado2023-01-161-1/+1
| | | | MemoryError (#101051)
* gh-99300: Use Py_NewRef() in Parser/ directory (#99330)Victor Stinner2022-11-101-2/+1
| | | | Replace Py_INCREF() with Py_NewRef() in C files of the Parser/ directory and in the PEG generator.
* gh-84623: Remove unused imports (#94132)Victor Stinner2022-06-224-12/+1
|
* gh-54781: Move Lib/lib2to3/tests/ to Lib/test/test_lib2to3/ (#94049)Victor Stinner2022-06-212-2/+2
| | | | | | | | * Move Lib/lib2to3/tests/ to Lib/test/test_lib2to3/. * Remove Lib/test/test_lib2to3.py. * Update imports. * all_project_files(): use different paths and sort files to make the tests more reproducible. * Update references to tests.
* gh-93103: Parser uses PyConfig.parser_debug instead of Py_DebugFlag (#93106)Victor Stinner2022-05-241-1/+1
| | | | | | | * Replace deprecated Py_DebugFlag with PyConfig.parser_debug in the parser. * Add Parser.debug member. * Add tok_state.debug member. * Py_FrozenMain(): Replace Py_VerboseFlag with PyConfig.verbose.
* gh-90473: Decrease recursion limit and skip tests on WASI (GH-92803)Christian Heimes2022-05-191-1/+5
|
* gh-91320: Use _PyCFunction_CAST() (#92251)Victor Stinner2022-05-031-2/+2
| | | | | | | | | | Replace "(PyCFunction)(void(*)(void))func" cast with _PyCFunction_CAST(func). Change generated by the command: sed -i -e \ 's!(PyCFunction)(void(\*)(void)) *\([A-Za-z0-9_]\+\)!_PyCFunction_CAST(\1)!g' \ $(find -name "*.c")
* bpo-46576: Speed up test_peg_generator by using a static library for shared ↵Jeremy Kloth2022-04-062-42/+101
| | | | | sources (GH-32338) Speed up test_peg_generator by using a static library for shared sources to avoid recompiling as much code.
* Update grammar_grapher with the new forced (&&) directive (#31704)Luca Chiodini2022-03-061-0/+3
|
* Allow the parser to avoid nested processing of invalid rules (GH-31252)Pablo Galindo Salgado2022-02-101-2/+15
|
* bpo-46576: bpo-46524: Disable compiler optimization within ↵Gregory P. Smith2022-02-021-0/+10
| | | | | | | | | | | | | | | | | | | | test_peg_generator. (#31015) Disable compiler optimization within test_peg_generator. This speed up test_peg_generator by always disabling compiler optimizations by using -O0 or equivalent when the test is building its own C extensions. A build not using --with-pydebug in order to speed up test execution winds up with this test taking a very long time as it would do repeated compilation of parser C code using the same optimization flags as CPython was built with. This speeds the test up 6-8x on gps-raspbian. Also incorporate's #31017's win32 conditional and flags. Co-authored-by: Kumar Aditya kumaraditya303
* bpo-46110: Restore commit e9898bf153d26059261ffef11f7643ae991e2a4cPablo Galindo Salgado2022-01-031-4/+11
| | | This restores commit e9898bf153d26059261ffef11f7643ae991e2a4c .
* Revert "bpo-46110: Add a recursion check to avoid stack overflow in the PEG ↵Pablo Galindo Salgado2022-01-031-11/+4
| | | | | parser (GH-30177)" (GH-30363) This reverts commit e9898bf153d26059261ffef11f7643ae991e2a4c temporarily as we want to confirm if this commit is the cause of a slowdown at startup time.
* bpo-46110: Add a recursion check to avoid stack overflow in the PEG parser ↵Pablo Galindo Salgado2021-12-201-4/+11
| | | | | (GH-30177) Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
* bpo-45866: pegen strips directory of "generated from" header (GH-29777)Victor Stinner2021-11-263-3/+7
| | | | | "make regen-all" now produces the same output when run from a directory other than the source tree: when building Python out of the source tree.
* Refactor parser compilation units into specific components (GH-29676)Pablo Galindo Salgado2021-11-212-1/+3
|
* bpo-44257: fix "assigment_expr" typo + regenerate the grammar, and remove ↵wim glenn2021-11-031-2/+0
| | | | | | unused imports (GH-29393) Co-authored-by: Wim Glenn <wglenn@jumptrading.com>
* bpo-45494: Fix parser crash when reporting errors involving invalid ↵Pablo Galindo Salgado2021-10-191-1/+1
| | | | | | | | | | | | continuation characters (GH-28993) There are two errors that this commit fixes: * The parser was not correctly computing the offset and the string source for E_LINECONT errors due to the incorrect usage of strtok(). * The parser was not correctly unwinding the call stack when a tokenizer exception happened in rules involving optionals ('?', [...]) as we always make them return valid results by using the comma operator. We need to check first if we don't have an error before continuing.
* Fix typos in the Tools directory (GH-28769)Christian Clauss2021-10-063-3/+3
| | | | | | | Like #28744 but for the Tools directory. [skip issue] Opening a related issue is pending python/psf-infra-meta#130 Automerge-Triggered-By: GH:pablogsal
* [Tools/peg_generator/pegen/parser.py] Fix typo: s/wether/whether/ (GH-28739)Ikko Ashimine2021-10-051-1/+1
|
* Extract visitors from the grammar nodes and call makers in the peg generator ↵Pablo Galindo Salgado2021-09-0511-240/+244
| | | | | (GH-28172) Simplify the peg generator logic by extracting as much visitors as possible to disentangle the flow and separate concerns.
* Update pegen to use the latest upstream developments (GH-27586)Pablo Galindo Salgado2021-08-1219-397/+549
|
* bpo-44345: Fix 'generated by' comment in parser.c (GH-26615)Akira Nonaka2021-06-091-1/+1
|
* bpo-44180: Fix edge cases in invalid assigment rules in the parser (GH-26283)Pablo Galindo2021-05-211-0/+2
| | | | | | | | | | | | | | | The invalid assignment rules are very delicate since the parser can easily raise an invalid assignment when a keyword argument is provided. As they are very deep into the grammar tree, is very difficult to specify in which contexts these rules can be used and in which don't. For that, we need to use a different version of the rule that doesn't do error checking in those situations where we don't want the rule to raise (keyword arguments and generator expressions). We also need to check if we are in left-recursive rule, as those can try to eagerly advance the parser even if the parse will fail at the end of the expression. Failing to do this allows the parser to start parsing a call as a tuple and incorrectly identify a keyword argument as an invalid assignment, before it realizes that it was not a tuple after all.
* bpo-38605: Revert making 'from __future__ import annotations' the default ↵Pablo Galindo2021-04-212-2/+2
| | | | | | (GH-25490) This reverts commits 044a1048ca93d466965afc027b91a5a9eb9ce23c and 1be456ae9d53bb1cba2b24fc86175c282d1c2169, adapting the code to changes that happened after it.
* bpo-43822: Improve syntax errors for missing commas (GH-25377)Pablo Galindo2021-04-151-0/+12
|
* Sanitize macros and debug functions in pegen.c (GH-25291)Pablo Galindo2021-04-091-0/+8
|
* bpo-43244: Remove the pyarena.h header (GH-25007)Victor Stinner2021-03-241-4/+4
| | | | | | | | | | | | | | | | | | Remove the pyarena.h header file with functions: * PyArena_New() * PyArena_Free() * PyArena_Malloc() * PyArena_AddPyObject() These functions were undocumented, excluded from the limited C API, and were only used internally by the compiler. Add pycore_pyarena.h header. Rename functions: * PyArena_New() => _PyArena_New() * PyArena_Free() => _PyArena_Free() * PyArena_Malloc() => _PyArena_Malloc() * PyArena_AddPyObject() => _PyArena_AddPyObject()
* bpo-43244: Remove parser_interface.h header file (GH-25001)Victor Stinner2021-03-241-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | Remove parser functions using the "struct _mod" type, because the AST C API was removed: * PyParser_ASTFromFile() * PyParser_ASTFromFileObject() * PyParser_ASTFromFilename() * PyParser_ASTFromString() * PyParser_ASTFromStringObject() These functions were undocumented and excluded from the limited C API. Add pycore_parser.h internal header file. Rename functions: * PyParser_ASTFromFileObject() => _PyParser_ASTFromFile() * PyParser_ASTFromStringObject() => _PyParser_ASTFromString() These functions are no longer exported (replace PyAPI_FUNC() with extern). Remove also _PyPegen_run_parser_from_file() function. Update test_peg_generator to use _PyPegen_run_parser_from_file_pointer() instead.
* bpo-43244: Add pycore_compile.h header file (GH-25000)Victor Stinner2021-03-231-1/+3
| | | | | | | | | | | | | | | | | | | | | | | | | | | Remove the compiler functions using "struct _mod" type, because the public AST C API was removed: * PyAST_Compile() * PyAST_CompileEx() * PyAST_CompileObject() * PyFuture_FromAST() * PyFuture_FromASTObject() These functions were undocumented and excluded from the limited C API. Rename functions: * PyAST_CompileObject() => _PyAST_Compile() * PyFuture_FromASTObject() => _PyFuture_FromAST() Moreover, _PyFuture_FromAST() is no longer exported (replace PyAPI_FUNC() with extern). _PyAST_Compile() remains exported for test_peg_generator. Remove also compatibility functions: * PyAST_Compile() * PyAST_CompileEx() * PyFuture_FromAST()
* bpo-43244: Fix test_peg_generators on Windows (GH-24913)Victor Stinner2021-03-181-3/+2
| | | | Don't redefine Py_DebugFlag, it's already defined in pydebug.h which is included by Python.h
* bpo-43244: Fix test_peg_generator for PyAST_Validate() (GH-24912)Victor Stinner2021-03-181-0/+3
| | | | | | | | | test_peg_generator now defines _Py_TEST_PEGEN macro when building C code to not call PyAST_Validate() in Parser/pegen.c. Moreover, it defines Py_BUILD_CORE_MODULE macro to get access to the internal C API. Remove "global_ast_state" from Python-ast.c when it's built by test_peg_generator: always get the AST state from the current interpreter.
* Remove unnecessary imports in the grammar parser (GH-24904)Elisha Hollander2021-03-172-8/+2
|
generated by cgit v0.12 at 2024-11-23 02:42:38 (GMT)