summaryrefslogtreecommitdiffstats
path: root/Parser
Commit message (Collapse)AuthorAgeFilesLines
* Validate the AST produced by the parser in debug mode (GH-21643)Pablo Galindo2020-07-271-0/+9
| | | This will improve the debug experience if something fails in the produced AST. Previously, errors in the produced AST can be felt much later like in the garbage collector or the compiler, making debugging them much more difficult.
* Fix trivial typo in the PEG string parser (GH-21508)Eric V. Smith2020-07-161-1/+1
|
* Fix possibly-unitialized warning in string_parser.c. (GH-21503)Benjamin Peterson2020-07-161-15/+16
| | | | | | | | | | | | | | | | | | | | GCC says ``` ../cpython/Parser/string_parser.c: In function ‘fstring_find_expr’: ../cpython/Parser/string_parser.c:404:93: warning: ‘cols’ may be used uninitialized in this function [-Wmaybe-uninitialized] 404 | p2->starting_col_offset = p->tok->first_lineno == p->tok->lineno ? t->col_offset + cols : cols; | ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~ ../cpython/Parser/string_parser.c:384:16: note: ‘cols’ was declared here 384 | int lines, cols; | ^~~~ ../cpython/Parser/string_parser.c:403:45: warning: ‘lines’ may be used uninitialized in this function [-Wmaybe-uninitialized] 403 | p2->starting_lineno = t->lineno + lines - 1; | ~~~~~~~~~~~~~~~~~~^~~ ../cpython/Parser/string_parser.c:384:9: note: ‘lines’ was declared here 384 | int lines, cols; | ^~~~~ ``` and, indeed, if `PyBytes_AsString` somehow fails, lines & cols will not be initialized.
* bpo-41215: Make assertion in the new parser more strict (GH-21364)Lysandros Nikolaou2020-07-061-1/+1
|
* bpo-41215: Don't use NULL by default in the PEG parser keyword list (GH-21355)Pablo Galindo2020-07-062-4/+7
| | | Automerge-Triggered-By: @lysnikolaou
* bpo-41204: Fix compiler warning in ast_type_init() (GH-21307)Victor Stinner2020-07-041-4/+5
|
* bpo-41194: Convert _ast extension to PEP 489 (GH-21293)Victor Stinner2020-07-031-46/+73
| | | | Convert the _ast extension module to PEP 489 "Multiphase initialization". Replace the global _ast state with a module state.
* bpo-41194: The _ast module cannot be loaded more than once (GH-21290)Victor Stinner2020-07-031-39/+32
| | | | | | | | Fix a crash in the _ast module: it can no longer be loaded more than once. It now uses a global state rather than a module state. * Move _ast module state: use a global state instead. * Set _astmodule.m_size to -1, so the extension cannot be loaded more than once.
* bpo-41194: Pass module state in Python-ast.c (GH-21284)Victor Stinner2020-07-031-80/+102
| | | | | | Rework asdl_c.py to pass the module state to functions in Python-ast.c, instead of using astmodulestate_global. Handle also PyState_AddModule() failure in init_types().
* bpo-35975: Only use cf_feature_version if PyCF_ONLY_AST in cf_flags (#21021)Guido van Rossum2020-06-281-2/+3
|
* bpo-41076: Pre-feed the parser with the f-string expression location (GH-21054)Lysandros Nikolaou2020-06-272-242/+25
| | | This commit changes the parsing of f-string expressions with the new parser. The parser gets pre-fed with the location of the expression itself (not the f-string, which was what we were doing before). This allows us to completely skip the shifting of the AST nodes after the parsing is completed.
* bpo-40769: Allow extra surrounding parentheses for invalid annotated ↵Batuhan Taskaya2020-06-271-205/+261
| | | | assignment rule (GH-20387)
* bpo-41132: Use pymalloc allocator in the f-string parser (GH-21173)Lysandros Nikolaou2020-06-272-10/+10
|
* bpo-41084: Adjust message when an f-string expression causes a SyntaxError ↵Lysandros Nikolaou2020-06-261-0/+21
| | | | | (GH-21084) Prefix the error message with `fstring: `, when parsing an f-string expression throws a `SyntaxError`.
* bpo-41119: Output correct error message for list/tuple followed by colon ↵Lysandros Nikolaou2020-06-251-320/+284
| | | | (GH-21160)
* bpo-40939: Rename PyPegen* functions to PyParser* (GH-21016)Lysandros Nikolaou2020-06-211-7/+22
| | | | | | Rename PyPegen* functions to PyParser*, so that we can remove the old set of PyParser* functions that were using the old parser.
* bpo-41060: Avoid SEGFAULT when calling GET_INVALID_TARGET in the grammar ↵Lysandros Nikolaou2020-06-212-7/+26
| | | | | | | | | (GH-21020) `GET_INVALID_TARGET` might unexpectedly return `NULL`, which if not caught will cause a SEGFAULT. Therefore, this commit introduces a new inline function `RAISE_SYNTAX_ERROR_INVALID_TARGET` that always checks for `GET_INVALID_TARGET` returning NULL and can be used in the grammar, replacing the long C ternary operation used till now.
* bpo-40939: Remove the old parser (Part 2) (GH-21005)Lysandros Nikolaou2020-06-201-189/+0
| | | Remove some remaining files and Makefile targets for the old parser
* bpo-40958: Avoid 'possible loss of data' warning on Windows (GH-20970)Lysandros Nikolaou2020-06-202-2/+2
|
* bpo-40334: Produce better error messages on invalid targets (GH-20106)Lysandros Nikolaou2020-06-183-1427/+1736
| | | | | | | | | | | | | | The following error messages get produced: - `cannot delete ...` for invalid `del` targets - `... is an illegal 'for' target` for invalid targets in for statements - `... is an illegal 'with' target` for invalid targets in with statements Additionally, a few `cut`s were added in various places before the invocation of the `invalid_*` rule, in order to speed things up. Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40958: Avoid buffer overflow in the parser when indexing the current ↵Pablo Galindo2020-06-162-13/+11
| | | | line (GH-20875)
* Remove old comment in string_parser.c (GH-20906)Pablo Galindo2020-06-161-5/+0
|
* bpo-36020: Remove snprintf macro in pyerrors.h (GH-20889)Victor Stinner2020-06-151-1/+1
| | | | | | | | | | On Windows, #include "pyerrors.h" no longer defines "snprintf" and "vsnprintf" macros. PyOS_snprintf() and PyOS_vsnprintf() should be used to get portable behavior. Replace snprintf() calls with PyOS_snprintf() and replace vsnprintf() calls with PyOS_vsnprintf().
* Improve readability and style in parser files (GH-20884)Pablo Galindo2020-06-152-119/+160
|
* bpo-40939: Remove the old parser (GH-20768)Pablo Galindo2020-06-1120-26697/+24309
| | | This commit removes the old parser, the deprecated parser module, the old parser compatibility flags and environment variables and all associated support code and documentation.
* bpo-40939: Remove PEG parser easter egg (__new_parser__) (#20802)Lysandros Nikolaou2020-06-111-44/+10
| | | It no longer serves a purpose (there's only one parser) and having "new" in any name will eventually look odd. Also, it impinges on a potential sub-namespace, `__new_...__`.
* bpo-40847: Consider a line with only a LINECONT a blank line (GH-20769)Lysandros Nikolaou2020-06-101-1/+2
| | | | | | | | | | A line with only a line continuation character should be considered a blank line at tokenizer level so that only a single NEWLINE token gets emitted. The old parser was working around the issue, but the new parser threw a `SyntaxError` for valid input. For example, an empty line following a line continuation character was interpreted as a `SyntaxError`. Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-39465: Use _PyInterpreterState_GET() (GH-20788)Victor Stinner2020-06-101-9/+9
| | | | | | | | | | | | Replace _PyThreadState_GET() with _PyInterpreterState_GET() in: * get_small_int() * gcmodule.c: add also get_gc_state() function * _PyTrash_deposit_object() * _PyTrash_destroy_chain() * warnings_get_state() * Py_GetRecursionLimit() Cleanup listnode.c: add 'parser' variable.
* Raise specialised syntax error for invalid lambda parameters (GH-20776)Pablo Galindo2020-06-101-407/+712
|
* bpo-40903: Handle multiple '=' in invalid assignment rules in the PEG parser ↵Pablo Galindo2020-06-082-234/+429
| | | | | (GH-20697) Automerge-Triggered-By: @pablogsal
* bpo-40904: Fix segfault in the new parser with f-string containing yield ↵Pablo Galindo2020-06-081-0/+3
| | | | statements with no value (GH-20701)
* bpo-40880: Fix invalid read in newline_in_string in pegen.c (#20666)Pablo Galindo2020-06-051-2/+2
| | | | | | | | | | | * bpo-40880: Fix invalid read in newline_in_string in pegen.c * Update Parser/pegen/pegen.c Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> * Add NEWS entry Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
* bpo-40883: Fix memory leak in fstring_compile_expr in parse_string.c (GH-20667)Pablo Galindo2020-06-051-0/+2
|
* bpo-40826: Add _PyOS_InterruptOccurred(tstate) function (GH-20599)Victor Stinner2020-06-031-4/+13
| | | | | | | | | | | | my_fgets() now calls _PyOS_InterruptOccurred(tstate) to check for pending signals, rather calling PyOS_InterruptOccurred(). my_fgets() is called with the GIL released, whereas PyOS_InterruptOccurred() must be called with the GIL held. test_repl: use text=True and avoid SuppressCrashReport in test_multiline_string_parsing(). Fix my_fgets() on Windows: fgets(fp) does crash if fileno(fp) is closed.
* bpo-40826: Fix GIL usage in PyOS_Readline() (GH-20579)Victor Stinner2020-06-011-29/+72
| | | | | | Fix GIL usage in PyOS_Readline(): lock the GIL to set an exception. Pass tstate to my_fgets() and _PyOS_WindowsConsoleReadline(). Cleanup these functions.
* bpo-40614: Respect feature version for f-string debug expressions (GH-20196)Shantanu2020-05-271-0/+5
| | | | Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
* Refactor error handling code in Parser/pegen/pegen.c (GH-20440)Lysandros Nikolaou2020-05-271-3/+8
| | | | | | Set p->error_indicator in various places, where it's needed, but it's not done. Automerge-Triggered-By: @gvanrossum
* bpo-40217: Ensure Py_VISIT(Py_TYPE(self)) is always called for ↵Pablo Galindo2020-05-271-0/+1
| | | | | | | | | PyType_FromSpec types (reverts GH-19414) (GH-20264) Heap types now always visit the type in tp_traverse. See added docs for details. This reverts commit 0169d3003be3d072751dd14a5c84748ab63a249f. Automerge-Triggered-By: @encukou
* Fix lookahead of soft keywords in the PEG parser (GH-20436)Pablo Galindo2020-05-262-0/+10
| | | Automerge-Triggered-By: @gvanrossum
* Add soft keywords (GH-20370)Guido van Rossum2020-05-262-0/+25
| | | | | | | These are like keywords but they only work in context; they are not reserved except when there is an exact match. This would enable things like match statements without reserving `match` (which would be bad for the `re.match()` function and probably lots of other places). Automerge-Triggered-By: @gvanrossum
* Fix peg_generator compiler warnings under MSVC (GH-20405)Ammar Askar2020-05-261-4/+0
|
* bpo-38964: Print correct filename on a SyntaxError in an fstring (GH-20399)Lysandros Nikolaou2020-05-261-5/+2
| | | | | | | | When a `SyntaxError` in the expression part of a fstring is found, the filename attribute of the `SyntaxError` is always `<fstring>`. With this commit, it gets changed to always have the name of the file the fstring resides in. Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
* bpo-40750: Do not expand the new parser debug flags if Py_BUILD_CORE is not ↵Pablo Galindo2020-05-251-1/+1
| | | | defined (GH-20393)
* bpo-40750: Support -d flag in the new parser (GH-20340)Pablo Galindo2020-05-253-23/+4378
|
* bpo-36290: Fix keytword collision handling in AST node constructors (GH-12382)Rémi Lapeyre2020-05-241-3/+24
|
* Use Py_ssize_t for the column number in the PEG support code (GH-20341)Pablo Galindo2020-05-241-1/+1
|
* bpo-40334: Produce better error messages for non-parenthesized genexps ↵Lysandros Nikolaou2020-05-223-1/+73
| | | | | | (GH-20153) The error message, generated for a non-parenthesized generator expression in function calls, was still the generic `invalid syntax`, when the generator expression wasn't appearing as the first argument in the call. With this patch, even on input like `f(a, b, c for c in d, e)`, the correct error message gets produced.
* bpo-40715: Reject dict unpacking on dict comprehensions (GH-20292)Batuhan Taskaya2020-05-211-203/+293
| | | | | Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
* bpo-40176: Improve error messages for trailing comma on from import (GH-20294)Batuhan Taskaya2020-05-211-150/+207
|
* Regenerate the parser (#20195)Pablo Galindo2020-05-181-0/+3
|
generated by cgit v0.12 at 2024-11-30 21:50:21 (GMT)