summaryrefslogtreecommitdiffstats
path: root/Parser
Commit message (Collapse)AuthorAgeFilesLines
* Allow pgen to produce a DOT format dump of the grammar (GH-18005)Pablo Galindo2020-01-143-2/+46
| | | Originally suggested by Anthony Shaw.
* Remove unused functions in Parser/parsetok.c (GH-17365)Emmanuel Arias2020-01-131-18/+0
|
* bpo-39307: Fix memory leak on error path in parsetok (GH-17953)Alex Henrie2020-01-131-0/+1
|
* bpo-39209: Manage correctly multi-line tokens in interactive mode (GH-17860)Pablo Galindo2020-01-061-0/+2
|
* bpo-36500: Simplify PCbuild/build.bat and prevent path separator changing in ↵Steve Dower2019-12-171-1/+2
| | | | comments (GH-17644)
* bpo-38673: dont switch to ps2 if the line starts with comment or whitespace ↵Batuhan Taşkaya2019-12-091-0/+6
| | | | | (GH-17421) https://bugs.python.org/issue38673
* bpo-36876: Moved Parser/listnode.c statics to interpreter state. (GH-16328)Vinay Sajip2019-11-071-10/+14
|
* closes bpo-38648: Remove double tp_free slot in Python-ast.c. (GH-17002)Max Bernstein2019-10-311-1/+0
| | | This looks like a typo due to copy-paste.
* bpo-11410: Standardize and use symbol visibility attributes across POSIX and ↵Vinay Sajip2019-10-151-2/+3
| | | | Windows. (GH-16347)
* bpo-38425: Fix ‘res’ may be used uninitialized warning (GH-16688)Dong-hee Na2019-10-101-1/+2
|
* bpo-38140: Make dict and weakref offsets opaque for C heap types (#16076)Eddie Elizondo2019-09-191-1/+7
| | | | | | * Make dict and weakref offsets opaque for C heap types * Add news
* Fix leaks in Python-ast.c (#16127)Eddie Elizondo2019-09-141-1/+5
|
* bpo-21120: Exclude Python-ast.h, ast.h and asdl.h from the limited API (#14634)Zackery Spytz2019-09-121-0/+2
| | | | The PyArena type is not part of the limited API, so these headers shouldn't be part of it either.
* bpo-38113: Update the Python-ast.c generator to PEP384 (gh-15957)Dino Viehland2019-09-111-113/+187
| | | Summary: This mostly migrates Python-ast.c to PEP384 and removes all statics from the whole file. This modifies the generator itself that generates the Python-ast.c. It leaves in the usage of _PyObject_LookupAttr even though it's not fully PEP384 compatible (this could always be shimmed in by anyone who needs it).
* bpo-38083: Minor improvements in asdl_c.py and Python-ast.c. (GH-15824)Serhiy Storchaka2019-09-101-10/+15
| | | | | * Use the const qualifier for constant C strings. * Intern field and attribute names. * Temporary incref a borrowed reference to a list item.
* Mark files as executable that are meant as scripts. (GH-15354)Greg Price2019-09-091-0/+0
| | | | | | | | | | | | | | | | | | | | | | | | | This is the converse of GH-15353 -- in addition to plenty of scripts in the tree that are marked with the executable bit (and so can be directly executed), there are a few that have a leading `#!` which could let them be executed, but it doesn't do anything because they don't have the executable bit set. Here's a command which finds such files and marks them. The first line finds files in the tree with a `#!` line *anywhere*; the next-to-last step checks that the *first* line is actually of that form. In between we filter out files that already have the bit set, and some files that are meant as fragments to be consumed by one or another kind of preprocessor. $ git grep -l '^#!' \ | grep -vxFf <( \ git ls-files --stage \ | perl -lane 'print $F[3] if (!/^100644/)' \ ) \ | grep -ve '\.in$' -e '^Doc/includes/' \ | while read f; do head -c2 "$f" | grep -qxF '#!' \ && chmod a+x "$f"; \ done
* Fix typo in the algorithm description (GH-15774)Pablo Galindo2019-09-091-1/+1
|
* Fix typos in the documentation of Parser/pgen (GH-15416)Shashi Ranjan2019-08-241-7/+7
| | | | Co-Authored-By: Antoine <43954001+awecx@users.noreply.github.com>
* Refactor Parser/pgen and add documentation and explanations (GH-15373)Pablo Galindo2019-08-227-332/+753
| | | | | | | | | | | | | | | | | | | | | | | | | | * Refactor Parser/pgen and add documentation and explanations To improve the readability and maintainability of the parser generator perform the following transformations: * Separate the metagrammar parser in its own class to simplify the parser generator logic. * Create separate classes for DFAs and NFAs and move methods that act exclusively on them from the parser generator to these classes. * Add docstrings and comment documenting the process to go from the grammar file into NFAs and then DFAs. Detail some of the algorithms and give some background explanations of some concepts that will helps readers not familiar with the parser generation process. * Select more descriptive names for some variables and variables. * PEP8 formatting and quote-style homogenization. The output of the parser generator remains the same (Include/graminit.h and Python/graminit.c remain untouched by running the new parser generator).
* Indent code inside if block. (GH-15284)Hansraj Das2019-08-151-1/+1
| | | Without indendation, seems like strcpy line is parallel to `if` condition.
* Fix `SyntaxError` indicator printing too many spaces for multi-line strings ↵Anthony Sottile2019-07-291-0/+2
| | | | (GH-14433)
* Remove duplicate call to strip method in Parser/pgen/token.py (GH-14938)Hansraj Das2019-07-241-6/+2
|
* bpo-37593: Swap the positions of posonlyargs and args in the constructor of ↵Pablo Galindo2019-07-141-1/+1
| | | | | ast.parameters nodes (GH-14778) https://bugs.python.org/issue37593
* bpo-37253: Remove PyAST_obj2mod_ex() function (GH-14020)Victor Stinner2019-06-131-6/+0
| | | | PyAST_obj2mod_ex() is similar to PyAST_obj2mod() with an additional 'feature_version' parameter which is unused.
* bpo-36974: tp_print -> tp_vectorcall_offset and tp_reserved -> tp_as_async ↵Jeroen Demeyer2019-05-311-2/+2
| | | | | | | | | (GH-13464) Automatically replace tp_print -> tp_vectorcall_offset tp_compare -> tp_as_async tp_reserved -> tp_as_async
* bpo-37050: Remove expr_text from FormattedValue ast node, use Constant node ↵Eric V. Smith2019-05-271-1/+1
| | | | | instead (GH-13597) When using the "=" debug functionality of f-strings, use another Constant node (or a merged constant node) instead of adding expr_text to the FormattedValue node.
* bpo-36842: Implement PEP 578 (GH-12613)Steve Dower2019-05-232-0/+13
| | | Adds sys.audit, sys.addaudithook, io.open_code, and associated C APIs.
* bpo-36878: Only allow text after `# type: ignore` if first character ASCII ↵Michael J. Sullivan2019-05-221-2/+3
| | | | | | | | | | | (GH-13504) This disallows things like `# type: ignoreé`, which seems wrong. Also switch to using Py_ISALNUM for the alnum check, for consistency with other code (and maybe correctness re: locale issues?). https://bugs.python.org/issue36878
* bpo-36878: Track extra text added to 'type: ignore' in the AST (GH-13479)Michael J. Sullivan2019-05-223-19/+35
| | | | | GH-13238 made extra text after a # type: ignore accepted by the parser. This finishes the job and actually plumbs the extra text through the parser and makes it available in the AST.
* bpo-34616: Add PyCF_ALLOW_TOP_LEVEL_AWAIT to allow top-level await (GH-13148)Matthias Bussonnier2019-05-211-0/+2
| | | Co-Authored-By: Yury Selivanov <yury@magic.io>
* bpo-2180: Treat line continuation at EOF as a `SyntaxError` (GH-13401)Anthony Sottile2019-05-181-1/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This makes the parser consistent with the tokenize module (already the case in `pypy`). sample ------ ```python x = 5\ ``` before ------ ```console $ python3 t.py $ python3 -mtokenize t.py t.py:2:0: error: EOF in multi-line statement ``` after ----- ```console $ ./python t.py File "t.py", line 3 x = 5\ ^ SyntaxError: unexpected EOF while parsing $ ./python -m tokenize t.py t.py:2:0: error: EOF in multi-line statement ``` https://bugs.python.org/issue2180
* bpo-36878: Allow extra text after `# type: ignore` comments (GH-13238)Michael J. Sullivan2019-05-111-8/+5
| | | | | | | In the parser, when using the type_comments=True option, recognize a TYPE_IGNORE as anything containing `# type: ignore` followed by a non-alphanumeric character. This is to allow ignores such as `# type: ignore[E1000]`.
* bpo-36817: Add f-string debugging using '='. (GH-13123)Eric V. Smith2019-05-081-1/+1
| | | If a "=" is specified a the end of an f-string expression, the f-string will evaluate to the text of the expression, followed by '=', followed by the repr of the value of the expression.
* bpo-36540: PEP 570 -- Implementation (GH-12701)Pablo Galindo2019-04-291-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This commit contains the implementation of PEP570: Python positional-only parameters. * Update Grammar/Grammar with new typedarglist and varargslist * Regenerate grammar files * Update and regenerate AST related files * Update code object * Update marshal.c * Update compiler and symtable * Regenerate importlib files * Update callable objects * Implement positional-only args logic in ceval.c * Regenerate frozen data * Update standard library to account for positional-only args * Add test file for positional-only args * Update other test files to account for positional-only args * Add News entry * Update inspect module and related tests
* fix warnings by adding more const (GH-12924)Inada Naoki2019-04-234-20/+16
|
* use `const` in graminit.c (GH-12713)tyomitch2019-04-231-3/+3
|
* bpo-36623: Clean parser headers and include files (GH-12253)Pablo Galindo2019-04-139-7/+10
| | | After the removal of pgen, multiple header and function prototypes that lack implementation or are unused are still lying around.
* bpo-36459: Fix a possible double PyMem_FREE() due to tokenizer.c's ↵Zackery Spytz2019-03-281-1/+0
| | | | | | tok_nextc() (12601) Remove the PyMem_FREE() call added in cb90c89. The buffer will be freed when PyTokenizer_Free() is called on the tokenizer state.
* bpo-36143: Regenerate Lib/keyword.py from the Grammar and Tokens file using ↵Pablo Galindo2019-03-251-0/+60
| | | | | | pgen (GH-12456) Now that the parser generator is written in Python (Parser/pgen) we can make use of it to regenerate the Lib/keyword file that contains the language keywords instead of parsing the autogenerated grammar files. This also allows checking in the CI that the autogenerated files are up to date.
* bpo-36385: Add ``elif`` sentence on to avoid multiple ``if`` (GH-12478)Emmanuel Arias2019-03-211-2/+2
| | | | | | | | | | | Currently, when arguments on Parser/asdl_c.py are parsed ``ìf`` sentence is used. This PR Propose to use ``elif`` to avoid multiple evaluting of the ifs. https://bugs.python.org/issue36385
* bpo-36367: Free buffer if realloc fails in tokenize.c (GH-12442)Pablo Galindo2019-03-191-2/+8
|
* bpo-36280: Add Constant.kind field (GH-12295)Guido van Rossum2019-03-131-1/+1
| | | | | | | | | | | | | | The value is a string for string and byte literals, None otherwise. It is 'u' for u"..." literals, 'b' for b"..." literals, '' for "..." literals. The 'r' (raw) prefix is ignored. Does not apply to f-strings. This appears sufficient to make mypy capable of using the stdlib ast module instead of typed_ast (assuming a mypy patch I'm working on). WIP: I need to make the tests pass. @ilevkivskyi @serhiy-storchaka https://bugs.python.org/issue36280
* Remove d_initial from the parser as it is unused (GH-12212)tyomitch2019-03-091-1/+1
| | | d_initial, the first state of a particular DFA in the parser has always been initialized to 0 in the old pgen as well as the new pgen. As this value is not used and the first state of each DFA is assumed to be the first element in the array representing it, remove d_initial from the parser to reduce complexity.
* bpo-35975: Support parsing earlier minor versions of Python 3 (GH-12086)Guido van Rossum2019-03-075-0/+96
| | | | | | | This adds a `feature_version` flag to `ast.parse()` (documented) and `compile()` (hidden) that allow tweaking the parser to support older versions of the grammar. In particular if `feature_version` is 5 or 6, the hacks for the `async` and `await` keyword from PEP 492 are reinstated. (For 7 or higher, these are unconditionally treated as keywords, but they are still special tokens rather than `NAME` tokens that the parser driver recognizes.) https://bugs.python.org/issue35975
* bpo-36187: Remove NamedStore. (GH-12167)Serhiy Storchaka2019-03-051-1/+1
| | | | | NamedStore has been replaced with Store. The difference between NamedStore and Store is handled when precess the NamedExpr node one level upper.
* Clean implementation of Parser/pgen and fix some style issues (GH-12156)Pablo Galindo2019-03-044-28/+14
|
* Remove some code which has been dead since 1994 (#12136)Alex Gaynor2019-03-021-11/+0
|
* bpo-35808: Retire pgen and use pgen2 to generate the parser (GH-11814)Pablo Galindo2019-03-0117-1727/+642
| | | | | Pgen is the oldest piece of technology in the CPython repository, building it requires various #if[n]def PGEN hacks in other parts of the code and it also depends more and more on CPython internals. This commit removes the old pgen C code and replaces it for a new version implemented in pure Python. This is a modified and adapted version of lib2to3/pgen2 that can generate grammar files compatibles with the current parser. This commit also eliminates all the #ifdef and code branches related to pgen, simplifying the code and making it more maintainable. The regen-grammar step now uses $(PYTHON_FOR_REGEN) that can be any version of the interpreter, so the new pgen code maintains compatibility with older versions of the interpreter (this also allows regenerating the grammar with the current CI solution that uses Python3.5). The new pgen Python module also makes use of the Grammar/Tokens file that holds the token specification, so is always kept in sync and avoids having to maintain duplicate token definitions.
* Fix potential memory leak in parsetok.c (GH-11832)Pablo Galindo2019-02-131-1/+2
|
* bpo-35879: Fix type comment leaks (GH-11728)Guido van Rossum2019-02-011-0/+1
| | | | | * Fix leak for # type: ignore * Fix the type comment leak
generated by cgit v0.12 at 2024-11-26 05:43:54 (GMT)