summaryrefslogtreecommitdiffstats
path: root/Parser/asdl_c.py
Commit message (Collapse)AuthorAgeFilesLines
* Fixed typo in "decclarations" (GH-28578)Rajendra arora2021-09-281-1/+1
|
* bpo-40528: fix is_simple(sum)s behavior for attributes (GH-26918)Batuhan Taskaya2021-06-271-6/+10
| | | | | | | | This is something I noticed while (now discontinued) experimenting with the idea of annotating operators with location information. Unfortunately without this addition, adding any `attributes` to stuff like `unaryop` doesn't change anything since the code assumes they are singletons and caches all instances. This patch fixes this assumption with including the attributes as well as constructor fields.
* bpo-40528: move asdl identifier collection to the new metadata system (GH-26858)Batuhan Taskaya2021-06-241-49/+51
|
* bpo-40528: Implement a metadata system for ASDL Generator (GH-20193)Batuhan Taskaya2021-06-221-24/+75
| | | | | | | | ASDL Generator was lack of proper annotation related to generated module. This patch implements a MetadataVisitor that produces a metadata object to pass to other visitors that are visiting that same module. For the inital patch, it dynamically retrieves int sequences (like cmpop), that was previously hardcoded. It offers an interface that is easy to extend.
* bpo-11105: Do not crash when compiling recursive ASTs (GH-20594)Batuhan Taskaya2021-06-031-4/+15
| | | | | | | When compiling an AST object with a direct / indirect reference cycles, on the conversion phase because of exceeding amount of calls, a segfault was raised. This patch adds recursion guards to places for preventing user inputs to not to crash AST but instead raise a RecursionError.
* bpo-43244: Rename pycore_ast.h functions to _PyAST_xxx() (GH-25252)Victor Stinner2021-04-071-17/+8
| | | | | | Rename AST functions of pycore_ast.h to use the "_PyAST_" prefix. Remove macros creating aliases without prefix. For example, Module() becomes _PyAST_Module(). Update Grammar/python.gram to use _PyAST_xxx() functions.
* bpo-43244: Remove Yield macro from pycore_ast.h (GH-25243)Victor Stinner2021-04-071-36/+31
| | | | | | | | * pycore_ast.h no longer defines the Yield macro. * Fix a compiler warning on Windows: "warning C4005: 'Yield': macro redefinition". * Python-ast.c now defines directly functions with their real _Py_xxx() name, rather than xxx(). * Remove "#undef Yield" in C files including pycore_ast.h.
* bpo-43244: Remove the pyarena.h header (GH-25007)Victor Stinner2021-03-241-3/+3
| | | | | | | | | | | | | | | | | | Remove the pyarena.h header file with functions: * PyArena_New() * PyArena_Free() * PyArena_Malloc() * PyArena_AddPyObject() These functions were undocumented, excluded from the limited C API, and were only used internally by the compiler. Add pycore_pyarena.h header. Rename functions: * PyArena_New() => _PyArena_New() * PyArena_Free() => _PyArena_Free() * PyArena_Malloc() => _PyArena_Malloc() * PyArena_AddPyObject() => _PyArena_AddPyObject()
* bpo-43244: Remove ast.h, asdl.h, Python-ast.h headers (GH-24933)Victor Stinner2021-03-231-28/+43
| | | | | | | | | | | | | | | | These functions were undocumented and excluded from the limited C API. Most names defined by these header files were not prefixed by "Py" and so could create names conflicts. For example, Python-ast.h defined a "Yield" macro which was conflict with the "Yield" name used by the Windows <winbase.h> header. Use the Python ast module instead. * Move Include/asdl.h to Include/internal/pycore_asdl.h. * Move Include/Python-ast.h to Include/internal/pycore_ast.h. * Remove ast.h header file. * pycore_symtable.h no longer includes Python-ast.h.
* bpo-43244: Fix test_peg_generator for PyAST_Validate() (GH-24912)Victor Stinner2021-03-181-30/+4
| | | | | | | | | test_peg_generator now defines _Py_TEST_PEGEN macro when building C code to not call PyAST_Validate() in Parser/pegen.c. Moreover, it defines Py_BUILD_CORE_MODULE macro to get access to the internal C API. Remove "global_ast_state" from Python-ast.c when it's built by test_peg_generator: always get the AST state from the current interpreter.
* bpo-43244: Rename pycore_ast.h to pycore_ast_state.h (GH-24907)Victor Stinner2021-03-171-4/+4
|
* bpo-42128: Structural Pattern Matching (PEP 634) (GH-22917)Brandt Bucher2021-02-261-1/+4
| | | | | Co-authored-by: Guido van Rossum <guido@python.org> Co-authored-by: Talin <viridia@gmail.com> Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
* bpo-1635741: _ast uses PyModule_AddObjectRef() (GH-23146)Victor Stinner2020-11-041-4/+2
| | | | Replace PyModule_AddObject() with PyModule_AddObjectRef() in the _ast module (Python-ast.c).
* bpo-41796: Call _PyAST_Fini() earlier to fix a leak (GH-23131)Victor Stinner2020-11-031-24/+45
| | | | | | | | | | | | | | | Call _PyAST_Fini() on all interpreters, not only on the main interpreter. Also, call it ealier to fix a reference leak. Python types contain a reference to themselves in in their PyTypeObject.tp_mro member. _PyAST_Fini() must called before the last GC collection to destroy AST types. _PyInterpreterState_Clear() now calls _PyAST_Fini(). It now also calls _PyWarnings_Fini() on subinterpeters, not only on the main interpreter. Add an assertion in AST init_types() to ensure that the _ast module is no longer used after _PyAST_Fini() has been called.
* bpo-41796: Make _ast module state per interpreter (GH-23024)Victor Stinner2020-11-021-82/+161
| | | | | | | | | | | The ast module internal state is now per interpreter. * Rename "astmodulestate" to "struct ast_state" * Add pycore_ast.h internal header: the ast_state structure is now declared in pycore_ast.h. * Add PyInterpreterState.ast (struct ast_state) * Remove get_ast_state() * Rename get_global_ast_state() to get_ast_state() * PyAST_obj2mod() now handles get_ast_state() failures
* bpo-42000: Cleanup the AST related C-code (GH-22641)Batuhan Taskaya2020-10-101-42/+0
| | | | | | | | - Use the proper asdl sequence when creating empty arguments - Remove reduntant casts (thanks to new typed asdl_sequences) - Remove MarshalPrototypeVisitor and some utilities from asdl generator - Fix the header of `Python/ast.c` (kept from pgen times) Automerge-Triggered-By: @pablogsal
* bpo-41746: Add type information to asdl_seq objects (GH-22223)Pablo Galindo2020-09-161-6/+55
| | | | | | | | | | | | | * Add new capability to the PEG parser to type variable assignments. For instance: ``` | a[asdl_stmt_seq*]=';'.small_stmt+ [';'] NEWLINE { a } ``` * Add new sequence types from the asdl definition (automatically generated) * Make `asdl_seq` type a generic aliasing pointer type. * Create a new `asdl_generic_seq` for the generic case using `void*`. * The old `asdl_seq_GET`/`ast_seq_SET` macros now are typed. * New `asdl_seq_GET_UNTYPED`/`ast_seq_SET_UNTYPED` macros for dealing with generic sequences. * Changes all possible `asdl_seq` types to use specific versions everywhere.
* bpo-41631: _ast module uses again a global state (#21961)Victor Stinner2020-09-151-41/+20
| | | | | | | | | | | | | | | | | Partially revert commit ac46eb4ad6662cf6d771b20d8963658b2186c48c: "bpo-38113: Update the Python-ast.c generator to PEP384 (gh-15957)". Using a module state per module instance is causing subtle practical problems. For example, the Mercurial project replaces the __import__() function to implement lazy import, whereas Python expected that "import _ast" always return a fully initialized _ast module. Add _PyAST_Fini() to clear the state at exit. The _ast module has no state (set _astmodule.m_size to 0). Remove astmodule_traverse(), astmodule_clear() and astmodule_free() functions.
* bpo-41204: Fix compiler warning in ast_type_init() (GH-21307)Victor Stinner2020-07-041-4/+5
|
* bpo-41194: Convert _ast extension to PEP 489 (GH-21293)Victor Stinner2020-07-031-46/+73
| | | | Convert the _ast extension module to PEP 489 "Multiphase initialization". Replace the global _ast state with a module state.
* bpo-41194: The _ast module cannot be loaded more than once (GH-21290)Victor Stinner2020-07-031-39/+32
| | | | | | | | Fix a crash in the _ast module: it can no longer be loaded more than once. It now uses a global state rather than a module state. * Move _ast module state: use a global state instead. * Set _astmodule.m_size to -1, so the extension cannot be loaded more than once.
* bpo-41194: Pass module state in Python-ast.c (GH-21284)Victor Stinner2020-07-031-80/+102
| | | | | | Rework asdl_c.py to pass the module state to functions in Python-ast.c, instead of using astmodulestate_global. Handle also PyState_AddModule() failure in init_types().
* bpo-40217: Ensure Py_VISIT(Py_TYPE(self)) is always called for ↵Pablo Galindo2020-05-271-0/+1
| | | | | | | | | PyType_FromSpec types (reverts GH-19414) (GH-20264) Heap types now always visit the type in tp_traverse. See added docs for details. This reverts commit 0169d3003be3d072751dd14a5c84748ab63a249f. Automerge-Triggered-By: @encukou
* bpo-36290: Fix keytword collision handling in AST node constructors (GH-12382)Rémi Lapeyre2020-05-241-3/+24
|
* bpo-40528: Improve AST generation script to do builds simultaneously (GH-19968)Batuhan Taskaya2020-05-181-85/+73
| | | | | | - Switch from getopt to argparse. - Removed the limitation of not being able to produce both C and H simultaneously. This will make it run faster since it parses the asdl definition once and uses the generated tree to generate both the header and the C source.
* bpo-40528: Improve and clear several aspects of the ASDL definition code for ↵Batuhan Taskaya2020-05-061-8/+2
| | | | the AST (GH-19952)
* bpo-40268: Remove unused structmember.h includes (GH-19530)Victor Stinner2020-04-151-1/+1
| | | | | | If only offsetof() is needed: include stddef.h instead. When structmember.h is used, add a comment explaining that PyMemberDef is used.
* bpo-39999: Improve compatibility of the ast module. (GH-19056)Serhiy Storchaka2020-03-221-3/+3
| | | | | | | * Re-add removed classes Suite, slice, Param, AugLoad and AugStore. * Add docstrings for dummy classes. * Add docstrings for attribute aliases. * Set __module__ to "ast" instead of "_ast".
* bpo-39638: Keep ASDL signatures in the AST nodes (GH-18515)Batuhan Taşkaya2020-03-161-8/+37
|
* bpo-36287: Make ast.dump() not output optional fields and attributes with ↵Serhiy Storchaka2020-03-091-0/+11
| | | | | | | default values. (GH-18843) The default values for optional fields and attributes of AST nodes are now set as class attributes (e.g. Constant.kind is set to None).
* bpo-38823: Fix refleaks in _ast initialization error path (GH-17276)Brandt Bucher2020-02-061-8/+18
|
* bpo-36500: Simplify PCbuild/build.bat and prevent path separator changing in ↵Steve Dower2019-12-171-1/+2
| | | | comments (GH-17644)
* closes bpo-38648: Remove double tp_free slot in Python-ast.c. (GH-17002)Max Bernstein2019-10-311-1/+0
| | | This looks like a typo due to copy-paste.
* bpo-38425: Fix ‘res’ may be used uninitialized warning (GH-16688)Dong-hee Na2019-10-101-1/+2
|
* bpo-38140: Make dict and weakref offsets opaque for C heap types (#16076)Eddie Elizondo2019-09-191-1/+7
| | | | | | * Make dict and weakref offsets opaque for C heap types * Add news
* Fix leaks in Python-ast.c (#16127)Eddie Elizondo2019-09-141-1/+5
|
* bpo-21120: Exclude Python-ast.h, ast.h and asdl.h from the limited API (#14634)Zackery Spytz2019-09-121-0/+2
| | | | The PyArena type is not part of the limited API, so these headers shouldn't be part of it either.
* bpo-38113: Update the Python-ast.c generator to PEP384 (gh-15957)Dino Viehland2019-09-111-113/+187
| | | Summary: This mostly migrates Python-ast.c to PEP384 and removes all statics from the whole file. This modifies the generator itself that generates the Python-ast.c. It leaves in the usage of _PyObject_LookupAttr even though it's not fully PEP384 compatible (this could always be shimmed in by anyone who needs it).
* bpo-38083: Minor improvements in asdl_c.py and Python-ast.c. (GH-15824)Serhiy Storchaka2019-09-101-10/+15
| | | | | * Use the const qualifier for constant C strings. * Intern field and attribute names. * Temporary incref a borrowed reference to a list item.
* Mark files as executable that are meant as scripts. (GH-15354)Greg Price2019-09-091-0/+0
| | | | | | | | | | | | | | | | | | | | | | | | | This is the converse of GH-15353 -- in addition to plenty of scripts in the tree that are marked with the executable bit (and so can be directly executed), there are a few that have a leading `#!` which could let them be executed, but it doesn't do anything because they don't have the executable bit set. Here's a command which finds such files and marks them. The first line finds files in the tree with a `#!` line *anywhere*; the next-to-last step checks that the *first* line is actually of that form. In between we filter out files that already have the bit set, and some files that are meant as fragments to be consumed by one or another kind of preprocessor. $ git grep -l '^#!' \ | grep -vxFf <( \ git ls-files --stage \ | perl -lane 'print $F[3] if (!/^100644/)' \ ) \ | grep -ve '\.in$' -e '^Doc/includes/' \ | while read f; do head -c2 "$f" | grep -qxF '#!' \ && chmod a+x "$f"; \ done
* bpo-37253: Remove PyAST_obj2mod_ex() function (GH-14020)Victor Stinner2019-06-131-6/+0
| | | | PyAST_obj2mod_ex() is similar to PyAST_obj2mod() with an additional 'feature_version' parameter which is unused.
* bpo-36974: tp_print -> tp_vectorcall_offset and tp_reserved -> tp_as_async ↵Jeroen Demeyer2019-05-311-2/+2
| | | | | | | | | (GH-13464) Automatically replace tp_print -> tp_vectorcall_offset tp_compare -> tp_as_async tp_reserved -> tp_as_async
* bpo-36842: Implement PEP 578 (GH-12613)Steve Dower2019-05-231-0/+4
| | | Adds sys.audit, sys.addaudithook, io.open_code, and associated C APIs.
* bpo-34616: Add PyCF_ALLOW_TOP_LEVEL_AWAIT to allow top-level await (GH-13148)Matthias Bussonnier2019-05-211-0/+2
| | | Co-Authored-By: Yury Selivanov <yury@magic.io>
* bpo-36385: Add ``elif`` sentence on to avoid multiple ``if`` (GH-12478)Emmanuel Arias2019-03-211-2/+2
| | | | | | | | | | | Currently, when arguments on Parser/asdl_c.py are parsed ``ìf`` sentence is used. This PR Propose to use ``elif`` to avoid multiple evaluting of the ifs. https://bugs.python.org/issue36385
* bpo-35975: Support parsing earlier minor versions of Python 3 (GH-12086)Guido van Rossum2019-03-071-0/+6
| | | | | | | This adds a `feature_version` flag to `ast.parse()` (documented) and `compile()` (hidden) that allow tweaking the parser to support older versions of the grammar. In particular if `feature_version` is 5 or 6, the hacks for the `async` and `await` keyword from PEP 492 are reinstated. (For 7 or higher, these are unconditionally treated as keywords, but they are still special tokens rather than `NAME` tokens that the parser driver recognizes.) https://bugs.python.org/issue35975
* bpo-35766 follow-up: Kill half-support for FunctionType in PyAST_obj2mod ↵Guido van Rossum2019-02-011-3/+2
| | | | | | | (#11714) See https://github.com/python/cpython/pull/11645/files/229874c612df868e7ae3e997e159915f49d16542#r252631862 https://bugs.python.org/issue35766
* bpo-35766: Merge typed_ast back into CPython (GH-11645)Guido van Rossum2019-01-311-2/+14
|
* bpo-33416: Add end positions to Python AST (GH-11605)Ivan Levkivskyi2019-01-221-3/+5
| | | | | | | | | | | | | | | | | | The majority of this PR is tediously passing `end_lineno` and `end_col_offset` everywhere. Here are non-trivial points: * It is not possible to reconstruct end positions in AST "on the fly", some information is lost after an AST node is constructed, so we need two more attributes for every AST node `end_lineno` and `end_col_offset`. * I add end position information to both CST and AST. Although it may be technically possible to avoid adding end positions to CST, the code becomes more cumbersome and less efficient. * Since the end position is not known for non-leaf CST nodes while the next token is added, this requires a bit of extra care (see `_PyNode_FinalizeEndPos`). Unless I made some mistake, the algorithm should be linear. * For statements, I "trim" the end position of suites to not include the terminal newlines and dedent (this seems to be what people would expect), for example in ```python class C: pass pass ``` the end line and end column for the class definition is (2, 8). * For `end_col_offset` I use the common Python convention for indexing, for example for `pass` the `end_col_offset` is 4 (not 3), so that `[0:4]` gives one the source code that corresponds to the node. * I added a helper function `ast.get_source_segment()`, to get source text segment corresponding to a given AST node. It is also useful for testing. An (inevitable) downside of this PR is that AST now takes almost 25% more memory. I think however it is probably justified by the benefits.
* bpo-35177, Python-ast.h: Fix "Yield" compiler warning (GH-10664)Victor Stinner2018-11-221-1/+1
| | | | | | | | Partially revert commit 5f2df88b63e50d23914e97ec778861a52abdeaad: add "#undef Yield" to .c files after including Python-ast.h. Fix the warning: winbase.h(102): warning C4005: 'Yield': macro redefinition
generated by cgit v0.12 at 2024-11-29 14:20:32 (GMT)