| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
| |
Co-authored-by: Ken Jin <28750310+Fidget-Spinner@users.noreply.github.com>
Co-authored-by: Guido van Rossum <gvanrossum@gmail.com>
|
|
|
|
|
|
|
|
| |
The traceback.c and traceback.py mechanisms now utilize the newly added code.co_positions and PyCode_Addr2Location
to print carets on the specific expressions involved in a traceback.
Co-authored-by: Pablo Galindo <Pablogsal@gmail.com>
Co-authored-by: Ammar Askar <ammar@ammaraskar.com>
Co-authored-by: Batuhan Taskaya <batuhanosmantaskaya@gmail.com>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
This PR is part of PEP 657 and augments the compiler to emit ending
line numbers as well as starting and ending columns from the AST
into compiled code objects. This allows bytecodes to be correlated
to the exact source code ranges that generated them.
This information is made available through the following public APIs:
* The `co_positions` method on code objects.
* The C API function `PyCode_Addr2Location`.
Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
Co-authored-by: Ammar Askar <ammar@ammaraskar.com>
|
|
|
|
|
|
|
|
|
|
|
|
| |
Add an internal _PyType_AllocNoTrack() function to allocate an object
without tracking it in the GC.
Modify dict_new() to use _PyType_AllocNoTrack(): dict subclasses are
now only tracked once all PyDictObject members are initialized.
Calling _PyObject_GC_UNTRACK() is no longer needed for the dict type.
Similar change in tuple_subtype_new() for tuple subclasses.
Replace tuple_gc_track() with _PyObject_GC_TRACK().
|
|
|
|
| |
(GH-26677)
|
|
|
|
|
|
| |
All uses of this flag are either setting it
or in doc or tests for it. So we should be
able to get rid of it completely.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Remove 4 C API private trashcan functions which were only kept for
the backward compatibility of the stable ABI with Python 3.8 and
older, since the trashcan API was not usable with the limited C API
on Python 3.8 and older. The trashcan API was excluded from the
limited C API in Python 3.9.
Removed functions:
* _PyTrash_deposit_object()
* _PyTrash_destroy_chain()
* _PyTrash_thread_deposit_object()
* _PyTrash_thread_destroy_chain()
The trashcan C API was never usable with the limited C API, since old
trashcan macros accessed directly PyThreadState members like
"_tstate->trash_delete_nesting", whereas the PyThreadState structure
is opaque in the limited C API.
Exclude also the PyTrash_UNWIND_LEVEL constant from the C API.
The trashcan C API was modified in Python 3.9 by commit
38965ec5411da60d312b59be281f3510d58e0cf1 and in Python 3.10 by commit
ed1a5a5baca8f61e9a99c5be3adc16b1801514fe to hide implementation
details.
|
|
|
| |
* bpo-44458: Ensure BUFFER_BLOCK_SIZE symbol is statically allocated.
|
|
|
| |
Managing it as a bare pointer to malloc'ed bytes is just too awkward in a few places.
|
|
|
|
|
|
|
|
| |
* Specialize obj.__class__ with LOAD_ATTR_SLOT
* Specialize instance attribute lookup with attribute on class, provided attribute on class is not an overriding descriptor.
* Add stat for how many times the unquickened instruction has executed.
|
|
|
|
| |
array. (GH-26771)
|
|
|
|
|
| |
Currently, if an arg value escapes (into the closure for an inner function) we end up allocating two indices in the fast locals even though only one gets used. Additionally, using the lower index would be better in some cases, such as with no-arg `super()`. To address this, we update the compiler to fix the offsets so each variable only gets one "fast local". As a consequence, now some cell offsets are interspersed with the locals (only when an arg escapes to an inner function).
https://bugs.python.org/issue43693
|
| |
|
|
|
|
|
|
|
|
|
|
| |
* Add specializations of LOAD_GLOBAL.
* Add more stats.
* Remove old opcache; it is no longer used.
* Add NEWS
|
| |
|
|
|
|
| |
Py_IS_TYPE() no longer uses Py_TYPE() to avoid a compiler warning:
no longer cast "const PyObject*" to "PyObject*".
|
| |
|
|
|
|
|
| |
* Move trace-info to thread state.
* Correct output for pdb when turning on tracing in middle of line
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Specialize LOAD_ATTR with LOAD_ATTR_SLOT and LOAD_ATTR_SPLIT_KEYS
* Move dict-common.h to internal/pycore_dict.h
* Add LOAD_ATTR_WITH_HINT specialized opcode.
* Quicken in function if loopy
* Specialize LOAD_ATTR for module attributes.
* Add specialization stats
|
|
|
|
|
|
|
|
|
| |
This was reverted in GH-26596 (commit 6d518bb) due to some bad memory accesses.
* Add the MAKE_CELL opcode. (gh-26396)
The memory accesses have been fixed.
https://bugs.python.org/issue43693
|
|
|
|
|
| |
offsets. (gh-26396)" (GH-26597)
This reverts commit 631f9938b1604d4f893417ec339b9e0fa9196fb1.
|
|
|
|
|
|
| |
(GH-26493)" (GH-26596)
This reverts commit f3fa63ec75fdbb4a08a10957a5c631bf0c4a5970 as is
causing crashes in some Windows tests in the buildbots.
|
|
|
|
|
|
|
| |
The plan is to eventually make PyCodeObject opaque in the public C-API, with the full struct moved to Include/internal/pycore_code.h. _PyLocalsPlusKinds and _PyLocalsPlusKind started off there but were needed on PyCodeObject, hence the duplication. This led to warnings with some compilers. (Apparently it does not trigger a warning on my install of GCC.)
This change eliminates the superfluous typedef.
https://bugs.python.org/issue43693
|
|
|
|
|
|
|
| |
(gh-26396)
This moves logic out of the frame initialization code and into the compiler and eval loop. Doing so simplifies the runtime code and allows us to optimize it better.
https://bugs.python.org/issue43693
|
|
|
|
|
|
|
|
|
|
| |
These were reverted in gh-26530 (commit 17c4edc) due to refleaks.
* 2c1e258 - Compute deref offsets in compiler (gh-25152)
* b2bf2bc - Add new internal code objects fields: co_fastlocalnames and co_fastlocalkinds. (gh-26388)
This change fixes the refleaks.
https://bugs.python.org/issue43693
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Add co_firstinstr field to code object.
* Implement barebones quickening.
* Use non-quickened bytecode when tracing.
* Add NEWS item
* Add new file to Windows build.
* Don't specialize instructions with EXTENDED_ARG.
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
b2bf2bc1ece673d387341e06c8d3c2bc6e259747 (GH-26530)
* Revert "bpo-43693: Compute deref offsets in compiler (gh-25152)"
This reverts commit b2bf2bc1ece673d387341e06c8d3c2bc6e259747.
* Revert "bpo-43693: Add new internal code objects fields: co_fastlocalnames and co_fastlocalkinds. (gh-26388)"
This reverts commit 2c1e2583fdc4db6b43d163239ea42b0e8394171f.
These two commits are breaking the refleak buildbots.
|
|
|
|
|
| |
Convert the Py_TYPE() and Py_SIZE() macros to static inline
functions. The Py_SET_TYPE() and Py_SET_SIZE() functions must now be
used to set an object type and size.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
co_fastlocalkinds. (gh-26388)
A number of places in the code base (notably ceval.c and frameobject.c) rely on mapping variable names to indices in the frame "locals plus" array (AKA fast locals), and thus opargs. Currently the compiler indirectly encodes that information on the code object as the tuples co_varnames, co_cellvars, and co_freevars. At runtime the dependent code must calculate the proper mapping from those, which isn't ideal and impacts performance-sensitive sections. This is something we can easily address in the compiler instead.
This change addresses the situation by replacing internal use of co_varnames, etc. with a single combined tuple of names in locals-plus order, along with a minimal array mapping each to its kind (local vs. cell vs. free). These two new PyCodeObject fields, co_fastlocalnames and co_fastllocalkinds, are not exposed to Python code for now, but co_varnames, etc. are still available with the same values as before (though computed lazily).
Aside from the (mild) performance impact, there are a number of other benefits:
* there's now a clear, direct relationship between locals-plus and variables
* code that relies on the locals-plus-to-name mapping is simpler
* marshaled code objects are smaller and serialize/de-serialize faster
Also note that we can take this approach further by expanding the possible values in co_fastlocalkinds to include specific argument types (e.g. positional-only, kwargs). Doing so would allow further speed-ups in _PyEval_MakeFrameVector(), which is where args get unpacked into the locals-plus array. It would also allow us to shrink marshaled code objects even further.
https://bugs.python.org/issue43693
|
| |
|
|
|
|
|
|
|
|
| |
* Store log2(size) instead of size in dict-keys.
* Use enum instead of function pointer to record kind of keys.
* Add version number to dict keys.
|
|
|
|
|
| |
This is an internal-only API that helps us manage the many values used to create a code object.
https://bugs.python.org/issue43693
|
|
|
|
|
|
|
|
|
|
| |
* Move up the comment about fields using in hashing/comparision.
* Group the fields more clearly.
* Add co_ncellvars and co_nfreevars.
* Raise ValueError if nlocals != len(varnames), rather than aborting.
|
| |
|
| |
|
|
|
|
|
|
|
|
|
|
|
| |
Py_FrozenMain was added to the Limited C API in [bpo-42591]() (3.10.0a4);
but to fix that issue it would be enough to add it to the regular C API.
The function is undocumented, tests were added very recently ([bpo-44131]()),
and most importantly, it is not present in all builds of Python, as
the linker sometimes omits it as unused.
It should be added back when these issues are fixed.
Note that this does not affect Python's regular C API.
|
|
|
|
|
|
|
|
|
|
|
|
| |
When the parser does a second pass to check for errors, these rules can
have some small side-effects as they may advance the parser more than
the point reached in the first pass. This can cause the tokenizer to ask
for extra tokens in interactive mode causing the tokenizer to show the
prompt instead of failing instantly.
To avoid this, add a new mode to the tokenizer that is activated in the
second pass and deactivates asking for new tokens when the interactive
line is finished. As the parsing should have reached the last line in
the first pass, the second pass should not need to ask for more tokens.
|
|
|
|
|
|
|
|
|
| |
Fix a crash at Python exit when a deallocator function removes the
last strong reference to a heap type.
Don't read type memory after calling basedealloc() since
basedealloc() can deallocate the type and free its memory.
_PyMem_IsPtrFreed() argument is now constant.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
* Remove 'zombie' frames. We won't need them once we are allocating fixed-size frames.
* Add co_nlocalplus field to code object to avoid recomputing size of locals + frees + cells.
* Move locals, cells and freevars out of frame object into separate memory buffer.
* Use per-threadstate allocated memory chunks for local variables.
* Move globals and builtins from frame object to per-thread stack.
* Move (slow) locals frame object to per-thread stack.
* Move internal frame functions to internal header.
|
|
|
|
|
|
|
|
|
| |
keywords (GH-26014)
* Add CALL_METHOD_KW
* Make CALL_METHOD branchless too since it shares the same code
* Place parentheses in STACK_SHRINK
|
|
|
| |
These APIs are deprecated since Python 3.3. They are not documented too.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
Deprecate the following functions to configure the Python
initialization:
* PySys_AddWarnOption()
* PySys_AddWarnOptionUnicode()
* PySys_AddXOption()
* PySys_HasWarnOptions()
* Py_SetPath()
* Py_SetProgramName()
* Py_SetPythonHome()
* Py_SetStandardStreamEncoding()
* _Py_SetProgramFullPath()
Use the new PyConfig API of the Python Initialization Configuration
instead (PEP 587).
|
| |
|
|
|
|
|
|
| |
- Reformat the C API and ABI Versioning page (and extend/clarify a bit)
- Rewrite the stable ABI docs into a general text on C API Compatibility
- Add a list of Limited API contents, and notes for the individual items.
- Replace `Include/README.rst` with a link to a devguide page with the same info
|
| |
|
|
|
|
|
|
|
|
| |
"Zero cost" exception handling.
* Uses a lookup table to determine how to handle exceptions.
* Removes SETUP_FINALLY and POP_TOP block instructions, eliminating (most of) the runtime overhead of try statements.
* Reduces the size of the frame object by about 60%.
|
|
|
|
|
|
|
|
|
|
|
|
| |
Remove deprecated `Py_UNICODE` APIs: `PyUnicode_Encode`,
`PyUnicode_EncodeUTF7`, `PyUnicode_EncodeUTF8`,
`PyUnicode_EncodeUTF16`, `PyUnicode_EncodeUTF32`,
`PyUnicode_EncodeLatin1`, `PyUnicode_EncodeMBCS`,
`PyUnicode_EncodeDecimal`, `PyUnicode_EncodeRawUnicodeEscape`,
`PyUnicode_EncodeCharmap`, `PyUnicode_EncodeUnicodeEscape`,
`PyUnicode_TransformDecimalToASCII`, `PyUnicode_TranslateCharmap`,
`PyUnicodeEncodeError_Create`, `PyUnicodeTranslateError_Create`.
See :pep:`393` and :pep:`624` for reference.
|
| |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| |
- Make case-swaps half the cost of any other edit
- Refactor Levenshtein code to not use memory allocator, and to bail early on no match.
- Add comments to Levenshtein distance code
- Add test cases for Levenshtein distance behind a debug macro
- Set threshold to `(name_size + item_size + 3) * MOVE_COST / 6`.
- Reasoning: similar to `difflib.SequenceMatcher.ratio()` >= 2/3:
```
"Multiset Jaccard similarity" >= 2/3
matching letters / total letters >= 2/3
(name_size - distance + item_size - distance) / (name_size + item_size) >= 2/3
1 - (2*distance) / (name_size + item_size) >= 2/3
1/3 >= (2*distance) / (name_size + item_size)
(name_size + item_size) / 6 >= distance
With rounding:
(name_size + item_size + 3) // 6 >= distance
```
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
|