summaryrefslogtreecommitdiffstats
path: root/Include
Commit message (Collapse)AuthorAgeFilesLines
* gh-76785: Drop PyInterpreterID_Type (gh-117101)Eric Snow2024-03-213-34/+2
| | | I added it quite a while ago as a strategy for managing interpreter lifetimes relative to the PEP 554 (now 734) implementation. Relatively recently I refactored that implementation to no longer rely on InterpreterID objects. Thus now I'm removing it.
* gh-115754: Add Py_GetConstant() function (#116883)Victor Stinner2024-03-214-5/+43
| | | | | | | | | | | | Add Py_GetConstant() and Py_GetConstantBorrowed() functions. In the limited C API version 3.13, getting Py_None, Py_False, Py_True, Py_Ellipsis and Py_NotImplemented singletons is now implemented as function calls at the stable ABI level to hide implementation details. Getting these constants still return borrowed references. Add _testlimitedcapi/object.c and test_capi/test_object.py to test Py_GetConstant() and Py_GetConstantBorrowed() functions.
* gh-105716: Update interp->threads.main After Fork (gh-117049)Eric Snow2024-03-211-0/+3
| | | | | I missed this in gh-109921. We also update Py_Exit() to call _PyInterpreterState_SetNotRunningMain(), if necessary.
* gh-76785: Clean Up Interpreter ID Conversions (gh-117048)Eric Snow2024-03-212-1/+7
| | | Mostly we unify the two different implementations of the conversion code (from PyObject * to int64_t. We also drop the PyArg_ParseTuple()-style converter function, as well as rename and move PyInterpreterID_LookUp().
* gh-105927: Limit PyWeakref_GetRef() to limited C API 3.13 (#117091)Victor Stinner2024-03-201-0/+3
|
* gh-117021: Fix integer overflow in PyLong_AsPid() on non-Windows 64-bit ↵Serhiy Storchaka2024-03-202-2/+19
| | | | platforms (GH-117064)
* GH-108362: Incremental Cycle GC (GH-116206)Mark Shannon2024-03-203-29/+38
|
* gh-115756: make PyCode_GetFirstFree an unstable API (GH-115781)Bogdan Romanyuk2024-03-191-1/+5
|
* gh-116941: Fix pyatomic_std.h syntax errors (#116967)Sam Gross2024-03-181-0/+2
|
* gh-116869: Make C API compatible with ISO C90 (#116950)Victor Stinner2024-03-182-3/+3
| | | | Make the C API compatible with -Werror=declaration-after-statement compiler flag again.
* gh-116916: Remove separate next_func_version counter (#116918)Guido van Rossum2024-03-181-1/+0
| | | | | Somehow we ended up with two separate counter variables tracking "the next function version". Most likely this was a historical accident where an old branch was updated incorrectly. This PR merges the two counters into a single one: `interp->func_state.next_version`.
* gh-116869: Fix redefinition of the _PyOptimizerObject type (#116963)Victor Stinner2024-03-181-2/+2
| | | | | | | | | | | | | | | Defining a type twice is a C11 feature and so makes the C API incompatible with C99. Fix the issue by only defining the type once. Example of warning (treated as an error): In file included from Include/Python.h:122: Include/cpython/optimizer.h:77:3: error: redefinition of typedef '_PyOptimizerObject' is a C11 feature [-Werror,-Wtypedef-redefinition] } _PyOptimizerObject; ^ build/Include/cpython/optimizer.h:60:35: note: previous definition is here typedef struct _PyOptimizerObject _PyOptimizerObject; ^
* gh-116809: Restore removed _PyErr_ChainExceptions1() function (#116900)Victor Stinner2024-03-162-3/+4
|
* gh-114271: Fix race in `Thread.join()` (#114839)mpage2024-03-163-37/+1
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | There is a race between when `Thread._tstate_lock` is released[^1] in `Thread._wait_for_tstate_lock()` and when `Thread._stop()` asserts[^2] that it is unlocked. Consider the following execution involving threads A, B, and C: 1. A starts. 2. B joins A, blocking on its `_tstate_lock`. 3. C joins A, blocking on its `_tstate_lock`. 4. A finishes and releases its `_tstate_lock`. 5. B acquires A's `_tstate_lock` in `_wait_for_tstate_lock()`, releases it, but is swapped out before calling `_stop()`. 6. C is scheduled, acquires A's `_tstate_lock` in `_wait_for_tstate_lock()` but is swapped out before releasing it. 7. B is scheduled, calls `_stop()`, which asserts that A's `_tstate_lock` is not held. However, C holds it, so the assertion fails. The race can be reproduced[^3] by inserting sleeps at the appropriate points in the threading code. To do so, run the `repro_join_race.py` from the linked repo. There are two main parts to this PR: 1. `_tstate_lock` is replaced with an event that is attached to `PyThreadState`. The event is set by the runtime prior to the thread being cleared (in the same place that `_tstate_lock` was released). `Thread.join()` blocks waiting for the event to be set. 2. `_PyInterpreterState_WaitForThreads()` provides the ability to wait for all non-daemon threads to exit. To do so, an `is_daemon` predicate was added to `PyThreadState`. This field is set each time a thread is created. `threading._shutdown()` now calls into `_PyInterpreterState_WaitForThreads()` instead of waiting on `_tstate_lock`s. [^1]: https://github.com/python/cpython/blob/441affc9e7f419ef0b68f734505fa2f79fe653c7/Lib/threading.py#L1201 [^2]: https://github.com/python/cpython/blob/441affc9e7f419ef0b68f734505fa2f79fe653c7/Lib/threading.py#L1115 [^3]: https://github.com/mpage/cpython/commit/81946532792f938cd6f6ab4c4ff92a4edf61314f --------- Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> Co-authored-by: Antoine Pitrou <antoine@python.org>
* GH-116422: Modify a few uops so that they can be supported by tier 2 with ↵Mark Shannon2024-03-151-2/+7
| | | | hot/cold splitting (GH-116832)
* gh-111696, PEP 737: Add %T and %N to PyUnicode_FromFormat() (#116839)Victor Stinner2024-03-141-0/+2
|
* gh-111696, PEP 737: Add PyType_GetModuleName() function (#116824)Victor Stinner2024-03-142-5/+2
| | | Co-authored-by: Eric Snow <ericsnowcurrently@gmail.com>
* GH-116422: Factor out eval breaker checks at end of calls into its own ↵Mark Shannon2024-03-143-144/+147
| | | | micro-op. (GH-116817)
* gh-111696, PEP 737: Add PyType_GetFullyQualifiedName() function (#116815)Victor Stinner2024-03-141-0/+3
| | | Rewrite tests on type names in Python, they were written in C.
* Post 3.13.0a5Thomas Wouters2024-03-121-1/+1
|
* Python 3.13.0a5v3.13.0a5Thomas Wouters2024-03-121-2/+2
|
* Fix code comment regarding DK_ENTRIES (GH-113960)Matthias Diener2024-03-121-1/+1
| | | fix code comment regarding dict entries
* gh-85283: Convert grp extension to the limited C API (#116611)Victor Stinner2024-03-124-6/+0
| | | | posixmodule.h: remove check on the limited C API, since these helpers are not part of the public C API.
* gh-110850: Fix _PyTime_FromSecondsDouble() API (#116606)Victor Stinner2024-03-111-2/+4
| | | | | | | | Return 0 on success. Set an exception and return -1 on error. Fix os.timerfd_settime(): properly report exceptions on _PyTime_FromSecondsDouble() failure. No longer export _PyTime_FromSecondsDouble().
* gh-116167: Allow disabling the GIL with `PYTHON_GIL=0` or `-X gil=0` (#116338)Brett Simmers2024-03-113-0/+20
| | | | | | | | | In free-threaded builds, running with `PYTHON_GIL=0` will now disable the GIL. Follow-up issues track work to re-enable the GIL when loading an incompatible extension, and to disable the GIL by default. In order to support re-enabling the GIL at runtime, all GIL-related data structures are initialized as usual, and disabling the GIL simply sets a flag that causes `take_gil()` and `drop_gil()` to return early.
* GH-116596: Better determination of escaping uops. (GH-116597)Mark Shannon2024-03-112-26/+26
|
* gh-112536: Define MI_TSAN to 1 for --with-mimalloc and ↵Donghee Na2024-03-111-0/+4
| | | | --with-thread-sanitizer (gh-116558)
* GH-116468: Use constants instead of `oparg` in stack effects when `oparg` is ↵Mark Shannon2024-03-111-5/+5
| | | | known to be a constant. (GH-116469)
* gh-111389: expose PyHASH_INF/BITS/MODULUS/IMAG macros as public (#111418)Sergey B Kirpichev2024-03-091-5/+11
| | | Co-authored-by: Victor Stinner <vstinner@python.org>
* gh-112075: Support freeing object memory via QSBR (#116344)Dino Viehland2024-03-082-3/+27
| | | Free objects with qsbr if shared
* gh-116381: Remove bad specializations, add fail stats (GH-116464)Ken Jin2024-03-076-77/+40
| | | * Remove bad specializations, add fail stats
* gh-116381: Specialize CONTAINS_OP (GH-116385)Ken Jin2024-03-068-139/+208
| | | | | | | | | | | * Specialize CONTAINS_OP * 📜🤖 Added by blurb_it. * Add PyAPI_FUNC for JIT --------- Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
* gh-115398: Revert `PyExpat_CAPI_MAGIC` version bump (GH-116411)Sebastian Pipping2024-03-061-1/+1
| | | | | | | | | | | | | | Revert "gh-115398: Increment PyExpat_CAPI_MAGIC for SetReparseDeferralEnabled addition (GH-116301)" This reverts part of commit eda2963378a3c292cf6bb202bb00e94e46ee6d90. Why? this comment buried in an earlier code review explains: I checked again how that value is used in practice, it's here: https://github.com/python/cpython/blob/0c80da4c14d904a367968955544dd6ae58c8101c/Modules/_elementtree.c#L4363-L4372 Based on that code my understanding is that loading bigger structs from the future is considered okay unless `PyExpat_CAPI_MAGIC` differs, which implies that (1) magic needs to stay the same to support loading the future from the past and (2) that `PyExpat_CAPI_MAGIC` should only ever change for changes that do not increase size (but keep it constant). To summarize, that supports your argument. I checked branches 3.8, 3.9, 3.10, 3.11, 3.12 now and they all have the same comparison code there so reverting that magic string bump will support seamless backporting.
* gh-115103: Delay reuse of mimalloc pages that store PyObjects (#115435)Sam Gross2024-03-063-1/+24
| | | | | | | | | | | | | | | | | | This implements the delayed reuse of mimalloc pages that contain Python objects in the free-threaded build. Allocations of the same size class are grouped in data structures called pages. These are different from operating system pages. For thread-safety, we want to ensure that memory used to store PyObjects remains valid as long as there may be concurrent lock-free readers; we want to delay using it for other size classes, in other heaps, or returning it to the operating system. When a mimalloc page becomes empty, instead of immediately freeing it, we tag it with a QSBR goal and insert it into a per-thread state linked list of pages to be freed. When mimalloc needs a fresh page, we process the queue and free any still empty pages that are now deemed safe to be freed. Pages waiting to be freed are still available for allocations of the same size class and allocating from a page prevent it from being freed. There is additional logic to handle abandoned pages when threads exit.
* GH-113710: Tier 2 optimizer: check the function instead of checking globals. ↵Mark Shannon2024-03-062-113/+110
| | | | (GH-116410)
* gh-115103: Enable internal mimalloc assertions in debug builds (#116343)Sam Gross2024-03-051-1/+1
| | | | | | | | This sets `MI_DEBUG` to `2` in debug builds to enable `mi_assert_internal()` calls. Expensive internal assertions are not enabled. This also disables an assertion in free-threaded builds that would be triggered by the free-threaded GC because we traverse heaps that are not owned by the current thread.
* chore: fix typos (#116345)cui fliter2024-03-056-7/+7
| | | Signed-off-by: cui fliter <imcusg@gmail.com>
* GH-115685: Split `_TO_BOOL_ALWAYS_TRUE` into micro-ops (GH-116352)Mark Shannon2024-03-053-23/+23
|
* GH-115819: Eliminate Boolean guards when value is known (GH-116355)Mark Shannon2024-03-051-0/+1
|
* GH-115685: Optimize `TO_BOOL` and variants based on truthiness of input. ↵Mark Shannon2024-03-051-0/+1
| | | | (GH-116311)
* gh-115398: Increment PyExpat_CAPI_MAGIC for SetReparseDeferralEnabled ↵Gregory P. Smith2024-03-041-1/+1
| | | | | | | | | | addition (GH-116301) * Increment PyExpat_CAPI_MAGIC due to SetReparseDeferralEnabled addition. This is a followup to git commit 6a95676bb526261434dd068d6c49927c44d24a9b from Github PR #115623. * RESTify news API list.
* GH-116226: include `pthread_stubs.h` in `pycore_pythreads.h` (#116227)Brett Cannon2024-03-011-1/+2
|
* gh-114271: Make `_thread.ThreadHandle` thread-safe in free-threaded builds ↵mpage2024-03-011-0/+13
| | | | | | | | | | | | | | | | | (GH-115190) Make `_thread.ThreadHandle` thread-safe in free-threaded builds We protect the mutable state of `ThreadHandle` using a `_PyOnceFlag`. Concurrent operations (i.e. `join` or `detach`) on `ThreadHandle` block until it is their turn to execute or an earlier operation succeeds. Once an operation has been applied successfully all future operations complete immediately. The `join()` method is now idempotent. It may be called multiple times but the underlying OS thread will only be joined once. After `join()` succeeds, any future calls to `join()` will succeed immediately. The internal thread handle `detach()` method has been removed.
* gh-115999: Disable the specializing adaptive interpreter in free-threaded ↵Brett Simmers2024-03-011-0/+5
| | | | | builds (#116013) For now, disable all specialization when the GIL might be disabled.
* gh-115398: Expose Expat >=2.6.0 reparse deferral API (CVE-2023-52425) ↵Sebastian Pipping2024-02-291-1/+3
| | | | | | | | | | | | | | | | | | | | | (GH-115623) Allow controlling Expat >=2.6.0 reparse deferral (CVE-2023-52425) by adding five new methods: - `xml.etree.ElementTree.XMLParser.flush` - `xml.etree.ElementTree.XMLPullParser.flush` - `xml.parsers.expat.xmlparser.GetReparseDeferralEnabled` - `xml.parsers.expat.xmlparser.SetReparseDeferralEnabled` - `xml.sax.expatreader.ExpatParser.flush` Based on the "flush" idea from https://github.com/python/cpython/pull/115138#issuecomment-1932444270 . ### Notes - Please treat as a security fix related to CVE-2023-52425. Includes code suggested-by: Snild Dolkow <snild@sony.com> and by core dev Serhiy Storchaka.
* gh-115685: Type/values propagate for TO_BOOL in tier 2 (GH-115686)Ken Jin2024-02-292-20/+23
|
* gh-116088: Insert bottom checks after all sym_set_...() calls (#116089)Guido van Rossum2024-02-291-4/+4
| | | | | | | | | This changes the `sym_set_...()` functions to return a `bool` which is `false` when the symbol is `bottom` after the operation. All calls to such functions now check this result and go to `hit_bottom`, a special error label that prints a different message and then reports that it wasn't able to optimize the trace. No executor will be produced in this case.
* GH-115802: JIT "small" code for Windows (GH-115964)Brandt Bucher2024-02-2916-44/+47
|
* gh-115859: Re-enable T2 optimizer pass by default (#116062)Guido van Rossum2024-02-281-0/+2
| | | | | | | | | | | | | This undoes the *temporary* default disabling of the T2 optimizer pass in gh-115860. - Add a new test that reproduces Brandt's example from gh-115859; it indeed crashes before gh-116028 with PYTHONUOPSOPTIMIZE=1 - Re-enable the optimizer pass in T2, stop checking PYTHONUOPSOPTIMIZE - Rename the env var to disable T2 entirely to PYTHON_UOPS_OPTIMIZE (must be explicitly set to 0 to disable) - Fix skipIf conditions on tests in test_opt.py accordingly - Export sym_is_bottom() (for debugging) - Fix various things in the `_BINARY_OP_` specializations in the abstract interpreter: - DECREF(temp) - out-of-space check after sym_new_const() - add sym_matches_type() checks, so even if we somehow reach a binary op with symbolic constants of the wrong type on the stack we won't trigger the type assert
* gh-115816: Improve internal symbols API in optimizer (#116028)Guido van Rossum2024-02-281-5/+7
| | | | | | | - Any `sym_set_...` call that attempts to set conflicting information cause the symbol to become `bottom` (contradiction). - All `sym_is...` and similar calls return false or NULL for `bottom`. - Everything's tested. - The tests still pass with `PYTHONUOPSOPTIMIZE=1`.
generated by cgit v0.12 at 2025-02-17 22:32:13 (GMT)