summaryrefslogtreecommitdiffstats
path: root/Tools
Commit message (Collapse)AuthorAgeFilesLines
* gh-117139: Convert the evaluation stack to stack refs (#118450)Ken Jin2024-06-2610-25/+56
| | | | | | | | | | | | | | | | | This PR sets up tagged pointers for CPython. The general idea is to create a separate struct _PyStackRef for everything on the evaluation stack to store the bits. This forces the C compiler to warn us if we try to cast things or pull things out of the struct directly. Only for free threading: We tag the low bit if something is deferred - that means we skip incref and decref operations on it. This behavior may change in the future if Mark's plans to defer all objects in the interpreter loop pans out. This implies a strict stack reference discipline is required. ALL incref and decref operations on stackrefs must use the stackref variants. It is unsafe to untag something then do normal incref/decref ops on it. The new incref and decref variants are called dup and close. They mimic a "handle" API operating on these stackrefs. Please read Include/internal/pycore_stackref.h for more information! --------- Co-authored-by: Mark Shannon <9448417+markshannon@users.noreply.github.com>
* GH-120982: Add stack check assertions to generated interpreter code (GH-120992)Mark Shannon2024-06-252-0/+4
|
* gh-120956: Avoid comparison of int to Py_ssize_t in parser (#120959)Lysandros Nikolaou2024-06-241-1/+1
|
* GH-117062: Make _JUMP_TO_TOP a general absolute jump (GH-120854)Brandt Bucher2024-06-242-6/+0
|
* gh-113993: Allow interned strings to be mortal, and fix related issues ↵Petr Viktorin2024-06-212-5/+36
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (GH-120520) * Add an InternalDocs file describing how interning should work and how to use it. * Add internal functions to *explicitly* request what kind of interning is done: - `_PyUnicode_InternMortal` - `_PyUnicode_InternImmortal` - `_PyUnicode_InternStatic` * Switch uses of `PyUnicode_InternInPlace` to those. * Disallow using `_Py_SetImmortal` on strings directly. You should use `_PyUnicode_InternImmortal` instead: - Strings should be interned before immortalization, otherwise you're possibly interning a immortalizing copy. - `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to `SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in backports, as they are now part of public API and version-specific ABI. * Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery. * Make sure the statically allocated string singletons are unique. This means these sets are now disjoint: - `_Py_ID` - `_Py_STR` (including the empty string) - one-character latin-1 singletons Now, when you intern a singleton, that exact singleton will be interned. * Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic). * Intern `_Py_STR` singletons at startup. * For free-threaded builds, intern `_Py_LATIN1_CHR` singletons at startup. * Beef up the tests. Cover internal details (marked with `@cpython_only`). * Add lots of assertions Co-Authored-By: Eric Snow <ericsnowcurrently@gmail.com>
* GH-120602: Support LLVM_VERSION_SUFFIX for JIT builds (GH-120604)Xarblu2024-06-201-1/+1
|
* Fix types in pegen parser generator (GH-120720)yf-yang2024-06-193-6/+6
|
* GH-119726: Deduplicate JIT trampolines for out-of-range jumps (GH-120250)Diego Russo2024-06-191-4/+16
|
* Ignore some failing tests in emulated JIT CI (GH-120375)Diego Russo2024-06-191-1/+4
|
* gh-120417: Remove unused imports in cases_generator (#120622)Victor Stinner2024-06-1711-30/+7
|
* gh-117657: Fix `__slots__` thread safety in free-threaded build (#119368)Daniele Parmeggiani2024-06-171-2/+0
| | | | Fix a race in `PyMember_GetOne` and `PyMember_SetOne` for `Py_T_OBJECT_EX`. These functions implement `__slots__` accesses for Python objects.
* gh-117657: Fix TSan reported data race on ioctl_works (#120175)Sam Gross2024-06-171-1/+0
|
* gh-120417: Remove unused imports in Tools (#120623)Victor Stinner2024-06-173-5/+2
|
* gh-120507: Double WASI memory (#120648)Victor Stinner2024-06-172-4/+3
| | | | Use 16 MiB stack with 40 MiB memory limit, instead of 8 MiB stack with 20 MiB memory limit.
* gh-120433: Mention ``chocolatey`` for installing llvm on Windows as an ↵Kirill Podoprigora2024-06-171-0/+6
| | | | alternative option (#120434)
* gh-117657: Add TSAN suppression for set_default_allocator_unlocked (#120500)AN Long2024-06-141-0/+2
| | | Add TSAN suppression for set_default_allocator_unlocked
* gh-117657: Make PyType_HasFeature atomic (GH-120210)Ken Jin2024-06-131-1/+0
| | | Make PyType_HasFeature atomic
* gh-71587: Drop local reference cache to `_strptime` module in `_datetime` ↵neonene2024-06-121-1/+0
| | | | | (gh-120224) The _strptime module object was cached in a static local variable (in the datetime.strptime() implementation). That's a problem when it crosses isolation boundaries, such as reinitializing the runtme or between interpreters. This change fixes the problem by dropping the static variable, instead always relying on the normal sys.modules cache (via PyImport_Import()).
* gh-117657: Make Py_TYPE and Py_SET_TYPE thread safe (GH-120165)Ken Jin2024-06-121-2/+0
| | | | Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Nadeshiko Manju <me@manjusaka.me>
* gh-117657: Fix TSAN race involving import lock (#118523)Sam Gross2024-06-061-4/+0
| | | | | This adds a `_PyRecursiveMutex` type based on `PyMutex` and uses that for the import lock. This fixes some data races in the free-threaded build and generally simplifies the import lock code.
* gh-117657: Fix race involving GC and heap initialization (#119923)Sam Gross2024-06-041-3/+0
| | | | | | | | | | | | The `_PyThreadState_Bind()` function is called before the first `PyEval_AcquireThread()` so it's not synchronized with the stop the world GC. We had a race where `gc_visit_heaps()` might visit a thread's heap while it's being initialized. Use a simple atomic int to avoid visiting heaps for threads that are not yet fully initialized (i.e., before `tstate_mimalloc_bind()` is called). The race was reproducible by running: `python Lib/test/test_importlib/partial/pool_in_threads.py`.
* gh-117398: Use Per-Interpreter State for the _datetime Static Types (gh-119929)Eric Snow2024-06-032-0/+2
| | | | | | | We make use of the same mechanism that we use for the static builtin types. This required a few tweaks. The relevant code could use some cleanup but I opted to avoid the significant churn in this change. I'll tackle that separately. This change is the final piece needed to make _datetime support multiple interpreters. I've updated the module slot accordingly.
* gh-117657: Fix race involving immortalizing objects (#119927)Sam Gross2024-06-031-2/+0
| | | | | | | | | The free-threaded build currently immortalizes objects that use deferred reference counting (see gh-117783). This typically happens once the first non-main thread is created, but the behavior can be suppressed for tests, in subinterpreters, or during a compile() call. This fixes a race condition involving the tracking of whether the behavior is suppressed.
* gh-117657: Avoid `sem_clockwait` in TSAN (#119915)Sam Gross2024-06-031-9/+0
| | | | The `sem_clockwait` function is not currently instrumented, which leads to false positives.
* gh-119679: Ensures correct import libraries are included in Windows install ↵Steve Dower2024-06-031-8/+14
| | | | packages (GH-119790)
* gh-116991: Improve `pygen --help` for `python` subparser (#116992)Nikita Sobolev2024-06-031-1/+4
|
* gh-117657: Fix data races report by TSAN unicode-hash (gh-119907)Donghee Na2024-06-031-1/+0
|
* gh-117657: Fix TSAN reported race in `_PyEval_IsGILEnabled`. (#119921)Sam Gross2024-06-021-1/+0
| | | | The GIL may be disabled concurrently with this call so we need to use a relaxed atomic load.
* gh-117657: Add TSAN suppression for `set_discard_entry` (#119908)Sam Gross2024-06-011-0/+2
| | | Seen in CI occasionally when running `test_weakref`.
* gh-117657: Fix TSAN race in QSBR assertion (#119887)Sam Gross2024-06-011-2/+0
| | | | Due to a limitation in TSAN, all reads from `PyThreadState.state` must be atomic to avoid reported races.
* gh-117657: Fix TSAN race in free-threaded GC (#119883)Sam Gross2024-06-011-3/+0
| | | | | Only call `gc_restore_tid()` from stop-the-world contexts. `worklist_pop()` can be called while other threads are running, so use a relaxed atomic to modify `ob_tid`.
* Bump types-psutil from 5.9.5.20240423 to 5.9.5.20240516 in /Tools (#119900)dependabot[bot]2024-06-011-1/+1
|
* build(deps-dev): bump types-setuptools from 69.5.0.20240423 to ↵dependabot[bot]2024-06-011-1/+1
| | | | 70.0.0.20240524 in /Tools (#119899)
* gh-97747: Improvements to WASM browser REPL. (#97665)Katie Bell2024-05-312-8/+69
| | | | | Improvements to WASM browser REPL. Adds a text box to write and run code outside the REPL, a stop button, and handling of Ctrl-D for EOF.
* gh-118235: Move RAISE_SYNTAX_ERROR actions to invalid rules and make sure ↵Petr Viktorin2024-05-301-0/+12
| | | | | | | | they stay there (GH-119731) The Full Grammar specification in the docs omits rule actions, so grammar rules that raise a syntax error looked like valid syntax. This was solved in ef940de by hiding those rules in the custom syntax highlighter. This moves all syntax-error alternatives to invalid rules, adds a validator that ensures that actions containing RAISE_SYNTAX_ERROR are in invalid rules, and reverts the syntax highlighter hack.
* gh-119689: generate stack effect metadata for pseudo instructions (#119691)Irit Katriel2024-05-295-20/+47
|
* gh-119661: Add _Py_SINGLETON() include in Argumenet Clinic (#119712)Victor Stinner2024-05-291-3/+9
| | | | | When the _Py_SINGLETON() is used, Argument Clinic now adds an explicit "pycore_runtime.h" include to get the macro. Previously, the macro may or may not be included indirectly by another include.
* gh-119396: Optimize unicode_repr() (#119617)Victor Stinner2024-05-281-0/+1
| | | | | | | | | | | | | | | | | | | | | | Use stringlib to specialize unicode_repr() for each string kind (UCS1, UCS2, UCS4). Benchmark: +-------------------------------------+---------+----------------------+ | Benchmark | ref | change2 | +=====================================+=========+======================+ | repr('abc') | 100 ns | 103 ns: 1.02x slower | +-------------------------------------+---------+----------------------+ | repr('a' * 100) | 369 ns | 369 ns: 1.00x slower | +-------------------------------------+---------+----------------------+ | repr(('a' + squote) * 100) | 1.21 us | 946 ns: 1.27x faster | +-------------------------------------+---------+----------------------+ | repr(('a' + nl) * 100) | 1.23 us | 907 ns: 1.36x faster | +-------------------------------------+---------+----------------------+ | repr(dquote + ('a' + squote) * 100) | 1.08 us | 858 ns: 1.25x faster | +-------------------------------------+---------+----------------------+ | Geometric mean | (ref) | 1.16x faster | +-------------------------------------+---------+----------------------+
* gh-117557: Improve error messages when a string, bytes or bytearray of ↵Serhiy Storchaka2024-05-281-3/+22
| | | | length 1 are expected (GH-117631)
* Fix typos in comments (#119645)Xie Yanbo2024-05-283-3/+3
|
* gh-117398: Statically Allocate the Datetime C-API (GH-119472)Eric Snow2024-05-231-0/+3
|
* gh-117142: ctypes: Clean up c-analyzer .tsv files (GH-117544)neonene2024-05-222-47/+0
| | | Co-authored-by: Petr Viktorin <encukou@gmail.com>
* gh-117505: Run ensurepip in isolated env in Windows installer (GH-118257)Michael Vincent2024-05-221-2/+2
| | | ensurepip forks a subprocess to run pip itself, but that subprocess only inherits a -I isolated mode flag (see _run_pip() in Lib/ensurepip/__init__.py), not the "-E -s" flags that the installer has been using. This means that parts of ensurepip don't actually run in an isolated environment and can make incorrect decisions based on packages installed in the user site-packages.
* gh-119213: Be More Careful About _PyArg_Parser.kwtuple Across Interpreters ↵Eric Snow2024-05-221-0/+2
| | | | | | | | | (gh-119331) _PyArg_Parser holds static global data generated for modules by Argument Clinic. The _PyArg_Parser.kwtuple field is a tuple object, even though it's stored within a static global. In some cases the tuple is statically allocated and thus it's okay that it gets shared by multiple interpreters. However, in other cases the tuple is set lazily, allocated from the heap using the active interprepreter at the point the tuple is needed. This is a problem once that interpreter is destroyed since _PyArg_Parser.kwtuple becomes at dangling pointer, leading to crashes. It isn't a problem if the tuple is allocated under the main interpreter, since its lifetime is bound to the lifetime of the runtime. The solution here is to temporarily switch to the main interpreter. The alternative would be to always statically allocate the tuple. This change also fixes a bug where only the most recent parser was added to the global linked list.
* gh-117657: Fix itertools.count thread safety (#119268)Arnon Yaari2024-05-211-1/+0
| | | Fix itertools.count in free-threading mode
* gh-112844: Update CPE references for external dependencies (#118521)Seth Michael Larson2024-05-201-1/+15
|
* GH-118943: Fix a race condition when generating jit_stencils.h (GH-118957)Brandt Bucher2024-05-161-7/+12
|
* Explain how to install LLVM on Fedora (GH-118983)Miro Hrončok2024-05-161-0/+6
| | | Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
* GH-118836: Fix JIT build error when SHT_NOTE section is present (GH-119000)Michał Górny2024-05-131-0/+1
|
* gh-117657: Log TSAN warnings to separate files and archive them (#118747)mpage2024-05-101-0/+3
| | | | | | | This ensures we don't lose races that occur in subprocesses or interleave races from workers running in parallel. Log files are collected and packaged into a zipfile that can be downloaded from the "Artifacts" section of the workflow run.
generated by cgit v0.12 at 2025-03-17 09:59:39 (GMT)