summaryrefslogtreecommitdiffstats
path: root/Python/clinic
Commit message (Collapse)AuthorAgeFilesLines
* gh-113993: Allow interned strings to be mortal, and fix related issues ↵Petr Viktorin2024-06-211-6/+46
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (GH-120520) * Add an InternalDocs file describing how interning should work and how to use it. * Add internal functions to *explicitly* request what kind of interning is done: - `_PyUnicode_InternMortal` - `_PyUnicode_InternImmortal` - `_PyUnicode_InternStatic` * Switch uses of `PyUnicode_InternInPlace` to those. * Disallow using `_Py_SetImmortal` on strings directly. You should use `_PyUnicode_InternImmortal` instead: - Strings should be interned before immortalization, otherwise you're possibly interning a immortalizing copy. - `_Py_SetImmortal` doesn't handle the `SSTATE_INTERNED_MORTAL` to `SSTATE_INTERNED_IMMORTAL` update, and those flags can't be changed in backports, as they are now part of public API and version-specific ABI. * Add private `_only_immortal` argument for `sys.getunicodeinternedsize`, used in refleak test machinery. * Make sure the statically allocated string singletons are unique. This means these sets are now disjoint: - `_Py_ID` - `_Py_STR` (including the empty string) - one-character latin-1 singletons Now, when you intern a singleton, that exact singleton will be interned. * Add a `_Py_LATIN1_CHR` macro, use it instead of `_Py_ID`/`_Py_STR` for one-character latin-1 singletons everywhere (including Clinic). * Intern `_Py_STR` singletons at startup. * For free-threaded builds, intern `_Py_LATIN1_CHR` singletons at startup. * Beef up the tests. Cover internal details (marked with `@cpython_only`). * Add lots of assertions Co-Authored-By: Eric Snow <ericsnowcurrently@gmail.com>
* gh-111201: A new Python REPL (GH-111567)Pablo Galindo Salgado2024-05-051-1/+19
| | | | | | | Co-authored-by: Łukasz Langa <lukasz@langa.pl> Co-authored-by: Marta Gómez Macías <mgmacias@google.com> Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com> Co-authored-by: Hugo van Kemenade <1324225+hugovk@users.noreply.github.com> Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
* gh-117514: Add `sys._is_gil_enabled()` function (#118514)Sam Gross2024-05-031-1/+29
| | | | | The function returns `True` or `False` depending on whether the GIL is currently enabled. In the default build, it always returns `True` because the GIL is always enabled.
* gh-105879: Add support for keyword arguments to eval and exec (#105885)Raphael Gaschignard2024-05-021-24/+59
| | | Co-authored-by: Jelle Zijlstra <jelle.zijlstra@gmail.com>
* gh-117680: make _PyInstructionSequence a PyObject and use it in tests (#117629)Irit Katriel2024-04-171-0/+304
|
* gh-117764: Fix and add signatures for many builtins (GH-117769)Serhiy Storchaka2024-04-122-8/+8
|
* gh-117764: Add signatures for some functions in the sys module (GH-117770)Serhiy Storchaka2024-04-121-5/+29
| | | Use Argument Clinic if possible.
* GH-114695: Add `sys._clear_internal_caches` (GH-115152)Brandt Bucher2024-02-121-1/+19
|
* gh-76763: Make chr() always raising ValueError for out-of-range values ↵Serhiy Storchaka2024-02-101-20/+1
| | | | | | (GH-114882) Previously it raised OverflowError for very large or very small values.
* gh-113626: Add allow_code parameter in marshal functions (GH-113648)Serhiy Storchaka2024-01-161-26/+212
| | | | | Passing allow_code=False prevents serialization and de-serialization of code objects which is incompatible between Python versions.
* bpo-34392: Add sys. _is_interned() (GH-8755)Serhiy Storchaka2023-12-041-1/+35
|
* gh-111089: Revert PyUnicode_AsUTF8() changes (#111833)Victor Stinner2023-11-073-6/+21
| | | | | | | | | | | | | | | | | | | | | * Revert "gh-111089: Use PyUnicode_AsUTF8() in Argument Clinic (#111585)" This reverts commit d9b606b3d04fc56fb0bcc479d7d6c14562edb5e2. * Revert "gh-111089: Use PyUnicode_AsUTF8() in getargs.c (#111620)" This reverts commit cde1071b2a72e8261ca66053ef61431b7f3a81fd. * Revert "gh-111089: PyUnicode_AsUTF8() now raises on embedded NUL (#111091)" This reverts commit d731579bfb9a497cfb0076cb6b221058a20088fe. * Revert "gh-111089: Add PyUnicode_AsUTF8() to the limited C API (#111121)" This reverts commit d8f32be5b6a736dc2fc9dca3f1bf176c82fc9b44. * Revert "gh-111089: Use PyUnicode_AsUTF8() in sqlite3 (#111122)" This reverts commit 37e4e20eaa8f27ada926d49e5971fecf0477ad26.
* gh-111089: Use PyUnicode_AsUTF8() in Argument Clinic (#111585)Victor Stinner2023-11-013-21/+6
| | | | | | | Replace PyUnicode_AsUTF8AndSize() with PyUnicode_AsUTF8() to remove the explicit check for embedded null characters. The change avoids to have to include explicitly <string.h> to get the strlen() function when using a recent version of the limited C API.
* gh-67565: Remove redundant C-contiguity checks (GH-105521)Furkan Onder2023-10-232-10/+2
| | | | Co-authored-by: Stefan Krah <skrah@bytereef.org>
* gh-110964: Remove private _PyArg functions (#110966)Victor Stinner2023-10-179-9/+21
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Move the following private functions and structures to pycore_modsupport.h internal C API: * _PyArg_BadArgument() * _PyArg_CheckPositional() * _PyArg_NoKeywords() * _PyArg_NoPositional() * _PyArg_ParseStack() * _PyArg_ParseStackAndKeywords() * _PyArg_Parser structure * _PyArg_UnpackKeywords() * _PyArg_UnpackKeywordsWithVararg() * _PyArg_UnpackStack() * _Py_ANY_VARARGS() Changes: * Python/getargs.h now includes pycore_modsupport.h to export functions. * clinic.py now adds pycore_modsupport.h when one of these functions is used. * Add pycore_modsupport.h includes when a C extension uses one of these functions. * Define Py_BUILD_CORE_MODULE in C extensions which now include directly or indirectly (via code generated by Argument Clinic) pycore_modsupport.h: * _csv * _curses_panel * _dbm * _gdbm * _multiprocessing.posixshmem * _sqlite.row * _statistics * grp * resource * syslog * _testcapi: bad_get() no longer uses METH_FASTCALL calling convention but METH_VARARGS. Replace _PyArg_UnpackStack() with PyArg_ParseTuple(). * _testcapi: add PYTESTCAPI_NEED_INTERNAL_API macro which is defined by _testcapi sub-modules which need the internal C API (pycore_modsupport.h): exceptions.c, float.c, vectorcall.c, watchers.c. * Remove Include/cpython/modsupport.h header file. Include/modsupport.h no longer includes the removed header file. * Fix mypy clinic.py
* gh-109595: Add -Xcpu_count=<n> cmdline for container users (#109667)Donghee Na2023-10-101-1/+29
| | | | | | --------- Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Gregory P. Smith [Google LLC] <greg@krypto.org>
* gh-108753: Enhance pystats (#108754)Victor Stinner2023-09-061-6/+18
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Statistics gathering is now off by default. Use the "-X pystats" command line option or set the new PYTHONSTATS environment variable to 1 to turn statistics gathering on at Python startup. Statistics are no longer dumped at exit if statistics gathering was off or statistics have been cleared. Changes: * Add PYTHONSTATS environment variable. * sys._stats_dump() now returns False if statistics are not dumped because they are all equal to zero. * Add PyConfig._pystats member. * Add tests on sys functions and on setting PyConfig._pystats to 1. * Add Include/cpython/pystats.h and Include/internal/pycore_pystats.h header files. * Rename '_py_stats' variable to '_Py_stats'. * Exclude Include/cpython/pystats.h from the Py_LIMITED_API. * Move pystats.h include from object.h to Python.h. * Add _Py_StatsOn() and _Py_StatsOff() functions. Remove '_py_stats_struct' variable from the API: make it static in specialize.c. * Document API in Include/pystats.h and Include/cpython/pystats.h. * Complete pystats documentation in Doc/using/configure.rst. * Don't write "all zeros" stats: if _stats_off() and _stats_clear() or _stats_dump() were called. * _PyEval_Fini() now always call _Py_PrintSpecializationStats() which does nothing if stats are all zeros. Co-authored-by: Michael Droettboom <mdboom@gmail.com>
* gh-107603: Argument Clinic: Only include pycore_gc.h if needed (#108726)Victor Stinner2023-08-319-46/+22
| | | | | | | | | | | | | | | | | | | | Argument Clinic now only includes pycore_gc.h if PyGC_Head is needed, and only includes pycore_runtime.h if _Py_ID() is needed. * Add 'condition' optional argument to Clinic.add_include(). * deprecate_keyword_use() includes pycore_runtime.h when using the _PyID() function. * Fix rendering of includes: comments start at the column 35. * Mark PC/clinic/_wmimodule.cpp.h and "Objects/stringlib/clinic/*.h.h" header files as generated in .gitattributes. Effects: * 42 header files generated by AC no longer include the internal C API, instead of 4 header files before. For example, Modules/clinic/_abc.c.h no longer includes the internal C API. * Fix _testclinic_depr.c.h: it now always includes pycore_runtime.h to get _Py_ID().
* gh-106320: Remove private AC converter functions (#108505)Victor Stinner2023-08-261-1/+2
| | | | | | | | | | | | | | Move these private functions to the internal C API (pycore_abstract.h): * _Py_convert_optional_to_ssize_t() * _PyNumber_Index() Argument Clinic now emits #include "pycore_abstract.h" when these functions are used. The parser of the c-analyzer tool now uses a list of files which use the limited C API, rather than a list of files using the internal C API.
* gh-108444: Argument Clinic uses PyLong_AsInt() (#108458)Victor Stinner2023-08-247-37/+37
| | | | Argument Clinic now uses the new public PyLong_AsInt(), rather than the old name _PyLong_AsInt().
* gh-107526: Revert "gh-100357: Convert several functions in bltinsmodule to ↵Nikita Sobolev2023-08-211-195/+1
| | | | AC" (#107542)
* gh-92536: Argument Clinic no longer emits PyUnicode_READY() (#105208)Victor Stinner2023-06-014-34/+4
| | | | | | Since Python 3.12, PyUnicode_READY() does nothing and always returns 0. Argument Clinic now also checks for .cpp files (PC/_wmimodule.cpp).
* gh-105069: Add a readline-like callable to the tokenizer to consume input ↵Pablo Galindo Salgado2023-05-301-17/+24
| | | | iteratively (#105070)
* gh-102856: Python tokenizer implementation for PEP 701 (#104323)Marta Gómez Macías2023-05-211-8/+14
| | | | | | | | | | | This commit replaces the Python implementation of the tokenize module with an implementation that reuses the real C tokenizer via a private extension module. The tokenize module now implements a compatibility layer that transforms tokens from the C tokenizer into Python tokenize tokens for backward compatibility. As the C tokenizer does not emit some tokens that the Python tokenizer provides (such as comments and non-semantic newlines), a new special mode has been added to the C tokenizer mode that currently is only used via the extension module that exposes it to the Python layer. This new mode forces the C tokenizer to emit these new extra tokens and add the appropriate metadata that is needed to match the old Python implementation. Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
* gh-84436: Implement Immortal Objects (gh-19474)Eddie Elizondo2023-04-221-1/+29
| | | | | | | | | This is the implementation of PEP683 Motivation: The PR introduces the ability to immortalize instances in CPython which bypasses reference counting. Tagging objects as immortal allows up to skip certain operations when we know that the object will be around for the entire execution of the runtime. Note that this by itself will bring a performance regression to the runtime due to the extra reference count checks. However, this brings the ability of having truly immutable objects that are useful in other contexts such as immutable data sharing between sub-interpreters.
* GH-103082: Implementation of PEP 669: Low Impact Monitoring for CPython ↵Mark Shannon2023-04-121-0/+311
| | | | | | | | | | (GH-103083) * The majority of the monitoring code is in instrumentation.c * The new instrumentation bytecodes are in bytecodes.c * legacy_tracing.c adapts the new API to the old sys.setrace and sys.setprofile APIs
* gh-98627: Add an Optional Check for Extension Module Subinterpreter ↵Eric Snow2023-02-161-1/+32
| | | | | | | | | | | | | | | Compatibility (gh-99040) Enforcing (optionally) the restriction set by PEP 489 makes sense. Furthermore, this sets the stage for a potential restriction related to a per-interpreter GIL. This change includes the following: * add tests for extension module subinterpreter compatibility * add _PyInterpreterConfig.check_multi_interp_extensions * add Py_RTFLAGS_MULTI_INTERP_EXTENSIONS * add _PyImport_CheckSubinterpIncompatibleExtensionAllowed() * fail iff the module does not implement multi-phase init and the current interpreter is configured to check https://github.com/python/cpython/issues/98627
* gh-39615: Add warnings.warn() skip_file_prefixes support (#100840)Gregory P. Smith2023-01-281-10/+40
| | | | | `warnings.warn()` gains the ability to skip stack frames based on code filename prefix rather than only a numeric `stacklevel=` via a new `skip_file_prefixes=` keyword argument.
* gh-86682: Adds sys._getframemodulename as an alternative to using _getframe ↵Steve Dower2023-01-131-1/+70
| | | | | (GH-99520) Also updates calls in collections, doctest, enum, and typing modules to use _getframemodulename first when available.
* gh-100776: Fix misleading default value in help(input) (#100788)Shantanu2023-01-081-2/+2
|
* gh-100357: Convert several functions in `bltinsmodule` to AC (#100358)Nikita Sobolev2022-12-241-1/+195
|
* Correct CVE-2020-10735 documentation (#100306)Jeremy Paige2022-12-181-2/+2
|
* bpo-15999: Accept arbitrary values for boolean parameters. (#15609)Serhiy Storchaka2022-12-031-3/+3
| | | builtins and extension module functions and methods that expect boolean values for parameters now accept any Python object rather than just a bool or int type. This is more consistent with how native Python code itself behaves.
* GH-98686: Quicken everything (GH-98687)Brandt Bucher2022-11-021-28/+1
|
* gh-96143: Improve perf profiler docs (#96445)Erlend E. Aasland2022-10-271-4/+6
|
* gh-97841: Add methoddef for _filters_mutated (gh-98115)Dong-hee Na2022-10-091-1/+18
|
* gh-96526: Clarify format and __format__ docstrings (gh-96648)Michael2022-10-031-5/+8
|
* Fix typos in `bltinmodule.c`. (GH-97766)Nikita Sobolev2022-10-031-5/+5
|
* gh-95778: CVE-2020-10735: Prevent DoS by very large int() (#96499)Gregory P. Smith2022-09-021-1/+77
| | | | | | | | | | | | | | | | Integer to and from text conversions via CPython's bignum `int` type is not safe against denial of service attacks due to malicious input. Very large input strings with hundred thousands of digits can consume several CPU seconds. This PR comes fresh from a pile of work done in our private PSRT security response team repo. Signed-off-by: Christian Heimes [Red Hat] <christian@python.org> Tons-of-polishing-up-by: Gregory P. Smith [Google] <greg@krypto.org> Reviews via the private PSRT repo via many others (see the NEWS entry in the PR). <!-- gh-issue-number: gh-95778 --> * Issue: gh-95778 <!-- /gh-issue-number --> I wrote up [a one pager for the release managers](https://docs.google.com/document/d/1KjuF_aXlzPUxTK4BMgezGJ2Pn7uevfX7g0_mvgHlL7Y/edit#). Much of that text wound up in the Issue. Backports PRs already exist. See the issue for links.
* gh-96143: Allow Linux perf profiler to see Python calls (GH-96123)Pablo Galindo Salgado2022-08-301-1/+74
| | | | | | | :warning: :warning: Note for reviewers, hackers and fellow systems/low-level/compiler engineers :warning: :warning: If you have a lot of experience with this kind of shenanigans and want to improve the **first** version, **please make a PR against my branch** or **reach out by email** or **suggest code changes directly on GitHub**. If you have any **refinements or optimizations** please, wait until the first version is merged before starting hacking or proposing those so we can keep this PR productive.
* GH-93503: Add thread-specific APIs to set profiling and tracing functions in ↵Pablo Galindo Salgado2022-08-241-1/+25
| | | | | | | | | | | | | | | | | the C-API (#93504) * gh-93503: Add APIs to set profiling and tracing functions in all threads in the C-API * Use a separate API * Fix NEWS entry * Add locks around the loop * Document ignoring exceptions * Use the new APIs in the sys module * Update docs
* gh-90928: Improve static initialization of keywords tuple in AC (#95907)Erlend E. Aasland2022-08-136-261/+111
|
* gh-90928: Statically Initialize the Keywords Tuple in Clinic-Generated Code ↵Eric Snow2022-08-118-23/+566
| | | | | | | | | | | | | | | | (gh-95860) We only statically initialize for core code and builtin modules. Extension modules still create the tuple at runtime. We'll solve that part of interpreter isolation separately. This change includes generated code. The non-generated changes are in: * Tools/clinic/clinic.py * Python/getargs.c * Include/cpython/modsupport.h * Makefile.pre.in (re-generate global strings after running clinic) * very minor tweaks to Modules/_codecsmodule.c and Python/Python-tokenize.c All other changes are generated code (clinic, global strings).
* gh-91102: Port 8-argument _warnings.warn_explicit to Argument Clinic (#92891)Oleg Iarygin2022-07-201-1/+82
| | | Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>
* GH-93841: Allow stats to be turned on and off, cleared and dumped at ↵Mark Shannon2022-06-211-1/+105
| | | | runtime. (GH-93843)
* gh-92203: Add closure support to exec(). (#92204)larryhastings2022-05-061-11/+26
| | | Add a closure keyword-only parameter to exec(). It can only be specified when exec-ing a code object that uses free variables. When specified, it must be a tuple, with exactly the number of cell variables referenced by the code object. closure has a default value of None, and it must be None if the code object doesn't refer to any free variables.
* gh-91320: Argument Clinic uses _PyCFunction_CAST() (#32210)Victor Stinner2022-05-036-39/+39
| | | | Replace "(PyCFunction)(void(*)(void))func" cast with _PyCFunction_CAST(func).
* Use FASTCALL for __import__ (GH-31752)Kumar Aditya2022-03-111-1/+80
|
* bpo-46328: Add sys.exception() (GH-30514)Irit Katriel2022-01-131-1/+23
|
* bpo-21736: Set __file__ on frozen stdlib modules. (gh-28656)Eric Snow2021-10-141-10/+27
| | | | | | | | | | | | | | | | | | | | | | | | Currently frozen modules do not have __file__ set. In their spec, origin is set to "frozen" and they are marked as not having a location. (Similarly, for frozen packages __path__ is set to an empty list.) However, for frozen stdlib modules we are able to extrapolate __file__ as long as we can determine the stdlib directory at runtime. (We now do so since gh-28586.) Having __file__ set is helpful for a number of reasons. Likewise, having a non-empty __path__ means we can import submodules of a frozen package from the filesystem (e.g. we could partially freeze the encodings module). This change sets __file__ (and adds to __path__) for frozen stdlib modules. It uses sys._stdlibdir (from gh-28586) and the frozen module alias information (from gh-28655). All that work is done in FrozenImporter (in Lib/importlib/_bootstrap.py). Also, if a frozen module is imported before importlib is bootstrapped (during interpreter initialization) then we fix up that module and its spec during the importlib bootstrapping step (i.e. imporlib._bootstrap._setup()) to match what gets set by FrozenImporter, including setting the file info (if the stdlib dir is known). To facilitate this, modules imported using PyImport_ImportFrozenModule() have __origname__ set using the frozen module alias info. __origname__ is popped off during importlib bootstrap. (To be clear, even with this change the new code to set __file__ during fixups in imporlib._bootstrap._setup() doesn't actually get triggered yet. This is because sys._stdlibdir hasn't been set yet in interpreter initialization at the point importlib is bootstrapped. However, we do fix up such modules at that point to otherwise match the result of importing through FrozenImporter, just not the __file__ and __path__ parts. Doing so will require changes in the order in which things happen during interpreter initialization. That can be addressed separately. Once it is, the file-related fixup code from this PR will kick in.) Here are things this change does not do: * set __file__ for non-stdlib modules (no way of knowing the parent dir) * set __file__ if the stdlib dir is not known (nor assume the expense of finding it) * relatedly, set __file__ if the stdlib is in a zip file * verify that the filename set to __file__ actually exists (too expensive) * update __path__ for frozen packages that alias a non-package (since there is no package dir) Other things this change skips, but we may do later: * set __file__ on modules imported using PyImport_ImportFrozenModule() * set co_filename when we unmarshal the frozen code object while importing the module (e.g. in FrozenImporter.exec_module()) -- this would allow tracebacks to show source lines * implement FrozenImporter.get_filename() and FrozenImporter.get_source() https://bugs.python.org/issue21736
generated by cgit v0.12 at 2025-08-14 02:47:07 (GMT)