summaryrefslogtreecommitdiffstats
Commit message (Collapse)AuthorAgeFilesLines
...
* gh-121352: use _Py_SourceLocation in symtable (#121353)Irit Katriel2024-07-042-133/+69
|
* gh-120754: Reduce system calls in full-file FileIO.readall() case (#120755)Cody Maloney2024-07-043-33/+60
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | This reduces the system call count of a simple program[0] that reads all the `.rst` files in Doc by over 10% (5706 -> 4734 system calls on my linux system, 5813 -> 4875 on my macOS) This reduces the number of `fstat()` calls always and seek calls most the time. Stat was always called twice, once at open (to error early on directories), and a second time to get the size of the file to be able to read the whole file in one read. Now the size is cached with the first call. The code keeps an optimization that if the user had previously read a lot of data, the current position is subtracted from the number of bytes to read. That is somewhat expensive so only do it on larger files, otherwise just try and read the extra bytes and resize the PyBytes as needeed. I built a little test program to validate the behavior + assumptions around relative costs and then ran it under `strace` to get a log of the system calls. Full samples below[1]. After the changes, this is everything in one `filename.read_text()`: ```python3 openat(AT_FDCWD, "cpython/Doc/howto/clinic.rst", O_RDONLY|O_CLOEXEC) = 3` fstat(3, {st_mode=S_IFREG|0644, st_size=343, ...}) = 0` ioctl(3, TCGETS, 0x7ffdfac04b40) = -1 ENOTTY (Inappropriate ioctl for device) lseek(3, 0, SEEK_CUR) = 0 read(3, ":orphan:\n\n.. This page is retain"..., 344) = 343 read(3, "", 1) = 0 close(3) = 0 ``` This does make some tradeoffs 1. If the file size changes between open() and readall(), this will still get all the data but might have more read calls. 2. I experimented with avoiding the stat + cached result for small files in general, but on my dev workstation at least that tended to reduce performance compared to using the fstat(). [0] ```python3 from pathlib import Path nlines = [] for filename in Path("cpython/Doc").glob("**/*.rst"): nlines.append(len(filename.read_text())) ``` [1] Before small file: ``` openat(AT_FDCWD, "cpython/Doc/howto/clinic.rst", O_RDONLY|O_CLOEXEC) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=343, ...}) = 0 ioctl(3, TCGETS, 0x7ffe52525930) = -1 ENOTTY (Inappropriate ioctl for device) lseek(3, 0, SEEK_CUR) = 0 lseek(3, 0, SEEK_CUR) = 0 fstat(3, {st_mode=S_IFREG|0644, st_size=343, ...}) = 0 read(3, ":orphan:\n\n.. This page is retain"..., 344) = 343 read(3, "", 1) = 0 close(3) = 0 ``` After small file: ``` openat(AT_FDCWD, "cpython/Doc/howto/clinic.rst", O_RDONLY|O_CLOEXEC) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=343, ...}) = 0 ioctl(3, TCGETS, 0x7ffdfac04b40) = -1 ENOTTY (Inappropriate ioctl for device) lseek(3, 0, SEEK_CUR) = 0 read(3, ":orphan:\n\n.. This page is retain"..., 344) = 343 read(3, "", 1) = 0 close(3) = 0 ``` Before large file: ``` openat(AT_FDCWD, "cpython/Doc/c-api/typeobj.rst", O_RDONLY|O_CLOEXEC) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=133104, ...}) = 0 ioctl(3, TCGETS, 0x7ffe52525930) = -1 ENOTTY (Inappropriate ioctl for device) lseek(3, 0, SEEK_CUR) = 0 lseek(3, 0, SEEK_CUR) = 0 fstat(3, {st_mode=S_IFREG|0644, st_size=133104, ...}) = 0 read(3, ".. highlight:: c\n\n.. _type-struc"..., 133105) = 133104 read(3, "", 1) = 0 close(3) = 0 ``` After large file: ``` openat(AT_FDCWD, "cpython/Doc/c-api/typeobj.rst", O_RDONLY|O_CLOEXEC) = 3 fstat(3, {st_mode=S_IFREG|0644, st_size=133104, ...}) = 0 ioctl(3, TCGETS, 0x7ffdfac04b40) = -1 ENOTTY (Inappropriate ioctl for device) lseek(3, 0, SEEK_CUR) = 0 lseek(3, 0, SEEK_CUR) = 0 read(3, ".. highlight:: c\n\n.. _type-struc"..., 133105) = 133104 read(3, "", 1) = 0 close(3) = 0 ``` Co-authored-by: Shantanu <12621235+hauntsaninja@users.noreply.github.com> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com> Co-authored-by: Victor Stinner <vstinner@python.org>
* gh-121141: add support for `copy.replace` to AST nodes (#121162)Bénédikt Tran2024-07-045-2/+837
|
* gh-117983: Defer import of threading for lazy module loading (#120233)Chris Markiewicz2024-07-032-1/+5
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | As noted in gh-117983, the import importlib.util can be triggered at interpreter startup under some circumstances, so adding threading makes it a potentially obligatory load. Lazy loading is not used in the stdlib, so this removes an unnecessary load for the majority of users and slightly increases the cost of the first lazily loaded module. An obligatory threading load breaks gevent, which monkeypatches the stdlib. Although unsupported, there doesn't seem to be an offsetting benefit to breaking their use case. For reference, here are benchmarks for the current main branch: ``` ❯ hyperfine -w 8 './python -c "import importlib.util"' Benchmark 1: ./python -c "import importlib.util" Time (mean ± σ): 9.7 ms ± 0.7 ms [User: 7.7 ms, System: 1.8 ms] Range (min … max): 8.4 ms … 13.1 ms 313 runs ``` And with this patch: ``` ❯ hyperfine -w 8 './python -c "import importlib.util"' Benchmark 1: ./python -c "import importlib.util" Time (mean ± σ): 8.4 ms ± 0.7 ms [User: 6.8 ms, System: 1.4 ms] Range (min … max): 7.2 ms … 11.7 ms 352 runs ``` Compare to: ``` ❯ hyperfine -w 8 './python -c pass' Benchmark 1: ./python -c pass Time (mean ± σ): 7.6 ms ± 0.6 ms [User: 5.9 ms, System: 1.6 ms] Range (min … max): 6.7 ms … 11.3 ms 390 runs ``` This roughly halves the import time of importlib.util.
* gh-118714: Make the pdb post-mortem restart/quit behavior more reasonable ↵Tian Gao2024-07-033-3/+25
| | | | (#118725)
* gh-112136: Restore removed _PyArg_Parser (#121262)Victor Stinner2024-07-038-24/+40
| | | | | | | Restore the private _PyArg_Parser structure and the private _PyArg_ParseTupleAndKeywordsFast() function, previously removed in Python 3.13 alpha 1. Recreate Include/cpython/modsupport.h header file.
* gh-121300: Add `replace` to `copy.__all__` (#121302)Max Muoto2024-07-032-3/+8
|
* gh-121201: Disable perf_trampoline on riscv64 for now (#121328)Stefano Rivera2024-07-034-5/+0
| | | | | | | Disable perf_trampoline on riscv64 for now Until support is added in perf_jit_trampoline.c gh-120089 was incomplete.
* GH-119726: Emit AArch64 trampolines out-of-line (GH-121280)Diego Russo2024-07-032-8/+8
|
* gh-121035: Update PNG image for logging flow diagram. (GH-121323)Vinay Sajip2024-07-031-0/+0
|
* gh-121245: a regression test for site.register_readline() (#121259)Sergey B Kirpichev2024-07-031-0/+27
|
* gh-121263: Macro-ify most stackref functions for MSVC (GH-121270)Ken Jin2024-07-031-58/+30
| | | Macro-ify most stackref functions for MSVC
* gh-121272: set ste_coroutine during symtable construction (#121297)Irit Katriel2024-07-032-3/+15
| | | compiler no longer modifies the symtable after this.
* gh-61103: Support float and long double complex types in ctypes module (#121248)Sergey B Kirpichev2024-07-0311-10/+135
| | | | | | | | | | | This amends 6988ff02a5: memory allocation for stginfo->ffi_type_pointer.elements in PyCSimpleType_init() should be more generic (perhaps someday fmt->pffi_type->elements will be not a two-elements array). It should finally resolve #61103. Co-authored-by: Victor Stinner <vstinner@python.org> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>
* gh-106597: Add more offsets to _Py_DebugOffsets (#121311)Gabriele N. Tornetta2024-07-032-0/+18
| | | | | | Add more offsets to _Py_DebugOffsets We add a few more offsets that are required by some out-of-process tools, such as [Austin](https://github.com/p403n1x87/austin).
* Docs: Add `os.splice` flags argument (#109847)Amin Alaee2024-07-031-1/+20
| | | | Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com> Co-authored-by: Blaise Pabon <blaise@gmail.com>
* docs: Fix "Py_TPFLAGS_MANAGED_WEAKREF is set in tp_flags" (#112237)da-woods2024-07-031-2/+2
|
* build(deps): bump hypothesis from 6.100.2 to 6.104.2 in /Tools (#121218)dependabot[bot]2024-07-031-1/+1
| | | | | | | | | | | | | | | Bumps [hypothesis](https://github.com/HypothesisWorks/hypothesis) from 6.100.2 to 6.104.2. - [Release notes](https://github.com/HypothesisWorks/hypothesis/releases) - [Commits](https://github.com/HypothesisWorks/hypothesis/compare/hypothesis-python-6.100.2...hypothesis-python-6.104.2) --- updated-dependencies: - dependency-name: hypothesis dependency-type: direct:production update-type: version-update:semver-minor ... Signed-off-by: dependabot[bot] <support@github.com> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* updated tp_flags initialization to use inplace or (#120625)byundojin2024-07-031-1/+1
|
* gh-111872: Document the max_children attribute for ↵AN Long2024-07-031-0/+6
| | | | `socketserver.ForkingMixIn` (#118134)
* gh-116181: Remove Py_BUILD_CORE_BUILTIN and Py_BUILD_CORE_MODULE in ↵AN Long2024-07-031-5/+0
| | | | rotatingtree.c (#121260)
* gh-121027: Make the functools.partial object a method descriptor (GH-121089)Serhiy Storchaka2024-07-036-40/+28
| | | Co-authored-by: d.grigonis <dgrigonis@users.noreply.github.com>
* GH-73991: Support copying directory symlinks on older Windows (#120807)Barney Gale2024-07-036-29/+40
| | | | | Check for `ERROR_INVALID_PARAMETER` when calling `_winapi.CopyFile2()` and raise `UnsupportedOperation`. In `Path.copy()`, handle this exception and fall back to the `PathBase.copy()` implementation.
* gh-121035: Further improve logging flow diagram with respect to dark/light ↵Vinay Sajip2024-07-022-2/+14
| | | | modes. (GH-121265)
* gh-115773: Add sizes to debug offset structure (#120112)Pablo Galindo Salgado2024-07-022-0/+19
|
* gh-117139: Add _PyTuple_FromStackRefSteal and use it (#121244)Sam Gross2024-07-028-33/+28
| | | Avoids the extra conversion from stack refs to PyObjects.
* gh-121272: move __future__ import validation from compiler to symtable (#121273)Irit Katriel2024-07-022-16/+24
|
* gh-121165: protect macro expansion of `ADJUST_INDICES` with do-while(0) ↵Bénédikt Tran2024-07-022-26/+36
| | | | (#121166)
* gh-121210: handle nodes with missing attributes/fields in `ast.compare` ↵Bénédikt Tran2024-07-023-4/+36
| | | | (#121211)
* gh-121245: Amend d611c4c8e9 (correct import) (#121255)Sergey B Kirpichev2024-07-022-2/+3
| | | Co-authored-by: Miro Hrončok <miro@hroncok.cz>
* Move get_signal_name() to test.support (#121251)Victor Stinner2024-07-026-58/+65
| | | | | | * Move get_signal_name() from test.libregrtest to test.support. * Use get_signal_name() in support.script_helper. * support.script_helper now decodes stdout and stderr from UTF-8, instead of ASCII, if a command failed.
* gh-121035: Improve logging flow diagram for dark/light modes. (GH-121254)Vinay Sajip2024-07-022-11/+62
|
* Fix phrasing in paragraphs with leading "similar" (#121135)Rafael Fontenelle2024-07-021-2/+2
|
* GH-119726: Use LDR for AArch64 trampolines (GH-121001)Diego Russo2024-07-012-22/+11
|
* gh-121196: Document `dict.fromkeys` params as pos-only (#121197)sobolevn2024-07-011-1/+1
|
* GH-116017: Get rid of _COLD_EXITs (GH-120960)Brandt Bucher2024-07-0116-337/+246
|
* gh-117657: Fix data races reported by TSAN in some set methods (#120914)AN Long2024-07-016-121/+81
| | | | | | Refactor the fast Unicode hash check into `_PyObject_HashFast` and use relaxed atomic loads in the free-threaded build. After this change, the TSAN doesn't report data races for this method.
* gh-121110: Temporarily Skip test_basic_multiple_interpreters_reset_each ↵Eric Snow2024-07-011-0/+7
| | | | | (gh-121236) This will allow Py_TRACE_REFS builds to pass the test suite, until the underlying issue can be resolved.
* gh-114104: clarify asynchronous comprehension docs to match runtime behavior ↵Danny Yang2024-07-011-4/+6
| | | | (#121175)
* gh-120743: Soft deprecate os.popen() function (#120744)Victor Stinner2024-07-013-1/+17
| | | Soft deprecate os.popen() and os.spawn*() functions.
* gh-121200: Fix test_expanduser_pwd2() of test_posixpath (#121228)Victor Stinner2024-07-012-3/+11
| | | | Call getpwnam() to get pw_dir, since it can be different than getpwall() pw_dir.
* gh-117657: Use critical section to make _socket.socket.close thread safe ↵AN Long2024-07-013-10/+41
| | | | (GH-120490)
* gh-121220: Mark test_threaded_weak_value_dict_copy() as CPU-heavy (#121221)Kirill Podoprigora2024-07-011-0/+2
| | | | | Mark test_threaded_weak_value_dict_copy() and test_threaded_weak_key_dict_copy() of test_weakref as CPU-heavy tests
* gh-117784: Only reference PHA functions ifndef SSL_VERIFY_POST_HANDSHAKE ↵Will Childs-Klein2024-07-012-11/+18
| | | | | | (GH-117785) With this change, builds with OpenSSL forks that don't have this functionalty (like AWS-LC or BoringSSL) will require less patching.
* build(deps-dev): bump types-setuptools from 70.0.0.20240524 to ↵dependabot[bot]2024-07-011-1/+1
| | | | | | 70.1.0.20240627 in /Tools (#121217) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* build(deps-dev): bump mypy from 1.10.0 to 1.10.1 in /Tools (#121216)dependabot[bot]2024-07-011-1/+1
| | | | Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* build(deps-dev): bump types-psutil from 5.9.5.20240516 to 6.0.0.20240621 in ↵dependabot[bot]2024-07-011-1/+1
| | | | | | /Tools (#121215) Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* gh-121200: Log pwd entry in test_expanduser_pwd2() (#121207)Victor Stinner2024-07-011-6/+7
| | | | Use subTest() to log the pwd entry in test_expanduser_pwd2() of test_posixpath to help debugging.
* gh-121084: Call _abc_registry_clear() when checking refleaks (#121191)Victor Stinner2024-07-011-3/+7
| | | | dash_R_cleanup() now calls _abc_registry_clear() before calling again register().
* gh-61103: Support double complex (_Complex) type in ctypes (#120894)Sergey B Kirpichev2024-07-0117-17/+316
| | | | | | | | | | | | | | | | | | Example: ```pycon >>> import ctypes >>> ctypes.__STDC_IEC_559_COMPLEX__ 1 >>> libm = ctypes.CDLL('libm.so.6') >>> libm.clog.argtypes = [ctypes.c_double_complex] >>> libm.clog.restype = ctypes.c_double_complex >>> libm.clog(1+1j) (0.34657359027997264+0.7853981633974483j) ``` Co-authored-by: Nice Zombies <nineteendo19d0@gmail.com> Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com> Co-authored-by: Victor Stinner <vstinner@python.org>
generated by cgit v0.12 at 2025-08-26 06:46:54 (GMT)