diff options
| author | Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com> | 2024-05-28 20:47:45 (GMT) |
|---|---|---|
| committer | GitHub <noreply@github.com> | 2024-05-28 20:47:45 (GMT) |
| commit | 0d0be6b3efeace4743329f81c08f9720cc221207 (patch) | |
| tree | afeec3b81a48161eef6c675658a9e7aac55aaf36 /Misc | |
| parent | c0e99617985d64e6134964f758ae0a1a20f9f433 (diff) | |
| download | cpython-0d0be6b3efeace4743329f81c08f9720cc221207.zip cpython-0d0be6b3efeace4743329f81c08f9720cc221207.tar.gz cpython-0d0be6b3efeace4743329f81c08f9720cc221207.tar.bz2 | |
[3.13] gh-119118: Fix performance regression in tokenize module (GH-119615) (#119682)
- Cache line object to avoid creating a Unicode object
for all of the tokens in the same line.
- Speed up byte offset to column offset conversion by using the
smallest buffer possible to measure the difference.
(cherry picked from commit d87b0151062e36e67f9e42e1595fba5bf23a485c)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
Diffstat (limited to 'Misc')
| -rw-r--r-- | Misc/NEWS.d/next/Library/2024-05-28-12-15-03.gh-issue-119118.FMKz1F.rst | 2 |
1 files changed, 2 insertions, 0 deletions
diff --git a/Misc/NEWS.d/next/Library/2024-05-28-12-15-03.gh-issue-119118.FMKz1F.rst b/Misc/NEWS.d/next/Library/2024-05-28-12-15-03.gh-issue-119118.FMKz1F.rst new file mode 100644 index 0000000..3cf6166 --- /dev/null +++ b/Misc/NEWS.d/next/Library/2024-05-28-12-15-03.gh-issue-119118.FMKz1F.rst @@ -0,0 +1,2 @@ +Fix performance regression in the :mod:`tokenize` module by caching the ``line`` +token attribute and calculating the column offset more efficiently. |