| Commit message (Collapse) | Author | Age | Files | Lines |
| |
|
|
|
|
|
|
| |
implicit encodings (GH-115824) (#115949)
gh-115823: Calculate correctly error locations when dealing with implicit encodings (GH-115824)
(cherry picked from commit 015b97d19a24a169cc3c0939119e1228791e4253)
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
|
| |
|
|
|
|
|
|
|
| |
(GH-113607) (#113652)
gh-113602: Bail out when the parser tries to override existing errors (GH-113607)
(cherry picked from commit 9ed36d533ab8b256f0a589b5be6d7a2fdcf4aff2)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
|
| |
|
|
|
|
|
|
|
| |
overwrite tokenizer errors (GH-112410) (#112466)
gh-112388: Fix an error that was causing the parser to try to overwrite tokenizer errors (GH-112410)
(cherry picked from commit 2c8b19174274c183eb652932871f60570123fe99)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
|
| |
|
|
|
|
|
|
|
| |
tokenize errors (GH-112409) (#112468)
gh-112387: Fix error positions for decoded strings with backwards tokenize errors (GH-112409)
(cherry picked from commit 45d648597b1146431bf3d91041e60d7f040e70bf)
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
Co-authored-by: Pablo Galindo Salgado <Pablogsal@gmail.com>
|
| |
|
|
|
|
|
|
|
| |
(GH-110832) (#110931)
(cherry picked from commit a1ac5590e0f8fe008e5562d22edab65d0c1c5507)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Filipe Laíns <lains@riseup.net>
Co-authored-by: Serhiy Storchaka <storchaka@gmail.com>
|
| |
|
|
|
|
|
| |
(GH-110754) (#110762)
(cherry picked from commit fb7843ee895ac7f6eeb58f356b1a320eea081cfc)
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
|
| |
|
|
|
| |
fix incorrect soft keywords (GH-109606). (#109752)
(cherry picked from commit b28ffaa193efc66f46ab90d383279174a11a11d7)
|
| |
|
|
|
|
|
|
| |
(#108263)
gh-108179: Add error message for parser stack overflows (GH-108256)
(cherry picked from commit 86617518c4ac824e2b6dc20691ba5a08df04f285)
Co-authored-by: Dennis Sweeney <36520290+sweeneyde@users.noreply.github.com>
|
| |
|
|
|
|
|
|
|
|
|
| |
This commit replaces the Python implementation of the tokenize module with an implementation
that reuses the real C tokenizer via a private extension module. The tokenize module now implements
a compatibility layer that transforms tokens from the C tokenizer into Python tokenize tokens for backward
compatibility.
As the C tokenizer does not emit some tokens that the Python tokenizer provides (such as comments and non-semantic newlines), a new special mode has been added to the C tokenizer mode that currently is only used via
the extension module that exposes it to the Python layer. This new mode forces the C tokenizer to emit these new extra tokens and add the appropriate metadata that is needed to match the old Python implementation.
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
|
| |
|
|
|
| |
(GH-103896)
Co-authored-by: Pablo Galindo <pablogsal@gmail.com>
|
| |
|
|
|
|
| |
Co-authored-by: Lysandros Nikolaou <lisandrosnik@gmail.com>
Co-authored-by: Batuhan Taskaya <isidentical@gmail.com>
Co-authored-by: Marta Gómez Macías <mgmacias@google.com>
Co-authored-by: sunmy2019 <59365878+sunmy2019@users.noreply.github.com>
|
| |
|
|
|
| |
in the tokenizer (GH-100065)
Automerge-Triggered-By: GH:pablogsal
|
| |
|
|
|
| |
Right now, the tokenizer only returns type and two pointers to the start and end of the token.
This PR modifies the tokenizer to return the type and set all of the necessary information,
so that the parser does not have to this.
|
| | |
|
| |
|
|
|
|
|
|
|
|
|
| |
* bpo-14916: interactive fd is not always stdin
related to https://github.com/python/cpython/pull/31006 merged bugfix
following https://bugs.python.org/issue14916
* 📜🤖 Added by blurb_it.
Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com>
|
| |
|
|
|
|
|
|
|
|
|
| |
errors from stdin (#94386)
* gh-94360: Fix a tokenizer crash when reading encoded files with syntax errors from stdin
Signed-off-by: Pablo Galindo <pablogsal@gmail.com>
* nitty nit
Co-authored-by: Łukasz Langa <lukasz@langa.pl>
|
| |
|
|
|
| |
buffers are uninitialized (GH-32129)
Automerge-Triggered-By: GH:pablogsal
|
| | |
|
| |
|
|
| |
'get_error_line_from_tokenizer_buffers' (#30545)
|
| |
|
|
|
| |
f-strings (GH-30529)
Automerge-Triggered-By: GH:pablogsal
|
| |
|
|
| |
not finished (GH-30378)
|
| |
|
|
| |
inside parentheses (GH-29757)
|
| | |
|
| |
|