Commit message (Collapse) | Author | Age | Files | Lines | |
---|---|---|---|---|---|
* | gh-100445: Improve error message for unterminated strings with escapes (#100446) | Shantanu | 2023-10-18 | 1 | -2/+16 |
| | |||||
* | gh-107450: Check for overflow in the tokenizer and fix overflow test (#110832) | Lysandros Nikolaou | 2023-10-16 | 1 | -0/+4 |
| | | | | Co-authored-by: Filipe LaĆns <lains@riseup.net> Co-authored-by: Serhiy Storchaka <storchaka@gmail.com> | ||||
* | gh-104169: Refactor tokenizer into lexer and wrappers (#110684) | Lysandros Nikolaou | 2023-10-11 | 6 | -0/+1805 |
* The lexer, which include the actual lexeme producing logic, goes into the `lexer` directory. * The wrappers, one wrapper per input mode (file, string, utf-8, and readline), go into the `tokenizer` directory and include logic for creating a lexer instance and managing the buffer for different modes. --------- Co-authored-by: Pablo Galindo <pablogsal@gmail.com> Co-authored-by: blurb-it[bot] <43283697+blurb-it[bot]@users.noreply.github.com> |