bpo-44201: Avoid side effects of "invalid_*" rules in the REPL (GH-26298)

When the parser does a second pass to check for errors, these rules can
have some small side-effects as they may advance the parser more than
the point reached in the first pass. This can cause the tokenizer to ask
for extra tokens in interactive mode causing the tokenizer to show the
prompt instead of failing instantly.

To avoid this, add a new mode to the tokenizer that is activated in the
second pass and deactivates asking for new tokens when the interactive
line is finished. As the parsing should have reached the last line in
the first pass, the second pass should not need to ask for more tokens.

1 files changed, 1 insertions, 0 deletions

diff --git a/Include/errcode.h b/Include/errcode.h
index f2671d6..2e07fc2 100644
--- a/Include/errcode.h
+++ b/Include/errcode.h
@@ -28,6 +28,7 @@ extern "C" {
 #define E_DECODE        22      /* Error in decoding into Unicode */
 #define E_LINECONT      25      /* Unexpected characters after a line continuation */
 #define E_BADSINGLE     27      /* Ill-formed single statement input */
+#define E_INTERACT_STOP 28      /* Interactive mode stopped tokenization */
 
 #ifdef __cplusplus
 }