Make identifiers str (not str8) objects throughout.

This affects the parser, various object implementations,
and all places that put identifiers into C string literals.

In testing, a number of crashes occurred as code would
fail when the recursion limit was reached (such as the
Unicode interning dictionary having key/value pairs where
key is not value). To solve these, I added an overflowed
flag, which allows for 50 more recursions after the
limit was reached and the exception was raised, and
a recursion_critical flag, which indicates that recursion
absolutely must be allowed, i.e. that a certain call
must not cause a stack overflow exception.

There are still some places where both str and str8 are
accepted as identifiers; these should eventually be
removed.

1 files changed, 11 insertions, 1 deletions

diff --git a/Objects/stringobject.c b/Objects/stringobject.c
index 60e6129..92bc95b 100644
--- a/Objects/stringobject.c
+++ b/Objects/stringobject.c
@@ -686,6 +686,11 @@ string_getbuffer(register PyObject *op)
 Py_ssize_t
 PyString_Size(register PyObject *op)
 {
+	if (PyUnicode_Check(op)) {
+		op = _PyUnicode_AsDefaultEncodedString(op, NULL);
+		if (!op)
+			return -1;
+	}
 	if (!PyString_Check(op))
 		return string_getsize(op);
 	return ((PyStringObject *)op) -> ob_size;
@@ -694,6 +699,11 @@ PyString_Size(register PyObject *op)
 /*const*/ char *
 PyString_AsString(register PyObject *op)
 {
+	if (PyUnicode_Check(op)) {
+		op = _PyUnicode_AsDefaultEncodedString(op, NULL);
+		if (!op)
+			return NULL;
+	}
 	if (!PyString_Check(op))
 		return string_getbuffer(op);
 	return ((PyStringObject *)op) -> ob_sval;
@@ -824,7 +834,7 @@ PyString_Repr(PyObject *obj, int smartquotes)
 {
 	static const char *hexdigits = "0123456789abcdef";
 	register PyStringObject* op = (PyStringObject*) obj;
-	Py_ssize_t length = PyUnicode_GET_SIZE(op);
+	Py_ssize_t length = PyString_GET_SIZE(op);
 	size_t newsize = 2 + 4 * op->ob_size;
 	PyObject *v;
 	if (newsize > PY_SSIZE_T_MAX || newsize / 4 != op->ob_size) {