[2992970] Restore the safety of Tcl_AppendObjToObj(x, x) for bytearrays.bug_2992970

Also moves overflow checking to TclAppendBytesToByteArray() and adds
the ability to call TABTBA() with bytes==NULL, for appending unspecified
bytes.  That is, the string grows, but the new bytes are of undetermined
value.  Like Tcl_NewByteArrayObj(NULL, length) this option is useful
for manipulating buffers.  The TABTBA growth algorithm is also enhanced
a bit, copying over a fuller implementation from GrowStringBuffer()
in tclStringObj.c

1 files changed, 31 insertions, 11 deletions

diff --git a/generic/tclStringObj.c b/generic/tclStringObj.c
index e495c2e..1512f0c 100644
--- a/generic/tclStringObj.c
+++ b/generic/tclStringObj.c
@@ -1281,23 +1281,43 @@ Tcl_AppendObjToObj(
 
     if ((TclIsPureByteArray(objPtr) || objPtr->bytes == tclEmptyStringRep)
 	    && TclIsPureByteArray(appendObjPtr)) {
-	unsigned char *bytesSrc;
-	int lengthSrc, lengthTotal;
+	int lengthSrc;
 
 	/*
-	 * We do not assume that objPtr and appendObjPtr must be distinct!
-	 * This makes this code a bit more complex than it otherwise would be,
-	 * but in turn makes it much safer.
+	 * You might expect the code here to be
+	 *
+
+	    bytes = Tcl_GetByteArrayFromObj(appendObjPtr, &length);
+	    TclAppendBytesToByteArray(objPtr, bytes, length);
+
+	 *
+	 * and essentially all of the time that would be fine.  However,
+	 * it would run into trouble in the case where objPtr and
+	 * appendObjPtr point to the same thing.  That may never be a
+	 * good idea.  It seems to violate Copy On Write, and we don't
+	 * have any tests for the situation, since making any Tcl commands
+	 * that call Tcl_AppendObjToObj() do that appears impossible
+	 * (They honor Copy On Write!).  For the sake of extensions that
+	 * go off into that realm, though, here's a more complex approach
+	 * that can handle all the cases.
 	 */
 
+	/* Get lengths */
 	(void) Tcl_GetByteArrayFromObj(objPtr, &length);
 	(void) Tcl_GetByteArrayFromObj(appendObjPtr, &lengthSrc);
-	lengthTotal = length + lengthSrc;
-	if (((length > lengthSrc) ? length : lengthSrc) > lengthTotal) {
-	    Tcl_Panic("max size for a Tcl value (%d bytes) exceeded", INT_MAX);
-	}
-	bytesSrc = Tcl_GetByteArrayFromObj(appendObjPtr, NULL);
-	TclAppendBytesToByteArray(objPtr, bytesSrc, lengthSrc);
+
+	/* Grow buffer enough for the append */
+	TclAppendBytesToByteArray(objPtr, NULL, lengthSrc);
+
+	/* Reset objPtr back to the original value */
+	Tcl_SetByteArrayLength(objPtr, length);
+
+	/*
+	 * Now do the append secure that buffer growth cannot cause
+	 * any trouble.
+	 */
+	TclAppendBytesToByteArray(objPtr,
+		Tcl_GetByteArrayFromObj(appendObjPtr, NULL), lengthSrc);
 	return;
     }