| Commit message (Collapse) | Author | Age | Files | Lines |
|---|
| |
|
|
| |
U+FFFF.
|
| |
|
|
| |
for lead bytes F0-F5, Tcl_UtfCharComplete() should guarantee that those 4 bytes are available, not 3.
|
| |\ |
|
| | |\
| | |
| | |
| | | |
TCL_UTF_MAX value, since we didn't figure out yet how it should behave for TCL_UTF_MAX>3.
|
| | | |
| | |
| | |
| | | |
correct, now they are. Make next/prev behavior the same for all TCL_UTF_MAX values, since the exact behavior for TCL_UTF_MAX>3 should be worked out further for Tcl 8.7 first, then everything agreed upon can be backported.
|
| | |\ \
| | |/
| | |
| | | |
the right answer. Add testcase 4.14 with similar corner-case, this one is OK.
|
| | |/
|/|
| |
| |
| | |
[6596c4af31e29b5d]. Just look at the Tcl_UtfAtIndex() implementation for TCL_UTF_MAX=4: It's not the same.
There are no test-cases for Tcl_UniCharAtIndex(), see [f45d0dc1a7], not really worth to write one, since the implementation of this function didn't change in 20 years.
|
| | |
| |
| |
| | |
testcases in 4 groups (utf-7.10, utf-7.15, utf-7.40 and utf-7.48) , where Tcl_UtfPrev() didn't jump to the beginning of the UTF-8 character, even though there was no limitation which prevented that. So, this is actually a bug-fix for the TIP #389 implementation.
|
| | | |
|
| | |
| |
| | |
For start bytes F0-F4, case TCL_UTF_MAX=4, Tcl_UtfToUniChar() reads 3 bytes but only advances 1 byte. So Tcl_UtfCharComplete() must make sure 3 bytes are available, not 1. Adapt Tcl_UtfCharComplete() accordingly. No change for TCL_UTF_MAX=[3|6]
|
| |\ \ |
|
| | |\ \
| | |/ |
|
| | | |\
| |_|/
|/| | |
|
| | |/
| |
| | |
recent reforms. Older efforts aborted.
|
| | |\
| | |
| | |
| | | |
in "utf16" build any more.
|
| | | |
| | |
| | | |
Make sure that Tcl_UtfPrev() never reads more than 3 trail bytes (or 4 when TCL_UTF_MAX > 4). Those are the same limits as for Tcl_UtfNext() and Tcl_UtfToUniChar()
|
| | |\ \ |
|
| | | | | |
|
| | |\ \ \ |
|
| | |\ \ \ \ |
|
| | |\ \ \ \ \ |
|
| | | | | | | | |
|
| | | | | | | | |
|
| |\ \ \ \ \ \ \ |
|
| | | | | | | | | |
|
| |\ \ \ \ \ \ \ \
| |/ / / / / / / |
|
| | | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | |
| | | | | | | | |
character, we cannot conclude that the next byte also will be, or can by
taken as a single byte. At least we cannot when TCL_UTF_MAX > 3 so that we
have room for valid two-byte sequences after incomplete sequence detection.
No need for conditional code, just use an algorithm that always works.
|
| | |_|_|_|_|_|/
|/| | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
more than 3 bytes. This is more consistant with what Tcl 8.7 does too.
For TCL_UTF_MAX==6: Make sure that Tcl_UtfNext()/Tcl_UtfPrev() never move more than 4 bytes.
For TCL_UTF_MAX==3: No change.
Introduce ucs2_utf16 test constraint, since many test results now become the same for ucs2 and utf16.
|
| |\ \ \ \ \ \ \
| |/ / / / / / |
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Tcl_NumUtfChars(). Don't use "-1" in the Tcl_NumUtfChars() calculation, since that raises more questions than it solves, but that's easy to be remedied as well: Juse use >= in stead of > in the comparation. Great idea, Don!
Backport more code formatting from Tcl 8.6 (e.g. use of CONST, which makes no sense any more in c-files)
|
| |\ \ \ \ \ \ \
| |/ / / / / / |
|
| | | | | | | | |
|
| |\ \ \ \ \ \ \
| |/ / / / / / |
|
| | | | | | | | |
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
Update the comments to describe what it does now, and cautions that callers
take into account.
|
| |\ \ \ \ \ \ \
| |/ / / / / / |
|
| | | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | |
| | | | | | | |
first in Tcl_UtfNext(), so if src[1] is invalid src[2] doesn't need to be checked any more.
Note: This order change, calling Invalid() first was wrong, and is corrected in later commits. Thanks, Don, for noticing this!
|
| | |_|_|_|_|/
|/| | | | |
| | | | | |
| | | | | | |
in TCL_UTF_MAX>3 builds.
|
| |\ \ \ \ \ \
| |/ / / / /
| | | | | /
| |_|_|_|/
|/| | | | |
|
| | | | | |
| | | | |
| | | | |
| | | | |
| | | | | |
"knownBug" testcase utf-6.93.1.
Rename tip389 selector to utf16, since that's what it actually is, in contrast to ucs2 and ucs4.
|
| | |_|_|/
|/| | |
| | | |
| | | | |
fixes all "knownBug" testcases related to tip389.
|
| | |_|/
|/| |
| | |
| | |
| | | |
TCL_UTF_MAX=4. (even though TCL_UTF_MAX=4 is unsupported, it would be nice to make it work)
Marked various test-cases as "knownBug", those work correctly in core-8-branch (8.7). The fix there could be backported. Low prio.
|
| | | |
| | |
| | |
| | |
| | | |
commit, it was not correct).
Perfectionalize TclUtfToUCS4()/TclUCS4Complete() and new (internal) function TclUCS4ToUtf(). They can help preventing bugs regarding splitting/joining surrogates. Used them in a few more places.
|
| | |/
|/|
| |
| |
| |
| | |
always for whatever testConstraints.
Fix one invalid use of TclUCS4Complete(), and let TclUtfToUCS4() handle (invalid) 4-byte sequences.
Test-case cleanup (removal of unnecessary quoting)
|
| |\ \
| | |
| | |
| | |
| | |
| | | |
bytes.
Tcl_UtfToUniChar() now never reads more than TCL_UTF_MAX bytes any more. The UtfToUtf encoder/decoder is adapted to do attitional checks (more tricky than in Tcl 8.7, since we want compatibility with earlier 8.6 releases).
Other callers of Tcl_UtfToUniChar() needs to be revised for the same problem. Most callers will need to change Tcl_UtfToUniChar() -> TclUtfToUCS4() and Tcl_UtfCharComplete() -> TclUCS4Complete(), but that's not done yet.
|
| | | |
| | |
| | |
| | | |
test-cases fail when we no longer check the validity of the 3th trail byte.
|
| | | | |
|
| | |\ \ |
|
| | |\ \ \
| | | | |
| | | | | |
Quick exit from Tcl_UtfToChar16()/Tcl_UtfToUniChar() when lead-byte is 0xF5 - 0xF7.
|
| | |\ \ \ \ |
|
