summaryrefslogtreecommitdiffstats
path: root/generic
Commit message (Collapse)AuthorAgeFilesLines
...
| * | | | | | | | Merge 8.6jan.nijtmans2020-05-035-41/+77
| |\ \ \ \ \ \ \ \
| * | | | | | | | | Seems almost correct. Still problem with "string index" for TCL_UTF_MAX>3jan.nijtmans2020-05-021-15/+10
| | | | | | | | | |
| * | | | | | | | | More fixes for [ed29806baf]. Not working yet. WIPjan.nijtmans2020-05-023-19/+48
| | | | | | | | | |
* | | | | | | | | | Merge 8.5jan.nijtmans2020-05-081-4/+4
|\ \ \ \ \ \ \ \ \ \ | |_|_|_|_|_|_|/ / / |/| | | | | | | / / | | |_|_|_|_|_|/ / | |/| | | | | | |
| * | | | | | | | More test-case cleanup. Enhance "testutfnext", so it can detect whether ↵jan.nijtmans2020-05-081-4/+4
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | bytes after the string-end are read. The command will return -1 in that case. No need for additional arguments any more.
* | | | | | | | | merge 8.5dgp2020-05-071-5/+6
|\ \ \ \ \ \ \ \ \ | |/ / / / / / / /
| * | | | | | | | Same trouble with Tcl_UtfToUniCharDstring. Test and fix.dgp2020-05-071-4/+5
| | | | | | | | |
* | | | | | | | | merge 8.5dgp2020-05-071-14/+15
|\ \ \ \ \ \ \ \ \ | |/ / / / / / / /
| * | | | | | | | Fix. Note that just because we get one positive detection of an incompletedgp2020-05-071-9/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | character, we cannot conclude that the next byte also will be, or can by taken as a single byte. At least we cannot when TCL_UTF_MAX > 3 so that we have room for valid two-byte sequences after incomplete sequence detection. No need for conditional code, just use an algorithm that always works.
* | | | | | | | | For TCL_UTF_MAX==4: Make sure that Tcl_UtfNext()/Tcl_UtfPrev() never move ↵jan.nijtmans2020-05-071-6/+6
| |_|_|_|_|_|/ / |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | more than 3 bytes. This is more consistant with what Tcl 8.7 does too. For TCL_UTF_MAX==6: Make sure that Tcl_UtfNext()/Tcl_UtfPrev() never move more than 4 bytes. For TCL_UTF_MAX==3: No change. Introduce ucs2_utf16 test constraint, since many test results now become the same for ucs2 and utf16.
* | | | | | | | Optimize Tcl_UtfToUniCharDString()jan.nijtmans2020-05-071-23/+26
|\ \ \ \ \ \ \ \ | |/ / / / / / /
| * | | | | | | Tighten optimization in Tcl_UtfToUniCharDString(), just as in ↵jan.nijtmans2020-05-071-81/+79
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Tcl_NumUtfChars(). Don't use "-1" in the Tcl_NumUtfChars() calculation, since that raises more questions than it solves, but that's easy to be remedied as well: Juse use >= in stead of > in the comparation. Great idea, Don! Backport more code formatting from Tcl 8.6 (e.g. use of CONST, which makes no sense any more in c-files)
* | | | | | | | merge 8.5dgp2020-05-061-14/+24
|\ \ \ \ \ \ \ \ | |/ / / / / / /
| * | | | | | | Tighten optimization in Tcl_NumUtfChars. Explain in comments.dgp2020-05-061-14/+25
| | | | | | | |
* | | | | | | | merge 8.5dgp2020-05-061-11/+26
|\ \ \ \ \ \ \ \ | |/ / / / / / /
| * | | | | | | Restore safe calls of Invalid().dgp2020-05-061-3/+10
| | | | | | | |
| * | | | | | | close forkdgp2020-05-061-23/+24
| |\ \ \ \ \ \ \
| * | | | | | | | The routine Invalid() has been revised to do something different.dgp2020-05-061-9/+17
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | Update the comments to describe what it does now, and cautions that callers take into account.
* | | | | | | | | Merge 8.5jan.nijtmans2020-05-061-20/+21
|\ \ \ \ \ \ \ \ \ | | |/ / / / / / / | |/| | | | | | |
| * | | | | | | | Backport testutfprev/testutfnex testcase improvements from 8.7. This makes ↵jan.nijtmans2020-05-061-23/+24
| |/ / / / / / / | | | | | | | | | | | | | | | | | | | | | | | | testdescriptions/testresults more equal among branches, so the real differences are more visible.
* | | | | | | | Merge 8.5. More usage of UCHAR() macro.jan.nijtmans2020-05-066-16/+18
|\ \ \ \ \ \ \ \ | |/ / / / / / /
| * | | | | | | Change Invalid() parameter type to "const char *". Also call Invalid() ↵jan.nijtmans2020-05-061-11/+13
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | first in Tcl_UtfNext(), so if src[1] is invalid src[2] doesn't need to be checked any more. Note: This order change, calling Invalid() first was wrong, and is corrected in later commits. Thanks, Don, for noticing this!
* | | | | | | | More usage of TclUtfToUCS4(), so we can use the whole Unicode range better ↵jan.nijtmans2020-05-054-36/+36
| |_|_|_|_|/ / |/| | | | | | | | | | | | | | | | | | | | in TCL_UTF_MAX>3 builds.
* | | | | | | Merge 8.5jan.nijtmans2020-05-051-2/+2
|\ \ \ \ \ \ \ | |/ / / / / / | | | | | / / | |_|_|_|/ / |/| | | | |
| * | | | | Properly protect "Invalid" function against lead bytes 0x80-0xBF. This fixes ↵jan.nijtmans2020-05-051-2/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | "knownBug" testcase utf-6.93.1. Rename tip389 selector to utf16, since that's what it actually is, in contrast to ucs2 and ucs4.
* | | | | | New internal function TclGetUCS4() only available when TCL_UTF_MAX=4. This ↵jan.nijtmans2020-05-045-14/+91
| |_|_|/ / |/| | | | | | | | | | | | | | fixes all "knownBug" testcases related to tip389.
* | | | | (partial) fix for [9d0cb35bb2]: Various issues with core-8-6-branch, ↵jan.nijtmans2020-05-041-16/+18
| |_|/ / |/| | | | | | | | | | | | | | | TCL_UTF_MAX=4. (even though TCL_UTF_MAX=4 is unsupported, it would be nice to make it work) Marked various test-cases as "knownBug", those work correctly in core-8-branch (8.7). The fix there could be backported. Low prio.
* | | | Re-join utf-6.93.0 and utf-6.93.1 (please disregard comment in previous ↵jan.nijtmans2020-05-035-41/+82
| | | | | | | | | | | | | | | | | | | | commit, it was not correct). Perfectionalize TclUtfToUCS4()/TclUCS4Complete() and new (internal) function TclUCS4ToUtf(). They can help preventing bugs regarding splitting/joining surrogates. Used them in a few more places.
* | | | Join test-cases utf-6.93.0 and utf-6.93.1, which MUST give the same answer ↵jan.nijtmans2020-05-022-2/+2
| |/ / |/| | | | | | | | | | | | | | always for whatever testConstraints. Fix one invalid use of TclUCS4Complete(), and let TclUtfToUCS4() handle (invalid) 4-byte sequences. Test-case cleanup (removal of unnecessary quoting)
* | | Fix first part of [ed29806baf]: Tcl_UtfToUniChar reads more than TCL_UTF_MAX ↵jan.nijtmans2020-05-013-26/+42
|\ \ \ | | | | | | | | | | | | | | | | | | | | bytes. Tcl_UtfToUniChar() now never reads more than TCL_UTF_MAX bytes any more. The UtfToUtf encoder/decoder is adapted to do attitional checks (more tricky than in Tcl 8.7, since we want compatibility with earlier 8.6 releases). Other callers of Tcl_UtfToUniChar() needs to be revised for the same problem. Most callers will need to change Tcl_UtfToUniChar() -> TclUtfToUCS4() and Tcl_UtfCharComplete() -> TclUCS4Complete(), but that's not done yet.
| * | | Fix [ed29806baf] by introducing TclUCS4Complete(). All other calls of ↵jan.nijtmans2020-05-013-9/+12
| | | | | | | | | | | | | | | | | | | | Tcl_UtfToUniChar() are suspicious, because those cannot handle 4-byte UTF-8 sequences reliable. So, there's more work to do, but this part can already be backported to Tcl 8.6 and see where we get.
| * | | First, prove that bug [ed29806baf] is present in 8.7 too. Let's see what ↵jan.nijtmans2020-04-301-3/+3
| | | | | | | | | | | | | | | | test-cases fail when we no longer check the validity of the 3th trail byte.
| * | | Let's not get out the src[3] check yet.jan.nijtmans2020-04-301-1/+1
| | | |
| * | | Merge 8.6jan.nijtmans2020-04-301-9/+9
| |\ \ \
| * \ \ \ Merge-mark 8.6 (Use of UNICODE_OUT_OF_RANGE() macro already was in 8.7).jan.nijtmans2020-04-291-11/+6
| |\ \ \ \ | | | | | | | | | | | | Quick exit from Tcl_UtfToChar16()/Tcl_UtfToUniChar() when lead-byte is 0xF5 - 0xF7.
| * \ \ \ \ Merge 8.6jan.nijtmans2020-04-281-2/+2
| |\ \ \ \ \
| * | | | | | Extend "testutfnext" test-command such that it can detect when it reads more ↵jan.nijtmans2020-04-281-3/+12
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | bytes than the end of the string. If you want to null-terminate string explitely, use \x00
* | | | | | | merge 8.5dgp2020-04-301-0/+4
|\ \ \ \ \ \ \ | |_|_|_|/ / / |/| | | | / / | | |_|_|/ / | |/| | | |
| * | | | | Add comments so I'll know again later why this is here.dgp2020-04-301-0/+4
| | | | | |
| * | | | | Backport many UNICODE_OUT_OF_RANGE() calls. This should fix ↵jan.nijtmans2020-04-291-103/+166
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | [69634d51fb74551b] for Tcl 8.5 (with TCL_UTF_MAX=4) too. Also fix some comments which were not up to date. No change at all in behavior for TCL_UTF_MAX=3.
* | | | | | Partial fix for [ed29806ba]: Tcl_UtfToUniChar reads more than TCL_UTF_MAX bytes.jan.nijtmans2020-04-301-9/+11
| |_|_|/ / |/| | | |
* | | | | Add UNICODE_OUT_OF_RANGE() calls to UCS4ToTitle() and friends. Backported ↵jan.nijtmans2020-04-291-21/+27
| |_|/ / |/| | | | | | | | | | | from 8.7. This fixes [69634d51fb]: handling out of range UCS4 values (at least, it's fixed in 8.6 now the same way as it's fixed in 8.7).
* | | | Merge 8.5jan.nijtmans2020-04-281-2/+2
|\ \ \ \ | |/ / /
| * | | Fix [b6038e5bc8]: mis-parse of surrogate pair escapes. Thanks Don!jan.nijtmans2020-04-281-2/+2
| |\ \ \
| | * | | Proposed fix.dgp2020-04-281-2/+2
| | |/ /
| | * | merge 8.6dgp2020-04-271-1/+1
| | |\ \
| | * \ \ merge 8.6dgp2020-04-271-45/+16
| | |\ \ \
| | * | | | Cherry-pick Tcl_UniCharAtIndex() implementation from [6596c4af31], but ↵jan.nijtmans2020-04-261-22/+4
| | | | | | | | | | | | | | | | | | | | | | | | adapted to the needs of TIPs 389/542.
| | * | | | Make reg-13.33/reg-13.34 work again. Still have to investigate what's the ↵jan.nijtmans2020-04-251-8/+2
| | | | | | | | | | | | | | | | | | | | | | | | problem.
| | * | | | Merge 8.6jan.nijtmans2020-04-251-39/+11
| | |\ \ \ \
generated by cgit v0.12 at 2025-08-31 02:44:26 (GMT)