summaryrefslogtreecommitdiffstats
path: root/generic/tclUtf.c
Commit message (Collapse)AuthorAgeFilesLines
* Merge 8.7jan.nijtmans2020-12-081-1/+1
|\
| * Add -finput-charset=UTF-8 and -fextended-identifiers to gcc (and clang). All ↵jan.nijtmans2020-12-081-1/+1
| | | | | | | | C sources can now use UTF-8, as far as gcc/clang/msvc support it. Not used yet
* | Merge 8.7jan.nijtmans2020-11-181-1/+14
|\ \ | |/
| * More usage of TclUtfToUCS4/TclUniCharToUCS4 in stead of it's UniChar ↵jan.nijtmans2020-11-181-1/+14
| | | | | | | | variants: This handles surrogate pairs better.
* | Merge 8.7. Use more TCL_INDEX_NONE in documentation/headers/code.jan.nijtmans2020-06-041-6/+6
|\ \ | |/
* | Merge 8.7jan.nijtmans2020-05-251-0/+14
|\ \ | |/
| * Fix compiled "string is <class>" for characters > U+FFFF. Add testcase ↵jan.nijtmans2020-05-251-0/+14
| |\ | | | | | | | | | exposing this bug.
| | * Fix compiled "string is <class>" for TCL_UTF_MAX=4 build, for characters > ↵jan.nijtmans2020-05-251-0/+16
| | | | | | | | | | | | U+FFFF.
| | * Tiny fix for TCL_UTF_MAX=4 build only: Since Tcl_UtfNext() verifies 4 bytes ↵jan.nijtmans2020-05-181-1/+1
| | | | | | | | | | | | for lead bytes F0-F5, Tcl_UtfCharComplete() should guarantee that those 4 bytes are available, not 3.
| | * Fix [ed29806baf]: Tcl_UtfToUniChar reads more than TCL_UTF_MAX bytesjan.nijtmans2020-05-131-25/+11
| | |\
| | | * Merge testcase cleanup. Make Tcl_UtfPrev() behave the same for any ↵jan.nijtmans2020-05-121-4/+4
| | | |\ | | | | | | | | | | | | | | | TCL_UTF_MAX value, since we didn't figure out yet how it should behave for TCL_UTF_MAX>3.
| | | * | Fix "knownBug" utf-4.11. Turns out a few other testcases where still not ↵jan.nijtmans2020-05-121-23/+5
| | | | | | | | | | | | | | | | | | | | correct, now they are. Make next/prev behavior the same for all TCL_UTF_MAX values, since the exact behavior for TCL_UTF_MAX>3 should be worked out further for Tcl 8.7 first, then everything agreed upon can be backported.
| | | * | Merge 8.6. Mark testcase utf-4.11 as "knownBug": this one still doesn't give ↵jan.nijtmans2020-05-111-36/+48
| | | |\ \ | | | | |/ | | | | | | | | | | the right answer. Add testcase 4.14 with similar corner-case, this one is OK.
* | | | | Merge 8.7jan.nijtmans2020-05-201-1/+1
|\ \ \ \ \ | |/ / / /
| * | | | Adapt some comments, which are not correct for Tcl 8.7 any morejan.nijtmans2020-05-201-1/+1
| | | | |
* | | | | Merge 8.7jan.nijtmans2020-05-121-10/+11
|\ \ \ \ \ | |/ / / /
| * | | | Little tweak to Tcl_UniCharAtIndex(): Protect against negative index, return ↵jan.nijtmans2020-05-121-0/+3
| | | | | | | | | | | | | | | | | | | | -1 in that case.
* | | | | Merge 8.7jan.nijtmans2020-05-121-3/+16
|\ \ \ \ \ | |/ / / /
| * | | | Merge 8.6jan.nijtmans2020-05-121-3/+14
| |\ \ \ \ | | |/ / /
| | * | | Revert implementation of Tcl_UniCharAtIndex() change done in this commit: ↵jan.nijtmans2020-05-121-1/+3
| | | |/ | | |/| | | | | | | | | | | | | [6596c4af31e29b5d]. Just look at the Tcl_UtfAtIndex() implementation for TCL_UTF_MAX=4: It's not the same. There are no test-cases for Tcl_UniCharAtIndex(), see [f45d0dc1a7], not really worth to write one, since the implementation of this function didn't change in 20 years.
* | | | Merge 8.7jan.nijtmans2020-05-111-2/+8
|\ \ \ \ | |/ / /
| * | | Merge 8.6jan.nijtmans2020-05-111-2/+8
| |\ \ \ | | |/ /
| | * | Tweak the Tcl_UtfPrev() implementation for TCL_UTF_MAX=4. This fixes 10 ↵jan.nijtmans2020-05-111-1/+1
| | | | | | | | | | | | | | | | testcases in 4 groups (utf-7.10, utf-7.15, utf-7.40 and utf-7.48) , where Tcl_UtfPrev() didn't jump to the beginning of the UTF-8 character, even though there was no limitation which prevented that. So, this is actually a bug-fix for the TIP #389 implementation.
* | | | Merge 8.7jan.nijtmans2020-05-111-3/+3
|\ \ \ \ | |/ / /
| * | | Merge 8.6jan.nijtmans2020-05-111-3/+3
| |\ \ \ | | |/ /
| | * | occurance -> occurrence.jan.nijtmans2020-05-111-3/+3
| | | |
* | | | Merge 8.7jan.nijtmans2020-05-101-8/+7
|\ \ \ \ | |/ / /
| * | | Merge 8.6jan.nijtmans2020-05-101-8/+7
| |\ \ \ | | |/ /
| | * | Tweak Invalid() function: No need for "return 0" twice in the function. jan.nijtmans2020-05-101-8/+31
| | | | | | | | | | | | For start bytes F0-F4, case TCL_UTF_MAX=4, Tcl_UtfToUniChar() reads 3 bytes but only advances 1 byte. So Tcl_UtfCharComplete() must make sure 3 bytes are available, not 1. Adapt Tcl_UtfCharComplete() accordingly. No change for TCL_UTF_MAX=[3|6]
| | * | Rebase to latest core-8-6-branch.jan.nijtmans2020-05-081-26/+12
| | |\ \
| | | * \ Merge changes from parent branchdgp2020-05-071-14/+16
| | | |\ \ | | | | |/
| | | | * Merge 8.6jan.nijtmans2020-05-071-19/+21
| | | | |\ | | | |_|/ | | |/| |
| | | * | New approach to fixing the regression reported in [31aa44375d] builds ondgp2020-05-071-17/+3
| | | |/ | | | | | | | | recent reforms. Older efforts aborted.
| | | * Merge 8.6. Some more tweaks to Tcl_UtfPrev(), so it cannot jump back 4 bytes ↵jan.nijtmans2020-05-071-63/+92
| | | |\ | | | | | | | | | | | | | | | in "utf16" build any more.
| | | * | Add 4 test-cases that could fool Tcl_UtfPrev (but ... actually they don't).jan.nijtmans2020-05-051-1/+1
| | | | | | | | | | | | | | | Make sure that Tcl_UtfPrev() never reads more than 3 trail bytes (or 4 when TCL_UTF_MAX > 4). Those are the same limits as for Tcl_UtfNext() and Tcl_UtfToUniChar()
| | | * | Merge 8.6jan.nijtmans2020-05-051-1/+1
| | | |\ \
| | | * | | More progress/simplificationjan.nijtmans2020-05-041-21/+2
| | | | | |
| | | * | | Merge 8.6jan.nijtmans2020-05-041-6/+7
| | | |\ \ \
| | | * \ \ \ Merge 8.6jan.nijtmans2020-05-041-6/+8
| | | |\ \ \ \
| | | * \ \ \ \ Merge 8.6jan.nijtmans2020-05-031-13/+55
| | | |\ \ \ \ \
| | | * | | | | | Seems almost correct. Still problem with "string index" for TCL_UTF_MAX>3jan.nijtmans2020-05-021-15/+10
| | | | | | | | |
| | | * | | | | | More fixes for [ed29806baf]. Not working yet. WIPjan.nijtmans2020-05-021-17/+41
| | | | | | | | |
* | | | | | | | | merge 8.7dgp2020-05-071-14/+16
|\ \ \ \ \ \ \ \ \ | |/ / / / / / / /
| * | | | | | | | Merge 8.6dgp2020-05-071-14/+16
| |\ \ \ \ \ \ \ \ | | |/ / / / / / /
| | * | | | | | | merge 8.5dgp2020-05-071-5/+6
| | |\ \ \ \ \ \ \
| | | * | | | | | | Same trouble with Tcl_UtfToUniCharDstring. Test and fix.dgp2020-05-071-4/+5
| | | | | | | | | |
| | * | | | | | | | merge 8.5dgp2020-05-071-14/+15
| | |\ \ \ \ \ \ \ \ | | | |/ / / / / / /
| | | * | | | | | | Fix. Note that just because we get one positive detection of an incompletedgp2020-05-071-9/+10
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | character, we cannot conclude that the next byte also will be, or can by taken as a single byte. At least we cannot when TCL_UTF_MAX > 3 so that we have room for valid two-byte sequences after incomplete sequence detection. No need for conditional code, just use an algorithm that always works.
| | * | | | | | | | For TCL_UTF_MAX==4: Make sure that Tcl_UtfNext()/Tcl_UtfPrev() never move ↵jan.nijtmans2020-05-071-6/+6
| | | |_|_|_|_|_|/ | | |/| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | more than 3 bytes. This is more consistant with what Tcl 8.7 does too. For TCL_UTF_MAX==6: Make sure that Tcl_UtfNext()/Tcl_UtfPrev() never move more than 4 bytes. For TCL_UTF_MAX==3: No change. Introduce ucs2_utf16 test constraint, since many test results now become the same for ucs2 and utf16.
* | | | | | | | | Merge 8.7jan.nijtmans2020-05-071-34/+40
|\ \ \ \ \ \ \ \ \ | |/ / / / / / / /
generated by cgit v0.12 at 2024-11-16 10:02:20 (GMT)