Harfbuzz-thai: Hide ZWJ and ZWNJ characters and show Inherited characters

Thai is not supposed to have ZWJ and ZWNJ characters or any other of the
Inherited Unicode Scripts
(http://www.verisigninc.com/assets/idn-inherited-unicode-script.pdf)
- they don't have a mapping to the thai encoding tis620 which libthai
requires.  However it is an unfortunate fact that there are many websites
etc that liberally place these ZWJ and ZWNJ characters throughout thai text
to force word boundaries, so we must also deal with them.

We deal with all Inherited characters by mapping them to the invalid code ~0
in tis620 encoding, following what libthai does internally in its own tis620
encoding functions, and then replacing this character with the original
unicode and setting dontPrint to true to hide the ZWJ and ZWNJ
characters.

Includes a unit test to check the behaviour.

Change-Id: I49e44da6e39443daa832e7f50c94804e13deabb4
Reviewed-by: Adrian Yanes <ext-adrian.yanes@nokia.com>
Reviewed-by: Lars Knoll <lars.knoll@nokia.com>

1 files changed, 8 insertions, 3 deletions

diff --git a/src/3rdparty/harfbuzz/src/harfbuzz-thai.c b/src/3rdparty/harfbuzz/src/harfbuzz-thai.c
index 386fd5c..bf6c35b 100644
--- a/src/3rdparty/harfbuzz/src/harfbuzz-thai.c
+++ b/src/3rdparty/harfbuzz/src/harfbuzz-thai.c
@@ -77,7 +77,7 @@ static void to_tis620(const HB_UChar16 *string, hb_uint32 len, const char *cstr)
         else if (string[i] >= 0xe01 && string[i] <= 0xe5b)
             result[i] = (unsigned char)(string[i] - 0xe00 + 0xa0);
         else
-            result[i] = '?';
+            result[i] = (unsigned char)~0; // Same encoding as libthai uses for invalid chars
     }
 
     result[len] = 0;
@@ -259,8 +259,13 @@ static HB_Bool HB_ThaiConvertStringToGlyphIndices (HB_ShaperItem *item)
         for (int lgi = 0; lgi < lgn; lgi++) {
             if ( rglyphs[lgi] == 0xdd/*TH_BLANK_BASE_GLYPH*/ ) {
                 glyphString[slen++] = C_DOTTED_CIRCLE;
-            }
-            else {
+            } else if (cstr[i] == (signed char)~0) {
+                // The only glyphs that should be passed to this function that cannot be mapped to
+                // tis620 are the ones of type Inherited class.  Pass these glyphs untouched.
+                glyphString[slen++] = string[i];
+                if (string[i] == 0x200D || string[i] == 0x200C)
+                    item->attributes[slen-1].dontPrint = true; // Hide ZWJ and ZWNJ characters
+            } else {
                 glyphString[slen++] = (HB_UChar16) thai_get_glyph_index (font_type, rglyphs[lgi]);
             }
         }