diff options
| author | stanton <stanton> | 1999-04-16 00:46:29 (GMT) |
|---|---|---|
| committer | stanton <stanton> | 1999-04-16 00:46:29 (GMT) |
| commit | 97464e6cba8eb0008cf2727c15718671992b913f (patch) | |
| tree | ce9959f2747257d98d52ec8d18bf3b0de99b9535 /tools/encoding/cjk.inf | |
| parent | a8c96ddb94d1483a9de5e340b740cb74ef6cafa7 (diff) | |
| download | tcl-97464e6cba8eb0008cf2727c15718671992b913f.zip tcl-97464e6cba8eb0008cf2727c15718671992b913f.tar.gz tcl-97464e6cba8eb0008cf2727c15718671992b913f.tar.bz2 | |
merged tcl 8.1 branch back into the main trunk
Diffstat (limited to 'tools/encoding/cjk.inf')
| -rw-r--r-- | tools/encoding/cjk.inf | 4467 |
1 files changed, 4467 insertions, 0 deletions
diff --git a/tools/encoding/cjk.inf b/tools/encoding/cjk.inf new file mode 100644 index 0000000..9fbe527 --- /dev/null +++ b/tools/encoding/cjk.inf @@ -0,0 +1,4467 @@ +--- BEGIN (CJK.INF VERSION 2.1 07/12/96) 185553 BYTES --- +CJK.INF Version 2.1 (July 12, 1996) + +Copyright (C) 1995-1996 Ken Lunde. All Rights Reserved. + +CJK is a registered trademark and service mark of The Research + Libraries Group, Inc. + +Online Companion to "Understanding Japanese Information Processing" +- ENGLISH: 1993, O'Reilly & Associates, Inc., ISBN 1-56592-043-0 +- JAPANESE: 1995, SOFTBANK Corporation, ISBN 4-89052-708-7 + + + This online document provides information on CJK (that is, +Chinese, Japanese, and Korean) character set standards and encoding +systems. In short, it provides detailed information on how CJK text is +handled electronically. I am happy to share this information with +others, and I would appreciate any comments/feedback on its content. +The current version (master copy) of this document is maintained at: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/doc/cjk.inf + +This file may also be obtained by contacting me directly using one of +the e-mail addresses listed in the CONTACT INFORMATION section. + + +TABLE OF CONTENTS + + VERSION HISTORY + RESTRICTIONS + CONTACT INFORMATION + WHAT HAPPENED TO JAPAN.INF? + DISCLAIMER + CONVENTIONS + INTRODUCTION + PART 1: WHAT'S UP WITH UJIP? + PART 2: CJK CHARACTER SET STANDARDS + 2.1: JAPANESE + 2.1.1: JIS X 0201-1976 + 2.1.2: JIS X 0208-1990 + 2.1.3: JIS X 0212-1990 + 2.1.4: JIS X 0221-1995 + 2.1.5: JIS X 0213-199X + 2.1.6: OBSOLETE STANDARDS + 2.2: CHINESE (PRC) + 2.2.1: GB 1988-89 + 2.2.2: GB 2312-80 + 2.2.3: GB 6345.1-86 + 2.2.4: GB 7589-87 + 2.2.5: GB 7590-87 + 2.2.6: GB 8565.2-88 + 2.2.7: GB/T 12345-90 + 2.2.8: GB/T 13131-9X + 2.2.9: GB/T 13132-9X + 2.2.10: GB 13000.1-93 + 2.2.11: ISO-IR-165:1992 + 2.2.12: OBSOLETE STANDARDS + 2.3: CHINESE (TAIWAN) + 2.3.1: BIG FIVE + 2.3.2: CNS 11643-1992 + 2.3.3: CNS 5205 + 2.3.4: OBSOLETE STANDARDS + 2.4: KOREAN + 2.4.1: KS C 5636-1993 + 2.4.2: KS C 5601-1992 + 2.4.3: KS C 5657-1991 + 2.4.4: GB 12052-89 + 2.4.5: KS C 5700-1995 + 2.4.6: OBSOLETE STANDARDS + 2.5: CJK + 2.5.1: ISO 10646-1:1993 + 2.5.2: CCCII + 2.5.3: ANSI Z39.64-1989 + 2.6: OTHER + 2.6.1: GB 8045-87 + 2.6.2: TCVN-5773:1993 + PART 3: CJK ENCODING SYSTEMS + 3.1: 7-BIT ISO 2022 ENCODING + 3.1.1: CODE SPACE + 3.1.2: ISO-REGISTERED ESCAPE SEQUENCES + 3.1.3: ISO-2022-JP AND ISO-2022-JP-2 + 3.1.4: ISO-2022-KR + 3.1.5: ISO-2022-CN AND ISO-2022-CN-EXT + 3.2: EUC ENCODING + 3.2.1: JAPANESE REPRESENTATION + 3.2.2: CHINESE (PRC) REPRESENTATION + 3.2.3: CHINESE (TAIWAN) REPRESENTATION + 3.2.4: KOREAN REPRESENTATION + 3.3: LOCALE-SPECIFIC ENCODINGS + 3.3.1: SHIFT-JIS + 3.3.2: HZ (HZ-GB-2312) + 3.3.3: zW + 3.3.4: BIG FIVE + 3.3.5: JOHAB + 3.3.6: N-BYTE HANGUL + 3.3.7: UCS-2 + 3.3.8: UCS-4 + 3.3.9: UTF-7 + 3.3.10: UTF-8 + 3.3.11: UTF-16 + 3.3.12: ANSI Z39.64-1989 + 3.3.13: BASE64 + 3.3.14: IBM DBCS-HOST + 3.3.15: IBM DBCS-PC + 3.3.16: IBM DBCS-/TBCS-EUC + 3.3.17: UNIFIED HANGUL CODE + 3.3.18: TRON CODE + 3.3.19: GBK + 3.4: CJK CODE PAGES + PART 4: CJK CHARACTER SET COMPATIBILITY ISSUES + 4.1: JAPANESE + 4.2: CHINESE (PRC) + 4.3: CHINESE (TAIWAN) + 4.4: KOREAN + 4.5: ISO 10646-1:1993 + 4.6: UNICODE + 4.7: CODE CONVERSION TIPS + PART 5: CJK-CAPABLE OPERATING SYSTEMS + 5.1: MS-DOS + 5.2: WINDOWS + 5.3: MACINTOSH + 5.4: UNIX AND X WINDOWS + 5.5: OTHERS + PART 6: CJK TEXT AND INTERNET SERVICES + 6.1: ELECTRONIC MAIL + 6.2: USENET NEWS + 6.3: GOPHER + 6.4: WORLD-WIDE WEB + 6.5: FILE TRANSFER TIPS + PART 7: CJK TEXT HANDLING SOFTWARE + 7.1: MULE + 7.2: CNPRINT + 7.3: MASS + 7.4: ADOBE TYPE MANAGER (ATM) + 7.5: MACINTOSH SOFTWARE + 7.6: MACBLUE TELNET + 7.7: CXTERM + 7.8: UW-DBM + 7.9: POSTSCRIPT + 7.10: NJWIN + PART 8: CJK PROGRAMMING ISSUES + 8.1: C AND C++ + 8.2: PERL + 8.3: JAVA + A FINAL NOTE + ACKNOWLEDGMENTS + APPENDIX A: INFORMATION SOURCES + A.1: USENET NEWSGROUPS AND MAILING LISTS + A.1.1: USENET NEWSGROUPS + A.1.2: MAILING LISTS + A.2: INTERNET RESOURCES + A.2.1: USEFUL FTP SITES + A.2.2: USEFUL TELNET SITES + A.2.3: USEFUL GOPHER SITES + A.2.4: USEFUL WWW SITES + A.2.5: USEFUL MAIL SERVERS + A.3: OTHER RESOURCES + A.3.1: BOOKS + A.3.2: MAGAZINES + A.3.3: JOURNALS + A.3.4: RFCs + A.3.5: FAQs + + +VERSION HISTORY + + The following is a complete listing of the earlier versions of +this document along with their release dates and sizes (in bytes): + + Document Version Release Date Size + ^^^^^^^^ ^^^^^^^ ^^^^^^^^^^^^ ^^^^ + JAPAN.INF 1.0 Unknown Unknown + JAPAN.INF 1.1 08/19/91 101,784 + JAPAN.INF 1.2 03/20/92 166,929 (JIS) or 165,639 (Shift-JIS/EUC) + CJK.INF 1.0 06/09/95 103,985 + CJK.INF 1.1 06/12/95 112,771 + CJK.INF 1.2 06/14/95 125,275 + CJK.INF 1.3 06/16/95 130,069 + CJK.INF 1.4 06/19/95 142,543 + CJK.INF 1.5 06/22/95 146,064 + CJK.INF 1.6 06/29/95 150,882 + CJK.INF 1.7 08/15/95 153,772 + CJK.INF 1.8 09/11/95 157,295 + CJK.INF 1.9 12/18/95 170,698 + CJK.INF 2.0 03/12/96 175,973 + +With the release of this version, all of the above are now considered +obsolete. Also, note the three-year gap between the last installment +of JAPAN.INF and the first installment of CJK.INF -- I was writing +UJIP and my PhD dissertation during those three years. Ah, so much for +excuses... + + +RESTRICTIONS + + This document is provided free-of-charge to *anyone*, but no +person or company is permitted to modify, sell, or otherwise +distribute it for profit or other purposes. This document may be +bundled with commercial products only with the prior consent from the +author, and provided that it is not modified in any way whatsoever. +The point here is that I worked long and hard on this document so that +lots of fine folks and companies can benefit from its contents -- not +profit from it. + + +CONTACT INFORMATION + + I would enjoy hearing from readers of this document, even if +it is just to say "hello" or whatever. I can be contacted as follows: + + Ken Lunde + Adobe Systems Incorporated + 1585 Charleston Road + P.O. Box 7900 + Mountain View, CA 94039-7900 USA + 415-962-3866 (office phone) + 415-960-0886 (facsimile) + lunde@adobe.com (preferred) + lunde@ora.com or ujip@ora.com + WWW Home Page: http://jasper.ora.com/lunde/ + +If you wonder what I do for my day job, read on. + I have been working for Adobe Systems for over four years now +(before that I was a graduate student at UW-Madison), and my current +position is Project Manager, CJK Type Development. + + +WHAT HAPPENED TO JAPAN.INF? + + Put bluntly, JAPAN.INF died. It first evolved into my first +book entitled "Understanding Japanese Information Processing" (this +book is now into its second printing, and the Japanese translation was +just published). After my book came out, I did attempt to update +JAPAN.INF, but the effort felt a bit futile. I decided that something +fresh was necessary. + JAPAN.INF also evolved into this document, which breaks the +Japanese barrier by providing similar information on Chinese and +Korean character sets and encodings. It fills the Chinese and Korean +gap, so to speak. My specialty (and hobby, believe it or not) is the +field of CJK character sets and encoding systems, so I felt that +shifting this document more towards those lines was appropriate use of +my (copious) free time (I wish there were more than 24 hours in a +day!). Besides, this document now becomes useful to a much broader +audience. + + +DISCLAIMER + + Ah yes, the ever popular disclaimer! Here's mine. Although I +list my address here at Adobe Systems Incorporated for contact +purposes, Adobe Systems does not endorse this document which I have +created, and have continued (and will continue) to update on a regular +basis (uh, yeah, I promise this time!). This document is a personal +endeavor to inform people of how CJK text can be handled on a variety +of platforms. + + +CONVENTIONS + + The notation that is used for detailing Internet resource +information, such as the Internet protocol type, site name, path, and +file follows the URL (Uniform Resource Locator) notation, namely: + + protocol://site-name/path/file + +An example URL is as follows: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/00README + +The protocol is FTP, the site-name is ftp.ora.com, the path is pub/ +examples/nutshell/ujip/, and the file is 00README. Also note that this +same notation is used for invoking FTP on WWW (World Wide Web) +browsing software, such as Mosaic, Netscape, or Lynx. + Note that most references to HTTP documents use the four- +letter file extension ".html". However, some HTTP documents are on +file systems that support only three-letter file extensions (can you +say "MS-DOS"?), so you may encounter just ".htm". This is just to let +you know that what you see is not a typo. + References to my book "Understanding Japanese Information +Processing" are (affectionately) abbreviated as UJIP. These references +also apply to the Japanese translation (UJIP-J). + Hexadecimal values are prefixed with 0x, and every two +hexadecimal digits represent a one-byte value. Other values can be +assumed to be in decimal notation. + Chinese characters are referred to as kanji (Japanese), hanzi +(Chinese), or hanja (Korean), depending on context. + References to ISO 10646-1:1993 also refer to Unicode +(usually). I have done this so that I do not have to repeat "Unicode" +in the same context as ISO 10646-1:1993. There are times, however, +when I need to distinguish ISO 10646-1:1993 from Unicode. + + +INTRODUCTION + + Electronic mail (e-mail), just one of the many Internet +resources, has become a very efficient means of communicating both +locally and world-wide. While it is very simple to send text which +uses only the 94 printable ASCII characters, character sets that +contain more than these ASCII characters pose special problems. + This document is primarily concerned with CJK character set +and encoding issues. Much of this sort of information is not easily +obtained. This represents one person's attempt at making such +information more widely available. + + +PART 1: WHAT'S UP WITH UJIP? + + UJIP (First Edition) was published in September 1993 by +O'Reilly & Associates, Incorporated. The second printing (*not* the +Second Edition) was subsequently published in March 1994. The page +count for both printings is unchanged at 470. + The following files contain the latest information about +changes (additions and corrections) made to UJIP and UJIP-J for +various printings, both for those that have taken place (such as for +the second printing of the English edition) and for those that are +planned (the first digit is the edition, and the second is the +printing): + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/errata/ujip-errata-1-2.txt + ftp://ftp.ora.com/pub/examples/nutshell/ujip/errata/ujip-errata-1-3.txt + ftp://ftp.ora.com/pub/examples/nutshell/ujip/errata/ujip-j-errata-1-2.txt + +I *highly* recommend that all readers of UJIP obtain these errata +files. Those without FTP access can request copies directly from me. + The Japanese translation of UJIP (UJIP-J), co-published by +O'Reilly & Associates, Incorporated and SOFTBANK Corporation, was just +released. The translation was done by my good friend Jack Halpern, +along with one of his colleagues, Takeo Suzuki. The Japanese edition +incorporates corrections and updates not yet found in the English +edition. The page count is 535. + Late-breaking news! I am currently working on UJIP Second +Edition (to be retitled as "Understanding CJK Information Processing" +and abbreviated UCJKIP). If all goes well, it should be available by +January 1997, and will be well over 700 pages. If there was something +you wanted to see in UJIP, now's your chance to send me a request... + + +PART 2: CJK CHARACTER SET STANDARDS + + These sections describe the character sets used in Japan, +China (PRC and Taiwan), and Korea. Exact numbers of characters are +provided for each character set standard (when known), as well as +tidbits of information not otherwise available. This provides the +basic foundations for understanding how CJK scripts are handled on +computer systems. + The two basic types of characters enumerated by CJK character +set standards are Chinese characters (kanji, hanzi, or hanja), which +number in the thousands (and, in some cases, tens of thousands), and +characters other than Chinese characters (symbols, numerals, kana +hangul, alphabets, and so on), which usually number in the hundreds +(there are thousands of pre-combined hangul, though). + If you happen to be running X Windows, it is very easy to +display these CJK character sets (if a bitmapped font for the +character set exists, that is). Here is what I usually do: + +o Obtain a BDF (Bitmap Distribution Format) font for the target + character set. Try the following URLs for starters: + + ftp://cair-archive.kaist.ac.kr/pub/hangul/fonts/ + ftp://etlport.etl.go.jp/pub/mule/fonts/ + ftp://ftp.ifcss.org/pub/software/fonts/{big5,cns,gb,misc,unicode}/bdf/ + ftp://ftp.kuis.kyoto-u.ac.jp/misc/fonts/jisksp-fonts/ + ftp://ftp.net.tsinghua.edu.cn/pub/Chinese/fonts/ + ftp://ftp.ora.com/pub/examples/nutshell/ujip/unix/ + ftp://ftp.technet.sg:/pub/chinese/fonts/ + http://ccic.ifcss.org/www/pub/software/fonts/ + + BDF files usually have the string "bdf" somewhere in their file + name, usually at the end. If the file is compressed (noticing that + it ends in .gz or .Z is a good indication), decompress it. BDF files + are text files. + +o Convert the BDF file to SNF (Server Natural Format) or PCF (Portable + Compiled Format) using the programs "bdftosnf" or "bdftopcf," + respectively. Example command lines are as follows: + + % bdftopcf jiskan16-1990.bdf > k16-90.pcf + % bdftosnf jiskan16-1990.bdf > k16-90.snf + + SNF files (and the "bdftosnf" program) are used on X11R4 and + earlier, and PCF files (and the "bdftopcf" program) are used on + X11R5 and later. + +o Copy the SNF or PCF file to a directory in the font search path (or + make a new path). Supposing I made a new directory called "fonts" in + my home directory, I then run "mkfontdir" on the directory + containing the SNF or PCF files as follows: + + % mkfontdir ~/fonts + + This creates a fonts.dir file in ~/fonts. I can now add this + directory to my font search path with the following command: + + % xset +fp ~/fonts + +o The command "xfd" (X Font Displayer) with the "-fn" switch followed + by a font name then invokes a window that displays all the + characters of the font. In the case of two-byte (CJK) fonts, one row + is displayed at a time. The following is an example command line: + + % xfd -fn -misc-fixed-medium-r-normal--16-150-75-75-c-160-jisx0208.1990-0 + + You can create a "fonts.alias" file in the same directory as the + "fonts.dir" file in order to shorten the name when accessing the + font. The alias "k16-90" could be used instead if the content of the + fonts.alias file is as follows: + + k16-90 -misc-fixed-medium-r-normal--16-150-75-75-c-160-jisx0208.1990-0 + + Don't forget to execute the following command in order to make the + X Font Server aware of the new alias: + + % xset fp rehash + + Now you can use a simpler command line for "xfd" as follows: + + % xfd -fn k16-90 + + The "X Window System User's Guide" (Volume 3 of the X Window +System series by O'Reilly & Associates, Inc.) provides detailed +information on managing fonts under X Windows (pp 123-160). The +article entitled "The X Administrator: Font Formats and Utilities" (pp +14-34 in "The X Resource," Issue 2), describes the BDF, SNF, and PCF +formats in great detail. + There is another bitmap format called HBF (Hanzi Bitmap +Format), which is similar to BDF, but optimized for fixed-width +(monospaced) fonts. It is described in the article entitled "The HBF +Font Format: Optimizing Fixed-pitch Font Support" (pp 113-123 in "The +X Resource," Issue 10), and also at the following URL: + + ftp://ftp.ifcss.org/pub/software/fonts/hbf-discussion/ + +HBF fonts can be found at the following URL: + + ftp://ftp.ifcss.org/pub/software/fonts/{big5,cns,gb,misc,unicode}/hbf/ + + Lastly, you may wish to check out my newly-developed CJK +Character Set Server, which generates various CJK character sets with +proper encoding applied. It is written in Perl, and accessed through +an HTML form. This server can be considered an upgrade to my JChar +tool (written in C). The URL is: + + http://jasper.ora.com/lunde/cjk-char.html + + +2.1: JAPANESE + + All (national) character set standards that originate in Japan +have names that begin with the three letters JIS. JIS is short for +"Japanese Industrial Standard." But it is JSA (Japanese Standards +Association) who publishes the corresponding manuals. Chapter 3 and +Appendixes H and J of UJIP provide more detailed information on +Japanese character set standards. + + +2.1.1: JIS X 0201-1976 + + JIS X 0201-1976 (formerly JIS C 6220-1969; reaffirmed in 1989; +and its revision [with no character set changes] is currently under +public review) enumerates two sets of characters: JIS-Roman and +half-width katakana. + JIS-Roman is the Japanese equivalent of the ASCII character +set, namely 128 characters consisting of the following: + +o 10 numerals +o 52 uppercase and lowercase characters of the Latin alphabet +o 32 symbols (punctuation and so on) +o 34 non-printing characters (white space and control characters) + +The term "white space" refers to characters that occupy space, but +have no appearance, such as tabs, spaces, and termination characters +(line feed, carriage return, and form feed). + So, how are JIS-Roman and ASCII different? The following +three codes are (usually) different: + + Code ASCII JIS-Roman + ^^^^ ^^^^^ ^^^^^^^^^ + 0x5C backslash yen symbol + 0x7C broken bar bar + 0x7E tilde overbar + + Half-width katakana consists of 63 characters that provide a +minimal set of characters necessary for expressing Japanese. The +shapes are compressed, and visually occupy a space half that of +*normal* Japanese characters. + + +2.1.2: JIS X 0208-1990 + + This basic Japanese character set standard enumerates 6,879 +characters, 6,355 of which are kanji separated into two levels. Kanji +in the first level are arranged by (most frequent) reading, and those +in the second level are arranged by radical then total number of +(remaining) strokes. + +o Row 1: 94 symbols +o Row 2: 53 symbols +o Row 3: 10 numerals and 52 uppercase and lowercase Latin alphabet +o Row 4: 83 hiragana +o Row 5: 86 katakana +o Row 6: 48 uppercase and lowercase Greek alphabet +o Row 7: 66 uppercase and lowercase Cyrillic (Russian) alphabet +o Row 8: 32 line-drawing elements +o Rows 16 through 47: 2,965 kanji (JIS Level 1 Kanji; last is 47-51) +o Rows 48 through 84: 3,390 kanji (JIS Level 2 Kanji; last is 84-06) + +Appendix B of UJIP provides a complete illustration of the JIS X +0208-1990 character set standard by KUTEN (row-cell) code. Appendix G +(pp 294-317) of "Developing International Software for Windows 95 and +Windows NT" by Nadine Kano illustrates the JIS X 0208-1990 character +set standard plus the Microsoft extensions by Shift-JIS code +(Microsoft calls this Code Page 932). + Earlier versions of this standard were dated 1978 (JIS C +6226-1978) and 1983 (JIS X 0208-1983, formerly JIS C 6226-1983). + JIS X 0208 went through a revision (from November 1995 until +February 1996), and is slated for publication sometime in 1996 (to +become JIS X 0208-1996). More information on this revision is +available at the following URL: + + ftp://ftp.tiu.ac.jp/jis/jisx0208/ + + +2.1.3: JIS X 0212-1990 + + This supplemental Japanese character set standard enumerates +6,067 characters, 5,801 of which are kanji ordered by radical then +total number of (remaining) strokes. All 5,801 kanji are unique when +compared to those in JIS X 0208-1990 (see Section 2.1.2). The +remaining 266 characters are categorized as non-kanji. + +o Row 2: 21 diacritics and symbols +o Row 6: 21 Greek characters with diacritics +o Row 7: 26 Eastern European characters +o Rows 9 through 11: 198 alphabetic characters +o Rows 16 through 77: 5,801 kanji (last is 77-67) + +Appendix C of UJIP provides a complete illustration of the JIS X +0212-1990 character set standard by KUTEN (row-cell) code. + The only commercial operating system that provides JIS X +0212-1990 support is BTRON by Personal Media Corporation: + + http://www.personal-media.co.jp/ + +Section 3.3.18 provides information about TRON Code (used by BTRON), +and details how it encodes the JIS X 0212-1990 character set. + + +2.1.4: JIS X 0221-1995 + + This document is, for all practical purposes, the Japanese +translation of ISO 10646-1:1993 (see Section 2.5.1). Like ISO +10646-1:1993, it is based on Unicode Version 1.1. + It is noteworthy that JIS X 0221-1995 enumerates subsets that +are applicable for Japanese use (a brief description of their contents +in parentheses): + +o BASIC JAPANESE (JIS X 0208-1990 and JIS X 0201-1976 -- characters + that can be created by means of combining are not included -- 6,884 + characters) +o JAPANESE NON IDEOGRAPHICS SUPPLEMENT (1,913 characters: all non- + kanji of JIS X 0212-1990 plus hundreds of non-JIS characters) +o JAPANESE IDEOGRAPHICS SUPPLEMENT 1 (918 frequently-used kanji from + JIS X 0212-1990, including 28 that are identical to kanji forms in + JIS C 6226-1978) +o JAPANESE IDEOGRAPHICS SUPPLEMENT 2 (the remainder of JIS X 0212- + 1990, namely 4,883 kanji) +o JAPANESE IDEOGRAPHICS SUPPLEMENT 3 (the remaining kanji of ISO + 10646-1:1993, namely 8,746 characters) +o FULLWIDTH ALPHANUMERICS (94 characters; for compatibility) +o HALFWIDTH KATAKANA (63 characters; for compatibility) + + Pages 893 through 993 provide Kangxi Zidian (a classic +300-year-old Chinese character dictionary containing approximately +50,000 characters) and Dai Kanwa Jiten (also known as Morohashi) +indexes for the entire Chinese character block, namely from 0x4E00 +through 0x9FA5. + At 25,750 Yen, it is actually cheaper than ISO 10646-1:1993! + + +2.1.5: JIS X 0213-199X + + I recently became aware that JSA plans to publish an extension +to JIS X 0208, containing approximately 2,000 characters (kanji and +non-kanji). A public review of this new standard is planned for Summer +1996. I would expect that its information will eventually be available +at the following URL: + + ftp://ftp.tiu.ac.jp/jis/ + + +2.1.6: OBSOLETE STANDARDS + + JIS C 6226-1978 and JIS X 0208-1983 (formerly JIS C 6226-1983) +have been superseded by JIS X 0208-1990. Section 4.1 provides details +on the changes made between these earlier versions of JIS X 0208. + JIS X 0221-1995 does not mean the end of JIS X 0201-1976, JIS +X 0208-1990, and JIS X 0212-1990. Instead, it will co-exist with those +standards. + + +2.2: CHINESE (PRC) + + All character set standards that originate in PRC have +designations that begin with "GB." "GB" is short for "Guo Biao" (which +is, in turn, short for "Guojia Biaojun") and means "National +Standard." A select few also have "/T" attached. The "T" presumably is +short for "Traditional." Section 2.2.11 describes ISO-IR-165:1992, +which is a variant of GB 2312-80. It is included here because of this +relationship. + Most people correlate GB character set standards with +simplified Chinese, but as you will see below, that is not always the +case. + There are three basic character sets, each one having a +simplified and traditional version. + + Character Set Set Number Character Forms + ^^^^^^^^^^^^^ ^^^^^^^^^^ ^^^^^^^^^^^^^^^ + GB 2312-80 0 Simplified + GB/T 12345-90 1 Traditional of GB 2312-80 + GB 7589-87 2 Simplified + GB/T 13131-9X 3 Traditional of GB 7589-87 + GB 7590-87 4 Simplified + GB/T 13132-9X 5 Traditional of GB 7590-87 + + +2.2.1: GB 1988-89 + + This character set, formerly GB 1988-80 and sometimes referred +to as GB-Roman, is the Chinese analog to ASCII and ISO 646. The main +difference is that the currency symbol (0x24), which is represented as +a dollar sign ($) in ASCII, is represented as a Chinese Yuan +(currency) symbol instead. GB 1988-89 is sometimes referred to as +GB-Roman. + + +2.2.2: GB 2312-80 + + This basic (simplified) Chinese character set standard +enumerates 7,445 characters, 6,763 of which are hanzi separated into +two levels. Hanzi in the first level are arranged by reading, and +those in the second level are arranges by radical then total number of +(remaining) strokes. GB 2312-80 is also known as the "Primary Set," +GB0 (zero), or just GB. + +o Row 1: 94 symbols +o Row 2: 72 numerals +o Row 3: 94 full-width GB 1988-89 characters (see Section 2.2.1) +o Row 4: 83 hiragana +o Row 5: 86 katakana +o Row 6: 48 uppercase and lowercase Greek alphabet +o Row 7: 66 uppercase and lowercase Cyrillic (Russian) alphabet +o Row 8: 26 Pinyin and 37 Bopomofo characters +o Row 9: 76 line-drawing elements (09-04 through 09-79) +o Rows 16 through 55: 3,755 hanzi (Level 1 Hanzi; last is 55-89) +o Rows 56 through 87: 3,008 hanzi (Level 2 Hanzi; last is 87-94) + +Compare some of the structure with JIS X 0208-1990, and you will find +many similarities, such as: + +o Hiragana, katakana, Greek, and Cyrillic characters are in Rows 4, 5, + 6, and 7, respectively +o Chinese characters begin at Row 16 +o Chinese characters are separated into two levels +o Level 1 arranged by reading +o Level 2 arranged by radical then total number of strokes + +The Japanese standard, JIS C 6226-1978, came out in 1978, which means +that it pre-dates GB 2312-80. The above similarities could not be by +coincidence, but rather by design. + Appendix G (pp 318-344) of "Developing International Software +for Windows 95 and Windows NT" by Nadine Kano illustrates the GB 2312- +80 character set standard by EUC code (Microsoft calls this Code Page +936). Code Page 936 incorporates the correction of the hanzi at 79-81, +and the correction of the order of 07-22 and 07-23 (see Section 2.2.3 +for more details). + + +2.2.3: GB 6345.1-86 + + This document specifies corrections and additions to GB +2312-80 (see Section 2.2.2). The following is a detailed enumeration +of the changes: + +o The form of "g" in Row 3 (position 71) was altered +o Row 8 has six additional Pinyin characters (08-27 through 08-32) +o Row 10 contains half-width versions of Row 3 (94 characters) +o Row 11 contains half-width versions of the Pinyin characters from + Row 8 (32 characters; 11-01 through 11-32) +o The hanzi at 79-81 was corrected to have a simplified left-side + radical (this was an error in GB 2312-80) + +Note that these changes affect the total number of characters in GB +2312-80 -- an increase of 132 characters. This now makes 7,577 as +the total number of characters in GB 2312-80 (7,445 plus 132). + There was, however, an undocumented correction made in GB +6345.1-86. The order of characters 07-22 and 07-23 (uppercase +Cyrillic) were reversed. This error is apparently in the first and +perhaps second printing of the GB 2312-80 manual, because the copy I +have is from the third printing, and this has been corrected. Page 145 +(Figure 113) of John Clews' "Language Automation Worldwide: The +Development of Character Set Standards" illustrates this error. +Developers should take special note of this -- I have seen GB 2312-80 +based font products that propagate this ordering error. + + +2.2.4: GB 7589-87 + + This character set enumerates 7,237 hanzi in Rows 16 through +92 (last is 92-93), and they are ordered by radical then total number +of (remaining) strokes. GB 7589-87 is also known as the "Second +Supplementary Set" or GB2. + + +2.2.5: GB 7590-87 + + This character set enumerates 7,039 hanzi in Rows 16 through +90 (last is 90-83), and they are ordered by radical then total number +of (remaining) strokes. GB 7590-87 is also known as the "Fourth +Supplementary Set" or GB4. + + +2.2.6: GB 8565.2-88 + + This standard makes additions to GB 2312-80 (these additions +are separate from those made in GB 6345.1-86 described in Section +2.2.3). GB 8565.2-88 is also known as GB8. In this case there are 705 +additions, indicated as follows: + +o Row 13 contains 50 hanzi from GB 7589-87 (last is 13-50) +o Row 14 contains 92 hanzi from GB 7590-87 (last is 14-92) +o Row 15 contains 69 non-hanzi indicating dates and times, plus 24 + miscellaneous hanzi (for personal/place names and radicals; last is + 15-93). +o Rows 90 through 94 contain 470 hanzi from GB 7589-87 (94 each) + +GB 8565.2-88 therefore provides a total of 8,150 characters (7,445 +plus 705). + + +2.2.7: GB/T 12345-90 + + This character set is nearly identical to GB 2312-80 (see +Section 2.2.2) in terms of the number and arrangement of characters, +but simplified hanzi are replaced by their traditional versions. GB/T +12345-90 is also known as the "Supplementary Set" or GB1. + The following are some interesting facts about this character +set (some instances of simplified/traditional pairs that appear below +are actually character form differences): + +o 29 vertical-use characters (punctuation and parentheses) included in + Row 6 (06-57 through 06-85). + +o 2,118 traditional hanzi replace simplified hanzi in Rows 16 through + 87. The "G1-Unique" appendix of the unofficial version (supplied to + the CJK-JRG for Han Unification purposes) is missing the following + four (specifies only 2,114): + + 0x5B3B 0x6D2F + 0x5E7C 0x6F71 + + But, ISO 10646-1:1993 ended up getting these hanzi included anyway, + with correct mappings. + +o Four simplified/traditional hanzi pairs (eight affected code points) + in rows 16 through 87 are swapped: + + 0x3A73 <-> 0x6161 + 0x5577 <-> 0x6167 + 0x5360 <-> 0x6245 (see the next bullet) + 0x4334 <-> 0x7761 + +o One hanzi (0x6245), after being swapped, had its left-side radical + unsimplified (this character, now at 0x5360, is considered part of + the 2,118 traditional hanzi from the second bullet): + + 0x6245 -> 0x5360 + +o 103 hanzi included in Rows 88 (94 characters) and 89 (9 characters; + 89-01 through 89-09). These are all related to characters between + Rows 16 and 87. + + - 41 simplified hanzi from Rows 16 through 87 moved to Rows 88 and + 89 (traditional hanzi are now at the original code points): + + 0x3327 -> 0x7827 0x3E5D -> 0x7846 0x4B49 -> 0x7869 + 0x3365 -> 0x7828 0x3F64 -> 0x7849 0x4C28 -> 0x786B + 0x3373 -> 0x7829 0x402F -> 0x784B 0x4D3F -> 0x786F + 0x3533 -> 0x782C 0x4030 -> 0x784C 0x4D72 -> 0x7871 + 0x356D -> 0x782D 0x406F -> 0x784E 0x5236 -> 0x7878 + 0x3637 -> 0x782F 0x4131 -> 0x7850 0x5374 -> 0x7879 + 0x3736 -> 0x7832 0x463B -> 0x785C 0x5438 -> 0x787C + 0x3761 -> 0x7833 0x463E -> 0x785D 0x5446 -> 0x787D + 0x3849 -> 0x7835 0x464B -> 0x785E 0x5622 -> 0x7921 + 0x3963 -> 0x7838 0x464D -> 0x785F 0x563B -> 0x7923 + 0x3B2E -> 0x783B 0x4653 -> 0x7860 0x5656 -> 0x7926 + 0x3C38 -> 0x7840 0x4837 -> 0x7866 0x567E -> 0x7928 + 0x3C5B -> 0x7842 0x4961 -> 0x7867 0x573C -> 0x7929 + 0x3C76 -> 0x7843 0x4A75 -> 0x7868 + + - 62 hanzi added to Rows 88 and 89 (the gaps from the above are + filled in). These were mostly to account for multiple traditional + hanzi collapsing into a single simplified form. + + - The following code point mappings illustrate how all of these 103 + hanzi are related to hanzi between Rows 16 and 87 (note how many + of these 103 hanzi map to a single code point): + + 0x7821 -> 0x305A 0x7844 -> 0x3D2A 0x7867 -> 0x4961 + 0x7822 -> 0x3065 0x7845 -> 0x3E21 0x7868 -> 0x4A75 + 0x7823 -> 0x316D 0x7846 -> 0x3E5D 0x7869 -> 0x4B49 + 0x7824 -> 0x3170 0x7847 -> 0x3E6D 0x786A -> 0x4B55 + 0x7825 -> 0x3237 0x7848 -> 0x3F4B 0x786B -> 0x4C28 + 0x7826 -> 0x3245 0x7849 -> 0x3F64 0x786C -> 0x4C28 + 0x7827 -> 0x3327 0x784A -> 0x4027 0x786D -> 0x4C28 + 0x7828 -> 0x3365 0x784B -> 0x402F 0x786E -> 0x4C33 + 0x7829 -> 0x3373 0x784C -> 0x4030 0x786F -> 0x4D3F + 0x782A -> 0x3376 0x784D -> 0x405B 0x7870 -> 0x4D45 + 0x782B -> 0x3531 0x784E -> 0x406F 0x7871 -> 0x4D72 + 0x782C -> 0x3533 0x784F -> 0x407A 0x7872 -> 0x4F35 + 0x782D -> 0x356D 0x7850 -> 0x4131 0x7873 -> 0x4F35 + 0x782E -> 0x362C 0x7851 -> 0x414B 0x7874 -> 0x4F4C + 0x782F -> 0x3637 0x7852 -> 0x4231 0x7875 -> 0x4F72 + 0x7830 -> 0x3671 0x7853 -> 0x425E 0x7876 -> 0x506B + 0x7831 -> 0x3722 0x7854 -> 0x4339 0x7877 -> 0x5229 + 0x7832 -> 0x3736 0x7855 -> 0x4349 0x7878 -> 0x5236 + 0x7833 -> 0x3761 0x7856 -> 0x4349 0x7879 -> 0x5374 + 0x7834 -> 0x3834 0x7857 -> 0x4349 0x787A -> 0x5379 + 0x7835 -> 0x3849 0x7858 -> 0x4356 0x787B -> 0x5375 + 0x7836 -> 0x3948 0x7859 -> 0x4366 0x787C -> 0x5438 + 0x7837 -> 0x394E 0x785A -> 0x436F 0x787D -> 0x5446 + 0x7838 -> 0x3963 0x785B -> 0x3159 0x787E -> 0x5460 + 0x7839 -> 0x6358 0x785C -> 0x463B 0x7921 -> 0x5622 + 0x783A -> 0x3A7A 0x785D -> 0x463E 0x7922 -> 0x563B + 0x783B -> 0x3B2E 0x785E -> 0x464B 0x7923 -> 0x563B + 0x783C -> 0x3B58 0x785F -> 0x464D 0x7924 -> 0x5642 + 0x783D -> 0x3B63 0x7860 -> 0x4653 0x7925 -> 0x5646 + 0x783E -> 0x3B71 0x7861 -> 0x4727 0x7926 -> 0x5656 + 0x783F -> 0x3C22 0x7862 -> 0x4729 0x7927 -> 0x566C + 0x7840 -> 0x3C38 0x7863 -> 0x4F4B 0x7928 -> 0x567E + 0x7841 -> 0x3C52 0x7864 -> 0x476F 0x7929 -> 0x573C + 0x7842 -> 0x3C5B 0x7865 -> 0x477A + 0x7843 -> 0x3C76 0x7866 -> 0x4837 + +So, if we total everything up, we see that GB/T 12345-90 has 2,180 +hanzi (2,118 are replacements for GB 2312-80 code points, and 62 are +additional) and 29 non-hanzi not found in GB 2312-80. + Note that the printing of the GB/T 12345-90 has some +character-form errors. The errors I am aware of are as follows: + + Code Point Description of Error + ^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^ + 0x4125 The upper-left element should be "tree" instead of + "warrior" + 0x596C The "bird" radical should not include the "fire" element + + +2.2.8: GB/T 13131-9X + + This character set is identical to GB 7589-87 (see Section +2.2.4) in terms of number of characters, but simplified hanzi are +replaced by their traditional versions. The exact number of such +substitutions is currently unknown to this author. GB/T 13131-9X is +also known as the "Third Supplementary Set" or GB3. + + +2.2.9: GB/T 13132-9X + + This character set is identical to GB 7590-87 (see Section +2.2.5) in terms of number of characters, but simplified hanzi are +replaced by their traditional versions. The exact number of such +substitutions is currently unknown to this author. GB/T 13132-9X is +also known as the "Fifth Supplementary Set" or GB5. + + +2.2.10: GB 13000.1-93 + + This document is, for all practical purposes, the Chinese +translation of ISO 10646-1:1993 (see Section 2.5.1). + + +2.2.11: ISO-IR-165:1992 + + This standard, also known as the CCITT Chinese Set, is a +variant of GB 2312-80 with the following characteristics: + +o GB 6345.1-86 modifications (including the undocumented one) and + additions, namely 132 characters (see Section 2.2.3) +o GB 8565.2-88 additions, namely 705 characters (see Section 2.2.6) +o Row 6 contains 22 background (shading) characters (06-60 through + 06-81) +o Row 12 contains 94 hanzi +o Row 13 contains 44 additional hanzi (13-51 through 13-94; fills the + row) +o Row 15 contains 1 additional hanzi (15-94) + +ISO-IR-165:1992 can therefore be considered a superset of GB 2312-80, +GB 6345.1-86, and GB 8565.2-88. This means 8,443 total characters +compared to the 7,445 in GB 2312-80, 7,577 in GB 6345.1-86, and the +8,150 in GB 8565.2-88. + + +2.2.12: OBSOLETE STANDARDS + + Most GB standards seem to be revised through other documents, +so it is hard to point to a standard and claim that it is obsolete. +The only revision I am aware of is the GB 1988-89 (the original was +named GB 1988-80). + + +2.3: CHINESE (TAIWAN) + + The sections below describe two major Taiwanese character +sets, namely Big Five and CNS 11643-1992. As you will learn they are +somewhat compatible. CCCII, also developed in Taiwan, is described in +Section 2.5.2. + + +2.3.1: BIG FIVE + + The Big Five character set is composed of 94 rows of 157 +characters each (the 157 characters of each row are encoded in an +initial group of 63 codes followed by the remaining 94 codes). The +following is a break-down of its contents: + +o Row 1: 157 symbols +o Row 2: 157 symbols +o Row 3: 94 symbols +o Rows 4 through 38: 5,401 hanzi (Level 1 Hanzi; last is 38-63) +o Rows 41 through 89: 7,652 hanzi (Level 2 Hanzi; last is 89-116) + +This forms what I consider to be the basic Big Five set. Actually, two +of the hanzi in Level 2 are duplicates, so there are actually only +7,650 unique hanzi in Level 2. + There are two major extensions to Big Five. The first really +has no name, and can be considered part of the basic Big Five set as +specified above. It adds the following characters: + +o Rows 38-39: 4 Japanese iteration marks, 83 hiragana, 86 katakana, 66 + uppercase and lowercase Cyrillic (Russian) alphabet, 10 circled + digits, and 10 parenthesized digits + + The other extension was developed by a company called ETen +Information System in Taiwan, and is actually considered to be the +most widely used version of Big Five. It provides the following +extensions to Big Five (different from the above extension): + +o Rows 38-40: 10 circled digits, 10 parenthesized digits, 10 lowercase + Roman numerals, 25 classical radicals, 15 Japanese-specific symbols, + 83 hiragana, 86 katakana, 66 uppercase and lowercase Cyrillic + (Russian) alphabet, 3 arrows, 10 radical-like hanzi elements, 40 + fraction-like digits, and 7 symbols +o Row 89: 7 hanzi, 33 double-lined line-drawing elements, and a black + box + + It is *very* important to note that while these two extensions +have many common portions (in particular, hiragana, katakana, the +Cyrillic alphabet, and so on), they do not share the same code points +for such characters. + Appendix G (pp 407-450) of "Developing International Software +for Windows 95 and Windows NT" by Nadine Kano illustrates the Big Five +character set standard by Big Five code (Microsoft calls this Code +Page 950). Code Page 950 incorporates some of the ETen extensions, +namely those in Row 89. + + +2.3.2: CNS 11643-1992 + + CNS 11643-1992 (also known as CNS 11643 X 5012), by +definition, consists of 16 planes of characters, seven of which have +character assignments. Each plane is a 94-row-by-94-cell matrix +capable of holding a total of 8,836 characters. CNS stands for +"Chinese National Standard." + CNS 11643-1992 specifies characters only in the first seven +planes. A break-down of characters, by plane, is as follows: + +o Plane 1: + - 438 symbols in Rows 1 through 6 + - 213 classical radicals in Rows 7 through 9 + - 33 graphic representations of control characters in Row 34 + - 5,401 hanzi in Rows 36 through 93 (last is 93-43) +o Plane 2: 7,650 hanzi in Rows 1 through 82 (last is 82-36) +o Plane 3: 6,148 hanzi in Rows 1 through 66 (last is 66-38) +o Plane 4: 7,298 hanzi in Rows 1 through 78 (last is 78-60) +o Plane 5: 8,603 hanzi in Rows 1 through 92 (last is 92-49) +o Plane 6: 6,388 hanzi in Rows 1 through 68 (last is 68-90) +o Plane 7: 6,539 hanzi in Rows 1 through 70 (last is 70-53) + +The total number of characters in CNS 11643-1992 is a staggering +48,711 characters, 48,027 of which are hanzi. Also note that number of +hanzi in Plane 1 is identical to Level 1 hanzi of Big Five (see +Section 2.3.1). The 2 extra hanzi in Level 2 hanzi of Big Five are +actually redundant, and are therefore not in CNS 11643-1992 Plane 2. + It is rumored that Plane 8 is currently being defined, and +will add yet more hanzi to this standard. + + +2.3.3: CNS 5205 + + This character set is Taiwan's analog to ASCII and ISO 646, +and is reportedly rarely used. How it differs from ASCII, if at all, +is unknown to this author. + + +2.3.4: OBSOLETE STANDARDS + + CNS 11643-1986 specified characters only in the first three +planes, as described in Section 2.3.2. Also, Plane 3 of CNS 11643-1992 +was called Plane 14 of CNS 11643-1986. + + +2.4: KOREAN + + The sections below describe the most current Korean character +sets, namely KS C 5636-1993, KS C 5601-1992, KS C 5657-1991, and KS C +5700-1995. "KS" stands for "Korean Standard." + + +2.4.1: KS C 5636-1993 + + This character set (published on January 6, 1993), formerly KS +C 5636-1989 (published on April 22, 1989) and sometimes referred to as +KS-Roman, is the Korean analog to ASCII and ISO 646-1991. The primary +difference is that the ASCII backslash (0x5C) is represented as a Won +symbol. + + +2.4.2: KS C 5601-1992 + + This basic Korean character set standard enumerates 8,224 +characters, 4,888 of which are hanja, and 2,350 of which are pre- +combined hangul. The hanja and hangul blocks are arranged by reading. +The following is a break-down of its contents: + +o Row 1: 94 symbols +o Row 2: 69 abbreviations and symbols +o Row 3: 94 full-width KS C 5636-1993 characters (see Section 2.4.1) +o Row 4: 94 hangul elements +o Row 5: 68 lowercase and uppercase Roman numerals and lowercase and + uppercase Greek alphabet +o Row 6: 68 line-drawing elements +o Row 7: 79 abbreviations +o Row 8: 91 phonetic symbols, circled characters, and fractions +o Row 9: 94 phonetic symbols, parenthesized characters, subscripts, + and superscripts +o Row 10: 83 hiragana +o Row 11: 86 katakana +o Row 12: 66 lowercase and uppercase Cyrillic (Russian) alphabet +o Rows 16 through 40: 2,350 pre-combined hangul (last is 40-94) +o Rows 42 through 93: 4,888 hanja (last is 93-94) + +Rows 41 and 94 are designated for user-defined characters. + There are many similarities with JIS X 0208-1990 and GB +2312-80, such as hiragana, katakana, Greek, and Cyrillic characters, +but they are assigned to different rows. + There is an interesting note about the hanja block (Rows 42 +through 93). Although there are 4,888 hanja, not all are unique. The +hanja block is arranged by reading, and in those cases when a hanja +has more than one reading, that hanja is duplicated (sometimes more +than once) in the same character set. There are 268 such cases of +duplicate hanja in KS C 5601-1992, meaning that it contains 4,620 +unique hanja. If you have a copy of the KS C 5601-1992 manual handy, +you can compare the following four code points: + + 0x6445 + 0x5162 + 0x5525 + 0x6879 + +While most of these cases involve two hanja instances, there are four +hanja that have three instances, and one (listed above) that has four! +This is the only CJK character set that has this property of +intentionally duplicating Chinese characters. See Section 4.4 for more +details. + Annex 3 of this standard defines the complete set of 11,172 +pre-combined hangul characters, also known as Johab. Johab refers to +the encoding method, and is almost like encoding all possible three- +letter words (meaning that most are nonsense). See Section 3.3.5 for +more details on Johab encoding. + + +2.4.3: KS C 5657-1991 + + This character set standard provides supplemental characters +for Korean writing, to include symbols, pre-combined hangul, and +hanja. The following is a break-down of its contents: + +o Rows 1 through 7: 613 lowercase and uppercase Latin characters with + diacritics (see note below) +o Rows 8 through 10: 273 lowercase and uppercase Greek characters with + diacritics +o Rows 11 through 13: 275 symbols +o Row 14: 27 compound hangul elements +o Rows 16 through 36: 1,930 pre-combined hangul (last is 36-50) +o Rows 37 through 54: 1,675 pre-combined hangul (last is 54-77; see + note below) +o Rows 55 through 85: 2,856 hanja (last is 85-36) + +The KS C 5657-1991 manual has a possible error (or at least an +inconsistency) for Rows 1 through 7. The manual says that there are +615 characters in that range, but I only counted 613. The difference +can be found on page 19 as the following two characters: + + Character Code Character + ^^^^^^^^^^^^^^ ^^^^^^^^^ + 0x2137 X + 0x217A TM + +An "X" doesn't belong there (it is already in KS C 5601-1992 at code +point 0x2358), and the trademark symbol is also part of KS C 5601-1992 +at code point 0x2262. This is why I feel that my count of 613 is more +accurate than what is explicitly stated in the manual on page 2. + Also, page 2 of the manual says that Rows 37 through 54 +contains 1,677 pre-combined hangul, but I only counted 1,675 (17 rows +of 94 characters plus a final row with 77 characters -- do the math +for yourself). + Here's another interesting note. My official copy of this +standard has all of its 2,856 hanja hand-written. + + +2.4.4: GB 12052-89 + + You may be asking yourself why a GB standard is listed under +the Korean section of this document. Well, there is a rather large +Korean population in China (Korea was considered part of China before +the 1890s), and they need a character set standard for communicating +using hangul. GB 12052-89 is a Korean character set standard +established by China (PRC), and enumerates a total of 5,979 +characters. + The following is the arrangement of this character set: + +o Row 1: 94 symbols +o Row 2: 72 numerals +o Row 3: 94 full-width ASCII characters +o Row 4: 83 hiragana +o Row 5: 86 katakana +o Row 6: 48 uppercase and lowercase Greek alphabet +o Row 7: 66 uppercase and lowercase Cyrillic (Russian) alphabet +o Row 8: 26 Pinyin and 37 Bopomofo characters +o Row 9: 76 line-drawing elements (09-04 through 09-79) +o Rows 16 through 37: 2,068 pre-combined hangul (Level 1 Hangul, Part + 1; last is 37-94) +o Rows 38 through 52: 1,356 pre-combined hangul (Level 1 Hangul, Part + 2; last is 52-40) +o Rows 53 through 71: 1,779 pre-combined hangul (Level 2 Hangul; last + is 71-87) +o Rows 71 through 72: 94 "Idu" hanja (71-89 through 72-88) + + There are a few interesting notes I can make about this +character set: + +o Rows 1 through 9 are identical to the same rows in GB 2312-80, + except that 03-04 is a dollar sign, not a Chinese Yuan (currency) + symbol. + +o The GB 12052-89 manual states on pp 1 and 3 that Rows 53 through 72 + contain 1,876 characters, but I only counted 1,873 (1,779 hangul + plus 94 hanja). + +o The total number of characters, 5,979, is correctly stated in the + manual although the hangul count is incorrect. + +o The arrangement and ordering of these hangul bear no relationship to + that of KS C 5601-1992. Both standards order by reading, which is + the only way in which they are similar. + + I am not aware to what extent this character set is being +used (and who might be using it). + + +2.4.5: KS C 5700-1995 + + Korea has developed a new character set standard called KS C +5700-1995. It is equivalent to ISO 10646-1:1993, but have pre-combined +hangul as provided (and ordered) in Unicode Version 2.0 (meaning that +all 11,172 hangul are in a contiguous block). + + +2.4.6: OBSOLETE STANDARDS + + KS C 5601-1986, KS C 5601-1987, and KS C 5601-1989 are the +same, character-set wise, to KS C 5601-1992. The 1992 edition provides +more material in the form of annexes. KS C 5601-1982, the original +version, enumerated only the 51 basic hangul elements in a one-byte 7- +and 8-bit encoding. This information is still part of KS C 5601-1992, +but in Annex 4. + There were two earlier multiple-byte standards called KS C +5619-1982 and KIPS. KS C 5619-1982 enumerated 51 hangul elements, +1,316 pre-combined hangul, and 1,672 hanja. KIPS (Korean Information +Processing System) enumerated 2,058 pre-combined hangul and 2,392 +hanja. Both have been rendered obsolete by KS C 5601-1987. + + +2.5: CJK + + The only true CJK character sets available today are CCCII, +ANSI Z39.64-1989 (also known as EACC or REACC), and ISO 10646-1:1993. +ISO 10646-1:1993 is unique in that it goes beyond CJK (Chinese +characters) to provide virtually all commonly-used alphabetic scripts. + Of these three, only ISO 10646-1:1993 is expected to gain +wide-spread acceptance. CCCII and ANSI Z39.64-1989 are still used +today, but primarily for bibliographic purposes. + + +2.5.1: ISO 10646-1:1993 + + Published by ISO (International Organization for +Standardization) in Switzerland, this character set enumerates over +34,000 characters. Its I-zone ("I" stands for "Ideograph") enumerates +approximately 21,000 Chinese characters, which is the result of a +massive effort by the CJK-JRG (CJK Joint Research Group) called "Han +Unification." The CJK-JRG is now called the IRG (Ideographic +Rapporteur Group), and is off doing additional research for future +Chinese character allocations to ISO 10646-1:1993. + The Basic Multilingual Plane (BMP) of ISO 10646-1:1993 is +equivalent to Unicode. While Unicode is comprised of a single plane of +characters (which doesn't allow much room for future expansion), ISO +10646-1:1993 contains hundreds of such planes. + One very nice feature of this standard's manual are the CJK +code correspondence tables in Section 26 (pp 262-698). Four columns +are provided for each ISO 10646-1:1993 I-zone code point -- simplified +Chinese, traditional Chinese, Japanese, and Korean. If the ISO +10646-1:1993 Chinese character maps to one of these locales, the +hexadecimal character code, (decimal) row-cell value, and glyph for +that locale is provided. The corresponding tables in Volume 2 of "The +Unicode Standard" provide character codes (sometimes the hexadecimal +character code, and sometimes the row-cell value) and a single +glyph. Quite unfortunate. I hear that a new edition of "The Unicode +Standard" is about to be released. I hope that this problem has been +addressed. + ISO 10646-1:1993 does not replace existing national character +set standards. It simply provides a single character set that is a +superset of *most* national character sets. For example, only a +fraction of the 48,027 hanzi in CNS 11643-1992 are included in ISO +10646-1:1993. I feel that it is best to think of ISO 10646-1:1993 as +"just another character set." My philosophy is to support the maximum +number of character sets and encodings as possible. + A note about ordering this standard. If you order through ANSI +in the United States, try to get an original manual. It is not easy, +though. You see, ANSI has duplication rights for ISO documents. +Photocopying Section 26 (pp 262-698) doesn't do the Chinese characters +much justice, and some characters become hard-to-read. Unfortunately, +there is no way to indicate that you want an original ISO document +through ANSI's ordering process, so some post-ordering haggling may +become necessary. + More information on ISO 10646-1:1993 can be found at the +following URL: + + http://www.unicode.org/ + + Japan, China (PRC), and Korea have developed their own +national standards that are based on ISO 10646-1:1993. They are +designated as JIS X 0221-1995 (see Section 2.1.4), GB 13000.1-93 (see +Section 2.2.10), and KS C 5700-1995 (see Section 2.4.5), respectively. + Note that these national-standard versions of Unicode are +aligned differently with its three versions: + + Unicode Version 1.0 + Unicode Version 1.1 <-> ISO 10646-1:1993, JIS X 0221-1995, GB 13000.1-93 + Unicode Version 2.0 <-> KS C 5700-1995 + +One of the major changes made for Unicode Version 2.0 is the inclusion +of all 11,172 hangul. Versions 1.1 has 6,656 hangul. + + +2.5.2: CCCII + + The Chinese Character Analysis Group in Taiwan developed CCCII +(Chinese Character Code for Information Interchange) in the 1980s. +This character set is composed of 94 planes that have 94 rows and 94 +cells (94 x 94 x 94 = 830,584 characters). Furthermore, every six +planes constitute a "layer" (6 x 94 x 94 = 53,016 characters). The +following is the contents of each of the 16 layers (the 16th layer +contains only four planes): + +o Layer 1: Symbols and Traditional Chinese characters +o Layer 2: Simplified Chinese characters from PRC +o Layers 3 through 12: Variant Chinese character forms +o Layer 13: Japanese kana and kokuji (Japanese-made kanji) +o Layer 14: Korean hangul +o Layer 15: Reserved +o Layer 16: Miscellaneous characters (Japanese and Korean) + + Layers 1 through 12 have a special meaning and relationship. +The same code point in these layers is designed to hold the same +character, but with different forms. Layer 1 code points contain the +traditional character forms, Layer 2 code points contain the +simplified character forms (if any), and Layers 3 through 12 contain +variant character forms (if any). For example, given a Chinese +character with three forms, its encoding and arrangement may be as +follows: + + Character Form Code Point Layer + ^^^^^^^^^^^^^^ ^^^^^^^^^^ ^^^^^ + Traditional 0x224E41 1 + Simplified 0x284E41 2 + Variant 0x2E4E41 3 + +Note how the second and third bytes (0x4E41) are identical in all +three instances -- only the first byte's value, which indicates the +layer, differs. Needless to say, this method of arrangement provides +easy access to related Chinese character forms. No wonder it is used +for bibliographic purposes. + The first layer is composed as follows: + +o Plane 1/Row 2: 56 mathematical symbols +o Plane 1/Row 3: The ASCII character set +o Plane 1/Row 11: 35 Chinese punctuation marks +o Plane 1/Rows 12 through 14: 214 classical radicals +o Plane 1/Row 15: 41 Chinese numerical symbols, 37 phonetic symbols, + and 4 tone marks +o Plane 1/Rows 16 through 67: 4,808 common Chinese characters +o Plane 1/Row 68 through Plane 3/Row 64: 17,032 less common Chinese + characters +o Plane 3/Row 65 through Plane 6/Row 5: 20,583 rare Chinese characters + +Note that Row 1 of all planes is reserved, and never assigned +characters. Take this into account when studying the above table +ranges that span planes (that is, skip Row 1). + In addition to the above, there are 11,517 simplified Chinese +characters in Layer 2 (3,625 are considered PRC simplified forms, and +the remaining 7,892 are regular simplified forms). This provides a +total of 53,940 Chinese characters. + Further information on CCCII (to include very interesting +historical notes) can be found on pp 146-149 of John Clews' "Language +Automation Worldwide: The Development of Character Set Standards" and +Chapter 6 of Huang & Huang's "An Introduction to Chinese, Japanese, +and Korean Computing." + + +2.5.3: ANSI Z39.64-1989 + + This national standard is designated as ANSI Z39.64-1989 and +named "East Asian Character Code" (EACC), but was originally known as +REACC (RLIN East Asian Character Code), that is, before it became a +national standard. RLIN stands for "Research Libraries Information +Network," which was developed by the Research Libraries Group (RLG) +located in Mountain View, California. + RLG's Home Page is at the following URL: + + http://www.rlg.org/ + + The structure of ANSI Z39.64-1989 is based on CCCII, but with +a few differences. Many consider it to be superior to and a +replacement for CCCII (see Section 2.5.2). + The ANSI Z39.64-1989 standard is available through ANSI, but +you should be aware that it is distributed in the form of several +microfiche. Not a terribly useful storage medium these days. I had my +set tranformed into tangible printed pages. You can also obtain this +standard through NISO (National Information Standards Organization) +Press Fulfillment. Their URL is: + + http://www.niso.org/ + + EACC has been designated by the Library of Congress as a +character set for use in USMARC (United States MAchine-Readable +Cataloging) records, and is used extensively by East Asian libraries +across North America. + EACC is also being used in Australia for the National CJK +Project. Check out the following URL for more details: + + http://www.nla.gov.au/1/asian/ncjk/cjkhome.html + + Further information on ANSI Z39.64-1989 (to include very +interesting historical notes) can be found on pp 150-156 of John +Clews' "Language Automation Worldwide: The Development of Character +Set Standards" (although a source at RLG tells me that some of Clews' +facts are wrong) and Chapter 6 of Huang & Huang's "An Introduction to +Chinese, Japanese, and Korean Computing." + The authoritative paper on EACC is "RLIN East Asian Character +Code and the RLIN CJK Thesaurus" by Karen Smith Yoshimura and Alan +Tucker, published in "Proceedings of the Second Asian-Pacific +Conference on Library Science," May 20-24,1985, Seoul, Korea. + + +2.6: OTHER + + This section includes character set standards that don't +properly fall under the above sections. + + +2.6.1: GB 8045-87 + + GB 8045-87 is a Mongolian character set standard established +by China (PRC). This standard enumerates 94 Mongolian characters. Of +these 94 characters, 12 are punctuation (vertically-oriented), and the +remaining 82 are characters specific to the Mongolian script. +Mongolian is written vertically like Chinese. + I do not discuss the encoding for GB 8045-87 in Part 3, so +will do it here. The GB 8045-87 manual describes a 7- and 8-bit +encoding. The 7-bit encoding puts these 94 characters in the standard +ASCII printable range, namely 0x21 through 0x7E. Code point 0x20 is +marked as "MSP" which stands for "Mongolian space." The 8-bit encoding +puts these 94 characters in the range 0xA1 through 0xFE, with the +"MSP" character at code point 0xA0. The GB 1988-89 set is then encoded +in the range 0x21 through 0x7E. + + +2.6.2: TCVN-5773:1993 + + TCVN-5773:1993 (also called NSCII, which is short for Nom +Standard Code for Information Interchange) is the Vietnamese analog to +ISO 10646-1:1993, but adds 1,775 Vietnamese-specific Chinese +characters. These 1,775 characters are encoded in the range 0xA000 +through 0xA6EE. + More information on TCVN-5773:1993 can be found at the +following URL: + + ftp://unicode.org/pub/MappingTables/EastAsiaMaps/ + +There are two files at the above URL that pertain to this standard. +The first is a README, and the second is a Macintosh HyperCard stack +(requires HyperCard): + + TCVN-NSCII.README + TCVN-NSCIIstack_1.0.sea.hqx + + +PART 3: CJK ENCODING SYSTEMS + + These sections describe the various systems for encoding the +character set standards listed in Part 2. The first two described, +7-bit ISO 2022 and EUC, are not specific to a locale, and in some +cases not specific to CJK. + The CJK Character Set Server at the following URL can generate +character sets based on encodings described in this section: + + http://jasper.ora.com/lunde/cjk-char.html + +I suggest that you use this as a way to obtain files that illustrate +these encodings in action. + But first, please take a peek at the following table, which is +an attempt to illustrate how two Chinese characters (that stand for +"kanji/hanzi/hanja") are encoded using the various methods presented +in the following sections (character codes as hexadecimal digits, and +escape sequences or shift sequences as printable characters): + +o Japanese (JIS X 0208-1990 & JIS X 0201-1976): + - 7-bit ISO 2022 <ESC> & @ <ESC> $ B 0x3441 0x3B7A <ESC> ( J + - ISO-2022-JP <ESC> $ B 0x3441 0x3B7A <ESC> ( J + - EUC 0xB4C1 0xBBFA + - Shift-JIS 0x8ABF 0x8E9A + +o Simplified Chinese (GB 2312-80 & GB 1988-89 or ASCII): + - 7-bit ISO 2022 <ESC> $ A 0x3A3A 0x5756 <ESC> ( T + - ISO-2022-CN <ESC> $ ) A <SO> 0x3A3A 0x5756 <SI> + - EUC 0xBABA 0xD7D6 + - HZ (HZ-GB-2312) ~{ 0x3A3A 0x5756 ~} + - zW zW 0x3A3A 0x5756 + +o Traditional Chinese (CNS 11643-1992): + - 7-bit ISO 2022 <ESC> $ ( G 0x6947 0x4773 <ESC> ( B + - ISO-2022-CN <ESC> $ ) G <SO> 0x6947 0x4773 <SI> + - EUC 0xE9C7 0xC7F3 or 0x8EA1E9C7 0x8EA1C7F3 + +o Traditional Chinese (Big Five): + - Big Five 0xBA7E 0xA672 + +o Korean (KS C 5601-1992 & ASCII): + - 7-bit ISO 2022 <ESC> $ ( C 0x7953 0x6D2E <ESC> ( B + - ISO-2022-KR <ESC> $ ) C <SO> 0x7953 0x6D2E <SI> + - EUC 0xF9D3 0xEDAE + - Johab 0xF7D3 0xF1AE + +o CJK (ISO 10646-1:1993, JIS X 0221-1995, GB 13000.1-93, or KS C + 5700-1995): + - UCS-2 0x6F22 0x5B57 + - UCS-4 0x00006F22 0x00005B57 + +The above should have given you a taste of what information the +following sections provide. + + +3.1: 7-BIT ISO 2022 ENCODING + + 7-bit ISO 2022 is the name commonly given to the encoding +system that uses escape sequences to shift between character sets. +(ISO 2022 encoded Japanese text is also known as "JIS" encoding, but +is different from ISO-2022-JP and ISO-2022-JP-2, and will be explained +in Section 3.1.3.) This encoding comes from the ISO 2022-1993 +standard. + An escape sequence, as the name implies, consists of an escape +character followed by a sequence of one or more characters. These +escape sequences are used to change character set of the text +stream. This may also mean a shift from one- to two-byte-per-character +mode (or vice versa). + 7-bit ISO 2022 Character sets fall into two types: one-byte +and two-byte. CJK character sets, for obvious reasons, fall into the +latter group. + One advantage that 7-bit ISO 2022 encoding has over other +encoding systems is that its escape sequences specify the character +set, thus specify the locale. 7-bit ISO 2022 encoding also encodes +text using only seven-bit bytes, which has the benefit of being able +to survive Internet travel (e-mail). + + +3.1.1: CODE SPACE + + Each byte in the representation of graphic (printable) +characters fall into the range 0x21 (decimal 33) through 0x7E (decimal +126). For one-byte character sets, this means a maximum of 94 +characters. For two-byte character sets, this means a maximum of 8,836 +characters (94 x 94 = 8,836). + + One-byte Characters Encoding Range + ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + first byte range 0x21-0x7E + + Two-byte Characters Encoding Range + ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + first byte range 0x21-0x7E + second byte range 0x21-0x7E + +White space and control characters (of which the "escape" character is +one) are still found in 0x00-0x20 and 0x7F. + + +3.1.2: ISO-REGISTERED ESCAPE SEQUENCES + + The following is a table that provides the ISO-registered +escape sequences for various one- and two-byte character sets +mentioned in Part 2 of this document (ISO registration numbers +provided in the fourth column): + + One-byte Character Set Escape Sequence Hexadecimal ISO Reg + ^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ ^^^^^^^^^^^ ^^^^^^^ + ASCII (ANSI X3.4-1986) <ESC> ( B 0x1B2842 6 + Half-width katakana <ESC> ( I 0x1B2849 13 + JIS X 0201-1976 Roman <ESC> ( J 0x1B284A 14 + GB 1988-89 Roman <ESC> ( T 0x1B2854 57 + + Two-byte Character Set Escape Sequence Hexadecimal ISO Reg + ^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ ^^^^^^^^^^^ ^^^^^^^ + JIS C 6226-1978 <ESC> $ @ 0x1B2440 42 + GB 2312-80 <ESC> $ A 0x1B2441 58 + JIS X 0208-1983 <ESC> $ B 0x1B2442 87 + KS C 5601-1992 <ESC> $ ( C 0x1B242843 149 + JIS X 0212-1990 <ESC> $ ( D 0x1B242844 159 + ISO-IR-165:1992 <ESC> $ ( E 0x1B242845 165 + JIS X 0208-1990 <ESC> & @ <ESC> $ B 0x1B26401B2442 168 + CNS 11643-1992 Plane 1 <ESC> $ ( G 0x1B242847 171 + CNS 11643-1992 Plane 2 <ESC> $ ( H 0x1B242848 172 + CNS 11643-1992 Plane 3 <ESC> $ ( I 0x1B242849 183 + CNS 11643-1992 Plane 4 <ESC> $ ( J 0x1B24284A 184 + CNS 11643-1992 Plane 5 <ESC> $ ( K 0x1B24284B 185 + CNS 11643-1992 Plane 6 <ESC> $ ( L 0x1B24284C 186 + CNS 11643-1992 Plane 7 <ESC> $ ( M 0x1B24284D 187 + +Note that the first four two-byte character sets do not use an opening +parenthesis (0x28 or "(") in their escape sequences, which means that +they don't follow the 7-bit ISO 2022 rules precisely. They are shorter +for historical reasons, and are retained for backwards compatibility. +Also note that not all of the CJK character set standards described in +Part 2 have ISO-registered escape sequences. + There are other encoding methods that are similar to 7-bit ISO +2022 in that they are suitable for Internet use, but are locale- +specific. These include HZ and zW encoding, both of which are specific +to the GB 2312-80 character set (see Sections 3.3.2 and 3.3.3). +ISO-2022-JP, ISO-2022-KR, ISO-2022-CN, and ISO-2022-CN-EXT are +described below. + + +3.1.3: ISO-2022-JP AND ISO-2022-JP-2 + + ISO-2022-JP is best described as a subset of 7-bit ISO 2022 +encoding for Japanese, and reflects how Japanese text is encoded for +e-mail messages. ISO-2022-JP-2 is an extension that supports +additional character sets. + There are only four escape sequences permitted in ISO-2022-JP, +indicated as follows: + + One-byte Character Set Escape Sequence Hexadecimal + ^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ ^^^^^^^^^^^ + ASCII (ANSI X3.4-1986) <ESC> ( B 0x1B2842 + JIS X 0201-1976 Roman <ESC> ( J 0x1B284A + + Two-byte Character Set Escape Sequence Hexadecimal + ^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ ^^^^^^^^^^^ + JIS C 6226-1978 <ESC> $ @ 0x1B2440 + JIS X 0208-1983 <ESC> $ B 0x1B2442 + +Note the lack of JIS X 0208-1990, JIS X 0212-1990, and half-width +katakana escape sequences. The JIS X 0208-1983 escape sequence is used +to indicate both JIS X 0208-1983 and JIS X 0208-1990 (for practical +reasons). + ISO-2022-JP-2 permits additional escape sequences, indicated +as follows: + + One-byte Character Set Escape Sequence Hexadecimal + ^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ ^^^^^^^^^^^ + ASCII (ANSI X3.4-1986) <ESC> ( B 0x1B2842 + JIS X 0201-1976 Roman <ESC> ( J 0x1B284A + + Two-byte Character Set Escape Sequence Hexadecimal + ^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ ^^^^^^^^^^^ + JIS C 6226-1978 <ESC> $ @ 0x1B2440 + JIS X 0208-1983 <ESC> $ B 0x1B2442 + JIS X 0212-1990 <ESC> $ ( D 0x1B242844 + GB 2312-80 <ESC> $ A 0x1B2441 + KS C 5601-1992 <ESC> $ ( C 0x1B242843 + +With the introduction of ISO-2022-KR (see Section 3.1.4), ISO-2022-CN +(see Section 3.1.5), and ISO-2022-CN-EXT (see Section 3.1.5), the +usefulness of supporting GB 2312-80 and KS C 5601-1992 can be +questioned. However, ISO-2022-JP-2 provides support for JIS X +0212-1990. + More detailed information on ISO-2022-JP encoding can be found +in RFC 1468. And, more detailed information on ISO-2022-JP-2 encoding +can be found in RFC 1554. + + +3.1.4: ISO-2022-KR + + ISO-2022-KR is similar to ISO-2022-JP (see Section 3.1.3) in +that it reflects how Korean text is encoded for e-mail messages. +However, its actual implementation is a bit different. Below is a +summary. + There are only two shift sequences used in ISO-2022-KR, +indicated as follows: + + One-byte Character Set Shift Sequence Hexadecimal + ^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ ^^^^^^^^^^^ + ASCII (ANSI X3.4-1986) <SI> 0x0F + + Two-byte Character Set Shift Sequence Hexadecimal + ^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ ^^^^^^^^^^^ + KS C 5601-1992 <SO> 0x0E + +Furthermore, the following designator sequence must appear only once, +at the beginning of a line, before any KS C 5601-1992 characters (this +usually means that it appears by itself on the first line of the +file): + + <ESC> $ ) C 0x1B242943 + +It almost looks the same as the KS C 5601-1992 escape sequence in +7-bit ISO 2022, but look again. The opening parenthesis (0x28 or "(") +is replaced by a closing parenthesis (0x29 or ")"). This designator +sequence serves a different purpose than an escape sequence. It is +like a flag indicating that "this document contains KS C 5601-1992 +characters." The <SO> and <SI> control characters actually perform the +switching between one- (ASCII) and two-byte (KS C 5601-1992) codes. + More detailed information on ISO-2022-KR encoding can be found +in RFC 1557. + + +3.1.5: ISO-2022-CN AND ISO-2022-CN-EXT + + ISO-2022-CN and ISO-2022-CN-EXT are similar to ISO-2022-JP +(see Section 3.1.3) and ISO-2022-KR (see Section 3.1.4) in that they +reflect how Chinese text is encoded for e-mail messages. + Like with ISO-2022-KR, there are only two shift sequences, +indicated as follows: + + One-byte Character Set Shift Sequence Hexadecimal + ^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ ^^^^^^^^^^^ + ASCII (ANSI X3.4-1986) <SI> 0x0F + + Two-byte Character Set Shift Sequence Hexadecimal + ^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ ^^^^^^^^^^^ + <Too Many to List> <SO> 0x0E + +But, unlike ISO-2022-KR, there are single shift sequences. Single +shift means that they are used before every (single) character, not +before sequences of characters. + + Single Shift Type Shift Sequence Hexadecimal + ^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ ^^^^^^^^^^^ + SS2 <ESC> N 0x1B4E + SS3 <ESC> O (not zero!) 0x1B4F + + ISO-2022-CN supports the following character sets using SO and +SS2 designations: + + Character Set Type Designation Sequence Hexadecimal + ^^^^^^^^^^^^^ ^^^^ ^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^ + GB 2312-80 SO <ESC> $ ) A 0x1B242941 + CNS 11643-1992 Plane 1 SO <ESC> $ ) G 0x1B242947 + CNS 11643-1992 Plane 2 SS2 <ESC> $ * H 0x1B242A48 + +The designator sequences must appear once on a line before any +instance of the character set it designates. If two lines contain +characters from the same character set, both lines must include the +designator sequence (this is so the text can be displayed correctly +when scroll back in a window). This is different behavior from +ISO-2022-KR where the designator sequence appears once in the entire +file (this is because ISO-2022-KR supports a single two-byte character +set). + ISO-2022-CN-EXT supports the following character sets using +SO, SS2, and SS3 designations (notice how ISO-2022-CN is still +supported in the same manner): + + Character Set Type Designation Sequence Hexadecimal + ^^^^^^^^^^^^^ ^^^^ ^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^ + GB 2312-80 SO <ESC> $ ) A 0x1B242941 + GB/T 12345-90 SO NOT REGISTERED + ISO-IR-165 SO <ESC> $ ) E 0x1B242945 + CNS 11643-1992 Plane 1 SO <ESC> $ ) G 0x1B242947 + CNS 11643-1992 Plane 2 SS2 <ESC> $ * H 0x1B242A48 + GB 7589-87 SS2 NOT REGISTERED + GB/T 13131-9X SS2 NOT REGISTERED + CNS 11643-1992 Plane 3 SS3 <ESC> $ + I 0x1B242B49 + CNS 11643-1992 Plane 4 SS3 <ESC> $ + J 0x1B242B4A + CNS 11643-1992 Plane 5 SS3 <ESC> $ + K 0x1B242B4B + CNS 11643-1992 Plane 6 SS3 <ESC> $ + L 0x1B242B4C + CNS 11643-1992 Plane 7 SS3 <ESC> $ + M 0x1B242B4D + GB 7590-87 SS3 NOT REGISTERED + GB/T 13132-9X SS3 NOT REGISTERED + +Support for character sets indicated as NOT REGISTERED will be added +once they are ISO-registered. + More detailed information on ISO-2022-CN and ISO-2022-CN-EXT +encodings can be found in RFC 1922. + + +3.2: EUC ENCODING + + EUC stands for "Extended UNIX Code," and is a rich encoding +system from ISO 2022-1993 that is designed to handle large or multiple +character sets. It is primarily used on UNIX systems, such as Sun's +Solaris. + EUC consists of four codes sets, numbered 0 through 3. The +only code set that is more or less fixed by definition is code set 0, +which is specified to contain ASCII or a locale's equivalent (such as +JIS X 0201-1976 for Japanese or GB 1988-89 for PRC Chinese). + It is quite common to append the locale name to "EUC" when +designating a specific instance of EUC encoding. Common designations +include EUC-JP, EUC-CN, EUC-KR, and EUC-TW. + + +3.2.1: JAPANESE REPRESENTATION + + The following table illustrates the Japanese representation of +EUC packed format: + + EUC Code Sets Encoding Range + ^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + Code set 0 (ASCII or JIS X 0201-1976 Roman): 0x21-0x7E + Code set 1 (JIS X 0208): 0xA1A1-0xFEFE + Code set 2 (half-width katakana): 0x8EA1-0x8EDF + Code set 3 (JIS X 0212-1990): 0x8FA1A1-0x8FFEFE + +An earlier version of EUC for Japanese used code set 3 as the user- +defined range. + + +3.2.2: CHINESE (PRC) REPRESENTATION + + The following table illustrates the Chinese (PRC) +representation of EUC packed format: + + EUC Code Sets Encoding Range + ^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + Code set 0 (ASCII or GB 1988-89): 0x21-0x7E + Code set 1 (GB 2312-80): 0xA1A1-0xFEFE + Code set 2: unused + Code set 3: unused + +Note how code sets 2 and 3 are unused. + The encoding used on Macintosh is quite similar, but has a +shortened two-byte range (0xA1A1 through 0xFCFE) plus additional +one-byte code points, namely 0x80 ("u" with dieresis), 0xFD +("copyright" symbol: "c" in a circle), 0xFE ("trademark" symbol: "TM" +as a superscript), and 0xFF ("ellipsis" symbol: three dots). + + +3.2.3: CHINESE (TAIWAN) REPRESENTATION + + The following table illustrates the Chinese (Taiwan) +representation of EUC packed format: + + EUC Code Sets Encoding Range + ^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + Code set 0 (ASCII): 0x21-0x7E + Code set 1 (CNS 11643-1992 Plane 1): 0xA1A1-0xFEFE + Code set 2 (CNS 11643-1992 Planes 1-16): 0x8EA1A1A1-0x8EB0FEFE + Code set 3: unused + +Note how CNS 11643-1992 Plane 1 is redundantly encoded in code set 1 +(two-byte) and code set 2 (four-byte). The second byte of code set 2 +indicates the plane number. For example, 0xA1 is Plane 1 and so on up +until 0xB0, which is Plane 16. + + +3.2.4: KOREAN REPRESENTATION + + The following table illustrates the Korean representation of +EUC packed format (this is also known as "Wansung" encoding -- the +Korean word "wansung" means "pre-compose"): + + EUC Code Sets Encoding Range + ^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + Code set 0 (ASCII or KS C 5636-1993): 0x21-0x7E + Code set 1 (KS C 5601-1992): 0xA1A1-0xFEFE + Code set 2: unused + Code set 3: unused + +Note how code sets 2 and 3 are unused. + The encoding used on Macintosh is quite similar, but has a +shortened two-byte range (0xA1A1 through 0xFDFE) plus additional +one-byte code points, namely 0x81 ("won" symbol), 0x82 (hyphen), 0x83 +("copyright" symbol: "c" in a circle), 0xFE ("trademark" symbol: "TM" +as a superscript), and 0xFF ("ellipsis" symbol: three dots). + See Section 3.3.17 for a description of Microsoft's extension +to this encoding, called Unified Hangul Code. + + +3.3: LOCALE-SPECIFIC ENCODINGS + + The encoding systems described in the following sections are +considered to be locale-specific, namely that are used to encode a +specific character set standard. This is not to say that they are not +widely used (actually, some of these are among the most widely used +encoding systems!), but rather that they are tied to a specific +character set. + + +3.3.1: SHIFT-JIS + + Shift-JIS (also known as MS Kanji, SJIS, or DBCS-PC) is the +encoding system used on machines that support MS-DOS or Windows, and +also for Macintosh (KanjiTalk or Japanese Language Kit). It was +originally developed by Microsoft Corporation as a way to support the +Japanese character set on MS-DOS. The following tables provide the +Shift-JIS encoding ranges: + + Two-byte Standard Characters Encoding Ranges + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ + first byte ranges 0x81-0x9F, 0xE0-0xEF + second byte ranges 0x40-0x7E, 0x80-0xFC + + Two-byte User-defined Characters Encoding Ranges + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ + first byte range 0xF0-0xFC + second byte ranges 0x40-0x7E, 0x80-0xFC + + One-byte Characters Encoding Range + ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + Half-width katakana 0xA1-0xDF + ASCII/JIS-Roman 0x21-0x7E + +It is important to note that the user-defined range does not +correspond to code points in other encodings that support Japanese, +such as 7-bit ISO 2022 or EUC. This is a portability problem. It is +also unique in that it does not support the JIS X 0212-1990 character +set standard. + The encoding used on Macintosh is quite similar to the above +table, but has additional one-byte code points, namely 0x80 +(backslash), 0xFD ("copyright" symbol: "c" in a circle), 0xFE +("trademark" symbol: "TM" as a superscript), and 0xFF ("ellipsis" +symbol: three dots). + + +3.3.2: HZ (HZ-GB-2312) + + HZ is a simple yet very powerful and reliable system for +encoding GB 2312-80 text which was developed by Fung Fung Lee +(lee@umunhum.stanford.edu). HZ encoding is commonly used when +exchanging e-mail or posting messages to Usenet News (specifically, to +alt.chinese.text). + The actual encoding ranges used for one- and two-byte +characters is almost identical to 7-bit ISO 2022 encoding (see Section +3.1.1). The first-byte range is limited to 0x21 through 0x77. But, +instead of using an escape sequence to shift between one- and two-byte +character modes, a simple string of two printable characters is used. + + One-byte Character Set Shift Sequence Hexadecimal + ^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ ^^^^^^^^^^^ + ASCII ~} 0x7E7D + + Two-byte Character Set Shift Sequence Hexadecimal + ^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ ^^^^^^^^^^^ + GB 2312-80 ~{ 0x7E7B + +The tilde character (0x7E) is interpreted as an escape character in HZ +encoding, so it has special meaning. If a tilde character is to appear +in one-byte-per-character mode, it must be doubled (so ~~ would appear +as just ~). This means that there are three escape sequences used in +HZ encoding: + + Escape Sequence Meaning + ^^^^^^^^^^^^^^^ ^^^^^^^ + ~~ ~ in one-byte-per-character mode + ~} Shift into one-byte-per-character mode + ~{ Shift into two-byte-per-character mode + +There is also a fourth escape sequence, namely ~ plus a newline +character (~\n). This escape sequence is a line-continuation marker to +be consumed with no output produced. + This method works without problems because the shift sequences +represent empty positions in the very last row of the GB 2312-80 table +(actually, the second- and third-from-last code points). HZ encoding +makes 77 of the 94 rows accessible, and because there are no defined +characters beyond row 77, this causes no problems. + The complete HZ specification is part of the HZ package, +described in RFC 1843, and available in HTML format. These are +available at the following URLs: + + ftp://ftp.ifcss.org/pub/software/unix/convert/HZ-2.0.tar.gz + ftp://ftp.ora.com/pub/examples/nutshell/ujip/Ch9/rfc-1843.txt + http://umunhum.stanford.edu/~lee/chicomp/HZ_spec.html + +In addition, RFC 1842 establishes "HZ-GB-2312" as the "charset" +parameter in MIME-encoded e-mail headers. Its properties are identical +to HZ encoding as described in RFC 1843. + + +3.3.3: zW + + zW encoding, developed by Ya-Gui Wei and Edmund Lai, is older +than and somewhat similar to HZ encoding (HZ is considered to be a +better encoding system, and users are encouraged to switch over to HZ +encoding). + zW encoding is named by how it encodes each line of GB 2312-80 +text, namely lines that contain Chinese text must begin with the two +characters "z" and "W" ("zW"). This encoding method does not permit +the mixture of one- (ASCII) and two-byte (GB 2312-80) characters on a +per-character basis, but rather on a per-line basis. That is, each +line can contain only Chinese or ASCII text, but not both. + More information on zW encoding can be found as part of the +ZWDOS package available at the following URL: + + ftp://ftp.ifcss.org/pub/software/dos/ZWDOS/ + + +3.3.4: BIG FIVE + + Big Five is the encoding system used on machines that support +MS-DOS or Windows, and also for Macintosh (such as the Chinese +Language Kit or the fully-localized operating system). + + Two-byte Standard Characters Encoding Ranges + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ + first byte range 0xA1-0xFE + second byte ranges 0x40-0x7E, 0xA1-0xFE + + One-byte Characters Encoding Range + ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + ASCII 0x21-0x7E + + The encoding used on Macintosh is quite similar to the above, +but has a slightly shortened two-byte range (second byte range up to +0xFC only) plus additional one-byte code points, namely 0x80 +(backslash), 0xFD ("copyright" symbol: "c" in a circle), 0xFE +("trademark" symbol: "TM" as a superscript), and 0xFF ("ellipsis" +symbol: three dots). + + +3.3.5: JOHAB + + Korean hangul characters are typically encoded in what is +known as pre-combined form, namely 2 or 3 hangul elements bound into a +single character. KS C 5601-1992 enumerates 2,350 such pre-combined +forms. While this number is felt to be sufficient for most purposes, +it does not account for the total number of possible permutations. The +encoding system that encodes all possible pre-combined hangul is known +as Johab encoding (also known as "two-byte combination code" -- the +Korean word "johab" means "combine"), and is described in Annex 3 of +the KS C 5601-1992 standard. This encoding is almost like encoding all +possible three-letter words in English -- while all combinations are +possible, only a fraction represent *real* words. + Pre-combined hangul can be composed of 19 different initial, +21 different medial, and 27 different final hangul elements (28, +actually, if you count the placeholder). This provides a maximum of +11,172 pre-combined hangul. Of these 67 hangul elements, 51 are unique +(some can occur in different positions). Each of these positions are +encoded using five bits each (five bits can encode up to 32 unique +objects). The encoding array looks as follows: + +o Bit 1: always on +o Bits 2-6: initial hangul element +o Bits 7-11: medial hangul element +o Bits 12-16: final hangul element + +Initial and final elements are consonants, and the medial elements are +vowels. This encoding must be treated as a 16-bite entity because the +bit array of the medial hangul element spans the first and second byte. + Johab encoding also provides the complete set of KS C 5601- +1992 symbols and hanja, but in different code points. Annex 3 of the +KS C 5601-1992 manual (pp 33-34) contains a complete symbol and hanja +mapping table between EUC and Johab code points. (The KS C 5601-1989 +manual did not have this.) The code space ranges for Johab encoding +are as follows: + + One-byte Characters Encoding Range + ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + ASCII or KS C 5636-1993 0x21-0x7E + + Two-byte Pre-combined Hangul Encoding Ranges + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ + first byte range 0x84-0xD3 + second byte ranges 0x41-0x7E, 0x81-0xFE + + Two-byte Symbols and Hanja Encoding Ranges + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ + first byte ranges 0xD8-0xDE, 0xE0-0xF9 + second byte ranges 0x31-0x7E, 0x91-0xFE + +Note that the second byte ranges encode a total of 188 characters, and +that the second byte ranges for hangul and symbols/hanja are slightly +different (yet the same size, namely 188 characters). + Here is a summary of the above table, which better describes +what is encoded where. Rows 0x84 through 0xD3 provide 80 rows of 188 +characters each (15,040 code points, which is more than enough for the +11,172 pre-combined hangul). Row 0xD8 provides 188 user-defined +positions, the same as Rows 41 and 94 in the standard KS C 5601-1992 +table. Rows 0xD9 through 0xDE encode Rows 1 through 12 of the standard +KS C 5601-1992 table (symbols). Rows 0xE0 through 0xF9 encode Rows 42 +through 94 of the KS C 5601-1992 table (hanja). The following URL +provides a complete mapping table for the KS C 5601-1992 symbols and +hanja: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/map/non-hangul-codes.txt + +The following URLs provides similar information (they are the same +file), but only for the 11,172 pre-combined hangul: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/map/hangul-codes.txt + ftp://unicode.org/pub/MappingTables/EastAsiaMaps/hangul-codes.txt + + Of further interest may be that Microsoft designates Johab +encoding as its Code Page 1361. Microsoft if planning to support Johab +encoding for Korean Windows NT. + + +3.3.6: N-BYTE HANGUL + + In the days before full two-byte capable operating systems, +each of the 51 basic hangul elements were encoding using a single +(7-bit) byte. The encoding range spans 0x40 through 0x7C, but there +are several unassigned gaps. This is known as the "N-byte Hangul" +code, and is described in Annex 4 (page 35) of the KS C 5601-1992 +manual. + The following table illustrates these 51 one-byte code points +(the pronunciation or meaning of the hangul element is provided in +parentheses) and how they map to the three 5-bit arrays in Johab +encoding (expressed as binary patterns): + + Element Initial Medial Final + ^^^^^^^ ^^^^^^^ ^^^^^^ ^^^^^ + 0x40 ("fill") 00001 00010 00001 + 0x41 (g) 00010 ***** 00010 + 0x42 (gg) 00011 ***** 00011 + 0x43 (gs) ***** ***** 00100 + 0x44 (n) 00100 ***** 00101 + 0x45 (nj) ***** ***** 00110 + 0x46 (nh) ***** ***** 00111 + 0x47 (d) 00101 ***** 01000 + 0x48 (dd) 00110 ***** ***** + 0x49 (r) 00111 ***** 01001 + 0x4A (rg) ***** ***** 01010 + 0x4B (rm) ***** ***** 01011 + 0x4C (rb) ***** ***** 01100 + 0x4D (rs) ***** ***** 01101 + 0x4E (rt) ***** ***** 01110 + 0x4F (rp) ***** ***** 01111 + 0x50 (rh) ***** ***** 10000 + 0x51 (m) 01000 ***** 10001 + 0x52 (b) 01001 ***** 10011 + 0x53 (bb) 01010 ***** ***** + 0x54 (bs) ***** ***** 10100 + 0x55 (s) 01011 ***** 10101 + 0x56 (ss) 01100 ***** 10110 + 0x57 (ng) 01101 ***** 10111 + 0x58 (j) 01110 ***** 11000 + 0x59 (jj) 01111 ***** ***** + 0x5A (c) 10000 ***** 11001 + 0x5B (k) 10001 ***** 11010 + 0x5C (t) 10010 ***** 11011 + 0x5D (p) 10011 ***** 11100 + 0x5E (h) 10100 ***** 11101 + 0x5F UNASSIGNED + 0x60 UNASSIGNED + 0x61 UNASSIGNED + 0x62 (a) ***** 00011 ***** + 0x63 (ae) ***** 00100 ***** + 0x64 (ya) ***** 00101 ***** + 0x65 (yae) ***** 00110 ***** + 0x66 (eo) ***** 00111 ***** + 0x67 (e) ***** 01010 ***** + 0x68 UNASSIGNED + 0x69 UNASSIGNED + 0x6A (yeo) ***** 01011 ***** + 0x6B (ye) ***** 01100 ***** + 0x6C (o) ***** 01101 ***** + 0x6D (wa) ***** 01110 ***** + 0x6E (wae) ***** 01111 ***** + 0x6F (oe) ***** 10010 ***** + 0x70 UNASSIGNED + 0x71 UNASSIGNED + 0x72 (yo) ***** 10011 ***** + 0x73 (u) ***** 10100 ***** + 0x74 (weo) ***** 10101 ***** + 0x75 (we) ***** 10110 ***** + 0x76 (wi) ***** 10111 ***** + 0x77 (yu) ***** 11010 ***** + 0x78 UNASSIGNED + 0x79 UNASSIGNED + 0x7A (eu) ***** 11011 ***** + 0x7B (yi) ***** 11100 ***** + 0x7C (i) ***** 11101 ***** + + There are utilities to convert N-byte Hangul code to other, +more widely-used, encoding methods. Pointers to these and other code +conversion utilities can be found in Section 4.7. + + +3.3.7: UCS-2 + + UCS-2 (Universal Character Set containing 2 bytes) encoding is +one way to encode ISO 10646-1:1993 text, and is considered identical +to Unicode encoding. Its encoding range, which is quite simple, is as +follows: + + ISO 10646-1:1993 Characters Encoding Range + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + first byte range 0x00-0xFF + second byte range 0x00-0xFF + +Yes, folks, the whole range of 65,536 possible code points are +available for encoding characters. The "signature" that indicates a +file using UCS-2 is as follows: + + 0xFEFF + + Escape sequences for UCS-2 have already been registered with +ISO, and are as follows: + + ISO 10646-1:1993 Escape Sequence Hexadecimal ISO Reg + ^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ ^^^^^^^^^^^ ^^^^^^^ + UCS-2 Level 1 <ESC> % / @ 0x1B252F40 162 + UCS-2 Level 2 <ESC> % / C 0x1B252F43 174 + UCS-2 Level 3 <ESC> % / E 0x1B252F45 176 + +So what do these three levels mean? Level 3 means all characters in +ISO 10646-1:1993 with no restrictions (0x0000 through 0xFFFF). + Level 2 begins to restrict the character set by not including +the following characters or character ranges: + + 0x0300-0x0345 0x09D7 0x0BD7 0x11A8-0x11F9 + 0x0360-0x0361 0x0A3C 0x0C55-0x0C56 0x20D0-0x20E1 + 0x0483-0x0486 0x0A70-0x0A71 0x0CD5-0x0CD6 0x302A-0x302F + 0x093C 0x0ABC 0x0D57 0x3099-0x309A + 0x0953-0x0954 0x0B3C 0x1100-0x1159 0xFE20-0xFE23 + 0x09BC 0x0B56-0x0B57 0x115F-0x11A2 + +These are all combining characters, and represent 364 code points. + Level 1 further restricts the character set by not including +the following characters or character ranges: + + 0x05B0-0x05B9 0x09BE-0x09C4 0x0B47-0x0B48 0x0D02-0x0D03 + 0x05BB-0x05BD 0x09C7-0x09C8 0x0B4B-0x0B4D 0x0D3E-0x0D43 + 0x05BF 0x09CB-0x09CD 0x0B82-0x0B83 0x0D46-0x0D48 + 0x05C1-0x05C2 0x09E2-0x09E3 0x0BBE-0x0BC2 0x0D4A-0x0D4D + 0x064B-0x0652 0x0A02 0x0BC6-0x0BC8 0x0E31 + 0x0670 0x0A3E-0x0A42 0x0BCA-0x0BCD 0x0E34-0x0E3A + 0x06D6-0x06E4 0x0A47-0x0A48 0x0C01-0x0C03 0x0E47-0x0E4E + 0x06E7-0x06E8 0x0A4B-0x0A4D 0x0C3E-0x0C44 0x0EB1 + 0x06EA-0x06ED 0x0A81-0x0A83 0x0C46-0x0C48 0x0EB4-0x0EB9 + 0x0901-0x0903 0x0ABE-0x0AC5 0x0C4A-0x0C4D 0x0EBB-0x0EBC + 0x093E-0x094D 0x0AC7-0x0AC9 0x0C82-0x0C83 0x0EC8-0x0ECD + 0x0951-0x0952 0x0ACB-0x0ACD 0x0CBE-0x0CC4 0xFB1E + 0x0962-0x0963 0x0B01-0x0B03 0x0CC6-0x0CC8 + 0x0981-0x0983 0x0B3E-0x0B43 0x0CCA-0x0CCD + +These, too, are all combining characters, and represent 586 code +points (222 above plus the 364 characters from the Level 2 +restriction). + + +3.3.8: UCS-4 + + UCS-4 (Universal Character Set containing 4 bytes) encoding is +another way to encode ISO 10646-1:1993 text, and is used for future +expansion of the character set. Its encoding range is as follows: + + ISO 10646-1:1993 Characters Encoding Range + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + first byte range 0x00-0x7F + second byte range 0x00-0xFF + third byte range 0x00-0xFF + fourth byte range 0x00-0xFF + +Note that the first byte range only goes up to 0x7F. This means that +UCS-4 is a 31-bit encoding. And, in case you're wondering, 31 bits +provide 2,147,483,648 code points. The "signature" that indicates a +file using UCS-4 is as follows: + + 0x0000 0xFEFF + + Escape sequences for UCS-4 have already been registered with +ISO, and are as follows: + + ISO 10646-1:1993 Escape Sequence Hexadecimal ISO Reg + ^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ ^^^^^^^^^^^ ^^^^^^^ + UCS-4 Level 1 <ESC> % / A 0x1B252F41 163 + UCS-4 Level 2 <ESC> % / D 0x1B252F44 175 + UCS-4 Level 3 <ESC> % / F 0x1B252F46 177 + +See the end of Section 3.3.7 for a description of these three levels. +But, in the case of UCS-4, simply prepend "0000" to all the values. + + +3.3.9: UTF-7 + + It turns out that *raw* ISO 10646-1:1993 encoding (that is, +UCS-2 or UCS-4) can cause problems because null bytes (0x00) are +possible (and frequent). Several UTFs (UCS Transformation Formats) +have been developed to deal with this and other problems. I must admit +that I don't know too much about UTFs, and what I provide below is +minimal, but does include pointers to more complete descriptions. + UTF-7 is a mail-safe 7-bit transformation format for UCS-2 +(including UTF-16). It uses straight ASCII for many ASCII characters, +and switches into a Base64 encoding of UCS-2 or UTF-16 for everything +else. It was designed to be usable in MIME-compliant e-mail headers as +well as message bodies, and to pass through gateways to non-ASCII mail +systems (like Bitnet). More detailed information on UTF-7 can be found +in RFC 1642, and a UTF-7 converter is available. The following URLs +provide this information: + + http://www.stonehand.com/unicode/standard/utf7.html + ftp://unicode.org/pub/Programs/ConvertUTF/ + + +3.3.10: UTF-8 + + UTF-8 (also known as UTF-2 or FSS-UTF -- FSS stands for "file +system safe") can represent any character in UCS-2 and UCS-4, and is +officially an annex to ISO 10646-1:1993. It is different from UTF-7 in +that it encodes character sets into 8-bit bytes. UCS-2 and UCS-4 have +problems with some file systems and utilities, so this UTF was +developed. + More detailed information on UTF-8 and its relationship with +ISO 10646-1:1993 can be found at the following URLs: + + http://www.stonehand.com/unicode/standard/utf8.html + ftp://unicode.org/pub/Programs/ConvertUTF/ + + X/Open Company Limited also published a document that +describes UTF-8 in detail (they call it FSS-UTF), and you can find +information about it at the following URL: + + http://www.xopen.co.uk/public/pubs/catalog/c501.htm + +The new programming language called Java supports Unicode through +UTF-8. More information on Java is at the following URL: + + http://www.javasoft.com/ + + +3.3.11: UTF-16 + + UTF-16 (formerly UCS-2E), like UTF-8, is now officially an +annex to ISO 10646-1:1993. From what I've read, UTF-16 transforms +UCS-4 into a 16-bit form. UTF-16 can then be further encoded in UTF-7 +or UTF-8 (but doing this is not according to the standard -- there is +little to gain by doing so). + More detailed information on UTF-16 and its relationship with +ISO 10646-1:1993 can be found at the following URLs: + + http://www.stonehand.com/unicode/standard/utf16.html + ftp://unicode.org/pub/Programs/ConvertUTF/ + + +3.3.12: ANSI Z39.64-1989 + + The encoding used for ANSI Z39.64-1989 (and CCCII) is three- +byte 7-bit ISO 2022, namely the following code space: + + Three-byte ANSI Z39.64-1989 Encoding Range + ^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + first byte range 0x21-0x7E + second byte range 0x21-0x7E + third byte range 0x21-0x7E + + +3.3.13: BASE64 + + Base64 encoding is mentioned here only because of its common +usage in e-mail headers, and relationship with MIME (Multi-purpose +Internet Mail Extensions). It is also a source of confusion. Base64 is +a method of encoding arbitrary bytes into the safest 64-character +ASCII subset, and is defined in RFC 1341 (which adapted it from RFC +1113). RFC 1341 was made obsolete by RFC 1521. RFC 1522 also provides +useful information, particularly for handling non-ASCII text, and +obsoletes RFC 1342. + Here is how it works. Every three bytes are encoded as a +four-byte sequence. That is, the 24 bits that make up the three bytes +are split into four 6-bit segments (6 bits can encode up to 64 +characters). Each 6-bit segment is then converted into a character in +the Base64 Alphabet (see below). There is a 65th character, "=", which +has a special purpose (it functions as a "pad" if a full three-byte +sequence is not found). This all may sound a bit like uuencoding, but +it is different. The Base64 Alphabet is as follows: + + ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789+/ + + My name, written in Japanese kanji, is as follows when it is +EUC-encoded (six bytes, expressed as three groups of hexadecimal +values, one group for each character): + + 0xBEAE 0xCED3 0xB7F5 + +When these three EUC-encoded characters are converted to Base64 +encoding, they appear as follows (eight bytes): + + vq7O07f1 + + Base64 encoding is most commonly used for encoding non-ASCII +text that appears in e-mail headers. Of all the portions of an e-mail +message, its header gets manipulated the most during transmission, and +Base64 encoding offers a safe way to further encode non-ASCII text so +that it is not altered by mail-routing software. This is where Base64 +encoding can cause confusion. For example, what goes through your mind +when you see the following chunk o' text? + + From: lunde@adobe.com (=?ISO-2022-JP?B?vq7O07f1?=) + +Many folks think that they are seeing ISO-2022-JP encoding. Not +true. The "ISO-2022-JP" portion is just a flag that indicates the +original encoding before Base64 encoding was applied. The actual +Base64-encoded portion is enclosed between question marks (?) as +follows: + + From: lunde@adobe.com (=?ISO-2022-JP?B?vq7O07f1?=) + >^^^^^^^^< + +The whole string enclosed in parentheses has several components, and +the following explains their purpose and relationships (using the +above string as an example): + + Component Explanation + ^^^^^^^^^ ^^^^^^^^^^^ + =? Signals start of encoded string + ISO-2022-JP Charset name ("ISO-2022-JP" is for Japanese) + ? Delimiter + B Encoding ("B" is for Base64) + ? Delimiter + vq7O07f1 Example string of type "charset" encoded by "encoding" + ?= Signals end of encoded string + + One typically does not need to worry about encoding text as +Base64 (MIME-compliant mailing software usually performs this task for +you). The problem is usually trying to decode Base64-encoded text. A +Base64 decoder is available in Perl at the following URL: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/perl/b64decode.pl + +Note that this program takes "raw" Base64 data as input. Any non- +Base64 stuff must be stripped. I usually run this from within Mule +("C-u M-| b64decode.pl") after defining a region around the Base64- +encoded material. I hope to replace this program soon with one that +automatically recognizes the Base64-encoded portions. + Most MIME-compliant e-mail software can decode Base64-encoded +text. + + +3.3.14: IBM DBCS-HOST + + The oldest two-byte encoding system is IBM's DBCS-Host. DBCS +stands for Double-Byte Character Set. DBCS-Host is still in use on +IBM's mainframe computer systems (hence the use of "Host"). + DBCS-Host encoding is EBCDIC-based, and uses Shift characters, +0x0E and 0x0F, to switch between one- and two-byte mode. Its encoding +specifications are as follows: + + Two-byte Characters Encoding Range + ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + first byte range 0x41-0xFE + second byte range 0x41-0xFE + + Two-byte "Space" Character Code Point + ^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^ + first- and second byte 0x4040 + + One-byte Characters Encoding Range + ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + EBCDIC 0x41-0xF9 + + Shifting Characters Code Point + ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + Two-byte 0x0E + One-byte 0x0F + +This same encoding specification is shared by all of IBM's CJK +character sets, namely for Japanese, Simplified Chinese, Traditional +Chinese, and Korean. + + +3.3.15: IBM DBCS-PC + + IBM's DBCS-PC encoding is used on IBM personal computers (that +is where the "PC" comes from). DBCS-PC encoding is ASCII-based, and +uses the values of characters' bytes themselves to switch between one- +and two-byte mode. Its encoding specifications are as follows: + + Two-byte Characters Encoding Ranges + ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ + first byte range 0x81-0xFE + second byte range 0x40-0x7E, 0x80-0xFE + + One-byte Characters Encoding Range + ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + ASCII 0x21-0x7E + +This same encoding specification is shared by all of IBM's CJK +character sets, namely for Japanese, Simplified Chinese, Traditional +Chinese, and Korean. + DBCS-PC encoding for Japanese, although conforming to the +above encoding specifications, actually uses the same encoding +specifications for Shift-JIS, to include the full user-defined range +(see Section 3.3.1 for more details on Shift-JIS encoding). One big +accommodation is the half-width katakana range, namely 0xA1 through +0xDF. Further, the DBCS-PC code space that is outside the Shift-JIS +specification is unused. + DBCS-PC encoding for Korean uses the equivalent of EUC code +set 1 code points (0xA1A1 through 0xFEFE) for those characters that +are common with KS C 5601-1992. Those characters that are not common +with KS C 5601-1992, namely IBM's extensions, are within the DBCS-PC +encoding space, but outside EUC encoding space (0x9A through 0xA0). +Many hanja and pre-combined hangul are part of IBM's Korean extension. + Note that DBCS-PC is sort of useless without a corresponding +SBCS (Single-Byte Character Set) for the one-byte range. Mixing DBCS +and SBCS results in a MBCS (Multiple-Byte Character Set). How these +are mixed to form MBCSs is detailed in Section 3.4. + + +3.3.16: IBM DBCS-/TBCS-EUC + + IBM has also developed DBCS-EUC and TBCS-EUC encodings. TBCS +stands for Triple-Byte Character Set. These essentially follow the EUC +encoding specifications, and were developed for use with IBM's AIX +(Advanced Interactive Executive) operating system, which is +UNIX-based. + Refer to Section 3.2 for all the details on EUC encoding. + + +3.3.17: UNIFIED HANGUL CODE + + Microsoft has developed what is called "Unified Hangul Code" +(UHC) for its Windows 95 operating system (this was also known as +"Extended Wansung"). It is the optional, not standard, character set +of Win95K. + UHC provides full compatibility with KS C 5601-1992 EUC +encoding (see Section 3.2.4), but adds additional encoding ranges for +holding additional pre-combined hangul (more precisely, the 8,822 that +are needed to fully support the Johab character set). The following is +a table that provides the encoding ranges for UHC encoding: + + Two-byte Standard Characters Encoding Ranges + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ + first byte range 0x81-0xFE + second byte ranges 0x41-0x5A, 0x61-0x7A, + and 0x81-0xFE + + One-byte Characters Encoding Range + ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + ASCII 0x21-0x7E + +Note that 0xA1A1 through 0xFEFE in the above encoding is still +identical, in terms of character-to-code allocation, with KS C 5601- +1992 in EUC encoding. + Appendix G (pp 345-406) of "Developing International Software +for Windows 95 and Windows NT" by Nadine Kano illustrates the KS C +5601-1992 character set standard plus these Microsoft extensions +(8,822 pre-combined hangul) by UHC code (Microsoft calls this Code +Page 949). + + +3.3.18: TRON CODE + + TRON (The Real-time Operating system Nucleus) is an OS +developed in Japan some time ago. Personal Media Corporation has done +work to develop BTRON (Business TRON), which is unique in that it is +the only commercially-available OS that supports JIS X 0212-1990. + TRON Code provides a one- and two-byte encoding space and a +method for switching between them. + The following is how the two-byte space in TRON Code is +allocated: + + A-Zone (8,836 characters; JIS X 0208-1990) Encoding Range + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + first byte range 0x21-0x7E + second byte range 0x21-0x7E + + B-Zone (11,844 characters; JIS X 0212-1990) Encoding Range + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + first byte range 0x80-0xFD + second byte range 0x21-0x7E + + C-Zone (11,844 characters; unassigned) Encoding Range + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + first byte range 0x21-0x7E + second byte range 0x80-0xFD + + D-Zone (15,876 characters; unassigned) Encoding Range + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + first byte range 0x80-0xFD + second byte range 0x80-0xFD + +Note how the B-Zone is larger that the conventional 94-by-94 +matrix. In fact, the JIS X 0212-1990 portion of the B-Zone is +restricted to 0xA121-0xFD7E (93-by-94 matrix -- 0xFE as a first-byte +value is unavailable, and you will see why in a minute). + TRON Code implements "language specifying codes" consisting of +two bytes as follows: + + Two-byte Japanese 0xFE21 + One-byte English 0xFE80 + +0xFE21 in a one-byte stream invokes two-byte Japanese mode, and 0xFE80 +in a two-byte stream invokes one-byte English mode. + The following is the one-byte encoding range for TRON Code: + + One-byte Characters 0x21-0x7E and 0x80-0xFD + +Control codes are in 0x00-0x20 and 0x7F (the usual ASCII control code +range). Also, 0xA0 is reserved as a fixed-width space character. + + +3.3.19: GBK + + GBK is an extension to GB 2312-80 that adds all ISO 10646- +1:1993 (GB 13000.1-93) hanzi not already in GB 2312-80. GBK is defined +as a normative annex of GB 13000.1-93 (see Section 2.2.10). The "K" in +"GBK" is the first sound in the Chinese word meaning "extension" (read +"Kuo Zhan"). + GBK is divided into five levels as follows: + + Level Encoded Range Total Code Points Total Encoded Characters + ^^^^^ ^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^ + GBK/1 0xA1A1-0xA9FE 846 717 + GBK/2 0xB0A1-0xF7FE 6,768 6,763 + GBK/3 0x8140-0xA0FE 6,080 6,080 + GBK/4 0xAA40-0xFEA0 8,160 8,160 + GBK/5 0xA840-0xA9A0 192 166 + + There are also 1,894 user-defined code points as follows: + + Encoded Range Total Code Points + ^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^^^ + 0xAAA1-0xAFFE 564 + 0xF8A1-0xFEFE 658 + 0xA140-0xA7A0 672 + + GBK thus provides a total of 23,940 code points, 21,886 of +which are assigned. + Each "row" in the GBK code table consists of 190 characters. +The following describes the encoding ranges of GBK in detail: + + Two-byte Standard Characters Encoding Ranges + ^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ + first byte range 0x81-0xFE + second byte ranges 0x40-0x7E and 0x80-0xFE + + One-byte Characters Encoding Range + ^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^ + ASCII 0x21-0x7E + +Note that the sub-range 0xA1A1-0xFEFE in the above encoding is still +identical, in terms of character-to-code allocation, with GB 2312-80 +in EUC encoding. GBK is therefore backward-compatible with GB 2312-80 +and forward-compatible with ISO 10646-1:1993. + GBK is the standard character set and encoding for the +Simplified Chinese version of Windows 95. + + +3.4: CJK CODE PAGES + + Many times one reads about references to "Code Pages" in +material about CJK (and other) character sets and encodings. These are +not literal pages, but rather references to a character set and +encoding combination. In the case of CJK Code Pages, they definitely +comprise more than one page! + Microsoft refers to its supported CJK character sets and +encodings through such Code Page designations. The following is a +listing of several Microsoft CJK Code Pages along with their +characteristics: + + Code Page Characteristics + ^^^^^^^^^ ^^^^^^^^^^^^^^^ + 932 JIS X 0208-1990 base, Shift-JIS encoding, Microsoft + extensions (NEC Row 13 and IBM select characters in + redundantly encoded in Rows 89 through 92 and Rows 115 + through 119) + 936 GB 2312-80 base, EUC encoding + 949 KS C 5601-1992 base, Unified Hangul Code encoding, + remaining 8,822 pre-combined hangul as extension (all of + this is referred to as Unified Hangul Code) + 950 Big Five base, Big Five encoding, Microsoft extensions + (actually, the ETen extensions of Row 89) + 1361 Johab base, Johab encoding + + IBM also uses Code Page designations, and, in fact, some +designations (and associated characteristics) are nearly identical to +those in the above table, most notably, Code Pages 932 and 936. IBM's +Code Page 932 does not include NEC Row 13 or IBM select characters in +Rows 89 through 92. + The best way to describe IBM Code Page designations is by +first listing the SBCS (Single-Byte Character Set) and DBCS (Double- +Byte Character Set) Code Page designations (those designated by "Host" +use EBCDIC-based encodings): + + IBM SBCS Code Page Characteristics + ^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ + 37 (US) SBCS-Host + 290 (Japanese) SBCS-Host + 833 (Korean) SBCS-Host + 836 (Simplified Chinese) SBCS-Host + 891 (Korean) SBCS-PC + 897 (Japanese) SBCS-PC + 903 (Simplified Chinese) SBCS-PC + 904 (Traditional Chinese) SBCS-PC + + IBM DBCS Code Page Characteristics + ^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ + 300 (Japanese) DBCS-Host + 301 (Japanese) DBCS-PC + 834 (Korean) DBCS-Host + 835 (Traditional Chinese) DBCS-Host + 837 (Simplified Chinese) DBCS-Host + 926 (Korean) DBCS-PC + 927 (Traditional Chinese) DBCS-PC + 928 (Simplified Chinese) DBCS-PC + +So far there appears to be no relationship with Microsoft's CJK Code +Pages, but when we combine the above SBCS and DBCS Code Pages into +MBCS (Multiple-Byte Character Set) Code Pages, things become a bit +more revealing: + + IBM MBCS Code Page Characteristics + ^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^^^^ + 930 (Japanese) MBCS-Host (Code Pages 300 and 290) + 932 (Japanese) MBCS-PC (Code Pages 301 and 897) + 933 (Korean) MBCS-Host (Code Pages 834 and 833) + 934 (Korean) MBCS-PC (Code Pages 926 and 891) + 938 (Traditional Chinese) MBCS-PC (Code Pages 927 and 904) + 936 (Simplified Chinese) MBCS-PC (Code Pages 928 and 903) + 5031 (Simplified Chinese) MBCS-Host (Code Pages 837 and 836) + 5033 (Traditional Chinese) MBCS-Host (Code Pages 835 and 37) + +So, you can now see that many of Microsoft's CJK Code Pages are +derived from those established by IBM. + More detailed information on the encoding specifications for +DBCS-Host and DBCS-PC can be found in Sections 3.3.14 and 3.3.15, +respectively. + + +PART 4: CJK CHARACTER SET COMPATIBILITY ISSUES + + The sections below provide detailed information about +compatibility issues between CJK character sets, to include tidbits of +useful information. + One thing to mention first is that conversion to and from +IBM's DBCS-Host (Section 3.3.14) and DBCS-PC (Section 3.3.15) +encodings is table-driven, and fully documented in the following IBM +publication: + +o IBM Corporation. "Character Data Representation Architecture - Level + 2, Registry." 1993. IBM order number SC09-1391-01. + +Unfortunately, the CJK-related tables are not supplied in machine- +readable format, and must be obtained from IBM directly. The only real +compatibility issue is trying to obtain the conversion tables from +IBM. + + +4.1: JAPANESE + + In general, when a Japanese character set was revised, +characters were simply added (usually appended at the end). However, +when JIS C 6226-1978 was revised in 1983 (to become JIS X 0208-1983), +a bit more happened (this is still a controversy). + A detailed treatment of the two main transitions, JIS C 6226- +1978 to JIS X 0208-1983 and JIS X 0208-1983 to JIS X 0208-1990, is +covered in Appendix J of UJIP. I provide machine-readable files that +detail these transitions at the following URL: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/AppJ/ + + An interesting side note here is that there is a reason why +there are many lists that illustrate JIS C 6226-1978 and JIS X 0208- +1983 kanji form differences. While most share the same basic set of +changes, there are some inconsistencies. Well, it turns out that JIS C +6226-1978 had ten printings, and not all of them shared the same kanji +forms. If comparisons between JIS C 6226-1978 and JIS X 0208-1983 were +made using different printings of the JIS C 6226-1978 manual, the +results can differ slightly. + There are also interesting correspondences between JIS X +0208-1990 and JIS X 0212-1990. 28 kanji that vanished during the JIS C +6226-1978 to JIS X 0208-1983 transition (they were replaced by +simplified versions) were restored in JIS X 0212-1990 (at totally +different code points). Appendix J of UJIP discusses this, and a file +at the following URL details the 28 mappings: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/AppJ/TJ2.jis + + +4.2: CHINESE (PRC) + + The basic PRC standard, GB 2312-80, has been revised, but not +through a later version of the standard. Instead, the revisions were +carried out in the form of three other documents. Specifically, they +are (in order of publication): + +o GB 6345.1-86 (see Section 2.2.3) +o GB 8565.2-88 (see Section 2.2.6) +o GB/T 12345-90 (see Section 2.2.7) + +Unless you are aware of these documents, figuring out what has been +corrected or added to GB 2312-80 is nearly impossible. + + +4.3: CHINESE (TAIWAN) + + The first question people think of with regard to Big Five and +CNS 11643-1992 is compatibility. It turns out that Planes 1 and 2 of +CNS 11643-1992 are more or less equivalent to Big Five, but a handful +of hanzi are in a different order. The following tables detail the +mapping from Big Five (with the ETen extension) to CNS 11643-1992 +(when using this conversion table, keep in mind the encoding space +ranges for both Big Five and CNS 11643-1992): + +Big Five Level 1 Correspondence to CNS 11643-1992 Plane 1: + + 0xA140-0xA1F5 <-> 0x2121-0x2256 + 0xA1F6 <-> 0x2258 + 0xA1F7 <-> 0x2257 + 0xA1F8-0xA2AE <-> 0x2259-0x234E + 0xA2AF-0xA3BF <-> 0x2421-0x2570 + 0xA3C0-0xA3E0 <-> 0x4221-0x4241 # Symbols for control characters + 0xA440-0xACFD <-> 0x4421-0x5322 # Level 1 Hanzi BEGIN + 0xACFE <-> 0x5753 + 0xAD40-0xAFCF <-> 0x5323-0x5752 + 0xAFD0-0xBBC7 <-> 0x5754-0x6B4F + 0xBBC8-0xBE51 <-> 0x6B51-0x6F5B + 0xBE52 <-> 0x6B50 + 0xBE53-0xC1AA <-> 0x6F5C-0x7534 + 0xC1AB-0xC2CA <-> 0x7536-0x7736 + 0xC2CB <-> 0x7535 + 0xC2CC-0xC360 <-> 0x7737-0x782C + 0xC361-0xC3B8 <-> 0x782E-0x7863 + 0xC3B9 <-> 0x7865 + 0xC3BA <-> 0x7864 + 0xC3BB-0xC455 <-> 0x7866-0x7961 + 0xC456 <-> 0x782D + 0xC457-0xC67E <-> 0x7962-0x7D4B # Level 1 Hanzi END + 0xC6A1-0xC6AA <-> 0x2621-0x262A # Circled numerals + 0xC6AB-0xC6B4 <-> 0x262B-0x2634 # Parenthesized numerals + 0xC6B5-0xC6BE <-> 0x2635-0x263E # Lowercase Roman numerals + 0xC6BF-0xC6C0 <-> 0x2723-0x2724 # 213 radicals BEGIN + 0xC6C1-0xC6C2 <-> 0x2726, 0x2728 + 0xC6C3-0xC6C5 <-> 0x272D-0x272F + 0xC6C6-0xC6C7 <-> 0x2734, 0x2737 + 0xC6C8-0xC6C9 <-> 0x273A, 0x273C + 0xC6CA-0xC6CB <-> 0x2742, 0x2747 + 0xC6CC-0xC6CD <-> 0x274E, 0x2753 + 0xC6CE-0xC6CF <-> 0x2754-0x2755 + 0xC6D0-0xC6D1 <-> 0x2759-0x275A + 0xC6D2-0xC6D3 <-> 0x2761, 0x2766 + 0xC6D4-0xC6D5 <-> 0x2829-0x282A + 0xC6D6-0xC6D7 <-> 0x2863, 0x286C # 213 radicals END + 0xC6D8-0xC6E6 -> ****** # Japanese symbols + 0xC6E7-0xC77A -> ****** # Hiragana + 0xC77B-0xC7F2 -> ****** # Katakana + 0xC7F3-0xC875 -> ****** # Cyrillic alphabet + 0xC876-0xC878 -> ****** # Symbols + 0xC87A -> ****** # Hanzi element + 0xC87C -> ****** # Hanzi element + 0xC87E-0xC8A1 -> ****** # Hanzi elements + 0xC8A3-0xC8A4 -> ****** # Hanzi elements + 0xC8A5-0xC8CC -> ****** # Combined numerals + 0xC8CD-0xC8D3 -> ****** # Japanese symbols + +Big Five Level 1 Correspondences to CNS 11643-1992 Plane 4: + + 0xC879 <-> 0x2123 # Hanzi element + 0xC87B <-> 0x2124 # Hanzi element + 0xC87D <-> 0x212A # Hanzi element + 0xC8A2 <-> 0x2152 # Hanzi element + +Big Five Level 2 Correspondence to CNS 11643-1992 Plane 1: + + 0xC94A -> 0x4442 # duplicate of 0xA461 + +Big Five Level 2 Correspondences to CNS 11643-1992 Plane 2: + + 0xC940-0xC949 <-> 0x2121-0x212A # Level 2 Hanzi BEGIN + 0xC94B-0xC96B <-> 0x212B-0x214B + 0xC96C-0xC9BD <-> 0x214D-0x217C + 0xC9BE <-> 0x214C + 0xC9BF-0xC9EC <-> 0x217D-0x224C + 0xC9ED-0xCAF6 <-> 0x224E-0x2438 + 0xCAF7 <-> 0x224D + 0xCAF8-0xD6CB <-> 0x2439-0x376E + 0xD6CC <-> 0x3E63 + 0xD6CD-0xD779 <-> 0x3770-0x387D + 0xD77A <-> 0x3F6A + 0xD77B-0xDADE <-> 0x387E-0x3E62 + 0xDADF <-> 0x376F + 0xDAE0-0xDBA6 <-> 0x3E64-0x3F69 + 0xDBA7-0xDDFB <-> 0x3F6B-0x4423 + 0xDDFC -> 0x4176 # duplicate of 0xDCD1 + 0xDDFD-0xE8A2 <-> 0x4424-0x554A + 0xE8A3-0xE975 <-> 0x554C-0x5721 + 0xE976-0xEB5A <-> 0x5723-0x5A27 + 0xEB5B-0xEBF0 <-> 0x5A29-0x5B3E + 0xEBF1 <-> 0x554B + 0xEBF2-0xECDD <-> 0x5B3F-0x5C69 + 0xECDE <-> 0x5722 + 0xECDF-0xEDA9 <-> 0x5C6A-0x5D73 + 0xEDAA-0xEEEA <-> 0x5D75-0x6038 + 0xEEEB <-> 0x642F + 0xEEEC-0xF055 <-> 0x6039-0x6242 + 0xF056 <-> 0x5D74 + 0xF057-0xF0CA <-> 0x6243-0x6336 + 0xF0CB <-> 0x5A28 + 0xF0CC-0xF162 <-> 0x6337-0x642E + 0xF163-0xF16A <-> 0x6430-0x6437 + 0xF16B <-> 0x6761 + 0xF16C-0xF267 <-> 0x6438-0x6572 + 0xF268 <-> 0x6934 + 0xF269-0xF2C2 <-> 0x6573-0x664C + 0xF2C3-0xF374 <-> 0x664E-0x6760 + 0xF375-0xF465 <-> 0x6762-0x6933 + 0xF466-0xF4B4 <-> 0x6935-0x6961 + 0xF4B5 <-> 0x664D + 0xF4B6-0xF4FC <-> 0x6962-0x6A4A + 0xF4FD-0xF662 <-> 0x6A4C-0x6C51 + 0xF663 <-> 0x6A4B + 0xF664-0xF976 <-> 0x6C52-0x7165 + 0xF977-0xF9C3 <-> 0x7167-0x7233 + 0xF9C4 <-> 0x7166 + 0xF9C5 <-> 0x7234 + 0xF9C6 <-> 0x7240 + 0xF9C7-0xF9D1 <-> 0x7235-0x723F + 0xF9D2-0xF9D5 <-> 0x7241-0x7244 # Level 2 Hanzi END + 0xF9DD-0xF9FE -> ****** # Symbols + +Big Five Level 2 Correspondence to CNS 11643-1992 Plane 3: + + 0xF9D6 <-> 0x4337 # ETen-specific hanzi + 0xF9D7 <-> 0x4F50 # ETen-specific hanzi + 0xF9D8 <-> 0x444E # ETen-specific hanzi + 0xF9D9 <-> 0x504A # ETen-specific hanzi + 0xF9DA <-> 0x2C5D # ETen-specific hanzi + 0xF9DB <-> 0x3D7E # ETen-specific hanzi + 0xF9DC <-> 0x4B5C # ETen-specific hanzi + +I adapted the above from material Ross Paterson (rap@doc.ic.ac.uk) +kindly made available at the following URL: + + http://www.ifcss.org:8001/www/pub/software/info/cjk-codes/ + +Check it out. Basically, I just changed the CNS 11643-1992 codes from +decimal row-cell values to hexadecimal codes, and corrected the +mappings to correspond to ETen's Big Five (which is considered to be +the most standard). + It turns out that corrections were made to Big Five (at least +in the ETen and Microsoft implementations thereof) which made it a bit +closer to CNS 11643-1992 as far as character ordering is concerned. +The following six lines of code correspondences: + + 0xCAF8-0xD6CB <-> 0x2439-0x376E + 0xD6CC <-> 0x3E63 + 0xD6CD-0xD779 <-> 0x3770-0x387D + 0xD77A <-> 0x3F6A + 0xD77B-0xDADE <-> 0x387E-0x3E62 + 0xDADF <-> 0x376F + +can now be expressed as the following three lines: + + 0xCAF8-0xD779 <-> 0x2439-0x387D + 0xD77A <-> 0x3F6A + 0xD77B-0xDBA6 <-> 0x387E-0x3F69 + +In essence, the ordering of Big Five characters 0xD6CC and 0xDADF were +reversed. This resulted in the same order as found in CNS 11643-1992 +Plane 2. + As for the two duplicate hanzi in Big Five (as indicated in +the above tables), they have been placed into a compatibility zone in +ISO 10646-1:1993 (this allows for round-trip conversion). The mapping +is as follows: + + Big Five ISO 10646-1:1993 + ^^^^^^^^ ^^^^^^^^^^^^^^^^ + 0xC94A -> 0xFA0C + 0xDDFC -> 0xFA0D + + Speaking of duplicate hanzi, Plane 1 of CNS 11643-1992 +contains 213 classical radicals in rows 27 through 29. However, 187 of +them map directly to hanzi code points in Planes 1, 2, and 3 (and +naturally to Big Five). Below is a detailed mapping of these 213 +radicals: + + Radical CNS 11643 Big Five Radical CNS 11643 Big Five + ^^^^^^^ ^^^^^^^^^ ^^^^^^^^ ^^^^^^^ ^^^^^^^^^ ^^^^^^^^ + 0x2721 -> 0x4421 0xA440 0x282E -> 0x4678 0xA5D8 + 0x2722 -> 0x2121 (3) ****** 0x282F -> 0x4679 0xA5D9 + 0x2723 -> 0x2122 (3) 0xC6BF 0x2830 -> 0x467A 0xA5DA + 0x2724 -> 0x2123 (3) 0xC6C0 0x2831 -> 0x467B 0xA5DB + 0x2725 -> 0x4422 0xA441 0x2832 -> 0x467C 0xA5DC + 0x2726 -> 0x2124 (3) 0xC6C1 0x2833 -> 0x2167 (2) 0xC9A8 + 0x2727 -> 0x4428 0xA447 0x2834 -> 0x467D 0xA5DD + 0x2728 -> ****** 0xC6C2 0x2835 -> 0x467E 0xA5DE + 0x2729 -> 0x4429 0xA448 0x2836 -> 0x4721 0xA5DF + 0x272A -> 0x442A 0xA449 0x2837 -> 0x484C 0xA6CB + 0x272B -> 0x442B 0xA44A 0x2838 -> 0x484D 0xA6CC + 0x272C -> 0x442C 0xA44B 0x2839 -> 0x484E 0xA6CD + 0x272D -> 0x2127 (3) 0xC6C3 0x283A -> 0x484F 0xA6CE + 0x272E -> 0x2128 (3) 0xC6C4 0x283B -> 0x2269 (2) 0xCA49 + 0x272F -> ****** 0xC6C5 0x283C -> 0x4850 0xA6CF + 0x2730 -> 0x442D 0xA44C 0x283D -> 0x4851 0xA6D0 + 0x2731 -> 0x2123 (2) 0xC942 0x283E -> 0x4852 0xA6D1 + 0x2732 -> 0x442E 0xA44D 0x283F -> 0x4854 0xA6D3 + 0x2733 -> 0x4430 0xA44F 0x2840 -> 0x4855 0xA6D4 + 0x2734 -> ****** 0xC6C6 0x2841 -> 0x4856 0xA6D5 + 0x2735 -> 0x4431 0xA450 0x2842 -> 0x4857 0xA6D6 + 0x2736 -> 0x2124 (2) 0xC943 0x2843 -> 0x4858 0xA6D7 + 0x2737 -> 0x2129 (3) 0xC6C7 0x2844 -> 0x485B 0xA6DA + 0x2738 -> 0x4432 0xA451 0x2845 -> 0x485C 0xA6DB + 0x2739 -> 0x4433 0xA452 0x2846 -> 0x485D 0xA6DC + 0x273A -> 0x212A (3) 0xC6C8 0x2847 -> 0x485E 0xA6DD + 0x273B -> 0x2125 (2) 0xC944 0x2848 -> 0x485F 0xA6DE + 0x273C -> 0x212B (3) 0xC6C9 0x2849 -> 0x4860 0xA6DF + 0x273D -> 0x4434 0xA453 0x284A -> 0x4861 0xA6E0 + 0x273E -> 0x4447 0xA466 0x284B -> 0x4862 0xA6E1 + 0x273F -> 0x212A (2) 0xC949 0x284C -> 0x4863 0xA6E2 + 0x2740 -> 0x4448 0xA467 0x284D -> 0x226A (2) 0xCA4A + 0x2741 -> 0x4449 0xA468 0x284E -> 0x226F (2) 0xCA4F + 0x2742 -> 0x213A (3) 0xC6CA 0x284F -> 0x4865 0xA6E4 + 0x2743 -> 0x444A 0xA469 0x2850 -> 0x4866 0xA6E5 + 0x2744 -> 0x444B 0xA46A 0x2851 -> 0x4867 0xA6E6 + 0x2745 -> 0x444C 0xA46B 0x2852 -> 0x4868 0xA6E7 + 0x2746 -> 0x444D 0xA46C 0x2853 -> 0x2270 (2) 0xCA50 + 0x2747 -> 0x213B (3) 0xC6CB 0x2854 -> 0x4B44 0xA8A3 + 0x2748 -> 0x4450 0xA46F 0x2855 -> 0x4B45 0xA8A4 + 0x2749 -> 0x4451 0xA470 0x2856 -> 0x4B46 0xA8A5 + 0x274A -> 0x4452 0xA471 0x2857 -> 0x4B47 0xA8A6 + 0x274B -> 0x4453 0xA472 0x2858 -> 0x4B48 0xA8A7 + 0x274C -> 0x212B (2) 0xC94B 0x2859 -> 0x4B49 0xA8A8 + 0x274D -> 0x4454 0xA473 0x285A -> 0x2524 (2) 0xCBA4 + 0x274E -> 0x213C (3) 0xC6CC 0x285B -> 0x4B4A 0xA8A9 + 0x274F -> 0x4456 0xA475 0x285C -> 0x4B4B 0xA8AA + 0x2750 -> 0x4457 0xA476 0x285D -> 0x4B4C 0xA8AB + 0x2751 -> 0x445A 0xA479 0x285E -> 0x4B4D 0xA8AC + 0x2752 -> 0x445B 0xA47A 0x285F -> 0x4B4E 0xA8AD + 0x2753 -> 0x213D (3) 0xC6CD 0x2860 -> 0x4B4F 0xA8AE + 0x2754 -> 0x213E (3) 0xC6CE 0x2861 -> 0x4B50 0xA8AF + 0x2755 -> 0x213F (3) 0xC6CF 0x2862 -> 0x4B51 0xA8B0 + 0x2756 -> 0x445C 0xA47B 0x2863 -> 0x272F (3) 0xC6D6 + 0x2757 -> 0x445D 0xA47C 0x2864 -> 0x4B57 0xA8B6 + 0x2758 -> 0x445E 0xA47D 0x2865 -> 0x4B5C 0xA8BB + 0x2759 -> 0x2140 (3) 0xC6D0 0x2866 -> 0x4B5D 0xA8BC + 0x275A -> 0x2142 (3) 0xC6D1 0x2867 -> 0x4B5E 0xA8BD + 0x275B -> 0x212C (2) 0xC94C 0x2868 -> 0x4F5A 0xAAF7 + 0x275C -> 0x4540 0xA4DF 0x2869 -> 0x4F5B 0xAAF8 + 0x275D -> 0x4541 0xA4E0 0x286A -> 0x4F5C 0xAAF9 + 0x275E -> 0x4542 0xA4E1 0x286B -> 0x4F5D 0xAAFA + 0x275F -> 0x4543 0xA4E2 0x286C -> 0x2A7D (3) 0xC6D7 + 0x2760 -> 0x4545 0xA4E4 0x286D -> 0x4F63 0xAB41 + 0x2761 -> 0x2167 (3) 0xC6D2 0x286E -> 0x4F64 0xAB42 + 0x2762 -> 0x4546 0xA4E5 0x286F -> 0x4F65 0xAB43 + 0x2763 -> 0x4547 0xA4E6 0x2870 -> 0x4F66 0xAB44 + 0x2764 -> 0x4548 0xA4E7 0x2871 -> 0x5372 0xADB1 + 0x2765 -> 0x4549 0xA4E8 0x2872 -> 0x5373 0xADB2 + 0x2766 -> 0x2169 (3) 0xC6D3 0x2873 -> 0x5374 0xADB3 + 0x2767 -> 0x454A 0xA4E9 0x2874 -> 0x5375 0xADB4 + 0x2768 -> 0x454B 0xA4EA 0x2875 -> 0x5376 0xADB5 + 0x2769 -> 0x454C 0xA4EB 0x2876 -> 0x5377 0xADB6 + 0x276A -> 0x454D 0xA4EC 0x2877 -> 0x5378 0xADB7 + 0x276B -> 0x454E 0xA4ED 0x2878 -> 0x5379 0xADB8 + 0x276C -> 0x454F 0xA4EE 0x2879 -> 0x537A 0xADB9 + 0x276D -> 0x4550 0xA4EF 0x287A -> 0x537B 0xADBA + 0x276E -> 0x213F (2) 0xC95F 0x287B -> 0x537C 0xADBB + 0x276F -> 0x4551 0xA4F0 0x287C -> 0x586B 0xB0A8 + 0x2770 -> 0x4552 0xA4F1 0x287D -> 0x586C 0xB0A9 + 0x2771 -> 0x4553 0xA4F2 0x287E -> 0x586D 0xB0AA + 0x2772 -> 0x4554 0xA4F3 0x2921 -> 0x334C (2) 0xD449 + 0x2773 -> 0x2141 (2) 0xC961 0x2922 -> 0x586E 0xB0AB + 0x2774 -> 0x4555 0xA4F4 0x2923 -> 0x334D (2) 0xD44A + 0x2775 -> 0x4556 0xA4F5 0x2924 -> 0x586F 0xB0AC + 0x2776 -> 0x4557 0xA4F6 0x2925 -> 0x5870 0xB0AD + 0x2777 -> 0x4558 0xA4F7 0x2926 -> 0x5E23 0xB3BD + 0x2778 -> 0x4559 0xA4F8 0x2927 -> 0x5E24 0xB3BE + 0x2779 -> 0x2142 (2) 0xC962 0x2928 -> 0x5E25 0xB3BF + 0x277A -> 0x455A 0xA4F9 0x2929 -> 0x5E26 0xB3C0 + 0x277B -> 0x455B 0xA4FA 0x292A -> 0x5E27 0xB3C1 + 0x277C -> 0x455C 0xA4FB 0x292B -> 0x5E28 0xB3C2 + 0x277D -> 0x455D 0xA4FC 0x292C -> 0x6327 0xB6C0 + 0x277E -> 0x4668 0xA5C8 0x292D -> 0x6328 0xB6C1 + 0x2821 -> 0x4669 0xA5C9 0x292E -> 0x6329 0xB6C2 + 0x2822 -> 0x466A 0xA5CA 0x292F -> 0x4155 (2) 0xDCB0 + 0x2823 -> 0x466B 0xA5CB 0x2930 -> 0x4875 (2) 0xE0EF + 0x2824 -> 0x466C 0xA5CC 0x2931 -> 0x676F 0xB9A9 + 0x2825 -> 0x466D 0xA5CD 0x2932 -> 0x6770 0xB9AA + 0x2826 -> 0x466E 0xA5CE 0x2933 -> 0x6771 0xB9AB + 0x2827 -> 0x4670 0xA5D0 0x2934 -> 0x6B7C 0xBBF3 + 0x2828 -> 0x4674 0xA5D4 0x2935 -> 0x6B7D 0xBBF4 + 0x2829 -> 0x225B (3) 0xC6D4 0x2936 -> 0x702F 0xBEA6 + 0x282A -> 0x225C (3) 0xC6D5 0x2937 -> 0x733E 0xC073 + 0x282B -> 0x4675 0xA5D5 0x2938 -> 0x733F 0xC074 + 0x282C -> 0x4676 0xA5D6 0x2939 -> 0x6142 (2) 0xEFB6 + 0x282D -> 0x4677 0xA5D7 + + +4.4: KOREAN + + The 268 duplicate hanja in KS C 5601-1992 can cause problems +when converting to and from other CJK character sets. When converting +from KS C 5601-1992, two or more hanja can collapse into a single code +point. When converting these 268 hanja to KS C 5601-1992, a decision +about which KS C 5601-1992 code point to map to must be made. The only +exception to this is mapping to and from ISO 10646-1:1993. That +standard encodes these 268 duplicate hanja in a compatibility zone, +namely from 0xF900 through 0xFA0B. + The following is a listing of 262 hanja that map to two or +more code points (four map to three code points, and one maps to four: +a total of 268 redundantly-encoded hanja) in KS C 5601-1992: + + Standard Extra Standard Extra Standard Extra + ^^^^^^^^ ^^^^^ ^^^^^^^^ ^^^^^ ^^^^^^^^ ^^^^^ + 0x4A39 -> 0x4D4F 0x5573 -> 0x6631 0x573C -> 0x6B29 + 0x4B3D -> 0x7A22 0x5574 -> 0x6633 0x573E -> 0x6B3A + 0x4C38 -> 0x7A66 0x5575 -> 0x6637 0x573F -> 0x6B3B + 0x4C5A -> 0x4B56 0x5576 -> 0x6638 0x5740 -> 0x6B3D + 0x4C78 -> 0x5050 0x5579 -> 0x663C 0x5741 -> 0x6B41 + 0x4D7A -> 0x4E2D 0x557B -> 0x6646 0x5743 -> 0x6B42 + 0x4E29 -> 0x7C29 0x557C -> 0x6647 0x5744 -> 0x6B46 + 0x4F23 -> 0x4F7B 0x557E -> 0x6652 0x5745 -> 0x6B47 + 0x4F4F -> 0x5022 0x5621 -> 0x6656 0x5747 -> 0x6B4C + 0x5038 0x5622 -> 0x6659 0x5748 -> 0x6B4F + 0x5142 -> 0x4B50 0x5623 -> 0x665F 0x5749 -> 0x6B50 + 0x5151 -> 0x505D 0x5624 -> 0x6661 0x574A -> 0x6B51 + 0x5159 -> 0x547C 0x5625 -> 0x6665 0x574C -> 0x6B58 + 0x5167 -> 0x552B 0x5626 -> 0x6664 0x574D -> 0x5270 + 0x522F -> 0x5155 0x5627 -> 0x6666 0x574E -> 0x5271 + 0x5233 -> 0x657C 0x5628 -> 0x6668 0x574F -> 0x5272 + 0x5234 -> 0x6644 0x562A -> 0x666A 0x5750 -> 0x5273 + 0x5235 -> 0x664A 0x562B -> 0x666B 0x5752 -> 0x5274 + 0x5236 -> 0x665C 0x562D -> 0x666F 0x5753 -> 0x5275 + 0x5237 -> 0x6676 0x562E -> 0x6671 0x5754 -> 0x5277 + 0x523A -> 0x6677 0x562F -> 0x6675 0x5755 -> 0x5278 + 0x523B -> 0x5638 0x5631 -> 0x6679 0x5757 -> 0x6C26 + 0x672C 0x5633 -> 0x6721 0x5759 -> 0x6C27 + 0x5241 -> 0x564D 0x5634 -> 0x6726 0x575B -> 0x6C2A + 0x5263 -> 0x6871 0x5635 -> 0x6729 0x575D -> 0x6C30 + 0x526E -> 0x6A74 0x5637 -> 0x672A 0x575E -> 0x6C31 + 0x526F -> 0x6B2A 0x563A -> 0x672D 0x5762 -> 0x6C35 + 0x527A -> 0x6C32 0x563B -> 0x6730 0x5765 -> 0x6C38 + 0x527B -> 0x6C49 0x563C -> 0x673F 0x5767 -> 0x6C3A + 0x527C -> 0x6C4A 0x563E -> 0x6746 0x576A -> 0x6C40 + 0x527E -> 0x7331 0x5640 -> 0x6747 0x576B -> 0x6C41 + 0x5321 -> 0x552E 0x5642 -> 0x674B 0x576C -> 0x6C45 + 0x5358 -> 0x7738 0x5643 -> 0x674D 0x576E -> 0x6C46 + 0x536B -> 0x7748 0x5644 -> 0x674F 0x5770 -> 0x6C55 + 0x5378 -> 0x7674 0x5645 -> 0x6750 0x5772 -> 0x6C5D + 0x5441 -> 0x5466 0x5647 -> 0x6753 0x5773 -> 0x6C5E + 0x5457 -> 0x7753 0x5649 -> 0x675F 0x5774 -> 0x6C61 + 0x547A -> 0x5154 0x564A -> 0x6764 0x5776 -> 0x6C64 + 0x547B -> 0x5158 0x564B -> 0x6766 0x5777 -> 0x6C67 + 0x547D -> 0x515B 0x564C -> 0x523E 0x5778 -> 0x6C68 + 0x547E -> 0x515C 0x564F -> 0x5242 0x5779 -> 0x6C77 + 0x5521 -> 0x515D 0x5650 -> 0x5243 0x577A -> 0x6C78 + 0x5522 -> 0x515E 0x5653 -> 0x5244 0x577C -> 0x6C7A + 0x5523 -> 0x515F 0x5654 -> 0x5246 0x5821 -> 0x6D21 + 0x5524 -> 0x5160 0x5655 -> 0x5247 0x5822 -> 0x6D22 + 0x5526 -> 0x5163 0x5656 -> 0x5248 0x5823 -> 0x6D23 + 0x5527 -> 0x5164 0x5657 -> 0x5249 0x5A72 -> 0x5B64 + 0x5528 -> 0x5165 0x5658 -> 0x524A 0x5C56 -> 0x5D25 + 0x552A -> 0x5166 0x565A -> 0x524B 0x5C5F -> 0x7870 + 0x552C -> 0x5168 0x565B -> 0x524D 0x5C74 -> 0x5D55 + 0x552D -> 0x5169 0x565C -> 0x524E 0x5D41 -> 0x5B45 + 0x552F -> 0x516A 0x565E -> 0x524F 0x5F2F -> 0x616D + 0x5530 -> 0x516B 0x565F -> 0x5250 0x5F52 -> 0x6D6E + 0x5531 -> 0x516D 0x5660 -> 0x5251 0x5F5D -> 0x5F61 + 0x5534 -> 0x516F 0x5661 -> 0x5252 0x5F63 -> 0x5E7E + 0x5535 -> 0x5170 0x5662 -> 0x5253 0x6063 -> 0x612D + 0x5536 -> 0x5172 0x5663 -> 0x5254 0x6672 + 0x5539 -> 0x5176 0x5665 -> 0x5255 0x607D -> 0x5F68 + 0x553D -> 0x517A 0x5666 -> 0x5256 0x6163 -> 0x574B + 0x5540 -> 0x517C 0x5667 -> 0x5257 0x6B52 + 0x5541 -> 0x517D 0x566B -> 0x5259 0x6226 -> 0x5E7C + 0x5543 -> 0x517E 0x566C -> 0x525A 0x6326 -> 0x6429 + 0x5544 -> 0x5222 0x566F -> 0x525E 0x635B -> 0x723D + 0x5545 -> 0x5223 0x5670 -> 0x525F 0x6427 -> 0x727A + 0x5546 -> 0x5227 0x5671 -> 0x5261 0x6442 -> 0x6777 + 0x5547 -> 0x5228 0x5674 -> 0x5262 0x6445 -> 0x5162 + 0x5548 -> 0x5229 0x5675 -> 0x6867 0x5525 + 0x5549 -> 0x522A 0x5676 -> 0x6868 0x6879 + 0x554D -> 0x522B 0x5677 -> 0x6870 0x6534 -> 0x652E + 0x554E -> 0x522D 0x5679 -> 0x6877 0x6636 -> 0x6C2F + 0x5552 -> 0x5232 0x567A -> 0x687B 0x6728 -> 0x6071 + 0x5553 -> 0x6531 0x567B -> 0x687E 0x6856 -> 0x6A41 + 0x5554 -> 0x6532 0x567E -> 0x6927 0x6C36 -> 0x5764 + 0x5555 -> 0x6539 0x5721 -> 0x692C 0x6C56 -> 0x666C + 0x5557 -> 0x653B 0x5723 -> 0x694C 0x6D29 -> 0x7427 + 0x5558 -> 0x653C 0x5724 -> 0x5264 0x6D33 -> 0x6E5B + 0x5559 -> 0x6544 0x5726 -> 0x5265 0x6F37 -> 0x746E + 0x555D -> 0x654E 0x5727 -> 0x5266 0x7263 -> 0x6375 + 0x555E -> 0x6550 0x5728 -> 0x5267 0x7333 -> 0x4B67 + 0x555F -> 0x6552 0x5729 -> 0x5268 0x7351 -> 0x5F33 + 0x5561 -> 0x6556 0x572B -> 0x5269 0x742C -> 0x7676 + 0x5564 -> 0x657A 0x572C -> 0x526A 0x7658 -> 0x6421 + 0x5565 -> 0x657B 0x5730 -> 0x526B 0x7835 -> 0x5C25 + 0x5566 -> 0x657E 0x5731 -> 0x6A65 0x786C -> 0x785B + 0x5569 -> 0x6621 0x5733 -> 0x6A77 0x7932 -> 0x5D74 + 0x556B -> 0x6624 0x5735 -> 0x6A7C 0x7A3C -> 0x7A21 + 0x556C -> 0x6627 0x5736 -> 0x6A7E 0x7B29 -> 0x6741 + 0x556F -> 0x662D 0x5738 -> 0x6B24 0x7C41 -> 0x4D68 + 0x5571 -> 0x662F 0x573A -> 0x6B27 0x7D3B -> 0x6977 + 0x5572 -> 0x6630 + +The above table represents a weekend of my time (but time well spent, +in my opinion). + + +4.5: ISO 10646-1:1993 + + The Chinese character subset of ISO 10646-1:1993 +has excellent round-trip conversion capability with the various +national character sets. Those national character sets with duplicate +characters, such as KS C 5601-1992 (268 hanja) and Big Five (2 hanzi), +have corresponding code points in ISO 10646-1:1993 within +a compatibility zone. See Sections 4.3 and 4.4 for more details. + Other issues regarding ISO 10646-1:1993 have to do with proper +character rendering (that is, how characters are displayed, printed, +or otherwise imaged). Many (sometimes) subtle character form +differences have been collapsed under ISO 10646-1:1993. Language or +locale was not one of the factors used in performing Han Unification. +This means that it is nearly impossible to create a single ISO 10646-1: +1993 font that meets the character form criteria of each of the four +CJK locales. An ISO 10646-1:1993 code point is not enough information +to render a Chinese character. If the font was specifically designed +for a single locale, it is a non-problem, but if there is any CJK +intent, text must be flagged for language or locale. + + +4.6: UNICODE + + One of the most interesting (and major) differences between +the current three flavors of Unicode are the number and arrangement of +pre-combined hangul. The following table provides a summary of the +differences: + + Unicode Number of Pre-combined Hangul UCS-2 Ranges + ^^^^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ ^^^^^^^^^^^^ + Version 1.0 2,350 Basic Hangul 0x3400-0x3D3D + + Version 1.1 2,350 Basic Hangul 0x3400-0x3D3D + 1,930 Supplemental Hangul A 0x3D2E-0x44B7 + 2,376 Supplemental Hangul B 0x44BE-0x4DFF + + Version 2.0 11,172 Hangul 0xAC00-0xD7A3 + +Of the above three versions, the most controversial is Version 2.0. +Why? Because it is located in the user-defined range of Unicode +(O-Zone: 16,384 code points in 0xA000-0xDFFF), and occupies +approximately two-thirds of its space. + The information in the above table is courtesy of the +following useful document: + + ftp://unicode.org/pub/MappingTables/EastAsiaMaps/Hangul-Codes.txt + +The same file is also mirrored at the following URL: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/map/hangul-codes.txt + + +4.7: CODE CONVERSION TIPS + + There are two types of conversions that can be performed. The +first type is converting between different encodings for the same +character set. This is usually without problems (but not always). The +second type is converting from one character set to another (it is not +usually relevant whether the underlying encoding has changed or not). +This usually involves the handling of characters that are in one +character set, but not the other. So, what to do? + I suggest JConv for handling Japanese code conversion (this +means converting between JIS, Shift-JIS, and EUC encodings). This is +in the category of different encodings for the same character set. The +following URLs provide executables or source code: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/mac/jconv-30.hqx + ftp://ftp.ora.com/pub/examples/nutshell/ujip/mac/jconv-dd-181.hqx + ftp://ftp.ora.com/pub/examples/nutshell/ujip/dos/jconv.exe + ftp://ftp.ora.com/pub/examples/nutshell/ujip/src/jconv.c + +There are other programs available that do the same basic thing as +JConv, such as kc and nkf. They are available at the following URL: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/unix/ + + For software and tables that handles Chinese code conversion +(this includes conversion to and from Japanese), I suggest browsing at +the following URLs: + + ftp://etlport.etl.go.jp/pub/iso-2022-cn/convert/ + ftp://ftp.ifcss.org/pub/software/dos/convert/ + ftp://ftp.ifcss.org/pub/software/mac/convert/ + ftp://ftp.ifcss.org/pub/software/ms-win/convert/ + ftp://ftp.ifcss.org/pub/software/unix/convert/ + ftp://ftp.ifcss.org/pub/software/vms/convert/ + ftp://ftp.net.tsinghua.edu.cn/pub/Chinese/convert/ + ftp://ftp.ora.com/pub/examples/nutshell/ujip/map/ + ftp://ftp.seed.net.tw/Pub/Chinese/DOS/code-convert/ + http://www.yajima.kuis.kyoto-u.ac.jp/staffs/yasuoka/CJK.html + +The latter URL has FTP links to tables created by Koichi Yasuoka +(yasuoka@kudpc.kyoto-u.ac.jp). + The following URLs provide utilities or tables for converting +between various Korean encodings (the last represent the same file): + + ftp://cair-archive.kaist.ac.kr/pub/hangul/code/ + ftp://ftp.ora.com/pub/examples/nutshell/ujip/map/non-hangul-codes.txt + ftp://ftp.ora.com/pub/examples/nutshell/ujip/map/hangul-codes.txt + ftp://unicode.org/pub/MappingTables/EastAsiaMaps/Hangul-Codes.txt + +A popular Korean code conversion utility seems to be "hcode" by +June-Yub Lee (jylee@cims.nyu.edu). + Finally, the following URLs provide many Unicode- and CJK- +related mapping tables: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/map/ + ftp://ftp.ora.com/pub/examples/nutshell/ujip/unicode/ + ftp://unicode.org/pub/MappingTables/ + http://www.yajima.kuis.kyoto-u.ac.jp/staffs/yasuoka/CJK.html + +Note that the official and authoritative Unicode mapping tables (from +Unicode values to various international, national and vendor +standards) are maintained by the Unicode Consortium at the following +URL: + + ftp://unicode.org/pub/MappingTables/ + +Version 2.0 of "The Unicode Standard" (to be published by Addison- +Wesley shortly) will include these mapping tables on CD-ROM. + + +PART 5: CJK-CAPABLE OPERATING SYSTEMS + + The first step in being able to display CJK text is to obtain +an operating system that handles such text (or an application that +sets up its own CJK-capable environment). Below I describe how +different types of machines can handle CJK text. + Actually, for the first few releases of CJK.INF, these +subsections will be far from complete (some may even be empty!). The +purpose of CJK.INF is to provide detailed information on character set +standards and encoding systems, so I therefore consider this sort of +information secondary. + + +5.1: MS-DOS + + I am not aware of any CJK-capable MS-DOS operating system, but +localized versions do exist. CJK support has been introduced with +Microsoft's Windows operating system (see Section 5.2). + + +5.2: WINDOWS + + Microsoft has CJK versions of its Windows operating system +available. The latest versions of their Windows operating system are +called Windows 95 and Windows NT. Windows 95 supports the same +character sets and encodings as in Windows Version 3.1 -- Windows NT +supports Unicode (ISO 10646-1:1993). Contact Microsoft Corporation for +more details. The URL of their WWW Home Page is: + + http://www.microsoft.com/ + +Nadine Kano's "Developing International Software for Windows 95 and +Windows NT" provides abundant reference material for how CJK is +supported in Windows 95 and Windows NT. Check it out. + TwinBridge is a package that adds CJK functionality to non-CJK +Windows. Demo versions of TwinBridge for Japanese and Chinese are at +the following URLs: + + ftp://ftp.netcom.com/pub/tw/twinbrg/Japanese/demo/tbjdemo.zip + ftp://ftp.netcom.com/pub/tw/twinbrg/Chinese/demo/tbcdemo.zip + + Another useful CJK add-on for Windows 95 is NJWIN (see Section +7.10) by Hongbo Data Systems. + + +5.3: MACINTOSH + + Macintosh is well-known as a computer that was designed to +handle multilingual texts. There are currently fully-localized +operating systems available for Japanese (KanjiTalk), Chinese +(simplified and traditional available), and Korean (HangulTalk). In +addition, Apple has developed "Language Kits" (*LK) for Chinese (CLK) +and Japanese (JLK). A Korean Language Kit (KLK) will be released +shortly. + These localized operating systems can usually be installed +together in order to make your system CJK-capable. + The common portion of these CJK-capable operating systems is a +technology Apple calls "WorldScript II" ("WorldScript I" is for one- +byte scripts). It provides the basic one- and two-byte functionality. + + +5.4: UNIX AND X WINDOWS + + The typical encoding system used on UNIX and X Windows is EUC +(see Section 3.2). Many systems, such as IBM's AIX, can be configured +to handle both EUC and Shift-JIS (for Japanese). In addition, X11R6 (X +Window System, Version 11, Release 6) has many CJK-capable features. + If you have a fast PC and a good amount of RAM (more than +4MB), you should consider replacing MS-DOS (and Microsoft Windows, +too, if you have it) with Linux, which is a full-blown UNIX operating +system that runs on Intel processors. You can even run X Windows +(X11R6). "Running Linux" by Matt Welsh and Lar Kaufman is an excellent +guide to installing and using Linux. The companion volume, "Linux +Network Administrator's Guide" by Olaf Kirch is also useful. Because +there is a fine line -- or no line at all -- between a user and System +Administrator when using Linux, "Essential System Administration" +Second Edition by AEleen Frisch is a must-have. + Linux and Linux information are available at the following +URLs: + + ftp://sunsite.unc.edu/pub/Linux/ + http://sunsite.unc.edu/mdw/linux.html + +I personally use Linux, and find it quite useful and powerful. My bias +comes from being a UNIX user. But, you can't beat the price (free), +and all of my favorite text-manipulation tools (such as Perl) are +readily available. + + +5.5: OTHERS + + No information yet. + + +PART 6: CJK TEXT AND INTERNET SERVICES + + Part 5 described how CJK text is handled on a machine +internally, but this part goes into the implications of handling such +text externally, namely for information interchange purposes. This +boils down to handling CJK text on Internet services. + For more detailed information on how these and other Internet +services are used, I suggest "The Whole Internet User's Guide & +Catalog" by Ed Krol. For more information on setting up and +maintaining these and other Internet services, I suggest "Managing +Internet Information Services" by Cricket Liu et al. + + +6.1: ELECTRONIC MAIL + + The most basic Internet service is electronic mail (henceforth +to be called "e-mail"), which is virtually guaranteed to be available +to all users regardless of their system. + Several Internet standards (called RFCs, short for Request For +Comments) have been developed to describe how CJK text is to be handled +over e-mail systems (see Section A.3.4). + The bottom-line is that most e-mail systems do not support +8-bit characters (that is, bytes that have their 8th bit set). Some do +offer 8-bit support, but you can never know what path your e-mail +might take while on route to its recipient. This means that 7-bit ISO +2022 (or equivalent) is the ideal encoding to use when sending CJK +text through e-mail. If your operating system processes another +encoding system, you must convert from that encoding to one that is +compatible with 7-bit ISO 2022. + However, even 7-bit ISO 2022 encoding can get mangled by +mail-routing software -- the escape character, sometimes even part of +the escape sequence (meaning more than just the escape character), is +stripped. The JConv tool described in Section 4.7 restores stripped +escape sequences for Japanese 7-bit ISO 2022. + If your mailing software is MIME-compliant, there is a means +to identify the character set and encoding of the message using the +"charset" parameter. Some valid "charset" values include the +following: + +o iso-2022-jp (see Section 3.1.3) +o iso-2022-jp-2 (see Section 3.1.3) +o iso-2022-kr (see Section 3.1.4) +o iso-2022-cn (see Section 3.1.5) +o iso-2022-cn-ext (see Section 3.1.5) +o iso-8859-1 + +Insertion of these values should happen automatically. + A last-ditch effort to send CJK text through e-mail is to use +uuencode or Base64 encoding (see Section 3.3.13). Base64 is something +that is usually done automatically by mailing software -- explicit +Base64 encoding is not common. The recipient must then run uudecode or +a Base64 decoder to get the original file (if such utilities are +available). + + +6.2: USENET NEWS + + Usenet News follows many of the same requirements as e-mail, +namely that 7-bit ISO 2022 encoding is ideal. However, some newsgroups +use specific encoding methods, such as: + + alt.chinese.text (HZ encoding used for Chinese text) + alt.chinese.text.big5 (Big Five encoding used for Chinese text) + chinese.flame (UTF-7) + chinese.text.unicode (UTF-8) + +Also, the newsgroups in Korean (all begin with "han.*") use EUC (EUC- +KR) because the news-handling software in Korea has been designed to +handle eight-bit characters correctly. Mailing list versions of Korean +newsgroups are likely to use ISO-2022-KR encoding. + One common problem with Usenet News is that the escape +characters used in 7-bit ISO 2022 encoding are sometimes stripped, +usually by the software used to post the article. This can be quite +annoying. There are programs available, such as JConv, that repair +such files by restoring the escape characters. + Another common problem are news readers that do not allow +escape characters to function. One simple solution is to "pipe" the +article through a display command, such as "more," "page," "less," or +"cat." This is done by typing a "pipe" character (|) followed by the +command name anywhere within the article being displayed. + + +6.3: GOPHER + + The World-Wide Web (WWW) has almost eliminated the need for +using Gopher, so I won't discuss it here. Not that I don't appreciate +Gopher servers, but what I mean is that WWW browsing software permits +access to Gopher sites. + + +6.4: WORLD-WIDE WEB + + First, there are two types of WWW browsers available. The most +common type is the graphics-based browser (examples include Mosaic and +Netscape). Graphics-based browsers have the unfortunate requirement of +a TCP/IP (SLIP and PPP support these protocols) connection. Lynx and +the W3 client for Emacs, which are text-based browsers, can be run +from the host computer through a standard terminal connection. They +don't display all the pretty pictures that folks put into their WWW +documents, but you get all the text (this is, in many ways, a blessing +in disguise -- transferring graphics is what slows down graphics-based +browsers the most). When the W3 client is run using Mule, it becomes a +fully CJK-capable WWW browser. Both Lynx and the W3 client for Emacs +are freely available. A Japanese-capable Lynx is available at the +following URL: + + ftp://ftp.ipc.chiba-u.ac.jp/pub.asada/www/lynx/ + +There is also a WWW page that provides information on Japanese-capable +Lynx. Its URL is as follows: + + http://www.icsd6.tj.chiba-u.ac.jp/lynx/ + + When WWW documents first came online, there was no method for +handling CJK character sets. This has, fortunately, changed. As of +this writing, two commercial WWW browsers support Japanese. They are +Infomosaic by Fujitsu Limited, and Netscape Navigator by Netscape +Communications Corporation (Version 1.1 added Japanese support). Both +are graphics-based browsers. The former can be ordered at the +following URL: + + http://www.fujitsu.co.jp/ + +The latter can be found at the following URLs: + + http://www.netscape.com/ + ftp://ftp.netscape.com/ + + One can also use a delegate server to *filter* Japanese codes +to the one supported by your browser. It is also possible to +"Japanize" existing WWW browsers using assorted tools and patches. +Katsuhiko Momoi (momoi@tigger.stcloud.msus.edu) has authored an +excellent guide to Japanizing WWW browsers. Its URL is: + + http://condor.stcloud.msus.edu:20020/netscape.html + +I *highly* suggest reading it. + Japanese-capable WWW browsers support automatic detection of +the three Japanese encoding methods (JIS, Shift-JIS, and EUC). Hey, +but, what about support for the "C" and "K" of CJK? Attempting to +answer this question provides us an answer to another question: "What +is the best encoding method to use for CJK WWW documents?" + Encoding methods such as EUC and Shift-JIS provide for mixing +only two character sets. This is because they provide no way to *flag* +or *tag* text for locale (character set) information. Without flagging +information, it is impossible to distinguish Japanese EUC from Chinese +or Korean EUC. However, the escape sequences used in 7-bit ISO 2022 +encoding explicitly provide locale information. 7-bit ISO 2022 is +ideal for static documents, which is exactly what one finds on WWW. + My personal recommendation (for the short-term) is to compose +WWW documents (also called HTML documents; HTML stands for Hyper Text +Markup Language) using 7-bit ISO 2022 encoding. The escape sequences +themselves act as explicit flags that indicate locale. However, some +WWW clients are confused by 7-bit ISO 2022 encoding, but the products +by Netscape Communications and Fujitsu Limited prove that this can +work. See the following URL for a description of this problem: + + http://www.ntt.jp/japan/note-on-JP/LibWWW-patch.html + + Check out the following URLs for information on and proposals +for international support for WWW: + + http://www.ebt.com:8080/docs/multilingual-www.html + http://www.w3.org/hypertext/WWW/International/Overview/ + + There is currently an RFC in the works (called an Internet +Draft) to address the problem of internationalizing HTML by using +Unicode. It is very promising. The latest draft is available at the +following URLs: + + ftp://ds.internic.net/internet-drafts/draft-ietf-html-i18n-04.txt.Z + ftp://ftp.isi.edu/internet-drafts/draft-ietf-html-i18n-04.txt + ftp://munnari.oz.au/internet-drafts/draft-ietf-html-i18n-04.txt.Z + ftp://nic.nordu.net/internet-drafts/draft-ietf-html-i18n-04.txt + +Note that some have been compressed. + + +6.5: FILE TRANSFER TIPS + + Although CJK encoding systems such as Shift-JIS and EUC make +extensive use of 8-bit bytes, that does not mean that you need to +treat the data as binary. Such files are simply to be treated as text, +and should be transferred in text mode (for example, FTP's ASCII mode, +which is also called "Type A Transfer"). + When text files are transferred in binary mode (such as FTP's +BINARY mode, which is also called Type I Transfer"), line termination +characters are left unaltered. For example, when transferring a text +file from UNIX to Macintosh, a text transfer will translate the UNIX +newline (0x0A) characters to Macintosh carriage return (0x0D) +characters, but a binary transfer will make no such modifications. +Text-style conversion is typically desired. + The most common types of files that need to be handled as +binary include tar archives (*.tar), compressed files (*.Z, *.gz, +*.zip, *.zoo, *.lzh, and so on), and executables (*.exe, *.bin, and so +on). + + +PART 7: CJK TEXT HANDLING SOFTWARE + + This section describes various CJK-capable software packages. +I expect this section to grow with future versions of this document. I +define "CJK-capable" as being able to support Chinese, Japanese, and +Korean text. + The descriptions I provide below are intentionally short. You +are encouraged to use the information pointers to obtain further +information or the software itself. + + +7.1: MULE + + Mule (multilingual enhancement to GNU Emacs), written by +Kenichi Handa (handa@etl.go.jp), is the first (and only?) CJK-capable +editor for UNIX systems, and is freely available under the terms of +the GNU General Public License. Mule was developed from Nemacs +(Nihongo Emacs). + Mule is available at the following URL: + + ftp://etlport.etl.go.jp/pub/mule/ + + Mule, beginning with Version 2.2, includes handy utilities +(any2ps and m2ps) for printing files in any of the encodings supported +by Mule (which is a lot of encodings, by the way). These programs use +BDF fonts. See the beginning of Part 2 for a list of URLs that have +CJK BDF fonts. + GNU Emacs is a fine editor, and Mule takes it several steps +further by providing multilingual support. I personally use Mule +together with SKK (for Japanese input) -- it is a superb combination. + + +7.2: CNPRINT + + CNPRINT, developed by Yidao Cai (cai@neurophys.wisc.edu), is a +utility to print CJK text (or convert it to a PostScript file), and is +available for MS-DOS, VMS, and UNIX systems. A wide range of encoding +methods are supported by CNPRINT. + CNPRINT is available at the following URLs: + + ftp://ftp.ifcss.org/pub/software/{dos,unix,vms}/print/ + ftp://neurophys.wisc.edu/[public.cn]/ + + +7.3: MASS + + MASS (Multilingual Application Support Service), developed at +the National University of Singapore, is a suite of software tools +that speed and ease the development of UNIX-based CJK (actually, more +than just CJK) applications. It supports a wide variety of character +sets and encodings, including ISO 10646-1:1993 (UCS-2, UTF-7, and +UTF-8), EACC, and CCCII. + More information on MASS, to include contact information for +its developers, can be found at the following URL: + + http://www.iss.nus.sg/RND/MLP/Projects/MASS/MASS.html + + +7.4: ADOBE TYPE MANAGER (ATM) + + Adobe Type Manager for Macintosh, beginning with Version 3.8, +is CJK-capable (as long as the underlying operating system is CJK- +capable). Actually, ATM generically supports CID-keyed fonts, which +are based on a newly-developed file specification for fonts with large +numbers of characters (like CJK fonts). See Section 7.9 for more +details. + ATM is very easy to obtain. It is bundled with fonts and +applications from Adobe Systems (chances are you have ATM if you +recently purchased an Adobe product). But what about Windows? The +Windows version of ATM should soon follow with identical +functionality. + + +7.5: MACINTOSH SOFTWARE + + WorldScript II, a System Extension introduced with System 7, +provides multi-byte script handling, namely CJK support. If a +Macintosh product claims to support WorldScript II, chances are it is +CJK-capable (provided that your operating system has the necessary +extensions loaded). + The CJK encodings that are supported by WorldScript II capable +applications are the same as made available by the underlying +Macintosh operating system. No import/export of other encodings is +supported at the operating system level. You must run separate +conversion utilities for both import and export. Anyway, below are +some products that are known to be CJK capable. + Nisus Writer, written by Nisus Software, is fully CJK-capable +as long as you have the appropriate scripts installed (such as CLK for +Chinese or JLK for Japanese). A "Language Key" (read "dongle") is also +required for Chinese and Korean (and some one-byte scripts such as +Arabic and Hebrew). A demo version of Nisus Writer is available at the +following URL: + + ftp://ftp.nisus-soft.com/pub/nisus/demos/ + +Give it a try! Updates are also available at the same FTP site. Nisus +Software can be contacted using the following e-mail address or +through their WWW page: + + info@nisus-soft.com + http://www.nisus-soft.com/ + +I also suggest reading "The Nisus Way" by Joe Kissell. Chapter 13 +provides detailed information about using Nisus Writer with +WorldScript, and includes a CD-ROM containing among other things a +trial (expires after 90 days) version of Nisus Writer and a +non-expiring version of Nisus Compact. + ClarisWorks by Claris Corporation, beginning with Version 4.0, +is compatible with WorldScript II and all Apple language kits. This +translates into full CJK support. The following URL provides a trial +version of ClarisWorks: + + ftp://ftp.claris.com/pub/USA-Macintosh/Trial_Software/ + +The following URL has detailed information on this and other Claris +products: + + http://www.claris.com/ + + The latest version of WordPerfect by Novell Incorporated is +also compatible with WorldScript II. The following URL has detailed +information: + + http://wp.novell.com/tree.htm + + +7.6: MACBLUE TELNET + + Although MacBlue Telnet (a modified version of NCSA Telnet) is +Macintosh software, I describe it separately because it does not +require the various Apple Language Kits or localized operating +systems. There are also input methods, adapted from cxterm (see +Section 7.7), available that cover the CJK spectrum (Japanese, +Simplified Chinese, Traditional Chinese, and Korean). + MacBlue Telnet is available at the following URL: + + ftp://ftp.ifcss.org/pub/software/mac/networking/MacBlueTelnet/ + +Its associated CJK input methods are at the following URL: + + ftp://ftp.ifcss.org/pub/software/mac/input/ + + +7.7: CXTERM + + This program, cxterm, is a CJK-capable xterm for X Windows +(works with X11R4, X11R5, and X11R6). It is based on the X11R6 xterm. +It is available at the following URL: + + ftp://ftp.ifcss.org/pub/software/x-win/cxterm/ + + The following URL is for a program that adds Unicode +capability to cxterm: + + ftp://ftp.ifcss.org/pub/software/unix/convert/hztty-2.0.tar.gz + +The following URL adds support for other encodings to cxterm: + + ftp://ftp.ifcss.org/pub/software/unix/convert/BeTTY-1.534.tar.gz + + +7.8: UW-DBM + + UW-DBM, for Windows 3.1, Windows 95, and Windows NT, is a +program that allows users to handle Chinese (Big Five, GB-2312-80, or +HZ code), Japanese (Shift-JIS), and Korean (KS C 5601-1992) +simultaneously. More information on UW-DBM is available at the +following URL: + + http://www.gy.com/ccd/win95/cjkw95.htm + + A demo version of UW-DBM is available at the following URL: + + ftp://ftp.aimnet.com/pub/users/chinabus/uwdbm40.zip + + +7.9: POSTSCRIPT + + With the introduction of CID-keyed Font Technology, PostScript +has become fully CJK capable. + Adobe Systems has developed the following CJK character +collection for CID-keyed fonts (font developers are encouraged to +conform to these specifications): + + Character Collection CIDs Supported Character Sets & Encodings + ^^^^^^^^^^^^^^^^^^^^ ^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + Adobe-GB1-1 9,897 GB 2312-80 and GB/T 12345-90; 7-bit ISO + 2022 and EUC + Adobe-CNS1-0 14,099 Big Five (ETen extensions) and CNS + 11643-1992 Planes 1 and 2; Big Five, + 7-bit ISO 2022, and EUC + Adobe-Japan1-2 8,720 JIS X 0208-1990; Shift-JIS, 7-bit ISO + 2022, and EUC + Adobe-Japan2-0 6,068 JIS X 0212-1990; 7-bit ISO 2022 and EUC + Adobe-Korea1-1 18,155 KS C 5601-1992 (Macintosh extensions + plus Johab); 7-bit ISO 2022, EUC, UHC, + and Johab + +Note that Macintosh and Windows do not support any of the encodings +for Adobe-Japan2-0, thus fonts based on that specification are +unusable for those platforms. + Adobe Systems also have a few things in the works (that is, +they are either proposed or in draft form), all of which are +supplements to above character collections (that is, they add CIDs): + + Character Collection CIDs Supported Character Sets & Encodings + ^^^^^^^^^^^^^^^^^^^^ ^^^^ ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + Adobe-CNS1-1 +6,018 Add CNS 11643-1992 Plane 3 support (30 + of the 6,148 hanzi are in Adobe-CNS1-0) + + To find out more about these CJK character collections or +CID-keyed font technology, contact the Adobe Developers Association. +Several CID-related documents have been published. ADA's contact +information is as follows: + + Adobe Developers Association + Adobe Systems Incorporated + 1585 Charleston Road + P.O. Box 7900 + Mountain View, CA 94039-7900 + USA + +1-415-961-4111 (phone) + +1-415-967-9231 (facsimile) + devsupp-person@adobe.com + http://www.adobe.com/Support/ + +Adobe Systems has recently developed the CID SDK (CID Software +Developers Kit), which is on a single CD-ROM. Contact the Adobe +Developers Association for information on obtaining a copy. + The complete CID-keyed font file specification and an overview +document are available at the following URLs (as a PostScript or PDF +[Adobe Acrobat] file, respectively): + + ftp://ftp.adobe.com/pub/adobe/DeveloperSupport/TechNotes/PSfiles/ + ftp://ftp.adobe.com/pub/adobe/DeveloperSupport/TechNotes/PDFfiles/ + +The file names (not provided above due to URL length) are: + + 5014.CMap_CIDFont_Spec.ps (complete CID engineering specification) + 5014.CMap_CIDFont_Spec.pdf + 5092.CID_Overview.ps (CID technology overview) + 5092.CID_Overview.pdf + +Other related files, most character collection specifications, are +available only in PDF format at the latter URL indicated above: + + 5004.AFM_Spec.pdf (Includes CID-keyed AFM specification) + 5078b.pdf (Adobe-Japan1-2 character collection) + 5079b.pdf (Adobe-GB1-0 character collection) + 5080b.pdf (Adobe-CNS1-0 character collection) + 5093b.pdf (Adobe-Korea1-0 character collection) + 5094.pdf (Adobe CJK CMap file descriptions) + 5097b.pdf (Adobe-Japan2-0 character collection) + +If you do not have Adobe Acrobat, there is a freely-available Acrobat +Reader (for Macintosh, Windows, MS-DOS, and UNIX) at the following +URL: + + ftp://ftp.adobe.com/pub/adobe/Applications/Acrobat/ + + I have also placed some CJK character collection materials, +including prototype Unicode (UCS-2 and UTF-8) CMap files, at the +following URL: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/adobe/ + +A sample (Adobe-Korea1-0) CIDFont is also available at the above URL. + There is also a somewhat brief description of CID-keyed fonts +at the end of Chapter 6 in UJIP. + + +7.10: NJWIN + + Hongbo Data Systems has recently release a ShareWare ($49 USD) +product called NJWIN whose purpose is to force the display of CJK text +in non-CJK applications running under US Windows 95. Actually, there +are two versions: full CJK and Japanese only. + NJWIN and its full description are available at the following +URL: + + http://www.njstar.com.au/njstar/njwin.htm + +Other (popular) URLs that carry NJWIN are as follows: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/windows/ + ftp://ftp.cc.monash.edu.au/pub/nihongo/ + + Hongbo Data Systems' e-mail address is: + + hongbo@njstar.com.au + +Their WWW Home Page is at the following URL: + + http://www.njstar.com.au/ + + +PART 8: CJK PROGRAMMING ISSUES + + This new section describes issues related to using specific +programming languages to process CJK text. + + +8.1: C AND C++ + + At one time I used C on a regular basis for my CJK programming +needs, and released three tools for others to use: JConv, JChar, and +JCode. While these tools are specific to Japanese, they can be easily +adapted for CJK use. Their source code is available at the following +URL: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/src/ + + I also provided several C code snippets in Chapter 7 of +UJIP. These are available in machine-readable form at the following +URL: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/Ch7/ + + +8.2: PERL + + Although Perl does not have any special CJK facilities (note +that most implementations of C and C++ do not either), it provides a +powerful programming environment that is useful for many CJK-related +tasks. + The noteworthy features of Perl are associative arrays and +regular expressions. These are features not found in C or C++, and +allow one to write meaningful code in little time. + JPerl is an implementation of Perl that provides two-byte +support for Japanese (EUC or Shift-JIS encoding). It is not ideal +because JPerl scripts often cannot run under (non-Japanese) Perl. + If you often write programs for internal use, I suggest that +you check out Perl to see if it can offer you something. Chances are +that it can. A good place to start looking at Perl are through books +on the subject (see Section A.3.1) and at the following URL: + + http://www.perl.com/ + + For those who like additional reading, "The Perl Journal" is +starting up, and information is at the following URL: + + http://work.media.mit.edu/the_perl_journal/ + + +8.3: JAVA + + I am just starting to learn about the Java programming +language (and rightly so since my wife is Javanese!). It seems to have +a lot to offer. + The most interesting aspects of Java are: + +o Built-in support for Unicode and UTF-8. +o The programmer must write code in the object-oriented paradigm. +o Provides a portable way to supply compiled code. +o Security features for Internet use. + +More information on Java are at the following URLs: + + http://www.gamelan.com/ + http://www.javasoft.com/ + +Oh, Gamelan is the name of Javanese music. + Of the books about Java published thus far, the one I consider +to be the best is "Java in a Nutshell" by David Flanagan. + One programming feature of Perl that I dearly miss in Java are +regexes (regular expressions). Luckily, some kind person wrote a regex +package for Java based on Perl regexes. Information on this Java regex +package is available at the following URL: + + http://www.win.net/~stevesoft/pat/ + + +A FINAL NOTE + + I hope that the information presented here will prove +useful. I would like to keep the electronic version of this document +as up-to-date as possible, and through readers' input, I am able to +do so. + Many readers will notice that I am very heavy into UNIX and +Macintosh (well, I recently got my first PC). If anyone has any +information on CJK-capable interfaces for other platforms, please feel +free to send it to me, and I will be sure to include it in the next +version of CJK.INF. Please include sources for the software or +documentation by providing addresses, phone numbers, FTP sites, and so +on. + Please do not hesitate to ask me further question concerning +any subject presented in this document. + + +ACKNOWLEDGMENTS + + I would like to express my deepest thanks to Kazumasa Utashiro +of Internet Initiative Japan (IIJ). He taught to me how to send and +receive Japanese text using the 7-bit ISO 2022 codes back in 1989. +With his help I was able to write JAPAN.INF, my book, and this +document in order to inform others about what he has taught me plus +more. + Next, I thank all the folks at O'Reilly & Associates for +publishing UJIP. Special thanks to Tim O'Reilly for accepting the book +proposal, and to Peter Mui for guiding me through the process. I have +had nothing but good experiences with "them there fine folks." + I got to know Jack Halpern through UJIP, and he subsequently +translated it into Japanese. Many thanks to him. + I am also grateful to my employer, Adobe Systems, for letting +me work on interesting CJK-related projects. I really like what I do +here. In particular, I want to thank Dan Mills, my manager, for +putting up with me for these past four years. + Lastly, I would also like to thank the countless people who +provided comments on JAPAN.INF, UJIP, and CJK.INF. I hope that this +new document lives up to the spirit of my previous efforts. + + +APPENDIX A: OTHER INFORMATION SOURCES + + One of the most useful types of information are pointers to +other information sources. This appendix provides just that. + + +A.1: USENET NEWSGROUPS AND MAILING LISTS + + Appendix L of UJIP provided information on a number of mailing +lists. This section supplements that appendix with information on +other useful mailing lists, and points out which ones in UJIP are +relevant to readers of CJK.INF. + + +A.1.1: USENET NEWSGROUPS + + The following Usenet Newsgroups typically have postings with +information relevant to issues discussed in CJK.INF (in alphabetical +order): + + alt.chinese.computing + alt.chinese.text (HZ encoding used for Chinese text) + alt.chinese.text.big5 (Big Five encoding used for Chinese text) + alt.japanese.text (JIS encoding used for Japanese text) + chinese.flame (UTF-7) + chinese.text.unicode (UTF-8) + comp.lang.c + comp.lang.c++ + comp.lang.java + comp.lang.perl.misc + comp.software.international + comp.std.internat + fj.editor.mule (JIS encoding used for Japanese text) + fj.kanji (JIS encoding used for Japanese text) + fj.net.infosystems.www.browsers (JIS encoding used for Japanese text) + fj.news.reader (JIS encoding used for Japanese text) + han.comp.hangul + han.sys.mac + sci.lang.japan (JIS encoding used for Japanese text) + + If your local news host does not provide a feed of the fj.* +newsgroups (shame on them!), or if you do not have access to Usenet +News, you can alternatively fetch them from the following URL: + + ftp://kuso.shef.ac.uk/pub/News/ + +The subdirectories correspond to the newsgroup name, but with the +"dots" being replaced by "slashes." For example, the "fj.binaries.mac" +newsgroup is archived in the "fj/binaries/mac" subdirectory. Many +thanks to Earl Kinmonth (jp1ek@sunc.shef.uc.uk) for this service. + There are some sites that carry full feeds of the fj.* +newsgroups, and permit public access (meaning that you can configure +your news reader to point to it). The only one I know of thus far is +as follows: + + ume.cc.tsukuba.ac.jp + + +A.1.2: MAILING LISTS + + The following are mailing lists that should interest readers +of this document (some are more active than others). The first line +after each entry indicates the address (or addresses) that can be used +for subscribing. The second line is the address for posting. + +o CCNET-L MAILING LIST + listserv@uga.uga.edu (or listserv@uga) + ccnet-l@uga.uga.edu + +o China Net Mailing List + majordomo@lists.mindspring.com + (See http://www.asia-net.com/ or jobs@asia-net.com) + +o EASUG (East Asian Software Users Group) Mailing List + easug-request@guvax.acc.georgetown.edu + easug@guvax.acc.georgetown.edu + +o EBTI-L (Electronic Buddhist Text Initiative) Mailing List + ebti-l-request@uxmail.ust.hk + ebti-l@uxmail.ust.hk + +o EFJ (Electronic Frontiers Japan) Mailing List + majordomo@lists.twics.com + efj@lists.twics.com + +o Hangul Mailing List (han.comp.hangul newsgroup) + majordomo@cair.kaist.ac.kr + hangul@cair.kaist.ac.kr + +o INSOFT-L Mailing List + majordomo@trans2.b30.ingr.com + insoft-l@trans2.b30 + +o ISO 10646 Mailing List + listproc@listproc.hcf.jhu.edu + iso10646@listproc.hcf.jhu.edu + +o Japan Net Mailing List + majordomo@lists.mindspring.com + (See http://www.asia-net.com/ or jobs@asia-net.com) + +o KanjiTalk Mailing List + kanjitalk-request@cs15.atr-sw.atr.co.jp (or kanjitalk-request@crl.go.jp) + kanjitalk@cs15.atr-sw.atr.co.jp (or kanjitalk@crl.go.jp) + +o Mac Mailing List (han.sys.mac newsgroup) + majordomo@krnic.net + mac@krnic.net + +o Mule Mailing List + mule-request@etl.go.jp + mule@etl.go.jp or mule-jp@etl.go.jp + +o NIHONGO Mailing List (sci.lang.japan newsgroup) + listserv@mitvma.mit.edu (or listserv@mitvma) + nihongo@mitvma.mit.edu + +o Nihongo-Hiroba Mailing List + listproc@mcfeeley.cc.utexas.edu + nihongo-hiroba@mcfeeley.cc.utexas.edu + +o Nisus Mailing List + listserv@dartmouth.edu + nisus@dartmouth.edu + +o TLUG (Tokyo Linux User's Group) Mailing List + majordomo@lists.twics.com + tlug@lists.twics.com + +o Unicode Mailing List + unicode-request@unicode.org + unicode@unicode.org + +o WNN User Mailing List + wnn-user-request@wnn.astem.or.jp + wnn-user-jp@wnn.astem.or.jp + +o WWW Multilingual Mailing List + www-mling-request@square.ntt.jp + www-mling@square.ntt.jp + +If the name of the mailing list is part of the subscription address +(such as "easug-request"), the message body should look like this: + + subscribe + +Including your name is optional. If username in the subscription +address is "listserv" or "majordomo" (these are names of mailing list +managing software), the mailing list name must appear after +"subscribe" in the message body as follows: + + subscribe ccnet-l + +Again, including your name is optional. + The following URL has information about Japanese-related +mailing lists: + + gopher://gan1.ncc.go.jp/11/INFO/mail-lists/ + + +A.2: INTERNET RESOURCES + + The Internet provides what I would consider to be the greatest +information resources of all. These can be subcategorized into FTP, +Telnet, Gopher, WWW, and e-mail. + + +A.2.1: USEFUL FTP SITES + + Below are the URLs for useful FTP sites. The directory +specified is the recommended place from which to start poking around +for useful files. + + ftp://cair-archive.kaist.ac.kr/pub/hangul/ + ftp://etlport.etl.go.jp/pub/mule/ + ftp://ftp.adobe.com/pub/adobe/ + ftp://ftp.cc.monash.edu.au/pub/nihongo/ + ftp://ftp.ifcss.org/pub/software/ + ftp://ftp.ora.com/pub/examples/nutshell/ujip/ + ftp://ftp.sra.co.jp/pub/ + ftp://ftp.uwtc.washington.edu/pub/Japanese/ + ftp://kuso.shef.ac.uk/pub/Japanese/ + ftp://unicode.org/pub/ + +This list is expected to grow. + + +A.2.2: USEFUL TELNET SITES + + For those who have a NIFTY-Serve account, there is now a very +convenient way to access NIFTY-Serve using telnet. The URL is as +follows: + + telnet://r2.niftyserve.or.jp/ + +Information about what NIFTY-Serve has to offer (and how to subscribe) +can be found at the following URL: + + http://www.nifty.co.jp/ + + Another information service with a similar access mechanism is +CompuServe, whose URL is as follows: + + telnet://compuserve.com/ + +You will need to press the return key to get the "Host Name:" prompt, +at which time you type "cis" (just follow the menus from this point +on). + You can also do a search on fj.* newsgroup articles at the +following URL: + + telnet://asahi-net.or.jp/ + +You login as "fj-db" once you are connected. + + +A.2.3: USEFUL GOPHER SITES + + I am not too much of a Gopher user. There, of course, is the +following: + + gopher://gopher.ora.com/ + +Another Gopher site provides information on Japanese-related mailing +lists: + + gopher://gan1.ncc.go.jp/11/INFO/mail-lists/ + +If you happen to know of others, please let me know. + + +A.2.4: USEFUL WWW SITES + + Because the World-Wide Web is a constantly changing place (and +more importantly, because I don't want to re-issue a new version of +this document every month!), I will maintain links to useful documents +at my WWW Home Page. Its URL is as follows: + + http://jasper.ora.com/lunde/ + +If you cannot get to my WWW Home Page, you couldn't get to any that I +would list here anyway. + + +A.2.5: USEFUL MAIL SERVERS + + In the past (that is, in JAPAN.INF) I included a full list of +the domains in the "jp" hierarchy. That took up a lot of space, and +changes very rapidly. You can now send a request to a mail server in +order to return the most current listing. The mail server is: + + mail-server@nic.ad.jp + +The most common command is "send," and the following arguments can be +supplied to retrieve specific documents (and should be in the message +body, not on the "Subject:" line): + + send help + send index + send jpnic/domain-list.txt + send jpnic/domain-list-e.txt + +The first sends back a help file, the second sends back a complete +index of files that can be retrieved (use this one to see what other +useful stuff is available), and the last two send back a complete +listing of domains in the "fj" hierarchy (the last one send it back in +English/romanized). + + +A.3: OTHER RESOURCES + + This section provides pointers to specific documentation +available electronically or in print. + + +A.3.1: BOOKS + + There are other useful reference materials available in print +or online, in addition to the various national and international +standards mentioned throughout this document. The following are books +that I recommend for further reading or mental stimulus. (Sorry for +plugging my own books in this list, but they are relevant.) + +o Clews, John. "Language Automation Worldwide: The Development of + Character Set Standards." SESAME Computer Projects. 1988. ISBN + 1-870095-01-4. + +o Flanagan, David. "Java in a Nutshell." O'Reilly & Associates, + Inc. 1996. ISBN 1-56592-183-6. + +o Frisch, AEleen. "Essential System Administration." Second Edition. + O'Reilly & Associates, Inc. 1995. ISBN 1-56592-127-5. + +o Huang, Jack & Timothy Huang. "An Introduction to Chinese, Japanese + and Korean Computing." World Scientific Computing. 1989. ISBN + 9971-50-664-5. + +o IBM Corporation. "Character Data Representation Architecture - Level + 2, Registry." 1993. IBM order number SC09-1391-01. + +o Kano, Nadine. "Developing International Software for Windows 95 and + Windows NT." Microsoft Press. 1995. ISBN 1-55615-840-8. + +o Kirch, Olaf. "Linux Network Administrator's Guide." O'Reilly & + Associates, Inc. 1995. ISBN 1-56592-087-2. + +o Kissell, Joe. "The Nisus Way." MIS:Press. 1996. ISBN 1-55828-455-9. + +o Krol, Ed. "The Whole Internet User's Guide & Catalog." Second + Edition. O'Reilly & Associates, Inc. 1994. ISBN 1-56592-063-5. + +o Liu, Cricket et al. "Managing Internet Information Services." + O'Reilly & Associates, Inc. 1994. ISBN 1-56592-062-7. + +o Lunde, Ken. "Understanding Japanese Information Processing." + O'Reilly & Associates, Incorporated. 1993. ISBN 1-56592-043-0. LCCN + PL524.5.L86 1993. + +o Lunde, Ken. "Nihongo Joho Shori." SOFTBANK Corporation. 1995. ISBN + 4-89052-708-7. + +o Luong, Tuoc V. et al. "Internationalization: Developing Software for + Global Markets." John Wiley & Sons, Incorporated. 1995. ISBN + 0-471-07661-9. + +o Schwartz, Randal L. "Learning Perl." O'Reilly & Associates, + Incorporated. 1993. ISBN 1-56592-042-2. + +o Stallman, Richard M. "GNU Emacs Manual." Tenth edition. Free + Software Foundation. 1994. ISBN 1-882114-04-3. + +o Tuthill, Bill. "Solaris International Developer's Guide." SunSoft + Press and PTR Prentice Hall. 1993. ISBN 0-13-031063-8. + +o Unicode Consortium, The. "The Unicode Standard: Worldwide Character + Encoding." Version 1.0. Volume 2. Addison-Wesley. 1992. ISBN + 0-201-60845-6. + +o Vromans, Johan. "Perl 5 Desktop Reference." O'Reilly & Associates, + Inc. 1996. ISBN 1-56592-187-9. + +o Wall, Larry & Randal L. Schwartz. "Programming Perl." O'Reilly & + Associates, Incorporated. 1991. ISBN 0-937175-64-1. + +o Welsh, Matt & Lar Kaufman. "Running Linux." O'Reilly & Associates, + Inc. 1995. ISBN 1-56592-100-3. + + If you want to get your hands on any of the national or +international standards mentioned in this document, I suggest the +following: + +o The American National Standards Institute can provide ISO, KS, and + JIS standards. Bear in mind that ISO standards will most likely + arrive as a photocopy of the original. + + ANSI + 11 West 42nd Street + New York, NY 10036 + USA + +1-212-642-4900 (phone) + +1-212-302-1286 (facsimile) + +o The International Organization for Standardization can provide + ISO standards. + + ISO + 1, rue de Varemb + Case postale 56 + CH-1211, Geneva 20 + SWITZERLAND + +41-22-749-01-11 (phone) + +41-22-733-34-30 (facsimile) + central@isocs.iso.ch (e-mail) + http://www.iso.ch/ (WWW) + +o Chinese (GB and CNS) standards are the hardest to obtain. It is + quite unfortunate. + + +A.3.2: MAGAZINES + +o "Computing Japan," published monthly, ISSN 1340-7228, + editors@cj.gol.com. + +o "MANGAJIN," published 10 times per year, ISSN 1051-8177. + +o "Multilingual Communications & Computing," published bi-monthly, + ISSN 1065-7657, info@multilingual.com. + +o "The Perl Journal," published quarterly, ISSN 1087-903X, + perl-journal-subscriptions@perl.com. + + +A.3.3: JOURNALS + +o "Chinese Information Processing" (CIP), published bi-monthly, ISSN + 1003-9082. (In Chinese.) + +o "Computer Processing of Chinese & Oriental Languages" (CPCOL), + co-published twice a year by World Scientific Publishing and Chinese + Language Computer Society (CLCS), ISSN 0715-9048. + +o "The Electronic Bodhidharma," published by the International + Research Institute for Zen (IRIZ) Buddhism, Hanazono University, + Japan. More information on the organization that publishes this + journal is available at the following URL: + + http://www.iijnet.or.jp/iriz/irizhtml/irizhome.htm + + +A.3.4: RFCs + + Many RFCs (Request For Comments) are relevant to this +document. They are: + +o RFC 1341: "MIME (Multipurpose Internet Mail Extensions): Mechanisms + for Specifying and Describing the Format of Internet Message + Bodies," by Nathaniel Borenstein and Ned Freed, June 1992. + +o RFC 1342: "Representation of Non-ASCII Text in Internet Message + Headers," by Keith Moore, June 1992. + +o RFC 1468: "Japanese Character Encoding for Internet Messages," by + Jun Murai et al., June 1993. + +o RFC 1521: "MIME (Multipurpose Internet Mail Extensions) Part One: + Mechanisms for Specifying and Describing the Format of Internet + Message Bodies," by Nathaniel Borenstein and Ned Freed, September + 1993. Obsoletes RFC 1341. + +o RFC 1522: "MIME (Multipurpose Internet Mail Extensions) Part Two: + Message Header Extensions for Non-ASCII Text," by Keith Moore, + September 1993. Obsoletes RFC 1342. + +o RFC 1554: "ISO-2022-JP-2: Multilingual Extension of ISO-2022-JP," by + Masataka Ohta and Kenichi Handa, December 1993. + +o RFC 1557: "Korean Character Encoding for Internet Messages," by + Uhhyung Choi et al., December 1993. + +o RFC 1642: "UTF-7: A Mail-Safe Transformation Format of Unicode," by + David Goldsmith and Mark Davis, July 1994. + +o RFC 1815: "Character Sets ISO-10646 and ISO-10646-J-1," by Masataka + Ohta, July 1995. + +o RFC 1842: "ASCII Printable Characters-Based Chinese Character + Encoding for Internet Messages," by Ya-Gui Wei et al., August 1995. + +o RFC 1843: "HZ - A Data Format for Exchanging Files of Arbitrarily + Mixed Chinese and ASCII Characters," by Fung Fung Lee, August 1995. + +o RFC 1922: "Chinese Character Encoding for Internet Messages," by + Haifeng Zhu et al., March 1996. + +These RFCs can be obtained from FTP archives that contain all RFC +documents, such as at the following URLs + + ftp://nic.ddn.mil/rfc/ + ftp://ftp.uu.net/inet/rfc/ + +But these specific ones are mirrored at the following URL for +convenience: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/Ch9/ + + +A.3.5: FAQs + + There are several FAQ (Frequently Asked Questions) files that +provide useful information. The following is a listing of some along +with their URLs: + +o "Japanese Language Information" FAQ (formerly the "sci.lang.japan" + FAQ) by Rafael Santos (santos@mickey.ai.kyutech.ac.jp) at: + + http://www.mickey.ai.kyutech.ac.jp/cgi-bin/japanese/ + + Update announcements are usually posted to the sci.lang.japan + newsgroup. + +o "Programming for Internationalization" FAQ by Michael Gschwind + (mike@vlsivie.tuwien.ac.at) at: + + ftp://ftp.vlsivie.tuwien.ac.at/pub/8bit/ISO-programming + + Also posted to the comp.software.international newsgroup. This and + other internationalization documents are also accessible through the + following URL: + + http://www.vlsivie.tuwien.ac.at/mike/i18n.html + +o Three FAQs about Internet Service Providers in Japan by Taki Naruto + (tn@panix.com), Jesse Casman (jcasman@unm.edu), and Kenji Yoshida + (kenny@mb.tokyo.infoweb.or.jp), respectively, at: + + http://www.panix.com/~tn/ispj.html + http://nobunaga.unm.edu/internet.html + http://cswww2.essex.ac.uk/users/whean/japan/net.html + +o "Internationalization Reference List" by Eugene Dorr + (gdorr@pgh.legent.com) at: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/doc/i18n-books.txt + + Note really a FAQ, but quite useful because it is a very complete + listing of I18N-related books. + +o "INSOFT-L Service" by Brian Tatro (btatro@tatro.com) at: + + http://iquest.com/~btatro/in2.html + + This includes a link to the FAQ for the INSOFT-L Mailing List (see + Section A.1.2). + +o "How to Use Japanese on the Internet with a PC: From Login to WWW" + by Hideki Hirayama (sgw01623@niftyserve.or.jp) at: + + ftp://ftp.ora.com/pub/examples/nutshell/ujip/faq/jpn-inet.FAQ + +o "Hangul and Internet in Korea" FAQ by Jungshik Shin + (jshin@minerva.cis.yale.edu) at: + + http://pantheon.cis.yale.edu/~jshin/faq/ +--- END (CJK.INF VERSION 2.1 07/12/96) 185553 BYTES --- |