summaryrefslogtreecommitdiffstats
path: root/Objects/unicodetype_db.h
diff options
context:
space:
mode:
authorEzio Melotti <ezio.melotti@gmail.com>2010-06-05 17:51:07 (GMT)
committerEzio Melotti <ezio.melotti@gmail.com>2010-06-05 17:51:07 (GMT)
commite57e50c8e77bc64e1ebab7a9ddf6f13fc3440c48 (patch)
tree3ff3d99d6b043463440cc7252747c88e128b0547 /Objects/unicodetype_db.h
parent3b818bfbfa8057f1163cb4665bf84b2cf230bb04 (diff)
downloadcpython-e57e50c8e77bc64e1ebab7a9ddf6f13fc3440c48.zip
cpython-e57e50c8e77bc64e1ebab7a9ddf6f13fc3440c48.tar.gz
cpython-e57e50c8e77bc64e1ebab7a9ddf6f13fc3440c48.tar.bz2
Update PyUnicode_DecodeUTF8 from RFC 2279 to RFC 3629.
1) #8271: when a byte sequence is invalid, only the start byte and all the valid continuation bytes are now replaced by U+FFFD, instead of replacing the number of bytes specified by the start byte. See http://www.unicode.org/versions/Unicode5.2.0/ch03.pdf (pages 94-95); 2) 5- and 6-bytes-long UTF-8 sequences are now considered invalid (no changes in behavior); 3) Add code and tests to reject surrogates (U+D800-U+DFFF) as defined in RFC 3629, but leave it commented out since it's not backward compatible; 4) Change the error messages "unexpected code byte" to "invalid start byte" and "invalid data" to "invalid continuation byte"; 5) Add an extensive set of tests in test_unicode; 6) Fix test_codeccallbacks because it was failing after this change.
Diffstat (limited to 'Objects/unicodetype_db.h')
0 files changed, 0 insertions, 0 deletions