summaryrefslogtreecommitdiffstats
path: root/Lib/test/test_codecs.py
Commit message (Collapse)AuthorAgeFilesLines
* Backport revision 42872:Walter Dörwald2006-03-061-5/+9
| | | | | If size is specified, try to read at least size characters. This is a alternative version of patch #1379332.
* Backport checkin:Walter Dörwald2005-08-301-1/+50
| | | | | | SF bug #1251300: On UCS-4 builds the "unicode-internal" codec will now complain about illegal code points. The codec now supports PEP 293 style error handlers. (This is a variant of the patch by Nik Haldimann that detects truncated data)
* Make IDNA return an empty string when the input is empty. Fixes #1163178.Martin v. Löwis2005-08-251-0/+6
|
* Backport checkin:Walter Dörwald2005-07-201-1/+18
| | | | | | Make attributes and local variables in the StreamReader str objects instead of unicode objects, so that codecs that do a str->str decoding won't promote the result to unicode. This fixes SF bug #1241507.
* Backport checkin (and the appropriate fix to the test):Walter Dörwald2005-04-211-0/+2
| | | | | | | | | | | | If the data read from the bytestream in readline() ends in a '\r' read one more byte, even if the user has passed a size parameter. This extra byte shouldn't cause a buffer overflow in the tokenizer. The original plan was to return a line ending in '\r', which might be recognizable as a complete line and skip any '\n' that was read afterwards. Unfortunately this didn't work, as the tokenizer only recognizes '\n' as line ends, which in turn lead to joined lines and SyntaxErrors, so this special treatment of a split '\r\n' has been dropped. (It can only happen with a temporarily exhausted bytestream now anyway.) Fixes parts of SF bugs #1163244 and #1175396.
* Backport checkin:Walter Dörwald2005-04-041-9/+82
| | | | | | | | | | | | | Fix for SF bug #1175396: readline() will now read one more character, if the last character read is "\r" (and size is None, i.e. we're allowed to call read() multiple times), so that we can return the correct line ending (this additional character might be a "\n"). If the stream is temporarily exhausted, we might return the wrong line ending (if the last character read is "\r" and the next one (after the byte stream provides more data) is "\n", but at least the atcr member ensures that we get the correct number of lines (i.e. this "\n" will not be treated as another line ending).
* Backport checkin:Walter Dörwald2005-03-141-0/+11
| | | | Reset internal buffers when seek() is called. This fixes SF bug #1156259.
* Backport checkin:Walter Dörwald2005-01-101-0/+30
| | | | Fix and test for SF bug #1098990: codec readline() splits lines apart.
* Backport checkin:Walter Dörwald2004-12-211-13/+80
| | | | | | | | | | | | | | | The changes to the stateful codecs in 2.4 resulted in StreamReader.readline() trying to return a complete line even if a size parameter was given (see http://www.python.org/sf/1076985). This leads to buffer overflows with long source lines under Windows if e.g. cp1252 is used as the source encoding. This patch reverts the behaviour of readline() to something that behaves more like Python 2.3: If a size parameter is given, read() is called only once. As a side effect of this, readline() now supports all types of linebreaks supported by unicode.splitlines(). Note that the tokenizer is still broken and it's possible to provoke segfaults (see http://www.python.org/sf/1089395).
* Trigger a few error cases in Modules/_codecsmodule.c.Walter Dörwald2004-10-281-0/+14
|
* SF #1048865: Fix a trivial typo that breaks StreamReader.readlines()Hye-Shik Chang2004-10-171-1/+12
|
* SF patch #998993: The UTF-8 and the UTF-16 stateful decoders now supportWalter Dörwald2004-09-071-1/+117
| | | | | | | | | | | decoding incomplete input (when the input stream is temporarily exhausted). codecs.StreamReader now implements buffering, which enables proper readline support for the UTF-16 decoders. codecs.StreamReader.read() has a new argument chars which specifies the number of characters to return. codecs.StreamReader.readline() and codecs.StreamReader.readlines() have a new argument keepends. Trailing "\n"s will be stripped from the lines if keepends is false. Added C APIs PyUnicode_DecodeUTF8Stateful and PyUnicode_DecodeUTF16Stateful.
* Add generic codecs.encode() and .decode() APIs that don't imposeMarc-André Lemburg2004-07-101-1/+11
| | | | any restriction on the return type (like unicode.encode() et al. do).
* Whitespace normalization.Tim Peters2004-07-081-1/+1
|
* Add test case for unicode(somestring, "idna").Martin v. Löwis2004-03-241-1/+6
|
* Combine the functionality of test_support.run_unittest()Walter Dörwald2003-05-011-7/+7
| | | | | | | | | | and test_support.run_classtests() into run_unittest() and use it wherever possible. Also don't use "from test.test_support import ...", but "from test import test_support" in a few spots. From SF patch #662807.
* Whitespace normalization.Tim Peters2003-04-241-2/+2
|
* Skip nameprep test 3.43, as we do allow unassigned characters. The testMartin v. Löwis2003-04-181-2/+4
| | | | fails only in UCS-2 mode, since it tests a non-BMP character.
* Implement IDNA (Internationalized Domain Names in Applications).Martin v. Löwis2003-04-181-0/+296
|
* Fix for [ 543344 ] Interpreter crashes when recoding; suggestedMarc-André Lemburg2003-02-041-0/+9
| | | | | | by Michael Stone (mbrierst). Python 2.1.4, 2.2.2 candidate.
* Check whether a string resize is necessary at the endWalter Dörwald2002-09-031-1/+8
| | | | | | | | | of PyString_DecodeEscape(). This prevents a call to _PyString_Resize() for the empty string, which would result in a PyErr_BadInternalCall(), because the empty string has more than one reference. This closes SF bug http://www.python.org/sf/603937
* Get rid of relative imports in all unittests. Now anything thatBarry Warsaw2002-07-231-1/+2
| | | | | | | | | | | imports e.g. test_support must do so using an absolute package name such as "import test.test_support" or "from test import test_support". This also updates the README in Lib/test, and gets rid of the duplicate data dirctory in Lib/test/data (replaced by Lib/email/test/data). Now Tim and Jack can have at it. :)
* Change the PyUnit-based tests to use the test_main() approach. ThisFred Drake2001-09-201-1/+7
| | | | | allows using the tests with unittest.py as a script. The tests will still run when run as a script themselves.
* Test by Martin v. Loewis for the new UTF-16 codec handling of BOMMarc-André Lemburg2001-06-191-0/+25
marks.
generated by cgit v0.12 at 2025-12-27 11:18:42 (GMT)