summaryrefslogtreecommitdiffstats
path: root/Lib/test/test_email
Commit message (Collapse)AuthorAgeFilesLines
* bpo-45001: Make email date parsing more robust against malformed input ↵Miss Islington (bot)2021-08-261-0/+2
| | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | (GH-27946) (GH-27974) Various date parsing utilities in the email module, such as email.utils.parsedate(), are supposed to gracefully handle invalid input, typically by raising an appropriate exception or by returning None. The internal email._parseaddr._parsedate_tz() helper used by some of these date parsing routines tries to be robust against malformed input, but unfortunately it can still crash ungracefully when a non-empty but whitespace-only input is passed. This manifests as an unexpected IndexError. In practice, this can happen when parsing an email with only a newline inside a ‘Date:’ header, which unfortunately happens occasionally in the real world. Here's a minimal example: $ python Python 3.9.6 (default, Jun 30 2021, 10:22:16) [GCC 11.1.0] on linux Type "help", "copyright", "credits" or "license" for more information. >>> import email.utils >>> email.utils.parsedate('foo') >>> email.utils.parsedate(' ') Traceback (most recent call last): File "<stdin>", line 1, in <module> File "/usr/lib/python3.9/email/_parseaddr.py", line 176, in parsedate t = parsedate_tz(data) File "/usr/lib/python3.9/email/_parseaddr.py", line 50, in parsedate_tz res = _parsedate_tz(data) File "/usr/lib/python3.9/email/_parseaddr.py", line 72, in _parsedate_tz if data[0].endswith(',') or data[0].lower() in _daynames: IndexError: list index out of range The fix is rather straight-forward: guard against empty lists, after splitting on whitespace, but before accessing the first element. (cherry picked from commit 989f6a3800f06b2bd31cfef7c3269a443ad94fac) Co-authored-by: wouter bolsterlee <wouter@bolsterl.ee>
* bpo-27321 Fix email.generator.py to not replace a non-existent header. ↵Miss Skeleton (bot)2020-10-191-0/+35
| | | | | | | | | | | | | | (GH-18074) This PR replaces GH-1977. The reason for the replacement is two-fold. The fix itself is different is that if the CTE header doesn't exist in the original message, it is inserted. This is important because the new CTE could be quoted-printable whereas the original is implicit 8bit. Also the tests are different. The test_nonascii_as_string_without_cte test in GH-1977 doesn't actually test the issue in that it passes without the fix. The test_nonascii_as_string_without_content_type_and_cte test is improved here, and even though it doesn't fail without the fix, it is included for completeness. Automerge-Triggered-By: @warsaw (cherry picked from commit bf838227c35212709dc43b3c3c57f8e1655c1d24) Co-authored-by: Mark Sapiro <mark@msapiro.net>
* bpo-40597: Allow email.contextmanager set_content() to set a null string. ↵Miss Islington (bot)2020-07-081-0/+13
| | | | | | | (GH-20542) (cherry picked from commit 4fa61a7732923f92de0f7830c12da48c4cec937f) Co-authored-by: Mark Sapiro <mark@msapiro.net>
* bpo-39040: Fix parsing of email mime headers with whitespace between ↵Miss Islington (bot)2020-05-291-0/+19
| | | | | | | | | | | | | | | | | | encoded-words. (gh-17620) * bpo-39040: Fix parsing of email headers with encoded-words inside a quoted string. It is fairly common to find malformed mime headers (especially content-disposition headers) where the parameter values, instead of being encoded to RFC standards, are "encoded" by doing RFC 2047 "encoded word" encoding, and then enclosing the whole thing in quotes. The processing of these malformed headers was incorrectly leaving the spaces between encoded words in the decoded text (whitespace between adjacent encoded words is supposed to be stripped on decoding). This changeset fixes the encoded word processing inside quoted strings (bare-quoted-string) to do correct RFC 2047 decoding by stripping that whitespace. (cherry picked from commit 21017ed904f734be9f195ae1274eb81426a9e776) Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
* bpo-39073: validate Address parts to disallow CRLF (GH-19007)Miss Islington (bot)2020-05-271-0/+19
| | | | | | Disallow CR or LF in email.headerregistry.Address arguments to guard against header injection attacks. (cherry picked from commit 614f17211c5fc0e5b828be1d3320661d1038fe8f) Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com>
* bpo-40597: email: Use CTE if lines are longer than max_line_length ↵Miss Islington (bot)2020-05-181-0/+15
| | | | | | | | | | | | | | | | | | consistently (gh-20038) (gh-20084) raw_data_manager (default for EmailPolicy, EmailMessage) does correct wrapping of 'text' parts as long as the message contains characters outside of 7bit US-ASCII set: base64 or qp Content-Transfer-Encoding is applied if the lines would be too long without it. It did not, however, do this for ascii-only text, which could result in lines that were longer than policy.max_line_length or even the rfc 998 maximum. This changeset fixes the heuristic so that if lines are longer than policy.max_line_length, it will always apply a content-transfer-encoding so that the lines are wrapped correctly. (cherry picked from commit 6f2f475d5a2cd7675dce844f3af436ba919ef92b) Co-authored-by: Arkadiusz Hiler <arek.l1@gmail.com>
* bpo-39793: use the same domain on make_msgid tests (GH-18698) (GH-19554)Miss Islington (bot)2020-04-161-4/+6
| | | | | | (cherry picked from commit 5565c30f0b25996a0e73477fc0e1e1aced52b926) Co-authored-by: Batuhan Taşkaya <batuhanosmantaskaya@gmail.com>
* bpo-38708: email: Fix a potential IndexError when parsing Message-ID (GH-17504)Miss Islington (bot)2019-12-091-0/+6
| | | | | | | Fix a potential IndexError when passing an empty value to the message-id parser. Instead, HeaderParseError should be raised. (cherry picked from commit 3ae4ea1931361dd2743e464790e739d9285501bf) Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
* bpo-38698: Add a new InvalidMessageID token to email header parser. (GH-17503)Miss Islington (bot)2019-12-091-3/+37
| | | | | | | This adds a new InvalidMessageID token to the email header parser which can be used to represent invalid message-id headers in the parse tree. (cherry picked from commit 68157da8b42b26408af5d157d2dba4fcf29c6320) Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
* bpo-38698: Prevent UnboundLocalError to pop up in parse_message_id (GH-17277)Miss Islington (bot)2019-12-051-0/+6
| | | | | | | | | parse_message_id() was improperly using a token defined inside an exception handler, which was raising `UnboundLocalError` on parsing an invalid value. https://bugs.python.org/issue38698 (cherry picked from commit bb815499af855b1759c02535f8d7a9d0358e74e8) Co-authored-by: Claudiu Popa <pcmanticore@gmail.com>
* bpo-38332: Catch KeyError from unknown cte in encoded-word. (GH-16503)Miss Islington (bot)2019-10-122-0/+14
| | | | | | KeyError should cause a failure in parsing the encoded word and should be caught and raised as a _InvalidEWError instead. (cherry picked from commit 65dcc8a8dc41d3453fd6b987073a5f1b30c5c0fd) Co-authored-by: Andrei Troie <andreitroie90@gmail.com>
* [3.8] bpo-37764: Fix infinite loop when parsing unstructured email headers. ↵Abhilash Raj2019-09-052-0/+37
| | | | | | | | | | | | | | | | (GH-15239) (GH-15686) Fixes a case in which email._header_value_parser.get_unstructured hangs the system for some invalid headers. This covers the cases in which the header contains either: - a case without trailing whitespace - an invalid encoded word https://bugs.python.org/issue37764 This fix should also be backported to 3.7 and 3.8 https://bugs.python.org/issue37764 (cherry picked from commit c5b242f87f31286ad38991bc3868cf4cfbf2b681) Co-authored-by: Ashwin Ramaswami <aramaswamis@gmail.com>
* [3.8] bpo-21315: Fix parsing of encoded words with missing leading ws ↵Ashwin Ramaswami2019-09-032-3/+24
| | | | | | | | | | | | | | | | | | | | | | | | | | (GH-13425) (GH-15655) * [bpo-21315](https://bugs.python.org/issue21315): Fix parsing of encoded words with missing leading ws. Because of missing leading whitespace, encoded word would get parsed as unstructured token. This patch fixes that by looking for encoded words when splitting tokens with whitespace. Missing trailing whitespace around encoded word now register a defect instead. Original patch suggestion by David R. Murray on [bpo-21315](https://bugs.python.org/issue21315). (cherry picked from commit 66c4f3f38b867d8329b28c032bb907fd1a2f22d2) Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com> (cherry picked from commit dc20fc4311dece19488299a7cd11317ffbe4d3c3) Co-authored-by: Miss Islington (bot) <31488909+miss-islington@users.noreply.github.com> https://bugs.python.org/issue21315
* [3.8] bpo-37482: Fix email address name with encoded words and special chars ↵bsiem2019-08-291-0/+24
| | | | | | | | | | | | | | | | | | | | | | (GH-14561) (GH-15380) Special characters in email address header display names are normally put within double quotes. However, encoded words (=?charset?x?...?=) are not allowed withing double quotes. When the header contains a word with special characters and another word that must be encoded, the first one must also be encoded. In the next example, the display name in the From header is quoted and therefore the comma is allowed; in the To header, the comma is not within quotes and not encoded, which is not allowed and therefore rejected by some mail servers. From: "Foo Bar, France" <foo@example.com> To: Foo Bar, =?utf-8?q?Espa=C3=B1a?= <foo@example.com> https://bugs.python.org/issue37482 (cherry picked from commit df0c21ff46c5c37b6913828ef8c7651f523432f8) Co-authored-by: bsiem <52461103+bsiem@users.noreply.github.com>
* bpo-32178: Fix IndexError trying to parse 'To' header starting with ':'. ↵Miss Islington (bot)2019-08-111-0/+8
| | | | | | | | | | (GH-15044) This should fix the IndexError trying to retrieve `DisplayName.display_name` and `DisplayName.value` when the `value` is basically an empty string. https://bugs.python.org/issue32178 (cherry picked from commit 09a1872a8007048dcdf825a476816c5e3498b8f8) Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
* bpo-34155: Dont parse domains containing @ (GH-13079)Miss Islington (bot)2019-08-092-0/+24
| | | | | | | | | | | | | | | | | | | | | | Before: >>> email.message_from_string('From: a@malicious.org@important.com', policy=email.policy.default)['from'].addresses (Address(display_name='', username='a', domain='malicious.org'),) >>> parseaddr('a@malicious.org@important.com') ('', 'a@malicious.org') After: >>> email.message_from_string('From: a@malicious.org@important.com', policy=email.policy.default)['from'].addresses (Address(display_name='', username='', domain=''),) >>> parseaddr('a@malicious.org@important.com') ('', 'a@') https://bugs.python.org/issue34155 (cherry picked from commit 8cb65d1381b027f0b09ee36bfed7f35bb4dec9a9) Co-authored-by: jpic <jpic@users.noreply.github.com>
* Fix IndexError when parsing unexpectedly ending quoted-string. (GH-14813)Miss Islington (bot)2019-07-171-0/+4
| | | | | | | This exception was caused because the input ended unexpectedly with only one single quote instead of a pair with some value inside it. (cherry picked from commit 719a062bcb7b08a56e6576dcd75f4244e6053209) Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
* bpo-37461: Fix infinite loop in parsing of specially crafted email headers ↵Miss Islington (bot)2019-07-171-0/+7
| | | | | | | | | | | | | (GH-14794) * bpo-37461: Fix infinite loop in parsing of specially crafted email headers. Some crafted email header would cause the get_parameter method to run in an infinite loop causing a DoS attack surface when parsing those headers. This patch fixes that by making sure the DQUOTE character is handled to prevent going into an infinite loop. (cherry picked from commit a4a994bd3e619cbaff97610a1cee8ffa87c672f5) Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
* Fix infinite loop in email folding logic (GH-12732)Miss Islington (bot)2019-07-161-0/+20
| | | | | | | | | | | | | As far as I can tell, this infinite loop would be triggered if: 1. The value being folded contains a single word (no spaces) longer than max_line_length 2. The max_line_length is shorter than the encoding's name + 9 characters. bpo-36564: https://bugs.python.org/issue36564 (cherry picked from commit f69d5c61981ea97d251db515c7ff280fcc17182d) Co-authored-by: Paul Ganssle <pganssle@users.noreply.github.com>
* Remove unused imports in tests (GH-14518) (GH-14520)Victor Stinner2019-07-011-1/+0
| | | (cherry picked from commit 8f4ef3b019ce380022018587571b0f970e668de3)
* bpo-29412: Fix indexError when parsing a header value ending unexpectedly ↵Miss Islington (bot)2019-06-261-0/+17
| | | | | | | | | (GH-14387) (GH-14411) * patched string index out of range error in get_word function of _header_value_parser.py and created tests in test__header_value_parser.py for CFWS. * Raise HeaderParseError instead of continuing when parsing a word. (cherry picked from commit 7213df7bbfd85378c6e42e1ac63144d5974bdcf6) Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com>
* [3.8] bpo-33972: Fix EmailMessage.iter_attachments raising AttributeError ↵Abhilash Raj2019-06-251-0/+9
| | | | | | | | | | | | | | (GH-14119) (GH-14380) When certain malformed messages have content-type set to 'mulitpart/*' but still have a single part body, iter_attachments can raise AttributeError. This patch fixes it by returning a None value instead when the body is single part. (cherry picked from commit 02257012f6d3821d816cb6a7e8461a88a05b9a08) Co-authored-by: Abhilash Raj <maxking@users.noreply.github.com> https://bugs.python.org/issue33972
* [3.8] bpo-36520: Email header folded incorrectly (GH-13608) (GH-13909)Abhilash Raj2019-06-111-0/+131
| | | | | | | | | | | | | | | | | | | | | | * [bpo-36520](https://bugs.python.org/issue36520): reset the encoded word offset when starting a new line during an email header folding operation * 📜🤖 Added by blurb_it. * [bpo-36520](https://bugs.python.org/issue36520): add an additional test case, and provide descriptive comments for the test_folding_with_utf8_encoding_* tests * [bpo-36520](https://bugs.python.org/issue36520): fix whitespace issue * [bpo-36520](https://bugs.python.org/issue36520): changes per reviewer request -- remove extraneous backslashes; add whitespace between terminating quotes and line-continuation backslashes; use "bpo-" instead of "issue GH-" in comments (cherry picked from commit f6713e84afc5addcfa8477dbdf2c027787f711c0) Co-authored-by: websurfer5 <49998481+websurfer5@users.noreply.github.com> https://bugs.python.org/issue36520
* bpo-30835: email: Fix AttributeError when parsing invalid CTE (GH-13598)Abhilash Raj2019-06-041-0/+9
| | | | | | | | | | | | | | | | | | | | * bpo-30835: email: Fix AttributeError when parsing invalid Content-Transfer-Encoding Parsing an email containing a multipart Content-Type, along with a Content-Transfer-Encoding containing an invalid (non-ASCII-decodable) byte will fail. email.feedparser.FeedParser._parsegen() gets the header and attempts to convert it to lowercase before comparing it with the accepted encodings, but as the header contains an invalid byte, it's returned as a Header object rather than a str. Cast the Content-Transfer-Encoding header to a str to avoid this. Found using the AFL fuzzer. Reported-by: Daniel Axtens <dja@axtens.net> Signed-off-by: Andrew Donnellan <andrew@donnellan.id.au> * Add email and NEWS entry for the bugfix.
* bpo-35805: Add parser for Message-ID email header. (GH-13397)Abhilash Raj2019-06-042-0/+100
| | | | | | | | | | | | | * bpo-35805: Add parser for Message-ID header. This parser is based on the definition of Identification Fields from RFC 5322 Sec 3.6.4. This should also prevent folding of Message-ID header using RFC 2047 encoded words and hence fix bpo-35805. * Prevent folding of non-ascii message-id headers. * Add fold method to MsgID token to prevent folding.
* bpo-27737: Allow whitespace only headers encoding (#13478)Batuhan Taşkaya2019-05-231-0/+3
|
* bpo-33524: Fix the folding of email header when max_line_length is 0 or None ↵Abhilash Raj2019-05-171-0/+13
| | | | | (#13391) and there are non-ascii characters in the header.
* bpo-33529, email: Fix infinite loop in email header encoding (GH-12020)Krzysztof Wojcik2019-05-142-4/+12
|
* bpo-34424: Handle different policy.linesep lengths correctly. (#8803)Jens Troeger2019-05-141-0/+22
|
* bpo-35317: Fix mktime() error in test_email (GH-10721)Victor Stinner2018-11-271-0/+2
| | | | | Fix mktime() overflow error in test_email: run test_localtime_daylight_true_dst_true() and test_localtime_daylight_false_dst_true() with a specific timezone.
* bpo-33476: Fix _header_value_parser when address group is missing final ';' ↵Dong-hee Na2018-07-281-0/+25
| | | | (GH-7484)
* bpo-27397: Make email module properly handle invalid-length base64 strings ↵Tal Einat2018-06-123-0/+32
| | | | | | | (#7583) When attempting to base64-decode a payload of invalid length (1 mod 4), properly recognize and handle it. The given data will be returned as-is, i.e. not decoded, along with a new defect, InvalidBase64LengthDefect.
* bpo-27931: Fix email address header parsing error (#5329)jayyyin2018-01-291-0/+17
| | | | Correctly handle addresses whose username is an empty quoted string.
* bpo-27240 Rewrite the email header folding algorithm. (#3488)R. David Murray2017-12-033-117/+216
| | | | | | | | | | | | | | | | | | | | | The original algorithm tried to delegate the folding to the tokens so that those tokens whose folding rules differed could specify the differences. However, this resulted in a lot of duplicated code because most of the rules were the same. The new algorithm moves all folding logic into a set of functions external to the token classes, but puts the information about which tokens can be folded in which ways on the tokens...with the exception of mime-parameters, which are a special case (which was not even implemented in the old folder). This algorithm can still probably be improved and hopefully simplified somewhat. Note that some of the test expectations are changed. I believe the changes are toward more desirable and consistent behavior: in general when (re) folding a line the canonical version of the tokens is generated, rather than preserving errors or extra whitespace.
* bpo-31370: Remove support for threads-less builds (#3385)Antoine Pitrou2017-09-071-4/+1
| | | | | | * Remove Setup.config * Always define WITH_THREAD for compatibility.
* bpo-30532: Fix whitespace folding in certain casesJoel Hillacre2017-06-261-0/+12
| | | Leading whitespace was incorrectly dropped during folding of certain lines in the _header_value_parser's folding algorithm. This makes the whitespace handling code consistent.
* [email] bpo-29478: Fix passing max_line_length=None from Compat32 policy ↵mircea-cosbuc2017-06-121-0/+7
| | | | | | (GH-595) If max_line_length=None is specified while using the Compat32 policy, it is no longer ignored.
* Merge #28407 Improve test coverage of make_msgid.R David Murray2016-12-181-0/+12
|\
| * #28407 Improve test coverage of make_msgid.R David Murray2016-12-181-0/+12
| | | | | | | | Patch by Dillon Brock.
| * #19003: Only replace \r and/or \n line endings in email.generator.R David Murray2016-09-111-0/+12
| | | | | | | | | | This is a further restoration of backward compatibility, as well as being correct per the RFCs.
* | Merge: #19003: Only replace \r and/or \n line endings in email.generator.R David Murray2016-09-111-0/+12
| |
* | #20476: Deal with the message_factory circular import differently.R David Murray2016-09-101-1/+1
| | | | | | | | | | | | | | It turns out we can't depend on email.message getting imported every place message_factory is needed, so to avoid a circular import we need to special case Policy.message_factory=None in the parser instead of using monkey patching. I had a feeling that was a bad idea when I did it.
* | #20476: add a message_factory policy attribute to email.R David Murray2016-09-092-46/+87
| |
* | Merge with 3.5Steve Dower2016-09-092-3/+32
|\ \ | |/
| * #28047: Fix calculation of base64 line length.R David Murray2016-09-092-3/+32
| | | | | | | | | | This is buggy in the old email code as well, but it doesn't affect anything there because only the default line length is ever used there.
* | #27364: fix "incorrect" uses of escape character in the stdlib.R David Murray2016-09-082-3/+3
| | | | | | | | | | | | | | And most of the tools. Patch by Emanual Barry, reviewed by me, Serhiy Storchaka, and Martin Panter.
* | Issue #27445: Merge from 3.5Berker Peksag2016-09-081-1/+4
|\ \ | |/
| * Issue #27445: Don't pass str(_charset) to MIMEText.set_payload()Berker Peksag2016-09-081-1/+4
| | | | | | | | Patch by Claude Paroz.
| * Issue #27895: Spelling fixes (Contributed by Ville Skyttä).Martin Panter2016-09-071-2/+2
| |
* | #24277: The new email API is no longer provisional.R David Murray2016-09-081-0/+20
| | | | | | | | | | | | | | This is a wholesale reorganization and editing of the email documentation to make the new API the standard one, and the old API the 'legacy' one. The default is still the compat32 policy, for backward compatibility. We will change that eventually.
generated by cgit v0.12 at 2026-03-05 14:05:02 (GMT)