| Commit message (Collapse) | Author | Age | Files | Lines | |
|---|---|---|---|---|---|
| * | Replace str.find()!=1 with the more readable "in" operator. | Raymond Hettinger | 2004-05-04 | 1 | -1/+1 |
| | | |||||
| * | SF patch #911431: robot.txt must be robots.txt | Raymond Hettinger | 2004-03-13 | 1 | -2/+2 |
| | | | | | (Contributed by George Yoshida.) | ||||
| * | Get rid of many apply() calls. | Guido van Rossum | 2003-02-27 | 1 | -1/+1 |
| | | |||||
| * | Remove import of re, it is not used | Neal Norwitz | 2002-05-31 | 1 | -1/+1 |
| | | |||||
| * | Patch 560023 adding docstrings. 2.2 Candidate (after verifying modules were ↵ | Raymond Hettinger | 2002-05-29 | 1 | -0/+17 |
| | | | | | not updated after 2.2). | ||||
| * | Convert a pile of obvious "yes/no" functions to return bool. | Tim Peters | 2002-04-04 | 1 | -6/+6 |
| | | |||||
| * | Correctly set default entry in all cases. | Martin v. Löwis | 2002-03-18 | 1 | -6/+9 |
| | | |||||
| * | Patch #499513: use readline() instead of readlines(). Removed the | Martin v. Löwis | 2002-03-18 | 1 | -16/+6 |
| | | | | | unnecessary redirection limit code which is already in FancyURLopener. | ||||
| * | Correct various errors: | Martin v. Löwis | 2002-02-28 | 1 | -6/+16 |
| | | | | | | | | - Use substring search, not re search for user-agent and paths. - Consider * entry last. Unquote, then requote URLs. - Treat empty Disallow as "allow everything". Add test cases. Fixes #523041 | ||||
| * | Remove unused import (PyChecker) | Andrew M. Kuchling | 2001-08-13 | 1 | -1/+0 |
| | | |||||
| * | Whitespace normalization. | Tim Peters | 2001-02-15 | 1 | -1/+1 |
| | | |||||
| * | The bulk of the credit for these changes goes to Bastian Kleineidam | Skip Montanaro | 2001-02-12 | 1 | -34/+89 |
| | | | | | | | | * restores urllib as the file fetcher (closes bug #132000) * allows checking URLs with empty paths (closes patches #103511 and 103721) * properly handle user agents with versions (e.g., SpamMeister/1.5) * added several more tests | ||||
| * | String method conversion. | Eric S. Raymond | 2001-02-09 | 1 | -8/+8 |
| | | |||||
| * | Whitespace normalization. | Tim Peters | 2001-01-21 | 1 | -10/+10 |
| | | |||||
| * | added __all__ lists to a number of Python modules | Skip Montanaro | 2001-01-20 | 1 | -0/+2 |
| | | | | | | | | | added test script and expected output file as well this closes patch 103297. __all__ attributes will be added to other modules without first submitting a patch, just adding the necessary line to the test script to verify more-or-less correct implementation. | ||||
| * | rewrite of robotparser.py by Bastian Kleineidam. Closes patch 102229. | Skip Montanaro | 2001-01-20 | 1 | -60/+179 |
| | | |||||
| * | Skip Montanaro: | Guido van Rossum | 2000-03-27 | 1 | -17/+17 |
| | | | | | | | | | | | | | The robotparser.py module currently lives in Tools/webchecker. In preparation for its migration to Lib, I made the following changes: * renamed the test() function _test * corrected the URLs in _test() so they refer to actual documents * added an "if __name__ == '__main__'" catcher to invoke _test() when run as a main program * added doc strings for the two main methods, parse and can_fetch * replaced usage of regsub and regex with corresponding re code | ||||
| * | Give in to tabnanny | Guido van Rossum | 1998-04-06 | 1 | -60/+60 |
| | | |||||
| * | Skip Montanaro's robots.txt parser. | Guido van Rossum | 1997-01-30 | 1 | -0/+97 |
 |
index : cpython.git | |
| https://github.com/python/cpython.git |
| summaryrefslogtreecommitdiffstats |