diff options
| author | Terry Jan Reedy <tjreedy@udel.edu> | 2016-06-11 19:06:08 (GMT) |
|---|---|---|
| committer | Terry Jan Reedy <tjreedy@udel.edu> | 2016-06-11 19:06:08 (GMT) |
| commit | 4da945f361cda03104548f49bad690a54b0209f7 (patch) | |
| tree | ed25742482aca03601d01a6c6fccc65a5e0ded2c /Doc/library/urllib.robotparser.rst | |
| parent | e670be2273f93dfc262e16ebc27f0d453f15fa21 (diff) | |
| parent | fa089b9b0b926c04e5d57812b7d7653472787965 (diff) | |
| download | cpython-4da945f361cda03104548f49bad690a54b0209f7.zip cpython-4da945f361cda03104548f49bad690a54b0209f7.tar.gz cpython-4da945f361cda03104548f49bad690a54b0209f7.tar.bz2 | |
Merge Issue #22558.
Diffstat (limited to 'Doc/library/urllib.robotparser.rst')
| -rw-r--r-- | Doc/library/urllib.robotparser.rst | 4 |
1 files changed, 4 insertions, 0 deletions
diff --git a/Doc/library/urllib.robotparser.rst b/Doc/library/urllib.robotparser.rst index c2e1bef..7d31932 100644 --- a/Doc/library/urllib.robotparser.rst +++ b/Doc/library/urllib.robotparser.rst @@ -4,8 +4,10 @@ .. module:: urllib.robotparser :synopsis: Load a robots.txt file and answer questions about fetchability of other URLs. + .. sectionauthor:: Skip Montanaro <skip@pobox.com> +**Source code:** :source:`Lib/urllib/robotparser.py` .. index:: single: WWW @@ -13,6 +15,8 @@ single: URL single: robots.txt +-------------- + This module provides a single class, :class:`RobotFileParser`, which answers questions about whether or not a particular user agent can fetch a URL on the Web site that published the :file:`robots.txt` file. For more details on the |