diff options
Diffstat (limited to 'Doc/library/robotparser.rst')
| -rw-r--r-- | Doc/library/robotparser.rst | 5 |
1 files changed, 2 insertions, 3 deletions
diff --git a/Doc/library/robotparser.rst b/Doc/library/robotparser.rst index 2451799..6cc7df8 100644 --- a/Doc/library/robotparser.rst +++ b/Doc/library/robotparser.rst @@ -15,9 +15,8 @@ This module provides a single class, :class:`RobotFileParser`, which answers questions about whether or not a particular user agent can fetch a URL on the -Web site that published the :file:`robots.txt` file. For more details on the -structure of :file:`robots.txt` files, see -http://www.robotstxt.org/wc/norobots.html. +Web site that published the :file:`robots.txt` file. For more details on the +structure of :file:`robots.txt` files, see http://www.robotstxt.org/orig.html. .. class:: RobotFileParser() |