summaryrefslogtreecommitdiffstats
path: root/Doc/library/robotparser.rst
diff options
context:
space:
mode:
authorBenjamin Peterson <benjamin@python.org>2008-04-25 01:59:09 (GMT)
committerBenjamin Peterson <benjamin@python.org>2008-04-25 01:59:09 (GMT)
commite41251e864e94885d785b5a9bf8f824753316296 (patch)
treef530db7682d71f4920b22b8d7f84c89727647ab5 /Doc/library/robotparser.rst
parent768db92b438038586c1580b711c528363a97d3f4 (diff)
downloadcpython-e41251e864e94885d785b5a9bf8f824753316296.zip
cpython-e41251e864e94885d785b5a9bf8f824753316296.tar.gz
cpython-e41251e864e94885d785b5a9bf8f824753316296.tar.bz2
Merged revisions 62490 via svnmerge from
svn+ssh://pythondev@svn.python.org/python/trunk ........ r62490 | benjamin.peterson | 2008-04-24 20:29:10 -0500 (Thu, 24 Apr 2008) | 2 lines reformat some documentation of classes so methods and attributes are under the class directive ........
Diffstat (limited to 'Doc/library/robotparser.rst')
-rw-r--r--Doc/library/robotparser.rst12
1 files changed, 6 insertions, 6 deletions
diff --git a/Doc/library/robotparser.rst b/Doc/library/robotparser.rst
index 6cc7df8..b3a9a60 100644
--- a/Doc/library/robotparser.rst
+++ b/Doc/library/robotparser.rst
@@ -25,35 +25,35 @@ structure of :file:`robots.txt` files, see http://www.robotstxt.org/orig.html.
single :file:`robots.txt` file.
- .. method:: RobotFileParser.set_url(url)
+ .. method:: set_url(url)
Sets the URL referring to a :file:`robots.txt` file.
- .. method:: RobotFileParser.read()
+ .. method:: read()
Reads the :file:`robots.txt` URL and feeds it to the parser.
- .. method:: RobotFileParser.parse(lines)
+ .. method:: parse(lines)
Parses the lines argument.
- .. method:: RobotFileParser.can_fetch(useragent, url)
+ .. method:: can_fetch(useragent, url)
Returns ``True`` if the *useragent* is allowed to fetch the *url* according to
the rules contained in the parsed :file:`robots.txt` file.
- .. method:: RobotFileParser.mtime()
+ .. method:: mtime()
Returns the time the ``robots.txt`` file was last fetched. This is useful for
long-running web spiders that need to check for new ``robots.txt`` files
periodically.
- .. method:: RobotFileParser.modified()
+ .. method:: modified()
Sets the time the ``robots.txt`` file was last fetched to the current time.