diff options
Diffstat (limited to 'Doc/lib/librobotparser.tex')
| -rw-r--r-- | Doc/lib/librobotparser.tex | 4 |
1 files changed, 2 insertions, 2 deletions
diff --git a/Doc/lib/librobotparser.tex b/Doc/lib/librobotparser.tex index a48ad85..8321dad 100644 --- a/Doc/lib/librobotparser.tex +++ b/Doc/lib/librobotparser.tex @@ -9,13 +9,13 @@ \sectionauthor{Skip Montanaro}{skip@mojam.com} \index{WWW} -\index{World-Wide Web} +\index{World Wide Web} \index{URL} \index{robots.txt} This module provides a single class, \class{RobotFileParser}, which answers questions about whether or not a particular user agent can fetch a URL on -the web site that published the \file{robots.txt} file. For more details on +the Web site that published the \file{robots.txt} file. For more details on the structure of \file{robots.txt} files, see \url{http://info.webcrawler.com/mak/projects/robots/norobots.html}. |