diff options
Diffstat (limited to 'Doc')
| -rw-r--r-- | Doc/library/formatter.rst | 8 | ||||
| -rw-r--r-- | Doc/library/html.entities.rst | 9 | ||||
| -rw-r--r-- | Doc/library/html.parser.rst | 8 | ||||
| -rw-r--r-- | Doc/library/htmllib.rst | 147 | ||||
| -rw-r--r-- | Doc/library/markup.rst | 2 | ||||
| -rw-r--r-- | Doc/library/sgmllib.rst | 253 | ||||
| -rw-r--r-- | Doc/library/urllib.rst | 5 |
7 files changed, 10 insertions, 422 deletions
diff --git a/Doc/library/formatter.rst b/Doc/library/formatter.rst index ba09b8e..c983ccc 100644 --- a/Doc/library/formatter.rst +++ b/Doc/library/formatter.rst @@ -1,4 +1,3 @@ - :mod:`formatter` --- Generic output formatting ============================================== @@ -6,12 +5,9 @@ :synopsis: Generic output formatter and device interface. -.. index:: single: HTMLParser (class in htmllib) - This module supports two interface definitions, each with multiple -implementations. The *formatter* interface is used by the :class:`HTMLParser` -class of the :mod:`htmllib` module, and the *writer* interface is required by -the formatter interface. +implementations: The *formatter* interface, and the *writer* interface which is +required by the formatter interface. Formatter objects transform an abstract flow of formatting events into specific output events on writer objects. Formatters manage several stack structures to diff --git a/Doc/library/html.entities.rst b/Doc/library/html.entities.rst index 0b2efeb..aa67bae 100644 --- a/Doc/library/html.entities.rst +++ b/Doc/library/html.entities.rst @@ -7,11 +7,10 @@ This module defines three dictionaries, ``name2codepoint``, ``codepoint2name``, -and ``entitydefs``. ``entitydefs`` is used by the :mod:`htmllib` module to -provide the :attr:`entitydefs` member of the :class:`html.parser.HTMLParser` -class. The definition provided here contains all the entities defined by XHTML -1.0 that can be handled using simple textual substitution in the Latin-1 -character set (ISO-8859-1). +and ``entitydefs``. ``entitydefs`` is used to provide the :attr:`entitydefs` +member of the :class:`html.parser.HTMLParser` class. The definition provided +here contains all the entities defined by XHTML 1.0 that can be handled using +simple textual substitution in the Latin-1 character set (ISO-8859-1). .. data:: entitydefs diff --git a/Doc/library/html.parser.rst b/Doc/library/html.parser.rst index fe6748a..78b7677 100644 --- a/Doc/library/html.parser.rst +++ b/Doc/library/html.parser.rst @@ -11,9 +11,6 @@ This module defines a class :class:`HTMLParser` which serves as the basis for parsing text files formatted in HTML (HyperText Mark-up Language) and XHTML. -Unlike the parser in :mod:`htmllib`, this parser is not based on the SGML parser -in :mod:`sgmllib`. - .. class:: HTMLParser() @@ -23,9 +20,8 @@ in :mod:`sgmllib`. begin and end. The :class:`HTMLParser` class is meant to be overridden by the user to provide a desired behavior. - Unlike the parser in :mod:`htmllib`, this parser does not check that end tags - match start tags or call the end-tag handler for elements which are closed - implicitly by closing an outer element. + This parser does not check that end tags match start tags or call the end-tag + handler for elements which are closed implicitly by closing an outer element. An exception is defined as well: diff --git a/Doc/library/htmllib.rst b/Doc/library/htmllib.rst deleted file mode 100644 index 5e6554a..0000000 --- a/Doc/library/htmllib.rst +++ /dev/null @@ -1,147 +0,0 @@ - -:mod:`htmllib` --- A parser for HTML documents -============================================== - -.. module:: htmllib - :synopsis: A parser for HTML documents. - - -.. index:: - single: HTML - single: hypertext - -.. index:: - module: sgmllib - module: formatter - single: SGMLParser (in module sgmllib) - -This module defines a class which can serve as a base for parsing text files -formatted in the HyperText Mark-up Language (HTML). The class is not directly -concerned with I/O --- it must be provided with input in string form via a -method, and makes calls to methods of a "formatter" object in order to produce -output. The :class:`HTMLParser` class is designed to be used as a base class -for other classes in order to add functionality, and allows most of its methods -to be extended or overridden. In turn, this class is derived from and extends -the :class:`SGMLParser` class defined in module :mod:`sgmllib`. The -:class:`HTMLParser` implementation supports the HTML 2.0 language as described -in :rfc:`1866`. Two implementations of formatter objects are provided in the -:mod:`formatter` module; refer to the documentation for that module for -information on the formatter interface. - -The following is a summary of the interface defined by -:class:`sgmllib.SGMLParser`: - -* The interface to feed data to an instance is through the :meth:`feed` method, - which takes a string argument. This can be called with as little or as much - text at a time as desired; ``p.feed(a); p.feed(b)`` has the same effect as - ``p.feed(a+b)``. When the data contains complete HTML markup constructs, these - are processed immediately; incomplete constructs are saved in a buffer. To - force processing of all unprocessed data, call the :meth:`close` method. - - For example, to parse the entire contents of a file, use:: - - parser.feed(open('myfile.html').read()) - parser.close() - -* The interface to define semantics for HTML tags is very simple: derive a class - and define methods called :meth:`start_tag`, :meth:`end_tag`, or :meth:`do_tag`. - The parser will call these at appropriate moments: :meth:`start_tag` or - :meth:`do_tag` is called when an opening tag of the form ``<tag ...>`` is - encountered; :meth:`end_tag` is called when a closing tag of the form ``<tag>`` - is encountered. If an opening tag requires a corresponding closing tag, like - ``<H1>`` ... ``</H1>``, the class should define the :meth:`start_tag` method; if - a tag requires no closing tag, like ``<P>``, the class should define the - :meth:`do_tag` method. - -The module defines a parser class and an exception: - - -.. class:: HTMLParser(formatter) - - This is the basic HTML parser class. It supports all entity names required by - the XHTML 1.0 Recommendation (http://www.w3.org/TR/xhtml1). It also defines - handlers for all HTML 2.0 and many HTML 3.0 and 3.2 elements. - - -.. exception:: HTMLParseError - - Exception raised by the :class:`HTMLParser` class when it encounters an error - while parsing. - - -.. seealso:: - - Module :mod:`formatter` - Interface definition for transforming an abstract flow of formatting events into - specific output events on writer objects. - - Module :mod:`html.parser` - Alternate HTML parser that offers a slightly lower-level view of the input, but - is designed to work with XHTML, and does not implement some of the SGML syntax - not used in "HTML as deployed" and which isn't legal for XHTML. - - Module :mod:`html.entities` - Definition of replacement text for XHTML 1.0 entities. - - Module :mod:`sgmllib` - Base class for :class:`HTMLParser`. - - -.. _html-parser-objects: - -HTMLParser Objects ------------------- - -In addition to tag methods, the :class:`HTMLParser` class provides some -additional methods and instance variables for use within tag methods. - - -.. attribute:: HTMLParser.formatter - - This is the formatter instance associated with the parser. - - -.. attribute:: HTMLParser.nofill - - Boolean flag which should be true when whitespace should not be collapsed, or - false when it should be. In general, this should only be true when character - data is to be treated as "preformatted" text, as within a ``<PRE>`` element. - The default value is false. This affects the operation of :meth:`handle_data` - and :meth:`save_end`. - - -.. method:: HTMLParser.anchor_bgn(href, name, type) - - This method is called at the start of an anchor region. The arguments - correspond to the attributes of the ``<A>`` tag with the same names. The - default implementation maintains a list of hyperlinks (defined by the ``HREF`` - attribute for ``<A>`` tags) within the document. The list of hyperlinks is - available as the data attribute :attr:`anchorlist`. - - -.. method:: HTMLParser.anchor_end() - - This method is called at the end of an anchor region. The default - implementation adds a textual footnote marker using an index into the list of - hyperlinks created by :meth:`anchor_bgn`. - - -.. method:: HTMLParser.handle_image(source, alt[, ismap[, align[, width[, height]]]]) - - This method is called to handle images. The default implementation simply - passes the *alt* value to the :meth:`handle_data` method. - - -.. method:: HTMLParser.save_bgn() - - Begins saving character data in a buffer instead of sending it to the formatter - object. Retrieve the stored data via :meth:`save_end`. Use of the - :meth:`save_bgn` / :meth:`save_end` pair may not be nested. - - -.. method:: HTMLParser.save_end() - - Ends buffering character data and returns all data saved since the preceding - call to :meth:`save_bgn`. If the :attr:`nofill` flag is false, whitespace is - collapsed to single spaces. A call to this method without a preceding call to - :meth:`save_bgn` will raise a :exc:`TypeError` exception. diff --git a/Doc/library/markup.rst b/Doc/library/markup.rst index 1a90031..2bb20e3 100644 --- a/Doc/library/markup.rst +++ b/Doc/library/markup.rst @@ -23,8 +23,6 @@ definition of the Python bindings for the DOM and SAX interfaces. html.parser.rst html.entities.rst - sgmllib.rst - htmllib.rst pyexpat.rst xml.dom.rst xml.dom.minidom.rst diff --git a/Doc/library/sgmllib.rst b/Doc/library/sgmllib.rst deleted file mode 100644 index 637aa91..0000000 --- a/Doc/library/sgmllib.rst +++ /dev/null @@ -1,253 +0,0 @@ - -:mod:`sgmllib` --- Simple SGML parser -===================================== - -.. module:: sgmllib - :synopsis: Only as much of an SGML parser as needed to parse HTML. - - -.. index:: single: SGML - -This module defines a class :class:`SGMLParser` which serves as the basis for -parsing text files formatted in SGML (Standard Generalized Mark-up Language). -In fact, it does not provide a full SGML parser --- it only parses SGML insofar -as it is used by HTML, and the module only exists as a base for the -:mod:`htmllib` module. Another HTML parser which supports XHTML and offers a -somewhat different interface is available in the :mod:`HTMLParser` module. - - -.. class:: SGMLParser() - - The :class:`SGMLParser` class is instantiated without arguments. The parser is - hardcoded to recognize the following constructs: - - * Opening and closing tags of the form ``<tag attr="value" ...>`` and - ``</tag>``, respectively. - - * Numeric character references of the form ``&#name;``. - - * Entity references of the form ``&name;``. - - * SGML comments of the form ``<!--text-->``. Note that spaces, tabs, and - newlines are allowed between the trailing ``>`` and the immediately preceding - ``--``. - -A single exception is defined as well: - - -.. exception:: SGMLParseError - - Exception raised by the :class:`SGMLParser` class when it encounters an error - while parsing. - -:class:`SGMLParser` instances have the following methods: - - -.. method:: SGMLParser.reset() - - Reset the instance. Loses all unprocessed data. This is called implicitly at - instantiation time. - - -.. method:: SGMLParser.setnomoretags() - - Stop processing tags. Treat all following input as literal input (CDATA). - (This is only provided so the HTML tag ``<PLAINTEXT>`` can be implemented.) - - -.. method:: SGMLParser.setliteral() - - Enter literal mode (CDATA mode). - - -.. method:: SGMLParser.feed(data) - - Feed some text to the parser. It is processed insofar as it consists of - complete elements; incomplete data is buffered until more data is fed or - :meth:`close` is called. - - -.. method:: SGMLParser.close() - - Force processing of all buffered data as if it were followed by an end-of-file - mark. This method may be redefined by a derived class to define additional - processing at the end of the input, but the redefined version should always call - :meth:`close`. - - -.. method:: SGMLParser.get_starttag_text() - - Return the text of the most recently opened start tag. This should not normally - be needed for structured processing, but may be useful in dealing with HTML "as - deployed" or for re-generating input with minimal changes (whitespace between - attributes can be preserved, etc.). - - -.. method:: SGMLParser.handle_starttag(tag, method, attributes) - - This method is called to handle start tags for which either a :meth:`start_tag` - or :meth:`do_tag` method has been defined. The *tag* argument is the name of - the tag converted to lower case, and the *method* argument is the bound method - which should be used to support semantic interpretation of the start tag. The - *attributes* argument is a list of ``(name, value)`` pairs containing the - attributes found inside the tag's ``<>`` brackets. - - The *name* has been translated to lower case. Double quotes and backslashes in - the *value* have been interpreted, as well as known character references and - known entity references terminated by a semicolon (normally, entity references - can be terminated by any non-alphanumerical character, but this would break the - very common case of ``<A HREF="url?spam=1&eggs=2">`` when ``eggs`` is a valid - entity name). - - For instance, for the tag ``<A HREF="http://www.cwi.nl/">``, this method would - be called as ``unknown_starttag('a', [('href', 'http://www.cwi.nl/')])``. The - base implementation simply calls *method* with *attributes* as the only - argument. - - -.. method:: SGMLParser.handle_endtag(tag, method) - - This method is called to handle endtags for which an :meth:`end_tag` method has - been defined. The *tag* argument is the name of the tag converted to lower - case, and the *method* argument is the bound method which should be used to - support semantic interpretation of the end tag. If no :meth:`end_tag` method is - defined for the closing element, this handler is not called. The base - implementation simply calls *method*. - - -.. method:: SGMLParser.handle_data(data) - - This method is called to process arbitrary data. It is intended to be - overridden by a derived class; the base class implementation does nothing. - - -.. method:: SGMLParser.handle_charref(ref) - - This method is called to process a character reference of the form ``&#ref;``. - The base implementation uses :meth:`convert_charref` to convert the reference to - a string. If that method returns a string, it is passed to :meth:`handle_data`, - otherwise ``unknown_charref(ref)`` is called to handle the error. - - -.. method:: SGMLParser.convert_charref(ref) - - Convert a character reference to a string, or ``None``. *ref* is the reference - passed in as a string. In the base implementation, *ref* must be a decimal - number in the range 0-255. It converts the code point found using the - :meth:`convert_codepoint` method. If *ref* is invalid or out of range, this - method returns ``None``. This method is called by the default - :meth:`handle_charref` implementation and by the attribute value parser. - - -.. method:: SGMLParser.convert_codepoint(codepoint) - - Convert a codepoint to a :class:`str` value. Encodings can be handled here if - appropriate, though the rest of :mod:`sgmllib` is oblivious on this matter. - - -.. method:: SGMLParser.handle_entityref(ref) - - This method is called to process a general entity reference of the form - ``&ref;`` where *ref* is an general entity reference. It converts *ref* by - passing it to :meth:`convert_entityref`. If a translation is returned, it calls - the method :meth:`handle_data` with the translation; otherwise, it calls the - method ``unknown_entityref(ref)``. The default :attr:`entitydefs` defines - translations for ``&``, ``&apos``, ``>``, ``<``, and ``"``. - - -.. method:: SGMLParser.convert_entityref(ref) - - Convert a named entity reference to a :class:`str` value, or ``None``. The - resulting value will not be parsed. *ref* will be only the name of the entity. - The default implementation looks for *ref* in the instance (or class) variable - :attr:`entitydefs` which should be a mapping from entity names to corresponding - translations. If no translation is available for *ref*, this method returns - ``None``. This method is called by the default :meth:`handle_entityref` - implementation and by the attribute value parser. - - -.. method:: SGMLParser.handle_comment(comment) - - This method is called when a comment is encountered. The *comment* argument is - a string containing the text between the ``<!--`` and ``-->`` delimiters, but - not the delimiters themselves. For example, the comment ``<!--text-->`` will - cause this method to be called with the argument ``'text'``. The default method - does nothing. - - -.. method:: SGMLParser.handle_decl(data) - - Method called when an SGML declaration is read by the parser. In practice, the - ``DOCTYPE`` declaration is the only thing observed in HTML, but the parser does - not discriminate among different (or broken) declarations. Internal subsets in - a ``DOCTYPE`` declaration are not supported. The *data* parameter will be the - entire contents of the declaration inside the ``<!``...\ ``>`` markup. The - default implementation does nothing. - - -.. method:: SGMLParser.report_unbalanced(tag) - - This method is called when an end tag is found which does not correspond to any - open element. - - -.. method:: SGMLParser.unknown_starttag(tag, attributes) - - This method is called to process an unknown start tag. It is intended to be - overridden by a derived class; the base class implementation does nothing. - - -.. method:: SGMLParser.unknown_endtag(tag) - - This method is called to process an unknown end tag. It is intended to be - overridden by a derived class; the base class implementation does nothing. - - -.. method:: SGMLParser.unknown_charref(ref) - - This method is called to process unresolvable numeric character references. - Refer to :meth:`handle_charref` to determine what is handled by default. It is - intended to be overridden by a derived class; the base class implementation does - nothing. - - -.. method:: SGMLParser.unknown_entityref(ref) - - This method is called to process an unknown entity reference. It is intended to - be overridden by a derived class; the base class implementation does nothing. - -Apart from overriding or extending the methods listed above, derived classes may -also define methods of the following form to define processing of specific tags. -Tag names in the input stream are case independent; the *tag* occurring in -method names must be in lower case: - - -.. method:: SGMLParser.start_tag(attributes) - :noindex: - - This method is called to process an opening tag *tag*. It has preference over - :meth:`do_tag`. The *attributes* argument has the same meaning as described for - :meth:`handle_starttag` above. - - -.. method:: SGMLParser.do_tag(attributes) - :noindex: - - This method is called to process an opening tag *tag* for which no - :meth:`start_tag` method is defined. The *attributes* argument has the same - meaning as described for :meth:`handle_starttag` above. - - -.. method:: SGMLParser.end_tag() - :noindex: - - This method is called to process a closing tag *tag*. - -Note that the parser maintains a stack of open elements for which no end tag has -been found yet. Only tags processed by :meth:`start_tag` are pushed on this -stack. Definition of an :meth:`end_tag` method is optional for these tags. For -tags processed by :meth:`do_tag` or by :meth:`unknown_tag`, no :meth:`end_tag` -method must be defined; if defined, it will not be used. If both -:meth:`start_tag` and :meth:`do_tag` methods exist for a tag, the -:meth:`start_tag` method takes precedence. - diff --git a/Doc/library/urllib.rst b/Doc/library/urllib.rst index e976cb1..646b048 100644 --- a/Doc/library/urllib.rst +++ b/Doc/library/urllib.rst @@ -389,14 +389,13 @@ URL Opener objects .. index:: single: HTML pair: HTTP; protocol - module: htmllib * The data returned by :func:`urlopen` or :func:`urlretrieve` is the raw data returned by the server. This may be binary data (such as an image), plain text or (for example) HTML. The HTTP protocol provides type information in the reply header, which can be inspected by looking at the :mailheader:`Content-Type` - header. If the returned data is HTML, you can use the module :mod:`htmllib` to - parse it. + header. If the returned data is HTML, you can use the module + :mod:`html.parser` to parse it. .. index:: single: FTP |