Merged revisions 61003-61033 via svnmerge from

svn+ssh://pythondev@svn.python.org/python/trunk

........
  r61004 | georg.brandl | 2008-02-23 19:47:04 +0100 (Sat, 23 Feb 2008) | 2 lines

  Documentation coverage builder, part 1.
........
  r61006 | andrew.kuchling | 2008-02-23 20:02:33 +0100 (Sat, 23 Feb 2008) | 1 line

  #1389051: IMAP module tries to read entire message in one chunk.  Patch by Fredrik Lundh.
........
  r61008 | andrew.kuchling | 2008-02-23 20:28:58 +0100 (Sat, 23 Feb 2008) | 1 line

  #1389051, #1092502: fix excessively large allocations when using read() on a socket
........
  r61011 | jeffrey.yasskin | 2008-02-23 20:40:54 +0100 (Sat, 23 Feb 2008) | 13 lines

  Prevent classes like:
      class RunSelfFunction(object):
          def __init__(self):
              self.thread = threading.Thread(target=self._run)
              self.thread.start()
          def _run(self):
              pass
  from creating a permanent cycle between the object and the thread by having the
  Thread delete its references to the object when it completes.

  As an example of the effect of this bug, paramiko.Transport inherits from
  Thread to avoid it.
........
  r61013 | jeffrey.yasskin | 2008-02-23 21:40:35 +0100 (Sat, 23 Feb 2008) | 3 lines

  Followup to r61011: Also avoid the reference cycle when the Thread's target
  raises an exception.
........
  r61017 | georg.brandl | 2008-02-23 22:59:11 +0100 (Sat, 23 Feb 2008) | 2 lines

  #2101: fix removeAttribute docs.
........
  r61018 | georg.brandl | 2008-02-23 23:05:38 +0100 (Sat, 23 Feb 2008) | 2 lines

  Add examples to modulefinder docs. Written for GHOP by Josip Dzolonga.
........
  r61019 | georg.brandl | 2008-02-23 23:09:24 +0100 (Sat, 23 Feb 2008) | 2 lines

  Use os.closerange() in popen2.
........
  r61020 | georg.brandl | 2008-02-23 23:14:02 +0100 (Sat, 23 Feb 2008) | 2 lines

  Use os.closerange().
........
  r61021 | georg.brandl | 2008-02-23 23:35:33 +0100 (Sat, 23 Feb 2008) | 3 lines

  In test_heapq and test_bisect, test both the Python and the C implementation.
  Originally written for GHOP by Josip Dzolonga, heavily patched by me.
........
  r61024 | facundo.batista | 2008-02-23 23:54:12 +0100 (Sat, 23 Feb 2008) | 3 lines


  Added simple test case. Thanks Benjamin Peterson.
........
  r61025 | georg.brandl | 2008-02-23 23:55:18 +0100 (Sat, 23 Feb 2008) | 2 lines

  #1825: correctly document msilib.add_data.
........
  r61027 | georg.brandl | 2008-02-24 00:02:23 +0100 (Sun, 24 Feb 2008) | 2 lines

  #1826: allow dotted attribute paths in operator.attrgetter.
........
  r61028 | georg.brandl | 2008-02-24 00:04:35 +0100 (Sun, 24 Feb 2008) | 2 lines

  #1506171: added operator.methodcaller().
........
  r61029 | georg.brandl | 2008-02-24 00:25:26 +0100 (Sun, 24 Feb 2008) | 2 lines

  Document import ./. threading issues. #1720705.
........
  r61032 | georg.brandl | 2008-02-24 00:43:01 +0100 (Sun, 24 Feb 2008) | 2 lines

  Specify what kind of warning -3 emits.
........
  r61033 | christian.heimes | 2008-02-24 00:59:45 +0100 (Sun, 24 Feb 2008) | 1 line

  MS Windows doesn't have mode_t but stat.st_mode is defined as unsigned short.
........

9 files changed, 149 insertions, 14 deletions

diff --git a/Doc/Makefile b/Doc/Makefile
index 95718d7..f69383e 100644
--- a/Doc/Makefile
+++ b/Doc/Makefile
@@ -12,7 +12,7 @@ PAPER        =
 ALLSPHINXOPTS = -b $(BUILDER) -d build/doctrees -D latex_paper_size=$(PAPER) \
                 $(SPHINXOPTS) . build/$(BUILDER)
 
-.PHONY: help checkout update build html web htmlhelp clean
+.PHONY: help checkout update build html web htmlhelp clean coverage
 
 help:
 	@echo "Please use \`make <target>' where <target> is one of"
@@ -22,6 +22,7 @@ help:
 	@echo "  latex     to make LaTeX files, you can set PAPER=a4 or PAPER=letter"
 	@echo "  changes   to make an overview over all changed/added/deprecated items"
 	@echo "  linkcheck to check all external links for integrity"
+	@echo "  coverage  to check documentation coverage for library and C API"
 
 checkout:
 	@if [ ! -d tools/sphinx ]; then \
@@ -74,9 +75,13 @@ changes: build
 
 linkcheck: BUILDER = linkcheck
 linkcheck: build
-	@echo "Link check complete; look for any errors in the above output "\
+	@echo "Link check complete; look for any errors in the above output " \
 	      "or in build/$(BUILDER)/output.txt"
 
+coverage: BUILDER = coverage
+coverage: build
+	@echo "Coverage finished; see c.txt and python.txt in build/coverage"
+
 clean:
 	-rm -rf build/*
 	-rm -rf tools/sphinx
diff --git a/Doc/README.txt b/Doc/README.txt
index 202a1ce..f452c4e 100644
--- a/Doc/README.txt
+++ b/Doc/README.txt
@@ -64,6 +64,9 @@ Available make targets are:
    deprecated items in the current version. This is meant as a help for the
    writer of the "What's New" document.
 
+ * "coverage", which builds a coverage overview for standard library modules
+   and C API.
+
 A "make update" updates the Subversion checkouts in `tools/`.
 
 
diff --git a/Doc/conf.py b/Doc/conf.py
index 1c8dd71..5b3b42e 100644
--- a/Doc/conf.py
+++ b/Doc/conf.py
@@ -13,7 +13,7 @@ sys.path.append('tools/sphinxext')
 # General configuration
 # ---------------------
 
-extensions = ['sphinx.addons.refcounting']
+extensions = ['sphinx.addons.refcounting', 'sphinx.addons.coverage']
 
 # General substitutions.
 project = 'Python'
@@ -140,3 +140,39 @@ latex_preamble = r'''
 
 # Documents to append as an appendix to all manuals.
 latex_appendices = ['glossary', 'about', 'license', 'copyright']
+
+# Options for the coverage checker
+# --------------------------------
+
+# The coverage checker will ignore all modules/functions/classes whose names
+# match any of the following regexes (using re.match).
+coverage_ignore_modules = [
+    r'[T|t][k|K]',
+    r'Tix',
+    r'distutils.*',
+]
+
+coverage_ignore_functions = [
+    'test($|_)',
+]
+
+coverage_ignore_classes = [
+]
+
+# Glob patterns for C source files for C API coverage, relative to this directory.
+coverage_c_path = [
+    '../Include/*.h',
+]
+
+# Regexes to find C items in the source files.
+coverage_c_regexes = {
+    'cfunction': (r'^PyAPI_FUNC\(.*\)\s+([^_][\w_]+)'),
+    'data': (r'^PyAPI_DATA\(.*\)\s+([^_][\w_]+)'),
+    'macro': (r'^#define ([^_][\w_]+)\(.*\)[\s|\\]'),
+}
+
+# The coverage checker will ignore all C items whose names match these regexes
+# (using re.match) -- the keys must be the same as in coverage_c_regexes.
+coverage_ignore_c_items = {
+#    'cfunction': [...]
+}
diff --git a/Doc/library/modulefinder.rst b/Doc/library/modulefinder.rst
index 9ae0788..d39f412 100644
--- a/Doc/library/modulefinder.rst
+++ b/Doc/library/modulefinder.rst
@@ -48,3 +48,65 @@ report of the imported modules will be printed.
 
    Analyze the contents of the *pathname* file, which must contain  Python code.
 
+.. attribute:: ModuleFinder.modules
+
+   A dictionary mapping module names to modules. See :ref:`modulefinder-example`
+
+
+.. _modulefinder-example:
+
+Example usage of :class:`ModuleFinder`
+--------------------------------------
+
+The script that is going to get analyzed later on (bacon.py)::
+
+   import re, itertools
+
+   try:
+       import baconhameggs
+   except ImportError:
+       pass
+
+   try:
+       import guido.python.ham
+   except ImportError:
+       pass
+
+
+The script that will output the report of bacon.py::
+
+   from modulefinder import ModuleFinder
+
+   finder = ModuleFinder()
+   finder.run_script('bacon.py')
+
+   print 'Loaded modules:'
+   for name, mod in finder.modules.iteritems():
+       print '%s: ' % name,
+       print ','.join(mod.globalnames.keys()[:3])
+
+   print '-'*50
+   print 'Modules not imported:'
+   print '\n'.join(finder.badmodules.iterkeys())
+
+Sample output (may vary depending on the architecture)::
+
+    Loaded modules:
+    _types:
+    copy_reg:  _inverted_registry,_slotnames,__all__
+    sre_compile:  isstring,_sre,_optimize_unicode
+    _sre:
+    sre_constants:  REPEAT_ONE,makedict,AT_END_LINE
+    sys:
+    re:  __module__,finditer,_expand
+    itertools:
+    __main__:  re,itertools,baconhameggs
+    sre_parse:  __getslice__,_PATTERNENDERS,SRE_FLAG_UNICODE
+    array:
+    types:  __module__,IntType,TypeType
+    ---------------------------------------------------
+    Modules not imported:
+    guido.python.ham
+    baconhameggs
+
+
diff --git a/Doc/library/msilib.rst b/Doc/library/msilib.rst
index 35e472c..a8972a8 100644
--- a/Doc/library/msilib.rst
+++ b/Doc/library/msilib.rst
@@ -65,7 +65,7 @@ structures.
 
 .. function:: init_database(name, schema, ProductName, ProductCode, ProductVersion, Manufacturer)
 
-   Create and return a new database *name*, initialize it  with *schema*,  and set
+   Create and return a new database *name*, initialize it with *schema*, and set
    the properties *ProductName*, *ProductCode*, *ProductVersion*, and
    *Manufacturer*.
 
@@ -77,13 +77,20 @@ structures.
    function returns.
 
 
-.. function:: add_data(database, records)
+.. function:: add_data(database, table, records)
 
-   Add all *records* to *database*.  *records* should be a list of tuples, each one
-   containing all fields of a record according to the schema of the table.  For
-   optional fields, ``None`` can be passed.
+   Add all *records* to the table named *table* in *database*.
 
-   Field values can be integers, strings, or instances of the Binary class.
+   The *table* argument must be one of the predefined tables in the MSI schema,
+   e.g. ``'Feature'``, ``'File'``, ``'Component'``, ``'Dialog'``, ``'Control'``,
+   etc.
+
+   *records* should be a list of tuples, each one containing all fields of a
+   record according to the schema of the table.  For optional fields,
+   ``None`` can be passed.
+
+   Field values can be int or long numbers, strings, or instances of the Binary
+   class.
 
 
 .. class:: Binary(filename)
diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst
index 9a613ab..ca056ab 100644
--- a/Doc/library/operator.rst
+++ b/Doc/library/operator.rst
@@ -419,10 +419,12 @@ expect a function argument.
 
    Return a callable object that fetches *attr* from its operand. If more than one
    attribute is requested, returns a tuple of attributes. After,
-   ``f=attrgetter('name')``, the call ``f(b)`` returns ``b.name``.  After,
-   ``f=attrgetter('name', 'date')``, the call ``f(b)`` returns ``(b.name,
+   ``f = attrgetter('name')``, the call ``f(b)`` returns ``b.name``.  After,
+   ``f = attrgetter('name', 'date')``, the call ``f(b)`` returns ``(b.name,
    b.date)``.
 
+   The attribute names can also contain dots; after ``f = attrgetter('date.month')``,
+   the call ``f(b)`` returns ``b.date.month``.
 
 .. function:: itemgetter(item[, args...])
 
@@ -443,6 +445,15 @@ Examples::
    [('orange', 1), ('banana', 2), ('apple', 3), ('pear', 5)]
 
 
+.. function:: methodcaller(name[, args...])
+
+   Return a callable object that calls the method *name* on its operand.  If
+   additional arguments and/or keyword arguments are given, they will be given
+   to the method as well.  After ``f = methodcaller('name')``, the call ``f(b)``
+   returns ``b.name()``.  After ``f = methodcaller('name', 'foo', bar=1)``, the
+   call ``f(b)`` returns ``b.name('foo', bar=1)``.
+
+
 .. _operator-map:
 
 Mapping Operators to Functions
diff --git a/Doc/library/thread.rst b/Doc/library/thread.rst
index 31d58e7..d7d140e 100644
--- a/Doc/library/thread.rst
+++ b/Doc/library/thread.rst
@@ -147,6 +147,11 @@ In addition to these methods, lock objects can also be used via the
   exception will be received by an arbitrary thread.  (When the :mod:`signal`
   module is available, interrupts always go to the main thread.)
 
+* The import machinery is not thread safe.  In general, an import may not
+  have the side effect of importing a module, and only the main thread
+  should import modules.  Imports within or caused by a thread other than
+  the main thread isn't safe.
+
 * Calling :func:`sys.exit` or raising the :exc:`SystemExit` exception is
   equivalent to calling :func:`exit`.
 
@@ -167,4 +172,3 @@ In addition to these methods, lock objects can also be used via the
 * When the main thread exits, it does not do any of its usual cleanup (except
   that :keyword:`try` ... :keyword:`finally` clauses are honored), and the
   standard I/O files are not flushed.
-
diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst
index 6f3e95b..56b2b99 100644
--- a/Doc/library/threading.rst
+++ b/Doc/library/threading.rst
@@ -555,6 +555,13 @@ the :meth:`setDaemon` method and retrieved with the :meth:`isDaemon` method.
 There is a "main thread" object; this corresponds to the initial thread of
 control in the Python program.  It is not a daemon thread.
 
+.. warning::
+   
+   The import machinery is not thread safe.  In general, an import may not
+   have the side effect of importing a module, and only the main thread
+   should import modules.  Imports within or caused by a thread other than
+   the main thread isn't safe.
+
 There is the possibility that "dummy thread objects" are created. These are
 thread objects corresponding to "alien threads", which are threads of control
 started outside the threading module, such as directly from C code.  Dummy
diff --git a/Doc/library/xml.dom.rst b/Doc/library/xml.dom.rst
index 5ddd6a8..768c9c8 100644
--- a/Doc/library/xml.dom.rst
+++ b/Doc/library/xml.dom.rst
@@ -641,8 +641,8 @@ of that class.
 
 .. method:: Element.removeAttribute(name)
 
-   Remove an attribute by name.  No exception is raised if there is no matching
-   attribute.
+   Remove an attribute by name.  If there is no matching attribute, a
+   :exc:`NotFoundErr` is raised.
 
 
 .. method:: Element.removeAttributeNode(oldAttr)