Merged revisions 66141,66145,66150,66180,66211,66217,66219,66226,66231,66244,66246,66249-66250,66264,66268,66272,66294,66306 via svnmerge from

svn+ssh://pythondev@svn.python.org/python/trunk

........
  r66141 | gregory.p.smith | 2008-09-02 00:29:51 -0500 (Tue, 02 Sep 2008) | 3 lines

  Issue #3678: Correctly pass LDFLAGS and LDLAST to the linker on shared
  library targets in the Makefile.
........
  r66145 | marc-andre.lemburg | 2008-09-02 05:32:34 -0500 (Tue, 02 Sep 2008) | 5 lines

  Add quotes around the file name to avoid issues with spaces.

  Closes #3719.
........
  r66150 | marc-andre.lemburg | 2008-09-02 07:11:19 -0500 (Tue, 02 Sep 2008) | 3 lines

  Add news item for #3719.
........
  r66180 | vinay.sajip | 2008-09-03 04:20:05 -0500 (Wed, 03 Sep 2008) | 1 line

  Issue #3726: Allowed spaces in separators in logging configuration files.
........
  r66211 | vinay.sajip | 2008-09-04 02:31:21 -0500 (Thu, 04 Sep 2008) | 1 line

  Issue #3772: Fixed regression problem in StreamHandler.emit().
........
  r66217 | andrew.kuchling | 2008-09-04 08:26:24 -0500 (Thu, 04 Sep 2008) | 1 line

  #3671: various corrections and markup fixes noted by Kent Johnson
........
  r66219 | hirokazu.yamamoto | 2008-09-04 09:25:30 -0500 (Thu, 04 Sep 2008) | 1 line

  Added NEWS
........
  r66226 | benjamin.peterson | 2008-09-04 18:31:27 -0500 (Thu, 04 Sep 2008) | 1 line

  flesh out the documentation on using 2to3
........
  r66231 | andrew.kuchling | 2008-09-05 10:15:56 -0500 (Fri, 05 Sep 2008) | 1 line

  #3671: Typo fix
........
  r66244 | jesse.noller | 2008-09-05 20:20:11 -0500 (Fri, 05 Sep 2008) | 2 lines

  Fix typo in multiprocessing doc, cancel_join_thread was missing _thread
........
  r66246 | benjamin.peterson | 2008-09-05 22:00:00 -0500 (Fri, 05 Sep 2008) | 1 line

  actually tell the name of the flag to use
........
  r66249 | andrew.kuchling | 2008-09-06 07:50:05 -0500 (Sat, 06 Sep 2008) | 1 line

  Various corrections
........
  r66250 | andrew.kuchling | 2008-09-06 08:04:02 -0500 (Sat, 06 Sep 2008) | 1 line

  #3040: include 'dest' argument in example; trim some trailing whitespace
........
  r66264 | benjamin.peterson | 2008-09-06 14:42:39 -0500 (Sat, 06 Sep 2008) | 1 line

  docs are pretty good about new-style classes these days
........
  r66268 | andrew.kuchling | 2008-09-06 15:28:01 -0500 (Sat, 06 Sep 2008) | 1 line

  #3669 from Robert Lehmann: simplify use of iterator in example
........
  r66272 | andrew.kuchling | 2008-09-06 16:26:02 -0500 (Sat, 06 Sep 2008) | 1 line

  #1317: describe the does_esmtp, ehlo_resp, esmtp_features, and helo_resp attributes
........
  r66294 | georg.brandl | 2008-09-07 12:00:17 -0500 (Sun, 07 Sep 2008) | 2 lines

  Add a new howto about Python and the web, by Marek Kubica.
........
  r66306 | mark.summerfield | 2008-09-08 09:45:37 -0500 (Mon, 08 Sep 2008) | 3 lines

  Added xrefs to each other.
........

13 files changed, 888 insertions, 76 deletions

diff --git a/Doc/howto/index.rst b/Doc/howto/index.rst
index 68aab60..5a1f397 100644
--- a/Doc/howto/index.rst
+++ b/Doc/howto/index.rst
@@ -21,4 +21,5 @@ Currently, the HOWTOs are:
    sockets.rst
    unicode.rst
    urllib2.rst
+   webservers.rst
 
diff --git a/Doc/howto/webservers.rst b/Doc/howto/webservers.rst
new file mode 100644
index 0000000..97c2267
--- /dev/null
+++ b/Doc/howto/webservers.rst
@@ -0,0 +1,697 @@
+*******************************
+  HOWTO Use Python in the web
+*******************************
+
+:Author: Marek Kubica
+
+.. topic:: Abstract
+
+   This document shows how Python fits into the web.  It presents some ways on
+   how to integrate Python with the web server and general practices useful for
+   developing web sites.
+
+
+Programming for the Web has become a hot topic since the raise of the "Web 2.0",
+which focuses on user-generated content on web sites.  It has always been
+possible to use Python for creating web sites, but it was a rather tedious task.
+Therefore, many so-called "frameworks" and helper tools were created to help
+developers creating sites faster and these sites being more robust.  This HOWTO
+describes some of the methods used to combine Python with a web server to create
+dynamic content.  It is not meant as a general introduction as this topic is far
+too broad to be covered in one single document.  However, a short overview of
+the most popular libraries is provided.
+
+.. seealso::
+
+   While this HOWTO tries to give an overview over Python in the Web, it cannot
+   always be as up to date as desired.  Web development in Python is moving
+   forward rapidly, so the wiki page on `Web Programming
+   <http://wiki.python.org/moin/WebProgramming>`_ might be more in sync with
+   recent development.
+
+
+The low-level view
+==================
+
+.. .. image:: http.png
+
+When a user enters a web site, his browser makes a connection to the site's
+webserver (this is called the *request*).  The server looks up the file in the
+file system and sends it back to the user's browser, which displays it (this is
+the *response*).  This is roughly how the unterlying protocol, HTTP works.
+
+Now, dynamic web sites are not files in the file system, but rather programs
+which are run by the web server when a request comes in.  They can do all sorts
+of useful things, like display the postings of a bulletin board, show your
+mails, configurate software or just display the current time.  These programs
+can be written in about any programming language the server supports, so it is
+easy to use Python for creating dynamic web sites.
+
+As most of HTTP servers are written in C or C++, they cannot execute Python code
+in a simple way -- a bridge is needed between the server and the program.  These
+bridges or rather interfaces define how programs interact with the server.  In
+the past there have been numerous attempts to create the best possible
+interface, but there are only a few worth mentioning.
+
+Not every web server supports every interface.  Many web servers do support only
+old, now-obsolete interfaces.  But they can often be extended using some
+third-party modules to support new interfaces.
+
+
+Common Gateway Interface
+------------------------
+
+This interface is the oldest one, supported by nearly every web server out of
+the box.  Programs using CGI to communicate with their web server need to be
+started by the server for every request.  So, every request starts a new Python
+interpreter -- which takes some time to start up -- thus making the whole
+interface only usable for low load situations.
+
+The upside of CGI is that it is simple -- writing a program which uses CGI is a
+matter of about three lines of code.  But this simplicity comes at a price: it
+does very few things to help the developer.
+
+Writing CGI programs, while still possible, is not recommended anymore.  With
+WSGI (more on that later) it is possible to write programs that emulate CGI, so
+they can be run as CGI if no better option is available.
+
+.. seealso::
+
+   The Python standard library includes some modules that are helpful for
+   creating plain CGI programs:
+
+   * :mod:`cgi` -- Handling of user input in CGI scripts
+   * :mod:`cgitb` -- Displays nice tracebacks when errors happen in of CGI
+     applications, instead of presenting a "500 Internal Server Error" message
+
+   The Python wiki features a page on `CGI scripts
+   <http://wiki.python.org/moin/CgiScripts>`_ with some additional information
+   about CGI in Python.
+
+   
+Simple script for testing CGI
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To test whether your web server works with CGI, you can use this short and
+simple CGI program::
+
+    #!/usr/bin/env python
+    # -*- coding: UTF-8 -*-
+
+    # enable debugging
+    import cgitb; cgitb.enable()
+
+    print "Content-Type: text/plain;charset=utf-8"
+    print
+
+    print "Hello World!"
+
+You need to write this code into a file with a ``.py`` or ``.cgi`` extension,
+this depends on your web server configuration.  Depending on your web server
+configuration, this file may also need to be in a ``cgi-bin`` folder, for
+security reasons.
+
+You might wonder what the ``cgitb`` line is about.  This line makes it possible
+to display a nice traceback instead of just crashing and displaying an "Internal
+Server Error" in the user's browser.  This is useful for debugging, but it might
+risk exposing some confident data to the user.  Don't use it when the script is
+ready for production use.  Still, you should *always* catch exceptions, and
+display proper error pages -- end-users don't like to see nondescript "Internal
+Server Errors" in their browsers.
+
+
+Setting up CGI on your own server
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+If you don't have your own web server, this does not apply to you.  You can
+check whether if works as-is and if not you need to talk to the administrator of
+your web server anyway. If it is a big hoster, you can try filing a ticket
+asking for Python support.
+
+If you're your own administrator or want to install it for testing purposes on
+your own computers, you have to configure it by yourself.  There is no one and
+single way on how to configure CGI, as there are many web servers with different
+configuration options.  The currently most widely used free web server is
+`Apache HTTPd <http://httpd.apache.org/>`_, Apache for short -- this is the one
+that most people use, it can be easily installed on nearly every system using
+the systems' package management.  But `lighttpd <http://www.lighttpd.net>`_ has
+been gaining attention since some time and is said to have a better performance.
+On many systems this server can also be installed using the package management,
+so manually compiling the web server is never needed.
+
+* On Apache you can take a look into the `Dynamic Content with CGI
+  <http://httpd.apache.org/docs/2.2/howto/cgi.html>`_ tutorial, where everything
+  is described.  Most of the time it is enough just to set ``+ExecCGI``.  The
+  tutorial also describes the most common gotchas that might arise.
+* On lighttpd you need to use the `CGI module
+  <http://trac.lighttpd.net/trac/wiki/Docs%3AModCGI>`_ which can be configured
+  in a straightforward way.  It boils down to setting ``cgi.assign`` properly.
+
+
+Common problems with CGI scripts
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Trying to use CGI sometimes leads to small annoyances that one might experience
+while trying to get these scripts to run.  Sometimes it happens that a seemingly
+correct script does not work as expected, which is caused by some small hidden
+reason that's difficult to spot.
+
+Some of these reasons are:
+
+* The Python script is not marked executable.  When CGI scripts are not
+  executable most of the web servers will let the user download it, instead of
+  running it and sending the output to the user.  For CGI scripts to run
+  properly the ``+x`` bit needs to be set.  Using ``chmod a+x your_script.py``
+  might already solve the problem.
+* The line endings must be of Unix-type.  This is important because the web
+  server checks the first line of the script (called shebang) and tries to run
+  the program specified there.  It gets easily confused by Windows line endings
+  (Carriage Return & Line Feed, also called CRLF), so you have to convert the
+  file to Unix line endings (only Line Feed, LF).  This can be done
+  automatically by uploading the file via FTP in text mode instead of binary
+  mode, but the preferred way is just telling your editor to save the files with
+  Unix line endings.  Most proper editors support this.
+* Your web server must be able to read the file, you need to make sure the
+  permissions are fine.  Often the server runs as user and group ``www-data``,
+  so it might be worth a try to change the file ownership or making the file
+  world readable by using ``chmod a+r your_script.py``.
+* The webserver must be able to know that the file you're trying to access is a
+  CGI script.  Check the configuration of your web server, maybe there is some
+  mistake.
+* The path to the interpreter in the shebang (``#!/usr/bin/env python``) must be
+  currect.  This line calls ``/usr/bin/env`` to find Python, but it'll fail if
+  there is no ``/usr/bin/env``.  If you know where your Python is installed, you
+  can also use that path.  The commands ``whereis python`` and ``type -p
+  python`` might also help to find where it is installed.  Once this is known,
+  the shebang line can be changed accordingly: ``#!/usr/bin/python``.
+* The file must not contain a BOM (Byte Order Mark). The BOM is meant for
+  determining the byte order of UTF-16 encodings, but some editors write this
+  also into UTF-8 files.  The BOM interferes with the shebang line, so be sure
+  to tell your editor not to write the BOM.
+* :ref:`mod-python` might be making problems.  mod_python is able to handle CGI
+  scripts by itself, but it can also be a source for problems.  Be sure you
+  disable it.
+
+
+.. _mod-python:
+
+mod_python
+----------
+
+People coming from PHP often find it hard to grasp how to use Python in the web.
+Their first thought is mostly `mod_python <http://www.modpython.org/>`_ because
+they think that this is the equivalent to ``mod_php``.  Actually it is not
+really.  It does embed the interpreter into the Apache process, thus speeding up
+requests by not having to start a Python interpreter every request.  On the
+other hand, it is by far not "Python intermixed with HTML" as PHP often does.
+The Python equivalent of that is a template engine.  mod_python itself is much
+more powerful and gives more access to Apache internals.  It can emulate CGI, it
+can work an a "Python Server Pages" mode similar to JSP which is "HTML
+intermangled with Python" and it has a "Publisher" which destignates one file to
+accept all requests and decide on what to do then.
+
+But mod_python has some problems.  Unlike the PHP interpreter the Python
+interpreter uses caching when executing files, so when changing a file the whole
+web server needs to be re-started to update.  Another problem ist the basic
+concept -- Apache starts some child processes to handle the requests and
+unfortunately every child process needs to load the whole Python interpreter
+even if it does not use it.  This makes the whole web server slower.  Another
+problem is that as mod_python is linked against a specific version of
+``libpython``, it is not possible to switch from an older version to a newer
+(e.g. 2.4 to 2.5) without recompiling mod_python.  mod_python is also bound to
+the Apache web server, so programs written for mod_python cannot easily run on
+other web servers.
+
+These are the reasons why mod_python should be avoided when writing new
+programs.  In some circumstances it might be still a good idea to use mod_python
+for deployment, but WSGI makes it possible to run WSGI programs under mod_python
+as well.
+
+
+FastCGI and SCGI
+----------------
+
+FastCGI and SCGI try to solve the performance problem of CGI in another way.
+Instead of embedding the interpreter into the web server, they create
+long-running processes which run in the background. There still is some module
+in the web server which makes it possible for the web server to "speak" with the
+background process.  As the background process is independent from the server,
+it can be written in any language of course also in Python.  The language just
+needs to have a library which handles the communication with the web server.
+
+The difference between FastCGI and SCGI is very small, as SCGI is essentially
+just a "simpler FastCGI".  But as the web server support for SCGI is limited
+most people use FastCGI instead, which works the same way.  Almost everything
+that applies to SCGI also applies to FastCGI as well, so we'll only write about
+the latter.
+
+These days, FastCGI is never used directly.  Just like ``mod_python`` it is only
+used for the deployment of WSGI applications.
+
+.. seealso::
+
+   * `FastCGI, SCGI, and Apache: Background and Future
+     <http://www.vmunix.com/mark/blog/archives/2006/01/02/fastcgi-scgi-and-apache-background-and-future/>`_
+     is a discussion on why the concept of FastCGI and SCGI is better that that
+     of mod_python.
+
+
+Setting up FastCGI
+^^^^^^^^^^^^^^^^^^
+
+Depending on the web server you need to have a special module.
+
+* Apache has both `mod_fastcgi <http://www.fastcgi.com/>`_ and `mod_fcgid
+  <http://fastcgi.coremail.cn/>`_.  ``mod_fastcgi`` is the original one, but it
+  has some licensing issues that's why it is sometimes considered non-free.
+  ``mod_fcgid`` is a smaller, compatible alternative. One of these modules needs
+  to be loaded by Apache.
+* lighttpd ships its own `FastCGI module
+  <http://trac.lighttpd.net/trac/wiki/Docs%3AModFastCGI>`_ as well as an `SCGI
+  module <http://trac.lighttpd.net/trac/wiki/Docs%3AModSCGI>`_.
+* nginx also supports `FastCGI
+  <http://wiki.codemongers.com/NginxSimplePythonFCGI>`_.
+
+Once you have installed and configured the module, you can test it with the
+following WSGI-application::
+
+    #!/usr/bin/env python
+    # -*- coding: UTF-8 -*-
+
+    from cgi import escape
+    import sys, os
+    from flup.server.fcgi import WSGIServer
+
+    def app(environ, start_response):
+        start_response('200 OK', [('Content-Type', 'text/html')])
+
+        yield '<h1>FastCGI Environment</h1>'
+        yield '<table>'
+        for k, v in sorted(environ.items()):
+             yield '<tr><th>%s</th><td>%s</td></tr>' % (escape(k), escape(v))
+        yield '</table>'
+
+    WSGIServer(app).run()
+
+This is a simple WSGI application, but you need to install `flup
+<http://pypi.python.org/pypi/flup/1.0>`_ first, as flup handles the low level
+FastCGI access.
+
+.. seealso::
+
+   There is some documentation on `setting up Django with FastCGI
+   <http://www.djangoproject.com/documentation/fastcgi/>`_, most of which can be
+   reused for other WSGI-compliant frameworks and libraries.  Only the
+   ``manage.py`` part has to be changed, the example used here can be used
+   instead. Django does more or less the exact same thing.
+
+
+mod_wsgi
+--------
+
+`mod_wsgi <http://www.modwsgi.org/>`_ is an attempt to get rid of the low level
+gateways.  As FastCGI, SCGI, mod_python are mostly used to deploy WSGI
+applications anyway, mod_wsgi was started to directly embed WSGI aplications
+into the Apache web server.  The benefit from this approach is that WSGI
+applications can be deployed much easier as is is specially designed to host
+WSGI applications -- unlike the other low level methods which have glue code to
+host WSGI applications (like flup which was mentioned before).  The downside is
+that mod_wsgi is limited to the Apache web server, other servers would need
+their own implementations of mod_wsgi.
+
+It supports two modes: the embedded mode in which it integrates with the Apache
+process and the daemon mode which is more FastCGI-like.  Contrary to FastCGI,
+mod_wsgi handles the worker-processes by itself which makes administration
+easier.
+
+
+.. _WSGI:
+
+Step back: WSGI
+===============
+
+WSGI was already mentioned several times so it has to be something important.
+In fact it really is, so now it's time to explain.
+
+The *Web Server Gateway Interface*, :pep:`333` or WSGI for short is currently
+the best possible way to Python web programming.  While it is great for
+programmers writing frameworks, the normal person does not need to get in direct
+contact with it.  But when choosing a framework for web development it is a good
+idea to take one which supports WSGI.
+
+The big profit from WSGI is the unification.  When your program is compatible
+with WSGI -- that means that your framework has support for WSGI, your program
+can be deployed on every web server interface for which there are WSGI wrappers.
+So you do not need to care about whether the user uses mod_python or FastCGI --
+with WSGI it just works on any gateway interface.  The Python standard library
+contains its own WSGI server :mod:`wsgiref`, which is a small web server that
+can be used for testing.
+
+A really great WSGI feature are the middlewares.  Middlewares are layers around
+your program which can add various functionality to it.  There is a `number of
+middlewares <http://wsgi.org/wsgi/Middleware_and_Utilities>`_ already available.
+For example, instead of writing your own session management (to identify a user
+in subsequent requests, as HTTP does not maintain state, so it does now know
+that the requests belong to the same user) you can just take one middleware,
+plug it in and you can rely an already existing functionality.  The same thing
+is compression -- say you want to compress your HTML using gzip, to save your
+server's bandwidth.  So you only need to plug-in a middleware and you're done.
+Authentication is also a problem easily solved using a middleware.
+
+So, generally -- although WSGI may seem complex, the initial phase of learning
+can be very rewarding as WSGI does already have solutions to many problems that
+might arise while writing web sites.
+
+
+WSGI Servers
+------------
+
+The code that is used to connect to various low level gateways like CGI or
+mod_python is called *WSGI server*.  One of these servers is ``flup`` which was
+already mentioned and supports FastCGI, SCGI as well as `AJP
+<http://en.wikipedia.org/wiki/Apache_JServ_Protocol>`_.  Some of these servers
+are written in Python as ``flup`` is, but there also exist others which are
+written in C and can be used as drop-in replacements.
+
+There are quite a lot of servers already available, so a Python web application
+can be deployed nearly everywhere.  This is one big advantage that Python has
+compared with other web techniques.
+
+.. seealso::
+
+   A good overview of all WSGI-related code can be found in the `WSGI wiki
+   <http://wsgi.org/wsgi>`_, which contains an extensive list of `WSGI servers
+   <http://wsgi.org/wsgi/Servers>`_, which can be used by *every* application
+   supporting WSGI.
+
+   You might be interested in some WSGI-supporting modules already contained in
+   the standard library, namely:
+    
+   * :mod:`wsgiref` -- some tiny utilities and servers for WSGI
+
+
+Case study: MoinMoin
+--------------------
+
+What does WSGI give the web application developer?  Let's take a look on one
+long existing web application written in Python without using WSGI.
+
+One of the most widely used wiki software is `MoinMoin <http://moinmo.in/>`_.
+It was created in 2000, so it predates WSGI by about three years.  While it now
+includes support for WSGI, older versions needed separate code to run on CGI,
+mod_python, FastCGI and standalone.  Now, this all is possible by using WSGI and
+the already-written gateways.  For running with on FastCGI ``flup`` can be used,
+for running a standalone server :mod:`wsgiref` is the way to go.
+
+
+Model-view-controller
+=====================
+
+The term *MVC* is often heard in statements like "framework *foo* supports MVC".
+While MVC is not really something technical but rather organisational, many web
+frameworks use this model to help the developer to bring structure into his
+program.  Bigger web applications can have lots of code so it is a good idea to
+have structure in the program right from the beginnings.  That way, even users
+of other frameworks (or even languages, as MVC is nothing Python-specific) can
+understand the existing code easier, as they are already familiar with the
+structure.
+
+MVC stands for three components:
+
+* The *model*.  This is the data that is meant to modify.  In Python frameworks
+  this component is often represented by the classes used by the
+  object-relational mapper.  So, all declarations go here.
+* The *view*.  This component's job is to display the data of the model to the
+  user.  Typically this component is represented by the templates.
+* The *controller*.  This is the layer between the user and the model.  The
+  controller reacts on user actions (like opening some specific URL) and tells
+  the model to modify the data if neccessary.
+
+While one might think that MVC is a complex design pattern, in fact it is not.
+It is used in Python because it has turned out to be useful for creating clean,
+maintainable web sites.
+
+.. note::
+
+   While not all Python frameworks explicitly support MVC, it is often trivial
+   to create a web site which uses the MVC pattern by seperating the data logic
+   (the model) from the user interaction logic (the controller) and the
+   templates (the view).  That's why it is important not to write unneccessary
+   Python code in the templates -- it is against MVC and creates more chaos.
+
+.. seealso::
+
+   The english Wikipedia has an article about the `Model-View-Controller pattern
+   <http://en.wikipedia.org/wiki/Model-view-controller>`_, which includes a long
+   list of web frameworks for different programming languages.
+
+
+Ingredients for web sites
+=========================
+
+Web sites are complex constructs, so tools were created to help the web site
+developer to make his work maintainable.  None of these tools are in any way
+Python specific, they also exist for other programming languages as well.  Of
+course, developers are not forced to use these tools and often there is no
+"best" tool, but it is worth informing yourself before choosing something
+because of the big number of helpers that the developer can use.
+
+
+.. seealso::
+
+   People have written far more components that can be combined than these
+   presented here.  The Python wiki has a page about these components, called
+   `Web Components <http://wiki.python.org/moin/WebComponents>`_.
+
+
+Templates
+---------
+
+Mixing of HTML and Python code is possible with some libraries.  While
+convenient at first, it leads to horribly unmaintainable code.  That's why
+templates exist.  Templates are, in the simplest case, just HTML files with
+placeholders.  The HTML is sent to the user's browser after filling out the
+placeholders.
+
+Python already includes such simple templates::
+
+    # a simple template
+    template = "<html><body><h1>Hello %s!</h1></body></html>"
+    print template % "Reader"
+
+The Python standard library also includes some more advanced templates usable
+through :class:`string.Template`, but in HTML templates it is needed to use
+conditional and looping contructs like Python's *for* and *if*.  So, some
+*template engine* is needed.
+
+Now, Python has a lot of template engines which can be used with or without a
+`framework`_.  Some of these are using a plain-text programming language which
+is very easy to learn as it is quite limited while others use XML so the
+template output is always guaranteed to be valid XML.  Some `frameworks`_ ship
+their own template engine or recommend one particular.  If one is not yet sure,
+using these is a good idea.
+
+.. note::
+
+   While Python has quite a lot of different template engines it usually does
+   not make sense to use a homebrewed template system.  The time needed to
+   evaluate all templating systems is not really worth it, better invest the
+   time in looking through the most popular ones.  Some frameworks have their
+   own template engine or have a recommentation for one.  It's wise to use
+   these.
+  
+   Popular template engines include:
+
+   * Mako
+   * Genshi
+   * Jinja
+
+.. seealso::
+
+   Lots of different template engines divide the attention between themselves
+   because it's easy to create them in Python.  The page `Templating
+   <http://wiki.python.org/moin/Templating>`_ in the wiki lists a big,
+   ever-growing number of these.
+
+
+Data persistence
+----------------
+
+*Data persistence*, while sounding very complicated is just about storing data.
+This data might be the text of blog entries, the postings of a bulletin board or
+the text of a wiki page.  As always, there are different ways to store
+informations on a web server.
+
+Often relational database engines like `MySQL <http://www.mysql.com/>`_ or
+`PostgreSQL <http://http://www.postgresql.org/>`_ are used due to their good
+performance handling very large databases consisting of up to millions of
+entries.  These are *queried* using a language called `SQL
+<http://en.wikipedia.org/wiki/SQL>`_.  Python programmers in general do not like
+SQL too much, they prefer to work with objects.  It is possible to save Python
+objects into a database using a technology called `ORM
+<http://en.wikipedia.org/wiki/Object-relational_mapping>`_.  ORM translates all
+object-oriented access into SQL code under the hood, the user does not need to
+think about it.  Most `frameworks`_ use ORMs and it works quite well.
+
+A second possibility is using files that are saved on the hard disk (sometimes
+called flatfiles).  This is very easy, but is not too fast.  There is even a
+small database engine called `SQLite <http://www.sqlite.org/>`_ which is bundled
+with Python in the :mod:`sqlite` module and uses only one file.  This database
+can be used to store objects via an ORM and has no other dependencies.  For
+smaller sites SQLite is just enough.  But it is not the only way in which data
+can be saved into the file systems.  Sometimes normal, plain text files are
+enough.
+
+The third and least used possibility are so-called object oriented databases.
+These databases store the *actual objects* instead of the relations that
+OR-mapping creates between rows in a database.  This has the advantage that
+nearly all objects can be saven in a straightforward way, unlike in relational
+databases where some objects are very hard to represent with ORMs.
+
+`Frameworks`_ often give the users hints on which method to choose, it is
+usually a good idea to stick to these unless there are some special requirements
+which require to use the one method and not the other.
+
+.. seealso::
+
+   * `Persistence Tools <http://wiki.python.org/moin/PersistenceTools>`_ lists
+     possibilities on how to save data in the file system, some of these modules
+     are part of the standard library
+   * `Database Programming <http://wiki.python.org/moin/DatabaseProgramming>`_
+     helps on choosing a method on how to save the data
+   * `SQLAlchemy <http://www.sqlalchemy.org/>`_, the most powerful OR-Mapper for
+     Python and `Elixir <http://elixir.ematia.de/>`_ which makes it easier to
+     use
+   * `SQLObject <http://www.sqlobject.org/>`_, another popular OR-Mapper
+   * `ZODB <https://launchpad.net/zodb>`_ and `Durus
+     <http://www.mems-exchange.org/software/durus/>`_, two object oriented
+     databases
+
+
+.. _framework:
+
+Frameworks
+==========
+
+As web sites can easily become quite large, there are so-called frameworks which
+were created to help the developer with making these sites.  Although the most
+well-known framework is Ruby on Rails, Python does also have its own frameworks
+which are partly inspired by Rails or which were existing a long time before
+Rails.
+
+Two possible approaches to web frameworks exist: the minimalistic approach and
+the all-inclusive approach (somtimes called *full-stack*). Frameworks which are
+all-inclusive give you everything you need to start working, like a template
+engine, some way to save and access data in databases and many features more.
+Most users are best off using these as they are widely used by lots of other
+users and well documented in form of books and tutorials.  Other web frameworks
+go the minimalistic approach trying to be as flexible as possible leaving the
+user the freedom to choose what's best for him.
+
+The majority of users is best off with all-inclusive framewors.  They bring
+everything along so a user can just jump in and start to code.  While they do
+have some limitations they can fullfill 80% of what one will ever want to
+perfectly.  They consist of various components which are designed to work
+together as good as possible.
+
+The multitude of web frameworks written in Python demonstrates that it is really
+easy to write one.  One of the most well-known web applications written in
+Python is `Zope <http://www.zope.org/>`_ which can be regarded as some kind of
+big framework.  But Zope was not the only framework, there were some others
+which are by now nearly forgotten.  These do not need to be mentioned anymore,
+because most people that used them moved on to newer ones.
+
+
+Some notable frameworks
+-----------------------
+
+There is an incredible number of frameworks, so there is no way to describe them
+all.  It is not even neccessary, as most of these frameworks are nothing special
+and everything that can be done with these can also be done with one of the
+popular ones.
+
+
+Django
+^^^^^^
+
+`Django <http://www.djangoproject.com/>`_ is a framework consisting of several
+tightly coupled elements which were written from scratch and work together very
+well.  It includes an ORM which is quite powerful while being simple to use and
+has a great online administration interface which makes it possible to edit the
+data in the database with a browser.  The template engine is text-based and is
+designed to be usable for page designers who cannot write Python.  It supports
+so-called template inheritance and filters (which work like Unix pipes).  Django
+has many handy features bundled, like creation of RSS feeds or generic views
+which make it possible to write web sites nearly without any Python code.
+
+It has a big, international community which has created many sites using Django.
+There are also quite a lot of add-on projects which extend Django's normal
+functionality.  This is partly due to Django's well written `online
+documentation <http://doc.djangoproject.com/>`_ and the `Django book
+<http://www.djangobook.com/>`_.
+
+
+.. note::
+
+   Although Django is an MVC-style framework, it calls the components
+   differently, which is described in the `Django FAQ
+   <http://www.djangoproject.com/documentation/faq/#django-appears-to-be-a-mvc-framework-but-you-call-the-controller-the-view-and-the-view-the-template-how-come-you-don-t-use-the-standard-names>`_.
+
+
+TurboGears
+^^^^^^^^^^
+
+The other popular web framework in Python is `TurboGears
+<http://www.turbogears.org/>`_.  It takes the approach of using already existing
+components and combining them with glue code to create a seamless experience.
+TurboGears gives the user more flexibility on which components to choose, the
+ORM can be switched between some easy to use but limited and complex but very
+powerful.  Same goes for the template engine.  One strong point about TurboGears
+is that the components that it consists of can be used easily in other projects
+without depending on TurboGears, for example the underlying web server CherryPy.
+
+The documentation can be found in the `TurboGears wiki
+<http://docs.turbogears.org/>`_, where links to screencasts can be found.
+TurboGears has also an active user community which can respond to most related
+questions.  There is also a `TurboGears book <http://turbogearsbook.com/>`_
+published, which is a good starting point.
+
+The plan for the next major version of TurboGears, version 2.0 is to switch to a
+more flexible base provided by another very flexible web framework called
+`Pylons <http://pylonshq.com/>`_.
+
+
+Other notable frameworks
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+These two are of course not the only frameworks that are available, there are
+also some less-popular frameworks worth mentioning.
+
+One of these is the already mentioned Zope, which has been around for quite a
+long time.  With Zope 2.x having been known as rather un-pythonic, the newer
+Zope 3.x tries to change that and therefore gets more acceptance from Python
+programmers.  These efforts already showed results, there is a project which
+connects Zope with WSGI called `Repoze <http://repoze.org/>`_ and another
+project called `Grok <http://grok.zope.org/>`_ which makes it possible for
+"normal" Python programmers use the very mature Zope components.
+
+Another framework that's already been mentioned is `Pylons`_.  Pylons is much
+like TurboGears with ab even stronger emphasis on flexibility, which is bought
+at the cost of being more difficult to use.  Nearly every component can be
+exchanged, which makes it neccessary to use the documentation of every single
+component, because there are so many Pylons combinations possible that can
+satisfy every requirement.  Pylons builds upon `Paste
+<http://pythonpaste.org/>`_, an extensive set of tools which are handy for WSGI.
+
+And that's still not everything.  The most up-to-date information can always be
+found in the Python wiki.
+
+.. seealso::
+    
+   The Python wiki contains an extensive list of `web frameworks
+   <http://wiki.python.org/moin/WebFrameworks>`_.
+
+   Most frameworks also have their own mailing lists and IRC channels, look out
+   for these on the projects' websites.  There is also a general "Python in the
+   Web" IRC channel on freenode called `#python.web
+   <http://wiki.python.org/moin/PoundPythonWeb>`_.
diff --git a/Doc/library/2to3.rst b/Doc/library/2to3.rst
index 99749b3..e8ea861 100644
--- a/Doc/library/2to3.rst
+++ b/Doc/library/2to3.rst
@@ -7,15 +7,21 @@
 
 2to3 is a Python program that reads Python 2.x source code and applies a series
 of *fixers* to transform it into valid Python 3.x code.  The standard library
-contains a rich set of fixers that will handle almost all code.  It is, however,
-possible to write your own fixers.
+contains a rich set of fixers that will handle almost all code.  2to3 supporting
+library :mod:`lib2to3` is, however, a flexible and generic library, so it is
+possible to write your own fixers for 2to3.  :mod:`lib2to3` could also be
+adapted to custom applications in which Python code needs to be edited
+automatically.
 
 
 Using 2to3
 ----------
 
-2to3 can be run with a list of files to transform or a directory to recursively
-traverse looking for files with the ``.py`` extension.
+2to3 will usually be installed with the Python interpreter as a script.  It is
+also located in the :file:`Tools/scripts` directory of the Python root.
+
+2to3's basic arguments are a list of files or directories to transform.  The
+directories are to recursively traversed for Python sources.
 
 Here is a sample Python 2.x source file, :file:`example.py`::
 
@@ -29,13 +35,14 @@ It can be converted to Python 3.x code via 2to3 on the command line::
 
    $ 2to3 example.py
 
-A diff against the original source file will be printed.  2to3 can also write
-the needed modifications right back to the source file.  (A backup of the
-original file will also be made.)  This is done with the :option:`-w` flag::
+A diff against the original source file is printed.  2to3 can also write the
+needed modifications right back to the source file.  (Of course, a backup of the
+original is also be made.)  Writing the changes back is enabled with the
+:option:`-w` flag::
 
    $ 2to3 -w example.py
 
-:file:`example.py` will now look like this::
+After transformation, :file:`example.py` looks like this::
 
    def greet(name):
        print("Hello, {0}!".format(name))
@@ -43,10 +50,10 @@ original file will also be made.)  This is done with the :option:`-w` flag::
    name = input()
    greet(name)
 
-Comments and and exact indentation will be preserved throughout the translation
+Comments and and exact indentation are preserved throughout the translation
 process.
 
-By default, 2to3 will run a set of predefined fixers.  The :option:`-l` flag
+By default, 2to3 runs a set of predefined fixers.  The :option:`-l` flag
 lists all avaible fixers.  An explicit set of fixers to run can be given by use
 of the :option:`-f` flag.  The following example runs only the ``imports`` and
 ``has_key`` fixers::
@@ -54,16 +61,30 @@ of the :option:`-f` flag.  The following example runs only the ``imports`` and
    $ 2to3 -f imports -f has_key example.py
 
 Some fixers are *explicit*, meaning they aren't run be default and must be
-listed on the command line.  Here, in addition to the default fixers, the
-``idioms`` fixer is run::
+listed on the command line to be run.  Here, in addition to the default fixers,
+the ``idioms`` fixer is run::
 
    $ 2to3 -f all -f idioms example.py
 
-Notice how ``all`` enables all default fixers.
+Notice how passing ``all`` enables all default fixers.
 
 Sometimes 2to3 will find will find a place in your source code that needs to be
 changed, but 2to3 cannot fix automatically.  In this case, 2to3 will print a
-warning beneath the diff for a file.
+warning beneath the diff for a file.  You should address the warning in order to
+have compliant 3.x code.
+
+2to3 can also refactor doctests.  To enable this mode, use the :option:`-d`
+flag.  Note that *only* doctests will be refactored.
+
+The :option:`-v` option enables the output of more information on the
+translation process.
+
+When the :option:`-p` is passed to it, 2to3 treats ``print`` as a function
+instead of a statement.  This is useful when ``from __future__ import
+print_function`` is being used.  If this option is not given, the print fixer
+will surround print calls in an extra set of parentheses because it cannot
+differentiate between the and print statement with parentheses (such as ``print
+("a" + "b" + "c")``) and a true function call.
 
 
 :mod:`lib2to3` - 2to3's library
diff --git a/Doc/library/getopt.rst b/Doc/library/getopt.rst
index cdc40bf..4bf5bef 100644
--- a/Doc/library/getopt.rst
+++ b/Doc/library/getopt.rst
@@ -11,7 +11,12 @@ This module helps scripts to parse the command line arguments in ``sys.argv``.
 It supports the same conventions as the Unix :cfunc:`getopt` function (including
 the special meanings of arguments of the form '``-``' and '``--``').  Long
 options similar to those supported by GNU software may be used as well via an
-optional third argument. This module provides two functions and an
+optional third argument.
+
+A more convenient, flexible, and powerful alternative is the
+:mod:`optparse` module.
+
+This module provides two functions and an
 exception:
 
 
diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
index a247e2f..3621412 100644
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -1859,7 +1859,7 @@ Joining processes that use queues
     Bear in mind that a process that has put items in a queue will wait before
     terminating until all the buffered items are fed by the "feeder" thread to
     the underlying pipe.  (The child process can call the
-    :meth:`Queue.cancel_join` method of the queue to avoid this behaviour.)
+    :meth:`Queue.cancel_join_thread` method of the queue to avoid this behaviour.)
 
     This means that whenever you use a queue you need to make sure that all
     items which have been put on the queue will eventually be removed before the
diff --git a/Doc/library/optparse.rst b/Doc/library/optparse.rst
index bd0b02a..de1a116 100644
--- a/Doc/library/optparse.rst
+++ b/Doc/library/optparse.rst
@@ -8,7 +8,7 @@
 
 
 ``optparse`` is a more convenient, flexible, and powerful library for parsing
-command-line options than ``getopt``.  ``optparse`` uses a more declarative
+command-line options than the old :mod:`getopt` module.  ``optparse`` uses a more declarative
 style of command-line parsing: you create an instance of :class:`OptionParser`,
 populate it with options, and parse the command line. ``optparse`` allows users
 to specify options in the conventional GNU/POSIX syntax, and additionally
@@ -92,7 +92,7 @@ argument
    ``sys.argv[1:]``, or of some other list provided as a substitute for
    ``sys.argv[1:]``".
 
-option   
+option
    an argument used to supply extra information to guide or customize the execution
    of a program.  There are many different syntaxes for options; the traditional
    Unix syntax is a hyphen ("-") followed by a single letter, e.g. ``"-x"`` or
@@ -464,7 +464,7 @@ user-friendly (documented) options::
                      action="store_true", dest="verbose", default=True,
                      help="make lots of noise [default]")
    parser.add_option("-q", "--quiet",
-                     action="store_false", dest="verbose", 
+                     action="store_false", dest="verbose",
                      help="be vewwy quiet (I'm hunting wabbits)")
    parser.add_option("-f", "--filename",
                      metavar="FILE", help="write output to FILE"),
@@ -1632,7 +1632,7 @@ arguments::
        setattr(parser.values, option.dest, value)
 
    [...]
-   parser.add_option("-c", "--callback",
+   parser.add_option("-c", "--callback", dest="vararg_attr",
                      action="callback", callback=vararg_callback)
 
 The main weakness with this particular implementation is that negative numbers
diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst
index 31f1ad6..74be645 100644
--- a/Doc/library/smtplib.rst
+++ b/Doc/library/smtplib.rst
@@ -171,6 +171,8 @@ An :class:`SMTP` instance has the following methods:
 
    Identify yourself to the SMTP server using ``HELO``.  The hostname argument
    defaults to the fully qualified domain name of the local host.
+   The message returned by the server is stored as the :attr:`helo_resp` attribute
+   of the object.
 
    In normal operation it should not be necessary to call this method explicitly.
    It will be implicitly called by the :meth:`sendmail` when necessary.
@@ -180,7 +182,13 @@ An :class:`SMTP` instance has the following methods:
 
    Identify yourself to an ESMTP server using ``EHLO``.  The hostname argument
    defaults to the fully qualified domain name of the local host.  Examine the
-   response for ESMTP option and store them for use by :meth:`has_extn`.
+   response for ESMTP option and store them for use by :meth:`has_extn`. 
+   Also sets several informational attributes: the message returned by 
+   the server is stored as the :attr:`ehlo_resp` attribute, :attr:`does_esmtp` 
+   is set to true or false depending on whether the server supports ESMTP, and
+   :attr:`esmtp_features` will be a dictionary containing the names of the
+   SMTP service extensions this server supports, and their
+   parameters (if any).
 
    Unless you wish to use :meth:`has_extn` before sending mail, it should not be
    necessary to call this method explicitly.  It will be implicitly called by
diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst
index baf12e8..3e3faca 100644
--- a/Doc/library/sqlite3.rst
+++ b/Doc/library/sqlite3.rst
@@ -419,7 +419,7 @@ A :class:`Connection` instance has the following attributes and methods:
       import sqlite3, os
 
       con = sqlite3.connect('existing_db.db')
-      full_dump = os.linesep.join([line for line in con.iterdump()])
+      full_dump = os.linesep.join(con.iterdump())
       f = open('dump.sql', 'w')
       f.writelines(full_dump)
       f.close()
diff --git a/Doc/whatsnew/2.6.rst b/Doc/whatsnew/2.6.rst
index 883eb90..af7991d 100644
--- a/Doc/whatsnew/2.6.rst
+++ b/Doc/whatsnew/2.6.rst
@@ -63,7 +63,7 @@ what it can, adding compatibility functions in a
 usages that will become unsupported in 3.0.
 
 Some significant new packages have been added to the standard library,
-such as the :mod:`multiprocessing` and :mod:`jsonlib` modules, but
+such as the :mod:`multiprocessing` and :mod:`json` modules, but
 there aren't many new features that aren't related to Python 3.0 in
 some way.
 
@@ -623,7 +623,7 @@ versa.)
 Two other classes, :class:`Pool` and :class:`Manager`, provide
 higher-level interfaces.  :class:`Pool` will create a fixed number of
 worker processes, and requests can then be distributed to the workers
-by calling :meth:`apply` or `apply_async` to add a single request,
+by calling :meth:`apply` or :meth:`apply_async` to add a single request,
 and :meth:`map` or :meth:`map_async` to add a number of
 requests.  The following code uses a :class:`Pool` to spread requests
 across 5 worker processes and retrieve a list of results::
@@ -977,10 +977,10 @@ sequence of bytes::
     bytearray(b'ABC')
     >>> b = bytearray(u'\u21ef\u3244', 'utf-8')
     >>> b
-    bytearray(b'\xe2\x87\xaf \xe3\x89\x84')
+    bytearray(b'\xe2\x87\xaf\xe3\x89\x84')
     >>> b[0] = '\xe3'
     >>> b
-    bytearray(b'\xe3\x87\xaf \xe3\x89\x84')
+    bytearray(b'\xe3\x87\xaf\xe3\x89\x84')
     >>> unicode(str(b), 'utf-8')
     u'\u31ef \u3244'
 
@@ -1975,7 +1975,7 @@ changes, or look through the Subversion logs for all the details.
 
 * A new function in the :mod:`heapq` module, ``merge(iter1, iter2, ...)``,
   takes any number of iterables returning data in sorted
-  order, and returns a new iterator that returns the contents of all
+  order, and returns a new generator that returns the contents of all
   the iterators, also in sorted order.  For example::
 
      heapq.merge([1, 3, 5, 9], [2, 8, 16]) ->
@@ -2014,56 +2014,56 @@ changes, or look through the Subversion logs for all the details.
   others, the missing values are set to *fillvalue*.  For example::
 
      itertools.izip_longest([1,2,3], [1,2,3,4,5]) ->
-       [(1, 1), (2, 2), (3, 3), (None, 4), (None, 5)]
+       (1, 1), (2, 2), (3, 3), (None, 4), (None, 5)
 
   ``product(iter1, iter2, ..., [repeat=N])`` returns the Cartesian product
   of the supplied iterables, a set of tuples containing
   every possible combination of the elements returned from each iterable. ::
 
      itertools.product([1,2,3], [4,5,6]) ->
-       [(1, 4), (1, 5), (1, 6),
+        (1, 4), (1, 5), (1, 6),
 	(2, 4), (2, 5), (2, 6),
-	(3, 4), (3, 5), (3, 6)]
+	(3, 4), (3, 5), (3, 6)
 
   The optional *repeat* keyword argument is used for taking the
   product of an iterable or a set of iterables with themselves,
   repeated *N* times.  With a single iterable argument, *N*-tuples
   are returned::
 
-     itertools.product([1,2], repeat=3)) ->
-       [(1, 1, 1), (1, 1, 2), (1, 2, 1), (1, 2, 2),
-        (2, 1, 1), (2, 1, 2), (2, 2, 1), (2, 2, 2)]
+     itertools.product([1,2], repeat=3) ->
+        (1, 1, 1), (1, 1, 2), (1, 2, 1), (1, 2, 2),
+        (2, 1, 1), (2, 1, 2), (2, 2, 1), (2, 2, 2)
 
   With two iterables, *2N*-tuples are returned. ::
 
-     itertools(product([1,2], [3,4], repeat=2) ->
-       [(1, 3, 1, 3), (1, 3, 1, 4), (1, 3, 2, 3), (1, 3, 2, 4),
+     itertools.product([1,2], [3,4], repeat=2) ->
+        (1, 3, 1, 3), (1, 3, 1, 4), (1, 3, 2, 3), (1, 3, 2, 4),
         (1, 4, 1, 3), (1, 4, 1, 4), (1, 4, 2, 3), (1, 4, 2, 4),
         (2, 3, 1, 3), (2, 3, 1, 4), (2, 3, 2, 3), (2, 3, 2, 4),
-        (2, 4, 1, 3), (2, 4, 1, 4), (2, 4, 2, 3), (2, 4, 2, 4)]
+        (2, 4, 1, 3), (2, 4, 1, 4), (2, 4, 2, 3), (2, 4, 2, 4)
 
   ``combinations(iterable, r)`` returns sub-sequences of length *r* from
   the elements of *iterable*. ::
 
     itertools.combinations('123', 2) ->
-      [('1', '2'), ('1', '3'), ('2', '3')]
+       ('1', '2'), ('1', '3'), ('2', '3')
 
     itertools.combinations('123', 3) ->
-      [('1', '2', '3')]
+       ('1', '2', '3')
 
     itertools.combinations('1234', 3) ->
-      [('1', '2', '3'), ('1', '2', '4'), ('1', '3', '4'),
-       ('2', '3', '4')]
+       ('1', '2', '3'), ('1', '2', '4'), ('1', '3', '4'),
+       ('2', '3', '4')
 
   ``permutations(iter[, r])`` returns all the permutations of length *r* of
   the iterable's elements.  If *r* is not specified, it will default to the
   number of elements produced by the iterable. ::
 
     itertools.permutations([1,2,3,4], 2) ->
-      [(1, 2), (1, 3), (1, 4),
+       (1, 2), (1, 3), (1, 4),
        (2, 1), (2, 3), (2, 4),
        (3, 1), (3, 2), (3, 4),
-       (4, 1), (4, 2), (4, 3)]
+       (4, 1), (4, 2), (4, 3)
 
   ``itertools.chain(*iterables)`` is an existing function in
   :mod:`itertools` that gained a new constructor in Python 2.6.
@@ -2073,7 +2073,7 @@ changes, or look through the Subversion logs for all the details.
   all the elements of the second, and so on. ::
 
     chain.from_iterable([[1,2,3], [4,5,6]]) ->
-       [1, 2, 3, 4, 5, 6]
+        1, 2, 3, 4, 5, 6
 
   (All contributed by Raymond Hettinger.)
 
@@ -2178,7 +2178,7 @@ changes, or look through the Subversion logs for all the details.
   :const:`UF_APPEND` to indicate that data can only be appended to the
   file.  (Contributed by M. Levinson.)
 
-  ``os.closerange(*low*, *high*)`` efficiently closes all file descriptors
+  ``os.closerange(low, high)`` efficiently closes all file descriptors
   from *low* to *high*, ignoring any errors and not including *high* itself.
   This function is now used by the :mod:`subprocess` module to make starting
   processes faster.  (Contributed by Georg Brandl; :issue:`1663329`.)
@@ -2311,12 +2311,12 @@ changes, or look through the Subversion logs for all the details.
   will be ignored, not copied.
 
   The :mod:`shutil` module also provides an :func:`ignore_patterns`
-  function for use with this new parameter.
-  :func:`ignore_patterns` takes an arbitrary number of glob-style patterns
-  and will ignore any files and directories that match any of these patterns.
-  The following example copies a directory tree, but skips both
-  :file:`.svn` directories and Emacs backup
-  files, which have names ending with '~'::
+  function for use with this new parameter.  :func:`ignore_patterns`
+  takes an arbitrary number of glob-style patterns and returns a
+  callable that will ignore any files and directories that match any
+  of these patterns.  The following example copies a directory tree,
+  but skips both :file:`.svn` directories and Emacs backup files,
+  which have names ending with '~'::
 
       shutil.copytree('Doc/library', '/tmp/library',
                       ignore=shutil.ignore_patterns('*~', '.svn'))
@@ -2523,13 +2523,15 @@ changes, or look through the Subversion logs for all the details.
 
   (Contributed by Dwayne Bailey; :issue:`1581073`.)
 
-* The :mod:`threading` module API is being changed to use properties such as
-  :attr:`daemon` instead of :meth:`setDaemon` and :meth:`isDaemon` methods, and
-  some methods have been renamed to use underscores instead of camel-case; for
-  example, the :meth:`activeCount` method is renamed to :meth:`active_count`.
-  The 2.6 version of the module supports the same properties and renamed
-  methods, but doesn't remove the old methods.  3.0 also fully supports both
-  APIs, and a date for the deprecation of the old APIs has not been set yet.
+* The :mod:`threading` module API is being changed to use properties
+  such as :attr:`daemon` instead of :meth:`setDaemon` and
+  :meth:`isDaemon` methods, and some methods have been renamed to use
+  underscores instead of camel-case; for example, the
+  :meth:`activeCount` method is renamed to :meth:`active_count`.  Both
+  the 2.6 and 3.0 versions of the module support the same properties
+  and renamed methods, but don't remove the old methods.  No date has been set
+  for the deprecation of the old APIs in Python 3.x; the old APIs won't 
+  be removed in any 2.x version.
   (Carried out by several people, most notably Benjamin Peterson.)
 
   The :mod:`threading` module's :class:`Thread` objects
@@ -2735,15 +2737,15 @@ of these built-in functions that can be imported when writing
 
 The functions in this module currently include:
 
-* ``ascii(*obj*)``: equivalent to :func:`repr`.  In Python 3.0,
+* ``ascii(obj)``: equivalent to :func:`repr`.  In Python 3.0,
   :func:`repr` will return a Unicode string, while :func:`ascii` will
   return a pure ASCII bytestring.
 
-* ``filter(*predicate*, *iterable*)``,
-  ``map(*func*, *iterable1*, ...)``: the 3.0 versions
+* ``filter(predicate, iterable)``,
+  ``map(func, iterable1, ...)``: the 3.0 versions
   return iterators, unlike the 2.x built-ins which return lists.
 
-* ``hex(*value*)``, ``oct(*value*)``: instead of calling the
+* ``hex(value)``, ``oct(value)``: instead of calling the
   :meth:`__hex__` or :meth:`__oct__` methods, these versions will
   call the :meth:`__index__` method and convert the result to hexadecimal
   or octal.  :func:`oct` will use the new ``0o`` notation for its
@@ -3210,7 +3212,8 @@ that may require changes to your code:
 Acknowledgements
 ================
 
-The author would like to thank the following people for offering suggestions,
-corrections and assistance with various drafts of this article:
-Georg Brandl, Steve Brown, Nick Coghlan, Jim Jewett, Antoine Pitrou.
+The author would like to thank the following people for offering
+suggestions, corrections and assistance with various drafts of this
+article: Georg Brandl, Steve Brown, Nick Coghlan, Jim Jewett, Kent
+Johnson, Chris Lambacher, Antoine Pitrou.
 
diff --git a/Lib/logging/__init__.py b/Lib/logging/__init__.py
index 552ce18..499c954 100644
--- a/Lib/logging/__init__.py
+++ b/Lib/logging/__init__.py
@@ -753,7 +753,7 @@ class StreamHandler(Handler):
                 self.stream.write(fs % msg)
             else:
                 try:
-                    if hasattr(self.stream, 'encoding'):
+                    if getattr(self.stream, 'encoding', None) is not None:
                         self.stream.write(fs % msg.encode(self.stream.encoding))
                     else:
                         self.stream.write(fs % msg)
diff --git a/Lib/logging/config.py b/Lib/logging/config.py
index a9a4e62..93d68a3 100644
--- a/Lib/logging/config.py
+++ b/Lib/logging/config.py
@@ -19,7 +19,7 @@ Configuration functions for the logging package for Python. The core package
 is based on PEP 282 and comments thereto in comp.lang.python, and influenced
 by Apache's log4j system.
 
-Copyright (C) 2001-2007 Vinay Sajip. All Rights Reserved.
+Copyright (C) 2001-2008 Vinay Sajip. All Rights Reserved.
 
 To use, simply 'import logging' and log away!
 """
@@ -98,6 +98,8 @@ def _resolve(name):
             found = getattr(found, n)
     return found
 
+def _strip_spaces(alist):
+    return map(lambda x: x.strip(), alist)
 
 def _create_formatters(cp):
     """Create and return formatters"""
@@ -105,9 +107,10 @@ def _create_formatters(cp):
     if not len(flist):
         return {}
     flist = flist.split(",")
+    flist = _strip_spaces(flist)
     formatters = {}
     for form in flist:
-        sectname = "formatter_%s" % form.strip()
+        sectname = "formatter_%s" % form
         opts = cp.options(sectname)
         if "format" in opts:
             fs = cp.get(sectname, "format", 1)
@@ -133,10 +136,11 @@ def _install_handlers(cp, formatters):
     if not len(hlist):
         return {}
     hlist = hlist.split(",")
+    hlist = _strip_spaces(hlist)
     handlers = {}
     fixups = [] #for inter-handler references
     for hand in hlist:
-        sectname = "handler_%s" % hand.strip()
+        sectname = "handler_%s" % hand
         klass = cp.get(sectname, "class")
         opts = cp.options(sectname)
         if "formatter" in opts:
@@ -189,8 +193,9 @@ def _install_loggers(cp, handlers, disable_existing_loggers):
     hlist = cp.get(sectname, "handlers")
     if len(hlist):
         hlist = hlist.split(",")
+        hlist = _strip_spaces(hlist)
         for hand in hlist:
-            log.addHandler(handlers[hand.strip()])
+            log.addHandler(handlers[hand])
 
     #and now the others...
     #we don't want to lose the existing loggers,
@@ -240,8 +245,9 @@ def _install_loggers(cp, handlers, disable_existing_loggers):
         hlist = cp.get(sectname, "handlers")
         if len(hlist):
             hlist = hlist.split(",")
+            hlist = _strip_spaces(hlist)
             for hand in hlist:
-                logger.addHandler(handlers[hand.strip()])
+                logger.addHandler(handlers[hand])
 
     #Disable any old loggers. There's no point deleting
     #them as other threads may continue to hold references
diff --git a/Lib/test/test_logging.py b/Lib/test/test_logging.py
index 55502d6..75d1366 100644
--- a/Lib/test/test_logging.py
+++ b/Lib/test/test_logging.py
@@ -587,6 +587,48 @@ class ConfigFileTest(BaseTest):
     # config5 specifies a custom handler class to be loaded
     config5 = config1.replace('class=StreamHandler', 'class=logging.StreamHandler')
 
+    # config6 uses ', ' delimiters in the handlers and formatters sections
+    config6 = """
+    [loggers]
+    keys=root,parser
+
+    [handlers]
+    keys=hand1, hand2
+
+    [formatters]
+    keys=form1, form2
+
+    [logger_root]
+    level=WARNING
+    handlers=
+
+    [logger_parser]
+    level=DEBUG
+    handlers=hand1
+    propagate=1
+    qualname=compiler.parser
+
+    [handler_hand1]
+    class=StreamHandler
+    level=NOTSET
+    formatter=form1
+    args=(sys.stdout,)
+
+    [handler_hand2]
+    class=FileHandler
+    level=NOTSET
+    formatter=form1
+    args=('test.blah', 'a')
+
+    [formatter_form1]
+    format=%(levelname)s ++ %(message)s
+    datefmt=
+
+    [formatter_form2]
+    format=%(message)s
+    datefmt=
+    """
+
     def apply_config(self, conf):
         try:
             fn = tempfile.mktemp(".ini")
@@ -653,6 +695,9 @@ class ConfigFileTest(BaseTest):
     def test_config5_ok(self):
         self.test_config1_ok(config=self.config5)
 
+    def test_config6_ok(self):
+        self.test_config1_ok(config=self.config6)
+
 class LogRecordStreamHandler(StreamRequestHandler):
 
     """Handler for a streaming logging request. It saves the log message in the
@@ -814,6 +859,31 @@ class MemoryTest(BaseTest):
             ('foo', 'DEBUG', '3'),
         ])
 
+class EncodingTest(BaseTest):
+    def test_encoding_plain_file(self):
+        # In Python 2.x, a plain file object is treated as having no encoding.
+        log = logging.getLogger("test")
+        fn = tempfile.mktemp(".log")
+        # the non-ascii data we write to the log.
+        data = "foo\x80"
+        try:
+            handler = logging.FileHandler(fn, encoding="utf8")
+            log.addHandler(handler)
+            try:
+                # write non-ascii data to the log.
+                log.warning(data)
+            finally:
+                log.removeHandler(handler)
+                handler.close()
+            # check we wrote exactly those bytes, ignoring trailing \n etc
+            f = open(fn, encoding="utf8")
+            try:
+                self.failUnlessEqual(f.read().rstrip(), data)
+            finally:
+                f.close()
+        finally:
+            if os.path.isfile(fn):
+                os.remove(fn)
 
 # Set the locale to the platform-dependent default.  I have no idea
 # why the test does this, but in any case we save the current locale
@@ -822,7 +892,8 @@ class MemoryTest(BaseTest):
 def test_main():
     run_unittest(BuiltinLevelsTest, BasicFilterTest,
                     CustomLevelsAndFiltersTest, MemoryHandlerTest,
-                    ConfigFileTest, SocketHandlerTest, MemoryTest)
+                    ConfigFileTest, SocketHandlerTest, MemoryTest,
+                    EncodingTest)
 
 if __name__ == "__main__":
     test_main()
diff --git a/Makefile.pre.in b/Makefile.pre.in
index dee3fd0..b5b46b5 100644
--- a/Makefile.pre.in
+++ b/Makefile.pre.in
@@ -410,14 +410,14 @@ $(LIBRARY): $(LIBRARY_OBJS)
 
 libpython$(VERSION).so: $(LIBRARY_OBJS)
 	if test $(INSTSONAME) != $(LDLIBRARY); then \
-		$(LDSHARED) -Wl,-h$(INSTSONAME) -o $(INSTSONAME) $(LIBRARY_OBJS) $(SHLIBS) $(LIBC) $(LIBM); \
+		$(LDSHARED) $(LDFLAGS) -Wl,-h$(INSTSONAME) -o $(INSTSONAME) $(LIBRARY_OBJS) $(SHLIBS) $(LIBC) $(LIBM) $(LDLAST); \
 		$(LN) -f $(INSTSONAME) $@; \
 	else\
-		$(LDSHARED) -o $@ $(LIBRARY_OBJS) $(SHLIBS) $(LIBC) $(LIBM); \
+		$(LDSHARED) $(LDFLAGS) -o $@ $(LIBRARY_OBJS) $(SHLIBS) $(LIBC) $(LIBM) $(LDLAST); \
 	fi
 
 libpython$(VERSION).sl: $(LIBRARY_OBJS)
-	$(LDSHARED) -o $@ $(LIBRARY_OBJS) $(SHLIBS) $(LIBC) $(LIBM)
+	$(LDSHARED) $(LDFLAGS) -o $@ $(LIBRARY_OBJS) $(SHLIBS) $(LIBC) $(LIBM) $(LDLAST)
 
 # This rule is here for OPENSTEP/Rhapsody/MacOSX. It builds a temporary
 # minimal framework (not including the Lib directory and such) in the current
@@ -451,8 +451,8 @@ $(PYTHONFRAMEWORKDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK): \
 # for a shared core library; otherwise, this rule is a noop.
 $(DLLLIBRARY) libpython$(VERSION).dll.a: $(LIBRARY_OBJS)
 	if test -n "$(DLLLIBRARY)"; then \
-		$(LDSHARED) -Wl,--out-implib=$@ -o $(DLLLIBRARY) $^ \
-			$(LIBS) $(MODLIBS) $(SYSLIBS); \
+		$(LDSHARED) $(LDFLAGS) -Wl,--out-implib=$@ -o $(DLLLIBRARY) $^ \
+			$(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST); \
 	else true; \
 	fi