Removed old dynamic linking chapter.

Integrated information on Windows DLLs from Chris Phoenix (Thanks!).

Added "Acknowledgements" area to thank contributing authors.

1 files changed, 148 insertions, 213 deletions

diff --git a/Doc/ext/ext.tex b/Doc/ext/ext.tex
index 646d951..4f527d5 100644
--- a/Doc/ext/ext.tex
+++ b/Doc/ext/ext.tex
@@ -19,6 +19,17 @@
 
 \input{copyright}
 
+%begin{latexonly}
+\vspace{1in}
+%end{latexonly}
+\strong{\large Acknowledgements}
+
+% XXX This needs to be checked and updated manually before each
+% release.
+
+The following people have contributed sections to this document:  Jim
+Fulton, Konrad Hinsen, Chris Phoenix, and Neil Schemenauer.
+
 \begin{abstract}
 
 \noindent
@@ -203,7 +214,7 @@ function call, since you should be able to tell from the return value.
 
 When a function \var{f} that calls another function \var{g} detects
 that the latter fails, \var{f} should itself return an error value
-(e.g. \NULL{} or \code{-1}).  It should \emph{not} call one of the
+(e.g.\ \NULL{} or \code{-1}).  It should \emph{not} call one of the
 \cfunction{PyErr_*()} functions --- one has already been called by \var{g}.
 \var{f}'s caller is then supposed to also return an error indication
 to \emph{its} caller, again \emph{without} calling \cfunction{PyErr_*()},
@@ -223,7 +234,7 @@ To ignore an exception set by a function call that failed, the exception
 condition must be cleared explicitly by calling \cfunction{PyErr_Clear()}. 
 The only time C code should call \cfunction{PyErr_Clear()} is if it doesn't
 want to pass the error on to the interpreter but wants to handle it
-completely by itself (e.g. by trying something else or pretending
+completely by itself (e.g.\ by trying something else or pretending
 nothing happened).
 
 Note that a failing \cfunction{malloc()} call must be turned into an
@@ -262,7 +273,7 @@ static PyObject *SpamError;
 \end{verbatim}
 
 and initialize it in your module's initialization function
-(\cfunction{initspam()}) with an exception object, e.g. (leaving out
+(\cfunction{initspam()}) with an exception object, e.g.\ (leaving out
 the error checking for now):
 
 \begin{verbatim}
@@ -742,20 +753,30 @@ Some example calls:
     long k, l;
     char *s;
     int size;
+\end{verbatim}
 
+\begin{verbatim}
     ok = PyArg_ParseTuple(args, ""); /* No arguments */
         /* Python call: f() */
+\end{verbatim}
 
+\begin{verbatim}
     ok = PyArg_ParseTuple(args, "s", &s); /* A string */
         /* Possible Python call: f('whoops!') */
+\end{verbatim}
 
+\begin{verbatim}
     ok = PyArg_ParseTuple(args, "lls", &k, &l, &s); /* Two longs and a string */
         /* Possible Python call: f(1, 2, 'three') */
+\end{verbatim}
 
+\begin{verbatim}
     ok = PyArg_ParseTuple(args, "(ii)s#", &i, &j, &s, &size);
         /* A pair of ints and a string, whose size is also returned */
         /* Possible Python call: f((1, 2), 'three') */
+\end{verbatim}
 
+\begin{verbatim}
     {
         char *file;
         char *mode = "r";
@@ -767,7 +788,9 @@ Some example calls:
            f('spam', 'w')
            f('spam', 'wb', 100000) */
     }
+\end{verbatim}
 
+\begin{verbatim}
     {
         int left, top, right, bottom, h, v;
         ok = PyArg_ParseTuple(args, "((ii)(ii))(ii)",
@@ -776,7 +799,9 @@ Some example calls:
         /* Possible Python call:
            f(((0, 0), (400, 300)), (10, 10)) */
     }
+\end{verbatim}
 
+\begin{verbatim}
     {
         Py_complex c;
         ok = PyArg_ParseTuple(args, "D:myfunction", &c);
@@ -1310,8 +1335,8 @@ interpreter. When modules are used as shared libraries, however, the
 symbols defined in one module may not be visible to another module.
 The details of visibility depend on the operating system; some systems
 use one global namespace for the Python interpreter and all extension
-modules (e.g. Windows), whereas others require an explicit list of
-imported symbols at module link time (e.g. AIX), or offer a choice of
+modules (e.g.\ Windows), whereas others require an explicit list of
+imported symbols at module link time (e.g.\ AIX), or offer a choice of
 different strategies (most Unices). And even if symbols are globally
 visible, the module whose functions one wishes to call might not have
 been loaded yet!
@@ -1513,7 +1538,7 @@ Python source code distribution).
 \chapter{Building C and \Cpp{} Extensions on \UNIX{}
          \label{building-on-unix}}
 
-\sectionauthor{Fim Fulton}{jim@Digicool.com}
+\sectionauthor{Jim Fulton}{jim@Digicool.com}
 
 
 %The make file make file, building C extensions on Unix
@@ -1537,7 +1562,7 @@ The make file make file, \file{Makefile.pre.in} uses metadata
 provided in a file named \file{Setup}.  The format of the \file{Setup}
 file is the same as the \file{Setup} (or \file{Setup.in}) file
 provided in the \file{Modules/} directory of the Python source
-distribution.  The \file{Setup} file contains variable definitions::
+distribution.  The \file{Setup} file contains variable definitions:
 
 \begin{verbatim}
 EC=/projects/ExtensionClass
@@ -1631,7 +1656,8 @@ Several compiler options are supported:
   \lineii{-C}{Tell the C pre-processor not to discard comments}
   \lineii{-D\var{name}=\var{value}}{Define a macro}
   \lineii{-I\var{dir}}{Specify an include directory, \var{dir}}
-  \lineii{-L\var{dir}}{Specify a library directory, \var{dir}}
+  \lineii{-L\var{dir}}{Specify a link-time library directory, \var{dir}}
+  \lineii{-R\var{dir}}{Specify a run-time library directory, \var{dir}}
   \lineii{-l\var{lib}}{Link a library, \var{lib}}
   \lineii{-U\var{name}}{Undefine a macro}
 \end{tableii}
@@ -1679,15 +1705,31 @@ people who do not have a source distribution of Python.
 Do not distribute a make file.  People building your modules
 should use \file{Makefile.pre.in} to build their own make file.
 
+Work is being done to make building and installing Python extensions
+easier for all platforms; this work in likely to supplant the current
+approach at some point in the future.  For more information or to
+participate in the effort, refer to
+\url{http://www.python.org/sigs/distutils-sig/} on the Python Web
+site.
 
-\chapter{Building C and \Cpp{} Extensions on Windows
-         \label{building-on-unix}}
 
-\sectionauthor{Neil Schemenauer}{neil_schemenauer@transcanada.com}
+\chapter{Building C and \Cpp{} Extensions on Windows
+         \label{building-on-windows}}
 
 
 This chapter briefly explains how to create a Windows extension module
-for Python using Microsoft Visual \Cpp{}.
+for Python using Microsoft Visual \Cpp{}, and follows with more
+detailed background information on how it works.  The explanatory
+material is useful for both the Windows programmer learning to build
+Python extensions and the \UNIX{} programming interested in producing
+software which can be successfully built on both \UNIX{} and Windows.
+
+\section{A Cookbook Approach \label{win-cookbook}}
+
+\sectionauthor{Neil Schemenauer}{neil_schemenauer@transcanada.com}
+
+This section provides a recipe for building a Python extension on
+Windows.
 
 Grab the binary installer from \url{http://www.python.org/} and
 install Python.  The binary installer has all of the required header
@@ -1729,6 +1771,100 @@ Refer to section 3 of the Python FAQ
 do this.
 
 
+\section{Differences Between \UNIX{} and Windows
+         \label{dynamic-linking}}
+\sectionauthor{Chris Phoenix}{cphoenix@best.com}
+
+
+\UNIX{} and Windows use completely different paradigms for run-time
+loading of code.  Before you try to build a module that can be
+dynamically loaded, be aware of how your system works.
+
+In \UNIX{}, a shared object (.so) file contains code to be used by the
+program, and also the names of functions and data that it expects to
+find in the program.  When the file is joined to the program, all
+references to those functions and data in the file's code are changed
+to point to the actual locations in the program where the functions
+and data are placed in memory.  This is basically a link operation.
+
+In Windows, a dynamic-link library (\file{.dll}) file has no dangling
+references.  Instead, an access to functions or data goes through a
+lookup table.  So the DLL code does not have to be fixed up at runtime
+to refer to the program's memory; instead, the code already uses the
+DLL's lookup table, and the lookup table is modified at runtime to
+point to the functions and data.
+
+In \UNIX{}, there is only one type of library file (\file{.a}) which
+contains code from several object files (\file{.o}).  During the link
+step to create a shared object file (\file{.so}), the linker may find
+that it doesn't know where an identifier is defined.  The linker will
+look for it in the object files in the libraries; if it finds it, it
+will include all the code from that object file.
+
+In Windows, there are two types of library, a static library and an
+import library (both called \file{.lib}).  A static library is like a
+\UNIX{} \file{.a} file; it contains code to be included as necessary.
+An import library is basically used only to reassure the linker that a
+certain identifier is legal, and will be present in the program when
+the DLL is loaded.  So the linker uses the information from the
+import library to build the lookup table for using identifiers that
+are not included in the DLL.  When an application or a DLL is linked,
+an import library may be generated, which will need to be used for all
+future DLLs that depend on the symbols in the application or DLL.
+
+Suppose you are building two dynamic-load modules, B and C, which should
+share another block of code A.  On \UNIX{}, you would \emph{not} pass
+\file{A.a} to the linker for \file{B.so} and \file{C.so}; that would
+cause it to be included twice, so that B and C would each have their
+own copy.  In Windows, building \file{A.dll} will also build
+\file{A.lib}.  You \emph{do} pass \file{A.lib} to the linker for B and
+C.  \file{A.lib} does not contain code; it just contains information
+which will be used at runtime to access A's code.  
+
+In Windows, using an import library is sort of like using \samp{import
+spam}; it gives you access to spam's names, but does not create a
+separate copy.  On \UNIX{}, linking with a library is more like
+\samp{from spam import *}; it does create a separate copy.
+
+
+\section{Using DLLs in Practice \label{win-dlls}}
+\sectionauthor{Chris Phoenix}{cphoenix@best.com}
+
+Windows Python is built in Microsoft Visual \Cpp{}; using other
+compilers may or may not work (though Borland seems to).  The rest of
+this section is MSV\Cpp{} specific.
+
+When creating DLLs in Windows, you must pass \file{python15.lib} to
+the linker.  To build two DLLs, spam and ni (which uses C functions
+found in spam), you could use these commands:
+
+\begin{verbatim}
+cl /LD /I/python/include spam.c ../libs/python15.lib
+cl /LD /I/python/include ni.c spam.lib ../libs/python15.lib
+\end{verbatim}
+
+The first command created three files: \file{spam.obj},
+\file{spam.dll} and \file{spam.lib}.  \file{Spam.dll} does not contain
+any Python functions (such as \cfunction{PyArg_ParseTuple()}), but it
+does know how to find the Python code thanks to \file{python15.lib}.
+
+The second command created \file{ni.dll} (and \file{.obj} and
+\file{.lib}), which knows how to find the necessary functions from
+spam, and also from the Python executable.
+
+Not every identifier is exported to the lookup table.  If you want any
+other modules (including Python) to be able to see your identifiers,
+you have to say \samp{_declspec(dllexport)}, as in \samp{void
+_declspec(dllexport) initspam(void)} or \samp{PyObject
+_declspec(dllexport) *NiGetSpamData(void)}.
+
+Developer Studio will throw in a lot of import libraries that you do
+not really need, adding about 100K to your executable.  To get rid of
+them, use the Project Settings dialog, Link tab, to specify
+\emph{ignore default libraries}.  Add the correct
+\file{msvcrt\var{xx}.lib} to the list of libraries.
+
+
 \chapter{Embedding Python in Another Application
          \label{embedding}}
 
@@ -1767,205 +1903,4 @@ will need to write the main program in \Cpp{}, and use the \Cpp{} compiler
 to compile and link your program.  There is no need to recompile Python
 itself using \Cpp{}.
 
-
-\chapter{Dynamic Loading
-         \label{dynload}}
-
-On most modern systems it is possible to configure Python to support
-dynamic loading of extension modules implemented in C.  When shared
-libraries are used dynamic loading is configured automatically;
-otherwise you have to select it as a build option (see below).  Once
-configured, dynamic loading is trivial to use: when a Python program
-executes \code{import spam}, the search for modules tries to find a
-file \file{spammodule.o} (\file{spammodule.so} when using shared
-libraries) in the module search path,%
-\indexiii{module}{search}{path}
-and if one is found, it is loaded into the executing binary and
-executed.  Once loaded, the module acts just like a built-in extension
-module.
-
-The advantages of dynamic loading are twofold: the ``core'' Python
-binary gets smaller, and users can extend Python with their own
-modules implemented in C without having to build and maintain their
-own copy of the Python interpreter.  There are also disadvantages:
-dynamic loading isn't available on all systems (this just means that
-on some systems you have to use static loading), and dynamically
-loading a module that was compiled for a different version of Python
-(e.g. with a different representation of objects) may dump core.
-
-
-\section{Configuring and Building the Interpreter for Dynamic Loading
-         \label{dynloadConfig}}
-
-There are three styles of dynamic loading: one using shared libraries,
-one using SGI IRIX 4 dynamic loading, and one using GNU dynamic
-loading.
-
-\subsection{Shared Libraries
-            \label{sharedlibs}}
-
-The following systems support dynamic loading using shared libraries:
-SunOS 4; Solaris 2; SGI IRIX 5 (but not SGI IRIX 4!), Linux, FreeBSD,
-NetBSD; and probably all systems derived from SVR4, or at least those
-SVR4 derivatives that support shared libraries (are there any that
-don't?).
-
-You don't need to do anything to configure dynamic loading on these
-systems --- the \file{configure} detects the presence of the
-\code{<dlfcn.h>} header file and automatically configures dynamic
-loading.
-
-\subsection{SGI IRIX 4 Dynamic Loading
-            \label{irixDynload}}
-
-Only SGI IRIX 4 supports dynamic loading of modules using SGI dynamic
-loading.  (SGI IRIX 5 might also support it but it is inferior to
-using shared libraries so there is no reason to; a small test didn't
-work right away so I gave up trying to support it.)
-
-Before you build Python, you first need to fetch and build the
-\code{dl} package written by Jack Jansen.  This is available by
-anonymous ftp from \url{ftp://ftp.cwi.nl/pub/dynload/}, file
-\file{dl-1.6.tar.Z}.  (The version number may change.)  Follow the
-instructions in the package's \file{README} file to build it.
-
-Once you have built \code{dl}, you can configure Python to use it.  To
-this end, you run the \program{configure} script with the option
-\code{--with-dl=\var{directory}} where \var{directory} is the absolute
-pathname of the \code{dl} directory.
-
-Now build and install Python as you normally would (see the
-\file{README} file in the toplevel Python directory.)
-
-\subsection{GNU Dynamic Loading
-            \label{gnuDynload}}
-
-GNU dynamic loading supports (according to its \file{README} file) the
-following hardware and software combinations: VAX (Ultrix), Sun 3
-(SunOS 3.4 and 4.0), Sparc (SunOS 4.0), Sequent Symmetry (Dynix), and
-Atari ST.  There is no reason to use it on a Sparc; I haven't seen a
-Sun 3 for years so I don't know if these have shared libraries or not.
-
-You need to fetch and build two packages.
-One is GNU DLD.  All development of this code has been done with DLD
-version 3.2.3, which is available by anonymous ftp from
-\url{ftp://ftp.cwi.nl/pub/dynload}, file
-\file{dld-3.2.3.tar.Z}.  (A more recent version of DLD is available
-via \url{http://www-swiss.ai.mit.edu/~jaffer/DLD.html} but this has
-not been tested.)
-The other package needed is an
-emulation of Jack Jansen's \code{dl} package that I wrote on top of
-GNU DLD 3.2.3.  This is available from the same host and directory,
-file \file{dl-dld-1.1.tar.Z}.  (The version number may change --- but I doubt
-it will.)  Follow the instructions in each package's \file{README}
-file to configure and build them.
-
-Now configure Python.  Run the \file{configure} script with the option
-\code{--with-dl-dld=\var{dl-directory},\var{dld-directory}} where
-\var{dl-directory} is the absolute pathname of the directory where you
-have built the \file{dl-dld} package, and \var{dld-directory} is that
-of the GNU DLD package.  The Python interpreter you build hereafter
-will support GNU dynamic loading.
-
-
-\section{Building a Dynamically Loadable Module
-         \label{makedynload}}
-
-Since there are three styles of dynamic loading, there are also three
-groups of instructions for building a dynamically loadable module.
-Instructions common for all three styles are given first.  Assuming
-your module is called \module{spam}, the source filename must be
-\file{spammodule.c}, so the object name is \file{spammodule.o}.  The
-module must be written as a normal Python extension module (as
-described earlier).
-
-Note that in all cases you will have to create your own Makefile that
-compiles your module file(s).  This Makefile will have to pass two
-\samp{-I} arguments to the C compiler which will make it find the
-Python header files.  If the Make variable \makevar{PYTHONTOP} points to
-the toplevel Python directory, your \makevar{CFLAGS} Make variable should
-contain the options \samp{-I\$(PYTHONTOP) -I\$(PYTHONTOP)/Include}.
-(Most header files are in the \file{Include/} subdirectory, but the
-\file{config.h} header lives in the toplevel directory.)
-
-
-\subsection{Shared Libraries
-            \label{linking}}
-
-You must link the \file{.o} file to produce a shared library.  This is 
-done using a special invocation of the \UNIX{} loader/linker,
-\manpage{ld}{1}.  Unfortunately the invocation differs slightly per
-system.
-
-On SunOS 4, use
-\begin{verbatim}
-ld spammodule.o -o spammodule.so
-\end{verbatim}
-
-On Solaris 2, use
-\begin{verbatim}
-ld -G spammodule.o -o spammodule.so
-\end{verbatim}
-
-On SGI IRIX 5, use
-\begin{verbatim}
-ld -shared spammodule.o -o spammodule.so
-\end{verbatim}
-
-On other systems, consult the manual page for \manpage{ld}{1} to find
-what flags, if any, must be used.
-
-If your extension module uses system libraries that haven't already
-been linked with Python (e.g. a windowing system), these must be
-passed to the \program{ld} command as \samp{-l} options after the
-\samp{.o} file.
-
-The resulting file \file{spammodule.so} must be copied into a directory
-along the Python module search path.
-
-
-\subsection{SGI IRIX 4 Dynamic Loading
-            \label{irixLinking}}
-
-\strong{IMPORTANT:} You must compile your extension module with the
-additional C flag \samp{-G0} (or \samp{-G 0}).  This instructs the
-assembler to generate position-independent code.
-
-You don't need to link the resulting \file{spammodule.o} file; just
-copy it into a directory along the Python module search path.%
-\indexiii{module}{search}{path}
-
-The first time your extension is loaded, it takes some extra time and
-a few messages may be printed.  This creates a file
-\file{spammodule.ld} which is an image that can be loaded quickly into
-the Python interpreter process.  When a new Python interpreter is
-installed, the \code{dl} package detects this and rebuilds
-\file{spammodule.ld}.  The file \file{spammodule.ld} is placed in the
-directory where \file{spammodule.o} was found, unless this directory is
-unwritable; in that case it is placed in a temporary
-directory.\footnote{Check the manual page of the \code{dl} package for
-details.}
-
-If your extension modules uses additional system libraries, you must
-create a file \file{spammodule.libs} in the same directory as the
-\file{spammodule.o}.  This file should contain one or more lines with
-whitespace-separated options that will be passed to the linker ---
-normally only \samp{-l} options or absolute pathnames of libraries
-(\samp{.a} files) should be used.
-
-
-\subsection{GNU Dynamic Loading
-            \label{gnuLinking}}
-
-Just copy \file{spammodule.o} into a directory along the Python module
-search path.%
-\indexiii{module}{search}{path}
-
-If your extension modules uses additional system libraries, you must
-create a file \file{spammodule.libs} in the same directory as the
-\file{spammodule.o}.  This file should contain one or more lines with
-whitespace-separated absolute pathnames of libraries (\samp{.a}
-files).  No \samp{-l} options can be used.
-
-
 \end{document}