Lots of new text and example code on embedding Python in C, contributed

by Albert Hofkamp.  Some editing has been done for style and markup
consistency.

This also supplies an example of importing modules and calling a function
defined in the module, so this closes SF bug #440037 as well.

(The long example code was moved to a separate file so that it would
format properly.)

2 files changed, 309 insertions, 1 deletions

diff --git a/Doc/ext/ext.tex b/Doc/ext/ext.tex
index 61e423e..0ffec46 100644
--- a/Doc/ext/ext.tex
+++ b/Doc/ext/ext.tex
@@ -2515,6 +2515,17 @@ them, use the Project Settings dialog, Link tab, to specify
 \chapter{Embedding Python in Another Application
      \label{embedding}}
 
+The previous chapters discussed how to extend Python, that is, how to
+extend the functionality of Python by attaching a library of C
+functions to it.  It is also possible to do it the other way around:
+enrich your C/\Cpp{} application by embedding Python in it.  Embedding
+provides your application with the ability to implement some of the
+functionality of your application in Python rather than C or \Cpp.
+This can be used for many purposes; one example would be to allow
+users to tailor the application to their needs by writing some scripts
+in Python.  You can also use it yourself if some of the functionality
+can be written in Python more easily.
+
 Embedding Python is similar to extending it, but not quite.  The
 difference is that when you extend Python, the main program of the
 application is still the Python interpreter, while if you embed
@@ -2525,7 +2536,7 @@ interpreter to run some Python code.
 So if you are embedding Python, you are providing your own main
 program.  One of the things this main program has to do is initialize
 the Python interpreter.  At the very least, you have to call the
-function \cfunction{Py_Initialize()} (on MacOS, call
+function \cfunction{Py_Initialize()} (on Mac OS, call
 \cfunction{PyMac_Initialize()} instead).  There are optional calls to
 pass command line arguments to Python.  Then later you can call the
 interpreter from any part of the application.
@@ -2542,6 +2553,237 @@ A simple demo of embedding Python can be found in the directory
 \file{Demo/embed/} of the source distribution.
 
 
+\begin{seealso}
+  \seetitle[../api/api.html]{Python/C API Reference Manual}{The
+            details of Python's C interface are given in this manual.
+            A great deal of necessary information can be found here.}
+\end{seealso}
+
+
+\section{Very High Level Embedding
+         \label{high-level-embedding}}
+
+The simplest form of embedding Python is the use of the very
+high level interface. This interface is intended to execute a
+Python script without needing to interact with the application
+directly. This can for example be used to perform some operation
+on a file.
+
+\begin{verbatim}
+#include <Python.h>
+
+int main()
+{
+  Py_Initialize();
+  PyRun_SimpleString("from time import time,ctime\n"
+                     "print 'Today is',ctime(time())\n");
+  Py_Finalize();
+  return 0;
+}
+\end{verbatim}
+
+The above code first initializes the Python interpreter with
+\cfunction{Py_Initialize()}, followed by the execution of a hard-coded
+Python script that print the date and time.  Afterwards, the
+\cfunction{Py_Finalize()} call shuts the interpreter down, followed by
+the end of the program.  In a real program, you may want to get the
+Python script from another source, perhaps a text-editor routine, a
+file, or a database.  Getting the Python code from a file can better
+be done by using the \cfunction{PyRun_SimpleFile()} function, which
+saves you the trouble of allocating memory space and loading the file
+contents.
+
+
+\section{Beyond Very High Level Embedding: An overview
+         \label{lower-level-embedding}}
+
+The high level interface gives you the ability to execute
+arbitrary pieces of Python code from your application, but
+exchanging data values is quite cumbersome to say the least. If
+you want that, you should use lower level calls. At the cost of
+having to write more C code, you can achieve almost anything.
+
+It should be noted that extending Python and embedding Python
+is quite the same activity, despite the different intent. Most
+topics discussed in the previous chapters are still valid. To
+show this, consider what the extension code from Python to C
+really does:
+
+\begin{enumerate}
+    \item Convert data values from Python to C,
+    \item Perform a function call to a C routine using the
+        converted values, and
+    \item Convert the data values from the call from C to Python.
+\end{enumerate}
+
+When embedding Python, the interface code does:
+
+\begin{enumerate}
+    \item Convert data values from C to Python,
+    \item Perform a function call to a Python interface routine
+        using the converted values, and
+    \item Convert the data values from the call from Python to C.
+\end{enumerate}
+
+As you can see, the data conversion steps are simply swapped to
+accomodate the different direction of the cross-language transfer.
+The only difference is the routine that you call between both
+data conversions. When extending, you call a C routine, when
+embedding, you call a Python routine.
+
+This chapter will not discuss how to convert data from Python
+to C and vice versa.  Also, proper use of references and dealing
+with errors is assumed to be understood.  Since these aspects do not
+differ from extending the interpreter, you can refer to earlier
+chapters for the required information.
+
+
+\section{Pure Embedding
+         \label{pure-embedding}}
+
+The first program aims to execute a function in a Python
+script. Like in the section about the very high level interface,
+the Python interpreter does not directly interact with the
+application (but that will change in th next section).
+
+The code to run a function defined in a Python script is:
+
+\verbatiminput{run-func.c}
+
+This code loads a Python script using \code{argv[1]}, and calls the
+function named in \code{argv[2]}.  Its integer arguments are the other
+values of the \code{argv} array.  If you compile and link this
+program (let's call the finished executable \program{call}), and use
+it to execute a Python script, such as:
+
+\begin{verbatim}
+def multiply(a,b):
+    print "Thy shall add", a, "times", b
+    c = 0
+    for i in range(0, a):
+        c = c + b
+    return c
+\end{verbatim}
+
+then the result should be:
+
+\begin{verbatim}
+$ call multiply 3 2
+Thy shall add 3 times 2
+Result of call: 6
+\end{verbatim} % $
+
+Although the program is quite large for its functionality, most of the
+code is for data conversion between Python and C, and for error
+reporting.  The interesting part with respect to embedding Python
+starts with
+
+\begin{verbatim}
+    Py_Initialize();
+    pName = PyString_FromString(argv[1]);
+    /* Error checking of pName left out */
+    pModule = PyImport_Import(pName);
+\end{verbatim}
+
+After initializing the interpreter, the script is loaded using
+\cfunction{PyImport_Import()}.  This routine needs a Python string
+as its argument, which is constructed using the
+\cfunction{PyString_FromString()} data conversion routine.
+
+\begin{verbatim}
+    pDict = PyModule_GetDict(pModule);
+    /* pDict is a borrowed reference */
+
+    pFunc = PyDict_GetItemString(pDict, argv[2]);
+    /* pFun is a borrowed reference */
+
+    if (pFunc && PyCallable_Check(pFunc)) {
+        ...
+    }
+\end{verbatim}
+
+Once the script is loaded, its dictionary is retrieved with
+\cfunction{PyModule_GetDict()}.  The dictionary is then searched using
+the normal dictionary access routines for the function name.  If the
+name exists, and the object retunred is callable, you can safely
+assume that it is a function.  The program then proceeds by
+constructing a tuple of arguments as normal.  The call to the python
+function is then made with:
+
+\begin{verbatim}
+    pValue = PyObject_CallObject(pFunc, pArgs);
+\end{verbatim}
+
+Upon return of the function, \code{pValue} is either \NULL{} or it
+contains a reference to the return value of the function.  Be sure to
+release the reference after examining the value.
+
+
+\section{Extending Embedded Python
+         \label{extending-with-embedding}}
+
+Until now, the embedded Python interpreter had no access to
+functionality from the application itself.  The Python API allows this
+by extending the embedded interpreter.  That is, the embedded
+interpreter gets extended with routines provided by the application.
+While it sounds complex, it is not so bad.  Simply forget for a while
+that the application starts the Python interpreter.  Instead, consider
+the application to be a set of subroutines, and write some glue code
+that gives Python access to those routines, just like you would write
+a normal Python extension.  For example:
+
+\begin{verbatim}
+static int numargs=0;
+
+/* Return the number of arguments of the application command line */
+static PyObject*
+emb_numargs(PyObject *self, PyObject *args)
+{
+    if(!PyArg_ParseTuple(args, ":numargs"))
+        return NULL;
+    return Py_BuildValue("i", numargs);
+}
+
+static PyMethodDef EmbMethods[]={
+    {"numargs", emb_numargs, METH_VARARGS},
+    {NULL,      NULL}
+};
+\end{verbatim}
+
+Insert the above code just above the \cfunction{main()} function.
+Also, insert the following two statements directly after
+\cfunction{Py_Initialize()}:
+
+\begin{verbatim}
+    numargs = argc;
+    Py_InitModule("emb", EmbMethods);
+\end{verbatim}
+
+These two lines initialize the \code{numargs} variable, and make the
+\function{emb.numargs()} function accessible to the embedded Python
+interpreter.  With these extensions, the Python script can do things
+like
+
+\begin{verbatim}
+import emb
+print "Number of arguments", emb.numargs()
+\end{verbatim}
+
+In a real application, the methods will expose an API of the
+application to Python.
+
+
+%\section{For the future}
+%
+%You don't happen to have a nice library to get textual
+%equivalents of numeric values do you :-) ?
+%Callbacks here ? (I may be using information from that section
+%?!)
+%threads
+%code examples do not really behave well if errors happen
+% (what to watch out for)
+
+
 \section{Embedding Python in \Cpp{}
      \label{embeddingInCplusplus}}
 
diff --git a/Doc/ext/run-func.c b/Doc/ext/run-func.c
new file mode 100644
index 0000000..fb2a9ae
--- /dev/null
+++ b/Doc/ext/run-func.c
@@ -0,0 +1,66 @@
+#include <stdio.h>
+#include <Python.h>
+
+int
+main(int argc, char *argv[])
+{
+    PyObject *pName, *pModule, *pDict, *pFunc;
+    PyObject *pArgs, *pValue;
+    int i, result;
+
+    if (argc < 3) {
+        fprintf(stderr,"Usage: call pythonfile funcname [args]\n");
+        return 1;
+    }
+
+    Py_Initialize();
+    pName = PyString_FromString(argv[1]);
+    /* Error checking of pName left out */
+
+    pModule = PyImport_Import(pName);
+    if (pModule != NULL) {
+        pDict = PyModule_GetDict(pModule);
+        /* pDict is a borrowed reference */
+
+        pFunc = PyDict_GetItemString(pDict, argv[2]);
+        /* pFun: Borrowed reference */
+
+        if (pFunc && PyCallable_Check(pFunc)) {
+            pArgs = PyTuple_New(argc - 3);
+            for (i = 0; i < argc - 3; ++i) {
+                pValue = PyInt_FromLong(atoi(argv[i + 3]));
+                if (!pValue) {
+                    fprintf(stderr, "Cannot convert argument\n");
+                    return 1;
+                }
+                /* pValue reference stolen here: */
+                PyTuple_SetItem(pArgs, i, pValue);
+            }
+            pValue = PyObject_CallObject(pFunc, pArgs);
+            if (pValue != NULL) {
+                printf("Result of call: %ld\n", PyInt_AsLong(pValue));
+                Py_DECREF(pValue);
+            }
+            else {
+                PyErr_Print();
+                fprintf(stderr,"Call failed\n");
+                return 1;
+            }
+            Py_DECREF(pArgs);
+            /* pDict and pFunc are borrowed and must not be Py_DECREF-ed */
+        }
+        else {
+            PyErr_Print();
+            fprintf(stderr, "Cannot find function \"%s\"\n", argv[2]);
+        }
+        Py_DECREF(pModule);
+    }
+    else {
+        PyErr_Print();
+        fprintf(stderr, "Failed to load \"%s\"\n", argv[1]);
+        return 1;
+    }
+    Py_DECREF(pName);
+    Py_Finalize();
+    return 0;
+}