1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
|
<HTML>
<HEAD>
<TITLE>Building Mac Python from source</TITLE>
</HEAD>
<BODY>
<H1>Building Mac Python from source</H1>
<HR>
This document explains how to build MacPython from source. This is
necessary if you want to write extension modules for 68K Python, and
is also necessary if you want to make modifications to the Python core.
Building Python is not something to be undertaken lightly,
you need a reasonable working
knowledge of the CodeWarrior development environment, a good net
connection and probably quite some time too. <p>
The information density in this file is high, so you should probably
print it and read it at your leasure. Most things are explained only
once (and probably in the wrong place:-). <p>
<blockquote>
First a warning: this information may become outdated if a new CodeWarrior is
released after MacPython. The
<a href="http://www.cwi.nl/~jack/macpython.html">MacPython homepage</a> will
hopefully have updated instructions in that case.
</blockquote>
I am very interested in feedback on this document, send your
comments to the <A
HREF="http://www.python.org/sigs/pythonmac-sig/">Mac Python Special
Interest Group</A>.
<H2>What you need.</H2>
The following things you definitely need:
<UL>
<LI> You need a MacPython source distribution, of course. You can
obtain one from <A HREF="ftp://ftp.cwi.nl/pub/jack/python/mac">
ftp://ftp.cwi.nl/pub/jack/python/mac</A> or from the companion webpage
at <A HREF="http://www.cwi.nl/~jack/macpython.html">
http://www.cwi.nl/~jack/macpython.html</A> (which has up-to-date links
to the other packages needed too) and possibly also from the standard
<A HREF="ftp://ftp.python.org/pub/python/mac">python.org ftp
site</A>. Everything you need is also included in the standard Python
source distribution, but the organization is different. Look in
directory <code>Mac/mwerks/projects</code> for the project files and
related stuff. <BR>
An alternative is to check the sources straight out of the CVS
repository, see below. Most of the packages mentioned here are also
available through CVS. Check the section on <a href="#cvs">CVS
repository use</a> below.
<LI> You need MetroWerks CodeWarrior. The current distribution has
been built with CodeWarrior Pro 4. Ordering information is
available on the <A HREF="http://www.metrowerks.com/">MetroWerks
homepage</A>. Building Python with MPW or Think/Symantec C is
probably impossible without major surgery.
<LI> You need GUSI, the Grand Unified Socket Interface, by Matthias
Neeracher. The original CWGUSI is
obtainable from <A
HREF="ftp://sunsite.cnlab-switch.ch/software/platform/macos/src">
ftp://sunsite.cnlab-switch.ch/software/platform/macos/src</A>.
At the moment Python is built with a slightly modified version of GUSI,
these modifications are available in folder <code>Python:Mac:GUSI-mods</code>. <br>
The modified GUSI is also in the MacPython cvs source repository, in the
directory <code>lib-src/CWGUSI</code>. However, some files contain slashes in
their names, something CVS seriously frowns upon, and each slash has been
replaced by <code>"_s_"</code>. There is a script
<code>Mac:scripts:fixgusidir.py</code> which you should run after checking CWGUSI
out
</UL>
<A NAME="optional">The MacPython project files are configured to
include a plethora of optional modules</A>, and these modules need a
number of extra packages. To use the project files as-is you have to
download these packages too. PPC and CFM68K Python have all such modules as
dynamically loaded modules, so if you don't need a certain package it
suffices to just refrain from builing the extension module. For static 68K
Python things are a bit more complicated: you have to edit the
interpreter project file to remove the reference to the module (and
the libraries it uses), and edit the <code>Mac:mwerks:mwerks_nonshared_config.h</code>
file to remove the <code>USE_...</code> line. Here are the locations for the various things
you need:
<UL>
<LI> Tcl and Tk are in a sad state on the Mac, the standard source distributions
simply don't compile, so I have created a distribution especially for use
with MacPython.
See the section on <A HREF="#tcltk">building Tcl/Tk Python</A>
below.
<LI> Waste, a TextEdit replacement written by Marco Piovanelli, <A
HREF="mailto:piovanel@kagi.com"><piovanel@kagi.com></A>. Python
was built using version 1.3, which you can obtain from <A
HREF="http://www.boingo.com/waste"><http://www.boingo.com/waste></A>
and various other places.
<LI> Gdbm library for the Mac. Available from Jack's Mac software page at
<A HREF="http://www.cwi.nl/~jack/macsoftware.html">
http://www.cwi.nl/~jack/macsoftware.html</A> and <A HREF="ftp://ftp.cwi.nl/pub/jack/mac">
ftp://ftp.cwi.nl/pub/jack/mac</A>. Also in the MacPython cvs repository at
<code>lib-src/gdbm</code>.
<LI> JPEG library by the Independent JPEG Group. A version including
Mac projects can be found at Jack's page mentioned above.
The most recent JPEG library can always be obtained from <A
HREF="ftp://ftp.uu.net/graphics/jpeg/">ftp://ftp.uu.net/graphics/jpeg/</A>. Again,
also in the MacPython cvs repository at <code>lib-src/jpeg</code>.
<LI> The netpbm/pbmplus, libtiff, zlib and png libraries. The netpbm distribution
(which includes libtiff) is generally available on Internet ftp
servers. For Python pbmplus, an older incarnation of netpbm, is
functionally identical to netpbm, since Python only uses the library
and not the complete applications. A distribution with correct
projects and library source only is available from, you guessed it, Jack's Mac software
page mentioned above. And, guessed it again, in the MacPython cvs repository
at <code>lib-src/netpbm</code>, etc. The only gotcha is that libtiff lives in
<code>lib-src/netpbm/libtiff</code>, for historical reasons.
</UL>
<H2>Setting Up</H2>
Now that you have collected everything you should start with building
the various parts. If you don't want to fix
access paths try to set things up as follows:
<PRE>
Top-level-folder:
CWGUSI
imglibs
jpeg
netpbm
libtiff
zlib
png
gdbm
Python
Modules
...
Mac
Modules
Build
...
Tcl/Tk Folder
tcl8.0
tk8.0
MoreFiles 1.4.3
Waste 1.3 distribution (if you want waste)
</PRE>
If your setup of the libraries is exactly the same as mine (which is
not very likely, unless you happen to work from the same CVS
repository) you can use the project <code>buildlibs.prj</code> in the
<code>:Mac:build.mac</code> folder to build all needed libraries in one
fell swoop, otherwise you will have to build the libraries one by
one. <p>
First build GUSI. If you didn't get the python-specific GUSI you have to
move the files from the "CWGUSI-mods" to the right
place in the CWGUSI distribution folder. Build the MSL version for your
platform (ppc, 68k, cfm68k). <p>
<em>Note:</em> always rebuild the CWGUSI libraries, even if you have
checked them out from the CVS repository. <P>
Next, in
<code>MoreFiles</code>, <code>libjpeg</code>, <code>pbmplus</code>,
<code>zlib</code>, <code>libpng</code>, <code>gdbm</code>,
and<code>libtiff</code> you build all projects. Usually the projects are in "mac"
subfolders, sometimes they are in the main folder. Tcl/tk is a special
case, see below. Of course, if you are only interested in one of
static 68K, CFM68K or PPC you can skip building the other libraries.
<H2><A NAME="tcltk">Building Tcl/Tk</H2>
The Tcl/Tk 8.3.0 source distribution does not work on the Mac. I have created
an archive of the sources that I used to build _tkinter for MacPython,
you can obtain this from <a
href="ftp://ftp.cwi.nl/pub/jack/python/mac/tcltk830src-for-python.sit">
ftp://ftp.cwi.nl/pub/jack/python/mac/tcltk830src-for-python.sit</a>. Only the
libraries needed to build _tkinter for PPC have been fixed. <P>
Note that if you use a different release of Tcl and Tk than the ones
I have used you may have to adapt the Python <code>tkresources.rsrc</code> file.
This is easiest done by building <code>SimpleTk</code> and copying the TEXT, ICON
and CRSR resources from it to <code>tkresources.rsrc</code>. This allows
the <code>_tkinter</code> module to work without an installed Tk/Tcl on your
machine. <P>
Also note that the <code>_tkinter.ppc.slb</code> that is normally distributed
in the <code>PlugIns</code> folder is the one from the Imaging extension,
which has some extra features needed by PIL (and which features should not
hinder normal operation).
</UL>
Build first the Tcl library, then
SimpleTcl (test it by typing <code>ls -l</code> in the window you get)
then the Tk library, then SimpleTk (which can again be tested with
<code>ls -l</code>). If this all worked you are all set to try
building Python.
<H2>Building Waste</H2>
You do not need to build the Waste libraries, as Python includes the
source modules themselves.
<H2>The organization of the Python source tree</H2>
Time for a short break, while we have a look at the organization of
the Python source tree. At the top level, we find the following
folders:
<DL>
<DT> Demo
<DD> Demo programs that are not Mac-specific. Some of these may not
work, the file <code>README-Mac</code> has some details.
<DT> Extensions
<DD> Extensions to the interpreter that are not Mac-specific. Contains
the <code>img</code>, <code>Imaging</code> and <code>Numeric</code> extensions
in this distribution. Nowadays, the extensions are all built in their own
folders (unlike in older distributions, where img was incorporated in the main
build procedure).
<DT> Grammar
<DD> The Python grammar. Included for reference only, you cannot build
the parser on a Mac.
<DT> Include
<DD> Machine-independent header files.
<DT> Modules
<DD> Machine-independent optional modules. Not all of these will work
on the Mac.
<DT> Objects
<DD> Machine-independent code for various objects. Most of these are
not really optional: the interpreter will not function without them.
<DT> Parser
<DD> The Python parser (machine-independent).
<DT> Python
<DD> The core interpreter. Most files are machine-independent, some
are unix-specific and not used on the Mac.
<DT> Tools
<DD> Tools for python developers. Contains <code>modulator</code>
which builds skeleton C extension modules and <code>bgen</code> which
generates complete interface modules from information in C header
files. There are some readme files, but more documentation is sorely
needed.
</DL>
All the mac-specific stuff lives in the <code>Mac</code> folder:
<DL>
<DT> Build
<DD> This is where the project files live and where you build the
libraries, shared libraries, executables and plugin modules. All the
resulting binaries, except for intermedeate results, are deposited in
the toplevel folder or the Mac:PlugIns folder (for plugin modules).
<DT> Compat
<DD> Unix-compatability routines. Some of these are not used anymore,
since CWGUSI provides a rather complete emulation, but you may need
these if you are trying to build a non-GUSI python.
<DT> Demo
<DD> Mac-specific demo programs, some of them annotated.
<DT> Include
<DD> Mac-specific but compiler-independent include files.
<DT> Lib
<DD> Mac-specific standard modules. The <code>toolbox</code> folder
contains modules specifically needed with various MacOS toolbox
interface modules.
<DT> Modules
<DD> Mac-specific builtin modules. Theoretically these are all
optional, but some are rather essential (like
<code>macmodule</code>). A lot of these modules are generated with
<code>bgen</code>, in which case the bgen input files are included so
you can attempt to regenerate them or extend them.
<DT> MPW
<DD> MPW-specific files. These have not been used or kept up-to-date
for a long time, so use at your own risk.
<DT> mwerks
<DD> Mwerks-specific sources and headers. Contains glue code for
Pythons shared-library architecture, a replacement for
<code>malloc</code> and a directory with various projects for building
variations on the Python interpreter. The <code>mwerks_*.h</code>
files here are the option-setting files for the various interpreters
and such, comparable to the unix command-line <code>-D</code> options
to the compiler. Each project uses the correct option file as its
"prefix file" in the "C/C++ language" settings. Disabling optional
modules (for the 68K interpreter), building non-GUSI interpreters and
various other things are accomplished by modifying these files (and
possibly changing the list of files included in the project window, of
course).
<DT> PlugIns
<DD> This is where the PPC and CFM68K dynamically-loaded plugin modules live.
<DT> Python
<DD> Mac-specific parts of the core interpreter.
<DT> Resources
<DD> Resource files needed to build the interpreter.
<DT> Scripts
<DD> A collection of various mac-specific Python scripts. Some are
essential, some are useful but few are documented, so you will have to
use your imagination to work them out.
<DT> Tools
<DD> A collection of tools, usually bigger than those in the scripts
folder. The important ones here are the IDE and macfreeze. The IDE is built
with the buildIDE.py script, which puts the resulting applet in the toplevel
folder. Macfreeze is usually invoked through the BuildApplication script,
but for more control over the freezing process you can run the main script here.
<DT> Unsupported
<DD> Modules that are not supported any longer but may still work with a little effort.
</DL>
<H2>Building the 68K interpreter</H2>
If you have all the optional libraries mentioned <A
HREF="#optional">above</A> loaded building Python for 68K macs is a
breeze: in the Mac:Build folder you build the libraries with buildlibs.prj
and then the interpreter with PythonStandalone.prj. <p>
If you were previously running another copy of this Python release,
from a binary installer for instance, you should
first remove the <code>Python XXX preferences</code> file from your
preference folder. Next, run the interpreter, in the toplevel folder. This will
create a correct initial preferences file. You are now all set, and
your tree should be completely compatible with a binary-only
distribution. Read the release notes
(<code>Relnotes-somethingorother</code>) and
<code>ReadMe</code> in the <code>Mac</code> folder. <p>
If something goes wrong you may end up with a garbled preferences file. Removing
it from the system folder and running Python once again will re-create it.
<H2>Building the PPC and CFM68K interpreter</H2>
First you optionally build the external libraries with buildlibs.prj. Next,
the projects for
interpreter, core library and applet skeleton are all linked together, so
building the fat target in <code>PythonEngine.prj</code>
will result in everything being built. The
resulting applications and fat shared library are deposited in the main
Python folder. Finally, you build all the plugins with the plugins.prj project.
For completeness sake here is a breakdown of the projects:
<DL>
<DT> PythonCore (with subprojects PythonCorePPC and PythonCoreCFM68K)
<DD> The shared library that contains the bulk of the interpreter and
its resources. It is a good idea to immedeately put an alias to this
shared library in the <code>Extensions</code> folder of your system
folder. Do exactly that: put an <em>alias</em> there, copying or
moving the file will cause you grief later if you rebuild the library and
forget to copy it to the extensions folder again. The InstallPython applet
will also do this, along with creating the plugin aliases.
<DT> PythonInterpeter
<DD> The interpreter. This is basically a routine to call out to the
shared library. Unlike in previous releases the same program is used for
creating applets (for which formerly PythonApplet was used). <p>
<DT> Plugin projects
<DD> Each plugin module has a separate project. The <code>Plugins.prj</code>
project tries to build them all, but is known to be flakey under CW Pro 4.
</DL>
After creating the alias to <code>PythonCore</code> you remove any old
<code>Python XXX Preferences</code> file from the <code>Preferences</code> folder
(if you had python installed on your system before) and run the interpreter once
to create the correct preferences file. <p>
Next, you have to build the extension modules.
The <code>PlugIns.ppc</code> project has all the
other projects as subprojects and builds everything (but see the gotcha above).
<p>
Finally, you must build the standard applets:
<code>EditPythonPrefs</code>, <code>BuildApplet</code>, etc. This is
easiest done with the <code>fullbuild</code> script from
<code>Mac:scripts</code>. <p>
<BLOCKQUOTE>
Actually, the <code>fullbuild</code> script can be used to build
everything, but you need a fully-functional interpreter before you can
use it (and one that isn't rebuilt in the process: you cannot rebuild
a running program). You could copy the 68K interpreter to a different
place and use that to run fullbuild, or use the standalone PPC python
for this. I tend to keep a standalone interpreter in a safe place for
this use only. <p>
</BLOCKQUOTE>
You are all set now, and should read the release notes and
<code>ReadMe</code> file from the <code>Mac</code> folder.
<H2>Rebuilding <code>.exp</code> files for PPC and CFM68K</H2>
Occasionally it may be necessary to rebuild your PythonCore <code>.exp</code>
file, a file that controls which symbols are exported by your PythonCore
shared library. Rebuild it if you get unexpected undefined symbols when you
are building a plugin module. <p>
Rebuilding the .exp file is done by first removing the file and removing the
reference to it in the project (in the "config" section). Next, build PythonCore.
This will create a new .exp file. Edit this file to remove the references to
the symbols <code>__initialize</code>, <code>__terminate</code>, <code>setjmp</code>,
<code>longjmp</code>, <code>vec_longjmp</code>, <code>main</code> and (for PPC) <code>__ptmf_null</code> or (for
CFM68K) <code>__start</code> and <code>dummy_init_routine</code>.
Next, add the .exp file to the project
again and rebuild PythonCore. <p>
This rather convoluted procedure is needed to ensure that plugin modules don't
accidentally link with those entrypoints from PythonCore, which will not work because
those routines have to be in the same code fragment as they are used from.
<H2><a name="cvs">Using the CVS source archive</a></H2>
It is possible to access the Python sources through remote CVS. The
advantage of this is that you get the very latest sources, so any bug
fixed or new features will be immedeately available. This is also the
disadvantage, of course: as this is the same tree as is used for
development it may sometimes be a little less stable. <p>
The CVS client of choice is Alexandre Parenteau's MacCVS. It can be
obtained through the <a href="http://www.wincvs.org">WinCVS
homepage</a>. MacCVS uses Internet Config to set file types correctly
based on the filename extension. In the maccvs preferences you should
also set (in the "binary files" section) "use mac encoding:
applesingle" and (in the "text files" section) "use ISO latin 1
conversion". <p>
It is also a good idea to disable Quicktime Exchange in the Quicktime
control panel. Quicktime Exchange will magically map some extensions to
filetypes, and this can seriously hinder you if, for instance, <code>.bmp</code>
is not a Windows bitmap file. <p>
The machine-independent Python sources are checked out from the main
Python CVS archive, see the <a
href="http://www.python.org/download/cvs.html">Source access via
CVS</a> page for details. When you check the sources out you will get
something like <code>Python:dist:src</code>, and under that the
<code>Modules</code>, <code>Lib</code>, etc hierarchy. The
<code>src</code> folder should be renamed to <code>Python</code>, and
is what this document refers to as the "toplevel Python folder". <P>
Next, <em>in a separate folder</em>, you check out the
mac-specific sources. You then move the <code>Mac</code> folder from this
checkout (the only folder with anything in it) to the Python source folder.
Note that the checking out in a separate folder and moving is necessary,
due to the way cvs works.
The CVS path to use for the mac stuff can be found
at the <a href="http://www.cwi.nl/~jack/macpython.html">MacPython
homepage</a>. Finally, you check out the external libraries needed in
the parent of the toplevel Python folder. The CVS path for these libraries is
also mentioned at the MacPython homepage. <p>
Neither of the pages mentioned above contains the passwords for the
CVS sites, for obvious reasons, but they do contain instructions on
how to obtain the passwords. <p>
You should end up with a folder structure as described at the top of this
document. <p>
Note that while the Mac folder is now a subfolder of your toplevel Python
folder this does not mean that they "act as one" as far as CVS is concerned.
To update all your sources you have to do a "cvs update" in the toplevel
Python folder and another one in the Mac folder. This is again a cvs problem:
it cannot deal with subpackages coming from different repositories. <p>
<H2>Odds and ends</H2>
Some remarks that I could not fit in elsewhere:
<UL>
<LI> It may be possible to use the <code>PythonCore</code> shared
library to embed Python in another program, if your program can live
with using GUSI for I/O. Use PythonCore in stead of your MSL C library
(or, at the very least, link it before the normal C library). Let me
know whether this works.
<LI> It is possible to build PPC extension modules without building a
complete Python. The binary distribution installer can optionally install
all the needed folders. A template for a dynamic module can be found in
<code>xx.prj</code>.
<LI> The Python shared library architecture is a variant of the architecture
described as "application with shared libraries and dropins" in the MetroWerks
"Targeting MacOS" documentation. The Python Application and applet-template use
the <code>MSL AppRuntime.Lib</code> runtime library (with properly set CFM
initialization and termination routines). PythonCore uses <code>MSL Runtime.Lib</code>,
which is really intended for standalone programs but which we fool into working by
providing a dummy main program.
It is linked statically into PythonCore (and exported to the applications and plugins)
so we do not have to distribute yet another shared library. Plugin modules use
<code>MSL ShlibRuntime.Lib</code> (not the dropin runtime: modules are never unloaded)
and obtain the rest from PythonCore. PythonCore uses a
non-standard initialization entry point, <code>__initialize_with_resources</code>, to
be able to obtain resources from the library file later on. Plugins can do the same
(_tkinter does) or use the standard <code>__initialize</code> entry point.
</UL>
</BODY>
</HTML>
|
