.. _socket-howto:
****************************
Socket Programming HOWTO
****************************
:Author: Gordon McMillan
.. topic:: Abstract
Sockets are used nearly everywhere, but are one of the most severely
misunderstood technologies around. This is a 10,000 foot overview of sockets.
It's not really a tutorial - you'll still have work to do in getting things
operational. It doesn't cover the fine points (and there are a lot of them), but
I hope it will give you enough background to begin using them decently.
Sockets
=======
I'm only going to talk about INET (i.e. IPv4) sockets, but they account for at least 99% of
the sockets in use. And I'll only talk about STREAM (i.e. TCP) sockets - unless you really
know what you're doing (in which case this HOWTO isn't for you!), you'll get
better behavior and performance from a STREAM socket than anything else. I will
try to clear up the mystery of what a socket is, as well as some hints on how to
work with blocking and non-blocking sockets. But I'll start by talking about
blocking sockets. You'll need to know how they work before dealing with
non-blocking sockets.
Part of the trouble with understanding these things is that "socket" can mean a
number of subtly different things, depending on context. So first, let's make a
distinction between a "client" socket - an endpoint of a conversation, and a
"server" socket, which is more like a switchboard operator. The client
application (your browser, for example) uses "client" sockets exclusively; the
web server it's talking to uses both "server" sockets and "client" sockets.
History
-------
Of the various forms of :abbr:`IPC (Inter Process Communication)`,
sockets are by far the most popular. On any given platform, there are
likely to be other forms of IPC that are faster, but for
cross-platform communication, sockets are about the only game in town.
They were invented in Berkeley as part of the BSD flavor of Unix. They spread
like wildfire with the Internet. With good reason --- the combination of sockets
with INET makes talking to arbitrary machines around the world unbelievably easy
(at least compared to other schemes).
Creating a Socket
=================
Roughly speaking, when you clicked on the link that brought you to this page,
your browser did something like the following::
# create an INET, STREAMing socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# now connect to the web server on port 80 - the normal http port
s.connect(("www.python.org", 80))
When the ``connect`` completes, the socket ``s`` can be used to send
in a request for the text of the page. The same socket will read the
reply, and then be destroyed. That's right, destroyed. Client sockets
are normally only used for one exchange (or a small set of sequential
exchanges).
What happens in the web server is a bit more complex. First, the web server
creates a "server socket"::
# create an INET, STREAMing socket
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# bind the socket to a public host, and a well-known port
serversocket.bind((socket.gethostname(), 80))
# become a server socket
serversocket.listen(5)
A couple things to notice: we used ``socket.gethostname()`` so that the socket
would be visible to the outside world. If we had used ``s.bind(('localhost',
80))`` or ``s.bind(('127.0.0.1', 80))`` we would still have a "server" socket,
but one that was only visible within the same machine. ``s.bind(('', 80))``
specifies that the socket is reachable by any address the machine happens to
have.
A second thing to note: low number ports are usually reserved for "well known"
services (HTTP, SNMP etc). If you're playing around, use a nice high number (4
digits).
Finally, the argument to ``listen`` tells the socket library that we want it to
queue up as many as 5 connect requests (the normal max) before refusing outside
connections. If the rest of the code is written properly, that should be plenty.
Now that we have a "server" socket, listening on port 80, we can enter the
mainloop of the web server::
while True:
# accept connections from outside
(clientsocket, address) = serversocket.accept()
# now do something with the clientsocket
# in this case, we'll pretend this is a threaded server
ct = client_thread(clientsocket)
ct.run()
There's actually 3 general ways in which this loop could work - dispatching a
thread to handle ``clientsocket``, create a new process to handle
``clientsocket``, or restructure this app to use non-blocking sockets, and
multiplex between our "server" socket and any active ``clientsocket``\ s using
``select``. More about that later. The important thing to understand now is
this: this is *all* a "server" socket does. It doesn't send any data. It doesn't
receive any data. It just produces "client" sockets. Each ``clientsocket`` is
created in response to some *other* "client" socket doing a ``connect()`` to the
host and port we're bound to. As soon as we've created that ``clientsocket``, we
go back to listening for more connections. The two "clients" are free to chat it
up - they are using some dynamically allocated port which will be recycled when
the conversation ends.
IPC
---
If you need fast IPC between two processes on one machine, you should look into
pipes or shared memory. If you do decide to use AF_INET sockets, bind the
"server" socket to ``'localhost'``. On most platforms, this will take a
shortcut around a couple of layers of network code and be quite a bit faster.
.. seealso::
The :mod:`multiprocessing` integrates cross-platform IPC into a higher-level
API.
Using a Socket
==============
The first thing to note, is that the web browser's "client" socket and the web
server's "client" socket are identical beasts. That is, this is a "peer to peer"
conversation. Or to put it another way, *as the designer, you will have to
decide what the rules of etiquette are for a conversation*. Normally, the
``connect``\ ing socket starts the conversation, by sending in a request, or
perhaps a signon. But that's a design decision - it's not a rule of sockets.
Now there are two sets of verbs to use for communication. You can use ``send``
and ``recv``, or you can transform your client socket into a file-like beast and
use ``read`` and ``write``. The latter is the way Java presents its sockets.
I'm not going to talk about it here, except to warn you that you need to use
``flush`` on sockets. These are buffered "files", and a common mistake is to
``write`` something, and then ``read`` for a reply. Without a ``flush`` in
there, you may wait forever for the reply, because the request may still be in
your output buffer.
Now we come to the major stumbling block of sockets - ``send`` and ``recv`` operate
on the network buffers. They do not necessarily handle all the bytes you hand
them (or expect from them), because their major focus is handling the network
buffers. In general, they return when the associated network buffers have been
filled (``send``) or emptied (``recv``). They then tell you how many bytes they
handled. It is *your* responsibility to call them again until your message has
been completely dealt with.
When a ``recv`` returns 0 bytes, it means the other side has closed (or is in
the process of closing) the connection. You will not receive any more data on
this connection. Ever. You may be able to send data successfully; I'll talk
more about this later.
A protocol like HTTP uses a socket for only one transfer. The client sends a
request, then reads a reply. That's it. The socket is discarded. This means that
a client can detect the end of the reply by receiving 0 bytes.
But if you plan to reuse your socket for further transfers, you need to realize
that *there is no* :abbr:`EOT (End of Transfer)` *on a socket.* I repeat: if a socket
``send`` or ``recv`` returns after handling 0 bytes, the connection has been
broken. If the connection has *not* been broken, you may wait on a ``recv``
forever, because the socket will *not* tell you that there's nothing more to
read (for now). Now if you think about that a bit, you'll come to realize a
fundamental truth of sockets: *messages must either be fixed length* (yuck), *or
be delimited* (shrug), *or indicate how long they are* (much better), *or end by
shutting down the connection*. The choice is entirely yours, (but some ways are
righter than others).
Assuming you don't want to end the connection, the simplest solution is a fixed
length message::
class MySocket:
"""demonstration class only
- coded for clarity, not efficiency
"""
def __init__(self, sock=None):
if sock is None:
self.sock = socket.socket(
socket.AF_INET, socket.SOCK_STREAM)
else:
self.sock = sock
def connect(self, host, port):
self.sock.connect((host, port))
def mysend(self, msg):
totalsent = 0
while totalsent < MSGLEN:
sent = self.sock.send(msg[totalsent:])
if sent == 0:
raise RuntimeError("socket connection broken")
totalsent = totalsent + sent
def myreceive(self):
chunks = []
bytes_recd = 0
while bytes_recd < MSGLEN:
chunk = self.sock.recv(min(MSGLEN - bytes_recd, 2048))
if chunk == b'':
raise RuntimeError("socket connection broken")
chunks.append(chunk)
bytes_recd = bytes_recd + len(chunk)
return b''.join(chunks)
The sending code here is usable for almost any messaging scheme - in Python you
send strings, and you can use ``len()`` to determine its length (even if it has
embedded ``\0`` characters). It's mostly the receiving code that gets more
complex. (And in C, it's not much worse, except you can't use ``strlen`` if the
message has embedded ``\0``\ s.)
The easiest enhancement is to make the first character of the message an
indicator of message type, and have the type determine the length. Now you have
two ``recv``\ s - the first to get (at least) that first character so you can
look up the length, and the second in a loop to get the rest. If you decide to
go the delimited route, you'll be receiving in some arbitrary chunk size, (4096
or 8192 is frequently a good match for network buffer sizes), and scanning what
you've received for a delimiter.
One complication to be aware of: if your conversational protocol allows multiple
messages to be sent back to back (without some kind of reply), and you pass
``recv`` an arbitrary chunk size, you may end up reading the start of a
following message. You'll need to put that aside and hold onto it, until it's
needed.
Prefixing the message with its length (say, as 5 numeric characters) gets more
complex, because (believe it or not), you may not get all 5 characters in one
``recv``. In playing around, you'll get away with it; but in high network loads,
your code will very quickly break unless you use two ``recv`` loops - the first
to determine the length, the second to get the data part of the message. Nasty.
This is also when you'll discover that ``send`` does not always manage to get
rid of everything in one pass. And despite having read this, you will eventually
get bit by it!
In the interests of space, building your character, (and preserving my
competitive position), these enhancements are left as an exercise for the
reader. Lets move on to cleaning up.
Binary Data
-----------
It is perfectly possible to send binary data over a socket. The major problem is
that not all machines use the same formats for binary data. For example, a
Motorola chip will represent a 16 bit integer with the value 1 as the two hex
bytes 00 01. Intel and DEC, however, are byte-reversed - that same 1 is 01 00.
Socket libraries have calls for converting 16 and 32 bit integers - ``ntohl,
htonl, ntohs, htons`` where "n" means *network* and "h" means *host*, "s" means
*short* and "l" means *long*. Where network order is host order, these do
nothing, but where the machine is byte-reversed, these swap the bytes around
appropriately.
In these days of 32 bit machines, the ascii representation of binary data is
frequently smaller than the binary representation. That's because a surprising
amount of the time, all those longs have the value 0, or maybe 1. The string "0"
would be two bytes, while binary is four. Of course, this doesn't fit well with
fixed-length messages. Decisions, decisions.
Disconnecting
=============
Strictly speaking, you're supposed to use ``shutdown`` on a socket before you
``close`` it. The ``shutdown`` is an advisory to the socket at the other end.
Depending on the argument you pass it, it can mean "I'm not going to send
anymore, but I'll still listen", or "I'm not listening, good riddance!". Most
socket libraries, however, are so used to programmers neglecting to use this
piece of etiquette that normally a ``close`` is the same as ``shutdown();
close()``. So in most situations, an explicit ``shutdown`` is not needed.
One way to use ``shutdown`` effectively is in an HTTP-like exchange. The client
sends a request and then does a ``shutdown(1)``. This tells the server "This
client is done sending, but can still receive." The server can detect "EOF" by
a receive of 0 bytes. It can assume it has the complete request. The server
sends a reply. If the ``send`` completes successfully then, indeed, the client
was still receiving.
Python takes the automatic shutdown a step further, and says that when a socket
is garbage collected, it will automatically do a ``close`` if it's needed. But
relying on this is a very bad habit. If your socket just disappears without
doing a ``close``, the socket at the other end may hang indefinitely, thinking
you're just being slow. *Please* ``close`` your sockets when you're done.
When Sockets Die
----------------
Probably the worst thing about using blocking sockets is what happens when the
other side comes down hard (without doing a ``close``). Your socket is likely to
hang. TCP is a reliable protocol, and it will wait a long, long time
before giving up on a connection. If you're using threads, the entire thread is
essentially dead. There's not much you can do about it. As long as you aren't
doing something dumb, like holding a lock while doing a blocking read, the
thread isn't really consuming much in the way of resources. Do *not* try to kill
the thread - part of the reason that threads are more efficient than processes
is that they avoid the overhead associated with the automatic recycling of
resources. In other words, if you do manage to kill the thread, your whole
process is likely to be screwed up.
Non-blocking Sockets
====================
If you've understood the preceding, you already know most of what you need to
know about the mechanics of using sockets. You'll still use the same calls, in
much the same ways. It's just that, if you do it right, your app will be almost
inside-out.
In Python, you use ``socket.setblocking(False)`` to make it non-blocking. In C, it's
more complex, (for one thing, you'll need to choose between the BSD flavor
``O_NONBLOCK`` and the almost indistinguishable POSIX flavor ``O_NDELAY``, which
is completely different from ``TCP_NODELAY``), but it's the exact same idea. You
do this after creating the socket, but before using it. (Actually, if you're
nuts, you can switch back and forth.)
The major mechanical difference is that ``send``, ``recv``, ``connect`` and
``accept`` can return without having done anything. You have (of course) a
number of choices. You can check return code and error codes and generally drive
yourself crazy. If you don't believe me, try it sometime. Your app will grow
large, buggy and suck CPU. So let's skip the brain-dead solutions and do it
right.
Use ``select``.
In C, coding ``select`` is fairly complex. In Python, it's a piece of cake, but
it's close enough to the C version that if you understand ``select`` in Python,
you'll have little trouble with it in C::
ready_to_read, ready_to_write, in_error = \
select.select(
potential_readers,
potential_writers,
potential_errs,
timeout)
You pass ``select`` three lists: the first contains all sockets that you might
want to try reading; the second all the sockets you might want to try writing
to, and the last (normally left empty) those that you want to check for errors.
You should note that a socket can go into more than one list. The ``select``
call is blocking, but you can give it a timeout. This is generally a sensible
thing to do - give it a nice long timeout (say a minute) unless you have good
reason to do otherwise.
In return, you will get three lists. They contain the sockets that are actually
readable, writable and in error. Each of these lists is a subset (possibly
empty) of the corresponding list you passed in.
If a socket is in the output readable list, you can be
as-close-to-certain-as-we-ever-get-in-this-business that a ``recv`` on that
socket will return *something*. Same idea for the writable list. You'll be able
to send *something*. Maybe not all you want to, but *something* is better than
nothing. (Actually, any reasonably healthy socket will return as writable - it
just means outbound network buffer space is available.)
If you have a "server" socket, put it in the potential_readers list. If it comes
out in the readable list, your ``accept`` will (almost certainly) work. If you
have created a new socket to ``connect`` to someone else, put it in the
potential_writers list. If it shows up in the writable list, you have a decent
chance that it has connected.
Actually, ``select`` can be handy even with blocking sockets. It's one way of
determining whether you will block - the socket returns as readable when there's
something in the buffers. However, this still doesn't help with the problem of
determining whether the other end is done, or just busy with something else.
**Portability alert**: On Unix, ``select`` works both with the sockets and
files. Don't try this on Windows. On Windows, ``select`` works with sockets
only. Also note that in C, many of the more advanced socket options are done
differently on Windows. In fact, on Windows I usually use threads (which work
very, very well) with my sockets.
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
# Top-level Makefile for Python
#
# As distributed, this file is called Makefile.pre.in; it is processed
# into the real Makefile by running the script ./configure, which
# replaces things like @spam@ with values appropriate for your system.
# This means that if you edit Makefile, your changes get lost the next
# time you run the configure script. Ideally, you can do:
#
# ./configure
# make
# make test
# make install
#
# If you have a previous version of Python installed that you don't
# want to overwrite, you can use "make altinstall" instead of "make
# install". Refer to the "Installing" section in the README file for
# additional details.
#
# See also the section "Build instructions" in the README file.
# === Variables set by makesetup ===
MODOBJS= _MODOBJS_
MODLIBS= _MODLIBS_
# === Variables set by configure
VERSION= @VERSION@
srcdir= @srcdir@
VPATH= @srcdir@
CC= @CC@
CXX= @CXX@
MAINCC= @MAINCC@
LINKCC= @LINKCC@
AR= @AR@
RANLIB= @RANLIB@
SVNVERSION= @SVNVERSION@
# Shell used by make (some versions default to the login shell, which is bad)
SHELL= /bin/sh
# Use this to make a link between python$(VERSION) and python in $(BINDIR)
LN= @LN@
# Portable install script (configure doesn't always guess right)
INSTALL= @INSTALL@
INSTALL_PROGRAM=@INSTALL_PROGRAM@
INSTALL_SCRIPT= @INSTALL_SCRIPT@
INSTALL_DATA= @INSTALL_DATA@
# Shared libraries must be installed with executable mode on some systems;
# rather than figuring out exactly which, we always give them executable mode.
# Also, making them read-only seems to be a good idea...
INSTALL_SHARED= ${INSTALL} -m 555
MAKESETUP= $(srcdir)/Modules/makesetup
# Compiler options
OPT= @OPT@
BASECFLAGS= @BASECFLAGS@
CFLAGS= $(BASECFLAGS) $(OPT) $(EXTRA_CFLAGS)
# Both CPPFLAGS and LDFLAGS need to contain the shell's value for setup.py to
# be able to build extension modules using the directories specified in the
# environment variables
CPPFLAGS= -I. -IInclude -I$(srcdir)/Include @CPPFLAGS@
LDFLAGS= @LDFLAGS@
LDLAST= @LDLAST@
SGI_ABI= @SGI_ABI@
CCSHARED= @CCSHARED@
LINKFORSHARED= @LINKFORSHARED@
# Extra C flags added for building the interpreter object files.
CFLAGSFORSHARED=@CFLAGSFORSHARED@
# C flags used for building the interpreter object files
PY_CFLAGS= $(CFLAGS) $(CPPFLAGS) $(CFLAGSFORSHARED) -DPy_BUILD_CORE
# Machine-dependent subdirectories
MACHDEP= @MACHDEP@
# Install prefix for architecture-independent files
prefix= @prefix@
# Install prefix for architecture-dependent files
exec_prefix= @exec_prefix@
# Install prefix for data files
datarootdir= @datarootdir@
# Expanded directories
BINDIR= $(exec_prefix)/bin
LIBDIR= $(exec_prefix)/lib
MANDIR= @mandir@
INCLUDEDIR= @includedir@
CONFINCLUDEDIR= $(exec_prefix)/include
SCRIPTDIR= $(prefix)/lib
# Detailed destination directories
BINLIBDEST= $(LIBDIR)/python$(VERSION)
LIBDEST= $(SCRIPTDIR)/python$(VERSION)
INCLUDEPY= $(INCLUDEDIR)/python$(VERSION)
CONFINCLUDEPY= $(CONFINCLUDEDIR)/python$(VERSION)
LIBP= $(LIBDIR)/python$(VERSION)
# Symbols used for using shared libraries
SO= @SO@
LDSHARED= @LDSHARED@
BLDSHARED= @BLDSHARED@
DESTSHARED= $(BINLIBDEST)/lib-dynload
# Executable suffix (.exe on Windows and Mac OS X)
EXE= @EXEEXT@
BUILDEXE= @BUILDEXEEXT@
# Short name and location for Mac OS X Python framework
UNIVERSALSDK=@UNIVERSALSDK@
PYTHONFRAMEWORK= @PYTHONFRAMEWORK@
PYTHONFRAMEWORKDIR= @PYTHONFRAMEWORKDIR@
PYTHONFRAMEWORKPREFIX= @PYTHONFRAMEWORKPREFIX@
PYTHONFRAMEWORKINSTALLDIR= @PYTHONFRAMEWORKINSTALLDIR@
# Deployment target selected during configure, to be checked
# by distutils. The export statement is needed to ensure that the
# deployment target is active during build.
MACOSX_DEPLOYMENT_TARGET=@CONFIGURE_MACOSX_DEPLOYMENT_TARGET@
@EXPORT_MACOSX_DEPLOYMENT_TARGET@export MACOSX_DEPLOYMENT_TARGET
# Options to enable prebinding (for fast startup prior to Mac OS X 10.3)
OTHER_LIBTOOL_OPT=@OTHER_LIBTOOL_OPT@
# Environment to run shared python without installed libraries
RUNSHARED= @RUNSHARED@
# Modes for directories, executables and data files created by the
# install process. Default to user-only-writable for all file types.
DIRMODE= 755
EXEMODE= 755
FILEMODE= 644
# configure script arguments
CONFIG_ARGS= @CONFIG_ARGS@
# Subdirectories with code
SRCDIRS= @SRCDIRS@
# Other subdirectories
SUBDIRSTOO= Include Lib Misc Demo
# Files and directories to be distributed
CONFIGFILES= configure configure.in acconfig.h pyconfig.h.in Makefile.pre.in
DISTFILES= README ChangeLog $(CONFIGFILES)
DISTDIRS= $(SUBDIRS) $(SUBDIRSTOO) Ext-dummy
DIST= $(DISTFILES) $(DISTDIRS)
LIBRARY= @LIBRARY@
LDLIBRARY= @LDLIBRARY@
BLDLIBRARY= @BLDLIBRARY@
DLLLIBRARY= @DLLLIBRARY@
LDLIBRARYDIR= @LDLIBRARYDIR@
INSTSONAME= @INSTSONAME@
LIBS= @LIBS@
LIBM= @LIBM@
LIBC= @LIBC@
SYSLIBS= $(LIBM) $(LIBC)
SHLIBS= @SHLIBS@
THREADOBJ= @THREADOBJ@
DLINCLDIR= @DLINCLDIR@
DYNLOADFILE= @DYNLOADFILE@
MACHDEP_OBJS= @MACHDEP_OBJS@
LIBOBJDIR= Python/
LIBOBJS= @LIBOBJS@
PYTHON= python$(EXE)
BUILDPYTHON= python$(BUILDEXE)
# The task to run while instrument when building the profile-opt target
PROFILE_TASK= Tools/pybench/pybench.py -n 2 --with-gc --with-syscheck
#PROFILE_TASK= Lib/test/regrtest.py
# === Definitions added by makesetup ===
##########################################################################
# Modules
MODULE_OBJS= \
Modules/config.o \
Modules/getpath.o \
Modules/main.o \
Modules/gcmodule.o
# Used of signalmodule.o is not available
SIGNAL_OBJS= @SIGNAL_OBJS@
##########################################################################
# Grammar
GRAMMAR_H= $(srcdir)/Include/graminit.h
GRAMMAR_C= $(srcdir)/Python/graminit.c
GRAMMAR_INPUT= $(srcdir)/Grammar/Grammar
##########################################################################
# Parser
PGEN= Parser/pgen$(EXE)
POBJS= \
Parser/acceler.o \
Parser/grammar1.o \
Parser/listnode.o \
Parser/node.o \
Parser/parser.o \
Parser/parsetok.o \
Parser/bitset.o \
Parser/metagrammar.o \
Parser/firstsets.o \
Parser/grammar.o \
Parser/pgen.o
PARSER_OBJS= $(POBJS) Parser/myreadline.o Parser/tokenizer.o
PGOBJS= \
Objects/obmalloc.o \
Python/mysnprintf.o \
Parser/tokenizer_pgen.o \
Parser/printgrammar.o \
Parser/pgenmain.o
PARSER_HEADERS= \
Parser/parser.h \
Parser/tokenizer.h
PGENOBJS= $(PGENMAIN) $(POBJS) $(PGOBJS)
##########################################################################
# AST
AST_H_DIR= $(srcdir)/Include
AST_H= $(AST_H_DIR)/Python-ast.h
AST_C_DIR= $(srcdir)/Python
AST_C= $(AST_C_DIR)/Python-ast.c
AST_ASDL= $(srcdir)/Parser/Python.asdl
ASDLGEN_FILES= $(srcdir)/Parser/asdl.py $(srcdir)/Parser/asdl_c.py
# XXX Note that a build now requires Python exist before the build starts
ASDLGEN= $(srcdir)/Parser/asdl_c.py
##########################################################################
# Python
PYTHON_OBJS= \
Python/_warnings.o \
Python/Python-ast.o \
Python/asdl.o \
Python/ast.o \
Python/bltinmodule.o \
Python/ceval.o \
Python/compile.o \
Python/codecs.o \
Python/errors.o \
Python/frozen.o \
Python/frozenmain.o \
Python/future.o \
Python/getargs.o \
Python/getcompiler.o \
Python/getcopyright.o \
Python/getmtime.o \
Python/getplatform.o \
Python/getversion.o \
Python/graminit.o \
Python/import.o \
Python/importdl.o \
Python/marshal.o \
Python/modsupport.o \
Python/mystrtoul.o \
Python/mysnprintf.o \
Python/peephole.o \
Python/pyarena.o \
Python/pyfpe.o \
Python/pymath.o \
Python/pystate.o \
Python/pythonrun.o \
Python/structmember.o \
Python/symtable.o \
Python/sysmodule.o \
Python/traceback.o \
Python/getopt.o \
Python/pystrcmp.o \
Python/pystrtod.o \
Python/formatter_unicode.o \
Python/$(DYNLOADFILE) \
$(LIBOBJS) \
$(MACHDEP_OBJS) \
$(THREADOBJ)
##########################################################################
# Objects
OBJECT_OBJS= \
Objects/abstract.o \
Objects/boolobject.o \
Objects/bytes_methods.o \
Objects/bytearrayobject.o \
Objects/bytesobject.o \
Objects/cellobject.o \
Objects/classobject.o \
Objects/cobject.o \
Objects/codeobject.o \
Objects/complexobject.o \
Objects/descrobject.o \
Objects/enumobject.o \
Objects/exceptions.o \
Objects/genobject.o \
Objects/fileobject.o \
Objects/floatobject.o \
Objects/frameobject.o \
Objects/funcobject.o \
Objects/iterobject.o \
Objects/listobject.o \
Objects/longobject.o \
Objects/dictobject.o \
Objects/memoryobject.o \
Objects/methodobject.o \
Objects/moduleobject.o \
Objects/object.o \
Objects/obmalloc.o \
Objects/rangeobject.o \
Objects/setobject.o \
Objects/sliceobject.o \
Objects/structseq.o \
Objects/tupleobject.o \
Objects/typeobject.o \
Objects/unicodeobject.o \
Objects/unicodectype.o \
Objects/weakrefobject.o
##########################################################################
# objects that get linked into the Python library
LIBRARY_OBJS= \
Modules/getbuildinfo.o \
$(PARSER_OBJS) \
$(OBJECT_OBJS) \
$(PYTHON_OBJS) \
$(MODULE_OBJS) \
$(SIGNAL_OBJS) \
$(MODOBJS)
#########################################################################
# Rules
# Default target
all: build_all
build_all: $(BUILDPYTHON) oldsharedmods sharedmods
# Compile a binary with gcc profile guided optimization.
profile-opt:
@echo "Building with support for profile generation:"
$(MAKE) clean
$(MAKE) build_all_generate_profile
@echo "Running benchmark to generate profile data:"
$(MAKE) profile-removal
$(MAKE) run_profile_task
@echo "Rebuilding with profile guided optimizations:"
$(MAKE) clean
$(MAKE) build_all_use_profile
build_all_generate_profile:
$(MAKE) all CFLAGS="$(CFLAGS) -fprofile-generate" LIBS="$(LIBS) -lgcov"
run_profile_task:
./$(BUILDPYTHON) $(PROFILE_TASK)
build_all_use_profile:
$(MAKE) all CFLAGS="$(CFLAGS) -fprofile-use"
coverage:
@echo "Building with support for coverage checking:"
$(MAKE) clean
$(MAKE) all CFLAGS="$(CFLAGS) -O0 -pg -fprofile-arcs -ftest-coverage" LIBS="$(LIBS) -lgcov"
# Build the interpreter
$(BUILDPYTHON): Modules/python.o $(LIBRARY) $(LDLIBRARY)
$(LINKCC) $(LDFLAGS) $(LINKFORSHARED) -o $@ \
Modules/python.o \
$(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST)
platform: $(BUILDPYTHON)
$(RUNSHARED) ./$(BUILDPYTHON) -E -c 'import sys ; from distutils.util import get_platform ; print(get_platform()+"-"+sys.version[0:3])' >platform
# Build the shared modules
sharedmods: $(BUILDPYTHON)
@case $$MAKEFLAGS in \
*s*) $(RUNSHARED) CC='$(CC)' LDSHARED='$(BLDSHARED)' OPT='$(OPT)' ./$(BUILDPYTHON) -E $(srcdir)/setup.py -q build;; \
*) $(RUNSHARED) CC='$(CC)' LDSHARED='$(BLDSHARED)' OPT='$(OPT)' ./$(BUILDPYTHON) -E $(srcdir)/setup.py build;; \
esac
# Build static library
# avoid long command lines, same as LIBRARY_OBJS
$(LIBRARY): $(LIBRARY_OBJS)
-rm -f $@
$(AR) cr $@ Modules/getbuildinfo.o
$(AR) cr $@ $(PARSER_OBJS)
$(AR) cr $@ $(OBJECT_OBJS)
$(AR) cr $@ $(PYTHON_OBJS)
$(AR) cr $@ $(MODULE_OBJS) $(SIGNAL_OBJS)
$(AR) cr $@ $(MODOBJS)
$(RANLIB) $@
libpython$(VERSION).so: $(LIBRARY_OBJS)
if test $(INSTSONAME) != $(LDLIBRARY); then \
$(LDSHARED) -Wl,-h$(INSTSONAME) -o $(INSTSONAME) $(LIBRARY_OBJS) $(SHLIBS) $(LIBC) $(LIBM); \
$(LN) -f $(INSTSONAME) $@; \
else\
$(LDSHARED) -o $@ $(LIBRARY_OBJS) $(SHLIBS) $(LIBC) $(LIBM); \
fi
libpython$(VERSION).sl: $(LIBRARY_OBJS)
$(LDSHARED) -o $@ $(LIBRARY_OBJS) $(SHLIBS) $(LIBC) $(LIBM)
# This rule is here for OPENSTEP/Rhapsody/MacOSX. It builds a temporary
# minimal framework (not including the Lib directory and such) in the current
# directory.
RESSRCDIR=Mac/Resources/framework
$(PYTHONFRAMEWORKDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK): \
$(LIBRARY) \
$(RESSRCDIR)/Info.plist
$(INSTALL) -d -m $(DIRMODE) $(PYTHONFRAMEWORKDIR)/Versions/$(VERSION)
if test "${UNIVERSALSDK}"; then \
$(CC) -o $(LDLIBRARY) -arch i386 -arch ppc -dynamiclib \
-isysroot "${UNIVERSALSDK}" \
-all_load $(LIBRARY) -Wl,-single_module \
-install_name $(DESTDIR)$(PYTHONFRAMEWORKINSTALLDIR)/Versions/$(VERSION)/$(PYTHONFRAMEWORK) \
-compatibility_version $(VERSION) \
-current_version $(VERSION); \
else \
/usr/bin/libtool -o $(LDLIBRARY) -dynamic $(OTHER_LIBTOOL_OPT) $(LIBRARY) \
@LIBTOOL_CRUFT@ ;\
fi
$(INSTALL) -d -m $(DIRMODE) \
$(PYTHONFRAMEWORKDIR)/Versions/$(VERSION)/Resources/English.lproj
$(INSTALL_DATA) $(RESSRCDIR)/Info.plist \
$(PYTHONFRAMEWORKDIR)/Versions/$(VERSION)/Resources/Info.plist
$(LN) -fsn $(VERSION) $(PYTHONFRAMEWORKDIR)/Versions/Current
$(LN) -fsn Versions/Current/$(PYTHONFRAMEWORK) $(PYTHONFRAMEWORKDIR)/$(PYTHONFRAMEWORK)
$(LN) -fsn Versions/Current/Headers $(PYTHONFRAMEWORKDIR)/Headers
$(LN) -fsn Versions/Current/Resources $(PYTHONFRAMEWORKDIR)/Resources
# This rule builds the Cygwin Python DLL and import library if configured
# for a shared core library; otherwise, this rule is a noop.
$(DLLLIBRARY) libpython$(VERSION).dll.a: $(LIBRARY_OBJS)
if test -n "$(DLLLIBRARY)"; then \
$(LDSHARED) -Wl,--out-implib=$@ -o $(DLLLIBRARY) $^ \
$(LIBS) $(MODLIBS) $(SYSLIBS); \
else true; \
fi
oldsharedmods: $(SHAREDMODS)
Makefile Modules/config.c: Makefile.pre \
$(srcdir)/Modules/config.c.in \
$(MAKESETUP) \
Modules/Setup.config \
Modules/Setup \
Modules/Setup.local
$(SHELL) $(MAKESETUP) -c $(srcdir)/Modules/config.c.in \
-s Modules \
Modules/Setup.config \
Modules/Setup.local \
Modules/Setup
@mv config.c Modules
@echo "The Makefile was updated, you may need to re-run make."
Modules/Setup: $(srcdir)/Modules/Setup.dist
@if test -f Modules/Setup; then \
echo "-----------------------------------------------"; \
echo "Modules/Setup.dist is newer than Modules/Setup;"; \
echo "check to make sure you have all the updates you"; \
echo "need in your Modules/Setup file."; \
echo "Usually, copying Modules/Setup.dist to Modules/Setup will work."; \
echo "-----------------------------------------------"; \
fi
############################################################################
# Special rules for object files
Modules/getbuildinfo.o: $(PARSER_OBJS) \
$(OBJECT_OBJS) \
$(PYTHON_OBJS) \
$(MODULE_OBJS) \
$(SIGNAL_OBJS) \
$(MODOBJS) \
$(srcdir)/Modules/getbuildinfo.c
$(CC) -c $(PY_CFLAGS) -DSVNVERSION=\"`LC_ALL=C $(SVNVERSION)`\" -o $@ $(srcdir)/Modules/getbuildinfo.c
Modules/getpath.o: $(srcdir)/Modules/getpath.c Makefile
$(CC) -c $(PY_CFLAGS) -DPYTHONPATH='"$(PYTHONPATH)"' \
-DPREFIX='"$(prefix)"' \
-DEXEC_PREFIX='"$(exec_prefix)"' \
-DVERSION='"$(VERSION)"' \
-DVPATH='"$(VPATH)"' \
-o $@ $(srcdir)/Modules/getpath.c
Modules/python.o: $(srcdir)/Modules/python.c
$(MAINCC) -c $(PY_CFLAGS) -o $@ $(srcdir)/Modules/python.c
$(GRAMMAR_H) $(GRAMMAR_C): $(PGEN) $(GRAMMAR_INPUT)
-@$(INSTALL) -d Include
-$(PGEN) $(GRAMMAR_INPUT) $(GRAMMAR_H) $(GRAMMAR_C)
$(PGEN): $(PGENOBJS)
$(CC) $(OPT) $(LDFLAGS) $(PGENOBJS) $(LIBS) -o $(PGEN)
Parser/grammar.o: $(srcdir)/Parser/grammar.c \
$(srcdir)/Include/token.h \
$(srcdir)/Include/grammar.h
Parser/metagrammar.o: $(srcdir)/Parser/metagrammar.c
Parser/tokenizer_pgen.o: $(srcdir)/Parser/tokenizer.c
Parser/pgenmain.o: $(srcdir)/Include/parsetok.h
$(AST_H): $(AST_ASDL) $(ASDLGEN_FILES)
$(ASDLGEN) -h $(AST_H_DIR) $(AST_ASDL)
$(AST_C): $(AST_ASDL) $(ASDLGEN_FILES)
$(ASDLGEN) -c $(AST_C_DIR) $(AST_ASDL)
Python/compile.o Python/symtable.o: $(GRAMMAR_H) $(AST_H)
Python/getplatform.o: $(srcdir)/Python/getplatform.c
$(CC) -c $(PY_CFLAGS) -DPLATFORM='"$(MACHDEP)"' -o $@ $(srcdir)/Python/getplatform.c
Python/importdl.o: $(srcdir)/Python/importdl.c
$(CC) -c $(PY_CFLAGS) -I$(DLINCLDIR) -o $@ $(srcdir)/Python/importdl.c
Objects/unicodectype.o: $(srcdir)/Objects/unicodectype.c \
$(srcdir)/Objects/unicodetype_db.h
BYTESTR_DEPS = \
$(srcdir)/Include/bytes_methods.h \
$(srcdir)/Objects/stringlib/count.h \
$(srcdir)/Objects/stringlib/ctype.h \
$(srcdir)/Objects/stringlib/eq.h \
$(srcdir)/Objects/stringlib/fastsearch.h \
$(srcdir)/Objects/stringlib/find.h \
$(srcdir)/Objects/stringlib/partition.h \
$(srcdir)/Objects/stringlib/stringdefs.h \
$(srcdir)/Objects/stringlib/string_format.h \
$(srcdir)/Objects/stringlib/transmogrify.h \
$(srcdir)/Objects/stringlib/unicodedefs.h \
$(srcdir)/Objects/stringlib/localeutil.h
Objects/stringobject.o: $(srcdir)/Objects/stringobject.c $(BYTESTR_DEPS)
Objects/bytesobject.o: $(srcdir)/Objects/bytesobject.c $(BYTESTR_DEPS)
Objects/unicodeobject.o: $(srcdir)/Objects/unicodeobject.c \
$(BYTESTR_DEPS)
Python/formatter_unicode.o: $(srcdir)/Python/formatter_unicode.c \
$(BYTESTR_DEPS)
############################################################################
# Header files
PYTHON_HEADERS= \
Include/Python-ast.h \
Include/Python.h \
Include/abstract.h \
Include/asdl.h \
Include/ast.h \
Include/bitset.h \
Include/boolobject.h \
Include/bytes_methods.h \
Include/bytearrayobject.h \
Include/bytesobject.h \
Include/cellobject.h \
Include/ceval.h \
Include/classobject.h \
Include/cobject.h \
Include/code.h \
Include/codecs.h \
Include/compile.h \
Include/complexobject.h \
Include/descrobject.h \
Include/dictobject.h \
Include/enumobject.h \
Include/errcode.h \
Include/eval.h \
Include/fileobject.h \
Include/floatobject.h \
Include/frameobject.h \
Include/funcobject.h \
Include/genobject.h \
Include/import.h \
Include/intrcheck.h \
Include/iterobject.h \
Include/listobject.h \
Include/longintrepr.h \
Include/longobject.h \
Include/marshal.h \
Include/memoryobject.h \
Include/metagrammar.h \
Include/methodobject.h \
Include/modsupport.h \
Include/moduleobject.h \
Include/node.h \
Include/object.h \
Include/objimpl.h \
Include/opcode.h \
Include/osdefs.h \
Include/parsetok.h \
Include/patchlevel.h \
Include/pgen.h \
Include/pgenheaders.h \
Include/pyarena.h \
Include/pydebug.h \
Include/pyerrors.h \
Include/pyfpe.h \
Include/pymath.h \
Include/pygetopt.h \
Include/pymem.h \
Include/pyport.h \
Include/pystate.h \
Include/pystrcmp.h \
Include/pystrtod.h \
Include/pythonrun.h \
Include/pythread.h \
Include/rangeobject.h \
Include/setobject.h \
Include/sliceobject.h \
Include/structmember.h \
Include/structseq.h \
Include/symtable.h \
Include/sysmodule.h \
Include/traceback.h \
Include/tupleobject.h \
Include/ucnhash.h \
Include/unicodeobject.h \
Include/warnings.h \
Include/weakrefobject.h \
pyconfig.h \
$(PARSER_HEADERS)
$(LIBRARY_OBJS) $(MODOBJS) Modules/python.o: $(PYTHON_HEADERS)
######################################################################
# Test the interpreter (twice, once without .pyc files, once with)
# In the past, we've had problems where bugs in the marshalling or
# elsewhere caused bytecode read from .pyc files to behave differently
# than bytecode generated directly from a .py source file. Sometimes
# the bytecode read from a .pyc file had the bug, somtimes the directly
# generated bytecode. This is sometimes a very shy bug needing a lot of
# sample data.
TESTOPTS= -l $(EXTRATESTOPTS)
TESTPROG= $(srcdir)/Lib/test/regrtest.py
TESTPYTHON= $(RUNSHARED) ./$(BUILDPYTHON) -E -tt -bb
test: all platform
-find $(srcdir)/Lib -name '*.py[co]' -print | xargs rm -f
-$(TESTPYTHON) $(TESTPROG) $(TESTOPTS)
$(TESTPYTHON) $(TESTPROG) $(TESTOPTS)
testall: all platform
-find $(srcdir)/Lib -name '*.py[co]' -print | xargs rm -f
$(TESTPYTHON) $(srcdir)/Lib/compileall.py
-find $(srcdir)/Lib -name '*.py[co]' -print | xargs rm -f
-$(TESTPYTHON) $(TESTPROG) $(TESTOPTS) -uall
$(TESTPYTHON) $(TESTPROG) $(TESTOPTS) -uall
# Run the unitests for both architectures in a Universal build on OSX
# Must be run on an Intel box.
testuniversal: all platform
if [ `arch` != 'i386' ];then \
echo "This can only be used on OSX/i386" ;\
exit 1 ;\
fi
-find $(srcdir)/Lib -name '*.py[co]' -print | xargs rm -f
-$(TESTPYTHON) $(TESTPROG) $(TESTOPTS) -uall
$(TESTPYTHON) $(TESTPROG) $(TESTOPTS) -uall
$(RUNSHARED) /usr/libexec/oah/translate ./$(BUILDPYTHON) -E -tt $(TESTPROG) $(TESTOPTS) -uall
# Like testall, but with a single pass only
# run an optional script to include some information about the build environment
buildbottest: all platform
-@if which pybuildbot.identify >/dev/null 2>&1; then \
pybuildbot.identify "CC='$(CC)'" "CXX='$(CXX)'"; \
fi
$(TESTPYTHON) $(TESTPROG) $(TESTOPTS) -uall -rw
QUICKTESTOPTS= $(TESTOPTS) -x test_thread test_signal test_strftime \
test_unicodedata test_re test_sre test_select test_poll \
test_struct test_zlib
quicktest: all platform
-find $(srcdir)/Lib -name '*.py[co]' -print | xargs rm -f
-$(TESTPYTHON) $(TESTPROG) $(QUICKTESTOPTS)
$(TESTPYTHON) $(TESTPROG) $(QUICKTESTOPTS)
MEMTESTOPTS= $(QUICKTESTOPTS) -x test_dl test___all__ test_fork1 \
test_longexp
memtest: all platform
-rm -f $(srcdir)/Lib/test/*.py[co]
-$(TESTPYTHON) $(TESTPROG) $(MEMTESTOPTS)
$(TESTPYTHON) $(TESTPROG) $(MEMTESTOPTS)
# Install everything
fullinstall: @FRAMEWORKINSTALLFIRST@ altinstall bininstall maninstall @FRAMEWORKINSTALLLAST@
# "make install" is an alias for "make altinstall" since we never want to
# overwrite Python 2.x.
install: altinstall
@echo "* Note: not installed as 'python'."
@echo "* Use 'make fullinstall' to install as 'python'."
@echo "* However, 'make fullinstall' is discouraged,"
@echo "* as it will clobber your Python 2.x installation."
# Install almost everything without disturbing previous versions
altinstall: @FRAMEWORKALTINSTALLFIRST@ altbininstall libinstall inclinstall libainstall \
sharedinstall oldsharedinstall @FRAMEWORKALTINSTALLLAST@
# Install shared libraries enabled by Setup
DESTDIRS= $(exec_prefix) $(LIBDIR) $(BINLIBDEST) $(DESTSHARED)
oldsharedinstall: $(DESTSHARED) $(SHAREDMODS)
@for i in X $(SHAREDMODS); do \
if test $$i != X; then \
echo $(INSTALL_SHARED) $$i $(DESTSHARED)/`basename $$i`; \
$(INSTALL_SHARED) $$i $(DESTDIR)$(DESTSHARED)/`basename $$i`; \
fi; \
done
$(DESTSHARED):
@for i in $(DESTDIRS); \
do \
if test ! -d $(DESTDIR)$$i; then \
echo "Creating directory $$i"; \
$(INSTALL) -d -m $(DIRMODE) $(DESTDIR)$$i; \
else true; \
fi; \
done
# Install the interpreter (by creating a hard link to python$(VERSION))
bininstall: altbininstall
-if test -f $(DESTDIR)$(BINDIR)/$(PYTHON) -o -h $(DESTDIR)$(BINDIR)/$(PYTHON); \
then rm -f $(DESTDIR)$(BINDIR)/$(PYTHON); \
else true; \
fi
(cd $(DESTDIR)$(BINDIR); $(LN) python$(VERSION)$(EXE) $(PYTHON))
-rm -f $(DESTDIR)$(BINDIR)/python-config
(cd $(DESTDIR)$(BINDIR); $(LN) -s python$(VERSION)-config python-config)
# Install the interpreter with $(VERSION) affixed
# This goes into $(exec_prefix)
altbininstall: $(BUILDPYTHON)
@for i in $(BINDIR) $(LIBDIR); \
do \
if test ! -d $(DESTDIR)$$i; then \
echo "Creating directory $$i"; \
$(INSTALL) -d -m $(DIRMODE) $(DESTDIR)$$i; \
else true; \
fi; \
done
$(INSTALL_PROGRAM) $(BUILDPYTHON) $(DESTDIR)$(BINDIR)/python$(VERSION)$(EXE)
if test -f libpython$(VERSION)$(SO); then \
if test "$(SO)" = .dll; then \
$(INSTALL_SHARED) libpython$(VERSION)$(SO) $(DESTDIR)$(BINDIR); \
else \
$(INSTALL_SHARED) libpython$(VERSION)$(SO) $(DESTDIR)$(LIBDIR)/$(INSTSONAME); \
if test libpython$(VERSION)$(SO) != $(INSTSONAME); then \
(cd $(DESTDIR)$(LIBDIR); $(LN) -sf $(INSTSONAME) libpython$(VERSION)$(SO)); \
fi \
fi; \
else true; \
fi
# Install the manual page
maninstall:
@for i in $(MANDIR) $(MANDIR)/man1; \
do \
if test ! -d $(DESTDIR)$$i; then \
echo "Creating directory $$i"; \
$(INSTALL) -d -m $(DIRMODE) $(DESTDIR)$$i; \
else true; \
fi; \
done
$(INSTALL_DATA) $(srcdir)/Misc/python.man \
$(DESTDIR)$(MANDIR)/man1/python.1
# Install the library
PLATDIR= plat-$(MACHDEP)
EXTRAPLATDIR= @EXTRAPLATDIR@
EXTRAMACHDEPPATH=@EXTRAMACHDEPPATH@
MACHDEPS= $(PLATDIR) $(EXTRAPLATDIR)
XMLLIBSUBDIRS= xml xml/dom xml/etree xml/parsers xml/sax
LIBSUBDIRS= tkinter site-packages test test/output test/data \
test/decimaltestdata \
encodings \
email email/mime email/test email/test/data \
html json json/tests http dbm xmlrpc \
sqlite3 sqlite3/test \
logging bsddb bsddb/test csv wsgiref \
lib2to3 lib2to3/fixes lib2to3/pgen2 lib2to3/tests \
ctypes ctypes/test ctypes/macholib idlelib idlelib/Icons \
distutils distutils/command distutils/tests $(XMLLIBSUBDIRS) \
setuptools setuptools/command setuptools/tests setuptools.egg-info \
curses $(MACHDEPS)
libinstall: build_all $(srcdir)/Lib/$(PLATDIR)
@for i in $(SCRIPTDIR) $(LIBDEST); \
do \
if test ! -d $(DESTDIR)$$i; then \
echo "Creating directory $$i"; \
$(INSTALL) -d -m $(DIRMODE) $(DESTDIR)$$i; \
else true; \
fi; \
done
@for d in $(LIBSUBDIRS); \
do \
a=$(srcdir)/Lib/$$d; \
if test ! -d $$a; then continue; else true; fi; \
b=$(LIBDEST)/$$d; \
if test ! -d $(DESTDIR)$$b; then \
echo "Creating directory $$b"; \
$(INSTALL) -d -m $(DIRMODE) $(DESTDIR)$$b; \
else true; \
fi; \
done
@for i in $(srcdir)/Lib/*.py $(srcdir)/Lib/*.doc $(srcdir)/Lib/*.egg-info ; \
do \
if test -x $$i; then \
$(INSTALL_SCRIPT) $$i $(DESTDIR)$(LIBDEST); \
echo $(INSTALL_SCRIPT) $$i $(LIBDEST); \
else \
$(INSTALL_DATA) $$i $(DESTDIR)$(LIBDEST); \
echo $(INSTALL_DATA) $$i $(LIBDEST); \
fi; \
done
@for d in $(LIBSUBDIRS); \
do \
a=$(srcdir)/Lib/$$d; \
if test ! -d $$a; then continue; else true; fi; \
if test `ls $$a | wc -l` -lt 1; then continue; fi; \
b=$(LIBDEST)/$$d; \
for i in $$a/*; \
do \
case $$i in \
*CVS) ;; \
*.py[co]) ;; \
*.orig) ;; \
*~) ;; \
*) \
if test -d $$i; then continue; fi; \
if test -x $$i; then \
echo $(INSTALL_SCRIPT) $$i $$b; \
$(INSTALL_SCRIPT) $$i $(DESTDIR)$$b; \
else \
echo $(INSTALL_DATA) $$i $$b; \
$(INSTALL_DATA) $$i $(DESTDIR)$$b; \
fi;; \
esac; \
done; \
done
$(INSTALL_DATA) $(srcdir)/LICENSE $(DESTDIR)$(LIBDEST)/LICENSE.txt
-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
./$(BUILDPYTHON) -Wi -tt $(DESTDIR)$(LIBDEST)/compileall.py \
-d $(LIBDEST) -f \
-x 'bad_coding|badsyntax|site-packages' $(DESTDIR)$(LIBDEST)
-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
./$(BUILDPYTHON) -Wi -tt -O $(DESTDIR)$(LIBDEST)/compileall.py \
-d $(LIBDEST) -f \
-x 'bad_coding|badsyntax|site-packages' $(DESTDIR)$(LIBDEST)
-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
./$(BUILDPYTHON) -Wi -t $(DESTDIR)$(LIBDEST)/compileall.py \
-d $(LIBDEST)/site-packages -f \
-x badsyntax $(DESTDIR)$(LIBDEST)/site-packages
-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
./$(BUILDPYTHON) -Wi -t -O $(DESTDIR)$(LIBDEST)/compileall.py \
-d $(LIBDEST)/site-packages -f \
-x badsyntax $(DESTDIR)$(LIBDEST)/site-packages
-PYTHONPATH=$(DESTDIR)$(LIBDEST) $(RUNSHARED) \
./$(BUILDPYTHON) -Wi -t -c "import lib2to3.pygram"
# Create the PLATDIR source directory, if one wasn't distributed..
$(srcdir)/Lib/$(PLATDIR):
mkdir $(srcdir)/Lib/$(PLATDIR)
cp $(srcdir)/Lib/plat-generic/regen $(srcdir)/Lib/$(PLATDIR)/regen
export PATH; PATH="`pwd`:$$PATH"; \
export PYTHONPATH; PYTHONPATH="`pwd`/Lib"; \
export DYLD_FRAMEWORK_PATH; DYLD_FRAMEWORK_PATH="`pwd`"; \
export EXE; EXE="$(BUILDEXE)"; \
cd $(srcdir)/Lib/$(PLATDIR); ./regen
# Install the include files
INCLDIRSTOMAKE=$(INCLUDEDIR) $(CONFINCLUDEDIR) $(INCLUDEPY) $(CONFINCLUDEPY)
inclinstall:
@for i in $(INCLDIRSTOMAKE); \
do \
if test ! -d $(DESTDIR)$$i; then \
echo "Creating directory $$i"; \
$(INSTALL) -d -m $(DIRMODE) $(DESTDIR)$$i; \
else true; \
fi; \
done
@for i in $(srcdir)/Include/*.h; \
do \
echo $(INSTALL_DATA) $$i $(INCLUDEPY); \
$(INSTALL_DATA) $$i $(DESTDIR)$(INCLUDEPY); \
done
$(INSTALL_DATA) pyconfig.h $(DESTDIR)$(CONFINCLUDEPY)/pyconfig.h
# Install the library and miscellaneous stuff needed for extending/embedding
# This goes into $(exec_prefix)
LIBPL= $(LIBP)/config
libainstall: all
@for i in $(LIBDIR) $(LIBP) $(LIBPL); \
do \
if test ! -d $(DESTDIR)$$i; then \
echo "Creating directory $$i"; \
$(INSTALL) -d -m $(DIRMODE) $(DESTDIR)$$i; \
else true; \
fi; \
done
@if test -d $(LIBRARY); then :; else \
if test "$(PYTHONFRAMEWORKDIR)" = no-framework; then \
if test "$(SO)" = .dll; then \
$(INSTALL_DATA) $(LDLIBRARY) $(DESTDIR)$(LIBPL) ; \
else \
$(INSTALL_DATA) $(LIBRARY) $(DESTDIR)$(LIBPL)/$(LIBRARY) ; \
$(RANLIB) $(DESTDIR)$(LIBPL)/$(LIBRARY) ; \
fi; \
else \
echo Skip install of $(LIBRARY) - use make frameworkinstall; \
fi; \
fi
$(INSTALL_DATA) Modules/config.c $(DESTDIR)$(LIBPL)/config.c
$(INSTALL_DATA) Modules/python.o $(DESTDIR)$(LIBPL)/python.o
$(INSTALL_DATA) $(srcdir)/Modules/config.c.in $(DESTDIR)$(LIBPL)/config.c.in
$(INSTALL_DATA) Makefile $(DESTDIR)$(LIBPL)/Makefile
$(INSTALL_DATA) Modules/Setup $(DESTDIR)$(LIBPL)/Setup
$(INSTALL_DATA) Modules/Setup.local $(DESTDIR)$(LIBPL)/Setup.local
$(INSTALL_DATA) Modules/Setup.config $(DESTDIR)$(LIBPL)/Setup.config
$(INSTALL_SCRIPT) $(srcdir)/Modules/makesetup $(DESTDIR)$(LIBPL)/makesetup
$(INSTALL_SCRIPT) $(srcdir)/install-sh $(DESTDIR)$(LIBPL)/install-sh
# Substitution happens here, as the completely-expanded BINDIR
# is not available in configure
sed -e "s,@EXENAME@,$(BINDIR)/python$(VERSION)$(EXE)," < $(srcdir)/Misc/python-config.in >python-config
$(INSTALL_SCRIPT) python-config $(DESTDIR)$(BINDIR)/python$(VERSION)-config
rm python-config
@if [ -s Modules/python.exp -a \
"`echo $(MACHDEP) | sed 's/^\(...\).*/\1/'`" = "aix" ]; then \
echo; echo "Installing support files for building shared extension modules on AIX:"; \
$(INSTALL_DATA) Modules/python.exp \
$(DESTDIR)$(LIBPL)/python.exp; \
echo; echo "$(LIBPL)/python.exp"; \
$(INSTALL_SCRIPT) $(srcdir)/Modules/makexp_aix \
$(DESTDIR)$(LIBPL)/makexp_aix; \
echo "$(LIBPL)/makexp_aix"; \
$(INSTALL_SCRIPT) $(srcdir)/Modules/ld_so_aix \
$(DESTDIR)$(LIBPL)/ld_so_aix; \
echo "$(LIBPL)/ld_so_aix"; \
echo; echo "See Misc/AIX-NOTES for details."; \
else true; \
fi
# Install the dynamically loadable modules
# This goes into $(exec_prefix)
sharedinstall:
$(RUNSHARED) ./$(BUILDPYTHON) -E $(srcdir)/setup.py install \
--prefix=$(prefix) \
--install-scripts=$(BINDIR) \
--install-platlib=$(DESTSHARED) \
--root=/$(DESTDIR)
# Here are a couple of targets for MacOSX again, to install a full
# framework-based Python. frameworkinstall installs everything, the
# subtargets install specific parts. Much of the actual work is offloaded to
# the Makefile in Mac
#
#
# This target is here for backward compatiblity, previous versions of Python
# hadn't integrated framework installation in the normal install process.
frameworkinstall: install
# On install, we re-make the framework
# structure in the install location, /Library/Frameworks/ or the argument to
# --enable-framework. If --enable-framework has been specified then we have
# automatically set prefix to the location deep down in the framework, so we
# only have to cater for the structural bits of the framework.
frameworkinstallframework: frameworkinstallstructure install frameworkinstallmaclib
frameworkinstallstructure: $(LDLIBRARY)
@if test "$(PYTHONFRAMEWORKDIR)" = no-framework; then \
echo Not configured with --enable-framework; \
exit 1; \
else true; \
fi
@for i in $(prefix)/Resources/English.lproj $(prefix)/lib; do\
if test ! -d $(DESTDIR)$$i; then \
echo "Creating directory $(DESTDIR)$$i"; \
$(INSTALL) -d -m $(DIRMODE) $(DESTDIR)$$i; \
else true; \
fi; \
done
$(LN) -fsn include/python$(VERSION) $(DESTDIR)$(prefix)/Headers
sed 's/%VERSION%/'"`$(RUNSHARED) ./$(BUILDPYTHON) -c 'import platform; print platform.python_version()'`"'/g' < $(RESSRCDIR)/Info.plist > $(DESTDIR)$(prefix)/Resources/Info.plist
$(LN) -fsn $(VERSION) $(DESTDIR)$(PYTHONFRAMEWORKINSTALLDIR)/Versions/Current
$(LN) -fsn Versions/Current/$(PYTHONFRAMEWORK) $(DESTDIR)$(PYTHONFRAMEWORKINSTALLDIR)/$(PYTHONFRAMEWORK)
$(LN) -fsn Versions/Current/Headers $(DESTDIR)$(PYTHONFRAMEWORKINSTALLDIR)/Headers
$(LN) -fsn Versions/Current/Resources $(DESTDIR)$(PYTHONFRAMEWORKINSTALLDIR)/Resources
$(INSTALL_SHARED) $(LDLIBRARY) $(DESTDIR)$(PYTHONFRAMEWORKPREFIX)/$(LDLIBRARY)
# This installs Mac/Lib into the framework
# Install a number of symlinks to keep software that expects a normal unix
# install (which includes python-config) happy.
frameworkinstallmaclib:
ln -fs "../../../$(PYTHONFRAMEWORK)" "$(DESTDIR)$(prefix)/lib/python$(VERSION)/config/libpython$(VERSION).a"
cd Mac && $(MAKE) installmacsubtree DESTDIR="$(DESTDIR)"
# This installs the IDE, the Launcher and other apps into /Applications
frameworkinstallapps:
cd Mac && $(MAKE) installapps DESTDIR="$(DESTDIR)"
# This install the unix python and pythonw tools in /usr/local/bin
frameworkinstallunixtools:
cd Mac && $(MAKE) installunixtools DESTDIR="$(DESTDIR)"
frameworkaltinstallunixtools:
cd Mac && $(MAKE) altinstallunixtools DESTDIR="$(DESTDIR)"
# This installs the Demos and Tools into the applications directory.
# It is not part of a normal frameworkinstall
frameworkinstallextras:
cd Mac && Make installextras DESTDIR="$(DESTDIR)"
# This installs a few of the useful scripts in Tools/scripts
scriptsinstall:
SRCDIR=$(srcdir) $(RUNSHARED) \
./$(BUILDPYTHON) $(srcdir)/Tools/scripts/setup.py install \
--prefix=$(prefix) \
--install-scripts=$(BINDIR) \
--root=/$(DESTDIR)
# Build the toplevel Makefile
Makefile.pre: Makefile.pre.in config.status
CONFIG_FILES=Makefile.pre CONFIG_HEADERS= $(SHELL) config.status
$(MAKE) -f Makefile.pre Makefile
# Run the configure script.
config.status: $(srcdir)/configure
$(SHELL) $(srcdir)/configure $(CONFIG_ARGS)
.PRECIOUS: config.status $(BUILDPYTHON) Makefile Makefile.pre
# Some make's put the object file in the current directory
.c.o:
$(CC) -c $(PY_CFLAGS) -o $@ $<
# Run reindent on the library
reindent:
./$(BUILDPYTHON) $(srcdir)/Tools/scripts/reindent.py -r $(srcdir)/Lib
# Rerun configure with the same options as it was run last time,
# provided the config.status script exists
recheck:
$(SHELL) config.status --recheck
$(SHELL) config.status
# Rebuild the configure script from configure.in; also rebuild pyconfig.h.in
autoconf:
(cd $(srcdir); autoconf)
(cd $(srcdir); autoheader)
# Create a tags file for vi
tags::
cd $(srcdir); \
ctags -w -t Include/*.h; \
for i in $(SRCDIRS); do ctags -w -t -a $$i/*.[ch]; \
done; \
sort -o tags tags
# Create a tags file for GNU Emacs
TAGS::
cd $(srcdir); \
etags Include/*.h; \
for i in $(SRCDIRS); do etags -a $$i/*.[ch]; done
# Sanitation targets -- clean leaves libraries, executables and tags
# files, which clobber removes those as well
pycremoval:
find $(srcdir) -name '*.py[co]' -exec rm -f {} ';'
rmtestturds:
-rm -f *BAD *GOOD *SKIPPED
-rm -rf OUT
-rm -f *.TXT
-rm -f *.txt
-rm -f gb-18030-2000.xml
docclean:
-rm -rf Doc/build
-rm -rf Doc/tools/sphinx Doc/tools/pygments Doc/tools/docutils
clean: pycremoval
find . -name '*.o' -exec rm -f {} ';'
find . -name '*.s[ol]' -exec rm -f {} ';'
find $(srcdir)/build -name 'fficonfig.h' -exec rm -f {} ';' || true
find $(srcdir)/build -name 'fficonfig.py' -exec rm -f {} ';' || true
profile-removal:
find . -name '*.gc??' -exec rm -f {} ';'
clobber: clean profile-removal
-rm -f $(BUILDPYTHON) $(PGEN) $(LIBRARY) $(LDLIBRARY) $(DLLLIBRARY) \
tags TAGS \
config.cache config.log pyconfig.h Modules/config.c
-rm -rf build platform
-rm -rf $(PYTHONFRAMEWORKDIR)
# Make things extra clean, before making a distribution:
# remove all generated files, even Makefile[.pre]
# Keep configure and Python-ast.[ch], it's possible they can't be generated
distclean: clobber
-rm -f core Makefile Makefile.pre config.status \
Modules/Setup Modules/Setup.local Modules/Setup.config
find $(srcdir) '(' -name '*.fdc' -o -name '*~' \
-o -name '[@,#]*' -o -name '*.old' \
-o -name '*.orig' -o -name '*.rej' \
-o -name '*.bak' ')' \
-exec rm -f {} ';'
# Check for smelly exported symbols (not starting with Py/_Py)
smelly: all
nm -p $(LIBRARY) | \
sed -n "/ [TDB] /s/.* //p" | grep -v "^_*Py" | sort -u; \
# Find files with funny names
funny:
find $(DISTDIRS) \
-name .svn -prune \
-o -type d \
-o -name '*.[chs]' \
-o -name '*.py' \
-o -name '*.doc' \
-o -name '*.dat' \
-o -name '*.el' \
-o -name '*.fd' \
-o -name '*.in' \
-o -name '*.gif' \
-o -name '*.txt' \
-o -name '*.xml' \
-o -name '*.xbm' \
-o -name '*.xpm' \
-o -name '*.uue' \
-o -name '*.decTest' \
-o -name '*,[vpt]' \
-o -name 'Setup' \
-o -name 'Setup.*' \
-o -name regen \
-o -name README \
-o -name Makefile \
-o -name ChangeLog \
-o -name Repository \
-o -name Root \
-o -name Entries \
-o -name Tag \
-o -name tags \
-o -name TAGS \
-o -name .cvsignore \
-o -name MANIFEST \
-o -print
# Perform some verification checks on any modified files.
check:
./$(BUILDPYTHON) $(srcdir)/Tools/scripts/patchcheck.py
# Dependencies
Python/thread.o: @THREADHEADERS@
# Declare targets that aren't real files
.PHONY: all build_all sharedmods oldsharedmods test quicktest memtest
.PHONY: install altinstall oldsharedinstall bininstall altbininstall
.PHONY: maninstall libinstall inclinstall libainstall sharedinstall
.PHONY: frameworkinstall frameworkinstallframework frameworkinstallstructure
.PHONY: frameworkinstallmaclib frameworkinstallapps frameworkinstallunixtools
.PHONY: frameworkaltinstallunixtools recheck autoconf clean clobber distclean
.PHONY: smelly funny
# IF YOU PUT ANYTHING HERE IT WILL GO AWAY
|