summaryrefslogtreecommitdiffstats
path: root/Doc/whatsnew/whatsnew25.tex
blob: bbe1ff6649150948f1945e7ceff4d2c264b1451e (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
2144
2145
2146
2147
2148
2149
2150
2151
2152
2153
2154
2155
2156
2157
2158
2159
2160
2161
2162
2163
2164
2165
2166
2167
2168
2169
2170
2171
2172
2173
2174
2175
2176
2177
2178
2179
2180
2181
2182
2183
2184
2185
2186
2187
2188
2189
2190
2191
2192
2193
2194
2195
2196
2197
2198
2199
2200
2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213
2214
2215
2216
2217
2218
2219
2220
2221
2222
2223
2224
2225
2226
2227
2228
2229
2230
2231
2232
2233
2234
2235
2236
2237
2238
2239
2240
2241
2242
2243
2244
2245
2246
2247
2248
2249
2250
2251
2252
2253
2254
2255
2256
2257
2258
2259
2260
2261
2262
2263
2264
2265
2266
2267
2268
2269
2270
2271
2272
2273
2274
2275
2276
2277
2278
2279
2280
2281
2282
2283
2284
2285
2286
2287
2288
2289
2290
2291
2292
2293
2294
2295
2296
2297
2298
2299
2300
2301
2302
2303
2304
2305
2306
2307
2308
2309
2310
2311
2312
2313
2314
2315
2316
2317
2318
2319
2320
2321
2322
2323
2324
2325
2326
2327
2328
2329
2330
2331
2332
2333
2334
2335
2336
2337
2338
2339
2340
2341
2342
2343
2344
2345
2346
2347
2348
2349
2350
2351
2352
2353
2354
2355
2356
2357
2358
2359
2360
2361
2362
2363
2364
2365
2366
2367
2368
2369
2370
2371
2372
2373
2374
2375
2376
2377
2378
2379
2380
2381
2382
2383
2384
2385
2386
2387
2388
2389
2390
2391
2392
2393
2394
2395
2396
2397
2398
2399
2400
2401
2402
2403
2404
2405
2406
2407
2408
2409
2410
2411
2412
2413
2414
2415
2416
2417
2418
2419
2420
2421
2422
2423
2424
2425
2426
2427
2428
2429
2430
2431
2432
2433
2434
2435
2436
2437
2438
2439
2440
2441
2442
2443
2444
2445
2446
2447
2448
2449
2450
2451
2452
2453
2454
2455
2456
2457
2458
2459
2460
2461
2462
2463
2464
2465
2466
2467
2468
2469
2470
2471
2472
2473
2474
2475
2476
2477
2478
2479
2480
2481
2482
2483
2484
2485
2486
2487
2488
2489
2490
2491
2492
2493
2494
2495
2496
2497
2498
2499
2500
2501
2502
2503
2504
2505
2506
2507
\documentclass{howto}
\usepackage{distutils}
% $Id$

% Fix XXX comments

\title{What's New in Python 2.5}
\release{1.0}
\author{A.M. Kuchling}
\authoraddress{\email{amk@amk.ca}}

\begin{document}
\maketitle
\tableofcontents

This article explains the new features in Python 2.5.  The final
release of Python 2.5 is scheduled for August 2006;
\pep{356} describes the planned release schedule.

The changes in Python 2.5 are an interesting mix of language and
library improvements. The library enhancements will be more important
to Python's user community, I think, because several widely-useful
packages were added.  New modules include ElementTree for XML
processing (section~\ref{module-etree}), the SQLite database module
(section~\ref{module-sqlite}), and the \module{ctypes} module for
calling C functions (section~\ref{module-ctypes}).

The language changes are of middling significance.  Some pleasant new
features were added, but most of them aren't features that you'll use
every day.  Conditional expressions were finally added to the language
using a novel syntax; see section~\ref{pep-308}.  The new
'\keyword{with}' statement will make writing cleanup code easier
(section~\ref{pep-343}).  Values can now be passed into generators
(section~\ref{pep-342}).  Imports are now visible as either absolute
or relative (section~\ref{pep-328}).  Some corner cases of exception
handling are handled better (section~\ref{pep-341}).  All these
improvements are worthwhile, but they're improvements to one specific
language feature or another; none of them are broad modifications to
Python's semantics.

As well as the language and library additions, other improvements and
bugfixes were made throughout the source tree.  A search through the
SVN change logs finds there were 334 patches applied and 443 bugs
fixed between Python 2.4 and 2.5.  (Both figures are likely to be
underestimates.)  

This article doesn't try to be a complete specification of the new
features; instead changes are briefly introduced using helpful
examples.  For full details, you should always refer to the
documentation for Python 2.5 at \url{http://docs.python.org}.
If you want to understand the complete implementation and design
rationale, refer to the PEP for a particular new feature.

Comments, suggestions, and error reports for this document are
welcome; please e-mail them to the author or open a bug in the Python
bug tracker.

%======================================================================
\section{PEP 308: Conditional Expressions\label{pep-308}}

For a long time, people have been requesting a way to write
conditional expressions, which are expressions that return value A or
value B depending on whether a Boolean value is true or false.  A
conditional expression lets you write a single assignment statement
that has the same effect as the following:

\begin{verbatim}
if condition:
    x = true_value
else:
    x = false_value
\end{verbatim}

There have been endless tedious discussions of syntax on both
python-dev and comp.lang.python.  A vote was even held that found the
majority of voters wanted conditional expressions in some form,
but there was no syntax that was preferred by a clear majority.
Candidates included C's \code{cond ? true_v : false_v},
\code{if cond then true_v else false_v}, and 16 other variations.

Guido van~Rossum eventually chose a surprising syntax:

\begin{verbatim}
x = true_value if condition else false_value
\end{verbatim}

Evaluation is still lazy as in existing Boolean expressions, so the
order of evaluation jumps around a bit.  The \var{condition}
expression in the middle is evaluated first, and the \var{true_value}
expression is evaluated only if the condition was true.  Similarly,
the \var{false_value} expression is only evaluated when the condition
is false.

This syntax may seem strange and backwards; why does the condition go
in the \emph{middle} of the expression, and not in the front as in C's
\code{c ? x : y}?  The decision was checked by applying the new syntax
to the modules in the standard library and seeing how the resulting
code read.  In many cases where a conditional expression is used, one
value seems to be the 'common case' and one value is an 'exceptional
case', used only on rarer occasions when the condition isn't met.  The
conditional syntax makes this pattern a bit more obvious:

\begin{verbatim}
contents = ((doc + '\n') if doc else '')
\end{verbatim}

I read the above statement as meaning ``here \var{contents} is 
usually assigned a value of \code{doc+'\e n'}; sometimes 
\var{doc} is empty, in which special case an empty string is returned.''  
I doubt I will use conditional expressions very often where there 
isn't a clear common and uncommon case.

There was some discussion of whether the language should require
surrounding conditional expressions with parentheses.  The decision
was made to \emph{not} require parentheses in the Python language's
grammar, but as a matter of style I think you should always use them.
Consider these two statements:

\begin{verbatim}
# First version -- no parens
level = 1 if logging else 0

# Second version -- with parens
level = (1 if logging else 0)
\end{verbatim}

In the first version, I think a reader's eye might group the statement
into 'level = 1', 'if logging', 'else 0', and think that the condition
decides whether the assignment to \var{level} is performed.  The
second version reads better, in my opinion, because it makes it clear
that the assignment is always performed and the choice is being made
between two values.

Another reason for including the brackets: a few odd combinations of
list comprehensions and lambdas could look like incorrect conditional
expressions. See \pep{308} for some examples.  If you put parentheses
around your conditional expressions, you won't run into this case.


\begin{seealso}

\seepep{308}{Conditional Expressions}{PEP written by
Guido van~Rossum and Raymond D. Hettinger; implemented by Thomas
Wouters.}

\end{seealso}


%======================================================================
\section{PEP 309: Partial Function Application\label{pep-309}}

The \module{functools} module is intended to contain tools for
functional-style programming.  

One useful tool in this module is the \function{partial()} function.
For programs written in a functional style, you'll sometimes want to
construct variants of existing functions that have some of the
parameters filled in.  Consider a Python function \code{f(a, b, c)};
you could create a new function \code{g(b, c)} that was equivalent to
\code{f(1, b, c)}.  This is called ``partial function application''.

\function{partial} takes the arguments
\code{(\var{function}, \var{arg1}, \var{arg2}, ...
\var{kwarg1}=\var{value1}, \var{kwarg2}=\var{value2})}.  The resulting
object is callable, so you can just call it to invoke \var{function}
with the filled-in arguments.

Here's a small but realistic example:

\begin{verbatim}
import functools

def log (message, subsystem):
    "Write the contents of 'message' to the specified subsystem."
    print '%s: %s' % (subsystem, message)
    ...

server_log = functools.partial(log, subsystem='server')
server_log('Unable to open socket')
\end{verbatim}

Here's another example, from a program that uses PyGTK.  Here a
context-sensitive pop-up menu is being constructed dynamically.  The
callback provided for the menu option is a partially applied version
of the \method{open_item()} method, where the first argument has been
provided.

\begin{verbatim}
...
class Application:
    def open_item(self, path):
       ...
    def init (self):
        open_func = functools.partial(self.open_item, item_path)
        popup_menu.append( ("Open", open_func, 1) )
\end{verbatim}


Another function in the \module{functools} module is the
\function{update_wrapper(\var{wrapper}, \var{wrapped})} function that
helps you write well-behaved decorators.  \function{update_wrapper()}
copies the name, module, and docstring attribute to a wrapper function
so that tracebacks inside the wrapped function are easier to
understand.  For example, you might write:

\begin{verbatim}
def my_decorator(f):
    def wrapper(*args, **kwds):
        print 'Calling decorated function'
        return f(*args, **kwds)
    functools.update_wrapper(wrapper, f)
    return wrapper
\end{verbatim}

\function{wraps()} is a decorator that can be used inside your own
decorators to copy the wrapped function's information.  An alternate 
version of the previous example would be:

\begin{verbatim}
def my_decorator(f):
    @functools.wraps(f)
    def wrapper(*args, **kwds):
        print 'Calling decorated function'
        return f(*args, **kwds)
    return wrapper
\end{verbatim}

\begin{seealso}

\seepep{309}{Partial Function Application}{PEP proposed and written by
Peter Harris; implemented by Hye-Shik Chang and Nick Coghlan, with
adaptations by Raymond Hettinger.}

\end{seealso}


%======================================================================
\section{PEP 314: Metadata for Python Software Packages v1.1\label{pep-314}}

Some simple dependency support was added to Distutils.  The
\function{setup()} function now has \code{requires}, \code{provides},
and \code{obsoletes} keyword parameters.  When you build a source
distribution using the \code{sdist} command, the dependency
information will be recorded in the \file{PKG-INFO} file.

Another new keyword parameter is \code{download_url}, which should be
set to a URL for the package's source code.  This means it's now
possible to look up an entry in the package index, determine the
dependencies for a package, and download the required packages.

\begin{verbatim}
VERSION = '1.0'
setup(name='PyPackage', 
      version=VERSION,
      requires=['numarray', 'zlib (>=1.1.4)'],
      obsoletes=['OldPackage']
      download_url=('http://www.example.com/pypackage/dist/pkg-%s.tar.gz'
                    % VERSION),
     )
\end{verbatim}

Another new enhancement to the Python package index at
\url{http://cheeseshop.python.org} is storing source and binary
archives for a package.  The new \command{upload} Distutils command
will upload a package to the repository.

Before a package can be uploaded, you must be able to build a
distribution using the \command{sdist} Distutils command.  Once that
works, you can run \code{python setup.py upload} to add your package
to the PyPI archive.  Optionally you can GPG-sign the package by
supplying the \longprogramopt{sign} and
\longprogramopt{identity} options.

Package uploading was implemented by Martin von~L\"owis and Richard Jones. 
 
\begin{seealso}

\seepep{314}{Metadata for Python Software Packages v1.1}{PEP proposed
and written by A.M. Kuchling, Richard Jones, and Fred Drake; 
implemented by Richard Jones and Fred Drake.}

\end{seealso}


%======================================================================
\section{PEP 328: Absolute and Relative Imports\label{pep-328}}

The simpler part of PEP 328 was implemented in Python 2.4: parentheses
could now be used to enclose the names imported from a module using
the \code{from ... import ...} statement, making it easier to import
many different names.

The more complicated part has been implemented in Python 2.5:
importing a module can be specified to use absolute or
package-relative imports.  The plan is to move toward making absolute
imports the default in future versions of Python.

Let's say you have a package directory like this:
\begin{verbatim}
pkg/
pkg/__init__.py
pkg/main.py
pkg/string.py
\end{verbatim}

This defines a package named \module{pkg} containing the
\module{pkg.main} and \module{pkg.string} submodules.  

Consider the code in the \file{main.py} module.  What happens if it
executes the statement \code{import string}?  In Python 2.4 and
earlier, it will first look in the package's directory to perform a
relative import, finds \file{pkg/string.py}, imports the contents of
that file as the \module{pkg.string} module, and that module is bound
to the name \samp{string} in the \module{pkg.main} module's namespace.

That's fine if \module{pkg.string} was what you wanted.  But what if
you wanted Python's standard \module{string} module?  There's no clean
way to ignore \module{pkg.string} and look for the standard module;
generally you had to look at the contents of \code{sys.modules}, which
is slightly unclean.   
Holger Krekel's \module{py.std} package provides a tidier way to perform
imports from the standard library, \code{import py ; py.std.string.join()},
but that package isn't available on all Python installations.

Reading code which relies on relative imports is also less clear,
because a reader may be confused about which module, \module{string}
or \module{pkg.string}, is intended to be used.  Python users soon
learned not to duplicate the names of standard library modules in the
names of their packages' submodules, but you can't protect against
having your submodule's name being used for a new module added in a
future version of Python.

In Python 2.5, you can switch \keyword{import}'s behaviour to 
absolute imports using a \code{from __future__ import absolute_import}
directive.  This absolute-import behaviour will become the default in
a future version (probably Python 2.7).  Once absolute imports 
are the default, \code{import string} will
always find the standard library's version.
It's suggested that users should begin using absolute imports as much
as possible, so it's preferable to begin writing \code{from pkg import
string} in your code.  

Relative imports are still possible by adding a leading period 
to the module name when using the \code{from ... import} form:

\begin{verbatim}
# Import names from pkg.string
from .string import name1, name2
# Import pkg.string
from . import string
\end{verbatim}

This imports the \module{string} module relative to the current
package, so in \module{pkg.main} this will import \var{name1} and
\var{name2} from \module{pkg.string}.  Additional leading periods
perform the relative import starting from the parent of the current
package.  For example, code in the \module{A.B.C} module can do:

\begin{verbatim}
from . import D                 # Imports A.B.D
from .. import E                # Imports A.E
from ..F import G               # Imports A.F.G
\end{verbatim}

Leading periods cannot be used with the \code{import \var{modname}} 
form of the import statement, only the \code{from ... import} form.

\begin{seealso}

\seepep{328}{Imports: Multi-Line and Absolute/Relative}
{PEP written by Aahz; implemented by Thomas Wouters.}

\seeurl{http://codespeak.net/py/current/doc/index.html}
{The py library by Holger Krekel, which contains the \module{py.std} package.}

\end{seealso}


%======================================================================
\section{PEP 338: Executing Modules as Scripts\label{pep-338}}

The \programopt{-m} switch added in Python 2.4 to execute a module as
a script gained a few more abilities.  Instead of being implemented in
C code inside the Python interpreter, the switch now uses an
implementation in a new module, \module{runpy}.

The \module{runpy} module implements a more sophisticated import
mechanism so that it's now possible to run modules in a package such
as \module{pychecker.checker}.  The module also supports alternative
import mechanisms such as the \module{zipimport} module.  This means
you can add a .zip archive's path to \code{sys.path} and then use the
\programopt{-m} switch to execute code from the archive.


\begin{seealso}

\seepep{338}{Executing modules as scripts}{PEP written and 
implemented by Nick Coghlan.}

\end{seealso}


%======================================================================
\section{PEP 341: Unified try/except/finally\label{pep-341}}

Until Python 2.5, the \keyword{try} statement came in two
flavours. You could use a \keyword{finally} block to ensure that code
is always executed, or one or more \keyword{except} blocks to catch 
specific exceptions.  You couldn't combine both \keyword{except} blocks and a
\keyword{finally} block, because generating the right bytecode for the
combined version was complicated and it wasn't clear what the
semantics of the combined should be.  

Guido van~Rossum spent some time working with Java, which does support the
equivalent of combining \keyword{except} blocks and a
\keyword{finally} block, and this clarified what the statement should
mean.  In Python 2.5, you can now write:

\begin{verbatim}
try:
    block-1 ...
except Exception1:
    handler-1 ...
except Exception2:
    handler-2 ...
else:
    else-block
finally:
    final-block 
\end{verbatim}

The code in \var{block-1} is executed.  If the code raises an
exception, the various \keyword{except} blocks are tested: if the
exception is of class \class{Exception1}, \var{handler-1} is executed;
otherwise if it's of class \class{Exception2}, \var{handler-2} is
executed, and so forth.  If no exception is raised, the
\var{else-block} is executed.  

No matter what happened previously, the \var{final-block} is executed
once the code block is complete and any raised exceptions handled.
Even if there's an error in an exception handler or the
\var{else-block} and a new exception is raised, the
code in the \var{final-block} is still run.

\begin{seealso}

\seepep{341}{Unifying try-except and try-finally}{PEP written by Georg Brandl; 
implementation by Thomas Lee.}

\end{seealso}


%======================================================================
\section{PEP 342: New Generator Features\label{pep-342}}

Python 2.5 adds a simple way to pass values \emph{into} a generator.
As introduced in Python 2.3, generators only produce output; once a
generator's code was invoked to create an iterator, there was no way to
pass any new information into the function when its execution is
resumed.  Sometimes the ability to pass in some information would be
useful.  Hackish solutions to this include making the generator's code
look at a global variable and then changing the global variable's
value, or passing in some mutable object that callers then modify.

To refresh your memory of basic generators, here's a simple example:

\begin{verbatim}
def counter (maximum):
    i = 0
    while i < maximum:
        yield i
        i += 1
\end{verbatim}

When you call \code{counter(10)}, the result is an iterator that
returns the values from 0 up to 9.  On encountering the
\keyword{yield} statement, the iterator returns the provided value and
suspends the function's execution, preserving the local variables.
Execution resumes on the following call to the iterator's 
\method{next()} method, picking up after the \keyword{yield} statement.

In Python 2.3, \keyword{yield} was a statement; it didn't return any
value.  In 2.5, \keyword{yield} is now an expression, returning a
value that can be assigned to a variable or otherwise operated on:

\begin{verbatim}
val = (yield i)
\end{verbatim}

I recommend that you always put parentheses around a \keyword{yield}
expression when you're doing something with the returned value, as in
the above example.  The parentheses aren't always necessary, but it's
easier to always add them instead of having to remember when they're
needed.

(\pep{342} explains the exact rules, which are that a
\keyword{yield}-expression must always be parenthesized except when it
occurs at the top-level expression on the right-hand side of an
assignment.  This means you can write \code{val = yield i} but have to
use parentheses when there's an operation, as in \code{val = (yield i)
+ 12}.)

Values are sent into a generator by calling its
\method{send(\var{value})} method.  The generator's code is then
resumed and the \keyword{yield} expression returns the specified
\var{value}.  If the regular \method{next()} method is called, the
\keyword{yield} returns \constant{None}.

Here's the previous example, modified to allow changing the value of
the internal counter.

\begin{verbatim}
def counter (maximum):
    i = 0
    while i < maximum:
        val = (yield i)
        # If value provided, change counter
        if val is not None:
            i = val
        else:
            i += 1
\end{verbatim}

And here's an example of changing the counter:

\begin{verbatim}
>>> it = counter(10)
>>> print it.next()
0
>>> print it.next()
1
>>> print it.send(8)
8
>>> print it.next()
9
>>> print it.next()
Traceback (most recent call last):
  File ``t.py'', line 15, in ?
    print it.next()
StopIteration
\end{verbatim}

Because \keyword{yield} will often be returning \constant{None}, you
should always check for this case.  Don't just use its value in
expressions unless you're sure that the \method{send()} method
will be the only method used resume your generator function.

In addition to \method{send()}, there are two other new methods on
generators:

\begin{itemize}

  \item \method{throw(\var{type}, \var{value}=None,
  \var{traceback}=None)} is used to raise an exception inside the
  generator; the exception is raised by the \keyword{yield} expression
  where the generator's execution is paused.

  \item \method{close()} raises a new \exception{GeneratorExit}
  exception inside the generator to terminate the iteration.  
  On receiving this
  exception, the generator's code must either raise
  \exception{GeneratorExit} or \exception{StopIteration}; catching the 
  exception and doing anything else is illegal and will trigger
  a \exception{RuntimeError}.  \method{close()} will also be called by 
  Python's garbage collector when the generator is garbage-collected.

  If you need to run cleanup code when a \exception{GeneratorExit} occurs,
  I suggest using a \code{try: ... finally:} suite instead of 
  catching \exception{GeneratorExit}.

\end{itemize}

The cumulative effect of these changes is to turn generators from
one-way producers of information into both producers and consumers.

Generators also become \emph{coroutines}, a more generalized form of
subroutines.  Subroutines are entered at one point and exited at
another point (the top of the function, and a \keyword{return}
statement), but coroutines can be entered, exited, and resumed at
many different points (the \keyword{yield} statements).  We'll have to
figure out patterns for using coroutines effectively in Python.

The addition of the \method{close()} method has one side effect that
isn't obvious.  \method{close()} is called when a generator is
garbage-collected, so this means the generator's code gets one last
chance to run before the generator is destroyed.  This last chance
means that \code{try...finally} statements in generators can now be
guaranteed to work; the \keyword{finally} clause will now always get a
chance to run.  The syntactic restriction that you couldn't mix
\keyword{yield} statements with a \code{try...finally} suite has
therefore been removed.  This seems like a minor bit of language
trivia, but using generators and \code{try...finally} is actually
necessary in order to implement the  \keyword{with} statement
described by PEP 343.  I'll look at this new statement in the following 
section.

Another even more esoteric effect of this change: previously, the
\member{gi_frame} attribute of a generator was always a frame object.
It's now possible for \member{gi_frame} to be \code{None}
once the generator has been exhausted.

\begin{seealso}

\seepep{342}{Coroutines via Enhanced Generators}{PEP written by 
Guido van~Rossum and Phillip J. Eby;
implemented by Phillip J. Eby.  Includes examples of 
some fancier uses of generators as coroutines.

Earlier versions of these features were proposed in 
\pep{288} by Raymond Hettinger and \pep{325} by Samuele Pedroni.
}

\seeurl{http://en.wikipedia.org/wiki/Coroutine}{The Wikipedia entry for 
coroutines.}

\seeurl{http://www.sidhe.org/\~{}dan/blog/archives/000178.html}{An
explanation of coroutines from a Perl point of view, written by Dan
Sugalski.}

\end{seealso}


%======================================================================
\section{PEP 343: The 'with' statement\label{pep-343}}

The '\keyword{with}' statement clarifies code that previously would
use \code{try...finally} blocks to ensure that clean-up code is
executed.  In this section, I'll discuss the statement as it will
commonly be used.  In the next section, I'll examine the
implementation details and show how to write objects for use with this
statement.

The '\keyword{with}' statement is a new control-flow structure whose
basic structure is:

\begin{verbatim}
with expression [as variable]:
    with-block
\end{verbatim}

The expression is evaluated, and it should result in an object that
supports the context management protocol.  This object may return a
value that can optionally be bound to the name \var{variable}.  (Note
carefully that \var{variable} is \emph{not} assigned the result of
\var{expression}.)  The object can then run set-up code
before \var{with-block} is executed and some clean-up code
is executed after the block is done, even if the block raised an exception.

To enable the statement in Python 2.5, you need 
to add the following directive to your module:

\begin{verbatim}
from __future__ import with_statement
\end{verbatim}

The statement will always be enabled in Python 2.6.

Some standard Python objects now support the context management
protocol and can be used with the '\keyword{with}' statement. File
objects are one example:

\begin{verbatim}
with open('/etc/passwd', 'r') as f:
    for line in f:
        print line
        ... more processing code ...
\end{verbatim}

After this statement has executed, the file object in \var{f} will
have been automatically closed, even if the 'for' loop
raised an exception part-way through the block.

The \module{threading} module's locks and condition variables 
also support the '\keyword{with}' statement:

\begin{verbatim}
lock = threading.Lock()
with lock:
    # Critical section of code
    ...
\end{verbatim}

The lock is acquired before the block is executed and always released once 
the block is complete.

The \module{decimal} module's contexts, which encapsulate the desired
precision and rounding characteristics for computations, provide a 
\method{context_manager()} method for getting a context manager:

\begin{verbatim}
import decimal

# Displays with default precision of 28 digits
v1 = decimal.Decimal('578')
print v1.sqrt()

ctx = decimal.Context(prec=16) 
with ctx.context_manager():
    # All code in this block uses a precision of 16 digits.
    # The original context is restored on exiting the block.
    print v1.sqrt()
\end{verbatim}

\subsection{Writing Context Managers\label{context-managers}}

Under the hood, the '\keyword{with}' statement is fairly complicated.
Most people will only use '\keyword{with}' in company with existing
objects and don't need to know these details, so you can skip the rest
of this section if you like.  Authors of new objects will need to
understand the details of the underlying implementation and should
keep reading.

A high-level explanation of the context management protocol is:

\begin{itemize}

\item The expression is evaluated and should result in an object
called a ``context manager''.  The context manager must have
\method{__enter__()} and \method{__exit__()} methods.

\item The context manager's \method{__enter__()} method is called.  The value
returned is assigned to \var{VAR}.  If no \code{'as \var{VAR}'} clause
is present, the value is simply discarded.

\item The code in \var{BLOCK} is executed.

\item If \var{BLOCK} raises an exception, the
\method{__exit__(\var{type}, \var{value}, \var{traceback})} is called
with the exception details, the same values returned by
\function{sys.exc_info()}.  The method's return value controls whether
the exception is re-raised: any false value re-raises the exception,
and \code{True} will result in suppressing it.  You'll only rarely
want to suppress the exception, because if you do
the author of the code containing the
'\keyword{with}' statement will never realize anything went wrong.

\item If \var{BLOCK} didn't raise an exception, 
the \method{__exit__()} method is still called,
but \var{type}, \var{value}, and \var{traceback} are all \code{None}.

\end{itemize}

Let's think through an example.  I won't present detailed code but
will only sketch the methods necessary for a database that supports
transactions.

(For people unfamiliar with database terminology: a set of changes to
the database are grouped into a transaction.  Transactions can be
either committed, meaning that all the changes are written into the
database, or rolled back, meaning that the changes are all discarded
and the database is unchanged.  See any database textbook for more
information.)

Let's assume there's an object representing a database connection.
Our goal will be to let the user write code like this:

\begin{verbatim}
db_connection = DatabaseConnection()
with db_connection as cursor:
    cursor.execute('insert into ...')
    cursor.execute('delete from ...')
    # ... more operations ...
\end{verbatim}

The transaction should be committed if the code in the block
runs flawlessly or rolled back if there's an exception.
Here's the basic interface
for \class{DatabaseConnection} that I'll assume:

\begin{verbatim}
class DatabaseConnection:
    # Database interface
    def cursor (self):
        "Returns a cursor object and starts a new transaction"
    def commit (self):
        "Commits current transaction"
    def rollback (self):
        "Rolls back current transaction"
\end{verbatim}

The \method {__enter__()} method is pretty easy, having only to start
a new transaction.  For this application the resulting cursor object
would be a useful result, so the method will return it.  The user can
then add \code{as cursor} to their '\keyword{with}' statement to bind
the cursor to a variable name.

\begin{verbatim}
class DatabaseConnection:
    ...
    def __enter__ (self):
        # Code to start a new transaction
        cursor = self.cursor()
        return cursor
\end{verbatim}

The \method{__exit__()} method is the most complicated because it's
where most of the work has to be done.  The method has to check if an
exception occurred.  If there was no exception, the transaction is
committed.  The transaction is rolled back if there was an exception.

In the code below, execution will just fall off the end of the
function, returning the default value of \code{None}.  \code{None} is
false, so the exception will be re-raised automatically.  If you
wished, you could be more explicit and add a \keyword{return}
statement at the marked location.

\begin{verbatim}
class DatabaseConnection:
    ...
    def __exit__ (self, type, value, tb):
        if tb is None:
            # No exception, so commit
            self.commit()
        else:
            # Exception occurred, so rollback.
            self.rollback()
            # return False
\end{verbatim}


\subsection{The contextlib module\label{module-contextlib}}

The new \module{contextlib} module provides some functions and a
decorator that are useful for writing objects for use with the
'\keyword{with}' statement.

The decorator is called \function{contextmanager}, and lets you write
a single generator function instead of defining a new class.  The generator
should yield exactly one value.  The code up to the \keyword{yield}
will be executed as the \method{__enter__()} method, and the value
yielded will be the method's return value that will get bound to the
variable in the '\keyword{with}' statement's \keyword{as} clause, if
any.  The code after the \keyword{yield} will be executed in the
\method{__exit__()} method.  Any exception raised in the block will be
raised by the \keyword{yield} statement.

Our database example from the previous section could be written 
using this decorator as:

\begin{verbatim}
from contextlib import contextmanager

@contextmanager
def db_transaction (connection):
    cursor = connection.cursor()
    try:
        yield cursor
    except:
        connection.rollback()
        raise
    else:
        connection.commit()

db = DatabaseConnection()
with db_transaction(db) as cursor:
    ...
\end{verbatim}

The \module{contextlib} module also has a \function{nested(\var{mgr1},
\var{mgr2}, ...)} function that combines a number of context managers so you
don't need to write nested '\keyword{with}' statements.  In this
example, the single '\keyword{with}' statement both starts a database
transaction and acquires a thread lock:

\begin{verbatim}
lock = threading.Lock()
with nested (db_transaction(db), lock) as (cursor, locked):
    ...
\end{verbatim}

Finally, the \function{closing(\var{object})} function
returns \var{object} so that it can be bound to a variable,
and calls \code{\var{object}.close()} at the end of the block.

\begin{verbatim}
import urllib, sys
from contextlib import closing

with closing(urllib.urlopen('http://www.yahoo.com')) as f:
    for line in f:
        sys.stdout.write(line)
\end{verbatim}

\begin{seealso}

\seepep{343}{The ``with'' statement}{PEP written by Guido van~Rossum
and Nick Coghlan; implemented by Mike Bland, Guido van~Rossum, and
Neal Norwitz.  The PEP shows the code generated for a '\keyword{with}'
statement, which can be helpful in learning how the statement works.}

\seeurl{../lib/module-contextlib.html}{The documentation 
for the \module{contextlib} module.}

\end{seealso}


%======================================================================
\section{PEP 352: Exceptions as New-Style Classes\label{pep-352}}

Exception classes can now be new-style classes, not just classic
classes, and the built-in \exception{Exception} class and all the
standard built-in exceptions (\exception{NameError},
\exception{ValueError}, etc.) are now new-style classes.

The inheritance hierarchy for exceptions has been rearranged a bit.
In 2.5, the inheritance relationships are:

\begin{verbatim}
BaseException       # New in Python 2.5
|- KeyboardInterrupt
|- SystemExit
|- Exception
   |- (all other current built-in exceptions)
\end{verbatim}

This rearrangement was done because people often want to catch all
exceptions that indicate program errors.  \exception{KeyboardInterrupt} and
\exception{SystemExit} aren't errors, though, and usually represent an explicit
action such as the user hitting Control-C or code calling
\function{sys.exit()}.  A bare \code{except:} will catch all exceptions,
so you commonly need to list \exception{KeyboardInterrupt} and
\exception{SystemExit} in order to re-raise them.  The usual pattern is:

\begin{verbatim}
try:
    ...
except (KeyboardInterrupt, SystemExit):
    raise
except: 
    # Log error...  
    # Continue running program...
\end{verbatim}

In Python 2.5, you can now write \code{except Exception} to achieve
the same result, catching all the exceptions that usually indicate errors 
but leaving \exception{KeyboardInterrupt} and
\exception{SystemExit} alone.  As in previous versions,
a bare \code{except:} still catches all exceptions.

The goal for Python 3.0 is to require any class raised as an exception
to derive from \exception{BaseException} or some descendant of
\exception{BaseException}, and future releases in the
Python 2.x series may begin to enforce this constraint.  Therefore, I
suggest you begin making all your exception classes derive from
\exception{Exception} now.  It's been suggested that the bare
\code{except:} form should be removed in Python 3.0, but Guido van~Rossum
hasn't decided whether to do this or not.

Raising of strings as exceptions, as in the statement \code{raise
"Error occurred"}, is deprecated in Python 2.5 and will trigger a
warning.  The aim is to be able to remove the string-exception feature
in a few releases.


\begin{seealso}

\seepep{352}{Required Superclass for Exceptions}{PEP written by 
Brett Cannon and Guido van~Rossum; implemented by Brett Cannon.}

\end{seealso}


%======================================================================
\section{PEP 353: Using ssize_t as the index type\label{pep-353}}

A wide-ranging change to Python's C API, using a new 
\ctype{Py_ssize_t} type definition instead of \ctype{int}, 
will permit the interpreter to handle more data on 64-bit platforms.
This change doesn't affect Python's capacity on 32-bit platforms.

Various pieces of the Python interpreter used C's \ctype{int} type to
store sizes or counts; for example, the number of items in a list or
tuple were stored in an \ctype{int}.  The C compilers for most 64-bit
platforms still define \ctype{int} as a 32-bit type, so that meant
that lists could only hold up to \code{2**31 - 1} = 2147483647 items.
(There are actually a few different programming models that 64-bit C
compilers can use -- see
\url{http://www.unix.org/version2/whatsnew/lp64_wp.html} for a
discussion -- but the most commonly available model leaves \ctype{int}
as 32 bits.)

A limit of 2147483647 items doesn't really matter on a 32-bit platform
because you'll run out of memory before hitting the length limit.
Each list item requires space for a pointer, which is 4 bytes, plus
space for a \ctype{PyObject} representing the item.  2147483647*4 is
already more bytes than a 32-bit address space can contain.

It's possible to address that much memory on a 64-bit platform,
however.  The pointers for a list that size would only require 16~GiB
of space, so it's not unreasonable that Python programmers might
construct lists that large.  Therefore, the Python interpreter had to
be changed to use some type other than \ctype{int}, and this will be a
64-bit type on 64-bit platforms.  The change will cause
incompatibilities on 64-bit machines, so it was deemed worth making
the transition now, while the number of 64-bit users is still
relatively small.  (In 5 or 10 years, we may \emph{all} be on 64-bit
machines, and the transition would be more painful then.)

This change most strongly affects authors of C extension modules.  
Python strings and container types such as lists and tuples 
now use \ctype{Py_ssize_t} to store their size.  
Functions such as \cfunction{PyList_Size()} 
now return \ctype{Py_ssize_t}.  Code in extension modules
may therefore need to have some variables changed to
\ctype{Py_ssize_t}.  

The \cfunction{PyArg_ParseTuple()} and \cfunction{Py_BuildValue()} functions
have a new conversion code, \samp{n}, for \ctype{Py_ssize_t}.  
\cfunction{PyArg_ParseTuple()}'s \samp{s\#} and \samp{t\#} still output
\ctype{int} by default, but you can define the macro 
\csimplemacro{PY_SSIZE_T_CLEAN} before including \file{Python.h} 
to make them return \ctype{Py_ssize_t}.

\pep{353} has a section on conversion guidelines that 
extension authors should read to learn about supporting 64-bit
platforms.

\begin{seealso}

\seepep{353}{Using ssize_t as the index type}{PEP written and implemented by Martin von~L\"owis.}

\end{seealso}


%======================================================================
\section{PEP 357: The '__index__' method\label{pep-357}}

The NumPy developers had a problem that could only be solved by adding
a new special method, \method{__index__}.  When using slice notation,
as in \code{[\var{start}:\var{stop}:\var{step}]}, the values of the
\var{start}, \var{stop}, and \var{step} indexes must all be either
integers or long integers.  NumPy defines a variety of specialized
integer types corresponding to unsigned and signed integers of 8, 16,
32, and 64 bits, but there was no way to signal that these types could
be used as slice indexes.

Slicing can't just use the existing \method{__int__} method because
that method is also used to implement coercion to integers.  If
slicing used \method{__int__}, floating-point numbers would also
become legal slice indexes and that's clearly an undesirable
behaviour.

Instead, a new special method called \method{__index__} was added.  It
takes no arguments and returns an integer giving the slice index to
use.  For example:

\begin{verbatim}
class C:
    def __index__ (self):
        return self.value  
\end{verbatim}

The return value must be either a Python integer or long integer.
The interpreter will check that the type returned is correct, and
raises a \exception{TypeError} if this requirement isn't met.

A corresponding \member{nb_index} slot was added to the C-level
\ctype{PyNumberMethods} structure to let C extensions implement this
protocol.  \cfunction{PyNumber_Index(\var{obj})} can be used in
extension code to call the \method{__index__} function and retrieve
its result.

\begin{seealso}

\seepep{357}{Allowing Any Object to be Used for Slicing}{PEP written 
and implemented by Travis Oliphant.}

\end{seealso}


%======================================================================
\section{Other Language Changes\label{other-lang}}

Here are all of the changes that Python 2.5 makes to the core Python
language.

\begin{itemize}

\item The \class{dict} type has a new hook for letting subclasses
provide a default value when a key isn't contained in the dictionary.
When a key isn't found, the dictionary's
\method{__missing__(\var{key})}
method will be called.  This hook is used to implement
the new \class{defaultdict} class in the \module{collections}
module.  The following example defines a dictionary 
that returns zero for any missing key:

\begin{verbatim}
class zerodict (dict):
    def __missing__ (self, key):
        return 0

d = zerodict({1:1, 2:2})
print d[1], d[2]   # Prints 1, 2
print d[3], d[4]   # Prints 0, 0
\end{verbatim}

\item Both 8-bit and Unicode strings have new \method{partition(sep)} 
and \method{rpartition(sep)} methods that simplify a common use case.

The \method{find(S)} method is often used to get an index which is
then used to slice the string and obtain the pieces that are before
and after the separator.  
\method{partition(sep)} condenses this
pattern into a single method call that returns a 3-tuple containing
the substring before the separator, the separator itself, and the
substring after the separator.  If the separator isn't found, the
first element of the tuple is the entire string and the other two
elements are empty.  \method{rpartition(sep)} also returns a 3-tuple
but starts searching from the end of the string; the \samp{r} stands
for 'reverse'.

Some examples:

\begin{verbatim}
>>> ('http://www.python.org').partition('://')
('http', '://', 'www.python.org')
>>> (u'Subject: a quick question').partition(':')
(u'Subject', u':', u' a quick question')
>>> ('file:/usr/share/doc/index.html').partition('://')
('file:/usr/share/doc/index.html', '', '')
>>> 'www.python.org'.rpartition('.')
('www.python', '.', 'org')
\end{verbatim}

(Implemented by Fredrik Lundh following a suggestion by Raymond Hettinger.)

\item The \method{startswith()} and \method{endswith()} methods
of string types now accept tuples of strings to check for.

\begin{verbatim}
def is_image_file (filename):
    return filename.endswith(('.gif', '.jpg', '.tiff'))
\end{verbatim}

(Implemented by Georg Brandl following a suggestion by Tom Lynn.)
% RFE #1491485

\item The \function{min()} and \function{max()} built-in functions
gained a \code{key} keyword parameter analogous to the \code{key}
argument for \method{sort()}.  This parameter supplies a function that
takes a single argument and is called for every value in the list;
\function{min()}/\function{max()} will return the element with the 
smallest/largest return value from this function.
For example, to find the longest string in a list, you can do:

\begin{verbatim}
L = ['medium', 'longest', 'short']
# Prints 'longest'
print max(L, key=len)              
# Prints 'short', because lexicographically 'short' has the largest value
print max(L)         
\end{verbatim}

(Contributed by Steven Bethard and Raymond Hettinger.)

\item Two new built-in functions, \function{any()} and
\function{all()}, evaluate whether an iterator contains any true or
false values.  \function{any()} returns \constant{True} if any value
returned by the iterator is true; otherwise it will return
\constant{False}.  \function{all()} returns \constant{True} only if
all of the values returned by the iterator evaluate as true.
(Suggested by Guido van~Rossum, and implemented by Raymond Hettinger.)

\item The result of a class's \method{__hash__()} method can now 
be either a long integer or a regular integer.  If a long integer is
returned, the hash of that value is taken.  In earlier versions the
hash value was required to be a regular integer, but in 2.5 the
\function{id()} built-in was changed to always return non-negative
numbers, and users often seem to use \code{id(self)} in
\method{__hash__()} methods (though this is discouraged).
% Bug #1536021

\item ASCII is now the default encoding for modules.  It's now 
a syntax error if a module contains string literals with 8-bit
characters but doesn't have an encoding declaration.  In Python 2.4
this triggered a warning, not a syntax error.  See \pep{263} 
for how to declare a module's encoding; for example, you might add 
a line like this near the top of the source file:

\begin{verbatim}
# -*- coding: latin1 -*-
\end{verbatim}

\item A new warning, \class{UnicodeWarning}, is triggered when 
you attempt to compare a Unicode string and an 8-bit string 
that can't be converted to Unicode using the default ASCII encoding.  
The result of the comparison is false:

\begin{verbatim}
>>> chr(128) == unichr(128)   # Can't convert chr(128) to Unicode
__main__:1: UnicodeWarning: Unicode equal comparison failed 
  to convert both arguments to Unicode - interpreting them 
  as being unequal
False
>>> chr(127) == unichr(127)   # chr(127) can be converted
True
\end{verbatim}

Previously this would raise a \class{UnicodeDecodeError} exception,
but in 2.5 this could result in puzzling problems when accessing a
dictionary.  If you looked up \code{unichr(128)} and \code{chr(128)}
was being used as a key, you'd get a \class{UnicodeDecodeError}
exception.  Other changes in 2.5 resulted in this exception being
raised instead of suppressed by the code in \file{dictobject.c} that
implements dictionaries.

Raising an exception for such a comparison is strictly correct, but
the change might have broken code, so instead 
\class{UnicodeWarning} was introduced.

(Implemented by Marc-Andr\'e Lemburg.)

\item One error that Python programmers sometimes make is forgetting
to include an \file{__init__.py} module in a package directory.
Debugging this mistake can be confusing, and usually requires running
Python with the \programopt{-v} switch to log all the paths searched.
In Python 2.5, a new \exception{ImportWarning} warning is triggered when
an import would have picked up a directory as a package but no
\file{__init__.py} was found.  This warning is silently ignored by default;
provide the \programopt{-Wd} option when running the Python executable
to display the warning message.
(Implemented by Thomas Wouters.)

\item The list of base classes in a class definition can now be empty.  
As an example, this is now legal:

\begin{verbatim}
class C():
    pass
\end{verbatim}
(Implemented by Brett Cannon.)

\end{itemize}


%======================================================================
\subsection{Interactive Interpreter Changes\label{interactive}}

In the interactive interpreter, \code{quit} and \code{exit} 
have long been strings so that new users get a somewhat helpful message
when they try to quit:

\begin{verbatim}
>>> quit
'Use Ctrl-D (i.e. EOF) to exit.'
\end{verbatim}

In Python 2.5, \code{quit} and \code{exit} are now objects that still
produce string representations of themselves, but are also callable.
Newbies who try \code{quit()} or \code{exit()} will now exit the
interpreter as they expect.  (Implemented by Georg Brandl.)

The Python executable now accepts the standard long options 
\longprogramopt{help} and \longprogramopt{version}; on Windows, 
it also accepts the \programopt{/?} option for displaying a help message.
(Implemented by Georg Brandl.)


%======================================================================
\subsection{Optimizations\label{opts}}

Several of the optimizations were developed at the NeedForSpeed
sprint, an event held in Reykjavik, Iceland, from May 21--28 2006.
The sprint focused on speed enhancements to the CPython implementation
and was funded by EWT LLC with local support from CCP Games.  Those
optimizations added at this sprint are specially marked in the
following list.

\begin{itemize}

\item When they were introduced 
in Python 2.4, the built-in \class{set} and \class{frozenset} types
were built on top of Python's dictionary type.  
In 2.5 the internal data structure has been customized for implementing sets,
and as a result sets will use a third less memory and are somewhat faster.
(Implemented by Raymond Hettinger.)

\item The speed of some Unicode operations, such as finding
substrings, string splitting, and character map encoding and decoding,
has been improved.  (Substring search and splitting improvements were
added by Fredrik Lundh and Andrew Dalke at the NeedForSpeed
sprint. Character maps were improved by Walter D\"orwald and
Martin von~L\"owis.)
% Patch 1313939, 1359618 

\item The \function{long(\var{str}, \var{base})} function is now
faster on long digit strings because fewer intermediate results are
calculated.  The peak is for strings of around 800--1000 digits where 
the function is 6 times faster.
(Contributed by Alan McIntyre and committed at the NeedForSpeed sprint.)
% Patch 1442927

\item The \module{struct} module now compiles structure format 
strings into an internal representation and caches this
representation, yielding a 20\% speedup.  (Contributed by Bob Ippolito
at the NeedForSpeed sprint.)

\item The \module{re} module got a 1 or 2\% speedup by switching to 
Python's allocator functions instead of the system's 
\cfunction{malloc()} and \cfunction{free()}.
(Contributed by Jack Diederich at the NeedForSpeed sprint.)

\item The code generator's peephole optimizer now performs
simple constant folding in expressions.  If you write something like
\code{a = 2+3}, the code generator will do the arithmetic and produce
code corresponding to \code{a = 5}.  (Proposed and implemented 
by Raymond Hettinger.)

\item Function calls are now faster because code objects now keep 
the most recently finished frame (a ``zombie frame'') in an internal
field of the code object, reusing it the next time the code object is
invoked.  (Original patch by Michael Hudson, modified by Armin Rigo
and Richard Jones; committed at the NeedForSpeed sprint.)
% Patch 876206

Frame objects are also slightly smaller, which may improve cache locality
and reduce memory usage a bit.  (Contributed by Neal Norwitz.)
% Patch 1337051

\item Python's built-in exceptions are now new-style classes, a change
that speeds up instantiation considerably.  Exception handling in
Python 2.5 is therefore about 30\% faster than in 2.4.
(Contributed by Richard Jones, Georg Brandl and Sean Reifschneider at
the NeedForSpeed sprint.)

\item Importing now caches the paths tried, recording whether 
they exist or not so that the interpreter makes fewer 
\cfunction{open()} and \cfunction{stat()} calls on startup.
(Contributed by Martin von~L\"owis and Georg Brandl.)
% Patch 921466

\end{itemize}


%======================================================================
\section{New, Improved, and Removed Modules\label{modules}}

The standard library received many enhancements and bug fixes in
Python 2.5.  Here's a partial list of the most notable changes, sorted
alphabetically by module name. Consult the \file{Misc/NEWS} file in
the source tree for a more complete list of changes, or look through
the SVN logs for all the details.

\begin{itemize}

\item The \module{audioop} module now supports the a-LAW encoding,
and the code for u-LAW encoding has been improved.  (Contributed by
Lars Immisch.)

\item The \module{codecs} module gained support for incremental
codecs.  The \function{codec.lookup()} function now
returns a \class{CodecInfo} instance instead of a tuple.
\class{CodecInfo} instances behave like a 4-tuple to preserve backward
compatibility but also have the attributes \member{encode},
\member{decode}, \member{incrementalencoder}, \member{incrementaldecoder},
\member{streamwriter}, and \member{streamreader}.  Incremental codecs 
can receive input and produce output in multiple chunks; the output is
the same as if the entire input was fed to the non-incremental codec.
See the \module{codecs} module documentation for details.
(Designed and implemented by Walter D\"orwald.)
% Patch  1436130

\item The \module{collections} module gained a new type,
\class{defaultdict}, that subclasses the standard \class{dict}
type.  The new type mostly behaves like a dictionary but constructs a
default value when a key isn't present, automatically adding it to the
dictionary for the requested key value.

The first argument to \class{defaultdict}'s constructor is a factory
function that gets called whenever a key is requested but not found.
This factory function receives no arguments, so you can use built-in
type constructors such as \function{list()} or \function{int()}.  For
example, 
you can make an index of words based on their initial letter like this:

\begin{verbatim}
words = """Nel mezzo del cammin di nostra vita
mi ritrovai per una selva oscura
che la diritta via era smarrita""".lower().split()

index = defaultdict(list)

for w in words:
    init_letter = w[0]
    index[init_letter].append(w)
\end{verbatim}

Printing \code{index} results in the following output:

\begin{verbatim}
defaultdict(<type 'list'>, {'c': ['cammin', 'che'], 'e': ['era'], 
        'd': ['del', 'di', 'diritta'], 'm': ['mezzo', 'mi'], 
        'l': ['la'], 'o': ['oscura'], 'n': ['nel', 'nostra'], 
        'p': ['per'], 's': ['selva', 'smarrita'], 
        'r': ['ritrovai'], 'u': ['una'], 'v': ['vita', 'via']}
\end{verbatim}

(Contributed by Guido van~Rossum.)

\item The \class{deque} double-ended queue type supplied by the
\module{collections} module now has a \method{remove(\var{value})}
method that removes the first occurrence of \var{value} in the queue,
raising \exception{ValueError} if the value isn't found.
(Contributed by Raymond Hettinger.)

\item New module: The \module{contextlib} module contains helper functions for use 
with the new '\keyword{with}' statement.  See
section~\ref{module-contextlib} for more about this module.

\item New module: The \module{cProfile} module is a C implementation of 
the existing \module{profile} module that has much lower overhead.
The module's interface is the same as \module{profile}: you run
\code{cProfile.run('main()')} to profile a function, can save profile
data to a file, etc.  It's not yet known if the Hotshot profiler,
which is also written in C but doesn't match the \module{profile}
module's interface, will continue to be maintained in future versions
of Python.  (Contributed by Armin Rigo.)

Also, the \module{pstats} module for analyzing the data measured by
the profiler now supports directing the output to any file object
by supplying a \var{stream} argument to the \class{Stats} constructor.
(Contributed by Skip Montanaro.)

\item The \module{csv} module, which parses files in
comma-separated value format, received several enhancements and a
number of bugfixes.  You can now set the maximum size in bytes of a
field by calling the \method{csv.field_size_limit(\var{new_limit})}
function; omitting the \var{new_limit} argument will return the
currently-set limit.  The \class{reader} class now has a
\member{line_num} attribute that counts the number of physical lines
read from the source; records can span multiple physical lines, so
\member{line_num} is not the same as the number of records read.

The CSV parser is now stricter about multi-line quoted
fields. Previously, if a line ended within a quoted field without a
terminating newline character, a newline would be inserted into the
returned field. This behavior caused problems when reading files that
contained carriage return characters within fields, so the code was
changed to return the field without inserting newlines. As a
consequence, if newlines embedded within fields are important, the
input should be split into lines in a manner that preserves the
newline characters.

(Contributed by Skip Montanaro and Andrew McNamara.)

\item The \class{datetime} class in the \module{datetime} 
module now has a \method{strptime(\var{string}, \var{format})} 
method for parsing date strings, contributed by Josh Spoerri.
It uses the same format characters as \function{time.strptime()} and
\function{time.strftime()}:

\begin{verbatim}
from datetime import datetime

ts = datetime.strptime('10:13:15 2006-03-07',
                       '%H:%M:%S %Y-%m-%d')
\end{verbatim}

\item The \method{SequenceMatcher.get_matching_blocks()} method
in the \module{difflib} module now guarantees to return a minimal list
of blocks describing matching subsequences.  Previously, the algorithm would
occasionally break a block of matching elements into two list entries.
(Enhancement by Tim Peters.)

\item The \module{doctest} module gained a \code{SKIP} option that
keeps an example from being executed at all.  This is intended for
code snippets that are usage examples intended for the reader and
aren't actually test cases.

An \var{encoding} parameter was added to the \function{testfile()}
function and the \class{DocFileSuite} class to specify the file's
encoding.  This makes it easier to use non-ASCII characters in 
tests contained within a docstring.  (Contributed by Bjorn Tillenius.)
% Patch 1080727

\item The \module{email} package has been updated to version 4.0.
% XXX need to provide some more detail here
(Contributed by Barry Warsaw.)

\item The \module{fileinput} module was made more flexible.
Unicode filenames are now supported, and a \var{mode} parameter that
defaults to \code{"r"} was added to the
\function{input()} function to allow opening files in binary or
universal-newline mode.  Another new parameter, \var{openhook},
lets you use a function other than \function{open()} 
to open the input files.  Once you're iterating over 
the set of files, the \class{FileInput} object's new
\method{fileno()} returns the file descriptor for the currently opened file.
(Contributed by Georg Brandl.)

\item In the \module{gc} module, the new \function{get_count()} function
returns a 3-tuple containing the current collection counts for the
three GC generations.  This is accounting information for the garbage
collector; when these counts reach a specified threshold, a garbage
collection sweep will be made.  The existing \function{gc.collect()}
function now takes an optional \var{generation} argument of 0, 1, or 2
to specify which generation to collect.
(Contributed by Barry Warsaw.)

\item The \function{nsmallest()} and 
\function{nlargest()} functions in the \module{heapq} module 
now support a \code{key} keyword parameter similar to the one
provided by the \function{min()}/\function{max()} functions
and the \method{sort()} methods.  For example:

\begin{verbatim}
>>> import heapq
>>> L = ["short", 'medium', 'longest', 'longer still']
>>> heapq.nsmallest(2, L)  # Return two lowest elements, lexicographically
['longer still', 'longest']
>>> heapq.nsmallest(2, L, key=len)   # Return two shortest elements
['short', 'medium']
\end{verbatim}

(Contributed by Raymond Hettinger.)

\item The \function{itertools.islice()} function now accepts
\code{None} for the start and step arguments.  This makes it more
compatible with the attributes of slice objects, so that you can now write
the following:

\begin{verbatim}
s = slice(5)     # Create slice object
itertools.islice(iterable, s.start, s.stop, s.step)
\end{verbatim}

(Contributed by Raymond Hettinger.)

\item The \function{format()} function in the \module{locale} module
has been modified and two new functions were added,
\function{format_string()} and \function{currency()}.

The \function{format()} function's \var{val} parameter could
previously be a string as long as no more than one \%char specifier
appeared; now the parameter must be exactly one \%char specifier with
no surrounding text.  An optional \var{monetary} parameter was also
added which, if \code{True}, will use the locale's rules for
formatting currency in placing a separator between groups of three
digits.

To format strings with multiple \%char specifiers, use the new
\function{format_string()} function that works like \function{format()}
but also supports mixing \%char specifiers with
arbitrary text.

A new \function{currency()} function was also added that formats a
number according to the current locale's settings.

(Contributed by Georg Brandl.)
% Patch 1180296

\item The \module{mailbox} module underwent a massive rewrite to add
the capability to modify mailboxes in addition to reading them.  A new
set of classes that include \class{mbox}, \class{MH}, and
\class{Maildir} are used to read mailboxes, and have an
\method{add(\var{message})} method to add messages,
\method{remove(\var{key})} to remove messages, and
\method{lock()}/\method{unlock()} to lock/unlock the mailbox.  The
following example converts a maildir-format mailbox into an mbox-format one:

\begin{verbatim}
import mailbox

# 'factory=None' uses email.Message.Message as the class representing
# individual messages.
src = mailbox.Maildir('maildir', factory=None)
dest = mailbox.mbox('/tmp/mbox')

for msg in src:
    dest.add(msg)
\end{verbatim}

(Contributed by Gregory K. Johnson.  Funding was provided by Google's
2005 Summer of Code.)

\item New module: the \module{msilib} module allows creating
Microsoft Installer \file{.msi} files and CAB files.  Some support
for reading the \file{.msi} database is also included.
(Contributed by Martin von~L\"owis.)

\item The \module{nis} module now supports accessing domains other
than the system default domain by supplying a \var{domain} argument to
the \function{nis.match()} and \function{nis.maps()} functions.
(Contributed by Ben Bell.)

\item The \module{operator} module's \function{itemgetter()} 
and \function{attrgetter()} functions now support multiple fields.  
A call such as \code{operator.attrgetter('a', 'b')}
will return a function 
that retrieves the \member{a} and \member{b} attributes.  Combining 
this new feature with the \method{sort()} method's \code{key} parameter 
lets you easily sort lists using multiple fields.
(Contributed by Raymond Hettinger.)

\item The \module{optparse} module was updated to version 1.5.1 of the
Optik library.  The \class{OptionParser} class gained an
\member{epilog} attribute, a string that will be printed after the
help message, and a \method{destroy()} method to break reference
cycles created by the object. (Contributed by Greg Ward.)

\item The \module{os} module underwent several changes.  The
\member{stat_float_times} variable now defaults to true, meaning that
\function{os.stat()} will now return time values as floats.  (This
doesn't necessarily mean that \function{os.stat()} will return times
that are precise to fractions of a second; not all systems support
such precision.)

Constants named \member{os.SEEK_SET}, \member{os.SEEK_CUR}, and
\member{os.SEEK_END} have been added; these are the parameters to the
\function{os.lseek()} function.  Two new constants for locking are
\member{os.O_SHLOCK} and \member{os.O_EXLOCK}.

Two new functions, \function{wait3()} and \function{wait4()}, were
added.  They're similar the \function{waitpid()} function which waits
for a child process to exit and returns a tuple of the process ID and
its exit status, but \function{wait3()} and \function{wait4()} return
additional information.  \function{wait3()} doesn't take a process ID
as input, so it waits for any child process to exit and returns a
3-tuple of \var{process-id}, \var{exit-status}, \var{resource-usage}
as returned from the \function{resource.getrusage()} function.
\function{wait4(\var{pid})} does take a process ID.
(Contributed by Chad J. Schroeder.)

On FreeBSD, the \function{os.stat()} function now returns 
times with nanosecond resolution, and the returned object
now has \member{st_gen} and \member{st_birthtime}.
The \member{st_flags} member is also available, if the platform supports it.
(Contributed by Antti Louko and  Diego Petten\`o.)
% (Patch 1180695, 1212117)

\item The Python debugger provided by the \module{pdb} module
can now store lists of commands to execute when a breakpoint is
reached and execution stops.  Once breakpoint \#1 has been created,
enter \samp{commands 1} and enter a series of commands to be executed,
finishing the list with \samp{end}.  The command list can include
commands that resume execution, such as \samp{continue} or
\samp{next}.  (Contributed by Gr\'egoire Dooms.)
% Patch 790710

\item The \module{pickle} and \module{cPickle} modules no
longer accept a return value of \code{None} from the
\method{__reduce__()} method; the method must return a tuple of
arguments instead.  The ability to return \code{None} was deprecated
in Python 2.4, so this completes the removal of the feature.

\item The \module{pkgutil} module, containing various utility
functions for finding packages, was enhanced to support PEP 302's
import hooks and now also works for packages stored in ZIP-format archives.
(Contributed by Phillip J. Eby.)

\item The pybench benchmark suite by Marc-Andr\'e~Lemburg is now
included in the \file{Tools/pybench} directory.  The pybench suite is
an improvement on the commonly used \file{pystone.py} program because
pybench provides a more detailed measurement of the interpreter's
speed.  It times particular operations such as function calls,
tuple slicing, method lookups, and numeric operations, instead of
performing many different operations and reducing the result to a
single number as \file{pystone.py} does.

\item The \module{pyexpat} module now uses version 2.0 of the Expat parser.
(Contributed by Trent Mick.)

\item The old \module{regex} and \module{regsub} modules, which have been 
deprecated ever since Python 2.0, have finally been deleted.  
Other deleted modules: \module{statcache}, \module{tzparse},
\module{whrandom}.

\item Also deleted: the \file{lib-old} directory,
which includes ancient modules such as \module{dircmp} and
\module{ni}, was removed.  \file{lib-old} wasn't on the default
\code{sys.path}, so unless your programs explicitly added the directory to 
\code{sys.path}, this removal shouldn't affect your code.

\item The \module{rlcompleter} module is no longer 
dependent on importing the \module{readline} module and
therefore now works on non-{\UNIX} platforms.
(Patch from Robert Kiendl.)
% Patch #1472854

\item The \module{SimpleXMLRPCServer} and \module{DocXMLRPCServer} 
classes now have a \member{rpc_paths} attribute that constrains
XML-RPC operations to a limited set of URL paths; the default is
to allow only \code{'/'} and \code{'/RPC2'}.  Setting 
\member{rpc_paths} to \code{None} or an empty tuple disables 
this path checking.
% Bug #1473048

\item The \module{socket} module now supports \constant{AF_NETLINK}
sockets on Linux, thanks to a patch from Philippe Biondi.  
Netlink sockets are a Linux-specific mechanism for communications
between a user-space process and kernel code; an introductory 
article about them is at \url{http://www.linuxjournal.com/article/7356}.
In Python code, netlink addresses are represented as a tuple of 2 integers, 
\code{(\var{pid}, \var{group_mask})}.

Two new methods on socket objects, \method{recv_buf(\var{buffer})} and
\method{recvfrom_buf(\var{buffer})}, store the received data in an object 
that supports the buffer protocol instead of returning the data as a
string.  This means you can put the data directly into an array or a
memory-mapped file.

Socket objects also gained \method{getfamily()}, \method{gettype()},
and \method{getproto()} accessor methods to retrieve the family, type,
and protocol values for the socket.

\item New module: the \module{spwd} module provides functions for
accessing the shadow password database on systems that support 
shadow passwords.

\item The \module{struct} is now faster because it 
compiles format strings into \class{Struct} objects
with \method{pack()} and \method{unpack()} methods.  This is similar
to how the \module{re} module lets you create compiled regular
expression objects.  You can still use the module-level 
\function{pack()} and \function{unpack()} functions; they'll create 
\class{Struct} objects and cache them.  Or you can use 
\class{Struct} instances directly:

\begin{verbatim}
s = struct.Struct('ih3s')

data = s.pack(1972, 187, 'abc')
year, number, name = s.unpack(data)
\end{verbatim}

You can also pack and unpack data to and from buffer objects directly
using the \method{pack_into(\var{buffer}, \var{offset}, \var{v1},
\var{v2}, ...)} and \method{unpack_from(\var{buffer}, \var{offset})}
methods.  This lets you store data directly into an array or a
memory-mapped file.

(\class{Struct} objects were implemented by Bob Ippolito at the
NeedForSpeed sprint.  Support for buffer objects was added by Martin
Blais, also at the NeedForSpeed sprint.)

\item The Python developers switched from CVS to Subversion during the 2.5
development process.  Information about the exact build version is
available as the \code{sys.subversion} variable, a 3-tuple of
\code{(\var{interpreter-name}, \var{branch-name},
\var{revision-range})}.  For example, at the time of writing my copy
of 2.5 was reporting \code{('CPython', 'trunk', '45313:45315')}.

This information is also available to C extensions via the 
\cfunction{Py_GetBuildInfo()} function that returns a 
string of build information like this:
\code{"trunk:45355:45356M, Apr 13 2006, 07:42:19"}.  
(Contributed by Barry Warsaw.)

\item Another new function, \function{sys._current_frames()}, returns
the current stack frames for all running threads as a dictionary
mapping thread identifiers to the topmost stack frame currently active
in that thread at the time the function is called.  (Contributed by
Tim Peters.)

\item The \class{TarFile} class in the \module{tarfile} module now has
an \method{extractall()} method that extracts all members from the
archive into the current working directory.  It's also possible to set
a different directory as the extraction target, and to unpack only a
subset of the archive's members.

The compression used for a tarfile opened in stream mode can now be
autodetected using the mode \code{'r|*'}.
% patch 918101
(Contributed by Lars Gust\"abel.)

\item The \module{threading} module now lets you set the stack size
used when new threads are created. The
\function{stack_size(\optional{\var{size}})} function returns the
currently configured stack size, and supplying the optional \var{size}
parameter sets a new value.  Not all platforms support changing the
stack size, but Windows, POSIX threading, and OS/2 all do.
(Contributed by Andrew MacIntyre.)
% Patch 1454481

\item The \module{unicodedata} module has been updated to use version 4.1.0
of the Unicode character database.  Version 3.2.0 is required 
by some specifications, so it's still available as 
\member{unicodedata.ucd_3_2_0}.

\item New module: the  \module{uuid} module generates 
universally unique identifiers (UUIDs) according to \rfc{4122}.  The
RFC defines several different UUID versions that are generated from a
starting string, from system properties, or purely randomly.  This
module contains a \class{UUID} class and 
functions named \function{uuid1()},
\function{uuid3()}, \function{uuid4()},  and 
\function{uuid5()} to generate different versions of UUID.  (Version 2 UUIDs 
are not specified in \rfc{4122} and are not supported by this module.)

\begin{verbatim}
>>> import uuid
>>> # make a UUID based on the host ID and current time
>>> uuid.uuid1()
UUID('a8098c1a-f86e-11da-bd1a-00112444be1e')

>>> # make a UUID using an MD5 hash of a namespace UUID and a name
>>> uuid.uuid3(uuid.NAMESPACE_DNS, 'python.org')
UUID('6fa459ea-ee8a-3ca4-894e-db77e160355e')

>>> # make a random UUID
>>> uuid.uuid4()
UUID('16fd2706-8baf-433b-82eb-8c7fada847da')

>>> # make a UUID using a SHA-1 hash of a namespace UUID and a name
>>> uuid.uuid5(uuid.NAMESPACE_DNS, 'python.org')
UUID('886313e1-3b8a-5372-9b90-0c9aee199e5d')
\end{verbatim}

(Contributed by Ka-Ping Yee.)

\item The \module{weakref} module's \class{WeakKeyDictionary} and
\class{WeakValueDictionary} types gained new methods for iterating
over the weak references contained in the dictionary. 
\method{iterkeyrefs()} and \method{keyrefs()} methods were
added to \class{WeakKeyDictionary}, and
\method{itervaluerefs()} and \method{valuerefs()} were added to
\class{WeakValueDictionary}.  (Contributed by Fred L.~Drake, Jr.)

\item The \module{webbrowser} module received a number of
enhancements.
It's now usable as a script with \code{python -m webbrowser}, taking a
URL as the argument; there are a number of switches 
to control the behaviour (\programopt{-n} for a new browser window, 
\programopt{-t} for a new tab).  New module-level functions,
\function{open_new()} and \function{open_new_tab()}, were added 
to support this.  The module's \function{open()} function supports an
additional feature, an \var{autoraise} parameter that signals whether
to raise the open window when possible. A number of additional
browsers were added to the supported list such as Firefox, Opera,
Konqueror, and elinks.  (Contributed by Oleg Broytmann and Georg
Brandl.)
% Patch #754022

\item The \module{xmlrpclib} module now supports returning 
      \class{datetime} objects for the XML-RPC date type.  Supply 
      \code{use_datetime=True} to the \function{loads()} function
      or the \class{Unmarshaller} class to enable this feature.
      (Contributed by Skip Montanaro.)
% Patch 1120353

\item The \module{zipfile} module now supports the ZIP64 version of the 
format, meaning that a .zip archive can now be larger than 4~GiB and
can contain individual files larger than 4~GiB.  (Contributed by
Ronald Oussoren.)
% Patch 1446489

\item The \module{zlib} module's \class{Compress} and \class{Decompress}
objects now support a \method{copy()} method that makes a copy of the 
object's internal state and returns a new 
\class{Compress} or \class{Decompress} object. 
(Contributed by Chris AtLee.)
% Patch 1435422

\end{itemize}



%======================================================================
\subsection{The ctypes package\label{module-ctypes}}

The \module{ctypes} package, written by Thomas Heller, has been added 
to the standard library.  \module{ctypes} lets you call arbitrary functions 
in shared libraries or DLLs.  Long-time users may remember the \module{dl} module, which 
provides functions for loading shared libraries and calling functions in them.  The \module{ctypes} package is much fancier.

To load a shared library or DLL, you must create an instance of the 
\class{CDLL} class and provide the name or path of the shared library
or DLL.  Once that's done, you can call arbitrary functions
by accessing them as attributes of the \class{CDLL} object.  

\begin{verbatim}
import ctypes

libc = ctypes.CDLL('libc.so.6')
result = libc.printf("Line of output\n")
\end{verbatim}

Type constructors for the various C types are provided: \function{c_int},
\function{c_float}, \function{c_double}, \function{c_char_p} (equivalent to \ctype{char *}), and so forth.  Unlike Python's types, the C versions are all mutable; you can assign to their \member{value} attribute
to change the wrapped value.  Python integers and strings will be automatically
converted to the corresponding C types, but for other types you 
must call the correct type constructor.  (And I mean \emph{must}; 
getting it wrong will often result in the interpreter crashing 
with a segmentation fault.)

You shouldn't use \function{c_char_p} with a Python string when the C function will be modifying the memory area, because Python strings are 
supposed to be immutable; breaking this rule will cause puzzling bugs.  When you need a modifiable memory area,
use \function{create_string_buffer()}:

\begin{verbatim}
s = "this is a string"
buf = ctypes.create_string_buffer(s)
libc.strfry(buf)
\end{verbatim}

C functions are assumed to return integers, but you can set
the \member{restype} attribute of the function object to 
change this:

\begin{verbatim}
>>> libc.atof('2.71828')
-1783957616
>>> libc.atof.restype = ctypes.c_double
>>> libc.atof('2.71828')
2.71828
\end{verbatim}

\module{ctypes} also provides a wrapper for Python's C API 
as the \code{ctypes.pythonapi} object.  This object does \emph{not} 
release the global interpreter lock before calling a function, because the lock must be held when calling into the interpreter's code.  
There's a \class{py_object()} type constructor that will create a 
\ctype{PyObject *} pointer.  A simple usage:

\begin{verbatim}
import ctypes

d = {}
ctypes.pythonapi.PyObject_SetItem(ctypes.py_object(d),
          ctypes.py_object("abc"),  ctypes.py_object(1))
# d is now {'abc', 1}.
\end{verbatim}

Don't forget to use \class{py_object()}; if it's omitted you end 
up with a segmentation fault.

\module{ctypes} has been around for a while, but people still write 
and distribution hand-coded extension modules because you can't rely on \module{ctypes} being present.
Perhaps developers will begin to write 
Python wrappers atop a library accessed through \module{ctypes} instead
of extension modules, now that \module{ctypes} is included with core Python.

\begin{seealso}

\seeurl{http://starship.python.net/crew/theller/ctypes/}
{The ctypes web page, with a tutorial, reference, and FAQ.}

\seeurl{../lib/module-ctypes.html}{The documentation 
for the \module{ctypes} module.}

\end{seealso}


%======================================================================
\subsection{The ElementTree package\label{module-etree}}

A subset of Fredrik Lundh's ElementTree library for processing XML has
been added to the standard library as \module{xml.etree}.  The
available modules are
\module{ElementTree}, \module{ElementPath}, and
\module{ElementInclude} from ElementTree 1.2.6.   
The \module{cElementTree} accelerator module is also included. 

The rest of this section will provide a brief overview of using
ElementTree.  Full documentation for ElementTree is available at
\url{http://effbot.org/zone/element-index.htm}.

ElementTree represents an XML document as a tree of element nodes.
The text content of the document is stored as the \member{.text}
and \member{.tail} attributes of 
(This is one of the major differences between ElementTree and 
the Document Object Model; in the DOM there are many different
types of node, including \class{TextNode}.)

The most commonly used parsing function is \function{parse()}, that
takes either a string (assumed to contain a filename) or a file-like
object and returns an \class{ElementTree} instance:

\begin{verbatim}
from xml.etree import ElementTree as ET

tree = ET.parse('ex-1.xml')

feed = urllib.urlopen(
          'http://planet.python.org/rss10.xml')
tree = ET.parse(feed)
\end{verbatim}

Once you have an \class{ElementTree} instance, you
can call its \method{getroot()} method to get the root \class{Element} node.

There's also an \function{XML()} function that takes a string literal
and returns an \class{Element} node (not an \class{ElementTree}).  
This function provides a tidy way to incorporate XML fragments,
approaching the convenience of an XML literal:

\begin{verbatim}
svg = ET.XML("""<svg width="10px" version="1.0">
             </svg>""")
svg.set('height', '320px')
svg.append(elem1)
\end{verbatim}

Each XML element supports some dictionary-like and some list-like
access methods.  Dictionary-like operations are used to access attribute
values, and list-like operations are used to access child nodes.

\begin{tableii}{c|l}{code}{Operation}{Result}
  \lineii{elem[n]}{Returns n'th child element.}
  \lineii{elem[m:n]}{Returns list of m'th through n'th child elements.}
  \lineii{len(elem)}{Returns number of child elements.}
  \lineii{list(elem)}{Returns list of child elements.}
  \lineii{elem.append(elem2)}{Adds \var{elem2} as a child.}
  \lineii{elem.insert(index, elem2)}{Inserts \var{elem2} at the specified location.}
  \lineii{del elem[n]}{Deletes n'th child element.}
  \lineii{elem.keys()}{Returns list of attribute names.}
  \lineii{elem.get(name)}{Returns value of attribute \var{name}.}
  \lineii{elem.set(name, value)}{Sets new value for attribute \var{name}.}
  \lineii{elem.attrib}{Retrieves the dictionary containing attributes.}
  \lineii{del elem.attrib[name]}{Deletes attribute \var{name}.}
\end{tableii}

Comments and processing instructions are also represented as
\class{Element} nodes.  To check if a node is a comment or processing
instructions:

\begin{verbatim}
if elem.tag is ET.Comment:
    ...
elif elem.tag is ET.ProcessingInstruction:
    ...
\end{verbatim}

To generate XML output, you should call the
\method{ElementTree.write()} method.  Like \function{parse()},
it can take either a string or a file-like object:

\begin{verbatim}
# Encoding is US-ASCII
tree.write('output.xml')

# Encoding is UTF-8
f = open('output.xml', 'w')
tree.write(f, encoding='utf-8')
\end{verbatim}

(Caution: the default encoding used for output is ASCII.  For general
XML work, where an element's name may contain arbitrary Unicode
characters, ASCII isn't a very useful encoding because it will raise
an exception if an element's name contains any characters with values
greater than 127.  Therefore, it's best to specify a different
encoding such as UTF-8 that can handle any Unicode character.)

This section is only a partial description of the ElementTree interfaces.
Please read the package's official documentation for more details.

\begin{seealso}

\seeurl{http://effbot.org/zone/element-index.htm}
{Official documentation for ElementTree.}

\end{seealso}


%======================================================================
\subsection{The hashlib package\label{module-hashlib}}

A new \module{hashlib} module, written by Gregory P. Smith, 
has been added to replace the
\module{md5} and \module{sha} modules.  \module{hashlib} adds support
for additional secure hashes (SHA-224, SHA-256, SHA-384, and SHA-512).
When available, the module uses OpenSSL for fast platform optimized
implementations of algorithms.  

The old \module{md5} and \module{sha} modules still exist as wrappers
around hashlib to preserve backwards compatibility.  The new module's
interface is very close to that of the old modules, but not identical.
The most significant difference is that the constructor functions
for creating new hashing objects are named differently.

\begin{verbatim}
# Old versions
h = md5.md5()   
h = md5.new()   

# New version 
h = hashlib.md5()

# Old versions
h = sha.sha()   
h = sha.new()   

# New version 
h = hashlib.sha1()

# Hash that weren't previously available
h = hashlib.sha224()
h = hashlib.sha256()
h = hashlib.sha384()
h = hashlib.sha512()

# Alternative form
h = hashlib.new('md5')          # Provide algorithm as a string
\end{verbatim}

Once a hash object has been created, its methods are the same as before:
\method{update(\var{string})} hashes the specified string into the 
current digest state, \method{digest()} and \method{hexdigest()}
return the digest value as a binary string or a string of hex digits,
and \method{copy()} returns a new hashing object with the same digest state.

\begin{seealso}

\seeurl{../lib/module-hashlib.html}{The documentation 
for the \module{hashlib} module.}

\end{seealso}


%======================================================================
\subsection{The sqlite3 package\label{module-sqlite}}

The pysqlite module (\url{http://www.pysqlite.org}), a wrapper for the
SQLite embedded database, has been added to the standard library under
the package name \module{sqlite3}.  

SQLite is a C library that provides a SQL-language database that
stores data in disk files without requiring a separate server process.
pysqlite was written by Gerhard H\"aring and provides a SQL interface
compliant with the DB-API 2.0 specification described by
\pep{249}. This means that it should be possible to write the first
version of your applications using SQLite for data storage.  If
switching to a larger database such as PostgreSQL or Oracle is
later necessary, the switch should be relatively easy.

If you're compiling the Python source yourself, note that the source
tree doesn't include the SQLite code, only the wrapper module.
You'll need to have the SQLite libraries and headers installed before
compiling Python, and the build process will compile the module when
the necessary headers are available.

To use the module, you must first create a \class{Connection} object
that represents the database.  Here the data will be stored in the 
\file{/tmp/example} file:

\begin{verbatim}
conn = sqlite3.connect('/tmp/example')
\end{verbatim}

You can also supply the special name \samp{:memory:} to create
a database in RAM.

Once you have a \class{Connection}, you can create a \class{Cursor} 
object and call its \method{execute()} method to perform SQL commands:

\begin{verbatim}
c = conn.cursor()

# Create table
c.execute('''create table stocks
(date timestamp, trans varchar, symbol varchar,
 qty decimal, price decimal)''')

# Insert a row of data
c.execute("""insert into stocks
          values ('2006-01-05','BUY','RHAT',100,35.14)""")
\end{verbatim}    

Usually your SQL operations will need to use values from Python
variables.  You shouldn't assemble your query using Python's string
operations because doing so is insecure; it makes your program
vulnerable to an SQL injection attack.  

Instead, use the DB-API's parameter substitution.  Put \samp{?} as a
placeholder wherever you want to use a value, and then provide a tuple
of values as the second argument to the cursor's \method{execute()}
method.  (Other database modules may use a different placeholder,
such as \samp{\%s} or \samp{:1}.) For example:

\begin{verbatim}    
# Never do this -- insecure!
symbol = 'IBM'
c.execute("... where symbol = '%s'" % symbol)

# Do this instead
t = (symbol,)
c.execute('select * from stocks where symbol=?', t)

# Larger example
for t in (('2006-03-28', 'BUY', 'IBM', 1000, 45.00),
          ('2006-04-05', 'BUY', 'MSOFT', 1000, 72.00),
          ('2006-04-06', 'SELL', 'IBM', 500, 53.00),
         ):
    c.execute('insert into stocks values (?,?,?,?,?)', t)
\end{verbatim}

To retrieve data after executing a SELECT statement, you can either 
treat the cursor as an iterator, call the cursor's \method{fetchone()}
method to retrieve a single matching row, 
or call \method{fetchall()} to get a list of the matching rows.

This example uses the iterator form:

\begin{verbatim}
>>> c = conn.cursor()
>>> c.execute('select * from stocks order by price')
>>> for row in c:
...    print row
...
(u'2006-01-05', u'BUY', u'RHAT', 100, 35.140000000000001)
(u'2006-03-28', u'BUY', u'IBM', 1000, 45.0)
(u'2006-04-06', u'SELL', u'IBM', 500, 53.0)
(u'2006-04-05', u'BUY', u'MSOFT', 1000, 72.0)
>>>
\end{verbatim}

For more information about the SQL dialect supported by SQLite, see 
\url{http://www.sqlite.org}.

\begin{seealso}

\seeurl{http://www.pysqlite.org}
{The pysqlite web page.}

\seeurl{http://www.sqlite.org}
{The SQLite web page; the documentation describes the syntax and the
available data types for the supported SQL dialect.}

\seeurl{../lib/module-sqlite3.html}{The documentation 
for the \module{sqlite3} module.}

\seepep{249}{Database API Specification 2.0}{PEP written by
Marc-Andr\'e Lemburg.}

\end{seealso}


%======================================================================
\subsection{The wsgiref package\label{module-wsgiref}}

% XXX should this be in a PEP 333 section instead?

The Web Server Gateway Interface (WSGI) v1.0 defines a standard
interface between web servers and Python web applications and is
described in \pep{333}.  The \module{wsgiref} package is a reference
implementation of the WSGI specification.

The package includes a basic HTTP server that will run a WSGI
application; this server is useful for debugging but isn't intended for 
production use.  Setting up a server takes only a few lines of code:

\begin{verbatim}
from wsgiref import simple_server

wsgi_app = ...

host = ''
port = 8000
httpd = simple_server.make_server(host, port, wsgi_app)
httpd.serve_forever()
\end{verbatim}

% XXX discuss structure of WSGI applications?  
% XXX provide an example using Django or some other framework?

\begin{seealso}

\seeurl{http://www.wsgi.org}{A central web site for WSGI-related resources.}

\seepep{333}{Python Web Server Gateway Interface v1.0}{PEP written by
Phillip J. Eby.}

\end{seealso}


% ======================================================================
\section{Build and C API Changes\label{build-api}}

Changes to Python's build process and to the C API include:

\begin{itemize}

\item The Python source tree was converted from CVS to Subversion, 
in a complex migration procedure that was supervised and flawlessly
carried out by Martin von~L\"owis.  The procedure was developed as
\pep{347}.

\item Coverity, a company that markets a source code analysis tool
called Prevent, provided the results of their examination of the Python
source code.  The analysis found about 60 bugs that 
were quickly fixed.  Many of the bugs were refcounting problems, often
occurring in error-handling code.  See
\url{http://scan.coverity.com} for the statistics.

\item The largest change to the C API came from \pep{353},
which modifies the interpreter to use a \ctype{Py_ssize_t} type
definition instead of \ctype{int}.  See the earlier
section~\ref{pep-353} for a discussion of this change.

\item The design of the bytecode compiler has changed a great deal, 
no longer generating bytecode by traversing the parse tree.  Instead
the parse tree is converted to an abstract syntax tree (or AST), and it is 
the abstract syntax tree that's traversed to produce the bytecode.

It's possible for Python code to obtain AST objects by using the 
\function{compile()} built-in and specifying \code{_ast.PyCF_ONLY_AST}
as the value of the 
\var{flags} parameter:

\begin{verbatim}
from _ast import PyCF_ONLY_AST
ast = compile("""a=0
for i in range(10):
    a += i
""", "<string>", 'exec', PyCF_ONLY_AST)

assignment = ast.body[0]
for_loop = ast.body[1]
\end{verbatim}

No official documentation has been written for the AST code yet, but
\pep{339} discusses the design.  To start learning about the code, read the
definition of the various AST nodes in \file{Parser/Python.asdl}.  A
Python script reads this file and generates a set of C structure
definitions in \file{Include/Python-ast.h}.  The
\cfunction{PyParser_ASTFromString()} and
\cfunction{PyParser_ASTFromFile()}, defined in
\file{Include/pythonrun.h}, take Python source as input and return the
root of an AST representing the contents.  This AST can then be turned
into a code object by \cfunction{PyAST_Compile()}.  For more
information, read the source code, and then ask questions on
python-dev.

% List of names taken from Jeremy's python-dev post at 
% http://mail.python.org/pipermail/python-dev/2005-October/057500.html
The AST code was developed under Jeremy Hylton's management, and
implemented by (in alphabetical order) Brett Cannon, Nick Coghlan,
Grant Edwards, John Ehresman, Kurt Kaiser, Neal Norwitz, Tim Peters,
Armin Rigo, and Neil Schemenauer, plus the participants in a number of
AST sprints at conferences such as PyCon.
 
\item Evan Jones's patch to obmalloc, first described in a talk
at PyCon DC 2005, was applied.  Python 2.4 allocated small objects in
256K-sized arenas, but never freed arenas.  With this patch, Python
will free arenas when they're empty.  The net effect is that on some
platforms, when you allocate many objects, Python's memory usage may
actually drop when you delete them and the memory may be returned to
the operating system.  (Implemented by Evan Jones, and reworked by Tim
Peters.)

Note that this change means extension modules must be more careful
when allocating memory.  Python's API has many different
functions for allocating memory that are grouped into families.  For
example, \cfunction{PyMem_Malloc()}, \cfunction{PyMem_Realloc()}, and
\cfunction{PyMem_Free()} are one family that allocates raw memory,
while \cfunction{PyObject_Malloc()}, \cfunction{PyObject_Realloc()},
and \cfunction{PyObject_Free()} are another family that's supposed to
be used for creating Python objects.  

Previously these different families all reduced to the platform's
\cfunction{malloc()} and \cfunction{free()} functions.  This meant 
it didn't matter if you got things wrong and allocated memory with the
\cfunction{PyMem} function but freed it with the \cfunction{PyObject}
function.  With 2.5's changes to obmalloc, these families now do different
things and mismatches will probably result in a segfault.  You should
carefully test your C extension modules with Python 2.5.

\item The built-in set types now have an official C API.  Call
\cfunction{PySet_New()} and \cfunction{PyFrozenSet_New()} to create a
new set, \cfunction{PySet_Add()} and \cfunction{PySet_Discard()} to
add and remove elements, and \cfunction{PySet_Contains} and
\cfunction{PySet_Size} to examine the set's state.
(Contributed by Raymond Hettinger.)

\item C code can now obtain information about the exact revision
of the Python interpreter by calling the 
\cfunction{Py_GetBuildInfo()} function that returns a 
string of build information like this:
\code{"trunk:45355:45356M, Apr 13 2006, 07:42:19"}.  
(Contributed by Barry Warsaw.)

\item Two new macros can be used to indicate C functions that are
local to the current file so that a faster calling convention can be
used.  \cfunction{Py_LOCAL(\var{type})} declares the function as
returning a value of the specified \var{type} and uses a fast-calling
qualifier. \cfunction{Py_LOCAL_INLINE(\var{type})} does the same thing
and also requests the function be inlined.  If
\cfunction{PY_LOCAL_AGGRESSIVE} is defined before \file{python.h} is
included, a set of more aggressive optimizations are enabled for the
module; you should benchmark the results to find out if these
optimizations actually make the code faster.  (Contributed by Fredrik
Lundh at the NeedForSpeed sprint.)

\item \cfunction{PyErr_NewException(\var{name}, \var{base},
\var{dict})} can now accept a tuple of base classes as its \var{base}
argument.  (Contributed by Georg Brandl.)

\item The \cfunction{PyErr_Warn()} function for issuing warnings
is now deprecated in favour of \cfunction{PyErr_WarnEx(category,
message, stacklevel)} which lets you specify the number of stack
frames separating this function and the caller.  A \var{stacklevel} of
1 is the function calling \cfunction{PyErr_WarnEx()}, 2 is the
function above that, and so forth.  (Added by Neal Norwitz.)

\item The CPython interpreter is still written in C, but 
the code can now be compiled with a {\Cpp} compiler without errors.  
(Implemented by Anthony Baxter, Martin von~L\"owis, Skip Montanaro.)

\item The \cfunction{PyRange_New()} function was removed.  It was
never documented, never used in the core code, and had dangerously lax
error checking.  In the unlikely case that your extensions were using
it, you can replace it by something like the following:
\begin{verbatim}
range = PyObject_CallFunction((PyObject*) &PyRange_Type, "lll", 
                              start, stop, step);
\end{verbatim}

\end{itemize}


%======================================================================
\subsection{Port-Specific Changes\label{ports}}

\begin{itemize}

\item MacOS X (10.3 and higher): dynamic loading of modules
now uses the \cfunction{dlopen()} function instead of MacOS-specific
functions.

\item MacOS X: a \longprogramopt{enable-universalsdk} switch was added
to the \program{configure} script that compiles the interpreter as a
universal binary able to run on both PowerPC and Intel processors.
(Contributed by Ronald Oussoren.)

\item Windows: \file{.dll} is no longer supported as a filename extension for 
extension modules.  \file{.pyd} is now the only filename extension that will
be searched for.

\end{itemize}


%======================================================================
\section{Porting to Python 2.5\label{porting}}

This section lists previously described changes that may require
changes to your code:

\begin{itemize}

\item ASCII is now the default encoding for modules.  It's now 
a syntax error if a module contains string literals with 8-bit
characters but doesn't have an encoding declaration.  In Python 2.4
this triggered a warning, not a syntax error.

\item Previously, the \member{gi_frame} attribute of a generator
was always a frame object.  Because of the \pep{342} changes
described in section~\ref{pep-342}, it's now possible
for \member{gi_frame} to be \code{None}.

\item A new warning, \class{UnicodeWarning}, is triggered when 
you attempt to compare a Unicode string and an 8-bit string that can't
be converted to Unicode using the default ASCII encoding.  Previously
such comparisons would raise a \class{UnicodeDecodeError} exception.

\item Library: the \module{csv} module is now stricter about multi-line quoted
fields.  If your files contain newlines embedded within fields, the
input should be split into lines in a manner which preserves the
newline characters.

\item Library: the \module{locale} module's 
\function{format()} function's would previously 
accept any string as long as no more than one \%char specifier
appeared.  In Python 2.5, the argument must be exactly one \%char
specifier with no surrounding text. 

\item Library: The \module{pickle} and \module{cPickle} modules no
longer accept a return value of \code{None} from the
\method{__reduce__()} method; the method must return a tuple of
arguments instead.  The modules also no longer accept the deprecated
\var{bin} keyword parameter.

\item Library: The \module{SimpleXMLRPCServer} and \module{DocXMLRPCServer} 
classes now have a \member{rpc_paths} attribute that constrains
XML-RPC operations to a limited set of URL paths; the default is
to allow only \code{'/'} and \code{'/RPC2'}.  Setting 
\member{rpc_paths} to \code{None} or an empty tuple disables 
this path checking.

\item C API: Many functions now use \ctype{Py_ssize_t} 
instead of \ctype{int} to allow processing more data on 64-bit
machines.  Extension code may need to make the same change to avoid
warnings and to support 64-bit machines.  See the earlier
section~\ref{pep-353} for a discussion of this change.

\item C API: 
The obmalloc changes mean that 
you must be careful to not mix usage 
of the \cfunction{PyMem_*()} and \cfunction{PyObject_*()}
families of functions. Memory allocated with 
one family's \cfunction{*_Malloc()} must be 
freed with the corresponding family's \cfunction{*_Free()} function.

\end{itemize}


%======================================================================
\section{Acknowledgements \label{acks}}

The author would like to thank the following people for offering
suggestions, corrections and assistance with various drafts of this
article: Georg Brandl, Nick Coghlan, Phillip J. Eby, Lars Gust\"abel,
Raymond Hettinger, Ralf W. Grosse-Kunstleve, Kent Johnson, Iain Lowe,
Martin von~L\"owis, Fredrik Lundh, Andrew McNamara, Skip Montanaro,
Gustavo Niemeyer, Paul Prescod, James Pryor, Mike Rovner, Scott
Weikart, Barry Warsaw, Thomas Wouters.

\end{document}