summaryrefslogtreecommitdiffstats
path: root/Doc/whatsnew/3.4.rst
blob: 65971700c8b583cfc31bbbea9c0cc78b11d3264f (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
1310
1311
1312
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359
1360
1361
1362
1363
1364
1365
1366
1367
1368
1369
1370
1371
1372
1373
1374
1375
1376
1377
1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484
1485
1486
1487
1488
1489
1490
1491
1492
1493
1494
1495
1496
1497
1498
1499
1500
1501
1502
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
1536
1537
1538
1539
1540
1541
1542
1543
1544
1545
1546
1547
1548
1549
1550
1551
1552
1553
1554
1555
1556
1557
1558
1559
1560
1561
1562
1563
1564
1565
1566
1567
1568
1569
1570
1571
1572
1573
1574
1575
1576
1577
1578
1579
1580
1581
1582
1583
1584
1585
1586
1587
1588
1589
1590
1591
1592
1593
1594
1595
1596
1597
1598
1599
1600
1601
1602
1603
1604
1605
1606
1607
1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643
1644
1645
1646
1647
1648
1649
1650
1651
1652
1653
1654
1655
1656
1657
1658
1659
1660
1661
1662
1663
1664
1665
1666
1667
1668
1669
1670
1671
1672
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687
1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
1719
1720
1721
1722
1723
1724
1725
1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
1958
1959
1960
1961
1962
1963
1964
1965
1966
1967
1968
1969
1970
1971
1972
1973
1974
1975
1976
1977
1978
1979
1980
1981
1982
1983
1984
1985
1986
1987
1988
1989
1990
1991
1992
1993
1994
1995
1996
1997
1998
1999
2000
2001
2002
2003
2004
2005
2006
2007
2008
2009
2010
2011
2012
2013
2014
2015
2016
2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
2032
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
2144
2145
2146
2147
2148
2149
2150
2151
2152
2153
2154
2155
2156
2157
2158
2159
2160
2161
2162
2163
2164
2165
2166
2167
2168
2169
2170
2171
2172
2173
2174
2175
2176
2177
2178
2179
2180
2181
2182
2183
2184
2185
2186
2187
2188
2189
2190
2191
2192
2193
2194
2195
2196
2197
2198
2199
2200
2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213
2214
2215
2216
2217
2218
2219
2220
2221
2222
2223
2224
2225
2226
2227
2228
2229
2230
2231
2232
2233
2234
2235
2236
2237
2238
2239
2240
2241
2242
2243
2244
2245
2246
2247
2248
2249
2250
2251
2252
2253
2254
2255
2256
2257
2258
2259
2260
2261
2262
2263
2264
2265
2266
2267
2268
2269
2270
2271
2272
2273
2274
2275
2276
2277
2278
2279
2280
2281
2282
2283
2284
2285
2286
2287
2288
2289
2290
2291
2292
2293
2294
2295
2296
2297
2298
2299
2300
2301
2302
2303
2304
2305
2306
2307
2308
2309
2310
2311
2312
2313
2314
2315
2316
2317
2318
2319
2320
2321
2322
2323
2324
2325
2326
2327
2328
2329
2330
2331
2332
2333
2334
2335
2336
2337
2338
2339
2340
2341
2342
2343
2344
2345
2346
2347
2348
2349
2350
2351
2352
2353
2354
2355
2356
2357
2358
2359
2360
2361
2362
2363
2364
2365
2366
2367
2368
2369
2370
2371
2372
2373
2374
2375
2376
2377
2378
2379
2380
2381
2382
2383
2384
2385
2386
2387
2388
2389
2390
2391
2392
2393
2394
2395
2396
2397
2398
2399
2400
2401
2402
2403
2404
2405
2406
2407
2408
2409
2410
2411
2412
2413
2414
2415
2416
2417
2418
2419
2420
2421
2422
2423
2424
2425
2426
2427
2428
2429
2430
2431
2432
2433
2434
2435
2436
2437
2438
2439
2440
2441
2442
2443
2444
2445
2446
2447
2448
2449
2450
2451
2452
2453
2454
2455
2456
2457
2458
2459
2460
2461
2462
2463
2464
2465
2466
2467
2468
2469
2470
2471
2472
2473
2474
2475
2476
2477
2478
2479
2480
2481
2482
2483
2484
2485
2486
2487
2488
2489
2490
2491
2492
2493
2494
2495
2496
2497
2498
2499
2500
2501
2502
2503
2504
2505
2506
2507
2508
2509
2510
2511
2512
2513
2514
2515
2516
2517
2518
2519
2520
2521
2522
2523
2524
2525
2526
2527
2528
2529
2530
2531
2532
2533
2534
2535
2536
2537
2538
2539
2540
2541
2542
2543
2544
2545
****************************
  What's New In Python 3.4
****************************

:Author: R. David Murray <rdmurray@bitdance.com> (Editor)

.. Rules for maintenance:

   * Anyone can add text to this document, but the maintainer reserves the
   right to rewrite any additions. In particular, for obscure or esoteric
   features, the maintainer may reduce any addition to a simple reference to
   the new documentation rather than explaining the feature inline.

   * While the maintainer will periodically go through Misc/NEWS
   and add changes, it's best not to rely on this. We know from experience
   that any changes that aren't in the What's New documentation around the
   time of the original release will remain largely unknown to the community
   for years, even if they're added later. We also know from experience that
   other priorities can arise, and the maintainer will run out of time to do
   updates -- in such cases, end users will be much better served by partial
   notifications that at least give a hint about new features to
   investigate.

   * This is not a complete list of every single change; completeness
   is the purpose of Misc/NEWS. The What's New should focus on changes that
   are visible to Python *users* and that *require* a feature release (i.e.
   most bug fixes should only be recorded in Misc/NEWS)

   * PEPs should not be marked Final until they have an entry in What's New.
   A placeholder entry that is just a section header and a link to the PEP
   (e.g ":pep:`397` has been implemented") is acceptable. If a PEP has been
   implemented and noted in What's New, don't forget to mark it as Final!

   * If you want to draw your new text to the attention of the
   maintainer, add 'XXX' to the beginning of the paragraph or
   section.

   * It's OK to add just a very brief note about a change.  For
   example: "The :ref:`~socket.transmogrify()` function was added to the
   :mod:`socket` module."  The maintainer will research the change and
   write the necessary text (if appropriate). The advantage of doing this
   is that even if no more descriptive text is ever added, readers will at
   least have a notification that the new feature exists and a link to the
   relevant documentation.

   * You can comment out your additions if you like, but it's not
   necessary (especially when a final release is some months away).

   * Credit the author of a patch or bugfix.   Just the name is
   sufficient; the e-mail address isn't necessary.

   * It's helpful to add the bug/patch number as a comment:

   The :ref:`~socket.transmogrify()` function was added to the
   :mod:`socket` module.  (Contributed by P.Y. Developer in :issue:`12345`.)

   This saves the maintainer the effort of going through the Mercurial log
   when researching a change.

   * Cross referencing tip: :ref:`mod.attr` will display as ``mod.attr``,
   while :ref:`~mod.attr` will display as ``attr``.

This article explains the new features in Python 3.4, compared to 3.3.
Python 3.4 was released on March 16, 2014.  For full details, see the
`changelog <https://docs.python.org/3.4/whatsnew/changelog.html>`_.


.. seealso::

   :pep:`429` -- Python 3.4 Release Schedule



Summary -- Release Highlights
=============================

.. This section singles out the most important changes in Python 3.4.
   Brevity is key.

New syntax features:

* No new syntax features were added in Python 3.4.

Other new features:

* :ref:`pip should always be available <whatsnew-pep-453>` (:pep:`453`).
* :ref:`Newly created file descriptors are non-inheritable <whatsnew-pep-446>`
  (:pep:`446`).
* command line option for :ref:`isolated mode <whatsnew-isolated-mode>`
  (:issue:`16499`).
* :ref:`improvements in the handling of codecs <codec-handling-improvements>`
  that are not text encodings (multiple issues).
* :ref:`A ModuleSpec Type <whatsnew-pep-451>` for the Import System
  (:pep:`451`).  (Affects importer authors.)
* The :mod:`marshal` format has been made :ref:`more compact and efficient
  <whatsnew-marshal-3>` (:issue:`16475`).

New library modules:

* :mod:`asyncio`: :ref:`New provisional API for asynchronous IO
  <whatsnew-asyncio>` (:pep:`3156`).
* :mod:`ensurepip`: :ref:`Bootstrapping the pip installer <whatsnew-ensurepip>`
  (:pep:`453`).
* :mod:`enum`: :ref:`Support for enumeration types <whatsnew-enum>`
  (:pep:`435`).
* :mod:`pathlib`: :ref:`Object-oriented filesystem paths <whatsnew-pathlib>`
  (:pep:`428`).
* :mod:`selectors`: :ref:`High-level and efficient I/O multiplexing
  <whatsnew-selectors>`, built upon the :mod:`select` module primitives (part
  of :pep:`3156`).
* :mod:`statistics`: A basic :ref:`numerically stable statistics library
  <whatsnew-statistics>` (:pep:`450`).
* :mod:`tracemalloc`: :ref:`Trace Python memory allocations
  <whatsnew-tracemalloc>` (:pep:`454`).

Significantly improved library modules:

* :ref:`Single-dispatch generic functions <whatsnew-singledispatch>` in
  :mod:`functools` (:pep:`443`).
* New :mod:`pickle` :ref:`protocol 4 <whatsnew-protocol-4>` (:pep:`3154`).
* :mod:`multiprocessing` now has :ref:`an option to avoid using os.fork
  on Unix <whatsnew-multiprocessing-no-fork>` (:issue:`8713`).
* :mod:`email` has a new submodule, :mod:`~email.contentmanager`, and
  a new :mod:`~email.message.Message` subclass
  (:class:`~email.contentmanager.EmailMessage`) that :ref:`simplify MIME
  handling <whatsnew_email_contentmanager>` (:issue:`18891`).
* The :mod:`inspect` and :mod:`pydoc` modules are now capable of
  correct introspection of a much wider variety of callable objects,
  which improves the output of the Python :func:`help` system.
* The :mod:`ipaddress` module API has been declared stable

Security improvements:

* :ref:`Secure and interchangeable hash algorithm <whatsnew-pep-456>`
  (:pep:`456`).
* :ref:`Make newly created file descriptors non-inheritable <whatsnew-pep-446>`
  (:pep:`446`) to avoid leaking file descriptors to child processes.
* New command line option for :ref:`isolated mode <whatsnew-isolated-mode>`,
  (:issue:`16499`).
* :mod:`multiprocessing` now has :ref:`an option to avoid using os.fork
  on Unix <whatsnew-multiprocessing-no-fork>`.  *spawn* and *forkserver* are
  more secure because they avoid sharing data with child processes.
* :mod:`multiprocessing` child processes on Windows no longer inherit
  all of the parent's inheritable handles, only the necessary ones.
* A new :func:`hashlib.pbkdf2_hmac` function provides
  the `PKCS#5 password-based key derivation function 2
  <https://en.wikipedia.org/wiki/PBKDF2>`_.
* :ref:`TLSv1.1 and TLSv1.2 support <whatsnew-tls-11-12>` for :mod:`ssl`.
* :ref:`Retrieving certificates from the Windows system cert store support
  <whatsnew34-win-cert-store>` for :mod:`ssl`.
* :ref:`Server-side SNI (Server Name Indication) support
  <whatsnew34-sni>` for :mod:`ssl`.
* The :class:`ssl.SSLContext` class has a :ref:`lot of improvements
  <whatsnew34-sslcontext>`.
* All modules in the standard library that support SSL now support server
  certificate verification, including hostname matching
  (:func:`ssl.match_hostname`) and CRLs (Certificate Revocation lists, see
  :func:`ssl.SSLContext.load_verify_locations`).

CPython implementation improvements:

* :ref:`Safe object finalization <whatsnew-pep-442>` (:pep:`442`).
* Leveraging :pep:`442`, in most cases :ref:`module globals are no longer set
  to None during finalization <whatsnew-pep-442>` (:issue:`18214`).
* :ref:`Configurable memory allocators <whatsnew-pep-445>` (:pep:`445`).
* :ref:`Argument Clinic <whatsnew-pep-436>` (:pep:`436`).

Please read on for a comprehensive list of user-facing changes, including many
other smaller improvements, CPython optimizations, deprecations, and potential
porting issues.



New Features
============

.. _whatsnew-pep-453:

PEP 453: Explicit Bootstrapping of PIP in Python Installations
--------------------------------------------------------------

Bootstrapping pip By Default
~~~~~~~~~~~~~~~~~~~~~~~~~~~~

The new :mod:`ensurepip` module (defined in :pep:`453`) provides a standard
cross-platform mechanism to bootstrap the pip installer into Python
installations and virtual environments. The version of ``pip`` included
with Python 3.4.0 is ``pip`` 1.5.4, and future 3.4.x maintenance releases
will update the bundled version to the latest version of ``pip`` that is
available at the time of creating the release candidate.

By default, the commands ``pipX`` and ``pipX.Y`` will be installed on all
platforms (where X.Y stands for the version of the Python installation),
along with the ``pip`` Python package and its dependencies. On Windows and
in virtual environments on all platforms, the unversioned ``pip`` command
will also be installed. On other platforms, the system wide unversioned
``pip`` command typically refers to the separately installed Python 2
version.

The ``pyvenv`` command line utility and the :mod:`venv`
module make use of the :mod:`ensurepip` module to make ``pip`` readily
available in virtual environments. When using the command line utility,
``pip`` is installed by default, while when using the :mod:`venv` module
:ref:`venv-api` installation of ``pip`` must be requested explicitly.

For CPython :ref:`source builds on POSIX systems <building-python-on-unix>`,
the ``make install`` and ``make altinstall`` commands bootstrap ``pip`` by
default.  This behaviour can be controlled through configure options, and
overridden through Makefile options.

On Windows and Mac OS X, the CPython installers now default to installing
``pip`` along with CPython itself (users may opt out of installing it
during the installation process). Window users will need to opt in to the
automatic ``PATH`` modifications to have ``pip`` available from the command
line by default, otherwise it can still be accessed through the Python
launcher for Windows as ``py -m pip``.

As `discussed in the PEP`__, platform packagers may choose not to install
these commands by default, as long as, when invoked, they provide clear and
simple directions on how to install them on that platform (usually using
the system package manager).

__ https://www.python.org/dev/peps/pep-0453/#recommendations-for-downstream-distributors

.. note::

   To avoid conflicts between parallel Python 2 and Python 3 installations,
   only the versioned ``pip3`` and ``pip3.4`` commands are bootstrapped by
   default when ``ensurepip`` is invoked directly - the ``--default-pip``
   option is needed to also request the unversioned ``pip`` command.
   ``pyvenv`` and the Windows installer ensure that the unqualified ``pip``
   command is made available in those environments, and ``pip`` can always be
   invoked via the ``-m`` switch rather than directly to avoid ambiguity on
   systems with multiple Python installations.


Documentation Changes
~~~~~~~~~~~~~~~~~~~~~

As part of this change, the :ref:`installing-index` and
:ref:`distributing-index` sections of the documentation have been
completely redesigned as short getting started and FAQ documents. Most
packaging documentation has now been moved out to the Python Packaging
Authority maintained `Python Packaging User Guide
<https://packaging.python.org>`__ and the documentation of the individual
projects.

However, as this migration is currently still incomplete, the legacy
versions of those guides remaining available as :ref:`install-index`
and :ref:`distutils-index`.

.. seealso::

   :pep:`453` -- Explicit bootstrapping of pip in Python installations
      PEP written by Donald Stufft and Nick Coghlan, implemented by
      Donald Stufft, Nick Coghlan, Martin von Löwis and Ned Deily.


.. _whatsnew-pep-446:

PEP 446: Newly Created File Descriptors Are Non-Inheritable
-----------------------------------------------------------

:pep:`446` makes newly created file descriptors :ref:`non-inheritable
<fd_inheritance>`.  In general, this is the behavior an application will
want: when launching a new process, having currently open files also
open in the new process can lead to all sorts of hard to find bugs,
and potentially to security issues.

However, there are occasions when inheritance is desired.  To support
these cases, the following new functions and methods are available:

* :func:`os.get_inheritable`, :func:`os.set_inheritable`
* :func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable`
* :meth:`socket.socket.get_inheritable`, :meth:`socket.socket.set_inheritable`

.. seealso::

   :pep:`446` -- Make newly created file descriptors non-inheritable
      PEP written and implemented by Victor Stinner.


.. _codec-handling-improvements:

Improvements to Codec Handling
------------------------------

Since it was first introduced, the :mod:`codecs` module has always been
intended to operate as a type-neutral dynamic encoding and decoding
system. However, its close coupling with the Python text model, especially
the type restricted convenience methods on the builtin :class:`str`,
:class:`bytes` and :class:`bytearray` types, has historically obscured that
fact.

As a key step in clarifying the situation, the :meth:`codecs.encode` and
:meth:`codecs.decode` convenience functions are now properly documented in
Python 2.7, 3.3 and 3.4. These functions have existed in the :mod:`codecs`
module (and have been covered by the regression test suite) since Python 2.4,
but were previously only discoverable through runtime introspection.

Unlike the convenience methods on :class:`str`, :class:`bytes` and
:class:`bytearray`, the :mod:`codecs` convenience functions support arbitrary
codecs in both Python 2 and Python 3, rather than being limited to Unicode text
encodings (in Python 3) or ``basestring`` <-> ``basestring`` conversions (in
Python 2).

In Python 3.4, the interpreter is able to identify the known non-text
encodings provided in the standard library and direct users towards these
general purpose convenience functions when appropriate::

    >>> b"abcdef".decode("hex")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    LookupError: 'hex' is not a text encoding; use codecs.decode() to handle arbitrary codecs

    >>> "hello".encode("rot13")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    LookupError: 'rot13' is not a text encoding; use codecs.encode() to handle arbitrary codecs

    >>> open("foo.txt", encoding="hex")
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    LookupError: 'hex' is not a text encoding; use codecs.open() to handle arbitrary codecs

In a related change, whenever it is feasible without breaking backwards
compatibility, exceptions raised during encoding and decoding operations
are wrapped in a chained exception of the same type that mentions the
name of the codec responsible for producing the error::

    >>> import codecs

    >>> codecs.decode(b"abcdefgh", "hex")
    Traceback (most recent call last):
      File "/usr/lib/python3.4/encodings/hex_codec.py", line 20, in hex_decode
        return (binascii.a2b_hex(input), len(input))
    binascii.Error: Non-hexadecimal digit found

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    binascii.Error: decoding with 'hex' codec failed (Error: Non-hexadecimal digit found)

    >>> codecs.encode("hello", "bz2")
    Traceback (most recent call last):
      File "/usr/lib/python3.4/encodings/bz2_codec.py", line 17, in bz2_encode
        return (bz2.compress(input), len(input))
      File "/usr/lib/python3.4/bz2.py", line 498, in compress
        return comp.compress(data) + comp.flush()
    TypeError: 'str' does not support the buffer interface

    The above exception was the direct cause of the following exception:

    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    TypeError: encoding with 'bz2' codec failed (TypeError: 'str' does not support the buffer interface)

Finally, as the examples above show, these improvements have permitted
the restoration of the convenience aliases for the non-Unicode codecs that
were themselves restored in Python 3.2. This means that encoding binary data
to and from its hexadecimal representation (for example) can now be written
as::

    >>> from codecs import encode, decode
    >>> encode(b"hello", "hex")
    b'68656c6c6f'
    >>> decode(b"68656c6c6f", "hex")
    b'hello'

The binary and text transforms provided in the standard library are detailed
in :ref:`binary-transforms` and :ref:`text-transforms`.

(Contributed by Nick Coghlan in :issue:`7475`, :issue:`17827`,
:issue:`17828` and :issue:`19619`.)


.. _whatsnew-pep-451:

PEP 451: A ModuleSpec Type for the Import System
------------------------------------------------

:pep:`451` provides an encapsulation of the information about a module that the
import machinery will use to load it (that is, a module specification).  This
helps simplify both the import implementation and several import-related APIs.
The change is also a stepping stone for `several future import-related
improvements`__.

__ https://mail.python.org/pipermail/python-dev/2013-November/130111.html

The public-facing changes from the PEP are entirely backward-compatible.
Furthermore, they should be transparent to everyone but importer authors.  Key
finder and loader methods have been deprecated, but they will continue working.
New importers should use the new methods described in the PEP.  Existing
importers should be updated to implement the new methods.  See the
:ref:`deprecated-3.4` section for a list of methods that should be replaced and
their replacements.


Other Language Changes
----------------------

Some smaller changes made to the core Python language are:

* Unicode database updated to UCD version 6.3.

* :func:`min` and :func:`max` now accept a *default* keyword-only argument that
  can be used to specify the value they return if the iterable they are
  evaluating has no elements.  (Contributed by Julian Berman in
  :issue:`18111`.)

* Module objects are now :mod:`weakref`'able.

* Module ``__file__`` attributes (and related values) should now always
  contain absolute paths by default, with the sole exception of
  ``__main__.__file__`` when a script has been executed directly using
  a relative path.  (Contributed by Brett Cannon in :issue:`18416`.)

* All the UTF-\* codecs (except UTF-7) now reject surrogates during both
  encoding and decoding unless the ``surrogatepass`` error handler is used,
  with the exception of the UTF-16 decoder (which accepts valid surrogate pairs)
  and the UTF-16 encoder (which produces them while encoding non-BMP characters).
  (Contributed by Victor Stinner, Kang-Hao (Kenny) Lu and Serhiy Storchaka in
  :issue:`12892`.)

* New German EBCDIC :ref:`codec <standard-encodings>` ``cp273``.  (Contributed
  by Michael Bierenfeld and Andrew Kuchling in :issue:`1097797`.)

* New Ukrainian :ref:`codec <standard-encodings>` ``cp1125``.  (Contributed by
  Serhiy Storchaka in :issue:`19668`.)

* :class:`bytes`.join() and :class:`bytearray`.join() now accept arbitrary
  buffer objects as arguments.  (Contributed by Antoine Pitrou in
  :issue:`15958`.)

* The :class:`int` constructor now accepts any object that has an ``__index__``
  method for its *base* argument.  (Contributed by Mark Dickinson in
  :issue:`16772`.)

* Frame objects now have a :func:`~frame.clear` method that clears all
  references to local variables from the frame.  (Contributed by Antoine Pitrou
  in :issue:`17934`.)

* :class:`memoryview` is now registered as a :class:`Sequence <collections.abc>`,
  and supports the :func:`reversed` builtin.  (Contributed by Nick Coghlan
  and Claudiu Popa in :issue:`18690` and :issue:`19078`.)

* Signatures reported by :func:`help` have been modified and improved in
  several cases as a result of the introduction of Argument Clinic and other
  changes to the :mod:`inspect` and :mod:`pydoc` modules.

* :meth:`~object.__length_hint__` is now part of the formal language
  specification (see :pep:`424`).  (Contributed by Armin Ronacher in
  :issue:`16148`.)


New Modules
===========


.. _whatsnew-asyncio:

asyncio
-------

The new :mod:`asyncio` module (defined in :pep:`3156`) provides a standard
pluggable event loop model for Python, providing solid asynchronous IO
support in the standard library, and making it easier for other event loop
implementations to interoperate with the standard library and each other.

For Python 3.4, this module is considered a :term:`provisional API`.

.. seealso::

   :pep:`3156` -- Asynchronous IO Support Rebooted: the "asyncio" Module
      PEP written and implementation led by Guido van Rossum.


.. _whatsnew-ensurepip:

ensurepip
---------

The new :mod:`ensurepip` module is the primary infrastructure for the
:pep:`453` implementation.  In the normal course of events end users will not
need to interact with this module, but it can be used to manually bootstrap
``pip`` if the automated bootstrapping into an installation or virtual
environment was declined.

:mod:`ensurepip` includes a bundled copy of ``pip``, up-to-date as of the first
release candidate of the release of CPython with which it ships (this applies
to both maintenance releases and feature releases).  ``ensurepip`` does not
access the internet.  If the installation has Internet access, after
``ensurepip`` is run the bundled ``pip`` can be used to upgrade ``pip`` to a
more recent release than the bundled one.  (Note that such an upgraded version
of ``pip`` is considered to be a separately installed package and will not be
removed if Python is uninstalled.)

The module is named *ensure*\ pip because if called when ``pip`` is already
installed, it does nothing.  It also has an ``--upgrade`` option that will
cause it to install the bundled copy of ``pip`` if the existing installed
version of ``pip`` is older than the bundled copy.


.. _whatsnew-enum:

enum
----

The new :mod:`enum` module (defined in :pep:`435`) provides a standard
implementation of enumeration types, allowing other modules (such as
:mod:`socket`) to provide more informative error messages and better
debugging support by replacing opaque integer constants with backwards
compatible enumeration values.

.. seealso::

   :pep:`435` -- Adding an Enum type to the Python standard library
      PEP written by Barry Warsaw, Eli Bendersky and Ethan Furman,
      implemented by Ethan Furman.


.. _whatsnew-pathlib:

pathlib
-------

The new :mod:`pathlib` module offers classes representing filesystem paths
with semantics appropriate for different operating systems.  Path classes are
divided between *pure paths*, which provide purely computational operations
without I/O, and *concrete paths*, which inherit from pure paths but also
provide I/O operations.

For Python 3.4, this module is considered a :term:`provisional API`.

.. seealso::

   :pep:`428` -- The pathlib module -- object-oriented filesystem paths
      PEP written and implemented by Antoine Pitrou.


.. _whatsnew-selectors:

selectors
---------

The new :mod:`selectors` module (created as part of implementing :pep:`3156`)
allows high-level and efficient I/O multiplexing, built upon the
:mod:`select` module primitives.


.. _whatsnew-statistics:

statistics
----------

The new :mod:`statistics` module (defined in :pep:`450`) offers some core
statistics functionality directly in the standard library. This module
supports calculation of the mean, median, mode, variance and standard
deviation of a data series.

.. seealso::

   :pep:`450` -- Adding A Statistics Module To The Standard Library
      PEP written and implemented by Steven D'Aprano

.. _whatsnew-tracemalloc:


tracemalloc
-----------

The new :mod:`tracemalloc` module (defined in :pep:`454`) is a debug tool to
trace memory blocks allocated by Python. It provides the following information:

* Trace where an object was allocated
* Statistics on allocated memory blocks per filename and per line number:
  total size, number and average size of allocated memory blocks
* Compute the differences between two snapshots to detect memory leaks

.. seealso::

   :pep:`454` -- Add a new tracemalloc module to trace Python memory allocations
      PEP written and implemented by Victor Stinner



Improved Modules
================


abc
---

New function :func:`abc.get_cache_token` can be used to know when to invalidate
caches that are affected by changes in the object graph.  (Contributed
by Łukasz Langa in :issue:`16832`.)

New class :class:`~abc.ABC` has :class:`~abc.ABCMeta` as its meta class.
Using ``ABC`` as a base class has essentially the same effect as specifying
``metaclass=abc.ABCMeta``, but is simpler to type and easier to read.
(Contributed by Bruno Dupuis in :issue:`16049`.)


aifc
----

The :meth:`~aifc.aifc.getparams` method now returns a namedtuple rather than a
plain tuple.  (Contributed by Claudiu Popa in :issue:`17818`.)

:func:`aifc.open` now supports the context management protocol: when used in a
:keyword:`with` block, the :meth:`~aifc.aifc.close` method of the returned
object will be called automatically at the end of the block.  (Contributed by
Serhiy Storchacha in :issue:`16486`.)

The :meth:`~aifc.aifc.writeframesraw` and :meth:`~aifc.aifc.writeframes`
methods now accept any :term:`bytes-like object`.  (Contributed by Serhiy
Storchaka in :issue:`8311`.)


argparse
--------

The :class:`~argparse.FileType` class now accepts *encoding* and
*errors* arguments, which are passed through to :func:`open`.  (Contributed
by Lucas Maystre in :issue:`11175`.)


audioop
-------

:mod:`audioop` now supports 24-bit samples.  (Contributed by Serhiy Storchaka
in :issue:`12866`.)

New :func:`~audioop.byteswap` function converts big-endian samples to
little-endian and vice versa.  (Contributed by Serhiy Storchaka in
:issue:`19641`.)

All :mod:`audioop` functions now accept any :term:`bytes-like object`.  Strings
are not accepted: they didn't work before, now they raise an error right away.
(Contributed by Serhiy Storchaka in :issue:`16685`.)


base64
------

The encoding and decoding functions in :mod:`base64` now accept any
:term:`bytes-like object` in cases where it previously required a
:class:`bytes` or :class:`bytearray` instance.  (Contributed by Nick Coghlan in
:issue:`17839`.)

New functions :func:`~base64.a85encode`, :func:`~base64.a85decode`,
:func:`~base64.b85encode`, and :func:`~base64.b85decode` provide the ability to
encode and decode binary data from and to ``Ascii85`` and the git/mercurial
``Base85`` formats, respectively.  The ``a85`` functions have options that can
be used to make them compatible with the variants of the ``Ascii85`` encoding,
including the Adobe variant.  (Contributed by Martin Morrison, the Mercurial
project, Serhiy Storchaka, and Antoine Pitrou in :issue:`17618`.)


collections
-----------

The :meth:`.ChainMap.new_child` method now accepts an *m* argument specifying
the child map to add to the chain.  This allows an existing mapping and/or a
custom mapping type to be used for the child.  (Contributed by Vinay Sajip in
:issue:`16613`.)


colorsys
--------

The number of digits in the coefficients for the RGB --- YIQ conversions have
been expanded so that they match the FCC NTSC versions.  The change in
results should be less than 1% and may better match results found elsewhere.
(Contributed by Brian Landers and Serhiy Storchaka in :issue:`14323`.)


contextlib
----------

The new :class:`contextlib.suppress` context manager helps to clarify the
intent of code that deliberately suppresses exceptions from a single
statement.  (Contributed by Raymond Hettinger in :issue:`15806` and
Zero Piraeus in :issue:`19266`.)

The new :func:`contextlib.redirect_stdout` context manager makes it easier
for utility scripts to handle inflexible APIs that write their output to
:data:`sys.stdout` and don't provide any options to redirect it.  Using the
context manager, the :data:`sys.stdout` output can be redirected to any
other stream or, in conjunction with :class:`io.StringIO`, to a string.
The latter can be especially useful, for example, to capture output
from a function that was written to implement a command line interface.
It is recommended only for utility scripts because it affects the
global state of :data:`sys.stdout`.  (Contributed by Raymond Hettinger
in :issue:`15805`.)

The :mod:`contextlib` documentation has also been updated to include a
:ref:`discussion <single-use-reusable-and-reentrant-cms>` of the
differences between single use, reusable and reentrant context managers.


dbm
---

:func:`dbm.open` objects now support the context management protocol.  When
used in a :keyword:`with` statement, the ``close`` method of the database
object will be called automatically at the end of the block.  (Contributed by
Claudiu Popa and Nick Coghlan in :issue:`19282`.)


dis
---

Functions :func:`~dis.show_code`, :func:`~dis.dis`, :func:`~dis.distb`, and
:func:`~dis.disassemble` now accept a keyword-only *file* argument that
controls where they write their output.

The :mod:`dis` module is now built around an :class:`~dis.Instruction` class
that provides object oriented access to the details of each individual bytecode
operation.

A new method, :func:`~dis.get_instructions`, provides an iterator that emits
the Instruction stream for a given piece of Python code.  Thus it is now
possible to write a program that inspects and manipulates a bytecode
object in ways different from those provided by the :mod:`~dis` module
itself.  For example::

    >>> import dis
    >>> for instr in dis.get_instructions(lambda x: x + 1):
    ...     print(instr.opname)
    LOAD_FAST
    LOAD_CONST
    BINARY_ADD
    RETURN_VALUE

The various display tools in the :mod:`dis` module have been rewritten to use
these new components.

In addition, a new application-friendly class :class:`~dis.Bytecode` provides
an object-oriented API for inspecting bytecode in both in human-readable form
and for iterating over instructions.  The :class:`~dis.Bytecode` constructor
takes the same arguments that :func:`~dis.get_instruction` does (plus an
optional *current_offset*), and the resulting object can be iterated to produce
:class:`~dis.Instruction` objects.  But it also has a :mod:`~dis.Bytecode.dis`
method, equivalent to calling :mod:`~dis.dis` on the constructor argument, but
returned as a multi-line string::

    >>> bytecode = dis.Bytecode(lambda x: x + 1, current_offset=3)
    >>> for instr in bytecode:
    ...     print('{} ({})'.format(instr.opname, instr.opcode))
    LOAD_FAST (124)
    LOAD_CONST (100)
    BINARY_ADD (23)
    RETURN_VALUE (83)
    >>> bytecode.dis().splitlines()       # doctest: +NORMALIZE_WHITESPACE
    ['  1           0 LOAD_FAST                0 (x)',
     '      -->     3 LOAD_CONST               1 (1)',
     '              6 BINARY_ADD',
     '              7 RETURN_VALUE']

:class:`~dis.Bytecode` also has a class method,
:meth:`~dis.Bytecode.from_traceback`, that provides the ability to manipulate a
traceback (that is, ``print(Bytecode.from_traceback(tb).dis())`` is equivalent
to ``distb(tb)``).

(Contributed by Nick Coghlan, Ryan Kelly and Thomas Kluyver in :issue:`11816`
and Claudiu Popa in :issue:`17916`.)

New function :func:`~dis.stack_effect` computes the effect on the Python stack
of a given opcode and argument, information that is not otherwise available.
(Contributed by Larry Hastings in :issue:`19722`.)


doctest
-------

A new :ref:`option flag <doctest-options>`, :data:`~doctest.FAIL_FAST`, halts
test running as soon as the first failure is detected.  (Contributed by R.
David Murray and Daniel Urban in :issue:`16522`.)

The :mod:`doctest` command line interface now uses :mod:`argparse`, and has two
new options, ``-o`` and ``-f``.  ``-o`` allows :ref:`doctest options
<doctest-options>` to be specified on the command line, and ``-f`` is a
shorthand for ``-o FAIL_FAST`` (to parallel the similar option supported by the
:mod:`unittest` CLI).  (Contributed by R. David Murray in :issue:`11390`.)

:mod:`doctest` will now find doctests in extension module ``__doc__`` strings.
(Contributed by Zachary Ware in :issue:`3158`.)


email
-----

:meth:`~email.message.Message.as_string` now accepts a *policy* argument to
override the default policy of the message when generating a string
representation of it.  This means that ``as_string`` can now be used in more
circumstances, instead of having to create and use a :mod:`~email.generator` in
order to pass formatting parameters to its ``flatten`` method.  (Contributed by
R. David Murray in :issue:`18600`.)

New method :meth:`~email.message.Message.as_bytes` added to produce a bytes
representation of the message in a fashion similar to how ``as_string``
produces a string representation.  It does not accept the *maxheaderlen*
argument, but does accept the *unixfrom* and *policy* arguments. The
:class:`~email.message.Message` :meth:`~email.message.Message.__bytes__` method
calls it, meaning that ``bytes(mymsg)`` will now produce the intuitive
result:  a bytes object containing the fully formatted message.  (Contributed
by R. David Murray in :issue:`18600`.)

The :meth:`.Message.set_param` message now accepts a *replace* keyword argument.
When specified, the associated header will be updated without changing
its location in the list of headers.  For backward compatibility, the default
is ``False``.  (Contributed by R. David Murray in :issue:`18891`.)


.. _whatsnew_email_contentmanager:

A pair of new subclasses of :class:`~email.message.Message` have been added
(:class:`.EmailMessage` and :class:`.MIMEPart`), along with a new sub-module,
:mod:`~email.contentmanager` and a new :mod:`~email.policy` attribute
:attr:`~email.policy.EmailPolicy.content_manager`.  All documentation is
currently in the new module, which is being added as part of email's new
:term:`provisional API`.  These classes provide a number of new methods that
make extracting content from and inserting content into email messages much
easier.  For details, see the :mod:`~email.contentmanager` documentation and
the :ref:`email-examples`.  These API additions complete the
bulk of the work that was planned as part of the email6 project.  The currently
provisional API is scheduled to become final in Python 3.5 (possibly with a few
minor additions in the area of error handling).  (Contributed by R. David
Murray in :issue:`18891`.)


filecmp
-------

A new :func:`~filecmp.clear_cache` function provides the ability to clear the
:mod:`filecmp` comparison cache, which uses :func:`os.stat` information to
determine if the file has changed since the last compare.  This can be used,
for example, if the file might have been changed and re-checked in less time
than the resolution of a particular filesystem's file modification time field.
(Contributed by Mark Levitt in :issue:`18149`.)

New module attribute :data:`~filecmp.DEFAULT_IGNORES` provides the list of
directories that are used as the default value for the *ignore* parameter of
the :func:`~filecmp.dircmp` function.  (Contributed by Eli Bendersky in
:issue:`15442`.)


functools
---------

The new :func:`~functools.partialmethod` descriptor brings partial argument
application to descriptors, just as :func:`~functools.partial` provides
for normal callables. The new descriptor also makes it easier to get
arbitrary callables (including :func:`~functools.partial` instances)
to behave like normal instance methods when included in a class definition.
(Contributed by Alon Horev and Nick Coghlan in :issue:`4331`.)

.. _whatsnew-singledispatch:

The new :func:`~functools.singledispatch` decorator brings support for
single-dispatch generic functions to the Python standard library. Where
object oriented programming focuses on grouping multiple operations on a
common set of data into a class, a generic function focuses on grouping
multiple implementations of an operation that allows it to work with
*different* kinds of data.

.. seealso::

   :pep:`443` -- Single-dispatch generic functions
      PEP written and implemented by Łukasz Langa.

:func:`~functools.total_ordering` now supports a return value of
:const:`NotImplemented` from the underlying comparison function.  (Contributed
by Katie Miller in :issue:`10042`.)

A pure-python version of the :func:`~functools.partial` function is now in the
stdlib; in CPython it is overridden by the C accelerated version, but it is
available for other implementations to use.  (Contributed by Brian Thorne in
:issue:`12428`.)


gc
--

New function :func:`~gc.get_stats` returns a list of three per-generation
dictionaries containing the collections statistics since interpreter startup.
(Contributed by Antoine Pitrou in :issue:`16351`.)


glob
----

A new function :func:`~glob.escape` provides a way to escape special characters
in a filename so that they do not become part of the globbing expansion but are
instead matched literally.  (Contributed by Serhiy Storchaka in :issue:`8402`.)


hashlib
-------

A new :func:`hashlib.pbkdf2_hmac` function provides
the `PKCS#5 password-based key derivation function 2
<https://en.wikipedia.org/wiki/PBKDF2>`_.  (Contributed by Christian
Heimes in :issue:`18582`.)

The :attr:`~hashlib.hash.name` attribute of :mod:`hashlib` hash objects is now
a formally supported interface.  It has always existed in CPython's
:mod:`hashlib` (although it did not return lower case names for all supported
hashes), but it was not a public interface and so some other Python
implementations have not previously supported it.  (Contributed by Jason R.
Coombs in :issue:`18532`.)


hmac
----

:mod:`hmac` now accepts ``bytearray`` as well as ``bytes`` for the *key*
argument to the :func:`~hmac.new` function, and the *msg* parameter to both the
:func:`~hmac.new` function and the :meth:`~hmac.HMAC.update` method now
accepts any type supported by the :mod:`hashlib` module.  (Contributed
by Jonas Borgström in :issue:`18240`.)

The *digestmod* argument to the :func:`hmac.new` function may now be any hash
digest name recognized by :mod:`hashlib`.  In addition, the current behavior in
which the value of *digestmod* defaults to ``MD5`` is deprecated: in a
future version of Python there will be no default value.  (Contributed by
Christian Heimes in :issue:`17276`.)

With the addition of :attr:`~hmac.HMAC.block_size` and :attr:`~hmac.HMAC.name`
attributes (and the formal documentation of the :attr:`~hmac.HMAC.digest_size`
attribute), the :mod:`hmac` module now conforms fully to the :pep:`247` API.
(Contributed by Christian Heimes in :issue:`18775`.)


html
----

New function :func:`~html.unescape` function converts HTML5 character references to
the corresponding Unicode characters.  (Contributed by Ezio Melotti in
:issue:`2927`.)

:class:`~html.parser.HTMLParser` accepts a new keyword argument
*convert_charrefs* that, when ``True``, automatically converts all character
references.  For backward-compatibility, its value defaults to ``False``, but
it will change to ``True`` in a future version of Python, so you are invited to
set it explicitly and update your code to use this new feature.  (Contributed
by Ezio Melotti in :issue:`13633`.)

The *strict* argument of :class:`~html.parser.HTMLParser` is now deprecated.
(Contributed by Ezio Melotti in :issue:`15114`.)


http
----

:meth:`~http.server.BaseHTTPRequestHandler.send_error` now accepts an
optional additional *explain* parameter which can be used to provide an
extended error description, overriding the hardcoded default if there is one.
This extended error description will be formatted using the
:attr:`~http.server.HTTP.error_message_format` attribute and sent as the body
of the error response.  (Contributed by Karl Cow in :issue:`12921`.)

The :mod:`http.server` :ref:`command line interface <http-server-cli>` now has
a ``-b/--bind`` option that causes the server to listen on a specific address.
(Contributed by Malte Swart in :issue:`17764`.)


idlelib and IDLE
----------------

Since idlelib implements the IDLE shell and editor and is not intended for
import by other programs, it gets improvements with every release. See
:file:`Lib/idlelib/NEWS.txt` for a cumulative list of changes since 3.3.0,
as well as changes made in future 3.4.x releases. This file is also available
from the IDLE :menuselection:`Help --> About IDLE` dialog.


importlib
---------

The :class:`~importlib.abc.InspectLoader` ABC defines a new method,
:meth:`~importlib.abc.InspectLoader.source_to_code` that accepts source
data and a path and returns a code object.  The default implementation
is equivalent to ``compile(data, path, 'exec', dont_inherit=True)``.
(Contributed by Eric Snow and Brett Cannon in :issue:`15627`.)

:class:`~importlib.abc.InspectLoader` also now has a default implementation
for the :meth:`~importlib.abc.InspectLoader.get_code` method.  However,
it will normally be desirable to override the default implementation
for performance reasons.  (Contributed by Brett Cannon in :issue:`18072`.)

The :func:`~importlib.reload` function has been moved from :mod:`imp` to
:mod:`importlib` as part of the :mod:`imp` module deprecation.  (Contributed by
Berker Peksag in :issue:`18193`.)

:mod:`importlib.util` now has a :data:`~importlib.util.MAGIC_NUMBER` attribute
providing access to the bytecode version number.  This replaces the
:func:`~imp.get_magic` function in the deprecated :mod:`imp` module.
(Contributed by Brett Cannon in :issue:`18192`.)

New :mod:`importlib.util` functions :func:`~importlib.util.cache_from_source`
and :func:`~importlib.util.source_from_cache` replace the same-named functions
in the deprecated :mod:`imp` module.  (Contributed by Brett Cannon in
:issue:`18194`.)

The :mod:`importlib` bootstrap :class:`.NamespaceLoader` now conforms to
the :class:`.InspectLoader` ABC, which means that ``runpy`` and
``python -m`` can now be used with namespace packages.  (Contributed
by Brett Cannon in :issue:`18058`.)

:mod:`importlib.util` has a new function :func:`~importlib.util.decode_source`
that decodes source from bytes using universal newline processing.  This is
useful for implementing :meth:`.InspectLoader.get_source` methods.

:class:`importlib.machinery.ExtensionFileLoader` now has a
:meth:`~importlib.machinery.ExtensionFileLoader.get_filename` method.  This was
inadvertently omitted in the original implementation.  (Contributed by Eric
Snow in :issue:`19152`.)


inspect
-------

The :mod:`inspect` module now offers a basic :ref:`command line interface
<inspect-module-cli>` to quickly display source code and other
information for modules, classes and functions.  (Contributed by Claudiu Popa
and Nick Coghlan in :issue:`18626`.)

:func:`~inspect.unwrap` makes it easy to unravel wrapper function chains
created by :func:`functools.wraps` (and any other API that sets the
``__wrapped__`` attribute on a wrapper function).  (Contributed by
Daniel Urban, Aaron Iles and Nick Coghlan in :issue:`13266`.)

As part of the implementation of the new :mod:`enum` module, the
:mod:`inspect` module now has substantially better support for custom
``__dir__`` methods and dynamic class attributes provided through
metaclasses.  (Contributed by Ethan Furman in :issue:`18929` and
:issue:`19030`.)

:func:`~inspect.getfullargspec` and :func:`~inspect.getargspec`
now use the :func:`~inspect.signature` API. This allows them to
support a much broader range of callables, including those with
``__signature__`` attributes, those with metadata provided by argument
clinic, :func:`functools.partial` objects and more. Note that, unlike
:func:`~inspect.signature`, these functions still ignore ``__wrapped__``
attributes, and report the already bound first argument for bound methods,
so it is still necessary to update your code to use
:func:`~inspect.signature` directly if those features are desired.
(Contributed by Yury Selivanov in :issue:`17481`.)

:func:`~inspect.signature` now supports duck types of CPython functions,
which adds support for functions compiled with Cython.  (Contributed
by Stefan Behnel and Yury Selivanov in :issue:`17159`.)


ipaddress
---------

:mod:`ipaddress` was added to the standard library in Python 3.3 as a
:term:`provisional API`. With the release of Python 3.4, this qualification
has been removed: :mod:`ipaddress` is now considered a stable API, covered
by the normal standard library requirements to maintain backwards
compatibility.

A new :attr:`~ipaddress.IPv4Address.is_global` property is ``True`` if
an address is globally routeable.  (Contributed by Peter Moody in
:issue:`17400`.)


logging
-------

The :class:`~logging.handlers.TimedRotatingFileHandler` has a new *atTime*
parameter that can be used to specify the time of day when rollover should
happen.  (Contributed by Ronald Oussoren in :issue:`9556`.)

:class:`~logging.handlers.SocketHandler` and
:class:`~logging.handlers.DatagramHandler` now support Unix domain sockets (by
setting *port* to ``None``).  (Contributed by Vinay Sajip in commit
ce46195b56a9.)

:func:`~logging.config.fileConfig` now accepts a
:class:`configparser.RawConfigParser` subclass instance for the *fname*
parameter.  This facilitates using a configuration file when logging
configuration is just a part of the overall application configuration, or where
the application modifies the configuration before passing it to
:func:`~logging.config.fileConfig`.  (Contributed by Vinay Sajip in
:issue:`16110`.)

Logging configuration data received from a socket via the
:func:`logging.config.listen` function can now be validated before being
processed by supplying a verification function as the argument to the new
*verify* keyword argument.  (Contributed by Vinay Sajip in :issue:`15452`.)


.. _whatsnew-marshal-3:

marshal
-------

The default :mod:`marshal` version has been bumped to 3.  The code implementing
the new version restores the Python2 behavior of recording only one copy of
interned strings and preserving the interning on deserialization, and extends
this "one copy" ability to any object type (including handling recursive
references).  This reduces both the size of ``.pyc`` files and the amount of
memory a module occupies in memory when it is loaded from a ``.pyc`` (or
``.pyo``) file.  (Contributed by Kristján Valur Jónsson in :issue:`16475`,
with additional speedups by Antoine Pitrou in :issue:`19219`.)


mmap
----

mmap objects can now be :mod:`weakref`\ ed.  (Contributed by Valerie Lambert in
:issue:`4885`.)


multiprocessing
---------------

.. _whatsnew-multiprocessing-no-fork:

On Unix two new :ref:`start methods <multiprocessing-start-methods>`,
``spawn`` and ``forkserver``, have been added for starting processes using
:mod:`multiprocessing`.  These make the mixing of processes with threads more
robust, and the ``spawn`` method matches the semantics that multiprocessing has
always used on Windows.  New function
:func:`~multiprocessing.get_all_start_methods` reports all start methods
available on the platform, :func:`~multiprocessing.get_start_method` reports
the current start method, and :func:`~multiprocessing.set_start_method` sets
the start method.  (Contributed by Richard Oudkerk in :issue:`8713`.)

:mod:`multiprocessing` also now has the concept of a ``context``, which
determines how child processes are created.  New function
:func:`~multiprocessing.get_context` returns a context that uses a specified
start method.  It has the same API as the :mod:`multiprocessing` module itself,
so you can use it to create :class:`~multiprocessing.pool.Pool`\ s and other
objects that will operate within that context.  This allows a framework and an
application or different parts of the same application to use multiprocessing
without interfering with each other.  (Contributed by Richard Oudkerk in
:issue:`18999`.)

Except when using the old *fork* start method, child processes no longer
inherit unneeded handles/file descriptors from their parents (part of
:issue:`8713`).

:mod:`multiprocessing` now relies on :mod:`runpy` (which implements the
``-m`` switch) to initialise ``__main__`` appropriately in child processes
when using the ``spawn`` or ``forkserver`` start methods. This resolves some
edge cases where combining multiprocessing, the ``-m`` command line switch,
and explicit relative imports could cause obscure failures in child
processes.  (Contributed by Nick Coghlan in :issue:`19946`.)


operator
--------

New function :func:`~operator.length_hint` provides an implementation of the
specification for how the :meth:`~object.__length_hint__` special method should
be used, as part of the :pep:`424` formal specification of this language
feature.  (Contributed by Armin Ronacher in :issue:`16148`.)

There is now a pure-python version of the :mod:`operator` module available for
reference and for use by alternate implementations of Python.  (Contributed by
Zachary Ware in :issue:`16694`.)


os
--

There are new functions to get and set the :ref:`inheritable flag
<fd_inheritance>` of a file descriptor (:func:`os.get_inheritable`,
:func:`os.set_inheritable`) or a Windows handle
(:func:`os.get_handle_inheritable`, :func:`os.set_handle_inheritable`).

New function :func:`~os.cpu_count` reports the number of CPUs available on the
platform on which Python is running (or ``None`` if the count can't be
determined).  The :func:`multiprocessing.cpu_count` function is now implemented
in terms of this function).  (Contributed by Trent Nelson, Yogesh Chaudhari,
Victor Stinner, and Charles-François Natali in :issue:`17914`.)

:func:`os.path.samestat` is now available on the Windows platform (and the
:func:`os.path.samefile` implementation is now shared between Unix and
Windows).  (Contributed by Brian Curtin in :issue:`11939`.)

:func:`os.path.ismount` now recognizes volumes mounted below a drive
root on Windows.  (Contributed by Tim Golden in :issue:`9035`.)

:func:`os.open` supports two new flags on platforms that provide them,
:data:`~os.O_PATH` (un-opened file descriptor), and :data:`~os.O_TMPFILE`
(unnamed temporary file; as of 3.4.0 release available only on Linux systems
with a kernel version of 3.11 or newer that have uapi headers).  (Contributed
by Christian Heimes in :issue:`18673` and Benjamin Peterson, respectively.)


pdb
---

:mod:`pdb` has been enhanced to handle generators, :keyword:`yield`, and
``yield from`` in a more useful fashion.  This is especially helpful when
debugging :mod:`asyncio` based programs.  (Contributed by Andrew Svetlov and
Xavier de Gaye in :issue:`16596`.)

The ``print`` command has been removed from :mod:`pdb`, restoring access to the
Python :func:`print` function from the pdb command line.  Python2's ``pdb`` did
not have a ``print`` command; instead, entering ``print`` executed the
``print`` statement.  In Python3 ``print`` was mistakenly made an alias for the
pdb :pdbcmd:`p` command.  ``p``, however, prints the ``repr`` of its argument,
not the ``str`` like the Python2 ``print`` command did.  Worse, the Python3
``pdb print`` command shadowed the Python3 ``print`` function, making it
inaccessible at the ``pdb`` prompt.  (Contributed by Connor Osborn in
:issue:`18764`.)


.. _whatsnew-protocol-4:

pickle
------

:mod:`pickle` now supports (but does not use by default) a new pickle protocol,
protocol 4.  This new protocol addresses a number of issues that were present
in previous protocols, such as the serialization of nested classes, very large
strings and containers, and classes whose :meth:`__new__` method takes
keyword-only arguments.  It also provides some efficiency improvements.

.. seealso::

   :pep:`3154` -- Pickle protocol 4
      PEP written by Antoine Pitrou and implemented by Alexandre Vassalotti.


plistlib
--------

:mod:`plistlib` now has an API that is similar to the standard pattern for
stdlib serialization protocols, with new :func:`~plistlib.load`,
:func:`~plistlib.dump`, :func:`~plistlib.loads`, and :func:`~plistlib.dumps`
functions.  (The older API is now deprecated.)  In addition to the already
supported XML plist format (:data:`~plistlib.FMT_XML`), it also now supports
the binary plist format (:data:`~plistlib.FMT_BINARY`).  (Contributed by Ronald
Oussoren and others in :issue:`14455`.)


poplib
------

Two new methods have been added to :mod:`poplib`: :meth:`~poplib.POP3.capa`,
which returns the list of capabilities advertised by the POP server, and
:meth:`~poplib.POP3.stls`, which switches a clear-text POP3 session into an
encrypted POP3 session if the POP server supports it.  (Contributed by Lorenzo
Catucci in :issue:`4473`.)


pprint
------

The :mod:`pprint` module's :class:`~pprint.PrettyPrinter` class and its
:func:`~pprint.pformat`, and :func:`~pprint.pprint` functions have a new
option, *compact*, that controls how the output is formatted.  Currently
setting *compact* to ``True`` means that sequences will be printed with as many
sequence elements as will fit within *width* on each (indented) line.
(Contributed by Serhiy Storchaka in :issue:`19132`.)

Long strings are now wrapped using Python's normal line continuation
syntax.  (Contributed by Antoine Pitrou in :issue:`17150`.)


pty
---

:func:`pty.spawn` now returns the status value from :func:`os.waitpid` on
the child process, instead of ``None``.  (Contributed by Gregory P. Smith.)


pydoc
-----

The :mod:`pydoc` module is now based directly on the :func:`inspect.signature`
introspection API, allowing it to provide signature information for a wider
variety of callable objects. This change also means that ``__wrapped__``
attributes are now taken into account when displaying help information.
(Contributed by Larry Hastings in :issue:`19674`.)

The :mod:`pydoc` module no longer displays the ``self`` parameter for
already bound methods. Instead, it aims to always display the exact current
signature of the supplied callable.  (Contributed by Larry Hastings in
:issue:`20710`.)

In addition to the changes that have been made to :mod:`pydoc` directly,
its handling of custom ``__dir__`` methods and various descriptor
behaviours has also been improved substantially by the underlying changes in
the :mod:`inspect` module.

As the :func:`help` builtin is based on :mod:`pydoc`, the above changes also
affect the behaviour of :func:`help`.


re
--

New :func:`~re.fullmatch` function and :meth:`.regex.fullmatch` method anchor
the pattern at both ends of the string to match.  This provides a way to be
explicit about the goal of the match, which avoids a class of subtle bugs where
``$`` characters get lost during code changes or the addition of alternatives
to an existing regular expression.  (Contributed by Matthew Barnett in
:issue:`16203`.)

The repr of :ref:`regex objects <re-objects>` now includes the pattern
and the flags; the repr of :ref:`match objects <match-objects>` now
includes the start, end, and the part of the string that matched.  (Contributed
by Hugo Lopes Tavares and Serhiy Storchaka in :issue:`13592` and
:issue:`17087`.)


resource
--------

New :func:`~resource.prlimit` function, available on Linux platforms with a
kernel version of 2.6.36 or later and glibc of 2.13 or later, provides the
ability to query or set the resource limits for processes other than the one
making the call.  (Contributed by Christian Heimes in :issue:`16595`.)

On Linux kernel version 2.6.36 or later, there are also some new
Linux specific constants: :attr:`~resource.RLIMIT_MSGQUEUE`,
:attr:`~resource.RLIMIT_NICE`, :attr:`~resource.RLIMIT_RTPRIO`,
:attr:`~resource.RLIMIT_RTTIME`, and :attr:`~resource.RLIMIT_SIGPENDING`.
(Contributed by Christian Heimes in :issue:`19324`.)

On FreeBSD version 9 and later, there some new FreeBSD specific constants:
:attr:`~resource.RLIMIT_SBSIZE`, :attr:`~resource.RLIMIT_SWAP`, and
:attr:`~resource.RLIMIT_NPTS`.  (Contributed by Claudiu Popa in
:issue:`19343`.)


select
------

:class:`~select.epoll` objects now support the context management protocol.
When used in a :keyword:`with` statement, the :meth:`~select.epoll.close`
method will be called automatically at the end of the block.  (Contributed
by Serhiy Storchaka in :issue:`16488`.)

:class:`~select.devpoll` objects now have :meth:`~select.devpoll.fileno` and
:meth:`~select.devpoll.close` methods, as well as a new attribute
:attr:`~select.devpoll.closed`.  (Contributed by Victor Stinner in
:issue:`18794`.)


shelve
------

:class:`~shelve.Shelf` instances may now be used in :keyword:`with` statements,
and will be automatically closed at the end of the :keyword:`with` block.
(Contributed by Filip Gruszczyński in :issue:`13896`.)


shutil
------

:func:`~shutil.copyfile` now raises a specific :exc:`~shutil.Error` subclass,
:exc:`~shutil.SameFileError`, when the source and destination are the same
file, which allows an application to take appropriate action on this specific
error.  (Contributed by Atsuo Ishimoto and Hynek Schlawack in
:issue:`1492704`.)


smtpd
-----

The :class:`~smtpd.SMTPServer` and :class:`~smtpd.SMTPChannel` classes now
accept a *map* keyword argument which, if specified, is passed in to
:class:`asynchat.async_chat` as its *map* argument.  This allows an application
to avoid affecting the global socket map.  (Contributed by Vinay Sajip in
:issue:`11959`.)


smtplib
-------

:exc:`~smtplib.SMTPException` is now a subclass of :exc:`OSError`, which allows
both socket level errors and SMTP protocol level errors to be caught in one
try/except statement by code that only cares whether or not an error occurred.
(Contributed by Ned Jackson Lovely in :issue:`2118`.)


socket
------

The socket module now supports the :data:`~socket.CAN_BCM` protocol on
platforms that support it.  (Contributed by Brian Thorne in :issue:`15359`.)

Socket objects have new methods to get or set their :ref:`inheritable flag
<fd_inheritance>`, :meth:`~socket.socket.get_inheritable` and
:meth:`~socket.socket.set_inheritable`.

The ``socket.AF_*`` and ``socket.SOCK_*`` constants are now enumeration values
using the new :mod:`enum` module.  This allows meaningful names to be printed
during debugging, instead of integer "magic numbers".

The :data:`~socket.AF_LINK` constant is now available on BSD and OSX.

:func:`~socket.inet_pton` and :func:`~socket.inet_ntop` are now supported
on Windows.  (Contributed by Atsuo Ishimoto in :issue:`7171`.)


sqlite3
-------

A new boolean parameter to the :func:`~sqlite3.connect` function, *uri*, can be
used to indicate that the *database* parameter is a ``uri`` (see the `SQLite
URI documentation <https://www.sqlite.org/uri.html>`_).  (Contributed by poq in
:issue:`13773`.)


ssl
---

.. _whatsnew-tls-11-12:

:data:`~ssl.PROTOCOL_TLSv1_1` and :data:`~ssl.PROTOCOL_TLSv1_2` (TLSv1.1 and
TLSv1.2 support) have been added; support for these protocols is only available if
Python is linked with OpenSSL 1.0.1 or later.  (Contributed by Michele Orrù and
Antoine Pitrou in :issue:`16692`.)

.. _whatsnew34-sslcontext:

New function :func:`~ssl.create_default_context` provides a standard way to
obtain an :class:`~ssl.SSLContext` whose settings are intended to be a
reasonable balance between compatibility and security.  These settings are
more stringent than the defaults provided by the :class:`~ssl.SSLContext`
constructor, and may be adjusted in the future, without prior deprecation, if
best-practice security requirements change.  The new recommended best
practice for using stdlib libraries that support SSL is to use
:func:`~ssl.create_default_context` to obtain an :class:`~ssl.SSLContext`
object, modify it if needed, and then pass it as the *context* argument
of the appropriate stdlib API.  (Contributed by Christian Heimes
in :issue:`19689`.)

:class:`~ssl.SSLContext` method :meth:`~ssl.SSLContext.load_verify_locations`
accepts a new optional argument *cadata*, which can be used to provide PEM or
DER encoded certificates directly via strings or bytes, respectively.
(Contributed by Christian Heimes in :issue:`18138`.)

New function :func:`~ssl.get_default_verify_paths` returns
a named tuple of the paths and environment variables that the
:meth:`~ssl.SSLContext.set_default_verify_paths` method uses to set
OpenSSL's default ``cafile`` and ``capath``.  This can be an aid in
debugging default verification issues.  (Contributed by Christian Heimes
in :issue:`18143`.)

:class:`~ssl.SSLContext` has a new method,
:meth:`~ssl.SSLContext.cert_store_stats`, that reports the number of loaded
``X.509`` certs, ``X.509 CA`` certs, and certificate revocation lists (``crl``\
s), as well as a :meth:`~ssl.SSLContext.get_ca_certs` method that returns a
list of the loaded ``CA`` certificates.  (Contributed by Christian Heimes in
:issue:`18147`.)

If OpenSSL 0.9.8 or later is available, :class:`~ssl.SSLContext` has a new
attribute :attr:`~ssl.SSLContext.verify_flags` that can be used to control the
certificate verification process by setting it to some combination of the new
constants :data:`~ssl.VERIFY_DEFAULT`, :data:`~ssl.VERIFY_CRL_CHECK_LEAF`,
:data:`~ssl.VERIFY_CRL_CHECK_CHAIN`, or :data:`~ssl.VERIFY_X509_STRICT`.
OpenSSL does not do any CRL verification by default.  (Contributed by
Christien Heimes in :issue:`8813`.)

New :class:`~ssl.SSLContext` method :meth:`~ssl.SSLContext.load_default_certs`
loads a set of default "certificate authority" (CA) certificates from default
locations, which vary according to the platform.  It can be used to load both
TLS web server authentication certificates
(``purpose=``:data:`~ssl.Purpose.SERVER_AUTH`) for a client to use to verify a
server, and certificates for a server to use in verifying client certificates
(``purpose=``:data:`~ssl.Purpose.CLIENT_AUTH`).  (Contributed by Christian
Heimes in :issue:`19292`.)

.. _whatsnew34-win-cert-store:

Two new windows-only functions, :func:`~ssl.enum_certificates` and
:func:`~ssl.enum_crls` provide the ability to retrieve certificates,
certificate information, and CRLs from the Windows cert store.  (Contributed
by Christian Heimes in :issue:`17134`.)

.. _whatsnew34-sni:

Support for server-side SNI (Server Name Indication) using the new
:meth:`ssl.SSLContext.set_servername_callback` method.
(Contributed by Daniel Black in :issue:`8109`.)

The dictionary returned by :meth:`.SSLSocket.getpeercert` contains additional
``X509v3`` extension items: ``crlDistributionPoints``, ``calIssuers``, and
``OCSP`` URIs.  (Contributed by Christian Heimes in :issue:`18379`.)


stat
----

The :mod:`stat` module is now backed by a C implementation in :mod:`_stat`. A C
implementation is required as most of the values aren't standardized and
are platform-dependent.  (Contributed by Christian Heimes in :issue:`11016`.)

The module supports new :mod:`~stat.ST_MODE` flags, :mod:`~stat.S_IFDOOR`,
:attr:`~stat.S_IFPORT`, and :attr:`~stat.S_IFWHT`.  (Contributed by
Christian Hiemes in :issue:`11016`.)


struct
------

New function :mod:`~struct.iter_unpack` and a new
:meth:`struct.Struct.iter_unpack` method on compiled formats provide streamed
unpacking of a buffer containing repeated instances of a given format of data.
(Contributed by Antoine Pitrou in :issue:`17804`.)


subprocess
----------

:func:`~subprocess.check_output` now accepts an *input* argument that can
be used to provide the contents of ``stdin`` for the command that is run.
(Contributed by Zack Weinberg in :issue:`16624`.)

:func:`~subprocess.getstatus` and :func:`~subprocess.getstatusoutput` now
work on Windows.  This change was actually inadvertently made in 3.3.4.
(Contributed by Tim Golden in :issue:`10197`.)


sunau
-----

The :meth:`~sunau.getparams` method now returns a namedtuple rather than a
plain tuple.  (Contributed by Claudiu Popa in :issue:`18901`.)

:meth:`sunau.open` now supports the context management protocol: when used in a
:keyword:`with` block, the ``close`` method of the returned object will be
called automatically at the end of the block.  (Contributed by Serhiy Storchaka
in :issue:`18878`.)

:meth:`.AU_write.setsampwidth` now supports 24 bit samples, thus adding
support for writing 24 sample using the module.  (Contributed by
Serhiy Storchaka in :issue:`19261`.)

The :meth:`~sunau.AU_write.writeframesraw` and
:meth:`~sunau.AU_write.writeframes` methods now accept any :term:`bytes-like
object`.  (Contributed by Serhiy Storchaka in :issue:`8311`.)


sys
---

New function :func:`sys.getallocatedblocks` returns the current number of
blocks allocated by the interpreter.  (In CPython with the default
``--with-pymalloc`` setting, this is allocations made through the
:c:func:`PyObject_Malloc` API.)  This can be useful for tracking memory leaks,
especially if automated via a test suite.  (Contributed by Antoine Pitrou
in :issue:`13390`.)

When the Python interpreter starts in :ref:`interactive mode
<tut-interactive>`, it checks for an :data:`~sys.__interactivehook__` attribute
on the :mod:`sys` module.  If the attribute exists, its value is called with no
arguments just before interactive mode is started.  The check is made after the
:envvar:`PYTHONSTARTUP` file is read, so it can be set there.  The :mod:`site`
module :ref:`sets it <rlcompleter-config>` to a function that enables tab
completion and history saving (in :file:`~/.python-history`) if the platform
supports :mod:`readline`.  If you do not want this (new) behavior, you can
override it in :envvar:`PYTHONSTARTUP`, :mod:`sitecustomize`, or
:mod:`usercustomize` by deleting this attribute from :mod:`sys` (or setting it
to some other callable).  (Contributed by Éric Araujo and Antoine Pitrou in
:issue:`5845`.)


tarfile
-------

The :mod:`tarfile` module now supports a simple :ref:`tarfile-commandline` when
called as a script directly or via ``-m``.  This can be used to create and
extract tarfile archives.  (Contributed by Berker Peksag in :issue:`13477`.)


textwrap
--------

The :class:`~textwrap.TextWrapper` class has two new attributes/constructor
arguments: :attr:`~textwrap.TextWrapper.max_lines`, which limits the number of
lines in the output, and :attr:`~textwrap.TextWrapper.placeholder`, which is a
string that will appear at the end of the output if it has been truncated
because of *max_lines*.  Building on these capabilities, a new convenience
function :func:`~textwrap.shorten` collapses all of the whitespace in the input
to single spaces and produces a single line of a given *width* that ends with
the *placeholder* (by default, ``[...]``).  (Contributed by Antoine Pitrou and
Serhiy Storchaka in :issue:`18585` and :issue:`18725`.)


threading
---------

The :class:`~threading.Thread` object representing the main thread can be
obtained from the new :func:`~threading.main_thread` function.  In normal
conditions this will be the thread from which the Python interpreter was
started.  (Contributed by Andrew Svetlov in :issue:`18882`.)


traceback
---------

A new :func:`traceback.clear_frames` function takes a traceback object
and clears the local variables in all of the frames it references,
reducing the amount of memory consumed.  (Contributed by Andrew Kuchling in
:issue:`1565525`.)


types
-----

A new :func:`~types.DynamicClassAttribute` descriptor provides a way to define
an attribute that acts normally when looked up through an instance object, but
which is routed to the *class* ``__getattr__`` when looked up through the
class.  This allows one to have properties active on a class, and have virtual
attributes on the class with the same name (see :mod:`Enum` for an example).
(Contributed by Ethan Furman in :issue:`19030`.)


urllib
------

:mod:`urllib.request` now supports ``data:`` URLs via the
:class:`~urllib.request.DataHandler` class.  (Contributed by Mathias Panzenböck
in :issue:`16423`.)

The http method that will be used by a :class:`~urllib.request.Request` class
can now be specified by setting a :class:`~urllib.request.Request.method`
class attribute on the subclass.  (Contributed by Jason R Coombs in
:issue:`18978`.)

:class:`~urllib.request.Request` objects are now reusable: if the
:attr:`~urllib.request.Request.full_url` or :attr:`~urllib.request.Request.data`
attributes are modified, all relevant internal properties are updated.  This
means, for example, that it is now possible to use the same
:class:`~urllib.request.Request` object in more than one
:meth:`.OpenerDirector.open` call with different *data* arguments, or to
modify a :class:`~urllib.request.Request`\ 's ``url`` rather than recomputing it
from scratch.  There is also a new
:meth:`~urllib.request.Request.remove_header` method that can be used to remove
headers from a :class:`~urllib.request.Request`.  (Contributed by Alexey
Kachayev in :issue:`16464`, Daniel Wozniak in :issue:`17485`, and Damien Brecht
and Senthil Kumaran in :issue:`17272`.)

:class:`~urllib.error.HTTPError` objects now have a
:attr:`~urllib.error.HTTPError.headers` attribute that provides access to the
HTTP response headers associated with the error.  (Contributed by
Berker Peksag in :issue:`15701`.)


unittest
--------

The :class:`~unittest.TestCase` class has a new method,
:meth:`~unittest.TestCase.subTest`, that produces a context manager whose
:keyword:`with` block becomes a "sub-test".  This context manager allows a test
method to dynamically generate subtests  by, say, calling the ``subTest``
context manager inside a loop.  A single test method can thereby produce an
indefinite number of separately-identified and separately-counted tests, all of
which will run even if one or more of them fail.  For example::

    class NumbersTest(unittest.TestCase):
        def test_even(self):
            for i in range(6):
                with self.subTest(i=i):
                    self.assertEqual(i % 2, 0)

will result in six subtests, each identified in the unittest verbose output
with a label consisting of the variable name ``i`` and a particular value for
that variable (``i=0``, ``i=1``, etc).  See :ref:`subtests` for the full
version of this example.  (Contributed by Antoine Pitrou in :issue:`16997`.)

:func:`unittest.main` now accepts an iterable of test names for
*defaultTest*, where previously it only accepted a single test name as a
string.  (Contributed by Jyrki Pulliainen in :issue:`15132`.)

If :class:`~unittest.SkipTest` is raised during test discovery (that is, at the
module level in the test file), it is now reported as a skip instead of an
error.  (Contributed by Zach Ware in :issue:`16935`.)

:meth:`~unittest.TestLoader.discover` now sorts the discovered files to provide
consistent test ordering.  (Contributed by Martin Melin and Jeff Ramnani in
:issue:`16709`.)

:class:`~unittest.TestSuite` now drops references to tests as soon as the test
has been run, if the test is successful.  On Python interpreters that do
garbage collection, this allows the tests to be garbage collected if nothing
else is holding a reference to the test.  It is possible to override this
behavior by creating a :class:`~unittest.TestSuite` subclass that defines a
custom ``_removeTestAtIndex`` method.  (Contributed by Tom Wardill, Matt
McClure, and Andrew Svetlov in :issue:`11798`.)

A new test assertion context-manager, :meth:`~unittest.TestCase.assertLogs`,
will ensure that a given block of code emits a log message using the
:mod:`logging` module.  By default the message can come from any logger and
have a priority of ``INFO`` or higher, but both the logger name and an
alternative minimum logging level may be specified.  The object returned by the
context manager can be queried for the :class:`~logging.LogRecord`\ s and/or
formatted messages that were logged.  (Contributed by Antoine Pitrou in
:issue:`18937`.)

Test discovery now works with namespace packages (Contributed by Claudiu Popa
in :issue:`17457`.)

:mod:`unittest.mock` objects now inspect their specification signatures when
matching calls, which means an argument can now be matched by either position
or name, instead of only by position.  (Contributed by Antoine Pitrou in
:issue:`17015`.)

:func:`~mock.mock_open` objects now have ``readline`` and ``readlines``
methods.  (Contributed by Toshio Kuratomi in :issue:`17467`.)


venv
----

:mod:`venv` now includes activation scripts for the ``csh`` and ``fish``
shells.  (Contributed by Andrew Svetlov in :issue:`15417`.)

:class:`~venv.EnvBuilder` and the :func:`~venv.create` convenience function
take a new keyword argument *with_pip*, which defaults to ``False``, that
controls whether or not :class:`~venv.EnvBuilder` ensures that ``pip`` is
installed in the virtual environment.  (Contributed by Nick Coghlan in
:issue:`19552` as part of the :pep:`453` implementation.)


wave
----

The :meth:`~wave.getparams` method now returns a namedtuple rather than a
plain tuple.  (Contributed by Claudiu Popa in :issue:`17487`.)

:meth:`wave.open` now supports the context management protocol.  (Contributed
by Claudiu Popa in :issue:`17616`.)

:mod:`wave` can now :ref:`write output to unseekable files
<wave-write-objects>`.  (Contributed by David Jones, Guilherme Polo, and Serhiy
Storchaka in :issue:`5202`.)

The :meth:`~wave.Wave_write.writeframesraw` and
:meth:`~wave.Wave_write.writeframes` methods now accept any :term:`bytes-like
object`.  (Contributed by Serhiy Storchaka in :issue:`8311`.)


weakref
-------

New :class:`~weakref.WeakMethod` class simulates weak references to bound
methods.  (Contributed by Antoine Pitrou in :issue:`14631`.)

New :class:`~weakref.finalize` class makes it possible to register a callback
to be invoked when an object is garbage collected, without needing to
carefully manage the lifecycle of the weak reference itself.  (Contributed by
Richard Oudkerk in :issue:`15528`.)

The callback, if any, associated with a :class:`~weakref.ref` is now
exposed via the :attr:`~weakref.ref.__callback__` attribute.  (Contributed
by Mark Dickinson in :issue:`17643`.)


xml.etree
---------

A new parser, :class:`~xml.etree.ElementTree.XMLPullParser`, allows a
non-blocking applications to parse XML documents.  An example can be
seen at :ref:`elementtree-pull-parsing`.  (Contributed by Antoine
Pitrou in :issue:`17741`.)

The :mod:`xml.etree.ElementTree` :func:`~xml.etree.ElementTree.tostring` and
:func:`~xml.etree.ElementTree.tostringlist` functions, and the
:class:`~xml.etree.ElementTree.ElementTree`
:meth:`~xml.etree.ElementTree.ElementTree.write` method, now have a
*short_empty_elements* :ref:`keyword-only parameter <keyword-only_parameter>`
providing control over whether elements with no content are written in
abbreviated (``<tag />``) or expanded (``<tag></tag>``) form.  (Contributed by
Ariel Poliak and Serhiy Storchaka in :issue:`14377`.)


zipfile
-------

The :meth:`~zipfile.PyZipFile.writepy` method of the
:class:`~zipfile.PyZipFile` class has a new *filterfunc* option that can be
used to control which directories and files are added to the archive.  For
example, this could be used to exclude test files from the archive.
(Contributed by Christian Tismer in :issue:`19274`.)

The *allowZip64* parameter to :class:`~zipfile.ZipFile` and
:class:`~zipfile.PyZipfile` is now ``True`` by default.  (Contributed by
William Mallard in :issue:`17201`.)



CPython Implementation Changes
==============================


.. _whatsnew-pep-445:

PEP 445: Customization of CPython Memory Allocators
---------------------------------------------------

:pep:`445` adds new C level interfaces to customize memory allocation in
the CPython interpreter.

.. seealso::

   :pep:`445` -- Add new APIs to customize Python memory allocators
      PEP written and implemented by Victor Stinner.


.. _whatsnew-pep-442:

PEP 442: Safe Object Finalization
---------------------------------

:pep:`442` removes the current limitations and quirks of object finalization
in CPython. With it, objects with :meth:`__del__` methods, as well as
generators with :keyword:`finally` clauses, can be finalized when they are
part of a reference cycle.

As part of this change, module globals are no longer forcibly set to
:const:`None` during interpreter shutdown in most cases, instead relying
on the normal operation of the cyclic garbage collector.  This avoids a
whole class of interpreter-shutdown-time errors, usually involving
``__del__`` methods, that have plagued Python since the cyclic GC
was first introduced.

.. seealso::

   :pep:`442` -- Safe object finalization
      PEP written and implemented by Antoine Pitrou.


.. _whatsnew-pep-456:

PEP 456: Secure and Interchangeable Hash Algorithm
--------------------------------------------------

:pep:`456` follows up on earlier security fix work done on Python's hash
algorithm to address certain DOS attacks to which public facing APIs backed by
dictionary lookups may be subject.  (See :issue:`14621` for the start of the
current round of improvements.)  The PEP unifies CPython's hash code to make it
easier for a packager to substitute a different hash algorithm, and switches
Python's default implementation to a SipHash implementation on platforms that
have a 64 bit data type.  Any performance differences in comparison with the
older FNV algorithm are trivial.

The PEP adds additional fields to the :attr:`sys.hash_info` struct sequence to
describe the hash algorithm in use by the currently executing binary.  Otherwise,
the PEP does not alter any existing CPython APIs.


.. _whatsnew-pep-436:

PEP 436: Argument Clinic
------------------------

"Argument Clinic" (:pep:`436`) is now part of the CPython build process
and can be used to simplify the process of defining and maintaining
accurate signatures for builtins and standard library extension modules
implemented in C.

Some standard library extension modules have been converted to use Argument
Clinic in Python 3.4, and :mod:`pydoc` and :mod:`inspect` have been updated
accordingly.

It is expected that signature metadata for programmatic introspection will
be added to additional callables implemented in C as part of Python 3.4
maintenance releases.

.. note::
   The Argument Clinic PEP is not fully up to date with the state of the
   implementation. This has been deemed acceptable by the release manager
   and core development team in this case, as Argument Clinic will not
   be made available as a public API for third party use in Python 3.4.

.. seealso::

   :pep:`436` -- The Argument Clinic DSL
       PEP written and implemented by Larry Hastings.


Other Build and C API Changes
-----------------------------

* The new :c:func:`PyType_GetSlot` function has been added to the stable ABI,
  allowing retrieval of function pointers from named type slots when using
  the limited API.  (Contributed by Martin von Löwis in :issue:`17162`.)

* The new :c:func:`Py_SetStandardStreamEncoding` pre-initialization API
  allows applications embedding the CPython interpreter to reliably force
  a particular encoding and error handler for the standard streams.
  (Contributed by Bastien Montagne and Nick Coghlan in :issue:`16129`.)

* Most Python C APIs that don't mutate string arguments are now correctly
  marked as accepting ``const char *`` rather than ``char *``.  (Contributed
  by Serhiy Storchaka in :issue:`1772673`.)

* A new shell version of ``python-config`` can be used even when a python
  interpreter is not available (for example, in cross compilation scenarios).

* :c:func:`PyUnicode_FromFormat` now supports width and precision
  specifications for ``%s``, ``%A``, ``%U``, ``%V``, ``%S``, and ``%R``.
  (Contributed by Ysj Ray and Victor Stinner in :issue:`7330`.)

* New function :c:func:`PyStructSequence_InitType2` supplements the
  existing :c:func:`PyStructSequence_InitType` function.  The difference
  is that it returns ``0`` on success and ``-1`` on failure.

* The CPython source can now be compiled using the address sanity checking
  features of recent versions of GCC and clang:  the false alarms in the small
  object allocator have been silenced.  (Contributed by Dhiru Kholia in
  :issue:`18596`.)

* The Windows build now uses `Address Space Layout Randomization
  <https://en.wikipedia.org/wiki/Address_space_layout_randomization>`_ and `Data Execution Prevention
  <https://en.wikipedia.org/wiki/Data_Execution_Prevention>`_.  (Contributed by
  Christian Heimes in :issue:`16632`.)

* New function :c:func:`PyObject_LengthHint` is the C API equivalent
  of :func:`operator.length_hint`.  (Contributed by Armin Ronacher in
  :issue:`16148`.)


.. _other-improvements-3.4:

Other Improvements
------------------

.. _whatsnew-isolated-mode:

* The :ref:`python <using-on-cmdline>` command has a new :ref:`option
  <using-on-misc-options>`, ``-I``, which causes it to run in "isolated mode",
  which means that :data:`sys.path` contains neither the script's directory nor
  the user's ``site-packages`` directory, and all :envvar:`PYTHON*` environment
  variables are ignored (it implies both ``-s`` and ``-E``).  Other
  restrictions may also be applied in the future, with the goal being to
  isolate the execution of a script from the user's environment.  This is
  appropriate, for example, when Python is used to run a system script.  On
  most POSIX systems it can and should be used in the ``#!`` line of system
  scripts.  (Contributed by Christian Heimes in :issue:`16499`.)

* Tab-completion is now enabled by default in the interactive interpreter
  on systems that support :mod:`readline`.  History is also enabled by default,
  and is written to (and read from) the file :file:`~/.python-history`.
  (Contributed by Antoine Pitrou and Éric Araujo in :issue:`5845`.)

* Invoking the Python interpreter with ``--version`` now outputs the version to
  standard output instead of standard error (:issue:`18338`). Similar changes
  were made to :mod:`argparse` (:issue:`18920`) and other modules that have
  script-like invocation capabilities (:issue:`18922`).

* The CPython Windows installer now adds ``.py`` to the :envvar:`PATHEXT`
  variable when extensions are registered, allowing users to run a python
  script at the windows command prompt by just typing its name without the
  ``.py`` extension.  (Contributed by Paul Moore in :issue:`18569`.)

* A new ``make`` target `coverage-report
  <https://devguide.python.org/coverage/#measuring-coverage-of-c-code-with-gcov-and-lcov>`_
  will build python, run the test suite, and generate an HTML coverage report
  for the C codebase using ``gcov`` and `lcov
  <http://ltp.sourceforge.net/coverage/lcov.php>`_.

* The ``-R`` option to the :ref:`python regression test suite <regrtest>` now
  also checks for memory allocation leaks, using
  :func:`sys.getallocatedblocks()`.  (Contributed by Antoine Pitrou in
  :issue:`13390`.)

* ``python -m`` now works with namespace packages.

* The :mod:`stat` module is now implemented in C, which means it gets the
  values for its constants from the C header files, instead of having the
  values hard-coded in the python module as was previously the case.

* Loading multiple python modules from a single OS module (``.so``, ``.dll``)
  now works correctly (previously it silently returned the first python
  module in the file).  (Contributed by Václav Šmilauer in :issue:`16421`.)

* A new opcode, :opcode:`LOAD_CLASSDEREF`, has been added to fix a bug in the
  loading of free variables in class bodies that could be triggered by certain
  uses of :ref:`__prepare__ <prepare>`.  (Contributed by Benjamin Peterson in
  :issue:`17853`.)

* A number of MemoryError-related crashes were identified and fixed by Victor
  Stinner using his :pep:`445`-based ``pyfailmalloc`` tool (:issue:`18408`,
  :issue:`18520`).

* The ``pyvenv`` command now accepts a ``--copies`` option
  to use copies rather than symlinks even on systems where symlinks are the
  default.  (Contributed by Vinay Sajip in :issue:`18807`.)

* The ``pyvenv`` command also accepts a ``--without-pip``
  option to suppress the otherwise-automatic bootstrapping of pip into
  the virtual environment.  (Contributed by Nick Coghlan in :issue:`19552`
  as part of the :pep:`453` implementation.)

* The encoding name is now optional in the value set for the
  :envvar:`PYTHONIOENCODING` environment variable.  This makes it possible to
  set just the error handler, without changing the default encoding.
  (Contributed by Serhiy Storchaka in :issue:`18818`.)

* The :mod:`bz2`, :mod:`lzma`, and :mod:`gzip` module ``open`` functions now
  support ``x`` (exclusive creation) mode.  (Contributed by Tim Heaney and
  Vajrasky Kok in :issue:`19201`, :issue:`19222`, and :issue:`19223`.)


Significant Optimizations
-------------------------

* The UTF-32 decoder is now 3x to 4x faster.  (Contributed by Serhiy Storchaka
  in :issue:`14625`.)

* The cost of hash collisions for sets is now reduced.  Each hash table
  probe now checks a series of consecutive, adjacent key/hash pairs before
  continuing to make random probes through the hash table.  This exploits
  cache locality to make collision resolution less expensive.
  The collision resolution scheme can be described as a hybrid of linear
  probing and open addressing.  The number of additional linear probes
  defaults to nine.  This can be changed at compile-time by defining
  LINEAR_PROBES to be any value.  Set LINEAR_PROBES=0 to turn-off
  linear probing entirely.  (Contributed by Raymond Hettinger in
  :issue:`18771`.)

* The interpreter starts about 30% faster. A couple of measures lead to the
  speedup. The interpreter loads fewer modules on startup, e.g. the :mod:`re`,
  :mod:`collections` and :mod:`locale` modules and their dependencies are no
  longer imported by default. The marshal module has been improved to load
  compiled Python code faster.  (Contributed by Antoine Pitrou, Christian
  Heimes and Victor Stinner in :issue:`19219`, :issue:`19218`, :issue:`19209`,
  :issue:`19205` and :issue:`9548`.)

* :class:`bz2.BZ2File` is now as fast or faster than the Python2 version for
  most cases.  :class:`lzma.LZMAFile` has also been optimized.  (Contributed by
  Serhiy Storchaka and Nadeem Vawda in :issue:`16034`.)

* :func:`random.getrandbits` is 20%-40% faster for small integers (the most
  common use case).  (Contributed by Serhiy Storchaka in :issue:`16674`.)

* By taking advantage of the new storage format for strings, pickling of
  strings is now significantly faster.  (Contributed by Victor Stinner and
  Antoine Pitrou in :issue:`15596`.)

* A performance issue in :meth:`io.FileIO.readall` has been solved.  This
  particularly affects Windows, and significantly speeds up the case of piping
  significant amounts of data through :mod:`subprocess`.  (Contributed
  by Richard Oudkerk in :issue:`15758`.)

* :func:`html.escape` is now 10x faster.  (Contributed by Matt Bryant in
  :issue:`18020`.)

* On Windows, the native ``VirtualAlloc`` is now used instead of the CRT
  ``malloc`` in ``obmalloc``.  Artificial benchmarks show about a 3% memory
  savings.

* :func:`os.urandom` now uses a lazily-opened persistent file descriptor
  so as to avoid using many file descriptors when run in parallel from
  multiple threads.  (Contributed by Antoine Pitrou in :issue:`18756`.)


.. _deprecated-3.4:

Deprecated
==========

This section covers various APIs and other features that have been deprecated
in Python 3.4, and will be removed in Python 3.5 or later.  In most (but not
all) cases, using the deprecated APIs will produce a :exc:`DeprecationWarning`
when the interpreter is run with deprecation warnings enabled (for example, by
using ``-Wd``).


Deprecations in the Python API
------------------------------

* As mentioned in :ref:`whatsnew-pep-451`, a number of :mod:`importlib`
  methods and functions are deprecated: :meth:`importlib.find_loader` is
  replaced by :func:`importlib.util.find_spec`;
  :meth:`importlib.machinery.PathFinder.find_module` is replaced by
  :meth:`importlib.machinery.PathFinder.find_spec`;
  :meth:`importlib.abc.MetaPathFinder.find_module` is replaced by
  :meth:`importlib.abc.MetaPathFinder.find_spec`;
  :meth:`importlib.abc.PathEntryFinder.find_loader` and
  :meth:`~importlib.abc.PathEntryFinder.find_module` are replaced by
  :meth:`importlib.abc.PathEntryFinder.find_spec`; all of the ``xxxLoader`` ABC
  ``load_module`` methods (:meth:`importlib.abc.Loader.load_module`,
  :meth:`importlib.abc.InspectLoader.load_module`,
  :meth:`importlib.abc.FileLoader.load_module`,
  :meth:`importlib.abc.SourceLoader.load_module`) should no longer be
  implemented, instead loaders should implement an
  ``exec_module`` method
  (:meth:`importlib.abc.Loader.exec_module`,
  :meth:`importlib.abc.InspectLoader.exec_module`
  :meth:`importlib.abc.SourceLoader.exec_module`) and let the import system
  take care of the rest; and
  :meth:`importlib.abc.Loader.module_repr`,
  :meth:`importlib.util.module_for_loader`, :meth:`importlib.util.set_loader`,
  and :meth:`importlib.util.set_package` are no longer needed because their
  functions are now handled automatically by the import system.

* The :mod:`imp` module is pending deprecation. To keep compatibility with
  Python 2/3 code bases, the module's removal is currently not scheduled.

* The :mod:`formatter` module is pending deprecation and is slated for removal
  in Python 3.6.

* ``MD5`` as the default *digestmod* for the :func:`hmac.new` function is
  deprecated.  Python 3.6 will require an explicit digest name or constructor as
  *digestmod* argument.

* The internal ``Netrc`` class in the :mod:`ftplib` module has been documented
  as deprecated in its docstring for quite some time.  It now emits a
  :exc:`DeprecationWarning` and will be removed completely in Python 3.5.

* The undocumented *endtime* argument to :meth:`subprocess.Popen.wait` should
  not have been exposed and is hopefully not in use; it is deprecated and
  will mostly likely be removed in Python 3.5.

* The *strict* argument of :class:`~html.parser.HTMLParser` is deprecated.

* The :mod:`plistlib` :func:`~plistlib.readPlist`,
  :func:`~plistlib.writePlist`, :func:`~plistlib.readPlistFromBytes`, and
  :func:`~plistlib.writePlistToBytes` functions are deprecated in favor of the
  corresponding new functions :func:`~plistlib.load`, :func:`~plistlib.dump`,
  :func:`~plistlib.loads`, and :func:`~plistlib.dumps`.  :func:`~plistlib.Data`
  is deprecated in favor of just using the :class:`bytes` constructor.

* The :mod:`sysconfig` key ``SO`` is deprecated, it has been replaced by
  ``EXT_SUFFIX``.

* The ``U`` mode accepted by various ``open`` functions is deprecated.
  In Python3 it does not do anything useful, and should be replaced by
  appropriate uses of :class:`io.TextIOWrapper` (if needed) and its *newline*
  argument.

* The *parser* argument of :func:`xml.etree.ElementTree.iterparse` has
  been deprecated, as has the *html* argument of
  :func:`~xml.etree.ElementTree.XMLParser`.  To prepare for the removal of the
  latter, all arguments to ``XMLParser`` should be passed by keyword.


Deprecated Features
-------------------

* Running :ref:`idle` with the ``-n`` flag (no subprocess) is deprecated.
  However, the feature will not be removed until :issue:`18823` is resolved.

* The site module adding a "site-python" directory to sys.path, if it
  exists, is deprecated (:issue:`19375`).



Removed
=======


Operating Systems No Longer Supported
-------------------------------------

Support for the following operating systems has been removed from the source
and build tools:

* OS/2 (:issue:`16135`).
* Windows 2000 (changeset e52df05b496a).
* Windows systems where ``COMSPEC`` points to ``command.com`` (:issue:`14470`).
* VMS (:issue:`16136`).


API and Feature Removals
------------------------

The following obsolete and previously deprecated APIs and features have been
removed:

* The unmaintained ``Misc/TextMate`` and ``Misc/vim`` directories have been
  removed (see the `devguide <https://devguide.python.org>`_
  for suggestions on what to use instead).

* The ``SO`` makefile macro is removed (it was replaced by the
  ``SHLIB_SUFFIX`` and ``EXT_SUFFIX`` macros) (:issue:`16754`).

* The ``PyThreadState.tick_counter`` field has been removed; its value has
  been meaningless since Python 3.2, when the "new GIL" was introduced
  (:issue:`19199`).

* ``PyLoader`` and ``PyPycLoader`` have been removed from :mod:`importlib`.
  (Contributed by Taras Lyapun in :issue:`15641`.)

* The *strict* argument to :class:`~http.client.HTTPConnection` and
  :class:`~http.client.HTTPSConnection` has been removed.  HTTP 0.9-style
  "Simple Responses" are no longer supported.

* The deprecated :mod:`urllib.request.Request` getter and setter methods
  ``add_data``, ``has_data``, ``get_data``, ``get_type``, ``get_host``,
  ``get_selector``, ``set_proxy``, ``get_origin_req_host``, and
  ``is_unverifiable`` have been removed (use direct attribute access instead).

* Support for loading the deprecated ``TYPE_INT64`` has been removed from
  :mod:`marshal`.  (Contributed by Dan Riti in :issue:`15480`.)

* :class:`inspect.Signature`: positional-only parameters are now required
  to have a valid name.

* :meth:`object.__format__` no longer accepts non-empty format strings, it now
  raises a :exc:`TypeError` instead.  Using a non-empty string has been
  deprecated since Python 3.2.  This change has been made to prevent a
  situation where previously working (but incorrect) code would start failing
  if an object gained a __format__ method, which means that your code may now
  raise a :exc:`TypeError` if you are using an ``'s'`` format code with objects
  that do not have a __format__ method that handles it.  See :issue:`7994` for
  background.

* :meth:`difflib.SequenceMatcher.isbjunk` and
  :meth:`difflib.SequenceMatcher.isbpopular` were deprecated in 3.2, and have
  now been removed: use ``x in sm.bjunk`` and
  ``x in sm.bpopular``, where *sm* is a :class:`~difflib.SequenceMatcher` object
  (:issue:`13248`).


Code Cleanups
-------------

* The unused and undocumented internal ``Scanner`` class has been removed from
  the :mod:`pydoc` module.

* The private and effectively unused ``_gestalt`` module has been removed,
  along with the private :mod:`platform` functions ``_mac_ver_lookup``,
  ``_mac_ver_gstalt``, and ``_bcd2str``, which would only have ever been called
  on badly broken OSX systems (see :issue:`18393`).

* The hardcoded copies of certain :mod:`stat` constants that were included in
  the :mod:`tarfile` module namespace have been removed.



Porting to Python 3.4
=====================

This section lists previously described changes and other bugfixes
that may require changes to your code.


Changes in 'python' Command Behavior
------------------------------------

* In a posix shell, setting the :envvar:`PATH` environment variable to
  an empty value is equivalent to not setting it at all.  However, setting
  :envvar:`PYTHONPATH` to an empty value was *not* equivalent to not setting it
  at all: setting :envvar:`PYTHONPATH` to an empty value was equivalent to
  setting it to ``.``, which leads to confusion when reasoning by analogy to
  how :envvar:`PATH` works.  The behavior now conforms to the posix convention
  for :envvar:`PATH`.

* The [X refs, Y blocks] output of a debug (``--with-pydebug``) build of the
  CPython interpreter is now off by default.  It can be re-enabled using the
  ``-X showrefcount`` option.  (Contributed by Ezio Melotti in :issue:`17323`.)

* The python command and most stdlib scripts (as well as :mod:`argparse`) now
  output ``--version`` information to ``stdout`` instead of ``stderr`` (for
  issue list see :ref:`other-improvements-3.4` above).


Changes in the Python API
-------------------------

* The ABCs defined in :mod:`importlib.abc` now either raise the appropriate
  exception or return a default value instead of raising
  :exc:`NotImplementedError` blindly. This will only affect code calling
  :func:`super` and falling through all the way to the ABCs. For compatibility,
  catch both :exc:`NotImplementedError` or the appropriate exception as needed.

* The module type now initializes the :attr:`__package__` and :attr:`__loader__`
  attributes to ``None`` by default. To determine if these attributes were set
  in a backwards-compatible fashion, use e.g.
  ``getattr(module, '__loader__', None) is not None``.  (:issue:`17115`.)

* :meth:`importlib.util.module_for_loader` now sets ``__loader__`` and
  ``__package__`` unconditionally to properly support reloading. If this is not
  desired then you will need to set these attributes manually. You can use
  :func:`importlib.util.module_to_load` for module management.

* Import now resets relevant attributes (e.g. ``__name__``, ``__loader__``,
  ``__package__``, ``__file__``, ``__cached__``) unconditionally when reloading.
  Note that this restores a pre-3.3 behavior in that it means a module is
  re-found when re-loaded (:issue:`19413`).

* Frozen packages no longer set ``__path__`` to a list containing the package
  name, they now set it to an empty list.  The previous behavior could cause
  the import system to do the wrong thing on submodule imports if there was
  also a directory with the same name as the frozen package.  The correct way
  to determine if a module is a package or not is to use ``hasattr(module,
  '__path__')`` (:issue:`18065`).

* Frozen modules no longer define a ``__file__`` attribute. It's semantically
  incorrect for frozen modules to set the attribute as they are not loaded from
  any explicit location. If you must know that a module comes from frozen code
  then you can see if the module's ``__spec__.location`` is set to ``'frozen'``,
  check if the loader is a subclass of
  :class:`importlib.machinery.FrozenImporter`,
  or if Python 2 compatibility is necessary you can use :func:`imp.is_frozen`.

* :func:`py_compile.compile` now raises :exc:`FileExistsError` if the file path
  it would write to is a symlink or a non-regular file. This is to act as a
  warning that import will overwrite those files with a regular file regardless
  of what type of file path they were originally.

* :meth:`importlib.abc.SourceLoader.get_source` no longer raises
  :exc:`ImportError` when the source code being loaded triggers a
  :exc:`SyntaxError` or :exc:`UnicodeDecodeError`. As :exc:`ImportError` is
  meant to be raised only when source code cannot be found but it should, it was
  felt to be over-reaching/overloading of that meaning when the source code is
  found but improperly structured. If you were catching ImportError before and
  wish to continue to ignore syntax or decoding issues, catch all three
  exceptions now.

* :func:`functools.update_wrapper` and :func:`functools.wraps` now correctly
  set the ``__wrapped__`` attribute to the function being wrapped, even if
  that function also had its ``__wrapped__`` attribute set. This means
  ``__wrapped__`` attributes now correctly link a stack of decorated
  functions rather than every ``__wrapped__`` attribute in the chain
  referring to the innermost function. Introspection libraries that
  assumed the previous behaviour was intentional can use
  :func:`inspect.unwrap` to access the first function in the chain that has
  no ``__wrapped__`` attribute.

* :func:`inspect.getfullargspec` has been reimplemented on top of
  :func:`inspect.signature` and hence handles a much wider variety of callable
  objects than it did in the past. It is expected that additional builtin and
  extension module callables will gain signature metadata over the course of
  the Python 3.4 series. Code that assumes that
  :func:`inspect.getfullargspec` will fail on non-Python callables may need
  to be adjusted accordingly.

* :class:`importlib.machinery.PathFinder` now passes on the current working
  directory to objects in :data:`sys.path_hooks` for the empty string. This
  results in :data:`sys.path_importer_cache` never containing ``''``, thus
  iterating through :data:`sys.path_importer_cache` based on :data:`sys.path`
  will not find all keys. A module's ``__file__`` when imported in the current
  working directory will also now have an absolute path, including when using
  ``-m`` with the interpreter (except for ``__main__.__file__`` when a script
  has been executed directly using a relative path) (Contributed by Brett
  Cannon in :issue:`18416`).  is specified on the command-line)
  (:issue:`18416`).

* The removal of the *strict* argument to :class:`~http.client.HTTPConnection`
  and :class:`~http.client.HTTPSConnection` changes the meaning of the
  remaining arguments if you are specifying them positionally rather than by
  keyword.  If you've been paying attention to deprecation warnings your code
  should already be specifying any additional arguments via keywords.

* Strings between ``from __future__ import ...`` statements now *always* raise
  a :exc:`SyntaxError`.  Previously if there was no leading docstring, an
  interstitial string would sometimes be ignored.  This brings CPython into
  compliance with the language spec; Jython and PyPy already were.
  (:issue:`17434`).

* :meth:`ssl.SSLSocket.getpeercert` and :meth:`ssl.SSLSocket.do_handshake`
  now raise an :exc:`OSError` with ``ENOTCONN`` when the ``SSLSocket`` is not
  connected, instead of the previous behavior of raising an
  :exc:`AttributeError`.  In addition, :meth:`~ssl.SSLSocket.getpeercert`
  will raise a :exc:`ValueError` if the handshake has not yet been done.

* :func:`base64.b32decode` now raises a :exc:`binascii.Error` when the
  input string contains non-b32-alphabet characters, instead of a
  :exc:`TypeError`.  This particular :exc:`TypeError` was missed when the other
  :exc:`TypeError`\ s were converted.  (Contributed by Serhiy Storchaka in
  :issue:`18011`.)  Note: this change was also inadvertently applied in Python
  3.3.3.

* The :attr:`~cgi.FieldStorage.file` attribute is now automatically closed when
  the creating :class:`cgi.FieldStorage` instance is garbage collected. If you
  were pulling the file object out separately from the :class:`cgi.FieldStorage`
  instance and not keeping the instance alive, then you should either store the
  entire :class:`cgi.FieldStorage` instance or read the contents of the file
  before the :class:`cgi.FieldStorage` instance is garbage collected.

* Calling ``read`` or ``write`` on a closed SSL socket now raises an
  informative :exc:`ValueError` rather than the previous more mysterious
  :exc:`AttributeError` (:issue:`9177`).

* :meth:`slice.indices` no longer produces an :exc:`OverflowError` for huge
  values.  As a consequence of this fix, :meth:`slice.indices` now raises a
  :exc:`ValueError` if given a negative length; previously it returned nonsense
  values (:issue:`14794`).

* The :class:`complex` constructor, unlike the :mod:`cmath` functions, was
  incorrectly accepting :class:`float` values if an object's ``__complex__``
  special method returned one.  This now raises a :exc:`TypeError`.
  (:issue:`16290`.)

* The :class:`int` constructor in 3.2 and 3.3 erroneously accepts :class:`float`
  values for the *base* parameter.  It is unlikely anyone was doing this, but
  if so, it will now raise a :exc:`TypeError` (:issue:`16772`).

* Defaults for keyword-only arguments are now evaluated *after* defaults for
  regular keyword arguments, instead of before.  Hopefully no one wrote any
  code that depends on the previous buggy behavior (:issue:`16967`).

* Stale thread states are now cleared after :func:`~os.fork`.  This may cause
  some system resources to be released that previously were incorrectly kept
  perpetually alive (for example, database connections kept in thread-local
  storage).  (:issue:`17094`.)

* Parameter names in ``__annotations__`` dicts are now mangled properly,
  similarly to ``__kwdefaults__``.  (Contributed by Yury Selivanov in
  :issue:`20625`.)

* :attr:`hashlib.hash.name` now always returns the identifier in lower case.
  Previously some builtin hashes had uppercase names, but now that it is a
  formal public interface the naming has been made consistent (:issue:`18532`).

* Because :mod:`unittest.TestSuite` now drops references to tests after they
  are run, test harnesses that re-use a :class:`~unittest.TestSuite` to re-run
  a set of tests may fail.   Test suites should not be re-used in this fashion
  since it means state is retained between test runs, breaking the test
  isolation that :mod:`unittest` is designed to provide.  However, if the lack
  of isolation is considered acceptable, the old behavior can be restored by
  creating a :mod:`~unittest.TestSuite` subclass that defines a
  ``_removeTestAtIndex`` method that does nothing (see
  :meth:`.TestSuite.__iter__`) (:issue:`11798`).

* :mod:`unittest` now uses :mod:`argparse` for command line parsing.  There are
  certain invalid command forms that used to work that are no longer allowed;
  in theory this should not cause backward compatibility issues since the
  disallowed command forms didn't make any sense and are unlikely to be in use.

* The :func:`re.split`, :func:`re.findall`, and :func:`re.sub` functions, and
  the :meth:`~re.match.group` and :meth:`~re.match.groups` methods of
  ``match`` objects now always return a *bytes* object when the string
  to be matched is a :term:`bytes-like object`.  Previously the return type
  matched the input type, so if your code was depending on the return value
  being, say, a ``bytearray``, you will need to change your code.

* :mod:`audioop` functions now raise an error immediately if passed string
  input, instead of failing randomly later on (:issue:`16685`).

* The new *convert_charrefs* argument to :class:`~html.parser.HTMLParser`
  currently defaults to ``False`` for backward compatibility, but will
  eventually be changed to default to ``True``.  It is recommended that you add
  this keyword, with the appropriate value, to any
  :class:`~html.parser.HTMLParser` calls in your code (:issue:`13633`).

* Since the *digestmod* argument to the :func:`hmac.new` function will in the
  future have no default, all calls to :func:`hmac.new` should be changed to
  explicitly specify a *digestmod* (:issue:`17276`).

* Calling :func:`sysconfig.get_config_var` with the ``SO`` key, or looking
  ``SO`` up in the results of a call to :func:`sysconfig.get_config_vars`
  is deprecated.  This key should be replaced by ``EXT_SUFFIX`` or
  ``SHLIB_SUFFIX``, depending on the context (:issue:`19555`).

* Any calls to ``open`` functions that specify ``U`` should be modified.
  ``U`` is ineffective in Python3 and will eventually raise an error if used.
  Depending on the function, the equivalent of its old Python2 behavior can be
  achieved using either a *newline* argument, or if necessary by wrapping the
  stream in :mod:`~io.TextIOWrapper` to use its *newline* argument
  (:issue:`15204`).

* If you use ``pyvenv`` in a script and desire that pip
  *not* be installed, you must add ``--without-pip`` to your command
  invocation.

* The default behavior of :func:`json.dump` and :func:`json.dumps` when
  an indent is specified has changed: it no longer produces trailing
  spaces after the item separating commas at the ends of lines.  This
  will matter only if you have tests that are doing white-space-sensitive
  comparisons of such output (:issue:`16333`).

* :mod:`doctest` now looks for doctests in extension module ``__doc__``
  strings, so if your doctest test discovery includes extension modules that
  have things that look like doctests in them you may see test failures you've
  never seen before when running your tests (:issue:`3158`).

* The :mod:`collections.abc` module has been slightly refactored as
  part of the Python startup improvements.  As a consequence of this, it is no
  longer the case that importing :mod:`collections` automatically imports
  :mod:`collections.abc`.  If your program depended on the (undocumented)
  implicit import, you will need to add an explicit ``import collections.abc``
  (:issue:`20784`).


Changes in the C API
--------------------

* :c:func:`PyEval_EvalFrameEx`, :c:func:`PyObject_Repr`, and
  :c:func:`PyObject_Str`, along with some other internal C APIs, now include
  a debugging assertion that ensures they are not used in situations where
  they may silently discard a currently active exception. In cases where
  discarding the active exception is expected and desired (for example,
  because it has already been saved locally with :c:func:`PyErr_Fetch` or
  is being deliberately replaced with a different exception), an explicit
  :c:func:`PyErr_Clear` call will be needed to avoid triggering the
  assertion when invoking these operations (directly or indirectly) and
  running against a version of Python that is compiled with assertions
  enabled.

* :c:func:`PyErr_SetImportError` now sets :exc:`TypeError` when its **msg**
  argument is not set. Previously only ``NULL`` was returned with no exception
  set.

* The result of the :c:data:`PyOS_ReadlineFunctionPointer` callback must
  now be a string allocated by :c:func:`PyMem_RawMalloc` or
  :c:func:`PyMem_RawRealloc`, or *NULL* if an error occurred, instead of a
  string allocated by :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`
  (:issue:`16742`)

* :c:func:`PyThread_set_key_value` now always set the value. In Python
  3.3, the function did nothing if the key already exists (if the current
  value is a non-NULL pointer).

* The ``f_tstate`` (thread state) field of the :c:type:`PyFrameObject`
  structure has been removed to fix a bug: see :issue:`14432` for the
  rationale.

Changed in 3.4.3
================

.. _pep-476:

PEP 476: Enabling certificate verification by default for stdlib http clients
-----------------------------------------------------------------------------

:mod:`http.client` and modules which use it, such as :mod:`urllib.request` and
:mod:`xmlrpc.client`, will now verify that the server presents a certificate
which is signed by a CA in the platform trust store and whose hostname matches
the hostname being requested by default, significantly improving security for
many applications.

For applications which require the old previous behavior, they can pass an
alternate context::

    import urllib.request
    import ssl

    # This disables all verification
    context = ssl._create_unverified_context()

    # This allows using a specific certificate for the host, which doesn't need
    # to be in the trust store
    context = ssl.create_default_context(cafile="/path/to/file.crt")

    urllib.request.urlopen("https://invalid-cert", context=context)