summaryrefslogtreecommitdiffstats
path: root/src/H5Fprivate.h
Commit message (Expand)AuthorAgeFilesLines
* [svn-r15800] Description:Quincey Koziol2008-10-071-39/+40
* [svn-r15796] Purpose: Close bug #1322Neil Fortner2008-10-071-2/+2
* [svn-r15746] Description:Quincey Koziol2008-10-011-0/+1
* [svn-r15704] I changed the return values of H5Fget_obj_ids and H5Fget_obj_cou...Raymond Lu2008-09-261-2/+2
* [svn-r15676] Description:Quincey Koziol2008-09-221-0/+46
* [svn-r15485] Purpose: Allow library to shut down properly when objects have r...Neil Fortner2008-08-191-2/+2
* [svn-r15468] Removed prototype for deleted function H5F_has_mountNeil Fortner2008-08-121-1/+0
* [svn-r15211] Description:Quincey Koziol2008-06-131-0/+2
* [svn-r14903] Undoing change committed in r14902.Scot Breitenfeld2008-04-301-0/+3
* [svn-r14902] Merged fortran_1_8 branch changes r14505:14901 into the trunk. N...Scot Breitenfeld2008-04-301-3/+0
* [svn-r14789] Add handling for external link:Vailin Choi2008-04-021-0/+3
* [svn-r14284] Description:Quincey Koziol2007-11-241-0/+2
* [svn-r13745] Description:Quincey Koziol2007-05-101-1/+1
* [svn-r13477] Description:Quincey Koziol2007-03-081-0/+3
* [svn-r13465] Description:Quincey Koziol2007-03-061-1/+7
* [svn-r13427] Description:Quincey Koziol2007-02-271-14/+11
* [svn-r13413] Description:Quincey Koziol2007-02-271-0/+20
* [svn-r13253] Updated all C and C++ style source code files with the THG copyr...Albert Cheng2007-02-071-2/+3
* [svn-r13015] Description:Quincey Koziol2006-12-041-0/+2
* [svn-r12935] Added list-to-btree conversion, changed SOHM apis, and pushed SO...James Laird2006-11-171-4/+5
* [svn-r12902] Checkin of Shared Object Header Message work.James Laird2006-11-131-0/+6
* [svn-r12866] Description:Quincey Koziol2006-11-061-6/+2
* [svn-r12842] Description:Quincey Koziol2006-11-021-142/+39
* [svn-r12739] Added Direct I/O driver to VFD. It's only supported by SGI Alti...Raymond Lu2006-10-101-1/+1
* [svn-r12708] Description:Quincey Koziol2006-10-031-0/+7
* [svn-r12656] Description:Quincey Koziol2006-09-111-0/+5
* [svn-r12362] Purpose:Quincey Koziol2006-05-221-0/+4
* [svn-r12292] Purpose:Quincey Koziol2006-04-201-1/+0
* [svn-r12080] Purpose:Quincey Koziol2006-03-111-0/+30
* [svn-r12004] Purpose:Quincey Koziol2006-03-041-42/+45
* [svn-r11850] Purpose:James Laird2006-01-021-0/+4
* [svn-r11712] Purpose:Quincey Koziol2005-11-151-2/+5
* [svn-r11470] Purpose:John Mainzer2005-09-271-0/+1
* [svn-r11307] Purpose:Quincey Koziol2005-08-291-4/+4
* [svn-r11245] Purpose:Quincey Koziol2005-08-131-11/+11
* [svn-r11219] Purpose:Quincey Koziol2005-08-081-0/+3
* [svn-r11215] Purpose:Quincey Koziol2005-08-081-0/+3
* [svn-r11093] Purpose:Quincey Koziol2005-07-211-2/+2
* [svn-r11057] Purpose:Quincey Koziol2005-07-091-0/+4
* [svn-r10971] Purpose: Bug fixRaymond Lu2005-06-221-0/+6
* [svn-r10847] Purpose: Bug fixRaymond Lu2005-06-021-0/+6
* [svn-r10841] Purpose:Quincey Koziol2005-06-021-0/+9
* [svn-r10688] Purpose:John Mainzer2005-04-281-4/+4
* [svn-r9354] Purpose:Quincey Koziol2004-10-011-0/+17
* [svn-r8981] Purpose:Quincey Koziol2004-08-021-1/+0
* [svn-r8932] Purpose:Quincey Koziol2004-07-221-1/+1
* [svn-r8892] Purpose:Quincey Koziol2004-07-161-0/+3
* [svn-r8853] Purpose:Quincey Koziol2004-07-101-3/+3
* [svn-r8844] Purpose:Quincey Koziol2004-07-091-3/+4
* [svn-r8823] Purpose:Quincey Koziol2004-07-071-0/+4
?h=v3.4.0b1&id=59b7ff2d4371933052717329ec12c7203b8d1cad&id2=23828f6d8e4ff915a675eb8964ce0934b680bcce'>Doc/library/concurrency.rst1
-rw-r--r--Doc/library/concurrent.futures.rst3
-rw-r--r--Doc/library/contextlib.rst227
-rw-r--r--Doc/library/datatypes.rst1
-rw-r--r--Doc/library/datetime.rst10
-rw-r--r--Doc/library/dbm.rst38
-rw-r--r--Doc/library/debug.rst1
-rw-r--r--Doc/library/development.rst1
-rw-r--r--Doc/library/dis.rst250
-rw-r--r--Doc/library/distribution.rst14
-rw-r--r--Doc/library/doctest.rst18
-rw-r--r--Doc/library/email.contentmanager.rst427
-rw-r--r--Doc/library/email.message.rst87
-rw-r--r--Doc/library/email.policy.rst45
-rw-r--r--Doc/library/email.rst1
-rw-r--r--Doc/library/ensurepip.rst125
-rw-r--r--Doc/library/enum.rst688
-rw-r--r--Doc/library/exceptions.rst6
-rw-r--r--Doc/library/faulthandler.rst3
-rw-r--r--Doc/library/filecmp.rst32
-rw-r--r--Doc/library/filesys.rst1
-rw-r--r--Doc/library/formatter.rst4
-rw-r--r--Doc/library/functions.rst55
-rw-r--r--Doc/library/functools.rst177
-rw-r--r--Doc/library/gc.rst47
-rw-r--r--Doc/library/glob.rst15
-rw-r--r--Doc/library/gzip.rst15
-rw-r--r--Doc/library/hashlib.rst67
-rw-r--r--Doc/library/hmac.rst44
-rw-r--r--Doc/library/html.entities.rst1
-rw-r--r--Doc/library/html.parser.rst4
-rw-r--r--Doc/library/html.rst11
-rw-r--r--Doc/library/http.client.rst24
-rw-r--r--Doc/library/http.server.rst24
-rw-r--r--Doc/library/idle.rst345
-rw-r--r--Doc/library/imp.rst41
-rw-r--r--Doc/library/importlib.rst515
-rw-r--r--Doc/library/index.rst1
-rw-r--r--Doc/library/inspect.rst45
-rw-r--r--Doc/library/io.rst57
-rw-r--r--Doc/library/ipaddress.rst11
-rw-r--r--Doc/library/ipc.rst3
-rw-r--r--Doc/library/json.rst72
-rw-r--r--Doc/library/logging.config.rst51
-rw-r--r--Doc/library/logging.handlers.rst14
-rw-r--r--Doc/library/lzma.rst15
-rw-r--r--Doc/library/marshal.rst9
-rw-r--r--Doc/library/multiprocessing.rst301
-rw-r--r--Doc/library/netrc.rst2
-rw-r--r--Doc/library/nntplib.rst3
-rw-r--r--Doc/library/numeric.rst1
-rw-r--r--Doc/library/operator.rst11
-rw-r--r--Doc/library/os.path.rst18
-rw-r--r--Doc/library/os.rst245
-rw-r--r--Doc/library/pathlib.rst873
-rw-r--r--Doc/library/pdb.rst14
-rw-r--r--Doc/library/pickle.rst8
-rw-r--r--Doc/library/plistlib.rst181
-rw-r--r--Doc/library/poplib.rst27
-rw-r--r--Doc/library/pprint.rst254
-rw-r--r--Doc/library/pty.rst3
-rw-r--r--Doc/library/py_compile.rst16
-rw-r--r--Doc/library/python.rst1
-rw-r--r--Doc/library/readline.rst20
-rw-r--r--Doc/library/resource.rst67
-rw-r--r--Doc/library/rlcompleter.rst16
-rw-r--r--Doc/library/select.rst55
-rw-r--r--Doc/library/selectors.rst239
-rw-r--r--Doc/library/shelve.rst16
-rw-r--r--Doc/library/shutil.rst22
-rw-r--r--Doc/library/site.rst23
-rw-r--r--Doc/library/smtpd.rst19
-rw-r--r--Doc/library/smtplib.rst4
-rw-r--r--Doc/library/socket.rst93
-rw-r--r--Doc/library/spwd.rst10
-rw-r--r--Doc/library/sqlite3.rst14
-rw-r--r--Doc/library/ssl.rst364
-rw-r--r--Doc/library/stat.rst51
-rw-r--r--Doc/library/statistics.rst408
-rw-r--r--Doc/library/stdtypes.rst6
-rw-r--r--Doc/library/struct.rst20
-rw-r--r--Doc/library/subprocess.rst34
-rw-r--r--Doc/library/sunau.rst14
-rw-r--r--Doc/library/sys.rst48
-rw-r--r--Doc/library/sysconfig.rst2
-rw-r--r--Doc/library/test.rst22
-rw-r--r--Doc/library/textwrap.rst66
-rw-r--r--Doc/library/threading.rst9
-rw-r--r--Doc/library/timeit.rst2
-rw-r--r--Doc/library/tkinter.rst26
-rw-r--r--Doc/library/traceback.rst9
-rw-r--r--Doc/library/tracemalloc.rst602
-rw-r--r--Doc/library/types.rst32
-rw-r--r--Doc/library/unicodedata.rst8
-rw-r--r--Doc/library/unittest.mock-examples.rst14
-rw-r--r--Doc/library/unittest.mock.rst31
-rw-r--r--Doc/library/unittest.rst173
-rw-r--r--Doc/library/urllib.error.rst7
-rw-r--r--Doc/library/urllib.request.rst123
-rw-r--r--Doc/library/venv.rst21
-rw-r--r--Doc/library/wave.rst39
-rw-r--r--Doc/library/weakref.rst266
-rw-r--r--Doc/library/xml.etree.elementtree.rst169
-rw-r--r--Doc/library/zipfile.rst35
-rw-r--r--Doc/license.rst89
-rw-r--r--Doc/reference/compound_stmts.rst17
-rw-r--r--Doc/reference/datamodel.rst33
-rw-r--r--Doc/reference/import.rst453
-rw-r--r--Doc/tools/sphinxext/indexsidebar.html2
-rw-r--r--Doc/tools/sphinxext/pyspecific.py2
-rw-r--r--Doc/tools/sphinxext/susp-ignored.csv15
-rw-r--r--Doc/tutorial/interactive.rst153
-rw-r--r--Doc/tutorial/interpreter.rst14
-rw-r--r--Doc/tutorial/modules.rst35
-rw-r--r--Doc/tutorial/stdlib.rst2
-rw-r--r--Doc/tutorial/stdlib2.rst2
-rw-r--r--Doc/using/cmdline.rst81
-rw-r--r--Doc/using/mac.rst6
-rw-r--r--Doc/using/venv-create.inc39
-rw-r--r--Doc/whatsnew/3.4.rst999
-rw-r--r--Doc/whatsnew/index.rst1
-rw-r--r--Include/Python-ast.h31
-rw-r--r--Include/Python.h1
-rw-r--r--Include/abstract.h48
-rw-r--r--Include/asdl.h15
-rw-r--r--Include/ast.h5
-rw-r--r--Include/bytearrayobject.h11
-rw-r--r--Include/bytesobject.h10
-rw-r--r--Include/codecs.h27
-rw-r--r--Include/compile.h15
-rw-r--r--Include/dictobject.h6
-rw-r--r--Include/fileobject.h5
-rw-r--r--Include/fileutils.h15
-rw-r--r--Include/frameobject.h5
-rw-r--r--Include/genobject.h2
-rw-r--r--Include/grammar.h8
-rw-r--r--Include/import.h24
-rw-r--r--Include/longobject.h19
-rw-r--r--Include/marshal.h5
-rw-r--r--Include/modsupport.h2
-rw-r--r--Include/object.h106
-rw-r--r--Include/objimpl.h104
-rw-r--r--Include/opcode.h2
-rw-r--r--Include/osdefs.h8
-rw-r--r--Include/parsetok.h49
-rw-r--r--Include/patchlevel.h10
-rw-r--r--Include/pydebug.h1
-rw-r--r--Include/pyerrors.h30
-rw-r--r--Include/pyhash.h147
-rw-r--r--Include/pymem.h95
-rw-r--r--Include/pyport.h64
-rw-r--r--Include/pystate.h46
-rw-r--r--Include/pythonrun.h45
-rw-r--r--Include/setobject.h9
-rw-r--r--Include/sliceobject.h3
-rw-r--r--Include/structseq.h2
-rw-r--r--Include/symtable.h10
-rw-r--r--Include/sysmodule.h3
-rw-r--r--Include/token.h2
-rw-r--r--Include/unicodeobject.h86
-rw-r--r--Include/warnings.h13
-rw-r--r--Lib/_bootlocale.py34
-rw-r--r--Lib/_collections_abc.py734
-rw-r--r--Lib/_dummy_thread.py4
-rw-r--r--Lib/_osx_support.py4
-rw-r--r--Lib/_pyio.py199
-rw-r--r--Lib/_sitebuiltins.py99
-rw-r--r--Lib/_strptime.py2
-rw-r--r--Lib/abc.py20
-rw-r--r--Lib/aifc.py41
-rw-r--r--Lib/argparse.py31
-rw-r--r--Lib/ast.py6
-rw-r--r--Lib/asynchat.py8
-rw-r--r--Lib/asyncio/__init__.py41
-rw-r--r--Lib/asyncio/base_events.py644
-rw-r--r--Lib/asyncio/base_subprocess.py166
-rw-r--r--Lib/asyncio/constants.py7
-rw-r--r--Lib/asyncio/events.py438
-rw-r--r--Lib/asyncio/futures.py343
-rw-r--r--Lib/asyncio/locks.py431
-rw-r--r--Lib/asyncio/log.py7
-rw-r--r--Lib/asyncio/proactor_events.py360
-rw-r--r--Lib/asyncio/protocols.py128
-rw-r--r--Lib/asyncio/queues.py284
-rw-r--r--Lib/asyncio/selector_events.py837
-rw-r--r--Lib/asyncio/streams.py398
-rw-r--r--Lib/asyncio/tasks.py637
-rw-r--r--Lib/asyncio/test_utils.py262
-rw-r--r--Lib/asyncio/transports.py211
-rw-r--r--Lib/asyncio/unix_events.py744
-rw-r--r--Lib/asyncio/windows_events.py468
-rw-r--r--Lib/asyncio/windows_utils.py192
-rw-r--r--Lib/asyncore.py29
-rwxr-xr-xLib/base64.py346
-rw-r--r--Lib/bdb.py36
-rw-r--r--Lib/bz2.py50
-rwxr-xr-xLib/cProfile.py53
-rwxr-xr-xLib/cgi.py13
-rw-r--r--Lib/chunk.py6
-rw-r--r--Lib/code.py2
-rw-r--r--Lib/codecs.py14
-rw-r--r--Lib/collections/__init__.py24
-rw-r--r--Lib/collections/abc.py730
-rw-r--r--Lib/colorsys.py18
-rw-r--r--Lib/compileall.py19
-rw-r--r--Lib/concurrent/futures/_base.py3
-rw-r--r--Lib/concurrent/futures/process.py11
-rw-r--r--Lib/concurrent/futures/thread.py2
-rw-r--r--Lib/configparser.py2
-rw-r--r--Lib/contextlib.py71
-rw-r--r--Lib/ctypes/__init__.py26
-rw-r--r--Lib/ctypes/test/__init__.py2
-rw-r--r--Lib/ctypes/test/test_checkretval.py2
-rw-r--r--Lib/ctypes/test/test_internals.py19
-rw-r--r--Lib/ctypes/test/test_macholib.py56
-rw-r--r--Lib/ctypes/test/test_win32.py2
-rw-r--r--Lib/ctypes/util.py5
-rw-r--r--Lib/datetime.py395
-rw-r--r--Lib/dbm/__init__.py16
-rw-r--r--Lib/dbm/dumb.py16
-rw-r--r--Lib/decimal.py4
-rw-r--r--Lib/difflib.py35
-rw-r--r--Lib/dis.py396
-rw-r--r--Lib/distutils/__init__.py2
-rw-r--r--Lib/distutils/archive_util.py71
-rw-r--r--Lib/distutils/ccompiler.py5
-rw-r--r--Lib/distutils/cmd.py6
-rw-r--r--Lib/distutils/command/bdist.py16
-rw-r--r--Lib/distutils/command/bdist_dumb.py19
-rw-r--r--Lib/distutils/command/build_ext.py26
-rw-r--r--Lib/distutils/command/build_py.py10
-rw-r--r--Lib/distutils/command/build_scripts.py2
-rw-r--r--Lib/distutils/command/install.py15
-rw-r--r--Lib/distutils/command/install_lib.py6
-rw-r--r--Lib/distutils/command/sdist.py9
-rw-r--r--Lib/distutils/command/upload.py2
-rw-r--r--Lib/distutils/config.py11
-rw-r--r--Lib/distutils/core.py7
-rw-r--r--Lib/distutils/cygwinccompiler.py28
-rw-r--r--Lib/distutils/dir_util.py4
-rw-r--r--Lib/distutils/dist.py35
-rw-r--r--Lib/distutils/emxccompiler.py315
-rw-r--r--Lib/distutils/errors.py4
-rw-r--r--Lib/distutils/file_util.py16
-rw-r--r--Lib/distutils/msvc9compiler.py2
-rw-r--r--Lib/distutils/spawn.py24
-rw-r--r--Lib/distutils/sysconfig.py38
-rw-r--r--Lib/distutils/tests/test_archive_util.py61
-rw-r--r--Lib/distutils/tests/test_bdist_dumb.py5
-rw-r--r--Lib/distutils/tests/test_build_py.py10
-rw-r--r--Lib/distutils/tests/test_config.py4
-rw-r--r--Lib/distutils/tests/test_dist.py45
-rw-r--r--Lib/distutils/tests/test_install.py6
-rw-r--r--Lib/distutils/tests/test_install_lib.py8
-rw-r--r--Lib/distutils/tests/test_sdist.py56
-rw-r--r--Lib/distutils/tests/test_sysconfig.py45
-rw-r--r--Lib/distutils/tests/test_upload.py13
-rw-r--r--Lib/distutils/tests/test_util.py2
-rw-r--r--Lib/distutils/util.py17
-rw-r--r--Lib/doctest.py59
-rw-r--r--Lib/email/_header_value_parser.py37
-rw-r--r--Lib/email/contentmanager.py249
-rw-r--r--Lib/email/feedparser.py27
-rw-r--r--Lib/email/iterators.py6
-rw-r--r--Lib/email/message.py260
-rw-r--r--Lib/email/parser.py3
-rw-r--r--Lib/email/policy.py13
-rw-r--r--Lib/email/utils.py24
-rw-r--r--Lib/encodings/aliases.py42
-rw-r--r--Lib/encodings/base64_codec.py1
-rw-r--r--Lib/encodings/bz2_codec.py1
-rw-r--r--Lib/encodings/cp037.py1
-rw-r--r--Lib/encodings/cp273.py307
-rw-r--r--Lib/encodings/cp500.py1
-rw-r--r--Lib/encodings/cp866.py52
-rw-r--r--Lib/encodings/hex_codec.py1
-rw-r--r--Lib/encodings/iso8859_1.py1
-rw-r--r--Lib/encodings/quopri_codec.py1
-rwxr-xr-xLib/encodings/rot_13.py1
-rw-r--r--Lib/encodings/uu_codec.py1
-rw-r--r--Lib/encodings/zlib_codec.py1
-rw-r--r--Lib/ensurepip/__init__.py92
-rw-r--r--Lib/ensurepip/__main__.py66
-rw-r--r--Lib/ensurepip/_bundled/pip-1.5.rc1-py2.py3-none-any.whlbin0 -> 931600 bytes-rw-r--r--Lib/ensurepip/_bundled/setuptools-1.3.2-py2.py3-none-any.whlbin0 -> 549754 bytes-rw-r--r--Lib/enum.py504
-rw-r--r--Lib/filecmp.py25
-rw-r--r--Lib/fileinput.py12
-rw-r--r--Lib/formatter.py3
-rw-r--r--Lib/fractions.py14
-rw-r--r--Lib/ftplib.py168
-rw-r--r--Lib/functools.py421
-rw-r--r--Lib/genericpath.py34
-rw-r--r--Lib/getpass.py99
-rw-r--r--Lib/gettext.py14
-rw-r--r--Lib/glob.py21
-rw-r--r--Lib/gzip.py115
-rw-r--r--Lib/hashlib.py107
-rw-r--r--Lib/hmac.py31
-rw-r--r--Lib/html/__init__.py123
-rw-r--r--Lib/html/parser.py55
-rw-r--r--Lib/http/client.py44
-rw-r--r--Lib/http/cookiejar.py31
-rw-r--r--Lib/http/server.py42
-rw-r--r--Lib/idlelib/EditorWindow.py4
-rw-r--r--Lib/idlelib/FileList.py2
-rw-r--r--Lib/idlelib/GrepDialog.py1
-rw-r--r--Lib/idlelib/IOBinding.py2
-rw-r--r--Lib/idlelib/NEWS.txt6
-rw-r--r--Lib/idlelib/PathBrowser.py2
-rw-r--r--Lib/idlelib/PyShell.py12
-rw-r--r--Lib/idlelib/TreeWidget.py4
-rw-r--r--Lib/idlelib/configHandler.py4
-rw-r--r--Lib/idlelib/help.txt556
-rw-r--r--Lib/idlelib/idlever.py2
-rw-r--r--Lib/idlelib/rpc.py8
-rw-r--r--Lib/idlelib/run.py4
-rw-r--r--Lib/imaplib.py12
-rw-r--r--Lib/imghdr.py2
-rw-r--r--Lib/imp.py220
-rw-r--r--Lib/importlib/__init__.py96
-rw-r--r--Lib/importlib/_bootstrap.py1683
-rw-r--r--Lib/importlib/abc.py344
-rw-r--r--Lib/importlib/machinery.py1
-rw-r--r--Lib/importlib/util.py105
-rw-r--r--Lib/inspect.py284
-rw-r--r--Lib/io.py2
-rw-r--r--Lib/ipaddress.py81
-rw-r--r--Lib/json/__init__.py30
-rw-r--r--Lib/json/decoder.py28
-rw-r--r--Lib/json/encoder.py56
-rw-r--r--Lib/json/scanner.py4
-rw-r--r--Lib/json/tool.py3
-rwxr-xr-xLib/keyword.py23
-rw-r--r--Lib/lib2to3/btm_utils.py6
-rw-r--r--Lib/lib2to3/fixer_util.py23
-rw-r--r--Lib/lib2to3/fixes/fix_intern.py21
-rw-r--r--Lib/lib2to3/fixes/fix_reload.py28
-rw-r--r--Lib/lib2to3/main.py4
-rw-r--r--Lib/lib2to3/pgen2/conv.py4
-rw-r--r--Lib/lib2to3/pgen2/driver.py2
-rw-r--r--Lib/lib2to3/pgen2/grammar.py10
-rw-r--r--Lib/lib2to3/pytree.py9
-rw-r--r--Lib/lib2to3/refactor.py6
-rwxr-xr-xLib/lib2to3/tests/pytree_idempotency.py2
-rw-r--r--Lib/lib2to3/tests/test_fixers.py59
-rw-r--r--Lib/linecache.py8
-rw-r--r--Lib/locale.py21
-rw-r--r--Lib/logging/__init__.py78
-rw-r--r--Lib/logging/config.py76
-rw-r--r--Lib/logging/handlers.py124
-rw-r--r--Lib/lzma.py137
-rw-r--r--Lib/macpath.py2
-rw-r--r--Lib/mailbox.py83
-rw-r--r--Lib/mailcap.py2
-rw-r--r--Lib/mimetypes.py2
-rw-r--r--Lib/modulefinder.py7
-rw-r--r--Lib/multiprocessing/__init__.py251
-rw-r--r--Lib/multiprocessing/connection.py147
-rw-r--r--Lib/multiprocessing/context.py348
-rw-r--r--Lib/multiprocessing/dummy/__init__.py31
-rw-r--r--Lib/multiprocessing/dummy/connection.py27
-rw-r--r--Lib/multiprocessing/forking.py474
-rw-r--r--Lib/multiprocessing/forkserver.py267
-rw-r--r--Lib/multiprocessing/heap.py56
-rw-r--r--Lib/multiprocessing/managers.py51
-rw-r--r--Lib/multiprocessing/pool.py133
-rw-r--r--Lib/multiprocessing/popen_fork.py83
-rw-r--r--Lib/multiprocessing/popen_forkserver.py69
-rw-r--r--Lib/multiprocessing/popen_spawn_posix.py72
-rw-r--r--Lib/multiprocessing/popen_spawn_win32.py99
-rw-r--r--Lib/multiprocessing/process.py83
-rw-r--r--Lib/multiprocessing/queues.py106
-rw-r--r--Lib/multiprocessing/reduction.py363
-rw-r--r--Lib/multiprocessing/resource_sharer.py158
-rw-r--r--Lib/multiprocessing/semaphore_tracker.py145
-rw-r--r--Lib/multiprocessing/sharedctypes.py40
-rw-r--r--Lib/multiprocessing/spawn.py258
-rw-r--r--Lib/multiprocessing/synchronize.py112
-rw-r--r--Lib/multiprocessing/util.py52
-rw-r--r--Lib/netrc.py5
-rw-r--r--Lib/nntplib.py4
-rw-r--r--Lib/ntpath.py60
-rw-r--r--Lib/nturl2path.py4
-rw-r--r--Lib/opcode.py4
-rw-r--r--Lib/operator.py412
-rw-r--r--Lib/os.py95
-rw-r--r--Lib/os2emxpath.py158
-rw-r--r--Lib/pathlib.py1284
-rwxr-xr-xLib/pdb.py47
-rw-r--r--Lib/pickle.py240
-rw-r--r--Lib/pickletools.py225
-rw-r--r--Lib/pkgutil.py107
-rw-r--r--Lib/plat-os2emx/IN.py82
-rw-r--r--Lib/plat-os2emx/SOCKET.py106
-rw-r--r--Lib/plat-os2emx/_emx_link.py79
-rw-r--r--Lib/plat-os2emx/grp.py182
-rw-r--r--Lib/plat-os2emx/pwd.py208
-rwxr-xr-xLib/plat-os2emx/regen7
-rwxr-xr-xLib/platform.py91
-rw-r--r--Lib/plistlib.py1098
-rw-r--r--Lib/poplib.py91
-rw-r--r--Lib/posixpath.py57
-rw-r--r--Lib/pprint.py124
-rwxr-xr-xLib/profile.py63
-rw-r--r--Lib/pstats.py12
-rw-r--r--Lib/pty.py22
-rw-r--r--Lib/py_compile.py77
-rwxr-xr-xLib/pydoc.py102
-rw-r--r--Lib/pydoc_data/topics.py18
-rwxr-xr-xLib/quopri.py2
-rw-r--r--Lib/random.py16
-rw-r--r--Lib/re.py1
-rw-r--r--Lib/rlcompleter.py5
-rw-r--r--Lib/runpy.py11
-rw-r--r--Lib/sched.py11
-rw-r--r--Lib/selectors.py431
-rw-r--r--Lib/shelve.py9
-rw-r--r--Lib/shutil.py39
-rw-r--r--Lib/site.py202
-rwxr-xr-xLib/smtpd.py21
-rw-r--r--Lib/smtplib.py12
-rw-r--r--Lib/sndhdr.py17
-rw-r--r--Lib/socket.py116
-rw-r--r--Lib/socketserver.py8
-rw-r--r--Lib/sqlite3/test/dbapi.py18
-rw-r--r--Lib/sre_compile.py270
-rw-r--r--Lib/sre_constants.py6
-rw-r--r--Lib/sre_parse.py75
-rw-r--r--Lib/ssl.py283
-rw-r--r--Lib/stat.py6
-rw-r--r--Lib/statistics.py610
-rw-r--r--Lib/struct.py1
-rw-r--r--Lib/subprocess.py355
-rw-r--r--Lib/sunau.py40
-rwxr-xr-xLib/symbol.py4
-rw-r--r--Lib/symtable.py3
-rw-r--r--Lib/sysconfig.py46
-rwxr-xr-xLib/tabnanny.py2
-rw-r--r--Lib/tarfile.py53
-rw-r--r--Lib/telnetlib.py273
-rw-r--r--Lib/tempfile.py35
-rw-r--r--Lib/test/__main__.py14
-rw-r--r--Lib/test/_test_multiprocessing.py (renamed from Lib/test/test_multiprocessing.py)588
-rw-r--r--Lib/test/audiodata/pluck-pcm24.aubin0 -> 19866 bytes-rw-r--r--Lib/test/audiotests.py147
-rw-r--r--Lib/test/badsyntax_future10.py3
-rw-r--r--Lib/test/buffer_tests.py16
-rw-r--r--Lib/test/bytecode_helper.py41
-rw-r--r--Lib/test/datetimetester.py4
-rw-r--r--Lib/test/final_a.py19
-rw-r--r--Lib/test/final_b.py19
-rw-r--r--Lib/test/fork_wait.py2
-rw-r--r--Lib/test/keycert3.pem73
-rw-r--r--Lib/test/keycert4.pem73
-rw-r--r--Lib/test/leakers/test_gestalt.py14
-rw-r--r--Lib/test/lock_tests.py20
-rw-r--r--Lib/test/make_ssl_certs.py118
-rw-r--r--Lib/test/mock_socket.py8
-rw-r--r--Lib/test/mp_fork_bomb.py5
-rw-r--r--Lib/test/multibytecodec_support.py2
-rw-r--r--Lib/test/pickletester.py53
-rw-r--r--Lib/test/pycacert.pem78
-rw-r--r--Lib/test/pycakey.pem28
-rwxr-xr-xLib/test/regrtest.py1153
-rw-r--r--Lib/test/revocation.crl11
-rw-r--r--Lib/test/script_helper.py36
-rw-r--r--Lib/test/sortperf.py6
-rw-r--r--Lib/test/ssl_servers.py10
-rw-r--r--Lib/test/string_tests.py27
-rw-r--r--Lib/test/subprocessdata/fd_status.py26
-rw-r--r--Lib/test/support/__init__.py252
-rw-r--r--Lib/test/test___all__.py30
-rw-r--r--Lib/test/test_abc.py16
-rw-r--r--Lib/test/test_aifc.py65
-rw-r--r--Lib/test/test_argparse.py36
-rwxr-xr-xLib/test/test_array.py4
-rw-r--r--Lib/test/test_ast.py98
-rw-r--r--Lib/test/test_asynchat.py35
-rw-r--r--Lib/test/test_asyncio/__init__.py31
-rw-r--r--Lib/test/test_asyncio/__main__.py5
-rw-r--r--Lib/test/test_asyncio/echo.py8
-rw-r--r--Lib/test/test_asyncio/echo2.py6
-rw-r--r--Lib/test/test_asyncio/echo3.py11
-rw-r--r--Lib/test/test_asyncio/sample.crt14
-rw-r--r--Lib/test/test_asyncio/sample.key15
-rw-r--r--Lib/test/test_asyncio/test_base_events.py684
-rw-r--r--Lib/test/test_asyncio/test_events.py1655
-rw-r--r--Lib/test/test_asyncio/test_futures.py347
-rw-r--r--Lib/test/test_asyncio/test_locks.py838
-rw-r--r--Lib/test/test_asyncio/test_proactor_events.py484
-rw-r--r--Lib/test/test_asyncio/test_queues.py470
-rw-r--r--Lib/test/test_asyncio/test_selector_events.py1554
-rw-r--r--Lib/test/test_asyncio/test_streams.py430
-rw-r--r--Lib/test/test_asyncio/test_tasks.py1518
-rw-r--r--Lib/test/test_asyncio/test_transports.py59
-rw-r--r--Lib/test/test_asyncio/test_unix_events.py1488
-rw-r--r--Lib/test/test_asyncio/test_windows_events.py139
-rw-r--r--Lib/test/test_asyncio/test_windows_utils.py141
-rw-r--r--Lib/test/test_asyncio/tests.txt13
-rw-r--r--Lib/test/test_asyncore.py25
-rw-r--r--Lib/test/test_atexit.py42
-rw-r--r--Lib/test/test_audioop.py221
-rw-r--r--Lib/test/test_base64.py362
-rw-r--r--Lib/test/test_bisect.py10
-rw-r--r--Lib/test/test_buffer.py6
-rw-r--r--Lib/test/test_builtin.py73
-rw-r--r--Lib/test/test_bytes.py34
-rw-r--r--Lib/test/test_bz2.py218
-rw-r--r--Lib/test/test_capi.py120
-rw-r--r--Lib/test/test_cmd_line.py100
-rw-r--r--Lib/test/test_cmd_line_script.py10
-rw-r--r--Lib/test/test_code_module.py14
-rw-r--r--Lib/test/test_codeccallbacks.py4
-rw-r--r--Lib/test/test_codecs.py383
-rw-r--r--Lib/test/test_coding.py63
-rw-r--r--Lib/test/test_collections.py126
-rw-r--r--Lib/test/test_colorsys.py32
-rw-r--r--Lib/test/test_compileall.py52
-rw-r--r--Lib/test/test_complex.py3
-rw-r--r--Lib/test/test_concurrent_futures.py22
-rw-r--r--Lib/test/test_contextlib.py141
-rw-r--r--Lib/test/test_cprofile.py2
-rw-r--r--Lib/test/test_crypt.py6
-rw-r--r--Lib/test/test_csv.py21
-rw-r--r--Lib/test/test_dbm.py2
-rw-r--r--Lib/test/test_dbm_dumb.py13
-rw-r--r--Lib/test/test_dbm_gnu.py11
-rw-r--r--Lib/test/test_dbm_ndbm.py13
-rw-r--r--Lib/test/test_decimal.py1
-rw-r--r--Lib/test/test_deque.py4
-rw-r--r--Lib/test/test_descr.py22
-rw-r--r--Lib/test/test_devpoll.py38
-rw-r--r--Lib/test/test_dis.py475
-rw-r--r--Lib/test/test_doctest.py281
-rw-r--r--Lib/test/test_dynamicclassattribute.py304
-rw-r--r--Lib/test/test_email/__init__.py26
-rw-r--r--Lib/test/test_email/test_contentmanager.py796
-rw-r--r--Lib/test/test_email/test_email.py33
-rw-r--r--Lib/test/test_email/test_headerregistry.py14
-rw-r--r--Lib/test/test_email/test_message.py742
-rw-r--r--Lib/test/test_email/test_policy.py1
-rw-r--r--Lib/test/test_email/torture_test.py2
-rw-r--r--Lib/test/test_ensurepip.py128
-rw-r--r--Lib/test/test_enum.py1325
-rw-r--r--Lib/test/test_enumerate.py10
-rw-r--r--Lib/test/test_epoll.py54
-rw-r--r--Lib/test/test_exceptions.py28
-rw-r--r--Lib/test/test_faulthandler.py77
-rw-r--r--Lib/test/test_fcntl.py19
-rw-r--r--Lib/test/test_file.py6
-rw-r--r--Lib/test/test_filecmp.py23
-rw-r--r--Lib/test/test_fileinput.py22
-rw-r--r--Lib/test/test_fileio.py20
-rw-r--r--Lib/test/test_finalization.py513
-rw-r--r--Lib/test/test_float.py1
-rw-r--r--Lib/test/test_fork1.py2
-rw-r--r--Lib/test/test_format.py30
-rw-r--r--Lib/test/test_fractions.py32
-rw-r--r--Lib/test/test_frame.py116
-rw-r--r--Lib/test/test_frozen.py79
-rw-r--r--Lib/test/test_ftplib.py54
-rw-r--r--Lib/test/test_funcattrs.py7
-rw-r--r--Lib/test/test_functools.py921
-rw-r--r--Lib/test/test_future.py8
-rw-r--r--Lib/test/test_gc.py80
-rw-r--r--Lib/test/test_gdb.py28
-rw-r--r--Lib/test/test_generators.py74
-rw-r--r--Lib/test/test_genericpath.py82
-rw-r--r--Lib/test/test_getargs2.py88
-rw-r--r--Lib/test/test_getpass.py155
-rw-r--r--Lib/test/test_glob.py22
-rw-r--r--[-rwxr-xr-x]Lib/test/test_gzip.py51
-rw-r--r--Lib/test/test_hash.py146
-rw-r--r--Lib/test/test_hashlib.py267
-rw-r--r--Lib/test/test_hmac.py64
-rw-r--r--Lib/test/test_html.py86
-rw-r--r--Lib/test/test_htmlparser.py32
-rw-r--r--Lib/test/test_httplib.py30
-rw-r--r--Lib/test/test_httpservers.py21
-rw-r--r--Lib/test/test_imaplib.py2
-rw-r--r--Lib/test/test_imp.py87
-rw-r--r--Lib/test/test_import.py102
-rw-r--r--Lib/test/test_importhooks.py250
-rw-r--r--Lib/test/test_importlib/__main__.py13
-rw-r--r--Lib/test/test_importlib/abc.py9
-rw-r--r--Lib/test/test_importlib/builtin/test_finder.py60
-rw-r--r--Lib/test/test_importlib/builtin/test_loader.py119
-rw-r--r--Lib/test/test_importlib/extension/test_case_sensitivity.py29
-rw-r--r--Lib/test/test_importlib/extension/test_finder.py20
-rw-r--r--Lib/test/test_importlib/extension/test_loader.py86
-rw-r--r--Lib/test/test_importlib/extension/test_path_hook.py20
-rw-r--r--Lib/test/test_importlib/extension/util.py1
-rw-r--r--Lib/test/test_importlib/frozen/test_finder.py49
-rw-r--r--Lib/test/test_importlib/frozen/test_loader.py128
-rw-r--r--Lib/test/test_importlib/import_/test___loader__.py48
-rw-r--r--Lib/test/test_importlib/import_/test___package__.py33
-rw-r--r--Lib/test/test_importlib/import_/test_api.py37
-rw-r--r--Lib/test/test_importlib/import_/test_caching.py30
-rw-r--r--Lib/test/test_importlib/import_/test_fromlist.py34
-rw-r--r--Lib/test/test_importlib/import_/test_meta_path.py28
-rw-r--r--Lib/test/test_importlib/import_/test_packages.py30
-rw-r--r--Lib/test/test_importlib/import_/test_path.py40
-rw-r--r--Lib/test/test_importlib/import_/test_relative_imports.py51
-rw-r--r--Lib/test/test_importlib/import_/util.py20
-rw-r--r--Lib/test/test_importlib/source/test_abc_loader.py906
-rw-r--r--Lib/test/test_importlib/source/test_case_sensitivity.py30
-rw-r--r--Lib/test/test_importlib/source/test_file_loader.py110
-rw-r--r--Lib/test/test_importlib/source/test_finder.py29
-rw-r--r--Lib/test/test_importlib/source/test_path_hook.py18
-rw-r--r--Lib/test/test_importlib/source/test_source_encoding.py20
-rw-r--r--Lib/test/test_importlib/source/util.py1
-rw-r--r--Lib/test/test_importlib/test_abc.py791
-rw-r--r--Lib/test/test_importlib/test_api.py395
-rw-r--r--Lib/test/test_importlib/test_locks.py71
-rw-r--r--Lib/test/test_importlib/test_spec.py968
-rw-r--r--Lib/test/test_importlib/test_util.py295
-rw-r--r--Lib/test/test_importlib/util.py25
-rw-r--r--Lib/test/test_inspect.py252
-rw-r--r--Lib/test/test_int.py41
-rw-r--r--Lib/test/test_io.py116
-rw-r--r--Lib/test/test_ioctl.py6
-rw-r--r--Lib/test/test_ipaddress.py12
-rw-r--r--Lib/test/test_iterlen.py62
-rw-r--r--Lib/test/test_itertools.py5
-rw-r--r--Lib/test/test_json/test_decode.py23
-rw-r--r--Lib/test/test_json/test_enum.py120
-rw-r--r--Lib/test/test_json/test_fail.py89
-rw-r--r--Lib/test/test_json/test_indent.py4
-rw-r--r--Lib/test/test_keyword.py138
-rw-r--r--Lib/test/test_keywordonlyarg.py12
-rw-r--r--Lib/test/test_kqueue.py29
-rw-r--r--Lib/test/test_largefile.py2
-rw-r--r--Lib/test/test_logging.py439
-rw-r--r--Lib/test/test_long.py2
-rw-r--r--Lib/test/test_lzma.py46
-rw-r--r--Lib/test/test_mailbox.py2
-rw-r--r--Lib/test/test_marshal.py163
-rw-r--r--Lib/test/test_memoryio.py12
-rw-r--r--Lib/test/test_memoryview.py9
-rw-r--r--Lib/test/test_mmap.py26
-rw-r--r--Lib/test/test_module.py64
-rw-r--r--Lib/test/test_multibytecodec.py73
-rw-r--r--Lib/test/test_multiprocessing_fork.py7
-rw-r--r--Lib/test/test_multiprocessing_forkserver.py7
-rw-r--r--Lib/test/test_multiprocessing_spawn.py7
-rw-r--r--Lib/test/test_namespace_pkgs.py13
-rw-r--r--Lib/test/test_nntplib.py2
-rw-r--r--Lib/test/test_normalization.py2
-rw-r--r--Lib/test/test_ntpath.py34
-rw-r--r--Lib/test/test_openpty.py2
-rw-r--r--Lib/test/test_operator.py122
-rw-r--r--Lib/test/test_optparse.py8
-rw-r--r--Lib/test/test_os.py329
-rw-r--r--Lib/test/test_ossaudiodev.py4
-rwxr-xr-xLib/test/test_pathlib.py1668
-rw-r--r--Lib/test/test_pdb.py348
-rw-r--r--Lib/test/test_peepholer.py295
-rw-r--r--Lib/test/test_pep247.py12
-rw-r--r--Lib/test/test_pep277.py4
-rw-r--r--Lib/test/test_pep352.py5
-rw-r--r--Lib/test/test_pkg.py20
-rw-r--r--Lib/test/test_pkgimport.py2
-rw-r--r--Lib/test/test_pkgutil.py22
-rw-r--r--Lib/test/test_plistlib.py497
-rw-r--r--Lib/test/test_poll.py20
-rw-r--r--Lib/test/test_poplib.py178
-rw-r--r--Lib/test/test_posix.py72
-rw-r--r--Lib/test/test_posixpath.py62
-rw-r--r--Lib/test/test_pprint.py50
-rw-r--r--Lib/test/test_print.py74
-rw-r--r--Lib/test/test_profile.py29
-rw-r--r--Lib/test/test_pty.py2
-rw-r--r--Lib/test/test_py_compile.py46
-rw-r--r--Lib/test/test_pyclbr.py4
-rw-r--r--Lib/test/test_pydoc.py221
-rw-r--r--Lib/test/test_random.py207
-rw-r--r--Lib/test/test_range.py2
-rw-r--r--Lib/test/test_re.py147
-rw-r--r--Lib/test/test_regrtest.py275
-rw-r--r--Lib/test/test_reprlib.py6
-rw-r--r--Lib/test/test_resource.py26
-rw-r--r--Lib/test/test_runpy.py14
-rw-r--r--Lib/test/test_sched.py5
-rw-r--r--Lib/test/test_scope.py17
-rw-r--r--Lib/test/test_select.py4
-rw-r--r--Lib/test/test_selectors.py430
-rw-r--r--Lib/test/test_set.py2
-rw-r--r--Lib/test/test_shelve.py13
-rw-r--r--Lib/test/test_shutil.py45
-rw-r--r--Lib/test/test_signal.py60
-rw-r--r--Lib/test/test_site.py39
-rw-r--r--Lib/test/test_slice.py114
-rw-r--r--Lib/test/test_smtplib.py23
-rw-r--r--Lib/test/test_sndhdr.py2
-rw-r--r--Lib/test/test_socket.py392
-rw-r--r--Lib/test/test_socketserver.py20
-rw-r--r--Lib/test/test_source_encoding.py (renamed from Lib/test/test_pep263.py)67
-rw-r--r--Lib/test/test_ssl.py703
-rw-r--r--Lib/test/test_stat.py61
-rw-r--r--Lib/test/test_statistics.py1534
-rw-r--r--Lib/test/test_strftime.py27
-rw-r--r--Lib/test/test_struct.py89
-rw-r--r--Lib/test/test_structseq.py2
-rw-r--r--Lib/test/test_subprocess.py206
-rw-r--r--Lib/test/test_sunau.py28
-rw-r--r--Lib/test/test_sundry.py39
-rw-r--r--Lib/test/test_super.py33
-rw-r--r--Lib/test/test_support.py1
-rw-r--r--Lib/test/test_syntax.py4
-rw-r--r--Lib/test/test_sys.py107
-rw-r--r--Lib/test/test_sysconfig.py63
-rw-r--r--Lib/test/test_tarfile.py10
-rw-r--r--Lib/test/test_tcl.py34
-rw-r--r--Lib/test/test_telnetlib.py101
-rw-r--r--Lib/test/test_tempfile.py29
-rw-r--r--Lib/test/test_textwrap.py159
-rw-r--r--Lib/test/test_thread.py2
-rw-r--r--Lib/test/test_threaded_import.py2
-rw-r--r--Lib/test/test_threading.py433
-rw-r--r--Lib/test/test_threadsignals.py2
-rw-r--r--Lib/test/test_time.py9
-rw-r--r--Lib/test/test_timeout.py2
-rw-r--r--Lib/test/test_traceback.py82
-rw-r--r--Lib/test/test_tracemalloc.py794
-rw-r--r--Lib/test/test_types.py37
-rw-r--r--Lib/test/test_ucn.py2
-rw-r--r--Lib/test/test_unicode.py373
-rw-r--r--Lib/test/test_unicodedata.py4
-rw-r--r--Lib/test/test_urllib.py103
-rw-r--r--Lib/test/test_urllib2.py137
-rw-r--r--Lib/test/test_urllib2_localnet.py27
-rw-r--r--Lib/test/test_urllib2net.py43
-rw-r--r--Lib/test/test_urllibnet.py19
-rwxr-xr-xLib/test/test_urlparse.py8
-rw-r--r--Lib/test/test_userdict.py9
-rw-r--r--Lib/test/test_venv.py120
-rw-r--r--Lib/test/test_wait3.py12
-rw-r--r--Lib/test/test_warnings.py32
-rw-r--r--Lib/test/test_weakref.py316
-rw-r--r--Lib/test/test_winreg.py28
-rw-r--r--Lib/test/test_winsound.py2
-rw-r--r--Lib/test/test_xml_etree.py200
-rw-r--r--Lib/test/test_xml_etree_c.py9
-rw-r--r--Lib/test/test_xmlrpc.py74
-rw-r--r--Lib/test/test_xmlrpc_net.py31
-rw-r--r--Lib/test/test_zipfile.py225
-rw-r--r--Lib/test/test_zipfile64.py4
-rw-r--r--Lib/test/test_zipimport.py37
-rw-r--r--Lib/test/test_zipimport_support.py6
-rw-r--r--Lib/test/tf_inherit_check.py2
-rw-r--r--Lib/textwrap.py73
-rw-r--r--Lib/threading.py188
-rw-r--r--Lib/timeit.py46
-rw-r--r--Lib/tkinter/__init__.py45
-rw-r--r--Lib/tkinter/filedialog.py4
-rw-r--r--Lib/tkinter/test/test_tkinter/test_widgets.py4
-rwxr-xr-xLib/token.py6
-rw-r--r--Lib/tokenize.py2
-rw-r--r--Lib/trace.py10
-rw-r--r--Lib/traceback.py259
-rw-r--r--Lib/tracemalloc.py464
-rw-r--r--Lib/turtle.py22
-rwxr-xr-xLib/turtledemo/__main__.py2
-rw-r--r--Lib/types.py57
-rw-r--r--Lib/unittest/__main__.py3
-rw-r--r--Lib/unittest/case.py379
-rw-r--r--Lib/unittest/loader.py76
-rw-r--r--Lib/unittest/main.py247
-rw-r--r--Lib/unittest/mock.py229
-rw-r--r--Lib/unittest/result.py16
-rw-r--r--Lib/unittest/suite.py19
-rw-r--r--Lib/unittest/test/__main__.py18
-rw-r--r--Lib/unittest/test/support.py38
-rw-r--r--Lib/unittest/test/test_assertions.py4
-rw-r--r--Lib/unittest/test/test_break.py4
-rw-r--r--Lib/unittest/test/test_case.py231
-rw-r--r--Lib/unittest/test/test_discovery.py169
-rw-r--r--Lib/unittest/test/test_functiontestcase.py4
-rw-r--r--Lib/unittest/test/test_loader.py4
-rw-r--r--Lib/unittest/test/test_program.py72
-rw-r--r--Lib/unittest/test/test_result.py92
-rw-r--r--Lib/unittest/test/test_runner.py16
-rw-r--r--Lib/unittest/test/test_setups.py7
-rw-r--r--Lib/unittest/test/test_skipping.py78
-rw-r--r--Lib/unittest/test/test_suite.py50
-rw-r--r--Lib/unittest/test/testmock/__main__.py18
-rw-r--r--Lib/unittest/test/testmock/testcallable.py4
-rw-r--r--Lib/unittest/test/testmock/testhelpers.py59
-rw-r--r--Lib/unittest/test/testmock/testmock.py143
-rw-r--r--Lib/unittest/test/testmock/testpatch.py1
-rw-r--r--Lib/unittest/test/testmock/testwith.py83
-rw-r--r--Lib/unittest/util.py37
-rw-r--r--Lib/urllib/error.py14
-rw-r--r--Lib/urllib/parse.py9
-rw-r--r--Lib/urllib/request.py191
-rw-r--r--Lib/uuid.py6
-rw-r--r--Lib/venv/__init__.py79
-rw-r--r--Lib/venv/scripts/nt/Activate.ps145
-rw-r--r--Lib/venv/scripts/nt/Deactivate.ps119
-rw-r--r--Lib/venv/scripts/nt/activate.bat2
-rw-r--r--Lib/venv/scripts/posix/activate4
-rw-r--r--Lib/venv/scripts/posix/activate.csh37
-rw-r--r--Lib/venv/scripts/posix/activate.fish74
-rw-r--r--Lib/warnings.py8
-rw-r--r--Lib/wave.py39
-rw-r--r--Lib/weakref.py198
-rw-r--r--Lib/webbrowser.py92
-rw-r--r--Lib/xml/dom/expatbuilder.py10
-rw-r--r--Lib/xml/etree/ElementInclude.py6
-rw-r--r--Lib/xml/etree/ElementPath.py15
-rw-r--r--Lib/xml/etree/ElementTree.py1431
-rw-r--r--Lib/xmlrpc/client.py9
-rw-r--r--Lib/xmlrpc/server.py7
-rw-r--r--Lib/zipfile.py196
-rw-r--r--Mac/BuildScript/README.txt12
-rwxr-xr-xMac/BuildScript/build-installer.py101
-rw-r--r--Mac/BuildScript/resources/ReadMe.txt29
-rw-r--r--Mac/BuildScript/resources/Welcome.rtf18
-rwxr-xr-xMac/BuildScript/scripts/postflight.ensurepip65
-rw-r--r--Mac/Makefile.in129
-rw-r--r--Mac/README10
-rw-r--r--Mac/Tools/Doc/HelpIndexingTool/Help_Indexing_Tool_Suite.py110
-rw-r--r--Mac/Tools/Doc/HelpIndexingTool/Miscellaneous_Standards.py49
-rw-r--r--Mac/Tools/Doc/HelpIndexingTool/Required_Suite.py32
-rw-r--r--Mac/Tools/Doc/HelpIndexingTool/Standard_Suite.py343
-rw-r--r--Mac/Tools/Doc/HelpIndexingTool/__init__.py78
-rw-r--r--Mac/Tools/Doc/HelpIndexingTool/odds_and_ends.py49
-rw-r--r--Mac/Tools/Doc/README35
-rw-r--r--Mac/Tools/Doc/setup.py213
-rw-r--r--Mac/Tools/bundlebuilder.py2
-rw-r--r--Mac/Tools/plistlib_generate_testdata.py94
-rw-r--r--Makefile.pre.in166
-rw-r--r--Misc/ACKS60
-rw-r--r--Misc/HISTORY4101
-rw-r--r--Misc/NEWS2565
-rw-r--r--Misc/README4
-rw-r--r--Misc/RPM/python-3.4.spec (renamed from Misc/RPM/python-3.3.spec)7
-rw-r--r--Misc/SpecialBuilds.txt4
-rw-r--r--Misc/TextMate/Python-Dev.tmbundle/Commands/2 to 3 - Module Deletion.tmCommand64
-rw-r--r--Misc/TextMate/Python-Dev.tmbundle/Commands/Build Docs.tmCommand23
-rw-r--r--Misc/TextMate/Python-Dev.tmbundle/Commands/Build.tmCommand25
-rw-r--r--Misc/TextMate/Python-Dev.tmbundle/Commands/Go to Issue.tmCommand20
-rw-r--r--Misc/TextMate/Python-Dev.tmbundle/Commands/Open Docs.tmCommand32
-rw-r--r--Misc/TextMate/Python-Dev.tmbundle/Commands/Open PEP.tmCommand23
-rw-r--r--Misc/TextMate/Python-Dev.tmbundle/Snippets/2 to 3 - Module Deletion (docs).tmSnippet16
-rw-r--r--Misc/TextMate/Python-Dev.tmbundle/info.plist37
-rw-r--r--Misc/Vim/python.vim148
-rw-r--r--Misc/Vim/syntax_test.py62
-rw-r--r--Misc/Vim/vim_syntax.py229
-rw-r--r--Misc/Vim/vimrc87
-rw-r--r--Misc/coverity_model.c132
-rw-r--r--Misc/python-config.in2
-rw-r--r--Misc/python-config.sh.in111
-rw-r--r--Misc/python-wing5.wpr18
-rw-r--r--Misc/python.man14
-rw-r--r--Modules/Setup.dist22
-rw-r--r--Modules/_bisectmodule.c5
-rw-r--r--Modules/_bz2module.c37
-rw-r--r--Modules/_codecsmodule.c6
-rw-r--r--Modules/_collectionsmodule.c412
-rw-r--r--Modules/_csv.c18
-rw-r--r--Modules/_ctypes/_ctypes.c80
-rw-r--r--Modules/_ctypes/callbacks.c23
-rw-r--r--Modules/_ctypes/callproc.c105
-rw-r--r--Modules/_ctypes/libffi_osx/ffi.c11
-rw-r--r--Modules/_ctypes/libffi_osx/x86/x86-ffi64.c11
-rw-r--r--Modules/_ctypes/stgdict.c4
-rw-r--r--Modules/_curses_panel.c8
-rw-r--r--Modules/_cursesmodule.c251
-rw-r--r--Modules/_datetimemodule.c175
-rw-r--r--Modules/_dbmmodule.c108
-rw-r--r--Modules/_elementtree.c407
-rw-r--r--Modules/_freeze_importlib.c6
-rw-r--r--Modules/_gdbmmodule.c38
-rw-r--r--Modules/_gestalt.c84
-rw-r--r--Modules/_hashopenssl.c252
-rw-r--r--Modules/_heapqmodule.c5
-rw-r--r--Modules/_io/_iomodule.c25
-rw-r--r--Modules/_io/_iomodule.h4
-rw-r--r--Modules/_io/bufferedio.c85
-rw-r--r--Modules/_io/fileio.c203
-rw-r--r--Modules/_io/iobase.c125
-rw-r--r--Modules/_io/textio.c105
-rw-r--r--Modules/_json.c177
-rw-r--r--Modules/_localemodule.c39
-rw-r--r--Modules/_lsprof.c32
-rw-r--r--Modules/_lzmamodule.c28
-rw-r--r--Modules/_multiprocessing/multiprocessing.c32
-rw-r--r--Modules/_multiprocessing/multiprocessing.h16
-rw-r--r--Modules/_multiprocessing/semaphore.c98
-rw-r--r--Modules/_operator.c (renamed from Modules/operator.c)125
-rw-r--r--Modules/_pickle.c258
-rw-r--r--Modules/_posixsubprocess.c142
-rw-r--r--Modules/_randommodule.c109
-rwxr-xr-xModules/_sha3/cleanup.py49
-rw-r--r--Modules/_sha3/keccak/KeccakF-1600-32-rvk.macros555
-rw-r--r--Modules/_sha3/keccak/KeccakF-1600-32-s1.macros1187
-rw-r--r--Modules/_sha3/keccak/KeccakF-1600-32-s2.macros1187
-rw-r--r--Modules/_sha3/keccak/KeccakF-1600-32.macros26
-rw-r--r--Modules/_sha3/keccak/KeccakF-1600-64.macros728
-rw-r--r--Modules/_sha3/keccak/KeccakF-1600-int-set.h6
-rw-r--r--Modules/_sha3/keccak/KeccakF-1600-interface.h46
-rw-r--r--Modules/_sha3/keccak/KeccakF-1600-opt32-settings.h6
-rw-r--r--Modules/_sha3/keccak/KeccakF-1600-opt32.c524
-rw-r--r--Modules/_sha3/keccak/KeccakF-1600-opt64-settings.h9
-rw-r--r--Modules/_sha3/keccak/KeccakF-1600-opt64.c510
-rw-r--r--Modules/_sha3/keccak/KeccakF-1600-simd128.macros651
-rw-r--r--Modules/_sha3/keccak/KeccakF-1600-simd64.macros517
-rw-r--r--Modules/_sha3/keccak/KeccakF-1600-unrolling.macros124
-rw-r--r--Modules/_sha3/keccak/KeccakF-1600-xop.macros573
-rw-r--r--Modules/_sha3/keccak/KeccakNISTInterface.c83
-rw-r--r--Modules/_sha3/keccak/KeccakNISTInterface.h72
-rw-r--r--Modules/_sha3/keccak/KeccakSponge.c266
-rw-r--r--Modules/_sha3/keccak/KeccakSponge.h76
-rwxr-xr-xModules/_sha3/keccak/brg_endian.h142
-rw-r--r--Modules/_sha3/keccak/crypto_hash.h (renamed from Modules/_ctypes/libffi/testsuite/libffi.call/a.out)0
-rw-r--r--Modules/_sha3/sha3module.c593
-rw-r--r--Modules/_sqlite/cache.c1
-rw-r--r--Modules/_sqlite/connection.c109
-rw-r--r--Modules/_sqlite/cursor.c17
-rw-r--r--Modules/_sqlite/module.c16
-rw-r--r--Modules/_sqlite/prepare_protocol.c1
-rw-r--r--Modules/_sqlite/row.c14
-rw-r--r--Modules/_sqlite/sqlitecompat.h63
-rw-r--r--Modules/_sqlite/statement.c33
-rw-r--r--Modules/_sqlite/util.c2
-rw-r--r--Modules/_sre.c1740
-rw-r--r--Modules/_ssl.c1408
-rw-r--r--Modules/_stat.c563
-rw-r--r--Modules/_struct.c359
-rw-r--r--Modules/_testbuffer.c62
-rw-r--r--Modules/_testcapimodule.c379
-rw-r--r--Modules/_testembed.c94
-rw-r--r--Modules/_testimportmultiple.c57
-rw-r--r--Modules/_threadmodule.c92
-rw-r--r--Modules/_tkinter.c666
-rw-r--r--Modules/_tracemalloc.c1394
-rw-r--r--Modules/_weakref.c60
-rw-r--r--Modules/_winapi.c43
-rw-r--r--Modules/arraymodule.c77
-rw-r--r--Modules/atexitmodule.c121
-rw-r--r--Modules/audioop.c1013
-rw-r--r--Modules/binascii.c23
-rw-r--r--Modules/cjkcodecs/_codecs_cn.c293
-rw-r--r--Modules/cjkcodecs/_codecs_hk.c93
-rw-r--r--Modules/cjkcodecs/_codecs_iso2022.c475
-rw-r--r--Modules/cjkcodecs/_codecs_jp.c459
-rw-r--r--Modules/cjkcodecs/_codecs_kr.c204
-rw-r--r--Modules/cjkcodecs/_codecs_tw.c93
-rw-r--r--Modules/cjkcodecs/alg_jisx0201.h75
-rw-r--r--Modules/cjkcodecs/cjkcodecs.h203
-rw-r--r--Modules/cjkcodecs/emu_jisx0213_2000.h33
-rw-r--r--Modules/cjkcodecs/mappings_cn.h2
-rw-r--r--Modules/cjkcodecs/mappings_jisx0213_pair.h2
-rw-r--r--Modules/cjkcodecs/multibytecodec.c344
-rw-r--r--Modules/cjkcodecs/multibytecodec.h27
-rw-r--r--Modules/faulthandler.c35
-rw-r--r--Modules/fcntlmodule.c172
-rw-r--r--Modules/gcmodule.c302
-rw-r--r--Modules/getpath.c130
-rw-r--r--Modules/grpmodule.c11
-rw-r--r--Modules/hashlib.h33
-rw-r--r--Modules/hashtable.c518
-rw-r--r--Modules/hashtable.h128
-rw-r--r--Modules/itertoolsmodule.c4
-rw-r--r--Modules/main.c73
-rw-r--r--Modules/md5module.c21
-rw-r--r--Modules/mmapmodule.c16
-rw-r--r--Modules/ossaudiodev.c13
-rw-r--r--Modules/overlapped.c1378
-rw-r--r--Modules/parsermodule.c154
-rw-r--r--Modules/posixmodule.c2588
-rw-r--r--Modules/pwdmodule.c9
-rw-r--r--Modules/pyexpat.c67
-rw-r--r--Modules/python.c27
-rw-r--r--Modules/readline.c168
-rw-r--r--Modules/resource.c177
-rw-r--r--Modules/selectmodule.c195
-rw-r--r--Modules/sha1module.c21
-rw-r--r--Modules/sha256module.c44
-rw-r--r--Modules/sha512module.c43
-rw-r--r--Modules/signalmodule.c44
-rw-r--r--Modules/socketmodule.c1212
-rw-r--r--Modules/socketmodule.h6
-rw-r--r--Modules/spwdmodule.c22
-rw-r--r--Modules/sre.h9
-rw-r--r--Modules/sre_lib.h1329
-rw-r--r--Modules/symtablemodule.c42
-rw-r--r--Modules/syslogmodule.c66
-rw-r--r--Modules/timemodule.c88
-rw-r--r--Modules/unicodedata.c70
-rw-r--r--Modules/unicodedata_db.h1988
-rw-r--r--Modules/unicodename_db.h33019
-rw-r--r--Modules/zipimport.c68
-rw-r--r--Modules/zlib/ChangeLog266
-rw-r--r--Modules/zlib/FAQ12
-rw-r--r--Modules/zlib/INDEX13
-rw-r--r--Modules/zlib/Makefile.in111
-rw-r--r--Modules/zlib/README24
-rw-r--r--Modules/zlib/adler32.c68
-rw-r--r--Modules/zlib/algorithm.txt4
-rw-r--r--Modules/zlib/compress.c2
-rwxr-xr-xModules/zlib/configure521
-rw-r--r--Modules/zlib/crc32.c83
-rw-r--r--Modules/zlib/crc32.h2
-rw-r--r--Modules/zlib/deflate.c267
-rw-r--r--Modules/zlib/deflate.h12
-rw-r--r--Modules/zlib/example.c90
-rw-r--r--Modules/zlib/gzguts.h103
-rw-r--r--Modules/zlib/gzio.c1026
-rw-r--r--Modules/zlib/gzlib.c197
-rw-r--r--Modules/zlib/gzread.c431
-rw-r--r--Modules/zlib/gzwrite.c196
-rw-r--r--Modules/zlib/infback.c16
-rw-r--r--Modules/zlib/inffast.c6
-rw-r--r--Modules/zlib/inffixed.h6
-rw-r--r--Modules/zlib/inflate.c136
-rw-r--r--Modules/zlib/inftrees.c54
-rw-r--r--Modules/zlib/inftrees.h2
-rw-r--r--Modules/zlib/make_vms.com403
-rw-r--r--Modules/zlib/minigzip.c213
-rw-r--r--Modules/zlib/trees.c54
-rw-r--r--Modules/zlib/uncompr.c2
-rw-r--r--Modules/zlib/zconf.h201
-rw-r--r--Modules/zlib/zconf.h.cmakein201
-rw-r--r--Modules/zlib/zconf.h.in201
-rw-r--r--Modules/zlib/zconf.in.h332
-rw-r--r--Modules/zlib/zlib.318
-rw-r--r--Modules/zlib/zlib.h345
-rw-r--r--Modules/zlib/zlib.map15
-rw-r--r--Modules/zlib/zutil.c26
-rw-r--r--Modules/zlib/zutil.h103
-rw-r--r--Modules/zlibmodule.c445
-rw-r--r--Objects/abstract.c138
-rw-r--r--Objects/bytearrayobject.c440
-rw-r--r--Objects/bytes_methods.c30
-rw-r--r--Objects/bytesobject.c263
-rw-r--r--Objects/classobject.c6
-rw-r--r--Objects/codeobject.c5
-rw-r--r--Objects/complexobject.c26
-rw-r--r--Objects/descrobject.c10
-rw-r--r--Objects/dictobject.c345
-rw-r--r--Objects/exceptions.c146
-rw-r--r--Objects/fileobject.c6
-rw-r--r--Objects/floatobject.c33
-rw-r--r--Objects/frameobject.c136
-rw-r--r--Objects/funcobject.c101
-rw-r--r--Objects/genobject.c105
-rw-r--r--Objects/iterobject.c13
-rw-r--r--Objects/listobject.c106
-rw-r--r--Objects/longobject.c187
-rw-r--r--Objects/memoryobject.c4
-rw-r--r--Objects/methodobject.c53
-rw-r--r--Objects/moduleobject.c109
-rw-r--r--Objects/namespaceobject.c62
-rw-r--r--Objects/object.c335
-rw-r--r--Objects/obmalloc.c584
-rw-r--r--Objects/rangeobject.c205
-rw-r--r--Objects/setobject.c516
-rw-r--r--Objects/sliceobject.c191
-rw-r--r--Objects/stringlib/asciilib.h1
-rw-r--r--Objects/stringlib/codecs.h116
-rw-r--r--Objects/stringlib/eq.h4
-rw-r--r--Objects/stringlib/fastsearch.h8
-rw-r--r--Objects/stringlib/find_max_char.h4
-rw-r--r--Objects/stringlib/join.h133
-rw-r--r--Objects/stringlib/partition.h10
-rw-r--r--Objects/stringlib/replace.h53
-rw-r--r--Objects/stringlib/split.h4
-rw-r--r--Objects/stringlib/stringdefs.h1
-rw-r--r--Objects/stringlib/transmogrify.h8
-rw-r--r--Objects/stringlib/ucs1lib.h1
-rw-r--r--Objects/stringlib/ucs2lib.h1
-rw-r--r--Objects/stringlib/ucs4lib.h1
-rw-r--r--Objects/stringlib/undef.h1
-rw-r--r--Objects/stringlib/unicode_format.h139
-rw-r--r--Objects/stringlib/unicodedefs.h6
-rw-r--r--Objects/structseq.c86
-rw-r--r--Objects/tupleobject.c121
-rw-r--r--Objects/typeobject.c368
-rw-r--r--Objects/unicodectype.c2
-rw-r--r--Objects/unicodeobject.c4701
-rw-r--r--Objects/unicodetype_db.h25
-rw-r--r--Objects/weakrefobject.c7
-rw-r--r--PC/VC6/_ctypes.dsp131
-rw-r--r--PC/VC6/_ctypes_test.dsp99
-rw-r--r--PC/VC6/_elementtree.dsp111
-rw-r--r--PC/VC6/_msi.dsp99
-rw-r--r--PC/VC6/_multiprocessing.dsp107
-rw-r--r--PC/VC6/_socket.dsp99
-rw-r--r--PC/VC6/_sqlite3.dsp131
-rw-r--r--PC/VC6/_ssl.dsp89
-rw-r--r--PC/VC6/_ssl.mak22
-rw-r--r--PC/VC6/_testcapi.dsp99
-rw-r--r--PC/VC6/_tkinter.dsp103
-rw-r--r--PC/VC6/build_ssl.py228
-rw-r--r--PC/VC6/build_tkinter.py81
-rw-r--r--PC/VC6/bz2.dsp99
-rw-r--r--PC/VC6/make_versioninfo.dsp108
-rw-r--r--PC/VC6/pcbuild.dsw306
-rw-r--r--PC/VC6/pyexpat.dsp111
-rw-r--r--PC/VC6/python.dsp100
-rw-r--r--PC/VC6/pythoncore.dsp780
-rw-r--r--PC/VC6/pythonw.dsp101
-rw-r--r--PC/VC6/readme.txt192
-rw-r--r--PC/VC6/rmpyc.py25
-rwxr-xr-xPC/VC6/rt.bat41
-rw-r--r--PC/VC6/select.dsp99
-rw-r--r--PC/VC6/tcl852.patch11
-rw-r--r--PC/VC6/unicodedata.dsp99
-rw-r--r--PC/VC6/w9xpopen.dsp97
-rw-r--r--PC/VC6/winsound.dsp99
-rw-r--r--PC/VS7.1/Uninstal.wse514
-rw-r--r--PC/VS7.1/_ctypes.vcproj311
-rw-r--r--PC/VS7.1/_ctypes_test.vcproj242
-rw-r--r--PC/VS7.1/_elementtree.vcproj264
-rw-r--r--PC/VS7.1/_msi.vcproj252
-rw-r--r--PC/VS7.1/_socket.vcproj254
-rw-r--r--PC/VS7.1/_sqlite3.vcproj283
-rw-r--r--PC/VS7.1/_ssl.mak38
-rw-r--r--PC/VS7.1/_ssl.vcproj84
-rw-r--r--PC/VS7.1/_testcapi.vcproj247
-rw-r--r--PC/VS7.1/_tkinter.vcproj261
-rw-r--r--PC/VS7.1/amd64_ml64.bat17
-rw-r--r--PC/VS7.1/build_ssl.bat12
-rw-r--r--PC/VS7.1/build_ssl.py181
-rw-r--r--PC/VS7.1/bz2.vcproj271
-rw-r--r--PC/VS7.1/db.build10
-rw-r--r--PC/VS7.1/field3.py35
-rw-r--r--PC/VS7.1/installer.bmpbin58806 -> 0 bytes-rw-r--r--PC/VS7.1/make_buildinfo.c92
-rw-r--r--PC/VS7.1/make_buildinfo.vcproj122
-rw-r--r--PC/VS7.1/make_versioninfo.vcproj142
-rw-r--r--PC/VS7.1/pcbuild.sln275
-rw-r--r--PC/VS7.1/pyexpat.vcproj263
-rw-r--r--PC/VS7.1/python.build20
-rw-r--r--PC/VS7.1/python.iss341
-rw-r--r--PC/VS7.1/python.vcproj274
-rw-r--r--PC/VS7.1/python20.wse3117
-rw-r--r--PC/VS7.1/pythoncore.vcproj826
-rw-r--r--PC/VS7.1/pythonw.vcproj261
-rw-r--r--PC/VS7.1/readme.txt337
-rw-r--r--PC/VS7.1/rmpyc.py25
-rwxr-xr-xPC/VS7.1/rt.bat52
-rw-r--r--PC/VS7.1/select.vcproj258
-rw-r--r--PC/VS7.1/unicodedata.vcproj247
-rw-r--r--PC/VS7.1/w9xpopen.vcproj121
-rw-r--r--PC/VS7.1/winsound.vcproj251
-rw-r--r--PC/VS8.0/_ctypes.vcproj705
-rw-r--r--PC/VS8.0/_elementtree.vcproj613
-rw-r--r--PC/VS8.0/_hashlib.vcproj537
-rw-r--r--PC/VS8.0/_msi.vcproj529
-rw-r--r--PC/VS8.0/_multiprocessing.vcproj545
-rw-r--r--PC/VS8.0/_socket.vcproj537
-rw-r--r--PC/VS8.0/_sqlite3.vcproj613
-rw-r--r--PC/VS8.0/_ssl.vcproj537
-rw-r--r--PC/VS8.0/_tkinter.vcproj541
-rw-r--r--PC/VS8.0/bdist_wininst.vcproj270
-rw-r--r--PC/VS8.0/build.bat17
-rw-r--r--PC/VS8.0/build_env.bat1
-rw-r--r--PC/VS8.0/build_pgo.bat41
-rw-r--r--PC/VS8.0/build_ssl.bat12
-rw-r--r--PC/VS8.0/build_ssl.py277
-rw-r--r--PC/VS8.0/build_tkinter.py85
-rw-r--r--PC/VS8.0/bz2.vcproj581
-rw-r--r--PC/VS8.0/debug.vsprops15
-rw-r--r--PC/VS8.0/env.bat5
-rw-r--r--PC/VS8.0/field3.py35
-rw-r--r--PC/VS8.0/idle.bat15
-rw-r--r--PC/VS8.0/kill_python.c178
-rw-r--r--PC/VS8.0/kill_python.vcproj279
-rw-r--r--PC/VS8.0/make_buildinfo.c116
-rw-r--r--PC/VS8.0/make_buildinfo.vcproj101
-rw-r--r--PC/VS8.0/make_versioninfo.vcproj324
-rw-r--r--PC/VS8.0/pcbuild.sln560
-rw-r--r--PC/VS8.0/pginstrument.vsprops34
-rw-r--r--PC/VS8.0/pgupdate.vsprops14
-rw-r--r--PC/VS8.0/pyd.vsprops28
-rw-r--r--PC/VS8.0/pyd_d.vsprops36
-rw-r--r--PC/VS8.0/pyexpat.vcproj553
-rw-r--r--PC/VS8.0/pyproject.vsprops87
-rw-r--r--PC/VS8.0/python.vcproj637
-rw-r--r--PC/VS8.0/pythoncore.vcproj1921
-rw-r--r--PC/VS8.0/pythonw.vcproj618
-rw-r--r--PC/VS8.0/release.vsprops15
-rw-r--r--PC/VS8.0/rmpyc.py25
-rw-r--r--PC/VS8.0/rt.bat52
-rw-r--r--PC/VS8.0/select.vcproj537
-rw-r--r--PC/VS8.0/sqlite3.vcproj537
-rw-r--r--PC/VS8.0/sqlite3.vsprops14
-rw-r--r--PC/VS8.0/ssl.vcproj189
-rw-r--r--PC/VS8.0/unicodedata.vcproj533
-rw-r--r--PC/VS8.0/w9xpopen.vcproj576
-rw-r--r--PC/VS8.0/winsound.vcproj523
-rw-r--r--PC/VS8.0/x64.vsprops22
-rw-r--r--PC/VS9.0/_sha3.vcproj (renamed from PC/VS8.0/_ctypes_test.vcproj)18
-rw-r--r--PC/VS9.0/_sqlite3.vcproj4
-rw-r--r--PC/VS9.0/_ssl.vcproj18
-rw-r--r--PC/VS9.0/_testimportmultiple.vcproj (renamed from PC/VS8.0/_testcapi.vcproj)10
-rw-r--r--PC/VS9.0/kill_python.c2
-rw-r--r--PC/VS9.0/pcbuild.sln44
-rw-r--r--PC/VS9.0/pyproject.vsprops6
-rw-r--r--PC/VS9.0/pythoncore.vcproj90
-rw-r--r--PC/VS9.0/w9xpopen.vcproj576
-rw-r--r--PC/_msi.c80
-rw-r--r--PC/bdist_wininst/install.c2
-rw-r--r--PC/config.c8
-rw-r--r--PC/example_nt/example.vcproj4
-rw-r--r--PC/getpathp.c43
-rw-r--r--PC/launcher.c254
-rwxr-xr-xPC/msvcrtmodule.c5
-rw-r--r--PC/os2emx/Makefile672
-rw-r--r--PC/os2emx/README.os2emx663
-rw-r--r--PC/os2emx/config.c164
-rw-r--r--PC/os2emx/dlfcn.c223
-rw-r--r--PC/os2emx/dlfcn.h51
-rw-r--r--PC/os2emx/dllentry.c42
-rw-r--r--PC/os2emx/getpathp.c418
-rw-r--r--PC/os2emx/pyconfig.h332
-rw-r--r--PC/os2emx/python33.def1314
-rw-r--r--PC/os2emx/pythonpm.c124
-rw-r--r--PC/os2vacpp/_tkinter.def8
-rw-r--r--PC/os2vacpp/config.c99
-rw-r--r--PC/os2vacpp/getpathp.c482
-rw-r--r--PC/os2vacpp/makefile1549
-rw-r--r--PC/os2vacpp/makefile.omk1047
-rw-r--r--PC/os2vacpp/pyconfig.h212
-rw-r--r--PC/os2vacpp/python.def479
-rw-r--r--PC/os2vacpp/readme.txt119
-rw-r--r--PC/pyconfig.h21
-rw-r--r--PC/python.mk5
-rw-r--r--PC/python3.def1395
-rw-r--r--PC/python3.mak12
-rw-r--r--PC/python34gen.py (renamed from PC/python33gen.py)8
-rw-r--r--PC/python34stub.def (renamed from PC/python33stub.def)3
-rw-r--r--PC/readme.txt13
-rw-r--r--PC/w9xpopen.c112
-rw-r--r--PC/winreg.c8
-rw-r--r--PCbuild/_overlapped.vcxproj234
-rw-r--r--PCbuild/_sha3.vcxproj (renamed from PCbuild/w9xpopen.vcxproj)165
-rw-r--r--PCbuild/_sqlite3.vcxproj1
-rw-r--r--PCbuild/_sqlite3.vcxproj.filters3
-rw-r--r--PCbuild/_ssl.vcxproj16
-rw-r--r--PCbuild/_testembed.vcxproj161
-rw-r--r--PCbuild/_testembed.vcxproj.filters22
-rw-r--r--PCbuild/_testimportmultiple.vcxproj218
-rw-r--r--PCbuild/_testimportmultiple.vcxproj.filters (renamed from PCbuild/w9xpopen.vcxproj.filters)6
-rw-r--r--PCbuild/build_ssl.bat4
-rw-r--r--PCbuild/build_tkinter.py6
-rw-r--r--PCbuild/kill_python.c2
-rw-r--r--PCbuild/pcbuild.sln90
-rw-r--r--PCbuild/pyproject.props11
-rw-r--r--PCbuild/python.vcxproj8
-rw-r--r--PCbuild/pythoncore.vcxproj14
-rw-r--r--PCbuild/pythoncore.vcxproj.filters23
-rw-r--r--PCbuild/readme.txt486
-rw-r--r--Parser/Python.asdl12
-rw-r--r--Parser/asdl.py33
-rwxr-xr-xParser/asdl_c.py69
-rw-r--r--Parser/grammar.c10
-rw-r--r--Parser/grammar1.c8
-rw-r--r--Parser/myreadline.c40
-rw-r--r--Parser/node.c2
-rw-r--r--Parser/parser.c36
-rw-r--r--Parser/parsetok.c93
-rw-r--r--Parser/pgenmain.c3
-rw-r--r--Parser/tokenizer.c29
-rw-r--r--Parser/tokenizer.h6
-rw-r--r--Python/Python-ast.c370
-rw-r--r--Python/_warnings.c245
-rw-r--r--Python/asdl.c4
-rw-r--r--Python/ast.c237
-rw-r--r--Python/bltinmodule.c161
-rw-r--r--Python/ceval.c2281
-rw-r--r--Python/codecs.c339
-rw-r--r--Python/compile.c459
-rw-r--r--Python/condvar.h2
-rw-r--r--Python/dynload_aix.c12
-rw-r--r--Python/dynload_os2.c42
-rw-r--r--Python/dynload_shlib.c36
-rw-r--r--Python/errors.c220
-rw-r--r--Python/fileutils.c380
-rw-r--r--Python/formatter_unicode.c37
-rw-r--r--Python/frozen.c4
-rw-r--r--Python/frozenmain.c58
-rw-r--r--Python/future.c53
-rw-r--r--Python/getargs.c92
-rw-r--r--Python/getcwd.c83
-rw-r--r--Python/import.c284
-rw-r--r--Python/importdl.c2
-rw-r--r--Python/importdl.h5
-rw-r--r--Python/importlib.h8422
-rw-r--r--Python/marshal.c799
-rw-r--r--Python/mystrtoul.c32
-rw-r--r--Python/opcode_targets.h4
-rw-r--r--Python/peephole.c55
-rw-r--r--Python/pyarena.c12
-rw-r--r--Python/pyhash.c427
-rw-r--r--Python/pystate.c127
-rw-r--r--Python/pystrtod.c6
-rw-r--r--Python/pythonrun.c542
-rw-r--r--Python/random.c103
-rw-r--r--Python/strdup.c2
-rw-r--r--Python/symtable.c187
-rw-r--r--Python/sysmodule.c229
-rw-r--r--Python/thread.c18
-rw-r--r--Python/thread_nt.h10
-rw-r--r--Python/thread_os2.h267
-rw-r--r--Python/thread_pth.h177
-rw-r--r--Python/thread_pthread.h12
-rw-r--r--Python/traceback.c33
-rw-r--r--README18
-rw-r--r--Tools/buildbot/external-amd64.bat10
-rw-r--r--Tools/buildbot/external-common.bat11
-rw-r--r--Tools/buildbot/external.bat8
-rwxr-xr-xTools/clinic/clinic.py2528
-rw-r--r--Tools/clinic/clinic_test.py713
-rw-r--r--Tools/freeze/bkfile.py4
-rwxr-xr-xTools/freeze/freeze.py4
-rw-r--r--Tools/gdb/libpython.py44
-rwxr-xr-xTools/i18n/msgfmt.py2
-rw-r--r--Tools/importbench/importbench.py31
-rw-r--r--Tools/msi/msi.py40
-rw-r--r--Tools/parser/unparse.py15
-rw-r--r--Tools/scripts/README126
-rwxr-xr-x[-rw-r--r--]Tools/scripts/analyze_dxp.py0
-rwxr-xr-xTools/scripts/byext.py2
-rwxr-xr-xTools/scripts/byteyears.py2
-rw-r--r--Tools/scripts/checkpip.py32
-rwxr-xr-xTools/scripts/checkpyc.py12
-rwxr-xr-xTools/scripts/copytime.py4
-rwxr-xr-xTools/scripts/finddiv.py2
-rwxr-xr-xTools/scripts/findlinksto.py2
-rwxr-xr-xTools/scripts/fixcid.py8
-rwxr-xr-xTools/scripts/ftpmirror.py20
-rwxr-xr-xTools/scripts/linktree.py6
-rwxr-xr-xTools/scripts/parse_html5_entities.py105
-rwxr-xr-xTools/scripts/pathfix.py10
-rwxr-xr-xTools/scripts/pindent.py4
-rwxr-xr-x[-rw-r--r--]Tools/scripts/pydocgui.pyw0
-rwxr-xr-xTools/scripts/reindent.py2
-rwxr-xr-xTools/scripts/treesync.py2
-rwxr-xr-xTools/scripts/untabify.py4
-rwxr-xr-xTools/scripts/which.py2
-rwxr-xr-x[-rw-r--r--]Tools/scripts/win_add2path.py2
-rwxr-xr-xTools/stringbench/stringbench.py2
-rw-r--r--Tools/unicode/gencodec.py18
-rw-r--r--Tools/unicode/makeunicodedata.py8
-rw-r--r--Tools/unicode/python-mappings/CP273.TXT258
-rwxr-xr-xconfigure567
-rw-r--r--configure.ac407
-rw-r--r--pyconfig.h.in34
-rw-r--r--setup.py31
1409 files changed, 146280 insertions, 101579 deletions
diff --git a/.gitignore b/.gitignore
index 49299ee..59d3832 100644
--- a/.gitignore
+++ b/.gitignore
@@ -15,11 +15,13 @@ Doc/tools/jinja2/
Doc/tools/pygments/
Doc/tools/sphinx/
Lib/lib2to3/*.pickle
+Lib/test/data/*
Lib/_sysconfigdata.py
Lib/plat-mac/errors.rsrc.df.rsrc
Makefile
Makefile.pre
Misc/python.pc
+Misc/python-config.sh
Modules/Setup
Modules/Setup.config
Modules/Setup.local
@@ -57,6 +59,8 @@ platform
pybuilddir.txt
pyconfig.h
python
+python-config
+python-config.py
python.exe
python-gdb.py
python.exe-gdb.py
diff --git a/.hgeol b/.hgeol
index 4e03690..aad79c2 100644
--- a/.hgeol
+++ b/.hgeol
@@ -26,6 +26,7 @@
**.psd = BIN
**.tar = BIN
**.wav = BIN
+**.whl = BIN
**.xar = BIN
**.zip = BIN
diff --git a/.hgignore b/.hgignore
index c42dd03..6128c2b 100644
--- a/.hgignore
+++ b/.hgignore
@@ -19,6 +19,8 @@ platform$
pyconfig.h$
python$
python.exe$
+python-config$
+python-config.py$
reflog.txt$
tags$
Lib/plat-mac/errors.rsrc.df.rsrc
@@ -28,12 +30,14 @@ Doc/tools/jinja/
Doc/tools/jinja2/
Doc/tools/pygments/
Misc/python.pc
+Misc/python-config.sh$
Modules/Setup$
Modules/Setup.config
Modules/Setup.local
Modules/config.c
Modules/ld_so_aix$
Parser/pgen$
+^lcov-report/
^core
^python-gdb.py
^python.exe-gdb.py
@@ -82,6 +86,9 @@ PCbuild/*.*sdf
PCbuild/Win32-temp-*
PCbuild/x64-temp-*
PCbuild/amd64
+PCbuild/ipch
+Tools/unicode/build/
+Tools/unicode/MAPPINGS/
BuildLog.htm
__pycache__
Modules/_freeze_importlib
@@ -89,3 +96,7 @@ Modules/_testembed
.coverage
coverage/
htmlcov/
+*.gcda
+*.gcno
+*.gcov
+coverage.info
diff --git a/.hgtags b/.hgtags
index 7801acf..2d7ca9b 100644
--- a/.hgtags
+++ b/.hgtags
@@ -118,3 +118,7 @@ d047928ae3f6314a13b6137051315453d0ae89b6 v3.3.2
fd53c500f8b80f54f3ecedec9da2e8c7e52a6888 v3.3.3rc1
d32442c0e60dfbd71234e807d3d1dedd227495a9 v3.3.3rc2
c3896275c0f61b2510a6c7e6c458a750359a91b8 v3.3.3
+46535f65e7f3bcdcf176f36d34bc1fed719ffd2b v3.4.0a1
+9265a2168e2cb2a84785d8717792acc661e6b692 v3.4.0a2
+dd9cdf90a5073510877e9dd5112f8e6cf20d5e89 v3.4.0a3
+e245b0d7209bb6d0e19316e1e2af1aa9c2139104 v3.4.0a4
diff --git a/.hgtouch b/.hgtouch
index 0af8c19..7e3a5e7 100644
--- a/.hgtouch
+++ b/.hgtouch
@@ -4,8 +4,8 @@
Python/importlib.h: Lib/importlib/_bootstrap.py Modules/_freeze_importlib.c
-Include/ast.h: Parser/Python.asdl Parser/asdl.py Parser/asdl_c.py
-Python/Python-ast.c: Include/ast.h
+Include/Python-ast.h: Parser/Python.asdl Parser/asdl.py Parser/asdl_c.py
+Python/Python-ast.c: Include/Python-ast.h
Python/opcode_targets.h: Python/makeopcodetargets.py Lib/opcode.py
diff --git a/Doc/README.txt b/Doc/README.txt
index c63fc60..64e307e 100644
--- a/Doc/README.txt
+++ b/Doc/README.txt
@@ -7,7 +7,8 @@ available at http://docs.python.org/download/.
Documentation on the authoring Python documentation, including information about
both style and markup, is available in the "Documenting Python" chapter of the
-documentation. There's also a chapter intended to point out differences to
+developers guide (http://docs.python.org/devguide/documenting.html).
+There's also a chapter intended to point out differences to
those familiar with the previous docs written in LaTeX.
diff --git a/Doc/c-api/concrete.rst b/Doc/c-api/concrete.rst
index 65904ee..2d56386 100644
--- a/Doc/c-api/concrete.rst
+++ b/Doc/c-api/concrete.rst
@@ -74,26 +74,35 @@ intrinsic to the Python language.
.. _mapobjects:
-Mapping Objects
-===============
+Container Objects
+=================
.. index:: object: mapping
.. toctree::
dict.rst
+ set.rst
.. _otherobjects:
-Other Objects
-=============
+Function Objects
+================
.. toctree::
- set.rst
function.rst
method.rst
+ cell.rst
+ code.rst
+
+
+Other Objects
+=============
+
+.. toctree::
+
file.rst
module.rst
iterator.rst
@@ -102,7 +111,6 @@ Other Objects
memoryview.rst
weakref.rst
capsule.rst
- cell.rst
gen.rst
datetime.rst
- code.rst
+
diff --git a/Doc/c-api/dict.rst b/Doc/c-api/dict.rst
index 6bacc32..5a9dca2 100644
--- a/Doc/c-api/dict.rst
+++ b/Doc/c-api/dict.rst
@@ -84,7 +84,7 @@ Dictionary Objects
on failure.
-.. c:function:: int PyDict_DelItemString(PyObject *p, char *key)
+.. c:function:: int PyDict_DelItemString(PyObject *p, const char *key)
Remove the entry in dictionary *p* which has a key specified by the string
*key*. Return ``0`` on success or ``-1`` on failure.
@@ -110,6 +110,15 @@ Dictionary Objects
:c:type:`char\*`, rather than a :c:type:`PyObject\*`.
+.. c:function:: PyObject* PyDict_SetDefault(PyObject *p, PyObject *key, PyObject *default)
+
+ This is the same as the Python-level :meth:`dict.setdefault`. If present, it
+ returns the value corresponding to *key* from the dictionary *p*. If the key
+ is not in the dict, it is inserted with value *defaultobj* and *defaultobj*
+ is returned. This function evaluates the hash function of *key* only once,
+ instead of evaluating it independently for the lookup and the insertion.
+
+
.. c:function:: PyObject* PyDict_Items(PyObject *p)
Return a :c:type:`PyListObject` containing all the items from the dictionary.
diff --git a/Doc/c-api/exceptions.rst b/Doc/c-api/exceptions.rst
index c3d978f..8658a58 100644
--- a/Doc/c-api/exceptions.rst
+++ b/Doc/c-api/exceptions.rst
@@ -235,7 +235,7 @@ in various ways. There is a separate error indicator for each thread.
Similar to :c:func:`PyErr_SetFromErrnoWithFilenameObject`, but the filename
is given as a C string. *filename* is decoded from the filesystem encoding
- (:func:`sys.getfilesystemencoding`).
+ (:func:`os.fsdecode`).
.. c:function:: PyObject* PyErr_SetFromWindowsErr(int ierr)
@@ -267,7 +267,7 @@ in various ways. There is a separate error indicator for each thread.
Similar to :c:func:`PyErr_SetFromWindowsErrWithFilenameObject`, but the
filename is given as a C string. *filename* is decoded from the filesystem
- encoding (:func:`sys.getfilesystemencoding`). Availability: Windows.
+ encoding (:func:`os.fsdecode`). Availability: Windows.
.. c:function:: PyObject* PyErr_SetExcFromWindowsErrWithFilenameObject(PyObject *type, int ierr, PyObject *filename)
@@ -293,20 +293,27 @@ in various ways. There is a separate error indicator for each thread.
.. versionadded:: 3.3
-.. c:function:: void PyErr_SyntaxLocationEx(char *filename, int lineno, int col_offset)
+.. c:function:: void PyErr_SyntaxLocationObject(PyObject *filename, int lineno, int col_offset)
Set file, line, and offset information for the current exception. If the
current exception is not a :exc:`SyntaxError`, then it sets additional
attributes, which make the exception printing subsystem think the exception
- is a :exc:`SyntaxError`. *filename* is decoded from the filesystem encoding
- (:func:`sys.getfilesystemencoding`).
+ is a :exc:`SyntaxError`.
- .. versionadded:: 3.2
+.. versionadded:: 3.4
+
+
+.. c:function:: void PyErr_SyntaxLocationEx(char *filename, int lineno, int col_offset)
+
+ Like :c:func:`PyErr_SyntaxLocationObject`, but *filename* is a byte string
+ decoded from the filesystem encoding (:func:`os.fsdecode`).
+
+.. versionadded:: 3.2
.. c:function:: void PyErr_SyntaxLocation(char *filename, int lineno)
- Like :c:func:`PyErr_SyntaxLocationExc`, but the col_offset parameter is
+ Like :c:func:`PyErr_SyntaxLocationEx`, but the col_offset parameter is
omitted.
@@ -355,15 +362,22 @@ in various ways. There is a separate error indicator for each thread.
documentation. There is no C API for warning control.
-.. c:function:: int PyErr_WarnExplicit(PyObject *category, const char *message, const char *filename, int lineno, const char *module, PyObject *registry)
+.. c:function:: int PyErr_WarnExplicitObject(PyObject *category, PyObject *message, PyObject *filename, int lineno, PyObject *module, PyObject *registry)
Issue a warning message with explicit control over all warning attributes. This
is a straightforward wrapper around the Python function
:func:`warnings.warn_explicit`, see there for more information. The *module*
and *registry* arguments may be set to *NULL* to get the default effect
- described there. *message* and *module* are UTF-8 encoded strings,
- *filename* is decoded from the filesystem encoding
- (:func:`sys.getfilesystemencoding`).
+ described there.
+
+ .. versionadded:: 3.4
+
+
+.. c:function:: int PyErr_WarnExplicit(PyObject *category, const char *message, const char *filename, int lineno, const char *module, PyObject *registry)
+
+ Similar to :c:func:`PyErr_WarnExplicitObject` except that *message* and
+ *module* are UTF-8 encoded strings, and *filename* is decoded from the
+ filesystem encoding (:func:`os.fsdecode`).
.. c:function:: int PyErr_WarnFormat(PyObject *category, Py_ssize_t stack_level, const char *format, ...)
diff --git a/Doc/c-api/file.rst b/Doc/c-api/file.rst
index cc190c9..6f2ecee 100644
--- a/Doc/c-api/file.rst
+++ b/Doc/c-api/file.rst
@@ -17,7 +17,7 @@ error reporting in the interpreter; third-party code is advised to access
the :mod:`io` APIs instead.
-.. c:function:: PyFile_FromFd(int fd, char *name, char *mode, int buffering, char *encoding, char *errors, char *newline, int closefd)
+.. c:function:: PyFile_FromFd(int fd, const char *name, const char *mode, int buffering, const char *encoding, const char *errors, const char *newline, int closefd)
Create a Python file object from the file descriptor of an already
opened file *fd*. The arguments *name*, *encoding*, *errors* and *newline*
diff --git a/Doc/c-api/import.rst b/Doc/c-api/import.rst
index 270152e..6cd2b8b 100644
--- a/Doc/c-api/import.rst
+++ b/Doc/c-api/import.rst
@@ -118,7 +118,7 @@ Importing Modules
encoded string instead of a Unicode object.
-.. c:function:: PyObject* PyImport_ExecCodeModule(char *name, PyObject *co)
+.. c:function:: PyObject* PyImport_ExecCodeModule(const char *name, PyObject *co)
.. index:: builtin: compile
@@ -145,7 +145,7 @@ Importing Modules
:c:func:`PyImport_ExecCodeModuleWithPathnames`.
-.. c:function:: PyObject* PyImport_ExecCodeModuleEx(char *name, PyObject *co, char *pathname)
+.. c:function:: PyObject* PyImport_ExecCodeModuleEx(const char *name, PyObject *co, const char *pathname)
Like :c:func:`PyImport_ExecCodeModule`, but the :attr:`__file__` attribute of
the module object is set to *pathname* if it is non-``NULL``.
@@ -162,7 +162,7 @@ Importing Modules
.. versionadded:: 3.3
-.. c:function:: PyObject* PyImport_ExecCodeModuleWithPathnames(char *name, PyObject *co, char *pathname, char *cpathname)
+.. c:function:: PyObject* PyImport_ExecCodeModuleWithPathnames(const char *name, PyObject *co, const char *pathname, const char *cpathname)
Like :c:func:`PyImport_ExecCodeModuleObject`, but *name*, *pathname* and
*cpathname* are UTF-8 encoded strings. Attempts are also made to figure out
@@ -246,7 +246,7 @@ Importing Modules
.. versionadded:: 3.3
-.. c:function:: int PyImport_ImportFrozenModule(char *name)
+.. c:function:: int PyImport_ImportFrozenModule(const char *name)
Similar to :c:func:`PyImport_ImportFrozenModuleObject`, but the name is a
UTF-8 encoded string instead of a Unicode object.
diff --git a/Doc/c-api/init.rst b/Doc/c-api/init.rst
index 705509f..6439d7f 100644
--- a/Doc/c-api/init.rst
+++ b/Doc/c-api/init.rst
@@ -86,6 +86,36 @@ Process-wide parameters
=======================
+.. c:function:: int Py_SetStandardStreamEncoding(char *encoding, char *errors)
+
+ .. index::
+ single: Py_Initialize()
+ single: main()
+ triple: stdin; stdout; sdterr
+
+ This function should be called before :c:func:`Py_Initialize`, if it is
+ called at all. It specifies which encoding and error handling to use
+ with standard IO, with the same meanings as in :func:`str.encode`.
+
+ It overrides :envvar:`PYTHONIOENCODING` values, and allows embedding code
+ to control IO encoding when the environment variable does not work.
+
+ ``encoding`` and/or ``errors`` may be NULL to use
+ :envvar:`PYTHONIOENCODING` and/or default values (depending on other
+ settings).
+
+ Note that :data:`sys.stderr` always uses the "backslashreplace" error
+ handler, regardless of this (or any other) setting.
+
+ If :c:func:`Py_Finalize` is called, this function will need to be called
+ again in order to affect subsequent calls to :c:func:`Py_Initialize`.
+
+ Returns 0 if successful, a nonzero value on error (e.g. calling after the
+ interpreter has already been initialized).
+
+ .. versionadded:: 3.4
+
+
.. c:function:: void Py_SetProgramName(wchar_t *name)
.. index::
@@ -329,7 +359,11 @@ Process-wide parameters
.. c:function:: void PySys_SetArgv(int argc, wchar_t **argv)
- This function works like :c:func:`PySys_SetArgvEx` with *updatepath* set to 1.
+ This function works like :c:func:`PySys_SetArgvEx` with *updatepath* set
+ to 1 unless the :program:`python` interpreter was started with the
+ :option:`-I`.
+
+ .. versionchanged:: 3.4 The *updatepath* value depends on :option:`-I`.
.. c:function:: void Py_SetPythonHome(wchar_t *home)
@@ -657,6 +691,20 @@ with sub-interpreters:
made on the main thread. This is mainly a helper/diagnostic function.
+.. c:function:: int PyGILState_Check()
+
+ Return 1 if the current thread is holding the GIL and 0 otherwise.
+ This function can be called from any thread at any time.
+ Only if it has had its Python thread state initialized and currently is
+ holding the GIL will it return 1.
+ This is mainly a helper/diagnostic function. It can be useful
+ for example in callback contexts or memory allocation functions when
+ knowing that the GIL is locked can allow the caller to perform sensitive
+ actions or otherwise behave differently.
+
+ .. versionadded:: 3.4
+
+
The following macros are normally used without a trailing semicolon; look for
example usage in the Python source distribution.
diff --git a/Doc/c-api/long.rst b/Doc/c-api/long.rst
index d5430fd..b348015 100644
--- a/Doc/c-api/long.rst
+++ b/Doc/c-api/long.rst
@@ -80,7 +80,7 @@ All integers are implemented as "long" integer objects of arbitrary size.
*NULL* on failure.
-.. c:function:: PyObject* PyLong_FromString(char *str, char **pend, int base)
+.. c:function:: PyObject* PyLong_FromString(const char *str, char **pend, int base)
Return a new :c:type:`PyLongObject` based on the string value in *str*, which
is interpreted according to the radix in *base*. If *pend* is non-*NULL*,
diff --git a/Doc/c-api/mapping.rst b/Doc/c-api/mapping.rst
index 0ef2961..2803fd0 100644
--- a/Doc/c-api/mapping.rst
+++ b/Doc/c-api/mapping.rst
@@ -22,7 +22,7 @@ Mapping Protocol
expression ``len(o)``.
-.. c:function:: int PyMapping_DelItemString(PyObject *o, char *key)
+.. c:function:: int PyMapping_DelItemString(PyObject *o, const char *key)
Remove the mapping for object *key* from the object *o*. Return ``-1`` on
failure. This is equivalent to the Python statement ``del o[key]``.
@@ -67,13 +67,13 @@ Mapping Protocol
the Python expression ``list(o.items())``.
-.. c:function:: PyObject* PyMapping_GetItemString(PyObject *o, char *key)
+.. c:function:: PyObject* PyMapping_GetItemString(PyObject *o, const char *key)
Return element of *o* corresponding to the object *key* or *NULL* on failure.
This is the equivalent of the Python expression ``o[key]``.
-.. c:function:: int PyMapping_SetItemString(PyObject *o, char *key, PyObject *v)
+.. c:function:: int PyMapping_SetItemString(PyObject *o, const char *key, PyObject *v)
Map the object *key* to the value *v* in object *o*. Returns ``-1`` on failure.
This is the equivalent of the Python statement ``o[key] = v``.
diff --git a/Doc/c-api/memory.rst b/Doc/c-api/memory.rst
index 8afa56a..a82e1c2 100644
--- a/Doc/c-api/memory.rst
+++ b/Doc/c-api/memory.rst
@@ -84,6 +84,48 @@ the C library allocator as shown in the previous example, the allocated memory
for the I/O buffer escapes completely the Python memory manager.
+Raw Memory Interface
+====================
+
+The following function sets are wrappers to the system allocator. These
+functions are thread-safe, the :term:`GIL <global interpreter lock>` does not
+need to be held.
+
+The default raw memory block allocator uses the following functions:
+:c:func:`malloc`, :c:func:`realloc` and :c:func:`free`; call ``malloc(1)`` when
+requesting zero bytes.
+
+.. versionadded:: 3.4
+
+.. c:function:: void* PyMem_RawMalloc(size_t n)
+
+ Allocates *n* bytes and returns a pointer of type :c:type:`void\*` to the
+ allocated memory, or *NULL* if the request fails. Requesting zero bytes
+ returns a distinct non-*NULL* pointer if possible, as if
+ ``PyMem_RawMalloc(1)`` had been called instead. The memory will not have
+ been initialized in any way.
+
+
+.. c:function:: void* PyMem_RawRealloc(void *p, size_t n)
+
+ Resizes the memory block pointed to by *p* to *n* bytes. The contents will
+ be unchanged to the minimum of the old and the new sizes. If *p* is *NULL*,
+ the call is equivalent to ``PyMem_RawMalloc(n)``; else if *n* is equal to
+ zero, the memory block is resized but is not freed, and the returned pointer
+ is non-*NULL*. Unless *p* is *NULL*, it must have been returned by a
+ previous call to :c:func:`PyMem_RawMalloc` or :c:func:`PyMem_RawRealloc`. If
+ the request fails, :c:func:`PyMem_RawRealloc` returns *NULL* and *p* remains
+ a valid pointer to the previous memory area.
+
+
+.. c:function:: void PyMem_RawFree(void *p)
+
+ Frees the memory block pointed to by *p*, which must have been returned by a
+ previous call to :c:func:`PyMem_RawMalloc` or :c:func:`PyMem_RawRealloc`.
+ Otherwise, or if ``PyMem_Free(p)`` has been called before, undefined
+ behavior occurs. If *p* is *NULL*, no operation is performed.
+
+
.. _memoryinterface:
Memory Interface
@@ -91,8 +133,16 @@ Memory Interface
The following function sets, modeled after the ANSI C standard, but specifying
behavior when requesting zero bytes, are available for allocating and releasing
-memory from the Python heap:
+memory from the Python heap.
+
+The default memory block allocator uses the following functions:
+:c:func:`malloc`, :c:func:`realloc` and :c:func:`free`; call ``malloc(1)`` when
+requesting zero bytes.
+
+.. warning::
+ The :term:`GIL <global interpreter lock>` must be held when using these
+ functions.
.. c:function:: void* PyMem_Malloc(size_t n)
@@ -155,6 +205,125 @@ versions and is therefore deprecated in extension modules.
:c:func:`PyMem_NEW`, :c:func:`PyMem_RESIZE`, :c:func:`PyMem_DEL`.
+Customize Memory Allocators
+===========================
+
+.. versionadded:: 3.4
+
+.. c:type:: PyMemAllocator
+
+ Structure used to describe a memory block allocator. The structure has
+ four fields:
+
+ +----------------------------------------------------------+---------------------------------------+
+ | Field | Meaning |
+ +==========================================================+=======================================+
+ | ``void *ctx`` | user context passed as first argument |
+ +----------------------------------------------------------+---------------------------------------+
+ | ``void* malloc(void *ctx, size_t size)`` | allocate a memory block |
+ +----------------------------------------------------------+---------------------------------------+
+ | ``void* realloc(void *ctx, void *ptr, size_t new_size)`` | allocate or resize a memory block |
+ +----------------------------------------------------------+---------------------------------------+
+ | ``void free(void *ctx, void *ptr)`` | free a memory block |
+ +----------------------------------------------------------+---------------------------------------+
+
+.. c:type:: PyMemAllocatorDomain
+
+ Enum used to identify an allocator domain. Domains:
+
+ * :c:data:`PYMEM_DOMAIN_RAW`: functions :c:func:`PyMem_RawMalloc`,
+ :c:func:`PyMem_RawRealloc` and :c:func:`PyMem_RawFree`
+ * :c:data:`PYMEM_DOMAIN_MEM`: functions :c:func:`PyMem_Malloc`,
+ :c:func:`PyMem_Realloc` and :c:func:`PyMem_Free`
+ * :c:data:`PYMEM_DOMAIN_OBJ`: functions :c:func:`PyObject_Malloc`,
+ :c:func:`PyObject_Realloc` and :c:func:`PyObject_Free`
+
+
+.. c:function:: void PyMem_GetAllocator(PyMemAllocatorDomain domain, PyMemAllocator *allocator)
+
+ Get the memory block allocator of the specified domain.
+
+
+.. c:function:: void PyMem_SetAllocator(PyMemAllocatorDomain domain, PyMemAllocator *allocator)
+
+ Set the memory block allocator of the specified domain.
+
+ The new allocator must return a distinct non-NULL pointer when requesting
+ zero bytes.
+
+ For the :c:data:`PYMEM_DOMAIN_RAW` domain, the allocator must be
+ thread-safe: the :term:`GIL <global interpreter lock>` is not held when the
+ allocator is called.
+
+ If the new allocator is not a hook (does not call the previous allocator),
+ the :c:func:`PyMem_SetupDebugHooks` function must be called to reinstall the
+ debug hooks on top on the new allocator.
+
+
+.. c:function:: void PyMem_SetupDebugHooks(void)
+
+ Setup hooks to detect bugs in the following Python memory allocator
+ functions:
+
+ - :c:func:`PyMem_RawMalloc`, :c:func:`PyMem_RawRealloc`,
+ :c:func:`PyMem_RawFree`
+ - :c:func:`PyMem_Malloc`, :c:func:`PyMem_Realloc`, :c:func:`PyMem_Free`
+ - :c:func:`PyObject_Malloc`, :c:func:`PyObject_Realloc`,
+ :c:func:`PyObject_Free`
+
+ Newly allocated memory is filled with the byte ``0xCB``, freed memory is
+ filled with the byte ``0xDB``. Additionnal checks:
+
+ - detect API violations, ex: :c:func:`PyObject_Free` called on a buffer
+ allocated by :c:func:`PyMem_Malloc`
+ - detect write before the start of the buffer (buffer underflow)
+ - detect write after the end of the buffer (buffer overflow)
+
+ The function does nothing if Python is not compiled is debug mode.
+
+
+Customize PyObject Arena Allocator
+==================================
+
+Python has a *pymalloc* allocator for allocations smaller than 512 bytes. This
+allocator is optimized for small objects with a short lifetime. It uses memory
+mappings called "arenas" with a fixed size of 256 KB. It falls back to
+:c:func:`PyMem_RawMalloc` and :c:func:`PyMem_RawRealloc` for allocations larger
+than 512 bytes. *pymalloc* is the default allocator used by
+:c:func:`PyObject_Malloc`.
+
+The default arena allocator uses the following functions:
+
+* :c:func:`VirtualAlloc` and :c:func:`VirtualFree` on Windows,
+* :c:func:`mmap` and :c:func:`munmap` if available,
+* :c:func:`malloc` and :c:func:`free` otherwise.
+
+.. versionadded:: 3.4
+
+.. c:type:: PyObjectArenaAllocator
+
+ Structure used to describe an arena allocator. The structure has
+ three fields:
+
+ +--------------------------------------------------+---------------------------------------+
+ | Field | Meaning |
+ +==================================================+=======================================+
+ | ``void *ctx`` | user context passed as first argument |
+ +--------------------------------------------------+---------------------------------------+
+ | ``void* alloc(void *ctx, size_t size)`` | allocate an arena of size bytes |
+ +--------------------------------------------------+---------------------------------------+
+ | ``void free(void *ctx, size_t size, void *ptr)`` | free an arena |
+ +--------------------------------------------------+---------------------------------------+
+
+.. c:function:: PyObject_GetArenaAllocator(PyObjectArenaAllocator *allocator)
+
+ Get the arena allocator.
+
+.. c:function:: PyObject_SetArenaAllocator(PyObjectArenaAllocator *allocator)
+
+ Set the arena allocator.
+
+
.. _memoryexamples:
Examples
diff --git a/Doc/c-api/module.rst b/Doc/c-api/module.rst
index e80e0ea..26c4384 100644
--- a/Doc/c-api/module.rst
+++ b/Doc/c-api/module.rst
@@ -35,13 +35,20 @@ There are only a few functions special to module objects.
single: __name__ (module attribute)
single: __doc__ (module attribute)
single: __file__ (module attribute)
+ single: __package__ (module attribute)
+ single: __loader__ (module attribute)
Return a new module object with the :attr:`__name__` attribute set to *name*.
- Only the module's :attr:`__doc__` and :attr:`__name__` attributes are filled in;
- the caller is responsible for providing a :attr:`__file__` attribute.
+ The module's :attr:`__name__`, :attr:`__doc__`, :attr:`__package__`, and
+ :attr:`__loader__` attributes are filled in (all but :attr:`__name__` are set
+ to ``None``); the caller is responsible for providing a :attr:`__file__`
+ attribute.
.. versionadded:: 3.3
+ .. versionchanged:: 3.4
+ :attr:`__package__` and :attr:`__loader__` are set to ``None``.
+
.. c:function:: PyObject* PyModule_New(const char *name)
diff --git a/Doc/c-api/object.rst b/Doc/c-api/object.rst
index 0aba360..be6d798 100644
--- a/Doc/c-api/object.rst
+++ b/Doc/c-api/object.rst
@@ -240,7 +240,7 @@ attribute is considered sufficient for this determination.
of the Python expression ``callable_object(*args)``.
-.. c:function:: PyObject* PyObject_CallFunction(PyObject *callable, char *format, ...)
+.. c:function:: PyObject* PyObject_CallFunction(PyObject *callable, const char *format, ...)
Call a callable Python object *callable*, with a variable number of C arguments.
The C arguments are described using a :c:func:`Py_BuildValue` style format
@@ -250,8 +250,11 @@ attribute is considered sufficient for this determination.
pass :c:type:`PyObject \*` args, :c:func:`PyObject_CallFunctionObjArgs` is a
faster alternative.
+ .. versionchanged:: 3.4
+ The type of *format* was changed from ``char *``.
-.. c:function:: PyObject* PyObject_CallMethod(PyObject *o, char *method, char *format, ...)
+
+.. c:function:: PyObject* PyObject_CallMethod(PyObject *o, const char *method, const char *format, ...)
Call the method named *method* of object *o* with a variable number of C
arguments. The C arguments are described by a :c:func:`Py_BuildValue` format
@@ -261,6 +264,9 @@ attribute is considered sufficient for this determination.
Note that if you only pass :c:type:`PyObject \*` args,
:c:func:`PyObject_CallMethodObjArgs` is a faster alternative.
+ .. versionchanged:: 3.4
+ The types of *method* and *format* were changed from ``char *``.
+
.. c:function:: PyObject* PyObject_CallFunctionObjArgs(PyObject *callable, ..., NULL)
@@ -342,6 +348,15 @@ attribute is considered sufficient for this determination.
returned. This is the equivalent to the Python expression ``len(o)``.
+.. c:function:: Py_ssize_t PyObject_LengthHint(PyObject *o, Py_ssize_t default)
+
+ Return an estimated length for the object *o*. First trying to return its
+ actual length, then an estimate using ``__length_hint__``, and finally
+ returning the default value. On error ``-1`` is returned. This is the
+ equivalent to the Python expression ``operator.length_hint(o, default)``.
+
+ .. versionadded:: 3.4
+
.. c:function:: PyObject* PyObject_GetItem(PyObject *o, PyObject *key)
Return element of *o* corresponding to the object *key* or *NULL* on failure.
diff --git a/Doc/c-api/typeobj.rst b/Doc/c-api/typeobj.rst
index b2b8038..48b13e5 100644
--- a/Doc/c-api/typeobj.rst
+++ b/Doc/c-api/typeobj.rst
@@ -465,6 +465,14 @@ type objects) *must* have the :attr:`ob_size` field.
:const:`Py_TPFLAGS_HAVE_VERSION_TAG`.
+ .. data:: Py_TPFLAGS_HAVE_FINALIZE
+
+ This bit is set when the :c:member:`~PyTypeObject.tp_finalize` slot is present in the
+ type structure.
+
+ .. versionadded:: 3.4
+
+
.. c:member:: char* PyTypeObject.tp_doc
An optional pointer to a NUL-terminated C string giving the docstring for this
@@ -968,6 +976,47 @@ type objects) *must* have the :attr:`ob_size` field.
This field is not inherited; it is calculated fresh by :c:func:`PyType_Ready`.
+.. c:member:: destructor PyTypeObject.tp_finalize
+
+ An optional pointer to an instance finalization function. Its signature is
+ :c:type:`destructor`::
+
+ void tp_finalize(PyObject *)
+
+ If :c:member:`~PyTypeObject.tp_finalize` is set, the interpreter calls it once when
+ finalizing an instance. It is called either from the garbage
+ collector (if the instance is part of an isolated reference cycle) or
+ just before the object is deallocated. Either way, it is guaranteed
+ to be called before attempting to break reference cycles, ensuring
+ that it finds the object in a sane state.
+
+ :c:member:`~PyTypeObject.tp_finalize` should not mutate the current exception status;
+ therefore, a recommended way to write a non-trivial finalizer is::
+
+ static void
+ local_finalize(PyObject *self)
+ {
+ PyObject *error_type, *error_value, *error_traceback;
+
+ /* Save the current exception, if any. */
+ PyErr_Fetch(&error_type, &error_value, &error_traceback);
+
+ /* ... */
+
+ /* Restore the saved exception. */
+ PyErr_Restore(error_type, error_value, error_traceback);
+ }
+
+ For this field to be taken into account (even through inheritance),
+ you must also set the :const:`Py_TPFLAGS_HAVE_FINALIZE` flags bit.
+
+ This field is inherited by subtypes.
+
+ .. versionadded:: 3.4
+
+ .. seealso:: "Safe object finalization" (:pep:`442`)
+
+
.. c:member:: PyObject* PyTypeObject.tp_cache
Unused. Not inherited. Internal use only.
diff --git a/Doc/c-api/unicode.rst b/Doc/c-api/unicode.rst
index 3649cfb..c7ed5e5 100644
--- a/Doc/c-api/unicode.rst
+++ b/Doc/c-api/unicode.rst
@@ -526,12 +526,23 @@ APIs:
The `"%lld"` and `"%llu"` format specifiers are only available
when :const:`HAVE_LONG_LONG` is defined.
+ .. note::
+ The width formatter unit is number of characters rather than bytes.
+ The precision formatter unit is number of bytes for ``"%s"`` and
+ ``"%V"`` (if the ``PyObject*`` argument is NULL), and a number of
+ characters for ``"%A"``, ``"%U"``, ``"%S"``, ``"%R"`` and ``"%V"``
+ (if the ``PyObject*`` argument is not NULL).
+
.. versionchanged:: 3.2
Support for ``"%lld"`` and ``"%llu"`` added.
.. versionchanged:: 3.3
Support for ``"%li"``, ``"%lli"`` and ``"%zi"`` added.
+ .. versionchanged:: 3.4
+ Support width and precision formatter for ``"%s"``, ``"%A"``, ``"%U"``,
+ ``"%V"``, ``"%S"``, ``"%R"`` added.
+
.. c:function:: PyObject* PyUnicode_FromFormatV(const char *format, va_list vargs)
diff --git a/Doc/c-api/veryhigh.rst b/Doc/c-api/veryhigh.rst
index 14ef8df..9f21b89 100644
--- a/Doc/c-api/veryhigh.rst
+++ b/Doc/c-api/veryhigh.rst
@@ -144,6 +144,37 @@ the same library that the Python runtime is using.
(:func:`sys.getfilesystemencoding`). Returns ``0`` at EOF.
+.. c:var:: int (*PyOS_InputHook)(void)
+
+ Can be set to point to a function with the prototype
+ ``int func(void)``. The function will be called when Python's
+ interpreter prompt is about to become idle and wait for user input
+ from the terminal. The return value is ignored. Overriding this
+ hook can be used to integrate the interpreter's prompt with other
+ event loops, as done in the :file:`Modules/_tkinter.c` in the
+ Python source code.
+
+
+.. c:var:: char* (*PyOS_ReadlineFunctionPointer)(FILE *, FILE *, const char *)
+
+ Can be set to point to a function with the prototype
+ ``char *func(FILE *stdin, FILE *stdout, char *prompt)``,
+ overriding the default function used to read a single line of input
+ at the interpreter's prompt. The function is expected to output
+ the string *prompt* if it's not *NULL*, and then read a line of
+ input from the provided standard input file, returning the
+ resulting string. For example, The :mod:`readline` module sets
+ this hook to provide line-editing and tab-completion features.
+
+ The result must be a string allocated by :c:func:`PyMem_RawMalloc` or
+ :c:func:`PyMem_RawRealloc`, or *NULL* if an error occurred.
+
+ .. versionchanged:: 3.4
+ The result must be allocated by :c:func:`PyMem_RawMalloc` or
+ :c:func:`PyMem_RawRealloc`, instead of being allocated by
+ :c:func:`PyMem_Malloc` or :c:func:`PyMem_Realloc`.
+
+
.. c:function:: struct _node* PyParser_SimpleParseString(const char *str, int start)
This is a simplified interface to
@@ -235,16 +266,15 @@ the same library that the Python runtime is using.
*optimize* set to ``-1``.
-.. c:function:: PyObject* Py_CompileStringExFlags(const char *str, const char *filename, int start, PyCompilerFlags *flags, int optimize)
+.. c:function:: PyObject* Py_CompileStringObject(const char *str, PyObject *filename, int start, PyCompilerFlags *flags, int optimize)
Parse and compile the Python source code in *str*, returning the resulting code
object. The start token is given by *start*; this can be used to constrain the
code which can be compiled and should be :const:`Py_eval_input`,
:const:`Py_file_input`, or :const:`Py_single_input`. The filename specified by
*filename* is used to construct the code object and may appear in tracebacks or
- :exc:`SyntaxError` exception messages, it is decoded from the filesystem
- encoding (:func:`sys.getfilesystemencoding`). This returns *NULL* if the
- code cannot be parsed or compiled.
+ :exc:`SyntaxError` exception messages. This returns *NULL* if the code
+ cannot be parsed or compiled.
The integer *optimize* specifies the optimization level of the compiler; a
value of ``-1`` selects the optimization level of the interpreter as given by
@@ -252,8 +282,15 @@ the same library that the Python runtime is using.
``__debug__`` is true), ``1`` (asserts are removed, ``__debug__`` is false)
or ``2`` (docstrings are removed too).
- .. versionadded:: 3.2
+ .. versionadded:: 3.4
+
+
+.. c:function:: PyObject* Py_CompileStringExFlags(const char *str, const char *filename, int start, PyCompilerFlags *flags, int optimize)
+
+ Like :c:func:`Py_CompileStringExFlags`, but *filename* is a byte string
+ decoded from the filesystem encoding (:func:`os.fsdecode`).
+ .. versionadded:: 3.2
.. c:function:: PyObject* PyEval_EvalCode(PyObject *co, PyObject *globals, PyObject *locals)
@@ -338,4 +375,3 @@ the same library that the Python runtime is using.
This bit can be set in *flags* to cause division operator ``/`` to be
interpreted as "true division" according to :pep:`238`.
-
diff --git a/Doc/data/refcounts.dat b/Doc/data/refcounts.dat
index a42584c..814c3b0 100644
--- a/Doc/data/refcounts.dat
+++ b/Doc/data/refcounts.dat
@@ -220,6 +220,11 @@ PyDict_GetItemString:PyObject*::0:
PyDict_GetItemString:PyObject*:p:0:
PyDict_GetItemString:const char*:key::
+PyDict_SetDefault:PyObject*::0:
+PyDict_SetDefault:PyObject*:p:0:
+PyDict_SetDefault:PyObject*:key:0:conditionally +1 if inserted into the dict
+PyDict_SetDefault:PyObject*:default:0:conditionally +1 if inserted into the dict
+
PyDict_Items:PyObject*::+1:
PyDict_Items:PyObject*:p:0:
@@ -503,13 +508,13 @@ PyImport_AddModule:const char*:name::
PyImport_Cleanup:void:::
PyImport_ExecCodeModule:PyObject*::+1:
-PyImport_ExecCodeModule:char*:name::
+PyImport_ExecCodeModule:const char*:name::
PyImport_ExecCodeModule:PyObject*:co:0:
PyImport_ExecCodeModuleEx:PyObject*::+1:
-PyImport_ExecCodeModuleEx:char*:name::
+PyImport_ExecCodeModuleEx:const char*:name::
PyImport_ExecCodeModuleEx:PyObject*:co:0:
-PyImport_ExecCodeModuleEx:char*:pathname::
+PyImport_ExecCodeModuleEx:const char*:pathname::
PyImport_GetMagicNumber:long:::
@@ -519,7 +524,7 @@ PyImport_Import:PyObject*::+1:
PyImport_Import:PyObject*:name:0:
PyImport_ImportFrozenModule:int:::
-PyImport_ImportFrozenModule:char*:::
+PyImport_ImportFrozenModule:const char*:::
PyImport_ImportModule:PyObject*::+1:
PyImport_ImportModule:const char*:name::
@@ -668,7 +673,7 @@ PyLong_FromUnsignedLongLong:PyObject*::+1:
PyLong_FromUnsignedLongLong:unsigned long long:v::
PyLong_FromString:PyObject*::+1:
-PyLong_FromString:char*:str::
+PyLong_FromString:const char*:str::
PyLong_FromString:char**:pend::
PyLong_FromString:int:base::
@@ -696,7 +701,7 @@ PyMapping_DelItemString:const char*:key::
PyMapping_GetItemString:PyObject*::+1:
PyMapping_GetItemString:PyObject*:o:0:
-PyMapping_GetItemString:char*:key::
+PyMapping_GetItemString:const char*:key::
PyMapping_HasKey:int:::
PyMapping_HasKey:PyObject*:o:0:
@@ -704,7 +709,7 @@ PyMapping_HasKey:PyObject*:key::
PyMapping_HasKeyString:int:::
PyMapping_HasKeyString:PyObject*:o:0:
-PyMapping_HasKeyString:char*:key::
+PyMapping_HasKeyString:const char*:key::
PyMapping_Items:PyObject*::+1:
PyMapping_Items:PyObject*:o:0:
@@ -717,7 +722,7 @@ PyMapping_Length:PyObject*:o:0:
PyMapping_SetItemString:int:::
PyMapping_SetItemString:PyObject*:o:0:
-PyMapping_SetItemString:char*:key::
+PyMapping_SetItemString:const char*:key::
PyMapping_SetItemString:PyObject*:v:+1:
PyMapping_Values:PyObject*::+1:
@@ -730,7 +735,7 @@ PyMarshal_ReadObjectFromFile:PyObject*::+1:
PyMarshal_ReadObjectFromFile:FILE*:file::
PyMarshal_ReadObjectFromString:PyObject*::+1:
-PyMarshal_ReadObjectFromString:char*:string::
+PyMarshal_ReadObjectFromString:const char*:string::
PyMarshal_ReadObjectFromString:int:len::
PyMarshal_WriteObjectToString:PyObject*::+1:
@@ -912,7 +917,7 @@ PyObject_Call:PyObject*:kw:0:
PyObject_CallFunction:PyObject*::+1:
PyObject_CallFunction:PyObject*:callable_object:0:
-PyObject_CallFunction:char*:format::
+PyObject_CallFunction:const char*:format::
PyObject_CallFunction::...::
PyObject_CallFunctionObjArgs:PyObject*::+1:
@@ -921,8 +926,8 @@ PyObject_CallFunctionObjArgs::...::
PyObject_CallMethod:PyObject*::+1:
PyObject_CallMethod:PyObject*:o:0:
-PyObject_CallMethod:char*:m::
-PyObject_CallMethod:char*:format::
+PyObject_CallMethod:const char*:m::
+PyObject_CallMethod:const char*:format::
PyObject_CallMethod::...::
PyObject_CallMethodObjArgs:PyObject*::+1:
diff --git a/Doc/distutils/builtdist.rst b/Doc/distutils/builtdist.rst
index d1ab7db..83c68ae 100644
--- a/Doc/distutils/builtdist.rst
+++ b/Doc/distutils/builtdist.rst
@@ -239,7 +239,8 @@ tedious and error-prone, so it's usually best to put them in the setup
configuration file, :file:`setup.cfg`\ ---see section :ref:`setup-config`. If
you distribute or package many Python module distributions, you might want to
put options that apply to all of them in your personal Distutils configuration
-file (:file:`~/.pydistutils.cfg`).
+file (:file:`~/.pydistutils.cfg`). If you want to temporarily disable
+this file, you can pass the :option:`--no-user-cfg` option to :file:`setup.py`.
There are three steps to building a binary RPM package, all of which are
handled automatically by the Distutils:
diff --git a/Doc/distutils/sourcedist.rst b/Doc/distutils/sourcedist.rst
index 1666436..9f7a38e 100644
--- a/Doc/distutils/sourcedist.rst
+++ b/Doc/distutils/sourcedist.rst
@@ -26,16 +26,16 @@ to create a gzipped tarball and a zip file. The available formats are:
+===========+=========================+=========+
| ``zip`` | zip file (:file:`.zip`) | (1),(3) |
+-----------+-------------------------+---------+
-| ``gztar`` | gzip'ed tar file | (2),(4) |
+| ``gztar`` | gzip'ed tar file | \(2) |
| | (:file:`.tar.gz`) | |
+-----------+-------------------------+---------+
-| ``bztar`` | bzip2'ed tar file | \(4) |
+| ``bztar`` | bzip2'ed tar file | |
| | (:file:`.tar.bz2`) | |
+-----------+-------------------------+---------+
| ``ztar`` | compressed tar file | \(4) |
| | (:file:`.tar.Z`) | |
+-----------+-------------------------+---------+
-| ``tar`` | tar file (:file:`.tar`) | \(4) |
+| ``tar`` | tar file (:file:`.tar`) | |
+-----------+-------------------------+---------+
Notes:
@@ -51,8 +51,16 @@ Notes:
of the standard Python library since Python 1.6)
(4)
- requires external utilities: :program:`tar` and possibly one of :program:`gzip`,
- :program:`bzip2`, or :program:`compress`
+ requires the :program:`compress` program. Notice that this format is now
+ pending for deprecation and will be removed in the future versions of Python.
+
+When using any ``tar`` format (``gztar``, ``bztar``, ``ztar`` or
+``tar``), under Unix you can specify the ``owner`` and ``group`` names
+that will be set for each member of the archive.
+
+For example, if you want all files of the archive to be owned by root::
+
+ python setup.py sdist --owner=root --group=root
.. _manifest:
diff --git a/Doc/extending/embedding.rst b/Doc/extending/embedding.rst
index bd66086..6cb686a 100644
--- a/Doc/extending/embedding.rst
+++ b/Doc/extending/embedding.rst
@@ -285,14 +285,14 @@ be directly useful to you:
* ``pythonX.Y-config --cflags`` will give you the recommended flags when
compiling::
- $ /opt/bin/python3.3-config --cflags
- -I/opt/include/python3.3m -I/opt/include/python3.3m -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
+ $ /opt/bin/python3.4-config --cflags
+ -I/opt/include/python3.4m -I/opt/include/python3.4m -DNDEBUG -g -fwrapv -O3 -Wall -Wstrict-prototypes
* ``pythonX.Y-config --ldflags`` will give you the recommended flags when
linking::
- $ /opt/bin/python3.3-config --ldflags
- -L/opt/lib/python3.3/config-3.3m -lpthread -ldl -lutil -lm -lpython3.3m -Xlinker -export-dynamic
+ $ /opt/bin/python3.4-config --ldflags
+ -L/opt/lib/python3.4/config-3.4m -lpthread -ldl -lutil -lm -lpython3.4m -Xlinker -export-dynamic
.. note::
To avoid confusion between several Python installations (and especially
diff --git a/Doc/extending/newtypes.rst b/Doc/extending/newtypes.rst
index f484ba4..45b5721 100644
--- a/Doc/extending/newtypes.rst
+++ b/Doc/extending/newtypes.rst
@@ -157,7 +157,8 @@ to :const:`Py_TPFLAGS_DEFAULT`. ::
Py_TPFLAGS_DEFAULT, /* tp_flags */
All types should include this constant in their flags. It enables all of the
-members defined by the current version of Python.
+members defined until at least Python 3.3. If you need further members,
+you will need to OR the corresponding flags.
We provide a doc string for the type in :c:member:`~PyTypeObject.tp_doc`. ::
@@ -928,8 +929,9 @@ Finalization and De-allocation
This function is called when the reference count of the instance of your type is
reduced to zero and the Python interpreter wants to reclaim it. If your type
-has memory to free or other clean-up to perform, put it here. The object itself
-needs to be freed here as well. Here is an example of this function::
+has memory to free or other clean-up to perform, you can put it here. The
+object itself needs to be freed here as well. Here is an example of this
+function::
static void
newdatatype_dealloc(newdatatypeobject * obj)
@@ -980,6 +982,22 @@ done. This can be done using the :c:func:`PyErr_Fetch` and
Py_TYPE(obj)->tp_free((PyObject*)self);
}
+.. note::
+ There are limitations to what you can safely do in a deallocator function.
+ First, if your type supports garbage collection (using :c:member:`~PyTypeObject.tp_traverse`
+ and/or :c:member:`~PyTypeObject.tp_clear`), some of the object's members can have been
+ cleared or finalized by the time :c:member:`~PyTypeObject.tp_dealloc` is called. Second, in
+ :c:member:`~PyTypeObject.tp_dealloc`, your object is in an unstable state: its reference
+ count is equal to zero. Any call to a non-trivial object or API (as in the
+ example above) might end up calling :c:member:`~PyTypeObject.tp_dealloc` again, causing a
+ double free and a crash.
+
+ Starting with Python 3.4, it is recommended not to put any complex
+ finalization code in :c:member:`~PyTypeObject.tp_dealloc`, and instead use the new
+ :c:member:`~PyTypeObject.tp_finalize` type method.
+
+ .. seealso::
+ :pep:`442` explains the new finalization scheme.
.. index::
single: string; object representation
diff --git a/Doc/faq/library.rst b/Doc/faq/library.rst
index c8ef9e7..8bd774b 100644
--- a/Doc/faq/library.rst
+++ b/Doc/faq/library.rst
@@ -211,7 +211,7 @@ using curses, but curses is a fairly large module to learn.
try:
c = sys.stdin.read(1)
print("Got character", repr(c))
- except IOError:
+ except OSError:
pass
finally:
termios.tcsetattr(fd, termios.TCSAFLUSH, oldterm)
@@ -224,7 +224,11 @@ using curses, but curses is a fairly large module to learn.
:func:`termios.tcsetattr` turns off stdin's echoing and disables canonical
mode. :func:`fcntl.fnctl` is used to obtain stdin's file descriptor flags
and modify them for non-blocking mode. Since reading stdin when it is empty
- results in an :exc:`IOError`, this error is caught and ignored.
+ results in an :exc:`OSError`, this error is caught and ignored.
+
+ .. versionchanged:: 3.3
+ *sys.stdin.read* used to raise :exc:`IOError`. Starting from Python 3.3
+ :exc:`IOError` is alias for :exc:`OSError`.
Threads
diff --git a/Doc/faq/programming.rst b/Doc/faq/programming.rst
index 3fd31b4..12f1a2d 100644
--- a/Doc/faq/programming.rst
+++ b/Doc/faq/programming.rst
@@ -1765,12 +1765,12 @@ When I edit an imported module and reimport it, the changes don't show up. Why
For reasons of efficiency as well as consistency, Python only reads the module
file on the first time a module is imported. If it didn't, in a program
consisting of many modules where each one imports the same basic module, the
-basic module would be parsed and re-parsed many times. To force rereading of a
+basic module would be parsed and re-parsed many times. To force re-reading of a
changed module, do this::
- import imp
+ import importlib
import modname
- imp.reload(modname)
+ importlib.reload(modname)
Warning: this technique is not 100% fool-proof. In particular, modules
containing statements like ::
@@ -1782,10 +1782,10 @@ module contains class definitions, existing class instances will *not* be
updated to use the new class definition. This can result in the following
paradoxical behaviour:
- >>> import imp
+ >>> import importlib
>>> import cls
>>> c = cls.C() # Create an instance of C
- >>> imp.reload(cls)
+ >>> importlib.reload(cls)
<module 'cls' from 'cls.py'>
>>> isinstance(c, cls.C) # isinstance is false?!?
False
diff --git a/Doc/glossary.rst b/Doc/glossary.rst
index 71e4686..df47051 100644
--- a/Doc/glossary.rst
+++ b/Doc/glossary.rst
@@ -302,6 +302,15 @@ Glossary
>>> sum(i*i for i in range(10)) # sum of squares 0, 1, 4, ... 81
285
+ generic function
+ A function composed of multiple functions implementing the same operation
+ for different types. Which implementation should be used during a call is
+ determined by the dispatch algorithm.
+
+ See also the :term:`single dispatch` glossary entry, the
+ :func:`functools.singledispatch` decorator, and :pep:`443`.
+
+
GIL
See :term:`global interpreter lock`.
@@ -663,20 +672,27 @@ Glossary
positional argument
See :term:`argument`.
- provisional package
- A provisional package is one which has been deliberately excluded from
+ provisional API
+ A provisional API is one which has been deliberately excluded from
the standard library's backwards compatibility guarantees. While major
- changes to such packages are not expected, as long as they are marked
+ changes to such interfaces are not expected, as long as they are marked
provisional, backwards incompatible changes (up to and including removal
- of the package) may occur if deemed necessary by core developers. Such
+ of the interface) may occur if deemed necessary by core developers. Such
changes will not be made gratuitously -- they will occur only if serious
- flaws are uncovered that were missed prior to the inclusion of the
- package.
+ fundamental flaws are uncovered that were missed prior to the inclusion
+ of the API.
+
+ Even for provisional APIs, backwards incompatible changes are seen as
+ a "solution of last resort" - every attempt will still be made to find
+ a backwards compatible resolution to any identified problems.
This process allows the standard library to continue to evolve over
time, without locking in problematic design errors for extended periods
of time. See :pep:`411` for more details.
+ provisional package
+ See :term:`provisional API`.
+
Python 3000
Nickname for the Python 3.x release line (coined long ago when the
release of version 3 was something in the distant future.) This is also
@@ -755,6 +771,10 @@ Glossary
mapping rather than a sequence because the lookups use arbitrary
:term:`immutable` keys rather than integers.
+ single dispatch
+ A form of :term:`generic function` dispatch where the implementation is
+ chosen based on the type of a single argument.
+
slice
An object usually containing a portion of a :term:`sequence`. A slice is
created using the subscript notation, ``[]`` with colons between numbers
diff --git a/Doc/howto/functional.rst b/Doc/howto/functional.rst
index d241f1a..0f4c4e4 100644
--- a/Doc/howto/functional.rst
+++ b/Doc/howto/functional.rst
@@ -3,7 +3,7 @@
********************************
:Author: A. M. Kuchling
-:Release: 0.31
+:Release: 0.32
In this document, we'll take a tour of Python's features suitable for
implementing programs in a functional style. After an introduction to the
@@ -15,9 +15,9 @@ concepts of functional programming, we'll look at language features such as
Introduction
============
-This section explains the basic concept of functional programming; if you're
-just interested in learning about Python language features, skip to the next
-section.
+This section explains the basic concept of functional programming; if
+you're just interested in learning about Python language features,
+skip to the next section on :ref:`functional-howto-iterators`.
Programming languages support decomposing problems in several different ways:
@@ -173,6 +173,8 @@ new programs by arranging existing functions in a new configuration and writing
a few functions specialized for the current task.
+.. _functional-howto-iterators:
+
Iterators
=========
@@ -670,7 +672,7 @@ indexes at which certain conditions are met::
:func:`sorted(iterable, key=None, reverse=False) <sorted>` collects all the
elements of the iterable into a list, sorts the list, and returns the sorted
-result. The *key*, and *reverse* arguments are passed through to the
+result. The *key* and *reverse* arguments are passed through to the
constructed list's :meth:`~list.sort` method. ::
>>> import random
@@ -836,7 +838,8 @@ Another group of functions chooses a subset of an iterator's elements based on a
predicate.
:func:`itertools.filterfalse(predicate, iter) <itertools.filterfalse>` is the
-opposite, returning all elements for which the predicate returns false::
+opposite of :func:`filter`, returning all elements for which the predicate
+returns false::
itertools.filterfalse(is_even, itertools.count()) =>
1, 3, 5, 7, 9, 11, 13, 15, ...
@@ -864,6 +867,77 @@ iterable's results. ::
itertools.dropwhile(is_even, itertools.count()) =>
1, 2, 3, 4, 5, 6, 7, 8, 9, 10, ...
+:func:`itertools.compress(data, selectors) <itertools.compress>` takes two
+iterators and returns only those elements of *data* for which the corresponding
+element of *selectors* is true, stopping whenever either one is exhausted::
+
+ itertools.compress([1,2,3,4,5], [True, True, False, False, True]) =>
+ 1, 2, 5
+
+
+Combinatoric functions
+----------------------
+
+The :func:`itertools.combinations(iterable, r) <itertools.combinations>`
+returns an iterator giving all possible *r*-tuple combinations of the
+elements contained in *iterable*. ::
+
+ itertools.combinations([1, 2, 3, 4, 5], 2) =>
+ (1, 2), (1, 3), (1, 4), (1, 5),
+ (2, 3), (2, 4), (2, 5),
+ (3, 4), (3, 5),
+ (4, 5)
+
+ itertools.combinations([1, 2, 3, 4, 5], 3) =>
+ (1, 2, 3), (1, 2, 4), (1, 2, 5), (1, 3, 4), (1, 3, 5), (1, 4, 5),
+ (2, 3, 4), (2, 3, 5), (2, 4, 5),
+ (3, 4, 5)
+
+The elements within each tuple remain in the same order as
+*iterable* returned them. For example, the number 1 is always before
+2, 3, 4, or 5 in the examples above. A similar function,
+:func:`itertools.permutations(iterable, r=None) <itertools.permutations>`,
+removes this constraint on the order, returning all possible
+arrangements of length *r*::
+
+ itertools.permutations([1, 2, 3, 4, 5], 2) =>
+ (1, 2), (1, 3), (1, 4), (1, 5),
+ (2, 1), (2, 3), (2, 4), (2, 5),
+ (3, 1), (3, 2), (3, 4), (3, 5),
+ (4, 1), (4, 2), (4, 3), (4, 5),
+ (5, 1), (5, 2), (5, 3), (5, 4)
+
+ itertools.permutations([1, 2, 3, 4, 5]) =>
+ (1, 2, 3, 4, 5), (1, 2, 3, 5, 4), (1, 2, 4, 3, 5),
+ ...
+ (5, 4, 3, 2, 1)
+
+If you don't supply a value for *r* the length of the iterable is used,
+meaning that all the elements are permuted.
+
+Note that these functions produce all of the possible combinations by
+position and don't require that the contents of *iterable* are unique::
+
+ itertools.permutations('aba', 3) =>
+ ('a', 'b', 'a'), ('a', 'a', 'b'), ('b', 'a', 'a'),
+ ('b', 'a', 'a'), ('a', 'a', 'b'), ('a', 'b', 'a')
+
+The identical tuple ``('a', 'a', 'b')`` occurs twice, but the two 'a'
+strings came from different positions.
+
+The :func:`itertools.combinations_with_replacement(iterable, r) <itertools.combinations_with_replacement>`
+function relaxes a different constraint: elements can be repeated
+within a single tuple. Conceptually an element is selected for the
+first position of each tuple and then is replaced before the second
+element is selected. ::
+
+ itertools.combinations_with_replacement([1, 2, 3, 4, 5], 2) =>
+ (1, 1), (1, 2), (1, 3), (1, 4), (1, 5),
+ (2, 2), (2, 3), (2, 4), (2, 5),
+ (3, 3), (3, 4), (3, 5),
+ (4, 4), (4, 5),
+ (5, 5)
+
Grouping elements
-----------------
@@ -986,6 +1060,17 @@ write the obvious :keyword:`for` loop::
for i in [1,2,3]:
product *= i
+A related function is `itertools.accumulate(iterable, func=operator.add) <itertools.accumulate`.
+It performs the same calculation, but instead of returning only the
+final result, :func:`accumulate` returns an iterator that also yields
+each partial result::
+
+ itertools.accumulate([1,2,3,4,5]) =>
+ 1, 3, 6, 10, 15
+
+ itertools.accumulate([1,2,3,4,5], operator.mul) =>
+ 1, 2, 6, 24, 120
+
The operator module
-------------------
@@ -1159,51 +1244,6 @@ features in Python 2.5.
.. comment
- Topics to place
- -----------------------------
-
- XXX os.walk()
-
- XXX Need a large example.
-
- But will an example add much? I'll post a first draft and see
- what the comments say.
-
-.. comment
-
- Original outline:
- Introduction
- Idea of FP
- Programs built out of functions
- Functions are strictly input-output, no internal state
- Opposed to OO programming, where objects have state
-
- Why FP?
- Formal provability
- Assignment is difficult to reason about
- Not very relevant to Python
- Modularity
- Small functions that do one thing
- Debuggability:
- Easy to test due to lack of state
- Easy to verify output from intermediate steps
- Composability
- You assemble a toolbox of functions that can be mixed
-
- Tackling a problem
- Need a significant example
-
- Iterators
- Generators
- The itertools module
- List comprehensions
- Small functions and the lambda statement
- Built-in functions
- map
- filter
-
-.. comment
-
Handy little function for printing part of an iterator -- used
while writing this document.
@@ -1214,5 +1254,3 @@ features in Python 2.5.
sys.stdout.write(str(elem))
sys.stdout.write(', ')
print(elem[-1])
-
-
diff --git a/Doc/howto/logging-cookbook.rst b/Doc/howto/logging-cookbook.rst
index 3f256e1..feb9b6c 100644
--- a/Doc/howto/logging-cookbook.rst
+++ b/Doc/howto/logging-cookbook.rst
@@ -704,9 +704,7 @@ the basis for code meeting your own specific requirements::
break
logger = logging.getLogger(record.name)
logger.handle(record) # No level or filter logic applied - just do it!
- except (KeyboardInterrupt, SystemExit):
- raise
- except:
+ except Exception:
import sys, traceback
print('Whoops! Problem:', file=sys.stderr)
traceback.print_exc(file=sys.stderr)
diff --git a/Doc/howto/logging.rst b/Doc/howto/logging.rst
index 165486a..c7a802a 100644
--- a/Doc/howto/logging.rst
+++ b/Doc/howto/logging.rst
@@ -900,10 +900,10 @@ provided:
disk files, rotating the log file at certain timed intervals.
#. :class:`~handlers.SocketHandler` instances send messages to TCP/IP
- sockets.
+ sockets. Since 3.4, Unix domain sockets are also supported.
#. :class:`~handlers.DatagramHandler` instances send messages to UDP
- sockets.
+ sockets. Since 3.4, Unix domain sockets are also supported.
#. :class:`~handlers.SMTPHandler` instances send messages to a designated
email address.
diff --git a/Doc/howto/urllib2.rst b/Doc/howto/urllib2.rst
index 74eeee0..7b217ea 100644
--- a/Doc/howto/urllib2.rst
+++ b/Doc/howto/urllib2.rst
@@ -507,7 +507,7 @@ than the URL you pass to .add_password() will also match. ::
-- ``ProxyHandler`` (if a proxy setting such as an :envvar:`http_proxy`
environment variable is set), ``UnknownHandler``, ``HTTPHandler``,
``HTTPDefaultErrorHandler``, ``HTTPRedirectHandler``, ``FTPHandler``,
- ``FileHandler``, ``HTTPErrorProcessor``.
+ ``FileHandler``, ``DataHandler``, ``HTTPErrorProcessor``.
``top_level_url`` is in fact *either* a full URL (including the 'http:' scheme
component and the hostname and optionally the port number)
diff --git a/Doc/includes/email-dir.py b/Doc/includes/email-dir.py
index cc5529e..3c7c770 100644
--- a/Doc/includes/email-dir.py
+++ b/Doc/includes/email-dir.py
@@ -8,7 +8,7 @@ import smtplib
# For guessing MIME type based on file name extension
import mimetypes
-from optparse import OptionParser
+from argparse import ArgumentParser
from email import encoders
from email.message import Message
@@ -22,44 +22,36 @@ COMMASPACE = ', '
def main():
- parser = OptionParser(usage="""\
+ parser = ArgumentParser(description="""\
Send the contents of a directory as a MIME message.
-
-Usage: %prog [options]
-
Unless the -o option is given, the email is sent by forwarding to your local
SMTP server, which then does the normal delivery process. Your local machine
must be running an SMTP server.
""")
- parser.add_option('-d', '--directory',
- type='string', action='store',
- help="""Mail the contents of the specified directory,
- otherwise use the current directory. Only the regular
- files in the directory are sent, and we don't recurse to
- subdirectories.""")
- parser.add_option('-o', '--output',
- type='string', action='store', metavar='FILE',
- help="""Print the composed message to FILE instead of
- sending the message to the SMTP server.""")
- parser.add_option('-s', '--sender',
- type='string', action='store', metavar='SENDER',
- help='The value of the From: header (required)')
- parser.add_option('-r', '--recipient',
- type='string', action='append', metavar='RECIPIENT',
- default=[], dest='recipients',
- help='A To: header value (at least one required)')
- opts, args = parser.parse_args()
- if not opts.sender or not opts.recipients:
- parser.print_help()
- sys.exit(1)
- directory = opts.directory
+ parser.add_argument('-d', '--directory',
+ help="""Mail the contents of the specified directory,
+ otherwise use the current directory. Only the regular
+ files in the directory are sent, and we don't recurse to
+ subdirectories.""")
+ parser.add_argument('-o', '--output',
+ metavar='FILE',
+ help="""Print the composed message to FILE instead of
+ sending the message to the SMTP server.""")
+ parser.add_argument('-s', '--sender', required=True,
+ help='The value of the From: header (required)')
+ parser.add_argument('-r', '--recipient', required=True,
+ action='append', metavar='RECIPIENT',
+ default=[], dest='recipients',
+ help='A To: header value (at least one required)')
+ args = parser.parse_args()
+ directory = args.directory
if not directory:
directory = '.'
# Create the enclosing (outer) message
outer = MIMEMultipart()
outer['Subject'] = 'Contents of directory %s' % os.path.abspath(directory)
- outer['To'] = COMMASPACE.join(opts.recipients)
- outer['From'] = opts.sender
+ outer['To'] = COMMASPACE.join(args.recipients)
+ outer['From'] = args.sender
outer.preamble = 'You will not see this in a MIME-aware mail reader.\n'
for filename in os.listdir(directory):
@@ -76,23 +68,19 @@ must be running an SMTP server.
ctype = 'application/octet-stream'
maintype, subtype = ctype.split('/', 1)
if maintype == 'text':
- fp = open(path)
- # Note: we should handle calculating the charset
- msg = MIMEText(fp.read(), _subtype=subtype)
- fp.close()
+ with open(path) as fp:
+ # Note: we should handle calculating the charset
+ msg = MIMEText(fp.read(), _subtype=subtype)
elif maintype == 'image':
- fp = open(path, 'rb')
- msg = MIMEImage(fp.read(), _subtype=subtype)
- fp.close()
+ with open(path, 'rb') as fp:
+ msg = MIMEImage(fp.read(), _subtype=subtype)
elif maintype == 'audio':
- fp = open(path, 'rb')
- msg = MIMEAudio(fp.read(), _subtype=subtype)
- fp.close()
+ with open(path, 'rb') as fp:
+ msg = MIMEAudio(fp.read(), _subtype=subtype)
else:
- fp = open(path, 'rb')
- msg = MIMEBase(maintype, subtype)
- msg.set_payload(fp.read())
- fp.close()
+ with open(path, 'rb') as fp:
+ msg = MIMEBase(maintype, subtype)
+ msg.set_payload(fp.read())
# Encode the payload using Base64
encoders.encode_base64(msg)
# Set the filename parameter
@@ -100,14 +88,12 @@ must be running an SMTP server.
outer.attach(msg)
# Now send or store the message
composed = outer.as_string()
- if opts.output:
- fp = open(opts.output, 'w')
- fp.write(composed)
- fp.close()
+ if args.output:
+ with open(args.output, 'w') as fp:
+ fp.write(composed)
else:
- s = smtplib.SMTP('localhost')
- s.sendmail(opts.sender, opts.recipients, composed)
- s.quit()
+ with smtplib.SMTP('localhost') as s:
+ s.sendmail(args.sender, args.recipients, composed)
if __name__ == '__main__':
diff --git a/Doc/includes/email-unpack.py b/Doc/includes/email-unpack.py
index 3653543..574a0b6 100644
--- a/Doc/includes/email-unpack.py
+++ b/Doc/includes/email-unpack.py
@@ -8,41 +8,27 @@ import email
import errno
import mimetypes
-from optparse import OptionParser
+from argparse import ArgumentParser
def main():
- parser = OptionParser(usage="""\
+ parser = ArgumentParser(description="""\
Unpack a MIME message into a directory of files.
-
-Usage: %prog [options] msgfile
""")
- parser.add_option('-d', '--directory',
- type='string', action='store',
- help="""Unpack the MIME message into the named
- directory, which will be created if it doesn't already
- exist.""")
- opts, args = parser.parse_args()
- if not opts.directory:
- parser.print_help()
- sys.exit(1)
+ parser.add_argument('-d', '--directory', required=True,
+ help="""Unpack the MIME message into the named
+ directory, which will be created if it doesn't already
+ exist.""")
+ parser.add_argument('msgfile')
+ args = parser.parse_args()
- try:
- msgfile = args[0]
- except IndexError:
- parser.print_help()
- sys.exit(1)
+ with open(args.msgfile) as fp:
+ msg = email.message_from_file(fp)
try:
- os.mkdir(opts.directory)
- except OSError as e:
- # Ignore directory exists error
- if e.errno != errno.EEXIST:
- raise
-
- fp = open(msgfile)
- msg = email.message_from_file(fp)
- fp.close()
+ os.mkdir(args.directory)
+ except FileExistsError:
+ pass
counter = 1
for part in msg.walk():
@@ -59,9 +45,8 @@ Usage: %prog [options] msgfile
ext = '.bin'
filename = 'part-%03d%s' % (counter, ext)
counter += 1
- fp = open(os.path.join(opts.directory, filename), 'wb')
- fp.write(part.get_payload(decode=True))
- fp.close()
+ with open(os.path.join(args.directory, filename), 'wb') as fp:
+ fp.write(part.get_payload(decode=True))
if __name__ == '__main__':
diff --git a/Doc/includes/mp_benchmarks.py b/Doc/includes/mp_benchmarks.py
deleted file mode 100644
index 3763ea9..0000000
--- a/Doc/includes/mp_benchmarks.py
+++ /dev/null
@@ -1,239 +0,0 @@
-#
-# Simple benchmarks for the multiprocessing package
-#
-# Copyright (c) 2006-2008, R Oudkerk
-# All rights reserved.
-#
-
-import time
-import multiprocessing
-import threading
-import queue
-import gc
-
-_timer = time.perf_counter
-
-delta = 1
-
-
-#### TEST_QUEUESPEED
-
-def queuespeed_func(q, c, iterations):
- a = '0' * 256
- c.acquire()
- c.notify()
- c.release()
-
- for i in range(iterations):
- q.put(a)
-
- q.put('STOP')
-
-def test_queuespeed(Process, q, c):
- elapsed = 0
- iterations = 1
-
- while elapsed < delta:
- iterations *= 2
-
- p = Process(target=queuespeed_func, args=(q, c, iterations))
- c.acquire()
- p.start()
- c.wait()
- c.release()
-
- result = None
- t = _timer()
-
- while result != 'STOP':
- result = q.get()
-
- elapsed = _timer() - t
-
- p.join()
-
- print(iterations, 'objects passed through the queue in', elapsed, 'seconds')
- print('average number/sec:', iterations/elapsed)
-
-
-#### TEST_PIPESPEED
-
-def pipe_func(c, cond, iterations):
- a = '0' * 256
- cond.acquire()
- cond.notify()
- cond.release()
-
- for i in range(iterations):
- c.send(a)
-
- c.send('STOP')
-
-def test_pipespeed():
- c, d = multiprocessing.Pipe()
- cond = multiprocessing.Condition()
- elapsed = 0
- iterations = 1
-
- while elapsed < delta:
- iterations *= 2
-
- p = multiprocessing.Process(target=pipe_func,
- args=(d, cond, iterations))
- cond.acquire()
- p.start()
- cond.wait()
- cond.release()
-
- result = None
- t = _timer()
-
- while result != 'STOP':
- result = c.recv()
-
- elapsed = _timer() - t
- p.join()
-
- print(iterations, 'objects passed through connection in',elapsed,'seconds')
- print('average number/sec:', iterations/elapsed)
-
-
-#### TEST_SEQSPEED
-
-def test_seqspeed(seq):
- elapsed = 0
- iterations = 1
-
- while elapsed < delta:
- iterations *= 2
-
- t = _timer()
-
- for i in range(iterations):
- a = seq[5]
-
- elapsed = _timer() - t
-
- print(iterations, 'iterations in', elapsed, 'seconds')
- print('average number/sec:', iterations/elapsed)
-
-
-#### TEST_LOCK
-
-def test_lockspeed(l):
- elapsed = 0
- iterations = 1
-
- while elapsed < delta:
- iterations *= 2
-
- t = _timer()
-
- for i in range(iterations):
- l.acquire()
- l.release()
-
- elapsed = _timer() - t
-
- print(iterations, 'iterations in', elapsed, 'seconds')
- print('average number/sec:', iterations/elapsed)
-
-
-#### TEST_CONDITION
-
-def conditionspeed_func(c, N):
- c.acquire()
- c.notify()
-
- for i in range(N):
- c.wait()
- c.notify()
-
- c.release()
-
-def test_conditionspeed(Process, c):
- elapsed = 0
- iterations = 1
-
- while elapsed < delta:
- iterations *= 2
-
- c.acquire()
- p = Process(target=conditionspeed_func, args=(c, iterations))
- p.start()
-
- c.wait()
-
- t = _timer()
-
- for i in range(iterations):
- c.notify()
- c.wait()
-
- elapsed = _timer() - t
-
- c.release()
- p.join()
-
- print(iterations * 2, 'waits in', elapsed, 'seconds')
- print('average number/sec:', iterations * 2 / elapsed)
-
-####
-
-def test():
- manager = multiprocessing.Manager()
-
- gc.disable()
-
- print('\n\t######## testing Queue.Queue\n')
- test_queuespeed(threading.Thread, queue.Queue(),
- threading.Condition())
- print('\n\t######## testing multiprocessing.Queue\n')
- test_queuespeed(multiprocessing.Process, multiprocessing.Queue(),
- multiprocessing.Condition())
- print('\n\t######## testing Queue managed by server process\n')
- test_queuespeed(multiprocessing.Process, manager.Queue(),
- manager.Condition())
- print('\n\t######## testing multiprocessing.Pipe\n')
- test_pipespeed()
-
- print()
-
- print('\n\t######## testing list\n')
- test_seqspeed(list(range(10)))
- print('\n\t######## testing list managed by server process\n')
- test_seqspeed(manager.list(list(range(10))))
- print('\n\t######## testing Array("i", ..., lock=False)\n')
- test_seqspeed(multiprocessing.Array('i', list(range(10)), lock=False))
- print('\n\t######## testing Array("i", ..., lock=True)\n')
- test_seqspeed(multiprocessing.Array('i', list(range(10)), lock=True))
-
- print()
-
- print('\n\t######## testing threading.Lock\n')
- test_lockspeed(threading.Lock())
- print('\n\t######## testing threading.RLock\n')
- test_lockspeed(threading.RLock())
- print('\n\t######## testing multiprocessing.Lock\n')
- test_lockspeed(multiprocessing.Lock())
- print('\n\t######## testing multiprocessing.RLock\n')
- test_lockspeed(multiprocessing.RLock())
- print('\n\t######## testing lock managed by server process\n')
- test_lockspeed(manager.Lock())
- print('\n\t######## testing rlock managed by server process\n')
- test_lockspeed(manager.RLock())
-
- print()
-
- print('\n\t######## testing threading.Condition\n')
- test_conditionspeed(threading.Thread, threading.Condition())
- print('\n\t######## testing multiprocessing.Condition\n')
- test_conditionspeed(multiprocessing.Process, multiprocessing.Condition())
- print('\n\t######## testing condition managed by a server process\n')
- test_conditionspeed(multiprocessing.Process, manager.Condition())
-
- gc.enable()
-
-if __name__ == '__main__':
- multiprocessing.freeze_support()
- test()
diff --git a/Doc/includes/mp_newtype.py b/Doc/includes/mp_newtype.py
index 7291743..1c796a5 100644
--- a/Doc/includes/mp_newtype.py
+++ b/Doc/includes/mp_newtype.py
@@ -1,11 +1,3 @@
-#
-# This module shows how to use arbitrary callables with a subclass of
-# `BaseManager`.
-#
-# Copyright (c) 2006-2008, R Oudkerk
-# All rights reserved.
-#
-
from multiprocessing import freeze_support
from multiprocessing.managers import BaseManager, BaseProxy
import operator
@@ -27,12 +19,10 @@ def baz():
# Proxy type for generator objects
class GeneratorProxy(BaseProxy):
- _exposed_ = ('next', '__next__')
+ _exposed_ = ['__next__']
def __iter__(self):
return self
def __next__(self):
- return self._callmethod('next')
- def __next__(self):
return self._callmethod('__next__')
# Function to return the operator module
@@ -90,8 +80,6 @@ def test():
op = manager.operator()
print('op.add(23, 45) =', op.add(23, 45))
print('op.pow(2, 94) =', op.pow(2, 94))
- print('op.getslice(range(10), 2, 6) =', op.getslice(list(range(10)), 2, 6))
- print('op.repeat(range(5), 3) =', op.repeat(list(range(5)), 3))
print('op._exposed_ =', op._exposed_)
##
diff --git a/Doc/includes/mp_pool.py b/Doc/includes/mp_pool.py
index 1578498..11101e1 100644
--- a/Doc/includes/mp_pool.py
+++ b/Doc/includes/mp_pool.py
@@ -1,10 +1,3 @@
-#
-# A test of `multiprocessing.Pool` class
-#
-# Copyright (c) 2006-2008, R Oudkerk
-# All rights reserved.
-#
-
import multiprocessing
import time
import random
@@ -46,269 +39,115 @@ def noop(x):
#
def test():
- print('cpu_count() = %d\n' % multiprocessing.cpu_count())
-
- #
- # Create pool
- #
-
PROCESSES = 4
print('Creating pool with %d processes\n' % PROCESSES)
- pool = multiprocessing.Pool(PROCESSES)
- print('pool = %s' % pool)
- print()
-
- #
- # Tests
- #
-
- TASKS = [(mul, (i, 7)) for i in range(10)] + \
- [(plus, (i, 8)) for i in range(10)]
-
- results = [pool.apply_async(calculate, t) for t in TASKS]
- imap_it = pool.imap(calculatestar, TASKS)
- imap_unordered_it = pool.imap_unordered(calculatestar, TASKS)
-
- print('Ordered results using pool.apply_async():')
- for r in results:
- print('\t', r.get())
- print()
-
- print('Ordered results using pool.imap():')
- for x in imap_it:
- print('\t', x)
- print()
-
- print('Unordered results using pool.imap_unordered():')
- for x in imap_unordered_it:
- print('\t', x)
- print()
-
- print('Ordered results using pool.map() --- will block till complete:')
- for x in pool.map(calculatestar, TASKS):
- print('\t', x)
- print()
-
- #
- # Simple benchmarks
- #
-
- N = 100000
- print('def pow3(x): return x**3')
- t = time.time()
- A = list(map(pow3, range(N)))
- print('\tmap(pow3, range(%d)):\n\t\t%s seconds' % \
- (N, time.time() - t))
+ with multiprocessing.Pool(PROCESSES) as pool:
+ #
+ # Tests
+ #
- t = time.time()
- B = pool.map(pow3, range(N))
- print('\tpool.map(pow3, range(%d)):\n\t\t%s seconds' % \
- (N, time.time() - t))
+ TASKS = [(mul, (i, 7)) for i in range(10)] + \
+ [(plus, (i, 8)) for i in range(10)]
- t = time.time()
- C = list(pool.imap(pow3, range(N), chunksize=N//8))
- print('\tlist(pool.imap(pow3, range(%d), chunksize=%d)):\n\t\t%s' \
- ' seconds' % (N, N//8, time.time() - t))
+ results = [pool.apply_async(calculate, t) for t in TASKS]
+ imap_it = pool.imap(calculatestar, TASKS)
+ imap_unordered_it = pool.imap_unordered(calculatestar, TASKS)
- assert A == B == C, (len(A), len(B), len(C))
- print()
+ print('Ordered results using pool.apply_async():')
+ for r in results:
+ print('\t', r.get())
+ print()
- L = [None] * 1000000
- print('def noop(x): pass')
- print('L = [None] * 1000000')
+ print('Ordered results using pool.imap():')
+ for x in imap_it:
+ print('\t', x)
+ print()
- t = time.time()
- A = list(map(noop, L))
- print('\tmap(noop, L):\n\t\t%s seconds' % \
- (time.time() - t))
+ print('Unordered results using pool.imap_unordered():')
+ for x in imap_unordered_it:
+ print('\t', x)
+ print()
- t = time.time()
- B = pool.map(noop, L)
- print('\tpool.map(noop, L):\n\t\t%s seconds' % \
- (time.time() - t))
+ print('Ordered results using pool.map() --- will block till complete:')
+ for x in pool.map(calculatestar, TASKS):
+ print('\t', x)
+ print()
- t = time.time()
- C = list(pool.imap(noop, L, chunksize=len(L)//8))
- print('\tlist(pool.imap(noop, L, chunksize=%d)):\n\t\t%s seconds' % \
- (len(L)//8, time.time() - t))
+ #
+ # Test error handling
+ #
- assert A == B == C, (len(A), len(B), len(C))
- print()
+ print('Testing error handling:')
- del A, B, C, L
-
- #
- # Test error handling
- #
-
- print('Testing error handling:')
-
- try:
- print(pool.apply(f, (5,)))
- except ZeroDivisionError:
- print('\tGot ZeroDivisionError as expected from pool.apply()')
- else:
- raise AssertionError('expected ZeroDivisionError')
-
- try:
- print(pool.map(f, list(range(10))))
- except ZeroDivisionError:
- print('\tGot ZeroDivisionError as expected from pool.map()')
- else:
- raise AssertionError('expected ZeroDivisionError')
-
- try:
- print(list(pool.imap(f, list(range(10)))))
- except ZeroDivisionError:
- print('\tGot ZeroDivisionError as expected from list(pool.imap())')
- else:
- raise AssertionError('expected ZeroDivisionError')
-
- it = pool.imap(f, list(range(10)))
- for i in range(10):
try:
- x = next(it)
+ print(pool.apply(f, (5,)))
except ZeroDivisionError:
- if i == 5:
- pass
- except StopIteration:
- break
+ print('\tGot ZeroDivisionError as expected from pool.apply()')
else:
- if i == 5:
- raise AssertionError('expected ZeroDivisionError')
-
- assert i == 9
- print('\tGot ZeroDivisionError as expected from IMapIterator.next()')
- print()
+ raise AssertionError('expected ZeroDivisionError')
- #
- # Testing timeouts
- #
-
- print('Testing ApplyResult.get() with timeout:', end=' ')
- res = pool.apply_async(calculate, TASKS[0])
- while 1:
- sys.stdout.flush()
try:
- sys.stdout.write('\n\t%s' % res.get(0.02))
- break
- except multiprocessing.TimeoutError:
- sys.stdout.write('.')
- print()
- print()
+ print(pool.map(f, list(range(10))))
+ except ZeroDivisionError:
+ print('\tGot ZeroDivisionError as expected from pool.map()')
+ else:
+ raise AssertionError('expected ZeroDivisionError')
- print('Testing IMapIterator.next() with timeout:', end=' ')
- it = pool.imap(calculatestar, TASKS)
- while 1:
- sys.stdout.flush()
try:
- sys.stdout.write('\n\t%s' % it.next(0.02))
- except StopIteration:
- break
- except multiprocessing.TimeoutError:
- sys.stdout.write('.')
- print()
- print()
-
- #
- # Testing callback
- #
-
- print('Testing callback:')
-
- A = []
- B = [56, 0, 1, 8, 27, 64, 125, 216, 343, 512, 729]
-
- r = pool.apply_async(mul, (7, 8), callback=A.append)
- r.wait()
-
- r = pool.map_async(pow3, list(range(10)), callback=A.extend)
- r.wait()
-
- if A == B:
- print('\tcallbacks succeeded\n')
- else:
- print('\t*** callbacks failed\n\t\t%s != %s\n' % (A, B))
-
- #
- # Check there are no outstanding tasks
- #
-
- assert not pool._cache, 'cache = %r' % pool._cache
-
- #
- # Check close() methods
- #
-
- print('Testing close():')
-
- for worker in pool._pool:
- assert worker.is_alive()
-
- result = pool.apply_async(time.sleep, [0.5])
- pool.close()
- pool.join()
-
- assert result.get() is None
-
- for worker in pool._pool:
- assert not worker.is_alive()
-
- print('\tclose() succeeded\n')
-
- #
- # Check terminate() method
- #
-
- print('Testing terminate():')
-
- pool = multiprocessing.Pool(2)
- DELTA = 0.1
- ignore = pool.apply(pow3, [2])
- results = [pool.apply_async(time.sleep, [DELTA]) for i in range(100)]
- pool.terminate()
- pool.join()
-
- for worker in pool._pool:
- assert not worker.is_alive()
-
- print('\tterminate() succeeded\n')
-
- #
- # Check garbage collection
- #
-
- print('Testing garbage collection:')
-
- pool = multiprocessing.Pool(2)
- DELTA = 0.1
- processes = pool._pool
- ignore = pool.apply(pow3, [2])
- results = [pool.apply_async(time.sleep, [DELTA]) for i in range(100)]
-
- results = pool = None
-
- time.sleep(DELTA * 2)
-
- for worker in processes:
- assert not worker.is_alive()
-
- print('\tgarbage collection succeeded\n')
+ print(list(pool.imap(f, list(range(10)))))
+ except ZeroDivisionError:
+ print('\tGot ZeroDivisionError as expected from list(pool.imap())')
+ else:
+ raise AssertionError('expected ZeroDivisionError')
+
+ it = pool.imap(f, list(range(10)))
+ for i in range(10):
+ try:
+ x = next(it)
+ except ZeroDivisionError:
+ if i == 5:
+ pass
+ except StopIteration:
+ break
+ else:
+ if i == 5:
+ raise AssertionError('expected ZeroDivisionError')
+
+ assert i == 9
+ print('\tGot ZeroDivisionError as expected from IMapIterator.next()')
+ print()
+
+ #
+ # Testing timeouts
+ #
+
+ print('Testing ApplyResult.get() with timeout:', end=' ')
+ res = pool.apply_async(calculate, TASKS[0])
+ while 1:
+ sys.stdout.flush()
+ try:
+ sys.stdout.write('\n\t%s' % res.get(0.02))
+ break
+ except multiprocessing.TimeoutError:
+ sys.stdout.write('.')
+ print()
+ print()
+
+ print('Testing IMapIterator.next() with timeout:', end=' ')
+ it = pool.imap(calculatestar, TASKS)
+ while 1:
+ sys.stdout.flush()
+ try:
+ sys.stdout.write('\n\t%s' % it.next(0.02))
+ except StopIteration:
+ break
+ except multiprocessing.TimeoutError:
+ sys.stdout.write('.')
+ print()
+ print()
if __name__ == '__main__':
multiprocessing.freeze_support()
-
- assert len(sys.argv) in (1, 2)
-
- if len(sys.argv) == 1 or sys.argv[1] == 'processes':
- print(' Using processes '.center(79, '-'))
- elif sys.argv[1] == 'threads':
- print(' Using threads '.center(79, '-'))
- import multiprocessing.dummy as multiprocessing
- else:
- print('Usage:\n\t%s [processes | threads]' % sys.argv[0])
- raise SystemExit(2)
-
test()
diff --git a/Doc/includes/mp_synchronize.py b/Doc/includes/mp_synchronize.py
deleted file mode 100644
index 81dbc38..0000000
--- a/Doc/includes/mp_synchronize.py
+++ /dev/null
@@ -1,278 +0,0 @@
-#
-# A test file for the `multiprocessing` package
-#
-# Copyright (c) 2006-2008, R Oudkerk
-# All rights reserved.
-#
-
-import time
-import sys
-import random
-from queue import Empty
-
-import multiprocessing # may get overwritten
-
-
-#### TEST_VALUE
-
-def value_func(running, mutex):
- random.seed()
- time.sleep(random.random()*4)
-
- mutex.acquire()
- print('\n\t\t\t' + str(multiprocessing.current_process()) + ' has finished')
- running.value -= 1
- mutex.release()
-
-def test_value():
- TASKS = 10
- running = multiprocessing.Value('i', TASKS)
- mutex = multiprocessing.Lock()
-
- for i in range(TASKS):
- p = multiprocessing.Process(target=value_func, args=(running, mutex))
- p.start()
-
- while running.value > 0:
- time.sleep(0.08)
- mutex.acquire()
- print(running.value, end=' ')
- sys.stdout.flush()
- mutex.release()
-
- print()
- print('No more running processes')
-
-
-#### TEST_QUEUE
-
-def queue_func(queue):
- for i in range(30):
- time.sleep(0.5 * random.random())
- queue.put(i*i)
- queue.put('STOP')
-
-def test_queue():
- q = multiprocessing.Queue()
-
- p = multiprocessing.Process(target=queue_func, args=(q,))
- p.start()
-
- o = None
- while o != 'STOP':
- try:
- o = q.get(timeout=0.3)
- print(o, end=' ')
- sys.stdout.flush()
- except Empty:
- print('TIMEOUT')
-
- print()
-
-
-#### TEST_CONDITION
-
-def condition_func(cond):
- cond.acquire()
- print('\t' + str(cond))
- time.sleep(2)
- print('\tchild is notifying')
- print('\t' + str(cond))
- cond.notify()
- cond.release()
-
-def test_condition():
- cond = multiprocessing.Condition()
-
- p = multiprocessing.Process(target=condition_func, args=(cond,))
- print(cond)
-
- cond.acquire()
- print(cond)
- cond.acquire()
- print(cond)
-
- p.start()
-
- print('main is waiting')
- cond.wait()
- print('main has woken up')
-
- print(cond)
- cond.release()
- print(cond)
- cond.release()
-
- p.join()
- print(cond)
-
-
-#### TEST_SEMAPHORE
-
-def semaphore_func(sema, mutex, running):
- sema.acquire()
-
- mutex.acquire()
- running.value += 1
- print(running.value, 'tasks are running')
- mutex.release()
-
- random.seed()
- time.sleep(random.random()*2)
-
- mutex.acquire()
- running.value -= 1
- print('%s has finished' % multiprocessing.current_process())
- mutex.release()
-
- sema.release()
-
-def test_semaphore():
- sema = multiprocessing.Semaphore(3)
- mutex = multiprocessing.RLock()
- running = multiprocessing.Value('i', 0)
-
- processes = [
- multiprocessing.Process(target=semaphore_func,
- args=(sema, mutex, running))
- for i in range(10)
- ]
-
- for p in processes:
- p.start()
-
- for p in processes:
- p.join()
-
-
-#### TEST_JOIN_TIMEOUT
-
-def join_timeout_func():
- print('\tchild sleeping')
- time.sleep(5.5)
- print('\n\tchild terminating')
-
-def test_join_timeout():
- p = multiprocessing.Process(target=join_timeout_func)
- p.start()
-
- print('waiting for process to finish')
-
- while 1:
- p.join(timeout=1)
- if not p.is_alive():
- break
- print('.', end=' ')
- sys.stdout.flush()
-
-
-#### TEST_EVENT
-
-def event_func(event):
- print('\t%r is waiting' % multiprocessing.current_process())
- event.wait()
- print('\t%r has woken up' % multiprocessing.current_process())
-
-def test_event():
- event = multiprocessing.Event()
-
- processes = [multiprocessing.Process(target=event_func, args=(event,))
- for i in range(5)]
-
- for p in processes:
- p.start()
-
- print('main is sleeping')
- time.sleep(2)
-
- print('main is setting event')
- event.set()
-
- for p in processes:
- p.join()
-
-
-#### TEST_SHAREDVALUES
-
-def sharedvalues_func(values, arrays, shared_values, shared_arrays):
- for i in range(len(values)):
- v = values[i][1]
- sv = shared_values[i].value
- assert v == sv
-
- for i in range(len(values)):
- a = arrays[i][1]
- sa = list(shared_arrays[i][:])
- assert a == sa
-
- print('Tests passed')
-
-def test_sharedvalues():
- values = [
- ('i', 10),
- ('h', -2),
- ('d', 1.25)
- ]
- arrays = [
- ('i', list(range(100))),
- ('d', [0.25 * i for i in range(100)]),
- ('H', list(range(1000)))
- ]
-
- shared_values = [multiprocessing.Value(id, v) for id, v in values]
- shared_arrays = [multiprocessing.Array(id, a) for id, a in arrays]
-
- p = multiprocessing.Process(
- target=sharedvalues_func,
- args=(values, arrays, shared_values, shared_arrays)
- )
- p.start()
- p.join()
-
- assert p.exitcode == 0
-
-
-####
-
-def test(namespace=multiprocessing):
- global multiprocessing
-
- multiprocessing = namespace
-
- for func in [test_value, test_queue, test_condition,
- test_semaphore, test_join_timeout, test_event,
- test_sharedvalues]:
-
- print('\n\t######## %s\n' % func.__name__)
- func()
-
- ignore = multiprocessing.active_children() # cleanup any old processes
- if hasattr(multiprocessing, '_debug_info'):
- info = multiprocessing._debug_info()
- if info:
- print(info)
- raise ValueError('there should be no positive refcounts left')
-
-
-if __name__ == '__main__':
- multiprocessing.freeze_support()
-
- assert len(sys.argv) in (1, 2)
-
- if len(sys.argv) == 1 or sys.argv[1] == 'processes':
- print(' Using processes '.center(79, '-'))
- namespace = multiprocessing
- elif sys.argv[1] == 'manager':
- print(' Using processes and a manager '.center(79, '-'))
- namespace = multiprocessing.Manager()
- namespace.Process = multiprocessing.Process
- namespace.current_process = multiprocessing.current_process
- namespace.active_children = multiprocessing.active_children
- elif sys.argv[1] == 'threads':
- print(' Using threads '.center(79, '-'))
- import multiprocessing.dummy as namespace
- else:
- print('Usage:\n\t%s [processes | manager | threads]' % sys.argv[0])
- raise SystemExit(2)
-
- test(namespace)
diff --git a/Doc/includes/mp_webserver.py b/Doc/includes/mp_webserver.py
deleted file mode 100644
index 651024d..0000000
--- a/Doc/includes/mp_webserver.py
+++ /dev/null
@@ -1,70 +0,0 @@
-#
-# Example where a pool of http servers share a single listening socket
-#
-# On Windows this module depends on the ability to pickle a socket
-# object so that the worker processes can inherit a copy of the server
-# object. (We import `multiprocessing.reduction` to enable this pickling.)
-#
-# Not sure if we should synchronize access to `socket.accept()` method by
-# using a process-shared lock -- does not seem to be necessary.
-#
-# Copyright (c) 2006-2008, R Oudkerk
-# All rights reserved.
-#
-
-import os
-import sys
-
-from multiprocessing import Process, current_process, freeze_support
-from http.server import HTTPServer
-from http.server import SimpleHTTPRequestHandler
-
-if sys.platform == 'win32':
- import multiprocessing.reduction # make sockets pickable/inheritable
-
-
-def note(format, *args):
- sys.stderr.write('[%s]\t%s\n' % (current_process().name, format % args))
-
-
-class RequestHandler(SimpleHTTPRequestHandler):
- # we override log_message() to show which process is handling the request
- def log_message(self, format, *args):
- note(format, *args)
-
-def serve_forever(server):
- note('starting server')
- try:
- server.serve_forever()
- except KeyboardInterrupt:
- pass
-
-
-def runpool(address, number_of_processes):
- # create a single server object -- children will each inherit a copy
- server = HTTPServer(address, RequestHandler)
-
- # create child processes to act as workers
- for i in range(number_of_processes - 1):
- Process(target=serve_forever, args=(server,)).start()
-
- # main process also acts as a worker
- serve_forever(server)
-
-
-def test():
- DIR = os.path.join(os.path.dirname(__file__), '..')
- ADDRESS = ('localhost', 8000)
- NUMBER_OF_PROCESSES = 4
-
- print('Serving at http://%s:%d using %d worker processes' % \
- (ADDRESS[0], ADDRESS[1], NUMBER_OF_PROCESSES))
- print('To exit press Ctrl-' + ['C', 'Break'][sys.platform=='win32'])
-
- os.chdir(DIR)
- runpool(ADDRESS, NUMBER_OF_PROCESSES)
-
-
-if __name__ == '__main__':
- freeze_support()
- test()
diff --git a/Doc/includes/mp_workers.py b/Doc/includes/mp_workers.py
index e66d97b..3b92269 100644
--- a/Doc/includes/mp_workers.py
+++ b/Doc/includes/mp_workers.py
@@ -1,16 +1,3 @@
-#
-# Simple example which uses a pool of workers to carry out some tasks.
-#
-# Notice that the results will probably not come out of the output
-# queue in the same in the same order as the corresponding tasks were
-# put on the input queue. If it is important to get the results back
-# in the original order then consider using `Pool.map()` or
-# `Pool.imap()` (which will save on the amount of code needed anyway).
-#
-# Copyright (c) 2006-2008, R Oudkerk
-# All rights reserved.
-#
-
import time
import random
diff --git a/Doc/includes/typestruct.h b/Doc/includes/typestruct.h
index 32647c0..fcb846a 100644
--- a/Doc/includes/typestruct.h
+++ b/Doc/includes/typestruct.h
@@ -70,4 +70,11 @@ typedef struct _typeobject {
PyObject *tp_subclasses;
PyObject *tp_weaklist;
+ destructor tp_del;
+
+ /* Type attribute cache version tag. Added in version 2.6 */
+ unsigned int tp_version_tag;
+
+ destructor tp_finalize;
+
} PyTypeObject;
diff --git a/Doc/library/2to3.rst b/Doc/library/2to3.rst
index 1e5f42d..fda214c 100644
--- a/Doc/library/2to3.rst
+++ b/Doc/library/2to3.rst
@@ -342,6 +342,10 @@ and off individually. They are described here in more detail.
Handles the move of :func:`reduce` to :func:`functools.reduce`.
+.. 2to3fixer:: reload
+
+ Converts :func:`reload` to :func:`imp.reload`.
+
.. 2to3fixer:: renames
Changes :data:`sys.maxint` to :data:`sys.maxsize`.
diff --git a/Doc/library/abc.rst b/Doc/library/abc.rst
index 9299124..7853d31 100644
--- a/Doc/library/abc.rst
+++ b/Doc/library/abc.rst
@@ -12,9 +12,9 @@
--------------
This module provides the infrastructure for defining :term:`abstract base
-classes <abstract base class>` (ABCs) in Python, as outlined in :pep:`3119`; see the PEP for why this
-was added to Python. (See also :pep:`3141` and the :mod:`numbers` module
-regarding a type hierarchy for numbers based on ABCs.)
+classes <abstract base class>` (ABCs) in Python, as outlined in :pep:`3119`;
+see the PEP for why this was added to Python. (See also :pep:`3141` and the
+:mod:`numbers` module regarding a type hierarchy for numbers based on ABCs.)
The :mod:`collections` module has some concrete classes that derive from
ABCs; these can, of course, be further derived. In addition the
@@ -23,7 +23,7 @@ a class or instance provides a particular interface, for example, is it
hashable or a mapping.
-This module provides the following class:
+This module provides the following classes:
.. class:: ABCMeta
@@ -58,6 +58,10 @@ This module provides the following class:
.. versionchanged:: 3.3
Returns the registered subclass, to allow usage as a class decorator.
+ .. versionchanged:: 3.4
+ To detect calls to :meth:`register`, you can use the
+ :func:`get_cache_token` function.
+
You can also override this method in an abstract base class:
.. method:: __subclasshook__(subclass)
@@ -127,6 +131,19 @@ This module provides the following class:
available as a method of ``Foo``, so it is provided separately.
+.. class:: ABC
+
+ A helper class that has :class:`ABCMeta` as its metaclass. With this class,
+ an abstract base class can be created by simply deriving from :class:`ABC`,
+ avoiding sometimes confusing metaclass usage.
+
+ Note that the type of :class:`ABC` is still :class:`ABCMeta`, therefore
+ inheriting from :class:`ABC` requires the usual precautions regarding metaclass
+ usage, as multiple inheritance may lead to metaclass conflicts.
+
+ .. versionadded:: 3.4
+
+
The :mod:`abc` module also provides the following decorators:
.. decorator:: abstractmethod
@@ -295,6 +312,19 @@ The :mod:`abc` module also provides the following decorators:
:func:`abstractmethod`, making this decorator redundant.
+The :mod:`abc` module also provides the following functions:
+
+.. function:: get_cache_token()
+
+ Returns the current abstract base class cache token.
+
+ The token is an opaque integer identifying the current version of the
+ abstract base class cache for virtual subclasses. This number changes
+ with every call to :meth:`ABCMeta.register` on any ABC.
+
+ .. versionadded:: 3.4
+
+
.. rubric:: Footnotes
.. [#] C++ programmers should note that Python's virtual base class
diff --git a/Doc/library/aifc.rst b/Doc/library/aifc.rst
index 8a3541b..9ffb5a3 100644
--- a/Doc/library/aifc.rst
+++ b/Doc/library/aifc.rst
@@ -30,8 +30,8 @@ sampling rate or frame rate is the number of times per second the sound is
sampled. The number of channels indicate if the audio is mono, stereo, or
quadro. Each frame consists of one sample per channel. The sample size is the
size in bytes of each sample. Thus a frame consists of
-*nchannels*\*\ *samplesize* bytes, and a second's worth of audio consists of
-*nchannels*\*\ *samplesize*\*\ *framerate* bytes.
+``nchannels * samplesize`` bytes, and a second's worth of audio consists of
+``nchannels * samplesize * framerate`` bytes.
For example, CD quality audio has a sample size of two bytes (16 bits), uses two
channels (stereo) and has a frame rate of 44,100 frames/second. This gives a
@@ -51,6 +51,11 @@ Module :mod:`aifc` defines the following function:
used for writing, the file object should be seekable, unless you know ahead of
time how many samples you are going to write in total and use
:meth:`writeframesraw` and :meth:`setnframes`.
+ The :func:`.open` function may be used in a :keyword:`with` statement. When
+ the :keyword:`with` block completes, the :meth:`~aifc.close` method is called.
+
+.. versionchanged:: 3.4
+ Support for the :keyword:`with` statement was added.
Objects returned by :func:`.open` when a file is opened for reading have the
following methods:
@@ -92,7 +97,9 @@ following methods:
.. method:: aifc.getparams()
- Return a tuple consisting of all of the above values in the above order.
+ Returns a :func:`~collections.namedtuple` ``(nchannels, sampwidth,
+ framerate, nframes, comptype, compname)``, equivalent to output of the
+ :meth:`get\*` methods.
.. method:: aifc.getmarkers()
@@ -218,12 +225,18 @@ number of frames must be filled in.
Write data to the output file. This method can only be called after the audio
file parameters have been set.
+ .. versionchanged:: 3.4
+ Any :term:`bytes-like object`\ s are now accepted.
+
.. method:: aifc.writeframesraw(data)
Like :meth:`writeframes`, except that the header of the audio file is not
updated.
+ .. versionchanged:: 3.4
+ Any :term:`bytes-like object`\ s are now accepted.
+
.. method:: aifc.close()
diff --git a/Doc/library/argparse.rst b/Doc/library/argparse.rst
index 0b18c6d..5b7d016 100644
--- a/Doc/library/argparse.rst
+++ b/Doc/library/argparse.rst
@@ -977,9 +977,9 @@ See the section on the default_ keyword argument for information on when the
``type`` argument is applied to default arguments.
To ease the use of various types of files, the argparse module provides the
-factory FileType which takes the ``mode=`` and ``bufsize=`` arguments of the
-:func:`open` function. For example, ``FileType('w')`` can be used to create a
-writable file::
+factory FileType which takes the ``mode=``, ``bufsize=``, ``encoding=`` and
+``errors=`` arguments of the :func:`open` function. For example,
+``FileType('w')`` can be used to create a writable file::
>>> parser = argparse.ArgumentParser()
>>> parser.add_argument('bar', type=argparse.FileType('w'))
@@ -1646,17 +1646,19 @@ Sub-commands
FileType objects
^^^^^^^^^^^^^^^^
-.. class:: FileType(mode='r', bufsize=None)
+.. class:: FileType(mode='r', bufsize=-1, encoding=None, errors=None)
The :class:`FileType` factory creates objects that can be passed to the type
argument of :meth:`ArgumentParser.add_argument`. Arguments that have
- :class:`FileType` objects as their type will open command-line arguments as files
- with the requested modes and buffer sizes::
+ :class:`FileType` objects as their type will open command-line arguments as
+ files with the requested modes, buffer sizes, encodings and error handling
+ (see the :func:`open` function for more details)::
>>> parser = argparse.ArgumentParser()
- >>> parser.add_argument('--output', type=argparse.FileType('wb', 0))
- >>> parser.parse_args(['--output', 'out'])
- Namespace(output=<_io.BufferedWriter name='out'>)
+ >>> parser.add_argument('--raw', type=argparse.FileType('wb', 0))
+ >>> parser.add_argument('out', type=argparse.FileType('w', encoding='UTF-8'))
+ >>> parser.parse_args(['--raw', 'raw.dat', 'file.txt'])
+ Namespace(out=<_io.TextIOWrapper name='file.txt' mode='w' encoding='UTF-8'>, raw=<_io.FileIO name='raw.dat' mode='wb'>)
FileType objects understand the pseudo-argument ``'-'`` and automatically
convert this into ``sys.stdin`` for readable :class:`FileType` objects and
diff --git a/Doc/library/asynchat.rst b/Doc/library/asynchat.rst
index 55c61d7..7b81074 100644
--- a/Doc/library/asynchat.rst
+++ b/Doc/library/asynchat.rst
@@ -10,6 +10,11 @@
--------------
+.. note::
+
+ This module exists for backwards compatibility only. For new code we
+ recommend using :mod:`asyncio`.
+
This module builds on the :mod:`asyncore` infrastructure, simplifying
asynchronous clients and servers and making it easier to handle protocols
whose elements are terminated by arbitrary strings, or are of variable length.
diff --git a/Doc/library/asyncio.rst b/Doc/library/asyncio.rst
new file mode 100644
index 0000000..7b8151f
--- /dev/null
+++ b/Doc/library/asyncio.rst
@@ -0,0 +1,586 @@
+:mod:`asyncio` -- Asynchronous I/O, event loop, coroutines and tasks
+====================================================================
+
+.. module:: asyncio
+ :synopsis: Asynchronous I/O, event loop, coroutines and tasks.
+
+.. versionadded:: 3.4
+
+**Source code:** :source:`Lib/asyncio/`
+
+--------------
+
+This module provides infrastructure for writing single-threaded concurrent
+code using coroutines, multiplexing I/O access over sockets and other
+resources, running network clients and servers, and other related primitives.
+
+Here is a more detailed list of the package contents:
+
+* a pluggable :ref:`event loop <event-loop>` with various system-specific
+ implementations;
+
+* :ref:`transport <transport>` and :ref:`protocol <protocol>` abstractions
+ (similar to those in `Twisted <http://twistedmatrix.com/>`_);
+
+* concrete support for TCP, UDP, SSL, subprocess pipes, delayed calls, and
+ others (some may be system-dependent);
+
+* a Future class that mimicks the one in the :mod:`concurrent.futures` module,
+ but adapted for use with the event loop;
+
+* coroutines and tasks based on ``yield from`` (:PEP:`380`), to help write
+ concurrent code in a sequential fashion;
+
+* cancellation support for Futures and coroutines;
+
+* :ref:`synchronization primitives <sync>` for use between coroutines in
+ a single thread, mimicking those in the :mod:`threading` module;
+
+* an interface for passing work off to a threadpool, for times when
+ you absolutely, positively have to use a library that makes blocking
+ I/O calls.
+
+
+Disclaimer
+----------
+
+Full documentation is not yet ready; we hope to have it written
+before Python 3.4 leaves beta. Until then, the best reference is
+:PEP:`3156`. For a motivational primer on transports and protocols,
+see :PEP:`3153`.
+
+
+.. XXX should the asyncio documentation come in several pages, as for logging?
+
+
+.. _event-loop:
+
+Event loops
+-----------
+
+The event loop is the central execution device provided by :mod:`asyncio`.
+It provides multiple facilities, amongst which:
+
+* Registering, executing and cancelling delayed calls (timeouts)
+
+* Creating client and server :ref:`transports <transport>` for various
+ kinds of communication
+
+* Launching subprocesses and the associated :ref:`transports <transport>`
+ for communication with an external program
+
+* Delegating costly function calls to a pool of threads
+
+Getting an event loop
+^^^^^^^^^^^^^^^^^^^^^
+
+The easiest way to get an event loop is to call the :func:`get_event_loop`
+function.
+
+.. XXX more docs
+
+Delayed calls
+^^^^^^^^^^^^^
+
+The event loop has its own internal clock for computing timeouts.
+Which clock is used depends on the (platform-specific) event loop
+implementation; ideally it is a monotonic clock. This will generally be
+a different clock than :func:`time.time`.
+
+.. method:: time()
+
+ Return the current time, as a :class:`float` value, according to the
+ event loop's internal clock.
+
+.. method:: call_later(delay, callback, *args)
+
+ Arrange for the *callback* to be called after the given *delay*
+ seconds (either an int or float).
+
+ A "handle" is returned: an opaque object with a :meth:`cancel` method
+ that can be used to cancel the call.
+
+ *callback* will be called exactly once per call to :meth:`call_later`.
+ If two callbacks are scheduled for exactly the same time, it is
+ undefined which will be called first.
+
+ The optional positional *args* will be passed to the callback when it
+ is called. If you want the callback to be called with some named
+ arguments, use a closure or :func:`functools.partial`.
+
+.. method:: call_at(when, callback, *args)
+
+ Arrange for the *callback* to be called at the given absolute timestamp
+ *when* (an int or float), using the same time reference as :meth:`time`.
+
+ This method's behavior is the same as :meth:`call_later`.
+
+Creating connections
+^^^^^^^^^^^^^^^^^^^^
+
+.. method:: create_connection(protocol_factory, host=None, port=None, **options)
+
+ Create a streaming transport connection to a given Internet *host* and
+ *port*. *protocol_factory* must be a callable returning a
+ :ref:`protocol <protocol>` instance.
+
+ This method returns a :ref:`coroutine <coroutine>` which will try to
+ establish the connection in the background. When successful, the
+ coroutine returns a ``(transport, protocol)`` pair.
+
+ The chronological synopsis of the underlying operation is as follows:
+
+ #. The connection is established, and a :ref:`transport <transport>`
+ is created to represent it.
+
+ #. *protocol_factory* is called without arguments and must return a
+ :ref:`protocol <protocol>` instance.
+
+ #. The protocol instance is tied to the transport, and its
+ :meth:`connection_made` method is called.
+
+ #. The coroutine returns successfully with the ``(transport, protocol)``
+ pair.
+
+ The created transport is an implementation-dependent bidirectional stream.
+
+ .. note::
+ *protocol_factory* can be any kind of callable, not necessarily
+ a class. For example, if you want to use a pre-created
+ protocol instance, you can pass ``lambda: my_protocol``.
+
+ *options* are optional named arguments allowing to change how the
+ connection is created:
+
+ * *ssl*: if given and not false, a SSL/TLS transport is created
+ (by default a plain TCP transport is created). If *ssl* is
+ a :class:`ssl.SSLContext` object, this context is used to create
+ the transport; if *ssl* is :const:`True`, a context with some
+ unspecified default settings is used.
+
+ * *server_hostname*, is only for use together with *ssl*,
+ and sets or overrides the hostname that the target server's certificate
+ will be matched against. By default the value of the *host* argument
+ is used. If *host* is empty, there is no default and you must pass a
+ value for *server_hostname*. If *server_hostname* is an empty
+ string, hostname matching is disabled (which is a serious security
+ risk, allowing for man-in-the-middle-attacks).
+
+ * *family*, *proto*, *flags* are the optional address family, protocol
+ and flags to be passed through to getaddrinfo() for *host* resolution.
+ If given, these should all be integers from the corresponding
+ :mod:`socket` module constants.
+
+ * *sock*, if given, should be an existing, already connected
+ :class:`socket.socket` object to be used by the transport.
+ If *sock* is given, none of *host*, *port*, *family*, *proto*, *flags*
+ and *local_addr* should be specified.
+
+ * *local_addr*, if given, is a ``(local_host, local_port)`` tuple used
+ to bind the socket to locally. The *local_host* and *local_port*
+ are looked up using getaddrinfo(), similarly to *host* and *port*.
+
+
+.. _protocol:
+
+Protocols
+---------
+
+:mod:`asyncio` provides base classes that you can subclass to implement
+your network protocols. Those classes are used in conjunction with
+:ref:`transports <transport>` (see below): the protocol parses incoming
+data and asks for the writing of outgoing data, while the transport is
+responsible for the actual I/O and buffering.
+
+When subclassing a protocol class, it is recommended you override certain
+methods. Those methods are callbacks: they will be called by the transport
+on certain events (for example when some data is received); you shouldn't
+call them yourself, unless you are implementing a transport.
+
+.. note::
+ All callbacks have default implementations, which are empty. Therefore,
+ you only need to implement the callbacks for the events in which you
+ are interested.
+
+
+Protocol classes
+^^^^^^^^^^^^^^^^
+
+.. class:: Protocol
+
+ The base class for implementing streaming protocols (for use with
+ e.g. TCP and SSL transports).
+
+.. class:: DatagramProtocol
+
+ The base class for implementing datagram protocols (for use with
+ e.g. UDP transports).
+
+.. class:: SubprocessProtocol
+
+ The base class for implementing protocols communicating with child
+ processes (through a set of unidirectional pipes).
+
+
+Connection callbacks
+^^^^^^^^^^^^^^^^^^^^
+
+These callbacks may be called on :class:`Protocol` and
+:class:`SubprocessProtocol` instances:
+
+.. method:: connection_made(transport)
+
+ Called when a connection is made.
+
+ The *transport* argument is the transport representing the
+ connection. You are responsible for storing it somewhere
+ (e.g. as an attribute) if you need to.
+
+.. method:: connection_lost(exc)
+
+ Called when the connection is lost or closed.
+
+ The argument is either an exception object or :const:`None`.
+ The latter means a regular EOF is received, or the connection was
+ aborted or closed by this side of the connection.
+
+:meth:`connection_made` and :meth:`connection_lost` are called exactly once
+per successful connection. All other callbacks will be called between those
+two methods, which allows for easier resource management in your protocol
+implementation.
+
+The following callbacks may be called only on :class:`SubprocessProtocol`
+instances:
+
+.. method:: pipe_data_received(fd, data)
+
+ Called when the child process writes data into its stdout or stderr pipe.
+ *fd* is the integer file descriptor of the pipe. *data* is a non-empty
+ bytes object containing the data.
+
+.. method:: pipe_connection_lost(fd, exc)
+
+ Called when one of the pipes communicating with the child process
+ is closed. *fd* is the integer file descriptor that was closed.
+
+.. method:: process_exited()
+
+ Called when the child process has exited.
+
+
+Data reception callbacks
+^^^^^^^^^^^^^^^^^^^^^^^^
+
+Streaming protocols
+"""""""""""""""""""
+
+The following callbacks are called on :class:`Protocol` instances:
+
+.. method:: data_received(data)
+
+ Called when some data is received. *data* is a non-empty bytes object
+ containing the incoming data.
+
+ .. note::
+ Whether the data is buffered, chunked or reassembled depends on
+ the transport. In general, you shouldn't rely on specific semantics
+ and instead make your parsing generic and flexible enough. However,
+ data is always received in the correct order.
+
+.. method:: eof_received()
+
+ Calls when the other end signals it won't send any more data
+ (for example by calling :meth:`write_eof`, if the other end also uses
+ asyncio).
+
+ This method may return a false value (including None), in which case
+ the transport will close itself. Conversely, if this method returns a
+ true value, closing the transport is up to the protocol. Since the
+ default implementation returns None, it implicitly closes the connection.
+
+ .. note::
+ Some transports such as SSL don't support half-closed connections,
+ in which case returning true from this method will not prevent closing
+ the connection.
+
+:meth:`data_received` can be called an arbitrary number of times during
+a connection. However, :meth:`eof_received` is called at most once
+and, if called, :meth:`data_received` won't be called after it.
+
+Datagram protocols
+""""""""""""""""""
+
+The following callbacks are called on :class:`DatagramProtocol` instances.
+
+.. method:: datagram_received(data, addr)
+
+ Called when a datagram is received. *data* is a bytes object containing
+ the incoming data. *addr* is the address of the peer sending the data;
+ the exact format depends on the transport.
+
+.. method:: error_received(exc)
+
+ Called when a previous send or receive operation raises an
+ :class:`OSError`. *exc* is the :class:`OSError` instance.
+
+ This method is called in rare conditions, when the transport (e.g. UDP)
+ detects that a datagram couldn't be delivered to its recipient.
+ In many conditions though, undeliverable datagrams will be silently
+ dropped.
+
+
+Flow control callbacks
+^^^^^^^^^^^^^^^^^^^^^^
+
+These callbacks may be called on :class:`Protocol` and
+:class:`SubprocessProtocol` instances:
+
+.. method:: pause_writing()
+
+ Called when the transport's buffer goes over the high-water mark.
+
+.. method:: resume_writing()
+
+ Called when the transport's buffer drains below the low-water mark.
+
+
+:meth:`pause_writing` and :meth:`resume_writing` calls are paired --
+:meth:`pause_writing` is called once when the buffer goes strictly over
+the high-water mark (even if subsequent writes increases the buffer size
+even more), and eventually :meth:`resume_writing` is called once when the
+buffer size reaches the low-water mark.
+
+.. note::
+ If the buffer size equals the high-water mark,
+ :meth:`pause_writing` is not called -- it must go strictly over.
+ Conversely, :meth:`resume_writing` is called when the buffer size is
+ equal or lower than the low-water mark. These end conditions
+ are important to ensure that things go as expected when either
+ mark is zero.
+
+
+.. _transport:
+
+Transports
+----------
+
+Transports are classed provided by :mod:`asyncio` in order to abstract
+various kinds of communication channels. You generally won't instantiate
+a transport yourself; instead, you will call a :class:`EventLoop` method
+which will create the transport and try to initiate the underlying
+communication channel, calling you back when it succeeds.
+
+Once the communication channel is established, a transport is always
+paired with a :ref:`protocol <protocol>` instance. The protocol can
+then call the transport's methods for various purposes.
+
+:mod:`asyncio` currently implements transports for TCP, UDP, SSL, and
+subprocess pipes. The methods available on a transport depend on
+the transport's kind.
+
+Methods common to all transports
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. method:: close(self)
+
+ Close the transport. If the transport has a buffer for outgoing
+ data, buffered data will be flushed asynchronously. No more data
+ will be received. After all buffered data is flushed, the
+ protocol's :meth:`connection_lost` method will be called with
+ :const:`None` as its argument.
+
+
+.. method:: get_extra_info(name, default=None)
+
+ Return optional transport information. *name* is a string representing
+ the piece of transport-specific information to get, *default* is the
+ value to return if the information doesn't exist.
+
+ This method allows transport implementations to easily expose
+ channel-specific information.
+
+Methods of readable streaming transports
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. method:: pause_reading()
+
+ Pause the receiving end of the transport. No data will be passed to
+ the protocol's :meth:`data_received` method until meth:`resume_reading`
+ is called.
+
+.. method:: resume_reading()
+
+ Resume the receiving end. The protocol's :meth:`data_received` method
+ will be called once again if some data is available for reading.
+
+Methods of writable streaming transports
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. method:: write(data)
+
+ Write some *data* bytes to the transport.
+
+ This method does not block; it buffers the data and arranges for it
+ to be sent out asynchronously.
+
+.. method:: writelines(list_of_data)
+
+ Write a list (or any iterable) of data bytes to the transport.
+ This is functionally equivalent to calling :meth:`write` on each
+ element yielded by the iterable, but may be implemented more efficiently.
+
+.. method:: write_eof()
+
+ Close the write end of the transport after flushing buffered data.
+ Data may still be received.
+
+ This method can raise :exc:`NotImplementedError` if the transport
+ (e.g. SSL) doesn't support half-closes.
+
+.. method:: can_write_eof()
+
+ Return :const:`True` if the transport supports :meth:`write_eof`,
+ :const:`False` if not.
+
+.. method:: abort()
+
+ Close the transport immediately, without waiting for pending operations
+ to complete. Buffered data will be lost. No more data will be received.
+ The protocol's :meth:`connection_lost` method will eventually be
+ called with :const:`None` as its argument.
+
+.. method:: set_write_buffer_limits(high=None, low=None)
+
+ Set the *high*- and *low*-water limits for write flow control.
+
+ These two values control when call the protocol's
+ :meth:`pause_writing` and :meth:`resume_writing` methods are called.
+ If specified, the low-water limit must be less than or equal to the
+ high-water limit. Neither *high* nor *low* can be negative.
+
+ The defaults are implementation-specific. If only the
+ high-water limit is given, the low-water limit defaults to a
+ implementation-specific value less than or equal to the
+ high-water limit. Setting *high* to zero forces *low* to zero as
+ well, and causes :meth:`pause_writing` to be called whenever the
+ buffer becomes non-empty. Setting *low* to zero causes
+ :meth:`resume_writing` to be called only once the buffer is empty.
+ Use of zero for either limit is generally sub-optimal as it
+ reduces opportunities for doing I/O and computation
+ concurrently.
+
+.. method:: get_write_buffer_size()
+
+ Return the current size of the output buffer used by the transport.
+
+Methods of datagram transports
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. method:: sendto(data, addr=None)
+
+ Send the *data* bytes to the remote peer given by *addr* (a
+ transport-dependent target address). If *addr* is :const:`None`, the
+ data is sent to the target address given on transport creation.
+
+ This method does not block; it buffers the data and arranges for it
+ to be sent out asynchronously.
+
+.. method:: abort()
+
+ Close the transport immediately, without waiting for pending operations
+ to complete. Buffered data will be lost. No more data will be received.
+ The protocol's :meth:`connection_lost` method will eventually be
+ called with :const:`None` as its argument.
+
+Methods of subprocess transports
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. method:: get_pid()
+
+ Return the subprocess process id as an integer.
+
+.. method:: get_returncode()
+
+ Return the subprocess returncode as an integer or :const:`None`
+ if it hasn't returned, similarly to the
+ :attr:`subprocess.Popen.returncode` attribute.
+
+.. method:: get_pipe_transport(fd)
+
+ Return the transport for the communication pipe correspondong to the
+ integer file descriptor *fd*. The return value can be a readable or
+ writable streaming transport, depending on the *fd*. If *fd* doesn't
+ correspond to a pipe belonging to this transport, :const:`None` is
+ returned.
+
+.. method:: send_signal(signal)
+
+ Send the *signal* number to the subprocess, as in
+ :meth:`subprocess.Popen.send_signal`.
+
+.. method:: terminate()
+
+ Ask the subprocess to stop, as in :meth:`subprocess.Popen.terminate`.
+ This method is an alias for the :meth:`close` method.
+
+ On POSIX systems, this method sends SIGTERM to the subprocess.
+ On Windows, the Windows API function TerminateProcess() is called to
+ stop the subprocess.
+
+.. method:: kill(self)
+
+ Kill the subprocess, as in :meth:`subprocess.Popen.kill`
+
+ On POSIX systems, the function sends SIGKILL to the subprocess.
+ On Windows, this method is an alias for :meth:`terminate`.
+
+
+.. _coroutine:
+
+Coroutines
+----------
+
+
+.. _sync:
+
+Synchronization primitives
+--------------------------
+
+
+Examples
+--------
+
+A :class:`Protocol` implementing an echo server::
+
+ class EchoServer(asyncio.Protocol):
+
+ TIMEOUT = 5.0
+
+ def timeout(self):
+ print('connection timeout, closing.')
+ self.transport.close()
+
+ def connection_made(self, transport):
+ print('connection made')
+ self.transport = transport
+
+ # start 5 seconds timeout timer
+ self.h_timeout = asyncio.get_event_loop().call_later(
+ self.TIMEOUT, self.timeout)
+
+ def data_received(self, data):
+ print('data received: ', data.decode())
+ self.transport.write(b'Re: ' + data)
+
+ # restart timeout timer
+ self.h_timeout.cancel()
+ self.h_timeout = asyncio.get_event_loop().call_later(
+ self.TIMEOUT, self.timeout)
+
+ def eof_received(self):
+ pass
+
+ def connection_lost(self, exc):
+ print('connection lost:', exc)
+ self.h_timeout.cancel()
+
diff --git a/Doc/library/asyncore.rst b/Doc/library/asyncore.rst
index 8f494d0..61079c5 100644
--- a/Doc/library/asyncore.rst
+++ b/Doc/library/asyncore.rst
@@ -13,6 +13,11 @@
--------------
+.. note::
+
+ This module exists for backwards compatibility only. For new code we
+ recommend using :mod:`asyncio`.
+
This module provides the basic infrastructure for writing asynchronous socket
service clients and servers.
diff --git a/Doc/library/audioop.rst b/Doc/library/audioop.rst
index edb3870..ca6cfb3 100644
--- a/Doc/library/audioop.rst
+++ b/Doc/library/audioop.rst
@@ -6,9 +6,16 @@
The :mod:`audioop` module contains some useful operations on sound fragments.
-It operates on sound fragments consisting of signed integer samples 8, 16 or 32
-bits wide, stored in bytes objects. All scalar items are integers, unless
-specified otherwise.
+It operates on sound fragments consisting of signed integer samples 8, 16, 24
+or 32 bits wide, stored in :term:`bytes-like object`\ s. All scalar items are
+integers, unless specified otherwise.
+
+.. versionchanged:: 3.4
+ Support for 24-bit samples was added.
+
+.. versionchanged:: 3.4
+ Any :term:`bytes-like object`\ s are now accepted by all functions in this
+ module. Strings no more supported.
.. index::
single: Intel/DVI ADPCM
@@ -35,7 +42,7 @@ The module defines the following variables and functions:
.. function:: add(fragment1, fragment2, width)
Return a fragment which is the addition of the two samples passed as parameters.
- *width* is the sample width in bytes, either ``1``, ``2`` or ``4``. Both
+ *width* is the sample width in bytes, either ``1``, ``2``, ``3`` or ``4``. Both
fragments should have the same length. Samples are truncated in case of overflow.
@@ -133,19 +140,19 @@ The module defines the following variables and functions:
.. function:: lin2lin(fragment, width, newwidth)
- Convert samples between 1-, 2- and 4-byte formats.
+ Convert samples between 1-, 2-, 3- and 4-byte formats.
.. note::
- In some audio formats, such as .WAV files, 16 and 32 bit samples are
+ In some audio formats, such as .WAV files, 16, 24 and 32 bit samples are
signed, but 8 bit samples are unsigned. So when converting to 8 bit wide
samples for these formats, you need to also add 128 to the result::
new_frames = audioop.lin2lin(frames, old_width, 1)
new_frames = audioop.bias(new_frames, 1, 128)
- The same, in reverse, has to be applied when converting from 8 to 16 or 32
- bit width samples.
+ The same, in reverse, has to be applied when converting from 8 to 16, 24
+ or 32 bit width samples.
.. function:: lin2ulaw(fragment, width)
diff --git a/Doc/library/base64.rst b/Doc/library/base64.rst
index 3b23e79..a7eed08 100644
--- a/Doc/library/base64.rst
+++ b/Doc/library/base64.rst
@@ -27,6 +27,10 @@ byte strings, but only using the Base64 standard alphabet.
ASCII-only Unicode strings are now accepted by the decoding functions of
the modern interface.
+.. versionchanged:: 3.4
+ Any :term:`bytes-like object`\ s are now accepted by all
+ encoding and decoding functions in this module.
+
The modern interface provides:
.. function:: b64encode(s, altchars=None)
@@ -128,6 +132,76 @@ The modern interface provides:
string.
+.. function:: a85encode(s, *, foldspaces=False, wrapcol=0, pad=False, adobe=False)
+
+ Encode a byte string using Ascii85.
+
+ *s* is the string to encode. The encoded byte string is returned.
+
+ *foldspaces* is an optional flag that uses the special short sequence 'y'
+ instead of 4 consecutive spaces (ASCII 0x20) as supported by 'btoa'. This
+ feature is not supported by the "standard" Ascii85 encoding.
+
+ *wrapcol* controls whether the output should have newline ('\n')
+ characters added to it. If this is non-zero, each output line will be
+ at most this many characters long.
+
+ *pad* controls whether the input string is padded to a multiple of 4
+ before encoding. Note that the ``btoa`` implementation always pads.
+
+ *adobe* controls whether the encoded byte sequence is framed with ``<~``
+ and ``~>``, which is used by the Adobe implementation.
+
+ .. versionadded:: 3.4
+
+
+.. function:: a85decode(s, *, foldspaces=False, adobe=False, ignorechars=b' \t\n\r\v')
+
+ Decode an Ascii85 encoded byte string.
+
+ *s* is the byte string to decode.
+
+ *foldspaces* is a flag that specifies whether the 'y' short sequence
+ should be accepted as shorthand for 4 consecutive spaces (ASCII 0x20).
+ This feature is not supported by the "standard" Ascii85 encoding.
+
+ *adobe* controls whether the input sequence is in Adobe Ascii85 format
+ (i.e. is framed with <~ and ~>).
+
+ *ignorechars* should be a byte string containing characters to ignore
+ from the input. This should only contain whitespace characters, and by
+ default contains all whitespace characters in ASCII.
+
+ .. versionadded:: 3.4
+
+
+.. function:: b85encode(s, pad=False)
+
+ Encode a byte string using base85, as used in e.g. git-style binary
+ diffs.
+
+ If *pad* is true, the input is padded with "\\0" so its length is a
+ multiple of 4 characters before encoding.
+
+ .. versionadded:: 3.4
+
+
+.. function:: b85decode(b)
+
+ Decode base85-encoded byte string. Padding is implicitly removed, if
+ necessary.
+
+ .. versionadded:: 3.4
+
+
+.. note::
+ Both Base85 and Ascii85 have an expansion factor of 5 to 4 (5 Base85 or
+ Ascii85 characters can encode 4 binary bytes), while the better-known
+ Base64 has an expansion factor of 6 to 4. They are therefore more
+ efficient when space expensive. They differ by details such as the
+ character map used for encoding.
+
+
The legacy interface:
.. function:: decode(input, output)
diff --git a/Doc/library/bz2.rst b/Doc/library/bz2.rst
index d06a39a..44e1331 100644
--- a/Doc/library/bz2.rst
+++ b/Doc/library/bz2.rst
@@ -37,8 +37,8 @@ All of the classes in this module may safely be accessed from multiple threads.
file object to read from or write to.
The *mode* argument can be any of ``'r'``, ``'rb'``, ``'w'``, ``'wb'``,
- ``'a'``, or ``'ab'`` for binary mode, or ``'rt'``, ``'wt'``, or ``'at'`` for
- text mode. The default is ``'rb'``.
+ ``'x'``, ``'xb'``, ``'a'`` or ``'ab'`` for binary mode, or ``'rt'``,
+ ``'wt'``, ``'xt'``, or ``'at'`` for text mode. The default is ``'rb'``.
The *compresslevel* argument is an integer from 1 to 9, as for the
:class:`BZ2File` constructor.
@@ -54,6 +54,9 @@ All of the classes in this module may safely be accessed from multiple threads.
.. versionadded:: 3.3
+ .. versionchanged:: 3.4
+ The ``'x'`` (exclusive creation) mode was added.
+
.. class:: BZ2File(filename, mode='r', buffering=None, compresslevel=9)
@@ -64,8 +67,9 @@ All of the classes in this module may safely be accessed from multiple threads.
be used to read or write the compressed data.
The *mode* argument can be either ``'r'`` for reading (default), ``'w'`` for
- overwriting, or ``'a'`` for appending. These can equivalently be given as
- ``'rb'``, ``'wb'``, and ``'ab'`` respectively.
+ overwriting, ``'x'`` for exclusive creation, or ``'a'`` for appending. These
+ can equivalently be given as ``'rb'``, ``'wb'``, ``'xb'`` and ``'ab'``
+ respectively.
If *filename* is a file object (rather than an actual file name), a mode of
``'w'`` does not truncate the file, and is instead equivalent to ``'a'``.
@@ -108,6 +112,9 @@ All of the classes in this module may safely be accessed from multiple threads.
The ``'a'`` (append) mode was added, along with support for reading
multi-stream files.
+ .. versionchanged:: 3.4
+ The ``'x'`` (exclusive creation) mode was added.
+
Incremental (de)compression
---------------------------
diff --git a/Doc/library/code.rst b/Doc/library/code.rst
index e869004..5b5d7cc 100644
--- a/Doc/library/code.rst
+++ b/Doc/library/code.rst
@@ -132,12 +132,15 @@ interpreter objects as well as the following additions.
.. method:: InteractiveConsole.interact(banner=None)
- Closely emulate the interactive Python console. The optional banner argument
+ Closely emulate the interactive Python console. The optional *banner* argument
specify the banner to print before the first interaction; by default it prints a
banner similar to the one printed by the standard Python interpreter, followed
by the class name of the console object in parentheses (so as not to confuse
this with the real interpreter -- since it's so close!).
+ .. versionchanged:: 3.4
+ To suppress printing any banner, pass an empty string.
+
.. method:: InteractiveConsole.push(line)
diff --git a/Doc/library/codecs.rst b/Doc/library/codecs.rst
index 009ae26..843de27 100644
--- a/Doc/library/codecs.rst
+++ b/Doc/library/codecs.rst
@@ -365,18 +365,23 @@ and implemented by all standard Python codecs:
| | in :pep:`383`. |
+-------------------------+-----------------------------------------------+
-In addition, the following error handlers are specific to a single codec:
+In addition, the following error handlers are specific to Unicode encoding
+schemes:
-+-------------------+---------+-------------------------------------------+
-| Value | Codec | Meaning |
-+===================+=========+===========================================+
-|``'surrogatepass'``| utf-8 | Allow encoding and decoding of surrogate |
-| | | codes in UTF-8. |
-+-------------------+---------+-------------------------------------------+
++-------------------+------------------------+-------------------------------------------+
+| Value | Codec | Meaning |
++===================+========================+===========================================+
+|``'surrogatepass'``| utf-8, utf-16, utf-32, | Allow encoding and decoding of surrogate |
+| | utf-16-be, utf-16-le, | codes in all the Unicode encoding schemes.|
+| | utf-32-be, utf-32-le | |
++-------------------+------------------------+-------------------------------------------+
.. versionadded:: 3.1
The ``'surrogateescape'`` and ``'surrogatepass'`` error handlers.
+.. versionchanged:: 3.4
+ The ``'surrogatepass'`` error handlers now works with utf-16\* and utf-32\* codecs.
+
The set of allowed values can be extended via :meth:`register_error`.
@@ -1022,6 +1027,10 @@ particular, the following variants typically exist:
+-----------------+--------------------------------+--------------------------------+
| cp1026 | ibm1026 | Turkish |
+-----------------+--------------------------------+--------------------------------+
+| cp1125 | 1125, ibm1125, cp866u, ruscii | Ukrainian |
+| | | |
+| | | .. versionadded:: 3.4 |
++-----------------+--------------------------------+--------------------------------+
| cp1140 | ibm1140 | Western Europe |
+-----------------+--------------------------------+--------------------------------+
| cp1250 | windows-1250 | Central and Eastern Europe |
@@ -1167,6 +1176,12 @@ particular, the following variants typically exist:
| utf_8_sig | | all languages |
+-----------------+--------------------------------+--------------------------------+
+.. versionchanged:: 3.4
+ The utf-16\* and utf-32\* encoders no longer allow surrogate code points
+ (U+D800--U+DFFF) to be encoded. The utf-32\* decoders no longer decode
+ byte sequences that correspond to surrogate code points.
+
+
Python Specific Encodings
-------------------------
@@ -1177,6 +1192,9 @@ common use case for codecs, the underlying codec infrastructure supports
arbitrary data transforms rather than just text encodings). For asymmetric
codecs, the stated purpose describes the encoding direction.
+Text Encodings
+^^^^^^^^^^^^^^
+
The following codecs provide :class:`str` to :class:`bytes` encoding and
:term:`bytes-like object` to :class:`str` decoding, similar to the Unicode text
encodings.
@@ -1223,57 +1241,83 @@ encodings.
| | | .. deprecated:: 3.3 |
+--------------------+---------+---------------------------+
-The following codecs provide :term:`bytes-like object` to :class:`bytes`
-mappings.
-
-
-.. tabularcolumns:: |l|L|L|
-
-+----------------------+---------------------------+------------------------------+
-| Codec | Purpose | Encoder/decoder |
-+======================+===========================+==============================+
-| base64_codec [#b64]_ | Convert operand to MIME | :meth:`base64.b64encode`, |
-| | base64 (the result always | :meth:`base64.b64decode` |
-| | includes a trailing | |
-| | ``'\n'``) | |
-+----------------------+---------------------------+------------------------------+
-| bz2_codec | Compress the operand | :meth:`bz2.compress`, |
-| | using bz2 | :meth:`bz2.decompress` |
-+----------------------+---------------------------+------------------------------+
-| hex_codec | Convert operand to | :meth:`base64.b16encode`, |
-| | hexadecimal | :meth:`base64.b16decode` |
-| | representation, with two | |
-| | digits per byte | |
-+----------------------+---------------------------+------------------------------+
-| quopri_codec | Convert operand to MIME | :meth:`quopri.encodestring`, |
-| | quoted printable | :meth:`quopri.decodestring` |
-+----------------------+---------------------------+------------------------------+
-| uu_codec | Convert the operand using | :meth:`uu.encode`, |
-| | uuencode | :meth:`uu.decode` |
-+----------------------+---------------------------+------------------------------+
-| zlib_codec | Compress the operand | :meth:`zlib.compress`, |
-| | using gzip | :meth:`zlib.decompress` |
-+----------------------+---------------------------+------------------------------+
-
-.. [#b64] Rather than accepting any :term:`bytes-like object`,
- ``'base64_codec'`` accepts only :class:`bytes` and :class:`bytearray` for
- encoding and only :class:`bytes`, :class:`bytearray`, and ASCII-only
- instances of :class:`str` for decoding
-
-
-The following codecs provide :class:`str` to :class:`str` mappings.
+.. _binary-transforms:
+
+Binary Transforms
+^^^^^^^^^^^^^^^^^
+
+The following codecs provide binary transforms: :term:`bytes-like object`
+to :class:`bytes` mappings.
+
+
+.. tabularcolumns:: |l|L|L|L|
+
++----------------------+------------------+------------------------------+------------------------------+
+| Codec | Aliases | Purpose | Encoder / decoder |
++======================+==================+==============================+==============================+
+| base64_codec [#b64]_ | base64, base_64 | Convert operand to MIME | :meth:`base64.b64encode` / |
+| | | base64 (the result always | :meth:`base64.b64decode` |
+| | | includes a trailing | |
+| | | ``'\n'``) | |
+| | | | |
+| | | .. versionchanged:: 3.4 | |
+| | | accepts any | |
+| | | :term:`bytes-like object` | |
+| | | as input for encoding and | |
+| | | decoding | |
++----------------------+------------------+------------------------------+------------------------------+
+| bz2_codec | bz2 | Compress the operand | :meth:`bz2.compress` / |
+| | | using bz2 | :meth:`bz2.decompress` |
++----------------------+------------------+------------------------------+------------------------------+
+| hex_codec | hex | Convert operand to | :meth:`base64.b16encode` / |
+| | | hexadecimal | :meth:`base64.b16decode` |
+| | | representation, with two | |
+| | | digits per byte | |
++----------------------+------------------+------------------------------+------------------------------+
+| quopri_codec | quopri, | Convert operand to MIME | :meth:`quopri.encodestring` /|
+| | quotedprintable, | quoted printable | :meth:`quopri.decodestring` |
+| | quoted_printable | | |
++----------------------+------------------+------------------------------+------------------------------+
+| uu_codec | uu | Convert the operand using | :meth:`uu.encode` / |
+| | | uuencode | :meth:`uu.decode` |
++----------------------+------------------+------------------------------+------------------------------+
+| zlib_codec | zip, zlib | Compress the operand | :meth:`zlib.compress` / |
+| | | using gzip | :meth:`zlib.decompress` |
++----------------------+------------------+------------------------------+------------------------------+
+
+.. [#b64] In addition to :term:`bytes-like objects <bytes-like object>`,
+ ``'base64_codec'`` also accepts ASCII-only instances of :class:`str` for
+ decoding
+
+.. versionadded:: 3.2
+ Restoration of the binary transforms.
+
+.. versionchanged:: 3.4
+ Restoration of the aliases for the binary transforms.
-.. tabularcolumns:: |l|L|
-+--------------------+---------------------------+
-| Codec | Purpose |
-+====================+===========================+
-| rot_13 | Returns the Caesar-cypher |
-| | encryption of the operand |
-+--------------------+---------------------------+
+.. _text-transforms:
+
+Text Transforms
+^^^^^^^^^^^^^^^
+
+The following codec provides a text transform: a :class:`str` to :class:`str`
+mapping.
+
+.. tabularcolumns:: |l|l|L|
+
++--------------------+---------+---------------------------+
+| Codec | Aliases | Purpose |
++====================+=========+===========================+
+| rot_13 | rot13 | Returns the Caesar-cypher |
+| | | encryption of the operand |
++--------------------+---------+---------------------------+
.. versionadded:: 3.2
- bytes-to-bytes and str-to-str codecs.
+ Restoration of the ``rot_13`` text transform.
+
+.. versionchanged:: 3.4
+ Restoration of the ``rot13`` alias.
:mod:`encodings.idna` --- Internationalized Domain Names in Applications
diff --git a/Doc/library/collections.abc.rst b/Doc/library/collections.abc.rst
index 4b21932..356f473 100644
--- a/Doc/library/collections.abc.rst
+++ b/Doc/library/collections.abc.rst
@@ -15,7 +15,7 @@
import itertools
__name__ = '<doctest>'
-**Source code:** :source:`Lib/collections/abc.py`
+**Source code:** :source:`Lib/_collections_abc.py`
--------------
diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst
index 7645f2a..80c6c76 100644
--- a/Doc/library/collections.rst
+++ b/Doc/library/collections.rst
@@ -76,14 +76,19 @@ The class can be used to simulate nested scopes and is useful in templating.
be modified to change which mappings are searched. The list should
always contain at least one mapping.
- .. method:: new_child()
+ .. method:: new_child(m=None)
- Returns a new :class:`ChainMap` containing a new :class:`dict` followed by
- all of the maps in the current instance. A call to ``d.new_child()`` is
- equivalent to: ``ChainMap({}, *d.maps)``. This method is used for
+ Returns a new :class:`ChainMap` containing a new map followed by
+ all of the maps in the current instance. If ``m`` is specified,
+ it becomes the new map at the front of the list of mappings; if not
+ specified, an empty dict is used, so that a call to ``d.new_child()``
+ is equivalent to: ``ChainMap({}, *d.maps)``. This method is used for
creating subcontexts that can be updated without altering values in any
of the parent mappings.
+ .. versionchanged:: 3.4
+ The optional ``m`` parameter was added.
+
.. attribute:: parents
Property returning a new :class:`ChainMap` containing all of the maps in
@@ -369,10 +374,6 @@ or subtracting from an empty counter.
.. seealso::
- * `Counter class <http://code.activestate.com/recipes/576611/>`_
- adapted for Python 2.5 and an early `Bag recipe
- <http://code.activestate.com/recipes/259174/>`_ for Python 2.4.
-
* `Bag class <http://www.gnu.org/software/smalltalk/manual-base/html_node/Bag.html>`_
in Smalltalk.
@@ -915,11 +916,6 @@ and more efficient to use a simple class declaration:
>>> class Status:
open, pending, closed = range(3)
-.. seealso::
-
- * `Named tuple recipe <http://code.activestate.com/recipes/500261/>`_
- adapted for Python 2.4.
-
* `Recipe for named tuple abstract base class with a metaclass mix-in
<http://code.activestate.com/recipes/577629-namedtupleabc-abstract-base-class-mix-in-for-named/>`_
by Jan Kaliszewski. Besides providing an :term:`abstract base class` for
@@ -982,10 +978,6 @@ The :class:`OrderedDict` constructor and :meth:`update` method both accept
keyword arguments, but their order is lost because Python's function call
semantics pass-in keyword arguments using a regular unordered dictionary.
-.. seealso::
-
- `Equivalent OrderedDict recipe <http://code.activestate.com/recipes/576693/>`_
- that runs on Python 2.4 or later.
:class:`OrderedDict` Examples and Recipes
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
diff --git a/Doc/library/concurrency.rst b/Doc/library/concurrency.rst
index f2a512f..0de281b 100644
--- a/Doc/library/concurrency.rst
+++ b/Doc/library/concurrency.rst
@@ -20,7 +20,6 @@ multitasking). Here's an overview:
subprocess.rst
sched.rst
queue.rst
- select.rst
The following are support modules for some of the above services:
diff --git a/Doc/library/concurrent.futures.rst b/Doc/library/concurrent.futures.rst
index c2f92b3..93538e4 100644
--- a/Doc/library/concurrent.futures.rst
+++ b/Doc/library/concurrent.futures.rst
@@ -164,6 +164,9 @@ uses a pool of processes to execute calls asynchronously.
allows it to side-step the :term:`Global Interpreter Lock` but also means that
only picklable objects can be executed and returned.
+The ``__main__`` module must be importable by worker subprocesses. This means
+that :class:`ProcessPoolExecutor` will not work in the interactive interpreter.
+
Calling :class:`Executor` or :class:`Future` methods from a callable submitted
to a :class:`ProcessPoolExecutor` will result in deadlock.
diff --git a/Doc/library/contextlib.rst b/Doc/library/contextlib.rst
index fba48f4..82efd0c 100644
--- a/Doc/library/contextlib.rst
+++ b/Doc/library/contextlib.rst
@@ -95,6 +95,83 @@ Functions and classes provided:
``page.close()`` will be called when the :keyword:`with` block is exited.
+.. function:: suppress(*exceptions)
+
+ Return a context manager that suppresses any of the specified exceptions
+ if they occur in the body of a with statement and then resumes execution
+ with the first statement following the end of the with statement.
+
+ As with any other mechanism that completely suppresses exceptions, this
+ context manager should be used only to cover very specific errors where
+ silently continuing with program execution is known to be the right
+ thing to do.
+
+ For example::
+
+ from contextlib import suppress
+
+ with suppress(FileNotFoundError):
+ os.remove('somefile.tmp')
+
+ with suppress(FileNotFoundError):
+ os.remove('someotherfile.tmp')
+
+ This code is equivalent to::
+
+ try:
+ os.remove('somefile.tmp')
+ except FileNotFoundError:
+ pass
+
+ try:
+ os.remove('someotherfile.tmp')
+ except FileNotFoundError:
+ pass
+
+ This context manager is :ref:`reentrant <reentrant-cms>`.
+
+ .. versionadded:: 3.4
+
+
+.. function:: redirect_stdout(new_target)
+
+ Context manager for temporarily redirecting :data:`sys.stdout` to
+ another file or file-like object.
+
+ This tool adds flexibility to existing functions or classes whose output
+ is hardwired to stdout.
+
+ For example, the output of :func:`help` normally is sent to *sys.stdout*.
+ You can capture that output in a string by redirecting the output to a
+ :class:`io.StringIO` object::
+
+ f = io.StringIO()
+ with redirect_stdout(f):
+ help(pow)
+ s = f.getvalue()
+
+ To send the output of :func:`help` to a file on disk, redirect the output
+ to a regular file::
+
+ with open('help.txt', 'w') as f:
+ with redirect_stdout(f):
+ help(pow)
+
+ To send the output of :func:`help` to *sys.stderr*::
+
+ with redirect_stdout(sys.stderr):
+ help(pow)
+
+ Note that the global side effect on :data:`sys.stdout` means that this
+ context manager is not suitable for use in library code and most threaded
+ applications. It also has no effect on the output of subprocesses.
+ However, it is still a useful approach for many utility scripts.
+
+ This context manager is :ref:`reusable but not reentrant <reusable-cms>`.
+
+ .. versionadded:: 3.4
+
+
.. class:: ContextDecorator()
A base class that enables a context manager to also be used as a decorator.
@@ -520,3 +597,153 @@ an explicit ``with`` statement.
The specification, background, and examples for the Python :keyword:`with`
statement.
+.. _single-use-reusable-and-reentrant-cms:
+
+Single use, reusable and reentrant context managers
+---------------------------------------------------
+
+Most context managers are written in a way that means they can only be
+used effectively in a :keyword:`with` statement once. These single use
+context managers must be created afresh each time they're used -
+attempting to use them a second time will trigger an exception or
+otherwise not work correctly.
+
+This common limitation means that it is generally advisable to create
+context managers directly in the header of the :keyword:`with` statement
+where they are used (as shown in all of the usage examples above).
+
+Files are an example of effectively single use context managers, since
+the first :keyword:`with` statement will close the file, preventing any
+further IO operations using that file object.
+
+Context managers created using :func:`contextmanager` are also single use
+context managers, and will complain about the underlying generator failing
+to yield if an attempt is made to use them a second time::
+
+ >>> from contextlib import contextmanager
+ >>> @contextmanager
+ ... def singleuse():
+ ... print("Before")
+ ... yield
+ ... print("After")
+ ...
+ >>> cm = singleuse()
+ >>> with cm:
+ ... pass
+ ...
+ Before
+ After
+ >>> with cm:
+ ... pass
+ ...
+ Traceback (most recent call last):
+ ...
+ RuntimeError: generator didn't yield
+
+
+.. _reentrant-cms:
+
+Reentrant context managers
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+More sophisticated context managers may be "reentrant". These context
+managers can not only be used in multiple :keyword:`with` statements,
+but may also be used *inside* a :keyword:`with` statement that is already
+using the same context manager.
+
+:class:`threading.RLock` is an example of a reentrant context manager, as are
+:func:`suppress` and :func:`redirect_stdout`. Here's a very simple example of
+reentrant use::
+
+ >>> from contextlib import redirect_stdout
+ >>> from io import StringIO
+ >>> stream = StringIO()
+ >>> write_to_stream = redirect_stdout(stream)
+ >>> with write_to_stream:
+ ... print("This is written to the stream rather than stdout")
+ ... with write_to_stream:
+ ... print("This is also written to the stream")
+ ...
+ >>> print("This is written directly to stdout")
+ This is written directly to stdout
+ >>> print(stream.getvalue())
+ This is written to the stream rather than stdout
+ This is also written to the stream
+
+Real world examples of reentrancy are more likely to involve multiple
+functions calling each other and hence be far more complicated than this
+example.
+
+Note also that being reentrant is *not* the same thing as being thread safe.
+:func:`redirect_stdout`, for example, is definitely not thread safe, as it
+makes a global modification to the system state by binding :data:`sys.stdout`
+to a different stream.
+
+
+.. _reusable-cms:
+
+Reusable context managers
+^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Distinct from both single use and reentrant context managers are "reusable"
+context managers (or, to be completely explicit, "reusable, but not
+reentrant" context managers, since reentrant context managers are also
+reusable). These context managers support being used multiple times, but
+will fail (or otherwise not work correctly) if the specific context manager
+instance has already been used in a containing with statement.
+
+:class:`threading.Lock` is an example of a reusable, but not reentrant,
+context manager (for a reentrant lock, it is necessary to use
+:class:`threading.RLock` instead).
+
+Another example of a reusable, but not reentrant, context manager is
+:class:`ExitStack`, as it invokes *all* currently registered callbacks
+when leaving any with statement, regardless of where those callbacks
+were added::
+
+ >>> from contextlib import ExitStack
+ >>> stack = ExitStack()
+ >>> with stack:
+ ... stack.callback(print, "Callback: from first context")
+ ... print("Leaving first context")
+ ...
+ Leaving first context
+ Callback: from first context
+ >>> with stack:
+ ... stack.callback(print, "Callback: from second context")
+ ... print("Leaving second context")
+ ...
+ Leaving second context
+ Callback: from second context
+ >>> with stack:
+ ... stack.callback(print, "Callback: from outer context")
+ ... with stack:
+ ... stack.callback(print, "Callback: from inner context")
+ ... print("Leaving inner context")
+ ... print("Leaving outer context")
+ ...
+ Leaving inner context
+ Callback: from inner context
+ Callback: from outer context
+ Leaving outer context
+
+As the output from the example shows, reusing a single stack object across
+multiple with statements works correctly, but attempting to nest them
+will cause the stack to be cleared at the end of the innermost with
+statement, which is unlikely to be desirable behaviour.
+
+Using separate :class:`ExitStack` instances instead of reusing a single
+instance avoids that problem::
+
+ >>> from contextlib import ExitStack
+ >>> with ExitStack() as outer_stack:
+ ... outer_stack.callback(print, "Callback: from outer context")
+ ... with ExitStack() as inner_stack:
+ ... inner_stack.callback(print, "Callback: from inner context")
+ ... print("Leaving inner context")
+ ... print("Leaving outer context")
+ ...
+ Leaving inner context
+ Callback: from inner context
+ Leaving outer context
+ Callback: from outer context
diff --git a/Doc/library/datatypes.rst b/Doc/library/datatypes.rst
index d0382e0..48af082 100644
--- a/Doc/library/datatypes.rst
+++ b/Doc/library/datatypes.rst
@@ -30,3 +30,4 @@ The following modules are documented in this chapter:
copy.rst
pprint.rst
reprlib.rst
+ enum.rst
diff --git a/Doc/library/datetime.rst b/Doc/library/datetime.rst
index 065e850..e4f1eb2 100644
--- a/Doc/library/datetime.rst
+++ b/Doc/library/datetime.rst
@@ -170,10 +170,12 @@ dates or times.
* ``0 <= seconds < 3600*24`` (the number of seconds in one day)
* ``-999999999 <= days <= 999999999``
- If any argument is a float and there are fractional microseconds, the fractional
- microseconds left over from all arguments are combined and their sum is rounded
- to the nearest microsecond. If no argument is a float, the conversion and
- normalization processes are exact (no information is lost).
+ If any argument is a float and there are fractional microseconds,
+ the fractional microseconds left over from all arguments are
+ combined and their sum is rounded to the nearest microsecond using
+ round-half-to-even tiebreaker. If no argument is a float, the
+ conversion and normalization processes are exact (no information is
+ lost).
If the normalized value of days lies outside the indicated range,
:exc:`OverflowError` is raised.
diff --git a/Doc/library/dbm.rst b/Doc/library/dbm.rst
index 81a05c7..f5496d5 100644
--- a/Doc/library/dbm.rst
+++ b/Doc/library/dbm.rst
@@ -73,33 +73,39 @@ Key and values are always stored as bytes. This means that when
strings are used they are implicitly converted to the default encoding before
being stored.
+These objects also support being used in a :keyword:`with` statement, which
+will automatically close them when done.
+
+.. versionchanged:: 3.4
+ Added native support for the context management protocol to the objects
+ returned by :func:`.open`.
+
The following example records some hostnames and a corresponding title, and
then prints out the contents of the database::
import dbm
# Open database, creating it if necessary.
- db = dbm.open('cache', 'c')
+ with dbm.open('cache', 'c') as db:
- # Record some values
- db[b'hello'] = b'there'
- db['www.python.org'] = 'Python Website'
- db['www.cnn.com'] = 'Cable News Network'
+ # Record some values
+ db[b'hello'] = b'there'
+ db['www.python.org'] = 'Python Website'
+ db['www.cnn.com'] = 'Cable News Network'
- # Note that the keys are considered bytes now.
- assert db[b'www.python.org'] == b'Python Website'
- # Notice how the value is now in bytes.
- assert db['www.cnn.com'] == b'Cable News Network'
+ # Note that the keys are considered bytes now.
+ assert db[b'www.python.org'] == b'Python Website'
+ # Notice how the value is now in bytes.
+ assert db['www.cnn.com'] == b'Cable News Network'
- # Often-used methods of the dict interface work too.
- print(db.get('python.org', b'not present'))
+ # Often-used methods of the dict interface work too.
+ print(db.get('python.org', b'not present'))
- # Storing a non-string key or value will raise an exception (most
- # likely a TypeError).
- db['www.yahoo.com'] = 4
+ # Storing a non-string key or value will raise an exception (most
+ # likely a TypeError).
+ db['www.yahoo.com'] = 4
- # Close when done.
- db.close()
+ # db is automatically closed when leaving the with statement.
.. seealso::
diff --git a/Doc/library/debug.rst b/Doc/library/debug.rst
index c69fb1c..88a2fa6 100644
--- a/Doc/library/debug.rst
+++ b/Doc/library/debug.rst
@@ -15,3 +15,4 @@ allowing you to identify bottlenecks in your programs.
profile.rst
timeit.rst
trace.rst
+ tracemalloc.rst
diff --git a/Doc/library/development.rst b/Doc/library/development.rst
index 2368769..06e7048 100644
--- a/Doc/library/development.rst
+++ b/Doc/library/development.rst
@@ -23,4 +23,3 @@ The list of modules described in this chapter is:
unittest.mock-examples.rst
2to3.rst
test.rst
- venv.rst
diff --git a/Doc/library/dis.rst b/Doc/library/dis.rst
index ec7112d..2365f0a 100644
--- a/Doc/library/dis.rst
+++ b/Doc/library/dis.rst
@@ -26,7 +26,8 @@ Example: Given the function :func:`myfunc`::
def myfunc(alist):
return len(alist)
-the following command can be used to get the disassembly of :func:`myfunc`::
+the following command can be used to display the disassembly of
+:func:`myfunc`::
>>> dis.dis(myfunc)
2 0 LOAD_GLOBAL 0 (len)
@@ -36,8 +37,75 @@ the following command can be used to get the disassembly of :func:`myfunc`::
(The "2" is a line number).
-The :mod:`dis` module defines the following functions and constants:
+Bytecode analysis
+-----------------
+The bytecode analysis API allows pieces of Python code to be wrapped in a
+:class:`Bytecode` object that provides easy access to details of the
+compiled code.
+
+.. class:: Bytecode(x, *, first_line=None, current_offset=None)
+
+ Analyse the bytecode corresponding to a function, method, string of
+ source code, or a code object (as returned by :func:`compile`).
+
+ This is a convenience wrapper around many of the functions listed below,
+ most notably :func:`get_instructions`, as iterating over a
+ :class:`Bytecode` instance yields the bytecode operations as
+ :class:`Instruction` instances.
+
+ If *first_line* is not None, it indicates the line number that should
+ be reported for the first source line in the disassembled code.
+ Otherwise, the source line information (if any) is taken directly from
+ the disassembled code object.
+
+ If *current_offset* is not None, it refers to an instruction offset
+ in the disassembled code. Setting this means :meth:`dis` will display
+ a "current instruction" marker against the specified opcode.
+
+ .. classmethod:: from_traceback(tb)
+
+ Construct a :class:`Bytecode` instance from the given traceback,
+ setting *current_offset* to the instruction responsible for the
+ exception.
+
+ .. data:: codeobj
+
+ The compiled code object.
+
+ .. data:: first_line
+
+ The first source line of the code object (if available)
+
+ .. method:: dis()
+
+ Return a formatted view of the bytecode operations (the same as
+ printed by :func:`dis`, but returned as a multi-line string).
+
+ .. method:: info()
+
+ Return a formatted multi-line string with detailed information about the
+ code object, like :func:`code_info`.
+
+Example::
+
+ >>> bytecode = dis.Bytecode(myfunc)
+ >>> for instr in bytecode:
+ ... print(instr.opname)
+ ...
+ LOAD_GLOBAL
+ LOAD_FAST
+ CALL_FUNCTION
+ RETURN_VALUE
+
+
+Analysis functions
+------------------
+
+The :mod:`dis` module also defines the following analysis functions that
+convert the input directly to the desired output. They can be useful if
+only a single operation is being performed, so the intermediate analysis
+object isn't useful:
.. function:: code_info(x)
@@ -51,17 +119,22 @@ The :mod:`dis` module defines the following functions and constants:
.. versionadded:: 3.2
-.. function:: show_code(x)
+.. function:: show_code(x, *, file=None)
Print detailed code object information for the supplied function, method,
- source code string or code object to stdout.
+ source code string or code object to *file* (or ``sys.stdout`` if *file*
+ is not specified).
- This is a convenient shorthand for ``print(code_info(x))``, intended for
- interactive exploration at the interpreter prompt.
+ This is a convenient shorthand for ``print(code_info(x), file=file)``,
+ intended for interactive exploration at the interpreter prompt.
.. versionadded:: 3.2
-.. function:: dis(x=None)
+ .. versionchanged:: 3.4
+ Added ``file`` parameter
+
+
+.. function:: dis(x=None, *, file=None)
Disassemble the *x* object. *x* can denote either a module, a class, a
method, a function, a code object, a string of source code or a byte sequence
@@ -72,16 +145,28 @@ The :mod:`dis` module defines the following functions and constants:
disassembled. If no object is provided, this function disassembles the last
traceback.
+ The disassembly is written as text to the supplied ``file`` argument if
+ provided and to ``sys.stdout`` otherwise.
+
+ .. versionchanged:: 3.4
+ Added ``file`` parameter
-.. function:: distb(tb=None)
+
+.. function:: distb(tb=None, *, file=None)
Disassemble the top-of-stack function of a traceback, using the last
traceback if none was passed. The instruction causing the exception is
indicated.
+ The disassembly is written as text to the supplied ``file`` argument if
+ provided and to ``sys.stdout`` otherwise.
+
+ .. versionchanged:: 3.4
+ Added ``file`` parameter
-.. function:: disassemble(code, lasti=-1)
- disco(code, lasti=-1)
+
+.. function:: disassemble(code, lasti=-1, *, file=None)
+ disco(code, lasti=-1, *, file=None)
Disassemble a code object, indicating the last instruction if *lasti* was
provided. The output is divided in the following columns:
@@ -97,6 +182,28 @@ The :mod:`dis` module defines the following functions and constants:
The parameter interpretation recognizes local and global variable names,
constant values, branch targets, and compare operators.
+ The disassembly is written as text to the supplied ``file`` argument if
+ provided and to ``sys.stdout`` otherwise.
+
+ .. versionchanged:: 3.4
+ Added ``file`` parameter
+
+
+.. function:: get_instructions(x, *, first_line=None)
+
+ Return an iterator over the instructions in the supplied function, method,
+ source code string or code object.
+
+ The iterator generates a series of :class:`Instruction` named tuples
+ giving the details of each operation in the supplied code.
+
+ If *first_line* is not None, it indicates the line number that should
+ be reported for the first source line in the disassembled code.
+ Otherwise, the source line information (if any) is taken directly from
+ the disassembled code object.
+
+ .. versionadded:: 3.4
+
.. function:: findlinestarts(code)
@@ -110,61 +217,60 @@ The :mod:`dis` module defines the following functions and constants:
Detect all offsets in the code object *code* which are jump targets, and
return a list of these offsets.
+.. _bytecodes:
-.. data:: opname
+Python Bytecode Instructions
+----------------------------
- Sequence of operation names, indexable using the bytecode.
+The :func:`get_instructions` function and :class:`Bytecode` class provide
+details of bytecode instructions as :class:`Instruction` instances:
+.. class:: Instruction
-.. data:: opmap
+ Details for a bytecode operation
- Dictionary mapping operation names to bytecodes.
+ .. data:: opcode
+ numeric code for operation, corresponding to the opcode values listed
+ below and the bytecode values in the :ref:`opcode_collections`.
-.. data:: cmp_op
- Sequence of all compare operation names.
+ .. data:: opname
+ human readable name for operation
-.. data:: hasconst
- Sequence of bytecodes that have a constant parameter.
+ .. data:: arg
+ numeric argument to operation (if any), otherwise None
-.. data:: hasfree
-
- Sequence of bytecodes that access a free variable.
+ .. data:: argval
-.. data:: hasname
+ resolved arg value (if known), otherwise same as arg
- Sequence of bytecodes that access an attribute by name.
-
-
-.. data:: hasjrel
- Sequence of bytecodes that have a relative jump target.
+ .. data:: argrepr
+ human readable description of operation argument
-.. data:: hasjabs
- Sequence of bytecodes that have an absolute jump target.
+ .. data:: offset
+ start index of operation within bytecode sequence
-.. data:: haslocal
- Sequence of bytecodes that access a local variable.
+ .. data:: starts_line
+ line started by this opcode (if any), otherwise None
-.. data:: hascompare
- Sequence of bytecodes of Boolean operations.
+ .. data:: is_jump_target
+ True if other code jumps to here, otherwise False
-.. _bytecodes:
+ .. versionadded:: 3.4
-Python Bytecode Instructions
-----------------------------
The Python compiler currently generates the following bytecode instructions.
@@ -506,12 +612,6 @@ the stack so that it is available for further iterations of the loop.
.. XXX explain the WHY stuff!
-.. opcode:: STORE_LOCALS
-
- Pops TOS from the stack and stores it as the current frame's ``f_locals``.
- This is used in class construction.
-
-
All of the following opcodes expect arguments. An argument is two bytes, with
the more significant byte last.
@@ -722,6 +822,13 @@ the more significant byte last.
Pushes a reference to the object the cell contains on the stack.
+.. opcode:: LOAD_CLASSDEREF (i)
+
+ Much like :opcode:`LOAD_DEREF` but first checks the locals dictionary before
+ consulting the cell. This is used for loading free variables in class
+ bodies.
+
+
.. opcode:: STORE_DEREF (i)
Stores TOS into the cell contained in slot *i* of the cell and free variable
@@ -820,3 +927,62 @@ the more significant byte last.
which don't take arguments ``< HAVE_ARGUMENT`` and those which do ``>=
HAVE_ARGUMENT``.
+.. _opcode_collections:
+
+Opcode collections
+------------------
+
+These collections are provided for automatic introspection of bytecode
+instructions:
+
+.. data:: opname
+
+ Sequence of operation names, indexable using the bytecode.
+
+
+.. data:: opmap
+
+ Dictionary mapping operation names to bytecodes.
+
+
+.. data:: cmp_op
+
+ Sequence of all compare operation names.
+
+
+.. data:: hasconst
+
+ Sequence of bytecodes that have a constant parameter.
+
+
+.. data:: hasfree
+
+ Sequence of bytecodes that access a free variable (note that 'free' in
+ this context refers to names in the current scope that are referenced by
+ inner scopes or names in outer scopes that are referenced from this scope.
+ It does *not* include references to global or builtin scopes).
+
+
+.. data:: hasname
+
+ Sequence of bytecodes that access an attribute by name.
+
+
+.. data:: hasjrel
+
+ Sequence of bytecodes that have a relative jump target.
+
+
+.. data:: hasjabs
+
+ Sequence of bytecodes that have an absolute jump target.
+
+
+.. data:: haslocal
+
+ Sequence of bytecodes that access a local variable.
+
+
+.. data:: hascompare
+
+ Sequence of bytecodes of Boolean operations.
diff --git a/Doc/library/distribution.rst b/Doc/library/distribution.rst
new file mode 100644
index 0000000..fb3f5df
--- /dev/null
+++ b/Doc/library/distribution.rst
@@ -0,0 +1,14 @@
+***********************************
+Software Packaging and Distribution
+***********************************
+
+These libraries help you with publishing and installing Python software.
+While these modules are designed to work in conjunction with the
+`Python Package Index <https://pypi.python.org>`__, they can also be used
+with a local index server, or without any index server at all.
+
+.. toctree::
+
+ distutils.rst
+ ensurepip.rst
+ venv.rst
diff --git a/Doc/library/doctest.rst b/Doc/library/doctest.rst
index 222c719..6138e30 100644
--- a/Doc/library/doctest.rst
+++ b/Doc/library/doctest.rst
@@ -495,7 +495,10 @@ Option Flags
A number of option flags control various aspects of doctest's behavior.
Symbolic names for the flags are supplied as module constants, which can be
or'ed together and passed to various functions. The names can also be used in
-:ref:`doctest directives <doctest-directives>`.
+:ref:`doctest directives <doctest-directives>`, and may be passed to the
+doctest command line interface via the ``-o`` option.
+
+.. versionadded:: 3.4 the ``-o`` command line option
The first group of options define test semantics, controlling aspects of how
doctest decides whether actual output matches an example's expected output:
@@ -633,6 +636,19 @@ The second group of options controls how test failures are reported:
the output is suppressed.
+.. data:: FAIL_FAST
+
+ When specified, exit after the first failing example and don't attempt to run
+ the remaining examples. Thus, the number of failures reported will be at most
+ 1. This flag may be useful during debugging, since examples after the first
+ failure won't even produce debugging output.
+
+ The doctest command line accepts the option ``-f`` as a shorthand for ``-o
+ FAIL_FAST``.
+
+ .. versionadded:: 3.4
+
+
.. data:: REPORTING_FLAGS
A bitmask or'ing together all the reporting flags above.
diff --git a/Doc/library/email.contentmanager.rst b/Doc/library/email.contentmanager.rst
new file mode 100644
index 0000000..f3ba794
--- /dev/null
+++ b/Doc/library/email.contentmanager.rst
@@ -0,0 +1,427 @@
+:mod:`email.contentmanager`: Managing MIME Content
+--------------------------------------------------
+
+.. module:: email.contentmanager
+ :synopsis: Storing and Retrieving Content from MIME Parts
+
+.. moduleauthor:: R. David Murray <rdmurray@bitdance.com>
+.. sectionauthor:: R. David Murray <rdmurray@bitdance.com>
+
+
+.. note::
+
+ The contentmanager module has been included in the standard library on a
+ :term:`provisional basis <provisional package>`. Backwards incompatible
+ changes (up to and including removal of the module) may occur if deemed
+ necessary by the core developers.
+
+.. versionadded:: 3.4
+ as a :term:`provisional module <provisional package>`.
+
+The :mod:`~email.message` module provides a class that can represent an
+arbitrary email message. That basic message model has a useful and flexible
+API, but it provides only a lower-level API for interacting with the generic
+parts of a message (the headers, generic header parameters, and the payload,
+which may be a list of sub-parts). This module provides classes and tools
+that provide an enhanced and extensible API for dealing with various specific
+types of content, including the ability to retrieve the content of the message
+as a specialized object type rather than as a simple bytes object. The module
+automatically takes care of the RFC-specified MIME details (required headers
+and parameters, etc.) for the certain common content types content properties,
+and support for additional types can be added by an application using the
+extension mechanisms.
+
+This module defines the eponymous "Content Manager" classes. The base
+:class:`.ContentManager` class defines an API for registering content
+management functions which extract data from ``Message`` objects or insert data
+and headers into ``Message`` objects, thus providing a way of converting
+between ``Message`` objects containing data and other representations of that
+data (Python data types, specialized Python objects, external files, etc). The
+module also defines one concrete content manager: :data:`raw_data_manager`
+converts between MIME content types and ``str`` or ``bytes`` data. It also
+provides a convenient API for managing the MIME parameters when inserting
+content into ``Message``\ s. It also handles inserting and extracting
+``Message`` objects when dealing with the ``message/rfc822`` content type.
+
+Another part of the enhanced interface is subclasses of
+:class:`~email.message.Message` that provide new convenience API functions,
+including convenience methods for calling the Content Managers derived from
+this module.
+
+.. note::
+
+ Although :class:`.EmailMessage` and :class:`.MIMEPart` are currently
+ documented in this module because of the provisional nature of the code, the
+ implementation lives in the :mod:`email.message` module.
+
+
+.. class:: EmailMessage(policy=default)
+
+ If *policy* is specified (it must be an instance of a :mod:`~email.policy`
+ class) use the rules it specifies to udpate and serialize the representation
+ of the message. If *policy* is not set, use the
+ :class:`~email.policy.default` policy, which follows the rules of the email
+ RFCs except for line endings (instead of the RFC mandated ``\r\n``, it uses
+ the Python standard ``\n`` line endings). For more information see the
+ :mod:`~email.policy` documentation.
+
+ This class is a subclass of :class:`~email.message.Message`. It adds
+ the following methods:
+
+
+ .. attribute:: is_attachment
+
+ Set to ``True`` if there is a :mailheader:`Content-Disposition` header
+ and its (case insensitive) value is ``attachment``, ``False`` otherwise.
+
+
+ .. method:: get_body(preferencelist=('related', 'html', 'plain'))
+
+ Return the MIME part that is the best candidate to be the "body" of the
+ message.
+
+ *preferencelist* must be a sequence of strings from the set ``related``,
+ ``html``, and ``plain``, and indicates the order of preference for the
+ content type of the part returned.
+
+ Start looking for candidate matches with the object on which the
+ ``get_body`` method is called.
+
+ If ``related`` is not included in *preferencelist*, consider the root
+ part (or subpart of the root part) of any related encountered as a
+ candidate if the (sub-)part matches a preference.
+
+ When encountering a ``multipart/related``, check the ``start`` parameter
+ and if a part with a matching :mailheader:`Content-ID` is found, consider
+ only it when looking for candidate matches. Otherwise consider only the
+ first (default root) part of the ``multipart/related``.
+
+ If a part has a :mailheader:``Content-Disposition`` header, only consider
+ the part a candidate match if the value of the header is ``inline``.
+
+ If none of the candidates matches any of the preferences in
+ *preferneclist*, return ``None``.
+
+ Notes: (1) For most applications the only *preferencelist* combinations
+ that really make sense are ``('plain',)``, ``('html', 'plain')``, and the
+ default, ``('related', 'html', 'plain')``. (2) Because matching starts
+ with the object on which ``get_body`` is called, calling ``get_body`` on
+ a ``multipart/related`` will return the object itself unless
+ *preferencelist* has a non-default value. (3) Messages (or message parts)
+ that do not specify a :mailheader:`Content-Type` or whose
+ :mailheader:`Content-Type` header is invalid will be treated as if they
+ are of type ``text/plain``, which may occasionally cause ``get_body`` to
+ return unexpected results.
+
+
+ .. method:: iter_attachments()
+
+ Return an iterator over all of the parts of the message that are not
+ candidate "body" parts. That is, skip the first occurrence of each of
+ ``text/plain``, ``text/html``, ``multipart/related``, or
+ ``multipart/alternative`` (unless they are explicitly marked as
+ attachments via :mailheader:`Content-Disposition: attachment`), and
+ return all remaining parts. When applied directly to a
+ ``multipart/related``, return an iterator over the all the related parts
+ except the root part (ie: the part pointed to by the ``start`` parameter,
+ or the first part if there is no ``start`` parameter or the ``start``
+ parameter doesn't match the :mailheader:`Content-ID` of any of the
+ parts). When applied directly to a ``multipart/alternative`` or a
+ non-``multipart``, return an empty iterator.
+
+
+ .. method:: iter_parts()
+
+ Return an iterator over all of the immediate sub-parts of the message,
+ which will be empty for a non-``multipart``. (See also
+ :meth:``~email.message.walk``.)
+
+
+ .. method:: get_content(*args, content_manager=None, **kw)
+
+ Call the ``get_content`` method of the *content_manager*, passing self
+ as the message object, and passing along any other arguments or keywords
+ as additional arguments. If *content_manager* is not specified, use
+ the ``content_manager`` specified by the current :mod:`~email.policy`.
+
+
+ .. method:: set_content(*args, content_manager=None, **kw)
+
+ Call the ``set_content`` method of the *content_manager*, passing self
+ as the message object, and passing along any other arguments or keywords
+ as additional arguments. If *content_manager* is not specified, use
+ the ``content_manager`` specified by the current :mod:`~email.policy`.
+
+
+ .. method:: make_related(boundary=None)
+
+ Convert a non-``multipart`` message into a ``multipart/related`` message,
+ moving any existing :mailheader:`Content-` headers and payload into a
+ (new) first part of the ``multipart``. If *boundary* is specified, use
+ it as the boundary string in the multipart, otherwise leave the boundary
+ to be automatically created when it is needed (for example, when the
+ message is serialized).
+
+
+ .. method:: make_alternative(boundary=None)
+
+ Convert a non-``multipart`` or a ``multipart/related`` into a
+ ``multipart/alternative``, moving any existing :mailheader:`Content-`
+ headers and payload into a (new) first part of the ``multipart``. If
+ *boundary* is specified, use it as the boundary string in the multipart,
+ otherwise leave the boundary to be automatically created when it is
+ needed (for example, when the message is serialized).
+
+
+ .. method:: make_mixed(boundary=None)
+
+ Convert a non-``multipart``, a ``multipart/related``, or a
+ ``multipart-alternative`` into a ``multipart/mixed``, moving any existing
+ :mailheader:`Content-` headers and payload into a (new) first part of the
+ ``multipart``. If *boundary* is specified, use it as the boundary string
+ in the multipart, otherwise leave the boundary to be automatically
+ created when it is needed (for example, when the message is serialized).
+
+
+ .. method:: add_related(*args, content_manager=None, **kw)
+
+ If the message is a ``multipart/related``, create a new message
+ object, pass all of the arguments to its :meth:`set_content` method,
+ and :meth:`~email.message.Message.attach` it to the ``multipart``. If
+ the message is a non-``multipart``, call :meth:`make_related` and then
+ proceed as above. If the message is any other type of ``multipart``,
+ raise a :exc:`TypeError`. If *content_manager* is not specified, use
+ the ``content_manager`` specified by the current :mod:`~email.policy`.
+ If the added part has no :mailheader:`Content-Disposition` header,
+ add one with the value ``inline``.
+
+
+ .. method:: add_alternative(*args, content_manager=None, **kw)
+
+ If the message is a ``multipart/alternative``, create a new message
+ object, pass all of the arguments to its :meth:`set_content` method, and
+ :meth:`~email.message.Message.attach` it to the ``multipart``. If the
+ message is a non-``multipart`` or ``multipart/related``, call
+ :meth:`make_alternative` and then proceed as above. If the message is
+ any other type of ``multipart``, raise a :exc:`TypeError`. If
+ *content_manager* is not specified, use the ``content_manager`` specified
+ by the current :mod:`~email.policy`.
+
+
+ .. method:: add_attachment(*args, content_manager=None, **kw)
+
+ If the message is a ``multipart/mixed``, create a new message object,
+ pass all of the arguments to its :meth:`set_content` method, and
+ :meth:`~email.message.Message.attach` it to the ``multipart``. If the
+ message is a non-``multipart``, ``multipart/related``, or
+ ``multipart/alternative``, call :meth:`make_mixed` and then proceed as
+ above. If *content_manager* is not specified, use the ``content_manager``
+ specified by the current :mod:`~email.policy`. If the added part
+ has no :mailheader:`Content-Disposition` header, add one with the value
+ ``attachment``. This method can be used both for explicit attachments
+ (:mailheader:`Content-Disposition: attachment` and ``inline`` attachments
+ (:mailheader:`Content-Disposition: inline`), by passing appropriate
+ options to the ``content_manager``.
+
+
+ .. method:: clear()
+
+ Remove the payload and all of the headers.
+
+
+ .. method:: clear_content()
+
+ Remove the payload and all of the :exc:`Content-` headers, leaving
+ all other headers intact and in their original order.
+
+
+.. class:: ContentManager()
+
+ Base class for content managers. Provides the standard registry mechanisms
+ to register converters between MIME content and other representations, as
+ well as the ``get_content`` and ``set_content`` dispatch methods.
+
+
+ .. method:: get_content(msg, *args, **kw)
+
+ Look up a handler function based on the ``mimetype`` of *msg* (see next
+ paragraph), call it, passing through all arguments, and return the result
+ of the call. The expectation is that the handler will extract the
+ payload from *msg* and return an object that encodes information about
+ the extracted data.
+
+ To find the handler, look for the following keys in the registry,
+ stopping with the first one found:
+
+ * the string representing the full MIME type (``maintype/subtype``)
+ * the string representing the ``maintype``
+ * the empty string
+
+ If none of these keys produce a handler, raise a :exc:`KeyError` for the
+ full MIME type.
+
+
+ .. method:: set_content(msg, obj, *args, **kw)
+
+ If the ``maintype`` is ``multipart``, raise a :exc:`TypeError`; otherwise
+ look up a handler function based on the type of *obj* (see next
+ paragraph), call :meth:`~email.message.EmailMessage.clear_content` on the
+ *msg*, and call the handler function, passing through all arguments. The
+ expectation is that the handler will transform and store *obj* into
+ *msg*, possibly making other changes to *msg* as well, such as adding
+ various MIME headers to encode information needed to interpret the stored
+ data.
+
+ To find the handler, obtain the type of *obj* (``typ = type(obj)``), and
+ look for the following keys in the registry, stopping with the first one
+ found:
+
+ * the type itself (``typ``)
+ * the type's fully qualified name (``typ.__module__ + '.' +
+ typ.__qualname__``).
+ * the type's qualname (``typ.__qualname__``)
+ * the type's name (``typ.__name__``).
+
+ If none of the above match, repeat all of the checks above for each of
+ the types in the :term:`MRO` (``typ.__mro__``). Finally, if no other key
+ yields a handler, check for a handler for the key ``None``. If there is
+ no handler for ``None``, raise a :exc:`KeyError` for the fully
+ qualified name of the type.
+
+ Also add a :mailheader:`MIME-Version` header if one is not present (see
+ also :class:`.MIMEPart`).
+
+
+ .. method:: add_get_handler(key, handler)
+
+ Record the function *handler* as the handler for *key*. For the possible
+ values of *key*, see :meth:`get_content`.
+
+
+ .. method:: add_set_handler(typekey, handler)
+
+ Record *handler* as the function to call when an object of a type
+ matching *typekey* is passed to :meth:`set_content`. For the possible
+ values of *typekey*, see :meth:`set_content`.
+
+
+.. class:: MIMEPart(policy=default)
+
+ This class represents a subpart of a MIME message. It is identical to
+ :class:`EmailMessage`, except that no :mailheader:`MIME-Version` headers are
+ added when :meth:`~EmailMessage.set_content` is called, since sub-parts do
+ not need their own :mailheader:`MIME-Version` headers.
+
+
+Content Manager Instances
+~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Currently the email package provides only one concrete content manager,
+:data:`raw_data_manager`, although more may be added in the future.
+:data:`raw_data_manager` is the
+:attr:`~email.policy.EmailPolicy.content_manager` provided by
+:attr:`~email.policy.EmailPolicy` and its derivatives.
+
+
+.. data:: raw_data_manager
+
+ This content manager provides only a minimum interface beyond that provided
+ by :class:`~email.message.Message` itself: it deals only with text, raw
+ byte strings, and :class:`~email.message.Message` objects. Nevertheless, it
+ provides significant advantages compared to the base API: ``get_content`` on
+ a text part will return a unicode string without the application needing to
+ manually decode it, ``set_content`` provides a rich set of options for
+ controlling the headers added to a part and controlling the content transfer
+ encoding, and it enables the use of the various ``add_`` methods, thereby
+ simplifying the creation of multipart messages.
+
+ .. method:: get_content(msg, errors='replace')
+
+ Return the payload of the part as either a string (for ``text`` parts), a
+ :class:`~email.message.EmailMessage` object (for ``message/rfc822``
+ parts), or a ``bytes`` object (for all other non-multipart types). Raise
+ a :exc:`KeyError` if called on a ``multipart``. If the part is a
+ ``text`` part and *errors* is specified, use it as the error handler when
+ decoding the payload to unicode. The default error handler is
+ ``replace``.
+
+ .. method:: set_content(msg, <'str'>, subtype="plain", charset='utf-8' \
+ cte=None, \
+ disposition=None, filename=None, cid=None, \
+ params=None, headers=None)
+ set_content(msg, <'bytes'>, maintype, subtype, cte="base64", \
+ disposition=None, filename=None, cid=None, \
+ params=None, headers=None)
+ set_content(msg, <'Message'>, cte=None, \
+ disposition=None, filename=None, cid=None, \
+ params=None, headers=None)
+ set_content(msg, <'list'>, subtype='mixed', \
+ disposition=None, filename=None, cid=None, \
+ params=None, headers=None)
+
+ Add headers and payload to *msg*:
+
+ Add a :mailheader:`Content-Type` header with a ``maintype/subtype``
+ value.
+
+ * For ``str``, set the MIME ``maintype`` to ``text``, and set the
+ subtype to *subtype* if it is specified, or ``plain`` if it is not.
+ * For ``bytes``, use the specified *maintype* and *subtype*, or
+ raise a :exc:`TypeError` if they are not specified.
+ * For :class:`~email.message.Message` objects, set the maintype to
+ ``message``, and set the subtype to *subtype* if it is specified
+ or ``rfc822`` if it is not. If *subtype* is ``partial``, raise an
+ error (``bytes`` objects must be used to construct
+ ``message/partial`` parts).
+ * For *<'list'>*, which should be a list of
+ :class:`~email.message.Message` objects, set the ``maintype`` to
+ ``multipart``, and the ``subtype`` to *subtype* if it is
+ specified, and ``mixed`` if it is not. If the message parts in
+ the *<'list'>* have :mailheader:`MIME-Version` headers, remove
+ them.
+
+ If *charset* is provided (which is valid only for ``str``), encode the
+ string to bytes using the specified character set. The default is
+ ``utf-8``. If the specified *charset* is a known alias for a standard
+ MIME charset name, use the standard charset instead.
+
+ If *cte* is set, encode the payload using the specified content transfer
+ encoding, and set the :mailheader:`Content-Transfer-Endcoding` header to
+ that value. For ``str`` objects, if it is not set use heuristics to
+ determine the most compact encoding. Possible values for *cte* are
+ ``quoted-printable``, ``base64``, ``7bit``, ``8bit``, and ``binary``.
+ If the input cannot be encoded in the specified encoding (eg: ``7bit``),
+ raise a :exc:`ValueError`. For :class:`~email.message.Message`, per
+ :rfc:`2046`, raise an error if a *cte* of ``quoted-printable`` or
+ ``base64`` is requested for *subtype* ``rfc822``, and for any *cte*
+ other than ``7bit`` for *subtype* ``external-body``. For
+ ``message/rfc822``, use ``8bit`` if *cte* is not specified. For all
+ other values of *subtype*, use ``7bit``.
+
+ .. note:: A *cte* of ``binary`` does not actually work correctly yet.
+ The ``Message`` object as modified by ``set_content`` is correct, but
+ :class:`~email.generator.BytesGenerator` does not serialize it
+ correctly.
+
+ If *disposition* is set, use it as the value of the
+ :mailheader:`Content-Disposition` header. If not specified, and
+ *filename* is specified, add the header with the value ``attachment``.
+ If it is not specified and *filename* is also not specified, do not add
+ the header. The only valid values for *disposition* are ``attachment``
+ and ``inline``.
+
+ If *filename* is specified, use it as the value of the ``filename``
+ parameter of the :mailheader:`Content-Disposition` header. There is no
+ default.
+
+ If *cid* is specified, add a :mailheader:`Content-ID` header with
+ *cid* as its value.
+
+ If *params* is specified, iterate its ``items`` method and use the
+ resulting ``(key, value)`` pairs to set additional paramters on the
+ :mailheader:`Content-Type` header.
+
+ If *headers* is specified and is a list of strings of the form
+ ``headername: headervalue`` or a list of ``header`` objects
+ (distinguised from strings by having a ``name`` attribute), add the
+ headers to *msg*.
diff --git a/Doc/library/email.message.rst b/Doc/library/email.message.rst
index 576531d..f278a0a 100644
--- a/Doc/library/email.message.rst
+++ b/Doc/library/email.message.rst
@@ -33,23 +33,31 @@ Here are the methods of the :class:`Message` class:
.. class:: Message(policy=compat32)
- The *policy* argument determiens the :mod:`~email.policy` that will be used
- to update the message model. The default value, :class:`compat32
- <email.policy.Compat32>` maintains backward compatibility with the
- Python 3.2 version of the email package. For more information see the
+ If *policy* is specified (it must be an instance of a :mod:`~email.policy`
+ class) use the rules it specifies to udpate and serialize the representation
+ of the message. If *policy* is not set, use the :class`compat32
+ <email.policy.Compat32>` policy, which maintains backward compatibility with
+ the Python 3.2 version of the email package. For more information see the
:mod:`~email.policy` documentation.
.. versionchanged:: 3.3 The *policy* keyword argument was added.
- .. method:: as_string(unixfrom=False, maxheaderlen=0)
+ .. method:: as_string(unixfrom=False, maxheaderlen=0, policy=None)
Return the entire message flattened as a string. When optional *unixfrom*
- is ``True``, the envelope header is included in the returned string.
- *unixfrom* defaults to ``False``. Flattening the message may trigger
- changes to the :class:`Message` if defaults need to be filled in to
- complete the transformation to a string (for example, MIME boundaries may
- be generated or modified).
+ is true, the envelope header is included in the returned string.
+ *unixfrom* defaults to ``False``. For backward compabitility reasons,
+ *maxheaderlen* defaults to ``0``, so if you want a different value you
+ must override it explicitly (the value specified for *max_line_length* in
+ the policy will be ignored by this method). The *policy* argument may be
+ used to override the default policy obtained from the message instance.
+ This can be used to control some of the formatting produced by the
+ method, since the specified *policy* will be passed to the ``Generator``.
+
+ Flattening the message may trigger changes to the :class:`Message` if
+ defaults need to be filled in to complete the transformation to a string
+ (for example, MIME boundaries may be generated or modified).
Note that this method is provided as a convenience and may not always
format the message the way you want. For example, by default it does
@@ -65,10 +73,58 @@ Here are the methods of the :class:`Message` class:
g.flatten(msg)
text = fp.getvalue()
+ If the message object contains binary data that is not encoded according
+ to RFC standards, the non-compliant data will be replaced by unicode
+ "unknown character" code points. (See also :meth:`.as_bytes` and
+ :class:`~email.generator.BytesGenerator`.)
+
+ .. versionchanged:: 3.4 the *policy* keyword argument was added.
+
.. method:: __str__()
- Equivalent to ``as_string(unixfrom=True)``.
+ Equivalent to :meth:`.as_string()`. Allows ``str(msg)`` to produce a
+ string containing the formatted message.
+
+
+ .. method:: as_bytes(unixfrom=False, policy=None)
+
+ Return the entire message flattened as a bytes object. When optional
+ *unixfrom* is true, the envelope header is included in the returned
+ string. *unixfrom* defaults to ``False``. The *policy* argument may be
+ used to override the default policy obtained from the message instance.
+ This can be used to control some of the formatting produced by the
+ method, since the specified *policy* will be passed to the
+ ``BytesGenerator``.
+
+ Flattening the message may trigger changes to the :class:`Message` if
+ defaults need to be filled in to complete the transformation to a string
+ (for example, MIME boundaries may be generated or modified).
+
+ Note that this method is provided as a convenience and may not always
+ format the message the way you want. For example, by default it does
+ not do the mangling of lines that begin with ``From`` that is
+ required by the unix mbox format. For more flexibility, instantiate a
+ :class:`~email.generator.BytesGenerator` instance and use its
+ :meth:`~email.generator.BytesGenerator.flatten` method directly.
+ For example::
+
+ from io import BytesIO
+ from email.generator import BytesGenerator
+ fp = BytesIO()
+ g = BytesGenerator(fp, mangle_from_=True, maxheaderlen=60)
+ g.flatten(msg)
+ text = fp.getvalue()
+
+ .. versionadded:: 3.4
+
+
+ .. method:: __bytes__()
+
+ Equivalent to :meth:`.as_bytes()`. Allows ``bytes(msg)`` to produce a
+ bytes object containing the formatted message.
+
+ .. versionadded:: 3.4
.. method:: is_multipart()
@@ -410,7 +466,8 @@ Here are the methods of the :class:`Message` class:
to ``False``.
- .. method:: set_param(param, value, header='Content-Type', requote=True, charset=None, language='')
+ .. method:: set_param(param, value, header='Content-Type', requote=True,
+ charset=None, language='', replace=False)
Set a parameter in the :mailheader:`Content-Type` header. If the
parameter already exists in the header, its value will be replaced with
@@ -427,6 +484,12 @@ Here are the methods of the :class:`Message` class:
language, defaulting to the empty string. Both *charset* and *language*
should be strings.
+ If *replace* is ``False`` (the default) the header is moved to the
+ end of the list of headers. If *replace* is ``True``, the header
+ will be updated in place.
+
+ .. versionchanged: 3.4 ``replace`` keyword was added.
+
.. method:: del_param(param, header='content-type', requote=True)
diff --git a/Doc/library/email.policy.rst b/Doc/library/email.policy.rst
index cb2023c..c2f9e6a 100644
--- a/Doc/library/email.policy.rst
+++ b/Doc/library/email.policy.rst
@@ -97,6 +97,17 @@ correct line separator characters when creating the binary string to feed into
``sendmail's`` ``stdin``, where the default policy would use ``\n`` line
separators.
+Some email package methods accept a *policy* keyword argument, allowing the
+policy to be overridden for that method. For example, the following code uses
+the :meth:`~email.message.Message.as_string` method of the *msg* object from
+the previous example and writes the message to a file using the native line
+separators for the platform on which it is running::
+
+ >>> import os
+ >>> with open('converted.txt', 'wb') as f:
+ ... f.write(msg.as_bytes(policy=msg.policy.clone(linesep=os.linesep)))
+ 17
+
Policy objects can also be combined using the addition operator, producing a
policy object whose settings are a combination of the non-default values of the
summed objects::
@@ -360,7 +371,7 @@ added matters. To illustrate::
to) :rfc:`5322`, :rfc:`2047`, and the current MIME RFCs.
This policy adds new header parsing and folding algorithms. Instead of
- simple strings, headers are custom objects with custom attributes depending
+ simple strings, headers are ``str`` subclasses with attributes that depend
on the type of the field. The parsing and folding algorithm fully implement
:rfc:`2047` and :rfc:`5322`.
@@ -397,6 +408,20 @@ added matters. To illustrate::
fields are treated as unstructured. This list will be completed before
the extension is marked stable.)
+ .. attribute:: content_manager
+
+ An object with at least two methods: get_content and set_content. When
+ the :meth:`~email.message.Message.get_content` or
+ :meth:`~email.message.Message.set_content` method of a
+ :class:`~email.message.Message` object is called, it calls the
+ corresponding method of this object, passing it the message object as its
+ first argument, and any arguments or keywords that were passed to it as
+ additional arguments. By default ``content_manager`` is set to
+ :data:`~email.contentmanager.raw_data_manager`.
+
+ .. versionadded 3.4
+
+
The class provides the following concrete implementations of the abstract
methods of :class:`Policy`:
@@ -416,7 +441,7 @@ added matters. To illustrate::
The name is returned unchanged. If the input value has a ``name``
attribute and it matches *name* ignoring case, the value is returned
unchanged. Otherwise the *name* and *value* are passed to
- ``header_factory``, and the resulting custom header object is returned as
+ ``header_factory``, and the resulting header object is returned as
the value. In this case a ``ValueError`` is raised if the input value
contains CR or LF characters.
@@ -424,7 +449,7 @@ added matters. To illustrate::
If the value has a ``name`` attribute, it is returned to unmodified.
Otherwise the *name*, and the *value* with any CR or LF characters
- removed, are passed to the ``header_factory``, and the resulting custom
+ removed, are passed to the ``header_factory``, and the resulting
header object is returned. Any surrogateescaped bytes get turned into
the unicode unknown-character glyph.
@@ -434,9 +459,9 @@ added matters. To illustrate::
A value is considered to be a 'source value' if and only if it does not
have a ``name`` attribute (having a ``name`` attribute means it is a
header object of some sort). If a source value needs to be refolded
- according to the policy, it is converted into a custom header object by
+ according to the policy, it is converted into a header object by
passing the *name* and the *value* with any CR and LF characters removed
- to the ``header_factory``. Folding of a custom header object is done by
+ to the ``header_factory``. Folding of a header object is done by
calling its ``fold`` method with the current policy.
Source values are split into lines using :meth:`~str.splitlines`. If
@@ -491,23 +516,23 @@ With all of these :class:`EmailPolicies <.EmailPolicy>`, the effective API of
the email package is changed from the Python 3.2 API in the following ways:
* Setting a header on a :class:`~email.message.Message` results in that
- header being parsed and a custom header object created.
+ header being parsed and a header object created.
* Fetching a header value from a :class:`~email.message.Message` results
- in that header being parsed and a custom header object created and
+ in that header being parsed and a header object created and
returned.
- * Any custom header object, or any header that is refolded due to the
+ * Any header object, or any header that is refolded due to the
policy settings, is folded using an algorithm that fully implements the
RFC folding algorithms, including knowing where encoded words are required
and allowed.
From the application view, this means that any header obtained through the
-:class:`~email.message.Message` is a custom header object with custom
+:class:`~email.message.Message` is a header object with extra
attributes, whose string value is the fully decoded unicode value of the
header. Likewise, a header may be assigned a new value, or a new header
created, using a unicode string, and the policy will take care of converting
the unicode string into the correct RFC encoded form.
-The custom header objects and their attributes are described in
+The header objects and their attributes are described in
:mod:`~email.headerregistry`.
diff --git a/Doc/library/email.rst b/Doc/library/email.rst
index a6cbbce..331d2ef 100644
--- a/Doc/library/email.rst
+++ b/Doc/library/email.rst
@@ -53,6 +53,7 @@ Contents of the :mod:`email` package documentation:
email.generator.rst
email.policy.rst
email.headerregistry.rst
+ email.contentmanager.rst
email.mime.rst
email.header.rst
email.charset.rst
diff --git a/Doc/library/ensurepip.rst b/Doc/library/ensurepip.rst
new file mode 100644
index 0000000..3756d4f
--- /dev/null
+++ b/Doc/library/ensurepip.rst
@@ -0,0 +1,125 @@
+:mod:`ensurepip` --- Bootstrapping the ``pip`` installer
+========================================================
+
+.. module:: ensurepip
+ :synopsis: Bootstrapping the ``pip`` installer into an existing Python
+ installation or virtual environment.
+
+The :mod:`ensurepip` package provides support for bootstrapping the ``pip``
+installer into an existing Python installation or virtual environment. This
+bootstrapping approach reflects the fact that ``pip`` is an independent
+project with its own release cycle, and the latest available stable version
+is bundled with maintenance and feature releases of the CPython reference
+interpreter.
+
+In most cases, end users of Python shouldn't need to invoke this module
+directly (as ``pip`` should be bootstrapped by default), but it may be
+needed if installing ``pip`` was skipped when installing Python (or
+when creating a virtual environment) or after explicitly uninstalling
+``pip``.
+
+.. versionadded:: 3.4
+
+.. note::
+
+ This module *does not* access the internet. All of the components
+ needed to bootstrap ``pip`` are included as internal parts of the
+ package.
+
+.. seealso::
+
+ :ref:`install-index`
+ The end user guide for installing Python packages
+
+ :pep:`453`: Explicit bootstrapping of pip in Python installations
+ The original rationale and specification for this module.
+
+
+Command line interface
+----------------------
+
+The command line interface is invoked using the interpreter's ``-m`` switch.
+
+The simplest possible invocation is::
+
+ python -m ensurepip
+
+This invocation will install ``pip`` if it is not already installed,
+but otherwise does nothing. To ensure the installed version of ``pip``
+is at least as recent as the one bundled with ``ensurepip``, pass the
+``--upgrade`` option::
+
+ python -m ensurepip --upgrade
+
+By default, ``pip`` is installed into the current virtual environment
+(if one is active) or into the system site packages (if there is no
+active virtual environment). The installation location can be controlled
+through two additional command line options:
+
+* ``--root <dir>``: Installs ``pip`` relative to the given root directory
+ rather than the root of the currently active virtual environment (if any)
+ or the default root for the current Python installation.
+* ``--user``: Installs ``pip`` into the user site packages directory rather
+ than globally for the current Python installation (this option is not
+ permitted inside an active virtual environment).
+
+By default, the scripts ``pipX`` and ``pipX.Y`` will be installed (where
+X.Y stands for the version of Python used to invoke ``ensurepip``). The
+scripts installed can be controlled through two additional command line
+options:
+
+* ``--altinstall``: if an alternate installation is requested, the ``pipX``
+ script will *not* be installed.
+
+* ``--default-pip``: if a "default pip" installation is requested, the
+ ``pip`` script will be installed in addition to the two regular scripts.
+
+Providing both of the script selection options will trigger an exception.
+
+
+Module API
+----------
+
+:mod:`ensurepip` exposes two functions for programmatic use:
+
+.. function:: version()
+
+ Returns a string specifying the bundled version of pip that will be
+ installed when bootstrapping an environment.
+
+.. function:: bootstrap(root=None, upgrade=False, user=False, \
+ altinstall=False, default_pip=False, \
+ verbosity=0)
+
+ Bootstraps ``pip`` into the current or designated environment.
+
+ *root* specifies an alternative root directory to install relative to.
+ If *root* is None, then installation uses the default install location
+ for the current environment.
+
+ *upgrade* indicates whether or not to upgrade an existing installation
+ of an earlier version of ``pip`` to the bundled version.
+
+ *user* indicates whether to use the user scheme rather than installing
+ globally.
+
+ By default, the scripts ``pipX`` and ``pipX.Y`` will be installed (where
+ X.Y stands for the current version of Python).
+
+ If *altinstall* is set, then ``pipX`` will *not* be installed.
+
+ If *default_pip* is set, then ``pip`` will be installed in addition to
+ the two regular scripts.
+
+ Setting both *altinstall* and *default_pip* will trigger
+ :exc:`ValueError`.
+
+ *verbosity* controls the level of output to :data:`sys.stdout` from the
+ bootstrapping operation.
+
+ .. note::
+
+ The bootstrapping process may install additional modules required by
+ ``pip``, but other software should not assume those dependencies will
+ always be present by default (as the dependencies may be removed in a
+ future version of ``pip``).
diff --git a/Doc/library/enum.rst b/Doc/library/enum.rst
new file mode 100644
index 0000000..c2030fa
--- /dev/null
+++ b/Doc/library/enum.rst
@@ -0,0 +1,688 @@
+:mod:`enum` --- Support for enumerations
+========================================
+
+.. module:: enum
+ :synopsis: Implementation of an enumeration class.
+
+.. :moduleauthor:: Ethan Furman <ethan@stoneleaf.us>
+.. :sectionauthor:: Barry Warsaw <barry@python.org>,
+.. :sectionauthor:: Eli Bendersky <eliben@gmail.com>,
+.. :sectionauthor:: Ethan Furman <ethan@stoneleaf.us>
+
+**Source code:** :source:`Lib/enum.py`
+
+----------------
+
+An enumeration is a set of symbolic names (members) bound to unique, constant
+values. Within an enumeration, the members can be compared by identity, and
+the enumeration itself can be iterated over.
+
+This module defines two enumeration classes that can be used to define unique
+sets of names and values: :class:`Enum` and :class:`IntEnum`. It also defines
+one decorator, :func:`unique`, that ensures only unique member values are
+present in an enumeration.
+
+
+Creating an Enum
+----------------
+
+Enumerations are created using the :keyword:`class` syntax, which makes them
+easy to read and write. An alternative creation method is described in
+`Functional API`_. To define an enumeration, subclass :class:`Enum` as
+follows::
+
+ >>> from enum import Enum
+ >>> class Color(Enum):
+ ... red = 1
+ ... green = 2
+ ... blue = 3
+ ...
+
+.. note:: Nomenclature
+
+ - The class :class:`Color` is an *enumeration* (or *enum*)
+ - The attributes :attr:`Color.red`, :attr:`Color.green`, etc., are
+ *enumeration members* (or *enum members*).
+ - The enum members have *names* and *values* (the name of
+ :attr:`Color.red` is ``red``, the value of :attr:`Color.blue` is
+ ``3``, etc.)
+
+.. note::
+
+ Even though we use the :keyword:`class` syntax to create Enums, Enums
+ are not normal Python classes. See `How are Enums different?`_ for
+ more details.
+
+Enumeration members have human readable string representations::
+
+ >>> print(Color.red)
+ Color.red
+
+...while their ``repr`` has more information::
+
+ >>> print(repr(Color.red))
+ <Color.red: 1>
+
+The *type* of an enumeration member is the enumeration it belongs to::
+
+ >>> type(Color.red)
+ <enum 'Color'>
+ >>> isinstance(Color.green, Color)
+ True
+ >>>
+
+Enum members also have a property that contains just their item name::
+
+ >>> print(Color.red.name)
+ red
+
+Enumerations support iteration, in definition order::
+
+ >>> class Shake(Enum):
+ ... vanilla = 7
+ ... chocolate = 4
+ ... cookies = 9
+ ... mint = 3
+ ...
+ >>> for shake in Shake:
+ ... print(shake)
+ ...
+ Shake.vanilla
+ Shake.chocolate
+ Shake.cookies
+ Shake.mint
+
+Enumeration members are hashable, so they can be used in dictionaries and sets::
+
+ >>> apples = {}
+ >>> apples[Color.red] = 'red delicious'
+ >>> apples[Color.green] = 'granny smith'
+ >>> apples == {Color.red: 'red delicious', Color.green: 'granny smith'}
+ True
+
+
+Programmatic access to enumeration members and their attributes
+---------------------------------------------------------------
+
+Sometimes it's useful to access members in enumerations programmatically (i.e.
+situations where ``Color.red`` won't do because the exact color is not known
+at program-writing time). ``Enum`` allows such access::
+
+ >>> Color(1)
+ <Color.red: 1>
+ >>> Color(3)
+ <Color.blue: 3>
+
+If you want to access enum members by *name*, use item access::
+
+ >>> Color['red']
+ <Color.red: 1>
+ >>> Color['green']
+ <Color.green: 2>
+
+If have an enum member and need its :attr:`name` or :attr:`value`::
+
+ >>> member = Color.red
+ >>> member.name
+ 'red'
+ >>> member.value
+ 1
+
+
+Duplicating enum members and values
+-----------------------------------
+
+Having two enum members with the same name is invalid::
+
+ >>> class Shape(Enum):
+ ... square = 2
+ ... square = 3
+ ...
+ Traceback (most recent call last):
+ ...
+ TypeError: Attempted to reuse key: 'square'
+
+However, two enum members are allowed to have the same value. Given two members
+A and B with the same value (and A defined first), B is an alias to A. By-value
+lookup of the value of A and B will return A. By-name lookup of B will also
+return A::
+
+ >>> class Shape(Enum):
+ ... square = 2
+ ... diamond = 1
+ ... circle = 3
+ ... alias_for_square = 2
+ ...
+ >>> Shape.square
+ <Shape.square: 2>
+ >>> Shape.alias_for_square
+ <Shape.square: 2>
+ >>> Shape(2)
+ <Shape.square: 2>
+
+.. note::
+
+ Attempting to create a member with the same name as an already
+ defined attribute (another member, a method, etc.) or attempting to create
+ an attribute with the same name as a member is not allowed.
+
+
+Ensuring unique enumeration values
+----------------------------------
+
+By default, enumerations allow multiple names as aliases for the same value.
+When this behavior isn't desired, the following decorator can be used to
+ensure each value is used only once in the enumeration:
+
+.. decorator:: unique
+
+A :keyword:`class` decorator specifically for enumerations. It searches an
+enumeration's :attr:`__members__` gathering any aliases it finds; if any are
+found :exc:`ValueError` is raised with the details::
+
+ >>> from enum import Enum, unique
+ >>> @unique
+ ... class Mistake(Enum):
+ ... one = 1
+ ... two = 2
+ ... three = 3
+ ... four = 3
+ ...
+ Traceback (most recent call last):
+ ...
+ ValueError: duplicate values found in <enum 'Mistake'>: four -> three
+
+
+Iteration
+---------
+
+Iterating over the members of an enum does not provide the aliases::
+
+ >>> list(Shape)
+ [<Shape.square: 2>, <Shape.diamond: 1>, <Shape.circle: 3>]
+
+The special attribute ``__members__`` is an ordered dictionary mapping names
+to members. It includes all names defined in the enumeration, including the
+aliases::
+
+ >>> for name, member in Shape.__members__.items():
+ ... name, member
+ ...
+ ('square', <Shape.square: 2>)
+ ('diamond', <Shape.diamond: 1>)
+ ('circle', <Shape.circle: 3>)
+ ('alias_for_square', <Shape.square: 2>)
+
+The ``__members__`` attribute can be used for detailed programmatic access to
+the enumeration members. For example, finding all the aliases::
+
+ >>> [name for name, member in Shape.__members__.items() if member.name != name]
+ ['alias_for_square']
+
+
+Comparisons
+-----------
+
+Enumeration members are compared by identity::
+
+ >>> Color.red is Color.red
+ True
+ >>> Color.red is Color.blue
+ False
+ >>> Color.red is not Color.blue
+ True
+
+Ordered comparisons between enumeration values are *not* supported. Enum
+members are not integers (but see `IntEnum`_ below)::
+
+ >>> Color.red < Color.blue
+ Traceback (most recent call last):
+ File "<stdin>", line 1, in <module>
+ TypeError: unorderable types: Color() < Color()
+
+Equality comparisons are defined though::
+
+ >>> Color.blue == Color.red
+ False
+ >>> Color.blue != Color.red
+ True
+ >>> Color.blue == Color.blue
+ True
+
+Comparisons against non-enumeration values will always compare not equal
+(again, :class:`IntEnum` was explicitly designed to behave differently, see
+below)::
+
+ >>> Color.blue == 2
+ False
+
+
+Allowed members and attributes of enumerations
+----------------------------------------------
+
+The examples above use integers for enumeration values. Using integers is
+short and handy (and provided by default by the `Functional API`_), but not
+strictly enforced. In the vast majority of use-cases, one doesn't care what
+the actual value of an enumeration is. But if the value *is* important,
+enumerations can have arbitrary values.
+
+Enumerations are Python classes, and can have methods and special methods as
+usual. If we have this enumeration::
+
+ >>> class Mood(Enum):
+ ... funky = 1
+ ... happy = 3
+ ...
+ ... def describe(self):
+ ... # self is the member here
+ ... return self.name, self.value
+ ...
+ ... def __str__(self):
+ ... return 'my custom str! {0}'.format(self.value)
+ ...
+ ... @classmethod
+ ... def favorite_mood(cls):
+ ... # cls here is the enumeration
+ ... return cls.happy
+ ...
+
+Then::
+
+ >>> Mood.favorite_mood()
+ <Mood.happy: 3>
+ >>> Mood.happy.describe()
+ ('happy', 3)
+ >>> str(Mood.funky)
+ 'my custom str! 1'
+
+The rules for what is allowed are as follows: _sunder_ names (starting and
+ending with a single underscore) are reserved by enum and cannot be used;
+all other attributes defined within an enumeration will become members of this
+enumeration, with the exception of *__dunder__* names and descriptors (methods
+are also descriptors).
+
+Note: if your enumeration defines :meth:`__new__` and/or :meth:`__init__` then
+whatever value(s) were given to the enum member will be passed into those
+methods. See `Planet`_ for an example.
+
+
+Restricted subclassing of enumerations
+--------------------------------------
+
+Subclassing an enumeration is allowed only if the enumeration does not define
+any members. So this is forbidden::
+
+ >>> class MoreColor(Color):
+ ... pink = 17
+ ...
+ Traceback (most recent call last):
+ ...
+ TypeError: Cannot extend enumerations
+
+But this is allowed::
+
+ >>> class Foo(Enum):
+ ... def some_behavior(self):
+ ... pass
+ ...
+ >>> class Bar(Foo):
+ ... happy = 1
+ ... sad = 2
+ ...
+
+Allowing subclassing of enums that define members would lead to a violation of
+some important invariants of types and instances. On the other hand, it makes
+sense to allow sharing some common behavior between a group of enumerations.
+(See `OrderedEnum`_ for an example.)
+
+
+Pickling
+--------
+
+Enumerations can be pickled and unpickled::
+
+ >>> from test.test_enum import Fruit
+ >>> from pickle import dumps, loads
+ >>> Fruit.tomato is loads(dumps(Fruit.tomato))
+ True
+
+The usual restrictions for pickling apply: picklable enums must be defined in
+the top level of a module, since unpickling requires them to be importable
+from that module.
+
+.. warning::
+
+ In order to support the singleton nature of enumeration members, pickle
+ protocol version 2 or higher must be used.
+
+
+Functional API
+--------------
+
+The :class:`Enum` class is callable, providing the following functional API::
+
+ >>> Animal = Enum('Animal', 'ant bee cat dog')
+ >>> Animal
+ <enum 'Animal'>
+ >>> Animal.ant
+ <Animal.ant: 1>
+ >>> Animal.ant.value
+ 1
+ >>> list(Animal)
+ [<Animal.ant: 1>, <Animal.bee: 2>, <Animal.cat: 3>, <Animal.dog: 4>]
+
+The semantics of this API resemble :class:`~collections.namedtuple`. The first
+argument of the call to :class:`Enum` is the name of the enumeration.
+
+The second argument is the *source* of enumeration member names. It can be a
+whitespace-separated string of names, a sequence of names, a sequence of
+2-tuples with key/value pairs, or a mapping (e.g. dictionary) of names to
+values. The last two options enable assigning arbitrary values to
+enumerations; the others auto-assign increasing integers starting with 1. A
+new class derived from :class:`Enum` is returned. In other words, the above
+assignment to :class:`Animal` is equivalent to::
+
+ >>> class Animals(Enum):
+ ... ant = 1
+ ... bee = 2
+ ... cat = 3
+ ... dog = 4
+ ...
+
+The reason for defaulting to ``1`` as the starting number and not ``0`` is
+that ``0`` is ``False`` in a boolean sense, but enum members all evaluate
+to ``True``.
+
+Pickling enums created with the functional API can be tricky as frame stack
+implementation details are used to try and figure out which module the
+enumeration is being created in (e.g. it will fail if you use a utility
+function in separate module, and also may not work on IronPython or Jython).
+The solution is to specify the module name explicitly as follows::
+
+ >>> Animals = Enum('Animals', 'ant bee cat dog', module=__name__)
+
+Derived Enumerations
+--------------------
+
+IntEnum
+^^^^^^^
+
+A variation of :class:`Enum` is provided which is also a subclass of
+:class:`int`. Members of an :class:`IntEnum` can be compared to integers;
+by extension, integer enumerations of different types can also be compared
+to each other::
+
+ >>> from enum import IntEnum
+ >>> class Shape(IntEnum):
+ ... circle = 1
+ ... square = 2
+ ...
+ >>> class Request(IntEnum):
+ ... post = 1
+ ... get = 2
+ ...
+ >>> Shape == 1
+ False
+ >>> Shape.circle == 1
+ True
+ >>> Shape.circle == Request.post
+ True
+
+However, they still can't be compared to standard :class:`Enum` enumerations::
+
+ >>> class Shape(IntEnum):
+ ... circle = 1
+ ... square = 2
+ ...
+ >>> class Color(Enum):
+ ... red = 1
+ ... green = 2
+ ...
+ >>> Shape.circle == Color.red
+ False
+
+:class:`IntEnum` values behave like integers in other ways you'd expect::
+
+ >>> int(Shape.circle)
+ 1
+ >>> ['a', 'b', 'c'][Shape.circle]
+ 'b'
+ >>> [i for i in range(Shape.square)]
+ [0, 1]
+
+For the vast majority of code, :class:`Enum` is strongly recommended,
+since :class:`IntEnum` breaks some semantic promises of an enumeration (by
+being comparable to integers, and thus by transitivity to other
+unrelated enumerations). It should be used only in special cases where
+there's no other choice; for example, when integer constants are
+replaced with enumerations and backwards compatibility is required with code
+that still expects integers.
+
+
+Others
+^^^^^^
+
+While :class:`IntEnum` is part of the :mod:`enum` module, it would be very
+simple to implement independently::
+
+ class IntEnum(int, Enum):
+ pass
+
+This demonstrates how similar derived enumerations can be defined; for example
+a :class:`StrEnum` that mixes in :class:`str` instead of :class:`int`.
+
+Some rules:
+
+1. When subclassing :class:`Enum`, mix-in types must appear before
+ :class:`Enum` itself in the sequence of bases, as in the :class:`IntEnum`
+ example above.
+2. While :class:`Enum` can have members of any type, once you mix in an
+ additional type, all the members must have values of that type, e.g.
+ :class:`int` above. This restriction does not apply to mix-ins which only
+ add methods and don't specify another data type such as :class:`int` or
+ :class:`str`.
+3. When another data type is mixed in, the :attr:`value` attribute is *not the
+ same* as the enum member itself, although it is equivalant and will compare
+ equal.
+4. %-style formatting: `%s` and `%r` call :class:`Enum`'s :meth:`__str__` and
+ :meth:`__repr__` respectively; other codes (such as `%i` or `%h` for
+ IntEnum) treat the enum member as its mixed-in type.
+5. :meth:`str.__format__` (or :func:`format`) will use the mixed-in
+ type's :meth:`__format__`. If the :class:`Enum`'s :func:`str` or
+ :func:`repr` is desired use the `!s` or `!r` :class:`str` format codes.
+
+
+Interesting examples
+--------------------
+
+While :class:`Enum` and :class:`IntEnum` are expected to cover the majority of
+use-cases, they cannot cover them all. Here are recipes for some different
+types of enumerations that can be used directly, or as examples for creating
+one's own.
+
+
+AutoNumber
+^^^^^^^^^^
+
+Avoids having to specify the value for each enumeration member::
+
+ >>> class AutoNumber(Enum):
+ ... def __new__(cls):
+ ... value = len(cls.__members__) + 1
+ ... obj = object.__new__(cls)
+ ... obj._value_ = value
+ ... return obj
+ ...
+ >>> class Color(AutoNumber):
+ ... red = ()
+ ... green = ()
+ ... blue = ()
+ ...
+ >>> Color.green.value == 2
+ True
+
+.. note::
+
+ The :meth:`__new__` method, if defined, is used during creation of the Enum
+ members; it is then replaced by Enum's :meth:`__new__` which is used after
+ class creation for lookup of existing members. Due to the way Enums are
+ supposed to behave, there is no way to customize Enum's :meth:`__new__`.
+
+
+OrderedEnum
+^^^^^^^^^^^
+
+An ordered enumeration that is not based on :class:`IntEnum` and so maintains
+the normal :class:`Enum` invariants (such as not being comparable to other
+enumerations)::
+
+ >>> class OrderedEnum(Enum):
+ ... def __ge__(self, other):
+ ... if self.__class__ is other.__class__:
+ ... return self.value >= other.value
+ ... return NotImplemented
+ ... def __gt__(self, other):
+ ... if self.__class__ is other.__class__:
+ ... return self.value > other.value
+ ... return NotImplemented
+ ... def __le__(self, other):
+ ... if self.__class__ is other.__class__:
+ ... return self.value <= other.value
+ ... return NotImplemented
+ ... def __lt__(self, other):
+ ... if self.__class__ is other.__class__:
+ ... return self.value < other.value
+ ... return NotImplemented
+ ...
+ >>> class Grade(OrderedEnum):
+ ... A = 5
+ ... B = 4
+ ... C = 3
+ ... D = 2
+ ... F = 1
+ ...
+ >>> Grade.C < Grade.A
+ True
+
+
+DuplicateFreeEnum
+^^^^^^^^^^^^^^^^^
+
+Raises an error if a duplicate member name is found instead of creating an
+alias::
+
+ >>> class DuplicateFreeEnum(Enum):
+ ... def __init__(self, *args):
+ ... cls = self.__class__
+ ... if any(self.value == e.value for e in cls):
+ ... a = self.name
+ ... e = cls(self.value).name
+ ... raise ValueError(
+ ... "aliases not allowed in DuplicateFreeEnum: %r --> %r"
+ ... % (a, e))
+ ...
+ >>> class Color(DuplicateFreeEnum):
+ ... red = 1
+ ... green = 2
+ ... blue = 3
+ ... grene = 2
+ ...
+ Traceback (most recent call last):
+ ...
+ ValueError: aliases not allowed in DuplicateFreeEnum: 'grene' --> 'green'
+
+.. note::
+
+ This is a useful example for subclassing Enum to add or change other
+ behaviors as well as disallowing aliases. If the only desired change is
+ disallowing aliases, the :func:`unique` decorator can be used instead.
+
+
+Planet
+^^^^^^
+
+If :meth:`__new__` or :meth:`__init__` is defined the value of the enum member
+will be passed to those methods::
+
+ >>> class Planet(Enum):
+ ... MERCURY = (3.303e+23, 2.4397e6)
+ ... VENUS = (4.869e+24, 6.0518e6)
+ ... EARTH = (5.976e+24, 6.37814e6)
+ ... MARS = (6.421e+23, 3.3972e6)
+ ... JUPITER = (1.9e+27, 7.1492e7)
+ ... SATURN = (5.688e+26, 6.0268e7)
+ ... URANUS = (8.686e+25, 2.5559e7)
+ ... NEPTUNE = (1.024e+26, 2.4746e7)
+ ... def __init__(self, mass, radius):
+ ... self.mass = mass # in kilograms
+ ... self.radius = radius # in meters
+ ... @property
+ ... def surface_gravity(self):
+ ... # universal gravitational constant (m3 kg-1 s-2)
+ ... G = 6.67300E-11
+ ... return G * self.mass / (self.radius * self.radius)
+ ...
+ >>> Planet.EARTH.value
+ (5.976e+24, 6378140.0)
+ >>> Planet.EARTH.surface_gravity
+ 9.802652743337129
+
+
+How are Enums different?
+------------------------
+
+Enums have a custom metaclass that affects many aspects of both derived Enum
+classes and their instances (members).
+
+
+Enum Classes
+^^^^^^^^^^^^
+
+The :class:`EnumMeta` metaclass is responsible for providing the
+:meth:`__contains__`, :meth:`__dir__`, :meth:`__iter__` and other methods that
+allow one to do things with an :class:`Enum` class that fail on a typical
+class, such as `list(Color)` or `some_var in Color`. :class:`EnumMeta` is
+responsible for ensuring that various other methods on the final :class:`Enum`
+class are correct (such as :meth:`__new__`, :meth:`__getnewargs__`,
+:meth:`__str__` and :meth:`__repr__`)
+
+
+Enum Members (aka instances)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+The most interesting thing about Enum members is that they are singletons.
+:class:`EnumMeta` creates them all while it is creating the :class:`Enum`
+class itself, and then puts a custom :meth:`__new__` in place to ensure
+that no new ones are ever instantiated by returning only the existing
+member instances.
+
+
+Finer Points
+^^^^^^^^^^^^
+
+Enum members are instances of an Enum class, and even though they are
+accessible as `EnumClass.member`, they are not accessible directly from
+the member::
+
+ >>> Color.red
+ <Color.red: 1>
+ >>> Color.red.blue
+ Traceback (most recent call last):
+ ...
+ AttributeError: 'Color' object has no attribute 'blue'
+
+Likewise, the :attr:`__members__` is only available on the class.
+
+If you give your :class:`Enum` subclass extra methods, like the `Planet`_
+class above, those methods will show up in a :func:`dir` of the member,
+but not of the class::
+
+ >>> dir(Planet)
+ ['EARTH', 'JUPITER', 'MARS', 'MERCURY', 'NEPTUNE', 'SATURN', 'URANUS', 'VENUS', '__class__', '__doc__', '__members__', '__module__']
+ >>> dir(Planet.EARTH)
+ ['__class__', '__doc__', '__module__', 'name', 'surface_gravity', 'value']
+
+A :meth:`__new__` method will only be used for the creation of the
+:class:`Enum` members -- after that it is replaced. This means if you wish to
+change how :class:`Enum` members are looked up you either have to write a
+helper function or a :func:`classmethod`.
diff --git a/Doc/library/exceptions.rst b/Doc/library/exceptions.rst
index 1854318..bcaed24 100644
--- a/Doc/library/exceptions.rst
+++ b/Doc/library/exceptions.rst
@@ -259,6 +259,12 @@ The following exceptions are the exceptions that are usually raised.
:exc:`VMSError`, :exc:`socket.error`, :exc:`select.error` and
:exc:`mmap.error` have been merged into :exc:`OSError`.
+ .. versionchanged:: 3.4
+
+ The :attr:`filename` attribute is now the original file name passed to
+ the function, instead of the name encoded to or decoded from the
+ filesystem encoding.
+
.. exception:: OverflowError
diff --git a/Doc/library/faulthandler.rst b/Doc/library/faulthandler.rst
index 3c33621..61bc503 100644
--- a/Doc/library/faulthandler.rst
+++ b/Doc/library/faulthandler.rst
@@ -27,6 +27,7 @@ tracebacks:
* Only the filename, the function name and the line number are
displayed. (no source code)
* It is limited to 100 frames and 100 threads.
+* The order is reversed: the most recent call is shown first.
By default, the Python traceback is written to :data:`sys.stderr`. To see
tracebacks, applications must be run in the terminal. A log file can
@@ -129,7 +130,7 @@ Example of a segmentation fault on Linux: ::
>>> ctypes.string_at(0)
Fatal Python error: Segmentation fault
- Current thread 0x00007fb899f39700:
+ Current thread 0x00007fb899f39700 (most recent call first):
File "/home/python/cpython/Lib/ctypes/__init__.py", line 486 in string_at
File "<stdin>", line 1 in <module>
Segmentation fault
diff --git a/Doc/library/filecmp.rst b/Doc/library/filecmp.rst
index 8a88f8c..8471a72 100644
--- a/Doc/library/filecmp.rst
+++ b/Doc/library/filecmp.rst
@@ -27,6 +27,10 @@ The :mod:`filecmp` module defines the following functions:
Note that no external programs are called from this function, giving it
portability and efficiency.
+ This function uses a cache for past comparisons and the results,
+ with a cache invalidation mechanism relying on stale signatures
+ or by explicitly calling :func:`clear_cache`.
+
.. function:: cmpfiles(dir1, dir2, common, shallow=True)
@@ -48,6 +52,15 @@ The :mod:`filecmp` module defines the following functions:
one of the three returned lists.
+.. function:: clear_cache()
+
+ .. versionadded:: 3.4
+
+ Clear the filecmp cache. This may be useful if a file is compared so quickly
+ after it is modified that it is within the mtime resolution of
+ the underlying filesystem.
+
+
.. _dircmp-objects:
The :class:`dircmp` class
@@ -55,28 +68,25 @@ The :class:`dircmp` class
.. class:: dircmp(a, b, ignore=None, hide=None)
- Construct a new directory comparison object, to compare the directories *a* and
- *b*. *ignore* is a list of names to ignore, and defaults to ``['RCS', 'CVS',
- 'tags']``. *hide* is a list of names to hide, and defaults to ``[os.curdir,
- os.pardir]``.
+ Construct a new directory comparison object, to compare the directories *a*
+ and *b*. *ignore* is a list of names to ignore, and defaults to
+ :attr:`filecmp.DEFAULT_IGNORES`. *hide* is a list of names to hide, and
+ defaults to ``[os.curdir, os.pardir]``.
The :class:`dircmp` class compares files by doing *shallow* comparisons
as described for :func:`filecmp.cmp`.
The :class:`dircmp` class provides the following methods:
-
.. method:: report()
Print (to :data:`sys.stdout`) a comparison between *a* and *b*.
-
.. method:: report_partial_closure()
Print a comparison between *a* and *b* and common immediate
subdirectories.
-
.. method:: report_full_closure()
Print a comparison between *a* and *b* and common subdirectories
@@ -133,7 +143,7 @@ The :class:`dircmp` class
.. attribute:: common_files
- Files in both *a* and *b*
+ Files in both *a* and *b*.
.. attribute:: common_funny
@@ -164,6 +174,12 @@ The :class:`dircmp` class
A dictionary mapping names in :attr:`common_dirs` to :class:`dircmp`
objects.
+.. attribute:: DEFAULT_IGNORES
+
+ .. versionadded:: 3.4
+
+ List of directories ignored by :class:`dircmp` by default.
+
Here is a simplified example of using the ``subdirs`` attribute to search
recursively through two directories to show common different files::
diff --git a/Doc/library/filesys.rst b/Doc/library/filesys.rst
index 58998a8..03e085b 100644
--- a/Doc/library/filesys.rst
+++ b/Doc/library/filesys.rst
@@ -12,6 +12,7 @@ in this chapter is:
.. toctree::
+ pathlib.rst
os.path.rst
fileinput.rst
stat.rst
diff --git a/Doc/library/formatter.rst b/Doc/library/formatter.rst
index 88be11c..bdc9e7b 100644
--- a/Doc/library/formatter.rst
+++ b/Doc/library/formatter.rst
@@ -4,6 +4,10 @@
.. module:: formatter
:synopsis: Generic output formatter and device interface.
+.. deprecated:: 3.4
+ Due to lack of usage, the formatter module has been deprecated and is slated
+ for removal in Python 3.6.
+
This module supports two interface definitions, each with multiple
implementations: The *formatter* interface, and the *writer* interface which is
diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst
index 4ebf3ca..ffee09e 100644
--- a/Doc/library/functions.rst
+++ b/Doc/library/functions.rst
@@ -543,6 +543,10 @@ are always available. They are listed here in alphabetical order.
:exc:`TypeError` exception is raised if the method is not found or if either
the *format_spec* or the return value are not strings.
+ .. versionadded:: 3.4
+ ``object().__format__(format_spec)`` raises :exc:`TypeError`
+ if *format_spec* is not empty string.
+
.. _func-frozenset:
.. function:: frozenset([iterable])
@@ -666,6 +670,12 @@ are always available. They are listed here in alphabetical order.
The integer type is described in :ref:`typesnumeric`.
+ .. versionchanged:: 3.4
+ If *base* is not an instance of :class:`int` and the *base* object has a
+ :meth:`base.__index__ <object.__index__>` method, that method is called
+ to obtain an integer for the base. Previous versions used
+ :meth:`base.__int__ <object.__int__>` instead of :meth:`base.__index__
+ <object.__index__>`.
.. function:: isinstance(object, classinfo)
@@ -748,19 +758,22 @@ are always available. They are listed here in alphabetical order.
already arranged into argument tuples, see :func:`itertools.starmap`\.
-.. function:: max(iterable, *[, key])
+.. function:: max(iterable, *[, default, key])
max(arg1, arg2, *args[, key])
Return the largest item in an iterable or the largest of two or more
arguments.
- If one positional argument is provided, *iterable* must be a non-empty
- iterable (such as a non-empty string, tuple or list). The largest item
- in the iterable is returned. If two or more positional arguments are
- provided, the largest of the positional arguments is returned.
+ If one positional argument is provided, it should be an :term:`iterable`.
+ The largest item in the iterable is returned. If two or more positional
+ arguments are provided, the smallest of the positional arguments is
+ returned.
- The optional keyword-only *key* argument specifies a one-argument ordering
- function like that used for :meth:`list.sort`.
+ There are two optional keyword-only arguments. The *key* argument specifies
+ a one-argument ordering function like that used for :meth:`list.sort`. The
+ *default* argument specifies an object to return if the provided iterable is
+ empty. If the iterable is empty and *default* is not provided, a
+ :exc:`ValueError` is raised.
If multiple items are maximal, the function returns the first one
encountered. This is consistent with other sort-stability preserving tools
@@ -776,19 +789,22 @@ are always available. They are listed here in alphabetical order.
:ref:`typememoryview` for more information.
-.. function:: min(iterable, *[, key])
+.. function:: min(iterable, *[, default, key])
min(arg1, arg2, *args[, key])
Return the smallest item in an iterable or the smallest of two or more
arguments.
- If one positional argument is provided, *iterable* must be a non-empty
- iterable (such as a non-empty string, tuple or list). The smallest item
- in the iterable is returned. If two or more positional arguments are
- provided, the smallest of the positional arguments is returned.
+ If one positional argument is provided, it should be an :term:`iterable`.
+ The smallest item in the iterable is returned. If two or more positional
+ arguments are provided, the smallest of the positional arguments is
+ returned.
- The optional keyword-only *key* argument specifies a one-argument ordering
- function like that used for :meth:`list.sort`.
+ There are two optional keyword-only arguments. The *key* argument specifies
+ a one-argument ordering function like that used for :meth:`list.sort`. The
+ *default* argument specifies an object to return if the provided iterable is
+ empty. If the iterable is empty and *default* is not provided, a
+ :exc:`ValueError` is raised.
If multiple items are minimal, the function returns the first one
encountered. This is consistent with other sort-stability preserving tools
@@ -963,6 +979,8 @@ are always available. They are listed here in alphabetical order.
:mod:`os.open` as *opener* results in functionality similar to passing
``None``).
+ The newly created file is :ref:`non-inheritable <fd_inheritance>`.
+
The following example uses the :ref:`dir_fd <dir_fd>` parameter of the
:func:`os.open` function to open a file relative to a given directory::
@@ -976,10 +994,6 @@ are always available. They are listed here in alphabetical order.
...
>>> os.close(dir_fd) # don't leak a file descriptor
- .. versionchanged:: 3.3
- The *opener* parameter was added.
- The ``'x'`` mode was added.
-
The type of :term:`file object` returned by the :func:`open` function
depends on the mode. When :func:`open` is used to open a file in a text
mode (``'w'``, ``'r'``, ``'wt'``, ``'rt'``, etc.), it returns a subclass of
@@ -1006,10 +1020,15 @@ are always available. They are listed here in alphabetical order.
and :mod:`shutil`.
.. versionchanged:: 3.3
+ The *opener* parameter was added.
+ The ``'x'`` mode was added.
:exc:`IOError` used to be raised, it is now an alias of :exc:`OSError`.
:exc:`FileExistsError` is now raised if the file opened in exclusive
creation mode (``'x'``) already exists.
+ .. versionchanged:: 3.4
+ The file is now non-inheritable.
+
.. XXX works for bytes too, but should it?
.. function:: ord(c)
diff --git a/Doc/library/functools.rst b/Doc/library/functools.rst
index 3c946e3..77cd838 100644
--- a/Doc/library/functools.rst
+++ b/Doc/library/functools.rst
@@ -6,6 +6,7 @@
.. moduleauthor:: Peter Harris <scav@blueyonder.co.uk>
.. moduleauthor:: Raymond Hettinger <python@rcn.com>
.. moduleauthor:: Nick Coghlan <ncoghlan@gmail.com>
+.. moduleauthor:: Łukasz Langa <lukasz@langa.pl>
.. sectionauthor:: Peter Harris <scav@blueyonder.co.uk>
**Source code:** :source:`Lib/functools.py`
@@ -133,15 +134,34 @@ The :mod:`functools` module defines the following functions:
@total_ordering
class Student:
+ def _is_valid_operand(self, other):
+ return (hasattr(other, "lastname") and
+ hasattr(other, "firstname"))
def __eq__(self, other):
+ if not self._is_valid_operand(other):
+ return NotImplemented
return ((self.lastname.lower(), self.firstname.lower()) ==
(other.lastname.lower(), other.firstname.lower()))
def __lt__(self, other):
+ if not self._is_valid_operand(other):
+ return NotImplemented
return ((self.lastname.lower(), self.firstname.lower()) <
(other.lastname.lower(), other.firstname.lower()))
+ .. note::
+
+ While this decorator makes it easy to create well behaved totally
+ ordered types, it *does* come at the cost of slower execution and
+ more complex stack traces for the derived comparison methods. If
+ performance benchmarking indicates this is a bottleneck for a given
+ application, implementing all six rich comparison methods instead is
+ likely to provide an easy speed boost.
+
.. versionadded:: 3.2
+ .. versionchanged:: 3.4
+ Returning NotImplemented from the underlying comparison function for
+ unrecognised types is now supported.
.. function:: partial(func, *args, **keywords)
@@ -174,6 +194,48 @@ The :mod:`functools` module defines the following functions:
18
+.. class:: partialmethod(func, *args, **keywords)
+
+ Return a new :class:`partialmethod` descriptor which behaves
+ like :class:`partial` except that it is designed to be used as a method
+ definition rather than being directly callable.
+
+ *func* must be a :term:`descriptor` or a callable (objects which are both,
+ like normal functions, are handled as descriptors).
+
+ When *func* is a descriptor (such as a normal Python function,
+ :func:`classmethod`, :func:`staticmethod`, :func:`abstractmethod` or
+ another instance of :class:`partialmethod`), calls to ``__get__`` are
+ delegated to the underlying descriptor, and an appropriate
+ :class:`partial` object returned as the result.
+
+ When *func* is a non-descriptor callable, an appropriate bound method is
+ created dynamically. This behaves like a normal Python function when
+ used as a method: the *self* argument will be inserted as the first
+ positional argument, even before the *args* and *keywords* supplied to
+ the :class:`partialmethod` constructor.
+
+ Example::
+
+ >>> class Cell(object):
+ ... @property
+ ... def alive(self):
+ ... return self._alive
+ ... def set_state(self, state):
+ ... self._alive = bool(state)
+ ... set_alive = partialmethod(set_state, True)
+ ... set_dead = partialmethod(set_state, False)
+ ...
+ >>> c = Cell()
+ >>> c.alive
+ False
+ >>> c.set_alive()
+ >>> c.alive
+ True
+
+ .. versionadded:: 3.4
+
+
.. function:: reduce(function, iterable[, initializer])
Apply *function* of two arguments cumulatively to the items of *sequence*, from
@@ -198,6 +260,111 @@ The :mod:`functools` module defines the following functions:
return value
+.. decorator:: singledispatch(default)
+
+ Transforms a function into a :term:`single-dispatch <single
+ dispatch>` :term:`generic function`.
+
+ To define a generic function, decorate it with the ``@singledispatch``
+ decorator. Note that the dispatch happens on the type of the first argument,
+ create your function accordingly::
+
+ >>> from functools import singledispatch
+ >>> @singledispatch
+ ... def fun(arg, verbose=False):
+ ... if verbose:
+ ... print("Let me just say,", end=" ")
+ ... print(arg)
+
+ To add overloaded implementations to the function, use the :func:`register`
+ attribute of the generic function. It is a decorator, taking a type
+ parameter and decorating a function implementing the operation for that
+ type::
+
+ >>> @fun.register(int)
+ ... def _(arg, verbose=False):
+ ... if verbose:
+ ... print("Strength in numbers, eh?", end=" ")
+ ... print(arg)
+ ...
+ >>> @fun.register(list)
+ ... def _(arg, verbose=False):
+ ... if verbose:
+ ... print("Enumerate this:")
+ ... for i, elem in enumerate(arg):
+ ... print(i, elem)
+
+ To enable registering lambdas and pre-existing functions, the
+ :func:`register` attribute can be used in a functional form::
+
+ >>> def nothing(arg, verbose=False):
+ ... print("Nothing.")
+ ...
+ >>> fun.register(type(None), nothing)
+
+ The :func:`register` attribute returns the undecorated function which
+ enables decorator stacking, pickling, as well as creating unit tests for
+ each variant independently::
+
+ >>> @fun.register(float)
+ ... @fun.register(Decimal)
+ ... def fun_num(arg, verbose=False):
+ ... if verbose:
+ ... print("Half of your number:", end=" ")
+ ... print(arg / 2)
+ ...
+ >>> fun_num is fun
+ False
+
+ When called, the generic function dispatches on the type of the first
+ argument::
+
+ >>> fun("Hello, world.")
+ Hello, world.
+ >>> fun("test.", verbose=True)
+ Let me just say, test.
+ >>> fun(42, verbose=True)
+ Strength in numbers, eh? 42
+ >>> fun(['spam', 'spam', 'eggs', 'spam'], verbose=True)
+ Enumerate this:
+ 0 spam
+ 1 spam
+ 2 eggs
+ 3 spam
+ >>> fun(None)
+ Nothing.
+ >>> fun(1.23)
+ 0.615
+
+ Where there is no registered implementation for a specific type, its
+ method resolution order is used to find a more generic implementation.
+ The original function decorated with ``@singledispatch`` is registered
+ for the base ``object`` type, which means it is used if no better
+ implementation is found.
+
+ To check which implementation will the generic function choose for
+ a given type, use the ``dispatch()`` attribute::
+
+ >>> fun.dispatch(float)
+ <function fun_num at 0x1035a2840>
+ >>> fun.dispatch(dict) # note: default implementation
+ <function fun at 0x103fe0000>
+
+ To access all registered implementations, use the read-only ``registry``
+ attribute::
+
+ >>> fun.registry.keys()
+ dict_keys([<class 'NoneType'>, <class 'int'>, <class 'object'>,
+ <class 'decimal.Decimal'>, <class 'list'>,
+ <class 'float'>])
+ >>> fun.registry[float]
+ <function fun_num at 0x1035a2840>
+ >>> fun.registry[object]
+ <function fun at 0x103fe0000>
+
+ .. versionadded:: 3.4
+
+
.. function:: update_wrapper(wrapper, wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)
Update a *wrapper* function to look like the *wrapped* function. The optional
@@ -212,8 +379,8 @@ The :mod:`functools` module defines the following functions:
To allow access to the original function for introspection and other purposes
(e.g. bypassing a caching decorator such as :func:`lru_cache`), this function
- automatically adds a __wrapped__ attribute to the wrapper that refers to
- the original function.
+ automatically adds a ``__wrapped__`` attribute to the wrapper that refers to
+ the function being wrapped.
The main intended use for this function is in :term:`decorator` functions which
wrap the decorated function and return the wrapper. If the wrapper function is
@@ -236,6 +403,11 @@ The :mod:`functools` module defines the following functions:
.. versionchanged:: 3.2
Missing attributes no longer trigger an :exc:`AttributeError`.
+ .. versionchanged:: 3.4
+ The ``__wrapped__`` attribute now always refers to the wrapped
+ function, even if that function defined a ``__wrapped__`` attribute.
+ (see :issue:`17482`)
+
.. decorator:: wraps(wrapped, assigned=WRAPPER_ASSIGNMENTS, updated=WRAPPER_UPDATES)
@@ -301,4 +473,3 @@ differences. For instance, the :attr:`__name__` and :attr:`__doc__` attributes
are not created automatically. Also, :class:`partial` objects defined in
classes behave like static methods and do not transform into bound methods
during instance attribute look-up.
-
diff --git a/Doc/library/gc.rst b/Doc/library/gc.rst
index 5878da5..8e3a4bd 100644
--- a/Doc/library/gc.rst
+++ b/Doc/library/gc.rst
@@ -67,6 +67,24 @@ The :mod:`gc` module provides the following functions:
returned.
+.. function:: get_stats()
+
+ Return a list of 3 per-generation dictionaries containing collection
+ statistics since interpreter start. At this moment, each dictionary will
+ contain the following items:
+
+ * ``collections`` is the number of times this generation was collected;
+
+ * ``collected`` is the total number of objects collected inside this
+ generation;
+
+ * ``uncollectable`` is the total number of objects which were found
+ to be uncollectable (and were therefore moved to the :data:`garbage`
+ list) inside this generation.
+
+ .. versionadded:: 3.4
+
+
.. function:: set_threshold(threshold0[, threshold1[, threshold2]])
Set the garbage collection thresholds (the collection frequency). Setting
@@ -158,24 +176,13 @@ values but should not rebind them):
.. data:: garbage
- A list of objects which the collector found to be unreachable but could not be
- freed (uncollectable objects). By default, this list contains only objects with
- :meth:`__del__` methods. Objects that have :meth:`__del__` methods and are
- part of a reference cycle cause the entire reference cycle to be uncollectable,
- including objects not necessarily in the cycle but reachable only from it.
- Python doesn't collect such cycles automatically because, in general, it isn't
- possible for Python to guess a safe order in which to run the :meth:`__del__`
- methods. If you know a safe order, you can force the issue by examining the
- *garbage* list, and explicitly breaking cycles due to your objects within the
- list. Note that these objects are kept alive even so by virtue of being in the
- *garbage* list, so they should be removed from *garbage* too. For example,
- after breaking cycles, do ``del gc.garbage[:]`` to empty the list. It's
- generally better to avoid the issue by not creating cycles containing objects
- with :meth:`__del__` methods, and *garbage* can be examined in that case to
- verify that no such cycles are being created.
-
- If :const:`DEBUG_SAVEALL` is set, then all unreachable objects will be added
- to this list rather than freed.
+ A list of objects which the collector found to be unreachable but could
+ not be freed (uncollectable objects). Starting with Python 3.4, this
+ list should be empty most of the time, except when using instances of
+ C extension types with a non-NULL ``tp_del`` slot.
+
+ If :const:`DEBUG_SAVEALL` is set, then all unreachable objects will be
+ added to this list rather than freed.
.. versionchanged:: 3.2
If this list is non-empty at interpreter shutdown, a
@@ -183,6 +190,10 @@ values but should not rebind them):
:const:`DEBUG_UNCOLLECTABLE` is set, in addition all uncollectable objects
are printed.
+ .. versionchanged:: 3.4
+ Following :pep:`442`, objects with a :meth:`__del__` method don't end
+ up in :attr:`gc.garbage` anymore.
+
.. data:: callbacks
A list of callbacks that will be invoked by the garbage collector before and
diff --git a/Doc/library/glob.rst b/Doc/library/glob.rst
index eff138b..abcbf38 100644
--- a/Doc/library/glob.rst
+++ b/Doc/library/glob.rst
@@ -25,6 +25,10 @@ For a literal match, wrap the meta-characters in brackets.
For example, ``'[?]'`` matches the character ``'?'``.
+.. seealso::
+ The :mod:`pathlib` module offers high-level path objects.
+
+
.. function:: glob(pathname)
Return a possibly-empty list of path names that match *pathname*, which must be
@@ -40,6 +44,17 @@ For example, ``'[?]'`` matches the character ``'?'``.
without actually storing them all simultaneously.
+.. function:: escape(pathname)
+
+ Escape all special characters (``'?'``, ``'*'`` and ``'['``).
+ This is useful if you want to match an arbitrary literal string that may
+ have special characters in it. Special characters in drive/UNC
+ sharepoints are not escaped, e.g. on Windows
+ ``escape('//?/c:/Quo vadis?.txt')`` returns ``'//?/c:/Quo vadis[?].txt'``.
+
+ .. versionadded:: 3.4
+
+
For example, consider a directory containing only the following files:
:file:`1.gif`, :file:`2.txt`, and :file:`card.gif`. :func:`glob` will produce
the following results. Notice how any leading components of the path are
diff --git a/Doc/library/gzip.rst b/Doc/library/gzip.rst
index ec16f6a..de5063a 100644
--- a/Doc/library/gzip.rst
+++ b/Doc/library/gzip.rst
@@ -35,8 +35,8 @@ The module defines the following items:
:class:`bytes` object), or an existing file object to read from or write to.
The *mode* argument can be any of ``'r'``, ``'rb'``, ``'a'``, ``'ab'``,
- ``'w'``, or ``'wb'`` for binary mode, or ``'rt'``, ``'at'``, or ``'wt'`` for
- text mode. The default is ``'rb'``.
+ ``'w'``, ``'wb'``, ``'x'`` or ``'xb'`` for binary mode, or ``'rt'``,
+ ``'at'``, ``'wt'``, or ``'xt'`` for text mode. The default is ``'rb'``.
The *compresslevel* argument is an integer from 0 to 9, as for the
:class:`GzipFile` constructor.
@@ -53,6 +53,9 @@ The module defines the following items:
Added support for *filename* being a file object, support for text mode,
and the *encoding*, *errors* and *newline* arguments.
+ .. versionchanged:: 3.4
+ Added support for the ``'x'``, ``'xb'`` and ``'xt'`` modes.
+
.. class:: GzipFile(filename=None, mode=None, compresslevel=9, fileobj=None, mtime=None)
@@ -73,8 +76,9 @@ The module defines the following items:
original filename is not included in the header.
The *mode* argument can be any of ``'r'``, ``'rb'``, ``'a'``, ``'ab'``, ``'w'``,
- or ``'wb'``, depending on whether the file will be read or written. The default
- is the mode of *fileobj* if discernible; otherwise, the default is ``'rb'``.
+ ``'wb'``, ``'x'``, or ``'xb'``, depending on whether the file will be read or
+ written. The default is the mode of *fileobj* if discernible; otherwise, the
+ default is ``'rb'``.
Note that the file is always opened in binary mode. To open a compressed file
in text mode, use :func:`.open` (or wrap your :class:`GzipFile` with an
@@ -125,6 +129,9 @@ The module defines the following items:
.. versionchanged:: 3.3
The :meth:`io.BufferedIOBase.read1` method is now implemented.
+ .. versionchanged:: 3.4
+ Added support for the ``'x'`` and ``'xb'`` modes.
+
.. function:: compress(data, compresslevel=9)
diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst
index 49fedec..b1daba1 100644
--- a/Doc/library/hashlib.rst
+++ b/Doc/library/hashlib.rst
@@ -32,6 +32,10 @@ digests. The modern term is secure hash.
Some algorithms have known hash collision weaknesses, refer to the "See
also" section at the end.
+
+Hash algorithms
+---------------
+
There is one constructor method named for each type of :dfn:`hash`. All return
a hash object with the same simple interface. For example: use :func:`sha1` to
create a SHA1 hash object. You can now feed this object with :term:`bytes-like
@@ -53,9 +57,13 @@ concatenation of the data fed to it so far using the :meth:`digest` or
.. index:: single: OpenSSL; (use in module hashlib)
Constructors for hash algorithms that are always present in this module are
-:func:`md5`, :func:`sha1`, :func:`sha224`, :func:`sha256`, :func:`sha384`, and
-:func:`sha512`. Additional algorithms may also be available depending upon the
-OpenSSL library that Python uses on your platform.
+:func:`md5`, :func:`sha1`, :func:`sha224`, :func:`sha256`, :func:`sha384`,
+:func:`sha512`, :func:`sha3_224`, :func:`sha3_256`, :func:`sha3_384`, and
+:func:`sha3_512`. Additional algorithms may also be available depending upon
+the OpenSSL library that Python uses on your platform.
+
+ .. versionchanged:: 3.4
+ Add sha3 family of hash algorithms.
For example, to obtain the digest of the byte string ``b'Nobody inspects the
spammish repetition'``::
@@ -122,6 +130,18 @@ returned by the constructors:
The internal block size of the hash algorithm in bytes.
+A hash object has the following attributes:
+
+.. attribute:: hash.name
+
+ The canonical name of this hash, always lowercase and always suitable as a
+ parameter to :func:`new` to create another hash of this type.
+
+ .. versionchanged:: 3.4
+ The name attribute has been present in CPython since its inception, but
+ until Python 3.4 was not formally specified, so may not exist on some
+ platforms.
+
A hash object has the following methods:
@@ -158,6 +178,45 @@ A hash object has the following methods:
compute the digests of data sharing a common initial substring.
+Key Derivation Function
+-----------------------
+
+Key derivation and key stretching algorithms are designed for secure password
+hashing. Naive algorithms such as ``sha1(password)`` are not resistant
+against brute-force attacks. A good password hashing function must be tunable,
+slow and include a salt.
+
+
+.. function:: pbkdf2_hmac(name, password, salt, rounds, dklen=None)
+
+ The function provides PKCS#5 password-based key derivation function 2. It
+ uses HMAC as pseudorandom function.
+
+ The string *name* is the desired name of the hash digest algorithm for
+ HMAC, e.g. 'sha1' or 'sha256'. *password* and *salt* are interpreted as
+ buffers of bytes. Applications and libraries should limit *password* to
+ a sensible value (e.g. 1024). *salt* should be about 16 or more bytes from
+ a proper source, e.g. :func:`os.urandom`.
+
+ The number of *rounds* should be chosen based on the hash algorithm and
+ computing power. As of 2013 a value of at least 100,000 rounds of SHA-256
+ have been suggested.
+
+ *dklen* is the length of the derived key. If *dklen* is ``None`` then the
+ digest size of the hash algorithm *name* is used, e.g. 64 for SHA-512.
+
+ >>> import hashlib, binascii
+ >>> dk = hashlib.pbkdf2_hmac('sha256', b'password', b'salt', 100000)
+ >>> binascii.hexlify(dk)
+ b'0394a2ede332c9a13eb82e9b24631604c31df978b4e2f0fbd2c549944f9d79a5'
+
+ .. versionadded:: 3.4
+
+ .. note:: A fast implementation of *pbkdf2_hmac* is available with OpenSSL.
+ The Python implementation uses an inline version of :mod:`hmac`. It is
+ about three times slower and doesn't release the GIL.
+
+
.. seealso::
Module :mod:`hmac`
@@ -173,3 +232,5 @@ A hash object has the following methods:
Wikipedia article with information on which algorithms have known issues and
what that means regarding their use.
+ http://www.ietf.org/rfc/rfc2898.txt
+ PKCS #5: Password-Based Cryptography Specification Version 2.0
diff --git a/Doc/library/hmac.rst b/Doc/library/hmac.rst
index 9575693..2e9b0b2 100644
--- a/Doc/library/hmac.rst
+++ b/Doc/library/hmac.rst
@@ -16,20 +16,33 @@ This module implements the HMAC algorithm as described by :rfc:`2104`.
.. function:: new(key, msg=None, digestmod=None)
- Return a new hmac object. *key* is a bytes object giving the secret key. If
- *msg* is present, the method call ``update(msg)`` is made. *digestmod* is
- the digest constructor or module for the HMAC object to use. It defaults to
- the :data:`hashlib.md5` constructor.
+ Return a new hmac object. *key* is a bytes or bytearray object giving the
+ secret key. If *msg* is present, the method call ``update(msg)`` is made.
+ *digestmod* is the digest name, digest constructor or module for the HMAC
+ object to use. It supports any name suitable to :func:`hashlib.new` and
+ defaults to the :data:`hashlib.md5` constructor.
+
+ .. versionchanged:: 3.4
+ Parameter *key* can be a bytes or bytearray object. Parameter *msg* can
+ be of any type supported by :mod:`hashlib`.
+
+ Paramter *digestmod* can be the name of a hash algorithm.
+
+ .. deprecated:: 3.4
+ MD5 as implicit default digest for *digestmod* is deprecated.
An HMAC object has the following methods:
.. method:: HMAC.update(msg)
- Update the hmac object with the bytes object *msg*. Repeated calls are
- equivalent to a single call with the concatenation of all the arguments:
+ Update the hmac object with *msg*. Repeated calls are equivalent to a
+ single call with the concatenation of all the arguments:
``m.update(a); m.update(b)`` is equivalent to ``m.update(a + b)``.
+ .. versionchanged:: 3.4
+ Parameter *msg* can be of any type supported by :mod:`hashlib`.
+
.. method:: HMAC.digest()
@@ -66,6 +79,25 @@ An HMAC object has the following methods:
compute the digests of strings that share a common initial substring.
+A hash object has the following attributes:
+
+.. attribute:: HMAC.digest_size
+
+ The size of the resulting HMAC digest in bytes.
+
+.. attribute:: HMAC.block_size
+
+ The internal block size of the hash algorithm in bytes.
+
+ .. versionadded:: 3.4
+
+.. attribute:: HMAC.name
+
+ The canonical name of this HMAC, always lowercase, e.g. ``hmac-md5``.
+
+ .. versionadded:: 3.4
+
+
This module also provides the following helper function:
.. function:: compare_digest(a, b)
diff --git a/Doc/library/html.entities.rst b/Doc/library/html.entities.rst
index 65ce817..ba8324c 100644
--- a/Doc/library/html.entities.rst
+++ b/Doc/library/html.entities.rst
@@ -20,6 +20,7 @@ This module defines four dictionaries, :data:`html5`,
Note that the trailing semicolon is included in the name (e.g. ``'gt;'``),
however some of the names are accepted by the standard even without the
semicolon: in this case the name is present with and without the ``';'``.
+ See also :func:`html.unescape`.
.. versionadded:: 3.3
diff --git a/Doc/library/html.parser.rst b/Doc/library/html.parser.rst
index e4154ef..0ea9644 100644
--- a/Doc/library/html.parser.rst
+++ b/Doc/library/html.parser.rst
@@ -74,7 +74,7 @@ as they are encountered::
def handle_data(self, data):
print("Encountered some data :", data)
- parser = MyHTMLParser(strict=False)
+ parser = MyHTMLParser()
parser.feed('<html><head><title>Test</title></head>'
'<body><h1>Parse me!</h1></body></html>')
@@ -272,7 +272,7 @@ examples::
def handle_decl(self, data):
print("Decl :", data)
- parser = MyHTMLParser(strict=False)
+ parser = MyHTMLParser()
Parsing a doctype::
diff --git a/Doc/library/html.rst b/Doc/library/html.rst
index 1107ca9..d0706bc 100644
--- a/Doc/library/html.rst
+++ b/Doc/library/html.rst
@@ -20,6 +20,17 @@ This module defines utilities to manipulate HTML.
.. versionadded:: 3.2
+
+.. function:: unescape(s)
+
+ Convert all named and numeric character references (e.g. ``&gt;``,
+ ``&#62;``, ``&x3e;``) in the string *s* to the corresponding unicode
+ characters. This function uses the rules defined by the HTML 5 standard
+ for both valid and invalid character references, and the :data:`list of
+ HTML 5 named character references <html.entities.html5>`.
+
+ .. versionadded:: 3.4
+
--------------
Submodules in the ``html`` package are:
diff --git a/Doc/library/http.client.rst b/Doc/library/http.client.rst
index 4daeeea..892e62f 100644
--- a/Doc/library/http.client.rst
+++ b/Doc/library/http.client.rst
@@ -27,7 +27,7 @@ HTTPS protocols. It is normally not used directly --- the module
The module provides the following classes:
-.. class:: HTTPConnection(host, port=None[, strict][, timeout], \
+.. class:: HTTPConnection(host, port=None[, timeout], \
source_address=None)
An :class:`HTTPConnection` instance represents one transaction with an HTTP
@@ -51,13 +51,13 @@ The module provides the following classes:
.. versionchanged:: 3.2
*source_address* was added.
- .. deprecated-removed:: 3.2 3.4
- The *strict* parameter is deprecated. HTTP 0.9-style "Simple Responses"
- are not supported anymore.
+ .. versionchanged:: 3.4
+ The *strict* parameter is removed. HTTP 0.9-style "Simple Responses" are
+ not supported.
.. class:: HTTPSConnection(host, port=None, key_file=None, \
- cert_file=None[, strict][, timeout], \
+ cert_file=None[, timeout], \
source_address=None, *, context=None, \
check_hostname=None)
@@ -89,19 +89,19 @@ The module provides the following classes:
This class now supports HTTPS virtual hosts if possible (that is,
if :data:`ssl.HAS_SNI` is true).
- .. deprecated-removed:: 3.2 3.4
- The *strict* parameter is deprecated. HTTP 0.9-style "Simple Responses"
- are not supported anymore.
+ .. versionchanged:: 3.4
+ The *strict* parameter is removed. HTTP 0.9-style "Simple Responses" are
+ not supported anymore.
-.. class:: HTTPResponse(sock, debuglevel=0[, strict], method=None, url=None)
+.. class:: HTTPResponse(sock, debuglevel=0, method=None, url=None)
Class whose instances are returned upon successful connection. Not
instantiated directly by user.
- .. deprecated-removed:: 3.2 3.4
- The *strict* parameter is deprecated. HTTP 0.9-style "Simple Responses"
- are not supported anymore.
+ .. versionchanged:: 3.4
+ The *strict* parameter is removed. HTTP 0.9 style "Simple Responses" are
+ not supported anymore.
The following exceptions are raised as appropriate:
diff --git a/Doc/library/http.server.rst b/Doc/library/http.server.rst
index 5aeb719..113ac44 100644
--- a/Doc/library/http.server.rst
+++ b/Doc/library/http.server.rst
@@ -170,12 +170,19 @@ of which this module provides three different variants:
.. versionadded:: 3.2
- .. method:: send_error(code, message=None)
+ .. method:: send_error(code, message=None, explain=None)
Sends and logs a complete error reply to the client. The numeric *code*
- specifies the HTTP error code, with *message* as optional, more specific text. A
- complete set of headers is sent, followed by text composed using the
- :attr:`error_message_format` class variable.
+ specifies the HTTP error code, with *message* as optional, more specific
+ text, usually referring to short message response. The *explain*
+ argument can be used to send a detailed information about the error in
+ response content body. A complete set of headers is sent, followed by
+ text composed using the :attr:`error_message_format` class variable.
+
+ .. versionchanged:: 3.4
+ The error response includes a Content-Length header.
+ Added the *explain* argument.
+
.. method:: send_response(code, message=None)
@@ -361,6 +368,15 @@ the previous example, this serves files relative to the current directory. ::
python -m http.server 8000
+By default, server binds itself to all interfaces. To restrict it to bind to a
+particular interface only, ``--bind ADDRESS`` argument can be used. For e.g, to
+restrict the server to bind only to localhost. ::
+
+ python -m http.server 8000 --bind 127.0.0.1
+
+.. versionadded:: 3.4
+ ``--bind`` argument was introduced.
+
.. class:: CGIHTTPRequestHandler(request, client_address, server)
diff --git a/Doc/library/idle.rst b/Doc/library/idle.rst
index 36d78b0..2718cef 100644
--- a/Doc/library/idle.rst
+++ b/Doc/library/idle.rst
@@ -16,70 +16,82 @@ IDLE has the following features:
* coded in 100% pure Python, using the :mod:`tkinter` GUI toolkit
-* cross-platform: works on Windows and Unix
+* cross-platform: works on Windows, Unix, and Mac OS X
-* multi-window text editor with multiple undo, Python colorizing and many other
- features, e.g. smart indent and call tips
+* multi-window text editor with multiple undo, Python colorizing,
+ smart indent, call tips, and many other features
* Python shell window (a.k.a. interactive interpreter)
-* debugger (not complete, but you can set breakpoints, view and step)
+* debugger (not complete, but you can set breakpoints, view and step)
Menus
-----
+IDLE has two window types, the Shell window and the Editor window. It is
+possible to have multiple editor windows simultaneously. IDLE's
+menus dynamically change based on which window is currently selected. Each menu
+documented below indicates which window type it is associated with. Click on
+the dotted line at the top of a menu to "tear it off": a separate window
+containing the menu is created (for Unix and Windows only).
-File menu
-^^^^^^^^^
+File menu (Shell and Editor)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
New file
- create a new file editing window
+ Create a new file editing window
Open...
- open an existing file
+ Open an existing file
Open module...
- open an existing module (searches sys.path)
+ Open an existing module (searches sys.path)
+
+Recent Files
+ Open a list of recent files
Class browser
- show classes and methods in current file
+ Show classes and methods in current file
Path browser
- show sys.path directories, modules, classes and methods
+ Show sys.path directories, modules, classes and methods
.. index::
single: Class browser
single: Path browser
Save
- save current window to the associated file (unsaved windows have a \* before and
- after the window title)
+ Save current window to the associated file (unsaved windows have a
+ \* before and after the window title)
Save As...
- save current window to new file, which becomes the associated file
+ Save current window to new file, which becomes the associated file
Save Copy As...
- save current window to different file without changing the associated file
+ Save current window to different file without changing the associated file
+
+Print Window
+ Print the current window
Close
- close current window (asks to save if unsaved)
+ Close current window (asks to save if unsaved)
Exit
- close all windows and quit IDLE (asks to save if unsaved)
+ Close all windows and quit IDLE (asks to save if unsaved)
-Edit menu
-^^^^^^^^^
+Edit menu (Shell and Editor)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Undo
- Undo last change to current window (max 1000 changes)
+ Undo last change to current window (a maximum of 1000 changes may be undone)
Redo
Redo last undone change to current window
Cut
- Copy selection into system-wide clipboard; then delete selection
+ Copy selection into system-wide clipboard; then delete the selection
Copy
Copy selection into system-wide clipboard
@@ -108,11 +120,30 @@ Replace...
Go to line
Ask for a line number and show that line
+Expand word
+ Expand the word you have typed to match another word in the same buffer;
+ repeat to get a different expansion
+
+Show call tip
+ After an unclosed parenthesis for a function, open a small window with
+ function parameter hints
+
+Show surrounding parens
+ Highlight the surrounding parenthesis
+
+Show Completions
+ Open a scroll window allowing selection keywords and attributes. See
+ Completions below.
+
+
+Format menu (Editor window only)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
Indent region
- Shift selected lines right 4 spaces
+ Shift selected lines right by the indent width (default 4 spaces)
Dedent region
- Shift selected lines left 4 spaces
+ Shift selected lines left by the indent width (default 4 spaces)
Comment out region
Insert ## in front of selected lines
@@ -121,67 +152,121 @@ Uncomment region
Remove leading # or ## from selected lines
Tabify region
- Turns *leading* stretches of spaces into tabs
+ Turns *leading* stretches of spaces into tabs. (Note: We recommend using
+ 4 space blocks to indent Python code.)
Untabify region
- Turn *all* tabs into the right number of spaces
+ Turn *all* tabs into the correct number of spaces
-Expand word
- Expand the word you have typed to match another word in the same buffer; repeat
- to get a different expansion
+Toggle tabs
+ Open a dialog to switch between indenting with spaces and tabs.
-Format Paragraph
- Reformat the current blank-line-separated paragraph
+New Indent Width
+ Open a dialog to change indent width. The accepted default by the Python
+ community is 4 spaces.
-Import module
- Import or reload the current module
+Format Paragraph
+ Reformat the current blank-line-separated paragraph. All lines in the
+ paragraph will be formatted to less than 80 columns.
-Run script
- Execute the current file in the __main__ namespace
+Strip trailing whitespace
+ Removes any space characters after the end of the last non-space character
.. index::
single: Import module
single: Run script
-Windows menu
-^^^^^^^^^^^^
+Run menu (Editor window only)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Zoom Height
- toggles the window between normal size (24x80) and maximum height.
+Python Shell
+ Open or wake up the Python Shell window
-The rest of this menu lists the names of all open windows; select one to bring
-it to the foreground (deiconifying it if necessary).
+Check module
+ Check the syntax of the module currently open in the Editor window. If the
+ module has not been saved IDLE will prompt the user to save the code.
+
+Run module
+ Restart the shell to clean the environment, then execute the currently
+ open module. If the module has not been saved IDLE will prompt the user
+ to save the code.
+Shell menu (Shell window only)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-Debug menu
-^^^^^^^^^^
+View Last Restart
+ Scroll the shell window to the last Shell restart
-* in the Python Shell window only
+Restart Shell
+ Restart the shell to clean the environment
+
+Debug menu (Shell window only)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Go to file/line
Look around the insert point for a filename and line number, open the file,
and show the line. Useful to view the source lines referenced in an
- exception traceback.
+ exception traceback. Available in the context menu of the Shell window.
-Debugger
- Run commands in the shell under the debugger.
+Debugger (toggle)
+ This feature is not complete and considered experimental. Run commands in
+ the shell under the debugger
Stack viewer
- Show the stack traceback of the last exception.
+ Show the stack traceback of the last exception
Auto-open Stack Viewer
- Open stack viewer on traceback.
+ Toggle automatically opening the stack viewer on unhandled exception
.. index::
single: stack viewer
single: debugger
+Options menu (Shell and Editor)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Configure IDLE
+ Open a configuration dialog. Fonts, indentation, keybindings, and color
+ themes may be altered. Startup Preferences may be set, and additional
+ help sources can be specified.
+
+Code Context (toggle)(Editor Window only)
+ Open a pane at the top of the edit window which shows the block context
+ of the section of code which is scrolling off the top of the window.
+
+Windows menu (Shell and Editor)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Zoom Height
+ Toggles the window between normal size (40x80 initial setting) and maximum
+ height. The initial size is in the Configure IDLE dialog under the general
+ tab.
+
+The rest of this menu lists the names of all open windows; select one to bring
+it to the foreground (deiconifying it if necessary).
+
+Help menu (Shell and Editor)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+About IDLE
+ Version, copyright, license, credits
+
+IDLE Help
+ Display a help file for IDLE detailing the menu options, basic editing and
+ navigation, and other tips.
+
+Python Docs
+ Access local Python documentation, if installed. Or will start a web browser
+ and open docs.python.org showing the latest Python documentation.
-Edit context menu
-^^^^^^^^^^^^^^^^^
+Additional help sources may be added here with the Configure IDLE dialog under
+the General tab.
-* Right-click in Edit window (Control-click on OS X)
+Editor Window context menu
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+* Right-click in Editor window (Control-click on OS X)
Cut
Copy selection into system-wide clipboard; then delete selection
@@ -207,8 +292,8 @@ Clear Breakpoint
single: breakpoints
-Shell context menu
-^^^^^^^^^^^^^^^^^^
+Shell Window context menu
+^^^^^^^^^^^^^^^^^^^^^^^^^
* Right-click in Python Shell window (Control-click on OS X)
@@ -225,19 +310,44 @@ Go to file/line
Same as in Debug menu.
-Basic editing and navigation
-----------------------------
+Editing and navigation
+----------------------
* :kbd:`Backspace` deletes to the left; :kbd:`Del` deletes to the right
+* :kbd:`C-Backspace` delete word left; :kbd:`C-Del` delete word to the right
+
* Arrow keys and :kbd:`Page Up`/:kbd:`Page Down` to move around
+* :kbd:`C-LeftArrow` and :kbd:`C-RightArrow` moves by words
+
* :kbd:`Home`/:kbd:`End` go to begin/end of line
* :kbd:`C-Home`/:kbd:`C-End` go to begin/end of file
-* Some :program:`Emacs` bindings may also work, including :kbd:`C-B`,
- :kbd:`C-P`, :kbd:`C-A`, :kbd:`C-E`, :kbd:`C-D`, :kbd:`C-L`
+* Some useful Emacs bindings are inherited from Tcl/Tk:
+
+ * :kbd:`C-a` beginning of line
+
+ * :kbd:`C-e` end of line
+
+ * :kbd:`C-k` kill line (but doesn't put it in clipboard)
+
+ * :kbd:`C-l` center window around the insertion point
+
+ * :kbd:`C-b` go backwards one character without deleting (usually you can
+ also use the cursor key for this)
+
+ * :kbd:`C-f` go forward one character without deleting (usually you can
+ also use the cursor key for this)
+
+ * :kbd:`C-p` go up one line (usually you can also use the cursor key for
+ this)
+
+ * :kbd:`C-d` delete next character
+
+Standard keybindings (like :kbd:`C-c` to copy and :kbd:`C-v` to paste)
+may work. Keybindings are selected in the Configure IDLE dialog.
Automatic indentation
@@ -246,27 +356,76 @@ Automatic indentation
After a block-opening statement, the next line is indented by 4 spaces (in the
Python Shell window by one tab). After certain keywords (break, return etc.)
the next line is dedented. In leading indentation, :kbd:`Backspace` deletes up
-to 4 spaces if they are there. :kbd:`Tab` inserts 1-4 spaces (in the Python
-Shell window one tab). See also the indent/dedent region commands in the edit
-menu.
-
+to 4 spaces if they are there. :kbd:`Tab` inserts spaces (in the Python
+Shell window one tab), number depends on Indent width. Currently tabs
+are restricted to four spaces due to Tcl/Tk limitations.
+
+See also the indent/dedent region commands in the edit menu.
+
+Completions
+^^^^^^^^^^^
+
+Completions are supplied for functions, classes, and attributes of classes,
+both built-in and user-defined. Completions are also provided for
+filenames.
+
+The AutoCompleteWindow (ACW) will open after a predefined delay (default is
+two seconds) after a '.' or (in a string) an os.sep is typed. If after one
+of those characters (plus zero or more other characters) a tab is typed
+the ACW will open immediately if a possible continuation is found.
+
+If there is only one possible completion for the characters entered, a
+:kbd:`Tab` will supply that completion without opening the ACW.
+
+'Show Completions' will force open a completions window, by default the
+:kbd:`C-space` will open a completions window. In an empty
+string, this will contain the files in the current directory. On a
+blank line, it will contain the built-in and user-defined functions and
+classes in the current name spaces, plus any modules imported. If some
+characters have been entered, the ACW will attempt to be more specific.
+
+If a string of characters is typed, the ACW selection will jump to the
+entry most closely matching those characters. Entering a :kbd:`tab` will
+cause the longest non-ambiguous match to be entered in the Editor window or
+Shell. Two :kbd:`tab` in a row will supply the current ACW selection, as
+will return or a double click. Cursor keys, Page Up/Down, mouse selection,
+and the scroll wheel all operate on the ACW.
+
+"Hidden" attributes can be accessed by typing the beginning of hidden
+name after a '.', e.g. '_'. This allows access to modules with
+``__all__`` set, or to class-private attributes.
+
+Completions and the 'Expand Word' facility can save a lot of typing!
+
+Completions are currently limited to those in the namespaces. Names in
+an Editor window which are not via ``__main__`` and :data:`sys.modules` will
+not be found. Run the module once with your imports to correct this situation.
+Note that IDLE itself places quite a few modules in sys.modules, so
+much can be found by default, e.g. the re module.
+
+If you don't like the ACW popping up unbidden, simply make the delay
+longer or disable the extension. Or another option is the delay could
+be set to zero. Another alternative to preventing ACW popups is to
+disable the call tips extension.
Python Shell window
^^^^^^^^^^^^^^^^^^^
-* :kbd:`C-C` interrupts executing command
+* :kbd:`C-c` interrupts executing command
-* :kbd:`C-D` sends end-of-file; closes window if typed at a ``>>>`` prompt
+* :kbd:`C-d` sends end-of-file; closes window if typed at a ``>>>`` prompt
+ (this is :kbd:`C-z` on Windows).
-* :kbd:`Alt-p` retrieves previous command matching what you have typed
+* :kbd:`Alt-/` (Expand word) is also useful to reduce typing
-* :kbd:`Alt-n` retrieves next
+ Command history
-* :kbd:`Return` while on any previous command retrieves that command
+ * :kbd:`Alt-p` retrieves previous command matching what you have typed. On
+ OS X use :kbd:`C-p`.
-* :kbd:`Alt-/` (Expand word) is also useful here
+ * :kbd:`Alt-n` retrieves next. On OS X use :kbd:`C-n`.
-.. index:: single: indentation
+ * :kbd:`Return` while on any previous command retrieves that command
Syntax colors
@@ -308,17 +467,17 @@ Startup
Upon startup with the ``-s`` option, IDLE will execute the file referenced by
the environment variables :envvar:`IDLESTARTUP` or :envvar:`PYTHONSTARTUP`.
-Idle first checks for ``IDLESTARTUP``; if ``IDLESTARTUP`` is present the file
-referenced is run. If ``IDLESTARTUP`` is not present, Idle checks for
+IDLE first checks for ``IDLESTARTUP``; if ``IDLESTARTUP`` is present the file
+referenced is run. If ``IDLESTARTUP`` is not present, IDLE checks for
``PYTHONSTARTUP``. Files referenced by these environment variables are
-convenient places to store functions that are used frequently from the Idle
+convenient places to store functions that are used frequently from the IDLE
shell, or for executing import statements to import common modules.
In addition, ``Tk`` also loads a startup file if it is present. Note that the
Tk file is loaded unconditionally. This additional file is ``.Idle.py`` and is
looked for in the user's home directory. Statements in this file will be
executed in the Tk namespace, so this file is not useful for importing functions
-to be used from Idle's Python shell.
+to be used from IDLE's Python shell.
Command line usage
@@ -349,3 +508,45 @@ If there are arguments:
the arguments are still available in ``sys.argv``.
+Additional help sources
+-----------------------
+
+IDLE includes a help menu entry called "Python Docs" that will open the
+extensive sources of help, including tutorials, available at docs.python.org.
+Selected URLs can be added or removed from the help menu at any time using the
+Configure IDLE dialog. See the IDLE help option in the help menu of IDLE for
+more information.
+
+
+Other preferences
+-----------------
+
+The font preferences, highlighting, keys, and general preferences can be
+changed via the Configure IDLE menu option. Be sure to note that
+keys can be user defined, IDLE ships with four built in key sets. In
+addition a user can create a custom key set in the Configure IDLE dialog
+under the keys tab.
+
+Extensions
+----------
+
+IDLE contains an extension facility. See the beginning of
+config-extensions.def in the idlelib directory for further information. The
+default extensions are currently:
+
+* FormatParagraph
+
+* AutoExpand
+
+* ZoomHeight
+
+* ScriptBinding
+
+* CallTips
+
+* ParenMatch
+
+* AutoComplete
+
+* CodeContext
+
diff --git a/Doc/library/imp.rst b/Doc/library/imp.rst
index ebdbd4d..6647b9e 100644
--- a/Doc/library/imp.rst
+++ b/Doc/library/imp.rst
@@ -1,6 +1,9 @@
:mod:`imp` --- Access the :ref:`import <importsystem>` internals
================================================================
+.. deprecated:: 3.4
+ The :mod:`imp` package is pending deprecation in favor of :mod:`importlib`.
+
.. module:: imp
:synopsis: Access the implementation of the import statement.
@@ -11,10 +14,6 @@ This module provides an interface to the mechanisms used to implement the
:keyword:`import` statement. It defines the following constants and functions:
-.. note::
- New programs should use :mod:`importlib` rather than this module.
-
-
.. function:: get_magic()
.. index:: pair: file; byte-code
@@ -22,6 +21,9 @@ This module provides an interface to the mechanisms used to implement the
Return the magic string value used to recognize byte-compiled code files
(:file:`.pyc` files). (This value may be different for each Python version.)
+ .. deprecated:: 3.4
+ Use :attr:`importlib.util.MAGIC_NUMBER` instead.
+
.. function:: get_suffixes()
@@ -101,8 +103,10 @@ This module provides an interface to the mechanisms used to implement the
using a :keyword:`try` ... :keyword:`finally` statement.
.. deprecated:: 3.3
- Unneeded as loaders should be used to load modules and
- :func:`find_module` is deprecated.
+ If previously used in conjunction with :func:`imp.find_module` then
+ call ``load_module()`` on the returned loader. If you wish to load a
+ module from a specific file, then use one of the file-based loaders found
+ in :mod:`importlib.machinery`.
.. function:: new_module(name)
@@ -110,6 +114,9 @@ This module provides an interface to the mechanisms used to implement the
Return a new empty module object called *name*. This object is *not* inserted
in ``sys.modules``.
+ .. deprecated:: 3.4
+ Use :class:`types.ModuleType` instead.
+
.. function:: reload(module)
@@ -176,6 +183,9 @@ This module provides an interface to the mechanisms used to implement the
Relies on both ``__name__`` and ``__loader__`` being defined on the module
being reloaded instead of just ``__name__``.
+ .. deprecated:: 3.4
+ Use :func:`importlib.reload` instead.
+
The following functions are conveniences for handling :pep:`3147` byte-compiled
file paths.
@@ -201,6 +211,9 @@ file paths.
If :attr:`sys.implementation.cache_tag` is ``None``, then
:exc:`NotImplementedError` is raised.
+ .. deprecated:: 3.4
+ Use :func:`importlib.util.cache_from_source` instead.
+
.. function:: source_from_cache(path)
@@ -216,14 +229,17 @@ file paths.
Raise :exc:`NotImplementedError` when
:attr:`sys.implementation.cache_tag` is not defined.
+ .. deprecated:: 3.4
+ Use :func:`importlib.util.source_from_cache` instead.
+
.. function:: get_tag()
Return the :pep:`3147` magic tag string matching this version of Python's
magic number, as returned by :func:`get_magic`.
- .. note::
- You may use :attr:`sys.implementation.cache_tag` directly starting
+ .. deprecated:: 3.4
+ Use :attr:`sys.implementation.cache_tag` directly starting
in Python 3.3.
@@ -251,6 +267,8 @@ that circular imports work without any deadlocks.
the most part. A global import lock is kept for some critical tasks,
such as initializing the per-module locks.
+.. deprecated:: 3.4
+
.. function:: acquire_lock()
@@ -269,6 +287,8 @@ that circular imports work without any deadlocks.
the most part. A global import lock is kept for some critical tasks,
such as initializing the per-module locks.
+.. deprecated:: 3.4
+
.. function:: release_lock()
@@ -280,6 +300,8 @@ that circular imports work without any deadlocks.
the most part. A global import lock is kept for some critical tasks,
such as initializing the per-module locks.
+.. deprecated:: 3.4
+
The following constants with integer values, defined in this module, are used
to indicate the search result of :func:`find_module`.
@@ -345,6 +367,9 @@ to indicate the search result of :func:`find_module`.
``None`` is inserted into ``sys.path_importer_cache`` instead of an
instance of :class:`NullImporter`.
+ .. deprecated:: 3.4
+ Insert ``None`` into ``sys.path_importer_cache`` instead.
+
.. _examples-imp:
diff --git a/Doc/library/importlib.rst b/Doc/library/importlib.rst
index efd027b..fc63a8b 100644
--- a/Doc/library/importlib.rst
+++ b/Doc/library/importlib.rst
@@ -90,7 +90,7 @@ Functions
Find the loader for a module, optionally within the specified *path*. If the
module is in :attr:`sys.modules`, then ``sys.modules[name].__loader__`` is
- returned (unless the loader would be ``None``, in which case
+ returned (unless the loader would be ``None`` or is not set, in which case
:exc:`ValueError` is raised). Otherwise a search using :attr:`sys.meta_path`
is done. ``None`` is returned if no loader is found.
@@ -99,6 +99,12 @@ Functions
will need to import all parent packages of the submodule and use the correct
argument to *path*.
+ .. versionadded:: 3.3
+
+ .. versionchanged:: 3.4
+ If ``__loader__`` is not set, raise :exc:`ValueError`, just like when the
+ attribute is set to ``None``.
+
.. function:: invalidate_caches()
Invalidate the internal caches of finders stored at
@@ -109,6 +115,74 @@ Functions
.. versionadded:: 3.3
+.. function:: reload(module)
+
+ Reload a previously imported *module*. The argument must be a module object,
+ so it must have been successfully imported before. This is useful if you
+ have edited the module source file using an external editor and want to try
+ out the new version without leaving the Python interpreter. The return value
+ is the module object (which can be different if re-importing causes a
+ different object to be placed in :data:`sys.modules`).
+
+ When :func:`reload` is executed:
+
+ * Python modules' code is recompiled and the module-level code re-executed,
+ defining a new set of objects which are bound to names in the module's
+ dictionary by reusing the :term:`loader` which originally loaded the
+ module. The ``init`` function of extension modules is not called a second
+ time.
+
+ * As with all other objects in Python the old objects are only reclaimed
+ after their reference counts drop to zero.
+
+ * The names in the module namespace are updated to point to any new or
+ changed objects.
+
+ * Other references to the old objects (such as names external to the module) are
+ not rebound to refer to the new objects and must be updated in each namespace
+ where they occur if that is desired.
+
+ There are a number of other caveats:
+
+ If a module is syntactically correct but its initialization fails, the first
+ :keyword:`import` statement for it does not bind its name locally, but does
+ store a (partially initialized) module object in ``sys.modules``. To reload
+ the module you must first :keyword:`import` it again (this will bind the name
+ to the partially initialized module object) before you can :func:`reload` it.
+
+ When a module is reloaded, its dictionary (containing the module's global
+ variables) is retained. Redefinitions of names will override the old
+ definitions, so this is generally not a problem. If the new version of a
+ module does not define a name that was defined by the old version, the old
+ definition remains. This feature can be used to the module's advantage if it
+ maintains a global table or cache of objects --- with a :keyword:`try`
+ statement it can test for the table's presence and skip its initialization if
+ desired::
+
+ try:
+ cache
+ except NameError:
+ cache = {}
+
+ It is legal though generally not very useful to reload built-in or
+ dynamically loaded modules (this is not true for e.g. :mod:`sys`,
+ :mod:`__main__`, :mod:`builtins` and other key modules where reloading is
+ frowned upon). In many cases, however, extension modules are not designed to
+ be initialized more than once, and may fail in arbitrary ways when reloaded.
+
+ If a module imports objects from another module using :keyword:`from` ...
+ :keyword:`import` ..., calling :func:`reload` for the other module does not
+ redefine the objects imported from it --- one way around this is to
+ re-execute the :keyword:`from` statement, another is to use :keyword:`import`
+ and qualified names (*module.name*) instead.
+
+ If a module instantiates instances of a class, reloading the module that
+ defines the class does not affect the method definitions of the instances ---
+ they continue to use the old class definition. The same is true for derived
+ classes.
+
+ .. versionadded:: 3.4
+
:mod:`importlib.abc` -- Abstract base classes related to import
---------------------------------------------------------------
@@ -132,8 +206,6 @@ ABC hierarchy::
+-- ExecutionLoader --+
+-- FileLoader
+-- SourceLoader
- +-- PyLoader (deprecated)
- +-- PyPycLoader (deprecated)
.. class:: Finder
@@ -149,6 +221,10 @@ ABC hierarchy::
module. Originally specified in :pep:`302`, this method was meant
for use in :data:`sys.meta_path` and in the path-based import subsystem.
+ .. versionchanged:: 3.4
+ Returns ``None`` when called instead of raising
+ :exc:`NotImplementedError`.
+
.. class:: MetaPathFinder
@@ -165,12 +241,19 @@ ABC hierarchy::
will be the value of :attr:`__path__` from the parent
package. If a loader cannot be found, ``None`` is returned.
+ .. versionchanged:: 3.4
+ Returns ``None`` when called instead of raising
+ :exc:`NotImplementedError`.
+
.. method:: invalidate_caches()
An optional method which, when called, should invalidate any internal
cache used by the finder. Used by :func:`importlib.invalidate_caches`
when invalidating the caches of all finders on :data:`sys.meta_path`.
+ .. versionchanged:: 3.4
+ Returns ``None`` when called instead of ``NotImplemented``.
+
.. class:: PathEntryFinder
@@ -178,7 +261,7 @@ ABC hierarchy::
it bears some similarities to :class:`MetaPathFinder`, ``PathEntryFinder``
is meant for use only within the path-based import subsystem provided
by :class:`PathFinder`. This ABC is a subclass of :class:`Finder` for
- compatibility.
+ compatibility reasons only.
.. versionadded:: 3.3
@@ -190,9 +273,12 @@ ABC hierarchy::
package. The loader may be ``None`` while specifying ``portion`` to
signify the contribution of the file system locations to a namespace
package. An empty list can be used for ``portion`` to signify the loader
- is not part of a package. If ``loader`` is ``None`` and ``portion`` is
- the empty list then no loader or location for a namespace package were
- found (i.e. failure to find anything for the module).
+ is not part of a namespace package. If ``loader`` is ``None`` and
+ ``portion`` is the empty list then no loader or location for a namespace
+ package were found (i.e. failure to find anything for the module).
+
+ .. versionchanged:: 3.4
+ Returns ``(None, [])`` instead of raising :exc:`NotImplementedError`.
.. method:: find_module(fullname)
@@ -224,12 +310,11 @@ ABC hierarchy::
from the import. If the loader inserted a module and the load fails, it
must be removed by the loader from :data:`sys.modules`; modules already
in :data:`sys.modules` before the loader began execution should be left
- alone. The :func:`importlib.util.module_for_loader` decorator handles
- all of these details.
+ alone (see :func:`importlib.util.module_for_loader`).
The loader should set several attributes on the module.
(Note that some of these attributes can change when a module is
- reloaded.)
+ reloaded):
- :attr:`__name__`
The name of the module.
@@ -249,20 +334,28 @@ ABC hierarchy::
- :attr:`__package__`
The parent package for the module/package. If the module is
top-level then it has a value of the empty string. The
- :func:`importlib.util.set_package` decorator can handle the details
- for :attr:`__package__`.
+ :func:`importlib.util.module_for_loader` decorator can handle the
+ details for :attr:`__package__`.
- :attr:`__loader__`
- The loader used to load the module.
- (This is not set by the built-in import machinery,
- but it should be set whenever a :term:`loader` is used.)
+ The loader used to load the module. The
+ :func:`importlib.util.module_for_loader` decorator can handle the
+ details for :attr:`__package__`.
+
+ .. versionchanged:: 3.4
+ Raise :exc:`ImportError` when called instead of
+ :exc:`NotImplementedError`.
.. method:: module_repr(module)
- An abstract method which when implemented calculates and returns the
- given module's repr, as a string.
+ An optional method which when implemented calculates and returns the
+ given module's repr, as a string. The module type's default repr() will
+ use the result of this method as appropriate.
+
+ .. versionadded:: 3.3
- .. versionadded: 3.3
+ .. versionchanged:: 3.4
+ Made optional instead of an abstractmethod.
.. class:: ResourceLoader
@@ -281,6 +374,9 @@ ABC hierarchy::
be found. The *path* is expected to be constructed using a module's
:attr:`__file__` attribute or an item from a package's :attr:`__path__`.
+ .. versionchanged:: 3.4
+ Raises :exc:`IOError` instead of :exc:`NotImplementedError`.
+
.. class:: InspectLoader
@@ -289,14 +385,21 @@ ABC hierarchy::
.. method:: get_code(fullname)
- An abstract method to return the :class:`code` object for a module.
- ``None`` is returned if the module does not have a code object
+ Return the code object for a module.
+ ``None`` should be returned if the module does not have a code object
(e.g. built-in module). :exc:`ImportError` is raised if loader cannot
find the requested module.
+ .. note::
+ While the method has a default implementation, it is suggested that
+ it be overridden if possible for performance.
+
.. index::
single: universal newlines; importlib.abc.InspectLoader.get_source method
+ .. versionchanged:: 3.4
+ No longer abstract and a concrete implementation is provided.
+
.. method:: get_source(fullname)
An abstract method to return the source of a module. It is returned as
@@ -305,12 +408,33 @@ ABC hierarchy::
if no source is available (e.g. a built-in module). Raises
:exc:`ImportError` if the loader cannot find the module specified.
+ .. versionchanged:: 3.4
+ Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
+
.. method:: is_package(fullname)
An abstract method to return a true value if the module is a package, a
false value otherwise. :exc:`ImportError` is raised if the
:term:`loader` cannot find the module.
+ .. versionchanged:: 3.4
+ Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
+
+ .. method:: source_to_code(data, path='<string>')
+
+ Create a code object from Python source.
+
+ The *data* argument can be whatever the :func:`compile` function
+ supports (i.e. string or bytes). The *path* argument should be
+ the "path" to where the source code originated from, which can be an
+ abstract concept (e.g. location in a zip file).
+
+ .. versionadded:: 3.4
+
+ .. method:: load_module(fullname)
+
+ Implementation of :meth:`Loader.load_module`.
+
.. class:: ExecutionLoader
@@ -328,6 +452,9 @@ ABC hierarchy::
the source file, regardless of whether a bytecode was used to load the
module.
+ .. versionchanged:: 3.4
+ Raises :exc:`ImportError` instead of :exc:`NotImplementedError`.
+
.. class:: FileLoader(fullname, path)
@@ -358,7 +485,7 @@ ABC hierarchy::
.. method:: get_data(path)
- Returns the open, binary file for *path*.
+ Reads *path* as a binary file and returns the bytes from it.
.. class:: SourceLoader
@@ -373,7 +500,8 @@ ABC hierarchy::
loading is not supported.
The abstract methods defined by this class are to add optional bytecode
- file support. Not implementing these optional methods causes the loader to
+ file support. Not implementing these optional methods (or causing them to
+ raise :exc:`NotImplementedError`) causes the loader to
only work with source code. Implementing the methods allows the loader to
work with source *and* bytecode files; it does not allow for *sourceless*
loading where only bytecode is provided. Bytecode files are an
@@ -390,10 +518,13 @@ ABC hierarchy::
- ``'size'`` (optional): the size in bytes of the source code.
Any other keys in the dictionary are ignored, to allow for future
- extensions.
+ extensions. If the path cannot be handled, :exc:`IOError` is raised.
.. versionadded:: 3.3
+ .. versionchanged:: 3.4
+ Raise :exc:`IOError` instead of :exc:`NotImplementedError`.
+
.. method:: path_mtime(path)
Optional abstract method which returns the modification time for the
@@ -402,7 +533,10 @@ ABC hierarchy::
.. deprecated:: 3.3
This method is deprecated in favour of :meth:`path_stats`. You don't
have to implement it, but it is still available for compatibility
- purposes.
+ purposes. Raise :exc:`IOError` if the path cannot be handled.
+
+ .. versionchanged:: 3.4
+ Raise :exc:`IOError` instead of :exc:`NotImplementedError`.
.. method:: set_data(path, data)
@@ -414,6 +548,9 @@ ABC hierarchy::
(:attr:`errno.EACCES`/:exc:`PermissionError`), do not propagate the
exception.
+ .. versionchanged:: 3.4
+ No longer raises :exc:`NotImplementedError` when called.
+
.. method:: get_code(fullname)
Concrete implementation of :meth:`InspectLoader.get_code`.
@@ -435,142 +572,6 @@ ABC hierarchy::
itself does not end in ``__init__``.
-.. class:: PyLoader
-
- An abstract base class inheriting from
- :class:`ExecutionLoader` and
- :class:`ResourceLoader` designed to ease the loading of
- Python source modules (bytecode is not handled; see
- :class:`SourceLoader` for a source/bytecode ABC). A subclass
- implementing this ABC will only need to worry about exposing how the source
- code is stored; all other details for loading Python source code will be
- handled by the concrete implementations of key methods.
-
- .. deprecated:: 3.2
- This class has been deprecated in favor of :class:`SourceLoader` and is
- slated for removal in Python 3.4. See below for how to create a
- subclass that is compatible with Python 3.1 onwards.
-
- If compatibility with Python 3.1 is required, then use the following idiom
- to implement a subclass that will work with Python 3.1 onwards (make sure
- to implement :meth:`ExecutionLoader.get_filename`)::
-
- try:
- from importlib.abc import SourceLoader
- except ImportError:
- from importlib.abc import PyLoader as SourceLoader
-
-
- class CustomLoader(SourceLoader):
- def get_filename(self, fullname):
- """Return the path to the source file."""
- # Implement ...
-
- def source_path(self, fullname):
- """Implement source_path in terms of get_filename."""
- try:
- return self.get_filename(fullname)
- except ImportError:
- return None
-
- def is_package(self, fullname):
- """Implement is_package by looking for an __init__ file
- name as returned by get_filename."""
- filename = os.path.basename(self.get_filename(fullname))
- return os.path.splitext(filename)[0] == '__init__'
-
-
- .. method:: source_path(fullname)
-
- An abstract method that returns the path to the source code for a
- module. Should return ``None`` if there is no source code.
- Raises :exc:`ImportError` if the loader knows it cannot handle the
- module.
-
- .. method:: get_filename(fullname)
-
- A concrete implementation of
- :meth:`importlib.abc.ExecutionLoader.get_filename` that
- relies on :meth:`source_path`. If :meth:`source_path` returns
- ``None``, then :exc:`ImportError` is raised.
-
- .. method:: load_module(fullname)
-
- A concrete implementation of :meth:`importlib.abc.Loader.load_module`
- that loads Python source code. All needed information comes from the
- abstract methods required by this ABC. The only pertinent assumption
- made by this method is that when loading a package
- :attr:`__path__` is set to ``[os.path.dirname(__file__)]``.
-
- .. method:: get_code(fullname)
-
- A concrete implementation of
- :meth:`importlib.abc.InspectLoader.get_code` that creates code objects
- from Python source code, by requesting the source code (using
- :meth:`source_path` and :meth:`get_data`) and compiling it with the
- built-in :func:`compile` function.
-
- .. method:: get_source(fullname)
-
- A concrete implementation of
- :meth:`importlib.abc.InspectLoader.get_source`. Uses
- :meth:`importlib.abc.ResourceLoader.get_data` and :meth:`source_path`
- to get the source code. It tries to guess the source encoding using
- :func:`tokenize.detect_encoding`.
-
-
-.. class:: PyPycLoader
-
- An abstract base class inheriting from :class:`PyLoader`.
- This ABC is meant to help in creating loaders that support both Python
- source and bytecode.
-
- .. deprecated:: 3.2
- This class has been deprecated in favor of :class:`SourceLoader` and to
- properly support :pep:`3147`. If compatibility is required with
- Python 3.1, implement both :class:`SourceLoader` and :class:`PyLoader`;
- instructions on how to do so are included in the documentation for
- :class:`PyLoader`. Do note that this solution will not support
- sourceless/bytecode-only loading; only source *and* bytecode loading.
-
- .. versionchanged:: 3.3
- Updated to parse (but not use) the new source size field in bytecode
- files when reading and to write out the field properly when writing.
-
- .. method:: source_mtime(fullname)
-
- An abstract method which returns the modification time for the source
- code of the specified module. The modification time should be an
- integer. If there is no source code, return ``None``. If the
- module cannot be found then :exc:`ImportError` is raised.
-
- .. method:: bytecode_path(fullname)
-
- An abstract method which returns the path to the bytecode for the
- specified module, if it exists. It returns ``None``
- if no bytecode exists (yet).
- Raises :exc:`ImportError` if the loader knows it cannot handle the
- module.
-
- .. method:: get_filename(fullname)
-
- A concrete implementation of
- :meth:`ExecutionLoader.get_filename` that relies on
- :meth:`PyLoader.source_path` and :meth:`bytecode_path`.
- If :meth:`source_path` returns a path, then that value is returned.
- Else if :meth:`bytecode_path` returns a path, that path will be
- returned. If a path is not available from both methods,
- :exc:`ImportError` is raised.
-
- .. method:: write_bytecode(fullname, bytecode)
-
- An abstract method which has the loader write *bytecode* for future
- use. If the bytecode is written, return ``True``. Return
- ``False`` if the bytecode could not be written. This method
- should not be called if :data:`sys.dont_write_bytecode` is true.
- The *bytecode* argument should be a bytes string or bytes array.
-
-
:mod:`importlib.machinery` -- Importers and path hooks
------------------------------------------------------
@@ -686,6 +687,10 @@ find and load modules.
Calls :meth:`importlib.abc.PathEntryFinder.invalidate_caches` on all
finders stored in :attr:`sys.path_importer_cache`.
+ .. versionchanged:: 3.4
+ Calls objects in :data:`sys.path_hooks` with the current working directory
+ for ``''`` (i.e. the empty string).
+
.. class:: FileFinder(path, \*loader_details)
@@ -800,7 +805,7 @@ find and load modules.
.. class:: ExtensionFileLoader(fullname, path)
- A concrete implementation of :class:`importlib.abc.InspectLoader` for
+ A concrete implementation of :class:`importlib.abc.ExecutionLoader` for
extension modules.
The *fullname* argument specifies the name of the module the loader is to
@@ -834,6 +839,70 @@ find and load modules.
Returns ``None`` as extension modules do not have source code.
+ .. method:: get_filename(fullname)
+
+ Returns :attr:`path`.
+
+ .. versionadded:: 3.4
+
+
+.. class:: ModuleSpec(name, loader, *, origin=None, loader_state=None, is_package=None)
+
+ A specification for a module's import-system-related state.
+
+ .. versionadded:: 3.4
+
+ .. attribute:: name
+
+ (``__name__``)
+
+ A string for the fully-qualified name of the module.
+
+ .. attribute:: loader
+
+ (``__loader__``)
+
+ The loader to use for loading. For namespace packages this should be
+ set to None.
+
+ .. attribute:: origin
+
+ (``__file__``)
+
+ Name of the place from which the module is loaded, e.g. "builtin" for
+ built-in modules and the filename for modules loaded from source.
+ Normally "origin" should be set, but it may be None (the default)
+ which indicates it is unspecified.
+
+ .. attribute:: submodule_search_locations
+
+ (``__path__``)
+
+ List of strings for where to find submodules, if a package (None
+ otherwise).
+
+ .. attribute:: loader_state
+
+ Container of extra module-specific data for use during loading (or
+ None).
+
+ .. attribute:: cached
+
+ (``__cached__``)
+
+ String for where the compiled module should be stored (or None).
+
+ .. attribute:: parent
+
+ (``__package__``)
+
+ (Read-only) Fully-qualified name of the package to which the module
+ belongs as a submodule (or None).
+
+ .. attribute:: has_location
+
+ (Read-only) Boolean indicating whether or not the module's "origin"
+ attribute refers to a loadable location.
:mod:`importlib.util` -- Utility code for importers
---------------------------------------------------
@@ -844,6 +913,51 @@ find and load modules.
This module contains the various objects that help in the construction of
an :term:`importer`.
+.. attribute:: MAGIC_NUMBER
+
+ The bytes which represent the bytecode version number. If you need help with
+ loading/writing bytecode then consider :class:`importlib.abc.SourceLoader`.
+
+ .. versionadded:: 3.4
+
+.. function:: cache_from_source(path, debug_override=None)
+
+ Return the :pep:`3147` path to the byte-compiled file associated with the
+ source *path*. For example, if *path* is ``/foo/bar/baz.py`` the return
+ value would be ``/foo/bar/__pycache__/baz.cpython-32.pyc`` for Python 3.2.
+ The ``cpython-32`` string comes from the current magic tag (see
+ :func:`get_tag`; if :attr:`sys.implementation.cache_tag` is not defined then
+ :exc:`NotImplementedError` will be raised). The returned path will end in
+ ``.pyc`` when ``__debug__`` is True or ``.pyo`` for an optimized Python
+ (i.e. ``__debug__`` is False). By passing in True or False for
+ *debug_override* you can override the system's value for ``__debug__`` for
+ extension selection.
+
+ *path* need not exist.
+
+ .. versionadded:: 3.4
+
+
+.. function:: source_from_cache(path)
+
+ Given the *path* to a :pep:`3147` file name, return the associated source code
+ file path. For example, if *path* is
+ ``/foo/bar/__pycache__/baz.cpython-32.pyc`` the returned path would be
+ ``/foo/bar/baz.py``. *path* need not exist, however if it does not conform
+ to :pep:`3147` format, a ``ValueError`` is raised. If
+ :attr:`sys.implementation.cache_tag` is not defined,
+ :exc:`NotImplementedError` is raised.
+
+ .. versionadded:: 3.4
+
+.. function:: decode_source(source_bytes)
+
+ Decode the given bytes representing source code and return it as a string
+ with universal newlines (as required by
+ :meth:`importlib.abc.InspectLoader.get_source`).
+
+ .. versionadded:: 3.4
+
.. function:: resolve_name(name, package)
Resolve a relative module name to an absolute one.
@@ -862,7 +976,7 @@ an :term:`importer`.
.. decorator:: module_for_loader
- A :term:`decorator` for a :term:`loader` method,
+ A :term:`decorator` for :meth:`importlib.abc.Loader.load_module`
to handle selecting the proper
module object to load with. The decorated method is expected to have a call
signature taking two positional arguments
@@ -873,55 +987,62 @@ an :term:`importer`.
The decorated method will take in the **name** of the module to be loaded
as expected for a :term:`loader`. If the module is not found in
- :data:`sys.modules` then a new one is constructed with its
- :attr:`__name__` attribute set to **name**, :attr:`__loader__` set to
- **self**, and :attr:`__package__` set if
- :meth:`importlib.abc.InspectLoader.is_package` is defined for **self** and
- does not raise :exc:`ImportError` for **name**. If a new module is not
- needed then the module found in :data:`sys.modules` will be passed into the
- method.
+ :data:`sys.modules` then a new one is constructed. Regardless of where the
+ module came from, :attr:`__loader__` set to **self** and :attr:`__package__`
+ is set based on what :meth:`importlib.abc.InspectLoader.is_package` returns
+ (if available). These attributes are set unconditionally to support
+ reloading.
If an exception is raised by the decorated method and a module was added to
- :data:`sys.modules` it will be removed to prevent a partially initialized
- module from being in left in :data:`sys.modules`. If the module was already
- in :data:`sys.modules` then it is left alone.
-
- Use of this decorator handles all the details of which module object a
- loader should initialize as specified by :pep:`302` as best as possible.
+ :data:`sys.modules`, then the module will be removed to prevent a partially
+ initialized module from being in left in :data:`sys.modules`. If the module
+ was already in :data:`sys.modules` then it is left alone.
.. versionchanged:: 3.3
:attr:`__loader__` and :attr:`__package__` are automatically set
(when possible).
-.. decorator:: set_loader
+ .. versionchanged:: 3.4
+ Set :attr:`__name__`, :attr:`__loader__` :attr:`__package__`
+ unconditionally to support reloading.
- A :term:`decorator` for a :term:`loader` method,
- to set the :attr:`__loader__`
- attribute on loaded modules. If the attribute is already set the decorator
- does nothing. It is assumed that the first positional argument to the
- wrapped method (i.e. ``self``) is what :attr:`__loader__` should be set to.
+ .. deprecated:: 3.4
+ The import machinery now directly performs all the functionality
+ provided by this function.
- .. note::
+.. decorator:: set_loader
- It is recommended that :func:`module_for_loader` be used over this
- decorator as it subsumes this functionality.
+ A :term:`decorator` for :meth:`importlib.abc.Loader.load_module`
+ to set the :attr:`__loader__`
+ attribute on the returned module. If the attribute is already set the
+ decorator does nothing. It is assumed that the first positional argument to
+ the wrapped method (i.e. ``self``) is what :attr:`__loader__` should be set
+ to.
+ .. versionchanged:: 3.4
+ Set ``__loader__`` if set to ``None``, as if the attribute does not
+ exist.
.. decorator:: set_package
- A :term:`decorator` for a :term:`loader` to set the :attr:`__package__`
- attribute on the module returned by the loader. If :attr:`__package__` is
- set and has a value other than ``None`` it will not be changed.
- Note that the module returned by the loader is what has the attribute
- set on and not the module found in :data:`sys.modules`.
+ A :term:`decorator` for :meth:`importlib.abc.Loader.load_module` to set the :attr:`__package__` attribute on the returned module. If :attr:`__package__`
+ is set and has a value other than ``None`` it will not be changed.
+
+.. function:: spec_from_loader(name, loader, *, origin=None, is_package=None)
+
+ A factory function for creating a :class:`ModuleSpec` instance based
+ on a loader. The parameters have the same meaning as they do for
+ ModuleSpec. The function uses available :term:`loader` APIs, such as
+ :meth:`InspectLoader.is_package`, to fill in any missing
+ information on the spec.
+
+ .. versionadded:: 3.4
- Reliance on this decorator is discouraged when it is possible to set
- :attr:`__package__` before importing. By
- setting it beforehand the code for the module is executed with the
- attribute set and thus can be used by global level code during
- initialization.
+.. function:: spec_from_file_location(name, location, *, loader=None, submodule_search_locations=None)
- .. note::
+ A factory function for creating a :class:`ModuleSpec` instance based
+ on the path to a file. Missing information will be filled in on the
+ spec by making use of loader APIs and by the implication that the
+ module will be file-based.
- It is recommended that :func:`module_for_loader` be used over this
- decorator as it subsumes this functionality.
+ .. versionadded:: 3.4
diff --git a/Doc/library/index.rst b/Doc/library/index.rst
index 1b25c6e..81289a5 100644
--- a/Doc/library/index.rst
+++ b/Doc/library/index.rst
@@ -65,6 +65,7 @@ the `Python Package Index <http://pypi.python.org/pypi>`_.
tk.rst
development.rst
debug.rst
+ distribution.rst
python.rst
custominterp.rst
modules.rst
diff --git a/Doc/library/inspect.rst b/Doc/library/inspect.rst
index 625ea6e..41c61b7 100644
--- a/Doc/library/inspect.rst
+++ b/Doc/library/inspect.rst
@@ -173,8 +173,9 @@ attributes:
.. note::
- :func:`getmembers` does not return metaclass attributes when the argument
- is a class (this behavior is inherited from the :func:`dir` function).
+ :func:`getmembers` will only return class attributes defined in the
+ metaclass when the argument is a class and those attributes have been
+ listed in the metaclass' custom :meth:`__dir__`.
.. function:: getmoduleinfo(path)
@@ -797,6 +798,23 @@ Classes and functions
.. versionadded:: 3.3
+.. function:: unwrap(func, *, stop=None)
+
+ Get the object wrapped by *func*. It follows the chain of :attr:`__wrapped__`
+ attributes returning the last object in the chain.
+
+ *stop* is an optional callback accepting an object in the wrapper chain
+ as its sole argument that allows the unwrapping to be terminated early if
+ the callback returns a true value. If the callback never returns a true
+ value, the last object in the chain is returned as usual. For example,
+ :func:`signature` uses this to stop unwrapping if any object in the
+ chain has a ``__signature__`` attribute defined.
+
+ :exc:`ValueError` is raised if a cycle is encountered.
+
+ .. versionadded:: 3.4
+
+
.. _inspect-stack:
The interpreter stack
@@ -829,6 +847,10 @@ index of the current line within that list.
finally:
del frame
+ If you want to keep the frame around (for example to print a traceback
+ later), you can also break reference cycles by using the
+ :meth:`frame.clear` method.
+
The optional *context* argument supported by most of these functions specifies
the number of lines of context to return, which are centered around the current
line.
@@ -985,3 +1007,22 @@ updated as expected:
return an empty dictionary.
.. versionadded:: 3.3
+
+
+.. _inspect-module-cli:
+
+Command Line Interface
+----------------------
+
+The :mod:`inspect` module also provides a basic introspection capability
+from the command line.
+
+.. program:: inspect
+
+By default, accepts the name of a module and prints the source of that
+module. A class or function within the module can be printed instead by
+appended a colon and the qualified name of the target object.
+
+.. cmdoption:: --details
+
+ Print information about the specified object rather than the source code
diff --git a/Doc/library/io.rst b/Doc/library/io.rst
index 73288f8..7925e33 100644
--- a/Doc/library/io.rst
+++ b/Doc/library/io.rst
@@ -283,10 +283,10 @@ I/O Base Classes
Return ``True`` if the stream can be read from. If False, :meth:`read`
will raise :exc:`OSError`.
- .. method:: readline(limit=-1)
+ .. method:: readline(size=-1)
- Read and return one line from the stream. If *limit* is specified, at
- most *limit* bytes will be read.
+ Read and return one line from the stream. If *size* is specified, at
+ most *size* bytes will be read.
The line terminator is always ``b'\n'`` for binary files; for text files,
the *newlines* argument to :func:`open` can be used to select the line
@@ -366,14 +366,14 @@ I/O Base Classes
In addition to the attributes and methods from :class:`IOBase`,
:class:`RawIOBase` provides the following methods:
- .. method:: read(n=-1)
+ .. method:: read(size=-1)
- Read up to *n* bytes from the object and return them. As a convenience,
- if *n* is unspecified or -1, :meth:`readall` is called. Otherwise,
- only one system call is ever made. Fewer than *n* bytes may be
- returned if the operating system call returns fewer than *n* bytes.
+ Read up to *size* bytes from the object and return them. As a convenience,
+ if *size* is unspecified or -1, :meth:`readall` is called. Otherwise,
+ only one system call is ever made. Fewer than *size* bytes may be
+ returned if the operating system call returns fewer than *size* bytes.
- If 0 bytes are returned, and *n* was not 0, this indicates end of file.
+ If 0 bytes are returned, and *size* was not 0, this indicates end of file.
If the object is in non-blocking mode and no bytes are available,
``None`` is returned.
@@ -442,10 +442,10 @@ I/O Base Classes
.. versionadded:: 3.1
- .. method:: read(n=-1)
+ .. method:: read(size=-1)
- Read and return up to *n* bytes. If the argument is omitted, ``None``, or
- negative, data is read and returned until EOF is reached. An empty
+ Read and return up to *size* bytes. If the argument is omitted, ``None``,
+ or negative, data is read and returned until EOF is reached. An empty
:class:`bytes` object is returned if the stream is already at EOF.
If the argument is positive, and the underlying raw stream is not
@@ -457,9 +457,9 @@ I/O Base Classes
A :exc:`BlockingIOError` is raised if the underlying raw stream is in
non blocking-mode, and has no data available at the moment.
- .. method:: read1(n=-1)
+ .. method:: read1(size=-1)
- Read and return up to *n* bytes, with at most one call to the underlying
+ Read and return up to *size* bytes, with at most one call to the underlying
raw stream's :meth:`~RawIOBase.read` method. This can be useful if you
are implementing your own buffering on top of a :class:`BufferedIOBase`
object.
@@ -522,6 +522,8 @@ Raw File I/O
:mod:`os.open` as *opener* results in functionality similar to passing
``None``).
+ The newly created file is :ref:`non-inheritable <fd_inheritance>`.
+
See the :func:`open` built-in function for examples on using the *opener*
parameter.
@@ -529,6 +531,9 @@ Raw File I/O
The *opener* parameter was added.
The ``'x'`` mode was added.
+ .. versionchanged:: 3.4
+ The file is now non-inheritable.
+
In addition to the attributes and methods from :class:`IOBase` and
:class:`RawIOBase`, :class:`FileIO` provides the following data
attributes:
@@ -601,21 +606,21 @@ than raw I/O does.
:class:`BufferedReader` provides or overrides these methods in addition to
those from :class:`BufferedIOBase` and :class:`IOBase`:
- .. method:: peek([n])
+ .. method:: peek([size])
Return bytes from the stream without advancing the position. At most one
single read on the raw stream is done to satisfy the call. The number of
bytes returned may be less or more than requested.
- .. method:: read([n])
+ .. method:: read([size])
- Read and return *n* bytes, or if *n* is not given or negative, until EOF
- or if the read call would block in non-blocking mode.
+ Read and return *size* bytes, or if *size* is not given or negative, until
+ EOF or if the read call would block in non-blocking mode.
- .. method:: read1(n)
+ .. method:: read1(size)
- Read and return up to *n* bytes with only one call on the raw stream. If
- at least one byte is buffered, only buffered bytes are returned.
+ Read and return up to *size* bytes with only one call on the raw stream.
+ If at least one byte is buffered, only buffered bytes are returned.
Otherwise, one raw stream read call is made.
@@ -734,17 +739,17 @@ Text I/O
.. versionadded:: 3.1
- .. method:: read(n)
+ .. method:: read(size)
- Read and return at most *n* characters from the stream as a single
- :class:`str`. If *n* is negative or ``None``, reads until EOF.
+ Read and return at most *size* characters from the stream as a single
+ :class:`str`. If *size* is negative or ``None``, reads until EOF.
- .. method:: readline(limit=-1)
+ .. method:: readline(size=-1)
Read until newline or EOF and return a single ``str``. If the stream is
already at EOF, an empty string is returned.
- If *limit* is specified, at most *limit* characters will be read.
+ If *size* is specified, at most *size* characters will be read.
.. method:: seek(offset, whence=SEEK_SET)
diff --git a/Doc/library/ipaddress.rst b/Doc/library/ipaddress.rst
index 769af72..826e4aa 100644
--- a/Doc/library/ipaddress.rst
+++ b/Doc/library/ipaddress.rst
@@ -161,7 +161,16 @@ write code that handles both IP versions correctly.
.. attribute:: is_private
``True`` if the address is allocated for private networks. See
- :RFC:`1918` (for IPv4) or :RFC:`4193` (for IPv6).
+ iana-ipv4-special-registry (for IPv4) or iana-ipv6-special-registry
+ (for IPv6).
+
+ .. attribute:: is_global
+
+ ``True`` if the address is allocated for public networks. See
+ iana-ipv4-special-registry (for IPv4) or iana-ipv6-special-registry
+ (for IPv6).
+
+ .. versionadded:: 3.4
.. attribute:: is_unspecified
diff --git a/Doc/library/ipc.rst b/Doc/library/ipc.rst
index 91ec693..2440bd4 100644
--- a/Doc/library/ipc.rst
+++ b/Doc/library/ipc.rst
@@ -18,6 +18,9 @@ The list of modules described in this chapter is:
socket.rst
ssl.rst
+ select.rst
+ selectors.rst
+ asyncio.rst
asyncore.rst
asynchat.rst
signal.rst
diff --git a/Doc/library/json.rst b/Doc/library/json.rst
index f652039..5d97ee8 100644
--- a/Doc/library/json.rst
+++ b/Doc/library/json.rst
@@ -42,8 +42,7 @@ Compact encoding::
Pretty printing::
>>> import json
- >>> print(json.dumps({'4': 5, '6': 7}, sort_keys=True,
- ... indent=4, separators=(',', ': ')))
+ >>> print(json.dumps({'4': 5, '6': 7}, sort_keys=True, indent=4))
{
"4": 5,
"6": 7
@@ -158,15 +157,13 @@ Basic Usage
.. versionchanged:: 3.2
Allow strings for *indent* in addition to integers.
- .. note::
-
- Since the default item separator is ``', '``, the output might include
- trailing whitespace when *indent* is specified. You can use
- ``separators=(',', ': ')`` to avoid this.
+ If specified, *separators* should be an ``(item_separator, key_separator)``
+ tuple. The default is ``(', ', ': ')`` if *indent* is ``None`` and
+ ``(',', ': ')`` otherwise. To get the most compact JSON representation,
+ you should specify ``(',', ':')`` to eliminate whitespace.
- If *separators* is an ``(item_separator, dict_separator)`` tuple, then it
- will be used instead of the default ``(', ', ': ')`` separators. ``(',',
- ':')`` is the most compact JSON representation.
+ .. versionchanged:: 3.4
+ Use ``(',', ': ')`` as default if *indent* is not ``None``.
*default(obj)* is a function that should return a serializable version of
*obj* or raise :exc:`TypeError`. The default simply raises :exc:`TypeError`.
@@ -248,6 +245,8 @@ Basic Usage
kwarg; otherwise :class:`JSONDecoder` is used. Additional keyword arguments
will be passed to the constructor of the class.
+ If the data being deserialized is not a valid JSON document, a
+ :exc:`ValueError` will be raised.
.. function:: loads(s, encoding=None, cls=None, object_hook=None, parse_float=None, parse_int=None, parse_constant=None, object_pairs_hook=None, **kw)
@@ -257,6 +256,8 @@ Basic Usage
The other arguments have the same meaning as in :func:`load`, except
*encoding* which is ignored and deprecated.
+ If the data being deserialized is not a valid JSON document, a
+ :exc:`ValueError` will be raised.
Encoders and Decoders
---------------------
@@ -354,23 +355,26 @@ Encoders and Decoders
.. _py-to-json-table:
- +-------------------+---------------+
- | Python | JSON |
- +===================+===============+
- | dict | object |
- +-------------------+---------------+
- | list, tuple | array |
- +-------------------+---------------+
- | str | string |
- +-------------------+---------------+
- | int, float | number |
- +-------------------+---------------+
- | True | true |
- +-------------------+---------------+
- | False | false |
- +-------------------+---------------+
- | None | null |
- +-------------------+---------------+
+ +----------------------------------------+---------------+
+ | Python | JSON |
+ +========================================+===============+
+ | dict | object |
+ +----------------------------------------+---------------+
+ | list, tuple | array |
+ +----------------------------------------+---------------+
+ | str | string |
+ +----------------------------------------+---------------+
+ | int, float, int- & float-derived Enums | number |
+ +----------------------------------------+---------------+
+ | True | true |
+ +----------------------------------------+---------------+
+ | False | false |
+ +----------------------------------------+---------------+
+ | None | null |
+ +----------------------------------------+---------------+
+
+ .. versionchanged:: 3.4
+ Added support for int- and float-derived Enum classes.
To extend this to recognize other objects, subclass and implement a
:meth:`default` method with another method that returns a serializable object
@@ -410,15 +414,13 @@ Encoders and Decoders
.. versionchanged:: 3.2
Allow strings for *indent* in addition to integers.
- .. note::
-
- Since the default item separator is ``', '``, the output might include
- trailing whitespace when *indent* is specified. You can use
- ``separators=(',', ': ')`` to avoid this.
-
If specified, *separators* should be an ``(item_separator, key_separator)``
- tuple. The default is ``(', ', ': ')``. To get the most compact JSON
- representation, you should specify ``(',', ':')`` to eliminate whitespace.
+ tuple. The default is ``(', ', ': ')`` if *indent* is ``None`` and
+ ``(',', ': ')`` otherwise. To get the most compact JSON representation,
+ you should specify ``(',', ':')`` to eliminate whitespace.
+
+ .. versionchanged:: 3.4
+ Use ``(',', ': ')`` as default if *indent* is not ``None``.
If specified, *default* is a function that gets called for objects that can't
otherwise be serialized. It should return a JSON encodable version of the
diff --git a/Doc/library/logging.config.rst b/Doc/library/logging.config.rst
index c00b532..8c15ee2 100644
--- a/Doc/library/logging.config.rst
+++ b/Doc/library/logging.config.rst
@@ -80,11 +80,23 @@ in :mod:`logging` itself) and defining handlers which are declared either in
.. function:: fileConfig(fname, defaults=None, disable_existing_loggers=True)
- Reads the logging configuration from a :mod:`configparser`\-format file
- named *fname*. This function can be called several times from an
- application, allowing an end user to select from various pre-canned
- configurations (if the developer provides a mechanism to present the choices
- and load the chosen configuration).
+ Reads the logging configuration from a :mod:`configparser`\-format file.
+ This function can be called several times from an application, allowing an
+ end user to select from various pre-canned configurations (if the developer
+ provides a mechanism to present the choices and load the chosen
+ configuration).
+
+ :param fname: A filename, or a file-like object, or an instance derived
+ from :class:`~configparser.RawConfigParser`. If a
+ ``RawConfigParser``-derived instance is passed, it is used as
+ is. Otherwise, a :class:`~configparser.Configparser` is
+ instantiated, and the configuration read by it from the
+ object passed in ``fname``. If that has a :meth:`readline`
+ method, it is assumed to be a file-like object and read using
+ :meth:`~configparser.ConfigParser.read_file`; otherwise,
+ it is assumed to be a filename and passed to
+ :meth:`~configparser.ConfigParser.read`.
+
:param defaults: Defaults to be passed to the ConfigParser can be specified
in this argument.
@@ -98,8 +110,17 @@ in :mod:`logging` itself) and defining handlers which are declared either in
their ancestors are explicitly named in the
logging configuration.
+ .. versionchanged:: 3.4
+ An instance of a subclass of :class:`~configparser.RawConfigParser` is
+ now accepted as a value for ``fname``. This facilitates:
+
+ * Use of a configuration file where logging configuration is just part
+ of the overall application configuration.
+ * Use of a configuration read from a file, and then modified by the using
+ application (e.g. based on command-line parameters or other aspects
+ of the runtime environment) before being passed to ``fileConfig``.
-.. function:: listen(port=DEFAULT_LOGGING_CONFIG_PORT)
+.. function:: listen(port=DEFAULT_LOGGING_CONFIG_PORT, verify=None)
Starts up a socket server on the specified port, and listens for new
configurations. If no port is specified, the module's default
@@ -110,6 +131,17 @@ in :mod:`logging` itself) and defining handlers which are declared either in
:meth:`~threading.Thread.join` when appropriate. To stop the server,
call :func:`stopListening`.
+ The ``verify`` argument, if specified, should be a callable which should
+ verify whether bytes received across the socket are valid and should be
+ processed. This could be done by encrypting and/or signing what is sent
+ across the socket, such that the ``verify`` callable can perform
+ signature verification and/or decryption. The ``verify`` callable is called
+ with a single argument - the bytes received across the socket - and should
+ return the bytes to be processed, or None to indicate that the bytes should
+ be discarded. The returned bytes could be the same as the passed in bytes
+ (e.g. when only verification is done), or they could be completely different
+ (perhaps if decryption were performed).
+
To send a configuration to the socket, read in the configuration file and
send it to the socket as a string of bytes preceded by a four-byte length
string packed in binary using ``struct.pack('>L', n)``.
@@ -126,7 +158,12 @@ in :mod:`logging` itself) and defining handlers which are declared either in
:func:`listen` socket and sending a configuration which runs whatever
code the attacker wants to have executed in the victim's process. This is
especially easy to do if the default port is used, but not hard even if a
- different port is used).
+ different port is used). To avoid the risk of this happening, use the
+ ``verify`` argument to :func:`listen` to prevent unrecognised
+ configurations from being applied.
+
+ .. versionchanged:: 3.4.
+ The ``verify`` argument was added.
.. function:: stopListening()
diff --git a/Doc/library/logging.handlers.rst b/Doc/library/logging.handlers.rst
index d0dc572..04d0f27 100644
--- a/Doc/library/logging.handlers.rst
+++ b/Doc/library/logging.handlers.rst
@@ -300,7 +300,7 @@ The :class:`TimedRotatingFileHandler` class, located in the
timed intervals.
-.. class:: TimedRotatingFileHandler(filename, when='h', interval=1, backupCount=0, encoding=None, delay=False, utc=False)
+.. class:: TimedRotatingFileHandler(filename, when='h', interval=1, backupCount=0, encoding=None, delay=False, utc=False, atTime=None)
Returns a new instance of the :class:`TimedRotatingFileHandler` class. The
specified file is opened and used as the stream for logging. On rotating it also
@@ -350,6 +350,12 @@ timed intervals.
If *delay* is true, then file opening is deferred until the first call to
:meth:`emit`.
+ If *atTime* is not ``None``, it must be a ``datetime.time`` instance which
+ specifies the time of day when rollover occurs, for the cases where rollover
+ is set to happen "at midnight" or "on a particular weekday".
+
+ .. versionchanged:: 3.4
+ *atTime* parameter was added.
.. method:: doRollover()
@@ -375,6 +381,9 @@ sends logging output to a network socket. The base class uses a TCP socket.
Returns a new instance of the :class:`SocketHandler` class intended to
communicate with a remote machine whose address is given by *host* and *port*.
+ .. versionchanged:: 3.4
+ If ``port`` is specified as ``None``, a Unix domain socket is created
+ using the value in ``host`` - otherwise, a TCP socket is created.
.. method:: close()
@@ -460,6 +469,9 @@ over UDP sockets.
Returns a new instance of the :class:`DatagramHandler` class intended to
communicate with a remote machine whose address is given by *host* and *port*.
+ .. versionchanged:: 3.4
+ If ``port`` is specified as ``None``, a Unix domain socket is created
+ using the value in ``host`` - otherwise, a TCP socket is created.
.. method:: emit()
diff --git a/Doc/library/lzma.rst b/Doc/library/lzma.rst
index 5fd5039..b201206 100644
--- a/Doc/library/lzma.rst
+++ b/Doc/library/lzma.rst
@@ -39,8 +39,8 @@ Reading and writing compressed files
opened, or it can be an existing file object to read from or write to.
The *mode* argument can be any of ``"r"``, ``"rb"``, ``"w"``, ``"wb"``,
- ``"a"`` or ``"ab"`` for binary mode, or ``"rt"``, ``"wt"``, or ``"at"`` for
- text mode. The default is ``"rb"``.
+ ``"x"``, ``"xb"``, ``"a"`` or ``"ab"`` for binary mode, or ``"rt"``,
+ ``"wt"``, ``"xt"``, or ``"at"`` for text mode. The default is ``"rb"``.
When opening a file for reading, the *format* and *filters* arguments have
the same meanings as for :class:`LZMADecompressor`. In this case, the *check*
@@ -57,6 +57,9 @@ Reading and writing compressed files
:class:`io.TextIOWrapper` instance with the specified encoding, error
handling behavior, and line ending(s).
+ .. versionchanged:: 3.4
+ Added support for the ``"x"``, ``"xb"`` and ``"xt"`` modes.
+
.. class:: LZMAFile(filename=None, mode="r", \*, format=None, check=-1, preset=None, filters=None)
@@ -69,8 +72,9 @@ Reading and writing compressed files
file will not be closed when the :class:`LZMAFile` is closed.
The *mode* argument can be either ``"r"`` for reading (default), ``"w"`` for
- overwriting, or ``"a"`` for appending. These can equivalently be given as
- ``"rb"``, ``"wb"``, and ``"ab"`` respectively.
+ overwriting, ``"x"`` for exclusive creation, or ``"a"`` for appending. These
+ can equivalently be given as ``"rb"``, ``"wb"``, ``"xb"`` and ``"ab"``
+ respectively.
If *filename* is a file object (rather than an actual file name), a mode of
``"w"`` does not truncate the file, and is instead equivalent to ``"a"``.
@@ -98,6 +102,9 @@ Reading and writing compressed files
byte of data will be returned, unless EOF has been reached. The exact
number of bytes returned is unspecified (the *size* argument is ignored).
+ .. versionchanged:: 3.4
+ Added support for the ``"x"`` and ``"xb"`` modes.
+
Compressing and decompressing data in memory
--------------------------------------------
diff --git a/Doc/library/marshal.rst b/Doc/library/marshal.rst
index 3b9e3d2..124eb61 100644
--- a/Doc/library/marshal.rst
+++ b/Doc/library/marshal.rst
@@ -40,10 +40,11 @@ this module. The following types are supported: booleans, integers, floating
point numbers, complex numbers, strings, bytes, bytearrays, tuples, lists, sets,
frozensets, dictionaries, and code objects, where it should be understood that
tuples, lists, sets, frozensets and dictionaries are only supported as long as
-the values contained therein are themselves supported; and recursive lists, sets
-and dictionaries should not be written (they will cause infinite loops). The
+the values contained therein are themselves supported.
singletons :const:`None`, :const:`Ellipsis` and :exc:`StopIteration` can also be
marshalled and unmarshalled.
+For format *version* lower than 3, recursive lists, sets and dictionaries cannot
+be written (see below).
There are functions that read/write files as well as functions operating on
strings.
@@ -103,7 +104,9 @@ In addition, the following constants are defined:
Indicates the format that the module uses. Version 0 is the historical
format, version 1 shares interned strings and version 2 uses a binary format
- for floating point numbers. The current version is 2.
+ for floating point numbers.
+ Version 3 adds support for object instancing and recursion.
+ The current version is 3.
.. rubric:: Footnotes
diff --git a/Doc/library/multiprocessing.rst b/Doc/library/multiprocessing.rst
index 3d1c209..2c44dc9 100644
--- a/Doc/library/multiprocessing.rst
+++ b/Doc/library/multiprocessing.rst
@@ -93,11 +93,108 @@ To show the individual process IDs involved, here is an expanded example::
p.start()
p.join()
-For an explanation of why (on Windows) the ``if __name__ == '__main__'`` part is
+For an explanation of why the ``if __name__ == '__main__'`` part is
necessary, see :ref:`multiprocessing-programming`.
+Contexts and start methods
+~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+Depending on the platform, :mod:`multiprocessing` supports three ways
+to start a process. These *start methods* are
+
+ *spawn*
+ The parent process starts a fresh python interpreter process. The
+ child process will only inherit those resources necessary to run
+ the process objects :meth:`~Process.run` method. In particular,
+ unnecessary file descriptors and handles from the parent process
+ will not be inherited. Starting a process using this method is
+ rather slow compared to using *fork* or *forkserver*.
+
+ Available on Unix and Windows. The default on Windows.
+
+ *fork*
+ The parent process uses :func:`os.fork` to fork the Python
+ interpreter. The child process, when it begins, is effectively
+ identical to the parent process. All resources of the parent are
+ inherited by the child process. Note that safely forking a
+ multithreaded process is problematic.
+
+ Available on Unix only. The default on Unix.
+
+ *forkserver*
+ When the program starts and selects the *forkserver* start method,
+ a server process is started. From then on, whenever a new process
+ is needed, the parent process connects to the server and requests
+ that it fork a new process. The fork server process is single
+ threaded so it is safe for it to use :func:`os.fork`. No
+ unnecessary resources are inherited.
+
+ Available on Unix platforms which support passing file descriptors
+ over Unix pipes.
+
+Before Python 3.4 *fork* was the only option available on Unix. Also,
+prior to Python 3.4, child processes would inherit all the parents
+inheritable handles on Windows.
+
+On Unix using the *spawn* or *forkserver* start methods will also
+start a *semaphore tracker* process which tracks the unlinked named
+semaphores created by processes of the program. When all processes
+have exited the semaphore tracker unlinks any remaining semaphores.
+Usually there should be none, but if a process was killed by a signal
+there may some "leaked" semaphores. (Unlinking the named semaphores
+is a serious matter since the system allows only a limited number, and
+they will not be automatically unlinked until the next reboot.)
+
+To select the a start method you use the :func:`set_start_method` in
+the ``if __name__ == '__main__'`` clause of the main module. For
+example::
+
+ import multiprocessing as mp
+
+ def foo(q):
+ q.put('hello')
+
+ if __name__ == '__main__':
+ mp.set_start_method('spawn')
+ q = mp.Queue()
+ p = mp.Process(target=foo, args=(q,))
+ p.start()
+ print(q.get())
+ p.join()
+
+:func:`set_start_method` should not be used more than once in the
+program.
+
+Alternatively, you can use :func:`get_context` to obtain a context
+object. Context objects have the same API as the multiprocessing
+module, and allow one to use multiple start methods in the same
+program. ::
+
+ import multiprocessing as mp
+
+ def foo(q):
+ q.put('hello')
+
+ if __name__ == '__main__':
+ ctx = mp.get_context('spawn')
+ q = ctx.Queue()
+ p = ctx.Process(target=foo, args=(q,))
+ p.start()
+ print(q.get())
+ p.join()
+
+Note that objects related to one context may not be compatible with
+processes for a different context. In particular, locks created using
+the *fork* context cannot be passed to a processes started using the
+*spawn* or *forkserver* start methods.
+
+A library which wants to use a particular start method should probably
+use :func:`get_context` to avoid interfering with the choice of the
+library user.
+
+
Exchanging objects between processes
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -274,15 +371,31 @@ processes in a few different ways.
For example::
from multiprocessing import Pool
+ from time import sleep
def f(x):
return x*x
if __name__ == '__main__':
- with Pool(processes=4) as pool: # start 4 worker processes
- result = pool.apply_async(f, [10]) # evaluate "f(10)" asynchronously
- print(result.get(timeout=1)) # prints "100" unless your computer is *very* slow
- print(pool.map(f, range(10))) # prints "[0, 1, 4,..., 81]"
+ # start 4 worker processes
+ with Pool(processes=4) as pool:
+
+ # print "[0, 1, 4,..., 81]"
+ print(pool.map(f, range(10)))
+
+ # print same numbers in arbitrary order
+ for i in pool.imap_unordered(f, range(10)):
+ print(i)
+
+ # evaluate "f(10)" asynchronously
+ res = pool.apply_async(f, [10])
+ print(res.get(timeout=1)) # prints "100"
+
+ # make worker sleep for 10 secs
+ res = pool.apply_async(sleep, 10)
+ print(res.get(timeout=1)) # raises multiprocessing.TimeoutError
+
+ # exiting the 'with'-block has stopped the pool
Note that the methods of a pool should only ever be used by the
process which created it.
@@ -731,6 +844,9 @@ Miscellaneous
Return the number of CPUs in the system. May raise
:exc:`NotImplementedError`.
+ .. seealso::
+ :func:`os.cpu_count`
+
.. function:: current_process()
Return the :class:`Process` object corresponding to the current process.
@@ -761,6 +877,43 @@ Miscellaneous
If the module is being run normally by the Python interpreter then
:func:`freeze_support` has no effect.
+.. function:: get_all_start_methods()
+
+ Returns a list of the supported start methods, the first of which
+ is the default. The possible start methods are ``'fork'``,
+ ``'spawn'`` and ``'forkserver'``. On Windows only ``'spawn'`` is
+ available. On Unix ``'fork'`` and ``'spawn'`` are always
+ supported, with ``'fork'`` being the default.
+
+ .. versionadded:: 3.4
+
+.. function:: get_context(method=None)
+
+ Return a context object which has the same attributes as the
+ :mod:`multiprocessing` module.
+
+ If *method* is *None* then the default context is returned.
+ Otherwise *method* should be ``'fork'``, ``'spawn'``,
+ ``'forkserver'``. :exc:`ValueError` is raised if the specified
+ start method is not available.
+
+ .. versionadded:: 3.4
+
+.. function:: get_start_method(allow_none=False)
+
+ Return the name of start method used for starting processes.
+
+ If the start method has not been fixed and *allow_none* is false,
+ then the start method is fixed to the default and the name is
+ returned. If the start method has not been fixed and *allow_none*
+ is true then *None* is returned.
+
+ The return value can be ``'fork'``, ``'spawn'``, ``'forkserver'``
+ or *None*. ``'fork'`` is the default on Unix, while ``'spawn'`` is
+ the default on Windows.
+
+ .. versionadded:: 3.4
+
.. function:: set_executable()
Sets the path of the Python interpreter to use when starting a child process.
@@ -769,8 +922,21 @@ Miscellaneous
set_executable(os.path.join(sys.exec_prefix, 'pythonw.exe'))
- before they can create child processes. (Windows only)
+ before they can create child processes.
+
+ .. versionchanged:: 3.4
+ Now supported on Unix when the ``'spawn'`` start method is used.
+
+.. function:: set_start_method(method)
+
+ Set the method which should be used to start child processes.
+ *method* can be ``'fork'``, ``'spawn'`` or ``'forkserver'``.
+
+ Note that this should be called at most once, and it should be
+ protected inside the ``if __name__ == '__main__'`` clause of the
+ main module.
+ .. versionadded:: 3.4
.. note::
@@ -1678,14 +1844,14 @@ Process Pools
One can create a pool of processes which will carry out tasks submitted to it
with the :class:`Pool` class.
-.. class:: Pool([processes[, initializer[, initargs[, maxtasksperchild]]]])
+.. class:: Pool([processes[, initializer[, initargs[, maxtasksperchild [, context]]]]])
A process pool object which controls a pool of worker processes to which jobs
can be submitted. It supports asynchronous results with timeouts and
callbacks and has a parallel map implementation.
*processes* is the number of worker processes to use. If *processes* is
- ``None`` then the number returned by :func:`cpu_count` is used. If
+ ``None`` then the number returned by :func:`os.cpu_count` is used. If
*initializer* is not ``None`` then each worker process will call
``initializer(*initargs)`` when it starts.
@@ -1698,6 +1864,13 @@ with the :class:`Pool` class.
unused resources to be freed. The default *maxtasksperchild* is None, which
means worker processes will live as long as the pool.
+ .. versionadded:: 3.4
+ *context* can be used to specify the context used for starting
+ the worker processes. Usually a pool is created using the
+ function :func:`multiprocessing.Pool` or the :meth:`Pool` method
+ of a context object. In both cases *context* is set
+ appropriately.
+
.. note::
Worker processes within a :class:`Pool` typically live for the complete
@@ -2189,43 +2362,8 @@ Below is an example session with logging turned on::
[INFO/MainProcess] sending shutdown message to manager
[INFO/SyncManager-...] manager exiting with exitcode 0
-In addition to having these two logging functions, the multiprocessing also
-exposes two additional logging level attributes. These are :const:`SUBWARNING`
-and :const:`SUBDEBUG`. The table below illustrates where theses fit in the
-normal level hierarchy.
-
-+----------------+----------------+
-| Level | Numeric value |
-+================+================+
-| ``SUBWARNING`` | 25 |
-+----------------+----------------+
-| ``SUBDEBUG`` | 5 |
-+----------------+----------------+
-
For a full table of logging levels, see the :mod:`logging` module.
-These additional logging levels are used primarily for certain debug messages
-within the multiprocessing module. Below is the same example as above, except
-with :const:`SUBDEBUG` enabled::
-
- >>> import multiprocessing, logging
- >>> logger = multiprocessing.log_to_stderr()
- >>> logger.setLevel(multiprocessing.SUBDEBUG)
- >>> logger.warning('doomed')
- [WARNING/MainProcess] doomed
- >>> m = multiprocessing.Manager()
- [INFO/SyncManager-...] child process calling self.run()
- [INFO/SyncManager-...] created temp directory /.../pymp-...
- [INFO/SyncManager-...] manager serving at '/.../pymp-djGBXN/listener-...'
- >>> del m
- [SUBDEBUG/MainProcess] finalizer calling ...
- [INFO/MainProcess] sending shutdown message to manager
- [DEBUG/SyncManager-...] manager received shutdown message
- [SUBDEBUG/SyncManager-...] calling <Finalize object, callback=unlink, ...
- [SUBDEBUG/SyncManager-...] finalizer calling <built-in function unlink> ...
- [SUBDEBUG/SyncManager-...] calling <Finalize object, dead>
- [SUBDEBUG/SyncManager-...] finalizer calling <function rmtree at 0x5aa730> ...
- [INFO/SyncManager-...] manager exiting with exitcode 0
The :mod:`multiprocessing.dummy` module
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
@@ -2246,8 +2384,10 @@ There are certain guidelines and idioms which should be adhered to when using
:mod:`multiprocessing`.
-All platforms
-~~~~~~~~~~~~~
+All start methods
+~~~~~~~~~~~~~~~~~
+
+The following applies to all start methods.
Avoid shared state
@@ -2281,11 +2421,13 @@ Joining zombie processes
Better to inherit than pickle/unpickle
- On Windows many types from :mod:`multiprocessing` need to be picklable so
- that child processes can use them. However, one should generally avoid
- sending shared objects to other processes using pipes or queues. Instead
- you should arrange the program so that a process which needs access to a
- shared resource created elsewhere can inherit it from an ancestor process.
+ When using the *spawn* or *forkserver* start methods many types
+ from :mod:`multiprocessing` need to be picklable so that child
+ processes can use them. However, one should generally avoid
+ sending shared objects to other processes using pipes or queues.
+ Instead you should arrange the program so that a process which
+ needs access to a shared resource created elsewhere can inherit it
+ from an ancestor process.
Avoid terminating processes
@@ -2332,15 +2474,17 @@ Joining processes that use queues
Explicitly pass resources to child processes
- On Unix a child process can make use of a shared resource created in a
- parent process using a global resource. However, it is better to pass the
- object as an argument to the constructor for the child process.
+ On Unix using the *fork* start method, a child process can make
+ use of a shared resource created in a parent process using a
+ global resource. However, it is better to pass the object as an
+ argument to the constructor for the child process.
- Apart from making the code (potentially) compatible with Windows this also
- ensures that as long as the child process is still alive the object will not
- be garbage collected in the parent process. This might be important if some
- resource is freed when the object is garbage collected in the parent
- process.
+ Apart from making the code (potentially) compatible with Windows
+ and the other start methods this also ensures that as long as the
+ child process is still alive the object will not be garbage
+ collected in the parent process. This might be important if some
+ resource is freed when the object is garbage collected in the
+ parent process.
So for instance ::
@@ -2399,17 +2543,19 @@ Beware of replacing :data:`sys.stdin` with a "file like object"
For more information, see :issue:`5155`, :issue:`5313` and :issue:`5331`
-Windows
-~~~~~~~
+The *spawn* and *forkserver* start methods
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-Since Windows lacks :func:`os.fork` it has a few extra restrictions:
+There are a few extra restriction which don't apply to the *fork*
+start method.
More picklability
- Ensure that all arguments to :meth:`Process.__init__` are picklable. This
- means, in particular, that bound or unbound methods cannot be used directly
- as the ``target`` argument on Windows --- just define a function and use
- that instead.
+ Ensure that all arguments to :meth:`Process.__init__` are
+ picklable. This means, in particular, that bound or unbound
+ methods cannot be used directly as the ``target`` (unless you use
+ the *fork* start method) --- just define a function and use that
+ instead.
Also, if you subclass :class:`~multiprocessing.Process` then make sure that
instances will be picklable when the :meth:`Process.start
@@ -2431,7 +2577,8 @@ Safe importing of main module
interpreter without causing unintended side effects (such a starting a new
process).
- For example, under Windows running the following module would fail with a
+ For example, using the *spawn* or *forkserver* start method
+ running the following module would fail with a
:exc:`RuntimeError`::
from multiprocessing import Process
@@ -2445,13 +2592,14 @@ Safe importing of main module
Instead one should protect the "entry point" of the program by using ``if
__name__ == '__main__':`` as follows::
- from multiprocessing import Process, freeze_support
+ from multiprocessing import Process, freeze_support, set_start_method
def foo():
print('hello')
if __name__ == '__main__':
freeze_support()
+ set_start_method('spawn')
p = Process(target=foo)
p.start()
@@ -2482,26 +2630,7 @@ Using :class:`~multiprocessing.pool.Pool`:
:language: python3
-Synchronization types like locks, conditions and queues:
-
-.. literalinclude:: ../includes/mp_synchronize.py
- :language: python3
-
-
An example showing how to use queues to feed tasks to a collection of worker
processes and collect the results:
.. literalinclude:: ../includes/mp_workers.py
-
-
-An example of how a pool of worker processes can each run a
-:class:`~http.server.SimpleHTTPRequestHandler` instance while sharing a single
-listening socket.
-
-.. literalinclude:: ../includes/mp_webserver.py
-
-
-Some simple benchmarks comparing :mod:`multiprocessing` with :mod:`threading`:
-
-.. literalinclude:: ../includes/mp_benchmarks.py
-
diff --git a/Doc/library/netrc.rst b/Doc/library/netrc.rst
index 564f101..23ffed6 100644
--- a/Doc/library/netrc.rst
+++ b/Doc/library/netrc.rst
@@ -29,7 +29,7 @@ the Unix :program:`ftp` program and other FTP clients.
This implements security behavior equivalent to that of ftp and other
programs that use :file:`.netrc`.
- .. versionchanged:: 3.3.3 Added the POSIX permission check.
+ .. versionchanged:: 3.4 Added the POSIX permission check.
.. exception:: NetrcParseError
diff --git a/Doc/library/nntplib.rst b/Doc/library/nntplib.rst
index 759d3b7..9ab2c94 100644
--- a/Doc/library/nntplib.rst
+++ b/Doc/library/nntplib.rst
@@ -1,4 +1,3 @@
-
:mod:`nntplib` --- NNTP protocol client
=======================================
@@ -71,7 +70,7 @@ The module itself defines the following classes:
reader-specific commands, such as ``group``. If you get unexpected
:exc:`NNTPPermanentError`\ s, you might need to set *readermode*.
:class:`NNTP` class supports the :keyword:`with` statement to
- unconditionally consume :exc:`socket.error` exceptions and to close the NNTP
+ unconditionally consume :exc:`OSError` exceptions and to close the NNTP
connection when done. Here is a sample on how using it:
>>> from nntplib import NNTP
diff --git a/Doc/library/numeric.rst b/Doc/library/numeric.rst
index 2732a84..7c76a47 100644
--- a/Doc/library/numeric.rst
+++ b/Doc/library/numeric.rst
@@ -23,3 +23,4 @@ The following modules are documented in this chapter:
decimal.rst
fractions.rst
random.rst
+ statistics.rst
diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst
index 03547ca..80bad71 100644
--- a/Doc/library/operator.rst
+++ b/Doc/library/operator.rst
@@ -11,6 +11,9 @@
import operator
from operator import itemgetter, iadd
+**Source code:** :source:`Lib/operator.py`
+
+--------------
The :mod:`operator` module exports a set of efficient functions corresponding to
the intrinsic operators of Python. For example, ``operator.add(x, y)`` is
@@ -235,6 +238,14 @@ their character equivalents.
.. XXX: find a better, readable, example
+.. function:: length_hint(obj, default=0)
+
+ Return an estimated length for the object *o*. First trying to return its
+ actual length, then an estimate using :meth:`object.__length_hint__`, and
+ finally returning the default value.
+
+ .. versionadded:: 3.4
+
The :mod:`operator` module also defines tools for generalized attribute and item
lookups. These are useful for making fast field extractors as arguments for
:func:`map`, :func:`sorted`, :meth:`itertools.groupby`, or other functions that
diff --git a/Doc/library/os.path.rst b/Doc/library/os.path.rst
index 36f2d14..03350d7 100644
--- a/Doc/library/os.path.rst
+++ b/Doc/library/os.path.rst
@@ -22,6 +22,11 @@ Functions such as :func:`expanduser` and :func:`expandvars` can be invoked
explicitly when an application desires shell-like path expansion. (See also
the :mod:`glob` module.)
+
+.. seealso::
+ The :mod:`pathlib` module offers high-level path objects.
+
+
.. note::
All of these functions accept either only bytes or only string objects as
@@ -42,7 +47,6 @@ the :mod:`glob` module.)
* :mod:`posixpath` for UNIX-style paths
* :mod:`ntpath` for Windows paths
* :mod:`macpath` for old-style MacOS paths
- * :mod:`os2emxpath` for OS/2 EMX paths
.. function:: abspath(path)
@@ -250,15 +254,14 @@ the :mod:`glob` module.)
On Unix, this is determined by the device number and i-node number and raises an
exception if a :func:`os.stat` call on either pathname fails.
- On Windows, two files are the same if they resolve to the same final path
- name using the Windows API call GetFinalPathNameByHandle. This function
- raises an exception if handles cannot be obtained to either file.
-
Availability: Unix, Windows.
.. versionchanged:: 3.2
Added Windows support.
+ .. versionchanged:: 3.4
+ Windows now uses the same implementation as all other platforms.
+
.. function:: sameopenfile(fp1, fp2)
@@ -277,7 +280,10 @@ the :mod:`glob` module.)
:func:`os.lstat`, or :func:`os.stat`. This function implements the
underlying comparison used by :func:`samefile` and :func:`sameopenfile`.
- Availability: Unix.
+ Availability: Unix, Windows.
+
+ .. versionchanged:: 3.4
+ Added Windows support.
.. function:: split(path)
diff --git a/Doc/library/os.rst b/Doc/library/os.rst
index 2c33366..67f94ac 100644
--- a/Doc/library/os.rst
+++ b/Doc/library/os.rst
@@ -54,7 +54,7 @@ Notes on the availability of these functions:
The name of the operating system dependent module imported. The following
names have currently been registered: ``'posix'``, ``'nt'``, ``'mac'``,
- ``'os2'``, ``'ce'``, ``'java'``.
+ ``'ce'``, ``'java'``.
.. seealso::
:attr:`sys.platform` has a finer granularity. :func:`os.uname` gives
@@ -685,17 +685,30 @@ as internal buffering of data.
.. function:: dup(fd)
- Return a duplicate of file descriptor *fd*.
+ Return a duplicate of file descriptor *fd*. The new file descriptor is
+ :ref:`non-inheritable <fd_inheritance>`.
+
+ On Windows, when duplicating a standard stream (0: stdin, 1: stdout,
+ 2: stderr), the new file descriptor is :ref:`inheritable
+ <fd_inheritance>`.
Availability: Unix, Windows.
+ .. versionchanged:: 3.4
+ The new file descriptor is now non-inheritable.
+
-.. function:: dup2(fd, fd2)
+.. function:: dup2(fd, fd2, inheritable=True)
Duplicate file descriptor *fd* to *fd2*, closing the latter first if necessary.
+ The file descriptor *fd2* is :ref:`inheritable <fd_inheritance>` by default,
+ or non-inheritable if *inheritable* is ``False``.
Availability: Unix, Windows.
+ .. versionchanged:: 3.4
+ Add the optional *inheritable* parameter.
+
.. function:: fchmod(fd, mode)
@@ -846,17 +859,21 @@ as internal buffering of data.
Open the file *file* and set various flags according to *flags* and possibly
its mode according to *mode*. When computing *mode*, the current umask value
is first masked out. Return the file descriptor for the newly opened file.
+ The new file descriptor is :ref:`non-inheritable <fd_inheritance>`.
For a description of the flag and mode values, see the C run-time documentation;
flag constants (like :const:`O_RDONLY` and :const:`O_WRONLY`) are defined in
- this module too (see :ref:`open-constants`). In particular, on Windows adding
+ the :mod:`os` module. In particular, on Windows adding
:const:`O_BINARY` is needed to open files in binary mode.
This function can support :ref:`paths relative to directory descriptors
- <dir_fd>`.
+ <dir_fd>` with the *dir_fd* parameter.
Availability: Unix, Windows.
+ .. versionchanged:: 3.4
+ The new file descriptor is now non-inheritable.
+
.. note::
This function is intended for low-level I/O. For normal usage, use the
@@ -867,25 +884,92 @@ as internal buffering of data.
.. versionadded:: 3.3
The *dir_fd* argument.
+The following constants are options for the *flags* parameter to the
+:func:`~os.open` function. They can be combined using the bitwise OR operator
+``|``. Some of them are not available on all platforms. For descriptions of
+their availability and use, consult the :manpage:`open(2)` manual page on Unix
+or `the MSDN <http://msdn.microsoft.com/en-us/library/z0kc8e3z.aspx>`_ on Windows.
+
+
+.. data:: O_RDONLY
+ O_WRONLY
+ O_RDWR
+ O_APPEND
+ O_CREAT
+ O_EXCL
+ O_TRUNC
+
+ These constants are available on Unix and Windows.
+
+
+.. data:: O_DSYNC
+ O_RSYNC
+ O_SYNC
+ O_NDELAY
+ O_NONBLOCK
+ O_NOCTTY
+ O_SHLOCK
+ O_EXLOCK
+ O_CLOEXEC
+
+ These constants are only available on Unix.
+
+ .. versionchanged:: 3.3
+ Add :data:`O_CLOEXEC` constant.
+
+.. data:: O_BINARY
+ O_NOINHERIT
+ O_SHORT_LIVED
+ O_TEMPORARY
+ O_RANDOM
+ O_SEQUENTIAL
+ O_TEXT
+
+ These constants are only available on Windows.
+
+
+.. data:: O_ASYNC
+ O_DIRECT
+ O_DIRECTORY
+ O_NOFOLLOW
+ O_NOATIME
+ O_PATH
+ O_TMPFILE
+
+ These constants are GNU extensions and not present if they are not defined by
+ the C library.
+
+ .. versionchanged:: 3.4
+ Add :data:`O_TMPFILE` constant. It's only available on Linux Kernel 3.11
+ or newer.
+
.. function:: openpty()
.. index:: module: pty
- Open a new pseudo-terminal pair. Return a pair of file descriptors ``(master,
- slave)`` for the pty and the tty, respectively. For a (slightly) more portable
- approach, use the :mod:`pty` module.
+ Open a new pseudo-terminal pair. Return a pair of file descriptors
+ ``(master, slave)`` for the pty and the tty, respectively. The new file
+ descriptors are :ref:`non-inheritable <fd_inheritance>`. For a (slightly) more
+ portable approach, use the :mod:`pty` module.
Availability: some flavors of Unix.
+ .. versionchanged:: 3.4
+ The new file descriptors are now non-inheritable.
+
.. function:: pipe()
- Create a pipe. Return a pair of file descriptors ``(r, w)`` usable for reading
- and writing, respectively.
+ Create a pipe. Return a pair of file descriptors ``(r, w)`` usable for
+ reading and writing, respectively. The new file descriptor is
+ :ref:`non-inheritable <fd_inheritance>`.
Availability: Unix, Windows.
+ .. versionchanged:: 3.4
+ The new file descriptors are now non-inheritable.
+
.. function:: pipe2(flags)
@@ -1080,78 +1164,6 @@ as internal buffering of data.
.. versionadded:: 3.3
-.. _open-constants:
-
-``open()`` flag constants
-~~~~~~~~~~~~~~~~~~~~~~~~~
-
-The following constants are options for the *flags* parameter to the
-:func:`~os.open` function. They can be combined using the bitwise OR operator
-``|``. Some of them are not available on all platforms. For descriptions of
-their availability and use, consult the :manpage:`open(2)` manual page on Unix
-or `the MSDN <http://msdn.microsoft.com/en-us/library/z0kc8e3z.aspx>`_ on Windows.
-
-
-.. data:: O_RDONLY
- O_WRONLY
- O_RDWR
- O_APPEND
- O_CREAT
- O_EXCL
- O_TRUNC
-
- These constants are available on Unix and Windows.
-
-
-.. data:: O_DSYNC
- O_RSYNC
- O_SYNC
- O_NDELAY
- O_NONBLOCK
- O_NOCTTY
- O_SHLOCK
- O_EXLOCK
- O_CLOEXEC
-
- These constants are only available on Unix.
-
- .. versionchanged:: 3.3
- Add :data:`O_CLOEXEC` constant.
-
-.. data:: O_BINARY
- O_NOINHERIT
- O_SHORT_LIVED
- O_TEMPORARY
- O_RANDOM
- O_SEQUENTIAL
- O_TEXT
-
- These constants are only available on Windows.
-
-
-.. data:: O_ASYNC
- O_DIRECT
- O_DIRECTORY
- O_NOFOLLOW
- O_NOATIME
-
- These constants are GNU extensions and not present if they are not defined by
- the C library.
-
-
-.. data:: RTLD_LAZY
- RTLD_NOW
- RTLD_GLOBAL
- RTLD_LOCAL
- RTLD_NODELETE
- RTLD_NOLOAD
- RTLD_DEEPBIND
-
- See the Unix manual page :manpage:`dlopen(3)`.
-
- .. versionadded:: 3.3
-
-
.. _terminal-size:
Querying the size of a terminal
@@ -1189,6 +1201,49 @@ Querying the size of a terminal
Height of the terminal window in characters.
+.. _fd_inheritance:
+
+Inheritance of File Descriptors
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+.. versionadded:: 3.4
+
+A file descriptor has an "inheritable" flag which indicates if the file descriptor
+can be inherited by child processes. Since Python 3.4, file descriptors
+created by Python are non-inheritable by default.
+
+On UNIX, non-inheritable file descriptors are closed in child processes at the
+execution of a new program, other file descriptors are inherited.
+
+On Windows, non-inheritable handles and file descriptors are closed in child
+processes, except for standard streams (file descriptors 0, 1 and 2: stdin, stdout
+and stderr), which are always inherited. Using :func:`spawn\* <spawnl>` functions,
+all inheritable handles and all inheritable file descriptors are inherited.
+Using the :mod:`subprocess` module, all file descriptors except standard
+streams are closed, and inheritable handles are only inherited if the
+*close_fds* parameter is ``False``.
+
+.. function:: get_inheritable(fd)
+
+ Get the "inheritable" flag of the specified file descriptor (a boolean).
+
+.. function:: set_inheritable(fd, inheritable)
+
+ Set the "inheritable" flag of the specified file descriptor.
+
+.. function:: get_handle_inheritable(handle)
+
+ Get the "inheritable" flag of the specified handle (a boolean).
+
+ Availability: Windows.
+
+.. function:: set_handle_inheritable(handle, inheritable)
+
+ Set the "inheritable" flag of the specified handle.
+
+ Availability: Windows.
+
+
.. _os-file-dir:
Files and Directories
@@ -3047,7 +3102,7 @@ information, consult your Unix manpages.
.. versionadded:: 3.3
-The following scheduling policies are exposed if they are a supported by the
+The following scheduling policies are exposed if they are supported by the
operating system.
.. data:: SCHED_OTHER
@@ -3156,10 +3211,6 @@ operating system.
Return the set of CPUs the process with PID *pid* (or the current process
if zero) is restricted to.
- .. seealso::
- :func:`multiprocessing.cpu_count` returns the number of CPUs in the
- system.
-
.. _os-path:
@@ -3197,6 +3248,13 @@ Miscellaneous System Information
Availability: Unix.
+.. function:: cpu_count()
+
+ Return the number of CPUs in the system. Returns None if undetermined.
+
+ .. versionadded:: 3.4
+
+
.. function:: getloadavg()
Return the number of processes in the system run queue averaged over the last
@@ -3295,6 +3353,19 @@ Higher-level operations on pathnames are defined in the :mod:`os.path` module.
The file path of the null device. For example: ``'/dev/null'`` for
POSIX, ``'nul'`` for Windows. Also available via :mod:`os.path`.
+.. data:: RTLD_LAZY
+ RTLD_NOW
+ RTLD_GLOBAL
+ RTLD_LOCAL
+ RTLD_NODELETE
+ RTLD_NOLOAD
+ RTLD_DEEPBIND
+
+ Flags for use with the :func:`~sys.setdlopenflags` and
+ :func:`~sys.getdlopenflags` functions. See the Unix manual page
+ :manpage:`dlopen(3)` for what the different flags mean.
+
+ .. versionadded:: 3.3
.. _os-miscfunc:
diff --git a/Doc/library/pathlib.rst b/Doc/library/pathlib.rst
new file mode 100644
index 0000000..7582315
--- /dev/null
+++ b/Doc/library/pathlib.rst
@@ -0,0 +1,873 @@
+
+:mod:`pathlib` --- Object-oriented filesystem paths
+===================================================
+
+.. module:: pathlib
+ :synopsis: Object-oriented filesystem paths
+
+.. index:: single: path; operations
+
+.. versionadded:: 3.4
+
+
+This module offers classes representing filesystem paths with semantics
+appropriate for different operating systems. Path classes are divided
+between :ref:`pure paths <pure-paths>`, which provide purely computational
+operations without I/O, and :ref:`concrete paths <concrete-paths>`, which
+inherit from pure paths but also provide I/O operations.
+
+The main point of entry is the :class:`Path` class, which will instantiate
+a :ref:`concrete path <concrete-paths>` for the current platform.
+
+.. note::
+ This module has been included in the standard library on a
+ :term:`provisional basis <provisional package>`. Backwards incompatible
+ changes (up to and including removal of the package) may occur if deemed
+ necessary by the core developers.
+
+.. seealso::
+ :pep:`428`: The pathlib module -- object-oriented filesystem paths.
+
+.. seealso::
+ For low-level path manipulation on strings, you can also use the
+ :mod:`os.path` module.
+
+
+Basic use
+---------
+
+Importing the main class::
+
+ >>> from pathlib import Path
+
+Listing subdirectories::
+
+ >>> p = Path('.')
+ >>> [x for x in p.iterdir() if x.is_dir()]
+ [PosixPath('.hg'), PosixPath('docs'), PosixPath('dist'),
+ PosixPath('__pycache__'), PosixPath('build')]
+
+Listing Python source files in this directory tree::
+
+ >>> list(p.glob('**/*.py'))
+ [PosixPath('test_pathlib.py'), PosixPath('setup.py'),
+ PosixPath('pathlib.py'), PosixPath('docs/conf.py'),
+ PosixPath('build/lib/pathlib.py')]
+
+Navigating inside a directory tree::
+
+ >>> p = Path('/etc')
+ >>> q = p / 'init.d' / 'reboot'
+ >>> q
+ PosixPath('/etc/init.d/reboot')
+ >>> q.resolve()
+ PosixPath('/etc/rc.d/init.d/halt')
+
+Querying path properties::
+
+ >>> q.exists()
+ True
+ >>> q.is_dir()
+ False
+
+Opening a file::
+
+ >>> with q.open() as f: f.readline()
+ ...
+ '#!/bin/bash\n'
+
+
+.. _pure-paths:
+
+Pure paths
+----------
+
+Pure path objects provide path-handling operations which don't actually
+access a filesystem. There are three ways to access these classes, which
+we also call *flavours*:
+
+
+.. class:: PurePosixPath
+
+ A subclass of :class:`PurePath`, this path flavour represents non-Windows
+ filesystem paths::
+
+ >>> PurePosixPath('/etc')
+ PurePosixPath('/etc')
+
+.. class:: PureWindowsPath
+
+ A subclass of :class:`PurePath`, this path flavour represents Windows
+ filesystem paths::
+
+ >>> PureWindowsPath('c:/Program Files/')
+ PureWindowsPath('c:/Program Files')
+
+.. class:: PurePath
+
+ A generic class that represents the system's path flavour (instantiating
+ it creates either a :class:`PurePosixPath` or a :class:`PureWindowsPath`)::
+
+ >>> PurePath('setup.py')
+ PurePosixPath('setup.py')
+
+
+Regardless of the system you're running on, you can instantiate all of
+these classes, since they don't provide any operation that does system calls.
+
+
+Constructing paths
+^^^^^^^^^^^^^^^^^^
+
+Path constructors accept an arbitrary number of positional arguments.
+When called without any argument, a path object points to the current
+directory::
+
+ >>> PurePath()
+ PurePosixPath('.')
+
+Any argument can be a string or bytes object representing an arbitrary number
+of path segments, but it can also be another path object::
+
+ >>> PurePath('foo', 'some/path', 'bar')
+ PurePosixPath('foo/some/path/bar')
+ >>> PurePath(Path('foo'), Path('bar'))
+ PurePosixPath('foo/bar')
+
+When several absolute paths are given, the last is taken as an anchor
+(mimicking :func:`os.path.join`'s behaviour)::
+
+ >>> PurePath('/etc', '/usr', 'lib64')
+ PurePosixPath('/usr/lib64')
+ >>> PureWindowsPath('c:/Windows', 'd:bar')
+ PureWindowsPath('d:bar')
+
+However, in a Windows path, changing the local root doesn't discard the
+previous drive setting::
+
+ >>> PureWindowsPath('c:/Windows', '/Program Files')
+ PureWindowsPath('c:/Program Files')
+
+Spurious slashes and single dots are collapsed, but double dots (``'..'``)
+are not, since this would change the meaning of a path in the face of
+symbolic links::
+
+ >>> PurePath('foo//bar')
+ PurePosixPath('foo/bar')
+ >>> PurePath('foo/./bar')
+ PurePosixPath('foo/bar')
+ >>> PurePath('foo/../bar')
+ PurePosixPath('foo/../bar')
+
+(a naïve approach would make ``PurePosixPath('foo/../bar')`` equivalent
+to ``PurePosixPath('bar')``, which is wrong if ``foo`` is a symbolic link
+to another directory)
+
+
+General properties
+^^^^^^^^^^^^^^^^^^
+
+Paths are immutable and hashable. Paths of a same flavour are comparable
+and orderable. These properties respect the flavour's case-folding
+semantics::
+
+ >>> PurePosixPath('foo') == PurePosixPath('FOO')
+ False
+ >>> PureWindowsPath('foo') == PureWindowsPath('FOO')
+ True
+ >>> PureWindowsPath('FOO') in { PureWindowsPath('foo') }
+ True
+ >>> PureWindowsPath('C:') < PureWindowsPath('d:')
+ True
+
+Paths of a different flavour compare unequal and cannot be ordered::
+
+ >>> PureWindowsPath('foo') == PurePosixPath('foo')
+ False
+ >>> PureWindowsPath('foo') < PurePosixPath('foo')
+ Traceback (most recent call last):
+ File "<stdin>", line 1, in <module>
+ TypeError: unorderable types: PureWindowsPath() < PurePosixPath()
+
+
+Operators
+^^^^^^^^^
+
+The slash operator helps create child paths, similarly to :func:`os.path.join`::
+
+ >>> p = PurePath('/etc')
+ >>> p
+ PurePosixPath('/etc')
+ >>> p / 'init.d' / 'apache2'
+ PurePosixPath('/etc/init.d/apache2')
+ >>> q = PurePath('bin')
+ >>> '/usr' / q
+ PurePosixPath('/usr/bin')
+
+The string representation of a path is the raw filesystem path itself
+(in native form, e.g. with backslashes under Windows), which you can
+pass to any function taking a file path as a string::
+
+ >>> p = PurePath('/etc')
+ >>> str(p)
+ '/etc'
+ >>> p = PureWindowsPath('c:/Program Files')
+ >>> str(p)
+ 'c:\\Program Files'
+
+Similarly, calling :class:`bytes` on a path gives the raw filesystem path as a
+bytes object, as encoded by :func:`os.fsencode`::
+
+ >>> bytes(p)
+ b'/etc'
+
+.. note::
+ Calling :class:`bytes` is only recommended under Unix. Under Windows,
+ the unicode form is the canonical representation of filesystem paths.
+
+
+Accessing individual parts
+^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+To access the individual "parts" (components) of a path, use the following
+property:
+
+.. data:: PurePath.parts
+
+ A tuple giving access to the path's various components::
+
+ >>> p = PurePath('/usr/bin/python3')
+ >>> p.parts
+ ('/', 'usr', 'bin', 'python3')
+
+ >>> p = PureWindowsPath('c:/Program Files/PSF')
+ >>> p.parts
+ ('c:\\', 'Program Files', 'PSF')
+
+ (note how the drive and local root are regrouped in a single part)
+
+
+Methods and properties
+^^^^^^^^^^^^^^^^^^^^^^
+
+Pure paths provide the following methods and properties:
+
+.. data:: PurePath.drive
+
+ A string representing the drive letter or name, if any::
+
+ >>> PureWindowsPath('c:/Program Files/').drive
+ 'c:'
+ >>> PureWindowsPath('/Program Files/').drive
+ ''
+ >>> PurePosixPath('/etc').drive
+ ''
+
+ UNC shares are also considered drives::
+
+ >>> PureWindowsPath('//host/share/foo.txt').drive
+ '\\\\host\\share'
+
+.. data:: PurePath.root
+
+ A string representing the (local or global) root, if any::
+
+ >>> PureWindowsPath('c:/Program Files/').root
+ '\\'
+ >>> PureWindowsPath('c:Program Files/').root
+ ''
+ >>> PurePosixPath('/etc').root
+ '/'
+
+ UNC shares always have a root::
+
+ >>> PureWindowsPath('//host/share').root
+ '\\'
+
+.. data:: PurePath.anchor
+
+ The concatenation of the drive and root::
+
+ >>> PureWindowsPath('c:/Program Files/').anchor
+ 'c:\\'
+ >>> PureWindowsPath('c:Program Files/').anchor
+ 'c:'
+ >>> PurePosixPath('/etc').anchor
+ '/'
+ >>> PureWindowsPath('//host/share').anchor
+ '\\\\host\\share\\'
+
+
+.. data:: PurePath.parents
+
+ An immutable sequence providing access to the logical ancestors of
+ the path::
+
+ >>> p = PureWindowsPath('c:/foo/bar/setup.py')
+ >>> p.parents[0]
+ PureWindowsPath('c:/foo/bar')
+ >>> p.parents[1]
+ PureWindowsPath('c:/foo')
+ >>> p.parents[2]
+ PureWindowsPath('c:/')
+
+
+.. data:: PurePath.parent
+
+ The logical parent of the path::
+
+ >>> p = PurePosixPath('/a/b/c/d')
+ >>> p.parent
+ PurePosixPath('/a/b/c')
+
+ You cannot go past an anchor, or empty path::
+
+ >>> p = PurePosixPath('/')
+ >>> p.parent
+ PurePosixPath('/')
+ >>> p = PurePosixPath('.')
+ >>> p.parent
+ PurePosixPath('.')
+
+ .. note::
+ This is a purely lexical operation, hence the following behaviour::
+
+ >>> p = PurePosixPath('foo/..')
+ >>> p.parent
+ PurePosixPath('foo')
+
+ If you want to walk an arbitrary filesystem path upwards, it is
+ recommended to first call :meth:`Path.resolve` so as to resolve
+ symlinks and eliminate `".."` components.
+
+
+.. data:: PurePath.name
+
+ A string representing the final path component, excluding the drive and
+ root, if any::
+
+ >>> PurePosixPath('my/library/setup.py').name
+ 'setup.py'
+
+ UNC drive names are not considered::
+
+ >>> PureWindowsPath('//some/share/setup.py').name
+ 'setup.py'
+ >>> PureWindowsPath('//some/share').name
+ ''
+
+
+.. data:: PurePath.suffix
+
+ The file extension of the final component, if any::
+
+ >>> PurePosixPath('my/library/setup.py').suffix
+ '.py'
+ >>> PurePosixPath('my/library.tar.gz').suffix
+ '.gz'
+ >>> PurePosixPath('my/library').suffix
+ ''
+
+
+.. data:: PurePath.suffixes
+
+ A list of the path's file extensions::
+
+ >>> PurePosixPath('my/library.tar.gar').suffixes
+ ['.tar', '.gar']
+ >>> PurePosixPath('my/library.tar.gz').suffixes
+ ['.tar', '.gz']
+ >>> PurePosixPath('my/library').suffixes
+ []
+
+
+.. data:: PurePath.stem
+
+ The final path component, without its suffix::
+
+ >>> PurePosixPath('my/library.tar.gz').stem
+ 'library.tar'
+ >>> PurePosixPath('my/library.tar').stem
+ 'library'
+ >>> PurePosixPath('my/library').stem
+ 'library'
+
+
+.. method:: PurePath.as_posix()
+
+ Return a string representation of the path with forward slashes (``/``)::
+
+ >>> p = PureWindowsPath('c:\\windows')
+ >>> str(p)
+ 'c:\\windows'
+ >>> p.as_posix()
+ 'c:/windows'
+
+
+.. method:: PurePath.as_uri()
+
+ Represent the path as a ``file`` URI. :exc:`ValueError` is raised if
+ the path isn't absolute.
+
+ >>> p = PurePosixPath('/etc/passwd')
+ >>> p.as_uri()
+ 'file:///etc/passwd'
+ >>> p = PureWindowsPath('c:/Windows')
+ >>> p.as_uri()
+ 'file:///c:/Windows'
+
+
+.. method:: PurePath.is_absolute()
+
+ Return whether the path is absolute or not. A path is considered absolute
+ if it has both a root and (if the flavour allows) a drive::
+
+ >>> PurePosixPath('/a/b').is_absolute()
+ True
+ >>> PurePosixPath('a/b').is_absolute()
+ False
+
+ >>> PureWindowsPath('c:/a/b').is_absolute()
+ True
+ >>> PureWindowsPath('/a/b').is_absolute()
+ False
+ >>> PureWindowsPath('c:').is_absolute()
+ False
+ >>> PureWindowsPath('//some/share').is_absolute()
+ True
+
+
+.. method:: PurePath.is_reserved()
+
+ With :class:`PureWindowsPath`, return True if the path is considered
+ reserved under Windows, False otherwise. With :class:`PurePosixPath`,
+ False is always returned.
+
+ >>> PureWindowsPath('nul').is_reserved()
+ True
+ >>> PurePosixPath('nul').is_reserved()
+ False
+
+ File system calls on reserved paths can fail mysteriously or have
+ unintended effects.
+
+
+.. method:: PurePath.joinpath(*other)
+
+ Calling this method is equivalent to combining the path with each of
+ the *other* arguments in turn::
+
+ >>> PurePosixPath('/etc').joinpath('passwd')
+ PurePosixPath('/etc/passwd')
+ >>> PurePosixPath('/etc').joinpath(PurePosixPath('passwd'))
+ PurePosixPath('/etc/passwd')
+ >>> PurePosixPath('/etc').joinpath('init.d', 'apache2')
+ PurePosixPath('/etc/init.d/apache2')
+ >>> PureWindowsPath('c:').joinpath('/Program Files')
+ PureWindowsPath('c:/Program Files')
+
+
+.. method:: PurePath.match(pattern)
+
+ Match this path against the provided glob-style pattern. Return True
+ if matching is successful, False otherwise.
+
+ If *pattern* is relative, the path can be either relative or absolute,
+ and matching is done from the right::
+
+ >>> PurePath('a/b.py').match('*.py')
+ True
+ >>> PurePath('/a/b/c.py').match('b/*.py')
+ True
+ >>> PurePath('/a/b/c.py').match('a/*.py')
+ False
+
+ If *pattern* is absolute, the path must be absolute, and the whole path
+ must match::
+
+ >>> PurePath('/a.py').match('/*.py')
+ True
+ >>> PurePath('a/b.py').match('/*.py')
+ False
+
+ As with other methods, case-sensitivity is observed::
+
+ >>> PureWindowsPath('b.py').match('*.PY')
+ True
+
+
+.. method:: PurePath.relative_to(*other)
+
+ Compute a version of this path relative to the path represented by
+ *other*. If it's impossible, ValueError is raised::
+
+ >>> p = PurePosixPath('/etc/passwd')
+ >>> p.relative_to('/')
+ PurePosixPath('etc/passwd')
+ >>> p.relative_to('/etc')
+ PurePosixPath('passwd')
+ >>> p.relative_to('/usr')
+ Traceback (most recent call last):
+ File "<stdin>", line 1, in <module>
+ File "pathlib.py", line 694, in relative_to
+ .format(str(self), str(formatted)))
+ ValueError: '/etc/passwd' does not start with '/usr'
+
+
+.. _concrete-paths:
+
+
+Concrete paths
+--------------
+
+Concrete paths are subclasses of the pure path classes. In addition to
+operations provided by the latter, they also provide methods to do system
+calls on path objects. There are three ways to instantiate concrete paths:
+
+
+.. class:: PosixPath
+
+ A subclass of :class:`Path` and :class:`PurePosixPath`, this class
+ represents concrete non-Windows filesystem paths::
+
+ >>> PosixPath('/etc')
+ PosixPath('/etc')
+
+.. class:: WindowsPath
+
+ A subclass of :class:`Path` and :class:`PureWindowsPath`, this class
+ represents concrete Windows filesystem paths::
+
+ >>> WindowsPath('c:/Program Files/')
+ WindowsPath('c:/Program Files')
+
+.. class:: Path
+
+ A subclass of :class:`PurePath`, this class represents concrete paths of
+ the system's path flavour (instantiating it creates either a
+ :class:`PosixPath` or a :class:`WindowsPath`)::
+
+ >>> Path('setup.py')
+ PosixPath('setup.py')
+
+
+You can only instantiate the class flavour that corresponds to your system
+(allowing system calls on non-compatible path flavours could lead to
+bugs or failures in your application)::
+
+ >>> import os
+ >>> os.name
+ 'posix'
+ >>> Path('setup.py')
+ PosixPath('setup.py')
+ >>> PosixPath('setup.py')
+ PosixPath('setup.py')
+ >>> WindowsPath('setup.py')
+ Traceback (most recent call last):
+ File "<stdin>", line 1, in <module>
+ File "pathlib.py", line 798, in __new__
+ % (cls.__name__,))
+ NotImplementedError: cannot instantiate 'WindowsPath' on your system
+
+
+Methods
+^^^^^^^
+
+Concrete paths provide the following methods in addition to pure paths
+methods. Many of these methods can raise an :exc:`OSError` if a system
+call fails (for example because the path doesn't exist):
+
+.. classmethod:: Path.cwd()
+
+ Return a new path object representing the current directory (as returned
+ by :func:`os.getcwd`)::
+
+ >>> Path.cwd()
+ PosixPath('/home/antoine/pathlib')
+
+
+.. method:: Path.stat()
+
+ Return information about this path (similarly to :func:`os.stat`).
+ The result is looked up at each call to this method.
+
+ >>> p = Path('setup.py')
+ >>> p.stat().st_size
+ 956
+ >>> p.stat().st_mtime
+ 1327883547.852554
+
+
+.. method:: Path.chmod(mode)
+
+ Change the file mode and permissions, like :func:`os.chmod`::
+
+ >>> p = Path('setup.py')
+ >>> p.stat().st_mode
+ 33277
+ >>> p.chmod(0o444)
+ >>> p.stat().st_mode
+ 33060
+
+
+.. method:: Path.exists()
+
+ Whether the path points to an existing file or directory::
+
+ >>> Path('.').exists()
+ True
+ >>> Path('setup.py').exists()
+ True
+ >>> Path('/etc').exists()
+ True
+ >>> Path('nonexistentfile').exists()
+ False
+
+ .. note::
+ If the path points to a symlink, :meth:`exists` returns whether the
+ symlink *points to* an existing file or directory.
+
+
+.. method:: Path.glob(pattern)
+
+ Glob the given *pattern* in the directory represented by this path,
+ yielding all matching files (of any kind)::
+
+ >>> sorted(Path('.').glob('*.py'))
+ [PosixPath('pathlib.py'), PosixPath('setup.py'), PosixPath('test_pathlib.py')]
+ >>> sorted(Path('.').glob('*/*.py'))
+ [PosixPath('docs/conf.py')]
+
+ The "``**``" pattern means "this directory and all subdirectories,
+ recursively". In other words, it enables recursive globbing::
+
+ >>> sorted(Path('.').glob('**/*.py'))
+ [PosixPath('build/lib/pathlib.py'),
+ PosixPath('docs/conf.py'),
+ PosixPath('pathlib.py'),
+ PosixPath('setup.py'),
+ PosixPath('test_pathlib.py')]
+
+ .. note::
+ Using the "``**``" pattern in large directory trees may consume
+ an inordinate amount of time.
+
+
+.. method:: Path.group()
+
+ Return the name of the group owning the file. :exc:`KeyError` is thrown
+ if the file's gid isn't found in the system database.
+
+
+.. method:: Path.is_dir()
+
+ Return True if the path points to a directory (or a symbolic link
+ pointing to a directory), False if it points to another kind of file.
+
+ False is also returned if the path doesn't exist or is a broken symlink;
+ other errors (such as permission errors) are propagated.
+
+
+.. method:: Path.is_file()
+
+ Return True if the path points to a regular file (or a symbolic link
+ pointing to a regular file), False if it points to another kind of file.
+
+ False is also returned if the path doesn't exist or is a broken symlink;
+ other errors (such as permission errors) are propagated.
+
+
+.. method:: Path.is_symlink()
+
+ Return True if the path points to a symbolic link, False otherwise.
+
+ False is also returned if the path doesn't exist; other errors (such
+ as permission errors) are propagated.
+
+
+.. method:: Path.is_socket()
+
+ Return True if the path points to a Unix socket (or a symbolic link
+ pointing to a Unix socket), False if it points to another kind of file.
+
+ False is also returned if the path doesn't exist or is a broken symlink;
+ other errors (such as permission errors) are propagated.
+
+
+.. method:: Path.is_fifo()
+
+ Return True if the path points to a FIFO (or a symbolic link
+ pointing to a FIFO), False if it points to another kind of file.
+
+ False is also returned if the path doesn't exist or is a broken symlink;
+ other errors (such as permission errors) are propagated.
+
+
+.. method:: Path.is_block_device()
+
+ Return True if the path points to a block device (or a symbolic link
+ pointing to a block device), False if it points to another kind of file.
+
+ False is also returned if the path doesn't exist or is a broken symlink;
+ other errors (such as permission errors) are propagated.
+
+
+.. method:: Path.is_char_device()
+
+ Return True if the path points to a character device (or a symbolic link
+ pointing to a character device), False if it points to another kind of file.
+
+ False is also returned if the path doesn't exist or is a broken symlink;
+ other errors (such as permission errors) are propagated.
+
+
+.. method:: Path.iterdir()
+
+ When the path points to a directory, yield path objects of the directory
+ contents::
+
+ >>> p = Path('docs')
+ >>> for child in p.iterdir(): child
+ ...
+ PosixPath('docs/conf.py')
+ PosixPath('docs/_templates')
+ PosixPath('docs/make.bat')
+ PosixPath('docs/index.rst')
+ PosixPath('docs/_build')
+ PosixPath('docs/_static')
+ PosixPath('docs/Makefile')
+
+.. method:: Path.lchmod(mode)
+
+ Like :meth:`Path.chmod` but, if the path points to a symbolic link, the
+ symbolic link's mode is changed rather than its target's.
+
+
+.. method:: Path.lstat()
+
+ Like :meth:`Path.stat` but, if the path points to a symbolic link, return
+ the symbolic link's information rather than its target's.
+
+
+.. method:: Path.mkdir(mode=0o777, parents=False)
+
+ Create a new directory at this given path. If *mode* is given, it is
+ combined with the process' ``umask`` value to determine the file mode
+ and access flags. If the path already exists, :exc:`OSError` is raised.
+
+ If *parents* is True, any missing parents of this path are created
+ as needed. If *parents* is False (the default), a missing parent raises
+ :exc:`OSError`.
+
+
+.. method:: Path.open(mode='r', buffering=-1, encoding=None, errors=None, newline=None)
+
+ Open the file pointed to by the path, like the built-in :func:`open`
+ function does::
+
+ >>> p = Path('setup.py')
+ >>> with p.open() as f:
+ ... f.readline()
+ ...
+ '#!/usr/bin/env python3\n'
+
+
+.. method:: Path.owner()
+
+ Return the name of the user owning the file. :exc:`KeyError` is thrown
+ if the file's uid isn't found in the system database.
+
+
+.. method:: Path.rename(target)
+
+ Rename this file or directory to the given *target*. *target* can be
+ either a string or another path object::
+
+ >>> p = Path('foo')
+ >>> p.open('w').write('some text')
+ 9
+ >>> target = Path('bar')
+ >>> p.rename(target)
+ >>> target.open().read()
+ 'some text'
+
+
+.. method:: Path.replace(target)
+
+ Rename this file or directory to the given *target*. If *target* points
+ to an existing file or directory, it will be unconditionally replaced.
+
+
+.. method:: Path.resolve()
+
+ Make the path absolute, resolving any symlinks. A new path object is
+ returned::
+
+ >>> p = Path()
+ >>> p
+ PosixPath('.')
+ >>> p.resolve()
+ PosixPath('/home/antoine/pathlib')
+
+ `".."` components are also eliminated (this is the only method to do so)::
+
+ >>> p = Path('docs/../setup.py')
+ >>> p.resolve()
+ PosixPath('/home/antoine/pathlib/setup.py')
+
+ If the path doesn't exist, :exc:`FileNotFoundError` is raised. If an
+ infinite loop is encountered along the resolution path,
+ :exc:`RuntimeError` is raised.
+
+
+.. method:: Path.rglob(pattern)
+
+ This is like calling :meth:`glob` with "``**``" added in front of the
+ given *pattern*:
+
+ >>> sorted(Path().rglob("*.py"))
+ [PosixPath('build/lib/pathlib.py'),
+ PosixPath('docs/conf.py'),
+ PosixPath('pathlib.py'),
+ PosixPath('setup.py'),
+ PosixPath('test_pathlib.py')]
+
+
+.. method:: Path.rmdir()
+
+ Remove this directory. The directory must be empty.
+
+
+.. method:: Path.symlink_to(target, target_is_directory=False)
+
+ Make this path a symbolic link to *target*. Under Windows,
+ *target_is_directory* must be True (default False) if the link's target
+ is a directory. Under POSIX, *target_is_directory*'s value is ignored.
+
+ >>> p = Path('mylink')
+ >>> p.symlink_to('setup.py')
+ >>> p.resolve()
+ PosixPath('/home/antoine/pathlib/setup.py')
+ >>> p.stat().st_size
+ 956
+ >>> p.lstat().st_size
+ 8
+
+ .. note::
+ The order of arguments (link, target) is the reverse
+ of :func:`os.symlink`'s.
+
+
+.. method:: Path.touch(mode=0o777, exist_ok=True)
+
+ Create a file at this given path. If *mode* is given, it is combined
+ with the process' ``umask`` value to determine the file mode and access
+ flags. If the file already exists, the function succeeds if *exist_ok*
+ is true (and its modification time is updated to the current time),
+ otherwise :exc:`OSError` is raised.
+
+
+.. method:: Path.unlink()
+
+ Remove this file or symbolic link. If the path points to a directory,
+ use :func:`Path.rmdir` instead.
diff --git a/Doc/library/pdb.rst b/Doc/library/pdb.rst
index f4e37ac..48a8a6b 100644
--- a/Doc/library/pdb.rst
+++ b/Doc/library/pdb.rst
@@ -41,7 +41,7 @@ of the debugger is::
.. versionchanged:: 3.3
Tab-completion via the :mod:`readline` module is available for commands and
command arguments, e.g. the current global and local names are offered as
- arguments of the ``print`` command.
+ arguments of the ``p`` command.
:file:`pdb.py` can also be invoked as a script to debug other scripts. For
example::
@@ -309,7 +309,7 @@ by the local file.
``end`` to terminate the commands. An example::
(Pdb) commands 1
- (com) print some_variable
+ (com) p some_variable
(com) end
(Pdb)
@@ -403,13 +403,19 @@ by the local file.
Print the argument list of the current function.
-.. pdbcommand:: p(rint) expression
+.. pdbcommand:: p expression
Evaluate the *expression* in the current context and print its value.
+ .. note::
+
+ ``print()`` can also be used, but is not a debugger command --- this executes the
+ Python :func:`print` function.
+
+
.. pdbcommand:: pp expression
- Like the :pdbcmd:`print` command, except the value of the expression is
+ Like the :pdbcmd:`p` command, except the value of the expression is
pretty-printed using the :mod:`pprint` module.
.. pdbcommand:: whatis expression
diff --git a/Doc/library/pickle.rst b/Doc/library/pickle.rst
index a99117c..9404a47 100644
--- a/Doc/library/pickle.rst
+++ b/Doc/library/pickle.rst
@@ -384,7 +384,8 @@ The following types can be pickled:
* tuples, lists, sets, and dictionaries containing only picklable objects
-* functions defined at the top level of a module
+* functions defined at the top level of a module (using :keyword:`def`, not
+ :keyword:`lambda`)
* built-in functions defined at the top level of a module
@@ -402,7 +403,7 @@ raised in this case. You can carefully raise this limit with
:func:`sys.setrecursionlimit`.
Note that functions (built-in and user-defined) are pickled by "fully qualified"
-name reference, not by value. This means that only the function name is
+name reference, not by value. [#]_ This means that only the function name is
pickled, along with the name of the module the function is defined in. Neither
the function's code, nor any of its function attributes are pickled. Thus the
defining module must be importable in the unpickling environment, and the module
@@ -850,6 +851,9 @@ The following example reads the resulting pickled data. ::
.. [#] Don't confuse this with the :mod:`marshal` module
+.. [#] This is why :keyword:`lambda` functions cannot be pickled: all
+ :keyword:`lambda` functions share the same name: ``<lambda>``.
+
.. [#] The exception raised will likely be an :exc:`ImportError` or an
:exc:`AttributeError` but it could be something else.
diff --git a/Doc/library/plistlib.rst b/Doc/library/plistlib.rst
index a267368..92de860 100644
--- a/Doc/library/plistlib.rst
+++ b/Doc/library/plistlib.rst
@@ -16,26 +16,21 @@
--------------
This module provides an interface for reading and writing the "property list"
-XML files used mainly by Mac OS X.
+files used mainly by Mac OS X and supports both binary and XML plist files.
-The property list (``.plist``) file format is a simple XML pickle supporting
+The property list (``.plist``) file format is a simple serialization supporting
basic object types, like dictionaries, lists, numbers and strings. Usually the
top level object is a dictionary.
-To write out and to parse a plist file, use the :func:`writePlist` and
-:func:`readPlist` functions.
+To write out and to parse a plist file, use the :func:`dump` and
+:func:`load` functions.
-To work with plist data in bytes objects, use :func:`writePlistToBytes`
-and :func:`readPlistFromBytes`.
+To work with plist data in bytes objects, use :func:`dumps`
+and :func:`loads`.
Values can be strings, integers, floats, booleans, tuples, lists, dictionaries
-(but only with string keys), :class:`Data` or :class:`datetime.datetime`
-objects. String values (including dictionary keys) have to be unicode strings --
-they will be written out as UTF-8.
-
-The ``<data>`` plist type is supported through the :class:`Data` class. This is
-a thin wrapper around a Python bytes object. Use :class:`Data` if your strings
-contain control characters.
+(but only with string keys), :class:`Data`, :class:`bytes`, :class:`bytesarray`
+or :class:`datetime.datetime` objects.
.. seealso::
@@ -45,37 +40,145 @@ contain control characters.
This module defines the following functions:
-.. function:: readPlist(pathOrFile)
+.. function:: load(fp, \*, fmt=None, use_builtin_types=True, dict_type=dict)
- Read a plist file. *pathOrFile* may either be a file name or a (readable and
- binary) file object. Return the unpacked root object (which usually is a
+ Read a plist file. *fp* should be a readable and binary file object.
+ Return the unpacked root object (which usually is a
dictionary).
- The XML data is parsed using the Expat parser from :mod:`xml.parsers.expat`
- -- see its documentation for possible exceptions on ill-formed XML.
- Unknown elements will simply be ignored by the plist parser.
+ The *fmt* is the format of the file and the following values are valid:
+
+ * :data:`None`: Autodetect the file format
+
+ * :data:`FMT_XML`: XML file format
+
+ * :data:`FMT_BINARY`: Binary plist format
+
+ If *use_builtin_types* is True (the default) binary data will be returned
+ as instances of :class:`bytes`, otherwise it is returned as instances of
+ :class:`Data`.
+
+ The *dict_type* is the type used for dictionaries that are read from the
+ plist file. The exact structure of the plist can be recovered by using
+ :class:`collections.OrderedDict` (although the order of keys shouldn't be
+ important in plist files).
+
+ XML data for the :data:`FMT_XML` format is parsed using the Expat parser
+ from :mod:`xml.parsers.expat` -- see its documentation for possible
+ exceptions on ill-formed XML. Unknown elements will simply be ignored
+ by the plist parser.
+
+ The parser for the binary format raises :exc:`InvalidFileException`
+ when the file cannot be parsed.
+
+ .. versionadded:: 3.4
+
+
+.. function:: loads(data, \*, fmt=None, use_builtin_types=True, dict_type=dict)
+
+ Load a plist from a bytes object. See :func:`load` for an explanation of
+ the keyword arguments.
+
+
+.. function:: dump(value, fp, \*, fmt=FMT_XML, sort_keys=True, skipkeys=False)
+
+ Write *value* to a plist file. *Fp* should be a writable, binary
+ file object.
+
+ The *fmt* argument specifies the format of the plist file and can be
+ one of the following values:
+
+ * :data:`FMT_XML`: XML formatted plist file
+
+ * :data:`FMT_BINARY`: Binary formatted plist file
+
+ When *sort_keys* is true (the default) the keys for dictionaries will be
+ written to the plist in sorted order, otherwise they will be written in
+ the iteration order of the dictionary.
+
+ When *skipkeys* is false (the default) the function raises :exc:`TypeError`
+ when a key of a dictionary is not a string, otherwise such keys are skipped.
+
+ A :exc:`TypeError` will be raised if the object is of an unsupported type or
+ a container that contains objects of unsupported types.
+
+ .. versionchanged:: 3.4
+ Added the *fmt*, *sort_keys* and *skipkeys* arguments.
+
+
+.. function:: dumps(value, \*, fmt=FMT_XML, sort_keys=True, skipkeys=False)
+
+ Return *value* as a plist-formatted bytes object. See
+ the documentation for :func:`dump` for an explanation of the keyword
+ arguments of this function.
+
+
+The following functions are deprecated:
+
+.. function:: readPlist(pathOrFile)
+
+ Read a plist file. *pathOrFile* may be either a file name or a (readable
+ and binary) file object. Returns the unpacked root object (which usually
+ is a dictionary).
+
+ This function calls :func:`load` to do the actual work, the the documentation
+ of :func:`that function <load>` for an explanation of the keyword arguments.
+
+ .. note::
+
+ Dict values in the result have a ``__getattr__`` method that defers
+ to ``__getitem_``. This means that you can use attribute access to
+ access items of these dictionaries.
+
+ .. deprecated: 3.4 Use :func:`load` instead.
.. function:: writePlist(rootObject, pathOrFile)
- Write *rootObject* to a plist file. *pathOrFile* may either be a file name
- or a (writable and binary) file object.
+ Write *rootObject* to an XML plist file. *pathOrFile* may be either a file name
+ or a (writable and binary) file object
- A :exc:`TypeError` will be raised if the object is of an unsupported type or
- a container that contains objects of unsupported types.
+ .. deprecated: 3.4 Use :func:`dump` instead.
.. function:: readPlistFromBytes(data)
Read a plist data from a bytes object. Return the root object.
+ See :func:`load` for a description of the keyword arguments.
+
+ .. note::
+
+ Dict values in the result have a ``__getattr__`` method that defers
+ to ``__getitem_``. This means that you can use attribute access to
+ access items of these dictionaries.
+
+ .. deprecated:: 3.4 Use :func:`loads` instead.
+
.. function:: writePlistToBytes(rootObject)
- Return *rootObject* as a plist-formatted bytes object.
+ Return *rootObject* as an XML plist-formatted bytes object.
+
+ .. deprecated:: 3.4 Use :func:`dumps` instead.
+
+ .. versionchanged:: 3.4
+ Added the *fmt*, *sort_keys* and *skipkeys* arguments.
+
+The following classes are available:
+
+.. class:: Dict([dict]):
+
+ Return an extended mapping object with the same value as dictionary
+ *dict*.
+
+ This class is a subclass of :class:`dict` where attribute access can
+ be used to access items. That is, ``aDict.key`` is the same as
+ ``aDict['key']`` for getting, setting and deleting items in the mapping.
+
+ .. deprecated:: 3.0
-The following class is available:
.. class:: Data(data)
@@ -86,6 +189,24 @@ The following class is available:
It has one attribute, :attr:`data`, that can be used to retrieve the Python
bytes object stored in it.
+ .. deprecated:: 3.4 Use a :class:`bytes` object instead
+
+
+The following constants are avaiable:
+
+.. data:: FMT_XML
+
+ The XML format for plist files.
+
+ .. versionadded:: 3.4
+
+
+.. data:: FMT_BINARY
+
+ The binary format for plist files
+
+ .. versionadded:: 3.4
+
Examples
--------
@@ -103,13 +224,15 @@ Generating a plist::
aTrueValue = True,
aFalseValue = False,
),
- someData = Data(b"<binary gunk>"),
- someMoreData = Data(b"<lots of binary gunk>" * 10),
+ someData = b"<binary gunk>",
+ someMoreData = b"<lots of binary gunk>" * 10,
aDate = datetime.datetime.fromtimestamp(time.mktime(time.gmtime())),
)
- writePlist(pl, fileName)
+ with open(fileName, 'wb') as fp:
+ dump(pl, fp)
Parsing a plist::
- pl = readPlist(pathOrFile)
+ with open(fileName, 'rb') as fp:
+ pl = load(fp)
print(pl["aKey"])
diff --git a/Doc/library/poplib.rst b/Doc/library/poplib.rst
index b4080d6..e248d6b 100644
--- a/Doc/library/poplib.rst
+++ b/Doc/library/poplib.rst
@@ -13,8 +13,11 @@
--------------
This module defines a class, :class:`POP3`, which encapsulates a connection to a
-POP3 server and implements the protocol as defined in :rfc:`1725`. The
-:class:`POP3` class supports both the minimal and optional command sets.
+POP3 server and implements the protocol as defined in :rfc:`1939`. The
+:class:`POP3` class supports both the minimal and optional command sets from
+:rfc:`1939`. The :class:`POP3` class also supports the `STLS` command introduced
+in :rfc:`2595` to enable encrypted communication on an already established connection.
+
Additionally, this module provides a class :class:`POP3_SSL`, which provides
support for connecting to POP3 servers that use SSL as an underlying protocol
layer.
@@ -97,6 +100,14 @@ An :class:`POP3` instance has the following methods:
Returns the greeting string sent by the POP3 server.
+.. method:: POP3.capa()
+
+ Query the server's capabilities as specified in :rfc:`2449`.
+ Returns a dictionary in the form ``{'name': ['param'...]}``.
+
+ .. versionadded:: 3.4
+
+
.. method:: POP3.user(username)
Send user command, response should indicate that a password is required.
@@ -176,6 +187,18 @@ An :class:`POP3` instance has the following methods:
the unique id for that message in the form ``'response mesgnum uid``, otherwise
result is list ``(response, ['mesgnum uid', ...], octets)``.
+.. method:: POP3.stls(context=None)
+
+ Start a TLS session on the active connection as specified in :rfc:`2595`.
+ This is only allowed before user authentication
+
+ *context* parameter is a :class:`ssl.SSLContext` object which allows
+ bundling SSL configuration options, certificates and private keys into
+ a single (potentially long-lived) structure.
+
+ .. versionadded:: 3.4
+
+
Instances of :class:`POP3_SSL` have no additional methods. The interface of this
subclass is identical to its parent.
diff --git a/Doc/library/pprint.rst b/Doc/library/pprint.rst
index 3a86331..447f8f7 100644
--- a/Doc/library/pprint.rst
+++ b/Doc/library/pprint.rst
@@ -14,8 +14,8 @@ The :mod:`pprint` module provides a capability to "pretty-print" arbitrary
Python data structures in a form which can be used as input to the interpreter.
If the formatted structures include objects which are not fundamental Python
types, the representation may not be loadable. This may be the case if objects
-such as files, sockets, classes, or instances are included, as well as many
-other built-in objects which are not representable as Python constants.
+such as files, sockets or classes are included, as well as many other
+objects which are not representable as Python literals.
The formatted representation keeps objects on a single line if it can, and
breaks them onto multiple lines if they don't fit within the allowed width.
@@ -29,14 +29,14 @@ The :mod:`pprint` module defines one class:
.. First the implementation class:
-.. class:: PrettyPrinter(indent=1, width=80, depth=None, stream=None)
+.. class:: PrettyPrinter(indent=1, width=80, depth=None, stream=None, *, \
+ compact=False)
Construct a :class:`PrettyPrinter` instance. This constructor understands
several keyword parameters. An output stream may be set using the *stream*
keyword; the only method used on the stream object is the file protocol's
:meth:`write` method. If not specified, the :class:`PrettyPrinter` adopts
- ``sys.stdout``. Three additional parameters may be used to control the
- formatted representation. The keywords are *indent*, *depth*, and *width*. The
+ ``sys.stdout``. The
amount of indentation added for each recursive level is specified by *indent*;
the default is one. Other values can cause output to look a little odd, but can
make nesting easier to spot. The number of levels which may be printed is
@@ -45,7 +45,12 @@ The :mod:`pprint` module defines one class:
the depth of the objects being formatted. The desired output width is
constrained using the *width* parameter; the default is 80 characters. If a
structure cannot be formatted within the constrained width, a best effort will
- be made.
+ be made. If *compact* is false (the default) each item of a long sequence
+ will be formatted on a separate line. If *compact* is true, as many items
+ as will fit within the *width* will be formatted on each output line.
+
+ .. versionchanged:: 3.4
+ Added the *compact* parameter.
>>> import pprint
>>> stuff = ['spam', 'eggs', 'lumberjack', 'knights', 'ni']
@@ -58,6 +63,12 @@ The :mod:`pprint` module defines one class:
'lumberjack',
'knights',
'ni']
+ >>> pp = pprint.PrettyPrinter(width=41, compact=True)
+ >>> pp.pprint(stuff)
+ [['spam', 'eggs', 'lumberjack',
+ 'knights', 'ni'],
+ 'spam', 'eggs', 'lumberjack', 'knights',
+ 'ni']
>>> tup = ('spam', ('eggs', ('lumberjack', ('knights', ('ni', ('dead',
... ('parrot', ('fresh fruit',))))))))
>>> pp = pprint.PrettyPrinter(depth=6)
@@ -65,23 +76,30 @@ The :mod:`pprint` module defines one class:
('spam', ('eggs', ('lumberjack', ('knights', ('ni', ('dead', (...)))))))
-The :class:`PrettyPrinter` class supports several derivative functions:
+The :mod:`pprint` module also provides several shortcut functions:
-.. function:: pformat(object, indent=1, width=80, depth=None)
+.. function:: pformat(object, indent=1, width=80, depth=None, *, compact=False)
- Return the formatted representation of *object* as a string. *indent*, *width*
- and *depth* will be passed to the :class:`PrettyPrinter` constructor as
- formatting parameters.
+ Return the formatted representation of *object* as a string. *indent*,
+ *width*, *depth* and *compact* will be passed to the :class:`PrettyPrinter`
+ constructor as formatting parameters.
+ .. versionchanged:: 3.4
+ Added the *compact* parameter.
-.. function:: pprint(object, stream=None, indent=1, width=80, depth=None)
+
+.. function:: pprint(object, stream=None, indent=1, width=80, depth=None, *, \
+ compact=False)
Prints the formatted representation of *object* on *stream*, followed by a
newline. If *stream* is ``None``, ``sys.stdout`` is used. This may be used
in the interactive interpreter instead of the :func:`print` function for
inspecting values (you can even reassign ``print = pprint.pprint`` for use
- within a scope). *indent*, *width* and *depth* will be passed to the
- :class:`PrettyPrinter` constructor as formatting parameters.
+ within a scope). *indent*, *width*, *depth* and *compact* will be passed
+ to the :class:`PrettyPrinter` constructor as formatting parameters.
+
+ .. versionchanged:: 3.4
+ Added the *compact* parameter.
>>> import pprint
>>> stuff = ['spam', 'eggs', 'lumberjack', 'knights', 'ni']
@@ -193,101 +211,141 @@ Example
-------
To demonstrate several uses of the :func:`pprint` function and its parameters,
-let's fetch information about a project from PyPI::
+let's fetch information about a project from `PyPI <https://pypi.python.org>`_::
>>> import json
>>> import pprint
>>> from urllib.request import urlopen
- >>> with urlopen('http://pypi.python.org/pypi/configparser/json') as url:
+ >>> with urlopen('http://pypi.python.org/pypi/Twisted/json') as url:
... http_info = url.info()
... raw_data = url.read().decode(http_info.get_content_charset())
>>> project_info = json.loads(raw_data)
- >>> result = {'headers': http_info.items(), 'body': project_info}
In its basic form, :func:`pprint` shows the whole object::
- >>> pprint.pprint(result)
- {'body': {'info': {'_pypi_hidden': False,
- '_pypi_ordering': 12,
- 'classifiers': ['Development Status :: 4 - Beta',
- 'Intended Audience :: Developers',
- 'License :: OSI Approved :: MIT License',
- 'Natural Language :: English',
- 'Operating System :: OS Independent',
- 'Programming Language :: Python',
- 'Programming Language :: Python :: 2',
- 'Programming Language :: Python :: 2.6',
- 'Programming Language :: Python :: 2.7',
- 'Topic :: Software Development :: Libraries',
- 'Topic :: Software Development :: Libraries :: Python Modules'],
- 'download_url': 'UNKNOWN',
- 'home_page': 'http://docs.python.org/py3k/library/configparser.html',
- 'keywords': 'configparser ini parsing conf cfg configuration file',
- 'license': 'MIT',
- 'name': 'configparser',
- 'package_url': 'http://pypi.python.org/pypi/configparser',
- 'platform': 'any',
- 'release_url': 'http://pypi.python.org/pypi/configparser/3.2.0r3',
- 'requires_python': None,
- 'stable_version': None,
- 'summary': 'This library brings the updated configparser from Python 3.2+ to Python 2.6-2.7.',
- 'version': '3.2.0r3'},
- 'urls': [{'comment_text': '',
- 'downloads': 47,
- 'filename': 'configparser-3.2.0r3.tar.gz',
- 'has_sig': False,
- 'md5_digest': '8500fd87c61ac0de328fc996fce69b96',
- 'packagetype': 'sdist',
- 'python_version': 'source',
- 'size': 32281,
- 'upload_time': '2011-05-10T16:28:50',
- 'url': 'http://pypi.python.org/packages/source/c/configparser/configparser-3.2.0r3.tar.gz'}]},
- 'headers': [('Date', 'Sat, 14 May 2011 12:48:52 GMT'),
- ('Server', 'Apache/2.2.16 (Debian)'),
- ('Content-Disposition', 'inline'),
- ('Connection', 'close'),
- ('Transfer-Encoding', 'chunked'),
- ('Content-Type', 'application/json; charset="UTF-8"')]}
+ >>> pprint.pprint(project_info)
+ {'info': {'_pypi_hidden': False,
+ '_pypi_ordering': 125,
+ 'author': 'Glyph Lefkowitz',
+ 'author_email': 'glyph@twistedmatrix.com',
+ 'bugtrack_url': '',
+ 'cheesecake_code_kwalitee_id': None,
+ 'cheesecake_documentation_id': None,
+ 'cheesecake_installability_id': None,
+ 'classifiers': ['Programming Language :: Python :: 2.6',
+ 'Programming Language :: Python :: 2.7',
+ 'Programming Language :: Python :: 2 :: Only'],
+ 'description': 'An extensible framework for Python programming, '
+ 'with special focus\r\n'
+ 'on event-based network programming and '
+ 'multiprotocol integration.',
+ 'docs_url': '',
+ 'download_url': 'UNKNOWN',
+ 'home_page': 'http://twistedmatrix.com/',
+ 'keywords': '',
+ 'license': 'MIT',
+ 'maintainer': '',
+ 'maintainer_email': '',
+ 'name': 'Twisted',
+ 'package_url': 'http://pypi.python.org/pypi/Twisted',
+ 'platform': 'UNKNOWN',
+ 'release_url': 'http://pypi.python.org/pypi/Twisted/12.3.0',
+ 'requires_python': None,
+ 'stable_version': None,
+ 'summary': 'An asynchronous networking framework written in Python',
+ 'version': '12.3.0'},
+ 'urls': [{'comment_text': '',
+ 'downloads': 71844,
+ 'filename': 'Twisted-12.3.0.tar.bz2',
+ 'has_sig': False,
+ 'md5_digest': '6e289825f3bf5591cfd670874cc0862d',
+ 'packagetype': 'sdist',
+ 'python_version': 'source',
+ 'size': 2615733,
+ 'upload_time': '2012-12-26T12:47:03',
+ 'url': 'https://pypi.python.org/packages/source/T/Twisted/Twisted-12.3.0.tar.bz2'},
+ {'comment_text': '',
+ 'downloads': 5224,
+ 'filename': 'Twisted-12.3.0.win32-py2.7.msi',
+ 'has_sig': False,
+ 'md5_digest': '6b778f5201b622a5519a2aca1a2fe512',
+ 'packagetype': 'bdist_msi',
+ 'python_version': '2.7',
+ 'size': 2916352,
+ 'upload_time': '2012-12-26T12:48:15',
+ 'url': 'https://pypi.python.org/packages/2.7/T/Twisted/Twisted-12.3.0.win32-py2.7.msi'}]}
The result can be limited to a certain *depth* (ellipsis is used for deeper
contents)::
- >>> pprint.pprint(result, depth=3)
- {'body': {'info': {'_pypi_hidden': False,
- '_pypi_ordering': 12,
- 'classifiers': [...],
- 'download_url': 'UNKNOWN',
- 'home_page': 'http://docs.python.org/py3k/library/configparser.html',
- 'keywords': 'configparser ini parsing conf cfg configuration file',
- 'license': 'MIT',
- 'name': 'configparser',
- 'package_url': 'http://pypi.python.org/pypi/configparser',
- 'platform': 'any',
- 'release_url': 'http://pypi.python.org/pypi/configparser/3.2.0r3',
- 'requires_python': None,
- 'stable_version': None,
- 'summary': 'This library brings the updated configparser from Python 3.2+ to Python 2.6-2.7.',
- 'version': '3.2.0r3'},
- 'urls': [{...}]},
- 'headers': [('Date', 'Sat, 14 May 2011 12:48:52 GMT'),
- ('Server', 'Apache/2.2.16 (Debian)'),
- ('Content-Disposition', 'inline'),
- ('Connection', 'close'),
- ('Transfer-Encoding', 'chunked'),
- ('Content-Type', 'application/json; charset="UTF-8"')]}
-
-Additionally, maximum *width* can be suggested. If a long object cannot be
-split, the specified width will be exceeded::
-
- >>> pprint.pprint(result['headers'], width=30)
- [('Date',
- 'Sat, 14 May 2011 12:48:52 GMT'),
- ('Server',
- 'Apache/2.2.16 (Debian)'),
- ('Content-Disposition',
- 'inline'),
- ('Connection', 'close'),
- ('Transfer-Encoding',
- 'chunked'),
- ('Content-Type',
- 'application/json; charset="UTF-8"')]
+ >>> pprint.pprint(project_info, depth=2)
+ {'info': {'_pypi_hidden': False,
+ '_pypi_ordering': 125,
+ 'author': 'Glyph Lefkowitz',
+ 'author_email': 'glyph@twistedmatrix.com',
+ 'bugtrack_url': '',
+ 'cheesecake_code_kwalitee_id': None,
+ 'cheesecake_documentation_id': None,
+ 'cheesecake_installability_id': None,
+ 'classifiers': [...],
+ 'description': 'An extensible framework for Python programming, '
+ 'with special focus\r\n'
+ 'on event-based network programming and '
+ 'multiprotocol integration.',
+ 'docs_url': '',
+ 'download_url': 'UNKNOWN',
+ 'home_page': 'http://twistedmatrix.com/',
+ 'keywords': '',
+ 'license': 'MIT',
+ 'maintainer': '',
+ 'maintainer_email': '',
+ 'name': 'Twisted',
+ 'package_url': 'http://pypi.python.org/pypi/Twisted',
+ 'platform': 'UNKNOWN',
+ 'release_url': 'http://pypi.python.org/pypi/Twisted/12.3.0',
+ 'requires_python': None,
+ 'stable_version': None,
+ 'summary': 'An asynchronous networking framework written in Python',
+ 'version': '12.3.0'},
+ 'urls': [{...}, {...}]}
+
+Additionally, maximum character *width* can be suggested. If a long object
+cannot be split, the specified width will be exceeded::
+
+ >>> pprint.pprint(project_info, depth=2, width=50)
+ {'info': {'_pypi_hidden': False,
+ '_pypi_ordering': 125,
+ 'author': 'Glyph Lefkowitz',
+ 'author_email': 'glyph@twistedmatrix.com',
+ 'bugtrack_url': '',
+ 'cheesecake_code_kwalitee_id': None,
+ 'cheesecake_documentation_id': None,
+ 'cheesecake_installability_id': None,
+ 'classifiers': [...],
+ 'description': 'An extensible '
+ 'framework for '
+ 'Python programming, '
+ 'with special '
+ 'focus\r\n'
+ 'on event-based '
+ 'network programming '
+ 'and multiprotocol '
+ 'integration.',
+ 'docs_url': '',
+ 'download_url': 'UNKNOWN',
+ 'home_page': 'http://twistedmatrix.com/',
+ 'keywords': '',
+ 'license': 'MIT',
+ 'maintainer': '',
+ 'maintainer_email': '',
+ 'name': 'Twisted',
+ 'package_url': 'http://pypi.python.org/pypi/Twisted',
+ 'platform': 'UNKNOWN',
+ 'release_url': 'http://pypi.python.org/pypi/Twisted/12.3.0',
+ 'requires_python': None,
+ 'stable_version': None,
+ 'summary': 'An asynchronous '
+ 'networking framework '
+ 'written in Python',
+ 'version': '12.3.0'},
+ 'urls': [{...}, {...}]}
diff --git a/Doc/library/pty.rst b/Doc/library/pty.rst
index 2b9385b..90baec5 100644
--- a/Doc/library/pty.rst
+++ b/Doc/library/pty.rst
@@ -45,6 +45,9 @@ The :mod:`pty` module defines the following functions:
a file descriptor. The defaults try to read 1024 bytes each time they are
called.
+ .. versionchanged:: 3.4
+ :func:`spawn` now returns the status value from :func:`os.waitpid`
+ on the child process.
Example
-------
diff --git a/Doc/library/py_compile.rst b/Doc/library/py_compile.rst
index 07ddc25..bae8450 100644
--- a/Doc/library/py_compile.rst
+++ b/Doc/library/py_compile.rst
@@ -28,7 +28,7 @@ byte-code cache files in the directory containing the source code.
.. function:: compile(file, cfile=None, dfile=None, doraise=False, optimize=-1)
- Compile a source file to byte-code and write out the byte-code cache file.
+ Compile a source file to byte-code and write out the byte-code cache file.
The source code is loaded from the file name *file*. The byte-code is
written to *cfile*, which defaults to the :PEP:`3147` path, ending in
``.pyc`` (``.pyo`` if optimization is enabled in the current interpreter).
@@ -41,6 +41,13 @@ byte-code cache files in the directory containing the source code.
is raised. This function returns the path to byte-compiled file, i.e.
whatever *cfile* value was used.
+ If the path that *cfile* becomes (either explicitly specified or computed)
+ is a symlink or non-regular file, :exc:`FileExistsError` will be raised.
+ This is to act as a warning that import will turn those paths into regular
+ files if it is allowed to write byte-compiled files to those paths. This is
+ a side-effect of import using file renaming to place the final byte-compiled
+ file into place to prevent concurrent file writing issues.
+
*optimize* controls the optimization level and is passed to the built-in
:func:`compile` function. The default of ``-1`` selects the optimization
level of the current interpreter.
@@ -50,6 +57,13 @@ byte-code cache files in the directory containing the source code.
default was *file* + ``'c'`` (``'o'`` if optimization was enabled).
Also added the *optimize* parameter.
+ .. versionchanged:: 3.4
+ Changed code to use :mod:`importlib` for the byte-code cache file writing.
+ This means file creation/writing semantics now match what :mod:`importlib`
+ does, e.g. permissions, write-and-move semantics, etc. Also added the
+ caveat that :exc:`FileExistsError` is raised if *cfile* is a symlink or
+ non-regular file.
+
.. function:: main(args=None)
diff --git a/Doc/library/python.rst b/Doc/library/python.rst
index b67fbfc..f307d7d 100644
--- a/Doc/library/python.rst
+++ b/Doc/library/python.rst
@@ -25,4 +25,3 @@ overview:
inspect.rst
site.rst
fpectl.rst
- distutils.rst
diff --git a/Doc/library/readline.rst b/Doc/library/readline.rst
index 1134619..692310b 100644
--- a/Doc/library/readline.rst
+++ b/Doc/library/readline.rst
@@ -190,28 +190,32 @@ Example
The following example demonstrates how to use the :mod:`readline` module's
history reading and writing functions to automatically load and save a history
-file named :file:`.pyhist` from the user's home directory. The code below would
-normally be executed automatically during interactive sessions from the user's
-:envvar:`PYTHONSTARTUP` file. ::
+file named :file:`.python_history` from the user's home directory. The code
+below would normally be executed automatically during interactive sessions
+from the user's :envvar:`PYTHONSTARTUP` file. ::
+ import atexit
import os
import readline
- histfile = os.path.join(os.path.expanduser("~"), ".pyhist")
+
+ histfile = os.path.join(os.path.expanduser("~"), ".python_history")
try:
readline.read_history_file(histfile)
except FileNotFoundError:
pass
- import atexit
+
atexit.register(readline.write_history_file, histfile)
- del os, histfile
+
+This code is actually automatically run when Python is run in
+:ref:`interactive mode <tut-interactive>` (see :ref:`rlcompleter-config`).
The following example extends the :class:`code.InteractiveConsole` class to
support history save/restore. ::
- import code
- import readline
import atexit
+ import code
import os
+ import readline
class HistoryConsole(code.InteractiveConsole):
def __init__(self, locals=None, filename="<console>",
diff --git a/Doc/library/resource.rst b/Doc/library/resource.rst
index ed85850..9bb5848 100644
--- a/Doc/library/resource.rst
+++ b/Doc/library/resource.rst
@@ -74,6 +74,27 @@ this module for those platforms.
``setrlimit`` may also raise :exc:`error` if the underlying system call
fails.
+.. function:: prlimit(pid, resource[, limits])
+
+ Combines :func:`setrlimit` and :func:`getrlimit` in one function and
+ supports to get and set the resources limits of an arbitrary process. If
+ *pid* is 0, then the call applies to the current process. *resource* and
+ *limits* have the same meaning as in :func:`setrlimit`, except that
+ *limits* is optional.
+
+ When *limits* is not given the function returns the *resource* limit of the
+ process *pid*. When *limits* is given the *resource* limit of the process is
+ set and the former resource limit is returned.
+
+ Raises :exc:`ProcessLookupError` when *pid* can't be found and
+ :exc:`PermissionError` when the user doesn't have ``CAP_SYS_RESOURCE`` for
+ the process.
+
+ Availability: Linux 2.6.36 or later with glibc 2.13 or later
+
+ .. versionadded:: 3.4
+
+
These symbols define resources whose consumption can be controlled using the
:func:`setrlimit` and :func:`getrlimit` functions described below. The values of
these symbols are exactly the constants used by C programs.
@@ -151,6 +172,52 @@ platform.
The maximum area (in bytes) of address space which may be taken by the process.
+.. data:: RLIMIT_MSGQUEUE
+
+ The number of bytes that can be allocated for POSIX message queues.
+
+ Availability: Linux 2.6.8 or later.
+
+ .. versionadded:: 3.4
+
+
+.. data:: RLIMIT_NICE
+
+ The ceiling for the process's nice level (calculated as 20 - rlim_cur).
+
+ Availability: Linux 2.6.12 or later.
+
+ .. versionadded:: 3.4
+
+
+.. data:: RLIMIT_RTPRIO
+
+ The ceiling of the real-time priority.
+
+ Availability: Linux 2.6.12 or later.
+
+ .. versionadded:: 3.4
+
+
+.. data:: RLIMIT_RTTIME
+
+ The time limit (in microseconds) on CPU time that a process can spend
+ under real-time scheduling without making a blocking syscall.
+
+ Availability: Linux 2.6.25 or later.
+
+ .. versionadded:: 3.4
+
+
+.. data:: RLIMIT_SIGPENDING
+
+ The number of signals which the process may queue.
+
+ Availability: Linux 2.6.8 or later.
+
+ .. versionadded:: 3.4
+
+
Resource Usage
--------------
diff --git a/Doc/library/rlcompleter.rst b/Doc/library/rlcompleter.rst
index 633088d..9ed01c7 100644
--- a/Doc/library/rlcompleter.rst
+++ b/Doc/library/rlcompleter.rst
@@ -27,18 +27,10 @@ Example::
readline.__name__ readline.parse_and_bind(
>>> readline.
-The :mod:`rlcompleter` module is designed for use with Python's interactive
-mode. A user can add the following lines to his or her initialization file
-(identified by the :envvar:`PYTHONSTARTUP` environment variable) to get
-automatic :kbd:`Tab` completion::
-
- try:
- import readline
- except ImportError:
- print("Module readline not available.")
- else:
- import rlcompleter
- readline.parse_and_bind("tab: complete")
+The :mod:`rlcompleter` module is designed for use with Python's
+:ref:`interactive mode <tut-interactive>`. Unless Python is run with the
+:option:`-S` option, the module is automatically imported and configured
+(see :ref:`rlcompleter-config`).
On platforms without :mod:`readline`, the :class:`Completer` class defined by
this module can still be used for custom purposes.
diff --git a/Doc/library/select.rst b/Doc/library/select.rst
index b1fd3a8..524e411 100644
--- a/Doc/library/select.rst
+++ b/Doc/library/select.rst
@@ -14,6 +14,14 @@ it also works for other file types (in particular, on Unix, it works on pipes).
It cannot be used on regular files to determine whether a file has grown since
it was last read.
+.. note::
+
+ The :mod:`selectors` module allows high-level and efficient I/O
+ multiplexing, built upon the :mod:`select` module primitives. Users are
+ encouraged to use the :mod:`selectors` module instead, unless they want
+ precise control over the OS-level primitives used.
+
+
The module defines the following:
@@ -37,8 +45,13 @@ The module defines the following:
increases this value, :c:func:`devpoll` may return an
incomplete list of active file descriptors.
+ The new file descriptor is :ref:`non-inheritable <fd_inheritance>`.
+
.. versionadded:: 3.3
+ .. versionchanged:: 3.4
+ The new file descriptor is now non-inheritable.
+
.. function:: epoll(sizehint=-1, flags=0)
(Only supported on Linux 2.5.44 and newer.) Return an edge polling object,
@@ -47,11 +60,17 @@ The module defines the following:
to :const:`EPOLL_CLOEXEC`, which causes the epoll descriptor to be closed
automatically when :func:`os.execve` is called. See section
:ref:`epoll-objects` below for the methods supported by epolling objects.
+ They also support the :keyword:`with` statement.
+ The new file descriptor is :ref:`non-inheritable <fd_inheritance>`.
.. versionchanged:: 3.3
Added the *flags* parameter.
+ .. versionchanged:: 3.4
+ Support for the :keyword:`with` statement was added.
+ The new file descriptor is now non-inheritable.
+
.. function:: poll()
@@ -66,6 +85,11 @@ The module defines the following:
(Only supported on BSD.) Returns a kernel queue object; see section
:ref:`kqueue-objects` below for the methods supported by kqueue objects.
+ The new file descriptor is :ref:`non-inheritable <fd_inheritance>`.
+
+ .. versionchanged:: 3.4
+ The new file descriptor is now non-inheritable.
+
.. function:: kevent(ident, filter=KQ_FILTER_READ, flags=KQ_EV_ADD, fflags=0, data=0, udata=0)
@@ -144,6 +168,27 @@ descriptors), ``/dev/poll`` is O(active file descriptors).
object.
+.. method:: devpoll.close()
+
+ Close the file descriptor of the polling object.
+
+ .. versionadded:: 3.4
+
+
+.. attribute:: devpoll.closed
+
+ ``True`` if the polling object is closed.
+
+ .. versionadded:: 3.4
+
+
+.. method:: devpoll.fileno()
+
+ Return the file descriptor number of the polling object.
+
+ .. versionadded:: 3.4
+
+
.. method:: devpoll.register(fd[, eventmask])
Register a file descriptor with the polling object. Future calls to the
@@ -241,6 +286,11 @@ Edge and Level Trigger Polling (epoll) Objects
Close the control file descriptor of the epoll object.
+.. attribute:: epoll.closed
+
+ ``True`` if the epoll object is closed.
+
+
.. method:: epoll.fileno()
Return the file descriptor number of the control fd.
@@ -360,6 +410,11 @@ Kqueue Objects
Close the control file descriptor of the kqueue object.
+.. attribute:: kqueue.closed
+
+ ``True`` if the kqueue object is closed.
+
+
.. method:: kqueue.fileno()
Return the file descriptor number of the control fd.
diff --git a/Doc/library/selectors.rst b/Doc/library/selectors.rst
new file mode 100644
index 0000000..38ca4d4
--- /dev/null
+++ b/Doc/library/selectors.rst
@@ -0,0 +1,239 @@
+:mod:`selectors` -- High-level I/O multiplexing
+===============================================
+
+.. module:: selectors
+ :synopsis: High-level I/O multiplexing.
+
+.. versionadded:: 3.4
+
+
+Introduction
+------------
+
+This module allows high-level and efficient I/O multiplexing, built upon the
+:mod:`select` module primitives. Users are encouraged to use this module
+instead, unless they want precise control over the OS-level primitives used.
+
+It defines a :class:`BaseSelector` abstract base class, along with several
+concrete implementations (:class:`KqueueSelector`, :class:`EpollSelector`...),
+that can be used to wait for I/O readiness notification on multiple file
+objects. In the following, "file object" refers to any object with a
+:meth:`fileno()` method, or a raw file descriptor. See :term:`file object`.
+
+:class:`DefaultSelector` is an alias to the most efficient implementation
+available on the current platform: this should be the default choice for most
+users.
+
+.. note::
+ The type of file objects supported depends on the platform: on Windows,
+ sockets are supported, but not pipes, whereas on Unix, both are supported
+ (some other types may be supported as well, such as fifos or special file
+ devices).
+
+.. seealso::
+
+ :mod:`select`
+ Low-level I/O multiplexing module.
+
+
+Classes
+-------
+
+Classes hierarchy::
+
+ BaseSelector
+ +-- SelectSelector
+ +-- PollSelector
+ +-- EpollSelector
+ +-- KqueueSelector
+
+
+In the following, *events* is a bitwise mask indicating which I/O events should
+be waited for on a given file object. It can be a combination of the constants
+below:
+
+ +-----------------------+-----------------------------------------------+
+ | Constant | Meaning |
+ +=======================+===============================================+
+ | :const:`EVENT_READ` | Available for read |
+ +-----------------------+-----------------------------------------------+
+ | :const:`EVENT_WRITE` | Available for write |
+ +-----------------------+-----------------------------------------------+
+
+
+.. class:: SelectorKey
+
+ A :class:`SelectorKey` is a :class:`~collections.namedtuple` used to
+ associate a file object to its underlying file decriptor, selected event
+ mask and attached data. It is returned by several :class:`BaseSelector`
+ methods.
+
+ .. attribute:: fileobj
+
+ File object registered.
+
+ .. attribute:: fd
+
+ Underlying file descriptor.
+
+ .. attribute:: events
+
+ Events that must be waited for on this file object.
+
+ .. attribute:: data
+
+ Optional opaque data associated to this file object: for example, this
+ could be used to store a per-client session ID.
+
+
+.. class:: BaseSelector
+
+ A :class:`BaseSelector` is used to wait for I/O event readiness on multiple
+ file objects. It supports file stream registration, unregistration, and a
+ method to wait for I/O events on those streams, with an optional timeout.
+ It's an abstract base class, so cannot be instantiated. Use
+ :class:`DefaultSelector` instead, or one of :class:`SelectSelector`,
+ :class:`KqueueSelector` etc. if you want to specifically use an
+ implementation, and your platform supports it.
+ :class:`BaseSelector` and its concrete implementations support the
+ :term:`context manager` protocol.
+
+ .. method:: register(fileobj, events, data=None)
+
+ Register a file object for selection, monitoring it for I/O events.
+
+ *fileobj* is the file object to monitor.
+ *events* is a bitwise mask of events to monitor.
+ *data* is an opaque object.
+
+ This returns a new :class:`SelectorKey` instance, or raises a
+ :exc:`ValueError` in case of invalid event mask or file descriptor, or
+ :exc:`KeyError` if the file object is already registered.
+
+ .. method:: unregister(fileobj)
+
+ Unregister a file object from selection, removing it from monitoring. A
+ file object shall be unregistered prior to being closed.
+
+ *fileobj* must be a file object previously registered.
+
+ This returns the associated :class:`SelectorKey` instance, or raises a
+ :exc:`KeyError` if the file object is not registered.
+
+ .. method:: modify(fileobj, events, data=None)
+
+ Change a registered file object's monitored events or attached data.
+
+ This is equivalent to :meth:`BaseSelector.unregister(fileobj)` followed
+ by :meth:`BaseSelector.register(fileobj, events, data)`, except that it
+ can be implemented more efficiently.
+
+ This returns a new :class:`SelectorKey` instance, or raises a
+ :exc:`ValueError` in case of invalid event mask or file descriptor, or
+ :exc:`KeyError` if the file object is not registered.
+
+ .. method:: select(timeout=None)
+
+ Wait until some registered file objects become ready, or the timeout
+ expires.
+
+ If ``timeout > 0``, this specifies the maximum wait time, in seconds.
+ If ``timeout <= 0``, the call won't block, and will report the currently
+ ready file objects.
+ If *timeout* is ``None``, the call will block until a monitored file object
+ becomes ready.
+
+ This returns a list of ``(key, events)`` tuples, one for each ready file
+ object.
+
+ *key* is the :class:`SelectorKey` instance corresponding to a ready file
+ object.
+ *events* is a bitmask of events ready on this file object.
+
+ .. method:: close()
+
+ Close the selector.
+
+ This must be called to make sure that any underlying resource is freed.
+ The selector shall not be used once it has been closed.
+
+ .. method:: get_key(fileobj)
+
+ Return the key associated with a registered file object.
+
+ This returns the :class:`SelectorKey` instance associated to this file
+ object, or raises :exc:`KeyError` if the file object is not registered.
+
+ .. method:: get_map()
+
+ Return a mapping of file objects to selector keys.
+
+ This returns a :class:`~collections.abc.Mapping` instance mapping
+ registered file objects to their associated :class:`SelectorKey`
+ instance.
+
+
+.. class:: DefaultSelector()
+
+ The default selector class, using the most efficient implementation
+ available on the current platform. This should be the default choice for
+ most users.
+
+
+.. class:: SelectSelector()
+
+ :func:`select.select`-based selector.
+
+
+.. class:: PollSelector()
+
+ :func:`select.poll`-based selector.
+
+
+.. class:: EpollSelector()
+
+ :func:`select.epoll`-based selector.
+
+ .. method:: fileno()
+
+ This returns the file descriptor used by the underlying
+ :func:`select.epoll` object.
+
+
+.. class:: KqueueSelector()
+
+ :func:`select.kqueue`-based selector.
+
+ .. method:: fileno()
+
+ This returns the file descriptor used by the underlying
+ :func:`select.kqueue` object.
+
+
+Examples of selector usage::
+
+ >>> import selectors
+ >>> import socket
+ >>>
+ >>> s = selectors.DefaultSelector()
+ >>> r, w = socket.socketpair()
+ >>>
+ >>> s.register(r, selectors.EVENT_READ)
+ SelectorKey(fileobj=<socket.socket fd=4, family=1, type=1, proto=0>, fd=4, events=1, data=None)
+ >>> s.register(w, selectors.EVENT_WRITE)
+ SelectorKey(fileobj=<socket.socket fd=5, family=1, type=1, proto=0>, fd=5, events=2, data=None)
+ >>>
+ >>> print(s.select())
+ [(SelectorKey(fileobj=<socket.socket fd=5, family=1, type=1, proto=0>, fd=5, events=2, data=None), 2)]
+ >>>
+ >>> for key, events in s.select():
+ ... if events & selectors.EVENT_WRITE:
+ ... key.fileobj.send(b'spam')
+ ...
+ 4
+ >>> for key, events in s.select():
+ ... if events & selectors.EVENT_READ:
+ ... print(key.fileobj.recv(1024))
+ ...
+ b'spam'
+ >>> s.close()
diff --git a/Doc/library/shelve.rst b/Doc/library/shelve.rst
index 048f80a..4ba9ddc 100644
--- a/Doc/library/shelve.rst
+++ b/Doc/library/shelve.rst
@@ -44,8 +44,11 @@ lots of shared sub-objects. The keys are ordinary strings.
.. note::
Do not rely on the shelf being closed automatically; always call
- :meth:`close` explicitly when you don't need it any more, or use a
- :keyword:`with` statement with :func:`contextlib.closing`.
+ :meth:`~Shelf.close` explicitly when you don't need it any more, or
+ use :func:`shelve.open` as a context manager::
+
+ with shelve.open('spam') as db:
+ db['eggs'] = 'eggs'
.. warning::
@@ -118,10 +121,15 @@ Restrictions
The *keyencoding* parameter is the encoding used to encode keys before they
are used with the underlying dict.
- .. versionadded:: 3.2
- The *keyencoding* parameter; previously, keys were always encoded in
+ :class:`Shelf` objects can also be used as context managers.
+
+ .. versionchanged:: 3.2
+ Added the *keyencoding* parameter; previously, keys were always encoded in
UTF-8.
+ .. versionchanged:: 3.4
+ Added context manager support.
+
.. class:: BsdDbShelf(dict, protocol=None, writeback=False, keyencoding='utf-8')
diff --git a/Doc/library/shutil.rst b/Doc/library/shutil.rst
index a1457d0..e4f348c 100644
--- a/Doc/library/shutil.rst
+++ b/Doc/library/shutil.rst
@@ -53,7 +53,7 @@ Directory and files operations
*dst* and return *dst*. *src* and *dst* are path names given as strings.
*dst* must be the complete target file name; look at :func:`shutil.copy`
for a copy that accepts a target directory path. If *src* and *dst*
- specify the same file, :exc:`Error` is raised.
+ specify the same file, :exc:`SameFileError` is raised.
The destination location must be writable; otherwise, an :exc:`OSError`
exception will be raised. If *dst* already exists, it will be replaced.
@@ -69,6 +69,19 @@ Directory and files operations
Added *follow_symlinks* argument.
Now returns *dst*.
+ .. versionchanged:: 3.4
+ Raise :exc:`SameFileError` instead of :exc:`Error`. Since the former is
+ a subclass of the latter, this change is backward compatible.
+
+
+.. exception:: SameFileError
+
+ This exception is raised if source and destination in :func:`copyfile`
+ are the same file.
+
+ .. versionadded:: 3.4
+
+
.. function:: copymode(src, dst, *, follow_symlinks=True)
Copy the permission bits from *src* to *dst*. The file contents, owner, and
@@ -380,11 +393,10 @@ provided by this module. ::
errors.extend(err.args[0])
try:
copystat(src, dst)
- except WindowsError:
- # can't copy file access times on Windows
- pass
except OSError as why:
- errors.extend((src, dst, str(why)))
+ # can't copy file access times on Windows
+ if why.winerror is None:
+ errors.extend((src, dst, str(why)))
if errors:
raise Error(errors)
diff --git a/Doc/library/site.rst b/Doc/library/site.rst
index 36b80c3..d93e938 100644
--- a/Doc/library/site.rst
+++ b/Doc/library/site.rst
@@ -38,6 +38,9 @@ Unix and Macintosh). For each of the distinct head-tail combinations, it sees
if it refers to an existing directory, and if so, adds it to ``sys.path`` and
also inspects the newly added path for configuration files.
+.. deprecated:: 3.4
+ Support for the "site-python" directory will be removed in 3.5.
+
If a file named "pyvenv.cfg" exists one directory above sys.executable,
sys.prefix and sys.exec_prefix are set to that directory and
it is also checked for site-packages and site-python (sys.base_prefix and
@@ -111,6 +114,23 @@ empty, and the path manipulations are skipped; however the import of
:mod:`sitecustomize` and :mod:`usercustomize` is still attempted.
+.. _rlcompleter-config:
+
+Readline configuration
+----------------------
+
+On systems that support :mod:`readline`, this module will also import and
+configure the :mod:`rlcompleter` module, if Python is started in
+:ref:`interactive mode <tut-interactive>` and without the :option:`-S` option.
+The default behavior is enable tab-completion and to use
+:file:`~/.python_history` as the history save file. To disable it, override
+the :data:`sys.__interactivehook__` attribute in your :mod:`sitecustomize`
+or :mod:`usercustomize` module or your :envvar:`PYTHONSTARTUP` file.
+
+
+Module contents
+---------------
+
.. data:: PREFIXES
A list of prefixes for site-packages directories.
@@ -153,8 +173,7 @@ empty, and the path manipulations are skipped; however the import of
Adds all the standard site-specific directories to the module search
path. This function is called automatically when this module is imported,
- unless the :program:`python` interpreter was started with the :option:`-S`
- flag.
+ unless the Python interpreter was started with the :option:`-S` flag.
.. versionchanged:: 3.3
This function used to be called unconditionnally.
diff --git a/Doc/library/smtpd.rst b/Doc/library/smtpd.rst
index f305818..3ebed06 100644
--- a/Doc/library/smtpd.rst
+++ b/Doc/library/smtpd.rst
@@ -27,7 +27,8 @@ SMTPServer Objects
------------------
-.. class:: SMTPServer(localaddr, remoteaddr, data_size_limit=33554432)
+.. class:: SMTPServer(localaddr, remoteaddr, data_size_limit=33554432,\
+ map=None)
Create a new :class:`SMTPServer` object, which binds to local address
*localaddr*. It will treat *remoteaddr* as an upstream SMTP relayer. It
@@ -38,6 +39,8 @@ SMTPServer Objects
accepted in a ``DATA`` command. A value of ``None`` or ``0`` means no
limit.
+ A dictionary can be specified in *map* to avoid using a global socket map.
+
.. method:: process_message(peer, mailfrom, rcpttos, data)
Raise :exc:`NotImplementedError` exception. Override this in subclasses to
@@ -53,6 +56,9 @@ SMTPServer Objects
Override this in subclasses to use a custom :class:`SMTPChannel` for
managing SMTP clients.
+ .. versionchanged:: 3.4
+ The *map* argument was added.
+
DebuggingServer Objects
-----------------------
@@ -90,11 +96,20 @@ MailmanProxy Objects
SMTPChannel Objects
-------------------
-.. class:: SMTPChannel(server, conn, addr)
+.. class:: SMTPChannel(server, conn, addr, data_size_limit=33554432,\
+ map=None))
Create a new :class:`SMTPChannel` object which manages the communication
between the server and a single SMTP client.
+ *conn* and *addr* are as per the instance variables described below.
+
+ *data_size_limit* specifies the maximum number of bytes that will be
+ accepted in a ``DATA`` command. A value of ``None`` or ``0`` means no
+ limit.
+
+ A dictionary can be specified in *map* to avoid using a global socket map.
+
To use a custom SMTPChannel implementation you need to override the
:attr:`SMTPServer.channel_class` of your :class:`SMTPServer`.
diff --git a/Doc/library/smtplib.rst b/Doc/library/smtplib.rst
index addc6be..a7d1538 100644
--- a/Doc/library/smtplib.rst
+++ b/Doc/library/smtplib.rst
@@ -110,8 +110,8 @@ A nice selection of exceptions is defined as well:
.. exception:: SMTPException
- The base exception class for all the other exceptions provided by this
- module.
+ Subclass of :exc:`OSError` that is the base exception class for all
+ the other exceptions provided by this module.
.. exception:: SMTPServerDisconnected
diff --git a/Doc/library/socket.rst b/Doc/library/socket.rst
index aa18ae4..b01238c 100644
--- a/Doc/library/socket.rst
+++ b/Doc/library/socket.rst
@@ -107,8 +107,8 @@ created. Socket addresses are represented as follows:
.. versionadded:: 3.3
-- Certain other address families (:const:`AF_BLUETOOTH`, :const:`AF_PACKET`)
- support specific representations.
+- Certain other address families (:const:`AF_BLUETOOTH`, :const:`AF_PACKET`,
+ :const:`AF_CAN`) support specific representations.
.. XXX document them!
@@ -257,6 +257,16 @@ The module :mod:`socket` exports the following constants and functions:
.. versionadded:: 3.3
+.. data:: CAN_BCM
+ CAN_BCM_*
+
+ CAN_BCM, in the CAN protocol family, is the broadcast manager (BCM) protocol.
+ Broadcast manager constants, documented in the Linux documentation, are also
+ defined in the socket module.
+
+ Availability: Linux >= 2.6.25.
+
+ .. versionadded:: 3.4
.. data:: AF_RDS
PF_RDS
@@ -283,6 +293,11 @@ The module :mod:`socket` exports the following constants and functions:
TIPC related constants, matching the ones exported by the C socket API. See
the TIPC documentation for more information.
+.. data:: AF_LINK
+
+ Availability: BSD, OSX.
+
+ .. versionadded:: 3.4
.. data:: has_ipv6
@@ -445,20 +460,29 @@ The module :mod:`socket` exports the following constants and functions:
``'udp'``, otherwise any protocol will match.
-.. function:: socket([family[, type[, proto]]])
+.. function:: socket(family=AF_INET, type=SOCK_STREAM, proto=0, fileno=None)
Create a new socket using the given address family, socket type and protocol
number. The address family should be :const:`AF_INET` (the default),
:const:`AF_INET6`, :const:`AF_UNIX`, :const:`AF_CAN` or :const:`AF_RDS`. The
socket type should be :const:`SOCK_STREAM` (the default),
:const:`SOCK_DGRAM`, :const:`SOCK_RAW` or perhaps one of the other ``SOCK_``
- constants. The protocol number is usually zero and may be omitted in that
- case or :const:`CAN_RAW` in case the address family is :const:`AF_CAN`.
+ constants. The protocol number is usually zero and may be omitted or in the
+ case where the address family is :const:`AF_CAN` the protocol should be one
+ of :const:`CAN_RAW` or :const:`CAN_BCM`.
+
+ The newly created socket is :ref:`non-inheritable <fd_inheritance>`.
.. versionchanged:: 3.3
The AF_CAN family was added.
The AF_RDS family was added.
+ .. versionchanged:: 3.4
+ The CAN_BCM protocol was added.
+
+ .. versionchanged:: 3.4
+ The socket is now non-inheritable.
+
.. function:: socketpair([family[, type[, proto]]])
@@ -468,12 +492,17 @@ The module :mod:`socket` exports the following constants and functions:
if defined on the platform; otherwise, the default is :const:`AF_INET`.
Availability: Unix.
+ The newly created sockets are :ref:`non-inheritable <fd_inheritance>`.
+
.. versionchanged:: 3.2
The returned socket objects now support the whole socket API, rather
than a subset.
+ .. versionchanged:: 3.4
+ The sockets are now non-inheritable.
-.. function:: fromfd(fd, family, type[, proto])
+
+.. function:: fromfd(fd, family, type, proto=0)
Duplicate the file descriptor *fd* (an integer as returned by a file object's
:meth:`fileno` method) and build a socket object from the result. Address
@@ -484,6 +513,11 @@ The module :mod:`socket` exports the following constants and functions:
a socket passed to a program as standard input or output (such as a server
started by the Unix inet daemon). The socket is assumed to be in blocking mode.
+ The newly created socket is :ref:`non-inheritable <fd_inheritance>`.
+
+ .. versionchanged:: 3.4
+ The socket is now non-inheritable.
+
.. function:: ntohl(x)
@@ -560,7 +594,7 @@ The module :mod:`socket` exports the following constants and functions:
both the value of *address_family* and the underlying implementation of
:c:func:`inet_pton`.
- Availability: Unix (maybe not all platforms).
+ Availability: Unix (maybe not all platforms), Windows.
.. function:: inet_ntop(address_family, packed_ip)
@@ -576,7 +610,7 @@ The module :mod:`socket` exports the following constants and functions:
specified address family, :exc:`ValueError` will be raised. A
:exc:`OSError` is raised for errors from the call to :func:`inet_ntop`.
- Availability: Unix (maybe not all platforms).
+ Availability: Unix (maybe not all platforms), Windows.
..
@@ -712,6 +746,11 @@ correspond to Unix system calls applicable to sockets.
*new* socket object usable to send and receive data on the connection, and
*address* is the address bound to the socket on the other end of the connection.
+ The newly created socket is :ref:`non-inheritable <fd_inheritance>`.
+
+ .. versionchanged:: 3.4
+ The socket is now non-inheritable.
+
.. method:: socket.bind(address)
@@ -757,6 +796,16 @@ correspond to Unix system calls applicable to sockets.
.. versionadded:: 3.2
+.. method:: socket.dup()
+
+ Duplicate the socket.
+
+ The newly created socket is :ref:`non-inheritable <fd_inheritance>`.
+
+ .. versionchanged:: 3.4
+ The socket is now non-inheritable.
+
+
.. method:: socket.fileno()
Return the socket's file descriptor (a small integer). This is useful with
@@ -767,6 +816,15 @@ correspond to Unix system calls applicable to sockets.
this limitation.
+.. method:: socket.get_inheritable()
+
+ Get the :ref:`inheritable flag <fd_inheritance>` of the socket's file
+ descriptor or socket's handle: ``True`` if the socket can be inherited in
+ child processes, ``False`` if it cannot.
+
+ .. versionadded:: 3.4
+
+
.. method:: socket.getpeername()
Return the remote address to which the socket is connected. This is useful to
@@ -1050,6 +1108,14 @@ correspond to Unix system calls applicable to sockets.
.. versionadded:: 3.3
+.. method:: socket.set_inheritable(inheritable)
+
+ Set the :ref:`inheritable flag <fd_inheritance>` of the socket's file
+ descriptor or socket's handle.
+
+ .. versionadded:: 3.4
+
+
.. method:: socket.setblocking(flag)
Set blocking or non-blocking mode of the socket: if *flag* is false, the
@@ -1331,7 +1397,16 @@ the interface::
s.ioctl(socket.SIO_RCVALL, socket.RCVALL_OFF)
The last example shows how to use the socket interface to communicate to a CAN
-network. This example might require special priviledge::
+network using the raw socket protocol. To use CAN with the broadcast
+manager protocol instead, open a socket with::
+
+ socket.socket(socket.AF_CAN, socket.SOCK_DGRAM, socket.CAN_BCM)
+
+After binding (:const:`CAN_RAW`) or connecting (:const:`CAN_BCM`) the socket, you
+can use the :meth:`socket.send`, and the :meth:`socket.recv` operations (and
+their counterparts) on the socket object as usual.
+
+This example might require special priviledge::
import socket
import struct
diff --git a/Doc/library/spwd.rst b/Doc/library/spwd.rst
index add611f..58be78f 100644
--- a/Doc/library/spwd.rst
+++ b/Doc/library/spwd.rst
@@ -19,9 +19,9 @@ below, see ``<shadow.h>``):
+-------+---------------+---------------------------------+
| Index | Attribute | Meaning |
+=======+===============+=================================+
-| 0 | ``sp_nam`` | Login name |
+| 0 | ``sp_namp`` | Login name |
+-------+---------------+---------------------------------+
-| 1 | ``sp_pwd`` | Encrypted password |
+| 1 | ``sp_pwdp`` | Encrypted password |
+-------+---------------+---------------------------------+
| 2 | ``sp_lstchg`` | Date of last change |
+-------+---------------+---------------------------------+
@@ -36,15 +36,15 @@ below, see ``<shadow.h>``):
+-------+---------------+---------------------------------+
| 6 | ``sp_inact`` | Number of days after password |
| | | expires until account is |
-| | | blocked |
+| | | disabled |
+-------+---------------+---------------------------------+
| 7 | ``sp_expire`` | Number of days since 1970-01-01 |
-| | | until account is disabled |
+| | | when account expires |
+-------+---------------+---------------------------------+
| 8 | ``sp_flag`` | Reserved |
+-------+---------------+---------------------------------+
-The sp_nam and sp_pwd items are strings, all others are integers.
+The sp_namp and sp_pwdp items are strings, all others are integers.
:exc:`KeyError` is raised if the entry asked for cannot be found.
The following functions are defined:
diff --git a/Doc/library/sqlite3.rst b/Doc/library/sqlite3.rst
index a2bd48c..6d1f76a 100644
--- a/Doc/library/sqlite3.rst
+++ b/Doc/library/sqlite3.rst
@@ -159,7 +159,7 @@ Module functions and constants
first blank for the column name: the column name would simply be "x".
-.. function:: connect(database[, timeout, detect_types, isolation_level, check_same_thread, factory, cached_statements])
+.. function:: connect(database[, timeout, detect_types, isolation_level, check_same_thread, factory, cached_statements, uri])
Opens a connection to the SQLite database file *database*. You can use
``":memory:"`` to open a database connection to a database that resides in RAM
@@ -195,6 +195,18 @@ Module functions and constants
for the connection, you can set the *cached_statements* parameter. The currently
implemented default is to cache 100 statements.
+ If *uri* is true, *database* is interpreted as a URI. This allows you
+ to specify options. For example, to open a database in read-only mode
+ you can use::
+
+ db = sqlite3.connect('file:path/to/database?mode=ro', uri=True)
+
+ More information about this feature, including a list of recognized options, can
+ be found in the `SQLite URI documentation <http://www.sqlite.org/uri.html>`_.
+
+ .. versionchanged:: 3.4
+ Added the *uri* parameter.
+
.. function:: register_converter(typename, callable)
diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst
index c4e1712..9091aff 100644
--- a/Doc/library/ssl.rst
+++ b/Doc/library/ssl.rst
@@ -26,7 +26,8 @@ probably additional platforms, as long as OpenSSL is installed on that platform.
Some behavior may be platform dependent, since calls are made to the
operating system socket APIs. The installed version of OpenSSL may also
- cause variations in behavior.
+ cause variations in behavior. For example, TLSv1.1 and TLSv1.2 come with
+ openssl version 1.0.1.
.. warning::
Don't use this module without reading the :ref:`ssl-security`. Doing so
@@ -183,14 +184,16 @@ instead.
.. table::
- ======================== ========= ========= ========== =========
- *client* / **server** **SSLv2** **SSLv3** **SSLv23** **TLSv1**
- ------------------------ --------- --------- ---------- ---------
- *SSLv2* yes no yes no
- *SSLv3* no yes yes no
- *SSLv23* yes no yes no
- *TLSv1* no no yes yes
- ======================== ========= ========= ========== =========
+ ======================== ========= ========= ========== ========= =========== ===========
+ *client* / **server** **SSLv2** **SSLv3** **SSLv23** **TLSv1** **TLSv1.1** **TLSv1.2**
+ ------------------------ --------- --------- ---------- --------- ----------- -----------
+ *SSLv2* yes no yes no no no
+ *SSLv3* no yes yes no no no
+ *SSLv23* yes no yes no no no
+ *TLSv1* no no yes yes no no
+ *TLSv1.1* no no yes no yes no
+ *TLSv1.2* no no yes no no yes
+ ======================== ========= ========= ========== ========= =========== ===========
.. note::
@@ -224,6 +227,45 @@ instead.
.. versionchanged:: 3.2
New optional argument *ciphers*.
+
+Context creation
+^^^^^^^^^^^^^^^^
+
+A convenience function helps create :class:`SSLContext` objects for common
+purposes.
+
+.. function:: create_default_context(purpose=Purpose.SERVER_AUTH, cafile=None, capath=None, cadata=None)
+
+ Return a new :class:`SSLContext` object with default settings for
+ the given *purpose*. The settings are chosen by the :mod:`ssl` module,
+ and usually represent a higher security level than when calling the
+ :class:`SSLContext` constructor directly.
+
+ *cafile*, *capath*, *cadata* represent optional CA certificates to
+ trust for certificate verification, as in
+ :meth:`SSLContext.load_verify_locations`. If all three are
+ :const:`None`, this function can choose to trust the system's default
+ CA certificates instead.
+
+ The settings in Python 3.4 are: :data:`PROTOCOL_TLSv1` with high encryption
+ cipher suites without RC4 and without unauthenticated cipher suites.
+ Passing :data:`~Purpose.SERVER_AUTH` as *purpose* sets
+ :data:`~SSLContext.verify_mode` to :data:`CERT_REQUIRED` and either
+ loads CA certificates (when at least one of *cafile*, *capath* or *cadata*
+ is given) or uses :meth:`SSLContext.load_default_certs` to load default
+ CA certificates.
+
+ .. note::
+ The protocol, options, cipher and other settings may change to more
+ restrictive values anytime without prior deprecation. The values
+ represent a fair balance between compatibility and security.
+
+ If your application needs specific settings, you should create a
+ :class:`SSLContext` and apply the settings yourself.
+
+ .. versionadded:: 3.4
+
+
Random generation
^^^^^^^^^^^^^^^^^
@@ -353,6 +395,61 @@ Certificate handling
Given a certificate as an ASCII PEM string, returns a DER-encoded sequence of
bytes for that same certificate.
+.. function:: get_default_verify_paths()
+
+ Returns a named tuple with paths to OpenSSL's default cafile and capath.
+ The paths are the same as used by
+ :meth:`SSLContext.set_default_verify_paths`. The return value is a
+ :term:`named tuple` ``DefaultVerifyPaths``:
+
+ * :attr:`cafile` - resolved path to cafile or None if the file doesn't exist,
+ * :attr:`capath` - resolved path to capath or None if the directory doesn't exist,
+ * :attr:`openssl_cafile_env` - OpenSSL's environment key that points to a cafile,
+ * :attr:`openssl_cafile` - hard coded path to a cafile,
+ * :attr:`openssl_capath_env` - OpenSSL's environment key that points to a capath,
+ * :attr:`openssl_capath` - hard coded path to a capath directory
+
+ .. versionadded:: 3.4
+
+.. function:: enum_certificates(store_name)
+
+ Retrieve certificates from Windows' system cert store. *store_name* may be
+ one of ``CA``, ``ROOT`` or ``MY``. Windows may provide additional cert
+ stores, too.
+
+ The function returns a list of (cert_bytes, encoding_type, trust) tuples.
+ The encoding_type specifies the encoding of cert_bytes. It is either
+ :const:`x509_asn` for X.509 ASN.1 data or :const:`pkcs_7_asn` for
+ PKCS#7 ASN.1 data. Trust specifies the purpose of the certificate as a set
+ of OIDS or exactly ``True`` if the certificate is trustworthy for all
+ purposes.
+
+ Example::
+
+ >>> ssl.enum_certificates("CA")
+ [(b'data...', 'x509_asn', {'1.3.6.1.5.5.7.3.1', '1.3.6.1.5.5.7.3.2'}),
+ (b'data...', 'x509_asn', True)]
+
+ Availability: Windows.
+
+ .. versionadded:: 3.4
+
+.. function:: enum_crls(store_name)
+
+ Retrieve CRLs from Windows' system cert store. *store_name* may be
+ one of ``CA``, ``ROOT`` or ``MY``. Windows may provide additional cert
+ stores, too.
+
+ The function returns a list of (cert_bytes, encoding_type, trust) tuples.
+ The encoding_type specifies the encoding of cert_bytes. It is either
+ :const:`x509_asn` for X.509 ASN.1 data or :const:`pkcs_7_asn` for
+ PKCS#7 ASN.1 data.
+
+ Availability: Windows.
+
+ .. versionadded:: 3.4
+
+
Constants
^^^^^^^^^
@@ -389,6 +486,38 @@ Constants
be passed, either to :meth:`SSLContext.load_verify_locations` or as a
value of the ``ca_certs`` parameter to :func:`wrap_socket`.
+.. data:: VERIFY_DEFAULT
+
+ Possible value for :attr:`SSLContext.verify_flags`. In this mode,
+ certificate revocation lists (CRLs) are not checked. By default OpenSSL
+ does neither require nor verify CRLs.
+
+ .. versionadded:: 3.4
+
+.. data:: VERIFY_CRL_CHECK_LEAF
+
+ Possible value for :attr:`SSLContext.verify_flags`. In this mode, only the
+ peer cert is check but non of the intermediate CA certificates. The mode
+ requires a valid CRL that is signed by the peer cert's issuer (its direct
+ ancestor CA). If no proper has been loaded
+ :attr:`SSLContext.load_verify_locations`, validation will fail.
+
+ .. versionadded:: 3.4
+
+.. data:: VERIFY_CRL_CHECK_CHAIN
+
+ Possible value for :attr:`SSLContext.verify_flags`. In this mode, CRLs of
+ all certificates in the peer cert chain are checked.
+
+ .. versionadded:: 3.4
+
+.. data:: VERIFY_X509_STRICT
+
+ Possible value for :attr:`SSLContext.verify_flags` to disable workarounds
+ for broken X.509 certificates.
+
+ .. versionadded:: 3.4
+
.. data:: PROTOCOL_SSLv2
Selects SSL version 2 as the channel encryption protocol.
@@ -414,9 +543,25 @@ Constants
.. data:: PROTOCOL_TLSv1
- Selects TLS version 1 as the channel encryption protocol. This is the most
+ Selects TLS version 1.0 as the channel encryption protocol.
+
+.. data:: PROTOCOL_TLSv1_1
+
+
+ Selects TLS version 1.1 as the channel encryption protocol.
+ Available only with openssl version 1.0.1+.
+
+ .. versionadded:: 3.4
+
+.. data:: PROTOCOL_TLSv1_2
+
+
+ Selects TLS version 1.2 as the channel encryption protocol. This is the most
modern version, and probably the best choice for maximum protection, if both
sides can speak it.
+ Available only with openssl version 1.0.1+.
+
+ .. versionadded:: 3.4
.. data:: OP_ALL
@@ -450,6 +595,22 @@ Constants
.. versionadded:: 3.2
+.. data:: OP_NO_TLSv1_1
+
+ Prevents a TLSv1.1 connection. This option is only applicable in conjunction
+ with :const:`PROTOCOL_SSLv23`. It prevents the peers from choosing TLSv1.1 as
+ the protocol version. Available only with openssl version 1.0.1+.
+
+ .. versionadded:: 3.4
+
+.. data:: OP_NO_TLSv1_2
+
+ Prevents a TLSv1.2 connection. This option is only applicable in conjunction
+ with :const:`PROTOCOL_SSLv23`. It prevents the peers from choosing TLSv1.2 as
+ the protocol version. Available only with openssl version 1.0.1+.
+
+ .. versionadded:: 3.4
+
.. data:: OP_CIPHER_SERVER_PREFERENCE
Use the server's cipher ordering preference, rather than the client's.
@@ -546,6 +707,37 @@ Constants
.. versionadded:: 3.2
+.. data:: ALERT_DESCRIPTION_HANDSHAKE_FAILURE
+ ALERT_DESCRIPTION_INTERNAL_ERROR
+ ALERT_DESCRIPTION_*
+
+ Alert Descriptions from :rfc:`5246` and others. The `IANA TLS Alert Registry
+ <http://www.iana.org/assignments/tls-parameters/tls-parameters.xml#tls-parameters-6>`_
+ contains this list and references to the RFCs where their meaning is defined.
+
+ Used as the return value of the callback function in
+ :meth:`SSLContext.set_servername_callback`.
+
+ .. versionadded:: 3.4
+
+.. data:: Purpose.SERVER_AUTH
+
+ Option for :func:`create_default_context` and
+ :meth:`SSLContext.load_default_certs`. This value indicates that the
+ context may be used to authenticate Web servers (therefore, it will
+ be used to create client-side sockets).
+
+ .. versionadded:: 3.4
+
+.. data:: Purpose.CLIENT_AUTH
+
+ Option for :func:`create_default_context` and
+ :meth:`SSLContext.load_default_certs`. This value indicates that the
+ context may be used to authenticate Web clients (therefore, it will
+ be used to create server-side sockets).
+
+ .. versionadded:: 3.4
+
SSL Sockets
-----------
@@ -584,7 +776,8 @@ SSL sockets also have the following additional methods and attributes:
.. method:: SSLSocket.getpeercert(binary_form=False)
If there is no certificate for the peer on the other end of the connection,
- returns ``None``.
+ return ``None``. If the SSL handshake hasn't been done yet, raise
+ :exc:`ValueError`.
If the ``binary_form`` parameter is :const:`False`, and a certificate was
received from the peer, this method returns a :class:`dict` instance. If the
@@ -641,6 +834,13 @@ SSL sockets also have the following additional methods and attributes:
The returned dictionary includes additional items such as ``issuer``
and ``notBefore``.
+ .. versionchanged:: 3.4
+ :exc:`ValueError` is raised when the handshake isn't done.
+
+ .. versionchanged:: 3.4
+ The returned dictionary includes additional X509v3 extension items
+ such as ``crlDistributionPoints``, ``caIssuers`` and ``OCSP`` URIs.
+
.. method:: SSLSocket.cipher()
Returns a three-value tuple containing the name of the cipher being used, the
@@ -711,11 +911,30 @@ to speed up repeated connections from the same clients.
Create a new SSL context. You must pass *protocol* which must be one
of the ``PROTOCOL_*`` constants defined in this module.
- :data:`PROTOCOL_SSLv23` is recommended for maximum interoperability.
+ :data:`PROTOCOL_SSLv23` is currently recommended for maximum
+ interoperability.
+
+ .. seealso::
+ :func:`create_default_context` lets the :mod:`ssl` module choose
+ security settings for a given purpose.
:class:`SSLContext` objects have the following methods and attributes:
+.. method:: SSLContext.cert_store_stats()
+
+ Get statistics about quantities of loaded X.509 certificates, count of
+ X.509 certificates flagged as CA certificates and certificate revocation
+ lists as dictionary.
+
+ Example for a context with one CA cert and one other cert::
+
+ >>> context.cert_store_stats()
+ {'crl': 0, 'x509_ca': 1, 'x509': 2}
+
+ .. versionadded:: 3.4
+
+
.. method:: SSLContext.load_cert_chain(certfile, keyfile=None, password=None)
Load a private key and the corresponding certificate. The *certfile*
@@ -746,12 +965,32 @@ to speed up repeated connections from the same clients.
.. versionchanged:: 3.3
New optional argument *password*.
-.. method:: SSLContext.load_verify_locations(cafile=None, capath=None)
+.. method:: SSLContext.load_default_certs(purpose=Purpose.SERVER_AUTH)
+
+ Load a set of default "certification authority" (CA) certificates from
+ default locations. On Windows it loads CA certs from the ``CA`` and
+ ``ROOT`` system stores. On other systems it calls
+ :meth:`SSLContext.set_default_verify_paths`. In the future the method may
+ load CA certificates from other locations, too.
+
+ The *purpose* flag specifies what kind of CA certificates are loaded. The
+ default settings :data:`Purpose.SERVER_AUTH` loads certificates, that are
+ flagged and trusted for TLS web server authentication (client side
+ sockets). :data:`Purpose.CLIENT_AUTH` loads CA certificates for client
+ certificate verification on the server side.
+
+ .. versionadded:: 3.4
+
+.. method:: SSLContext.load_verify_locations(cafile=None, capath=None, cadata=None)
Load a set of "certification authority" (CA) certificates used to validate
other peers' certificates when :data:`verify_mode` is other than
:data:`CERT_NONE`. At least one of *cafile* or *capath* must be specified.
+ This method can also load certification revocation lists (CRLs) in PEM or
+ or DER format. In order to make use of CRLs, :attr:`SSLContext.verify_flags`
+ must be configured properly.
+
The *cafile* string, if present, is the path to a file of concatenated
CA certificates in PEM format. See the discussion of
:ref:`ssl-certificates` for more information about how to arrange the
@@ -762,6 +1001,25 @@ to speed up repeated connections from the same clients.
following an `OpenSSL specific layout
<http://www.openssl.org/docs/ssl/SSL_CTX_load_verify_locations.html>`_.
+ The *cadata* object, if present, is either an ASCII string of one or more
+ PEM-encoded certificates or a bytes-like object of DER-encoded
+ certificates. Like with *capath* extra lines around PEM-encoded
+ certificates are ignored but at least one certificate must be present.
+
+ .. versionchanged:: 3.4
+ New optional argument *cadata*
+
+.. method:: SSLContext.get_ca_certs(binary_form=False)
+
+ Get a list of loaded "certification authority" (CA) certificates. If the
+ ``binary_form`` parameter is :const:`False` each list
+ entry is a dict like the output of :meth:`SSLSocket.getpeercert`. Otherwise
+ the method returns a list of DER-encoded certificates. The returned list
+ does not contain certificates from *capath* unless a certificate was
+ requested and loaded by a SSL connection.
+
+ .. versionadded:: 3.4
+
.. method:: SSLContext.set_default_verify_paths()
Load a set of default "certification authority" (CA) certificates from
@@ -799,6 +1057,56 @@ to speed up repeated connections from the same clients.
.. versionadded:: 3.3
+.. method:: SSLContext.set_servername_callback(server_name_callback)
+
+ Register a callback function that will be called after the TLS Client Hello
+ handshake message has been received by the SSL/TLS server when the TLS client
+ specifies a server name indication. The server name indication mechanism
+ is specified in :rfc:`6066` section 3 - Server Name Indication.
+
+ Only one callback can be set per ``SSLContext``. If *server_name_callback*
+ is ``None`` then the callback is disabled. Calling this function a
+ subsequent time will disable the previously registered callback.
+
+ The callback function, *server_name_callback*, will be called with three
+ arguments; the first being the :class:`ssl.SSLSocket`, the second is a string
+ that represents the server name that the client is intending to communicate
+ (or :const:`None` if the TLS Client Hello does not contain a server name)
+ and the third argument is the original :class:`SSLContext`. The server name
+ argument is the IDNA decoded server name.
+
+ A typical use of this callback is to change the :class:`ssl.SSLSocket`'s
+ :attr:`SSLSocket.context` attribute to a new object of type
+ :class:`SSLContext` representing a certificate chain that matches the server
+ name.
+
+ Due to the early negotiation phase of the TLS connection, only limited
+ methods and attributes are usable like
+ :meth:`SSLSocket.selected_npn_protocol` and :attr:`SSLSocket.context`.
+ :meth:`SSLSocket.getpeercert`, :meth:`SSLSocket.getpeercert`,
+ :meth:`SSLSocket.cipher` and :meth:`SSLSocket.compress` methods require that
+ the TLS connection has progressed beyond the TLS Client Hello and therefore
+ will not contain return meaningful values nor can they be called safely.
+
+ The *server_name_callback* function must return ``None`` to allow the
+ TLS negotiation to continue. If a TLS failure is required, a constant
+ :const:`ALERT_DESCRIPTION_* <ALERT_DESCRIPTION_INTERNAL_ERROR>` can be
+ returned. Other return values will result in a TLS fatal error with
+ :const:`ALERT_DESCRIPTION_INTERNAL_ERROR`.
+
+ If there is a IDNA decoding error on the server name, the TLS connection
+ will terminate with an :const:`ALERT_DESCRIPTION_INTERNAL_ERROR` fatal TLS
+ alert message to the client.
+
+ If an exception is raised from the *server_name_callback* function the TLS
+ connection will terminate with a fatal TLS alert message
+ :const:`ALERT_DESCRIPTION_HANDSHAKE_FAILURE`.
+
+ This method will raise :exc:`NotImplementedError` if the OpenSSL library
+ had OPENSSL_NO_TLSEXT defined when it was built.
+
+ .. versionadded:: 3.4
+
.. method:: SSLContext.load_dh_params(dhfile)
Load the key generation parameters for Diffie-Helman (DH) key exchange.
@@ -862,6 +1170,18 @@ to speed up repeated connections from the same clients.
>>> stats['hits'], stats['misses']
(0, 0)
+.. method:: SSLContext.get_ca_certs(binary_form=False)
+
+ Returns a list of dicts with information of loaded CA certs. If the
+ optional argument is True, returns a DER-encoded copy of the CA
+ certificate.
+
+ .. note::
+ Certificates in a capath directory aren't loaded unless they have
+ been used at least once.
+
+ .. versionadded:: 3.4
+
.. attribute:: SSLContext.options
An integer representing the set of SSL options enabled on this context.
@@ -878,6 +1198,15 @@ to speed up repeated connections from the same clients.
The protocol version chosen when constructing the context. This attribute
is read-only.
+.. attribute:: SSLContext.verify_flags
+
+ The flags for certificate verification operations. You can set flags like
+ :data:`VERIFY_CRL_CHECK_LEAF` by ORing them together. By default OpenSSL
+ does neither require nor verify certificate revocation lists (CRLs).
+ Available only with openssl version 0.9.8+.
+
+ .. versionadded:: 3.4
+
.. attribute:: SSLContext.verify_mode
Whether to try to verify other peers' certificates and how to behave
@@ -1343,3 +1672,12 @@ successful call of :func:`~ssl.RAND_add`, :func:`~ssl.RAND_bytes` or
`RFC 4366: Transport Layer Security (TLS) Extensions <http://www.ietf.org/rfc/rfc4366>`_
Blake-Wilson et. al.
+
+ `RFC 5246: The Transport Layer Security (TLS) Protocol Version 1.2 <http://www.ietf.org/rfc/rfc5246>`_
+ T. Dierks et. al.
+
+ `RFC 6066: Transport Layer Security (TLS) Extensions <http://www.ietf.org/rfc/rfc6066>`_
+ D. Eastlake
+
+ `IANA TLS: Transport Layer Security (TLS) Parameters <http://www.iana.org/assignments/tls-parameters/tls-parameters.xml>`_
+ IANA
diff --git a/Doc/library/stat.rst b/Doc/library/stat.rst
index 6c20aa2..24769f6 100644
--- a/Doc/library/stat.rst
+++ b/Doc/library/stat.rst
@@ -6,7 +6,8 @@
os.lstat() and os.fstat().
.. sectionauthor:: Skip Montanaro <skip@automatrix.com>
-**Source code:** :source:`Lib/stat.py`
+**Source code:** :source:`Modules/_stat.c`
+ :source:`Lib/stat.py`
--------------
@@ -15,6 +16,9 @@ results of :func:`os.stat`, :func:`os.fstat` and :func:`os.lstat` (if they
exist). For complete details about the :c:func:`stat`, :c:func:`fstat` and
:c:func:`lstat` calls, consult the documentation for your system.
+.. versionchanged:: 3.4
+ The stat module is backed by a C implementation.
+
The :mod:`stat` module defines the following functions to test for specific file
types:
@@ -53,6 +57,24 @@ types:
Return non-zero if the mode is from a socket.
+.. function:: S_ISDOOR(mode)
+
+ Return non-zero if the mode is from a door.
+
+ .. versionadded:: 3.4
+
+.. function:: S_ISPORT(mode)
+
+ Return non-zero if the mode is from an event port.
+
+ .. versionadded:: 3.4
+
+.. function:: S_ISWHT(mode)
+
+ Return non-zero if the mode is from a whiteout.
+
+ .. versionadded:: 3.4
+
Two additional functions are defined for more general manipulation of the file's
mode:
@@ -113,6 +135,10 @@ readable string:
.. versionadded:: 3.3
+ .. versionchanged:: 3.4
+ The function supports :data:`S_IFDOOR`, :data:`S_IFPORT` and
+ :data:`S_IFWHT`.
+
All the variables below are simply symbolic indexes into the 10-tuple returned
by :func:`os.stat`, :func:`os.fstat` or :func:`os.lstat`.
@@ -210,6 +236,29 @@ Use of the functions above is more portable than use of the first set of flags:
FIFO.
+.. data:: S_IFDOOR
+
+ Door.
+
+ .. versionadded:: 3.4
+
+.. data:: S_IFPORT
+
+ Event port.
+
+ .. versionadded:: 3.4
+
+.. data:: S_IFWHT
+
+ Whiteout.
+
+ .. versionadded:: 3.4
+
+.. note::
+
+ :data:`S_IFDOOR`, :data:`S_IFPORT` or :data:`S_IFWHT` are defined as
+ 0 when the platform does not have support for the file types.
+
The following flags can also be used in the *mode* argument of :func:`os.chmod`:
.. data:: S_ISUID
diff --git a/Doc/library/statistics.rst b/Doc/library/statistics.rst
new file mode 100644
index 0000000..e6c5959
--- /dev/null
+++ b/Doc/library/statistics.rst
@@ -0,0 +1,408 @@
+:mod:`statistics` --- Mathematical statistics functions
+=======================================================
+
+.. module:: statistics
+ :synopsis: mathematical statistics functions
+.. moduleauthor:: Steven D'Aprano <steve+python@pearwood.info>
+.. sectionauthor:: Steven D'Aprano <steve+python@pearwood.info>
+
+.. versionadded:: 3.4
+
+.. testsetup:: *
+
+ from statistics import *
+ __name__ = '<doctest>'
+
+**Source code:** :source:`Lib/statistics.py`
+
+--------------
+
+This module provides functions for calculating mathematical statistics of
+numeric (:class:`Real`-valued) data.
+
+Averages and measures of central location
+-----------------------------------------
+
+These functions calculate an average or typical value from a population
+or sample.
+
+======================= =============================================
+:func:`mean` Arithmetic mean ("average") of data.
+:func:`median` Median (middle value) of data.
+:func:`median_low` Low median of data.
+:func:`median_high` High median of data.
+:func:`median_grouped` Median, or 50th percentile, of grouped data.
+:func:`mode` Mode (most common value) of discrete data.
+======================= =============================================
+
+Measures of spread
+------------------
+
+These functions calculate a measure of how much the population or sample
+tends to deviate from the typical or average values.
+
+======================= =============================================
+:func:`pstdev` Population standard deviation of data.
+:func:`pvariance` Population variance of data.
+:func:`stdev` Sample standard deviation of data.
+:func:`variance` Sample variance of data.
+======================= =============================================
+
+
+Function details
+----------------
+
+Note: The functions do not require the data given to them to be sorted.
+However, for reading convenience, most of the examples show sorted sequences.
+
+.. function:: mean(data)
+
+ Return the sample arithmetic mean of *data*, a sequence or iterator of
+ real-valued numbers.
+
+ The arithmetic mean is the sum of the data divided by the number of data
+ points. It is commonly called "the average", although it is only one of many
+ different mathematical averages. It is a measure of the central location of
+ the data.
+
+ If *data* is empty, :exc:`StatisticsError` will be raised.
+
+ Some examples of use:
+
+ .. doctest::
+
+ >>> mean([1, 2, 3, 4, 4])
+ 2.8
+ >>> mean([-1.0, 2.5, 3.25, 5.75])
+ 2.625
+
+ >>> from fractions import Fraction as F
+ >>> mean([F(3, 7), F(1, 21), F(5, 3), F(1, 3)])
+ Fraction(13, 21)
+
+ >>> from decimal import Decimal as D
+ >>> mean([D("0.5"), D("0.75"), D("0.625"), D("0.375")])
+ Decimal('0.5625')
+
+ .. note::
+
+ The mean is strongly affected by outliers and is not a robust estimator
+ for central location: the mean is not necessarily a typical example of the
+ data points. For more robust, although less efficient, measures of
+ central location, see :func:`median` and :func:`mode`. (In this case,
+ "efficient" refers to statistical efficiency rather than computational
+ efficiency.)
+
+ The sample mean gives an unbiased estimate of the true population mean,
+ which means that, taken on average over all the possible samples,
+ ``mean(sample)`` converges on the true mean of the entire population. If
+ *data* represents the entire population rather than a sample, then
+ ``mean(data)`` is equivalent to calculating the true population mean μ.
+
+
+.. function:: median(data)
+
+ Return the median (middle value) of numeric data, using the common "mean of
+ middle two" method. If *data* is empty, :exc:`StatisticsError` is raised.
+
+ The median is a robust measure of central location, and is less affected by
+ the presence of outliers in your data. When the number of data points is
+ odd, the middle data point is returned:
+
+ .. doctest::
+
+ >>> median([1, 3, 5])
+ 3
+
+ When the number of data points is even, the median is interpolated by taking
+ the average of the two middle values:
+
+ .. doctest::
+
+ >>> median([1, 3, 5, 7])
+ 4.0
+
+ This is suited for when your data is discrete, and you don't mind that the
+ median may not be an actual data point.
+
+ .. seealso:: :func:`median_low`, :func:`median_high`, :func:`median_grouped`
+
+
+.. function:: median_low(data)
+
+ Return the low median of numeric data. If *data* is empty,
+ :exc:`StatisticsError` is raised.
+
+ The low median is always a member of the data set. When the number of data
+ points is odd, the middle value is returned. When it is even, the smaller of
+ the two middle values is returned.
+
+ .. doctest::
+
+ >>> median_low([1, 3, 5])
+ 3
+ >>> median_low([1, 3, 5, 7])
+ 3
+
+ Use the low median when your data are discrete and you prefer the median to
+ be an actual data point rather than interpolated.
+
+
+.. function:: median_high(data)
+
+ Return the high median of data. If *data* is empty, :exc:`StatisticsError`
+ is raised.
+
+ The high median is always a member of the data set. When the number of data
+ points is odd, the middle value is returned. When it is even, the larger of
+ the two middle values is returned.
+
+ .. doctest::
+
+ >>> median_high([1, 3, 5])
+ 3
+ >>> median_high([1, 3, 5, 7])
+ 5
+
+ Use the high median when your data are discrete and you prefer the median to
+ be an actual data point rather than interpolated.
+
+
+.. function:: median_grouped(data, interval=1)
+
+ Return the median of grouped continuous data, calculated as the 50th
+ percentile, using interpolation. If *data* is empty, :exc:`StatisticsError`
+ is raised.
+
+ .. doctest::
+
+ >>> median_grouped([52, 52, 53, 54])
+ 52.5
+
+ In the following example, the data are rounded, so that each value represents
+ the midpoint of data classes, e.g. 1 is the midpoint of the class 0.5-1.5, 2
+ is the midpoint of 1.5-2.5, 3 is the midpoint of 2.5-3.5, etc. With the data
+ given, the middle value falls somewhere in the class 3.5-4.5, and
+ interpolation is used to estimate it:
+
+ .. doctest::
+
+ >>> median_grouped([1, 2, 2, 3, 4, 4, 4, 4, 4, 5])
+ 3.7
+
+ Optional argument *interval* represents the class interval, and defaults
+ to 1. Changing the class interval naturally will change the interpolation:
+
+ .. doctest::
+
+ >>> median_grouped([1, 3, 3, 5, 7], interval=1)
+ 3.25
+ >>> median_grouped([1, 3, 3, 5, 7], interval=2)
+ 3.5
+
+ This function does not check whether the data points are at least
+ *interval* apart.
+
+ .. impl-detail::
+
+ Under some circumstances, :func:`median_grouped` may coerce data points to
+ floats. This behaviour is likely to change in the future.
+
+ .. seealso::
+
+ * "Statistics for the Behavioral Sciences", Frederick J Gravetter and
+ Larry B Wallnau (8th Edition).
+
+ * Calculating the `median <http://www.ualberta.ca/~opscan/median.html>`_.
+
+ * The `SSMEDIAN
+ <https://projects.gnome.org/gnumeric/doc/gnumeric-function-SSMEDIAN.shtml>`_
+ function in the Gnome Gnumeric spreadsheet, including `this discussion
+ <https://mail.gnome.org/archives/gnumeric-list/2011-April/msg00018.html>`_.
+
+
+.. function:: mode(data)
+
+ Return the most common data point from discrete or nominal *data*. The mode
+ (when it exists) is the most typical value, and is a robust measure of
+ central location.
+
+ If *data* is empty, or if there is not exactly one most common value,
+ :exc:`StatisticsError` is raised.
+
+ ``mode`` assumes discrete data, and returns a single value. This is the
+ standard treatment of the mode as commonly taught in schools:
+
+ .. doctest::
+
+ >>> mode([1, 1, 2, 3, 3, 3, 3, 4])
+ 3
+
+ The mode is unique in that it is the only statistic which also applies
+ to nominal (non-numeric) data:
+
+ .. doctest::
+
+ >>> mode(["red", "blue", "blue", "red", "green", "red", "red"])
+ 'red'
+
+
+.. function:: pstdev(data, mu=None)
+
+ Return the population standard deviation (the square root of the population
+ variance). See :func:`pvariance` for arguments and other details.
+
+ .. doctest::
+
+ >>> pstdev([1.5, 2.5, 2.5, 2.75, 3.25, 4.75])
+ 0.986893273527251
+
+
+.. function:: pvariance(data, mu=None)
+
+ Return the population variance of *data*, a non-empty iterable of real-valued
+ numbers. Variance, or second moment about the mean, is a measure of the
+ variability (spread or dispersion) of data. A large variance indicates that
+ the data is spread out; a small variance indicates it is clustered closely
+ around the mean.
+
+ If the optional second argument *mu* is given, it should be the mean of
+ *data*. If it is missing or ``None`` (the default), the mean is
+ automatically calculated.
+
+ Use this function to calculate the variance from the entire population. To
+ estimate the variance from a sample, the :func:`variance` function is usually
+ a better choice.
+
+ Raises :exc:`StatisticsError` if *data* is empty.
+
+ Examples:
+
+ .. doctest::
+
+ >>> data = [0.0, 0.25, 0.25, 1.25, 1.5, 1.75, 2.75, 3.25]
+ >>> pvariance(data)
+ 1.25
+
+ If you have already calculated the mean of your data, you can pass it as the
+ optional second argument *mu* to avoid recalculation:
+
+ .. doctest::
+
+ >>> mu = mean(data)
+ >>> pvariance(data, mu)
+ 1.25
+
+ This function does not attempt to verify that you have passed the actual mean
+ as *mu*. Using arbitrary values for *mu* may lead to invalid or impossible
+ results.
+
+ Decimals and Fractions are supported:
+
+ .. doctest::
+
+ >>> from decimal import Decimal as D
+ >>> pvariance([D("27.5"), D("30.25"), D("30.25"), D("34.5"), D("41.75")])
+ Decimal('24.815')
+
+ >>> from fractions import Fraction as F
+ >>> pvariance([F(1, 4), F(5, 4), F(1, 2)])
+ Fraction(13, 72)
+
+ .. note::
+
+ When called with the entire population, this gives the population variance
+ σ². When called on a sample instead, this is the biased sample variance
+ s², also known as variance with N degrees of freedom.
+
+ If you somehow know the true population mean μ, you may use this function
+ to calculate the variance of a sample, giving the known population mean as
+ the second argument. Provided the data points are representative
+ (e.g. independent and identically distributed), the result will be an
+ unbiased estimate of the population variance.
+
+
+.. function:: stdev(data, xbar=None)
+
+ Return the sample standard deviation (the square root of the sample
+ variance). See :func:`variance` for arguments and other details.
+
+ .. doctest::
+
+ >>> stdev([1.5, 2.5, 2.5, 2.75, 3.25, 4.75])
+ 1.0810874155219827
+
+
+.. function:: variance(data, xbar=None)
+
+ Return the sample variance of *data*, an iterable of at least two real-valued
+ numbers. Variance, or second moment about the mean, is a measure of the
+ variability (spread or dispersion) of data. A large variance indicates that
+ the data is spread out; a small variance indicates it is clustered closely
+ around the mean.
+
+ If the optional second argument *xbar* is given, it should be the mean of
+ *data*. If it is missing or ``None`` (the default), the mean is
+ automatically calculated.
+
+ Use this function when your data is a sample from a population. To calculate
+ the variance from the entire population, see :func:`pvariance`.
+
+ Raises :exc:`StatisticsError` if *data* has fewer than two values.
+
+ Examples:
+
+ .. doctest::
+
+ >>> data = [2.75, 1.75, 1.25, 0.25, 0.5, 1.25, 3.5]
+ >>> variance(data)
+ 1.3720238095238095
+
+ If you have already calculated the mean of your data, you can pass it as the
+ optional second argument *xbar* to avoid recalculation:
+
+ .. doctest::
+
+ >>> m = mean(data)
+ >>> variance(data, m)
+ 1.3720238095238095
+
+ This function does not attempt to verify that you have passed the actual mean
+ as *xbar*. Using arbitrary values for *xbar* can lead to invalid or
+ impossible results.
+
+ Decimal and Fraction values are supported:
+
+ .. doctest::
+
+ >>> from decimal import Decimal as D
+ >>> variance([D("27.5"), D("30.25"), D("30.25"), D("34.5"), D("41.75")])
+ Decimal('31.01875')
+
+ >>> from fractions import Fraction as F
+ >>> variance([F(1, 6), F(1, 2), F(5, 3)])
+ Fraction(67, 108)
+
+ .. note::
+
+ This is the sample variance s² with Bessel's correction, also known as
+ variance with N-1 degrees of freedom. Provided that the data points are
+ representative (e.g. independent and identically distributed), the result
+ should be an unbiased estimate of the true population variance.
+
+ If you somehow know the actual population mean μ you should pass it to the
+ :func:`pvariance` function as the *mu* parameter to get the variance of a
+ sample.
+
+Exceptions
+----------
+
+A single exception is defined:
+
+.. exception:: StatisticsError
+
+ Subclass of :exc:`ValueError` for statistics-related exceptions.
+
+..
+ # This modelines must appear within the last ten lines of the file.
+ kate: indent-width 3; remove-trailing-space on; replace-tabs on; encoding utf-8;
diff --git a/Doc/library/stdtypes.rst b/Doc/library/stdtypes.rst
index db3ef88..6635b0e 100644
--- a/Doc/library/stdtypes.rst
+++ b/Doc/library/stdtypes.rst
@@ -1526,7 +1526,7 @@ expression support in the :mod:`re` module).
at that position.
-.. method:: str.expandtabs([tabsize])
+.. method:: str.expandtabs(tabsize=8)
Return a copy of the string where all tab characters are replaced by one or
more spaces, depending on the current column and the given tab size. Tab
@@ -2462,6 +2462,10 @@ copying.
.. versionchanged:: 3.3
One-dimensional memoryviews with formats 'B', 'b' or 'c' are now hashable.
+ .. versionchanged:: 3.4
+ memoryview is now registered automatically with
+ :class:`collections.abc.Sequence`
+
:class:`memoryview` has several methods:
.. method:: __eq__(exporter)
diff --git a/Doc/library/struct.rst b/Doc/library/struct.rst
index 994506c..f2ea361 100644
--- a/Doc/library/struct.rst
+++ b/Doc/library/struct.rst
@@ -66,6 +66,19 @@ The module defines the following exception and functions:
format (``len(buffer[offset:])`` must be at least ``calcsize(fmt)``).
+.. function:: iter_unpack(fmt, buffer)
+
+ Iteratively unpack from the buffer *buffer* according to the format
+ string *fmt*. This function returns an iterator which will read
+ equally-sized chunks from the buffer until all its contents have been
+ consumed. The buffer's size in bytes must be a multiple of the amount
+ of data required by the format, as reflected by :func:`calcsize`.
+
+ Each iteration yields a tuple as specified by the format string.
+
+ .. versionadded:: 3.4
+
+
.. function:: calcsize(fmt)
Return the size of the struct (and hence of the bytes object produced by
@@ -388,6 +401,13 @@ The :mod:`struct` module also defines the following type:
(``len(buffer[offset:])`` must be at least :attr:`self.size`).
+ .. method:: iter_unpack(buffer)
+
+ Identical to the :func:`iter_unpack` function, using the compiled format.
+ (``len(buffer)`` must be a multiple of :attr:`self.size`).
+
+ .. versionadded:: 3.4
+
.. attribute:: format
The format string used to construct this Struct object.
diff --git a/Doc/library/subprocess.rst b/Doc/library/subprocess.rst
index 49a3657..2e88319 100644
--- a/Doc/library/subprocess.rst
+++ b/Doc/library/subprocess.rst
@@ -116,7 +116,7 @@ use cases, the underlying :class:`Popen` interface can be used directly.
*timeout* was added.
-.. function:: check_output(args, *, stdin=None, stderr=None, shell=False, universal_newlines=False, timeout=None)
+.. function:: check_output(args, *, input=None, stdin=None, stderr=None, shell=False, universal_newlines=False, timeout=None)
Run command with arguments and return its output.
@@ -129,15 +129,21 @@ use cases, the underlying :class:`Popen` interface can be used directly.
in :ref:`frequently-used-arguments` (hence the use of keyword-only notation
in the abbreviated signature). The full function signature is largely the
same as that of the :class:`Popen` constructor - this functions passes all
- supplied arguments other than *timeout* directly through to that interface.
- In addition, *stdout* is not permitted as an argument, as it is used
- internally to collect the output from the subprocess.
+ supplied arguments other than *input* and *timeout* directly through to
+ that interface. In addition, *stdout* is not permitted as an argument, as
+ it is used internally to collect the output from the subprocess.
The *timeout* argument is passed to :meth:`Popen.wait`. If the timeout
expires, the child process will be killed and then waited for again. The
:exc:`TimeoutExpired` exception will be re-raised after the child process
has terminated.
+ The *input* argument is passed to :meth:`Popen.communicate` and thus to the
+ subprocess's stdin. If used it must be a byte sequence, or a string if
+ ``universal_newlines=True``. When used, the internal :class:`Popen` object
+ is automatically created with ``stdin=PIPE``, and the *stdin* argument may
+ not be used as well.
+
Examples::
>>> subprocess.check_output(["echo", "Hello World!"])
@@ -146,6 +152,10 @@ use cases, the underlying :class:`Popen` interface can be used directly.
>>> subprocess.check_output(["echo", "Hello World!"], universal_newlines=True)
'Hello World!\n'
+ >>> subprocess.check_output(["sed", "-e", "s/foo/bar/"],
+ ... input=b"when in the course of fooman events\n")
+ b'when in the course of barman events\n'
+
>>> subprocess.check_output("exit 1", shell=True)
Traceback (most recent call last):
...
@@ -167,10 +177,6 @@ use cases, the underlying :class:`Popen` interface can be used directly.
... shell=True)
'ls: non_existent_file: No such file or directory\n'
- .. versionadded:: 3.1
-
- ..
-
.. warning::
Invoking the system shell with ``shell=True`` can be a security hazard
@@ -183,9 +189,13 @@ use cases, the underlying :class:`Popen` interface can be used directly.
read in the current process, the child process may block if it
generates enough output to the pipe to fill up the OS pipe buffer.
+ .. versionadded:: 3.1
+
.. versionchanged:: 3.3
*timeout* was added.
+ .. versionchanged:: 3.4
+ *input* was added.
.. data:: DEVNULL
@@ -834,8 +844,6 @@ The :mod:`subprocess` module exposes the following constants.
The new process has a new console, instead of inheriting its parent's
console (the default).
- This flag is always set when :class:`Popen` is created with ``shell=True``.
-
.. data:: CREATE_NEW_PROCESS_GROUP
A :class:`Popen` ``creationflags`` parameter to specify that a new process
@@ -1050,9 +1058,9 @@ handling consistency are valid for these functions.
Return ``(status, output)`` of executing *cmd* in a shell.
- Execute the string *cmd* in a shell with :class:`Popen` and return a 2-tuple
- ``(status, output)`` via :func:`Popen.communicate`. Universal newlines mode
- is used; see the notes on :ref:`frequently-used-arguments` for more details.
+ Execute the string *cmd* in a shell with :meth:`Popen.check_output` and
+ return a 2-tuple ``(status, output)``. Universal newlines mode is used;
+ see the notes on :ref:`frequently-used-arguments` for more details.
A trailing newline is stripped from the output.
The exit status for the command can be interpreted
diff --git a/Doc/library/sunau.rst b/Doc/library/sunau.rst
index 4bdb99b..15c06b5 100644
--- a/Doc/library/sunau.rst
+++ b/Doc/library/sunau.rst
@@ -150,8 +150,9 @@ AU_read objects, as returned by :func:`.open` above, have the following methods:
.. method:: AU_read.getparams()
- Returns a tuple ``(nchannels, sampwidth, framerate, nframes, comptype,
- compname)``, equivalent to output of the :meth:`get\*` methods.
+ Returns a :func:`~collections.namedtuple` ``(nchannels, sampwidth,
+ framerate, nframes, comptype, compname)``, equivalent to output of the
+ :meth:`get\*` methods.
.. method:: AU_read.readframes(n)
@@ -211,6 +212,9 @@ AU_write objects, as returned by :func:`.open` above, have the following methods
Set the sample width (in bytes.)
+ .. versionchanged:: 3.4
+ Added support for 24-bit samples.
+
.. method:: AU_write.setframerate(n)
@@ -246,11 +250,17 @@ AU_write objects, as returned by :func:`.open` above, have the following methods
Write audio frames, without correcting *nframes*.
+ .. versionchanged:: 3.4
+ Any :term:`bytes-like object`\ s are now accepted.
+
.. method:: AU_write.writeframes(data)
Write audio frames and make sure *nframes* is correct.
+ .. versionchanged:: 3.4
+ Any :term:`bytes-like object`\ s are now accepted.
+
.. method:: AU_write.close()
diff --git a/Doc/library/sys.rst b/Doc/library/sys.rst
index bc1d9fe..b885de8 100644
--- a/Doc/library/sys.rst
+++ b/Doc/library/sys.rst
@@ -380,6 +380,21 @@ always available.
.. versionadded:: 3.1
+.. function:: getallocatedblocks()
+
+ Return the number of memory blocks currently allocated by the interpreter,
+ regardless of their size. This function is mainly useful for tracking
+ and debugging memory leaks. Because of the interpreter's internal
+ caches, the result can vary from call to call; you may have to call
+ :func:`_clear_type_cache()` and :func:`gc.collect()` to get more
+ predictable results.
+
+ If a Python build or implementation cannot reasonably compute this
+ information, :func:`getallocatedblocks()` is allowed to return 0 instead.
+
+ .. versionadded:: 3.4
+
+
.. function:: getcheckinterval()
Return the interpreter's "check interval"; see :func:`setcheckinterval`.
@@ -396,9 +411,10 @@ always available.
.. function:: getdlopenflags()
- Return the current value of the flags that are used for :c:func:`dlopen` calls.
- The flag constants are defined in the :mod:`ctypes` and :mod:`DLFCN` modules.
- Availability: Unix.
+ Return the current value of the flags that are used for
+ :c:func:`dlopen` calls. Symbolic names for the flag values can be
+ found in the :mod:`os` module (``RTLD_xxx`` constants, e.g.
+ :data:`os.RTLD_LAZY`). Availability: Unix.
.. function:: getfilesystemencoding()
@@ -578,9 +594,20 @@ always available.
| :const:`imag` | multiplier used for the imaginary part of a |
| | complex number |
+---------------------+--------------------------------------------------+
+ | :const:`algorithm` | name of the algorithm for hashing of str, bytes, |
+ | | and memoryview |
+ +---------------------+--------------------------------------------------+
+ | :const:`hash_bits` | internal output size of the hash algorithm |
+ +---------------------+--------------------------------------------------+
+ | :const:`seed_bits` | size of the seed key of the hash algorithm |
+ +---------------------+--------------------------------------------------+
+
.. versionadded:: 3.2
+ .. versionchanged: 3.4
+ Added *algorithm*, *hash_bits* and *seed_bits*
+
.. data:: hexversion
@@ -664,6 +691,16 @@ always available.
.. versionadded:: 3.1
+.. data:: __interactivehook__
+
+ When present, this function is automatically called (with no arguments)
+ when the interpreter is launched in :ref:`interactive mode <tut-interactive>`.
+ This is done after the :envvar:`PYTHONSTARTUP` file is read, so that you
+ can set this hook there.
+
+ .. versionadded:: 3.4
+
+
.. function:: intern(string)
Enter *string* in the table of "interned" strings and return the interned string
@@ -811,8 +848,6 @@ always available.
Windows ``'win32'``
Windows/Cygwin ``'cygwin'``
Mac OS X ``'darwin'``
- OS/2 ``'os2'``
- OS/2 EMX ``'os2emx'``
================ ===========================
.. versionchanged:: 3.3
@@ -883,7 +918,7 @@ always available.
the interpreter loads extension modules. Among other things, this will enable a
lazy resolving of symbols when importing a module, if called as
``sys.setdlopenflags(0)``. To share symbols across extension modules, call as
- ``sys.setdlopenflags(os.RTLD_GLOBAL)``. Symbolic names for the flag modules
+ ``sys.setdlopenflags(os.RTLD_GLOBAL)``. Symbolic names for the flag values
can be found in the :mod:`os` module (``RTLD_xxx`` constants, e.g.
:data:`os.RTLD_LAZY`).
@@ -1093,7 +1128,6 @@ always available.
| :const:`name` | Name of the thread implementation: |
| | |
| | * ``'nt'``: Windows threads |
- | | * ``'os2'``: OS/2 threads |
| | * ``'pthread'``: POSIX threads |
| | * ``'solaris'``: Solaris threads |
+------------------+---------------------------------------------------------+
diff --git a/Doc/library/sysconfig.rst b/Doc/library/sysconfig.rst
index c47dcce..535ac54 100644
--- a/Doc/library/sysconfig.rst
+++ b/Doc/library/sysconfig.rst
@@ -83,8 +83,6 @@ Python currently supports seven schemes:
located under the user home directory.
- *nt*: scheme for NT platforms like Windows.
- *nt_user*: scheme for NT platforms, when the *user* option is used.
-- *os2*: scheme for OS/2 platforms.
-- *os2_home*: scheme for OS/2 patforms, when the *user* option is used.
Each scheme is itself composed of a series of paths and each path has a unique
identifier. Python currently uses eight paths:
diff --git a/Doc/library/test.rst b/Doc/library/test.rst
index c1270f4..2c51549 100644
--- a/Doc/library/test.rst
+++ b/Doc/library/test.rst
@@ -442,13 +442,6 @@ The :mod:`test.support` module defines the following functions:
A decorator for running tests that require support for symbolic links.
-.. function:: suppress_crash_popup()
-
- A context manager that disables Windows Error Reporting dialogs using
- `SetErrorMode <http://msdn.microsoft.com/en-us/library/windows/desktop/ms680621%28v=vs.85%29.aspx>`_.
- On other platforms it's a no-op.
-
-
.. decorator:: anticipate_failure(condition)
A decorator to conditionally mark tests with
@@ -593,6 +586,21 @@ The :mod:`test.support` module defines the following classes:
Temporarily unset the environment variable ``envvar``.
+.. class:: SuppressCrashReport()
+
+ A context manager used to try to prevent crash dialog popups on tests that
+ are expected to crash a subprocess.
+
+ On Windows, it disables Windows Error Reporting dialogs using
+ `SetErrorMode <http://msdn.microsoft.com/en-us/library/windows/desktop/ms680621.aspx>`_.
+
+ On UNIX, :func:`resource.setrlimit` is used to set
+ :attr:`resource.RLIMIT_CORE`'s soft limit to 0 to prevent coredump file
+ creation.
+
+ On both platforms, the old value is restored by :meth:`__exit__`.
+
+
.. class:: WarningsRecorder()
Class used to record warnings for unit tests. See documentation of
diff --git a/Doc/library/textwrap.rst b/Doc/library/textwrap.rst
index c625254..1ba42a3 100644
--- a/Doc/library/textwrap.rst
+++ b/Doc/library/textwrap.rst
@@ -10,11 +10,11 @@
--------------
-The :mod:`textwrap` module provides two convenience functions, :func:`wrap` and
-:func:`fill`, as well as :class:`TextWrapper`, the class that does all the work,
-and two utility functions, :func:`dedent` and :func:`indent`. If you're just wrapping or filling one
-or two text strings, the convenience functions should be good enough;
-otherwise, you should use an instance of :class:`TextWrapper` for efficiency.
+The :mod:`textwrap` module provides some convenience functions,
+as well as :class:`TextWrapper`, the class that does all the work.
+If you're just wrapping or filling one or two text strings, the convenience
+functions should be good enough; otherwise, you should use an instance of
+:class:`TextWrapper` for efficiency.
.. function:: wrap(text, width=70, **kwargs)
@@ -39,19 +39,24 @@ otherwise, you should use an instance of :class:`TextWrapper` for efficiency.
In particular, :func:`fill` accepts exactly the same keyword arguments as
:func:`wrap`.
-Both :func:`wrap` and :func:`fill` work by creating a :class:`TextWrapper`
-instance and calling a single method on it. That instance is not reused, so for
-applications that wrap/fill many text strings, it will be more efficient for you
-to create your own :class:`TextWrapper` object.
-Text is preferably wrapped on whitespaces and right after the hyphens in
-hyphenated words; only then will long words be broken if necessary, unless
-:attr:`TextWrapper.break_long_words` is set to false.
+.. function:: shorten(text, width=70, *, placeholder=" [...]")
+
+ Collapse and truncate the given text to fit in the given width.
+
+ The text first has its whitespace collapsed. If it then fits in
+ the *width*, it is returned unchanged. Otherwise, as many words
+ as possible are joined and then the *placeholder* is appended::
+
+ >>> textwrap.shorten("Hello world!", width=12)
+ 'Hello world!'
+ >>> textwrap.shorten("Hello world!", width=11)
+ 'Hello [...]'
+ >>> textwrap.shorten("Hello world", width=10, placeholder="...")
+ 'Hello...'
+
+ .. versionadded:: 3.4
-Two additional utility function, :func:`dedent` and :func:`indent`, are
-provided to remove indentation from strings that have unwanted whitespace
-to the left of the text and to add an arbitrary prefix to selected lines
-in a block of text.
.. function:: dedent(text)
@@ -102,6 +107,16 @@ in a block of text.
+ world
+:func:`wrap`, :func:`fill` and :func:`shorten` work by creating a
+:class:`TextWrapper` instance and calling a single method on it. That
+instance is not reused, so for applications that process many text
+strings, it may be more efficient to create your own
+:class:`TextWrapper` object.
+
+Text is preferably wrapped on whitespaces and right after the hyphens in
+hyphenated words; only then will long words be broken if necessary, unless
+:attr:`TextWrapper.break_long_words` is set to false.
+
.. class:: TextWrapper(**kwargs)
The :class:`TextWrapper` constructor accepts a number of optional keyword
@@ -235,7 +250,23 @@ in a block of text.
was to always allow breaking hyphenated words.
- :class:`TextWrapper` also provides two public methods, analogous to the
+ .. attribute:: max_lines
+
+ (default: ``None``) If not ``None``, then the text be will truncated to
+ *max_lines* lines.
+
+ .. versionadded:: 3.4
+
+
+ .. attribute:: placeholder
+
+ (default: ``' [...]'``) String that will be appended to the last line of
+ text if it will be truncated.
+
+ .. versionadded:: 3.4
+
+
+ :class:`TextWrapper` also provides some public methods, analogous to the
module-level convenience functions:
.. method:: wrap(text)
@@ -251,4 +282,3 @@ in a block of text.
Wraps the single paragraph in *text*, and returns a single string
containing the wrapped paragraph.
-
diff --git a/Doc/library/threading.rst b/Doc/library/threading.rst
index 7c326d1..4a3b3ea 100644
--- a/Doc/library/threading.rst
+++ b/Doc/library/threading.rst
@@ -57,6 +57,15 @@ This module defines the following functions:
and threads that have not yet been started.
+.. function:: main_thread()
+
+ Return the main :class:`Thread` object. In normal conditions, the
+ main thread is the thread from which the Python interpreter was
+ started.
+
+ .. versionadded:: 3.4
+
+
.. function:: settrace(func)
.. index:: single: trace function
diff --git a/Doc/library/timeit.rst b/Doc/library/timeit.rst