summaryrefslogtreecommitdiffstats
path: root/Doc/library/compileall.rst
blob: 22d1c6bfa416807f52eb6227dea8909b3c376431 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
:mod:`compileall` --- Byte-compile Python libraries
===================================================

.. module:: compileall
   :synopsis: Tools for byte-compiling all Python source files in a directory tree.

**Source code:** :source:`Lib/compileall.py`

--------------

This module provides some utility functions to support installing Python
libraries.  These functions compile Python source files in a directory tree.
This module can be used to create the cached byte-code files at library
installation time, which makes them available for use even by users who don't
have write permission to the library directories.


Command-line use
----------------

This module can work as a script (using :program:`python -m compileall`) to
compile Python sources.

.. program:: compileall

.. cmdoption:: directory ...
               file ...

   Positional arguments are files to compile or directories that contain
   source files, traversed recursively.  If no argument is given, behave as if
   the command line was ``-l <directories from sys.path>``.

.. cmdoption:: -l

   Do not recurse into subdirectories, only compile source code files directly
   contained in the named or implied directories.

.. cmdoption:: -f

   Force rebuild even if timestamps are up-to-date.

.. cmdoption:: -q

   Do not print the list of files compiled. If passed once, error messages will
   still be printed. If passed twice (``-qq``), all output is suppressed.

.. cmdoption:: -d destdir

   Directory prepended to the path to each file being compiled.  This will
   appear in compilation time tracebacks, and is also compiled in to the
   byte-code file, where it will be used in tracebacks and other messages in
   cases where the source file does not exist at the time the byte-code file is
   executed.

.. cmdoption:: -x regex

   regex is used to search the full path to each file considered for
   compilation, and if the regex produces a match, the file is skipped.

.. cmdoption:: -i list

   Read the file ``list`` and add each line that it contains to the list of
   files and directories to compile.  If ``list`` is ``-``, read lines from
   ``stdin``.

.. cmdoption:: -b

   Write the byte-code files to their legacy locations and names, which may
   overwrite byte-code files created by another version of Python.  The default
   is to write files to their :pep:`3147` locations and names, which allows
   byte-code files from multiple versions of Python to coexist.

.. cmdoption:: -r

   Control the maximum recursion level for subdirectories.
   If this is given, then ``-l`` option will not be taken into account.
   :program:`python -m compileall <directory> -r 0` is equivalent to
   :program:`python -m compileall <directory> -l`.

.. cmdoption:: -j N

   Use *N* workers to compile the files within the given directory.
   If ``0`` is used, then the result of :func:`os.cpu_count()`
   will be used.

.. cmdoption:: --invalidation-mode [timestamp|checked-hash|unchecked-hash]

   Control how the generated pycs will be invalidated at runtime. The default
   setting, ``timestamp``, means that ``.pyc`` files with the source timestamp
   and size embedded will be generated. The ``checked-hash`` and
   ``unchecked-hash`` values cause hash-based pycs to be generated. Hash-based
   pycs embed a hash of the source file contents rather than a timestamp. See
   :ref:`pyc-invalidation` for more information on how Python validates bytecode
   cache files at runtime.

.. versionchanged:: 3.2
   Added the ``-i``, ``-b`` and ``-h`` options.

.. versionchanged:: 3.5
   Added the  ``-j``, ``-r``, and ``-qq`` options.  ``-q`` option
   was changed to a multilevel value.  ``-b`` will always produce a
   byte-code file ending in ``.pyc``, never ``.pyo``.

.. versionchanged:: 3.7
   Added the ``--invalidation-mode`` parameter.


There is no command-line option to control the optimization level used by the
:func:`compile` function, because the Python interpreter itself already
provides the option: :program:`python -O -m compileall`.

Similarly, the :func:`compile` function respects the :attr:`sys.pycache_prefix`
setting. The generated bytecode cache will only be useful if :func:`compile` is
run with the same :attr:`sys.pycache_prefix` (if any) that will be used at
runtime.

Public functions
----------------

.. function:: compile_dir(dir, maxlevels=10, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, workers=1, invalidation_mode=py_compile.PycInvalidationMode.TIMESTAMP)

   Recursively descend the directory tree named by *dir*, compiling all :file:`.py`
   files along the way. Return a true value if all the files compiled successfully,
   and a false value otherwise.

   The *maxlevels* parameter is used to limit the depth of the recursion; it
   defaults to ``10``.

   If *ddir* is given, it is prepended to the path to each file being compiled
   for use in compilation time tracebacks, and is also compiled in to the
   byte-code file, where it will be used in tracebacks and other messages in
   cases where the source file does not exist at the time the byte-code file is
   executed.

   If *force* is true, modules are re-compiled even if the timestamps are up to
   date.

   If *rx* is given, its search method is called on the complete path to each
   file considered for compilation, and if it returns a true value, the file
   is skipped.

   If *quiet* is ``False`` or ``0`` (the default), the filenames and other
   information are printed to standard out. Set to ``1``, only errors are
   printed. Set to ``2``, all output is suppressed.

   If *legacy* is true, byte-code files are written to their legacy locations
   and names, which may overwrite byte-code files created by another version of
   Python.  The default is to write files to their :pep:`3147` locations and
   names, which allows byte-code files from multiple versions of Python to
   coexist.

   *optimize* specifies the optimization level for the compiler.  It is passed to
   the built-in :func:`compile` function.

   The argument *workers* specifies how many workers are used to
   compile files in parallel. The default is to not use multiple workers.
   If the platform can't use multiple workers and *workers* argument is given,
   then sequential compilation will be used as a fallback.  If *workers* is
   lower than ``0``, a :exc:`ValueError` will be raised.

   *invalidation_mode* should be a member of the
   :class:`py_compile.PycInvalidationMode` enum and controls how the generated
   pycs are invalidated at runtime.

   .. versionchanged:: 3.2
      Added the *legacy* and *optimize* parameter.

   .. versionchanged:: 3.5
      Added the *workers* parameter.

   .. versionchanged:: 3.5
      *quiet* parameter was changed to a multilevel value.

   .. versionchanged:: 3.5
      The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
      no matter what the value of *optimize* is.

   .. versionchanged:: 3.6
      Accepts a :term:`path-like object`.

   .. versionchanged:: 3.7
      The *invalidation_mode* parameter was added.

.. function:: compile_file(fullname, ddir=None, force=False, rx=None, quiet=0, legacy=False, optimize=-1, invalidation_mode=py_compile.PycInvalidationMode.TIMESTAMP)

   Compile the file with path *fullname*. Return a true value if the file
   compiled successfully, and a false value otherwise.

   If *ddir* is given, it is prepended to the path to the file being compiled
   for use in compilation time tracebacks, and is also compiled in to the
   byte-code file, where it will be used in tracebacks and other messages in
   cases where the source file does not exist at the time the byte-code file is
   executed.

   If *rx* is given, its search method is passed the full path name to the
   file being compiled, and if it returns a true value, the file is not
   compiled and ``True`` is returned.

   If *quiet* is ``False`` or ``0`` (the default), the filenames and other
   information are printed to standard out. Set to ``1``, only errors are
   printed. Set to ``2``, all output is suppressed.

   If *legacy* is true, byte-code files are written to their legacy locations
   and names, which may overwrite byte-code files created by another version of
   Python.  The default is to write files to their :pep:`3147` locations and
   names, which allows byte-code files from multiple versions of Python to
   coexist.

   *optimize* specifies the optimization level for the compiler.  It is passed to
   the built-in :func:`compile` function.

   *invalidation_mode* should be a member of the
   :class:`py_compile.PycInvalidationMode` enum and controls how the generated
   pycs are invalidated at runtime.

   .. versionadded:: 3.2

   .. versionchanged:: 3.5
      *quiet* parameter was changed to a multilevel value.

   .. versionchanged:: 3.5
      The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
      no matter what the value of *optimize* is.

   .. versionchanged:: 3.7
      The *invalidation_mode* parameter was added.

.. function:: compile_path(skip_curdir=True, maxlevels=0, force=False, quiet=0, legacy=False, optimize=-1, invalidation_mode=py_compile.PycInvalidationMode.TIMESTAMP)

   Byte-compile all the :file:`.py` files found along ``sys.path``. Return a
   true value if all the files compiled successfully, and a false value otherwise.

   If *skip_curdir* is true (the default), the current directory is not included
   in the search.  All other parameters are passed to the :func:`compile_dir`
   function.  Note that unlike the other compile functions, ``maxlevels``
   defaults to ``0``.

   .. versionchanged:: 3.2
      Added the *legacy* and *optimize* parameter.

   .. versionchanged:: 3.5
      *quiet* parameter was changed to a multilevel value.

   .. versionchanged:: 3.5
      The *legacy* parameter only writes out ``.pyc`` files, not ``.pyo`` files
      no matter what the value of *optimize* is.

   .. versionchanged:: 3.7
      The *invalidation_mode* parameter was added.

To force a recompile of all the :file:`.py` files in the :file:`Lib/`
subdirectory and all its subdirectories::

   import compileall

   compileall.compile_dir('Lib/', force=True)

   # Perform same compilation, excluding files in .svn directories.
   import re
   compileall.compile_dir('Lib/', rx=re.compile(r'[/\\][.]svn'), force=True)

   # pathlib.Path objects can also be used.
   import pathlib
   compileall.compile_dir(pathlib.Path('Lib/'), force=True)

.. seealso::

   Module :mod:`py_compile`
      Byte-compile a single source file.