summaryrefslogtreecommitdiffstats
path: root/Doc/library/2to3.rst
blob: a13fb650246605ce4f0854ae4b3a4a3b95cbef82 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
.. _2to3-reference:

2to3 - Automated Python 2 to 3 code translation
===============================================

.. sectionauthor:: Benjamin Peterson <benjamin@python.org>

2to3 is a Python program that reads Python 2.x source code and applies a series
of *fixers* to transform it into valid Python 3.x code.  The standard library
contains a rich set of fixers that will handle almost all code.  2to3 supporting
library :mod:`lib2to3` is, however, a flexible and generic library, so it is
possible to write your own fixers for 2to3.  :mod:`lib2to3` could also be
adapted to custom applications in which Python code needs to be edited
automatically.


.. _2to3-using:

Using 2to3
----------

2to3 will usually be installed with the Python interpreter as a script.  It is
also located in the :file:`Tools/scripts` directory of the Python root.

2to3's basic arguments are a list of files or directories to transform.  The
directories are to recursively traversed for Python sources.

Here is a sample Python 2.x source file, :file:`example.py`::

   def greet(name):
       print "Hello, {0}!".format(name)
   print "What's your name?"
   name = raw_input()
   greet(name)

It can be converted to Python 3.x code via 2to3 on the command line::

   $ 2to3 example.py

A diff against the original source file is printed.  2to3 can also write the
needed modifications right back to the source file.  (A backup of the original
file is made unless :option:`-n` is also given.)  Writing the changes back is
enabled with the :option:`-w` flag::

   $ 2to3 -w example.py

After transformation, :file:`example.py` looks like this::

   def greet(name):
       print("Hello, {0}!".format(name))
   print("What's your name?")
   name = input()
   greet(name)

Comments and exact indentation are preserved throughout the translation process.

By default, 2to3 runs a set of :ref:`predefined fixers <2to3-fixers>`.  The
:option:`-l` flag lists all available fixers.  An explicit set of fixers to run
can be given with :option:`-f`.  Likewise the :option:`-x` explicitly disables a
fixer.  The following example runs only the ``imports`` and ``has_key`` fixers::

   $ 2to3 -f imports -f has_key example.py

This command runs every fixer except the ``apply`` fixer::

   $ 2to3 -x apply example.py

Some fixers are *explicit*, meaning they aren't run by default and must be
listed on the command line to be run.  Here, in addition to the default fixers,
the ``idioms`` fixer is run::

   $ 2to3 -f all -f idioms example.py

Notice how passing ``all`` enables all default fixers.

Sometimes 2to3 will find a place in your source code that needs to be changed,
but 2to3 cannot fix automatically.  In this case, 2to3 will print a warning
beneath the diff for a file.  You should address the warning in order to have
compliant 3.x code.

2to3 can also refactor doctests.  To enable this mode, use the :option:`-d`
flag.  Note that *only* doctests will be refactored.  This also doesn't require
the module to be valid Python.  For example, doctest like examples in a reST
document could also be refactored with this option.

The :option:`-v` option enables output of more information on the translation
process.

Since some print statements can be parsed as function calls or statements, 2to3
cannot always read files containing the print function.  When 2to3 detects the
presence of the ``from __future__ import print_function`` compiler directive, it
modifies its internal grammar to interpret :func:`print` as a function.  This
change can also be enabled manually with the :option:`-p` flag.  Use
:option:`-p` to run fixers on code that already has had its print statements
converted.

The :option:`-o` or :option:`--output-dir` option allows specification of an
alternate directory for processed output files to be written to.  The
:option:`-n` flag is required when using this as backup files do not make sense
when not overwriting the input files.

.. versionadded:: 3.2.3
   The :option:`-o` option was added.

The :option:`-W` or :option:`--write-unchanged-files` flag tells 2to3 to always
write output files even if no changes were required to the file.  This is most
useful with :option:`-o` so that an entire Python source tree is copied with
translation from one directory to another.
This option implies the :option:`-w` flag as it would not make sense otherwise.

.. versionadded:: 3.2.3
   The :option:`-W` flag was added.

The :option:`--add-suffix` option specifies a string to append to all output
filenames.  The :option:`-n` flag is required when specifying this as backups
are not necessary when writing to different filenames.  Example::

   $ 2to3 -n -W --add-suffix=3 example.py

Will cause a converted file named ``example.py3`` to be written.

.. versionadded:: 3.2.3
   The :option:`--add-suffix` option was added.

To translate an entire project from one directory tree to another use::

   $ 2to3 --output-dir=python3-version/mycode -W -n python2-version/mycode


.. _2to3-fixers:

Fixers
------

Each step of transforming code is encapsulated in a fixer.  The command ``2to3
-l`` lists them.  As :ref:`documented above <2to3-using>`, each can be turned on
and off individually.  They are described here in more detail.


.. 2to3fixer:: apply

   Removes usage of :func:`apply`.  For example ``apply(function, *args,
   **kwargs)`` is converted to ``function(*args, **kwargs)``.

.. 2to3fixer:: basestring

   Converts :class:`basestring` to :class:`str`.

.. 2to3fixer:: buffer

   Converts :class:`buffer` to :class:`memoryview`.  This fixer is optional
   because the :class:`memoryview` API is similar but not exactly the same as
   that of :class:`buffer`.

.. 2to3fixer:: callable

   Converts ``callable(x)`` to ``isinstance(x, collections.Callable)``, adding
   an import to :mod:`collections` if needed. Note ``callable(x)`` has returned
   in Python 3.2, so if you do not intend to support Python 3.1, you can disable
   this fixer.

.. 2to3fixer:: dict

   Fixes dictionary iteration methods.  :meth:`dict.iteritems` is converted to
   :meth:`dict.items`, :meth:`dict.iterkeys` to :meth:`dict.keys`, and
   :meth:`dict.itervalues` to :meth:`dict.values`.  Similarly,
   :meth:`dict.viewitems`, :meth:`dict.viewkeys` and :meth:`dict.viewvalues` are
   converted respectively to :meth:`dict.items`, :meth:`dict.keys` and
   :meth:`dict.values`.  It also wraps existing usages of :meth:`dict.items`,
   :meth:`dict.keys`, and :meth:`dict.values` in a call to :class:`list`.

.. 2to3fixer:: except

   Converts ``except X, T`` to ``except X as T``.

.. 2to3fixer:: exec

   Converts the ``exec`` statement to the :func:`exec` function.

.. 2to3fixer:: execfile

   Removes usage of :func:`execfile`.  The argument to :func:`execfile` is
   wrapped in calls to :func:`open`, :func:`compile`, and :func:`exec`.

.. 2to3fixer:: exitfunc

   Changes assignment of :attr:`sys.exitfunc` to use of the :mod:`atexit`
   module.

.. 2to3fixer:: filter

   Wraps :func:`filter` usage in a :class:`list` call.

.. 2to3fixer:: funcattrs

   Fixes function attributes that have been renamed.  For example,
   ``my_function.func_closure`` is converted to ``my_function.__closure__``.

.. 2to3fixer:: future

   Removes ``from __future__ import new_feature`` statements.

.. 2to3fixer:: getcwdu

   Renames :func:`os.getcwdu` to :func:`os.getcwd`.

.. 2to3fixer:: has_key

   Changes ``dict.has_key(key)`` to ``key in dict``.

.. 2to3fixer:: idioms

   This optional fixer performs several transformations that make Python code
   more idiomatic.  Type comparisons like ``type(x) is SomeClass`` and
   ``type(x) == SomeClass`` are converted to ``isinstance(x, SomeClass)``.
   ``while 1`` becomes ``while True``.  This fixer also tries to make use of
   :func:`sorted` in appropriate places.  For example, this block ::

       L = list(some_iterable)
       L.sort()

   is changed to ::

      L = sorted(some_iterable)

.. 2to3fixer:: import

   Detects sibling imports and converts them to relative imports.

.. 2to3fixer:: imports

   Handles module renames in the standard library.

.. 2to3fixer:: imports2

   Handles other modules renames in the standard library.  It is separate from
   the :2to3fixer:`imports` fixer only because of technical limitations.

.. 2to3fixer:: input

   Converts ``input(prompt)`` to ``eval(input(prompt))``

.. 2to3fixer:: intern

   Converts :func:`intern` to :func:`sys.intern`.

.. 2to3fixer:: isinstance

   Fixes duplicate types in the second argument of :func:`isinstance`.  For
   example, ``isinstance(x, (int, int))`` is converted to ``isinstance(x,
   (int))``.

.. 2to3fixer:: itertools_imports

   Removes imports of :func:`itertools.ifilter`, :func:`itertools.izip`, and
   :func:`itertools.imap`.  Imports of :func:`itertools.ifilterfalse` are also
   changed to :func:`itertools.filterfalse`.

.. 2to3fixer:: itertools

   Changes usage of :func:`itertools.ifilter`, :func:`itertools.izip`, and
   :func:`itertools.imap` to their built-in equivalents.
   :func:`itertools.ifilterfalse` is changed to :func:`itertools.filterfalse`.

.. 2to3fixer:: long

   Strips the ``L`` prefix on long literals and renames :class:`long` to
   :class:`int`.

.. 2to3fixer:: map

   Wraps :func:`map` in a :class:`list` call.  It also changes ``map(None, x)``
   to ``list(x)``.  Using ``from future_builtins import map`` disables this
   fixer.

.. 2to3fixer:: metaclass

   Converts the old metaclass syntax (``__metaclass__ = Meta`` in the class
   body) to the new (``class X(metaclass=Meta)``).

.. 2to3fixer:: methodattrs

   Fixes old method attribute names.  For example, ``meth.im_func`` is converted
   to ``meth.__func__``.

.. 2to3fixer:: ne

   Converts the old not-equal syntax, ``<>``, to ``!=``.

.. 2to3fixer:: next

   Converts the use of iterator's :meth:`~iterator.next` methods to the
   :func:`next` function.  It also renames :meth:`next` methods to
   :meth:`~object.__next__`.

.. 2to3fixer:: nonzero

   Renames :meth:`~object.__nonzero__` to :meth:`~object.__bool__`.

.. 2to3fixer:: numliterals

   Converts octal literals into the new syntax.

.. 2to3fixer:: operator

   Converts calls to various functions in the :mod:`operator` module to other,
   but equivalent, function calls.  When needed, the appropriate ``import``
   statements are added, e.g. ``import collections``.  The following mapping
   are made:

   ==================================  ==========================================
   From                                To
   ==================================  ==========================================
   ``operator.isCallable(obj)``        ``hasattr(obj, '__call__')``
   ``operator.sequenceIncludes(obj)``  ``operator.contains(obj)``
   ``operator.isSequenceType(obj)``    ``isinstance(obj, collections.Sequence)``
   ``operator.isMappingType(obj)``     ``isinstance(obj, collections.Mapping)``
   ``operator.isNumberType(obj)``      ``isinstance(obj, numbers.Number)``
   ``operator.repeat(obj, n)``         ``operator.mul(obj, n)``
   ``operator.irepeat(obj, n)``        ``operator.imul(obj, n)``
   ==================================  ==========================================

.. 2to3fixer:: paren

   Add extra parenthesis where they are required in list comprehensions.  For
   example, ``[x for x in 1, 2]`` becomes ``[x for x in (1, 2)]``.

.. 2to3fixer:: print

   Converts the ``print`` statement to the :func:`print` function.

.. 2to3fixer:: raise

   Converts ``raise E, V`` to ``raise E(V)``, and ``raise E, V, T`` to ``raise
   E(V).with_traceback(T)``.  If ``E`` is a tuple, the translation will be
   incorrect because substituting tuples for exceptions has been removed in 3.0.

.. 2to3fixer:: raw_input

   Converts :func:`raw_input` to :func:`input`.

.. 2to3fixer:: reduce

   Handles the move of :func:`reduce` to :func:`functools.reduce`.

.. 2to3fixer:: renames

   Changes :data:`sys.maxint` to :data:`sys.maxsize`.

.. 2to3fixer:: repr

   Replaces backtick repr with the :func:`repr` function.

.. 2to3fixer:: set_literal

   Replaces use of the :class:`set` constructor with set literals.  This fixer
   is optional.

.. 2to3fixer:: standard_error

   Renames :exc:`StandardError` to :exc:`Exception`.

.. 2to3fixer:: sys_exc

   Changes the deprecated :data:`sys.exc_value`, :data:`sys.exc_type`,
   :data:`sys.exc_traceback` to use :func:`sys.exc_info`.

.. 2to3fixer:: throw

   Fixes the API change in generator's :meth:`throw` method.

.. 2to3fixer:: tuple_params

   Removes implicit tuple parameter unpacking.  This fixer inserts temporary
   variables.

.. 2to3fixer:: types

   Fixes code broken from the removal of some members in the :mod:`types`
   module.

.. 2to3fixer:: unicode

   Renames :class:`unicode` to :class:`str`.

.. 2to3fixer:: urllib

   Handles the rename of :mod:`urllib` and :mod:`urllib2` to the :mod:`urllib`
   package.

.. 2to3fixer:: ws_comma

   Removes excess whitespace from comma separated items.  This fixer is
   optional.

.. 2to3fixer:: xrange

   Renames :func:`xrange` to :func:`range` and wraps existing :func:`range`
   calls with :class:`list`.

.. 2to3fixer:: xreadlines

   Changes ``for x in file.xreadlines()`` to ``for x in file``.

.. 2to3fixer:: zip

   Wraps :func:`zip` usage in a :class:`list` call.  This is disabled when
   ``from future_builtins import zip`` appears.


:mod:`lib2to3` - 2to3's library
-------------------------------

.. module:: lib2to3
   :synopsis: the 2to3 library
.. moduleauthor:: Guido van Rossum
.. moduleauthor:: Collin Winter
.. moduleauthor:: Benjamin Peterson <benjamin@python.org>


.. note::

   The :mod:`lib2to3` API should be considered unstable and may change
   drastically in the future.

.. XXX What is the public interface anyway?