25.4. 2to3 - Automated Python 2 to 3 code translation¶
2to3 is a Python program that reads Python 2.x source code and applies a series
of fixers to transform it into valid Python 3.x code. The standard library
contains a rich set of fixers that will handle almost all code. 2to3 supporting
library lib2to3
is, however, a flexible and generic library, so it is
possible to write your own fixers for 2to3. lib2to3
could also be
adapted to custom applications in which Python code needs to be edited
automatically.
25.4.1. Using 2to3¶
2to3 will usually be installed with the Python interpreter as a script. It is
also located in the Tools/scripts
directory of the Python root.
2to3’s basic arguments are a list of files or directories to transform. The directories are recursively traversed for Python sources.
Here is a sample Python 2.x source file, example.py
:
def greet(name):
print "Hello, {0}!".format(name)
print "What's your name?"
name = raw_input()
greet(name)
It can be converted to Python 3.x code via 2to3 on the command line:
$ 2to3 example.py
A diff against the original source file is printed. 2to3 can also write the
needed modifications right back to the source file. (A backup of the original
file is made unless -n
is also given.) Writing the changes back is
enabled with the -w
flag:
$ 2to3 -w example.py
After transformation, example.py
looks like this:
def greet(name):
print("Hello, {0}!".format(name))
print("What's your name?")
name = input()
greet(name)
Comments and exact indentation are preserved throughout the translation process.
By default, 2to3 runs a set of predefined fixers. The
-l
flag lists all available fixers. An explicit set of fixers to run
can be given with -f
. Likewise the -x
explicitly disables a
fixer. The following example runs only the imports
and has_key
fixers:
$ 2to3 -f imports -f has_key example.py
This command runs every fixer except the apply
fixer:
$ 2to3 -x apply example.py
Some fixers are explicit, meaning they aren’t run by default and must be
listed on the command line to be run. Here, in addition to the default fixers,
the idioms
fixer is run:
$ 2to3 -f all -f idioms example.py
Notice how passing all
enables all default fixers.
Sometimes 2to3 will find a place in your source code that needs to be changed, but 2to3 cannot fix automatically. In this case, 2to3 will print a warning beneath the diff for a file. You should address the warning in order to have compliant 3.x code.
2to3 can also refactor doctests. To enable this mode, use the -d
flag. Note that only doctests will be refactored. This also doesn’t require
the module to be valid Python. For example, doctest like examples in a reST
document could also be refactored with this option.
The -v
option enables output of more information on the translation
process.
Since some print statements can be parsed as function calls or statements, 2to3
cannot always read files containing the print function. When 2to3 detects the
presence of the from __future__ import print_function
compiler directive, it
modifies its internal grammar to interpret print()
as a function. This
change can also be enabled manually with the -p
flag. Use
-p
to run fixers on code that already has had its print statements
converted.
The -o
or --output-dir
option allows specification of an
alternate directory for processed output files to be written to. The
-n
flag is required when using this as backup files do not make sense
when not overwriting the input files.
New in version 2.7.3: The -o
option was added.
The -W
or --write-unchanged-files
flag tells 2to3 to always
write output files even if no changes were required to the file. This is most
useful with -o
so that an entire Python source tree is copied with
translation from one directory to another.
This option implies the -w
flag as it would not make sense otherwise.
New in version 2.7.3: The -W
flag was added.
The --add-suffix
option specifies a string to append to all output
filenames. The -n
flag is required when specifying this as backups
are not necessary when writing to different filenames. Example:
$ 2to3 -n -W --add-suffix=3 example.py
Will cause a converted file named example.py3
to be written.
New in version 2.7.3: The --add-suffix
option was added.
To translate an entire project from one directory tree to another use:
$ 2to3 --output-dir=python3-version/mycode -W -n python2-version/mycode
25.4.2. Fixers¶
Each step of transforming code is encapsulated in a fixer. The command 2to3
-l
lists them. As documented above, each can be turned on
and off individually. They are described here in more detail.
-
apply
¶ Removes usage of
apply()
. For exampleapply(function, *args, **kwargs)
is converted tofunction(*args, **kwargs)
.
-
asserts
¶ Replaces deprecated
unittest
method names with the correct ones.From To failUnlessEqual(a, b)
assertEqual(a, b)
assertEquals(a, b)
assertEqual(a, b)
failIfEqual(a, b)
assertNotEqual(a, b)
assertNotEquals(a, b)
assertNotEqual(a, b)
failUnless(a)
assertTrue(a)
assert_(a)
assertTrue(a)
failIf(a)
assertFalse(a)
failUnlessRaises(exc, cal)
assertRaises(exc, cal)
failUnlessAlmostEqual(a, b)
assertAlmostEqual(a, b)
assertAlmostEquals(a, b)
assertAlmostEqual(a, b)
failIfAlmostEqual(a, b)
assertNotAlmostEqual(a, b)
assertNotAlmostEquals(a, b)
assertNotAlmostEqual(a, b)
-
basestring
¶ Converts
basestring
tostr
.
-
buffer
¶ Converts
buffer
tomemoryview
. This fixer is optional because thememoryview
API is similar but not exactly the same as that ofbuffer
.
-
callable
¶ Converts
callable(x)
toisinstance(x, collections.Callable)
, adding an import tocollections
if needed. Notecallable(x)
has returned in Python 3.2, so if you do not intend to support Python 3.1, you can disable this fixer.
-
dict
¶ Fixes dictionary iteration methods.
dict.iteritems()
is converted todict.items()
,dict.iterkeys()
todict.keys()
, anddict.itervalues()
todict.values()
. Similarly,dict.viewitems()
,dict.viewkeys()
anddict.viewvalues()
are converted respectively todict.items()
,dict.keys()
anddict.values()
. It also wraps existing usages ofdict.items()
,dict.keys()
, anddict.values()
in a call tolist
.
-
except
¶ Converts
except X, T
toexcept X as T
.
-
execfile
¶ Removes usage of
execfile()
. The argument toexecfile()
is wrapped in calls toopen()
,compile()
, andexec()
.
-
exitfunc
¶ Changes assignment of
sys.exitfunc
to use of theatexit
module.
-
funcattrs
¶ Fixes function attributes that have been renamed. For example,
my_function.func_closure
is converted tomy_function.__closure__
.
-
future
¶ Removes
from __future__ import new_feature
statements.
-
getcwdu
¶ Renames
os.getcwdu()
toos.getcwd()
.
-
has_key
¶ Changes
dict.has_key(key)
tokey in dict
.
-
idioms
¶ This optional fixer performs several transformations that make Python code more idiomatic. Type comparisons like
type(x) is SomeClass
andtype(x) == SomeClass
are converted toisinstance(x, SomeClass)
.while 1
becomeswhile True
. This fixer also tries to make use ofsorted()
in appropriate places. For example, this blockL = list(some_iterable) L.sort()
is changed to
L = sorted(some_iterable)
-
import
¶ Detects sibling imports and converts them to relative imports.
-
imports
¶ Handles module renames in the standard library.
-
imports2
¶ Handles other modules renames in the standard library. It is separate from the
imports
fixer only because of technical limitations.
-
input
¶ Converts
input(prompt)
toeval(input(prompt))
.
-
isinstance
¶ Fixes duplicate types in the second argument of
isinstance()
. For example,isinstance(x, (int, int))
is converted toisinstance(x, (int))
.
-
itertools_imports
¶ Removes imports of
itertools.ifilter()
,itertools.izip()
, anditertools.imap()
. Imports ofitertools.ifilterfalse()
are also changed toitertools.filterfalse()
.
-
itertools
¶ Changes usage of
itertools.ifilter()
,itertools.izip()
, anditertools.imap()
to their built-in equivalents.itertools.ifilterfalse()
is changed toitertools.filterfalse()
.
-
map
¶ Wraps
map()
in alist
call. It also changesmap(None, x)
tolist(x)
. Usingfrom future_builtins import map
disables this fixer.
-
metaclass
¶ Converts the old metaclass syntax (
__metaclass__ = Meta
in the class body) to the new (class X(metaclass=Meta)
).
-
methodattrs
¶ Fixes old method attribute names. For example,
meth.im_func
is converted tometh.__func__
.
-
ne
¶ Converts the old not-equal syntax,
<>
, to!=
.
-
next
¶ Converts the use of iterator’s
next()
methods to thenext()
function. It also renamesnext()
methods to__next__()
.
-
nonzero
¶ Renames
__nonzero__()
to__bool__()
.
-
numliterals
¶ Converts octal literals into the new syntax.
-
paren
¶ Add extra parenthesis where they are required in list comprehensions. For example,
[x for x in 1, 2]
becomes[x for x in (1, 2)]
.
-
raise
¶ Converts
raise E, V
toraise E(V)
, andraise E, V, T
toraise E(V).with_traceback(T)
. IfE
is a tuple, the translation will be incorrect because substituting tuples for exceptions has been removed in Python 3.
-
raw_input
¶ Converts
raw_input()
toinput()
.
-
reduce
¶ Handles the move of
reduce()
tofunctools.reduce()
.
-
renames
¶ Changes
sys.maxint
tosys.maxsize
.
-
standarderror
¶ Renames
StandardError
toException
.
-
sys_exc
¶ Changes the deprecated
sys.exc_value
,sys.exc_type
,sys.exc_traceback
to usesys.exc_info()
.
-
throw
¶ Fixes the API change in generator’s
throw()
method.
-
tuple_params
¶ Removes implicit tuple parameter unpacking. This fixer inserts temporary variables.
-
ws_comma
¶ Removes excess whitespace from comma separated items. This fixer is optional.
-
xreadlines
¶ Changes
for x in file.xreadlines()
tofor x in file
.