summaryrefslogtreecommitdiffstats
path: root/Doc/tutorial/inputoutput.rst
blob: b3bf0ef195fec328f203e734b38a6d32a9f16565 (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
.. _tut-io:

****************
Input and Output
****************

There are several ways to present the output of a program; data can be printed
in a human-readable form, or written to a file for future use. This chapter will
discuss some of the possibilities.


.. _tut-formatting:

Fancier Output Formatting
=========================

So far we've encountered two ways of writing values: *expression statements* and
the :func:`print` function.  (A third way is using the :meth:`write` method
of file objects; the standard output file can be referenced as ``sys.stdout``.
See the Library Reference for more information on this.)

Often you'll want more control over the formatting of your output than simply
printing space-separated values.  There are two ways to format your output; the
first way is to do all the string handling yourself; using string slicing and
concatenation operations you can create any layout you can imagine.  The
string type has some methods that perform useful operations for padding
strings to a given column width; these will be discussed shortly.  The second
way is to use the :meth:`str.format` method.

The :mod:`string` module contains a :class:`~string.Template` class which offers
yet another way to substitute values into strings.

One question remains, of course: how do you convert values to strings? Luckily,
Python has ways to convert any value to a string: pass it to the :func:`repr`
or :func:`str` functions.

The :func:`str` function is meant to return representations of values which are
fairly human-readable, while :func:`repr` is meant to generate representations
which can be read by the interpreter (or will force a :exc:`SyntaxError` if
there is no equivalent syntax).  For objects which don't have a particular
representation for human consumption, :func:`str` will return the same value as
:func:`repr`.  Many values, such as numbers or structures like lists and
dictionaries, have the same representation using either function.  Strings, in
particular, have two distinct representations.

Some examples::

   >>> s = 'Hello, world.'
   >>> str(s)
   'Hello, world.'
   >>> repr(s)
   "'Hello, world.'"
   >>> str(1/7)
   '0.14285714285714285'
   >>> x = 10 * 3.25
   >>> y = 200 * 200
   >>> s = 'The value of x is ' + repr(x) + ', and y is ' + repr(y) + '...'
   >>> print(s)
   The value of x is 32.5, and y is 40000...
   >>> # The repr() of a string adds string quotes and backslashes:
   ... hello = 'hello, world\n'
   >>> hellos = repr(hello)
   >>> print(hellos)
   'hello, world\n'
   >>> # The argument to repr() may be any Python object:
   ... repr((x, y, ('spam', 'eggs')))
   "(32.5, 40000, ('spam', 'eggs'))"

Here are two ways to write a table of squares and cubes::

   >>> for x in range(1, 11):
   ...     print(repr(x).rjust(2), repr(x*x).rjust(3), end=' ')
   ...     # Note use of 'end' on previous line
   ...     print(repr(x*x*x).rjust(4))
   ...
    1   1    1
    2   4    8
    3   9   27
    4  16   64
    5  25  125
    6  36  216
    7  49  343
    8  64  512
    9  81  729
   10 100 1000

   >>> for x in range(1, 11):
   ...     print('{0:2d} {1:3d} {2:4d}'.format(x, x*x, x*x*x))
   ...
    1   1    1
    2   4    8
    3   9   27
    4  16   64
    5  25  125
    6  36  216
    7  49  343
    8  64  512
    9  81  729
   10 100 1000

(Note that in the first example, one space between each column was added by the
way :func:`print` works: it always adds spaces between its arguments.)

This example demonstrates the :meth:`str.rjust` method of string
objects, which right-justifies a string in a field of a given width by padding
it with spaces on the left.  There are similar methods :meth:`str.ljust` and
:meth:`str.center`.  These methods do not write anything, they just return a
new string.  If the input string is too long, they don't truncate it, but
return it unchanged; this will mess up your column lay-out but that's usually
better than the alternative, which would be lying about a value.  (If you
really want truncation you can always add a slice operation, as in
``x.ljust(n)[:n]``.)

There is another method, :meth:`str.zfill`, which pads a numeric string on the
left with zeros.  It understands about plus and minus signs::

   >>> '12'.zfill(5)
   '00012'
   >>> '-3.14'.zfill(7)
   '-003.14'
   >>> '3.14159265359'.zfill(5)
   '3.14159265359'

Basic usage of the :meth:`str.format` method looks like this::

   >>> print('We are the {} who say "{}!"'.format('knights', 'Ni'))
   We are the knights who say "Ni!"

The brackets and characters within them (called format fields) are replaced with
the objects passed into the :meth:`str.format` method.  A number in the
brackets can be used to refer to the position of the object passed into the
:meth:`str.format` method. ::

   >>> print('{0} and {1}'.format('spam', 'eggs'))
   spam and eggs
   >>> print('{1} and {0}'.format('spam', 'eggs'))
   eggs and spam

If keyword arguments are used in the :meth:`str.format` method, their values
are referred to by using the name of the argument. ::

   >>> print('This {food} is {adjective}.'.format(
   ...       food='spam', adjective='absolutely horrible'))
   This spam is absolutely horrible.

Positional and keyword arguments can be arbitrarily combined::

   >>> print('The story of {0}, {1}, and {other}.'.format('Bill', 'Manfred',
                                                          other='Georg'))
   The story of Bill, Manfred, and Georg.

``'!a'`` (apply :func:`ascii`), ``'!s'`` (apply :func:`str`) and ``'!r'``
(apply :func:`repr`) can be used to convert the value before it is formatted::

   >>> import math
   >>> print('The value of PI is approximately {}.'.format(math.pi))
   The value of PI is approximately 3.14159265359.
   >>> print('The value of PI is approximately {!r}.'.format(math.pi))
   The value of PI is approximately 3.141592653589793.

An optional ``':'`` and format specifier can follow the field name. This allows
greater control over how the value is formatted.  The following example
rounds Pi to three places after the decimal.

   >>> import math
   >>> print('The value of PI is approximately {0:.3f}.'.format(math.pi))
   The value of PI is approximately 3.142.

Passing an integer after the ``':'`` will cause that field to be a minimum
number of characters wide.  This is useful for making tables pretty. ::

   >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 7678}
   >>> for name, phone in table.items():
   ...     print('{0:10} ==> {1:10d}'.format(name, phone))
   ...
   Jack       ==>       4098
   Dcab       ==>       7678
   Sjoerd     ==>       4127

If you have a really long format string that you don't want to split up, it
would be nice if you could reference the variables to be formatted by name
instead of by position.  This can be done by simply passing the dict and using
square brackets ``'[]'`` to access the keys ::

   >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678}
   >>> print('Jack: {0[Jack]:d}; Sjoerd: {0[Sjoerd]:d}; '
   ...       'Dcab: {0[Dcab]:d}'.format(table))
   Jack: 4098; Sjoerd: 4127; Dcab: 8637678

This could also be done by passing the table as keyword arguments with the '**'
notation. ::

   >>> table = {'Sjoerd': 4127, 'Jack': 4098, 'Dcab': 8637678}
   >>> print('Jack: {Jack:d}; Sjoerd: {Sjoerd:d}; Dcab: {Dcab:d}'.format(**table))
   Jack: 4098; Sjoerd: 4127; Dcab: 8637678

This is particularly useful in combination with the built-in function
:func:`vars`, which returns a dictionary containing all local variables.

For a complete overview of string formatting with :meth:`str.format`, see
:ref:`formatstrings`.


Old string formatting
---------------------

The ``%`` operator can also be used for string formatting. It interprets the
left argument much like a :c:func:`sprintf`\ -style format string to be applied
to the right argument, and returns the string resulting from this formatting
operation. For example::

   >>> import math
   >>> print('The value of PI is approximately %5.3f.' % math.pi)
   The value of PI is approximately 3.142.

More information can be found in the :ref:`old-string-formatting` section.


.. _tut-files:

Reading and Writing Files
=========================

.. index::
   builtin: open
   object: file

:func:`open` returns a :term:`file object`, and is most commonly used with
two arguments: ``open(filename, mode)``.

::

   >>> f = open('workfile', 'w')

.. XXX str(f) is <io.TextIOWrapper object at 0x82e8dc4>

   >>> print(f)
   <open file 'workfile', mode 'w' at 80a0960>

The first argument is a string containing the filename.  The second argument is
another string containing a few characters describing the way in which the file
will be used.  *mode* can be ``'r'`` when the file will only be read, ``'w'``
for only writing (an existing file with the same name will be erased), and
``'a'`` opens the file for appending; any data written to the file is
automatically added to the end.  ``'r+'`` opens the file for both reading and
writing. The *mode* argument is optional; ``'r'`` will be assumed if it's
omitted.

Normally, files are opened in :dfn:`text mode`, that means, you read and write
strings from and to the file, which are encoded in a specific encoding (the
default being UTF-8).  ``'b'`` appended to the mode opens the file in
:dfn:`binary mode`: now the data is read and written in the form of bytes
objects.  This mode should be used for all files that don't contain text.

In text mode, the default when reading is to convert platform-specific line
endings (``\n`` on Unix, ``\r\n`` on Windows) to just ``\n``.  When writing in
text mode, the default is to convert occurrences of ``\n`` back to
platform-specific line endings.  This behind-the-scenes modification
to file data is fine for text files, but will corrupt binary data like that in
:file:`JPEG` or :file:`EXE` files.  Be very careful to use binary mode when
reading and writing such files.


.. _tut-filemethods:

Methods of File Objects
-----------------------

The rest of the examples in this section will assume that a file object called
``f`` has already been created.

To read a file's contents, call ``f.read(size)``, which reads some quantity of
data and returns it as a string or bytes object.  *size* is an optional numeric
argument.  When *size* is omitted or negative, the entire contents of the file
will be read and returned; it's your problem if the file is twice as large as
your machine's memory. Otherwise, at most *size* bytes are read and returned.
If the end of the file has been reached, ``f.read()`` will return an empty
string (``''``).  ::

   >>> f.read()
   'This is the entire file.\n'
   >>> f.read()
   ''

``f.readline()`` reads a single line from the file; a newline character (``\n``)
is left at the end of the string, and is only omitted on the last line of the
file if the file doesn't end in a newline.  This makes the return value
unambiguous; if ``f.readline()`` returns an empty string, the end of the file
has been reached, while a blank line is represented by ``'\n'``, a string
containing only a single newline.  ::

   >>> f.readline()
   'This is the first line of the file.\n'
   >>> f.readline()
   'Second line of the file\n'
   >>> f.readline()
   ''

For reading lines from a file, you can loop over the file object. This is memory
efficient, fast, and leads to simple code::

   >>> for line in f:
   ...     print(line, end='')
   ...
   This is the first line of the file.
   Second line of the file

If you want to read all the lines of a file in a list you can also use
``list(f)`` or ``f.readlines()``.

``f.write(string)`` writes the contents of *string* to the file, returning
the number of characters written. ::

   >>> f.write('This is a test\n')
   15

To write something other than a string, it needs to be converted to a string
first::

   >>> value = ('the answer', 42)
   >>> s = str(value)
   >>> f.write(s)
   18

``f.tell()`` returns an integer giving the file object's current position in the file
represented as number of bytes from the beginning of the file when in `binary mode` and
an opaque number when in `text mode`.

To change the file object's position, use ``f.seek(offset, from_what)``.  The position is computed
from adding *offset* to a reference point; the reference point is selected by
the *from_what* argument.  A *from_what* value of 0 measures from the beginning
of the file, 1 uses the current file position, and 2 uses the end of the file as
the reference point.  *from_what* can be omitted and defaults to 0, using the
beginning of the file as the reference point. ::

   >>> f = open('workfile', 'rb+')
   >>> f.write(b'0123456789abcdef')
   16
   >>> f.seek(5)     # Go to the 6th byte in the file
   5
   >>> f.read(1)
   b'5'
   >>> f.seek(-3, 2) # Go to the 3rd byte before the end
   13
   >>> f.read(1)
   b'd'

In text files (those opened without a ``b`` in the mode string), only seeks
relative to the beginning of the file are allowed (the exception being seeking
to the very file end with ``seek(0, 2)``) and the only valid *offset* values are
those returned from the ``f.tell()``, or zero. Any other *offset* value produces
undefined behaviour.


When you're done with a file, call ``f.close()`` to close it and free up any
system resources taken up by the open file.  After calling ``f.close()``,
attempts to use the file object will automatically fail. ::

   >>> f.close()
   >>> f.read()
   Traceback (most recent call last):
     File "<stdin>", line 1, in ?
   ValueError: I/O operation on closed file

It is good practice to use the :keyword:`with` keyword when dealing with file
objects.  This has the advantage that the file is properly closed after its
suite finishes, even if an exception is raised on the way.  It is also much
shorter than writing equivalent :keyword:`try`\ -\ :keyword:`finally` blocks::

    >>> with open('workfile', 'r') as f:
    ...     read_data = f.read()
    >>> f.closed
    True

File objects have some additional methods, such as :meth:`~file.isatty` and
:meth:`~file.truncate` which are less frequently used; consult the Library
Reference for a complete guide to file objects.


.. _tut-json:

Saving structured data with :mod:`json`
---------------------------------------

.. index:: module: json

Strings can easily be written to and read from a file.  Numbers take a bit more
effort, since the :meth:`read` method only returns strings, which will have to
be passed to a function like :func:`int`, which takes a string like ``'123'``
and returns its numeric value 123.  When you want to save more complex data
types like nested lists and dictionaries, parsing and serializing by hand
becomes complicated.

Rather than having users constantly writing and debugging code to save
complicated data types to files, Python allows you to use the popular data
interchange format called `JSON (JavaScript Object Notation)
<http://json.org>`_.  The standard module called :mod:`json` can take Python
data hierarchies, and convert them to string representations; this process is
called :dfn:`serializing`.  Reconstructing the data from the string representation
is called :dfn:`deserializing`.  Between serializing and deserializing, the
string representing the object may have been stored in a file or data, or
sent over a network connection to some distant machine.

.. note::
   The JSON format is commonly used by modern applications to allow for data
   exchange.  Many programmers are already familiar with it, which makes
   it a good choice for interoperability.

If you have an object ``x``, you can view its JSON string representation with a
simple line of code::

   >>> json.dumps([1, 'simple', 'list'])
   '[1, "simple", "list"]'

Another variant of the :func:`~json.dumps` function, called :func:`~json.dump`,
simply serializes the object to a :term:`text file`.  So if ``f`` is a
:term:`text file` object opened for writing, we can do this::

   json.dump(x, f)

To decode the object again, if ``f`` is a :term:`text file` object which has
been opened for reading::

   x = json.load(f)

This simple serialization technique can handle lists and dictionaries, but
serializing arbitrary class instances in JSON requires a bit of extra effort.
The reference for the :mod:`json` module contains an explanation of this.

.. seealso::

   :mod:`pickle` - the pickle module

   Contrary to :ref:`JSON <tut-json>`, *pickle* is a protocol which allows
   the serialization of arbitrarily complex Python objects.  As such, it is
   specific to Python and cannot be used to communicate with applications
   written in other languages.  It is also insecure by default:
   deserializing pickle data coming from an untrusted source can execute
   arbitrary code, if the data was crafted by a skilled attacker.