summaryrefslogtreecommitdiffstats
path: root/Doc/library/nntplib.rst
blob: 479796503b148ec639328ebc6d412281a219f73d (plain)
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
:mod:`nntplib` --- NNTP protocol client
=======================================

.. module:: nntplib
   :synopsis: NNTP protocol client (requires sockets).


.. index::
   pair: NNTP; protocol
   single: Network News Transfer Protocol

**Source code:** :source:`Lib/nntplib.py`

--------------

This module defines the class :class:`NNTP` which implements the client side of
the Network News Transfer Protocol.  It can be used to implement a news reader
or poster, or automated news processors.  It is compatible with :rfc:`3977`
as well as the older :rfc:`977` and :rfc:`2980`.

Here are two small examples of how it can be used.  To list some statistics
about a newsgroup and print the subjects of the last 10 articles::

   >>> s = nntplib.NNTP('news.gmane.org')
   >>> resp, count, first, last, name = s.group('gmane.comp.python.committers')
   >>> print('Group', name, 'has', count, 'articles, range', first, 'to', last)
   Group gmane.comp.python.committers has 1096 articles, range 1 to 1096
   >>> resp, overviews = s.over((last - 9, last))
   >>> for id, over in overviews:
   ...     print(id, nntplib.decode_header(over['subject']))
   ...
   1087 Re: Commit privileges for Łukasz Langa
   1088 Re: 3.2 alpha 2 freeze
   1089 Re: 3.2 alpha 2 freeze
   1090 Re: Commit privileges for Łukasz Langa
   1091 Re: Commit privileges for Łukasz Langa
   1092 Updated ssh key
   1093 Re: Updated ssh key
   1094 Re: Updated ssh key
   1095 Hello fellow committers!
   1096 Re: Hello fellow committers!
   >>> s.quit()
   '205 Bye!'

To post an article from a binary file (this assumes that the article has valid
headers, and that you have right to post on the particular newsgroup)::

   >>> s = nntplib.NNTP('news.gmane.org')
   >>> f = open('article.txt', 'rb')
   >>> s.post(f)
   '240 Article posted successfully.'
   >>> s.quit()
   '205 Bye!'

The module itself defines the following classes:


.. class:: NNTP(host, port=119, user=None, password=None, readermode=None, usenetrc=False, [timeout])

   Return a new :class:`NNTP` object, representing a connection
   to the NNTP server running on host *host*, listening at port *port*.
   An optional *timeout* can be specified for the socket connection.
   If the optional *user* and *password* are provided, or if suitable
   credentials are present in :file:`/.netrc` and the optional flag *usenetrc*
   is true, the ``AUTHINFO USER`` and ``AUTHINFO PASS`` commands are used
   to identify and authenticate the user to the server.  If the optional
   flag *readermode* is true, then a ``mode reader`` command is sent before
   authentication is performed.  Reader mode is sometimes necessary if you are
   connecting to an NNTP server on the local machine and intend to call
   reader-specific commands, such as ``group``.  If you get unexpected
   :exc:`NNTPPermanentError`\ s, you might need to set *readermode*.
   :class:`NNTP` class supports the :keyword:`with` statement to
   unconditionally consume :exc:`OSError` exceptions and to close the NNTP
   connection when done. Here is a sample on how using it:

    >>> from nntplib import NNTP
    >>> with NNTP('news.gmane.org') as n:
    ...     n.group('gmane.comp.python.committers')
    ...
    ('211 1755 1 1755 gmane.comp.python.committers', 1755, 1, 1755, 'gmane.comp.python.committers')
    >>>


   .. versionchanged:: 3.2
      *usenetrc* is now ``False`` by default.

   .. versionchanged:: 3.3
      Support for the :keyword:`with` statement was added.

.. class:: NNTP_SSL(host, port=563, user=None, password=None, ssl_context=None, readermode=None, usenetrc=False, [timeout])

   Return a new :class:`NNTP_SSL` object, representing an encrypted
   connection to the NNTP server running on host *host*, listening at
   port *port*.  :class:`NNTP_SSL` objects have the same methods as
   :class:`NNTP` objects.  If *port* is omitted, port 563 (NNTPS) is used.
   *ssl_context* is also optional, and is a :class:`~ssl.SSLContext` object.
   Please read :ref:`ssl-security` for best practices.
   All other parameters behave the same as for :class:`NNTP`.

   Note that SSL-on-563 is discouraged per :rfc:`4642`, in favor of
   STARTTLS as described below.  However, some servers only support the
   former.

   .. versionadded:: 3.2

   .. versionchanged:: 3.4
      The class now supports hostname check with
      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
      :data:`ssl.HAS_SNI`).

.. exception:: NNTPError

   Derived from the standard exception :exc:`Exception`, this is the base
   class for all exceptions raised by the :mod:`nntplib` module.  Instances
   of this class have the following attribute:

   .. attribute:: response

      The response of the server if available, as a :class:`str` object.


.. exception:: NNTPReplyError

   Exception raised when an unexpected reply is received from the server.


.. exception:: NNTPTemporaryError

   Exception raised when a response code in the range 400--499 is received.


.. exception:: NNTPPermanentError

   Exception raised when a response code in the range 500--599 is received.


.. exception:: NNTPProtocolError

   Exception raised when a reply is received from the server that does not begin
   with a digit in the range 1--5.


.. exception:: NNTPDataError

   Exception raised when there is some error in the response data.


.. _nntp-objects:

NNTP Objects
------------

When connected, :class:`NNTP` and :class:`NNTP_SSL` objects support the
following methods and attributes.

Attributes
^^^^^^^^^^

.. attribute:: NNTP.nntp_version

   An integer representing the version of the NNTP protocol supported by the
   server.  In practice, this should be ``2`` for servers advertising
   :rfc:`3977` compliance and ``1`` for others.

   .. versionadded:: 3.2

.. attribute:: NNTP.nntp_implementation

   A string describing the software name and version of the NNTP server,
   or :const:`None` if not advertised by the server.

   .. versionadded:: 3.2

Methods
^^^^^^^

The *response* that is returned as the first item in the return tuple of almost
all methods is the server's response: a string beginning with a three-digit
code.  If the server's response indicates an error, the method raises one of
the above exceptions.

Many of the following methods take an optional keyword-only argument *file*.
When the *file* argument is supplied, it must be either a :term:`file object`
opened for binary writing, or the name of an on-disk file to be written to.
The method will then write any data returned by the server (except for the
response line and the terminating dot) to the file; any list of lines,
tuples or objects that the method normally returns will be empty.

.. versionchanged:: 3.2
   Many of the following methods have been reworked and fixed, which makes
   them incompatible with their 3.1 counterparts.


.. method:: NNTP.quit()

   Send a ``QUIT`` command and close the connection.  Once this method has been
   called, no other methods of the NNTP object should be called.


.. method:: NNTP.getwelcome()

   Return the welcome message sent by the server in reply to the initial
   connection.  (This message sometimes contains disclaimers or help information
   that may be relevant to the user.)


.. method:: NNTP.getcapabilities()

   Return the :rfc:`3977` capabilities advertised by the server, as a
   :class:`dict` instance mapping capability names to (possibly empty) lists
   of values. On legacy servers which don't understand the ``CAPABILITIES``
   command, an empty dictionary is returned instead.

      >>> s = NNTP('news.gmane.org')
      >>> 'POST' in s.getcapabilities()
      True

   .. versionadded:: 3.2


.. method:: NNTP.login(user=None, password=None, usenetrc=True)

   Send ``AUTHINFO`` commands with the user name and password.  If *user*
   and *password* are None and *usenetrc* is true, credentials from
   ``~/.netrc`` will be used if possible.

   Unless intentionally delayed, login is normally performed during the
   :class:`NNTP` object initialization and separately calling this function
   is unnecessary.  To force authentication to be delayed, you must not set
   *user* or *password* when creating the object, and must set *usenetrc* to
   False.

   .. versionadded:: 3.2


.. method:: NNTP.starttls(ssl_context=None)

   Send a ``STARTTLS`` command.  This will enable encryption on the NNTP
   connection.  The *ssl_context* argument is optional and should be a
   :class:`ssl.SSLContext` object.  Please read :ref:`ssl-security` for best
   practices.

   Note that this may not be done after authentication information has
   been transmitted, and authentication occurs by default if possible during a
   :class:`NNTP` object initialization.  See :meth:`NNTP.login` for information
   on suppressing this behavior.

   .. versionadded:: 3.2

   .. versionchanged:: 3.4
      The method now supports hostname check with
      :attr:`ssl.SSLContext.check_hostname` and *Server Name Indication* (see
      :data:`ssl.HAS_SNI`).

.. method:: NNTP.newgroups(date, *, file=None)

   Send a ``NEWGROUPS`` command.  The *date* argument should be a
   :class:`datetime.date` or :class:`datetime.datetime` object.
   Return a pair ``(response, groups)`` where *groups* is a list representing
   the groups that are new since the given *date*. If *file* is supplied,
   though, then *groups* will be empty.

      >>> from datetime import date, timedelta
      >>> resp, groups = s.newgroups(date.today() - timedelta(days=3))
      >>> len(groups)
      85
      >>> groups[0]
      GroupInfo(group='gmane.network.tor.devel', last='4', first='1', flag='m')


.. method:: NNTP.newnews(group, date, *, file=None)

   Send a ``NEWNEWS`` command.  Here, *group* is a group name or ``'*'``, and
   *date* has the same meaning as for :meth:`newgroups`.  Return a pair
   ``(response, articles)`` where *articles* is a list of message ids.

   This command is frequently disabled by NNTP server administrators.


.. method:: NNTP.list(group_pattern=None, *, file=None)

   Send a ``LIST`` or ``LIST ACTIVE`` command.  Return a pair
   ``(response, list)`` where *list* is a list of tuples representing all
   the groups available from this NNTP server, optionally matching the
   pattern string *group_pattern*.  Each tuple has the form
   ``(group, last, first, flag)``, where *group* is a group name, *last*
   and *first* are the last and first article numbers, and *flag* usually
   takes one of these values:

   * ``y``: Local postings and articles from peers are allowed.
   * ``m``: The group is moderated and all postings must be approved.
   * ``n``: No local postings are allowed, only articles from peers.
   * ``j``: Articles from peers are filed in the junk group instead.
   * ``x``: No local postings, and articles from peers are ignored.
   * ``=foo.bar``: Articles are filed in the ``foo.bar`` group instead.

   If *flag* has another value, then the status of the newsgroup should be
   considered unknown.

   This command can return very large results, especially if *group_pattern*
   is not specified.  It is best to cache the results offline unless you
   really need to refresh them.

   .. versionchanged:: 3.2
      *group_pattern* was added.


.. method:: NNTP.descriptions(grouppattern)

   Send a ``LIST NEWSGROUPS`` command, where *grouppattern* is a wildmat string as
   specified in :rfc:`3977` (it's essentially the same as DOS or UNIX shell wildcard
   strings).  Return a pair ``(response, descriptions)``, where *descriptions*
   is a dictionary mapping group names to textual descriptions.

      >>> resp, descs = s.descriptions('gmane.comp.python.*')
      >>> len(descs)
      295
      >>> descs.popitem()
      ('gmane.comp.python.bio.general', 'BioPython discussion list (Moderated)')


.. method:: NNTP.description(group)

   Get a description for a single group *group*.  If more than one group matches
   (if 'group' is a real wildmat string), return the first match.   If no group
   matches, return an empty string.

   This elides the response code from the server.  If the response code is needed,
   use :meth:`descriptions`.


.. method:: NNTP.group(name)

   Send a ``GROUP`` command, where *name* is the group name.  The group is
   selected as the current group, if it exists.  Return a tuple
   ``(response, count, first, last, name)`` where *count* is the (estimated)
   number of articles in the group, *first* is the first article number in
   the group, *last* is the last article number in the group, and *name*
   is the group name.


.. method:: NNTP.over(message_spec, *, file=None)

   Send an ``OVER`` command, or an ``XOVER`` command on legacy servers.
   *message_spec* can be either a string representing a message id, or
   a ``(first, last)`` tuple of numbers indicating a range of articles in
   the current group, or a ``(first, None)`` tuple indicating a range of
   articles starting from *first* to the last article in the current group,
   or :const:`None` to select the current article in the current group.

   Return a pair ``(response, overviews)``.  *overviews* is a list of
   ``(article_number, overview)`` tuples, one for each article selected
   by *message_spec*.  Each *overview* is a dictionary with the same number
   of items, but this number depends on the server.  These items are either
   message headers (the key is then the lower-cased header name) or metadata
   items (the key is then the metadata name prepended with ``":"``).  The
   following items are guaranteed to be present by the NNTP specification:

   * the ``subject``, ``from``, ``date``, ``message-id`` and ``references``
     headers
   * the ``:bytes`` metadata: the number of bytes in the entire raw article
     (including headers and body)
   * the ``:lines`` metadata: the number of lines in the article body

   The value of each item is either a string, or :const:`None` if not present.

   It is advisable to use the :func:`decode_header` function on header
   values when they may contain non-ASCII characters::

      >>> _, _, first, last, _ = s.group('gmane.comp.python.devel')
      >>> resp, overviews = s.over((last, last))
      >>> art_num, over = overviews[0]
      >>> art_num
      117216
      >>> list(over.keys())
      ['xref', 'from', ':lines', ':bytes', 'references', 'date', 'message-id', 'subject']
      >>> over['from']
      '=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?= <martin@v.loewis.de>'
      >>> nntplib.decode_header(over['from'])
      '"Martin v. Löwis" <martin@v.loewis.de>'

   .. versionadded:: 3.2


.. method:: NNTP.help(*, file=None)

   Send a ``HELP`` command.  Return a pair ``(response, list)`` where *list* is a
   list of help strings.


.. method:: NNTP.stat(message_spec=None)

   Send a ``STAT`` command, where *message_spec* is either a message id
   (enclosed in ``'<'`` and ``'>'``) or an article number in the current group.
   If *message_spec* is omitted or :const:`None`, the current article in the
   current group is considered.  Return a triple ``(response, number, id)``
   where *number* is the article number and *id* is the message id.

      >>> _, _, first, last, _ = s.group('gmane.comp.python.devel')
      >>> resp, number, message_id = s.stat(first)
      >>> number, message_id
      (9099, '<20030112190404.GE29873@epoch.metaslash.com>')


.. method:: NNTP.next()

   Send a ``NEXT`` command.  Return as for :meth:`.stat`.


.. method:: NNTP.last()

   Send a ``LAST`` command.  Return as for :meth:`.stat`.


.. method:: NNTP.article(message_spec=None, *, file=None)

   Send an ``ARTICLE`` command, where *message_spec* has the same meaning as
   for :meth:`.stat`.  Return a tuple ``(response, info)`` where *info*
   is a :class:`~collections.namedtuple` with three attributes *number*,
   *message_id* and *lines* (in that order).  *number* is the article number
   in the group (or 0 if the information is not available), *message_id* the
   message id as a string, and *lines* a list of lines (without terminating
   newlines) comprising the raw message including headers and body.

      >>> resp, info = s.article('<20030112190404.GE29873@epoch.metaslash.com>')
      >>> info.number
      0
      >>> info.message_id
      '<20030112190404.GE29873@epoch.metaslash.com>'
      >>> len(info.lines)
      65
      >>> info.lines[0]
      b'Path: main.gmane.org!not-for-mail'
      >>> info.lines[1]
      b'From: Neal Norwitz <neal@metaslash.com>'
      >>> info.lines[-3:]
      [b'There is a patch for 2.3 as well as 2.2.', b'', b'Neal']


.. method:: NNTP.head(message_spec=None, *, file=None)

   Same as :meth:`article()`, but sends a ``HEAD`` command.  The *lines*
   returned (or written to *file*) will only contain the message headers, not
   the body.


.. method:: NNTP.body(message_spec=None, *, file=None)

   Same as :meth:`article()`, but sends a ``BODY`` command.  The *lines*
   returned (or written to *file*) will only contain the message body, not the
   headers.


.. method:: NNTP.post(data)

   Post an article using the ``POST`` command.  The *data* argument is either
   a :term:`file object` opened for binary reading, or any iterable of bytes
   objects (representing raw lines of the article to be posted).  It should
   represent a well-formed news article, including the required headers.  The
   :meth:`post` method automatically escapes lines beginning with ``.`` and
   appends the termination line.

   If the method succeeds, the server's response is returned.  If the server
   refuses posting, a :class:`NNTPReplyError` is raised.


.. method:: NNTP.ihave(message_id, data)

   Send an ``IHAVE`` command. *message_id* is the id of the message to send
   to the server (enclosed in  ``'<'`` and ``'>'``).  The *data* parameter
   and the return value are the same as for :meth:`post()`.


.. method:: NNTP.date()

   Return a pair ``(response, date)``.  *date* is a :class:`~datetime.datetime`
   object containing the current date and time of the server.


.. method:: NNTP.slave()

   Send a ``SLAVE`` command.  Return the server's *response*.


.. method:: NNTP.set_debuglevel(level)

   Set the instance's debugging level.  This controls the amount of debugging
   output printed.  The default, ``0``, produces no debugging output.  A value of
   ``1`` produces a moderate amount of debugging output, generally a single line
   per request or response.  A value of ``2`` or higher produces the maximum amount
   of debugging output, logging each line sent and received on the connection
   (including message text).


The following are optional NNTP extensions defined in :rfc:`2980`.  Some of
them have been superseded by newer commands in :rfc:`3977`.


.. method:: NNTP.xhdr(hdr, str, *, file=None)

   Send an ``XHDR`` command.  The *hdr* argument is a header keyword, e.g.
   ``'subject'``.  The *str* argument should have the form ``'first-last'``
   where *first* and *last* are the first and last article numbers to search.
   Return a pair ``(response, list)``, where *list* is a list of pairs ``(id,
   text)``, where *id* is an article number (as a string) and *text* is the text of
   the requested header for that article. If the *file* parameter is supplied, then
   the output of the  ``XHDR`` command is stored in a file.  If *file* is a string,
   then the method will open a file with that name, write to it  then close it.
   If *file* is a :term:`file object`, then it will start calling :meth:`write` on
   it to store the lines of the command output. If *file* is supplied, then the
   returned *list* is an empty list.


.. method:: NNTP.xover(start, end, *, file=None)

   Send an ``XOVER`` command.  *start* and *end* are article numbers
   delimiting the range of articles to select.  The return value is the
   same of for :meth:`over()`.  It is recommended to use :meth:`over()`
   instead, since it will automatically use the newer ``OVER`` command
   if available.


.. method:: NNTP.xpath(id)

   Return a pair ``(resp, path)``, where *path* is the directory path to the
   article with message ID *id*.  Most of the time, this extension is not
   enabled by NNTP server administrators.

   .. deprecated:: 3.3
      The XPATH extension is not actively used.


.. XXX deprecated:

   .. method:: NNTP.xgtitle(name, *, file=None)

      Process an ``XGTITLE`` command, returning a pair ``(response, list)``, where
      *list* is a list of tuples containing ``(name, title)``. If the *file* parameter
      is supplied, then the output of the  ``XGTITLE`` command is stored in a file.
      If *file* is a string,  then the method will open a file with that name, write
      to it  then close it.  If *file* is a :term:`file object`, then it will start
      calling :meth:`write` on it to store the lines of the command output. If *file*
      is supplied, then the returned *list* is an empty list. This is an optional NNTP
      extension, and may not be supported by all servers.

      RFC2980 says "It is suggested that this extension be deprecated".  Use
      :meth:`descriptions` or :meth:`description` instead.


Utility functions
-----------------

The module also defines the following utility function:


.. function:: decode_header(header_str)

   Decode a header value, un-escaping any escaped non-ASCII characters.
   *header_str* must be a :class:`str` object.  The unescaped value is
   returned.  Using this function is recommended to display some headers
   in a human readable form::

      >>> decode_header("Some subject")
      'Some subject'
      >>> decode_header("=?ISO-8859-15?Q?D=E9buter_en_Python?=")
      'Débuter en Python'
      >>> decode_header("Re: =?UTF-8?B?cHJvYmzDqG1lIGRlIG1hdHJpY2U=?=")
      'Re: problème de matrice'