diff options
Diffstat (limited to 'Doc')
| -rw-r--r-- | Doc/c-api/concrete.rst | 10 | ||||
| -rw-r--r-- | Doc/library/collections.rst | 128 | ||||
| -rw-r--r-- | Doc/library/decimal.rst | 583 | ||||
| -rw-r--r-- | Doc/library/fileinput.rst | 5 | ||||
| -rw-r--r-- | Doc/library/functions.rst | 3 | ||||
| -rw-r--r-- | Doc/library/hashlib.rst | 2 | ||||
| -rw-r--r-- | Doc/library/operator.rst | 76 | ||||
| -rw-r--r-- | Doc/library/os.rst | 7 | ||||
| -rw-r--r-- | Doc/library/simplexmlrpcserver.rst | 2 | ||||
| -rw-r--r-- | Doc/library/ssl.rst | 457 | ||||
| -rw-r--r-- | Doc/library/undoc.rst | 3 | ||||
| -rw-r--r-- | Doc/reference/index.rst | 2 | ||||
| -rw-r--r-- | Doc/reference/simple_stmts.rst | 83 | ||||
| -rw-r--r-- | Doc/whatsnew/2.5.rst | 2 | ||||
| -rw-r--r-- | Doc/whatsnew/2.6.rst | 470 |
15 files changed, 1422 insertions, 411 deletions
diff --git a/Doc/c-api/concrete.rst b/Doc/c-api/concrete.rst index ba41fc5..6cd4be4 100644 --- a/Doc/c-api/concrete.rst +++ b/Doc/c-api/concrete.rst @@ -1953,12 +1953,12 @@ Tuple Objects ``PyTuple_Pack(2, a, b)`` is equivalent to ``Py_BuildValue("(OO)", a, b)``. -.. cfunction:: int PyTuple_Size(PyObject *p) +.. cfunction:: Py_ssize_t PyTuple_Size(PyObject *p) Take a pointer to a tuple object, and return the size of that tuple. -.. cfunction:: int PyTuple_GET_SIZE(PyObject *p) +.. cfunction:: Py_ssize_t PyTuple_GET_SIZE(PyObject *p) Return the size of the tuple *p*, which must be non-*NULL* and point to a tuple; no error checking is performed. @@ -2458,7 +2458,7 @@ change in future releases of Python. immediately after file object creation. -.. cfunction:: int PyFile_Encoding(PyFileObject *p, char *enc) +.. cfunction:: int PyFile_SetEncoding(PyFileObject *p, const char *enc) Set the file's encoding for Unicode output to *enc*. Return 1 on success and 0 on failure. @@ -3371,7 +3371,7 @@ The following functions and macros are available for instances of :class:`set` or :class:`frozenset` or instances of their subtypes. -.. cfunction:: int PySet_Size(PyObject *anyset) +.. cfunction:: Py_ssize_t PySet_Size(PyObject *anyset) .. index:: builtin: len @@ -3380,7 +3380,7 @@ or :class:`frozenset` or instances of their subtypes. :class:`set`, :class:`frozenset`, or an instance of a subtype. -.. cfunction:: int PySet_GET_SIZE(PyObject *anyset) +.. cfunction:: Py_ssize_t PySet_GET_SIZE(PyObject *anyset) Macro form of :cfunc:`PySet_Size` without error checking. diff --git a/Doc/library/collections.rst b/Doc/library/collections.rst index fe57f12..227f721 100644 --- a/Doc/library/collections.rst +++ b/Doc/library/collections.rst @@ -395,11 +395,14 @@ Setting the :attr:`default_factory` to :class:`set` makes the .. _named-tuple-factory: -:func:`NamedTuple` datatype factory function --------------------------------------------- +:func:`NamedTuple` factory function +----------------------------------- +Named tuples assign meaning to each position in a tuple and allow for more readable, +self-documenting code. They can be used wherever regular tuples are used, and +they add the ability to access fields by name instead of position index. -.. function:: NamedTuple(typename, fieldnames) +.. function:: NamedTuple(typename, fieldnames, [verbose]) Returns a new tuple subclass named *typename*. The new subclass is used to create tuple-like objects that have fields accessable by attribute lookup as @@ -407,43 +410,88 @@ Setting the :attr:`default_factory` to :class:`set` makes the helpful docstring (with typename and fieldnames) and a helpful :meth:`__repr__` method which lists the tuple contents in a ``name=value`` format. - The *fieldnames* are specified in a single string and are separated by spaces. - Any valid Python identifier may be used for a field name. - - Example:: - - >>> Point = NamedTuple('Point', 'x y') - >>> Point.__doc__ # docstring for the new datatype - 'Point(x, y)' - >>> p = Point(11, y=22) # instantiate with positional or keyword arguments - >>> p[0] + p[1] # works just like the tuple (11, 22) - 33 - >>> x, y = p # unpacks just like a tuple - >>> x, y - (11, 22) - >>> p.x + p.y # fields also accessable by name - 33 - >>> p # readable __repr__ with name=value style - Point(x=11, y=22) - - The use cases are the same as those for tuples. The named factories assign - meaning to each tuple position and allow for more readable, self-documenting - code. Named tuples can also be used to assign field names to tuples returned - by the :mod:`csv` or :mod:`sqlite3` modules. For example:: - - from itertools import starmap - import csv - EmployeeRecord = NamedTuple('EmployeeRecord', 'name age title department paygrade') - for record in starmap(EmployeeRecord, csv.reader(open("employees.csv", "rb"))): - print(record) - - To cast an individual record stored as :class:`list`, :class:`tuple`, or some - other iterable type, use the star-operator [#]_ to unpack the values:: - - >>> Color = NamedTuple('Color', 'name code') - >>> m = dict(red=1, green=2, blue=3) - >>> print(Color(*m.popitem())) - Color(name='blue', code=3) + The *fieldnames* are specified in a single string with each fieldname separated by + a space and/or comma. Any valid Python identifier may be used for a field name. + + If *verbose* is true, the *NamedTuple* call will print the class definition. + + *NamedTuple* instances do not have per-instance dictionaries, so they are + lightweight, requiring no more memory than regular tuples. + +Example:: + + >>> Point = NamedTuple('Point', 'x y', True) + class Point(tuple): + 'Point(x, y)' + __slots__ = () + __fields__ = ('x', 'y') + def __new__(cls, x, y): + return tuple.__new__(cls, (x, y)) + def __repr__(self): + return 'Point(x=%r, y=%r)' % self + def __replace__(self, field, value): + 'Return a new Point object replacing one field with a new value' + return Point(**dict(zip(('x', 'y'), self) + [(field, value)])) + x = property(itemgetter(0)) + y = property(itemgetter(1)) + + >>> p = Point(11, y=22) # instantiate with positional or keyword arguments + >>> p[0] + p[1] # indexable like the regular tuple (11, 22) + 33 + >>> x, y = p # unpack like a regular tuple + >>> x, y + (11, 22) + >>> p.x + p.y # fields also accessable by name + 33 + >>> p # readable __repr__ with a name=value style + Point(x=11, y=22) + +Named tuples are especially useful for assigning field names to result tuples returned +by the :mod:`csv` or :mod:`sqlite3` modules:: + + from itertools import starmap + import csv + EmployeeRecord = NamedTuple('EmployeeRecord', 'name age title department paygrade') + for record in starmap(EmployeeRecord, csv.reader(open("employees.csv", "rb"))): + print(emp.name, emp.title) + +When casting a single record to a *NamedTuple*, use the star-operator [#]_ to unpack +the values:: + + >>> t = [11, 22] + >>> Point(*t) # the star-operator unpacks any iterable object + Point(x=11, y=22) + +In addition to the methods inherited from tuples, named tuples support +an additonal method and an informational read-only attribute. + +.. method:: somenamedtuple.replace(field, value) + + Return a new instance of the named tuple replacing the named *field* with a new *value*:: + + >>> p = Point(x=11, y=22) + >>> p.__replace__('x', 33) + Point(x=33, y=22) + + >>> for recordnum, record in inventory: + ... inventory[recordnum] = record.replace('total', record.price * record.quantity) + +.. attribute:: somenamedtuple.__fields__ + + Return a tuple of strings listing the field names. This is useful for introspection, + for converting a named tuple instance to a dictionary, and for combining named tuple + types to create new named tuple types:: + + >>> p.__fields__ # view the field names + ('x', 'y') + >>> dict(zip(p.__fields__, p)) # convert to a dictionary + {'y': 22, 'x': 11} + + >>> Color = NamedTuple('Color', 'red green blue') + >>> pixel_fields = ' '.join(Point.__fields__ + Color.__fields__) # combine fields + >>> Pixel = NamedTuple('Pixel', pixel_fields) + >>> Pixel(11, 22, 128, 255, 0) + Pixel(x=11, y=22, red=128, green=255, blue=0)' .. rubric:: Footnotes diff --git a/Doc/library/decimal.rst b/Doc/library/decimal.rst index 444b20a..bbac0d4 100644 --- a/Doc/library/decimal.rst +++ b/Doc/library/decimal.rst @@ -14,7 +14,7 @@ The :mod:`decimal` module provides support for decimal floating point -arithmetic. It offers several advantages over the :class:`float()` datatype: +arithmetic. It offers several advantages over the :class:`float` datatype: * Decimal numbers can be represented exactly. In contrast, numbers like :const:`1.1` do not have an exact representation in binary floating point. End @@ -22,7 +22,7 @@ arithmetic. It offers several advantages over the :class:`float()` datatype: :const:`1.1000000000000001` as it does with binary floating point. * The exactness carries over into arithmetic. In decimal floating point, ``0.1 - + 0.1 + 0.1 - 0.3`` is exactly equal to zero. In binary floating point, result + + 0.1 + 0.1 - 0.3`` is exactly equal to zero. In binary floating point, the result is :const:`5.5511151231257827e-017`. While near to zero, the differences prevent reliable equality testing and differences can accumulate. For this reason, decimal would be preferred in accounting applications which have strict @@ -36,7 +36,7 @@ arithmetic. It offers several advantages over the :class:`float()` datatype: 1.20`` gives :const:`1.5600`. * Unlike hardware based binary floating point, the decimal module has a user - settable precision (defaulting to 28 places) which can be as large as needed for + alterable precision (defaulting to 28 places) which can be as large as needed for a given problem:: >>> getcontext().prec = 6 @@ -56,7 +56,7 @@ context for arithmetic, and signals. A decimal number is immutable. It has a sign, coefficient digits, and an exponent. To preserve significance, the coefficient digits do not truncate -trailing zeroes. Decimals also include special values such as +trailing zeros. Decimals also include special values such as :const:`Infinity`, :const:`-Infinity`, and :const:`NaN`. The standard also differentiates :const:`-0` from :const:`+0`. @@ -65,7 +65,7 @@ rules, limits on exponents, flags indicating the results of operations, and trap enablers which determine whether signals are treated as exceptions. Rounding options include :const:`ROUND_CEILING`, :const:`ROUND_DOWN`, :const:`ROUND_FLOOR`, :const:`ROUND_HALF_DOWN`, :const:`ROUND_HALF_EVEN`, -:const:`ROUND_HALF_UP`, and :const:`ROUND_UP`. +:const:`ROUND_HALF_UP`, :const:`ROUND_UP`, and :const:`ROUND_05UP`. Signals are groups of exceptional conditions arising during the course of computation. Depending on the needs of the application, signals may be ignored, @@ -82,11 +82,11 @@ reset them before monitoring a calculation. .. seealso:: - IBM's General Decimal Arithmetic Specification, `The General Decimal Arithmetic - Specification <http://www2.hursley.ibm.com/decimal/decarith.html>`_. + * IBM's General Decimal Arithmetic Specification, `The General Decimal Arithmetic + Specification <http://www2.hursley.ibm.com/decimal/decarith.html>`_. - IEEE standard 854-1987, `Unofficial IEEE 854 Text - <http://www.cs.berkeley.edu/~ejr/projects/754/private/drafts/854-1987/dir.html>`_. + * IEEE standard 854-1987, `Unofficial IEEE 854 Text + <http://www.cs.berkeley.edu/~ejr/projects/754/private/drafts/854-1987/dir.html>`_. .. % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -123,6 +123,8 @@ representation error). Decimal numbers include special values such as Decimal("3.14") >>> Decimal(str(2.0 ** 0.5)) Decimal("1.41421356237") + >>> Decimal(2) ** Decimal("0.5") + Decimal("1.414213562373095048801688724") >>> Decimal("NaN") Decimal("NaN") >>> Decimal("-Infinity") @@ -172,6 +174,17 @@ floating point flying circus:: >>> c % a Decimal("0.77") +And some mathematic functions are also available to Decimal:: + + >>> Decimal(2).sqrt() + Decimal("1.414213562373095048801688724") + >>> Decimal(1).exp() + Decimal("2.718281828459045235360287471") + >>> Decimal("10").ln() + Decimal("2.302585092994045684017991455") + >>> Decimal("10").log10() + Decimal("1") + The :meth:`quantize` method rounds a number to a fixed exponent. This method is useful for monetary applications that often round results to a fixed number of places:: @@ -285,7 +298,7 @@ Decimal objects The *context* precision does not affect how many digits are stored. That is determined exclusively by the number of digits in *value*. For example, - ``Decimal("3.00000")`` records all five zeroes even if the context precision is + ``Decimal("3.00000")`` records all five zeros even if the context precision is only three. The purpose of the *context* argument is determining what to do if *value* is a @@ -295,7 +308,7 @@ Decimal objects Once constructed, :class:`Decimal` objects are immutable. -Decimal floating point objects share many properties with the other builtin +Decimal floating point objects share many properties with the other built-in numeric types such as :class:`float` and :class:`int`. All of the usual math operations and special methods apply. Likewise, decimal objects can be copied, pickled, printed, used as dictionary keys, used as set elements, compared, @@ -315,50 +328,363 @@ also have a number of specialized methods: .. method:: Decimal.as_tuple() - Returns a tuple representation of the number: ``(sign, digittuple, exponent)``. + Return a tuple representation of the number: ``(sign, digit_tuple, exponent)``. + +.. method:: Decimal.canonical() + + Return the canonical encoding of the argument. Currently, the + encoding of a :class:`Decimal` instance is always canonical, so + this operation returns its argument unchanged. + + .. versionadded:: 2.6 .. method:: Decimal.compare(other[, context]) - Compares like :meth:`__cmp__` but returns a decimal instance:: + Compare the values of two Decimal instances. This operation + behaves in the same way as the usual comparison method + :meth:`__cmp__`, except that :meth:`compare` returns a Decimal + instance rather than an integer, and if either operand is a NaN + then the result is a NaN:: a or b is a NaN ==> Decimal("NaN") a < b ==> Decimal("-1") a == b ==> Decimal("0") a > b ==> Decimal("1") +.. method:: Decimal.compare_signal(other[, context]) + + This operation is identical to the :meth:`compare` method, except + that all NaNs signal. That is, if neither operand is a signaling + NaN then any quiet NaN operand is treated as though it were a + signaling NaN. + + .. versionadded:: 2.6 + +.. method:: Decimal.compare_total(other) + + Compare two operands using their abstract representation rather + than their numerical value. Similar to the :meth:`compare` method, + but the result gives a total ordering on :class:`Decimal` + instances. Two :class:`Decimal` instances with the same numeric + value but different representations compare unequal in this + ordering:: + + >>> Decimal("12.0").compare_total(Decimal("12")) + Decimal("-1") + + Quiet and signaling NaNs are also included in the total ordering. + The result of this function is ``Decimal("0")`` if both operands + have the same representation, ``Decimal("-1")`` if the first + operand is lower in the total order than the second, and + ``Decimal("1")`` if the first operand is higher in the total order + than the second operand. See the specification for details of the + total order. + + .. versionadded:: 2.6 + +.. method:: Decimal.compare_total_mag(other) + + Compare two operands using their abstract representation rather + than their value as in :meth:`compare_total`, but ignoring the sign + of each operand. ``x.compare_total_mag(y)`` is equivalent to + ``x.copy_abs().compare_total(y.copy_abs())``. + + .. versionadded:: 2.6 + +.. method:: Decimal.copy_abs() + + Return the absolute value of the argument. This operation is + unaffected by the context and is quiet: no flags are changed and no + rounding is performed. + + .. versionadded:: 2.6 + +.. method:: Decimal.copy_negate() + + Return the negation of the argument. This operation is unaffected + by the context and is quiet: no flags are changed and no rounding + is performed. + + .. versionadded:: 2.6 + +.. method:: Decimal.copy_sign(other) + + Return a copy of the first operand with the sign set to be the + same as the sign of the second operand. For example:: + + >>> Decimal("2.3").copy_sign(Decimal("-1.5")) + Decimal("-2.3") + + This operation is unaffected by the context and is quiet: no flags + are changed and no rounding is performed. + + .. versionadded:: 2.6 + +.. method:: Decimal.exp([context]) + + Return the value of the (natural) exponential function ``e**x`` at the + given number. The result is correctly rounded using the + :const:`ROUND_HALF_EVEN` rounding mode. + + >>> Decimal(1).exp() + Decimal("2.718281828459045235360287471") + >>> Decimal(321).exp() + Decimal("2.561702493119680037517373933E+139") + + .. versionadded:: 2.6 + +.. method:: Decimal.fma(other, third[, context]) + + Fused multiply-add. Return self*other+third with no rounding of + the intermediate product self*other. + + >>> Decimal(2).fma(3, 5) + Decimal("11") + + .. versionadded:: 2.6 + +.. method:: Decimal.is_canonical() + + Return :const:`True` if the argument is canonical and + :const:`False` otherwise. Currently, a :class:`Decimal` instance + is always canonical, so this operation always returns + :const:`True`. + + .. versionadded:: 2.6 + +.. method:: is_finite() + + Return :const:`True` if the argument is a finite number, and + :const:`False` if the argument is an infinity or a NaN. + + .. versionadded:: 2.6 + +.. method:: is_infinite() + + Return :const:`True` if the argument is either positive or + negative infinity and :const:`False` otherwise. + + .. versionadded:: 2.6 + +.. method:: is_nan() + + Return :const:`True` if the argument is a (quiet or signaling) + NaN and :const:`False` otherwise. + + .. versionadded:: 2.6 + +.. method:: is_normal() + + Return :const:`True` if the argument is a *normal* finite number. + Return :const:`False` if the argument is zero, subnormal, infinite + or a NaN. + + .. versionadded:: 2.6 + +.. method:: is_qnan() + + Return :const:`True` if the argument is a quiet NaN, and + :const:`False` otherwise. + + .. versionadded:: 2.6 + +.. method:: is_signed() + + Return :const:`True` if the argument has a negative sign and + :const:`False` otherwise. Note that zeros and NaNs can both carry + signs. + + .. versionadded:: 2.6 + +.. method:: is_snan() + + Return :const:`True` if the argument is a signaling NaN and + :const:`False` otherwise. + + .. versionadded:: 2.6 + +.. method:: is_subnormal() + + Return :const:`True` if the argument is subnormal, and + :const:`False` otherwise. + + .. versionadded:: 2.6 + +.. method:: is_zero() + + Return :const:`True` if the argument is a (positive or negative) + zero and :const:`False` otherwise. + + .. versionadded:: 2.6 + +.. method:: Decimal.ln([context]) + + Return the natural (base e) logarithm of the operand. The result + is correctly rounded using the :const:`ROUND_HALF_EVEN` rounding + mode. + + .. versionadded:: 2.6 + +.. method:: Decimal.log10([context]) + + Return the base ten logarithm of the operand. The result is + correctly rounded using the :const:`ROUND_HALF_EVEN` rounding mode. + + .. versionadded:: 2.6 + +.. method: Decimal.logb([context]) + + For a nonzero number, return the adjusted exponent of its operand + as a :class:`Decimal` instance. If the operand is a zero then + ``Decimal("-Infinity")`` is returned and the + :const:`DivisionByZero` flag is raised. If the operand is an + infinity then ``Decimal("Infinity")`` is returned. + + .. versionadded:: 2.6 + +.. method:: Decimal.logical_and(other[, context]) + + :meth:`logical_and` is a logical operation which takes two + *logical operands* (see :ref:`logical_operands_label`). The result + is the digit-wise ``and`` of the two operands. + + .. versionadded:: 2.6 + +.. method:: Decimal.logical_invert(other[, context]) + + :meth:`logical_invert` is a logical operation. The argument must + be a *logical operand* (see :ref:`logical_operands_label`). The + result is the digit-wise inversion of the operand. + + .. versionadded:: 2.6 + +.. method:: Decimal.logical_or(other[, context]) + + :meth:`logical_or` is a logical operation which takes two *logical + operands* (see :ref:`logical_operands_label`). The result is the + digit-wise ``or`` of the two operands. + + .. versionadded:: 2.6 + +.. method:: Decimal.logical_xor(other[, context]) + + :meth:`logical_xor` is a logical operation which takes two + *logical operands* (see :ref:`logical_operands_label`). The result + is the digit-wise exclusive or of the two operands. + + .. versionadded:: 2.6 .. method:: Decimal.max(other[, context]) Like ``max(self, other)`` except that the context rounding rule is applied - before returning and that :const:`NaN` values are either signalled or ignored + before returning and that :const:`NaN` values are either signaled or ignored (depending on the context and whether they are signaling or quiet). +.. method:: Decimal.max_mag(other[, context]) + + Similar to the :meth:`max` method, but the comparison is done using + the absolute values of the operands. + + .. versionadded:: 2.6 .. method:: Decimal.min(other[, context]) Like ``min(self, other)`` except that the context rounding rule is applied - before returning and that :const:`NaN` values are either signalled or ignored + before returning and that :const:`NaN` values are either signaled or ignored (depending on the context and whether they are signaling or quiet). +.. method:: Decimal.min_mag(other[, context]) + + Similar to the :meth:`min` method, but the comparison is done using + the absolute values of the operands. + + .. versionadded:: 2.6 + +.. method:: Decimal.next_minus([context]) + + Return the largest number representable in the given context (or + in the current thread's context if no context is given) that is smaller + than the given operand. + + .. versionadded:: 2.6 + +.. method:: Decimal.next_plus([context]) + + Return the smallest number representable in the given context (or + in the current thread's context if no context is given) that is + larger than the given operand. + + .. versionadded:: 2.6 + +.. method:: Decimal.next_toward(other[, context]) + + If the two operands are unequal, return the number closest to the + first operand in the direction of the second operand. If both + operands are numerically equal, return a copy of the first operand + with the sign set to be the same as the sign of the second operand. + + .. versionadded:: 2.6 .. method:: Decimal.normalize([context]) - Normalize the number by stripping the rightmost trailing zeroes and converting + Normalize the number by stripping the rightmost trailing zeros and converting any result equal to :const:`Decimal("0")` to :const:`Decimal("0e0")`. Used for producing canonical values for members of an equivalence class. For example, ``Decimal("32.100")`` and ``Decimal("0.321000e+2")`` both normalize to the equivalent value ``Decimal("32.1")``. +.. method:: Decimal.number_class([context]) + + Return a string describing the *class* of the operand. The + returned value is one of the following ten strings. + + * ``"-Infinity"``, indicating that the operand is negative infinity. + * ``"-Normal"``, indicating that the operand is a negative normal number. + * ``"-Subnormal"``, indicating that the operand is negative and subnormal. + * ``"-Zero"``, indicating that the operand is a negative zero. + * ``"+Zero"``, indicating that the operand is a positive zero. + * ``"+Subnormal"``, indicating that the operand is positive and subnormal. + * ``"+Normal"``, indicating that the operand is a positive normal number. + * ``"+Infinity"``, indicating that the operand is positive infinity. + * ``"NaN"``, indicating that the operand is a quiet NaN (Not a Number). + * ``"sNaN"``, indicating that the operand is a signaling NaN. + + .. versionadded:: 2.6 + +.. method:: Decimal.quantize(exp[, rounding[, context[, watchexp]]]) + + Returns a value equal to the first operand after rounding and + having the exponent of the second operand. -.. method:: Decimal.quantize(exp [, rounding[, context[, watchexp]]]) + >>> Decimal("1.41421356").quantize(Decimal("1.000")) + Decimal("1.414") - Quantize makes the exponent the same as *exp*. Searches for a rounding method - in *rounding*, then in *context*, and then in the current context. + Unlike other operations, if the length of the coefficient after the + quantize operation would be greater than precision, then an + :const:`InvalidOperation` is signaled. This guarantees that, unless + there is an error condition, the quantized exponent is always equal + to that of the right-hand operand. - If *watchexp* is set (default), then an error is returned whenever the resulting - exponent is greater than :attr:`Emax` or less than :attr:`Etiny`. + Also unlike other operations, quantize never signals Underflow, + even if the result is subnormal and inexact. + If the exponent of the second operand is larger than that of the + first then rounding may be necessary. In this case, the rounding + mode is determined by the ``rounding`` argument if given, else by + the given ``context`` argument; if neither argument is given the + rounding mode of the current thread's context is used. + + If watchexp is set (default), then an error is returned whenever + the resulting exponent is greater than Emax or less than Etiny. + +.. method:: Decimal.radix() + + Return ``Decimal(10)``, the radix (base) in which the + :class:`Decimal` class does all its arithmetic. Included for + compatibility with the specification. + + .. versionadded:: 2.6 .. method:: Decimal.remainder_near(other[, context]) @@ -368,16 +694,49 @@ also have a number of specialized methods: If both are equally close, the one chosen will have the same sign as *self*. +.. method:: Decimal.rotate(other[, context]) + + Return the result of rotating the digits of the first operand by + an amount specified by the second operand. The second operand + must be an integer in the range -precision through precision. The + absolute value of the second operand gives the number of places to + rotate. If the second operand is positive then rotation is to the + left; otherwise rotation is to the right. The coefficient of the + first operand is padded on the left with zeros to length precision + if necessary. The sign and exponent of the first operand are + unchanged. + + .. versionadded:: 2.6 .. method:: Decimal.same_quantum(other[, context]) Test whether self and other have the same exponent or whether both are :const:`NaN`. +.. method:: Decimal.scaleb(other[, context]) + + Return the first operand with exponent adjusted by the second. + Equivalently, return the first operand multiplied by ``10**other``. + The second operand must be an integer. + + .. versionadded:: 2.6 + +.. method:: Decimal.shift(other[, context]) + + Return the result of shifting the digits of the first operand by + an amount specified by the second operand. The second operand must + be an integer in the range -precision through precision. The + absolute value of the second operand gives the number of places to + shift. If the second operand is positive then the shift is to the + left; otherwise the shift is to the right. Digits shifted into the + coefficient are zeros. The sign and exponent of the first operand + are unchanged. + + .. versionadded:: 2.6 .. method:: Decimal.sqrt([context]) - Return the square root to full precision. + Return the square root of the argument to full precision. .. method:: Decimal.to_eng_string([context]) @@ -388,13 +747,53 @@ also have a number of specialized methods: to 3 digits left of the decimal place. For example, converts ``Decimal('123E+1')`` to ``Decimal("1.23E+3")`` - .. method:: Decimal.to_integral([rounding[, context]]) + Identical to the :meth:`to_integral_value` method. The ``to_integral`` + name has been kept for compatibility with older versions. + +.. method:: Decimal.to_integral_exact([rounding[, context]]) + + Round the argument to the nearest integer, signaling + :const:`Inexact` or :const:`Rounded` as appropriate if rounding + occurs. The rounding mode is determined by the ``rounding`` + parameter if given, else by the given ``context``. If neither + parameter is given then the rounding mode of the current context is + used. + + .. versionadded:: 2.6 + +.. method:: Decimal.to_integral_value([rounding[, context]]) + Rounds to the nearest integer without signaling :const:`Inexact` or :const:`Rounded`. If given, applies *rounding*; otherwise, uses the rounding method in either the supplied *context* or the current context. + .. versionchanged:: 2.6 + renamed from ``to_integral`` to ``to_integral_value``. The old name + remains valid for compatibility. + +.. method:: Decimal.trim() + + Returns its argument with *insignificant* trailing zeros removed. + Here, a trailing zero is considered insignificant either if it + follows the decimal point, or if the exponent of the argument (that + is, the last element of the :meth:`as_tuple` representation) is + positive. + + .. versionadded:: 2.6 + +.. _logical_operands_label: + +Logical operands +^^^^^^^^^^^^^^^^ + +The :meth:`logical_and`, :meth:`logical_invert`, :meth:`logical_or`, +and :meth:`logical_xor` methods expect their arguments to be *logical +operands*. A *logical operand* is a :class:`Decimal` instance whose +exponent and sign are both zero, and whose digits are all either +:const:`0` or :const:`1`. + .. % %%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%% @@ -509,6 +908,8 @@ In addition to the three supplied contexts, new contexts can be created with the * :const:`ROUND_HALF_EVEN` (to nearest with ties going to nearest even integer), * :const:`ROUND_HALF_UP` (to nearest with ties going away from zero), or * :const:`ROUND_UP` (away from zero). + * :const:`ROUND_05UP` (away from zero if last digit after rounding towards zero + would have been 0 or 5; otherwise towards zero) The *traps* and *flags* fields list any signals to be set. Generally, new contexts should only set traps and leave the flags clear. @@ -520,9 +921,16 @@ In addition to the three supplied contexts, new contexts can be created with the :const:`1`, exponents are printed with a capital :const:`E`; otherwise, a lowercase :const:`e` is used: :const:`Decimal('6.02e+23')`. -The :class:`Context` class defines several general purpose methods as well as a -large number of methods for doing arithmetic directly in a given context. + .. versionchanged:: 2.6 + The :const:`ROUND_05UP` rounding mode was added. +The :class:`Context` class defines several general purpose methods as +well as a large number of methods for doing arithmetic directly in a +given context. In addition, for each of the :class:`Decimal` methods +described above (with the exception of the :meth:`adjusted` and +:meth:`as_tuple` methods) there is a corresponding :class:`Context` +method. For example, ``C.exp(x)`` is equivalent to +``x.exp(context=C)``. .. method:: Context.clear_flags() @@ -533,6 +941,9 @@ large number of methods for doing arithmetic directly in a given context. Return a duplicate of the context. +.. method:: Context.copy_decimal(num) + + Return a copy of the Decimal instance num. .. method:: Context.create_decimal(num) @@ -581,42 +992,19 @@ those for the :class:`Decimal` class and are only briefly recounted here. Return the sum of *x* and *y*. -.. method:: Context.compare(x, y) - - Compares values numerically. - - Like :meth:`__cmp__` but returns a decimal instance:: - - a or b is a NaN ==> Decimal("NaN") - a < b ==> Decimal("-1") - a == b ==> Decimal("0") - a > b ==> Decimal("1") - - .. method:: Context.divide(x, y) Return *x* divided by *y*. -.. method:: Context.divmod(x, y) - - Divides two numbers and returns the integer part of the result. - - -.. method:: Context.max(x, y) +.. method:: Context.divide_int(x, y) - Compare two values numerically and return the maximum. + Return *x* divided by *y*, truncated to an integer. - If they are numerically equal then the left-hand operand is chosen as the - result. +.. method:: Context.divmod(x, y) -.. method:: Context.min(x, y) - - Compare two values numerically and return the minimum. - - If they are numerically equal then the left-hand operand is chosen as the - result. + Divides two numbers and returns the integer part of the result. .. method:: Context.minus(x) @@ -629,14 +1017,6 @@ those for the :class:`Decimal` class and are only briefly recounted here. Return the product of *x* and *y*. -.. method:: Context.normalize(x) - - Normalize reduces an operand to its simplest form. - - Essentially a :meth:`plus` operation with all trailing zeros removed from the - result. - - .. method:: Context.plus(x) Plus corresponds to the unary prefix plus operator in Python. This operation @@ -646,33 +1026,31 @@ those for the :class:`Decimal` class and are only briefly recounted here. .. method:: Context.power(x, y[, modulo]) - Return ``x ** y`` to the *modulo* if given. - - The right-hand operand must be a whole number whose integer part (after any - exponent has been applied) has no more than 9 digits and whose fractional part - (if any) is all zeros before any rounding. The operand may be positive, - negative, or zero; if negative, the absolute value of the power is used, and the - left-hand operand is inverted (divided into 1) before use. - - If the increased precision needed for the intermediate calculations exceeds the - capabilities of the implementation then an :const:`InvalidOperation` condition - is signaled. - - If, when raising to a negative power, an underflow occurs during the division - into 1, the operation is not halted at that point but continues. + Return ``x`` to the power of ``y``, reduced modulo ``modulo`` if + given. + With two arguments, compute ``x**y``. If ``x`` is negative then + ``y`` must be integral. The result will be inexact unless ``y`` is + integral and the result is finite and can be expressed exactly in + 'precision' digits. The result should always be correctly rounded, + using the rounding mode of the current thread's context. -.. method:: Context.quantize(x, y) + With three arguments, compute ``(x**y) % modulo``. For the three + argument form, the following restrictions on the arguments hold: - Returns a value equal to *x* after rounding and having the exponent of *y*. + - all three arguments must be integral + - ``y`` must be nonnegative + - at least one of ``x`` or ``y`` must be nonzero + - ``modulo`` must be nonzero and have at most 'precision' digits - Unlike other operations, if the length of the coefficient after the quantize - operation would be greater than precision, then an :const:`InvalidOperation` is - signaled. This guarantees that, unless there is an error condition, the - quantized exponent is always equal to that of the right-hand operand. + The result of ``Context.power(x, y, modulo)`` is identical to + the result that would be obtained by computing ``(x**y) % + modulo`` with unbounded precision, but is computed more + efficiently. It is always exact. - Also unlike other operations, quantize never signals Underflow, even if the - result is subnormal and inexact. + .. versionchanged:: 2.6 + ``y`` may now be nonintegral in ``x**y``. + Stricter requirements for the three-argument version. .. method:: Context.remainder(x, y) @@ -682,47 +1060,10 @@ those for the :class:`Decimal` class and are only briefly recounted here. The sign of the result, if non-zero, is the same as that of the original dividend. - -.. method:: Context.remainder_near(x, y) - - Computed the modulo as either a positive or negative value depending on which is - closest to zero. For instance, ``Decimal(10).remainder_near(6)`` returns - ``Decimal("-2")`` which is closer to zero than ``Decimal("4")``. - - If both are equally close, the one chosen will have the same sign as *self*. - - -.. method:: Context.same_quantum(x, y) - - Test whether *x* and *y* have the same exponent or whether both are - :const:`NaN`. - - -.. method:: Context.sqrt(x) - - Return the square root of *x* to full precision. - - .. method:: Context.subtract(x, y) Return the difference between *x* and *y*. - -.. method:: Context.to_eng_string() - - Convert to engineering-type string. - - Engineering notation has an exponent which is a multiple of 3, so there are up - to 3 digits left of the decimal place. For example, converts - ``Decimal('123E+1')`` to ``Decimal("1.23E+3")`` - - -.. method:: Context.to_integral(x) - - Rounds to the nearest integer without signaling :const:`Inexact` or - :const:`Rounded`. - - .. method:: Context.to_sci_string(x) Converts a number to a string using scientific notation. @@ -755,7 +1096,7 @@ condition. Typically, clamping occurs when an exponent falls outside the context's :attr:`Emin` and :attr:`Emax` limits. If possible, the exponent is reduced to - fit by adding zeroes to the coefficient. + fit by adding zeros to the coefficient. .. class:: DecimalException @@ -908,7 +1249,7 @@ Special values The number system for the :mod:`decimal` module provides special values including :const:`NaN`, :const:`sNaN`, :const:`-Infinity`, :const:`Infinity`, -and two zeroes, :const:`+0` and :const:`-0`. +and two zeros, :const:`+0` and :const:`-0`. Infinities can be constructed directly with: ``Decimal('Infinity')``. Also, they can arise from dividing by zero when the :exc:`DivisionByZero` signal is diff --git a/Doc/library/fileinput.rst b/Doc/library/fileinput.rst index fd84139..a14e05b 100644 --- a/Doc/library/fileinput.rst +++ b/Doc/library/fileinput.rst @@ -7,8 +7,9 @@ .. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org> -This module implements a helper class and functions to quickly write a loop over -standard input or a list of files. +This module implements a helper class and functions to quickly write a +loop over standard input or a list of files. If you just want to read or +write one file see :func:`open`. The typical use is:: diff --git a/Doc/library/functions.rst b/Doc/library/functions.rst index b5061fd..5d5dc1b 100644 --- a/Doc/library/functions.rst +++ b/Doc/library/functions.rst @@ -746,7 +746,8 @@ available. They are listed here in alphabetical order. Python enforces that the mode, after stripping ``'U'``, begins with ``'r'``, ``'w'`` or ``'a'``. - See also the :mod:`fileinput` module. + See also the :mod:`fileinput` module, the :mod:`os` module, and the + :mod:`os.path` module. .. function:: ord(c) diff --git a/Doc/library/hashlib.rst b/Doc/library/hashlib.rst index de2394e..0f8a742 100644 --- a/Doc/library/hashlib.rst +++ b/Doc/library/hashlib.rst @@ -36,7 +36,7 @@ concatenation of the data fed to it so far using the :meth:`digest` or Feeding string objects is to :meth:`update` is not supported, as hashes work on bytes, not on characters. -.. index:: single: OpenSSL +.. index:: single: OpenSSL; (use in module hashlib) Constructors for hash algorithms that are always present in this module are :func:`md5`, :func:`sha1`, :func:`sha224`, :func:`sha256`, :func:`sha384`, and diff --git a/Doc/library/operator.rst b/Doc/library/operator.rst index f396978..40acc6b 100644 --- a/Doc/library/operator.rst +++ b/Doc/library/operator.rst @@ -47,18 +47,18 @@ The logical operations are also generally applicable to all objects, and support truth tests, identity tests, and boolean operations: -.. function:: not_(o) - __not__(o) +.. function:: not_(obj) + __not__(obj) - Return the outcome of :keyword:`not` *o*. (Note that there is no + Return the outcome of :keyword:`not` *obj*. (Note that there is no :meth:`__not__` method for object instances; only the interpreter core defines this operation. The result is affected by the :meth:`__bool__` and :meth:`__len__` methods.) -.. function:: truth(o) +.. function:: truth(obj) - Return :const:`True` if *o* is true, and :const:`False` otherwise. This is + Return :const:`True` if *obj* is true, and :const:`False` otherwise. This is equivalent to using the :class:`bool` constructor. @@ -75,10 +75,10 @@ truth tests, identity tests, and boolean operations: The mathematical and bitwise operations are the most numerous: -.. function:: abs(o) - __abs__(o) +.. function:: abs(obj) + __abs__(obj) - Return the absolute value of *o*. + Return the absolute value of *obj*. .. function:: add(a, b) @@ -106,12 +106,12 @@ The mathematical and bitwise operations are the most numerous: Return ``a // b``. -.. function:: inv(o) - invert(o) - __inv__(o) - __invert__(o) +.. function:: inv(obj) + invert(obj) + __inv__(obj) + __invert__(obj) - Return the bitwise inverse of the number *o*. This is equivalent to ``~o``. + Return the bitwise inverse of the number *obj*. This is equivalent to ``~obj``. .. function:: lshift(a, b) @@ -132,10 +132,10 @@ The mathematical and bitwise operations are the most numerous: Return ``a * b``, for *a* and *b* numbers. -.. function:: neg(o) - __neg__(o) +.. function:: neg(obj) + __neg__(obj) - Return *o* negated. + Return *obj* negated. .. function:: or_(a, b) @@ -144,10 +144,10 @@ The mathematical and bitwise operations are the most numerous: Return the bitwise or of *a* and *b*. -.. function:: pos(o) - __pos__(o) +.. function:: pos(obj) + __pos__(obj) - Return *o* positive. + Return *obj* positive. .. function:: pow(a, b) @@ -373,24 +373,30 @@ objects. ... pass ... >>> import operator - >>> o = C() - >>> operator.isMappingType(o) + >>> obj = C() + >>> operator.isMappingType(obj) True +.. note:: + + Python 3 is expected to introduce abstract base classes for + collection types, so it should be possible to write, for example, + ``isinstance(obj, collections.Mapping)`` and ``isinstance(obj, + collections.Sequence)``. -.. function:: isCallable(o) +.. function:: isCallable(obj) .. deprecated:: 2.0 Use the :func:`callable` built-in function instead. - Returns true if the object *o* can be called like a function, otherwise it + Returns true if the object *obj* can be called like a function, otherwise it returns false. True is returned for functions, bound and unbound methods, class objects, and instance objects which support the :meth:`__call__` method. -.. function:: isMappingType(o) +.. function:: isMappingType(obj) - Returns true if the object *o* supports the mapping interface. This is true for + Returns true if the object *obj* supports the mapping interface. This is true for dictionaries and all instance objects defining :meth:`__getitem__`. .. warning:: @@ -400,9 +406,9 @@ objects. useful than it otherwise might be. -.. function:: isNumberType(o) +.. function:: isNumberType(obj) - Returns true if the object *o* represents a number. This is true for all + Returns true if the object *obj* represents a number. This is true for all numeric types implemented in C. .. warning:: @@ -412,9 +418,9 @@ objects. useful than it otherwise might be. -.. function:: isSequenceType(o) +.. function:: isSequenceType(obj) - Returns true if the object *o* supports the sequence protocol. This returns true + Returns true if the object *obj* supports the sequence protocol. This returns true for all objects which define sequence methods in C, and for all instance objects defining :meth:`__getitem__`. @@ -484,7 +490,7 @@ Python syntax and the functions in the :mod:`operator` module. +-----------------------+-------------------------+---------------------------------+ | Concatenation | ``seq1 + seq2`` | ``concat(seq1, seq2)`` | +-----------------------+-------------------------+---------------------------------+ -| Containment Test | ``o in seq`` | ``contains(seq, o)`` | +| Containment Test | ``obj in seq`` | ``contains(seq, obj)`` | +-----------------------+-------------------------+---------------------------------+ | Division | ``a / b`` | ``div(a, b)`` (without | | | | ``__future__.division``) | @@ -508,11 +514,11 @@ Python syntax and the functions in the :mod:`operator` module. +-----------------------+-------------------------+---------------------------------+ | Identity | ``a is not b`` | ``is_not(a, b)`` | +-----------------------+-------------------------+---------------------------------+ -| Indexed Assignment | ``o[k] = v`` | ``setitem(o, k, v)`` | +| Indexed Assignment | ``obj[k] = v`` | ``setitem(obj, k, v)`` | +-----------------------+-------------------------+---------------------------------+ -| Indexed Deletion | ``del o[k]`` | ``delitem(o, k)`` | +| Indexed Deletion | ``del obj[k]`` | ``delitem(obj, k)`` | +-----------------------+-------------------------+---------------------------------+ -| Indexing | ``o[k]`` | ``getitem(o, k)`` | +| Indexing | ``obj[k]`` | ``getitem(obj, k)`` | +-----------------------+-------------------------+---------------------------------+ | Left Shift | ``a << b`` | ``lshift(a, b)`` | +-----------------------+-------------------------+---------------------------------+ @@ -534,11 +540,11 @@ Python syntax and the functions in the :mod:`operator` module. +-----------------------+-------------------------+---------------------------------+ | Slicing | ``seq[i:j]`` | ``getslice(seq, i, j)`` | +-----------------------+-------------------------+---------------------------------+ -| String Formatting | ``s % o`` | ``mod(s, o)`` | +| String Formatting | ``s % obj`` | ``mod(s, obj)`` | +-----------------------+-------------------------+---------------------------------+ | Subtraction | ``a - b`` | ``sub(a, b)`` | +-----------------------+-------------------------+---------------------------------+ -| Truth Test | ``o`` | ``truth(o)`` | +| Truth Test | ``obj`` | ``truth(obj)`` | +-----------------------+-------------------------+---------------------------------+ | Ordering | ``a < b`` | ``lt(a, b)`` | +-----------------------+-------------------------+---------------------------------+ diff --git a/Doc/library/os.rst b/Doc/library/os.rst index 387fe54..2d87d8c 100644 --- a/Doc/library/os.rst +++ b/Doc/library/os.rst @@ -8,9 +8,10 @@ This module provides a more portable way of using operating system dependent functionality than importing a operating system dependent built-in module like -:mod:`posix` or :mod:`nt`. (If you just want to read or write a file see -:func:`open`, and if you want to manipulate paths, see the :mod:`os.path` -module.) +:mod:`posix` or :mod:`nt`. If you just want to read or write a file see +:func:`open`, if you want to manipulate paths, see the :mod:`os.path` +module, and if you want to read all the lines in all the files on the +command line see the :mod:`fileinput` module. This module searches for an operating system dependent built-in module like :mod:`mac` or :mod:`posix` and exports the same functions and data as found diff --git a/Doc/library/simplexmlrpcserver.rst b/Doc/library/simplexmlrpcserver.rst index 2288907..8533bf8 100644 --- a/Doc/library/simplexmlrpcserver.rst +++ b/Doc/library/simplexmlrpcserver.rst @@ -142,7 +142,7 @@ server:: import xmlrpclib - s = xmlrpclib.Server('http://localhost:8000') + s = xmlrpclib.ServerProxy('http://localhost:8000') print(s.pow(2,3)) # Returns 2**3 = 8 print(s.add(2,3)) # Returns 5 print(s.div(5,2)) # Returns 5//2 = 2 diff --git a/Doc/library/ssl.rst b/Doc/library/ssl.rst index 852a905..851f742 100644 --- a/Doc/library/ssl.rst +++ b/Doc/library/ssl.rst @@ -9,6 +9,10 @@ .. sectionauthor:: Bill Janssen <bill.janssen@gmail.com> +.. index:: single: OpenSSL; (use in module ssl) + +.. index:: TLS, SSL, Transport Layer Security, Secure Sockets Layer + This module provides access to Transport Layer Security (often known as "Secure Sockets Layer") encryption and peer authentication facilities for network sockets, both client-side and server-side. @@ -19,18 +23,125 @@ platforms, as long as OpenSSL is installed on that platform. .. note:: Some behavior may be platform dependent, since calls are made to the operating - system socket APIs. + system socket APIs. The installed version of OpenSSL may also cause + variations in behavior. This section documents the objects and functions in the ``ssl`` module; for more general information about TLS, SSL, and certificates, the -reader is referred to the documents in the :ref:`ssl-references` section. +reader is referred to the documents in the "See Also" section at +the bottom. -This module defines a class, :class:`ssl.sslsocket`, which is -derived from the :class:`socket.socket` type, and supports additional +This module provides a class, :class:`ssl.SSLSocket`, which is +derived from the :class:`socket.socket` type, and provides +a socket-like wrapper that also encrypts and decrypts the data +going over the socket with SSL. It supports additional :meth:`read` and :meth:`write` methods, along with a method, :meth:`getpeercert`, -to retrieve the certificate of the other side of the connection. +to retrieve the certificate of the other side of the connection, and +a method, :meth:`cipher`, to retrieve the cipher being used for the +secure connection. + +Functions, Constants, and Exceptions +------------------------------------ -This module defines the following functions, exceptions, and constants: +.. exception:: SSLError + + Raised to signal an error from the underlying SSL implementation. This + signifies some problem in the higher-level + encryption and authentication layer that's superimposed on the underlying + network connection. This error is a subtype of :exc:`socket.error`, which + in turn is a subtype of :exc:`IOError`. + +.. function:: wrap_socket (sock, keyfile=None, certfile=None, server_side=False, cert_reqs=CERT_NONE, ssl_version={see docs}, ca_certs=None) + + Takes an instance ``sock`` of :class:`socket.socket`, and returns an instance of :class:`ssl.SSLSocket`, a subtype + of :class:`socket.socket`, which wraps the underlying socket in an SSL context. + For client-side sockets, the context construction is lazy; if the underlying socket isn't + connected yet, the context construction will be performed after :meth:`connect` is called + on the socket. For server-side sockets, if the socket has no remote peer, it is assumed + to be a listening socket, and the server-side SSL wrapping is automatically performed + on client connections accepted via the :meth:`accept` method. :func:`wrap_socket` may + raise :exc:`SSLError`. + + The ``keyfile`` and ``certfile`` parameters specify optional files which contain a certificate + to be used to identify the local side of the connection. See the discussion of :ref:`ssl-certificates` + for more information on how the certificate is stored in the ``certfile``. + + Often the private key is stored + in the same file as the certificate; in this case, only the ``certfile`` parameter need be + passed. If the private key is stored in a separate file, both parameters must be used. + If the private key is stored in the ``certfile``, it should come before the first certificate + in the certificate chain:: + + -----BEGIN RSA PRIVATE KEY----- + ... (private key in base64 encoding) ... + -----END RSA PRIVATE KEY----- + -----BEGIN CERTIFICATE----- + ... (certificate in base64 PEM encoding) ... + -----END CERTIFICATE----- + + The parameter ``server_side`` is a boolean which identifies whether server-side or client-side + behavior is desired from this socket. + + The parameter ``cert_reqs`` specifies whether a certificate is + required from the other side of the connection, and whether it will + be validated if provided. It must be one of the three values + :const:`CERT_NONE` (certificates ignored), :const:`CERT_OPTIONAL` (not required, + but validated if provided), or :const:`CERT_REQUIRED` (required and + validated). If the value of this parameter is not :const:`CERT_NONE`, then + the ``ca_certs`` parameter must point to a file of CA certificates. + + The ``ca_certs`` file contains a set of concatenated "certification authority" certificates, + which are used to validate certificates passed from the other end of the connection. + See the discussion of :ref:`ssl-certificates` for more information about how to arrange + the certificates in this file. + + The parameter ``ssl_version`` specifies which version of the SSL protocol to use. + Typically, the server chooses a particular protocol version, and the client + must adapt to the server's choice. Most of the versions are not interoperable + with the other versions. If not specified, for client-side operation, the + default SSL version is SSLv3; for server-side operation, SSLv23. These + version selections provide the most compatibility with other versions. + + Here's a table showing which versions in a client (down the side) + can connect to which versions in a server (along the top): + + .. table:: + + ======================== ========= ========= ========== ========= + *client* / **server** **SSLv2** **SSLv3** **SSLv23** **TLSv1** + *SSLv2* yes no yes* no + *SSLv3* yes yes yes no + *SSLv23* yes no yes no + *TLSv1* no no yes yes + ======================== ========= ========= ========== ========= + + `*` In some older versions of OpenSSL (for instance, 0.9.7l on OS X 10.4), + an SSLv2 client could not connect to an SSLv23 server. + +.. function:: RAND_status() + + Returns True if the SSL pseudo-random number generator has been + seeded with 'enough' randomness, and False otherwise. You can use + :func:`ssl.RAND_egd` and :func:`ssl.RAND_add` to increase the randomness + of the pseudo-random number generator. + +.. function:: RAND_egd(path) + + If you are running an entropy-gathering daemon (EGD) somewhere, and ``path`` + is the pathname of a socket connection open to it, this will read + 256 bytes of randomness from the socket, and add it to the SSL pseudo-random number generator + to increase the security of generated secret keys. This is typically only + necessary on systems without better sources of randomness. + + See http://egd.sourceforge.net/ or http://prngd.sourceforge.net/ for + sources of entropy-gathering daemons. + +.. function:: RAND_add(bytes, entropy) + + Mixes the given ``bytes`` into the SSL pseudo-random number generator. + The parameter ``entropy`` (a float) is a lower bound on the entropy + contained in string (so you can always use :const:`0.0`). + See :rfc:`1750` for more information on sources of entropy. .. function:: cert_time_to_seconds(timestring) @@ -48,12 +159,27 @@ This module defines the following functions, exceptions, and constants: 'Wed May 9 00:00:00 2007' >>> -.. exception:: sslerror +.. function:: get_server_certificate (addr, ssl_version=PROTOCOL_SSLv3, ca_certs=None) - Raised to signal an error from the underlying SSL implementation. This - signifies some problem in the higher-level - encryption and authentication layer that's superimposed on the underlying - network connection. + Given the address ``addr`` of an SSL-protected server, as a + (*hostname*, *port-number*) pair, fetches the server's certificate, + and returns it as a PEM-encoded string. If ``ssl_version`` is + specified, uses that version of the SSL protocol to attempt to + connect to the server. If ``ca_certs`` is specified, it should be + a file containing a list of root certificates, the same format as + used for the same parameter in :func:`wrap_socket`. The call will + attempt to validate the server certificate against that set of root + certificates, and will fail if the validation attempt fails. + +.. function:: DER_cert_to_PEM_cert (DER_cert_bytes) + + Given a certificate as a DER-encoded blob of bytes, returns a PEM-encoded + string version of the same certificate. + +.. function:: PEM_cert_to_DER_cert (PEM_cert_string) + + Given a certificate as an ASCII PEM string, returns a DER-encoded + sequence of bytes for that same certificate. .. data:: CERT_NONE @@ -84,13 +210,16 @@ This module defines the following functions, exceptions, and constants: .. data:: PROTOCOL_SSLv23 - Selects SSL version 2 or 3 as the channel encryption protocol. This is a setting to use for maximum compatibility - with the other end of an SSL connection, but it may cause the specific ciphers chosen for the encryption to be - of fairly low quality. + Selects SSL version 2 or 3 as the channel encryption protocol. + This is a setting to use with servers for maximum compatibility + with the other end of an SSL connection, but it may cause the + specific ciphers chosen for the encryption to be of fairly low + quality. .. data:: PROTOCOL_SSLv3 Selects SSL version 3 as the channel encryption protocol. + For clients, this is the maximally compatible SSL variant. .. data:: PROTOCOL_TLSv1 @@ -99,6 +228,69 @@ This module defines the following functions, exceptions, and constants: protection, if both sides can speak it. +SSLSocket Objects +----------------- + +.. method:: SSLSocket.read([nbytes=1024]) + + Reads up to ``nbytes`` bytes from the SSL-encrypted channel and returns them. + +.. method:: SSLSocket.write(data) + + Writes the ``data`` to the other side of the connection, using the + SSL channel to encrypt. Returns the number of bytes written. + +.. method:: SSLSocket.getpeercert(binary_form=False) + + If there is no certificate for the peer on the other end of the + connection, returns ``None``. + + If the the parameter ``binary_form`` is :const:`False`, and a + certificate was received from the peer, this method returns a + :class:`dict` instance. If the certificate was not validated, the + dict is empty. If the certificate was validated, it returns a dict + with the keys ``subject`` (the principal for which the certificate + was issued), and ``notAfter`` (the time after which the certificate + should not be trusted). The certificate was already validated, so + the ``notBefore`` and ``issuer`` fields are not returned. If a + certificate contains an instance of the *Subject Alternative Name* + extension (see :rfc:`3280`), there will also be a + ``subjectAltName`` key in the dictionary. + + The "subject" field is a tuple containing the sequence of relative + distinguished names (RDNs) given in the certificate's data + structure for the principal, and each RDN is a sequence of + name-value pairs:: + + {'notAfter': 'Feb 16 16:54:50 2013 GMT', + 'subject': ((('countryName', u'US'),), + (('stateOrProvinceName', u'Delaware'),), + (('localityName', u'Wilmington'),), + (('organizationName', u'Python Software Foundation'),), + (('organizationalUnitName', u'SSL'),), + (('commonName', u'somemachine.python.org'),))} + + If the ``binary_form`` parameter is :const:`True`, and a + certificate was provided, this method returns the DER-encoded form + of the entire certificate as a sequence of bytes, or :const:`None` if the + peer did not provide a certificate. This return + value is independent of validation; if validation was required + (:const:`CERT_OPTIONAL` or :const:`CERT_REQUIRED`), it will have + been validated, but if :const:`CERT_NONE` was used to establish the + connection, the certificate, if present, will not have been validated. + +.. method:: SSLSocket.cipher() + + Returns a three-value tuple containing the name of the cipher being + used, the version of the SSL protocol that defines its use, and the + number of secret bits being used. If no connection has been + established, returns ``None``. + + +.. index:: single: certificates + +.. index:: single: X509 certificate + .. _ssl-certificates: Certificates @@ -127,8 +319,12 @@ can use a certificate to prove who they are. The other side of a network connection can also be required to produce a certificate, and that certificate can be validated to the satisfaction of the client or server that requires such validation. -The connection can be set to fail automatically if such -validation is not achieved. +The connection attempt can be set to raise an exception if +the validation fails. Validation is done +automatically, by the underlying OpenSSL framework; the +application need not concern itself with its mechanics. +But the application does usually need to provide +sets of certificates to allow this process to take place. Python uses files to contain certificates. They should be formatted as "PEM" (see :rfc:`1422`), which is a base-64 encoded form wrapped @@ -167,108 +363,55 @@ certificate, you need to provide a "CA certs" file, filled with the certificate chains for each issuer you are willing to trust. Again, this file just contains these chains concatenated together. For validation, Python will use the first chain it finds in the file which matches. -Some "standard" root certificates are available at -http://www.thawte.com/roots/ (for Thawte roots) and -http://www.verisign.com/support/roots.html (for Verisign roots). -See also :rfc:`4158` for more discussion of the way in which +Some "standard" root certificates are available from various certification +authorities: +`CACert.org <http://www.cacert.org/index.php?id=3>`_, +`Thawte <http://www.thawte.com/roots/>`_, +`Verisign <http://www.verisign.com/support/roots.html>`_, +`Positive SSL <http://www.PositiveSSL.com/ssl-certificate-support/cert_installation/UTN-USERFirst-Hardware.crt>`_ (used by python.org), +`Equifax and GeoTrust <http://www.geotrust.com/resources/root_certificates/index.asp>`_. + +In general, if you are using +SSL3 or TLS1, you don't need to put the full chain in your "CA certs" file; +you only need the root certificates, and the remote peer is supposed to +furnish the other certificates necessary to chain from its certificate to +a root certificate. +See :rfc:`4158` for more discussion of the way in which certification chains can be built. - -sslsocket Objects ------------------ - -.. class:: sslsocket(sock [, keyfile=None, certfile=None, server_side=False, cert_reqs=CERT_NONE, ssl_version=PROTOCOL_SSLv23, ca_certs=None]) - - Takes an instance ``sock`` of :class:`socket.socket`, and returns an instance of a subtype - of :class:`socket.socket` which wraps the underlying socket in an SSL context. - For client-side sockets, the context construction is lazy; if the underlying socket isn't - connected yet, the context construction will be performed after :meth:`connect` is called - on the socket. - - The ``keyfile`` and ``certfile`` parameters specify optional files which contain a certificate - to be used to identify the local side of the connection. See the above discussion of :ref:`ssl-certificates` - for more information on how the certificate is stored in the ``certfile``. - - Often the private key is stored - in the same file as the certificate; in this case, only the ``certfile`` parameter need be - passed. If the private key is stored in a separate file, both parameters must be used. - If the private key is stored in the ``certfile``, it should come before the first certificate - in the certificate chain:: - - -----BEGIN RSA PRIVATE KEY----- - ... (private key in base64 encoding) ... - -----END RSA PRIVATE KEY----- - -----BEGIN CERTIFICATE----- - ... (certificate in base64 PEM encoding) ... - -----END CERTIFICATE----- - - The parameter ``server_side`` is a boolean which identifies whether server-side or client-side - behavior is desired from this socket. - - The parameter ``cert_reqs`` specifies whether a certificate is - required from the other side of the connection, and whether it will - be validated if provided. It must be one of the three values - :const:`CERT_NONE` (certificates ignored), :const:`CERT_OPTIONAL` (not required, - but validated if provided), or :const:`CERT_REQUIRED` (required and - validated). If the value of this parameter is not :const:`CERT_NONE`, then - the ``ca_certs`` parameter must point to a file of CA certificates. - - The parameter ``ssl_version`` specifies which version of the SSL protocol to use. Typically, - the server specifies this, and a client connecting to it must use the same protocol. An - SSL server using :const:`PROTOCOL_SSLv23` can understand a client connecting via SSL2, SSL3, or TLS1, - but a client using :const:`PROTOCOL_SSLv23` can only connect to an SSL2 server. - - The ``ca_certs`` file contains a set of concatenated "certification authority" certificates, - which are used to validate certificates passed from the other end of the connection. - See the above discussion of :ref:`ssl-certificates` for more information about how to arrange - the certificates in this file. - -.. method:: sslsocket.read([nbytes]) - - Reads up to ``nbytes`` bytes from the SSL-encrypted channel and returns them. - -.. method:: sslsocket.write(data) - - Writes the ``data`` to the other side of the connection, using the SSL channel to encrypt. Returns the number - of bytes written. - -.. method:: sslsocket.getpeercert() - - If there is no certificate for the peer on the other end of the connection, returns ``None``. - If a certificate was received from the peer, but not validated, returns an empty ``dict`` instance. - If a certificate was received and validated, returns a ``dict`` instance with the fields - ``subject`` (the principal for which the certificate was issued), ``issuer`` (the signer of - the certificate), ``notBefore`` (the time before which the certificate should not be trusted), - and ``notAfter`` (the time after which the certificate should not be trusted) filled in. - - The "subject" and "issuer" fields are tuples containing the name-value fields - given in the certificate's data structure for each principal:: - - {'issuer': (('countryName', u'US'), - ('stateOrProvinceName', u'Delaware'), - ('localityName', u'Wilmington'), - ('organizationName', u'Python Software Foundation'), - ('organizationalUnitName', u'SSL'), - ('commonName', u'somemachine.python.org')), - 'notAfter': 'Feb 16 16:54:50 2013 GMT', - 'notBefore': 'Aug 27 16:54:50 2007 GMT', - 'subject': (('countryName', u'US'), - ('stateOrProvinceName', u'Delaware'), - ('localityName', u'Wilmington'), - ('organizationName', u'Python Software Foundation'), - ('organizationalUnitName', u'SSL'), - ('commonName', u'somemachine.python.org')), - 'version': 2} - - This certificate is said to be *self-signed*, because the subject - and issuer are the same entity. The *version* field refers to the X509 version - that's used for the certificate. - -.. method:: sslsocket.ssl_shutdown() - - Closes the SSL context (if any) over the socket, but leaves the socket connection - open for further use, if both sides are willing. This is different from :meth:`socket.socket.shutdown`, - which will close the connection, but leave the local socket available for further use. +If you are going to create a server that provides SSL-encrypted +connection services, you will need to acquire a certificate for that +service. There are many ways of acquiring appropriate certificates, +such as buying one from a certification authority. Another common +practice is to generate a self-signed certificate. The simplest +way to do this is with the OpenSSL package, using something like +the following:: + + % openssl req -new -x509 -days 365 -nodes -out cert.pem -keyout cert.pem + Generating a 1024 bit RSA private key + .......++++++ + .............................++++++ + writing new private key to 'cert.pem' + ----- + You are about to be asked to enter information that will be incorporated + into your certificate request. + What you are about to enter is what is called a Distinguished Name or a DN. + There are quite a few fields but you can leave some blank + For some fields there will be a default value, + If you enter '.', the field will be left blank. + ----- + Country Name (2 letter code) [AU]:US + State or Province Name (full name) [Some-State]:MyState + Locality Name (eg, city) []:Some City + Organization Name (eg, company) [Internet Widgits Pty Ltd]:My Organization, Inc. + Organizational Unit Name (eg, section) []:My Group + Common Name (eg, YOUR name) []:myserver.mygroup.myorganization.com + Email Address []:ops@myserver.mygroup.myorganization.com + % + +The disadvantage of a self-signed certificate is that it is its +own root certificate, and no one else will have it in their cache +of known (and trusted) root certificates. Examples @@ -295,12 +438,17 @@ sends some bytes, and reads part of the response:: import socket, ssl, pprint s = socket.socket(socket.AF_INET, socket.SOCK_STREAM) - ssl_sock = ssl.sslsocket(s, ca_certs="/etc/ca_certs_file", cert_reqs=ssl.CERT_REQUIRED) + + # require a certificate from the server + ssl_sock = ssl.wrap_socket(s, + ca_certs="/etc/ca_certs_file", + cert_reqs=ssl.CERT_REQUIRED) ssl_sock.connect(('www.verisign.com', 443)) print(repr(ssl_sock.getpeername())) pprint.pprint(ssl_sock.getpeercert()) + print(pprint.pformat(ssl_sock.getpeercert())) # Set a simple HTTP request -- use httplib in actual code. ssl_sock.write("""GET / HTTP/1.0\r @@ -310,35 +458,29 @@ sends some bytes, and reads part of the response:: # read all the data returned by the server. data = ssl_sock.read() - # note that closing the sslsocket will also close the underlying socket + # note that closing the SSLSocket will also close the underlying socket ssl_sock.close() -As of September 4, 2007, the certificate printed by this program +As of September 6, 2007, the certificate printed by this program looked like this:: - {'issuer': (('countryName', u'US'), - ('organizationName', u'VeriSign, Inc.'), - ('organizationalUnitName', u'VeriSign Trust Network'), - ('organizationalUnitName', - u'Terms of use at https://www.verisign.com/rpa (c)06'), - ('commonName', - u'VeriSign Class 3 Extended Validation SSL SGC CA')), - 'notAfter': 'May 8 23:59:59 2009 GMT', - 'notBefore': 'May 9 00:00:00 2007 GMT', - 'subject': (('serialNumber', u'2497886'), - ('1.3.6.1.4.1.311.60.2.1.3', u'US'), - ('1.3.6.1.4.1.311.60.2.1.2', u'Delaware'), - ('countryName', u'US'), - ('postalCode', u'94043'), - ('stateOrProvinceName', u'California'), - ('localityName', u'Mountain View'), - ('streetAddress', u'487 East Middlefield Road'), - ('organizationName', u'VeriSign, Inc.'), - ('organizationalUnitName', u'Production Security Services'), - ('organizationalUnitName', - u'Terms of use at www.verisign.com/rpa (c)06'), - ('commonName', u'www.verisign.com')), - 'version': 2} + {'notAfter': 'May 8 23:59:59 2009 GMT', + 'subject': ((('serialNumber', u'2497886'),), + (('1.3.6.1.4.1.311.60.2.1.3', u'US'),), + (('1.3.6.1.4.1.311.60.2.1.2', u'Delaware'),), + (('countryName', u'US'),), + (('postalCode', u'94043'),), + (('stateOrProvinceName', u'California'),), + (('localityName', u'Mountain View'),), + (('streetAddress', u'487 East Middlefield Road'),), + (('organizationName', u'VeriSign, Inc.'),), + (('organizationalUnitName', + u'Production Security Services'),), + (('organizationalUnitName', + u'Terms of use at www.verisign.com/rpa (c)06'),), + (('commonName', u'www.verisign.com'),))} + +which is a fairly poorly-formed ``subject`` field. Server-side operation ^^^^^^^^^^^^^^^^^^^^^ @@ -354,12 +496,15 @@ to connect:: bindsocket.listen(5) When one did, you'd call :meth:`accept` on the socket to get the new socket from the other -end, and use :func:`sslsocket` to create a server-side SSL context for it:: +end, and use :func:`wrap_socket` to create a server-side SSL context for it:: while True: newsocket, fromaddr = bindsocket.accept() - connstream = ssl.sslsocket(newsocket, server_side=True, certfile="mycertfile", - keyfile="mykeyfile", ssl_protocol=ssl.PROTOCOL_TLSv1) + connstream = ssl.wrap_socket(newsocket, + server_side=True, + certfile="mycertfile", + keyfile="mykeyfile", + ssl_protocol=ssl.PROTOCOL_TLSv1) deal_with_client(connstream) Then you'd read data from the ``connstream`` and do something with it till you are finished with the client (or the client is finished with you):: @@ -370,7 +515,8 @@ Then you'd read data from the ``connstream`` and do something with it till you a # null data means the client is finished with us while data: if not do_something(connstream, data): - # we'll assume do_something returns False when we're finished with client + # we'll assume do_something returns False + # when we're finished with client break data = connstream.read() # finished with client @@ -379,16 +525,19 @@ Then you'd read data from the ``connstream`` and do something with it till you a And go back to listening for new client connections. -.. _ssl-references: +.. seealso:: -References ----------- + Class :class:`socket.socket` + Documentation of underlying :mod:`socket` class -Class :class:`socket.socket` - Documentation of underlying :mod:`socket` class + `Introducing SSL and Certificates using OpenSSL <http://old.pseudonym.org/ssl/wwwj-index.html>`_ + Frederick J. Hirsch -`Introducing SSL and Certificates using OpenSSL <http://old.pseudonym.org/ssl/wwwj-index.html>`_, by Frederick J. Hirsch + `RFC 1422: Privacy Enhancement for Internet Electronic Mail: Part II: Certificate-Based Key Management <http://www.ietf.org/rfc/rfc1422>`_ + Steve Kent -`Privacy Enhancement for Internet Electronic Mail: Part II: Certificate-Based Key Management`, :rfc:`1422`, by Steve Kent + `RFC 1750: Randomness Recommendations for Security <http://www.ietf.org/rfc/rfc1750>`_ + D. Eastlake et. al. -`Internet X.509 Public Key Infrastructure Certificate and CRL Profile`, :rfc:`3280`, Housley et. al. + `RFC 3280: Internet X.509 Public Key Infrastructure Certificate and CRL Profile <http://www.ietf.org/rfc/rfc3280>`_ + Housley et. al. diff --git a/Doc/library/undoc.rst b/Doc/library/undoc.rst index 3cc94bb..942a924 100644 --- a/Doc/library/undoc.rst +++ b/Doc/library/undoc.rst @@ -19,9 +19,6 @@ Miscellaneous useful utilities Some of these are very old and/or not very robust; marked with "hmm." -:mod:`bdb` - --- A generic Python debugger base class (used by pdb). - :mod:`ihooks` --- Import hook support (for :mod:`rexec`; may become obsolete). diff --git a/Doc/reference/index.rst b/Doc/reference/index.rst index 18bf053..a179d21 100644 --- a/Doc/reference/index.rst +++ b/Doc/reference/index.rst @@ -17,7 +17,7 @@ write a Python extension module, and the :ref:`c-api-index` describes the interfaces available to C/C++ programmers in detail. .. toctree:: - :maxdepth: 2 + :maxdepth: 3 introduction.rst lexical_analysis.rst diff --git a/Doc/reference/simple_stmts.rst b/Doc/reference/simple_stmts.rst index b586871..5693f0d 100644 --- a/Doc/reference/simple_stmts.rst +++ b/Doc/reference/simple_stmts.rst @@ -62,47 +62,6 @@ using the built-in :func:`repr` function and the resulting string is written to standard output on a line by itself (except if the result is ``None``, so that procedure calls do not cause any output.) - -.. _assert: - -Assert statements -================= - -.. index:: - statement: assert - pair: debugging; assertions - single: __debug__ - exception: AssertionError - -Assert statements are a convenient way to insert debugging assertions into a -program: - -.. productionlist:: - assert_stmt: "assert" `expression` ["," `expression`] - -The simple form, ``assert expression``, is equivalent to :: - - if __debug__: - if not expression: raise AssertionError - -The extended form, ``assert expression1, expression2``, is equivalent to :: - - if __debug__: - if not expression1: raise AssertionError(expression2) - -These equivalences assume that :data:`__debug__` and :exc:`AssertionError` refer -to the built-in variables with those names. In the current implementation, the -built-in variable :data:`__debug__` is ``True`` under normal circumstances, -``False`` when optimization is requested (command line option ``-O``). The -current code generator emits no code for an assert statement when optimization -is requested at compile time. Note that it is unnecessary to include the source -code for the expression that failed in the error message; it will be displayed -as part of the stack trace. - -Assignments to :data:`__debug__` are illegal. The value for the built-in -variable is determined when the interpreter starts. - - .. _assignment: Assignment statements @@ -308,6 +267,48 @@ instance variable. For example:: a.x += 1 # writes a.x as 4 leaving A.x as 3 +.. _assert: + +The :keyword:`assert` statement +=============================== + +.. index:: + statement: assert + pair: debugging; assertions + +Assert statements are a convenient way to insert debugging assertions into a +program: + +.. productionlist:: + assert_stmt: "assert" `expression` ["," `expression`] + +The simple form, ``assert expression``, is equivalent to :: + + if __debug__: + if not expression: raise AssertionError + +The extended form, ``assert expression1, expression2``, is equivalent to :: + + if __debug__: + if not expression1: raise AssertionError, expression2 + +.. index:: + single: __debug__ + exception: AssertionError + +These equivalences assume that ``__debug__`` and :exc:`AssertionError` refer to +the built-in variables with those names. In the current implementation, the +built-in variable ``__debug__`` is ``True`` under normal circumstances, +``False`` when optimization is requested (command line option -O). The current +code generator emits no code for an assert statement when optimization is +requested at compile time. Note that it is unnecessary to include the source +code for the expression that failed in the error message; it will be displayed +as part of the stack trace. + +Assignments to ``__debug__`` are illegal. The value for the built-in variable +is determined when the interpreter starts. + + .. _pass: The :keyword:`pass` statement diff --git a/Doc/whatsnew/2.5.rst b/Doc/whatsnew/2.5.rst index f0429ec..497c612 100644 --- a/Doc/whatsnew/2.5.rst +++ b/Doc/whatsnew/2.5.rst @@ -637,7 +637,7 @@ precision and rounding characteristics for computations:: print v.sqrt() -.. _context-managers: +.. _new-25-context-managers: Writing Context Managers ------------------------ diff --git a/Doc/whatsnew/2.6.rst b/Doc/whatsnew/2.6.rst index 154b103..2dc5844 100644 --- a/Doc/whatsnew/2.6.rst +++ b/Doc/whatsnew/2.6.rst @@ -2,6 +2,9 @@ What's New in Python 2.6 **************************** +.. % XXX mention switch to reST for documentation +.. % XXX mention switch to Roundup for bug tracking + :Author: A.M. Kuchling :Release: |release| :Date: |today| @@ -67,12 +70,367 @@ new feature. .. % sets module deprecated .. % ====================================================================== +Python 3.0 +================ + +.. % XXX add general comment about Python 3.0 features in 2.6 + +.. % XXX mention -3 switch + +A new command-line switch, :option:`-3`, enables warnings +about features that will be removed in Python 3.0. You can run code +with this switch to see how much work will be necessary to port +code to 3.0. + +.. seealso:: + + The 3xxx series of PEPs, which describes the development process for + Python 3.0 and various features that have been accepted, rejected, + or are still under consideration. + +PEP 343: The 'with' statement +============================= + +The previous version, Python 2.5, added the ':keyword:`with`' +statement an optional feature, to be enabled by a ``from __future__ +import generators`` directive. In 2.6 the statement no longer need to +be specially enabled; this means that :keyword:`with` is now always a +keyword. The rest of this section is a copy of the corresponding +section from "What's New in Python 2.5" document; if you read +it back when Python 2.5 came out, you can skip the rest of this +section. + +The ':keyword:`with`' statement clarifies code that previously would use +``try...finally`` blocks to ensure that clean-up code is executed. In this +section, I'll discuss the statement as it will commonly be used. In the next +section, I'll examine the implementation details and show how to write objects +for use with this statement. + +The ':keyword:`with`' statement is a new control-flow structure whose basic +structure is:: + + with expression [as variable]: + with-block + +The expression is evaluated, and it should result in an object that supports the +context management protocol (that is, has :meth:`__enter__` and :meth:`__exit__` +methods. + +The object's :meth:`__enter__` is called before *with-block* is executed and +therefore can run set-up code. It also may return a value that is bound to the +name *variable*, if given. (Note carefully that *variable* is *not* assigned +the result of *expression*.) + +After execution of the *with-block* is finished, the object's :meth:`__exit__` +method is called, even if the block raised an exception, and can therefore run +clean-up code. + +Some standard Python objects now support the context management protocol and can +be used with the ':keyword:`with`' statement. File objects are one example:: + + with open('/etc/passwd', 'r') as f: + for line in f: + print line + ... more processing code ... + +After this statement has executed, the file object in *f* will have been +automatically closed, even if the :keyword:`for` loop raised an exception part- +way through the block. + +.. note:: + + In this case, *f* is the same object created by :func:`open`, because + :meth:`file.__enter__` returns *self*. + +The :mod:`threading` module's locks and condition variables also support the +':keyword:`with`' statement:: + + lock = threading.Lock() + with lock: + # Critical section of code + ... + +The lock is acquired before the block is executed and always released once the +block is complete. + +The new :func:`localcontext` function in the :mod:`decimal` module makes it easy +to save and restore the current decimal context, which encapsulates the desired +precision and rounding characteristics for computations:: + + from decimal import Decimal, Context, localcontext + + # Displays with default precision of 28 digits + v = Decimal('578') + print v.sqrt() + + with localcontext(Context(prec=16)): + # All code in this block uses a precision of 16 digits. + # The original context is restored on exiting the block. + print v.sqrt() + + +.. _new-26-context-managers: + +Writing Context Managers +------------------------ + +Under the hood, the ':keyword:`with`' statement is fairly complicated. Most +people will only use ':keyword:`with`' in company with existing objects and +don't need to know these details, so you can skip the rest of this section if +you like. Authors of new objects will need to understand the details of the +underlying implementation and should keep reading. + +A high-level explanation of the context management protocol is: + +* The expression is evaluated and should result in an object called a "context + manager". The context manager must have :meth:`__enter__` and :meth:`__exit__` + methods. + +* The context manager's :meth:`__enter__` method is called. The value returned + is assigned to *VAR*. If no ``'as VAR'`` clause is present, the value is simply + discarded. + +* The code in *BLOCK* is executed. + +* If *BLOCK* raises an exception, the :meth:`__exit__(type, value, traceback)` + is called with the exception details, the same values returned by + :func:`sys.exc_info`. The method's return value controls whether the exception + is re-raised: any false value re-raises the exception, and ``True`` will result + in suppressing it. You'll only rarely want to suppress the exception, because + if you do the author of the code containing the ':keyword:`with`' statement will + never realize anything went wrong. + +* If *BLOCK* didn't raise an exception, the :meth:`__exit__` method is still + called, but *type*, *value*, and *traceback* are all ``None``. + +Let's think through an example. I won't present detailed code but will only +sketch the methods necessary for a database that supports transactions. + +(For people unfamiliar with database terminology: a set of changes to the +database are grouped into a transaction. Transactions can be either committed, +meaning that all the changes are written into the database, or rolled back, +meaning that the changes are all discarded and the database is unchanged. See +any database textbook for more information.) + +Let's assume there's an object representing a database connection. Our goal will +be to let the user write code like this:: + + db_connection = DatabaseConnection() + with db_connection as cursor: + cursor.execute('insert into ...') + cursor.execute('delete from ...') + # ... more operations ... + +The transaction should be committed if the code in the block runs flawlessly or +rolled back if there's an exception. Here's the basic interface for +:class:`DatabaseConnection` that I'll assume:: + + class DatabaseConnection: + # Database interface + def cursor (self): + "Returns a cursor object and starts a new transaction" + def commit (self): + "Commits current transaction" + def rollback (self): + "Rolls back current transaction" + +The :meth:`__enter__` method is pretty easy, having only to start a new +transaction. For this application the resulting cursor object would be a useful +result, so the method will return it. The user can then add ``as cursor`` to +their ':keyword:`with`' statement to bind the cursor to a variable name. :: + + class DatabaseConnection: + ... + def __enter__ (self): + # Code to start a new transaction + cursor = self.cursor() + return cursor + +The :meth:`__exit__` method is the most complicated because it's where most of +the work has to be done. The method has to check if an exception occurred. If +there was no exception, the transaction is committed. The transaction is rolled +back if there was an exception. + +In the code below, execution will just fall off the end of the function, +returning the default value of ``None``. ``None`` is false, so the exception +will be re-raised automatically. If you wished, you could be more explicit and +add a :keyword:`return` statement at the marked location. :: + + class DatabaseConnection: + ... + def __exit__ (self, type, value, tb): + if tb is None: + # No exception, so commit + self.commit() + else: + # Exception occurred, so rollback. + self.rollback() + # return False + + +.. _module-contextlib: + +The contextlib module +--------------------- + +The new :mod:`contextlib` module provides some functions and a decorator that +are useful for writing objects for use with the ':keyword:`with`' statement. + +The decorator is called :func:`contextmanager`, and lets you write a single +generator function instead of defining a new class. The generator should yield +exactly one value. The code up to the :keyword:`yield` will be executed as the +:meth:`__enter__` method, and the value yielded will be the method's return +value that will get bound to the variable in the ':keyword:`with`' statement's +:keyword:`as` clause, if any. The code after the :keyword:`yield` will be +executed in the :meth:`__exit__` method. Any exception raised in the block will +be raised by the :keyword:`yield` statement. + +Our database example from the previous section could be written using this +decorator as:: + + from contextlib import contextmanager + + @contextmanager + def db_transaction (connection): + cursor = connection.cursor() + try: + yield cursor + except: + connection.rollback() + raise + else: + connection.commit() + + db = DatabaseConnection() + with db_transaction(db) as cursor: + ... + +The :mod:`contextlib` module also has a :func:`nested(mgr1, mgr2, ...)` function +that combines a number of context managers so you don't need to write nested +':keyword:`with`' statements. In this example, the single ':keyword:`with`' +statement both starts a database transaction and acquires a thread lock:: + + lock = threading.Lock() + with nested (db_transaction(db), lock) as (cursor, locked): + ... + +Finally, the :func:`closing(object)` function returns *object* so that it can be +bound to a variable, and calls ``object.close`` at the end of the block. :: + + import urllib, sys + from contextlib import closing + + with closing(urllib.urlopen('http://www.yahoo.com')) as f: + for line in f: + sys.stdout.write(line) + + +.. seealso:: + + :pep:`343` - The "with" statement + PEP written by Guido van Rossum and Nick Coghlan; implemented by Mike Bland, + Guido van Rossum, and Neal Norwitz. The PEP shows the code generated for a + ':keyword:`with`' statement, which can be helpful in learning how the statement + works. + + The documentation for the :mod:`contextlib` module. + +.. % ====================================================================== + +.. _pep-3110: + +PEP 3110: Exception-Handling Changes +===================================================== + +One error that Python programmers occasionally make +is the following:: + + try: + ... + except TypeError, ValueError: + ... + +The author is probably trying to catch both +:exc:`TypeError` and :exc:`ValueError` exceptions, but this code +actually does something different: it will catch +:exc:`TypeError` and bind the resulting exception object +to the local name ``"ValueError"``. The correct code +would have specified a tuple:: + + try: + ... + except (TypeError, ValueError): + ... + +This error is possible because the use of the comma here is ambiguous: +does it indicate two different nodes in the parse tree, or a single +node that's a tuple. + +Python 3.0 changes the syntax to make this unambiguous by replacing +the comma with the word "as". To catch an exception and store the +exception object in the variable ``exc``, you must write:: + + try: + ... + except TypeError as exc: + ... + +Python 3.0 will only support the use of "as", and therefore interprets +the first example as catching two different exceptions. Python 2.6 +supports both the comma and "as", so existing code will continue to +work. + +.. seealso:: + + :pep:`3110` - Catching Exceptions in Python 3000 + PEP written and implemented by Collin Winter. + +.. % ====================================================================== + +.. _pep-3119: + +PEP 3119: Abstract Base Classes +===================================================== + +XXX + +.. seealso:: + + :pep:`3119` - Introducing Abstract Base Classes + PEP written by Guido van Rossum and Talin. + Implemented by XXX. + Backported to 2.6 by Benjamin Aranguren (with Alex Martelli). Other Language Changes ====================== Here are all of the changes that Python 2.6 makes to the core Python language. +* Changes to the :class:`Exception` interface + as dictated by :pep:`352` continue to be made. For 2.6, + the :attr:`message` attribute is being deprecated in favor of the + :attr:`args` attribute. + +* When calling a function using the ``**`` syntax to provide keyword + arguments, you are no longer required to use a Python dictionary; + any mapping will now work:: + + >>> def f(**kw): + ... print sorted(kw) + ... + >>> ud=UserDict.UserDict() + >>> ud['a'] = 1 + >>> ud['b'] = 'string' + >>> f(**ud) + ['a', 'b'] + + .. % Patch 1686487 + +* The :func:`compile` built-in function now accepts keyword arguments + as well as positional parameters. (Contributed by XXX.) + + .. % Patch 1444529 + * The :func:`complex` constructor now accepts strings containing parenthesized complex numbers, letting ``complex(repr(cmplx))`` will now round-trip values. For example, ``complex('(3+4j)')`` @@ -87,6 +445,15 @@ Here are all of the changes that Python 2.6 makes to the core Python language. .. % Patch 1193128 +* The built-in :func:`dir` function now checks for a :meth:`__dir__` + method on the objects it receives. This method must return a list + of strings containing the names of valid attributes for the object, + and lets the object control the value that :func:`dir` produces. + Objects that have :meth:`__getattr__` or :meth:`__getattribute__` + methods. + + .. % Patch 1591665 + * An obscure change: when you use the the :func:`locals` function inside a :keyword:`class` statement, the resulting dictionary no longer returns free variables. (Free variables, in this case, are variables referred to in the @@ -160,6 +527,11 @@ complete list of changes, or look through the CVS logs for all the details. (Contributed by Raymond Hettinger.) +* An optional ``timeout`` parameter was added to the + :class:`httplib.HTTPConnection` and :class:`HTTPSConnection` + class constructors, specifying a timeout measured in seconds. + (Added by Facundo Batista.) + * A new function in the :mod:`itertools` module: ``izip_longest(iter1, iter2, ...[, fillvalue])`` makes tuples from each of the elements; if some of the iterables are shorter than others, the missing values are set to *fillvalue*. @@ -176,6 +548,15 @@ complete list of changes, or look through the CVS logs for all the details. .. % Patch #1490190 +* The :func:`os.walk` function now has a "followlinks" parameter. If + set to True, it will follow symlinks pointing to directories and + visit the directory's contents. For backward compatibility, the + parameter's default value is false. Note that the function can fall + into an infinite recursion if there's a symlink that points to a + parent directory. + + .. % Patch 1273829 + * In the :mod:`os.path` module, the :func:`splitext` function has been changed to not split on leading period characters. This produces better results when operating on Unix's dot-files. @@ -191,6 +572,12 @@ complete list of changes, or look through the CVS logs for all the details. .. % Patch 1339796 + On Windows, :func:`os.path.expandvars` will now expand environment variables + in the form "%var%", and "~user" will be expanded into the + user's home directory path. (Contributed by XXX.) + + .. % Patch 957650 + * New functions in the :mod:`posix` module: :func:`chflags` and :func:`lchflags` are wrappers for the corresponding system calls (where they're available). Constants for the flag values are defined in the :mod:`stat` module; some @@ -217,13 +604,78 @@ complete list of changes, or look through the CVS logs for all the details. .. % Patch #957003 +* The :mod:`tarfile` module now supports POSIX.1-2001 (pax) and + POSIX.1-1988 (ustar) format tarfiles, in addition to the GNU tar + format that was already supported. The default format + is GNU tar; specify the ``format`` parameter to open a file + using a different format:: + + tar = tarfile.open("output.tar", "w", format=tarfile.PAX_FORMAT) + + The new ``errors`` parameter lets you specify an error handling + scheme for character conversions: the three standard ways Python can + handle errors ``'strict'``, ``'ignore'``, ``'replace'`` , or the + special value ``'utf-8'``, which replaces bad characters with their + UTF-8 representation. Character conversions occur because the PAX + format supports Unicode filenames, defaulting to UTF-8 encoding. + + The :meth:`TarFile.add` method now accepts a ``exclude`` argument that's + a function that can be used to exclude certain filenames from + an archive. + The function must take a filename and return true if the file + should be excluded or false if it should be archived. + The function is applied to both the name initially passed to :meth:`add` + and to the names of files in recursively-added directories. + + (All changes contributed by Lars Gustäbel). + * An optional ``timeout`` parameter was added to the :class:`telnetlib.Telnet` class constructor, specifying a timeout measured in seconds. (Added by Facundo Batista.) -* The :mod:`test.test_support` module now contains a :func:`EnvironmentVarGuard` +* The :class:`tempfile.NamedTemporaryFile` class usually deletes + the temporary file it created when the file is closed. This + behaviour can now be changed by passing ``delete=False`` to the + constructor. (Contributed by Damien Miller.) + + .. % Patch #1537850 + +* The :mod:`test.test_support` module now contains a + :func:`EnvironmentVarGuard` context manager that supports temporarily changing environment variables and - automatically restores them to their old values. (Contributed by Brett Cannon.) + automatically restores them to their old values. + + Another context manager, :class:`TransientResource`, can surround calls + to resources that may or may not be available; it will catch and + ignore a specified list of exceptions. For example, + a network test may ignore certain failures when connecting to an + external web site:: + + with test_support.TransientResource(IOError, errno=errno.ETIMEDOUT): + f = urllib.urlopen('https://sf.net') + ... + + (Contributed by Brett Cannon.) + +* The :mod:`textwrap` module can now preserve existing whitespace + at the beginnings and ends of the newly-created lines + by specifying ``drop_whitespace=False`` + as an argument:: + + >>> S = """This sentence has a bunch of extra whitespace.""" + >>> print textwrap.fill(S, width=15) + This sentence + has a bunch + of extra + whitespace. + >>> print textwrap.fill(S, drop_whitespace=False, width=15) + This sentence + has a bunch + of extra + whitespace. + >>> + + .. % Patch #1581073 * The :mod:`timeit` module now accepts callables as well as strings for the statement being timed and for the setup code. @@ -235,6 +687,20 @@ complete list of changes, or look through the CVS logs for all the details. .. % Patch #1533909 +* An optional ``timeout`` parameter was added to the + :func:`urllib.urlopen` function and the + :class:`urllib.ftpwrapper` class constructor, as well as the + :func:`urllib2.urlopen` function. The parameter specifies a timeout + measured in seconds. For example:: + + >>> u = urllib2.urlopen("http://slow.example.com", timeout=3) + Traceback (most recent call last): + ... + urllib2.URLError: <urlopen error timed out> + >>> + + (Added by Facundo Batista.) + .. % ====================================================================== .. % whole new modules get described in \subsections here |