summaryrefslogtreecommitdiffstats
path: root/Doc/tutorial/floatingpoint.rst
diff options
context:
space:
mode:
authorRaymond Hettinger <python@rcn.com>2009-06-29 01:50:51 (GMT)
committerRaymond Hettinger <python@rcn.com>2009-06-29 01:50:51 (GMT)
commitb8aff6a6aa6df5a53d97caf0777a7fedd0dab6a0 (patch)
tree650bb714427ec396346dbfc35b1fd8ce9eb5dcb2 /Doc/tutorial/floatingpoint.rst
parent5afe30e07cf27abc032ed0bb73aa7fe429605beb (diff)
downloadcpython-b8aff6a6aa6df5a53d97caf0777a7fedd0dab6a0.zip
cpython-b8aff6a6aa6df5a53d97caf0777a7fedd0dab6a0.tar.gz
cpython-b8aff6a6aa6df5a53d97caf0777a7fedd0dab6a0.tar.bz2
Clean-up floating point tutorial
Diffstat (limited to 'Doc/tutorial/floatingpoint.rst')
-rw-r--r--Doc/tutorial/floatingpoint.rst51
1 files changed, 28 insertions, 23 deletions
diff --git a/Doc/tutorial/floatingpoint.rst b/Doc/tutorial/floatingpoint.rst
index a8a4202..e877a99 100644
--- a/Doc/tutorial/floatingpoint.rst
+++ b/Doc/tutorial/floatingpoint.rst
@@ -50,7 +50,7 @@ decimal value 0.1 cannot be represented exactly as a base 2 fraction. In base
Stop at any finite number of bits, and you get an approximation. On most
machines today, floats are approximated using a binary fraction with
-the numerator using the first 53 bits following the most significant bit and
+the numerator using the first 53 bits starting with the most significant bit and
with the denominator as a power of two. In the case of 1/10, the binary fraction
is ``3602879701896397 / 2 ** 55`` which is close to but not exactly
equal to the true value of 1/10.
@@ -230,12 +230,8 @@ as ::
and recalling that *J* has exactly 53 bits (is ``>= 2**52`` but ``< 2**53``),
the best value for *N* is 56::
- >>> 2**52
- 4503599627370496
- >>> 2**53
- 9007199254740992
- >>> 2**56/10
- 7205759403792794.0
+ >>> 2**52 <= 2**56 // 10 < 2**53
+ True
That is, 56 is the only value for *N* that leaves *J* with exactly 53 bits. The
best possible value for *J* is then that quotient rounded::
@@ -250,14 +246,13 @@ by rounding up::
>>> q+1
7205759403792794
-Therefore the best possible approximation to 1/10 in 754 double precision is
-that over 2\*\*56, or ::
+Therefore the best possible approximation to 1/10 in 754 double precision is::
- 7205759403792794 / 72057594037927936
+ 7205759403792794 / 2 ** 56
Dividing both the numerator and denominator by two reduces the fraction to::
- 3602879701896397 / 36028797018963968
+ 3602879701896397 / 2 ** 55
Note that since we rounded up, this is actually a little bit larger than 1/10;
if we had not rounded up, the quotient would have been a little bit smaller than
@@ -269,24 +264,34 @@ above, the best 754 double approximation it can get::
>>> 0.1 * 2 ** 55
3602879701896397.0
-If we multiply that fraction by 10\*\*60, we can see the value of out to
-60 decimal digits::
+If we multiply that fraction by 10\*\*55, we can see the value out to
+55 decimal digits::
- >>> 3602879701896397 * 10 ** 60 // 2 ** 55
+ >>> 3602879701896397 * 10 ** 55 // 2 ** 55
1000000000000000055511151231257827021181583404541015625
-meaning that the exact number stored in the computer is approximately equal to
-the decimal value 0.100000000000000005551115123125. Rounding that to 17
-significant digits gives the 0.10000000000000001 that Python displays (well,
-will display on any 754-conforming platform that does best-possible input and
-output conversions in its C library --- yours may not!).
+meaning that the exact number stored in the computer is equal to
+the decimal value 0.1000000000000000055511151231257827021181583404541015625.
+Instead of displaying the full decimal value, many languages (including
+older versions of Python), round the result to 17 significant digits::
+
+ >>> format(0.1, '.17f')
+ '0.10000000000000001'
The :mod:`fractions` and :mod:`decimal` modules make these calculations
easy::
>>> from decimal import Decimal
>>> from fractions import Fraction
- >>> print(Fraction.from_float(0.1))
- 3602879701896397/36028797018963968
- >>> print(Decimal.from_float(0.1))
- 0.1000000000000000055511151231257827021181583404541015625
+
+ >>> Fraction.from_float(0.1)
+ Fraction(3602879701896397, 36028797018963968)
+
+ >>> (0.1).as_integer_ratio()
+ (3602879701896397, 36028797018963968)
+
+ >>> Decimal.from_float(0.1)
+ Decimal('0.1000000000000000055511151231257827021181583404541015625')
+
+ >>> format(Decimal.from_float(0.1), '.17')
+ '0.10000000000000001'