diff options
author | Georg Brandl <georg@python.org> | 2008-08-10 12:16:45 (GMT) |
---|---|---|
committer | Georg Brandl <georg@python.org> | 2008-08-10 12:16:45 (GMT) |
commit | bae1b94d9e0a1d1e5264ce08be236efbfb649d60 (patch) | |
tree | 637c8fd2d23a1771795dfe516050adaa74350f76 /Doc/tutorial | |
parent | 3d1c7dec4af44134390addc25b002dfd22c4c3f8 (diff) | |
download | cpython-bae1b94d9e0a1d1e5264ce08be236efbfb649d60.zip cpython-bae1b94d9e0a1d1e5264ce08be236efbfb649d60.tar.gz cpython-bae1b94d9e0a1d1e5264ce08be236efbfb649d60.tar.bz2 |
Remove long integer output.
Diffstat (limited to 'Doc/tutorial')
-rw-r--r-- | Doc/tutorial/floatingpoint.rst | 12 |
1 files changed, 6 insertions, 6 deletions
diff --git a/Doc/tutorial/floatingpoint.rst b/Doc/tutorial/floatingpoint.rst index 150e8fb..c9408c9 100644 --- a/Doc/tutorial/floatingpoint.rst +++ b/Doc/tutorial/floatingpoint.rst @@ -173,24 +173,24 @@ and recalling that *J* has exactly 53 bits (is ``>= 2**52`` but ``< 2**53``), the best value for *N* is 56:: >>> 2**52 - 4503599627370496L + 4503599627370496 >>> 2**53 - 9007199254740992L + 9007199254740992 >>> 2**56/10 - 7205759403792793L + 7205759403792794.0 That is, 56 is the only value for *N* that leaves *J* with exactly 53 bits. The best possible value for *J* is then that quotient rounded:: >>> q, r = divmod(2**56, 10) >>> r - 6L + 6 Since the remainder is more than half of 10, the best approximation is obtained by rounding up:: >>> q+1 - 7205759403792794L + 7205759403792794 Therefore the best possible approximation to 1/10 in 754 double precision is that over 2\*\*56, or :: @@ -211,7 +211,7 @@ If we multiply that fraction by 10\*\*30, we can see the (truncated) value of its 30 most significant decimal digits:: >>> 7205759403792794 * 10**30 / 2**56 - 100000000000000005551115123125L + 100000000000000005551115123125 meaning that the exact number stored in the computer is approximately equal to the decimal value 0.100000000000000005551115123125. Rounding that to 17 |