Forum: Ruby rounding errors?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
8903a75dedc9892d0702dfe027c95011?d=identicon&s=25 Pete Hodgson (moredip)
on 2009-06-06 04:07
:~$ ruby -e "p ((0.29)*100).to_i"
28

Really?! I know that digital storage of floating point numbers can lead
to things like this, but this one seems a little extreme to me.

:~$ ruby -v
ruby 1.8.7 (2008-08-11 patchlevel 72) [i486-linux]

Cheers,
Pete
9e2504e0b74e5384af09ce8a660afac4?d=identicon&s=25 Pascal J. Bourguignon (Guest)
on 2009-06-06 04:15
(Received via mailing list)
Pete Hodgson <ruby-forum@thepete.net> writes:

> :~$ ruby -e "p ((0.29)*100).to_i"
> 28
>
> Really?! I know that digital storage of floating point numbers can lead
> to things like this, but this one seems a little extreme to me.

You may try a true Lisp instead of a Matzacred Lisp:

$ clisp -q -norc -x '(values (truncate (* 0.29 100)))'
29
E7559e558ececa67c40f452483b9ac8c?d=identicon&s=25 Gary Wright (Guest)
on 2009-06-06 04:21
(Received via mailing list)
On Jun 5, 2009, at 10:07 PM, Pete Hodgson wrote:

> :~$ ruby -e "p ((0.29)*100).to_i"
> 28
>
> Really?! I know that digital storage of floating point numbers can
> lead
> to things like this, but this one seems a little extreme to me.


 >> sprintf "%0.30f", 0.29
=> "0.289999999999999980015985556747"
 >> sprintf "%0.30f", 0.29*100
=> "28.999999999999996447286321199499"
 >> sprintf "%0.30f", (0.29*100).to_i
=> "28.000000000000000000000000000000"

If you want to really understand what is going on:
<http://docs.sun.com/source/806-3568/ncg_goldberg.html>

Gary Wright
47b1910084592eb77a032bc7d8d1a84e?d=identicon&s=25 Joel VanderWerf (Guest)
on 2009-06-06 05:09
(Received via mailing list)
Pascal J. Bourguignon wrote:
> $ clisp -q -norc -x '(values (truncate (* 0.29 100)))'
> 29
>

There's nothing strange here, just IEEE floats, which don't represent
every decimal float:

irb(main):001:0> (0.29)*100
=> 29.0
irb(main):002:0> ((0.29)*100).to_i
=> 28
irb(main):003:0> ((0.29)*100).round
=> 29
irb(main):004:0> ((0.29)*100).floor
=> 28
irb(main):006:0> "%20.18f" % ((0.29)*100)
=> "28.999999999999996447"
irb(main):007:0> "%20.18f" % 0.29
=> "0.289999999999999980"
Ef0db53920b243d6758c2f6b1306df0d?d=identicon&s=25 Steve Ross (cwd)
on 2009-06-06 05:47
(Received via mailing list)
On Jun 5, 2009, at 7:15 PM, Pascal J. Bourguignon wrote:

>
> $ clisp -q -norc -x '(values (truncate (* 0.29 100)))'
> 29
>
> --
> __Pascal Bourguignon__

The result of the calculation is undefined in pretty much any
language. The loose definition of the expression is "return a value
100 times a value approximating 29 and truncated to an integer using
the algorithm of your choice." So, you claim the *only* right answer
must be the one yielded by Lisp. However, the correct answer is: The
result of evaluating this expression is implementation-specific.

In C:

#include <stdio.h>

int main(void) {
   double d = 0.29;
   float  f = 0.29;

   printf("double truncated to integer is: %i\n", (int)(d * 100));
   printf("float truncated to integer is: %i\n", (int)(f * 100));

   return 0;
}

Would ANSI[1] C results be correct enough to be used as a gold
standard? Well, if so, the result of the program above is:

$ ./foo
double truncated to integer is: 28
float truncated to integer is: 29

The moral of the story is not "use Lisp". The moral is use floating
point numbers with great care. Oh, geez, I totally forgot, this is a
Ruby list, not a C list.

[1] gcc version 4.0.1 (Apple Inc. build 5488)
9e2504e0b74e5384af09ce8a660afac4?d=identicon&s=25 Pascal J. Bourguignon (Guest)
on 2009-06-06 10:03
(Received via mailing list)
"s.ross" <cwdinfo@gmail.com> writes:

>>
> 100 times a value approximating 29 and truncated to an integer using
> the algorithm of your choice." So, you claim the *only* right answer
> must be the one yielded by Lisp. However, the correct answer is: The
> result of evaluating this expression is implementation-specific.


My point being that there are some implementations that are more
useful and helpful than others.
E0d864d9677f3c1482a20152b7cac0e2?d=identicon&s=25 Robert Klemme (Guest)
on 2009-06-06 13:25
(Received via mailing list)
On 06.06.2009 05:46, s.ross wrote:
> The result of the calculation is undefined in pretty much any
> language. The loose definition of the expression is "return a value
> 100 times a value approximating 29 and truncated to an integer using
> the algorithm of your choice." So, you claim the *only* right answer
> must be the one yielded by Lisp.

Agreed.  And, btw, with BigDecimal even Ruby comes up with more
"natural" results:

require 'bigdecimal'
x = BigDecimal.new "0.29"
x *= 100
puts x
printf "%f %10.8f\n", x, x

->

0.29E2
29.000000 29.00000000

> However, the correct answer is: The
> result of evaluating this expression is implementation-specific.

Actually, this is only half true: if the implementation is IEEE-754
standards conform then it must yield certain results - regardless of
implementation.  I believe nowadays most if not all math libraries
conform to the standard so you should expect to see the same results
from all implementations.  I would assume that differences are mostly
due to numeric data type (see your C example) and conversion for
printing (e.g. how many places are output by default).

http://en.wikipedia.org/wiki/IEEE_754
http://grouper.ieee.org/groups/754/

> The moral of the story is not "use Lisp". The moral is use floating
> point numbers with great care.

Exactly.

Kind regards

  robert
This topic is locked and can not be replied to.