What's wrong with these calculations?

Ruby
1

Hi

I thought, why search for two hours being totally frustrated if I can
just ask…

why does this happen:

73.07-63.00
=> 10.07 # ok, normal!

73.07-64.00
=> 9.06999999999999 # why oh why?

What should I do about it?

Thanks
-c

2

Ok

I found this thread which explains this problem:
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/9654

So, now I understand why this happens, but I still would like to get
the proper result. How can I achieve that?

Thanks
-c

Op 5-dec-07, om 12:30 heeft Ivo D. het volgende geschreven:

3

On Dec 5, 2007 5:00 PM, Ivo D. [email protected] wrote:

=> 9.06999999999999 # why oh why?
Thats folly of floats, you can’t do much apart from :

sprintf(“%.2f”,(73.07-64.00)).to_f

–
Let them talk of their oriental summer climes of everlasting
conservatories; give me the privilege of making my own summer with my
own coals.

http://gnufied.org

4

On Dec 5, 2007 6:54 AM, hemant [email protected] wrote:

=> 10.07 # ok, normal!
class Float
def r2p places
sprintf(“%.#{places}f”,self).to_f
end
end

Also look into using the BigDecimal class. It does special processing to
keep floating point precision.

Jason

5

On Dec 5, 4:54 am, hemant [email protected] wrote:

why does this happen:
generally, I add a method to Float class like this:

class Float
def r2p places
sprintf(“%.#{places}f”,self).to_f
end
end

Note that, due to the nature of floats, calling to_f on that string
brings you back into the problem again:

x = ( 73.07 - 64.00 )
y = x.r2p( 2 )
p x, y, “%.20f” % y

#=> 9.06999999999999
#=> 9.07
#=> “9.07000000000000028422”

6

On Dec 5, 2007 5:22 PM, hemant [email protected] wrote:

73.07-64.00
=> 9.06999999999999 # why oh why?

Thats folly of floats, you can’t do much apart from :

sprintf(“%.2f”,(73.07-64.00)).to_f

generally, I add a method to Float class like this:

class Float
def r2p places
sprintf(“%.#{places}f”,self).to_f
end
end

7

Thanks to everyone helping to find me/us a nice solution.

I’m using a method to ‘round’ the float now.

btw, I noticed the same ‘problem’ in mysql, so just because someone
may be searching here, the page handling that info is here:
http://dev.mysql.com/doc/refman/5.0/en/problems-with-float.html

-c

8

On Dec 5, 7:31 am, Jason R. [email protected] wrote:

wrote:

73.07-64.00
sprintf(“%.#{places}f”,self).to_f
#=> 9.06999999999999
#=> 9.07
#=> “9.07000000000000028422”

require ‘bigdecimal’
require ‘bigdecimal/math’
include BigMath

(BigDecimal(“73.07”) - BigDecimal(“64.00”)).to_f #=> 9.07

http://www.ruby-doc.org/stdlib/libdoc/bigdecimal/rdoc/index.html

irb(main):002:0> require ‘bigdecimal’
irb(main):003:0> require ‘bigdecimal/math’
irb(main):004:0> include BigMath
irb(main):005:0> y = (BigDecimal(“73.07”) - BigDecimal(“64.00”)).to_f
=> 9.07
irb(main):006:0> “%.20f” % y
=> “9.07000000000000028422”

You might not see the problem on that particular output, but the
potential inaccuracy still exists as long as you’re in the Float
domain.

9

On Dec 5, 2007 9:15 AM, Phrogz [email protected] wrote:

can

Note that, due to the nature of floats, calling to_f on that string

require ‘bigdecimal’
require ‘bigdecimal/math’
include BigMath

(BigDecimal(“73.07”) - BigDecimal(“64.00”)).to_f #=> 9.07

http://www.ruby-doc.org/stdlib/libdoc/bigdecimal/rdoc/index.html

Jason