# What's wrong with these calculations?

**URL:** <https://www.ruby-forum.com/t/whats-wrong-with-these-calculations/122190>\
**Category:** Ruby\
**Created:** [December 5, 2007, 12:33pm UTC](https://www.ruby-forum.com/t/whats-wrong-with-these-calculations/122190 "2007-12-05T12:33:10Z")\
**Posts on this page:** 9\
**Page:** 1

<div class="post-metadata">

**Author:** ![Ivo\_D](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/d78d45/32.png) [@Ivo\_D](https://www.ruby-forum.com/u/Ivo_D)\
**Post date:** [December 5, 2007, 12:33pm UTC](https://www.ruby-forum.com/t/whats-wrong-with-these-calculations/122190/1 "2007-12-05T12:33:10Z")

</div>

Hi

I thought, why search for two hours being totally frustrated if I can  
just ask…

why does this happen:

> > 73.07-63.00  
> > =\> 10.07 # ok, normal!
> 
> > 73.07-64.00  
> > =\> 9.06999999999999 # why oh why?

What should I do about it?

Thanks  
-c

---

<div class="post-metadata">

**Author:** ![Ivo\_D](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/d78d45/32.png) [@Ivo\_D](https://www.ruby-forum.com/u/Ivo_D)\
**Post date:** [December 5, 2007, 12:49pm UTC](https://www.ruby-forum.com/t/whats-wrong-with-these-calculations/122190/2 "2007-12-05T12:49:40Z")

</div>

Ok

I found this thread which explains this problem:  
[http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/9654](http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-core/9654)

So, now I understand why this happens, but I still would like to get  
the proper result. How can I achieve that?

Thanks  
-c

Op 5-dec-07, om 12:30 heeft Ivo D. het volgende geschreven:

---

<div class="post-metadata">

**Author:** ![Ivo\_D](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/d78d45/32.png) [@Ivo\_D](https://www.ruby-forum.com/u/Ivo_D)\
**Post date:** [December 5, 2007, 12:52pm UTC](https://www.ruby-forum.com/t/whats-wrong-with-these-calculations/122190/3 "2007-12-05T12:52:36Z")

</div>

On Dec 5, 2007 5:00 PM, Ivo D. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> =\> 9.06999999999999 # why oh why?  
> Thats folly of floats, you can’t do much apart from :

sprintf(“%.2f”,(73.07-64.00)).to\_f

–  
Let them talk of their oriental summer climes of everlasting  
conservatories; give me the privilege of making my own summer with my  
own coals.

[http://gnufied.org](http://gnufied.org)

---

<div class="post-metadata">

**Author:** ![Ivo\_D](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/d78d45/32.png) [@Ivo\_D](https://www.ruby-forum.com/u/Ivo_D)\
**Post date:** [December 5, 2007, 1:36pm UTC](https://www.ruby-forum.com/t/whats-wrong-with-these-calculations/122190/4 "2007-12-05T13:36:03Z")

</div>

On Dec 5, 2007 6:54 AM, hemant [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> > > =\> 10.07 # ok, normal!  
> > > class Float  
> > > def r2p places  
> > > sprintf(“%.#{places}f”,self).to\_f  
> > > end  
> > > end

Also look into using the BigDecimal class. It does special processing to  
keep floating point precision.

Jason

---

<div class="post-metadata">

**Author:** ![Ivo\_D](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/d78d45/32.png) [@Ivo\_D](https://www.ruby-forum.com/u/Ivo_D)\
**Post date:** [December 5, 2007, 3:18pm UTC](https://www.ruby-forum.com/t/whats-wrong-with-these-calculations/122190/5 "2007-12-05T15:18:21Z")

</div>

On Dec 5, 4:54 am, hemant [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> > > why does this happen:  
> > > generally, I add a method to Float class like this:
> 
> class Float  
> def r2p places  
> sprintf(“%.#{places}f”,self).to\_f  
> end  
> end

Note that, due to the nature of floats, calling to\_f on that string  
brings you back into the problem again:

x = ( 73.07 - 64.00 )  
y = x.r2p( 2 )  
p x, y, “%.20f” % y

#=\> 9.06999999999999  
#=\> 9.07  
#=\> “9.07000000000000028422”

---

<div class="post-metadata">

**Author:** ![Ivo\_D](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/d78d45/32.png) [@Ivo\_D](https://www.ruby-forum.com/u/Ivo_D)\
**Post date:** [December 5, 2007, 12:54pm UTC](https://www.ruby-forum.com/t/whats-wrong-with-these-calculations/122190/6 "2007-12-05T12:54:56Z")

</div>

On Dec 5, 2007 5:22 PM, hemant [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> > > > 73.07-64.00  
> > > > =\> 9.06999999999999 # why oh why?
> 
> Thats folly of floats, you can’t do much apart from :
> 
> sprintf(“%.2f”,(73.07-64.00)).to\_f

generally, I add a method to Float class like this:

class Float  
def r2p places  
sprintf(“%.#{places}f”,self).to\_f  
end  
end

---

<div class="post-metadata">

**Author:** ![Ivo\_D](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/d78d45/32.png) [@Ivo\_D](https://www.ruby-forum.com/u/Ivo_D)\
**Post date:** [December 5, 2007, 4:30pm UTC](https://www.ruby-forum.com/t/whats-wrong-with-these-calculations/122190/7 "2007-12-05T16:30:41Z")

</div>

Thanks to everyone helping to find me/us a nice solution.

I’m using a method to ‘round’ the float now.

btw, I noticed the same ‘problem’ in mysql, so just because someone  
may be searching here, the page handling that info is here:  
[http://dev.mysql.com/doc/refman/5.0/en/problems-with-float.html](http://dev.mysql.com/doc/refman/5.0/en/problems-with-float.html)

-c

---

<div class="post-metadata">

**Author:** ![Ivo\_D](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/d78d45/32.png) [@Ivo\_D](https://www.ruby-forum.com/u/Ivo_D)\
**Post date:** [December 5, 2007, 4:00pm UTC](https://www.ruby-forum.com/t/whats-wrong-with-these-calculations/122190/8 "2007-12-05T16:00:39Z")

</div>

On Dec 5, 7:31 am, Jason R. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> > wrote:
> > 
> > > > > > > 73.07-64.00  
> > > > > > > sprintf(“%.#{places}f”,self).to\_f  
> > > > > > > #=\> 9.06999999999999  
> > > > > > > #=\> 9.07  
> > > > > > > #=\> “9.07000000000000028422”
> 
> require ‘bigdecimal’  
> require ‘bigdecimal/math’  
> include BigMath
> 
> (BigDecimal(“73.07”) - BigDecimal(“64.00”)).to\_f #=\> 9.07
> 
> [http://www.ruby-doc.org/stdlib/libdoc/bigdecimal/rdoc/index.html](http://www.ruby-doc.org/stdlib/libdoc/bigdecimal/rdoc/index.html)

irb(main):002:0\> require ‘bigdecimal’  
irb(main):003:0\> require ‘bigdecimal/math’  
irb(main):004:0\> include BigMath  
irb(main):005:0\> y = (BigDecimal(“73.07”) - BigDecimal(“64.00”)).to\_f  
=\> 9.07  
irb(main):006:0\> “%.20f” % y  
=\> “9.07000000000000028422”

You might not see the problem on that particular output, but the  
potential inaccuracy still exists as long as you’re in the Float  
domain.

---

<div class="post-metadata">

**Author:** ![Ivo\_D](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/d78d45/32.png) [@Ivo\_D](https://www.ruby-forum.com/u/Ivo_D)\
**Post date:** [December 5, 2007, 3:32pm UTC](https://www.ruby-forum.com/t/whats-wrong-with-these-calculations/122190/9 "2007-12-05T15:32:30Z")

</div>

On Dec 5, 2007 9:15 AM, Phrogz [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> can
> 
> > 
> 
> Note that, due to the nature of floats, calling to\_f on that string

require ‘bigdecimal’  
require ‘bigdecimal/math’  
include BigMath

(BigDecimal(“73.07”) - BigDecimal(“64.00”)).to\_f #=\> 9.07

[http://www.ruby-doc.org/stdlib/libdoc/bigdecimal/rdoc/index.html](http://www.ruby-doc.org/stdlib/libdoc/bigdecimal/rdoc/index.html)

Jason