hi.

i’m confused by float point calculation.

like this!

irb(main):001:0> 123.6 - 123

=> 0.599999999999994

how to i get result 0.6 ?

hi.

i’m confused by float point calculation.

like this!

irb(main):001:0> 123.6 - 123

=> 0.599999999999994

how to i get result 0.6 ?

Hi,

On Tue, Jul 8, 2008 at 10:39 AM, Hyun chul Park [email protected]

wrote:

how to i get result 0.6 ?

try using BigDecimal, like:

irb(main):001:0> require ‘bigdecimal’

=> true

irb(main):002:0> x = BigDecimal(“123.6”) - BigDecimal(“123”)

=> #BigDecimal:8b13c,‘0.6E0’,4(16)

irb(main):003:0> x.to_f

=> 0.6

zoran

-------- Original-Nachricht --------

Datum: Tue, 8 Jul 2008 17:39:50 +0900

Von: Hyun chul Park [email protected]

An: [email protected]

Betreff: Floating point calculation problem

Hi –

how to i get result 0.6 ?

you may want to take a look at this:

http://docs.sun.com/source/806-3568/ncg_goldberg.html

If you want to do exact calculations, you can use fractions

http://www.ruby-doc.org/stdlib/libdoc/rational/rdoc/classes/Rational.html

To convert between the two, for each real number (Float), you can use

its continued fraction

approximation.

Here’s a link to a Ruby program that calculates digits of Pi from its

continued fraction representation

(and also a Java one, which is much longer):

http://www.cs.utsa.edu/~wagner/pi/pi_cont.html

Best regards,

Axel

Zoran Regvart wrote:

Hi,

On Tue, Jul 8, 2008 at 10:39 AM, Hyun chul Park [email protected]

wrote:how to i get result 0.6 ?

try using BigDecimal, like:irb(main):001:0> require ‘bigdecimal’

=> true

irb(main):002:0> x = BigDecimal(“123.6”) - BigDecimal(“123”)

=> #BigDecimal:8b13c,‘0.6E0’,4(16)

irb(main):003:0> x.to_f

=> 0.6zoran

thank you, you’re reply.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

Sponsor our Newsletter | Privacy Policy | Terms of Service | Remote Ruby Jobs