Truncate trouble

Ruby
1

This

(‘40.12’.to_f*100).to_i

results in this integer :

4011

This is a bit of surprise. I’m using 1.8.4 on windows. Is this a bug?
Is there another floating type that avoids this problem? I;m not
seeing anything in the docs that might have warned me.

Greg

2

On Mar 19, 1:37 pm, “Greg L.” [email protected] wrote:

This

(‘40.12’.to_f*100).to_i

results in this integer :

4011

What I find interesting is that the #inspect result of both f1 and f2
below are the same, when they internally are clearly not.

irb(main):001:0> f1 = ‘40.12’.to_f
=> 40.12
irb(main):002:0> f1100
=> 4012.0
irb(main):003:0> (f1100).to_i
=> 4011
irb(main):004:0> (f1*100).round.to_i
=> 4012

irb(main):005:0> 4012.0.to_i
=> 4012

irb(main):006:0> f2 = ‘40.12000000000001’.to_f
=> 40.12
irb(main):007:0> (f2*100).to_i
=> 4012

3

On Tue, Mar 20, 2007 at 04:40:08AM +0900, Greg L. wrote:

This

(‘40.12’.to_f*100).to_i

results in this integer :

4011

This is a bit of surprise. I’m using 1.8.4 on windows. Is this a bug?

Only standard numeric rounding error. The problem is that decimal 0.1
cannot
be represented exactly in binary floating point - it’s a recurring
fraction,
like 1/3 in decimal - so you get only an approximation.

Is there another floating type that avoids this problem? I;m not
seeing anything in the docs that might have warned me.

irb(main):002:0> require ‘bigdecimal’
=> true
irb(main):003:0> f = BigDecimal.new(‘40.12’)
=> #BigDecimal:b7cb7548,‘0.4012E2’,8(8)
irb(main):004:0> (f*100).to_i
=> 4012

BigDecimal stores decimal numbers as decimal. It’s not exactly well
documented though (I only came across it because Ruby on Rails uses it).

Another solution is to multiply all your numbers by 100 at source, i.e.
works in cents rather than dollars or whatever.

Regards,

Brian.

4

On 19.03.2007 20:37, Greg L. wrote:

seeing anything in the docs that might have warned me.
No, that’s a normal rounding error that comes with floats:

irb(main):001:0> f=‘40.12’.to_f
=> 40.12
irb(main):002:0> “%20.10f” % f
=> " 40.1200000000"
irb(main):003:0> “%40.30f” % f
=> " 40.119999999999997442046151263639"
irb(main):004:0> “%40.30f” % (f*100)
=> " 4011.999999999999545252649113535881"

If you look at the last line it becomes clear why you get 4011 as
result.

Kind regards

robert