Weird issue with converting floats to integer

Any idea why this calculates the integer the way it does?

irb> (“291.15”.to_f * 100.0).to_i
=> 29114

Thanks,
Tom

TomRossi7 wrote in post #970207:

Any idea why this calculates the integer the way it does?

irb> (“291.15”.to_f * 100.0).to_i
=> 29114

Because of the vagaries of floating-point arithmetic:
irb> (“291.15”.to_f * 100.0) < 29115
=> true
irb> printf “%.30f”, (“291.15”.to_f * 100.0)
29114.999999999996362021192908287048

It bears repeating: never, ever use Floats for arithmetic. If you can’t
use fixed-point math, then use BigDecimal.

Thanks,
Tom

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

Or just change the way you calculate to get at the level of accuracy
that you want/need:

irb> (“291.15”.to_f * 1000.0).to_i/10
=> 29115

Jeff

I have currency information that I plan to store as an integer –
ironically to avoid issues like this. The user input will always be
in a decimal form. I’m thinking I will just do this (‘291.15’.to_f *
100).round instead. That should round out any of the weird float
issues.

Please quote when replying.

TomRossi7 wrote in post #970346:

I have currency information that I plan to store as an integer –
ironically to avoid issues like this. The user input will always be
in a decimal form. I’m thinking I will just do this (‘291.15’.to_f *
100).round instead. That should round out any of the weird float
issues.

I think there are better ways of doing that – such as using a DECIMAL
field in the database. Then no conversion is necessary and the data is
stored as fixed-point.

Or, if you’d rather store the data as an integral number of pennies
(though that may have i18n issues – there are a few currencies, such as
the Kuwaiti dinar, that are subdivided into 1000, not 100), then a quick
way would be to normalize the string so that it contains 2 decimal
places (so “123” would become “123.00”), then drop the decimal point.

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

Please quote when replying.

Jeff B.systems wrote in post #970344:

Or just change the way you calculate to get at the level of accuracy
that you want/need:

irb> (“291.15”.to_f * 1000.0).to_i/10
=> 29115

That’s incredibly silly IMHO. Besides, it will not always be the same
for all values, will it? I think BigDecimal is the right answer here.

Jeff

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

TomRossi7 wrote in post #970346:

I have currency information that I plan to store as an integer –
ironically to avoid issues like this. The user input will always be
in a decimal form. I’m thinking I will just do this (‘291.15’.to_f *
100).round instead. That should round out any of the weird float
issues.

As was said before by others in this thread, using the BigDecimal type
really is the most reliable, and simple, solution. Don’t reinvent
the wheel with calculating in pennies and rounding etc. …

In the migration, do this:

class AddAmountToPayment < ActiveRecord::Migration
def self.up
add_column :payments, :amount, :decimal, :precision => 12, :scale =>
2
end

This which will work automatically correct when reading
user input in your forms and when doing internal calculations:

$ rails c
Loading development environment (Rails 3.0.3)
001:0> Payment.columns.select{|c| c.sql_type =~ /decimal/}
=> [#<ActiveRecord::ConnectionAdapters::MysqlColumn:0xb6c1e844
@sql_type=“decimal(12,2)”, @scale=2, @name=“amount”, @limit=12,
@precision=12, @primary=false, @type=:decimal, @default=nil,
@null=true>]

002:0> payment = Payment.new(:amount => “291.15”)
=> #<Payment id: nil, user_id: nil, testing: nil, created_at: nil,
updated_at: nil, amount: #BigDecimal:b6c09318,‘0.29115E3’,8(12)>

003:0> payment.amount
=> #BigDecimal:b6c07e14,‘0.29115E3’,8(12)

004:0> payment.amount.to_s
=> “291.15”

019:0> (payment.amount - BigDecimal(“291”) - BigDecimal(“0.15”)).to_s
=> “0.0”
020:0> ((291.15 - 291) - 0.15).to_s
=> “-2.27318164292001e-14” # Rounding errors with float !!!

HTH,

Peter

On 23 December 2010 03:41, Marnen Laibow-Koser [email protected]
wrote:

29114.999999999996362021192908287048

It bears repeating: never, ever use Floats for arithmetic. If you can’t
use fixed-point math, then use BigDecimal.

I have to disagree here. I would rephrase it as almost never use
Floats The one time when they would be appropriate is if you have a
large amount of arithmetic to perform. The sort of thing that might
occur in complex statistical analysis for example or engineering
calculations of some sort. On most (possibly all) computers floating
point consumes vastly less processor time to perform than BigDecimal.
This is a rare occurence in Rails apps I expect so I agree that for
most people Marnen’s advice would be appropriate.

Colin

On Dec 22, 10:34pm, TomRossi7 [email protected] wrote:

Any idea why this calculates the integer the way it does?

irb> (“291.15”.to_f * 100.0).to_i
=> 29114

As Marnen’s already pointed out, the issue is that the result is a
tiny bit smaller than 29115, so to_i truncates off the entire
fractional part and gives the wrong answer. In general, if you want to
convert a decimal to an integer the way most people would on paper,
use .round.to_i, which will correctly round the float to the nearest
integer.

On the currency issue, it’s typically a good idea to avoid floating-
point throughout the process if you’re trying to get a reliable fixed-
point result. Decimal columns are definitely a good idea for this sort
of thing.

–Matt J.

Colin L. wrote in post #970376:

On 23 December 2010 03:41, Marnen Laibow-Koser [email protected]
wrote:

29114.999999999996362021192908287048

It bears repeating: never, ever use Floats for arithmetic. If you can’t
use fixed-point math, then use BigDecimal.

I have to disagree here.

I knew someone would. :slight_smile:

I would rephrase it as almost never use
Floats The one time when they would be appropriate is if you have a
large amount of arithmetic to perform. The sort of thing that might
occur in complex statistical analysis for example or engineering
calculations of some sort. On most (possibly all) computers floating
point consumes vastly less processor time to perform than BigDecimal.

Then use fixed-point math. IEEE floats are 100% inappropriate for any
sort of mathematical calculation. And who cares how fast they are if
they’re inaccurate? Inaccurate engineering calculations are worse than
none at all – do you want to drive over that bridge? :slight_smile:

If you want to use IEEE floats for arithmetic, then the onus is on you
to do the requisite numerical analysis to figure out how much error you
are introducing, and to inform your users of that fact.

This is a rare occurence in Rails apps

Yes indeed!

I expect so I agree that for
most people Marnen’s advice would be appropriate.

Colin

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

Marnen Laibow-Koser wrote in post #970377:

Then use fixed-point math. IEEE floats are 100% inappropriate for any
sort of mathematical calculation. And who cares how fast they are if
they’re inaccurate? Inaccurate engineering calculations are worse than
none at all – do you want to drive over that bridge? :slight_smile:

Let’s break this down to a comparison of accuracy vs. precision.

Take for example the value 1/3.

First using floating point math:

x = 1.0
=> 1.0
y = 3.0
=> 3.0
z = x / y
=> 0.3333333333333333

Now with Ruby’s BigDecimal math:
x = BigDecimal.new(‘1’)
=> #BigDecimal:102c078,‘0.1E1’,4(8)
y = BigDecimal.new(‘3’)
=> #BigDecimal:10388f0,‘0.3E1’,4(8)
z = x / y
=> #BigDecimal:103680c,‘0.33333333E0’,8(16)

In the case of float we have 16 decimal digits of precision, but in the
case of BigDecimal we have only eight digits of precision.

Yes, in the case of BigDecimal we have a value representing “precisely”
the value 0.33333333, but that result is obviously less accurate than
the result of the floating point calculation.

Over large aggregations this can make a difference. It is certainly the
case that IEEE floats introduce error due to the inherent storage
limitations, but doing precise mathematics as fixed point has it’s own
complications as well.

Ruby’s BigDecimal class does not provide the needed accuracy for complex
mathematics. The difference is that BigDecimal always stores precise
values within the limits of it storage space. Where floats store
imprecise representations, but representations with a fixed precision.

I don’t think either of these classes are appropriate for the sort of
math mentioned above. But, they both have their uses.

Robert W. wrote in post #970390:

Marnen Laibow-Koser wrote in post #970377:

Then use fixed-point math. IEEE floats are 100% inappropriate for any
sort of mathematical calculation. And who cares how fast they are if
they’re inaccurate? Inaccurate engineering calculations are worse than
none at all – do you want to drive over that bridge? :slight_smile:

Let’s break this down to a comparison of accuracy vs. precision.

Take for example the value 1/3.

First using floating point math:

x = 1.0
=> 1.0
y = 3.0
=> 3.0
z = x / y
=> 0.3333333333333333

Now with Ruby’s BigDecimal math:
x = BigDecimal.new(‘1’)
=> #BigDecimal:102c078,‘0.1E1’,4(8)
y = BigDecimal.new(‘3’)
=> #BigDecimal:10388f0,‘0.3E1’,4(8)
z = x / y
=> #BigDecimal:103680c,‘0.33333333E0’,8(16)

In the case of float we have 16 decimal digits of precision, but in the
case of BigDecimal we have only eight digits of precision.

Are you sure? Or are only eight digits displayed?

Yes, in the case of BigDecimal we have a value representing “precisely”
the value 0.33333333, but that result is obviously less accurate than
the result of the floating point calculation.

Not obvious at all, since I don’t know how many 3s the BigDecimal is
actually storing.

Over large aggregations this can make a difference.

Yes, it certainly can.

It is certainly the
case that IEEE floats introduce error due to the inherent storage
limitations, but doing precise mathematics as fixed point has it’s own
complications as well.

Sure. BigDecimal is the best of both worlds.

Ruby’s BigDecimal class does not provide the needed accuracy for complex
mathematics.

Of course it does. Why do you think it does not?

Anyway, if you need more than BigDecimal, then start climbing into the
realms of symbolic math with Rational.

The difference is that BigDecimal always stores precise
values within the limits of it storage space. Where floats store
imprecise representations, but representations with a fixed precision.

I don’t think either of these classes are appropriate for the sort of
math mentioned above.

Why is BigDecimal inappropriate? What would you use instead?

But, they both have their uses.

I’m not sure Float has a proper use case at all. :slight_smile:

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

Sent from my iPhone

On 24 December 2010 03:59, Marnen Laibow-Koser [email protected]
wrote:

x = BigDecimal.new(‘1’)
=> #BigDecimal:102c078,‘0.1E1’,4(8)
y = BigDecimal.new(‘3’)
=> #BigDecimal:10388f0,‘0.3E1’,4(8)
z = x / y
=> #BigDecimal:103680c,‘0.33333333E0’,8(16)

In the case of float we have 16 decimal digits of precision, but in the
case of BigDecimal we have only eight digits of precision.

Are you sure? Or are only eight digits displayed?

$ rails console
Loading development environment (Rails 3.0.3)
ruby-1.9.2-p0 > x=BigDecimal(‘1’)/BigDecimal(‘3’)
=> #BigDecimal:ad823a4,‘0.33333333E0’,8(16)
ruby-1.9.2-p0 > xBigDecimal(‘3’)
=> #BigDecimal:ad2b270,‘0.99999999E0’,8(20)
ruby-1.9.2-p0 > x
BigDecimal(‘3’) == 1
=> false
ruby-1.9.2-p0 > x*BigDecimal(‘3’) - 1
=> #BigDecimal:aa4fdf8,‘-0.1E-7’,4(20)

Colin

On 24 December 2010 03:59, Marnen Laibow-Koser [email protected]
wrote:

[…]
I’m not sure Float has a proper use case at all. :slight_smile:

Consider table containing the masses of objects. Columns description
and mass (kg), with the following records
apple, 0.2
earth, 5.9742E24
sun, 1.98892E30
super massive black hole, 1.0E39

What column type would be appropriate for storing the mass?

Colin

On 23 December 2010 21:34, Marnen Laibow-Koser [email protected]
wrote:

I knew someone would. :slight_smile:
You probably guessed it might be me :slight_smile:

they’re inaccurate? Inaccurate engineering calculations are worse than
none at all – do you want to drive over that bridge? :slight_smile:

We are getting off topic for Rails apps but I think if you looked into
most engineering and scientific software you would find they use
floating point arithmetic. Fixed point is not appropriate as
intermediate calculations need guaranteed relative error, not
absolute error. That is what floats provide.

If you want to use IEEE floats for arithmetic, then the onus is on you
to do the requisite numerical analysis to figure out how much error you
are introducing, and to inform your users of that fact.

Yes of course. As you said, the bridge designer needs to know the
possible errors in his calculations.

Colin

Colin L. wrote in post #970449:

On 24 December 2010 03:59, Marnen Laibow-Koser [email protected]
wrote:

[…]
I’m not sure Float has a proper use case at all. :slight_smile:

Consider table containing the masses of objects. Columns description
and mass (kg), with the following records
apple, 0.2
earth, 5.9742E24
sun, 1.98892E30
super massive black hole, 1.0E39

What column type would be appropriate for storing the mass?

Irrelevant. If I understand you correctly, you’re asking about DB
column types, not Ruby column types.

Anyway, the answer is: it depends. If I didn’t need arbitrary
precision, I’d store them as floats in the DB, and then change to
BigDecimal before doing any math on them.

If I did need arbitrary precision, and the DB didn’t provide a
BigDecimal-like type, I’d store them as strings.

Colin

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

Sent from my iPhone

Colin L. wrote in post #970447:

On 24 December 2010 03:59, Marnen Laibow-Koser [email protected]
wrote:

x = BigDecimal.new(‘1’)
=> #BigDecimal:102c078,‘0.1E1’,4(8)
y = BigDecimal.new(‘3’)
=> #BigDecimal:10388f0,‘0.3E1’,4(8)
z = x / y
=> #BigDecimal:103680c,‘0.33333333E0’,8(16)

In the case of float we have 16 decimal digits of precision, but in the
case of BigDecimal we have only eight digits of precision.

Are you sure? Or are only eight digits displayed?

$ rails console
Loading development environment (Rails 3.0.3)
ruby-1.9.2-p0 > x=BigDecimal(‘1’)/BigDecimal(‘3’)
=> #BigDecimal:ad823a4,‘0.33333333E0’,8(16)
ruby-1.9.2-p0 > xBigDecimal(‘3’)
=> #BigDecimal:ad2b270,‘0.99999999E0’,8(20)
ruby-1.9.2-p0 > x
BigDecimal(‘3’) == 1
=> false
ruby-1.9.2-p0 > x*BigDecimal(‘3’) - 1
=> #BigDecimal:aa4fdf8,‘-0.1E-7’,4(20)

Interesting. However, the BigDecimal constructor allows specification
of the minimum number of significant figures, so in this case we could
have done BigDecimal.new(‘1’, 16).

Colin

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

Sent from my iPhone

Colin L. wrote in post #970496:

On 24 December 2010 14:25, Marnen Laibow-Koser [email protected]
wrote:

=> #BigDecimal:ad2b270,‘0.99999999E0’,8(20)
ruby-1.9.2-p0 > xBigDecimal(‘3’) == 1
=> false
ruby-1.9.2-p0 > x
BigDecimal(‘3’) - 1
=> #BigDecimal:aa4fdf8,‘-0.1E-7’,4(20)

Interesting. However, the BigDecimal constructor allows specification
of the minimum number of significant figures, so in this case we could
have done BigDecimal.new(‘1’, 16).

ruby-1.9.2-p0 > x = BigDecimal.new(‘1’, 16)/BigDecimal.new(‘3’,16)
=> #BigDecimal:a146bbc,‘0.33333333E0’,8(16)
ruby-1.9.2-p0 > x*BigDecimal(‘3’,16) - 1
=> #BigDecimal:a114acc,‘-0.1E-7’,4(20)

It doesn’t make any difference how many digits you have, 1/3 as a
decimal number multiplied by 3 will never be equal to 1.

I’m aware of that, of course. In this particular case, we could use a
Rational to fix that. But that won’t necessarily work with arbitrary
floating-point numbers…

…unless we recast them as Rationals and do all math that way. Hmm.

I am not sure why the above still gives only 8 digits accuracy
however, I would have thought it aught to be 16.

Yeah, that’s peculiar. I’ll have to run some tests.

(I can almost guarantee that Rubinius would get that one right, since I
wrote most of the BigDecimal library at one time. Don’t know if they’re
still using it.)

Colin

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

Sent from my iPhone

On 24 December 2010 14:25, Marnen Laibow-Koser [email protected]
wrote:

=> #BigDecimal:ad2b270,‘0.99999999E0’,8(20)
ruby-1.9.2-p0 > xBigDecimal(‘3’) == 1
=> false
ruby-1.9.2-p0 > x
BigDecimal(‘3’) - 1
=> #BigDecimal:aa4fdf8,‘-0.1E-7’,4(20)

Interesting. However, the BigDecimal constructor allows specification
of the minimum number of significant figures, so in this case we could
have done BigDecimal.new(‘1’, 16).

ruby-1.9.2-p0 > x = BigDecimal.new(‘1’, 16)/BigDecimal.new(‘3’,16)
=> #BigDecimal:a146bbc,‘0.33333333E0’,8(16)
ruby-1.9.2-p0 > x*BigDecimal(‘3’,16) - 1
=> #BigDecimal:a114acc,‘-0.1E-7’,4(20)

It doesn’t make any difference how many digits you have, 1/3 as a
decimal number multiplied by 3 will never be equal to 1.

I am not sure why the above still gives only 8 digits accuracy
however, I would have thought it aught to be 16.

Colin

On Dec 24, 2:21pm, Marnen Laibow-Koser [email protected] wrote:

Colin L. wrote in post #970449:

If I did need arbitrary precision, and the DB didn’t provide a
BigDecimal-like type, I’d store them as strings.

Which still doesn’t allow you to store all numbers with arbitrary
precision (in fact putting my mathematical hat on, most numbers can’t
be stored like this). You’ll have to deal with numerical error
eventually

Fred