how come <%= 19043.0 - 19032.6 %>
display 10.4000000000015
how can I avoid this and get a proper result?
Thanks
On 9/26/06, rosoft2001 [email protected] wrote:
how come <%= 19043.0 - 19032.6 %>
It’s a floating point error. See here:
http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems
display 10.4000000000015
how can I avoid this and get a proper result?
Thanks
Use integer arithmetic instead. For example, use cents instead of
dollars by multiplying by 100 and converting to Integer. For other
numbers you will need to decide what precision you need. 4 decimal
places? Multiply by 10,000.
Cheers,
Dave
On 9/25/06, rosoft2001 [email protected] wrote:
how come <%= 19043.0 - 19032.6 %>
display 10.4000000000015
how can I avoid this and get a proper result?
Look up bigdecimal
http://www.ruby-doc.org/stdlib/
John W Higgins
[email protected]
On 9/26/06, John H. [email protected] wrote:
Look up bigdecimal
You’ll also need edge rails if you plan on storing accurate values as
decimal/numeric in your db.
Isak