I’m sure this has been hashed out somewhere on the list before, but
my searches have turned up fruitless. I had a strange issue come up
while diagnosing a unit test failure.
Notice the first dialog compared with the rest of this IRB session.
What causes -3930.0 to be converted to a Fixnum as -3929 only after
it’s been multiplied by 100 or 100.0? Even weirder is that it seems
to only be this number. Can I workaround this?
Rounding errors are typical for floating point numbers. In this case the
result
of -39.30*100 ist a little bit smaller than 3930, so to_i works correct.
Rounding errors are typical for floating point numbers. In this case
the result of -39.30*100 ist a little bit smaller than 3930, so to_i
works correct.
When I think of rounding error, I think of lost precision due to
rounding too early in a series of floating point calculations. When
multiplying a non-repeating Float point by 100, shouldn’t the decimal
move over two places without introducing extra precision?
And how can 39.30100 really be 3930.000000000000454747350886464…,
but 39.31100 be 3931.0?
BigDecimal seems to be unaffected, so I guess I’ll use that as a
workaround:
rounding too early in a series of floating point calculations. When
multiplying a non-repeating Float point by 100, shouldn’t the decimal
move over two places without introducing extra precision?
And how can 39.30100 really be 3930.000000000000454747350886464…,
but 39.31100 be 3931.0?
Because 39.30 is a base 10 representation of a number and there
is no exact representation of that number in base 2. The Ruby
parser has to do a conversion to base 2 for 39.30 long before
the multiplication occurs so you aren’t really multiplying
39.30*100 but instead you are multiplying
ruby supports integers and floating point numbers (wc in turn are
supported by the hw architecture). imho, there’s a big gap/hole for the
representation of plain decimals (usurped by floating point in this
case).
#--------------------------------------
>> (39.30*100).to_d.to_i
=> 3930
#--------------------------------------
indeed. 99% of the my time, i find no use for floats (maybe i’m scared
of surprises).
i’d suggest a global $DECIMAL=1 to instruct ruby to bigdecimal mode so
that
39.30100 == 3930
=> true
(39.30100).to_i # no need for to_d there, 39.30 is decimal,
right?
=> 3930
i really do not care if it would be slower. i chosed ruby anyway.
rounding too early in a series of floating point calculations. When
multiplying a non-repeating Float point by 100, shouldn’t the decimal
move over two places without introducing extra precision?
And how can 39.30*100 really be 3930.000000000000454747350886464…,
Almost all programming languages use binary floating point, so 0.30
can’t be
represented exactly. A very few languages use binary-coded decimal.
Examples: Decimal BASIC and Business BASIC (I think). The BASIC
that came with the old 8-bit Atari used binary-coded decimal, so it
would
have had no problem with 3930-39.30*100.
Since computers of today are so much faster than 8-bit computers,
one would think that they could afford to incur the speed penalty
associated with binary-coded decimal.
