 # Negative Float conversion to Fixnum

I’m sure this has been hashed out somewhere on the list before, but
my searches have turned up fruitless. I had a strange issue come up
while diagnosing a unit test failure.

Notice the first dialog compared with the rest of this IRB session.
What causes -3930.0 to be converted to a Fixnum as -3929 only after
it’s been multiplied by 100 or 100.0? Even weirder is that it seems
to only be this number. Can I workaround this?

(-39.30 * 100).to_i
=> -3929

(-39.30 * 100.0).to_i
=> -3929

-3930.0.to_i
=> -3930

(-39.31 * 100).to_i
=> -3931

(-9999.99 * 100).to_i
=> -999999
\$ ruby -v
ruby 1.8.5 (2006-12-04 patchlevel 2) [i686-linux]

Thanks.

-Drew

Drew R. schrieb:

=> -3929

Thanks.

-Drew

irb(main):001:0> 3930-39.30*100
=> 4.54747350886464e-013

Rounding errors are typical for floating point numbers. In this case the
result
of -39.30*100 ist a little bit smaller than 3930, so to_i works correct.

Drew R. schrieb:

Can I workaround this?

If you always expect a result for which “x * 100” is a number without a
fractional part, you can use “(-39.30 * 100).round” instead.

Wolfgang Nádasi-Donner

Wolfgang NÃ¡dasi-Donner wrote:

irb(main):001:0> 3930-39.30*100
=> 4.54747350886464e-013

Rounding errors are typical for floating point numbers. In this case
the result of -39.30*100 ist a little bit smaller than 3930, so to_i
works correct.

When I think of rounding error, I think of lost precision due to
rounding too early in a series of floating point calculations. When
multiplying a non-repeating Float point by 100, shouldn’t the decimal
move over two places without introducing extra precision?

And how can 39.30100 really be 3930.000000000000454747350886464…,
but 39.31
100 be 3931.0?

BigDecimal seems to be unaffected, so I guess I’ll use that as a
workaround:

(39.30*100).to_i
=> 3929

(39.30*100).to_d.to_i
=> 3930

-Drew

On Sat, 20 Jan 2007, Drew R. wrote:

rounding too early in a series of floating point calculations. When
multiplying a non-repeating Float point by 100, shouldn’t the decimal
move over two places without introducing extra precision?

no. check out

http://en.wikipedia.org/wiki/Floating_point#Computer_representation

-a

On Jan 19, 2007, at 4:50 PM, Drew R. wrote:

And how can 39.30100 really be 3930.000000000000454747350886464…,
but 39.31
100 be 3931.0?

Because 39.30 is a base 10 representation of a number and there
is no exact representation of that number in base 2. The Ruby
parser has to do a conversion to base 2 for 39.30 long before
the multiplication occurs so you aren’t really multiplying
39.30*100 but instead you are multiplying

39.2999999999999971578290569595992565155029296875 * 100

to get

3929.99999999999971578290569595992565155029296875

which when truncated to an integer is

3929

Gary W.

From: Drew R. [mailto:[email protected]] :

#--------------------------------------

# => 3929

#--------------------------------------

mixing integers and float is like mixing water w gasoline. expect
disasters in all directions.

irb(main):018:0> 39.30100 == 3930
=> false
irb(main):019:0> 39.30
100 == 3939
=> false

ruby supports integers and floating point numbers (wc in turn are
supported by the hw architecture). imho, there’s a big gap/hole for the
representation of plain decimals (usurped by floating point in this
case).

#--------------------------------------

# => 3930

#--------------------------------------

indeed. 99% of the my time, i find no use for floats (maybe i’m scared
of surprises).

i’d suggest a global \$DECIMAL=1 to instruct ruby to bigdecimal mode so
that

39.30*100 == 3930
=> true

(39.30*100).to_i # no need for to_d there, 39.30 is decimal,
right? => 3930

i really do not care if it would be slower. i chosed ruby anyway.

kind regards -botp

Drew R. wrote:

rounding too early in a series of floating point calculations. When
multiplying a non-repeating Float point by 100, shouldn’t the decimal
move over two places without introducing extra precision?

And how can 39.30*100 really be 3930.000000000000454747350886464…,

Almost all programming languages use binary floating point, so 0.30
can’t be
represented exactly. A very few languages use binary-coded decimal.
Examples: Decimal BASIC and Business BASIC (I think). The BASIC
that came with the old 8-bit Atari used binary-coded decimal, so it
would
have had no problem with 3930-39.30*100.

Since computers of today are so much faster than 8-bit computers,
one would think that they could afford to incur the speed penalty
associated with binary-coded decimal.

On Feb 8, 2007, at 12:50 PM, Drew R. wrote:

class Float
FUDGE = 1e-3
def ==(x)
(self-x).abs < FUDGE
end
end

This is better for my tests anyway. assert_equal now works on its
own.

Test::Unit already includes an assertion just for this. It’s called
assert_in_delta().

James Edward G. II

Wolfgang NÃ¡dasi-Donner wrote:

Drew R. schrieb:

Can I workaround this?

If you always expect a result for which “x * 100” is a number without
a fractional part, you can use “(-39.30 * 100).round” instead.

Hal F. gave me a good idea:

class Float
FUDGE = 1e-3
def ==(x)
(self-x).abs < FUDGE
end
end

This is better for my tests anyway. assert_equal now works on its
own.

-Drew

Footnotes:
 http://lnk.nu/amazon.com/dng

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.