Ruby to_i problem

Ruby
1

I’m having an issue with a float being converted to an int,

The maths is part of a vector rotation of a little video game, which x,y
coords are rotated at a given angle.

For what ever reason the first instance gives an error, and every
instance there after works as it should

This cant be right?

ruby 1.8.7
[irb]

This works the way I would think

1.0.o_i
=> 1

This doesn’t

x = -1
y = -1
angle = 90 * Math::PI / 180
n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
=> 1.0

this is where its werid

n_x.to_i
=> 0
[/irb]

any further 1.0.to_i works

2

Phil Cooper-king wrote:

any further 1.0.to_i works

You’re experiencing IEEE Floating point aritmetic. Look at this example:

ruby -v: ruby 1.9.1p378 (2010-01-10 revision 26273) [x86_64-linux]
irb(main):001:0> x = -1
=> -1
irb(main):002:0> y = -1
=> -1
irb(main):003:0> angle = 90 * Math::PI / 180
=> 1.5707963267949
irb(main):004:0> n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
=> 1.0
irb(main):005:0> sprintf("%.50f", n_x)
=> “0.99999999999999988897769753748434595763683319091797”
irb(main):006:0> n_x.to_i
=> 0
irb(main):007:0> n_x.round
=> 1
irb(main):008:0> sprintf("%.50f", 1.0)
=> “1.00000000000000000000000000000000000000000000000000”
irb(main):009:0>

Your result isn’t that exactly 1 as you think.

Marvin

3

On Tue, Feb 16, 2010 at 10:16 AM, Phil Cooper-king [email protected]
wrote:

For what ever reason the first instance gives an error, and every
instance there after works as it should

What You Think Is Happening Is Not What Is Actually Happening :slight_smile:

ruby 1.8.7
[irb]

This works the way I would think

1.0.o_i
=> 1

Yep, and every time you say 1.0.to_i you will get 1.

This doesn’t

x = -1
y = -1
angle = 90 * Math::PI / 180
n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
=> 1.0

irb is showing you an approximation. It’s confusing because irb is
showing you n_x to be 1.0, when in fact:

sprintf “%0.25f”, n_x
=> “0.9999999999999998889776975”

this is where its werid

n_x.to_i
=> 0

Yep:

0.9.to_i
=> 0

any further 1.0.to_i works

1.0.to_i and n_x.to_i (from your example) are two different things.

Search the group archives for “floating point”, you’re sure to find a
huge pile of other people with similar problems, and likely good
solutions.

Ben

4

On Tue, Feb 16, 2010 at 7:16 PM, Phil Cooper-king [email protected]
wrote:

angle = 90 * Math::PI / 180
n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
=> 1.0

this is where its werid

n_x.to_i
=> 0
[/irb]

any further 1.0.to_i works

You are facing a problem about the inaccuracy inherent to floating
point numbers:

irb(main):001:0> 0.9999.to_i
=> 0
irb(main):002:0> x = -1
=> -1
irb(main):003:0> y = -1
=> -1
irb(main):004:0> angle = 90 * Math::PI / 180
=> 1.5707963267949
irb(main):005:0> n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
=> 1.0
irb(main):009:0> “%.20f” % n_x
=> “0.99999999999999988898”
irb(main):010:0>
irb(main):010:0> n_x
=> 1.0

n_x is very close to 1.0 but it’s not. to_i rounds the number down, so
you get 0. You can check the archives for a more thorough explanation
about this, there’s been plenty of discussion in the past.

Jesus.

5

thanks guys, this makes it clear