Ruby to_i problem

I’m having an issue with a float being converted to an int,

The maths is part of a vector rotation of a little video game, which x,y
coords are rotated at a given angle.

For what ever reason the first instance gives an error, and every
instance there after works as it should

This cant be right?

ruby 1.8.7
[irb]

This works the way I would think

1.0.o_i
=> 1

This doesn’t

x = -1
y = -1
angle = 90 * Math::PI / 180
n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
=> 1.0

this is where its werid

n_x.to_i
=> 0
[/irb]

any further 1.0.to_i works

Phil Cooper-king wrote:

any further 1.0.to_i works

You’re experiencing IEEE Floating point aritmetic. Look at this example:

ruby -v: ruby 1.9.1p378 (2010-01-10 revision 26273) [x86_64-linux]
irb(main):001:0> x = -1
=> -1
irb(main):002:0> y = -1
=> -1
irb(main):003:0> angle = 90 * Math::PI / 180
=> 1.5707963267949
irb(main):004:0> n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
=> 1.0
irb(main):005:0> sprintf("%.50f", n_x)
=> “0.99999999999999988897769753748434595763683319091797”
irb(main):006:0> n_x.to_i
=> 0
irb(main):007:0> n_x.round
=> 1
irb(main):008:0> sprintf("%.50f", 1.0)
=> “1.00000000000000000000000000000000000000000000000000”
irb(main):009:0>

Your result isn’t that exactly 1 as you think.

Marvin

On Tue, Feb 16, 2010 at 10:16 AM, Phil Cooper-king [email protected]
wrote:

For what ever reason the first instance gives an error, and every
instance there after works as it should

What You Think Is Happening Is Not What Is Actually Happening :slight_smile:

ruby 1.8.7
[irb]

This works the way I would think

1.0.o_i
=> 1

Yep, and every time you say 1.0.to_i you will get 1.

This doesn’t

x = -1
y = -1
angle = 90 * Math::PI / 180
n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
=> 1.0

irb is showing you an approximation. It’s confusing because irb is
showing you n_x to be 1.0, when in fact:

sprintf “%0.25f”, n_x
=> “0.9999999999999998889776975”

this is where its werid

n_x.to_i
=> 0

Yep:

0.9.to_i
=> 0

any further 1.0.to_i works

1.0.to_i and n_x.to_i (from your example) are two different things.

Search the group archives for “floating point”, you’re sure to find a
huge pile of other people with similar problems, and likely good
solutions.

Ben

On Tue, Feb 16, 2010 at 7:16 PM, Phil Cooper-king [email protected]
wrote:

angle = 90 * Math::PI / 180
n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
=> 1.0

this is where its werid

n_x.to_i
=> 0
[/irb]

any further 1.0.to_i works

You are facing a problem about the inaccuracy inherent to floating
point numbers:

irb(main):001:0> 0.9999.to_i
=> 0
irb(main):002:0> x = -1
=> -1
irb(main):003:0> y = -1
=> -1
irb(main):004:0> angle = 90 * Math::PI / 180
=> 1.5707963267949
irb(main):005:0> n_x = (Math::cos(angle) * x) - (Math::sin(angle) * y)
=> 1.0
irb(main):009:0> “%.20f” % n_x
=> “0.99999999999999988898”
irb(main):010:0>
irb(main):010:0> n_x
=> 1.0

n_x is very close to 1.0 but it’s not. to_i rounds the number down, so
you get 0. You can check the archives for a more thorough explanation
about this, there’s been plenty of discussion in the past.

Jesus.

thanks guys, this makes it clear

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs