Floating point conversion error?

Ruby
1

(2241.18 * 100).to_i
=> 224117

Anyone know what is up with that?

2

Brandon H. wrote:

(2241.18 * 100).to_i

=> 224117

Anyone know what is up with that?

Floating point arithmetic isn’t entirely accurate.

“%.30f” % (2241.18 * 100)
=> “224117.999999999970896169543266296387”

And since 224117.999999999970896169543266296387.to_i is 224117, that’s
what
you get.

HTH,
Sebastian

3

Sebastian H. wrote:

Brandon H. wrote:

(2241.18 * 100).to_i

=> 224117

Anyone know what is up with that?

Floating point arithmetic isn’t entirely accurate.

“%.30f” % (2241.18 * 100)
=> “224117.999999999970896169543266296387”

And since 224117.999999999970896169543266296387.to_i is 224117, that’s
what
you get.

HTH,
Sebastian

Thanks… found this after poking around… guess I should have spent
more time looking.

http://www.ruby-forum.com/topic/132986#new

4

Hello!

On Jan 23, 2008 4:25 PM, Brandon H. [email protected] wrote:

(2241.18 * 100).to_i
=> 224117

Anyone know what is up with that?

Floating point numbers are never 100% accurate. See
Floating-point arithmetic - Wikipedia for a detailed
explanation.

5

On Jan 23, 2008, at 10:25 AM, Brandon H. wrote:

(2241.18 * 100).to_i
=> 224117

Anyone know what is up with that?

What Every Computer Scientist Should Know About Floating-Point
Arithmetic

http://docs.sun.com/source/806-3568/ncg_goldberg.html