(2241.18 * 100).to_i

=> 224117

Anyone know what is up with that?

(2241.18 * 100).to_i

=> 224117

Anyone know what is up with that?

Brandon H. wrote:

(2241.18 * 100).to_i

=> 224117

Anyone know what is up with that?

Floating point arithmetic isn’t entirely accurate.

“%.30f” % (2241.18 * 100)

=> “224117.999999999970896169543266296387”

And since 224117.999999999970896169543266296387.to_i is 224117, that’s

what

you get.

HTH,

Sebastian

Sebastian H. wrote:

Brandon H. wrote:

(2241.18 * 100).to_i

=> 224117

Anyone know what is up with that?

Floating point arithmetic isn’t entirely accurate.

“%.30f” % (2241.18 * 100)

=> “224117.999999999970896169543266296387”And since 224117.999999999970896169543266296387.to_i is 224117, that’s

what

you get.HTH,

Sebastian

Thanks… found this after poking around… guess I should have spent

more time looking.

Hello!

On Jan 23, 2008 4:25 PM, Brandon H. [email protected] wrote:

(2241.18 * 100).to_i

=> 224117Anyone know what is up with that?

Floating point numbers are never 100% accurate. See

Floating-point arithmetic - Wikipedia for a detailed

explanation.

On Jan 23, 2008, at 10:25 AM, Brandon H. wrote:

(2241.18 * 100).to_i

=> 224117Anyone know what is up with that?

What Every Computer Scientist Should Know About Floating-Point

Arithmetic

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

Sponsor our Newsletter | Privacy Policy | Terms of Service | Remote Ruby Jobs