(2241.18 * 100).to_i
=> 224117
Anyone know what is up with that?
(2241.18 * 100).to_i
=> 224117
Anyone know what is up with that?
Brandon H. wrote:
(2241.18 * 100).to_i
=> 224117
Anyone know what is up with that?
Floating point arithmetic isn’t entirely accurate.
“%.30f” % (2241.18 * 100)
=> “224117.999999999970896169543266296387”
And since 224117.999999999970896169543266296387.to_i is 224117, that’s
what
you get.
HTH,
Sebastian
Sebastian H. wrote:
Brandon H. wrote:
(2241.18 * 100).to_i
=> 224117
Anyone know what is up with that?
Floating point arithmetic isn’t entirely accurate.
“%.30f” % (2241.18 * 100)
=> “224117.999999999970896169543266296387”And since 224117.999999999970896169543266296387.to_i is 224117, that’s
what
you get.HTH,
Sebastian
Thanks… found this after poking around… guess I should have spent
more time looking.
Hello!
On Jan 23, 2008 4:25 PM, Brandon H. [email protected] wrote:
(2241.18 * 100).to_i
=> 224117Anyone know what is up with that?
Floating point numbers are never 100% accurate. See
http://en.wikipedia.org/wiki/Floating_point for a detailed
explanation.
On Jan 23, 2008, at 10:25 AM, Brandon H. wrote:
(2241.18 * 100).to_i
=> 224117Anyone know what is up with that?
What Every Computer Scientist Should Know About Floating-Point
Arithmetic
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.
Sponsor our Newsletter | Privacy Policy | Terms of Service | Remote Ruby Jobs