Hey, How come 2 - 2.01 = -0.00999999999999979 and not -0.01 ??? Can someone explain this to me please?
on 2007-10-18 14:49
on 2007-10-18 14:55
On 10/18/07, Michal Sza <nicpon@nicpon.net> wrote: > Hey, > How come 2 - 2.01 = -0.00999999999999979 and not -0.01 > ??? > Can someone explain this to me please? http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/273263
on 2007-10-18 14:57
Michal Sza wrote: > Hey, > How come 2 - 2.01 = -0.00999999999999979 and not -0.01 > ??? > Can someone explain this to me please? Because 0.01 cannot be represented accurately in binary. See http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems.
on 2007-10-18 14:58
On Thu, Oct 18, 2007 at 09:49:37PM +0900, Michal Sza wrote: > Hey, > How come 2 - 2.01 = -0.00999999999999979 and not -0.01 > ??? > Can someone explain this to me please? http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems --Greg
on 2007-10-18 15:01
On Thu, 18 Oct 2007 21:49:37 +0900, Michal Sza <nicpon@nicpon.net> wrote: > Hey, > How come 2 - 2.01 = -0.00999999999999979 and not -0.01 > ??? > Can someone explain this to me please? Because you are using floating point. Reas this article for an explanation: "What Every Computer Scientist Should Know About Floating-Point Arithmetic" http://docs.sun.com/source/806-3568/ncg_goldberg.html Kristian
on 2007-10-18 15:02
On Oct 18, 8:49 am, Michal Sza <nic...@nicpon.net> wrote: > Hey, > How come 2 - 2.01 = -0.00999999999999979 and not -0.01 > ??? > Can someone explain this to me please? > -- > Posted viahttp://www.ruby-forum.com/. This question comes up a lot. Google "floating point errors" for some good links.
Please log in before posting. Registration is free and takes only a minute.
Existing account
(Switch to SSL-encrypted connection)
NEW: Do you have a Google/GoogleMail or Yahoo account? No registration required!
Log in with Google account | Log in with Yahoo account
Log in with Google account | Log in with Yahoo account
No account? Register here.