Hey, How come 2 - 2.01 = -0.00999999999999979 and not -0.01 ??? Can someone explain this to me please?
on 18.10.2007 14:49
on 18.10.2007 14:55
On 10/18/07, Michal Sza <nicpon@nicpon.net> wrote: > Hey, > How come 2 - 2.01 = -0.00999999999999979 and not -0.01 > ??? > Can someone explain this to me please? http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/273263
on 18.10.2007 14:57
Michal Sza wrote: > Hey, > How come 2 - 2.01 = -0.00999999999999979 and not -0.01 > ??? > Can someone explain this to me please? Because 0.01 cannot be represented accurately in binary. See http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems.
on 18.10.2007 14:58
On Thu, Oct 18, 2007 at 09:49:37PM +0900, Michal Sza wrote: > Hey, > How come 2 - 2.01 = -0.00999999999999979 and not -0.01 > ??? > Can someone explain this to me please? http://en.wikipedia.org/wiki/Floating_point#Accuracy_problems --Greg
on 18.10.2007 15:01
On Thu, 18 Oct 2007 21:49:37 +0900, Michal Sza <nicpon@nicpon.net> wrote: > Hey, > How come 2 - 2.01 = -0.00999999999999979 and not -0.01 > ??? > Can someone explain this to me please? Because you are using floating point. Reas this article for an explanation: "What Every Computer Scientist Should Know About Floating-Point Arithmetic" http://docs.sun.com/source/806-3568/ncg_goldberg.html Kristian
on 18.10.2007 15:02
On Oct 18, 8:49 am, Michal Sza <nic...@nicpon.net> wrote: > Hey, > How come 2 - 2.01 = -0.00999999999999979 and not -0.01 > ??? > Can someone explain this to me please? > -- > Posted viahttp://www.ruby-forum.com/. This question comes up a lot. Google "floating point errors" for some good links.