Hi, Some more not so fun rounding issues. Using Floats: >> ((30 / 1.16) * 1.16) == 30 => true Cool, so you'd think it would work with BigDecimal: >> ((30 / BigDecimal("1.16")) * BigDecimal("1.16")) == 30 => false >> ((30 / BigDecimal("1.16")) * BigDecimal("1.16")).to_s => "30.0000000000000000000000002" I'm clearly missing something here as I thought BigDecimal was supposed to fix this type of thing. Tested in: ruby 1.8.6 (2008-03-03 patchlevel 114) [universal-darwin9.0] ruby 1.9.1p0 (2009-01-30 revision 21907) [i386-darwin9.6.0] Can anyone offer an explanation? Cheers, sam

on 2009-05-06 05:01

on 2009-05-06 07:06

Hi, At Wed, 6 May 2009 10:01:54 +0900, Samuel L. wrote in [ruby-talk:335867]: > >> ((30 / BigDecimal("1.16")) * BigDecimal("1.16")) == 30 > => false > >> ((30 / BigDecimal("1.16")) * BigDecimal("1.16")).to_s > => "30.0000000000000000000000002" > > I'm clearly missing something here as I thought BigDecimal was supposed > to fix this type of thing. BigDecimal is another kind of floating point number, which uses decimal base instead of binary, but has finite digits. Therefore, it is impossible to represent exactly a recurring decimal theoretically.

on 2009-05-06 07:30

On Tue, May 5, 2009 at 9:01 PM, Samuel L. <removed_email_address@domain.invalid> wrote: > > ruby 1.9.1p0 (2009-01-30 revision 21907) [i386-darwin9.6.0] > > Can anyone offer an explanation? BigDecimal is still a kind of float, albeit a decimal float rather than a binary float. So it helps with problems when the fractional part of a number can be expressed exactly as a finite string of decimal digits, but just as certain fractional parts can't be expressed exactly as a finite string of binary digits, there are some which have the same problem when the base is 10. For example 1/3 cannot be expressed as a decimal float, no matter how many digits 0.33333..... Similarly 3000/116 (which is the same as 30/1.16) can't be expressed by a finite sequence of decimal digits. It comes out as 25.862068965517241379310344827[5862068965517241379310344827]... where the digits in the brackets repeat infinitely. -- Rick DeNatale Blog: http://talklikeaduck.denhaven2.com/ Twitter: http://twitter.com/RickDeNatale WWR: http://www.workingwithrails.com/person/9021-rick-denatale LinkedIn: http://www.linkedin.com/in/rickdenatale

on 2009-05-06 14:08

Rick Denatale wrote: > On Tue, May 5, 2009 at 9:01 PM, Samuel L. <removed_email_address@domain.invalid> wrote: >> >> Can anyone offer an explanation? > > BigDecimal is still a kind of float, albeit a decimal float rather > than a binary float. > Many thanks Nobuyoshi and Rick, that makes complete sense now. I guess the safest rule of thumb is if you're doing comparisons for humans, use humanized numbers (i.e. integers :-) Cheers, sam

on 2009-05-06 14:11

Yeah I think they would be better off with rationals Blog: http://random8.zenunit.com/ Learn: http://sensei.zenunit.com/ Twitter: http://twitter.com/random8r On 06/05/2009, at 1:30 PM, Rick DeNatale <removed_email_address@domain.invalid>

on 2009-05-06 15:07

2009/5/6 Samuel L. <removed_email_address@domain.invalid>: > Many thanks Nobuyoshi and Rick, that makes complete sense now. > > I guess the safest rule of thumb is if you're doing comparisons for > humans, use humanized numbers (i.e. integers :-) Or define a maximum error, i.e. do not rely on identical values but rather the difference of the quotient. Kind regards robert