Hi,

Some more not so fun rounding issues.

Using Floats:

((30 / 1.16) * 1.16) == 30

=> true

Cool, so you’d think it would work with BigDecimal:

((30 / BigDecimal(“1.16”)) * BigDecimal(“1.16”)) == 30

=> false

((30 / BigDecimal(“1.16”)) * BigDecimal(“1.16”)).to_s

=> “30.0000000000000000000000002”

I’m clearly missing something here as I thought BigDecimal was supposed

to fix this type of thing.

Tested in:

ruby 1.8.6 (2008-03-03 patchlevel 114) [universal-darwin9.0]

ruby 1.9.1p0 (2009-01-30 revision 21907) [i386-darwin9.6.0]

Can anyone offer an explanation?

Cheers, sam

Hi,

At Wed, 6 May 2009 10:01:54 +0900,

Samuel L. wrote in [ruby-talk:335867]:

((30 / BigDecimal(“1.16”)) * BigDecimal(“1.16”)) == 30

=> false

((30 / BigDecimal(“1.16”)) * BigDecimal(“1.16”)).to_s

=> “30.0000000000000000000000002”

I’m clearly missing something here as I thought BigDecimal was supposed

to fix this type of thing.

BigDecimal is another kind of floating point number, which uses

decimal base instead of binary, but has finite digits.

Therefore, it is impossible to represent exactly a recurring

decimal theoretically.

On Tue, May 5, 2009 at 9:01 PM, Samuel L. removed_email_address[email protected] wrote:

ruby 1.9.1p0 (2009-01-30 revision 21907) [i386-darwin9.6.0]

Can anyone offer an explanation?

BigDecimal is still a kind of float, albeit a decimal float rather

than a binary float.

So it helps with problems when the fractional part of a number can be

expressed exactly as a finite string of decimal digits, but just as

certain fractional parts can’t be expressed exactly as a finite string

of binary digits, there are some which have the same problem when the

base is 10.

For example 1/3 cannot be expressed as a decimal float, no matter how

many digits 0.33333…

Similarly 3000/116 (which is the same as 30/1.16) can’t be expressed

by a finite sequence of decimal digits.

It comes out as

25.862068965517241379310344827[5862068965517241379310344827]…

where the digits in the brackets repeat infinitely.

–

Rick DeNatale

Blog: http://talklikeaduck.denhaven2.com/

Twitter: http://twitter.com/RickDeNatale

WWR: http://www.workingwithrails.com/person/9021-rick-denatale

LinkedIn: http://www.linkedin.com/in/rickdenatale

Yeah I think they would be better off with rationals

Blog: http://random8.zenunit.com/

Learn: http://sensei.zenunit.com/

Twitter: http://twitter.com/random8r

On 06/05/2009, at 1:30 PM, Rick DeNatale [email protected]

Rick Denatale wrote:

On Tue, May 5, 2009 at 9:01 PM, Samuel L. [email protected] wrote:

Can anyone offer an explanation?

BigDecimal is still a kind of float, albeit a decimal float rather

than a binary float.

Many thanks Nobuyoshi and Rick, that makes complete sense now.

I guess the safest rule of thumb is if you’re doing comparisons for

humans, use humanized numbers (i.e. integers

Cheers, sam

2009/5/6 Samuel L. [email protected]:

Many thanks Nobuyoshi and Rick, that makes complete sense now.

I guess the safest rule of thumb is if you’re doing comparisons for

humans, use humanized numbers (i.e. integers

Or define a maximum error, i.e. do not rely on identical values but

rather the difference of the quotient.

Kind regards

robert