# Computers + floating-point = a chore

**URL:** <https://www.ruby-forum.com/t/computers-floating-point-a-chore/117975>\
**Category:** Ruby\
**Created:** [October 22, 2007, 10:29pm UTC](https://www.ruby-forum.com/t/computers-floating-point-a-chore/117975 "2007-10-22T22:29:00Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Yossef\_M](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/a87d85/32.png) [@Yossef\_M](https://www.ruby-forum.com/u/Yossef_M)\
**Post date:** [October 22, 2007, 10:29pm UTC](https://www.ruby-forum.com/t/computers-floating-point-a-chore/117975/1 "2007-10-22T22:29:00Z")

</div>

I recall seeing a few posts about the “fun” involving floating-point  
representation of numbers, but this is something I found a little  
surprising:

Cassady:~ yossef$ irb --prompt simple

> > require ‘bigdecimal’  
> > =\> true
> 
> > bd = BigDecimal(‘19.76’)  
> > =\> #BigDecimal:9ccc,‘0.1976E2’,8(8)
> 
> > bd \<=\> 19.76  
> > =\> -1
> 
> > bd.to\_f \<=\> 19.76  
> > =\> -1
> 
> > bd.to\_f  
> > =\> 19.76
> 
> > 19.76 \<=\> 19.76  
> > =\> 0

Odd, isn’t it? What’s going on here?

> > 19.75999999999999999999999999999999999  
> > =\> 19.76

Ah, maybe this is it.

> > 19.75999999999999999999999999999999999 \<=\> 19.76  
> > =\> 0

Nope!  
Well, a bit of experimentation led me to this, which demonstrates two  
threshold points.

> > 19.7599999999999  
> > =\> 19.7599999999999
> 
> > 19.75999999999999  
> > =\> 19.76
> 
> > 19.75999999999999 \<=\> 19.76  
> > =\> -1
> 
> > 19.759999999999999 \<=\> 19.76  
> > =\> -1
> 
> > 19.7599999999999999 \<=\> 19.76  
> > =\> 0

It appears 19.75999999999999 (that’s 12 9s, or 14 decimal places  
total) is just close enough to _display_ rounded up, but not actually  
_become_ that number. Adding two more 9s makes it close enough to  
actually become that number.

No question or call for help, really. I just wanted to point out  
something that intrigued me.

---

<div class="post-metadata">

**Author:** ![Yossef\_M](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/a87d85/32.png) [@Yossef\_M](https://www.ruby-forum.com/u/Yossef_M)\
**Post date:** [October 23, 2007, 1:43am UTC](https://www.ruby-forum.com/t/computers-floating-point-a-chore/117975/2 "2007-10-23T01:43:14Z")

</div>

floating point math is always like this.  
Any language or framework that purports to have no trouble with it is  
actually using objects (or structs) consisting of integers for each  
of its aspects. All of the aspects are assembled.

---

<div class="post-metadata">

**Author:** ![Yossef\_M](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/a87d85/32.png) [@Yossef\_M](https://www.ruby-forum.com/u/Yossef_M)\
**Post date:** [October 23, 2007, 11:28am UTC](https://www.ruby-forum.com/t/computers-floating-point-a-chore/117975/3 "2007-10-23T11:28:43Z")

</div>

On Oct 22, 6:42 pm, John J. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
wrote:

> floating point math is always like this.  
> Any language or framework that purports to have no trouble with it is  
> actually using objects (or structs) consisting of integers for each  
> of its aspects. All of the aspects are assembled.

Yes, I know the troubles with floating point. I think what I found  
most interesting was a number that claimed to be 19.76 when it  
actually wasn’t, simply because it’s more convenient (?) to display it  
as such.

---

<div class="post-metadata">

**Author:** ![Yossef\_M](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/a87d85/32.png) [@Yossef\_M](https://www.ruby-forum.com/u/Yossef_M)\
**Post date:** [October 23, 2007, 4:11pm UTC](https://www.ruby-forum.com/t/computers-floating-point-a-chore/117975/4 "2007-10-23T16:11:10Z")

</div>

On Oct 22, 10:14 pm, Gary W. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> to 15 digits of precession and also works hard to strip trailing zero’s:  
> irb(main):005:0\>  
> {  
> if (!(e = strchr(buf, ‘e’))) {  
> while (p[-1]==‘0’ && ISDIGIT(p[-2]))  
> p–;  
> memmove(p, e, strlen(e)+1);  
> return rb\_str\_new2(buf);
> 
> }
> 
> Gary W.

Thanks for this, Gary. I wasn’t expecting that Float#inspect would  
call #to\_s, and obviously didn’t even bother to test anything out with  
sprintf.

This also explains another Float niggle, which is why it will display  
“Infinity” but trying to use Infinity as a constant doesn’t work.

Maybe I should spend more time looking at the Ruby source.

---

<div class="post-metadata">

**Author:** ![Yossef\_M](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/y/a87d85/32.png) [@Yossef\_M](https://www.ruby-forum.com/u/Yossef_M)\
**Post date:** [October 23, 2007, 11:51am UTC](https://www.ruby-forum.com/t/computers-floating-point-a-chore/117975/5 "2007-10-23T11:51:02Z")

</div>

On Oct 22, 2007, at 10:30 PM, Yossef M. wrote:

> Yes, I know the troubles with floating point. I think what I found  
> most interesting was a number that claimed to be 19.76 when it  
> actually wasn’t, simply because it’s more convenient (?) to display it  
> as such.

Well, the culprit is actually Float#to\_s, which has it’s own idea of  
how to convert floating point numbers to text. In particular it  
defaults  
to 15 digits of precession and also works hard to strip trailing zero’s:

$ irb  
irb(main):001:0\> a = 19.76  
=\> 19.76  
irb(main):002:0\> a.to\_s  
=\> “19.76”  
irb(main):003:0\> sprintf “%#.15g”, a  
=\> “19.7600000000000”  
irb(main):004:0\> sprintf “%#.32g”, a  
=\> “19.760000000000001563194018672220”  
irb(main):005:0\>

You can see from this session that 15 digits of precision is not  
enough to ‘show’ that a isn’t really 19.76.

What was surprising to me is that Float#to\_s doesn’t just use a standard  
sprintf format string:

static VALUE  
flo\_to\_s(flt)  
VALUE flt;  
{  
char buf[32];  
double value = RFLOAT(flt)-\>value;  
char \*p, \*e;

```
 if (isinf(value))
     return rb_str_new2(value < 0 ? "-Infinity" : "Infinity");
 else if(isnan(value))
     return rb_str_new2("NaN");

 sprintf(buf, "%#.15g", value); /* ensure to print decimal point */
 if (!(e = strchr(buf, 'e'))) {
     e = buf + strlen(buf);
 }
 if (!ISDIGIT(e[-1])) { /* reformat if ended with decimal point

```

(ex 111111111111111.) \*/  
sprintf(buf, “%#.14e”, value);  
if (!(e = strchr(buf, ‘e’))) {  
e = buf + strlen(buf);  
}  
}  
p = e;  
while (p[-1]==‘0’ && ISDIGIT(p[-2]))  
p–;  
memmove(p, e, strlen(e)+1);  
return rb\_str\_new2(buf);  
}

Gary W.