# Float precision

**URL:** <https://www.ruby-forum.com/t/float-precision/148375>\
**Category:** Ruby\
**Created:** [September 12, 2008, 10:17am UTC](https://www.ruby-forum.com/t/float-precision/148375 "2008-09-12T10:17:45Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![difei](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/a4c791/32.png) [@difei](https://www.ruby-forum.com/u/difei)\
**Post date:** [September 12, 2008, 10:17am UTC](https://www.ruby-forum.com/t/float-precision/148375/1 "2008-09-12T10:17:45Z")

</div>

Hi all,

Is there a direct method to get something like:

var = sprintf("%.2f", 92.22312).to\_f

Thanks alot!

Difei

---

<div class="post-metadata">

**Author:** ![difei](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/a4c791/32.png) [@difei](https://www.ruby-forum.com/u/difei)\
**Post date:** [September 12, 2008, 12:46pm UTC](https://www.ruby-forum.com/t/float-precision/148375/2 "2008-09-12T12:46:00Z")

</div>

The round function in the float class does not provide a precision  
feature, but you can do something like:  
(100\*9.23234234234532).round/100.0

According to my benchmarks this is at least 1/3 faster than your  
approach

Thomas

---

<div class="post-metadata">

**Author:** ![difei](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/a4c791/32.png) [@difei](https://www.ruby-forum.com/u/difei)\
**Post date:** [September 12, 2008, 3:20pm UTC](https://www.ruby-forum.com/t/float-precision/148375/3 "2008-09-12T15:20:59Z")

</div>

thomas peklak wrote:

> The round function in the float class does not provide a precision  
> feature, but you can do something like:  
> (100\*9.23234234234532).round/100.0
> 
> According to my benchmarks this is at least 1/3 faster than your  
> approach
> 
> Thomas

thanks for the sharing.

---

<div class="post-metadata">

**Author:** ![difei](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/a4c791/32.png) [@difei](https://www.ruby-forum.com/u/difei)\
**Post date:** [September 12, 2008, 9:18pm UTC](https://www.ruby-forum.com/t/float-precision/148375/4 "2008-09-12T21:18:31Z")

</div>

thomas peklak wrote:

> The round function in the float class does not provide a precision  
> feature, but you can do something like:  
> (100\*9.23234234234532).round/100.0
> 
> According to my benchmarks this is at least 1/3 faster than your  
> approach
> 
> Thomas

Just out of curiosity, does it make any difference on benchmarks with  
Ruby using division versus multiplication? It used to be much faster to  
multiply by .01 than divide by 100. I don’t know if this has changed  
with newer processors and compilers.

---

<div class="post-metadata">

**Author:** ![difei](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/a4c791/32.png) [@difei](https://www.ruby-forum.com/u/difei)\
**Post date:** [September 15, 2008, 12:26am UTC](https://www.ruby-forum.com/t/float-precision/148375/5 "2008-09-15T00:26:39Z")

</div>

thomas peklak wrote:

> You are right, multiplying with 0.01 is even fast, see the benchmarks:

Thanks for the information. It looks like multiplication is still twice  
as fast as division. I am sure that with today’s computers this only  
matters with large numbers of operations but on a IBM PC-1 it made a lot  
of difference.

---

<div class="post-metadata">

**Author:** ![difei](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/a4c791/32.png) [@difei](https://www.ruby-forum.com/u/difei)\
**Post date:** [September 13, 2008, 11:08am UTC](https://www.ruby-forum.com/t/float-precision/148375/6 "2008-09-13T11:08:49Z")

</div>

You are right, multiplying with 0.01 is even fast, see the benchmarks:

require ‘benchmark’

times = 1000000  
float = 9.234234765765765764  
Benchmark.bm do |r|  
r.report(‘SPf :’) {  
times.times do  
f1 = sprintf(’%.2f’ ,float).to\_f  
end  
}  
r.report(‘_100 :’) {  
times.times do  
f2 = (100 \* float).round/100.0  
end  
}  
r.report(’_.01 :’) {  
times.times do  
f3 = (100 \* float).round\*0.01  
end  
}  
end

```
    user system total real

```

SPf : 5.590000 0.050000 5.640000 ( 5.720373)  
\*100 : 3.760000 0.040000 3.800000 ( 3.829771)  
\*.01 : 1.770000 0.010000 1.780000 ( 1.811917)

Thomas

---

<div class="post-metadata">

**Author:** ![difei](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/a4c791/32.png) [@difei](https://www.ruby-forum.com/u/difei)\
**Post date:** [March 3, 2011, 11:24pm UTC](https://www.ruby-forum.com/t/float-precision/148375/7 "2011-03-03T23:24:02Z")

</div>

Robert K. wrote in post #727777:

> 2008/9/13 thomas peklak [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):
> 
> > You are right, multiplying with 0.01 is even fast, see the benchmarks:
> > 
> > require ‘benchmark’  
> > user system total real  
> > SPf : 5.590000 0.050000 5.640000 ( 5.720373)  
> > \*100 : 3.760000 0.040000 3.800000 ( 3.829771)  
> > \*.01 : 1.770000 0.010000 1.780000 ( 1.811917)
> 
> It’s even more efficient to calculate with the full precision and  
> round only on output. 🙂
> 
> Cheers
> 
> robert

This doesn’t seem to work for certain numbers though… is there a way  
to round this 7.17… e+25:

puts “#{cd}\t#{(100 \* thr.max).round\*0.01}”

output ..

1000 7.17897987691824e+25  
200 3.35  
4000 0.99  
20000 0.69

---

<div class="post-metadata">

**Author:** ![difei](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/a4c791/32.png) [@difei](https://www.ruby-forum.com/u/difei)\
**Post date:** [September 16, 2008, 12:56pm UTC](https://www.ruby-forum.com/t/float-precision/148375/8 "2008-09-16T12:56:18Z")

</div>

2008/9/13 thomas peklak [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):

> You are right, multiplying with 0.01 is even fast, see the benchmarks:
> 
> require ‘benchmark’  
> user system total real  
> SPf : 5.590000 0.050000 5.640000 ( 5.720373)  
> \*100 : 3.760000 0.040000 3.800000 ( 3.829771)  
> \*.01 : 1.770000 0.010000 1.780000 ( 1.811917)

It’s even more efficient to calculate with the full precision and  
round only on output. 🙂

Cheers

robert

---

<div class="post-metadata">

**Author:** ![difei](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/a4c791/32.png) [@difei](https://www.ruby-forum.com/u/difei)\
**Post date:** [April 28, 2011, 10:04am UTC](https://www.ruby-forum.com/t/float-precision/148375/9 "2011-04-28T10:04:16Z")

</div>

Joshua S. wrote in post #995421:

> Not surprisingly, it’s extremely quicker to convert to a string and  
> match a regular expression.
> 
> ```
> user system total real
> 
> ```
> 
> spf : 1.560000 0.000000 1.560000 ( 1.557252)  
> \*100 : 0.320000 0.000000 0.320000 ( 0.322573)  
> \*.01 : 0.350000 0.000000 0.350000 ( 0.343289)  
> regex: 0.000000 0.000000 0.000000 ( 0.000019)

You should be suspicious of a result which indicates it’s 5 orders of  
magnitude faster.

> r.report(‘regex:’) {  
> f4 = float.to\_s.match(/(^-?\d+(.\d{1,2})?)/)[1].to\_f  
> }

Ahem, you forgot the “times.times do…end” loop in the benchmark 🙂

---

<div class="post-metadata">

**Author:** ![difei](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/a4c791/32.png) [@difei](https://www.ruby-forum.com/u/difei)\
**Post date:** [April 28, 2011, 12:32am UTC](https://www.ruby-forum.com/t/float-precision/148375/10 "2011-04-28T00:32:44Z")

</div>

Not surprisingly, it’s extremely quicker to convert to a string and  
match a regular expression.

```
  user system total real

```

spf : 1.560000 0.000000 1.560000 ( 1.557252)  
\*100 : 0.320000 0.000000 0.320000 ( 0.322573)  
\*.01 : 0.350000 0.000000 0.350000 ( 0.343289)  
regex: 0.000000 0.000000 0.000000 ( 0.000019)

The expression  
(^-?\d+(.\d{1,2})?) matches all the following:

12  
1.2  
1.234567890  
12.34567890  
123.4567890  
1234.567890  
-12  
-1.2  
-1.234567890  
-12.34567890  
-123.4567890  
-1234.567890

* * *

require ‘benchmark’

times = 1000000  
float = 9.234234765765765764  
Benchmark.bm do |r|  
r.report(‘spf :’) {  
times.times do  
f1 = sprintf(’%.2f’ ,float).to\_f  
end  
}  
r.report(‘_100 :’) {  
times.times do  
f2 = (100 \* float).round/100.0  
end  
}  
r.report(’_.01 :’) {  
times.times do  
f3 = (100 \* float).round\*0.01  
end  
}  
r.report(‘regex:’) {  
f4 = float.to\_s.match(/(^-?\d+(.\d{1,2})?)/)[1].to\_f  
}  
end

---

<div class="post-metadata">

**Author:** ![difei](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/a4c791/32.png) [@difei](https://www.ruby-forum.com/u/difei)\
**Post date:** [April 28, 2011, 7:37pm UTC](https://www.ruby-forum.com/t/float-precision/148375/11 "2011-04-28T19:37:31Z")

</div>

Brian C. wrote in post #995481:

> Joshua S. wrote in post #995421:
> 
> > Not surprisingly, it’s extremely quicker to convert to a string and  
> > match a regular expression.
> > 
> > ```
> > user system total real
> > 
> > ```
> > 
> > spf : 1.560000 0.000000 1.560000 ( 1.557252)  
> > \*100 : 0.320000 0.000000 0.320000 ( 0.322573)  
> > \*.01 : 0.350000 0.000000 0.350000 ( 0.343289)  
> > regex: 0.000000 0.000000 0.000000 ( 0.000019)
> 
> You should be suspicious of a result which indicates it’s 5 orders of  
> magnitude faster.
> 
> > r.report(‘regex:’) {  
> > f4 = float.to\_s.match(/(^-?\d+(.\d{1,2})?)/)[1].to\_f  
> > }
> 
> Ahem, you forgot the “times.times do…end” loop in the benchmark 🙂

Wow. Yea … scratch that.

```
  user system total real

```

spf : 1.570000 0.000000 1.570000 ( 1.569649)  
\*100 : 0.330000 0.000000 0.330000 ( 0.323864)  
\*.01 : 0.340000 0.000000 0.340000 ( 0.342777)  
regex: 4.120000 0.010000 4.130000 ( 4.127264)
