# Help interpreting benchmark results

**URL:** <https://www.ruby-forum.com/t/help-interpreting-benchmark-results/161273>\
**Category:** Ruby\
**Created:** [February 24, 2009, 11:35pm UTC](https://www.ruby-forum.com/t/help-interpreting-benchmark-results/161273 "2009-02-24T23:35:57Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![alce](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/f07891/32.png) [@alce](https://www.ruby-forum.com/u/alce)\
**Post date:** [February 24, 2009, 11:35pm UTC](https://www.ruby-forum.com/t/help-interpreting-benchmark-results/161273/1 "2009-02-24T23:35:57Z")

</div>

I ran this simple benchmark:

Benchmark.bm do |x|  
x.report(‘Curl::Easy.http\_get’) do  
Curl::Easy.http\_get(text)  
end

x.report(‘RestClient.get’) do  
RestClient.get(text)  
end  
end

RestClient uses the standard Net:HTTP library so the benchmark is really  
trying to compare Net::HTTP vs curb (libcurl bindings).

These are the results I get:

```
                              user system total real

```

Curl::Easy.http\_get 0.070000 0.130000 0.200000 ( 26.319104)  
RestClient.get 0.970000 0.580000 1.550000 ( 23.557923)

I get these results consistently where curb’s user, system and total  
times are dramatically lower than Net::HTTP’s. However, Net:HTTP is  
reported to run faster by the real column. Shouldn’t the reported real  
time be consistent with the other columns?

I’m running this benchmark on a 2.4Ghz Core 2 Duo with 2GB RAM MacBook  
Pro, if that helps.

Thanks,  
Juan

---

<div class="post-metadata">

**Author:** ![alce](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/f07891/32.png) [@alce](https://www.ruby-forum.com/u/alce)\
**Post date:** [February 25, 2009, 12:39pm UTC](https://www.ruby-forum.com/t/help-interpreting-benchmark-results/161273/2 "2009-02-25T12:39:47Z")

</div>

On 24.02.2009, at 23:35, Juan Alvarez wrote:

> time be consistent with the other columns?  
> No. “real” measures the time that is passed from start to finish.

Pseudocode  
real\_start = Time.now  
yield # run test  
real\_end = Time.now  
real\_start.to\_f - real\_end.to\_f

If the Kernel of your OS schedules another processes for, say 20  
seconds,  
then the difference between total and real is \>= 20 seconds.  
I think the time called “real” should not be trusted in an evaluation.

regards, Sandor  
Szücs

---

<div class="post-metadata">

**Author:** ![alce](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/f07891/32.png) [@alce](https://www.ruby-forum.com/u/alce)\
**Post date:** [February 25, 2009, 1:22pm UTC](https://www.ruby-forum.com/t/help-interpreting-benchmark-results/161273/3 "2009-02-25T13:22:14Z")

</div>

2009/2/25 Sandor Szücs [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):

> > times are dramatically lower than Net::HTTP’s. However, Net:HTTP is
> 
> If the Kernel of your OS schedules another processes for, say 20 seconds,  
> then the difference between total and real is \>= 20 seconds.  
> I think the time called “real” should not be trusted in an evaluation.

Not so fast. If another process has been running that may well be a  
process spawned by our process.

13:19:09 Temp$ ruby19 bb.rb  
user system total real  
other process 0.000000 0.000000 0.139000 ( 10.375000)  
in process 4.594000 0.000000 4.594000 ( 4.765000)  
13:19:30 Temp$ cat bb.rb  
require ‘benchmark’

Benchmark.bm 20 do |b|  
b.report ‘other process’ do  
system \*%w{sleep 10}  
end

b.report ‘in process’ do  
10\_000\_000.times {|i| i + 1}  
end  
end  
13:19:35 Temp$

I’d check the curl lib - I would assume that it uses the command line  
utility of the same name - which then is another process.

Kind regards

robert

---

<div class="post-metadata">

**Author:** ![alce](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/f07891/32.png) [@alce](https://www.ruby-forum.com/u/alce)\
**Post date:** [February 25, 2009, 9:57pm UTC](https://www.ruby-forum.com/t/help-interpreting-benchmark-results/161273/4 "2009-02-25T21:57:43Z")

</div>

On 25.02.2009, at 13:20, Robert K. wrote:

> > If the Kernel of your OS schedules another processes for, say 20  
> > seconds,  
> > then the difference between total and real is \>= 20 seconds.  
> > I think the time called “real” should not be trusted in an  
> > evaluation.
> 
> Not so fast. If another process has been running that may well be a  
> process spawned by our process.

Full ack that’s right, if a process forks, like system() do you can  
measure  
that one too.

Benchmark.bm(20) do |b|  
b.report ‘fork’ do  
pid = Process.fork { sleep 3 }  
Process.waitpid pid  
end  
end  
user system total real  
fork 0.000000 0.010000 0.020000 ( 3.008273)

regards, Sandor  
Szücs