Help interpreting benchmark results

Ruby
1

I ran this simple benchmark:

Benchmark.bm do |x|
x.report(‘Curl::Easy.http_get’) do
Curl::Easy.http_get(text)
end

x.report(‘RestClient.get’) do
RestClient.get(text)
end
end

RestClient uses the standard Net:HTTP library so the benchmark is really
trying to compare Net::HTTP vs curb (libcurl bindings).

These are the results I get:

                              user     system      total        real

Curl::Easy.http_get 0.070000 0.130000 0.200000 ( 26.319104)
RestClient.get 0.970000 0.580000 1.550000 ( 23.557923)

I get these results consistently where curb’s user, system and total
times are dramatically lower than Net::HTTP’s. However, Net:HTTP is
reported to run faster by the real column. Shouldn’t the reported real
time be consistent with the other columns?

I’m running this benchmark on a 2.4Ghz Core 2 Duo with 2GB RAM MacBook
Pro, if that helps.

Thanks,
Juan

2

On 24.02.2009, at 23:35, Juan Alvarez wrote:

time be consistent with the other columns?
No. “real” measures the time that is passed from start to finish.

Pseudocode
real_start = Time.now
yield # run test
real_end = Time.now
real_start.to_f - real_end.to_f

If the Kernel of your OS schedules another processes for, say 20
seconds,
then the difference between total and real is >= 20 seconds.
I think the time called “real” should not be trusted in an evaluation.

regards, Sandor
Szücs

3

2009/2/25 Sandor Szücs [email protected]:

times are dramatically lower than Net::HTTP’s. However, Net:HTTP is

If the Kernel of your OS schedules another processes for, say 20 seconds,
then the difference between total and real is >= 20 seconds.
I think the time called “real” should not be trusted in an evaluation.

Not so fast. If another process has been running that may well be a
process spawned by our process.

13:19:09 Temp$ ruby19 bb.rb
user system total real
other process 0.000000 0.000000 0.139000 ( 10.375000)
in process 4.594000 0.000000 4.594000 ( 4.765000)
13:19:30 Temp$ cat bb.rb
require ‘benchmark’

Benchmark.bm 20 do |b|
b.report ‘other process’ do
system *%w{sleep 10}
end

b.report ‘in process’ do
10_000_000.times {|i| i + 1}
end
end
13:19:35 Temp$

I’d check the curl lib - I would assume that it uses the command line
utility of the same name - which then is another process.

Kind regards

robert

4

On 25.02.2009, at 13:20, Robert K. wrote:

If the Kernel of your OS schedules another processes for, say 20
seconds,
then the difference between total and real is >= 20 seconds.
I think the time called “real” should not be trusted in an
evaluation.

Not so fast. If another process has been running that may well be a
process spawned by our process.

Full ack that’s right, if a process forks, like system() do you can
measure
that one too.

Benchmark.bm(20) do |b|
b.report ‘fork’ do
pid = Process.fork { sleep 3 }
Process.waitpid pid
end
end
user system total real
fork 0.000000 0.010000 0.020000 ( 3.008273)

regards, Sandor
Szücs