Forum: Ruby Help interpreting benchmark results

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
B404a438e7106c61e31fa6ebcc089a5f?d=identicon&s=25 Juan Alvarez (soltikoff)
on 2009-02-24 23:35
I ran this simple benchmark:

Benchmark.bm do |x|
  x.report('Curl::Easy.http_get') do
    Curl::Easy.http_get(text)
  end

  x.report('RestClient.get') do
    RestClient.get(text)
  end
end

RestClient uses the standard Net:HTTP library so the benchmark is really
trying to compare Net::HTTP vs curb (libcurl bindings).

These are the results I get:

                                  user     system      total        real
Curl::Easy.http_get   0.070000   0.130000   0.200000 ( 26.319104)
RestClient.get          0.970000   0.580000   1.550000 ( 23.557923)

I get these results consistently where curb's user, system and total
times are dramatically lower than Net::HTTP's. However, Net:HTTP is
reported to run faster by the real column. Shouldn't the reported real
time be consistent with the other columns?

I'm running this benchmark on a 2.4Ghz Core 2 Duo with  2GB RAM MacBook
Pro, if that helps.

Thanks,
Juan
Cb6bbc826cd7d9238a2fae344958f7ec?d=identicon&s=25 Sandor Szücs (Guest)
on 2009-02-25 12:39
(Received via mailing list)
On 24.02.2009, at 23:35, Juan Alvarez wrote:

> time be consistent with the other columns?
No. "real" measures the time that is passed from start to finish.

Pseudocode
   real_start = Time.now
   yield # run test
   real_end = Time.now
   real_start.to_f - real_end.to_f


If the Kernel of your OS schedules another processes for, say 20
seconds,
then the difference between total and real is >= 20 seconds.
I think the time called "real" should not be trusted in an evaluation.

regards, Sandor
Szücs
E0d864d9677f3c1482a20152b7cac0e2?d=identicon&s=25 Robert Klemme (Guest)
on 2009-02-25 13:22
(Received via mailing list)
2009/2/25 Sandor Szücs <sandor.szuecs@fu-berlin.de>:
>> times are dramatically lower than Net::HTTP's. However, Net:HTTP is
>
>
> If the Kernel of your OS schedules another processes for, say 20 seconds,
> then the difference between total and real is >= 20 seconds.
> I think the time called "real" should not be trusted in an evaluation.

Not so fast.  If another process has been running that may well be a
process spawned by our process.

13:19:09 Temp$ ruby19 bb.rb
                          user     system      total        real
other process         0.000000   0.000000   0.139000 ( 10.375000)
in process            4.594000   0.000000   4.594000 (  4.765000)
13:19:30 Temp$ cat bb.rb
require 'benchmark'

Benchmark.bm 20 do |b|
  b.report 'other process' do
    system *%w{sleep 10}
  end

  b.report 'in process' do
    10_000_000.times {|i| i + 1}
  end
end
13:19:35 Temp$

I'd check the curl lib - I would assume that it uses the command line
utility of the same name - which then is another process.

Kind regards

robert
Cb6bbc826cd7d9238a2fae344958f7ec?d=identicon&s=25 Sandor Szücs (Guest)
on 2009-02-25 21:57
(Received via mailing list)
On 25.02.2009, at 13:20, Robert Klemme wrote:

>>
>> If the Kernel of your OS schedules another processes for, say 20
>> seconds,
>> then the difference between total and real is >= 20 seconds.
>> I think the time called "real" should not be trusted in an
>> evaluation.
>
> Not so fast.  If another process has been running that may well be a
> process spawned by our process.

Full ack that's right, if a process forks, like system() do you can
measure
that one too.

Benchmark.bm(20) do |b|
   b.report 'fork' do
     pid = Process.fork { sleep 3 }
     Process.waitpid pid
   end
end
                           user     system      total        real
fork                  0.000000   0.010000   0.020000 (  3.008273)


regards, Sandor
Szücs
This topic is locked and can not be replied to.