Cucumber - Performance


#1

Hi,

Small question on performance. I’ve just run the CS example provided
with Cucumber and it took about 45-50 seconds before the first output
appeared in the console, but once this first output had appeared the
tests completed quickly.

I’m wondering whether this is the expected performance or whether the
plan is to optimize it at some time in the future?

Ta,

Colin


#2

On Wed, Jan 7, 2009 at 10:31 AM, Colin J. removed_email_address@domain.invalid
wrote:

The reason it’s so slow is that IronRuby is slow. That’s where the
optimisation will have to happen. It runs “fast” on JRuby, MRI and YARV.

Aslak


#3

On 7 Jan 2009, at 09:31, Colin J. wrote:

Hi,

Small question on performance. I’ve just run the CS example provided
with Cucumber and it took about 45-50 seconds before the first
output appeared in the console, but once this first output had
appeared the tests completed quickly.

I’m wondering whether this is the expected performance or whether
the plan is to optimize it at some time in the future?

I normally just use MRI to run Cucumber, but I have tried JRuby a
couple of times and noticed that was a bit slower to start up. I’d
guess it’s something similar with IronRuby - .NET apps are often slow
to start up the first time in my experience, as the JIT compilation
kicks in or whatever magic it is these days.

Have you tried running any other Ruby tools through IronRuby? How do
they perform?

If you think it’s a bug in Cucumber (even if one specific to running
on the IronRuby platform) please raise a ticket at lighthouse[1].
Obviously the more you can do to diagnose the fault, the faster it’s
likely to be fixed.

[1]http://rspec.lighthouseapp.com/projects/16211-cucumber/overview

Matt W.
http://blog.mattwynne.net
http://www.songkick.com


#4

Matt W. wrote:

with Cucumber and it took about 45-50 seconds before the first output
My guess is that it’s the initial start-up that’s slow. This is a
great reason to use something like autotest, which will keep the
process loaded up. I wonder how that hell you could make that work
with C# code though! Watch the DLLs?!
Autotest doesn’t keep the process running - it invokes a new process by
shelling out (Kernel#`)

Scott


#5

On 7 Jan 2009, at 11:46, aslak hellesoy wrote:

I’m wondering whether this is the expected performance or whether
the plan is to optimize it at some time in the future?

The reason it’s so slow is that IronRuby is slow. That’s where the
optimisation will have to happen. It runs “fast” on JRuby, MRI and
YARV.

My guess is that it’s the initial start-up that’s slow. This is a
great reason to use something like autotest, which will keep the
process loaded up. I wonder how that hell you could make that work
with C# code though! Watch the DLLs?!

Matt W.
http://blog.mattwynne.net
http://www.songkick.com


#6

My guess is that it’s the initial start-up that’s slow. This is a great
reason to use something like autotest, which will keep the process loaded
up. I wonder how that hell you could make that work with C# code though!
Watch the DLLs?!

coming late on this one - if this is useful to someone, you can quite
easily watch the dll’s using the built-in FileSystemWatcher
(http://msdn.microsoft.com/en-us/library/system.io.filesystemwatcher.aspx).

cheers,

– Thibaut