Performance testing with cucumber

Rails
1

Hi,

Ive looked around at the various tools for load testing etc but i need
something a bit more specific. Something like cucumber and httperv
mixed together. What i would like to do is write some cucumber specs
for filling in various forms in my application but simulate say 20
users doing this simultaneously. Are there any specific tools out
there or hacks to create something like this?

JB

2

On 10 Jun 2011, at 12:32, johnnybutler7 wrote:

Ive looked around at the various tools for load testing etc but i need
something a bit more specific. Something like cucumber and httperv
mixed together. What i would like to do is write some cucumber specs
for filling in various forms in my application but simulate say 20
users doing this simultaneously. Are there any specific tools out
there or hacks to create something like this?

First of all, running load testing on different methods or a whole
scenario is a waste of time, since it doesn’t give you any indication
on which method is underperforming under high server load. Also,
filling in forms using Selenium (or just simulating it using Cucumber)
will also yield unreliable results (running the code that has nothing
to do with the actual server also takes time). What you want is
targetted performance tests on one particular form url, which
automatically brings us back to httperf.

If you are looking for load testing with form data, httperf supports
this. Just google “httperf post data” and you’ll get plenty of
information. Yes, the way to do it is awful, a nice wrapper around it
would be nice.

Best regards

Peter De Berdt

3

thanks Peter

4

On Jun 10, 11:49am, Peter De Berdt [email protected] wrote:

scenario is a waste of time, since it doesn’t give you any indication
on which method is underperforming under high server load. Also,
filling in forms using Selenium (or just simulating it using Cucumber)
will also yield unreliable results (running the code that has nothing
to do with the actual server also takes time). What you want is
targetted performance tests on one particular form url, which
automatically brings us back to httperf.

Depends on your app. I’ve definitely had cases where testing 1 action
in isolation yielded a good result (ie high number of transactions per
second), and testing a 2nd action in isolation was fine too, but
running both actions at the same time (which is what happened in real
world usage of the app) caused performance to drop through the floor
because of database contention.
For me the key is getting load tests as close as possible to real
world usage. For some apps, replaying log files can work well and has
the advantage that you’re in no danger of forgetting to simulate a
certain type of activity. In a more complicated case, a team I was
once on built a set of automated agents designed to replicate normal
user behaviour. Doing so accurately, and maintaining that accuracy
over time is difficult though. Tools like jmeter or grinder may be of
interest.

Fred

5

On Fri, Jun 10, 2011 at 3:49 AM, Peter De Berdt
[email protected] wrote:

the way to do it is awful, a nice wrapper around it would be nice.
Have you tried http://blitz.io? Comes with a ruby gem so you can
integrate load testing into your continuous deployment.

K.

http://blitz.io
http://twitter.com/pcapr
http://labs.mudynamics.com