Forum: Ruby load testing tool - Please advise?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Cf63da956b6ba955687a2f2f262928cb?d=identicon&s=25 Mmcolli00 Mom (mmcolli00)
on 2009-04-10 18:32
Will you point me in the right direction to which program will be good
for load testing? I have many regression tests that will test for one
user and one ie session at a time, however I am looking for a program
that will work with watir and drive several applications using multiple
concurrent users.

Thanks MC
9dec3df8319c613f6f4f14a27da0fdb4?d=identicon&s=25 Kyle Schmitt (Guest)
on 2009-04-11 03:24
(Received via mailing list)
I'd go straight for JMeter.

WATiR works by driving a browser, and you just aren't really going to
be able to stress test a webapp that way.  Even with several machines
running the tests.

Jmeter works by simulating user sessions, it isn't a whole browser.
It handles sessions and cookies, but other than that just sends http
commands to the server.  Because of it's design, it can easily
generate a lot of traffic from multiple different sessions on a single
machine, and really give the server a working over.

It takes a few days to get really good at, but it's free (open
source), currently maintained, and very _very_ powerful.  It's heavily
java based, and any scripting done in it is though java-esqe scripting
languages, not ruby.  That said, it's easy to make it read values from
csv and xml files for dynamic tests, so you can do a whole lot without
much scripting.

When I worked as a test-developer it's what I used for stress testing,
while relying on WATiR for other testing.

--Kyle
9dec3df8319c613f6f4f14a27da0fdb4?d=identicon&s=25 Kyle Schmitt (Guest)
on 2009-04-11 03:30
(Received via mailing list)
Just one more thought..

JMeter allows you to record sessions.  A few times I recorded the
actions of WATiR tests through JMeter, and then used those as the
basis of some stress tests.  It worked OK, but my WATiR tests didn't
really simulate real user behaviour.  If your tests do, then that may
be a good way of getting going.

--Kyle
D1309ef77561e3cdf0ddcab60b244e52?d=identicon&s=25 Mike Stephens (aspirer)
on 2009-04-11 12:37
Kyle Schmitt wrote:
>
> WATiR works by driving a browser, and you just aren't really going to
> be able to stress test a webapp that way.

Could you expand on that? I would have thought using a browser is the
very best simulation of a user you could have.
147cc8db2770d16f0ffe663be01d598d?d=identicon&s=25 Ghost Steven (kittens)
on 2009-04-11 15:21
Mike Stephens wrote:
> Could you expand on that? I would have thought using a browser is the
> very best simulation of a user you could have.

http://wtr.rubyforge.org/
9dec3df8319c613f6f4f14a27da0fdb4?d=identicon&s=25 Kyle Schmitt (Guest)
on 2009-04-11 15:39
(Received via mailing list)
Well, if you think about it, what does the server care about?  Sending
data, and receiving commands.

As far as it's concerned, there is no difference between firefox opera
and a 15 line ruby script, as long as a program asks  for the same
files, and sends the right commands.

If your goal is to test the web application server under load, that's
what you want.  You don't care if something isn't displayed right, or
if javascript does what it's supposed to, because the server doesn't
care.  You test all of those things: javascripts working right,
rendering, does this drop down show up at the right spot, does this
form work like we expect, etc; using watir or browser based tests.
You test for performance and concurrency problems etc using stress
tests.  For instance, you want to test that 100 users simultaneously
submitting a new recipe to your "biscuit enthusiast" site will
actually take the submissions, and not crash with a mysql error.

It's two different types of tests, using two different types of tools.
Aafa8848c4b764f080b1b31a51eab73d?d=identicon&s=25 Phlip (Guest)
on 2009-04-11 15:45
(Received via mailing list)
Mike Stephens wrote:
> Kyle Schmitt wrote:
>> WATiR works by driving a browser, and you just aren't really going to
>> be able to stress test a webapp that way.
>
> Could you expand on that? I would have thought using a browser is the
> very best simulation of a user you could have.

Watir runs a browser and your web server. Both those systems have
timeouts and
latencies that insulate your website from ultra high use.

Another soak test could use Mechanize (IIRC) to beat your server itself
up via
HTTP. And, of course, you can use unit tests to beat up your code...
7067095abfc198a023c3a9b5e31e67ea?d=identicon&s=25 Peter Booth (peterbooth)
on 2009-04-12 06:46
(Received via mailing list)
My two cents:

Load testing is hard, and often load tests produce results that are
not credible.
Watir isn't right for load testing, and there are many alternatives to
choose from.

There are two key questions:

1. What's the goal of the testing?
  B:  identify Bottlenecks
  C:  estimate Capacity

2. Are you wanting to
  S: isolate a Single page/user operation
or  G: test a Group of pages / paths

BS      -->   all you need is ApacheBenchmark, ab
BG     -->   JMeter, Grinder and Loadrunner all support remote agents,
scripts.
    Grinder is Java based, like JMeter, but configured with jython
scripts
    Loadrunner is commercial, expensive, and visualizes results in a way
that
    can sometimes help you to see patterns quickly.
CG,CS  -->   JMeter, Grinder and Loadrunner produce data that
overestimates throughput.
    This is because they use a closed system to model workload
    where a "user agent" issues a request, and then won't issue a second
    until first is complete. If you want to estimate absolute capacity
then you need
    a workload generator that can drive your system to the point of
overload, and the best
    I know is httperf, an open source product created by HP. httperf
uses an open model to
    drive traffic probabilistically

The best thing about load testing Ruby is "Ruby is slow and Rails
doesn't scale" (irony intentional). With Java its more likely that you
will need five or ten test clients to produce enough traffic to stress
your application. The HP Tech Report on httperf discusses some of the
issues that make it difficult do do effective load testing.

http://www.hpl.hp.com/techreports/98/HPL-98-61.pdf

If your application uses Rails then I'd strongly recommend buying a
New Relic Silver subscription for at least month. It's a great time-
saver when diagnosing performance issues.

Hope this helps,

Peter Booth



I would strongly recommend using
Cf63da956b6ba955687a2f2f262928cb?d=identicon&s=25 Mmcolli00 Mom (mmcolli00)
on 2009-04-13 15:52
> I would strongly recommend using

Using what?
9dec3df8319c613f6f4f14a27da0fdb4?d=identicon&s=25 Kyle Schmitt (Guest)
on 2009-04-13 16:01
(Received via mailing list)
On 4/11/09, Peter Booth <pbooth@nocoincidences.com> wrote:
> My two cents:
>
> Load testing is hard, and often load tests produce results that are
> not credible.

Hear hear
Cf63da956b6ba955687a2f2f262928cb?d=identicon&s=25 Mmcolli00 Mom (mmcolli00)
on 2009-04-13 17:45
> Hear hear

LOL ;-) thanks
This topic is locked and can not be replied to.