Will you point me in the right direction to which program will be good for load testing? I have many regression tests that will test for one user and one ie session at a time, however I am looking for a program that will work with watir and drive several applications using multiple concurrent users. Thanks MC
on 2009-04-10 18:32
on 2009-04-11 03:24
I'd go straight for JMeter. WATiR works by driving a browser, and you just aren't really going to be able to stress test a webapp that way. Even with several machines running the tests. Jmeter works by simulating user sessions, it isn't a whole browser. It handles sessions and cookies, but other than that just sends http commands to the server. Because of it's design, it can easily generate a lot of traffic from multiple different sessions on a single machine, and really give the server a working over. It takes a few days to get really good at, but it's free (open source), currently maintained, and very _very_ powerful. It's heavily java based, and any scripting done in it is though java-esqe scripting languages, not ruby. That said, it's easy to make it read values from csv and xml files for dynamic tests, so you can do a whole lot without much scripting. When I worked as a test-developer it's what I used for stress testing, while relying on WATiR for other testing. --Kyle
on 2009-04-11 03:30
Just one more thought.. JMeter allows you to record sessions. A few times I recorded the actions of WATiR tests through JMeter, and then used those as the basis of some stress tests. It worked OK, but my WATiR tests didn't really simulate real user behaviour. If your tests do, then that may be a good way of getting going. --Kyle
on 2009-04-11 12:37
Kyle Schmitt wrote: > > WATiR works by driving a browser, and you just aren't really going to > be able to stress test a webapp that way. Could you expand on that? I would have thought using a browser is the very best simulation of a user you could have.
on 2009-04-11 15:21
Mike Stephens wrote: > Could you expand on that? I would have thought using a browser is the > very best simulation of a user you could have. http://wtr.rubyforge.org/
on 2009-04-11 15:39
on 2009-04-11 15:45
Mike Stephens wrote: > Kyle Schmitt wrote: >> WATiR works by driving a browser, and you just aren't really going to >> be able to stress test a webapp that way. > > Could you expand on that? I would have thought using a browser is the > very best simulation of a user you could have. Watir runs a browser and your web server. Both those systems have timeouts and latencies that insulate your website from ultra high use. Another soak test could use Mechanize (IIRC) to beat your server itself up via HTTP. And, of course, you can use unit tests to beat up your code...
on 2009-04-12 06:46
My two cents: Load testing is hard, and often load tests produce results that are not credible. Watir isn't right for load testing, and there are many alternatives to choose from. There are two key questions: 1. What's the goal of the testing? B: identify Bottlenecks C: estimate Capacity 2. Are you wanting to S: isolate a Single page/user operation or G: test a Group of pages / paths BS --> all you need is ApacheBenchmark, ab BG --> JMeter, Grinder and Loadrunner all support remote agents, scripts. Grinder is Java based, like JMeter, but configured with jython scripts Loadrunner is commercial, expensive, and visualizes results in a way that can sometimes help you to see patterns quickly. CG,CS --> JMeter, Grinder and Loadrunner produce data that overestimates throughput. This is because they use a closed system to model workload where a "user agent" issues a request, and then won't issue a second until first is complete. If you want to estimate absolute capacity then you need a workload generator that can drive your system to the point of overload, and the best I know is httperf, an open source product created by HP. httperf uses an open model to drive traffic probabilistically The best thing about load testing Ruby is "Ruby is slow and Rails doesn't scale" (irony intentional). With Java its more likely that you will need five or ten test clients to produce enough traffic to stress your application. The HP Tech Report on httperf discusses some of the issues that make it difficult do do effective load testing. http://www.hpl.hp.com/techreports/98/HPL-98-61.pdf If your application uses Rails then I'd strongly recommend buying a New Relic Silver subscription for at least month. It's a great time- saver when diagnosing performance issues. Hope this helps, Peter Booth I would strongly recommend using
on 2009-04-13 15:52
> I would strongly recommend using Using what?
on 2009-04-13 16:01
On 4/11/09, Peter Booth <firstname.lastname@example.org> wrote: > My two cents: > > Load testing is hard, and often load tests produce results that are > not credible. Hear hear
on 2009-04-13 17:45
> Hear hear LOL ;-) thanks