Abstract test runner?


We have a quite large test suite built on Watir (and eventually moving
to our soon-to-be-released JRuby wrapper for HtmlUnit). Currently the
tests are written using Test::Unit, but we are looking at other
solutions - rspec, both spec and story frameworks, and RubyFIT.

I suspect that we will have tests written using more than one solution
(at least initiallly, while still undecided), and I think for some
scenarios (i.e. lots of similar but different input data going through
the exact same testing process) FIT could work really well. But if
we’re using more than one framwork, we don’t want to look at seperate
reports from each framework.

What I’d like to have is some kind of abstract TestRunner that could
execute tests, and provide a common interface to results from the
different frameworks for a reporting tool. This of course limits what
kind of result data is recorded - but at a minimum could be just
passed/failed + the «test name».

This would allow us to build up some infrastructure around running
tests and storing/presenting (non-detailed) results, without being
bound to one framework.

So maybe something like this, using strategies:

t = TestRunner.new(RSpecRunner.new)

(or it could figure out what kind of runner to use from the file
name). Perhaps also with an output strategy:

t = TestRunner.new(RSpecRunner.new, DatabaseOutputter.new)


Having this, we’ll also be able to play with future frameworks and
easily plug them into our «simple reports» by just writing a
«NewFramework»Runner class.

Any interest in this? Perhaps a solution already exists that I’m not
aware of.


This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs