Test Harness?


#1

I’m starting to feel my way around in Ruby after having been a Perler
for a
very long time. I’m trying to figure out how to write a proper test
harness
in Ruby and so far I’m running up against a brick wall. All my classes
have
good unit tests using Test::Unit. However, I want to create a harness
that
I can use to run all of the unit tests at a go and give me a
success/failure
report across all tests. Every time I make a significant change I have
to
remember to run all my unit tests, which is error prone.

I have written test harnesses in Perl using Test::Harness::Straps that
does
exactly what I want - for Perl. Is there a similar library for Ruby?
If
so, I have yet to find it.

Thanks,

Troy


#2

On Mar 9, 2006, at 10:56 AM, Troy D. wrote:

report across all tests. Every time I make a significant change I
have to
remember to run all my unit tests, which is error prone.

I have written test harnesses in Perl using Test::Harness::Straps
that does
exactly what I want - for Perl. Is there a similar library for
Ruby? If
so, I have yet to find it.

Making a test suite is trivial. Just add a final like “ts_all.rb” to
your test folder that looks something like this:

#!/usr/local/bin/ruby -w

require “test/unit”

require “tc_file_one.rb”
require “tc_file_two.rb”

END

Execute that file to run all of the tests. Hope that helps.

James Edward G. II


#3

perhaps this will also help
http://www.zenspider.com/ZSS/Products/ZenTest/


#4

Troy D. wrote:

I’m starting to feel my way around in Ruby after having been a Perler
for a
very long time. I’m trying to figure out how to write a proper test
harness
in Ruby and so far I’m running up against a brick wall. All my classes
have
good unit tests using Test::Unit. However, I want to create a harness
that
I can use to run all of the unit tests at a go and give me a
success/failure
report across all tests. Every time I make a significant change I have
to
remember to run all my unit tests, which is error prone.

I use rake. Create a Rakefile with the following:


require ‘rake/testtask’

task :default => :test

Rake::TestTask.new(:test) do |t|
t.test_files = FileList[‘test/test_*.rb’]
t.verbose = true
end

The above assumes all your tests are in the test directory and begin
with ‘test_’. Change as needed for your situation.

To invoke, just type

rake

or

rake test

Add other tasks as needed.

(See http://docs.rubyrake.org for more details on rake)


– Jim W.


#5

On 3/9/06, Troy D. removed_email_address@domain.invalid wrote:

so, I have yet to find it.
In addition to the other suggestions you’ve received, there’s always
this simple approach.

Put your code in a lib directory below your project directory.
Put your test code in a test directory.
cd to to your lib directory
run “testrb …/test/*.rb”

I should probably learn how to use Rake though.


#6

On Fri, 2006-03-10 at 01:56 +0900, Troy D. wrote:

I’m starting to feel my way around in Ruby after having been a Perler for a
very long time. I’m trying to figure out how to write a proper test harness
in Ruby and so far I’m running up against a brick wall. All my classes have
good unit tests using Test::Unit. However, I want to create a harness that
I can use to run all of the unit tests at a go and give me a success/failure
report across all tests. Every time I make a significant change I have to
remember to run all my unit tests, which is error prone.

I see you’ve had plenty of suggestions, but heres yet another variation:

$ ruby -e "Dir['test/tc*.rb'].each { |tc| require tc }"
Loaded suite -e
Started
.......................................
Finished in 0.25893 seconds.

39 tests, 101 assertions, 0 failures, 0 errors

That ‘Loaded suite -e’ thing is a shame though…


#7

or, use Rake and a task like:

Rake::TestTask.new(:unittests) do |t|
t.test_files = FileList[‘test/test*.rb’]
t.warning = true
t.verbose = false
end

then when you want to run your tests do:
$ rake unittests
(in /home/eylerpm/notes/ruby/provisioning)
Loaded suite
/usr/lib/ruby/gems/1.8/gems/rake-0.7.0/lib/rake/rake_test_loader
Started

Finished in 2.307442 seconds.

17 tests, 55 assertions, 0 failures, 0 errors
$


#8

On Mar 9, 2006, at 8:56 AM, Troy D. wrote:

report across all tests. Every time I make a significant change I
have to
remember to run all my unit tests, which is error prone.

I have written test harnesses in Perl using Test::Harness::Straps
that does
exactly what I want - for Perl. Is there a similar library for
Ruby? If
so, I have yet to find it.

testrb (which comes with Ruby and Test::Unit) does this for you. You
only need to specify a directory and have your test files named
properly (which is test_*.rb).

$ ls
test/
$ ls test
test_one.rb test_two.rb
$ testrb test
Loaded suite test
Started

Finished in 0.003477 seconds.

2 tests, 2 assertions, 0 failures, 0 errors
$

If you named your tests wrong you can use the -p argument:

$ ls test
one_test.rb two_test.rb
$ testrb test -p /_test.rb/
Loaded suite test
Started

Finished in 0.002296 seconds.

2 tests, 2 assertions, 0 failures, 0 errors


Eric H. - removed_email_address@domain.invalid - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com


#9

On Fri, 2006-03-10 at 08:29 +0900, Daniel N. wrote:

Is there a way to pass Rake tasks arguments?

You can do the easy way:

$ cat Rakefile
task :default do
puts ENV[‘SOMEVAR’]
end

$ rake
(in /home/rosco/dev/ruby/ngtest)
nil

$ rake SOMEVAR=321
(in /home/rosco/dev/ruby/ngtest)
321

Another sneaky thing I’ve done before when embedding rake in other stuff
is to have a task that creates a Rake rule that matches everything, so
that all arguments following that task are grabbed by that rule.

task :create do
  rule '' do |t|
    argument_passed = t.name

    # do stuff with argument...
  end
end

This is only good when you don’t want to have further tasks on the same
commandline, but works pretty nice in some cases e.g.

rake create myproject

I’d like to do something where I break my unit tests up into test
suites by just dropping them into folders and then being able to
invoke a Rake task like rake suite “sweet” or rake suite
“sweet/saccharine”, if I wanted a test suite inside of a test suite.

Or, alternatively, I’d like to be able to invoke test cases named
“sucrose” by rake test_case “sucrose”

The Rake test-task lets you specify the name for the task, so it’s
pretty easy to split up your tests into groups in the Rakefile by simply
defining separate tasks for e.g. functional and unit tests. I often have
a couple of separate tasks, with a single task bringing them together
via task dependencies as an :alltest task.

Also, as others have mentioned, testrb is pretty handy for this stuff:

$ testrb tests/tc*.rb -n ‘/test_ruby_xml.*/’
47 tests, 339 assertions, 0 failures, 0 errors

$ testrb tests/tc*.rb -n ‘/test_ruby_xml_document.*/’
15 tests, 129 assertions, 0 failures, 0 errors

(output trimmed for brevity)


#10

Daniel N. wrote:

Is there a way to pass Rake tasks arguments?

I’d like to do something where I break my unit tests up into test
suites by just dropping them into folders and then being able to
invoke a Rake task like rake suite “sweet” or rake suite
“sweet/saccharine”, if I wanted a test suite inside of a test suite.

Or, alternatively, I’d like to be able to invoke test cases named
“sucrose” by rake test_case “sucrose”

Disclaimer: I’ve read Rake tutorials several times in the past but
have yet to arse myself to start using it (which I should, because I
think I’m starting to exert more effor than is worthwhile keeping
tests organzied without rake), so I might’ve forgotten the part where
it says you can do exactly what I’m asking.

This page: http://rake.rubyforge.org/classes/Rake/TestTask.html pretty
well describes how the built-in test task takes arguments.


– Jim W.


#11

Is there a way to pass Rake tasks arguments?

I’d like to do something where I break my unit tests up into test
suites by just dropping them into folders and then being able to
invoke a Rake task like rake suite “sweet” or rake suite
“sweet/saccharine”, if I wanted a test suite inside of a test suite.

Or, alternatively, I’d like to be able to invoke test cases named
“sucrose” by rake test_case “sucrose”

Disclaimer: I’ve read Rake tutorials several times in the past but
have yet to arse myself to start using it (which I should, because I
think I’m starting to exert more effor than is worthwhile keeping
tests organzied without rake), so I might’ve forgotten the part where
it says you can do exactly what I’m asking.


#12

Thanks to everyone for the ideas on this. I’ve implemented the rake
suggestion, and it mostly does what I’m looking for.

Thanks,

Troy