Is it possible to dynamically extend Test::Unit test cases?

Ruby
1

Hello list,

I’ve got some “normal” test cases that look like this

class Testcases < Test::Unit::TestCase

def setup
…
end
def teardown
…
end
def test_1
…
end
def test_2
…
end
end

The test cases run as expected, but now I need to add some new test
cases at run time. What I think I need to do is something like this

require ‘testcases’
t = Testcases.new
t.send(:define_method, “test_defined_at_run_time”, “”)

but I’m getting “wrong number of arguments (0 for 1)” when I try to
create the instance of Testcases.

Is there some way around this?

Thanks in advance

David M.

2

David M. wrote:

The test cases run as expected, but now I need to add some new test
cases at run time.

Why? Just curious…

What I think I need to do is something like this

Try this:

class MySuite < Test::Unit::TestCase

[:concept_1, :concept_2, :concept_3].each do |thang|
define_method “test_#{thang}” do
assert_concept thang
end
end

def assert_concept(thang)
# now convert the thang symbol to a real thing and test it
end

end

t.send(:define_method, “test_defined_at_run_time”, “”)

You are trying too hard. define_method is easier than that to call.

And note I put most of the processing outside the define_method. Its
only job
is to construct a test case name and pass in a thang. This helps make
assert_concept() more comprehensible.

3

Thanks Phlip,

The reason I need to do this is that we’ve got a small Watir-based DSL
written to allow us to drive an app through code that looks sort of
like:
login(“fred”, “password”)
click_tab(“Reports”)
click_drilldown(“Asia”)
open_report(“Some report title”)
…

It essentially lets us construct test cases in something like plain
English.

We built a few test cases using the DSL with a “normal”
Test::Unit::TestCase approach, then showed them to our testers.
Everyone was pretty excited about it; we can generate our own test
data using the DSL, the testers can comprehend the DSL without having
to dig into the nuts and bolts of the application itself, we can
finally build a full regression test suite for an application that’s
basically a pig to drive using normal automation testing tools like
QTP, the scripts we write using the DSL are easy to maintain over
time, and everyone’s happy.

Once our testers got a look at that, they pointed out what should’ve
been obvious all along: we can now get actual business users to write
a lot of the test cases using the DSL, rather than using specialist
testers. Rather than writing huge business requirements documents
that have a habit of getting misinterpreted, we can get the business
users to create what are essentially test cases using the DSL, and
that gives the developers a reasonably unambiguous description of how
things are supposed to work - we’ll save a whole lot of time and money
we’re currently wasting translating between business-speak and
developer-speak.

The only problem was the “scaffolding” code; apparently business users
are incapable of writing/extending code that looks like

class Testcases < Test::Unit::TestCase
def test_1
<>
end
def test_2
<>
end
…
end

but they are capable of creating a bunch of test cases in individual
files that contain nothing but the DSL commands. They’ll use e.g.
Notepad to create test cases in individual files that look like:
login(‘fred’, ‘password’)
click_tab(‘Reports’)
…

Fine with me - I just work here…

So now I’ve got a situation where we’re going to have business users
generating loads of test cases using our DSL (without any of that
nasty complicated Test::Unit::TestCase stuff), saving them in flat
files, and we need to run be able to run some unspecified number of
test cases that will change over time. What I need to be able to do
is something like:

Dir.glob("app_test_cases/**/*.app).each do |test_script_file|
<<grab the content of each file, build a new Test::Unit::TestCase
wrapper around it and eval it>>
end

That’s no problem - I’ve got most of this working already; all I
needed was the way to dynamically add new test cases, and you’ve now
given me a way to do that. I’ll have a play with it tomorrow.

Thanks again

David M.

2008/7/15 phlip [email protected]:

4

David,

test/spec or shoulda or rspec can be used to provide a more “dsl-ish”
front
end for your tests.

Bret

5

On Jul 15, 7:33 am, Bret P. [email protected] wrote:

[Note: parts of this message were removed to make it a legal post.]

David,

test/spec or shoulda or rspec can be used to provide a more “dsl-ish” front
end for your tests.

Bret

Bret’s right.

I would also argue that the xUnit narrative for workflow tests is
inadequate.

You could object model your AUT

g_mail.goto_url
g_mail.welcome_page.sign_up_for_google_mail_link.click
gmail.create_account_page.first_name_field.set(‘aidy’)
gmail.create_account_page.last_name_field.set(‘lewis’)

example: http://wiki.openqa.org/display/WTR/Example+Frameworks

You can then wrap this up in Rspec story runner tests

acceptance test:

Given a user at gmail
When clicks to create a account
And enters first and second name
Then #…

steps

Given “a user at gmail” do
g_mail.goto_url
end

When “clicks to create a account” do
g_mail.welcome_page.sign_up_for_google_mail_link.click
end

When “enters first and second name” do
gmail.create_account.page.first_name_field.set(‘aidy’)
gmail.create_account.page.last_name_field.set(‘lewis’)
end

…

I will example this and post on Watir site

Aidy

6

Thanks Bret and Aidy

I’ll check out the options you’ve suggested today. I’ve only ever
needed test/unit in the past, so have never bothered to look into
alternate test frameworks.

Regards

David M.

2008/7/16 aidy [email protected]:

7

On Jul 15, 3:19 pm, David M. [email protected] wrote:

...

cases at run time. What I think I need to do is something like this
Thanks in advance

David M.

I took the following test:

require ‘test/unit/ui/console/testrunner’
require “test/unit”
class TestCases < Test::Unit::TestCase
def test_case_1
p “case 1”
end
end
eval “class TestCase2 < TestCases;def test_case_2;p ‘case 2’;end;end;”
Test::Unit::UI::Console::TestRunner.run(TestCases)
Test::Unit::UI::Console::TestRunner.run(TestCase2)

It gives:
Loaded suite TestCases
Started
“case 1”
.
Finished in 0.000433 seconds.

1 tests, 0 assertions, 0 failures, 0 errors
Loaded suite
Started
“case 1”
.“case 2”
.
Finished in 0.001912 seconds.

2 tests, 0 assertions, 0 failures, 0 errors

It works well.and following is the same

require ‘test/unit/ui/console/testrunner’
require “test/unit”
class TestCases < Test::Unit::TestCase
def test_case_1
p “case 1”
end
end
test_case_2 = Class.new(TestCases) do
def test_case_2
p “case 2”
end
end
Test::Unit::UI::Console::TestRunner.run(TestCases)
Test::Unit::UI::Console::TestRunner.run(test_case_2)

if you take a look into the testcase.rb
def initialize(test_method_name)
so,I wrote the following code:

require ‘test/unit/ui/console/testrunner’
require “test/unit”
class TestCases < Test::Unit::TestCase
def test_case_1
p “case 1”
end
def test_case_2
p “case 2”
end
end
test_case_2 = TestCases.new(:test_case_2)
Test::Unit::UI::Console::TestRunner.run(TestCases)
Test::Unit::UI::Console::TestRunner.run(test_case_2)

the result:
Loaded suite TestCases
Started
“case 1”
.“case 2”
.
Finished in 0.00051 seconds.

2 tests, 0 assertions, 0 failures, 0 errors
Loaded suite test_case_2(TestCases)
Started
“case 2”
.
Finished in 0.000314 seconds.

1 tests, 0 assertions, 0 failures, 0 errors

so the initialize function takes a methods name,and only the method is
running if you run the instance
but if you run the Class,the test* methods will all run through the
TestCase.suite [testcase.rb]

I hope this will help

8

Hi Nick

If you haven’t already, you might look at systir [1] – does almost
exactly what you’re describing. It’s a little old, and perhaps not
maintained much, but it could give you an idea of how to do the
test/unit testcase definition more cleanly.

/Nick

[1]:http://www.atomicobject.com/pages/System+Testing+in+Ruby- Hide quoted text -

  • Show quoted text -

The issues I have with the more procedural frameworks is that you are
increasing your code base and thus maintenance.

For example

set_terms_of_use_checkbox
clear_terms_of_use_checkbox

Will wrap the same object in two methods

Whereas this:

gmail.sign_up_page.terms_of_use_checkbox.set
gmail.sign_up_page.terms_of_use_checkbox.clear

Is one object with two different methods operated upon it.

Aidy

9

On Tue, Jul 15, 2008 at 7:39 AM, David M. [email protected]
wrote:

end
Fine with me - I just work here…
wrapper around it and eval it>>
end

That’s no problem - I’ve got most of this working already; all I
needed was the way to dynamically add new test cases, and you’ve now
given me a way to do that. I’ll have a play with it tomorrow.

If you haven’t already, you might look at systir 1 – does almost
exactly what you’re describing. It’s a little old, and perhaps not
maintained much, but it could give you an idea of how to do the
test/unit testcase definition more cleanly.

/Nick

10

On 15 Jul, 22:33, David M. [email protected] wrote:

2008/7/16aidy[email protected]:

g_mail.welcome_page.sign_up_for_google_mail_link.click
When clicks to create a account
g_mail.welcome_page.sign_up_for_google_mail_link.click

Aidy- Hide quoted text -

  • Show quoted text -

I have added an example of the Rspec story runner that is used with
the AUT object model and Watir

http://wiki.openqa.org/display/WTR/Example+Frameworks

Aidy