Why does a test fail when I predicted it's exception class?

[richard]

Hi,

I have a test suite that invokes on test set that specifies

assert_raise(ZeroDivisionError, MyCalc.new.calc)

When I run the test suite, instead of “1 test, 0 failures, 0 errors”,
I get the following. The code follows this output.

What’s up?
Richard

K:_Projects\Ruby\TestUnitTesting\BasicTest>ruby MyCalcTest.rb
.\BasicCalc.rb:5:in /': divided by 0 (ZeroDivisionError) from .\BasicCalc.rb:5:incalc’
from .\BasicCalc.rb:9
from
K:/_Utilities/Ruby_1.8.2-15/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
gem_original_require' from K:/_Utilities/Ruby_1.8.2-15/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:inrequire’
from ./tc_TestSet.rb:1
from
K:/_Utilities/Ruby_1.8.2-15/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:in
gem_original_require' from K:/_Utilities/Ruby_1.8.2-15/ruby/lib/ruby/site_ruby/1.8/rubygems/custom_require.rb:27:inrequire’
from MyCalcTest.rb:4

BasicCalc.rb

BasicCalc.rb

class MyCalc
def calc
2/0
end
end

tc_TestSet.rb

require ‘.\BasicCalc.rb’

class Test1 < Test::Unit::TestCase # The long-winded way
def test_excep
assert_raise(ZeroDivisionError, MyCalc.new.calc)
end
end

MyCalcTest.rb

Reverse-Polish Evaluation Test

require ‘test/unit’
require ‘tc_TestSet’

[richard]

On 12/3/06, Richard [email protected]
wrote:

You are using assert_raise in the wrong way. The call you want to test
must be wrapped in a block so that it’s exception can be catched and
compared to the expected one.

Try something like

assert_raise(RuntimeError) {raise ""}

/Robert

[richard]

On 12/2/06, Richard [email protected]
wrote:

tc_TestSet.rb

require ‘.\BasicCalc.rb’

class Test1 < Test::Unit::TestCase # The long-winded way
def test_excep
assert_raise(ZeroDivisionError, MyCalc.new.calc)
end
end

You need to run the code that raises the exception in a block.

assert_raise(ZeroDivisionError) { MyCalc.new.calc }

should do the trick.

[richard]

Hi Robert & Louis,

You guys were right on! Below is one of my actual test sets, the one
with the exception testing. It is more complicated than the toy I
posted, so it took me a few syntax-errors to get it to work. But then
I got my desired: 17 tests, 17 assertions, 0 failures, 0 errors

Thank you very much for your responses.

Best wishes,
Richard

require ‘.\RevPolishEvaluator.rb’

class ErrorTests < Test::Unit::TestCase # The succinct way
def test1_noarg; assert_raise(ArgumentError) {Polish.new().eval}; end
def test2_badop; assert_raise(ArgumentError) {Polish.new(%w{2 3 $ 7 –
*}).eval}; end
def test3_missingdata; assert_raise(ArgumentError) {Polish.new(%w{2
+}).eval}; end
def test4_divby0; assert_raise(ZeroDivisionError) {Polish.new(%w{2 0
/}).eval}; end
end

[richard]

On Dec 2, 2006, at 20:40 , Richard wrote:

7 –
*}).eval}; end
def test3_missingdata; assert_raise(ArgumentError) {Polish.new(%w{2
+}).eval}; end
def test4_divby0; assert_raise(ZeroDivisionError) {Polish.new(%w{2 0
/}).eval}; end
end

Better:

require ‘test/unit’
require ‘rev_polish_evaluator’

class TestPolish < Test::Unit::TestCase

def test_eval_bad_operator
# …
end

def test_eval_divide_by_zero
# …
end

def test_eval_no_input
e = assert_raise ArgumentError do Polish.new.eval end
assert_equal ‘no arguments supplied’, e.message
end

def test_eval_missing_data
# …
end

end

Written this way your tests:

• follow the test/unit naming scheme (TestBlah where Blah is the
  class you’re testing)

• be in the same order as the output (alphabetical)

• will work with the -n flag:

  $ ruby test_polish.rb -n /eval/

• will work with testrb:

  $ testrb .

–
Eric H. - [email protected] - http://blog.segment7.net

I LIT YOUR GEM ON FIRE!

[richard]

On Dec 3, 2006, at 15:55 , Richard wrote:

def test_eval_bad_operator

1. I take it that to conform to unit-test’s expectations, the test
   names should begin with test_ and not test1_, test2_ etc.

Numbering tests is not typical behavior (I’ve seen it very rarely).
Using #setup and #teardown its easy to avoid the need to number tests.

The only restrictions test/unit has are that the must start with
‘test’ and accept 0 arguments.

2. How about me using test_1_Whatever, test_2_SomethingElse, etc,?
   Wouldn’t that conform equally well and cause any error msgs
   produced by
   Test to be presented in lexicographical order (with fewer than 10
   tests; double digits if I wanted up to 99 tests, etc.)?

I match test methods to implementation methods and test classes to
implementation classes. This allows easy auditing (for example with
ZenTest) or even visually of your test coverage.

def test_foo() end

matches

def foo() end

and if I need to test a couple edge-cases of foo:

def test_foo_empty_foopy() end
def test_foo_no_blah() end

And everything stays in run order.

3. I like to produce tests in one-line format if they’ll fit on my
   screen reasonably so that I can write and scan them more quickly. Do
   see any substantive problem in my continuing to do that?

I don’t care, they’re your tests (but I hate ; so I had to get rid of
them.)

I sometimes write:

def some_method() do_the_stuff end

because the ; is ugly, but almost always for test stub objects.

Most people add newlines.

K:_Projects\Ruby\TestUnitTesting\ReversePolishEvaluator>
====== end ============

I thought the problem might be that I had no errors, so that there was
nothing for Test to report. So I introduced an erroneous assertion
(expressed in three lines rather than my preferred one-line format)
but still got nothing.

You ran your implementation, not your tests.

Then I ran this expanded test through SciTE and got (both in one-line
format as well as three-line format):

==== SciTE output ====

ruby RevPolishEvaluatorTest.rb
Loaded suite RevPolishEvaluatorTest
Started
…F…
Finished in 0.031 seconds.

This one ran with -n

Do you have any idea why the command with the “-n” switch failed on my
system? My code follows.

Try again running your test file

Again, thanks for your suggestions. I look forward to your comments.

Oh, also you can make your test shorter with setup:

=== tc_TestSet2.rb ===
require ‘.\RevPolishEvaluator.rb’

class ErrorTests < Test::Unit::TestCase # The succinct way

def setup() @p = Polish.new end

def test_1_noarg; assert_raise(ArgumentError) {Polish.new().eval}; end

def test_1_noarg() assert_raise ArgumentError do @p.eval end end

(As you can see, I hate punctuation.)

–
Eric H. - [email protected] - http://blog.segment7.net

I LIT YOUR GEM ON FIRE!

[richard]

Hi Eric,

Thanks for weighing in on my question.

def test_eval_no_input
e = assert_raise ArgumentError do Polish.new.eval end
assert_equal ‘no arguments supplied’, e.message
end

I like the additions here a lot:

1. Capturing the exception object
2. Checking not only the correct exception type is raised but also
   checking the programmed message passed to ‘raise’

def test_eval_bad_operator

1. I take it that to conform to unit-test’s expectations, the test
   names should begin with test_ and not test1_, test2_ etc.

2. How about me using test_1_Whatever, test_2_SomethingElse, etc,?
   Wouldn’t that conform equally well and cause any error msgs produced by
   Test to be presented in lexicographical order (with fewer than 10
   tests; double digits if I wanted up to 99 tests, etc.)?

3. I like to produce tests in one-line format if they’ll fit on my
   screen reasonably so that I can write and scan them more quickly. Do
   see any substantive problem in my continuing to do that?

$ ruby test_polish.rb -n /eval/

I’m running Ruby_1.8.2-15 over WinXP-Pro/SP2. I tried this command and
got nothing:

=== Command Window ====
K:_Projects\Ruby\TestUnitTesting\ReversePolishEvaluator>ruby
RevPolishEvaluator.rb -n /eval/

K:_Projects\Ruby\TestUnitTesting\ReversePolishEvaluator>
====== end ============

I thought the problem might be that I had no errors, so that there was
nothing for Test to report. So I introduced an erroneous assertion
(expressed in three lines rather than my preferred one-line format)
but still got nothing.

Then I ran this expanded test through SciTE and got (both in one-line
format as well as three-line format):

==== SciTE output ====

ruby RevPolishEvaluatorTest.rb
Loaded suite RevPolishEvaluatorTest
Started
…F…
Finished in 0.031 seconds.

1. Failure:
   test_5_dummy(ErrorTests) [./tc_TestSet2.rb:12]:
   <1> expected but was
   <2>.

18 tests, 18 assertions, 1 failures, 0 errors

Exit code: 1
=== end ===

Do you have any idea why the command with the “-n” switch failed on my
system? My code follows.

Again, thanks for your suggestions. I look forward to your comments.

Best wishes,
Richard

=== RevPolishEvaluator.rb ===

Code same as posted earlier

=== end ===

=== RevPolishEvaluatorTest.rb ===

Reverse-Polish Evaluation Test

require ‘test/unit’
require ‘tc_TestSet1’ # <= 13 tests that produced no errors;
file not shown here
require ‘tc_TestSet2’
=== end ===

=== tc_TestSet2.rb ===
require ‘.\RevPolishEvaluator.rb’

class ErrorTests < Test::Unit::TestCase # The succinct way
def test_1_noarg; assert_raise(ArgumentError) {Polish.new().eval}; end
def test_2_badop; assert_raise(ArgumentError) {Polish.new(%w{2 3 $ 7
– *}).eval}; end
def test_3_missingdata
e = assert_raise(ArgumentError) {Polish.new(%w{2 +}).eval}
assert_equal(‘Stack underflow’, e.message) #
<= Had to change the msg to confirm to my coding
end
def test_4_divby0; assert_raise(ZeroDivisionError) {Polish.new(%w{2 0
/}).eval}; end
def test_5_dummy
assert_equal(1, 2)
end
end
=== end ===

[richard]

Hi,

> Your answers here solved a problem I posted yesterday about another > messed up test. I have now deleted that post because "I've seen the > light." > > Below is a toy testing setup. I stuck with my numbered approach for > the moment because it fit these tests. This setup worked quite nicely. > If you're in the mood to comment on them, I'd be a grateful recipient. > But I think I'm OK on this topic now. Incidentally, I thought about > trying to use lamda expressions to reduce the duplication, but enough > is enough.

Why do you want to number your tests? Your tests should be written in
such a way that they don’t depend upon the order they run, so
numbering them to try and get a specific order is a bad idea. If its
just to get a nice output when you run the suite, you should look at
tweaking the test runner or autotest or whatever you use to run your
tests.

    @n = n

“TestSetupTeardown”

    @instance1 = Foo.new(1)
    @instance2 = Foo.new(2)

end
def teardown
@instance1 = nil
@instance2 = nil

You don’t need to set these to nil, as a new instance of the test
class will be create for each test case. You normally only see
teardown used for things like closing a database connection, making
sure a stubbed method gets removed, etc.

end
def test1
assert_equal(“I’m Foo#bar with Foo’s Constant1”,
@instance1.bar)
assert_equal(“I’m Foo#bar with Foo’s Constant2”,
@instance2.bar)
end
end

• Rob

[richard]

Hi Erik,

Hi Eric,

Using #setup and #teardown its easy to avoid the need to number tests.

Thanks for mentioning them.

The only restrictions test/unit has are that the must start with
‘test’ and accept 0 arguments.
Excellent

I match test methods to implementation methods and test classes to
implementation classes. This allows easy auditing [snip]
I like that, too.

And everything stays in run order.
Ah, that’s what I didn’t know. I thought my numbering might be
required to achieve that.

I don’t care, they’re your tests [snip]
Great.

$ ruby test_polish.rb -n /eval/
Do you have any idea why the command with the “-n” switch failed on my
system? My code follows.

Try again running your test file
Woops!

Your answers here solved a problem I posted yesterday about another
messed up test. I have now deleted that post because “I’ve seen the
light.”

Below is a toy testing setup. I stuck with my numbered approach for
the moment because it fit these tests. This setup worked quite nicely.
If you’re in the mood to comment on them, I’d be a grateful recipient.
But I think I’m OK on this topic now. Incidentally, I thought about
trying to use lamda expressions to reduce the duplication, but enough
is enough.

Again, thank you for providing such thoughtful comments.

Best wishes,
Richard

SetupTeardown.rb

class Foo
MyConst = “Foo’s Constant”
def initialize(n)
@n = n
end
def bar
“I’m Foo#bar with #{MyConst}#{@n}”
end
end

puts Foo.new(1).bar if FILE == $0

TestSetupTeardown.rb

“TestSetupTeardown”

require ‘test/unit’
require ‘tc_Test1’

tc_Test.rb

require ‘./SetupTeardown’
require ‘test/unit’
class TestST < Test::Unit::TestCase
def setup
@instance1 = Foo.new(1)
@instance2 = Foo.new(2)
end
def teardown
@instance1 = nil
@instance2 = nil
end
def test1
assert_equal(“I’m Foo#bar with Foo’s Constant1”,
@instance1.bar)
assert_equal(“I’m Foo#bar with Foo’s Constant2”,
@instance2.bar)
end
end

Result (using SciTE)

ruby TestSetupTeardown.rb
Loaded suite TestSetupTeardown
Started
.
Finished in 0.0 seconds.

1 tests, 2 assertions, 0 failures, 0 errors

Exit code: 0

[richard]

Hi Robm,

Thanks for adding your views to this discussion.

Why do you want to number your tests? Your tests should be written in
such a way that they don’t depend upon the order they run, so
numbering them to try and get a specific order is a bad idea. If its
just to get a nice output when you run the suite, you should look at
tweaking the test runner or autotest or whatever you use to run your
tests.

I like the numbering so that when I start to develop a slew of tests
for a real app and get a failure(s), I can quickly locate the offending
test in cases where the app is right and the test needs to be
corrected. Maybe when I get some real experience in Test Driven
Development I’ll find that’s not as helpful as I anticipate it to be.

I don’t do it to get the tests to run in any particular order.

def teardown
@instance1 = nil
@instance2 = nil

You don’t need to set these to nil, as a new instance of the test
class will be create for each test case. You normally only see
teardown used for things like closing a database connection, making
sure a stubbed method gets removed, etc.
OK.

Again, thanks for responding

Best wishes,
Richard