Test::Unit : assert_aborts


#1

Is there an existing way to assert that abort is invoked with
Test::Unit. Here’s what I came up with.

Place in “test_helper.rb” for example.


Three redefinitions to be able to assert aborts.

class AbortException < Exception
end

class Test::Unit::TestCase
def assert_aborts(msg_or_pattern)
asserted = false
caught_exception = ‘none’
begin
yield if block_given? # if there is no block, there will not be
any abort
either
rescue AbortException => e
caught_exception = e
if msg_or_pattern.is_a? String
assert_equal msg_or_pattern, e.to_s.sub(/^[a-z_]*: /,’’)
return
end
if msg_or_pattern.is_a? Regexp
assert_match msg_or_pattern, e.to_s
return
end
end
flunk “Expected to handle abort with >>#{ msg_or_pattern }<<. Caught
exception >>#{ caught_exception }<< but didn’t handle”
end
end

module Kernel
def abort(msg)
raise AbortException.new(msg)
end
end

Then in a test

def test_aborting
assert_aborts ‘fatal error encountered’ do
trigger_abort # …
end
end

Stephan


#2

Have you tried

assert_raises(AbortException) { ... }

?


#3

Brian C. wrote:

Have you tried

assert_raises(AbortException) { ... }

?

Thanks – if you are suggesting to leave out the method assert_aborts: I
thought it might be good to check the abort message. Otherwise, please
let me know.

Stephan


#4

?

Thanks – if you are suggesting to leave out the method assert_aborts:
I
thought it might be good to check the abort message. Otherwise, please
let me know.

See assert_raise_message in Test::Unit 2.x.

gem install test-unit

Regards,

Dan


#5

Daniel B. wrote:

?

Thanks – if you are suggesting to leave out the method assert_aborts:
I
thought it might be good to check the abort message. Otherwise, please
let me know.

See assert_raise_message in Test::Unit 2.x.

gem install test-unit

Regards,

Dan

Ok thanks. Should I submit patches to Ruby?

  1. assert_aborts through redefining Kernel#abort
  2. assert_raise_message accepts regular expression.

Stephan


#6

On May 24, 2009, at 4:44 AM, Stephan W. wrote:

let me know.
It’s possible to check the message even in old versions of
Test::Unit. For example:

class SpecificError < RuntimeError; end

require “test/unit”

class TestErrorHandling < Test::Unit::TestCase
def test_error_type_and_message
error = assert_raise(SpecificError) do
raise SpecificError, “Magic message goes here…”
end
assert_match(/magic/i, error.message)
end
end

END

As for testing for abort(), I wouldn’t. What are you really trying to
figure out, if the code would exit with an error message? Then check
that. Throw a StringIO in $stderr and check for a message and see if
Ruby is planning to exit. abort() raises the same Exception exit
does, so just check for that:

require “test/unit”

class TestErrorHandling < Test::Unit::TestCase
def test_error_type_and_message
assert_raise(SystemExit) do
abort “Bye.”
end
end
end

END

Hope that helps.

James Edward G. II


#7

James G. wrote:

On May 24, 2009, at 4:44 AM, Stephan W. wrote:

let me know.
It’s possible to check the message even in old versions of
Test::Unit. For example:

class SpecificError < RuntimeError; end

require “test/unit”

class TestErrorHandling < Test::Unit::TestCase
def test_error_type_and_message
error = assert_raise(SpecificError) do
raise SpecificError, “Magic message goes here…”
end
assert_match(/magic/i, error.message)
end
end

END

As for testing for abort(), I wouldn’t. What are you really trying to
figure out, if the code would exit with an error message? Then check
that. Throw a StringIO in $stderr and check for a message and see if
Ruby is planning to exit. abort() raises the same Exception exit
does, so just check for that:

require “test/unit”

class TestErrorHandling < Test::Unit::TestCase
def test_error_type_and_message
assert_raise(SystemExit) do
abort “Bye.”
end
end
end

END

Hope that helps.

Ok, thanks a lot!

You mean something like this.

require “test/unit”
require ‘stringio’

class TestErrorHandling < Test::Unit::TestCase
def test_error_type_and_message_II
assert_aborts(/bye/i) do
abort “Bye.”
end
end

private

def assert_aborts(pattern)
save_stderr = $stderr
begin
$stderr = StringIO.new
e = assert_raise(SystemExit) do
yield if block_given?
end
assert_match pattern, e.message
ensure
$stderr=save_stderr
end
end
end

END

Stephan

James Edward G. II


#8

James G. wrote:

On May 24, 2009, at 6:17 PM, Stephan W. wrote:

end
end
assert_match pattern, e.message
ensure
$stderr=save_stderr
end
end
end

END

Sort of like that, yeah. Basically what I was saying is that I feel
an assert_aborts() method tests an implementation detail.

It doesn’t really matter if I use abort() or some output method and
then call exit() myself. There may be good reasons to do that too,
say if you are printing a complex error message and using printf()
would make it easier to format.

We really just want to know if the user saw an error and if the
program is quitting, so it’s better to test for that.

Ok, thanks a lot; that makes sense.

Stephan

James Edward G. II


#9

On May 24, 2009, at 6:17 PM, Stephan W. wrote:

end
end
assert_match pattern, e.message
ensure
$stderr=save_stderr
end
end
end

END

Sort of like that, yeah. Basically what I was saying is that I feel
an assert_aborts() method tests an implementation detail.

It doesn’t really matter if I use abort() or some output method and
then call exit() myself. There may be good reasons to do that too,
say if you are printing a complex error message and using printf()
would make it easier to format.

We really just want to know if the user saw an error and if the
program is quitting, so it’s better to test for that.

James Edward G. II


#10

I just found this thread and could only run the code with ruby 1.9.3 but
my
server has 1.8.7 so I needed to tweak it a bit:

require ‘test/unit’
require “stringio”

class TestA < Test::Unit::TestCase
def assert_aborts(message)
e = assert_raises SystemExit do
previous_stderr = $stderr
$stderr = StringIO.new
yield
end
assert_equal message, e.message
ensure
$stderr = previous_stderr
end

def test_a
assert_aborts “hi” do
abort “hi”
end
end
end

Also one can change the assert_equal for an assert_match to use a regex.

Thanks :slight_smile:


#11

On May 23, 2009, at 21:54, Stephan W. wrote:

 if msg_or_pattern.is_a? String

exception >>#{ caught_exception }<< but didn’t handle"
end
end

module Kernel
def abort(msg)
raise AbortException.new(msg)
end
end

You’re doing far, far too much work!

$ ruby -rstringio -e ‘$stderr = StringIO.new; begin; abort “hi”;
rescue Exception; p $!; end’
#<SystemExit: hi>

Capturing $stderr (then ignoring it) and checking if SystemExit is
raised and has the right message is enough:

$ cat test.rb
require ‘minitest/autorun’

class TestA < MiniTest::Unit::TestCase
def assert_aborts(message)
e = assert_raises SystemExit do
capture_io do
yield
end
end

 assert_equal message, e.message

end

def test_a
assert_aborts “hi” do
abort “hi”
end
end
end
$ ruby19 test.rb
Loaded suite test
Started
.
Finished in 0.001654 seconds.

1 tests, 2 assertions, 0 failures, 0 errors, 0 skips