Any way to continue on Assertion Failure


#1

Hi,

I wonder if there is any way to tell the test script to continue even
when encountering an assertion failure? The reason I am asking this is
because I have a test flow that can take a long time to run and it would
be very useful if I can perform all verification at the end of test flow
in one shot rather than fail one and exit immediately. Of course, I
could break each verification point into separate test case, but then it
will double or triple the overall execution time. Does RSpec support
this?

Thanks,

–Gary


#2

On Tue, May 26, 2009 at 7:33 PM, Gary L. removed_email_address@domain.invalid wrote:

Hi,

I wonder if there is any way to tell the test script to continue even when
encountering an assertion failure? The reason I am asking this is because I
have a test flow that can take a long time to run and it would be very
useful if I can perform all verification at the end of test flow in one shot
rather than fail one and exit immediately. Of course, I could break each
verification point into separate test case, but then it will double or
triple the overall execution time. Does RSpec support this?

Nope. RSpec is all about isolated examples of small bits of behaviour.


#3

On Tue, May 26, 2009 at 7:58 PM, David C. removed_email_address@domain.invalid
wrote:

Nope. RSpec is all about isolated examples of small bits of behaviour.

I don’t mean there is no way to do it, btw. Just that RSpec doesn’t
offer a way out of the box. Each example is, essentially, run in a
begin/rescue block. RSpec expectation matchers raise errors on
failure.

To get what you’re after, you could write some sort of wrapper like
this:

def report_later(&block)
begin
yield
rescue Exception => e
# report the error somewhere but don’t raise
end
end

describe Something do
it “should do something” do
report_later do
# example code with expectations go here
end
end
end

If you do this, though, you’re really working against the intent of
RSpec. The trick about having a series of expectations within one
example is that if one fails early, you can’t really trust the state
of everything that comes after. This is one reason why RSpec is
designed the way it is - to focus on one expectation in each example.

HTH,
David


#4

On 27/05/2009, at 9:33 AM, Gary L. wrote:

Isn’t this usual behaviour?

Julian.


Learn: http://sensei.zenunit.com/
Last updated 20-May-09 (Rails, Basic Unix)
Blog: http://random8.zenunit.com/
Twitter: http://twitter.com/random8r


#5

On Wed, May 27, 2009 at 4:07 AM, Matt W. removed_email_address@domain.invalid wrote:

very useful if I can perform all verification at the end of test flow in one
I’ve misunderstood something. Normal RSpec test runs will catch all the test
set_up_state
do_something_to_system_under_test

assert_that_foo_was_done
assert_that_bar_was_done
end

I don’t know that I would call this an antipattern unless set_up_state
is duplicated across examples. I usually don’t factor things out of an
example unless it’s duplicated in another example (then I extract to a
before) or I’m working in a specific context (which I always give a
before since it provides meaning of the context in an explicit way).

end

it “should have done bar” do
assert_that_bar_was_done
end

Most of the time I would leave the “do_something_to_system_under_test”
inside each example. I find it easier to read, and a more useful
example (since you don’t have to go look else where to see how the
method is called), and it becomes easier to maintain should the object
change in a way that requires a specific pre-condition to be
introduced that aren’t needed by the other examples (you don’t have to
modify every other example).

another 2 cents to throw in the water fountain,


rspec-users mailing list
removed_email_address@domain.invalid
http://rubyforge.org/mailman/listinfo/rspec-users


Zach D.
http://www.continuousthinking.com (personal)
http://www.mutuallyhuman.com (hire me)
http://ideafoundry.info/behavior-driven-development (first rate BDD
training)
@zachdennis (twitter)


#6

On 27 May 2009, at 03:38, Julian L. wrote:

verification point into separate test case, but then it will double
or triple the overall execution time. Does RSpec support this?

Isn’t this usual behaviour?

Julian.

Yeah I’m a little confused by the question (and David’s response) so
maybe I’ve misunderstood something. Normal RSpec test runs will catch
all the test failures and report them at the end of the run. I wonder
whether the OP is talking about a situation where the interaction with
the system takes a long time, so that using lots of examples of the
desired behaviour causes that interaction to be run several times
making the whole run very slow.

I also wonder whether he’s just thinking in the test/unit mindset,
which I see a lot, where you have examples like this

it "should do foo and bar" do set_up_state do_something_to_system_under_test

assert_that_foo_was_done
assert_that_bar_was_done
end

Here’s my normal RSpec flow:

before(:each) do
set_up_state
do_something_to_system_under_test
end

it “should have done foo” do
assert_that_foo_was_done
end

it “should have done bar” do
assert_that_bar_was_done
end

if set_up_state and/or do_something_to_system_under_test take ages,
you could use a before(:all) block instead, but that comes with
obvious leaky state disadvantages.

Matt W.
http://beta.songkick.com
http://blog.mattwynne.net


#7

Let’s say I’ve got this spec:

describe Team, “add_player” do
it “should add a player to the team” do
team = Team.new
player = Player.new :name => “Pat”
team.add_player player
team.should have(1).players
team.players.first.name.should == “Pat”
end
end

and for some reason
team.should have(1).players
fails. Do I get any value from seeing the second expectation fail?
What if it passes?

My suggestion is to break up your long running test flow into bits
that you can test individually, that run quickly and independently of
each other. Get those tested, and have an integration test which
makes sure that everything works together.

Pat