Forum: RSpec Any way to continue on Assertion Failure

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Gary L. (Guest)
on 2009-05-27 03:35
(Received via mailing list)
Hi,

I wonder if there is any way to tell the test script to continue even
when encountering an assertion failure?  The reason I am asking this is
because I have a test flow that can take a long time to run and it would
be very useful if I can perform all verification at the end of test flow
in one shot rather than fail one and exit immediately.  Of course, I
could break each verification point into separate test case, but then it
will double or triple the overall execution time.  Does RSpec support
this?

Thanks,

--Gary
David C. (Guest)
on 2009-05-27 04:02
(Received via mailing list)
On Tue, May 26, 2009 at 7:33 PM, Gary L. <removed_email_address@domain.invalid> 
wrote:
> Hi,
>
> I wonder if there is any way to tell the test script to continue even when
> encountering an assertion failure?  The reason I am asking this is because I
> have a test flow that can take a long time to run and it would be very
> useful if I can perform all verification at the end of test flow in one shot
> rather than fail one and exit immediately.  Of course, I could break each
> verification point into separate test case, but then it will double or
> triple the overall execution time.  Does RSpec support this?

Nope. RSpec is all about isolated examples of small bits of behaviour.
David C. (Guest)
on 2009-05-27 04:31
(Received via mailing list)
On Tue, May 26, 2009 at 7:58 PM, David C. <removed_email_address@domain.invalid>
wrote:
>
> Nope. RSpec is all about isolated examples of small bits of behaviour.

I don't mean there is no way to do it, btw. Just that RSpec doesn't
offer a way out of the box. Each example is, essentially, run in a
begin/rescue block. RSpec expectation matchers raise errors on
failure.

To get what you're after, you *could* write some sort of wrapper like
this:

def report_later(&block)
  begin
    yield
  rescue Exception => e
    # report the error somewhere but don't raise
  end
end

describe Something do
  it "should do something" do
    report_later do
      # example code with expectations go here
    end
  end
end

If you do this, though, you're really working against the intent of
RSpec. The trick about having a series of expectations within one
example is that if one fails early, you can't really trust the state
of everything that comes after. This is one reason why RSpec is
designed the way it is - to focus on one expectation in each example.

HTH,
David
Julian L. (Guest)
on 2009-05-27 06:43
(Received via mailing list)
On 27/05/2009, at 9:33 AM, Gary L. wrote:

>
Isn't this usual behaviour?

Julian.



----------------------------------------------
Learn: http://sensei.zenunit.com/
Last updated 20-May-09 (Rails, Basic Unix)
Blog: http://random8.zenunit.com/
Twitter: http://twitter.com/random8r
Matt W. (Guest)
on 2009-05-27 12:13
(Received via mailing list)
On 27 May 2009, at 03:38, Julian L. wrote:

>> verification point into separate test case, but then it will double
>> or triple the overall execution time.  Does RSpec support this?
>>
>
> Isn't this usual behaviour?
>
> Julian.

Yeah I'm a little confused by the question (and David's response) so
maybe I've misunderstood something. Normal RSpec test runs will catch
all the test failures and report them at the end of the run. I wonder
whether the OP is talking about a situation where the interaction with
the system takes a long time, so that using lots of examples of the
desired behaviour causes that interaction to be run several times
making the whole run very slow.

I also wonder whether he's just thinking in the test/unit mindset,
which I see a lot, where you have examples like this

<antipattern>
it "should do foo and bar" do
   set_up_state
   do_something_to_system_under_test

   assert_that_foo_was_done
   assert_that_bar_was_done
end
</antipattern>

Here's my normal RSpec flow:

before(:each) do
   set_up_state
   do_something_to_system_under_test
end

it "should have done foo" do
   assert_that_foo_was_done
end

it "should have done bar" do
   assert_that_bar_was_done
end

if set_up_state and/or do_something_to_system_under_test take ages,
you could use a before(:all) block instead, but that comes with
obvious leaky state disadvantages.

Matt W.
http://beta.songkick.com
http://blog.mattwynne.net
Zach D. (Guest)
on 2009-05-31 05:13
(Received via mailing list)
On Wed, May 27, 2009 at 4:07 AM, Matt W. <removed_email_address@domain.invalid> 
wrote:
>>> very useful if I can perform all verification at the end of test flow in one
> I've misunderstood something. Normal RSpec test runs will catch all the test
>  set_up_state
>  do_something_to_system_under_test
>
>  assert_that_foo_was_done
>  assert_that_bar_was_done
> end
> </antipattern>

I don't know that I would call this an antipattern unless set_up_state
is duplicated across examples. I usually don't factor things out of an
example unless it's duplicated in another example (then I extract to a
before) or I'm working in a specific context (which I always give a
before since it provides meaning of the context in an explicit way).

> end
>
> it "should have done bar" do
>  assert_that_bar_was_done
> end

Most of the time I would leave the "do_something_to_system_under_test"
inside each example. I find it easier to read, and a more useful
example (since you don't have to go look else where to see how the
method is called), and it becomes easier to maintain should the object
change in a way that requires a specific pre-condition to be
introduced that aren't needed by the other examples (you don't have to
modify every other example).

another 2 cents to throw in the water fountain,

>
> _______________________________________________
> rspec-users mailing list
> removed_email_address@domain.invalid
> http://rubyforge.org/mailman/listinfo/rspec-users
>



--
Zach D.
http://www.continuousthinking.com (personal)
http://www.mutuallyhuman.com (hire me)
http://ideafoundry.info/behavior-driven-development (first rate BDD
training)
@zachdennis (twitter)
Pat M. (Guest)
on 2009-06-01 23:03
(Received via mailing list)
Let's say I've got this spec:

describe Team, "add_player" do
  it "should add a player to the team" do
    team = Team.new
    player = Player.new :name => "Pat"
    team.add_player player
    team.should have(1).players
    team.players.first.name.should == "Pat"
  end
end

and for some reason
team.should have(1).players
fails.  Do I get any value from seeing the second expectation fail?
What if it passes?

My suggestion is to break up your long running test flow into bits
that you can test individually, that run quickly and independently of
each other.  Get those tested, and have an integration test which
makes sure that everything works together.

Pat
This topic is locked and can not be replied to.