Cucumber step definitions vs. RSpec examples


#1

Hi,

So, I’m confused. I’ve been trying to use Cucumber and RSpec, but step
definitions and RSpec examples just seem to be overlapping. What should
go
in either one of these?

For instance, let’s say I’m testing if a form exists on a page. What
goes in
Cucumber and what goes in RSpec?

What I did for now is put Webrat methods in the step definitions, like
fill_in, select, etc, without submitting since I’m not testing
processing
yet; and then put selector matchers in the RSpec examples.

But I have no idea if that’s right. How do you separate these and keep
them
from duplicating one another?

Thanks,
Brandon


#2

On Sat, Mar 28, 2009 at 9:49 AM, Brandon O.
removed_email_address@domain.invalid wrote:

So, I’m confused. I’ve been trying to use Cucumber and RSpec, but step
definitions and RSpec examples just seem to be overlapping. What should go
in either one of these?

This is a subjective judgment call that doesn’t have an absolute
‘right’ answer. Ask a dozen programmers how they test and you’re
likely to get at least three dozen answers.

I believe the purists would say that the duplication is a feature.
The gist of this point of view is that RSpec’s main strength is on the
unit testing level: you want to test that model code works separate
from your controller code, and your controller code separate from your
view code. Then if one fails, you know exactly what class went wrong
and what to fix.

Cucumber’s main strength is on the acceptance or integration level:
you want to make sure it all comes together correctly and creates the
right application behavior, which is very hard to prove with unit
testing alone. But if you only did Cucumber tests, your test to see
if a form exists on a page (to use your example) would be too broad.
If it failed, where do you look first? Is the view broken? The
controller code? Do you even have a model for the view to operate on?
Etc.

So you need to have bot levels. You specify your overall behavior
with Cucumber, then you specify the isolated behaviors of the pieces
to make that work, and then you write your code. That’s the
full-round-trip, complete by-the-RSpec-book BDD answer.

My own view… Well, my view is that I’ve tried that and been driven
nuts by it. Writing specs for controllers and views can take me many
times longer than writing the code. Mocking out the models for
controllers is frustrating and fragile to me, and view specs are just
obvious and tedious. I don’t have any fun writing those specs. And
if I’m not having fun, I find myself not motivated to program.

So I stopped writing those specs, and now I mostly just write model
specs and Cucumber features. It usually doesn’t take that long for me
to look at a failing Cucumber test and figure out what went wrong –
from the failure text, a stacktrace, or sometimes just manually
running the action and eyeballing it. Sometimes, if the controller
action is unusual, I will write a spec for it, but only if my own
instincts tell me it’s warranted. (Or if something failed in an odd
way and I want to make sure that doesn’t happen again.)

That’s my current answer based on past experience. Next month I might
have a different way of doing things. I’m not going to make the case
here that I’m right and you should never write controller/view specs
for ordinary actions. (If nothing else, doing it a bunch of times –
and getting some things wrong, then fixing them – is a great way to
learn how controllers and views really work.)

I think the real answer is to try it a few different ways, decide what
works within your comfort level and leads to good code, and let
yourself experiment. And then, if you develop an effective working
pattern down the road that you think makes good general sense, share
it with the community as one more way of doing things. >8->


Have Fun,
Steve E. (removed_email_address@domain.invalid)
ESCAPE POD - The Science Fiction Podcast Magazine
http://www.escapepod.org


#3

On Sat, Mar 28, 2009 at 1:49 PM, Brandon O.
removed_email_address@domain.invalid wrote:

fill_in, select, etc, without submitting since I’m not testing processing
removed_email_address@domain.invalid
http://rubyforge.org/mailman/listinfo/rspec-users

So far I’ve not found a situation where I want to just test a form’s
existence at the acceptance level - I always just submit the form.
Then I might write a couple RSpec examples to verify the
presence/absence of that form in certain situations.

Occasionally there is duplication between your acceptance and unit
tests - I think that’s okay. Acceptance tests verify the behavior at
the application level, and unit tests generally do a better job of
defect localization. I try to minimize that duplication though, and
frequently I will sacrifice the defect localization and only test that
behavior through Cucumber. Then I go back and write the unit test the
first time I experience a problem and it takes me longer to track down
than it should have :slight_smile:

Pat


#4

testing alone. But if you only did Cucumber tests, your test to see
if a form exists on a page (to use your example) would be too broad.
If it failed, where do you look first? Is the view broken? The
controller code? Do you even have a model for the view to operate on?
Etc.

Thank you very much. That is much more understandable. This made me
realize
I forgot the controller testing for the feature I’m talking about.

from the failure text, a stacktrace, or sometimes just manually
learn how controllers and views really work.)

Yeah, that’s what I’m worried about. I wrote probably 5 examples just to
test the existence of one field and ensure it had the options I wanted.
I’m
worried about it taking way too long. But I do like the security
provided by
extensive unit tests.

Thanks,
Brandon


#5

On Sat, Mar 28, 2009 at 9:49 AM, Brandon O.
removed_email_address@domain.invalid wrote:

Hi,

So, I’m confused. I’ve been trying to use Cucumber and RSpec, but step
definitions and RSpec examples just seem to be overlapping. What should go
in either one of these?

Cucumber is intended to drive out the behaviour of the application and
RSpec is used to drive out the design and behaviour for the components
and objects that make up that application. Cucumber and RSpec are
bound to overlap. They’re providing you insights at different levels
about how the application is expected to be working.

For instance, let’s say I’m testing if a form exists on a page. What goes in
Cucumber and what goes in RSpec?

Does the scenario successfully drive the implementation of the form
and the page its on? If so, I would probably stick with the scenario.
However, if it doesn’t then I drop down to a view spec and spec
whatever else the page needs that the scenario doesn’t cover.

Scenarios usually don’t communicate well the expectations against a
page for showing and hiding information, etc based on permissions or
roles. RSpec works great for this. It provides a wonderful way to
simply and concisely make each of those expectations clear rather than
some strange side-effect of a scenario which is focused on something
else.

What I did for now is put Webrat methods in the step definitions, like
fill_in, select, etc, without submitting since I’m not testing processing
yet; and then put selector matchers in the RSpec examples.

But I have no idea if that’s right. How do you separate these and keep them
from duplicating one another?

At some level duplication will happen, especially in a web app since
the way you verify the application is working is by
POSTing/GETting/etc and verifying output on the page itself AND it’s
also what you have to do when you creating the individual views and
the controllers that make up those pages and respond to those
requests.

You may find that a scenario drives you to make a view with a form and
a controller with a #create action to process it. This may be adequate
and you may be able to confidently move forward without writing a view
spec or a controller spec. I know I have found this it the case many
times.

On the flip side, when something additional or conditional occurs that
is usually a good time to drive that with an example in a spec.

I use my scenarios to drive my app from the outside, and then specs to
drive the implementation for each of the underlying pieces. The more
I’ve practiced the easier and more natural it has become to know when
to do or not to do something. Also, the only way to not have something
feel like it’s taking too long is to know your tools. These things
won’t just come because you want it to. It takes practice.
Remember to practice.

http://www.vimeo.com/3756344


Zach D.
http://www.continuousthinking.com
http://www.mutuallyhuman.com


#6

On Sun, Mar 29, 2009 at 3:47 PM, Stephen E. removed_email_address@domain.invalid wrote:

testing alone. But if you only did Cucumber tests, your test to see
My own view… Well, my view is that I’ve tried that and been driven
running the action and eyeballing it. Sometimes, if the controller

I think the real answer is to try it a few different ways, decide what
works within your comfort level and leads to good code, and let
yourself experiment. And then, if you develop an effective working
pattern down the road that you think makes good general sense, share
it with the community as one more way of doing things. >8->

I like a lot of what Steve is saying. To summarize and communicate in
my own way…

It is good to know how and when to do something. Like writing view
specs and controller specs. If you don’t know how then you’ll never
know when. Sometimes learning to take small steps seems ridiculous,
but it’s by being able to work in small steps that you can be
confident about making larger ones.

Here are three questions I ask myself when I get frustrated:

1 - Am I doing it wrong?
2 - Should I be doing this in the first place?
3 - Am I experiencing a skills deficiency?

Usually I think it’s #1 and then I quickly jump to #2 when I don’t get
it resolved in 5 minutes of googling. And then I find out a day later
it was because of #3 and I needed to go home and practice a little
more so I better understood what I was doing, why I was doing it, and
how I could leverage the tools to make it easier.

We all learn to make good decisions by first making bad ones, and
practice lets us do that w/o putting giant turds in an application’s
production codebase.


Zach D.
http://www.continuousthinking.com
http://www.mutuallyhuman.com


#7

On Mon, Mar 30, 2009 at 7:08 AM, Brandon O.
removed_email_address@domain.invalid wrote:

For me, it’s either TDD or BDD, honestly. I’ve experienced the assurance
having tests for nearly every line of code affords me, and don’t want to
forgo that. But I like the semantics of BDD a lot better, after reading the
RSpec book.

Interesting. I don’t think what you’re saying is invalid, but I do
think it’s an unusual perspective to consider TDD and BDD as an
either/or. Depending on whom you ask, BDD is most often seen either
as an evolution/elaboration on TDD, or else the differences are viewed
as purely semantic.

Either way, though, by practicing BDD you’re generally doing
everything you’d do in TDD. You’re still specifying
success-vs.-failure conditions before you write the code, and then
writing only the code necessary to achieve success. Whether you call
those conditions ‘behaviors’ or ‘tests’ doesn’t change that
fundamental pattern.


Have Fun,
Steve E. (removed_email_address@domain.invalid)
ESCAPE POD - The Science Fiction Podcast Magazine
http://www.escapepod.org


#8

specs and controller specs. If you don’t know how then you’ll never
Usually I think it’s #1 and then I quickly jump to #2 when I don’t get
it resolved in 5 minutes of googling. And then I find out a day later
it was because of #3 and I needed to go home and practice a little
more so I better understood what I was doing, why I was doing it, and
how I could leverage the tools to make it easier.

We all learn to make good decisions by first making bad ones, and
practice lets us do that w/o putting giant turds in an application’s
production codebase.

Thanks for that explanation.

For me, it’s either TDD or BDD, honestly. I’ve experienced the assurance
having tests for nearly every line of code affords me, and don’t want to
forgo that. But I like the semantics of BDD a lot better, after reading
the
RSpec book.

Thanks,
Brandon


#9

as an evolution/elaboration on TDD, or else the differences are viewed
as purely semantic.

Either way, though, by practicing BDD you’re generally doing
everything you’d do in TDD. You’re still specifying
success-vs.-failure conditions before you write the code, and then
writing only the code necessary to achieve success. Whether you call
those conditions ‘behaviors’ or ‘tests’ doesn’t change that
fundamental pattern.

Yeah I guess I didn’t express that quite right, and I see what you mean.
Either way, I mean I have to do some kind of test driven development, in
whatever form it is.

Brandon