Working outside-in with Cucumber and RSpec

Testing RSpec
1

Hi,

Going to dive into Cucumber soon, and just reading through all the
docs now.

Can anyone point to an example of writing Cucumber features first and
then “when you get down to nitty gritty details, drop down one
abstraction level and use RSpec to write some specs for your
classes” (as per Cucumber wiki)? I’m trying to get a good feeling for
the process of starting with a feature, dropping straight down to
implementing code, and then somehow coming halfway back up and filling
in the specs. Which levels of granularity are we talking about here?
If you write a feature and then write enough code to make the feature
pass, how do the specs still have traction to drive the design?

Cheers,
-Tom

2

There are screencasts on peepcode and also Pat M. has done one,
but I think they may be a bit out of date, and more focussed on just
the mechanics of getting up and running with the old story runner
rather than the workflow once you get going.

Ben M.'s blog[1] has some useful experiences to draw on.

In answer to your question though, if this isn’t too vague, I think
this really sums it up:

Start with a plain-text feature, then implement the steps behind it
one by one. As you implement each step you’ll find you need the system
to do something. Maybe your step says “Given there is a User”. Do you
have a User class yet? Then you’d better make one. Each time you
change the system itself (as opposed to the steps), drive out that
change by writing rspec examples that fail because you haven’t written
the code yet. This is crucial: resist the urge to touch the code until
you have a failing spec. You can then use the usual TDD cycle to
change the classes that make up the system, then finally surface back
up to the feature, run it, and see what you need to do next.

The point of the cucumber features is not to exhaustively cover every
single path in your code - that’s the job of the specs (examples). The
features give you a safety net to tell you that it all fits together.

Does that make any more sense or have I just regurgetated Aslak’s
words from the Cucumber readme?!

[1]http://www.benmabey.com/

3

On Sat, Oct 25, 2008 at 5:12 AM, Matt W. [email protected] wrote:

Start with a plain-text feature, then implement the steps behind it one by
one. As you implement each step you’ll find you need the system to do
something. Maybe your step says “Given there is a User”. Do you have a User
class yet? Then you’d better make one. Each time you change the system
itself (as opposed to the steps), drive out that change by writing rspec
examples that fail because you haven’t written the code yet. This is
crucial: resist the urge to touch the code until you have a failing spec.

At the risk of creating controversy, confusion, etc, I’ve been playing
with the boundaries of this notion of “resisting the urge to touch the
code until you have a failing spec.”

This really needs a blog post, so I’ll try and drum one up, but the
short version is this:

In the past I’ve been very disciplined about following this rule, as
it is something that I learned back in my FitNesse/JUnit days. The
thing is, those two tools are completely decoupled. The FitNesse tests
(which map to cucumber in our ruby/bdd land) could be changed at any
time by any number of non-developer folks. They were also stored
outside of the SCM repo that the code was in.

So there was a risk to manage, which was that if somebody deleted or
changed a FitNesse test, there could be a coverage problem at the code
level, therefore high coverage from JUnit was essential.

These days I work on scenarios w/ customers, but they are ultimately
checked in directly with the rest of the code and run before a push
(I’m doing a lot of solo dev right now, so I’m not using CI since I’m
essentially integrating with every change I make :slight_smile: ). Since it’s all
part of the same code base, I don’t feel the need to create a code
example for User just to bring a User into existence.

Using webrat within cucumber scenarios for a rails app largely
eliminates the need for view specs as well. The combination of these
tools usually provides excellent feedback and, in my view, sufficient
fault isolation, when something falls apart in a view.

Of course, as soon as an object has to do something interesting (other
than mere existence), I’ll usually drop down to rspec. I don’t have a
“rule” for when to do this. I’m just going by feel, and so far have
not been bitten by this approach.

And once we have a disconnected in-browser editor for scenarios, I’m
sure I’ll go back to being more disciplined about starting from a code
example in rspec.

But in the mean time, cucumber steps ARE code examples, and as long as
they live closely bound to the code (in the same SCM repo), I think
it’s OK to consider them your starting point (example first) and not
require an rspec code example just to bring an object into existence.

And that was the short version :slight_smile:

FWIW,
David

4

On 25 Oct 2008, at 14:33, David C. wrote:

On Sat, Oct 25, 2008 at 5:12 AM, Matt W. [email protected]
wrote:

examples that fail because you haven’t written the code yet. This is
crucial: resist the urge to touch the code until you have a failing
spec.

At the risk of creating controversy, confusion, etc, I’ve been playing
with the boundaries of this notion of “resisting the urge to touch the
code until you have a failing spec.”

Crazy talk! Somebody kick this heretic off the list!

:wink:

I actually know what you mean, and found myself this week driving out
what I’d initially intended as a spike from a cucumber scenario, using
rspec examples to shape it later. I think what you’re talking about is
a pretty ‘ri’ technique[1] though - I strongly believe you need to
have experienced full-blown disciplined 100% TDD before you could
honestly and responsibly know when to pull back like you’re describing.

[1]Shuhari - Wikipedia

cheers,
Matt

5

On Sat, Oct 25, 2008 at 9:12 AM, Matt W. [email protected] wrote:

Crazy talk! Somebody kick this heretic off the list!

:wink:

I actually know what you mean, and found myself this week driving out what
I’d initially intended as a spike from a cucumber scenario, using rspec
examples to shape it later. I think what you’re talking about is a pretty
‘ri’ technique[1] though - I strongly believe you need to have experienced
full-blown disciplined 100% TDD before you could honestly and responsibly
know when to pull back like you’re describing.

Excellent point. That’s actually something Kent Beck talks about in
TDD by Example - it is important to first develop the discipline and
skill to do TDD in very, very small steps before you start taking
bigger steps. That way, when you do push the envelope too far (and you
will), you can easily back up a step and work your way through the
problem with a more granular approach.

6

On Oct 25, 2008, at 6:19 pm, David C. wrote:

know when to pull back like you’re describing.

Excellent point. That’s actually something Kent Beck talks about in
TDD by Example - it is important to first develop the discipline and
skill to do TDD in very, very small steps before you start taking
bigger steps. That way, when you do push the envelope too far (and you
will), you can easily back up a step and work your way through the
problem with a more granular approach.

I started this week doing BDD training with someone I’ve worked with
for a while. We held off on the BDD because we knew we had a
greenfield project round the corner.

Cucumber was fine. It takes minutes to knock up a feature file and
steps.

But the next thing we hit was the controller (in Merb, which adds a
bit of overhead while it’s new). This was AGONISINGLY painful. And,
pretty much every time I teach BDD (especially in a web environment)
is the same: the time from me explaining things and writing some code
myself to having a lightbulb come on so they can write the same code
themself is at least one order of magnitude longer than it would take
to write that code (or generate it…) with no specs. This means the
first impression I give of BDD is that it’s frustrating and slow.
Clearly since I’m the common factor in these situations, it’s my fault
it goes wrong…

Anyway this thread made me think of ways round this problem, and I
wondered if anyone had feedback on them.

  • Become a superhumanly good programmer and learn every feature of
    ever framework

Ok, this is a non-starter :slight_smile:

  • Use canned code

This could reduce some typing, but not much, seeing how little you
have to write with the RSpec TextMate bundle. And writing more than
strictly necessary is a bad demo of incremental BDD.

  • Start with pure-Ruby code

I’ve done this before, and it makes it much quicker to that important
keyboard handover. But unfortunately it eats into time that could be
spent “building value”. There are two situations where this is bad:
companies where the reaction is “why are you learning when you could
be coding???”, and companies that have no, or even negative, slack time.

  • Skip the controller specs.

Eeek, more heresy! Can you tell it’s Hallowe’en soon?

But… you can’t break a controller without breaking a Cucumber file,
that would be an Epic Fail at the feature level. And the controllers
shouldn’t be doing more than deciding what response to return, so they
should always be covered fully by Cucumber.

Matt - as it happens, I used a tai chi analogue of Shuhari, “no shape,
no form”, yesterday when I mentioned this to said victim^H^H^H^H^H^H
client. That is, when you have mastered drills, you apply them how
you need them, not in the way they were practised. But I’m scared
that this will give the wrong impression. If I say we’re going to
skip controller specs because they don’t add enough value, how do I
know that won’t be interpreted as “controller specs are too hard,
don’t write specs when it’s hard”?

I know the person I’m working with, and I know he will understand what
I’m saying. But it’s still sending the wrong message.

Any words of wisdom will be much appreciated…

Thanks
Ashley

–
http://www.patchspace.co.uk/

7

On Sat, Oct 25, 2008 at 10:12 AM, Matt W. [email protected] wrote:

[ . . . ] I think what you’re talking about is a pretty
‘ri’ technique[1] though - I strongly believe you need to have experienced
full-blown disciplined 100% TDD before you could honestly and responsibly
know when to pull back like you’re describing.

[1]Shuhari - Wikipedia

Thank you for posting this link. That’s one of the most valuable
things I’ve read all week.

–
Have Fun,
Steve E. ([email protected])
ESCAPE POD - The Science Fiction Podcast Magazine
http://www.escapepod.org

8

If I say we’re going to
skip controller specs because they don’t add enough value, how do I
know that won’t be interpreted as “controller specs are too hard,
don’t write specs when it’s hard”?

I have a three-headed defense of rarely writing controller specs
anymore:

  • Controllers are already exercised by cucumber
  • Most of the logic is provided by the framework.
  • Controllers tend to be procedural instead of OO

The first one should be obvious. As for the second, since I’m not
actually writing most of the code, I’m not that worried about it
breaking.

Procedural code is no fun to test. You have to use a lot of mocks, and
your example looks more or less the same as the production code. Blech.
Rails controllers don’t encapsulate state or iteration, the only
encapsulation-worthy behavior usually is a simple if statement.
Controllers also typically aren’t very cohesive, you’re never going to
call more than one method on them. Put simply, controllers, as used in
Rails, are a collection of transaction scripts grouped together by the
resource on which they operate.

Now, when I don’t write cucumber features, I absolutely do write
controller specs. But I’ll have those hit the db, with integrate_views
turned on, acting essentially as an integration test. The thing is,
there isn’t a lot of value in writing both features and controller
specs, unless your controllers have some pretty complex logic (which
they shouldn’t when using Rails!)

So I guess that were I to be teaching someone this stuff, I would
explain the benefits of testing, and evaluate how controller specs add
value. Specifically, they don’t add regression value over features, and
they provide marginal design value, particularly if you follow typical
Rails controller patterns. I would explain that I don’t see much
benefit to the controller specs, and that they’re kinda tough to test
because they’re not very OO. And I’d use that as an opportunity to
notice smells - when you’re writing specs and it’s difficult, and you
control all the code, then your code could probably use some design
improvements.

But yeah, I like working outside-in :slight_smile:

Pat

9

On Oct 26, 2008, at 12:03 am, Ben M. wrote:

laborious and it becomes extremely painful if you put actual
or been motivated enough to refactor them into the right level.
Again, I can totally understand your point of view and even agree
with it. I think if a team is experienced and disciplined enough
then writing controller specs could be skipped… just keep a look
out for “broken windows” or else the whole controller neighborhood
will start seeping tiny facets of business logic. :slight_smile:

Hmm, after hearing both sides of the argument, I’m inclined to keep
writing controller specs, but to gloss over them in their basic CRUD
form. They add so little value initially, they are a terrible
pedagogical example. Models are much more fun anyway :slight_smile:

Pat’s breakdown,

  • Controllers are already exercised by cucumber
  • Most of the logic is provided by the framework.
  • Controllers tend to be procedural instead of OO

is useful. The last two make me think that the controller code you
write should really look like:

class Items < Application

 # ...

 def show
   Show.new(self, content_type, params).render
 end

 # ...

 class Show
   def initalize(controller, content_type, params)
     @controller = controller
     @content_type = content_type
     @id = id
   end

   def render
     @item = Item.get(params[:id])
     raise NotFound unless @item
     controller.display @item
   end
 end

end

Forgive the unrunnable botched code example :slight_smile:

Has issues with assigning instance variables to the controller, and
not sure what you’d do about the content type (but you could handle it
in a much more OO way. In fact, if the controller strategy was
changed, you could have the show method just return a class:

 def show
   Show
 end

or even do it by convention (look for Items::Show).

This way, you get an isolated bit of code you can spec, without
worrying about all the magic that gets bundled in.

Hmmm, anyway, should get back to writing real code… but I’m going to
keep this at the back of my mind.

So anyway, controller specs it is, just de-emphasised when I’m
teaching BDD.

Thanks people.

Oh, and to an earlier point in this thread… I can’t remember the
last time I wrote a view spec. Any semi-interesting logic that for
some reason needs to be in the view I will place in a helper and
then spec that out.

+1

I found a long time ago they don’t add enough value to justify the
effort in maintaining them. Most of the things you write view specs
for aren’t likely to break due to human error, but are often need
changing due to inconsequential HTML changes.

Cheers
Ashley

–
http://www.patchspace.co.uk/

10

On 26 Oct 2008, at 10:39, Ashley M. wrote:

reasons for doing this is a) allow for interface discovery with
aspect I think they provide. In the past I have been able to
Hmm, after hearing both sides of the argument, I’m inclined to keep
writing controller specs, but to gloss over them in their basic CRUD
form. They add so little value initially, they are a terrible
pedagogical example. Models are much more fun anyway :slight_smile:

I don’t know. I’ve got inheritance in my controllers, for example I
have a MediaController which is subclassed by ImagesController and
VideosController. The specs allowed me to factor out this base class.

I think controllers are OK for doing BDD:

describe “the show action”
describe “when the video is for an artist”
describe “when the video cannot be found”
describe “when the video exists in the database”
describe “when the video is for a concert”

etc.

TBH, I have only just begun to glimpse the idea that cucumber might
mean I don’t need to get 100% coverage from unit-level specs… so I
may just need more time to get my head around the idea.

I guess it also depends on the rest of your team: it sounds as though
David can trust the rest of his team to write well designed code -
it’s just him! Personally I feel I need to set the rest of my team a
good example while they get through the ‘shu’ stage. Pat - are you
going solo too?

cheers,
Matt

11

Pat M. wrote:

  • Most of the logic is provided by the framework.
    Controllers also typically aren’t very cohesive, you’re never going to

But yeah, I like working outside-in :slight_smile:

Pat

rspec-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/rspec-users

I agree with most of this. For a purely RESTful controller I like to
use a plugin, like make_resourceful, that will take care of all the
boilerplate code that becomes very tedious to spec out and write. Since
the plugins are tested I can just rely that coverage alongside my
scenarios going through the full stack. However, once I need to go out
of the ordinary RESTful controller I usually always still write specs
for those actions using mocks. My two reasons for doing this is a)
allow for interface discovery with mocks and b) to prevent logic from
seeping into the controllers. Writing code examples for controllers in
a purely interaction-based way can be laborious and it becomes extremely
painful if you put actual business logic in them. I think that pain is
a good thing because it reminds less experienced people on your team
(and more experienced people too sometimes :slight_smile: ) that controllers
shouldn’t get too big and that they should be pushing the responsibility
into other objects (be that AR or regular ruby objects.)

So while I agree completely that controller specs don’t offer regression
value over scenarios I do place more value on the design aspect I think
they provide. In the past I have been able to detect feature envy and a
number of other code smells within my controllers via my controller
examples that I don’t think I would of noticed and/or been motivated
enough to refactor them into the right level. Again, I can totally
understand your point of view and even agree with it. I think if a team
is experienced and disciplined enough then writing controller specs
could be skipped… just keep a look out for “broken windows” or else
the whole controller neighborhood will start seeping tiny facets of
business logic. :slight_smile:

Oh, and to an earlier point in this thread… I can’t remember the last
time I wrote a view spec. Any semi-interesting logic that for some
reason needs to be in the view I will place in a helper and then spec
that out.

+1 on working outside-in. :slight_smile:

-Ben

12

On Oct 26, 2008, at 9:09 pm, Matt W. wrote:

describe “when the video is for a concert”

etc.

Hmm, what’s the code like behind that? I try to avoid inheritance, or
at least rarely feel the need for it. There are many ways to
implement those specs, and one of them involves a lot of ifs and
cases. (A valid if-statement would probably be for “when the video
cannot be found” and “when the video exists in the database”.)

TBH, I have only just begun to glimpse the idea that cucumber might
mean I don’t need to get 100% coverage from unit-level specs… so I
may just need more time to get my head around the idea.

Me too, especially when multiple content types and authentication
schemes come into play… not got that far in my current projects
though.

I guess it also depends on the rest of your team: it sounds as
though David can trust the rest of his team to write well designed
code - it’s just him! Personally I feel I need to set the rest of my
team a good example while they get through the ‘shu’ stage. Pat -
are you going solo too?

I’m in an unusual situation in this sense. One of my Merb projects is
purely personal, right now. So for that I just want to make my app
work, and the code not to cause me grief. The other is a two day a
week BDD coaching/pair programming development with an otherwise solo
developer. So I have to make the specs damn near perfect, because
it’s all the RSpec he’s seen. Due to Merb still being a bit of an
unknown, that’s making me lean towards neurotic perfectionism in my
own code, at the moment.

And I know just as soon as I’ve made them all perfect, David will
redefine perfection again :slight_smile:

Ashley

–
http://www.patchspace.co.uk/

13

On Oct 27, 2008, at 12:26 am, David C. wrote:

Due to Merb still being a bit of an unknown, that’s making me lean
towards neurotic perfectionism in my own code, at the moment.

And I know just as soon as I’ve made them all perfect, David will
redefine perfection again

Are you calling me neurotic?

Ha, that was unfortunate ordering of sentences on my part! Nah, but
you do have an annoying habit of finding a better way of doing
something just as I’ve got my head round the first :slight_smile:

Ashley

–
http://www.patchspace.co.uk/

14

Sent from my iPhone

On Oct 26, 2008, at 5:09 PM, Ashley M.
[email protected] wrote:

describe “when the video is for an artist”
cannot be found" and “when the video exists in the database”.)

day a week BDD coaching/pair programming development with an
otherwise solo developer. So I have to make the specs damn near
perfect, because it’s all the RSpec he’s seen. Due to Merb still
being a bit of an unknown, that’s making me lean towards neurotic
perfectionism in my own code, at the moment.

And I know just as soon as I’ve made them all perfect, David will
redefine perfection again

Are you calling me neurotic?

15

On Mon, Oct 27, 2008 at 9:35 AM, Pat M. [email protected] wrote:

Matt W. [email protected] writes:

Pat - are you going solo too?

Nope, I’m trying to teach RSpec/BDD to an organization that currently
doesn’t use it and has 0% test coverage. It’s interesting, to say the
least. They’re good devs, but even so, the effects of not writing tests
first (or even at all) are painfully evident when trying to get the
codebase under test, and especially to change stuff. Really puts your
“working effectively with legacy code” chops to work :slight_smile:

One major point which I failed to make is that a heavily determining
factor for me is confidence. How much confidence does a new test
inspire? Clearly, a full-stack acceptance test ought to inspire a
good deal of confidence. A mocked-out unit test considerably less so.

As I’ve said before, I see three major benefits to unit tests:

  • Drive the design of objects
  • Provide regression tests
  • Serve as examples of how to use the code

Mock based controller specs don’t provide as much regression value as
acceptance tests, unless you’ve got a bunch of weird little pathways
that you want to unit test. But most Rails apps these days aren’t
like that - we have REST + “skinny controller / fat model” to thank
for that.

Ben, I think it was you who mentioned that you like using mocks in
controller specs because it helps with interface discovery. While I
agree that mocks can be a great tool for interface discovery, I’m
finding that to be decreasingly important in controllers. They’re
just so damn simple, for the most part. Remember, mocks are just one
of the tools we use to improve design. Refactoring is another one.
I’ve found that when writing controllers, mocks provide little upfront
design value, and the bulk of design value I get comes from
refactoring.

Now, I will add that I’ve written a crapload of controller specs over
the last few years, and have developed a personal style that minimizes
logic in the controllers. So I don’t generally spend a lot of brain
cycles on controller design. They become patterns…think Cypher from
the Matrix: “all I see is blonde, brunette, redhead…”

When it comes to controllers specs, mocks provide the most value by
isolating from the model and db - the specs run faster, and you don’t
have to worry about model validations. But if you minimize controller
logic, you can write acceptance tests that give you confidence that
your controllers work, and then controller specs become unnecessary
overhead.

Pat

16

On Oct 27, 2008, at 5:20 pm, Pat M. wrote:

When it comes to controllers specs, mocks provide the most value by
isolating from the model and db - the specs run faster, and you don’t
have to worry about model validations. But if you minimize controller
logic, you can write acceptance tests that give you confidence that
your controllers work, and then controller specs become unnecessary
overhead.

What’s the consensus here then, controller specs yay or nay?

1 nay from Pat

1 act of fence sitting from me…

Ashley

–
http://www.patchspace.co.uk/

17

On Tue, Oct 28, 2008 at 2:32 AM, Ashley M.
[email protected] wrote:

What’s the consensus here then, controller specs yay or nay?

1 nay from Pat

1 act of fence sitting from me…

I think the nay from Pat was conditional, as it should be.

I think the answer is: if you have to ask, then you should use them :slight_smile:
Otherwise, don’t use them when you feel confident that you don’t need
them.

FWIW,
David

18

On Wed, Oct 29, 2008 at 10:32 PM, David C. [email protected]
wrote:

overhead.
I think the answer is: if you have to ask, then you should use them :slight_smile:
Otherwise, don’t use them when you feel confident that you don’t need
them.

When working outside in, the granularity increases the further in you
get. On the very outside it’s cumbersome to test for edge cases.

Start on the outside with Cucumber and whenever you come across an
edge case, make a judgement on where the right abstraction level is to
test it. That may be controller, view, model or somewhere else.

Don’t write specs just cuz

Aslak

19

Controllers are such a funny thing. Right now they are necessary evil.
Once
you start feeling pain with controllers you start changing your
thinking.
Somehow everyone who starts learning Rails ends up with bloated
controllers
even if they’ve read all of the posts and articles on “skinny
controllers/fat models”.

IME, only then do people start rethinking the path they’ve taken. Now
they
start to roll up their sleeves and find out how they can really make
skinny
controllers and fat models.

So they go write skinny controllers (in terms of what goes inside each
action) and really bloated models. And the cycle repeats itself. Now
they
write skinny controllers (again in terms of what goes inside each
action)
and relatively chunky models. The key here is that they now don’t shove
everything into “one” model. They start using more than one if
necessary.
And sometimes it happens to be an AR object and other times just a PORO.

Lastly, sometime in all of this they get bitten by the dormant monster
that
are controller filters and controller mixins. Sigh, now controller
examples really suck to maintain and write.

It seems like 1 out every 3 Rails developers will move the unnecessary
logic
that makes up these filters and mixins into their own objects, and they
let
the controller just use those objects, rather than having the logic end
up
embedded into the controller itself. This is what I do.

At the end of this cycle folks start writing cookie cutter controller
actions since their controllers really are simple and skinny. The
controller
actions all look like they’ve been generated from the same template. And
so
do our controller specs. Now we can write textmate snippets, generators,
etc. to do the tedious work for us, and we can stop doing it from hand
each
time.

And at this point this is a tell-tale sign that an abstraction is
needed.
Ultimately, more than an abstraction probably. I think we need a shift
in
how we think about mapping incoming requests to real objects doing real
work
is in order. I know that there are plugins like make_resourceful which
try
to eliminate some of pain that comes with redundancy, but this fills a
void
in the current controller implementation in Rails right now. We need
bigger
shifts in thinking to occur IMO. Anyways back on topic…

Mock-based examples will never provide anywhere near the value as a
regression test as an integration test (ie: cucumber features/acceptance
tests). That’s ok. They’re not intended to.

Integration tests will never provide anywhere near the value as design
and
discovery agents as object level examples. That’s ok. They’re not
intended
to.

I agree with Ben M. that about receiving value from having controller
examples. It helps the design of the controller stay simple and clean
over
time. To me this is important, and I value it more than the few minutes
it
takes me to write some controller examples.

I disagree with everyone who doesn’t write view specs. :slight_smile: For me, the
risk
of not having them is too high compared to the few minutes it takes to
spec
a view. Granted, there is overlap between features and view specs, but I
consider that par for the course.

20

“David C.” [email protected] writes:

What’s the consensus here then, controller specs yay or nay?

1 nay from Pat

I think the nay from Pat was conditional, as it should be.

I think the answer is: if you have to ask, then you should use them :slight_smile:
Otherwise, don’t use them when you feel confident that you don’t need
them.

yup