(rspec2/rails3) spec'ing the details of a controller that is not purely *skinny* by design

Testing RSpec
1

Looks like I didn’t read the instructions before posting…
I’m hoping that the “moderator” is /dev/null - otherwise this might
come up as a repost.
Anyway.

I want my controller to set certain attributes on #create and #update
(mostly created_by, updated_by… etc.) – implementing this is easy.
Testing it before implementing it has been nearly impossible for me.

As a side note: I have put this logic into the controller on purpose.
I do believe in the fat model/skinny controller paradigm but believe
that for this situation, it’s the controllers responsibility for a
variety of reasons. Not the least of which being the controller has
access to the session and knows who’s doing what to whom. I could just
put the current_user_id in a hidden field on the forms but I can’t
help but think that would lead to tampering… Anyway.

My controller does something like this for create
@object = Object.new(params[:object])
@object.created_by = current_user
if @object.save… yadda yadda yadda…

I have my model spec testing for the presence of created_by but I feel
compelled to test that my controller is correctly setting current_user
on its appropriate actions. Does this seem reasonable? If so, what’s
the proper way to test it? I tried testing first and though my
failings ended up with an implementation first but that’s neither here
nor there.

Any insight?
Thanks,

  • FJM
2

El 08/07/2010, a las 18:17, Frank J. Mattia
escribió:

My controller does something like this for create
@object = Object.new(params[:object])
@object.created_by = current_user
if @object.save… yadda yadda yadda…

I have my model spec testing for the presence of created_by but I feel
compelled to test that my controller is correctly setting current_user
on its appropriate actions. Does this seem reasonable? If so, what’s
the proper way to test it?

Seems reasonable to me. You could test with either an interaction based
approach (use mocks and stubs to confirm that @object receives the
“created_by” message with the expected param) or a state based approach
(hit the controller action and then inspect the system state afterwards
to confirm that it is in the expected state).

Interaction based approach:

  • may seem like more work due to the set-up of the mocks and stubs

  • runs faster because you can make it so that your controller test
    doesn’t even hit the database

  • may cause you to replicate internal implementation details in your
    tests

  • may shield you from finding out about actual breakage in your system
    because it insulates your controller from the actual API on the real
    mocked/stubbed class

State based approach:

  • may seem like less work due to easier set-up

  • may run slower, which can be important if you have an enormous spec
    suite

  • helps focus your specs on externally observable behavior rather than
    internal implementation details

  • couples your controller tests to your actual model instances, so
    breakage in your models will cause more than just model specs to fail

Personally I prefer a state based approach in general, but I look at
things on a case-by-case basis. Generally if I pick the wrong one I soon
find out about it because the specs I end up writing are
stomach-churning.

And there is a feedback loop here too: if both of these approaches
seem difficult, then it is a sign that the code you’re testing could
probably be rethought to make it more easily testable.

Cheers,
Wincent

3

Seems reasonable to me. You could test with either an interaction based approach (use mocks and stubs to confirm that @object receives the “created_by” message with the expected param) or a state based approach (hit the controller action and then inspect the system state afterwards to confirm that it is in the expected state).
What I was trying to do on my own felt a lot like what you’re calling
an interaction based approach. I want to leave the behavior testing of
my app to cuke features and I’m thinking that if I’m writing a
controller spec, then it would probably be right to test the internals
of my controllers. (As an aside, I’ve only been using RSpec for ~3
weeks now and testing in general for maybe a month. All of this is
pretty new to me… I find myself quickly becoming ‘opinionated’ about
things so if any of this sounds like nonsense, a pointer in a good
direction would be happily accepted.)

Here’s a stripped down excerpt from my controller spec. I know how
Order.should_receive(:method) works, but how do I translate that to an
instance variable? I’ve tried assigns(:order).should_receive to no
avail. Also, where do I put it? Above the post line won’t work because
@order doesn’t exist but after doesn’t work either because my action
has already been run.

describe OrdersController, :type => :controller do
context “when a user who is authenticated” do
before(:each) do
sign_in_user
end
context “requests POST /orders” do
context “with valid parameters” do
before(:each) do
Order.stub(:new) { mock_order(:save => true) }
end

    it "should explicitly set created_by" do
      post :create
    end
  end
end

end
end

I’m still reading the docs and will make some more attempts to get it
right. Thank you for your help.

Cheers,

  • FJM
4 
  sign_in_user
  end
end

end
end

it “should explicitly set created_by” do
controller.stub(:current_user) { mock_user }
mock_order.should_receive(:created_by=).with(mock_user)
post :create
end

This is my newly working spec. Does this look well thought out or is
there some glaring pitfall to doing it this way?

Cheers,

  • FJM
5

of my controllers.

I wouldn’t say that “behavior” is only for Cucumber. Back when Cucumber
hadn’t even been imagined yet, RSpec was created as a “Behavior-Driven
Development” library. Behavior has always been central to what we do
with RSpec, even at the lowest levels.

I think an important question to ask yourself when writing specs is
“Why am I doing this?”, and that leads on to “What am I interested
in specifying here?”. For me it’s all about behavior.

In retrospect, that’s absolutely right. I think what I should have
said was I want separation of granularity in my behaviors. The model
being the most fine grained and the features being the most coarse -
with just enough overlap as to not leave any holes but not so much as
to make senseless duplication.

The specs have two purposes, then: to document/show the behavior (when read),
and to confirm/test the behavior (when executed). If I find myself writing
a test that neither shows nor tests behavior, then I probably shouldn’t be writing it.

That’s been a good barometer for me too.

This is one of the costs of the interaction-based approached, and you have to weigh up that cost against the benefits.

Out of curiosity, how would you do this with a state based approach?
I’ve tried checking the value of created_by after post :create but my
assign(:order) has no idea what :created_by is.
assigns(:order).created_by.should eq(mock_current_user) just doesn’t
do what I thought it would.

Cheers,

  • FJM
6

El 09/07/2010, a las 04:25, Frank J. Mattia
escribió:

Seems reasonable to me. You could test with either an interaction based approach (use mocks and stubs to confirm that @object receives the “created_by” message with the expected param) or a state based approach (hit the controller action and then inspect the system state afterwards to confirm that it is in the expected state).

What I was trying to do on my own felt a lot like what you’re calling
an interaction based approach. I want to leave the behavior testing of
my app to cuke features and I’m thinking that if I’m writing a
controller spec, then it would probably be right to test the internals
of my controllers.

I wouldn’t say that “behavior” is only for Cucumber. Back when Cucumber
hadn’t even been imagined yet, RSpec was created as a “Behavior-Driven
Development” library. Behavior has always been central to what we do
with RSpec, even at the lowest levels.

I think an important question to ask yourself when writing specs is
“Why am I doing this?”, and that leads on to “What am I interested
in specifying here?”. For me it’s all about behavior. The specs have
two purposes, then: to document/show the behavior (when read), and to
confirm/test the behavior (when executed). If I find myself writing a
test that neither shows nor tests behavior, then I probably shouldn’t be
writing it.

So, with the focus always on the behavior of the action, you can proceed
to do either interaction-based or state-based testing, depending on what
feels right for each particular case. As I mentioned in my earlier
email, there are costs and benefits of each approach, and long running
debates about which method to use when, but I think the important thing
is to always try to use the best tool for the job at hand.

(As an aside, I’ve only been using RSpec for ~3
weeks now and testing in general for maybe a month. All of this is
pretty new to me… I find myself quickly becoming ‘opinionated’ about
things so if any of this sounds like nonsense, a pointer in a good
direction would be happily accepted.)

I think the best way to learn is to do what you’re already doing:
writing specs (lots of them), thinking about what you’re doing, and
when you have doubts asking questions or looking for better ways. As
time goes by you’ll not only get better at it, but your thinking will
probably evolve too. I know I’ve gone back and forth many times over the
years, and sometimes I get a surprise when I stumble over an old blog
post of mine on testing and see what I posted.

I think if you look at what the most influential people in the
Ruby/RSpec testing community have been saying over the years, you’ll see
how their ideas have been metamorphosing and changing over the years
too. It’s a natural part of thoughtful testing, I think.

El 09/07/2010, a las 06:07, Frank J. Mattia
escribió:

it “should explicitly set created_by” do
controller.stub(:current_user) { mock_user }
mock_order.should_receive(:created_by=).with(mock_user)
post :create
end

This is my newly working spec. Does this look well thought out or is
there some glaring pitfall to doing it this way?

Well, this is the way mocking and stubbing works. If you want to set an
expectation on an object that doesn’t exist yet at the time you’re
setting up the spec, you have to “chain” things so as to inject a mock
of your own at the right place, which you’ve done here. The amount of
work you have to do setting this up will vary from case to case.

This is one of the costs of the interaction-based approached, and you
have to weigh up that cost against the benefits.

Cheers,
Wincent

7

  • I use factories heavily (in this example Factory Girl) to make it easier to write tests. I don’t worry about the slowness of using real model instances until the spec suite actually becomes large enough for it to be a problem. I also don’t worry about the risk of cascading failures (broken model breaks controller spec) because I value the simplicity that comes with keeping mocking and stubbing to a minimum in the specs, and I am wary of the insulation that mocking and stubbing bring and can actually hide real failures (eg. API changes but spec keeps passing anyway). I still do mock and stub, but generally only where it is not easier to just verify state (and it often is easier).

    …

The controller/action is treated as a “black box” which I never look inside. Each “it” block basically just follows this pattern:

  1. Feed params into the black box
  2. Check one piece of state afterwards

By trying to keep only one “should” assertion in each “it” block I get nice granularity in the event of failure.

It is not “better” nor “the right” way, it is just “a” way of doing it. I’ve also written controller specs where I ended up mocking left, right and center, but if I can take the state-based approach I generally prefer it. The app I’m currently working on has about 3,000 examples, and the suite runs fast enough that I don’t yet feel the need to change my emphasis away from state-based testing.

Thanks for all your help. I’ve been toying around with a few different
approaches to these two ways and I’m starting to make sense of it all.
I too have been using Factory Girl (after spending a day creating my
own factory implementation and not knowing what it was). I can’t say
I’m sold on rr but it does look promising. After trying mocha for all
of 32 seconds I just couldn’t take it (any_instance made me twitch)
and went back to rspec mocking & stubbing… I’ll keep it in the back
of my mind for a day that I’m feeling experimental.

Anyway, thanks again. You’ve been a huge help.

Cheers,

  • FJM
8

El 09/07/2010, a las 14:29, Frank J. Mattia
escribió:

This is one of the costs of the interaction-based approached, and you have to weigh up that cost against the benefits.

Out of curiosity, how would you do this with a state based approach?
I’ve tried checking the value of created_by after post :create but my
assign(:order) has no idea what :created_by is.
assigns(:order).created_by.should eq(mock_current_user) just doesn’t
do what I thought it would.

Ok, some qualifications:

  • I use factories heavily (in this example Factory Girl) to make it
    easier to write tests. I don’t worry about the slowness of using real
    model instances until the spec suite actually becomes large enough for
    it to be a problem. I also don’t worry about the risk of cascading
    failures (broken model breaks controller spec) because I value the
    simplicity that comes with keeping mocking and stubbing to a minimum in
    the specs, and I am wary of the insulation that mocking and stubbing
    bring and can actually hide real failures (eg. API changes but spec
    keeps passing anyway). I still do mock and stub, but generally only
    where it is not easier to just verify state (and it often is easier).

  • I use RR for mocking because I like the syntax. Even in a state-based
    approach, you’ll see I have to stub out the “current_user” method on the
    controller.

  • I use a “before” block that sets up @user, @params, and does the
    stubbing, so that I can reuse the same stuff in many different “it”
    blocks.

  • And I try to keep each “it” block as short as possible, just do the
    post then inspect the state afterwards. In this case the state that I am
    checking for is the externally visible stuff like what got assigned
    (assigns), what got rendered (render_template) or redirected
    (redirect_to), what flash or cookies got set, and so on. If need be, I
    can query the database.

  • And finally, note that this is written for Rails 3/RSpec 2, which I’ve
    been using solidly for the last month and have now forgotten what Rails
    2/RSpec 1 specs looked like!

So with all that said, this is more or less what my state-based approach
would look like:

before do
@user = User.make!
stub(controller).current_user { @user }
@params = { :post => { :title => ‘foo’, :body => ‘bar’ } }
end

it ‘should set created_by’ do
post :create, @params
assigns[:post].created_by.should == @user
end

…

The controller/action is treated as a “black box” which I never look
inside. Each “it” block basically just follows this pattern:

  1. Feed params into the black box
  2. Check one piece of state afterwards

By trying to keep only one “should” assertion in each “it” block I get
nice granularity in the event of failure.

It is not “better” nor “the right” way, it is just “a” way of doing it.
I’ve also written controller specs where I ended up mocking left, right
and center, but if I can take the state-based approach I generally
prefer it. The app I’m currently working on has about 3,000 examples,
and the suite runs fast enough that I don’t yet feel the need to change
my emphasis away from state-based testing.

Cheers,
Wincent

9

El 10/07/2010, a las 05:29, Phillip K.
escribió:

It is not “better” nor “the right” way, it is just “a” way of doing it. I’ve also written controller specs where I ended up mocking left, right and center, but if I can take the state-based approach I generally prefer it. The app I’m currently working on has about 3,000 examples, and the suite runs fast enough that I don’t yet feel the need to change my emphasis away from state-based testing.

I’m curious, Wincent, what you consider to be “fast enough”? You have about twice as many examples as I have in a project, and I’m starting to think that it’s taking too long to run the whole suite. I do tons of mocking and stubbing (and grow very weary of it), but can’t imagine how long my suite would take if I didn’t. Currently, if I run with RCov, it’s in the 6-7 minute range. Without RCov, it’s between 5-6 depending on whether it’s running in Ruby 1.8.7 or 1.9.1 (1.9.1 being faster). I am using PostgreSQL in testing, though, so I know that’s adding a bit. I was using SQLite in testing until recently, and that was still up over 3-4 minutes. [I switched to PostgreSQL in testing so I could take advantage of PostgreSQL specific features. There is zero chance that this app will ever need to run on anything other than PG, so portability is not a concern.] There are a couple of tests that take quite a bit of time (one generates a PDF and verifies that it is where it is supposed to be and another creates a bunch of records to calculate statistics for), but even without them, the suite is taking over three minutes.

So how long do your 3k examples take?

About 4 minutes for the entire thing, so, yes, the suite as a whole is
far from providing “instant feedback”. The vast majority of the
examples, however, run very very fast. I’d say that 80% of the time is
due to the slowest 20% of the specs. The turn-around time for individual
specs (eg. change a model, run model specs; change a controller, run
controller specs) is obviously quite fine.

I think the biggest speed gain wouldn’t be from doing more mocking and
stubbing, but actually from swapping in an in-memory database instead of
MySQL. Not sure how hard that would be, to be honest. (Wondering if it’s
possible to run get MySQL to run from a RAM disk, seeing as I
unfortunately do have some MySQL-specific stuff in there that prevents
me from swapping away from it for testing.)

Slowest specs, of course, are the acceptance ones (running through
Steak/Capybara/Culerity/Celerity). This app is quite an old one written
before the days of Cucumber and Story Runner, so the acceptance coverage
is way behind the rest of the specs. I expect that once I catch up on
the acceptance side it will get much, much, much slower. I can see
myself with 4 minute runs of the whole suite minus the acceptance specs,
followed by 10 or 20 minute runs of the acceptance specs.

Cheers,
Wincent

10

On 2010-07-09 8:38 AM, Wincent C. wrote:

This is one of the costs of the interaction-based approached, and you have to weigh up that cost against the benefits.

 stub(controller).current_user { @user }

The controller/action is treated as a “black box” which I never look inside. Each “it” block basically just follows this pattern:

  1. Feed params into the black box
  2. Check one piece of state afterwards

By trying to keep only one “should” assertion in each “it” block I get nice granularity in the event of failure.

It is not “better” nor “the right” way, it is just “a” way of doing it. I’ve also written controller specs where I ended up mocking left, right and center, but if I can take the state-based approach I generally prefer it. The app I’m currently working on has about 3,000 examples, and the suite runs fast enough that I don’t yet feel the need to change my emphasis away from state-based testing.

I’m curious, Wincent, what you consider to be “fast enough”? You have
about twice as many examples as I have in a project, and I’m starting to
think that it’s taking too long to run the whole suite. I do tons of
mocking and stubbing (and grow very weary of it), but can’t imagine how
long my suite would take if I didn’t. Currently, if I run with RCov,
it’s in the 6-7 minute range. Without RCov, it’s between 5-6 depending
on whether it’s running in Ruby 1.8.7 or 1.9.1 (1.9.1 being faster). I
am using PostgreSQL in testing, though, so I know that’s adding a bit. I
was using SQLite in testing until recently, and that was still up over
3-4 minutes. [I switched to PostgreSQL in testing so I could take advantage of PostgreSQL specific features. There is zero chance that this app will ever need to run on anything other than PG, so portability is not a concern.] There are a couple of tests that take quite a bit of
time (one generates a PDF and verifies that it is where it is supposed
to be and another creates a bunch of records to calculate statistics
for), but even without them, the suite is taking over three minutes.

So how long do your 3k examples take?

Peace,
Phillip

11

El 10/07/2010, a las 12:21, Wincent C.
escribió:

I think the biggest speed gain wouldn’t be from doing more mocking and stubbing, but actually from swapping in an in-memory database instead of MySQL. Not sure how hard that would be, to be honest. (Wondering if it’s possible to run get MySQL to run from a RAM disk, seeing as I unfortunately do have some MySQL-specific stuff in there that prevents me from swapping away from it for testing.)

So I thought I’d try setting this up based on notes found here (nothing
Rails specific here, post is written by a Django developer):

http://kotega.com/blog/2010/apr/12/mysql-ramdisk-osx/

Alas, the before-and-after timings don’t bring the kind of speed up I
was hoping for:

normal disk

real 2m51.720s
user 2m30.692s
sys 0m2.653s

ram disk

real 2m42.268s
user 2m31.590s
sys 0m2.499s

It’s possible that one could get better performance by tuning some MySQL
parameters, but I am sceptical about the gains being very impressive.

Cheers,
Wincent

12

Awesome discussion you’ve started here, very fun read for a new rspec
follower.

Just wanted to throw in that I’ve found setting a
“cattr_accessor :current_user” in the user model to hold the current
user to be very handy in terms of simplifying controllers.

-Kevin

13

On 12 Jul 2010, at 06:43, Kevin wrote:

Awesome discussion you’ve started here, very fun read for a new rspec
follower.

Just wanted to throw in that I’ve found setting a
“cattr_accessor :current_user” in the user model to hold the current
user to be very handy in terms of simplifying controllers.

-Kevin

Probably not the best idea if you’re planning on having the app run on
more than one thread or process though :slight_smile:

cheers,
Matt

http://blog.mattwynne.net
+44(0)7974 430184