Controller testing and mocks


#1

So far I’ve been doing my controller testing against “real” (testing)
objects; however, I’ve been beginning to wonder if I should “default” to
using mocks in most cases – both for performence and to enforce a
looser coupling.

In part this is driven by my viewpoint that rail’s “functional” tests
are still “unit” tests, so they should be tightly focused on the
controller and not on the layers below it. Under this philosophy, it
should not be hitting the DB, etc.

Now I’m more use to mocks from EasyMock in Java or SimpleTest in PHP
which are both of the dynamic mock generation approach, rather than the
built-in rails hand-code approach. These other approaches I think make
it slightly easier to have widespread use of simple/custom mocks; and I
can see how attempting to do the same in Rails (without additional
plugins) might be a mistake.

Using a tool like Test::Rails from the ZenTest gem would seem to push
even stronger towards the mock’d approach as the view and controller
tests are seperated and thus more easily recongnizeable as unit tests.

How have other people apprached this issue? Do you mock with abandon in
the controller tests, or only when absolutely required, or have you
found some sweet-spot between the two extremes that you can share?

Eric


#2

On Jun 14, 2006, at 7:25 AM, Eric D. Nielsen wrote:

How have other people apprached this issue? Do you mock with
abandon in
the controller tests, or only when absolutely required, or have you
found some sweet-spot between the two extremes that you can share?

I use mocks to replace methods that cause problems during testing due to
their activity being inappropriate for testing, or won’t force the code
to journey through the code path I’d like to test (think error
handling).

A good example of this is credit card processing. You probably don’t
want to hit you processing each time you test, and you’d probably like
to test error handling if they’re unavailable. With Mocks you can
replace the method that actually calls out to them, and have the new
one respect an attribute that you can set in the test that causes the
mock method to return error codes.

As for testing other layers via functional tests, how could that be
avoided? Even in unit tests, I don’t have very many methods that don’t
depend on other methods I’ve written, so it’s very rare when tests
only run through a single method.


– Tom M.


#3

provides food rules of thumb (at least for me!) to decide to go mock or
not:

sub(‘food’,‘good’) !!!


#4

I found out that this paper provides food rules of thumb (at least for
me!)
to decide to go mock or not:
http://www.pragmaticprogrammer.com/articles/may_02_mock.pdf

Use mock when…
The real object has nondeterministic behavior.
The real object is difficult to set up.
The real object has behavior that is hard to trigger (for example, a
network
error).
The real object is slow.
The real object has (or is) a user interface.
The test needs to ask the real object about how it was used (for
example, a
test might need to check to see that a callback function was actually
called).
The real object does not yet exist.

Thibaut


#5

I have a test that grabs signups in the last week, aggregated by day. If
I go
against a database, I have to fix the database in time and fool the
system
into thinking the current date is different from today. However, to test
the
aggregation, I need to have a fair amount of data.

How is this scenario best handled? Mocks?

Thanks,

Steve

View this message in context:
http://www.nabble.com/Controller-testing-and-mocks…-t1786480.html#a4869333
Sent from the RubyOnRails Users forum at Nabble.com.


#6

Steve R. wrote:

I have a test that grabs signups in the last week, aggregated by day. If
I go
against a database, I have to fix the database in time and fool the
system
into thinking the current date is different from today. However, to test
the
aggregation, I need to have a fair amount of data.

How is this scenario best handled? Mocks?

Thanks,

Steve

It depends :). If its a controller based action, I’ld probably use
mocks (but that’s what I would have done in different languages, I
started this thread to learn more about how rails developers tend to use
them…) If you’re talking about testing the model level code that
generates the list for the controller/view, then I’d use dynamic
fixtures.


#7

Tom M. wrote:

On Jun 14, 2006, at 7:25 AM, Eric D. Nielsen wrote:

How have other people apprached this issue? Do you mock with
abandon in
the controller tests, or only when absolutely required, or have you
found some sweet-spot between the two extremes that you can share?

I use mocks to replace methods that cause problems during testing due to
their activity being inappropriate for testing, or won’t force the code
to journey through the code path I’d like to test (think error
handling).

A good example of this is credit card processing. You probably don’t
want to hit you processing each time you test, and you’d probably like
to test error handling if they’re unavailable. With Mocks you can
replace the method that actually calls out to them, and have the new
one respect an attribute that you can set in the test that causes the
mock method to return error codes.

I agree that both your examples above (forcing hard to force error
conditions or tests depending on third party network services) are
places where mocks are absolutely required.

As for testing other layers via functional tests, how could that be
avoided? Even in unit tests, I don’t have very many methods that don’t
depend on other methods I’ve written, so it’s very rare when tests
only run through a single method.

Well in some sense it comes back to the “Humble Dialog” approach used
for testing user-interfaces in either web or desktop applications.
Assuming you’ve pushed functionality to “lowest” level (Model <
Controller < View) the tests required at the higher levels become
successively more trivial.

Tightly focused unit tests will often only exercise a single method, but
even if they don’t they shouldn’t be exercising code “too far” removed
from the object under test. I often use the “Law of Demeter” as a
guideline for determining the appropriate “reach” of testing.

As to how this can be acheived, lets take the example of a simple
“login” action in some controller:

If following TDD, we’d normally drive the development with two tests:

  1. If the login fails, did we do the appropriate thing
  2. If the login succeeds, did we do the appropriate thing.

Lets assume that the User model will handle logins. As a client of the
User we don’t care how they authenticate, only that they do so. As a
result a mock of User with configurable return values (Mock as Actor)
can easily sub-in for the actual User without a need for hitting the
database.

So we could use a Mock like (hypthetical code, not tested for actual
run-ability)
class User #note does not extend ActiveRecord::Base
def self.set_expected_return_for_login(value)
@@return_val = value
end
def login(username, password)
@@return_val
end
end

Now we only use the “published” api relevant to us at our controller
test. (Using class variables to avoid issues with figuring out a
good/simple way to handle the mock injection in this case.)
With test code like

def successful_login
User.set_expected_return_for_login(true)
post :action, {:username=>‘foo’,:password=>‘bar’)
assert_redirected_to :where_ever
assert …
end

As a result you force the path you want to test; you don’t rely on the
details of how the lower layer generates the response. Of course, most
mock frameworks have improved support to automate the process of setting
return values/expectations on methods and do some checking to make sure
that the method being knocked out exists in the production code and that
the parameter lists match, etc. Thus the mock can detect if you api
changes in many cases.

Of course you also have your integration/acceptence tests that also help
to catch places where you mocks masked an api change (say if you flipped
the order of two arguements in a function call)

Taken to an extreme this would lead to the controller tests being
de-coupled from the database and never needing to use the DB, fixtures,
etc…


#8

On Wed, Jun 14, 2006 at 10:29:26AM -0700, s.ross wrote:

I have a test that grabs signups in the last week, aggregated by day. If I go
against a database, I have to fix the database in time and fool the system
into thinking the current date is different from today. However, to test the
aggregation, I need to have a fair amount of data.

How is this scenario best handled? Mocks?

Dynamic fixtures, as mentioned. An example:

my_drifting_object:
id: 3
name: Fred Nerk

36 hours ago

signup_date: <%= (Time.now() - 60 * 60 * 36).strftime(’%Y-%m-%d
%H:%M:%S’) %>

The only issue I’ve had with this is that, if you want to test boundary
conditions, there can be a few seconds between when this fixture data is
inserted and when your test runs, so if you set your signup_date to the
last
second you want to catch, it might end up dropping off the list if your
test
is a bit slow in running.

  • Matt

#9

On Wed, Jun 14, 2006 at 04:25:24PM +0200, Eric D. Nielsen wrote:

In part this is driven by my viewpoint that rail’s “functional” tests
are still “unit” tests, so they should be tightly focused on the
controller and not on the layers below it. Under this philosophy, it
should not be hitting the DB, etc.

So you’re going to mock out your model objects for your functional
tests?
In the words of a famous computer, “DANGER! Danger, Will Robinson!”.
This
policy is almost certainly going to cause you Pain and Suffering, as
your
models will change, but since your controllers are tested using mocks
which
don’t change, your controllers will still test OK but will fail when
actually used.

A good rule of thumb is: Mock as little as possible. Never, ever, mock
anything you don’t absolutely have to, because you have to then maintain
two
codebases in parallel, and if they get out of sync you’re stuffed.

Now I’m more use to mocks from EasyMock in Java or SimpleTest in PHP
which are both of the dynamic mock generation approach, rather than the
built-in rails hand-code approach.

I did the hand-code thing for a while, but I yearned for SimpleTest’s
call-chain verification, so now I’m using Test::Unit::MockObject. It is
a
fantastic mock object library – dynamic as all hell, like Ruby itself,
and
not nearly as verbose as the Python mock library I used a while ago
(whose
name I have mercifully forgotten). I like T::U::MockObject so much I
added
a couple of major improvements to it – I’ve sent them upstream, but a
new
release hasn’t come out yet. Let me know if you’d like my modified
version
in the interim.

How have other people apprached this issue? Do you mock with abandon in
the controller tests, or only when absolutely required, or have you
found some sweet-spot between the two extremes that you can share?

Mocks are essentially an acknowledgement that you’ve failed in your
quest
for perfect testing. I’m strong enough to acknowledge my failures, but
I
still don’t like failing more often than I have to.

  • Matt

#10

Matthew P. wrote:

On Wed, Jun 14, 2006 at 04:25:24PM +0200, Eric D. Nielsen wrote:

In part this is driven by my viewpoint that rail’s “functional” tests
are still “unit” tests, so they should be tightly focused on the
controller and not on the layers below it. Under this philosophy, it
should not be hitting the DB, etc.

So you’re going to mock out your model objects for your functional
tests?
In the words of a famous computer, “DANGER! Danger, Will Robinson!”.
This
policy is almost certainly going to cause you Pain and Suffering, as
your
models will change, but since your controllers are tested using mocks
which
don’t change, your controllers will still test OK but will fail when
actually used.

Well it doesn’t have to be as dangerous as you point out (but it can
be). In most of my previous non-Rails based projects, I’ve used a more
mapping based ORM layer. Thus I can use “real” model objects, while
using a mock for the ORM to avoid the database access overhead. I
haven’t figured out how/if to apply this to ActiveRecord style
Object/Relational systems, as the database access is intrinsic to the
class.

As mentioned in the my OP, I feel that the Law of Demeter should cover
these situations – if using a test double to replace an object at two
removes from the class under test lets you get into a situation where
you have passing unit tests, but failing integration/acceptence tests,
that just means you are missing unit tests on the class mediating in the
delegation.

I’ll have to check out T:U:MockObject, thanks for the hint.

Eric