Forum: RSpec Specifying mocks

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Be3698f145a80c1230fd667c87d0f0c8?d=identicon&s=25 Tom Stuart (Guest)
on 2009-05-29 11:17
(Received via mailing list)
Hi,

One of the problems with mocks, as far as I can tell, is that they
might go out of sync with the real object they're mocking. Is it
possible and sane to detect this by running each spec against its
corresponding mock? Does anyone already do this?

For example: your Account object has a particular behavioural
specification; you perhaps have some convenient helper method for
creating a mock Account which conforms to that specification with
various stubbed responses; and the specifications of other objects use
that mock Account when describing behaviours which involve interaction
with Accounts. What happens when you want to change the behaviour of
Account? Naively, you update the specification and implementation of
Account, your specs all pass, and you think you're done, except you're
not: the Account mock is now misrepresenting the behaviour of Account
all over the system, with the result that you've got failures that the
specs don't reveal.

Of course you have integration tests that will show these failures at
a higher level but it doesn't feel fundamentally like a problem that
RSpec has no business solving. Why not establish a base case for
RSpec's inductive demonstration of correctness by running the Account
spec against the standard Account mock? That way you'll automatically
be told, by RSpec, whenever the actual behaviour of your mock doesn't
match the specified behaviour of the mocked object, and you can let
your integration tests concentrate on the hard stuff (stateful
interactions between many objects) instead of checking that you
haven't forgotten to keep your specs up-to-date.

This assumes a priori that it makes engineering sense to have a
globally-available DRY mock helper for each object (i.e. class) rather
than building up your mocks piecemeal in every spec; I've found that I
always start off a project by doing the latter but end up refactoring
into the former once I get sick of mocking the same stuff over and
over again. Maybe that's the wrong way to do things altogether, or
maybe this is a process issue that I've misunderstood (i.e. the answer
is always to use integration tests to detect this kind of failure), so
stop me if I'm being stupid.

Cheers,
-Tom
5d38ab152e1e3e219512a9859fcd93af?d=identicon&s=25 David Chelimsky (Guest)
on 2009-05-29 13:12
(Received via mailing list)
Hi Tom,

On Fri, May 29, 2009 at 4:11 AM, Tom Stuart <tom@experthuman.com> wrote:
> Hi,
>
> One of the problems with mocks, as far as I can tell, is that they might go
> out of sync with the real object they're mocking.

Have you read "Mock Roles, not Objects"?

http://mockobjects.com/files/mockrolesnotobjects.pdf

> Is it possible and sane to
> detect this by running each spec against its corresponding mock? Does anyone
> already do this?

"it's corresponding mock" suggests a single mock object for each real
implementation. There's nothing stopping you from writing your own
mock objects to play this role. IMO, this is not what a dynamic mock
objects framework is for.

> For example: your Account object has a particular behavioural specification;
> you perhaps have some convenient helper method for creating a mock Account
> which conforms to that specification with various stubbed responses; and the
> specifications of other objects use that mock Account when describing
> behaviours which involve interaction with Accounts. What happens when you
> want to change the behaviour of Account? Naively, you update the
> specification and implementation of Account, your specs all pass, and you
> think you're done, except you're not: the Account mock is now
> misrepresenting the behaviour of Account all over the system, with the
> result that you've got failures that the specs don't reveal.

First of all, let's make a distinction between behaviour and method
signatures. Just because two objects sport the same methods doesn't
mean they behave the same way. In order to verify that a mock Account
honors the same contract as a real Account, you'd have to have some
very general specs like "Account balance should be an instance of the
Money class." This would not really get you very far in terms of
specifying the real account, and it's exactly the opposite of what
we're getting at with BDD - focus on what an object does (behaviour),
not what it is (structure).

So really, what we're talking about is concern over method signatures
straying. Theoretically, this could be solved with some audit
mechanism, but then we're tying mocks to specific objects. What about
when we use mocks in their most powerful way, to specify an object's
contract with polymorphic collaborators?

Consider the case of an Account in a banking application. Checking
accounts and savings accounts have some significant differences in
terms of their behaviour, yet a Transfer will probably only use the
parts that are common to both:

def transfer(source_account, target_account, amount)
  source_account.debit(amount)
  target_account.credit(amount)
end

an admittedly naive implementation, but it demonstrates the point. The
source account may be a checking account. It may be a savings account.
It may be a checking account with overdraw privileges. It may be a
checking account with overdraw privileges that draw on the target
account. Etc, etc. What mocks/stubs let us do here is set up the
context for each example exactly how we want it to be, with objects
that respond how we program them to, but aren't necessarily of a
specific class.

> Of course you have integration tests that will show these failures at a
> higher level but it doesn't feel fundamentally like a problem that RSpec has
> no business solving.

My personal opinion is that RSpec has no business solving this :)

Personal opinion aside, let's say we set out to solve this. We'd need
some auditing mechanism that says the object being mocked has all the
same APIs as the mock object. For RSpec, or any other Ruby framework,
we have a serious obstacle to being able to do this universally and
reliably:

Ruby

We're dealing with a language that lets us modify an object's
behaviour on the fly. This means that it is entirely possible (and
quite common) for a given method not to exist after a class definition
is loaded, but to be added later in the process via a dynamic method
definition or a mixin. If the point at which we do the audit, that
method has not yet been added to the object, we're screwed.

Would that be a problem all the time? Certainly not. But it means we'd
be depending on something that is not very reliable and/or restricting
our use of Ruby's most powerful features.

> globally-available DRY mock helper for each object (i.e. class) rather than
> building up your mocks piecemeal in every spec; I've found that I always
> start off a project by doing the latter but end up refactoring into the
> former once I get sick of mocking the same stuff over and over again. Maybe
> that's the wrong way to do things altogether, or maybe this is a process
> issue that I've misunderstood (i.e. the answer is always to use integration
> tests to detect this kind of failure), so stop me if I'm being stupid.

The most powerful use of mocks is as a design tool, allowing you to
focus on the object at hand and invent its collaborators as you go,
without having to go out and develop them just yet. As you suggest
above, even in your own process, you tend to build up mocks piecemeal
and then refactor towards a single, global helper to create a mock
Account (for example). This lets you keep focus on the task at hand,
and refactor to eliminate duplication as the duplication appears.
Seems like a perfectly viable approach to me.

Of course that doesn't solve the auditing problem I describe above,
and it also pushes you towards a one to one mapping between mocked
object and mock object, which reduces the power of mocks in terms of
polymorphism.

So that's my 1.8 cents (recession, and all). Looking forward to some
other opinions.

Cheers,
David
Be3698f145a80c1230fd667c87d0f0c8?d=identicon&s=25 Tom Stuart (Guest)
on 2009-05-29 14:58
(Received via mailing list)
Hi David,

On 29 May 2009, at 12:10, David Chelimsky wrote:
> Have you read "Mock Roles, not Objects"?
> "its corresponding mock" suggests a single mock object for each real
> implementation. There's nothing stopping you from writing your own
> mock objects to play this role. IMO, this is not what a dynamic mock
> objects framework is for.

Agreed -- I entirely concur that the main benefit of such a framework
is the ability to speculatively mock collaborators (while thinking in
terms of their role) before you have implemented them or perhaps even
know what they are -- but in practice I find that these collaborating
roles end up falling into one-one correspondence with actual classes,
which doesn't seem to contradict the core message of "Mock Roles, not
Objects". Maybe that just indicates I'm not working on designs that
are sufficiently sophisticated, sufficiently polymorphic or
sufficiently aspect-oriented to feel the burn of the divergence of
concepts.

Ultimately I think I'm agreeing with you, but perhaps just using the
wrong word: I might better have described Account as a role rather
than necessarily an object/class. The original point applies, namely
"how can I be sure that my mock (for a given role) is in sync with my
specification (of that role in the object(s) which perform it)";
instead of suggesting a single mock object for each real
implementation, I'm suggesting a single mock object for each role.

And speaking of using the wrong words: maybe what I actually mean is
"specifying stubs" instead of "specifying mocks", in as much as those
global helpers I talked about don't really return a "mock" at all,
just a stub which has been prepared for use as a mock in any spec
which requires it (cf Spec::Rails' mock_model). But any such spec will
exercise those stubbed methods which are important for the role's
behaviour, assuming one expectation per example, so it's important
that the stubbed behaviour accurately reflects reality, and unless you
are rigorous about keeping the stubbed behaviour updated to reflect
the specified behaviour they'll drift out of sync and the specs won't
be making the right assumptions about collaborators any more.

> Just because two objects sport the same methods doesn't
> mean they behave the same way. In order to verify that a mock Account
> honors the same contract as a real Account, you'd have to have some
> very general specs like "Account balance should be an instance of the
> Money class."

Why? I can't quite get my head around the issues of stubbing stateful
interactions, which is preventing me from thinking of a good example,
but what's (abstractly) wrong with:

class Account
   def credit(cents)
     self.balance = self.balance + cents
     return self.balance
   end
end

describe 'Account#credit' do
   before(:each) do
     @account = Account.new(:balance => 100)
   end
   specify { @account.credit(10).should == 110 }
end

def mock_account
   account = mock('Account')
   account.stub!(:credit).with(10).and_return(110)
   return account
end

It seems like there's nothing stopping you stubbing the detailed
behaviour of Account in a way that will allow the stub to pass the
specs -- it's just that the stub is only good for the canned
responses, necessarily a strict subset of the real implementation's
behaviour over all inputs.

Given the above, I could easily change spec and implementation to

class Account
   def credit(cents)
     self.balance = self.balance - cents
     return self.balance
   end
end

describe 'Account#credit' do
   before(:each) do
     @account = Account.new(:balance => 100)
   end
   specify { @account.credit(10).should == 90 }
end

and the spec will pass, and so will every spec that's using my Account
stub for mocking Account behaviour, but the system as a whole is
screwed, right? Because the behaviour's changed but the stub (plus the
behaviour that is mocked on it by other specs) has stayed the same.

> So really, what we're talking about is concern over method
> signatures straying.

Sorry, that's not what I meant at all: I intended to talk specifically
about cases where the signature stays the same but the underlying
behaviour changes.

> What about when we use mocks in their most powerful way, to specify
> an object's
> contract with polymorphic collaborators?

In that case there's a role (an object fulfilling that contract) for
which you might end up creating a shared stub that's used in the
specifications of all objects which act as the client in that specific
collaboration. Individual examples in those specifications will set
individual expectations on that stub in order to specify the
interaction, presumably clobbering a stubbed method in the process,
but the remaining stubbed methods will be exercised during each
example, which gives you a way of knowing that the mocked behaviour
lines up with what's been canned in the stub.

You can't do much about the detailed, piecemeal mocking that happens
inside such specs -- aside from the aforementioned reliance upon the
parts of the stub's canned behaviour for which an expectation hasn't
explicitly been set, there's no way to check that the expectations
match up with the specification(s) of the object(s) which performs the
role you're mocking, so you have to rely on integration specs -- but
if all of these specs are starting with the same stub, you can at
least run the role's spec over the stub to make sure it's accurate.
Why wouldn't you?

> Personal opinion aside, let's say we set out to solve this. We'd need
> some auditing mechanism that says the object being mocked has all the
> same APIs as the mock object.

As I say, that's not what I meant, and it's probably my fault for
saying "mock". What I meant is that a stub can provide a snapshot of
all of the behaviour of an object (or role), and I often end up in the
situation where the same stub is being used as a starting point for
mocking interactions in lots of specs, but right now there's no
mechanism for spotting when that stub is spreading misinformation
because it's become desynchronised from the specification of the
object (or role) which it represents.

For example: Spec::Rails' mock_model. (Should be called stub_model but
isn't.) It's very convenient because it stubs out a bunch of
"behaviour" that you'd otherwise have to stub out yourself in all of
your model specs before you started adding expectations. It conforms
to a tiny subset of ActiveRecord::Base's notional specification:
@record.should_receive(:new_record?).and_return(false) etc. If "the
ActiveRecord::Base spec" changes, mock_model needs updating, but we
can't discover that automatically because we never run "the
ActiveRecord::Base spec" over the object that's returned by
mock_model. (This is a stupid example because there is no
ActiveRecord::Base spec, but you get the idea.)

So is my fundamental mistake that I'm being lazy by trying to
concentrate all of this stub setup in one helper method instead of
doing piecemeal stubbing in every individual spec that might care? It
seems superficially like good practice to concentrate all of the
detail in one place like this, so that when a role's behaviour changes
you just update one piece of stub setup to reflect the change rather
than chase around the specs of every single collaborator, but maybe
that's just serving to obscure the problem that all of the actual
mocking has to happen in the individual specs and that this is the
stuff you really care about, not the behaviour of the shared stub?

Cheers,
-Tom
Cdf378de2284d8acf137122e541caa28?d=identicon&s=25 Matt Wynne (mattwynne)
on 2009-05-30 10:14
(Received via mailing list)
On 29 May 2009, at 13:51, Tom Stuart wrote:

> framework is the ability to speculatively mock collaborators (while
> wrong word: I might better have described Account as a role rather
> spec which requires it (cf Spec::Rails' mock_model). But any such
>> honors the same contract as a real Account, you'd have to have some
>    return self.balance
> def mock_account
>
>  before(:each) do
>
>
> You can't do much about the detailed, piecemeal mocking that happens
>> some auditing mechanism that says the object being mocked has all the
>
> ActiveRecord::Base spec, but you get the idea.)
> and that this is the stuff you really care about, not the behaviour
> of the shared stub?

On my team, we've built up a StubFactory which works a lot like the
FactoryGirl plug-in and creates 'stock' stubbed objects which we use
when we need a generic stub of a given role. This is a useful tool,
and it saves a lot of noise in our tests when we just need something
that quacks enough like an Account or whatever in order to let a test
run through, so that you can focus on the detail of the *specific*
collaboration behaviour you're concerned about in that test.

So that's one part of this - I definitely think it's pragmatic and
valuable to factor out common mocking setup into a single place if
that's what you seem to be duplicating a lot in your tests.

The auditing though... I've been around the loop with this one myself
- before I really discovered the value of acceptance tests I was
really keen on the idea, but to be honest this just isn't a problem I
come across very often now. By the time my acceptance tests have
pointed me down to the unit tests / classes I need to work on, I'm
(especially with a pair to work with) sufficiently focussed to
remember enough the of collaborations between objects and keep them in
step with the mocks / stubs. If I do forget something, I get pretty
rapid feedback when I step back up a level and run the acceptance tests.

I appreciate that the argument 'I can keep all the collaboration in my
head' is a bit wooly - but equally if the collaborations you're
mocking are so complex that you can't, then perhaps that in itself is
a whiff that something is wrong with your design?

Matt Wynne
http://beta.songkick.com
http://blog.mattwynne.net
This topic is locked and can not be replied to.