Surprising mock behavior

[markthomas]

I’m writing my first controller spec - for a controller that already
exists, and in the process have observed some behavior I find a little
surprising. I’d like to know whether I’m interpreting the situation
correctly…

In my controller I have some stuff written to a file, i.e.

file = File.new(…)
…
file.puts “a string”
…
file.puts “another string”

etc

In my spec I mock the file object -

file = mock(‘file’)
File.stub!(:new).and_return(file)

and then check that the expected messages are being received -

file.should_receive(:puts).with(“a string”).once
file.should_receive(:puts).with(“another string”).once

etc.

Here’s what I’m puzzled about. If I don’t include the expectation for
the first string in the spec, the spec will fail the expectation for the
second string. It seems as if “should_receive” is queuing up the
messages that come into the file object and when it tests an expectation
it just looks at the next one in line. If it doesn’t match then the
expectation will fail.

Is that really what happens? Or have I missed some important insight?
Seems counterintuitive to me. What is strange is that I was working away
on this yesterday and just picking out a number of particular puts to
test that I was concerned about. It seemed to be working the way I
expected it to. Then late yesterday I started seeing some spec failures
I didn’t understand and my investigation led me to the conclusion above.

Interested to know if what I think I’m seeing is consistent with other
people’s experience and with the intended behavior of mocks.

Mark.

[markthomas]

Actually, this is evidently not the whole story. I actually have two
examples in the same spec, and I just realized that the other one has
instances of file.should_receive(:puts) that don’t seem to suffer the
same limitation I described. So it looks like there is something about
the example I described that is causing it to behave strangely. Will
investigate further.

Mark.

[markthomas]

On 2008-10-17, at 14:55, Mark T. wrote:

…snip…
It seems as if “should_receive” is queuing up the messages that come
into the file object and when it tests an expectation it just looks
at the next one in line. If it doesn’t match then the expectation
will fail.

Hi Mark. From my understanding and experiences with RSpec, that’s
correct behaviour. Essentially, you’re defining an ordered list of
messages you expect to be called on the ‘file’ object.

Cheers,
Nick

[markthomas]

On Fri, Oct 17, 2008 at 2:55 PM, Mark T. [email protected]
wrote:

and then check that the expected messages are being received -
file.should_receive(:puts).with(“a string”).once
file.should_receive(:puts).with(“another string”).once

Here’s what I’m puzzled about. If I don’t include the expectation for the
first string in the spec, the spec will fail the expectation for the second
string. It seems as if “should_receive” is queuing up the messages that come
into the file object and when it tests an expectation it just looks at the
next one in line. If it doesn’t match then the expectation will fail.

That sounds right to me. You declared ‘file’ as a mock. Mocks are
bratty little children that treat it as an error and throw a tantrum
if you don’t give them everything they expect, no more nor less. (As
contrasted to stubs, which are couch potatoes that will respond if you
call them but don’t complain if you don’t.)

So when you create a mock, you need to be very thorough about it.
Every message has to be accounted for somehow. If it gets more
messages than you tell it, or different messages, it’ll error. If it
doesn’t get enough messages, it’ll error. This is correct behavior.

If the exact parameters aren’t part of your test plan, you could
always say file.should_receive(:puts).twice and leave off the
parameter expectation. Or just file.should_receive(:puts) if you know
it should happen but don’t care how often, or
file.should_receive(:puts).any_number_of_times if it could be zero as
well.

Or, if that all sounds like too much work, make ‘file’ a stub object
instead of a mock object and stub the :puts method:

file = stub(‘file’, :puts => true)
File.stub!(:new).and_return(file)

…and then only set should_receive for the things you care about in
specific tests.

Does that help, or is the behavior still anomalous even knowing that
mocks are super-finicky?

–
Have Fun,
Steve E. ([email protected])
ESCAPE POD - The Science Fiction Podcast Magazine
http://www.escapepod.org

[markthomas]

On Fri, Oct 17, 2008 at 3:37 PM, Stephen E. [email protected] wrote:

next one in line. If it doesn’t match then the expectation will fail.

That sounds right to me. You declared ‘file’ as a mock. Mocks are
bratty little children that treat it as an error and throw a tantrum
if you don’t give them everything they expect, no more nor less.

Are you a proud parent?

–
Zach D.
http://www.continuousthinking.com

Homepage | Mutually Human

[markthomas]

On Fri, Oct 17, 2008 at 11:55 AM, Mark T.
[email protected]wrote:

It seems as if “should_receive” is queuing up the messages that come into
the file object and when it tests an expectation it just looks at the next
one in line. If it doesn’t match then the expectation will fail.

Yup. Pretty cool, huh?

///ark

[markthomas]

On Fri, Oct 17, 2008 at 3:54 PM, Zach D. [email protected]
wrote:

On Fri, Oct 17, 2008 at 3:37 PM, Stephen E. [email protected] wrote:

That sounds right to me. You declared ‘file’ as a mock. Mocks are
bratty little children that treat it as an error and throw a tantrum
if you don’t give them everything they expect, no more nor less.

Are you a proud parent?

Yes, very. And I mock my three-year-old’s behavior often. I also
tell him what he should_receive; what he returns, alas, is not yet
deterministic.

–
Have Fun,
Steve E. ([email protected])
ESCAPE POD - The Science Fiction Podcast Magazine
http://www.escapepod.org

[markthomas]

Stephen E. wrote:

next one in line. If it doesn’t match then the expectation will fail.
messages than you tell it, or different messages, it’ll error. If it
doesn’t get enough messages, it’ll error. This is correct behavior.

Thanks for the explanation Stephen. However, if that is the intention,
I’m puzzled by something else - as I said the spec fails if I don’t
include an expectation for each output message. However it turns out
that that’s actually not always true. What I’ve observed is that it
behaves differently if I include a “should_not_receive(’…’)”
expectation somewhere in the spec. In that case it seems that I can have
as many “file.puts()” in the component being tested as I like without
specifying expectations for them, and they pass just fine. In fact I did
have such a situation in my initial spec and I think that’s what led me
to my mistaken understanding of how should_receive is meant to work. But
I’m struggling to understand what the rationale is that explains both of
these cases.

That aside, I also can’t help questioning the way the “should_receive”
expectation is expressed. Maybe specifying every message sent to the
mock is absolutely the right way to test the component. But in view of
the general philosophy of expressing expectations in a way that reflects
what they actually mean, in my mind this doesn’t quite hit the mark. If
you say “should receive”, the way I read that is that if the object
/does /receive what you specify then it should pass. But that’s clearly
not what happens. Nor is it an expectation on what will be received
next. If that were the case you might call the method
“should_next_receive”. However, in fact, as long as all messages are
accounted for, you can reorder the individual “should_receive”
expectations any way you like and the spec will still pass.

In fact “should_receive” does not appear to be an expectation on a
single message at all (even if you say “should_receive().once”, and
leaving aside the exception with “should_not_receive” I noted above). I
think a better way to think about this is that the total set of
“should_receive” calls are together an expectation on the totality of
messages received by the object. In view of this, I wonder if a better
way to formulate this test might be to say something like -

object.should_receive :method => :method_name, :with_each_of => [arg1,
arg2,… argN]

where arg1, arg2 etc represent the parameters for each individual call
to :method_name. i.e. declare the whole of what we really expect the
object to receive in a single call to should_receive.

You could take this one step further and declare all of the required
calls to any number of methods on the object in a single expectation, by
making the argument to should_receive an array -

object.should_receive [{:method => :method1, :with_each_of => […]},
{:method => :method2, :with_each_of => […]}, …]

Yeah it gets a little wordy, but if I’m understanding the behavior
correctly, this is what we are actually trying to test.

Does this make sense?

Mark.

[markthomas]

On Sat, Oct 18, 2008 at 9:49 PM, Mark T. [email protected]
wrote:

[…] What I’ve observed is that it behaves differently
if I include a “should_not_receive(‘…’)” expectation somewhere in the
spec. In that case it seems that I can have as many “file.puts()” in the
component being tested as I like without specifying expectations for them,
and they pass just fine.

Hmm, that does seem weird. Without seeing your actual code I think
it’d be difficult for anyone to say if it’s a bug, or expected
behavior with unexpected results, or something else causing the issue,
etc. It smells like a bug to me the way you describe it –
specifying messages you shouldn’t get ought not change the expectation
of messages you do get – but then, I’ve often been flummoxed by
test behavior in my own code when it was really my misunderstanding
causing the problem. Maybe file a Lighthouse ticket with code samples
clearly showing failing and passing tests.

That aside, I also can’t help questioning the way the “should_receive”
expectation is expressed. Maybe specifying every message sent to the mock is
absolutely the right way to test the component. But in view of the general
philosophy of expressing expectations in a way that reflects what they
actually mean, in my mind this doesn’t quite hit the mark. [ . . . ]

I think the difference in perspective here is that you’re focusing in
very closely on the individual “should_receive” calls. That’s not the
level at which this expectation is set. It’s not the presence of a
single should_receive that makes the mock want to know about every
message it gets. It’s the fact that it’s a mock. Unless you tell
them otherwise, that’s just the way mocks work. If you don’t want to
have to specify everything, then that’s what stubs are for.

In view of this, I wonder if a better way to formulate this test
might be to say something like -

object.should_receive :method => :method_name, :with_each_of => [arg1,
arg2,… argN]

That’s not a bad idea, but I suspect your tests in which you’re
calling ‘puts’ over and over might be a bit of an edge case. Most
spec examples are much simpler, and I suspect it’s uncommon to have a
need to call a given method more than once in a single example. Even
if you’re testing an iteration, you don’t have to use a large loop
in a test case. One or two iterations and logical induction ought to
suffice to prove that it works.

–
Have Fun,
Steve E. ([email protected])
ESCAPE POD - The Science Fiction Podcast Magazine
http://www.escapepod.org

[markthomas]

I’m top posting… I wish I could inline post, but you provided a lot
of generalizations for how you think things are working on your code,
but you don’t actually post concrete code (the should_receive and
should_not_receive case you mentioned that wasn’t acting like you’d
expect). Perhaps this will help:

• Mocks are not ordered by default.
• If you want ordered message expectations on a mock you have to
  explicitly tell them to be ordered.
• You cannot enforce ordering across mocks.

So without explicitly ordering, message expectations can be fulfilled
in any order. For example the below will pass even though the
expectation for 1 and 3 does not match the order in when the bar
method is called:

file = Object.new
file.should_receive(:bar).with("1")
file.should_receive(:bar).with("3")

file.bar "3"
file.bar "1"

Once a message expectation is fulfilled, if the object receives
another message matching that expectation it will force it to fail.
For example, the following would fail even though there are two
expectations for “bar” to be called with “1”, which match the two
calls to file.bar:

file = Object.new
file.should_receive(:bar).with("1")
file.should_receive(:bar).with("1")

file.bar "1"
file.bar "1"

Using should_not_receive and should_receive on the same message gets
tricky, because you can trick yourself into thinking the thing is
passing when it shouldn’t be. For example the below will fail because
even though you call file.bar with “2”, that matches the expectation
that file.bar should not be called with “3”:

file = Object.new
file.should_receive(:bar).with("1")
file.should_not_receive(:bar).with("3")

file.bar "1"
file.bar "2"

Hopefully this helps clarify some things for you about rspec’s mocks.

However…

object.should_receive :method => :method_name, :with_each_of => [arg1,
arg2,… argN]

I really don’t like this. This is hard to read and can lead to
something that is very convoluted. Can you share the actual code and
spec you’re having issues with so we can try to provide concrete help
back to you?

Zach

On Sat, Oct 18, 2008 at 9:49 PM, Mark T. [email protected]
wrote:

Here’s what I’m puzzled about. If I don’t include the expectation for the
bratty little children that treat it as an error and throw a tantrum
Thanks for the explanation Stephen. However, if that is the intention, I’m

“should_receive” expectations any way you like and the spec will still pass.
arg2,… argN]
{:method => :method2, :with_each_of => […]}, …]
http://rubyforge.org/mailman/listinfo/rspec-users

–
Zach D.
http://www.continuousthinking.com

Homepage | Mutually Human

[markthomas]

Stephen E. wrote:

clearly showing failing and passing tests.

Will do.

It’s not the presence of a
single should_receive that makes the mock want to know about every
message it gets. It’s the fact that it’s a mock. Unless you tell
them otherwise, that’s just the way mocks work.

Thanks, I think that’s really the key concept I’ve been missing. I see
how that changes the perspective on should_receive.

If you don’t want to
have to specify everything, then that’s what stubs are for.

I think I understand your point here, though am I right that a stub is
restricted to specifying only one response to any particular method? You
can’t specify parameters in the way you do using :with in should_receive
on a mock. OTOH taking a different tack, it seems from the
documentation (http://rspec.info/documentation/mocks/) that the
:null_object option to mock() may allow you to leave some messages
unspecified. I’ll check on this.

In view of this, I wonder if a better way to formulate this test
might be to say something like -

object.should_receive :method => :method_name, :with_each_of => [arg1,
arg2,… argN]

That’s not a bad idea, but I suspect your tests in which you’re
calling ‘puts’ over and over might be a bit of an edge case.

Yeah, I think that’s true. I was actually trying to debug the spec,
rather than object being tested. Still, it has turned out to be a useful
learning exercise.

[markthomas]

Zach D. wrote:

I’m top posting… I wish I could inline post, but you provided a lot
of generalizations for how you think things are working on your code,
but you don’t actually post concrete code (the should_receive and
should_not_receive case you mentioned that wasn’t acting like you’d
expect). Perhaps this will help:

• Mocks are not ordered by default.

Yeah that’s consistent with what I’m seeing. I mentioned ordering only
because it was one of the things that came up in my mind as I was
searching for a way to understand the different behaviors I was seeing.

• If you want ordered message expectations on a mock you have to
  explicitly tell them to be ordered.

Really, you can do that? I’m curious about how.

file.bar "3"
file.bar "1"

Ok got that.

file.bar "1"
file.bar "1"

This one kind of surprises me, but I understand your point.

file.bar "1"
file.bar "2"

Did you mean to say that will not fail since “2” will match “not 3”?
If so that’s really good to know. I had imagined that should_not_receive
was purely a statement about something that doesn’t happen, not about
something different happening. That may explain what I had thought was
anomalous behavior. I’ll have to take another look at that.

something that is very convoluted. Can you share the actual code and
spec you’re having issues with so we can try to provide concrete help
back to you?

Steve’s comments helped me understand why this approach isn’t necessary.

Mark.

[markthomas]

On Sun, Oct 19, 2008 at 1:34 AM, Mark T. [email protected]
wrote:

Yeah that’s consistent with what I’m seeing. I mentioned ordering only
because it was one of the things that came up in my mind as I was searching
for a way to understand the different behaviors I was seeing.

• If you want ordered message expectations on a mock you have to
  explicitly tell them to be ordered.

Really, you can do that? I’m curious about how.

Checkout http://rspec.info/documentation/mocks/message_expectations.html

And look at the Ordering section.

file.should_receive(:bar).with(“3”)
For example, the following would fail even though there are two

file.should_not_receive(:bar).with(“3”)

file.bar “1”
file.bar “2”

Did you mean to say that will not fail since “2” will match “not 3”?

Yes, that’s what I meant.

–
Zach D.
http://www.continuousthinking.com

Homepage | Mutually Human

[markthomas]

Can you please post an example of the spec and production code that
isn’t behaving as you expect?

Pat

[markthomas]

Zach D. wrote:

Thanks for that pointer. Somehow I never made it that far into the
document tree. If you hadn’t mentioned it I would not have realized that
was there. No wonder I was having trouble!

M.

[markthomas]

Pat M. wrote:

Can you please post an example of the spec and production code that
isn’t behaving as you expect?

Pat

Sure, sorry been tied up with business travel this week. Here’s my
controller…

---

class SubscribersController < ApplicationController

def test

file = File.new("test.txt", "w")


file.puts "one"
file.puts "two"
file.puts "three"
file.puts "four"
file.puts "five"
file.puts "six"

render :action => "index"

end

end

---

and a spec containing two examples that illustrate what I was talking
about -

---

describe SubscribersController do

it "does not pass" do

    file = mock('file')
    File.stub!(:new).and_return(file)

    file.should_receive(:puts).with("one")
    file.should_receive(:puts).with("two")
    file.should_receive(:puts).with("six")

    get 'test'

end


it "does pass" do

    file = mock('file')
    File.stub!(:new).and_return(file)

    file.should_receive(:puts).with("one")
    file.should_receive(:puts).with("two")
    file.should_receive(:puts).with("six")
    file.should_not_receive(:puts).with("ten")

    get 'test'

end

end

---

The first example fails with error message “Mock ‘file’ expected :puts
with (“one”) but received it with (“three”)” (The message seems a little
strange since “one” is clearly received, but I can live with that. I
understand the point Steve made that a mock should have all messages
specified in expectations).

The second example passes fine, which is the thing I still find
surprising. It seems as if “should_not_receive” doesn’t just match the
absence of the specified message but also the presence of any message
other than the specified message.

Mark.