Forum: RSpec Surprising mock behavior

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
07dd367bcc4ae602d640ec8c1b280df6?d=identicon&s=25 Mark Thomson (Guest)
on 2008-10-17 20:56
(Received via mailing list)
I'm writing my first controller spec - for a controller that already
exists, and in the process have observed some behavior I find a little
surprising. I'd like to know whether I'm interpreting the situation
correctly...

In my controller I have some stuff written to a file, i.e.

file = File.new(...)
...
file.puts "a string"
...
file.puts "another string"

etc


In my spec I mock the file object -

file = mock('file')
File.stub!(:new).and_return(file)

and then check that the expected messages are being received -

file.should_receive(:puts).with("a string").once
file.should_receive(:puts).with("another string").once

etc.


Here's what I'm puzzled about. If I don't include the expectation for
the first string in the spec, the spec will fail the expectation for the
second string. It seems as if "should_receive" is queuing up the
messages that come into the file object and when it tests an expectation
it just looks at the next one in line. If it doesn't match then the
expectation will fail.

Is that really what happens? Or have I missed some important insight?
Seems counterintuitive to me. What is strange is that I was working away
on this yesterday and just picking out a number of particular puts to
test that I was concerned about. It seemed to be working the way I
expected it to. Then late yesterday I started seeing some spec failures
I didn't understand and my investigation led me to the conclusion above.

Interested to know if what I think I'm seeing is consistent with other
people's experience and with the intended behavior of mocks.

Mark.
07dd367bcc4ae602d640ec8c1b280df6?d=identicon&s=25 Mark Thomson (Guest)
on 2008-10-17 21:11
(Received via mailing list)
Actually, this is evidently not the whole story. I actually have two
examples in the same spec, and I just realized that the other one has
instances of file.should_receive(:puts) that don't seem to suffer the
same limitation I described. So it looks like there is something about
the example I described that is causing it to behave strangely. Will
investigate further.

Mark.
49de4cd2f26705785cbef2b15a9df7aa?d=identicon&s=25 Nick Hoffman (Guest)
on 2008-10-17 21:15
(Received via mailing list)
On 2008-10-17, at 14:55, Mark Thomson wrote:
> ..snip..
> It seems as if "should_receive" is queuing up the messages that come
> into the file object and when it tests an expectation it just looks
> at the next one in line. If it doesn't match then the expectation
> will fail.

Hi Mark. From my understanding and experiences with RSpec, that's
correct behaviour. Essentially, you're defining an ordered list of
messages you expect to be called on the 'file' object.

Cheers,
Nick
994e42bda994be2cd1d791f18ee6d561?d=identicon&s=25 Stephen Eley (Guest)
on 2008-10-17 21:38
(Received via mailing list)
On Fri, Oct 17, 2008 at 2:55 PM, Mark Thomson <mark.thomson@ieee.org>
wrote:
>
> and then check that the expected messages are being received -
> file.should_receive(:puts).with("a string").once
> file.should_receive(:puts).with("another string").once
>
> Here's what I'm puzzled about. If I don't include the expectation for the
> first string in the spec, the spec will fail the expectation for the second
> string. It seems as if "should_receive" is queuing up the messages that come
> into the file object and when it tests an expectation it just looks at the
> next one in line. If it doesn't match then the expectation will fail.

That sounds right to me.  You declared 'file' as a mock.  Mocks are
bratty little children that treat it as an error and throw a tantrum
if you don't give them everything they expect, no more nor less.  (As
contrasted to stubs, which are couch potatoes that will respond if you
call them but don't complain if you don't.)

So when you create a mock, you need to be very thorough about it.
Every message has to be accounted for somehow.  If it gets more
messages than you tell it, or different messages, it'll error.  If it
doesn't get enough messages, it'll error.  This is correct behavior.

If the exact parameters aren't part of your test plan, you could
always say file.should_receive(:puts).twice and leave off the
parameter expectation.  Or just file.should_receive(:puts) if you know
it should happen but don't care how often, or
file.should_receive(:puts).any_number_of_times if it could be zero as
well.

Or, if that all sounds like too much work, make 'file' a stub object
instead of a mock object and stub the :puts method:

file = stub('file', :puts => true)
File.stub!(:new).and_return(file)

...and then only set should_receive for the things you care about in
specific tests.

Does that help, or is the behavior still anomalous even knowing that
mocks are super-finicky?


--
Have Fun,
   Steve Eley (sfeley@gmail.com)
   ESCAPE POD - The Science Fiction Podcast Magazine
   http://www.escapepod.org
F86901feca747abbb5c6c020362ef2e7?d=identicon&s=25 Zach Dennis (zdennis)
on 2008-10-17 21:55
(Received via mailing list)
On Fri, Oct 17, 2008 at 3:37 PM, Stephen Eley <sfeley@gmail.com> wrote:
>> next one in line. If it doesn't match then the expectation will fail.
>
> That sounds right to me.  You declared 'file' as a mock.  Mocks are
> bratty little children that treat it as an error and throw a tantrum
> if you don't give them everything they expect, no more nor less.

Are you a proud parent? :)

--
Zach Dennis
http://www.continuousthinking.com
http://www.mutuallyhuman.com
994e42bda994be2cd1d791f18ee6d561?d=identicon&s=25 Stephen Eley (Guest)
on 2008-10-17 22:12
(Received via mailing list)
On Fri, Oct 17, 2008 at 3:54 PM, Zach Dennis <zach.dennis@gmail.com>
wrote:
> On Fri, Oct 17, 2008 at 3:37 PM, Stephen Eley <sfeley@gmail.com> wrote:
>>
>> That sounds right to me.  You declared 'file' as a mock.  Mocks are
>> bratty little children that treat it as an error and throw a tantrum
>> if you don't give them everything they expect, no more nor less.
>
> Are you a proud parent? :)

Yes, very.  And I mock my three-year-old's behavior often.  I also
tell him what he should_receive; what he returns, alas, is not yet
deterministic.

--
Have Fun,
   Steve Eley (sfeley@gmail.com)
   ESCAPE POD - The Science Fiction Podcast Magazine
   http://www.escapepod.org
48641c4be1fbe167929fb16c9fd94990?d=identicon&s=25 Mark Wilden (Guest)
on 2008-10-18 06:34
(Received via mailing list)
On Fri, Oct 17, 2008 at 11:55 AM, Mark Thomson
<mark.thomson@ieee.org>wrote:


> It seems as if "should_receive" is queuing up the messages that come into
> the file object and when it tests an expectation it just looks at the next
> one in line. If it doesn't match then the expectation will fail.
>

Yup. Pretty cool, huh?

///ark
07dd367bcc4ae602d640ec8c1b280df6?d=identicon&s=25 Mark Thomson (Guest)
on 2008-10-19 03:50
(Received via mailing list)
Stephen Eley wrote:
>> next one in line. If it doesn't match then the expectation will fail.
> messages than you tell it, or different messages, it'll error.  If it
> doesn't get enough messages, it'll error.  This is correct behavior.
>

Thanks for the explanation Stephen. However, if that is the intention,
I'm puzzled by something else - as I said the spec fails if I don't
include an expectation for each output message. However it turns out
that that's actually not always true. What I've observed is that it
behaves differently if I include a "should_not_receive('...')"
expectation somewhere in the spec. In that case it seems that I can have
as many "file.puts()" in the component being tested as I like without
specifying expectations for them, and they pass just fine. In fact I did
have such a situation in my initial spec and I think that's what led me
to my mistaken understanding of how should_receive is meant to work. But
I'm struggling to understand what the rationale is that explains both of
these cases.

That aside, I also can't help questioning the way the "should_receive"
expectation is expressed. Maybe specifying every message sent to the
mock is absolutely the right way to test the component. But in view of
the general philosophy of expressing expectations in a way that reflects
what they actually mean, in my mind this doesn't quite hit the mark. If
you say "should receive", the way I read that is that if the object
/does /receive what you specify then it should pass. But that's clearly
not what happens. Nor is it  an expectation on what will be received
next. If that were the case you might call the method
"should_next_receive". However, in fact, as long as all messages are
accounted for, you can reorder the individual "should_receive"
expectations any way you like and the spec will still pass.

In fact "should_receive" does not appear to be an expectation on a
single message at all (even if you say "should_receive().once", and
leaving aside the exception with "should_not_receive" I noted above). I
think a better way to think about this is that the total set of
"should_receive" calls are _together_ an expectation on the totality of
messages received by the object. In view of this, I wonder if a better
way to formulate this test might be to say something like -

object.should_receive :method => :method_name, :with_each_of => [arg1,
arg2,... argN]

where arg1, arg2 etc represent the parameters for each individual call
to :method_name. i.e. declare the whole of what we _really_ expect the
object to receive in a single call to should_receive.

You could take this one step further and declare all of the required
calls to any number of methods on the object in a single expectation, by
making the argument to should_receive an array -

object.should_receive [{:method => :method1, :with_each_of => [...]},
{:method => :method2, :with_each_of => [...]}, ...]

Yeah it gets a little wordy, but if I'm understanding the behavior
correctly, this is what we are actually trying to test.

Does this make sense?

Mark.
994e42bda994be2cd1d791f18ee6d561?d=identicon&s=25 Stephen Eley (Guest)
on 2008-10-19 04:52
(Received via mailing list)
On Sat, Oct 18, 2008 at 9:49 PM, Mark Thomson <mark.thomson@ieee.org>
wrote:
>
> [...]  What I've observed is that it behaves differently
> if I include a "should_not_receive('...')" expectation somewhere in the
> spec. In that case it seems that I can have as many "file.puts()" in the
> component being tested as I like without specifying expectations for them,
> and they pass just fine.

Hmm, that does seem weird.  Without seeing your actual code I think
it'd be difficult for anyone to say if it's a bug, or expected
behavior with unexpected results, or something else causing the issue,
etc.  It _smells_ like a bug to me the way you describe it --
specifying messages you shouldn't get ought not change the expectation
of messages you _do_ get -- but then, I've often been flummoxed by
test behavior in my own code when it was really my misunderstanding
causing the problem.  Maybe file a Lighthouse ticket with code samples
clearly showing failing and passing tests.



> That aside, I also can't help questioning the way the "should_receive"
> expectation is expressed. Maybe specifying every message sent to the mock is
> absolutely the right way to test the component. But in view of the general
> philosophy of expressing expectations in a way that reflects what they
> actually mean, in my mind this doesn't quite hit the mark.  [ . . . ]

I think the difference in perspective here is that you're focusing in
very closely on the individual "should_receive" calls.  That's not the
level at which this expectation is set.  It's not the presence of a
single should_receive that makes the mock want to know about every
message it gets.  It's the fact that *it's a mock.*  Unless you tell
them otherwise, that's just the way mocks work.  If you don't want to
have to specify everything, then that's what stubs are for.



> In view of this, I wonder if a better way to formulate this test
> might be to say something like -
>
> object.should_receive :method => :method_name, :with_each_of => [arg1,
> arg2,... argN]

That's not a bad idea, but I suspect your tests in which you're
calling 'puts' over and over might be a bit of an edge case.  Most
spec examples are much simpler, and I suspect it's uncommon to have a
need to call a given method more than once in a single example.  Even
if you're testing an iteration, you don't have to use a _large_ loop
in a test case.  One or two iterations and logical induction ought to
suffice to prove that it works.


--
Have Fun,
   Steve Eley (sfeley@gmail.com)
   ESCAPE POD - The Science Fiction Podcast Magazine
   http://www.escapepod.org
F86901feca747abbb5c6c020362ef2e7?d=identicon&s=25 Zach Dennis (zdennis)
on 2008-10-19 05:41
(Received via mailing list)
I'm top posting... I wish I could inline post, but you provided a lot
of generalizations for how you think things are working on your code,
but you don't actually post concrete code (the should_receive and
should_not_receive case you mentioned that wasn't acting like you'd
expect). Perhaps this will help:

* Mocks are not ordered by default.
* If you want ordered message expectations on a mock you have to
explicitly tell them to be ordered.
* You cannot enforce ordering across mocks.

So without explicitly ordering, message expectations can be fulfilled
in any order. For example the below will pass even though the
expectation for 1 and 3 does not match the order in when the bar
method is called:

    file = Object.new
    file.should_receive(:bar).with("1")
    file.should_receive(:bar).with("3")

    file.bar "3"
    file.bar "1"

Once a message expectation is fulfilled, if the object receives
another message matching that expectation it will force it to fail.
For example, the following would fail even though there are two
expectations for "bar" to be called with "1", which match the two
calls to file.bar:

    file = Object.new
    file.should_receive(:bar).with("1")
    file.should_receive(:bar).with("1")

    file.bar "1"
    file.bar "1"

Using should_not_receive and should_receive on the same message gets
tricky, because you can trick yourself into thinking the thing is
passing when it shouldn't be. For example the below will fail because
even though you call file.bar with "2", that matches the expectation
that file.bar should not be called with "3":

    file = Object.new
    file.should_receive(:bar).with("1")
    file.should_not_receive(:bar).with("3")

    file.bar "1"
    file.bar "2"

Hopefully this helps clarify some things for you about rspec's mocks.

However...

> object.should_receive :method => :method_name, :with_each_of => [arg1,
> arg2,... argN]

I really don't like this. This is hard to read and can lead to
something that is very convoluted. Can you share the actual code and
spec you're having issues with so we can try to provide concrete help
back to you?

Zach


On Sat, Oct 18, 2008 at 9:49 PM, Mark Thomson <mark.thomson@ieee.org>
wrote:
>>> Here's what I'm puzzled about. If I don't include the expectation for the
>> bratty little children that treat it as an error and throw a tantrum
> Thanks for the explanation Stephen. However, if that is the intention, I'm
>
> "should_receive" expectations any way you like and the spec will still pass.
> arg2,... argN]
> {:method => :method2, :with_each_of => [...]}, ...]
> http://rubyforge.org/mailman/listinfo/rspec-users
>



--
Zach Dennis
http://www.continuousthinking.com
http://www.mutuallyhuman.com
07dd367bcc4ae602d640ec8c1b280df6?d=identicon&s=25 Mark Thomson (Guest)
on 2008-10-19 06:57
(Received via mailing list)
Stephen Eley wrote:
> clearly showing failing and passing tests.
>
>
>
Will do.

>   It's not the presence of a
> single should_receive that makes the mock want to know about every
> message it gets.  It's the fact that *it's a mock.*  Unless you tell
> them otherwise, that's just the way mocks work.

Thanks, I think that's really the key concept I've been missing. I see
how that changes the perspective on should_receive.

> If you don't want to
> have to specify everything, then that's what stubs are for.
>

I think I understand your point here, though am I right that a stub is
restricted to specifying only one response to any particular method? You
can't specify parameters in the way you do using :with in should_receive
on a mock.  OTOH taking a different tack, it seems from the
documentation (http://rspec.info/documentation/mocks/) that the
:null_object option to mock() may allow you to leave some messages
unspecified. I'll check on this.

>> In view of this, I wonder if a better way to formulate this test
>> might be to say something like -
>>
>> object.should_receive :method => :method_name, :with_each_of => [arg1,
>> arg2,... argN]
>>
>
> That's not a bad idea, but I suspect your tests in which you're
> calling 'puts' over and over might be a bit of an edge case.

Yeah, I think that's true. I was actually trying to debug the spec,
rather than object being tested. Still, it has turned out to be a useful
learning exercise.
07dd367bcc4ae602d640ec8c1b280df6?d=identicon&s=25 Mark Thomson (Guest)
on 2008-10-19 07:35
(Received via mailing list)
Zach Dennis wrote:
> I'm top posting... I wish I could inline post, but you provided a lot
> of generalizations for how you think things are working on your code,
> but you don't actually post concrete code (the should_receive and
> should_not_receive case you mentioned that wasn't acting like you'd
> expect). Perhaps this will help:
>
> * Mocks are not ordered by default.
>

Yeah that's consistent with what I'm seeing. I mentioned ordering only
because it was one of the things that came up in my mind as I was
searching for a way to understand the different behaviors I was seeing.

> * If you want ordered message expectations on a mock you have to
> explicitly tell them to be ordered.
>

Really, you can do that? I'm curious about how.

>
>     file.bar "3"
>     file.bar "1"
>
>
Ok got that.

>     file.bar "1"
>     file.bar "1"
>

This one kind of surprises me, but I understand your point.

>     file.bar "1"
>     file.bar "2"
>

Did you mean to say that will *not* fail since "2" will match "not 3"?
If so that's really good to know. I had imagined that should_not_receive
was purely a statement about something that doesn't happen, not about
something different happening. That may explain what I had thought was
anomalous behavior. I'll have to take another look at that.


> something that is very convoluted. Can you share the actual code and
> spec you're having issues with so we can try to provide concrete help
> back to you?
>
Steve's comments helped me understand why this approach isn't necessary.

Mark.
F86901feca747abbb5c6c020362ef2e7?d=identicon&s=25 Zach Dennis (zdennis)
on 2008-10-19 16:41
(Received via mailing list)
On Sun, Oct 19, 2008 at 1:34 AM, Mark Thomson <mark.thomson@ieee.org>
wrote:
>>
>
> Yeah that's consistent with what I'm seeing. I mentioned ordering only
> because it was one of the things that came up in my mind as I was searching
> for a way to understand the different behaviors I was seeing.
>
>> * If you want ordered message expectations on a mock you have to
>> explicitly tell them to be ordered.
>>
>
> Really, you can do that? I'm curious about how.

Checkout http://rspec.info/documentation/mocks/message_expe...

And look at the Ordering section.

>>    file.should_receive(:bar).with("3")
>> For example, the following would fail even though there are two
>
>>    file.should_not_receive(:bar).with("3")
>>
>>    file.bar "1"
>>    file.bar "2"
>>
>
> Did you mean to say that will *not* fail since "2" will match "not 3"?

Yes, that's what I meant.

--
Zach Dennis
http://www.continuousthinking.com
http://www.mutuallyhuman.com
42172acdf3c6046f84d644cb0b94642c?d=identicon&s=25 Pat Maddox (pergesu)
on 2008-10-19 17:59
(Received via mailing list)
Can you please post an example of the spec and production code that
isn't behaving as you expect?

Pat
07dd367bcc4ae602d640ec8c1b280df6?d=identicon&s=25 Mark Thomson (Guest)
on 2008-10-19 18:06
(Received via mailing list)
Zach Dennis wrote:
>
Thanks for that pointer. Somehow I never made it that far into the
document tree. If you hadn't mentioned it I would not have realized that
was there. No wonder I was having trouble!

M.
07dd367bcc4ae602d640ec8c1b280df6?d=identicon&s=25 Mark Thomson (Guest)
on 2008-10-24 06:16
(Received via mailing list)
Pat Maddox wrote:
> Can you please post an example of the spec and production code that
> isn't behaving as you expect?
>
> Pat
>

Sure, sorry been tied up with business travel this week. Here's my
controller...


----
class SubscribersController < ApplicationController

  def test

    file = File.new("test.txt", "w")


    file.puts "one"
    file.puts "two"
    file.puts "three"
    file.puts "four"
    file.puts "five"
    file.puts "six"

    render :action => "index"

  end

end

-----

and a spec containing two examples that illustrate what I was talking
about -


-----

describe SubscribersController do

    it "does not pass" do

        file = mock('file')
        File.stub!(:new).and_return(file)

        file.should_receive(:puts).with("one")
        file.should_receive(:puts).with("two")
        file.should_receive(:puts).with("six")

        get 'test'

    end


    it "does pass" do

        file = mock('file')
        File.stub!(:new).and_return(file)

        file.should_receive(:puts).with("one")
        file.should_receive(:puts).with("two")
        file.should_receive(:puts).with("six")
        file.should_not_receive(:puts).with("ten")

        get 'test'

    end

end

----

The first example fails with error message "Mock 'file' expected :puts
with ("one") but received it with ("three")" (The message seems a little
strange since "one" is clearly received, but I can live with that. I
understand the point Steve made that a mock should have all messages
specified in expectations).

The second example passes fine, which is the thing I still find
surprising. It seems as if "should_not_receive" doesn't just match the
absence of the specified message but also the presence of any message
other than the specified message.

Mark.
This topic is locked and can not be replied to.