Hello RSpec folks,
I’ve only been introduced to the world of mock objects since Wednesday
of last week, so go easy on me if I come off as ignorant.
So, I’m a big fan of testing, especially since it has really helped to
do refactoring in the past. But, I’m running into an issue that I’m
hoping could be cleared up. Let’s say that I have some code like the
following:
mock_engine = mock(“Engine”)
mock_engine.should_receive(:ignite)
car = Car.new(mock_engine)
car.start
Internally, the car class calls “ignite” on the engine that was passed
to the constructor. We run the spec, everything is green. So far, so
good.
Now, here’s what I don’t understand: suppose I refactor the Engine class
and rename the “ignite” method to “turn_on”. If I re-run the example
from above, the test is still green - but it shouldn’t be, since the
“ignite” method doesn’t exist on the Engine class anymore.
So I’m wondering…what am I doing wrong? I feel like my tests should
help catch these sorts of things when I do a refactoring - my hunch is
that I’m missing a step here.
mock_engine.should_receive(:ignite)
from above, the test is still green - but it shouldn’t be, since the
“ignite” method doesn’t exist on the Engine class anymore.
So I’m wondering…what am I doing wrong? I feel like my tests should
help catch these sorts of things when I do a refactoring - my hunch is
that I’m missing a step here.
The step your missing is that using mocks is different then classical,
black box testing.
When you use a mock object, you’re making a tradeoff. One of the
things you gain is isolation. For instance, if you were testing a
webservice or a database, you may not care about it being around for
this test. Another thing you’re gaining is pure speed (of the
running test case) - you simply won’t have to go through the full
stack. On the other hand, that’s also what your loosing - meaning
that if that API changes, your screwed.
This is why anyone who knows anything about testing will recommend
several different layers of tests - some very close to the metal which
will use mocks (which can be repeatedly run quickly), and others which
are closer on the integration side of the spectrum.
Hi Scott,
Cool - I see what you’re saying here. The only thing that I’m a bit
confused still is that it seems like, at least if your system is
starting to get larger, you’d really want your fast unit test to help
you catch API changes like this to help you make updates faster.
Having to run a suite of more expensive integration tests just to catch
API changes seems a little funny. But I guess I’m also hoping that
there’s some way for the mocks to help with that sort of thing - it’s my
understanding that some other frameworks out there help you with that
sort of stuff. One example mentioned to me was JMock – granted, that’s
Java, but still - if it’s possible in Java, Ruby should be able to do it
too.
there’s some way for the mocks to help with that sort of thing -
it’s my
understanding that some other frameworks out there help you with that
sort of stuff. One example mentioned to me was JMock – granted,
that’s
Java, but still - if it’s possible in Java, Ruby should be able to
do it
too.
Most certainly.
David, et. all:
Why don’t we have a partial mock which will raise an error (or at
least a warning) when stubbing an object who’s class doesn’t
respond_to? the method given? I feel like this sort of simple
dependency has been brought up 1000 times on the list before, but
never been explicitly stated.
API changes seems a little funny. But I guess I’m also hoping that
Why don’t we have a partial mock which will raise an error (or at least a
warning) when stubbing an object who’s class doesn’t respond_to? the method
given? I feel like this sort of simple dependency has been brought up 1000
times on the list before, but never been explicitly stated.
WDYT?
This has come up before.
The problem is that we’re dealing with a dynamic language and the
class definition may or may not include the definition of that method
at the point that the mock framework would do such an evaluation,
especially if thing are getting mocked that might otherwise be loading
modules and extending behaviour.
I’d rather have it be a known problem, but a consistent problem, then
a partially solved problem that will inevitably cause more pain that
it does today
Having to run a suite of more expensive integration tests just to
Most certainly.
David, et. all:
Why don’t we have a partial mock which will raise an error (or at
least a warning) when stubbing an object who’s class doesn’t
respond_to? the method given? I feel like this sort of simple
dependency has been brought up 1000 times on the list before, but
never been explicitly stated.
Also, I’d be game for implementing this, and think it would be a good
default behavior for the #mock method when passed a class name, but
not a literal string.
Why don’t we have a partial mock which will raise an error (or at
least a warning) when stubbing an object who’s class doesn’t
respond_to? the method given? I feel like this sort of simple
dependency has been brought up 1000 times on the list before, but
never been explicitly stated.
WDYT?
Scott
Hey cool! I doubt you’d want me poking around in your codebase, but if
there’s somewhere you can point me to help, I’d love to take a look.
But I doubt I’d be of much help. Can’t hurt to look though.
you catch API changes like this to help you make updates faster.
Java, but still - if it’s possible in Java, Ruby should be able to
the method
at the point that the mock framework would do such an evaluation,
especially if thing are getting mocked that might otherwise be loading
modules and extending behaviour.
I’d rather have it be a known problem, but a consistent problem, then
a partially solved problem that will inevitably cause more pain that
it does today
Yeah, that’s a good point.
I wonder if possibly passing some sort of flag to the stub, which is
off by default (or on) might solve this problem.
I’d rather have it be a known problem, but a consistent problem, then
a partially solved problem that will inevitably cause more pain that
it does today
+1
This is one of the down-sides of working in a dynamic language. We
have to suck it up, IMO.
Now, here’s what I don’t understand: suppose I refactor the Engine
class
and rename the “ignite” method to “turn_on”. If I re-run the example
from above, the test is still green - but it shouldn’t be, since the
“ignite” method doesn’t exist on the Engine class anymore.
So I’m wondering…what am I doing wrong? I feel like my tests should
help catch these sorts of things when I do a refactoring - my hunch is
that I’m missing a step here.
I asked Steve Freeman (author of the original JMock) about this very
question the other week. His reply was straightfowrard - “that’s just
crap testing!”. In other words, as Scott said, if you’re using mocks
for your unit tests, you have to also use acceptance tests to prove
that the whole stack integrates.
Thanks goodness for cucumber. I have already found it a massive
blessing when refactoring.
Actually, I take back my previous comment. Where would you stub a
method on an object where the method isn’t even loaded by further
(or previous) stubbing?
Any concrete examples come to mind?
How about if you haven’t implemented that object yet?
you catch API changes like this to help you make updates faster.
Java, but still - if it’s possible in Java, Ruby should be able to
the method
at the point that the mock framework would do such an evaluation,
especially if thing are getting mocked that might otherwise be loading
modules and extending behaviour.
I’d rather have it be a known problem, but a consistent problem, then
a partially solved problem that will inevitably cause more pain that
it does today
Actually, I take back my previous comment. Where would you stub a
method on an object where the method isn’t even loaded by further
(or previous) stubbing?
In Rails, at least, it’s those damn class-macro type things which do
all the method loading, but those are never stubbed out (they can’t
be, since the code needs to be loaded first).
Actually, I take back my previous comment. Where would you stub a
method on an object where the method isn’t even loaded by further
(or previous) stubbing?
Any concrete examples come to mind?
How about if you haven’t implemented that object yet?
Actually, I take back my previous comment. Where would you stub a
method on an object where the method isn’t even loaded by further
(or previous) stubbing?
Any concrete examples come to mind?
How about if you haven’t implemented that object yet?
It would only work if given a concrete class name (i.e. mock(Foo) as
opposed to mock(‘foo’))
No idea why you couldn’t pass a flag for regression testing:
No idea why you couldn’t pass a flag for regression testing:
mock(Foo, :check_methods => true)
Scott
It seems to me this way, too - though I’m obviously biased.
It’s true that a lot of the dynamic stuff could be problematic - what if
the flag was only for checking methods directly implemented on that
class? Though I guess that’d lead down a potential dark path where you’d
then also want the “mock” method to potentially check all mixins and
validate, etc. etc.
On the other hand, maybe it could serve as a gentle remonstrance to
folks to not overdo it with the “i am an 3l337 ninj4 haxx0r”
metaprogramming? Just a thought.
the flag was only for checking methods directly implemented on that
class? Though I guess that’d lead down a potential dark path where
you’d
then also want the “mock” method to potentially check all mixins and
validate, etc. etc.
On the other hand, maybe it could serve as a gentle remonstrance to
folks to not overdo it with the “i am an 3l337 ninj4 haxx0r”
metaprogramming? Just a thought.
+1
Tests are supposed to influence the way we write production code.
Actually, I take back my previous comment. Where would you stub a
method on an object where the method isn’t even loaded by further
(or previous) stubbing?
Any concrete examples come to mind?
How about if you haven’t implemented that object yet?
Of course, this also depends on what context.
Often in Rails I find myself using mocks when I have no need to - the
dependent object is already built, and I’m using a mock just so that I
don’t hit the database at all.
Scott
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.
Can’t hurt to look though.