MVC testing vs. models & Webrat+RailsStory?

Testing RSpec
1

I just read David C.'s slides from RailsConf. I was at
RailsConf, but was unable to attend the session. The slides are
excellent, and I’ve recently become VERY interested in stories. In
particular I’m a bit unhappy with regular view testing (even with the
easier notion of it in RSpec). I was wondering what folks think about
testing the full stack these days, and any recommendations you have?

Specifically, I’m wondering, or contemplating, if I do unit tests for
my models, and then I use WebRat plus RailsStory, do I even need to
then do functional testing of my controllers and views? I can see
that I might want to do some particular view testing that ensured
certain elements were on a page or something, but in terms of actually
testing the workings of my app/site, it seems model unit tests
combined with RailsStory+Webrat will do a pretty darn thorough job. I
would then consider going to Selenium to test some of the detailed
JavaScript as needed.

Thoughts? What am I missing, or what are the downsides of such an
approach?

–
Christopher B.
Cobalt Edge LLC
http://cobaltedge.com

2

On Thu, Jun 19, 2008 at 9:31 AM, Christopher B.
[email protected] wrote:

Thoughts? What am I missing, or what are the downsides of such an approach?
In the project I’m working on, we lean pretty heavily on model/helper
specs and webrat-driven plain text stories.

We still do some controller specs, but only in those instances where
we deviate from the standard REST design, or to test authentication
requirements. With a custom it_should_… example generator, that can
be as simple as:

describe FooController do
describe “GET :index” do
it_should_require_admin_access(:get, :index)
end
end

We currently have no view specs at all, though I am beginning to
regret that decision somewhat. Initially we avoided them because the
effort required to mock everything out wasn’t jiving well with rapid
prototyping, but I’m considering going back and pushing some things
into view specs using stub_model(). There are a lot of simple
requirements that are a good fit for view specs which we are currently
doing in the user stories, solely because the infrastructure was
already there.

As far as controllers go, I wouldn’t say we’ve seen any downside to
relying on stories instead of isolated specs; I’m sure you’ve seen how
90% of every controller spec ends up looking the same, and they start
to be a headache to maintain. Maybe the shoulda-esque rspec plugins
out there would help, but when I can easily just exercise the full
stack, I haven’t bothered to try. I saw one big win of not testing
controllers in isolation just this week, though, when I had to shift a
lot of an application’s view paths around; with standard controller
specs, I would’ve been spending a lot of typing updating the render
expectations, but with stories it wasn’t an issue: the right markup
either showed up or didn’t.

The primary downside for us has been speed; it takes a few minutes to
run the full story suite. The ability to selectively run just a few
stories, and then a good CI server, alleviates that for the most part.

I always ramble too much so I’ll stop here. HTH,

k

3

Kyle, thanks much for sharing your experience. You mention the speed
and so on. I’ve read that it is slow. Question: does Autotest work
the same way with stories, or have a way to detect what file(s)
changed and run the appropriate stories, or because they’re
integration tests, would it just re-run the entire story suite as it
really doesn’t have a way to correlate source files to stories? I
presume the latter.

I guess what I’d wonder, in terms of dealing with the speed issue is:

  • Can I run a single story via TextMate like I do with specs? That
    would allow me to concentrate on one particular thing if I needed
    quick turn around time.
  • I just always have autotest running, and most of the time, I just
    let it do its thing, listening for any failures (I use Mac’s speech to
    alert me :slight_smile: So, the speed isn’t so bad as I typically just keep
    working until I hear a problem, then go fix that.
  • I am a huge fan of CI servers, and have one setup for every project
    I control. I do like to know my suite passes before I push my changes
    up, but the CI server is a great way to ensure everything passes on a
    clean system. I suppose if the story runner was really taking very
    long times, that I’d be ok pushing and seeing if the CI server caught
    a problem.

I’d like to hear other’s thoughts as well, but your experience seems
consistent with my theory of the direction I want to go: unit test
models and helpers, drop controller and view tests in favor of Webrat

  • RailsStory, and then, as needed drop in RSpec view tests for things
    that can’t be as efficiently/effectively tested in stories.

Final bit/question (for now ;-)… With Webrat, does it pay attention
to whether or not div’s and such are hidden/displayed in terms of
whether it lets you enter values in them? In this particular project
I use a fair bit of disclosure style UI, where some fields are shown
after a use takes some other action or presses a button, etc. But
they’re all there, just not visible. So, wondering if I can simply
tell Webrat to put data into them, and not worry about having to
effect a button press that does its work via JavaScript. I can move
to Selenium to handle that, but that’s actually something I’m not
wanting to do yet (I’ve used it in the past, and it’s just a ton of
overhead, truly slow, etc.).

On Thu, Jun 19, 2008 at 8:41 AM, Kyle H.
[email protected] wrote:

JavaScript as needed.

into view specs using stub_model(). There are a lot of simple
controllers in isolation just this week, though, when I had to shift a

k

rspec-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/rspec-users

–
Christopher B.
Cobalt Edge LLC
http://cobaltedge.com

4

Just found answer to one of my questions below on being able to run
stories in TextMate. The new bundle for stories looks cool:

On Thu, Jun 19, 2008 at 9:56 AM, Christopher B.
[email protected] wrote:

  • Can I run a single story via TextMate like I do with specs? That
    long times, that I’d be ok pushing and seeing if the CI server caught
    whether it lets you enter values in them? In this particular project

On Thu, Jun 19, 2008 at 9:31 AM, Christopher B.

describe FooController do
requirements that are a good fit for view specs which we are currently
lot of an application’s view paths around; with standard controller
k
Cobalt Edge LLC
http://cobaltedge.com

–
Christopher B.
Cobalt Edge LLC
http://cobaltedge.com

5

Christopher B. wrote:

Just found answer to one of my questions below on being able to run
stories in TextMate. The new bundle for stories looks cool:
GitHub - bmabey/rspec-story-tmbundle: Textmate Bundle for RSpec plain text stories

Hey Chris,
There is actually a bug for running the stories that accidentally got
introduced recently… We’ll be pushing the fix out to github soon.
We are also adding more flexibility to how you can structure your
stories directories. Anyways, look for that as the running is currently
broken.

-Ben

6

On 6/19/08, Christopher B. [email protected] wrote:

Thoughts? What am I missing, or what are the downsides of such an approach?
Having spent the last 3 months working on a project which heavily
relied on Story Driven Development (currently 78 story files which
encompass 401 scenarios) one thing that I have found is that the build
has become quite slow. We made a decision early on that we wouldn’t
bother with view specs, as they felt redundant with our stories.
Looking back now I think that was a mistake. If I were to do it over
I would leverage stories for the happy path, as well as to catch
expected error conditions, but I would also rely heavily on
Controller/View/Helper specs to test all of the branches and abnormal
use cases.

Basically I think that you should test as much stuff at the unit level
as possible, and leverage the integration tools where appropriate, but
don’t overuse without expecting some decrease in performance.

–
Josh K.
phone: 509-979-1593
email: [email protected]
web: http://joshknowles.com

7

On Thu, Jun 19, 2008 at 11:56 AM, Christopher B.
[email protected] wrote:

quick turn around time.

  • I just always have autotest running, and most of the time, I just
    let it do its thing, listening for any failures (I use Mac’s speech to
    alert me :slight_smile: So, the speed isn’t so bad as I typically just keep
    working until I hear a problem, then go fix that.

I have not attempted to set up autotest + stories, but it should be
possible to set up autotest to watch stories/**/*.story and rerun the
corresponding story when that changes; like you say, it’d be
difficult to map implementation files to their stories.

I do miss autotest at times, but up+enter on your console to rerun
some focused stories is not bad at all. I use a command-line runner
without the maintenance requirements of the *.rb runner files
(GitHub - pd/story: command line runner for rspec stories), so I can pretty easily select the
individual foo.story, a full feature, or any mix I want, then just
rerun those when I’m ready. Actually, I save files so often that
waiting on the stories to catch up would probably be annoying.

I’d like to hear other’s thoughts as well, but your experience seems
consistent with my theory of the direction I want to go: unit test
models and helpers, drop controller and view tests in favor of Webrat

  • RailsStory, and then, as needed drop in RSpec view tests for things
    that can’t be as efficiently/effectively tested in stories.

As Josh K. also said, dropping the unit level specs has turned
out to be a mistake. I wouldn’t say it’s a BIG mistake, and the main
issue is just waiting too damned long before you are comfortable
pushing your code (which leads to saying “ah screw it” and then “ah
crap it failed on CI” and then …). But a mistake nonetheless.

If I started a fresh project today, I would still probably ignore view
specs early on; they’re cumbersome and take a lot of tweaking when
you’re just barely figuring out what it is you want to accomplish.
Once your ideas solidify a bit, tho, view specs using stub_model()
seem perfectly good these days. For controller specs, I think I’d
continue to do exactly what I’m already doing: spec the deviations
from the pretty-much-scaffolding REST controllers, and wrap up
authentication/authorization specs in a tidy DSL to generate that
stuff for me. Thin controllers deserve thin controller specs! =)

overhead, truly slow, etc.).
Yes, I take extensive advantage of that. So you can click ‘edit’ next
to something on the screen, which unhides a form, but the form is
already on the page; webrat is entirely unaware of the CSS and will
just fill it out and submit it. It’s a minor headache when there are
10 ‘edit’ links and 10 corresponding forms, cuz you’ll need to dig
through and be sure to fill out the right fields. Other than that,
tho, nothin but sunshine.

k