I’ve been using rspec / rails for just over a week now, and I’m loving
the specification framework. The way I can group examples together
feels really natural, and I’m finding the TDD flow terrific.
Thus far I’ve used the describe / it “should…” syntax to basically do
TDD of my controllers, views and models, with splendid isolation using
the mocking framework. I’ve yet to write any ‘integrations tests’, in
fact I’m not yet clear how much value they have.
What originally attracted me to rspec was hearing Dan N. talk about
the plain text stories, and I had some fun in my previous life as a C#
programmer using them to drive watir tests of an ASP.NET app.
What I’m not sure about is how they’re most appropriately used in rails.
Are people generally using them, as I was in the .NET world, to drive
selenium/watir acceptance tests, or do they have value as replacements
to the rails ‘integration tests’?
How about webrat? or some other glue / tool that I haven’t heard of yet?
Just a quick answer from anyone out there using these tools would be
great in order to get a feel for the current state of the art, and some
fresh fuel for my googling!
Sorry if this post is a bit rambling… thanks for bearing with me as I
scramble up the learning curve!
Are people generally using them, as I was in the .NET world, to drive
selenium/watir acceptance tests, or do they have value as replacements
to the rails ‘integration tests’?
I’m using stories as acceptance tests and also replacing rails
integration tests. I find with stories I generally end up with good
coverage of all the things I would do for integration tests.
How about webrat? or some other glue / tool that I haven’t heard of yet?
Webrat’s great but you often have to turn to something that handles
JavaScript. I ended up using Selenium to handle this (Selenium-rc). I
find in a big ajaxy web app, webrat accounts for about 10% of my
stories.
Dealing with the running tests in both Webrat and a browser testing
framework requires some separation when running the stories. I’ve been
meaning to make a post on this…
Webrat has started to look at how its syntax can be used to drive
Selenium/Watir (checkout the latest in git). How useful this is and
where it goes I’m unsure but its interesting!
Interesting. On the surface HtmlUnit looks just like webrat - what’s the
difference?
HtmlUnit is quite different. It is a language/framework agnostic way to
test any webapp including the app’s JS. All of this, even the JS, is
in memory and does not require a browser just like webrat. It does the
JS by using Rhino, Java’s implementation of JS. At the moment, webrat
is tied to rails integration testing framework (their are forks that
allow merb testing as well that I have used successfully.) As Joseph
said their is currently some work to wrap the webrat API around
Selenuin, mechanize, etc, that would break it’s dependence on the
integration testing framework in rails. Having it tied to the
integration framework does give you some flexibility though and is most
likely faster.
To answer your original question on how people are using it… In the
past project I did I ended up with 100+ scenarios all using webrat. We
decided to make the entire site with UJS (unobtrusive JS) which lent it
self to testing it with webrat. The extra JS we added we have not yet
wrapped stories around it so we have a manual testing cycle before each
major deployment. On the current project I’m on we are going to be
using JS heavily and will most likely leverage HtmlUnit (Celerity) in
our stories to keep them all in-memory. If your app is not extremely JS
dependent then I would recommend the UJS + webrat route.
At the moment I’m yet to see something like HtmlUnit on the ruby
platform. I watch Celerity in envy
Have you seen Johnson yet?
It’s goal is to wrap Mozilla’s JS engine written in C, SpiderMonkey, in
ruby. With this equivalent solution in place like Java’s Rhino it will
be possible to create an HtmlUnit like testing framwork entirely in Ruby
and perhaps just extend webrat to get JS functionality. I have not yet
realll played around with it and I haven’t been able to find any posts
about how to use it… but it is something to keep your eye on.
Johnson seems young so, as I said earlier, we will probably try to use
the more mature HtmlUnit (a’la JRuby) to do our in-memory JS testing.
I’m using stories with Webrat, and really like it. As mentioned, it
doesn’t cover the JavaScript bit, but so far that’s ok, we don’t have
enough that I can’t just test it manually (dread!).
But, yes, since starting to use stories, I’d guess I’ve written maybe
two or three controller tests, and have wound up deleting many of my
view tests (and don’t use Rails integration tests at all). So,
essentially, what it’s boiling down to for me, is that I have my
examples for models, and then use stories for everything else. There
have been a few tricky cases to do with stories, but otherwise I just
like it far better, and feel it’s a much better and more effective way
to test since it’s going to emulate what really happens on your site.
One of the things that’s driven me to using stories so much more is
the fragile nature of the other tests, in that it seems like view and
controller tests break so much more easily with various changes,
whereas stories don’t. This likely depends on how much you test the
precise text and such on a page.
But, yes, since starting to use stories, I’d guess I’ve written maybe
two or three controller tests, and have wound up deleting many of my
view tests (and don’t use Rails integration tests at all). So,
essentially, what it’s boiling down to for me, is that I have my
examples for models, and then use stories for everything else. There
have been a few tricky cases to do with stories, but otherwise I just
like it far better, and feel it’s a much better and more effective way
to test since it’s going to emulate what really happens on your site.
I’ve not used stories in anger yet (although my current personal
project is gathering a large selection of text stories as I think of
new functionality).
The thing that’s been holding me back is the granularity.
Do you try and write a scenario for every possible case?
For example, if my story was about signing up for an account, would
you write a sign up story with scenarios for success and scenarios
for failure (and let your controller/model specs deal with the
individual reasons that the signup may fail) or would you have
scenarios for “username is taken”, “password and confirmation do not
match” etc (effectively making the other tests slightly redundant, as
Christopher mentioned)?
Its a subtlety I’m only just getting my head around, but there’s a
wealth of useful experience behind these posts.
Thanks Matt (and Ben), that’s very helpful.
I’m definitely in the “declarative, token for conversation” camp
then. So I would write a scenario for success and one for failure
and let my models (and/or controllers) deal with what failure
actually means.
Heh, I like this stuff - it’s got me enthused about programming in a
way that haven’t been since I first came across Rails.
What originally attracted me to rspec was hearing Dan N. talk about
the plain text stories, and I had some fun in my previous life as a C#
programmer using them to drive watir tests of an ASP.NET app.
What I’m not sure about is how they’re most appropriately used in rails.
I’m just getting back into RSpec, and stories are new to me. I’m
finding a
lot of inspiration from Ben M.'s article/tutorial, which uses RSpec
with
webrat:
