Hey all,
Does anyone have an example(s) of the plain text story runner for
rails? I have read David’s blog
(http://blog.davidchelimsky.net/articles/2007/10/22/plain-text-stories-on-rails)
about it but I would like to see some examples with all of the step
matchers included… I couldn’t find any more examples on google
talking about the plain text stories and the example rails app in trunk
seems to still be using the other format. Does anyone have any links or
pasties for me? Thanks,
Ben
I just ran across this blog which helped me out a lot in answering some
of my questions about the plain text story runner:
http://www.kerrybuckley.com/2007/11/07/driving-selenium-from-the-rspec-story-runner-rbehave/
The Selenium integration is also an interesting idea that you might want
to read about if you haven’t seen this post. I’m still trying to
decided if using Selenium is worth the extra effort (my main goal would
be to test the JS during the acceptance tests) and if the regular rails
integration testing is good enough. I’d be interested to hear what
other user’s on this list think in terms of Selenium or any other
browser based framework and there experience with them.
-Ben
On Nov 7, 2007 10:18 PM, Ben M. [email protected] wrote:
I just ran across this blog which helped me out a lot in answering some
of my questions about the plain text story runner:
http://www.kerrybuckley.com/2007/11/07/driving-selenium-from-the-rspec-story-runner-rbehave/
It’s great to see this conversation advancing so quickly, however
please keep in mind that this is all very young and just because
someone blog’s something (including ME) doesn’t mean that it’s “the
way”. While Kerry’s example is very clean and obviously works well for
him, it does go against the grain of the philosohy that Dan is
espousing: expressing stories in the business domain rather than the
UI domain (btw Dan, that was brilliantly put).
In my own experience with tools like FitNesse, I’ve seen success and
failure regardless of the level or focus of abstraction. In the end,
what I think drives the success of tools in this (Acceptance Testing)
space is the level of commitment and interaction that customers have
with the tools and the stories they are expressing. These are called
CUSTOMER Acceptance Tests, after all. And just as TDD is for
developers, and you’re probably doing TDD right if it keeps you
productive throughout the life of a project (including the years of
maintenance), the same goes for Stories - if the customer gets the way
the stories are expressed and you are not finding yourself battling
the environment or wasting time debugging your way through a labyrinth
of test fixtures every time something changes, then you’re probably on
to something useful.
The Selenium integration is also an interesting idea that you might want
to read about if you haven’t seen this post. I’m still trying to
decided if using Selenium is worth the extra effort (my main goal would
be to test the JS during the acceptance tests)
One cool thing about having the steps disconnected from the expression
of the stories is that you can have more than one implementation of
any given step and use the one you want based on any arbitrary runtime
conditions. There is no direct support for making this decision built
into the Story Framework, but this IS Ruby after all.
This would theoretically allow you to write stories at an abstract
level and decide whether to run each scenario in memory or in a
browser based on whether or not there is additional business value in
running them in browser.
and if the regular rails
integration testing is good enough. I’d be interested to hear what
other user’s on this list think in terms of Selenium or any other
browser based framework and there experience with them.
Generally speaking, in-browser tests tend to be incredibly brittle and
run dog-slow. They simply have no choice but to be tied directly to
low-level implementation details like html structure and element IDs.
So I’d recommend only testing what you absolutely must in-browser. But
given our ajax-laden world, “what you absolutely must” may well turn
out to be a lot!
FWIW,
David
David C. wrote:
way". While Kerry’s example is very clean and obviously works well for
him, it does go against the grain of the philosohy that Dan is
espousing: expressing stories in the business domain rather than the
UI domain (btw Dan, that was brilliantly put).
What exactly do you and Dan mean when you say that the stories should be
expressed in the business domain rather than the UI domain? (BTW, is
there a link to a Dan N. post abou that?) From my understanding you
are saying that the steps that say “user types in such and such” and
user “hits the login button” are too UI specific and don’t really have
much to do with the business domain. Is that correct?
I think that this is a part of writing stories that I am somewhat
struggling with, meaning how specific should these stories be in
explaining the view? At RubyConf during your presentation (great job,
BTW!) you mentioned how you like to spec out your views first. I agree
that is is a really nice way to drive the development process. Now with
the introduction of the story runner we are now letting the stories
drive the process. So my thinking is that you let the stories drive all
of the development, including that of the views. Is this assumption
wrong? This feels right in one way because I will not be adding a form
to a view unless the story specifies it and thereby drives BDD… On the
other hand this level of granularity seems a bit overkill and clutters
the stories a bit. Where is the happy medium, in your opinion, between
keeping the stories concise and letting them drive every aspect of the
development? Maybe I am wrong in saying that the stories should drive
EVERY aspect of the development?
into the Story Framework, but this IS Ruby after all.
This would theoretically allow you to write stories at an abstract
level and decide whether to run each scenario in memory or in a
browser based on whether or not there is additional business value in
running them in browser.
Yes, good point. The step matchers really do open up a whole new world
of possibilities.
So I’d recommend only testing what you absolutely must in-browser. But
given our ajax-laden world, “what you absolutely must” may well turn
out to be a lot!FWIW,
David
I have heard that Selenium suites can become out of hand and end of
being more hassle than what there worth. Thanks for the recommendation,
I think that sounds like a sensible one.
Thanks for taking the time to discuss this,
Ben
On Nov 8, 2007 12:17 AM, Ben M. [email protected] wrote:
David C. wrote:
… the philosohy that Dan is
espousing: expressing stories in the business domain rather than the
UI domain (btw Dan, that was brilliantly put).
What exactly do you and Dan mean when you say that the stories should be
expressed in the business domain rather than the UI domain? (BTW, is
there a link to a Dan N. post abou that?)
Sorry - there are two different threads going on about this topic
right now - the other one is on the rspec-devel list and that was
where Dan posted that statement:
http://rubyforge.org/pipermail/rspec-devel/2007-November/004259.html
Keep in mind that User Stories are oft described as “a token for a
conversation”, and that even with acceptance criteria spelled out,
whether the button says “Create New Account” or “Enter” is rarely of
sufficient business value to express that in a story. That’s the sort
of detail that comes out when you say to your customer “the entry form
is done, why don’t you come by so I can show it to you,” as opposed to
in the iteration planning meeting.
In the end it really does boil down to what the customer feels is
necessary in order to accept the software. If the customer really DOES
care about what the button says, he or she may want that expressed in
a story. But even then, saying “And I press Create New Account” still
leaves things flexible enough so that you could be describing a web
app, a GUI app, a touch-screen app, etc. The only thing this doesn’t
work for would be a command line app. So maybe “And I tell the system
to Create New Account” would leave room for that as well. Or maybe,
“And I fold my arms and command the system to Create New Account.” I
kinda like that one!
From my understanding you
are saying that the steps that say “user types in such and such” and
user “hits the login button” are too UI specific and don’t really have
much to do with the business domain. Is that correct?
Unless the business IS software, like a text editor, yes.
I think that this is a part of writing stories that I am somewhat
struggling with, meaning how specific should these stories be in
explaining the view? At RubyConf during your presentation (great job,
BTW!) you mentioned how you like to spec out your views first.
Ah - confusion. I DO like to spec my views first - when I get down to
the Spec Framework.
In our example at RailsConf EU, we talked about a Cup Tracker (as in
Rugby, which was going on at the time). Take a look at the example
story:
http://rspec.rubyforge.org/svn/branches/railsconfeu2007/tags/chapter1/stories/plan_cup.rb
This story really rides this whole line very nicely. We’re describing
software that presents a view of something abstract, and we do so with
some detail about what it presents - but there is nothing in it that
says “when I click this button” or “when I enter this text.”
Now look at this version (same story, w/ the steps filled in):
http://rspec.rubyforge.org/svn/branches/railsconfeu2007/tags/chapter4/stories/plan_cup.rb
Note how the implementations of the steps are starting to get into the
views. That’s where that level of detail starts to play out.
Now look at this:
That spec was arrived at with the very granular red-green-refactor
process of TDD with a focus, naturally, on behaviour, design and
documentation (It just occurs to me that behaviour, design and
documentation are B, D and D - interesting …).
Now some people look at that spec and scream “holy crap - look at all
the duplication - all that mocking - so brittle!.” I won’t disagree
that there is lots of duplication with other parts of the system, a
lot of mocking, and changes to the views would require changes to this
spec (brittle). And, looking back at this, I might want to break that
spec up into more granular examples. While it speaks very well looking
at the code, running it would only tell you that “/cups/show.rhtml
should display the chart.”
But check this out! The implementation of that spec, with all that
mocking, IS the spec for the controller and model. That single spec
tells us about the structure that the model should expose (NOT the
actual structure necessarily - just what it should expose to things
like views that want the models to be easy to use) and what the
controller should provide for the view.
So this is all about discovery - starting from the outside and moving
in. Implementing the steps that describe domain concepts help us to
discover some details of the views. Implementing the view specs help
us to discover the services we want from our controller and the APIs
we want from our model. I find that to be very compelling.
Where is the happy medium, in your opinion, between
keeping the stories concise and letting them drive every aspect of the
development? Maybe I am wrong in saying that the stories should drive
EVERY aspect of the development?
Well, they should drive every aspect, but not directly. You’re not
going to describe database table structure in your stories, but in the
end those structures end up as they do because of the stories.
The Selenium integration is also an interesting idea that you might want
to read about if you haven’t seen this post. I’m still trying to
decided if using Selenium is worth the extra effort (my main goal would
be to test the JS during the acceptance tests)
Generally speaking, in-browser tests tend to be incredibly brittle and
run dog-slow. They simply have no choice but to be tied directly to
low-level implementation details like html structure and element IDs.So I’d recommend only testing what you absolutely must in-browser. But
given our ajax-laden world, “what you absolutely must” may well turn
out to be a lot!
I have heard that Selenium suites can become out of hand and end of
being more hassle than what there worth. Thanks for the recommendation,
I think that sounds like a sensible one.Thanks for taking the time to discuss this,
My pleasure!
Cheers,
David
It seems that what I’m coming to understand of the direction of this
story concept is that there is a lot of emphasis being put on ensuring
we keep things at the business level. I can appreciate the elegance
of it certainly, but when I think of what I would really want to gain
from this story-based testing process, I feel like it cuts out room
for some really cool ideas. Maybe you can help me see where they
would fit.
I have in mind a scenario where a customer says “I tried to log in and
it blew up!” I take what they say with a business perspective and
check my story and say “hey I’ve got a login story and it works” but
that doesn’t leave room for the fact that, slighty away from the norm
of the usual login procedure, this particular user clicked another
link in the middle of the process that changed some background
variable, breaking the process at the final login step. I would then
want to write a story similar to the usual login, but with the ACTUAL
details of what the customer did to ensure I could duplicate, fix, and
then never have this problem creep up again. My story, in order to be
of use to me, would HAVE to have steps like “click this button”
because, quite frankly, as a developer I find that’s the sort of thing
that can break my code and so I would want a story to account for it.
A user following my intended user-flow, then deviating in a way I
didn’t predict. From the business end, they still followed the login
process, they just had one little different quirk along the way.
Is there a way to get this kind of thing but still keep to the ideal
business perspective? Would I just simply “word” it in a way that the
simple little thing they did had a business description? Or is that
an indication that I have stupid things on my page that shouldn’t even
be an option? Or is there a “happy medium” here as well between
business and ui perspectives of stories?
Glenn
On Nov 8, 2007 10:01 AM, Glenn F. [email protected] wrote:
check my story and say “hey I’ve got a login story and it works” but
A user following my intended user-flow, then deviating in a way I
didn’t predict. From the business end, they still followed the login
process, they just had one little different quirk along the way.Is there a way to get this kind of thing but still keep to the ideal
business perspective?
Ideals are just that - ideals.
While we’re exploring the boundaries of this and trying to understand
pragmatic use, keep in mind that it is just a tool. The Story
Framework is never going to say to you “no, no, no - you can’t use the
word ‘button’ in a story.” Nor will the Story Police come around and
arrest you.
That said, if the thing that the user did can be expressed in business
terms, that might be more useful in the end because the story becomes
more portable. Say, for example, you start with a web app but
eventually the business invests in making it a desktop app or a kiosk
app. It’d be nice to make sure the new version doesn’t have the same
problem you are trying to expose with this story.
Thanks a lot David, this helps a lot. I have been trying to do just as
you recommended and letting the stories drive every aspect implicitly
without me explicitly defining the model methods in it.
The confusion I was having was that when you say you are going to follow
the failing stories to drive the process the first thing that usually
will fail will be in a model… Meaning, you usually have a given clause
that uses the model to set up the DB. Pat M., in his Story Runner
screencast, showed how to effectively stub out the model in the story
runner with a “class ModelName; end” type approach. However, this does
not always work depending on the type of methods you will be calling on
the model… Hmm… actually I guess you could just add something like
this, that would make that class/object act like a mock that receives
anything:
#Top of story file that uses UndefinedModel
class UndefinedModel
def initialize(*args); end
def method_missing
#I eat all methods
end
end
So stuff like this should work just fine:
Given…
u = UndefinedModel.new(:some_attribute => “value”
u.some_other_method
Doing this I suppose would allow the story runner to pass all of the
steps until it hit the view steps which in that case, like you
suggested, you could drop down to the view specs and then work your way
towards defining the real model.
I’ll play around this approach and see where it takes me. It should be
easy to leverage the current mocking/stubbing framework and create some
helpers to allow for such a process to be relatively painless. Thanks
again!
-Ben
Hi Glenn.
On Nov 8, 2007 4:01 PM, Glenn F. [email protected] wrote:
check my story and say “hey I’ve got a login story and it works” but
that doesn’t leave room for the fact that, slighty away from the norm
of the usual login procedure, this particular user clicked another
link in the middle of the process that changed some background
variable, breaking the process at the final login step. I would then
want to write a story similar to the usual login, but with the ACTUAL
details of what the customer did to ensure I could duplicate, fix, and
then never have this problem creep up again.
I completely agree.
My story, in order to be
of use to me, would HAVE to have steps like “click this button”
because, quite frankly, as a developer I find that’s the sort of thing
that can break my code and so I would want a story to account for it.
Yes, exactly. In this example, the UI domain *is *the most useful domain
to
express the (bogus) behaviour, so it’s the right thing to do. The
scenario
might be something like:
Scenario: press the Explode button whilst filling in form
Given I navigate to the Submit order page [I’m loving these infix
parameters!]
And I set first name to Dan and second name to North
When I click the Explode button
Then nothing should happen
And the current page title should be Submit order
Now, the whole of this scenario is in UI domain terms. It doesn’t matter
what business function I’m trying to achieve here - the description is
about
some bogus behaviour in my implementation (the Explode button should be
disabled whilst I’m capturing data)
So it’s fine to express scenarios in either business domain terms to
describe actual business flow, or UI terms to describe the user’s
interactions with the application. The confusion starts when you mix the
two
domains in the same story or at least in the same scenario.
A user following my intended user-flow, then deviating in a way I
didn’t predict. From the business end, they still followed the login
process, they just had one little different quirk along the way.
Yes, and it’s that quirk in this particular UI that you want to
capture
with this scenario.
Is there a way to get this kind of thing but still keep to the ideal
business perspective?
You’re trying to do a different thing here. The audience is different
(probably a tester or UI designer rather than a business stakeholder)
and
they’ll be using a different vocabulary.
Would I just simply “word” it in a way that the
simple little thing they did had a business description? Or is that
an indication that I have stupid things on my page that shouldn’t even
be an option? Or is there a “happy medium” here as well between
business and ui perspectives of stories?
See above - as long as you keep the distinction between domains when you
are
describing a scenario, you’ll probably be ok.
Glenn
Cheers,
Dan
That was exactly what I needed, thanks a lot!
Glenn
On Nov 8, 2007 8:01 AM, Glenn F. [email protected] wrote:
of use to me, would HAVE to have steps like “click this button”
because, quite frankly, as a developer I find that’s the sort of thing
that can break my code and so I would want a story to account for it.
A user following my intended user-flow, then deviating in a way I
didn’t predict. From the business end, they still followed the login
process, they just had one little different quirk along the way.
Hey Glenn,
Halted/canceled business processes are sometimes just as, or even
more, interesting than completed business processes. So for example,
in your login story you could have a “canceled login” scenario.
Scenario user canceled the login process
Given the user goes to the login screen
And the user canceled login
When the user submits his credentials
Then …
So now you have a story that captures what’s going on in business
terms. Maybe you don’t want the behavior of them canceling it, and
perhaps you call it “suspends the login process” so that when he
finally submits his credentials the authentication goes as expected.
If you have a bunch of ways in which the user can suspend the login
process, you could create a separate “suspend login” story. Put each
way the user can suspend the process in its own scenario. Of course,
if you have many ways that the user can suspend logging in, your
authentication is probably too tightly coupled to the rest of your app
anyway.
Pat