Step matchers

[pergesu]

I think we all know that the readability of steps isn’t great right
now, and in fact there’s a very recent thread that discusses just
that. It was that recent thread that prompted me to explore this a
bit.

The basic idea is that you define step matchers, which have a regex,
and then you match step names against that regex. Kind of tough for
me to explain so I’ll just link to some code

spec: Parked at Loopia
impl: Parked at Loopia

Instead of writing
Given “a user named __ who is __ years old”, “Pat M.”, 22

it allows you to write
Given “a user named Pat M. who is 22 years old”

I wrote it out as a separate matcher because it was just easiest to do
it that way while I explored this approach, no messing around with
RSpec internals to get it to really work. However if we went this
route the structure would certainly be different.

Hopefully you can get the idea from the example code. Ideally what I
would like is to write step libraries that are external to the stories
themselves. The stories would be much clearer because the
implementation would not be embedded, and the step names themselves
would make a lot more sense.

wdyt?

Pat

[pergesu]

On 10/14/07, Pat M. [email protected] wrote:

impl: Parked at Loopia
route the structure would certainly be different.

Hopefully you can get the idea from the example code. Ideally what I
would like is to write step libraries that are external to the stories
themselves. The stories would be much clearer because the
implementation would not be embedded, and the step names themselves
would make a lot more sense.

wdyt?

LOL - I just suggested something like this in the other thread you
cite. Though your idea strikes me as far more flexible and usable.
Well done!!!

Cheers,
David

[pergesu]

On 10/14/07, David C. [email protected] wrote:

spec: Parked at Loopia
RSpec internals to get it to really work. However if we went this
LOL - I just suggested something like this in the other thread you
cite. Though your idea strikes me as far more flexible and usable.
Well done!!!

This just occurs to me. If we do away with the need for special
characters/positioning, blocks, etc, we should be able to do just
this, no?

Given a savings account with 100 dollars
When the account owner asks for 101 dollars
Then the account owner should receive 0 dollars
Then the account should have 100 dollars

Look mom - no quotes!

Cheers,
David

[pergesu]

On 10/14/07, David C. [email protected] wrote:

it that way while I explored this approach, no messing around with

Then the account owner should receive 0 dollars
Then the account should have 100 dollars

Look mom - no quotes!

How would you plan to implement this? Read in a text file, strip the
leading Given/When/Then and match each line?

The one catch I see is that because stories are structured, you still
need something to demarcate steps within a scenario and scenarios
within a story.

In general, I prefer your syntax, making it look like a plain-text
spec instead of Ruby code. I question whether it’s a big enough win
to justify writing a parser instead of simply writing valid Ruby. At
this point, personally I’d say no, but I can see how it would be nice
for customers and spec readers/maintainers.

Actually, would you mind showing an example of a full spec with this
syntax? Maybe something like:

Story: Creating a post
As a user
I want post to my blog
So that I can share information with my fellow Rubyists

Scenario: anonymous user
Given no blog posts in the system

When I POST to /posts with title=Post Title, body=Post body

Then the page should show Post title
And the page should show Post body
And the page should show Posted by: 'anonymous'

Scenario: logged in user
Given no blog posts in the system
And a user named Pat
And logged in as Pat

When I POST to /posts with title=Custom Title, body=Custom body
And the page should show Custom Title
And the page should show Custom body
And the page should show Posted by: 'Pat'

Actually a parser for this would be quite simple, and I definitely
prefer it to the Ruby version. It makes sense to use the interpreter
when you’re defining step implementations inline, but if you’re not
then it doesn’t really add any value and looks/feels weird.

One other thing I thought of is that we can combine spec matchers with
your ? syntax idea. Basically, regexps don’t really get you anything
in this type of situation, since you’re going to be matching the whole
string, and regexp doesn’t convert data types for you. You could
define a step like

Step “I like to eat, eat, eat ? and ?” do |food1, food2|
@fav_foods ||= [food1, food2]
end

and then when it gets used in the spec, it’s simply
Given I like to eat, eat, eat apples and bananas

Under the hood, the ? get converted to (.*), which again works fine
since you’re matching the entire string exactly anyway. It’s a lot
more readable than full-blown regexps.

Pat

[pergesu]

On 10/14/2007 8:09 PM, Pat M. wrote:

In general, I prefer your syntax, making it look like a plain-text
spec instead of Ruby code. I question whether it’s a big enough win
to justify writing a parser instead of simply writing valid Ruby.

ANT-LR! ANT-LR! ANT-LR!

Jay

[pergesu]

On 10/14/07, Pat M. [email protected] wrote:

me to explain so I’ll just link to some code
I wrote it out as a separate matcher because it was just easiest to do
wdyt?
When the account owner asks for 101 dollars
within a story.
Story: Creating a post
And the page should show Post body
And the page should show Posted by: ‘Pat’
define a step like
more readable than full-blown regexps.

Pat

Also, a 100% plain text spec makes it easier to implement something like
Uses accounting vocabulary

for importing step libraries without it looking or feeling like real
code. That’s another feature I want out of this, but I probably won’t
be needing it this week. Still it’ll be nice to have at some point.

Pat

[pergesu]

On 10/14/07, Pat M. [email protected] wrote:

me to explain so I’ll just link to some code
I wrote it out as a separate matcher because it was just easiest to do
wdyt?
When the account owner asks for 101 dollars
within a story.
Story: Creating a post
And the page should show Post body
And the page should show Posted by: ‘Pat’
At a high level, this is exactly what I came up with:

Story: employee accrues 1 day per month

As an employee
I want to accrue 1 day per month during my first year
So that I can replenish my self

Scenario: accrual after one month
Given an employee
When employee has worked 1 month
Then employee should accrue 1 day vacation time

Scenario: accrual after 2 months
Given an employee
When employee has worked 2 months
Then employee should accrue 1 days vacation time

The Story: and Scenario: indicators are quite clear. This should be a
no-brainer to parse.

I think the real potential pitfalls in this approach have to do with
StepMatchers that have similar expressions and the user writes a step
with a one-word difference that invokes the wrong step and results in
inaccurate feedback.

That said, the ability for customers to be able to write
stories/scenarios in plain text is the holy grail IMO. That is the
promise of FIT. I’ve seen failure and success with FitNesse. The
success required customers who were able to grok tables and wikis. It
fell down when they didn’t care about these things, or when they
didn’t understand how to structure things w/ correct setups,
teardowns, etc.

A lot remains to be seen, but I think this plain text approach will
eliminate a LOT of that. Especially situations where deveoper-centric
things like setup became part of the customer facing FitNesse pages.
With the Story Runner, we’d be able to hide all of that out of the
view of the customer. Great stuff!

Actually a parser for this would be quite simple

Dead simple. It would also allow us to do away with methods like
Given, When and Then, which some people have objected to (because of
the capitalization), because the stories are no longer expressed
directly in Ruby. Internally, the parser could use a StepFactory to do
things like create_given, create_when, etc (or however we decide to
name these).

I’m really excited about this idea!

Cheers,
David

[pergesu]

On 10/14/07, David C. [email protected] wrote:

and then you match step names against that regex. Kind of tough for

Given a savings account with 100 dollars
need something to demarcate steps within a scenario and scenarios

Then the page should show Post title
And the page should show Custom body

Scenario: accrual after one month
no-brainer to parse.
fell down when they didn’t care about these things, or when they

Actually a parser for this would be quite simple
Cheers,
David

I’m working with a customer who’s got a decent-sized Rails app with
absolutely 0 lines of test code. The first thing we’ll be doing is
writing a bunch of user stories together. I’m going to do it in this
new format, so I ought to have at least a basic implementation in a
couple of days as a matter of necessity

re: step matchers that are close - hopefully we can make the step
names brittle enough that it won’t be too much of an issue. If you
saw my test code, it uses a line begin and end anchor, and because
most of the regexp is exact text then it has to match pretty
precisely. We’ll see what happens with that. Another idea I
mentioned earlier was that of “vocabularies” which would allow you to
import a step library into a story. I can certainly envision writing
two steps that have similar, or even the same name, but that would
execute differently depending on context. Sticking the steps in
different vocabularies would prevent clashes.

Pat

[pergesu]

El 15/10/2007, a las 5:49, [email protected]
escribió:

I’m really excited about this idea!

Cheers,
David

I’m working with a customer who’s got a decent-sized Rails app with
absolutely 0 lines of test code. The first thing we’ll be doing is
writing a bunch of user stories together. I’m going to do it in this
new format, so I ought to have at least a basic implementation in a
couple of days as a matter of necessity

I’ve read this thread with some interest but I don’t really get
exactly what’s being proposed, in the sense of how this would look in
practice.

• The customer/client (not necessarily with any programming
  knowledge) writes the stories in a format which is (almost) plain text.
• The developer then writes custom “step matchers”; where do they go?
• How much of parsing can be generalized and done by RSpec itself
  without requiring the developer to spend too much time writing the
  matchers?

Basically the idea of neat and readable stories is very appealing,
but I don’t really understand the mechanics of what’s being proposed.
Can someone please clarify?

Wincent

[pergesu]

On 10/14/07, David C. [email protected] wrote:

On 10/14/07, Pat M. [email protected] wrote:

Actually a parser for this would be quite simple

Dead simple. It would also allow us to do away with methods like
Given, When and Then, which some people have objected to (because of
the capitalization), because the stories are no longer expressed
directly in Ruby. Internally, the parser could use a StepFactory to do
things like create_given, create_when, etc (or however we decide to
name these).

Jay mentioned antlr. This parser is so simple though that I doubt we
would need/want that. There’s not really any parsing at all in fact.
You just look at each line, figure out what token it is, and pass the
following string to a constructor. It should be like 60 lines of Ruby
code, and we don’t have any external dependencies.

Pat

[pergesu]

On 15 Oct 2007, at 10:25, Wincent C. wrote:

• The customer/client (not necessarily with any programming
  knowledge) writes the stories in a format which is (almost) plain
  text.
• The developer then writes custom “step matchers”; where do they go?
• How much of parsing can be generalized and done by RSpec itself
  without requiring the developer to spend too much time writing the
  matchers?

I’m a bit sceptical about all this (not to suggest that Wincent
necessarily is!). I don’t fully grasp the implications of the
proposal either but superficially it smells like using a sledgehammer
to crack the rather straightforward nut of having something that
works like string interpolation.

What’s the problem with the alternating ‘string’, parameter,
‘string’, parameter, ‘string’ syntax? It might be less aesthetically
beautiful than punctuation-free plain text but conceptually it
expresses exactly what you’re trying to achieve without all of that
tedious mucking about with matching. I’m biased, I suppose; as a Ruby
programmer a big part of the conceptual beauty of examples comes from
them being written in native code, so I’d be sad to see that go out
of the window with scenarios, but I appreciate the pull of the
customer-facing aspect. Regardless I believe (without evidence) that
trying to pretend we’re doing something fundamentally different to
writing a program when constructing a scenario is a recipe for
confusion at best, and the “but customers will balk at apostrophes
and commas!” sentiment feels slightly too specious to justify it.

Cheers,
-Tom

[pergesu]

On 10/15/07, Wincent C. [email protected] wrote:

I’ve read this thread with some interest but I don’t really get
exactly what’s being proposed, in the sense of how this would look in
practice.

• The customer/client (not necessarily with any programming
  knowledge) writes the stories in a format which is (almost) plain text.

Why almost? Because there is required syntax? We’re asking them to be
able to write this:

Story: employee accrues 1 day per month

As an employee
I want to accrue 1 day per month during my first year
So that I can replenish my self

Scenario: accrual after one month
Given an employee
When employee has worked 1 month
Then employee should accrue 1 day vacation time

Scenario: accrual after 2 months
Given an employee
When employee has worked 2 months
Then employee should accrue 1 days vacation time

• The developer then writes custom “step matchers”; where do they go?

TBD. Probably in a directory under stories named steps or step_matchers.

• How much of parsing can be generalized and done by RSpec itself
  without requiring the developer to spend too much time writing the
  matchers?

At first this will be all on the developer. But this actually solves
the problem of sharing steps across stories. I’m sure that, as we gain
experience with this, people will create Step Libraries (coined by Pat
Maddox earlier in this thread, I believe) that would serve general use
well. I can especially see this evolving in the area of rails
controllers, though I would want to see them be higher level than
this:

When admin gets /users

I’d much rather see this:

When admin navigates to the user list

In this case, the step would look in a Hash like this:

paths = { ‘the user list’ => ‘/users’ }

This would mean keep the stories at the right level of abstraction and
make them easier to maintain if/when paths change.

Basically the idea of neat and readable stories is very appealing,
but I don’t really understand the mechanics of what’s being proposed.
Can someone please clarify?

There are some pieces missing. Pat’s initial description (the
beginning of this thread) recognizes Steps, but I’d like to see them
include the mechanics. Something like this:

matcher = StepMatcher.new(“/^(.) navigates to (.)$”) do
login_as arg1
get arg2
end

Then the runner would translate this:

When admin navigates to the user list

to this:

find_when(‘admin navigates to the user list’)

which would find the matcher defined above and eventually do this:

matcher.run_step ‘admin navigates to the user list’

at which point ‘admin’ and ‘the user list’ would be extracted and
thrown at the block.

Obviously there are pieces missing here, but I think that this could
prove very easy to use.

Cheers,
David

[pergesu]

El 15/10/2007, a las 14:21, “Pat M.” [email protected]
escribió:

name these).

Jay mentioned antlr. This parser is so simple though that I doubt we
would need/want that. There’s not really any parsing at all in fact.
You just look at each line, figure out what token it is, and pass the
following string to a constructor. It should be like 60 lines of Ruby
code, and we don’t have any external dependencies.

I’ve done a couple of month’s work with ANTLR (3.0) this year and
based on my experiences I’d say it’s not the right tool for this kind
of job. You use ANTLR if:

1. Parsing speed is really important.
2. You need to recognize a relatively complex grammar (most likely a
   context-sensitive language).
3. You don’t mind being tied to Java tool-chain (even though your
   output target language is non-Java, the ANTLR tool itself is Java).
4. You have lots of time on your hands to develop, debug and fine-
   tune the delicate and complex machinery that is an ANTLR-generated
   recognizer.

In short, I found ANTLR fiendishly difficult to use, so for a simple
task like this, especially one where parsing is not going to be a
bottleneck, the simplest possible solution (parsing with hand-written
Ruby) is going to be the best one.

Cheers,
Wincent

[pergesu]

On 10/15/2007 10:11 AM, Wincent C. wrote:

El 15/10/2007, a las 14:21, “Pat M.” [email protected] escribió:

Jay mentioned antlr. This parser is so simple though that I doubt we
would need/want that. There’s not really any parsing at all in fact.
You just look at each line, figure out what token it is, and pass the
following string to a constructor. It should be like 60 lines of Ruby
code, and we don’t have any external dependencies.

I’ve done a couple of month’s work with ANTLR (3.0) this year and
based on my experiences I’d say it’s not the right tool for this kind
of job. You use ANTLR if:

Yeah, I think you’re right. I’d used ANTLR about eight years ago and
got the impression from casual browsing that 3.0 not only solved all
those problems, but was moving toward a ruby-hosted toolchain. Guess
not.

Plus, I’ve never written a “real” parser, so I always try to find ways
to avoid doing so

Jay

[pergesu]

On 10/15/07, Wincent C. [email protected] wrote:

things like create_given, create_when, etc (or however we decide to
of job. You use ANTLR if:

1. Parsing speed is really important.
2. You need to recognize a relatively complex grammar (most likely a
   context-sensitive language).
3. You don’t mind being tied to Java tool-chain

Not a chance in this case.

Cheers,
David

[pergesu]

On 10/15/07, Tom S. [email protected] wrote:

necessarily is!). I don’t fully grasp the implications of the
them being written in native code, so I’d be sad to see that go out
of the window with scenarios, but I appreciate the pull of the
customer-facing aspect. Regardless I believe (without evidence) that
trying to pretend we’re doing something fundamentally different to
writing a program when constructing a scenario is a recipe for
confusion at best, and the “but customers will balk at apostrophes
and commas!” sentiment feels slightly too specious to justify it.

I can tell you from my experience with FitNesse that this is sentiment
does play out. Customers really get the idea that the ability to write
their own acceptance tests is very powerful, but their eyes glaze over
as soon as you start talking about programmatic things like setup and
teardown (which are front and center in a FitNese page).

We wanted the customer to feel ownership of the FitNesse pages - to
feel free to add a new page with a new scenario to check their
assumptions or to explore their questions about whether the fact that
the system can do A means that it can also do B. This simply never
happened. They ran the suites periodically, but they never used it as
a tool to explore the system.

Having this expression of stories and scenarios appearing devoid of
programmatic ideas has great potential to help the customers feel
ownership over stories/scenarios. Of course, there is an underlying
relationship to syntax that they’d have to understand and, ideally,
the system would help them out with (i.e. if they do write a new
scenario with steps that don’t match any existing steps, they’d be
alerted). But that’s all a bit down the road.

Thoughts?

David

[pergesu]

On 10/15/07, Wincent C. [email protected] wrote:

• The developer then writes custom “step matchers”; where do they go?
  out loud and know what it means; turn off code folding and a
  worried about this becoming very cumbersome and error-prone compared
  Scenario “…” do
  to have “strings like $this one” wasn’t bad.
  Then the runner would translate this:

get args[1]
But I’d be even happier if this weren’t stored in a separate file and
were just part of the story file:

When “$user navigates to $path” do |args|
login_as args[0]
get args[1]
end

Part of this is to separate the programming ‘noise’ from the text, so
if we do head down this path (which remains to be seen) I doubt these
would end up in the same file.

Thanks for taking the time to explain all this, David!

Well, it’s more like exploring at this point. None of this is written
in anything but the 1s and 0s in this thread.

But you’re welcome!

Cheers,
David

[pergesu]

On 10/15/07, David C. [email protected] wrote:

name these).
couple of days as a matter of necessity

without requiring the developer to spend too much time writing the
When admin gets /users
make them easier to maintain if/when paths change.
matcher = StepMatcher.new(“/^(.) navigates to (.)$”) do
find_when(‘admin navigates to the user list’)

which would find the matcher defined above and eventually do this:

matcher.run_step ‘admin navigates to the user list’

at which point ‘admin’ and ‘the user list’ would be extracted and
thrown at the block.

Obviously there are pieces missing here, but I think that this could
prove very easy to use.

I love this idea. I think Wincent’s right that it might be a bit of
trouble to maintain things in a couple different places, and in that
case I think it makes sense just to use the current way. However I
think your idea is phenomenal for sharing steps within the code base
and standardizing stories at a very high level.

Also, I know that in my first email the step matchers didn’t include
the mechanics, but I definitely thought we should combine the two.
Perhaps I didn’t express that well enough. I just wanted to share my
“here’s how we could solve the ugly args problem” idea, and figure out
the details later.

Pat

[pergesu]

On 10/15/07, David C. [email protected] wrote:

Having this expression of stories and scenarios appearing devoid of
programmatic ideas has great potential to help the customers feel
ownership over stories/scenarios. Of course, there is an underlying
relationship to syntax that they’d have to understand and, ideally,
the system would help them out with (i.e. if they do write a new
scenario with steps that don’t match any existing steps, they’d be
alerted). But that’s all a bit down the road.

Thoughts?

I for one really like where this seems to be heading. Reminds me
much of a php project with similar ambitions called Arbiter,
http://arbiter.sf.net. What’s nice about this, is the customer plays
a hand in defining what the syntax is, deepening the buy in since I
can conceivably write matchers to work with the language the
customer uses rather than attempt to constrain them into using a
new vocabulary.

I haven’t read all the posts in detail or given Pat’s pasties a thorough
study but I’d assume there would/could also be a non-match catch
all matcher that could be used to identify steps that appear to
require a new matcher or perhaps needing to be reworded, etc. ?

-Mike

[pergesu]

El 15/10/2007, a las 14:21, “David C.” [email protected]
escribió:

Given an employee
When employee has worked 2 months
Then employee should accrue 1 days vacation time

And as far as requirements go the leading whitespace could be treated
as totally optional (because you can rest assured that people will
get it wrong, use tabs instead of spaces, or spaces instead of tabs,
or mix them up etc).

• The developer then writes custom “step matchers”; where do they go?

TBD. Probably in a directory under stories named steps or
step_matchers.

My main concern here is that you’re now having to keep two files in
sync to have the stories work properly. The great thing about the
Story Runner in its current form (and RSpec too) is that you can
start off by writing a skeleton using a natural-language-like Ruby
DSL, and then you flesh it out with code to fulfill its purpose. In
other words, turn on code folding and a non-programmer can read it
out loud and know what it means; turn off code folding and a
programmer can see what’s happening inside. It’s great because your
spec files are both readable by all and executable by the computer!

But if your stories are stored in separate files, you no longer have
that. If you edit the story you have to regenerate the story’s
specification in a way that preserves existing changes. Discrepancies
between the files will break the stories. And on top of this you have
to worry about a third place, the place where you define your step
matchers (unless you define them in your “implementation” file, but
you’re still having to worry about two files then). So I am a little
worried about this becoming very cumbersome and error-prone compared
to the rest of RSpec.

If I understand the proposals correctly then this will be great for
non-programmer customers, but for programmers it will actually be
awkward and uncomfortable, which would be a shame because one of
RSpec’s great selling points is that it makes TDD/BDD pleasant.

I can’t help but thinking that the best solution will be of the form:

Story “…” do
Scenario “…” do
Given “…” do; end
And “…” do; end
When “…” do; end
Then “…” do; end
end
end

Where, whatever the format ends up being, the customer knows that the
user-editable bits are the ones between the quotes, and the bits
between “do” and “end” are for the programmer. I thought the proposal
to have “strings like $this one” wasn’t bad.

at which point ‘admin’ and ‘the user list’ would be extracted and
thrown at the block.

Obviously there are pieces missing here, but I think that this could
prove very easy to use.

I’d rather see:

matcher = StepMatcher.new “$user navigates to $path” do |args|
login_as args[0]
get args[1]
end

Where StepMatcher can easily translate:

“$user navigates to $path”

into a regex like:

/\A([^ ])+ navigates to ([^ ]+)\z/

But I’d be even happier if this weren’t stored in a separate file and
were just part of the story file:

When “$user navigates to $path” do |args|
login_as args[0]
get args[1]
end

Thanks for taking the time to explain all this, David!

(And on a side note: I recently switched to the digest version of the
mailing list, so I am aware that these replies aren’t showing up as
continuations of existing threads… sorry about that! Does anyone
know of a way to avoid this when replying to the digest?)

Cheers,
Wincent