Step matchers

[pergesu]

On 10/15/07, Alvin S. [email protected] wrote:

it that way while I explored this approach, no messing around with

When the account owner asks for 101 dollars

When the accont owner asks for 101 dollars

How can we help the user to recognize the typo?

At first I would imagine you just get an UnknownStep error (or
similar). That would trigger a conversation w/ a developer who would
peruse the step matchers. Perhaps the next step would be a command to
print out all the step matcher expressions so it becomes easier to
find the expression (or perhaps that this When has no step defined for
it yet). Eventually we could make it smarter, but that would be a way
down the road.

[pergesu]

Message: 7

On 10/14/07, Pat M. [email protected] wrote:

spec: Parked at Loopia
RSpec internals to get it to really work. However if we went this
LOL - I just suggested something like this in the other thread you
Then the account owner should receive 0 dollars
Then the account should have 100 dollars

Look mom - no quotes!

I like the ideas presented so far.

What happens when a user makes a typo in a step?

eg: a missing ‘u’ in account

When the accont owner asks for 101 dollars

How can we help the user to recognize the typo?

Alvin.

[pergesu]

El 15/10/2007, a las 17:01, “David C.” [email protected]
escribió:

Part of this is to separate the programming ‘noise’ from the text, so
if we do head down this path (which remains to be seen) I doubt these
would end up in the same file.

Thinking about this, the only way to really know how all this will
work is to try it out. That is, we can speculate about what users
will do depending on what kind of “interface” we give them, but to
really find out the only way will be to actually put it into practice.

But having to trial these two possibilities isn’t as onerous as it
may sound. You see, if you want to separate the “noise” from the
story then you’re going to need to “compile” it into an intermediate
form anyway. That is, you might start with “source” file, “stories/
foo.story”, that looks like:

Story: …
Scenario: …
Given: …
When: …
Then: …

But you’ll need to “compile” that into something executable at some
point anyway (probably “stories/foo.rb”, or, if you prefer, in a
subdirectory):

Story “…” do
Scenario “…” do
etc…

The programmer then fleshes out the “rb” file, working in a way very
similar to how he/she can work today with describe/it blocks,
starting with the skeleton and gradually adding the block bodies.

If the customer updates the story file then the corresponding Ruby
file should be updated too. I personally hate this kind of fragile
dependency, but it may be a necessary evil. Doing it manually is
error-prone, and I suppose this could be done in an automated fashion
but the tool for doing it would necessarily be relatively “dumb”;
rigorously parsing Ruby is a non-trivial task, so the parser would
have to be “fuzzy”, and I guess that makes that approach error-prone
too. But it seems an inevitable consequence of having non-executable
stories.

Developing a little bit the idea I previously posted, rather than
defining StepMatchers explicitly in this Ruby file it would probably
be nice if the programmer could just write something like:

Given /\A(.+) navigates to (.+)\z/ do |args|
login_as args[0]
get args[1]
end

In other words, that the whole StepMatcher thing be effectively
encapsulated behind the Given, When, Then classes. For a line in the
story file like this:

Given: “admin navigates to user list”

We could start out with a basic skeletal Ruby implementation like this:

“admin navigates to user list”

Given /admin navigates to user list/

And the programmer then customizes the regex and provides a body. The
comment is just a possible idea, so that you can see which story line
the regex was originally based on even after you’ve customized the
regex.

The other thought which occurred to me was that RSpec itself really
has two groups of “customers”. The developers who use RSpec are one
group, and the developers’ customers is the other. If this idea works
then we can provide a developer-friendly interface (.rb files) and a
friendly-to-non-developers interface as well (.story files).

Anyway, just some ideas that I wanted to throw out there.

Cheers,
Wincent

[pergesu]

On 10/15/07, James H. [email protected] wrote:

able to write this:
Then employee should accrue 1 day vacation time

other words, turn on code folding and a non-programmer can read it
accompanying code, and then opened up various folds to show the
implementation.

I don’t want my customers to need a code editor to look at specs. I
also don’t want them to have to understand what :type => RailsStory,
%{this stuff}, and do…end are.

Ideally, I would show a customer some stories I wrote up, explaining
that they’re organized that way “because it helps keep my thinking
organized.” Then I run a command and it says all the stories pass.
Then I take out some line, run the code again, and a message pops up
saying, “The page should show ‘page name’ but it didn’t.” Then a
lightbulb goes off for my customer.

The whole point of RSpec, to me, is mapping the code as closely as
possible to what goes on in my brain. Implementing specs in Ruby is
great because I’m a programmer, I think that way, and I want the
facilities of a powerful language. Stories, on the other hand, are
customer-facing. If we want to map stories as closely as possible to
how customers think, that means using plain text and no code. This
hasn’t really been possible with the available tools in the past, and
I think we’re at a point where it finally is a reality. That’s what I
find so exciting about this. If you need to use code to express
certain parts, you’re better off just using a real language. That’s
where fitnesse falls apart in my mind. The goals are noble, but you
can’t do everything entirely in text/tables and you end up with leaky
abstractions. But now, perhaps you can express everything entirely
in plain text, and that is immensely powerful.

Pat

[pergesu]

On 10/15/07, Wincent C. [email protected] wrote:

• The developer then writes custom “step matchers”; where do they go?
  out loud and know what it means; turn off code folding and a
  programmer can see what’s happening inside. It’s great because your
  spec files are both readable by all and executable by the computer!

Just wanted to chime in and say, as a regular user of the folding
features of the editor, this is what really resonated with me when
Story Runner was first introduced. In fact, I recently did a short
presentation to some coworkers and did exactly what is described here:
put a story up with folding turned on to show the “story” without
accompanying code, and then opened up various folds to show the
implementation.

James

[pergesu]

On 10/15/2007 6:11 PM, Pat M. wrote:

also don’t want them to have to understand what :type => RailsStory,
%{this stuff}, and do…end are.

But how hard would it be to allow .story files to contain an eval block?
Then, programmers who are writing stories themselves can keep them in
one file, and those with end-customers can use two files.

Maybe there’s even some way to have the syntax of these blocks match
Ruby well enough that today’s editor-folding-modes work. Dunno.

Jay

[pergesu]

On 10/15/07, Pat M. [email protected] wrote:

Well, I’m not sure that anyone was suggesting that we ditch the
current way. I certainly wasn’t. If you want to embed code then you
can.

Actually - if we do go down this path, I’d like to see the embedded
code go away. The structure that exists right now is built around
supporting embedded code. The structure needed in the case of using a
parser could be much simpler and, personally, I don’t see a reason to
be supporting both approaches if the parsing approach has enough
flexibility built into it that we can essentially hook up any
arbitrary code to it.

When you realize that you don’t need to embed code (because the steps
are extracted elsewhere), the reasons for expressing stories in code
at all falls away.

Hear, hear!

Cheers,
David

[pergesu]

On 10/15/07, Jay L. [email protected] wrote:

I don’t want my customers to need a code editor to look at specs. I
also don’t want them to have to understand what :type => RailsStory,
%{this stuff}, and do…end are.

But how hard would it be to allow .story files to contain an eval block?
Then, programmers who are writing stories themselves can keep them in
one file, and those with end-customers can use two files.

Well, I’m not sure that anyone was suggesting that we ditch the
current way. I certainly wasn’t. If you want to embed code then you
can.

Although, this new way solves a more common problem than just making
specs pretty for customers. Sharing steps right now is kind of a
pain. You have to write a helper method somewhere, and then in every
single story file you have to do

Given “a user named”, “Pat” do |name|
create_user name
end

Which is still too much duplication for my taste.

When you realize that you don’t need to embed code (because the steps
are extracted elsewhere), the reasons for expressing stories in code
at all falls away.

Pat

[pergesu]

On 10/15/07, Zach D. [email protected] wrote:

To me adding a story parser to parse a text file adds overhead to the
rspec team and to developers and customers using it. In a way I fear
that the textual freedom of a raw text file will lead to many gray
area’s both on the rspec’s implementation of it and also for
developers trying to explaining to their customers why or why they
cannot do something. I don’t know if this is greater then the
conversation that already has to take place though. I personally don’t
know customer’s who write the final draft of an acceptance test.

The parser will be trivial. You’re looking at 60 lines tops.

Wincet Colaitua brings up a good point [3] in regards to StepMatchers:

“My main concern here is that you’re now having to keep two files in
sync to have the stories work properly.”

Perhaps there was some confusion with how I implemented that initial
StepMatcher. I’ve since explained that ultimately it should contain
both the matching and mechanics. Unless Wincent was referring to
keeping the plain-text stories in sync with the StepMatchers. On the
surface that sounds valid, but at the same time if you’ve got

Given “a user named”, “Wincent” do |name|
@user = User.create :name => name
end

… some other scenario

Given “a user named”, “Pat”

You still have to keep stuff in sync. It’s easy here of course
because it’s all in the same file…but then what happens when you
recognize you’re using this “a user named” step in every single one of
your stories? You extract it to another file, and then now you have
to make sure the story steps are in sync with the extracted steps.

• creates an easy one to one mapping between a description and a method

to find a method. If one exists, call it. If one doesn’t exist strip
the first word, and try again. Repeat until there are no words left.
If no method was matched then raise an exception with the original
description string.

See, I would much rather mentally map
Given a user named Wincent to
StepMatcher.new(“a user named ?”)
than to
def a_user_named___(name)

It seems to me that mapping to a method leaves you with the sync
issues but in a far less readable format. Another thought is that if
you have a full-fledged object, you can do basic things like
decomposing methods.

The idea behind this is that you create a clean way to write and
maintain Story’s. It minimizes the “cody” aspects while still allowing
you to write executable ruby code.

The main point I’m trying to get across, and I believe Dave is (obv
jump in if I’m wrong), is that once we start making these sorts of
enhancements then there’s no reason to make it Ruby executable code
anymore. If we don’t need do…end blocks, and we don’t need
arguments, there’s really nothing left and the Ruby interpreter is no
longer adding any value and should be refactored away.

The helper methods which are implemented follow a simple convention
for translating a sentence description into a helper method and the
implementation of the story’s are implemented in well named helper
methods (which i think adds to maintainability over StepMatchers).

How does auto-mapping to method names add to maintainability over
auto-mapping to strings? I’m not seeing that, and I’d like to
understand your perspective.

Pat

[pergesu]

On 10/15/07, Pat M. [email protected] wrote:

The parser will be trivial. You’re looking at 60 lines tops.
surface that sounds valid, but at the same time if you’ve got
because it’s all in the same file…but then what happens when you

http://continuous.rubyforge.org/svn/trunk/test_unit_story_runner/

• Scenario’s do use do/end blocks to nest their story parts.
  Given a user named Wincent to
  StepMatcher.new(“a user named ?”)
  than to
  def a_user_named___(name)

It seems to me that mapping to a method leaves you with the sync
issues but in a far less readable format.

I don’t like:
def a_user_named___(name)

I prefer:
def a_user_named_wincent
@user = a_user_named “Wincent”
end

IMO arguments for helper method arguments should be used carefully. I
don’t think things that simply identify should be used as arguments. I
would rather use things that affect the outcome or behavior of what I
am trying to test. For example:

Given “a user named Wincent”
When “he makes a guess of”, 200_000

def make_a_guess_of(guess)
# …
end

Another thought is that if
you have a full-fledged object, you can do basic things like
decomposing methods.

Are you relating this to StepMatchers or another concept? I see
StepMatchers as a way to decompose based on a string pattern, not
necessarily method names.

longer adding any value and should be refactored away.
Syntax highlighting is a very nice value to be added for a developer.
Although it is a secondary citizen to what is trying to be achieved.
Right now I’m teetering on the fence of this. Is any value lost to the
developer worth the value picked up by the customer is the argument I
am trying to consider. I don’t have an answer yet. Still mulling it
over.

The helper methods which are implemented follow a simple convention
for translating a sentence description into a helper method and the
implementation of the story’s are implemented in well named helper
methods (which i think adds to maintainability over StepMatchers).

How does auto-mapping to method names add to maintainability over
auto-mapping to strings? I’m not seeing that, and I’d like to
understand your perspective.

I am worried about maintainability as the list of possible arguments
grow. One argument is pretty straight forward.
StepMatcher.new(“a user named ?”) # username

It’s common to find permutations of story part descriptions and based
on where those ? are it could become cumbersome to track down:
StepMatcher.new(“a ? named ?”) # account type, name

This and others like it could make it difficult to track down
StepMatchers. The lack of place holders allowed in method names makes
it an easier mapping then StepMatchers, even if is only by a small
margin.

I’m not against StepMatchers, just trying to determine if the value
they add outweighs the burden they add. I do like think of step
matchers in the syntax of:

step “a user named ?” do |username|

end

[pergesu]

I agree with the start of this thread with Pat M. stating,

“I think we all know that the readability of steps isn’t great right
now…”

My argument in the thread [0, 1] that may have sparked Pat’s initial
implementation of StepMatcher’s asked the question about what value
is being added by adding arbitrary arguments to story part
descriptions. It seems that adding arbitrary arguments, and increasing
the “cody-ness” in our story files is a step in the wrong direction. I
like where Pat is going.

David C. has pointed out that doing something similar to what
Pat has implemented could allow us to remove quotes completed as the
stories themselves could live in a completely “code-free” text file
[2].

Although this is the ideal it seems to be a less pragmatic solution
then one which strives to accomplish both the customer ideal (or holy
grail as Dave refers to it) and one that works well with developers
and their tools. A more pragmatic solution appears to be striking a
good balance between both audiences. A way to allow customers to
clearly define and update stories and a way for developers to
implement them without becoming burdened with unnecessary steps in the
process.

To me adding a story parser to parse a text file adds overhead to the
rspec team and to developers and customers using it. In a way I fear
that the textual freedom of a raw text file will lead to many gray
area’s both on the rspec’s implementation of it and also for
developers trying to explaining to their customers why or why they
cannot do something. I don’t know if this is greater then the
conversation that already has to take place though. I personally don’t
know customer’s who write the final draft of an acceptance test.

Wincet Colaitua brings up a good point [3] in regards to StepMatchers:

“My main concern here is that you’re now having to keep two files in
sync to have the stories work properly.”

It is difficult to maintain consistency among plain english sentences
when there is no meaning behind the sentence structure. It seems to be
an additional burden to the developer to have remember that how to
conform to a potential regular expression defined somewhere in a step
matcher. It is not a simple one to one translation, especially as
different people write different kinds of StepMatchers. Each
StepMatcher may have it’s own idiot which could become painful to
maintain or figure out after the fact.

Wincent also brings up in the same post [3]

“The great thing about the Story Runner in its current form (and RSpec
too)
is that you can start off by writing a skeleton using a
natural-language-like
Ruby DSL, and then you flesh it out with code to fulfill its purpose”

This is the part of Story Runner that I want to hold onto. In my
opinion a worthwhile balance to start exploring is one that does the
following things:

• creates an easy one to one mapping between a description and a method
• remove do/end blocks, they are not needed and are a negative for the
  customer
• keeps story’s clean and tidy, not full of unnecessary code artifacts

An implementation of this is the test/unit StoryRunner project that
I’ve been working on.
http://continuous.rubyforge.org/svn/trunk/test_unit_story_runner/

It is similar to rspec’s implementation but it does the three things
listed above. A nicely color formatted example can be found at
http://pastie.caboo.se/107537

A few things to note about the example:

• Story’s do not use do/end blocks to organize scenarios. It is not
  needed. Scenario’s belong to the last Story declaration preceding it.

• Scenario’s do use do/end blocks to nest their story parts.

• Descriptions used for Given, Then, When and And translate to a
  method call. The rules are simple. Take the description, downcase it
  and strip out punctuation. Replace whitespace with underscores and try
  to find a method. If one exists, call it. If one doesn’t exist strip
  the first word, and try again. Repeat until there are no words left.
  If no method was matched then raise an exception with the original
  description string.

The idea behind this is that you create a clean way to write and
maintain Story’s. It minimizes the “cody” aspects while still allowing
you to write executable ruby code.

The helper methods which are implemented follow a simple convention
for translating a sentence description into a helper method and the
implementation of the story’s are implemented in well named helper
methods (which i think adds to maintainability over StepMatchers).

The test/unit story runner still allows argument passing, but it is
encouraged when it adds value and not for arbitrary pieces of
information which do nothing for the test but increase the difficulty
of readability.

I really enjoy this thread and I’m very excited to see what will come
out of these recent threads,

–
Zach D.
http://www.continuousthinking.com

0 - http://rubyforge.org/pipermail/rspec-users/2007-October/003690.html
1 - http://rubyforge.org/pipermail/rspec-users/2007-October/003703.html
2 - http://rubyforge.org/pipermail/rspec-users/2007-October/003708.html
3 - http://rubyforge.org/pipermail/rspec-users/2007-October/003729.html

[pergesu]

El 16/10/2007, a las 2:44, “Pat M.” [email protected]
escribió:

keeping the plain-text stories in sync with the StepMatchers.
Yep, that’s what I was referring to.

If the conventions are kept reasonably tight then this concern could
be largely ameliorated with a good automated tool for generating an
“.rb” file from an “.story” file, or updating an existing “.rb” file
from an updated “.story” file.

So, for example, if the story file contained (excerpt only):

Scenario: “foo bar baz”

The tool would generate a file with:

Scenario “foo bar baz” do
end

If the customer added another scenario and the tool was re-run, then
another Scenario block would be added to the “.rb” file.

Now imagine that the programmer modifies the scenario in the “.rb” file:

Scenario “foo bar ?” do |arg|
end

Here I’m assuming that the whole matcher machinery you brought up is
effectively handled behind the scenes by the Scenario module. The
string basically defines a matcher (on the fly if need be) that uses
a regex like this one, where “?” (or whatever symbol we choose; it
could be “$baz”, for example) has been replaced with “.+”:

/foo bar .+/

Ideally, the tool would be smart enough to recognize that the
modified scenario in the “.rb” file corresponds to the one in the
“.story” file because:

“foo bar baz”

Matches this regexp.

The tool would complain about cases where the “.rb” file had more
scenarios than the “.story” file, listing the excess ones (this could
happen if the customer deletes a scenario or modifies it enough that
it no longer matches up.

I really think it’s important that this thing, whatever it ends up
looking like, be nice for programmers to use, not just programmer’s
customers.

Cheers,
Wincent

[pergesu]

On 10/15/07, Wincent C. [email protected] wrote:

Scenario “foo bar baz” do
end

Personally, I’d prefer to avoid this sort of mapping. It seems to me
that the simplest thing would be to have plain text stories/scenarios
with libraries of step matchers.

[pergesu]

On 10/15/07, Zach D. [email protected] wrote:

keeping the plain-text stories in sync with the StepMatchers. On the
You still have to keep stuff in sync. It’s easy here of course

Ruby DSL, and then you flesh it out with code to fulfill its purpose"
I’ve been working on.

See, I would much rather mentally map

Given “a user named Wincent”

jump in if I’m wrong), is that once we start making these sorts of
over.

it an easier mapping then StepMatchers, even if is only by a small
margin.

I’m not against StepMatchers, just trying to determine if the value
they add outweighs the burden they add. I do like think of step
matchers in the syntax of:

step “a user named ?” do |username|

end

What are your thoughts about using symbol identifiers rather then
question marks? I think this increases readability and gets rid of
ambiguity at least for me.

step “a user named :username” do |username|
…
end

[pergesu]

On 10/15/07, Zach D. [email protected] wrote:

What are your thoughts about using symbol identifiers rather then
question marks? I think this increases readability and gets rid of
ambiguity at least for me.

step “a user named :username” do |username|
…
end

I think that I’m leaning towards this one, which is similar.

define_given “a $account_type account with $amount dollars” do
|account_type, amount|
…
end

Reasoning:

define_given expresses the fact that we’re defining a step better than
‘given’ does. It also defines a specific type of step, which I think
will help to clarify intent, as well as organize steps in larger
groups.

“I prefer the $dollar_sign_prefix” because it feels more like a token
to me than “a :symbol embedded in a string does”

[pergesu]

On 10/15/07, Wincent C. [email protected] wrote:

[pergesu]

On 10/15/07, James H. [email protected] wrote:

On 10/15/07, Wincent C. [email protected] wrote:

I really think it’s important that this thing, whatever it ends up
looking like, be nice for programmers to use, not just programmer’s
customers.

+1

I think what we’re proposing would be nice for programmers. Right now
we have stories that look like this:

http://pastie.caboo.se/107589

This is from the BDD talk we did at RailsConfEU. Now on the one hand,
we have readable strings and good reuse of steps. But on the other
hand, there is simply no way to look at that file and instantly grok
this:

Story: Plan cup

As a cup organizer
I want to declare how many teams are in the cup
So that I can lay out the chart

Scenario: set up a 4 team cup structure
Given a new cup with max teams of 4
When I ask to see it
Then It should have a row count of 3
And The round 1 row should have a column count of 4
And The round 2 row should have a column count of 2
And The winner row should have a column count of 1

Scenario: set up an 8 team cup structure
Given a new cup with max teams of 8
When I ask to see it
Then It should have a row count of 4
And The round 1 row should have a column count of 8
And The round 2 row should have a column count of 4
And The round 3 row should have a column count of 2
And The winner row should have a column count of 1

Of course, this would be more readable:

Story: Plan cup

As a cup organizer
I want to declare how many teams are in the cup
So that I can lay out the chart

Scenario: set up a 4 team cup structure
Given a new cup with room for 4 teams
When I ask to see the chart
Then the chart should have 3 rows
And the 1st row should have 4 columns
And the 2nd row should have 2 columns
And the 3rd row should have 1 column

Scenario: set up an 8 team cup structure
Given a new cup with room for 4 teams
When I ask to see the chart
Then the chart should have 4 rows
And the 1st row should have 8 columns
And the 2nd row should have 4 columns
And the 3rd row should have 2 column
And the 4th row should have 1 column

There are a few things to note here. 1st: The pastie’d version is the
result of not having mid-string variable values. 2nd: Things like
“Then It” (w/ the 2nd word capitalized) were the result of us not
seeing things in a unified way.

Lastly, code that intermingles different levels of abstraction is
harder to read than code which maintains a consistent level of
abstraction. You wouldn’t want to see a method like this:

def save
validate_attributes
prepare_connection
exec(“insert into things (name, …) values (‘#{name}’, …)”)
end

For the same reason, it is difficult to read the second scenario in
the pastie. We get all excited that we get to reuse steps from the
first scenario, but the sad truth is that the result is not easy to
read, and is therefore not all that friendly to anyone, developers and
customers alike.

Separating the abstract representation of the steps from their
detailed implementation is a worthwhile goal because it makes things
more readable, which implicitly makes things more developer-friendly.

Of course, that doesn’t necessarily lead us straight to plain text,
but at the very least we’d want block-less steps in scenarios.

Cheers,
David

[pergesu]

On 10/15/07, Wincent C. [email protected] wrote:

I really think it’s important that this thing, whatever it ends up
looking like, be nice for programmers to use, not just programmer’s
customers.

+1

[pergesu]

Hi all. I’m with David in that plain-text specs are my holy grail. I
have actually been experimenting with this idea since I first saw the
story runner. My interpreter (spike!) would execute “specs” against
“proofs”, but I tried to put a bit more into the grammar.
Specifically, the interpreter would manage the flow of the spec (I
used the terms create/expect/act/verify) by setting up and reusing the
“class variables” between steps. For example:

“given a dog” interprets as

1. instantiate a Dog and assign it to class variables @he, @she, @it

“given a dog with a bone: Fido” interprets as

1. instantiate a Dog and assign it to class variable @fido
2. find a matcher for “:dog with a bone” and call it with @fido

“when Fido barks at Felix” interprets as

1. find a matcher for “:dog barks at :victim” and call it with @fido,
   @felix

I found that two modes of “define_then” statements were needed: one
called before its whens (interaction testing) and one called after its
whens (state testing).

Is that going too far? Yes my parser and interpreter were more than 60
lines

What I can say is that coding “libraries” of “step matchers” takes
little away from the maintainability (or programmer happiness!) For
example, the runner can warn things like:

not yet implemented: act “:dog with a bone” (to support “given a dog
with a bone: Fido”)

Cheers,

Josh

[pergesu]

On 10/16/07, Josh C. [email protected] wrote:

Is that going too far? Yes my parser and interpreter were more than 60 lines

This does seem more complex than what we’re thinking of doing. Are you
using this on projects? Would you mind posting a couple of specs so we
can see what they actually look like?

What I can say is that coding “libraries” of “step matchers” takes
little away from the maintainability (or programmer happiness!) For
example, the runner can warn things like:

not yet implemented: act “:dog with a bone” (to support “given a dog
with a bone: Fido”)

We’ll definitely do something like that, though the messaging won’t be
as granular (saying which part of a step is missing).

Cool ideas.

Cheers,
David