Documentation for Plain-Text Stories

Testing RSpec
21

aslak hellesoy wrote:

I’m inviting you to provide some constructive feedback about how we
could improve it to make the transition easier for people like
yourself.

I’ll give it a try, I’ll port some of my stories to cucumber and see how
it goes.

A natural part of software evolution is that every now and then a
tool, library or program gets replaced by something better. While it’s
certainly possible to maintain two different products, it takes a lot
more time. Time that I don’t have. Am I to understand from your email
that you’re stepping up and offering to help maintain and govern the
future of the story runner?

True, but Story runner was only released in the last version of RSpec so
it isn’t exactly old.
Depending on how hard it is to port from story runner to cucumber, I’ll
have to make a decision to
either port, or maintain the old code, it is a trade off of time. Right
now I have my entire
integration testing suite written using story runner.

On an initial look at cucumber it looks like the steps remain pretty
much the same, although I see
you added Before and After, and I was using before story and after story
listeners for setup, not
sure what scope the Before and After have. I also use a before Scenario
listener to clear the
context before each scenario.

What exactly is your understanding of deprecation and how have you
concluded that you will have rewrite several thousands of LOC?

What I have seen in the Ruby world (and Java) is that deprecation means
the next release prints some
warning that the feature is deprecated if you use it, then the following
release it is removed entirely.

I’ll site the Ruby Gem Sequel as an example of some recent deprecations,
that caused me to go
through many lines of code and update them as the feature I was using
was deprecated, and I
initially got a lot of warnings, then all my scripts stopped working
after an upgrade.

Maybe I won’t have to rewrite “thousands” of lines of code, at least I
hope not, if the steps remain
the same, I guess I’ll need to massage them and rewrite many of the
scenarios to fit the new format.

I have to say my reaction is due to the fact that tests are something
you want to write once and
forget, I don’t mind writing new tests using cucumber moving forward,
but to have to revisit old
tests is not a good use of my time. If you were to reword deprecation to
frozen I’d be a lot happier
:slight_smile: then I can leave my old tests alone and use cucumber for new tests.

–
Jim M., http://blog.wolfman.com

22

What do you think is the best thing to do?

I would prefer to merge it into RSpec whenever it’s ready. One of the
strong points about RSpec is that it’s a complete BDD toolkit: example
framework for developer testing, feature runner for customer test, and
the mock objects framework. People can get started with one part of
the toolkit, and gradually use the other pieces as their wants and
needs develop. I think it was a big win when RBehave merged into
RSpec. If Cucumber represents significant improvements over the
current Story Runner, I’d like to see those improvements make it into
RSpec.

Pat

23

On 8/19/08, Pat M. [email protected] wrote:

I would prefer to merge it into RSpec whenever it’s ready.

For whatever its worth I agree with Pat. Most of the short-comings of
the existing story runner that Aslak describes in the Cucumber README
are things that we at weplay have had to hack together. Having some
standardization for running a single scenario, or all scenarios, would
make developing tools on-top of the framework that much easier.

Josh

–
Josh K.
phone: 509-979-1593
email: [email protected]
web: http://joshknowles.com

24

On Aug 19, 2008, at 6:28 AM, Jim M. wrote:

tool, library or program gets replaced by something better. While
it’s
certainly possible to maintain two different products, it takes a lot
more time. Time that I don’t have. Am I to understand from your email
that you’re stepping up and offering to help maintain and govern the
future of the story runner?

True, but Story runner was only released in the last version of
RSpec so it isn’t exactly old.

It was introduced in version 1.1.0 - about 8 months ago. That’s not
old I agree with you.

Depending on how hard it is to port from story runner to cucumber,
I’ll have to make a decision to either port, or maintain the old
code, it is a trade off of time. Right now I have my entire
integration testing suite written using story runner.

I would recommend that you keep using the story runner until the
effort required to switch is so small that you can do it in an hour or
two.

On an initial look at cucumber it looks like the steps remain pretty
much the same, although I see you added

They do remain pretty much the same, but I’m considering to move them
from the toplevel scope and into classes (I prefer pure Ruby
constructs instead of inventing our own cleverness).

class MySteps < Cucumber::Steps # Same as Story runnner steps_for(:my)
Given /bla bla/ do
end
end

Before and After, and I was using before story and after story
listeners for setup, not sure what scope the Before and After have.
I also use a before Scenario listener to clear the context before
each scenario.

Right now the scope of Before/After is global - i.e. all Before and
After blocks will run before and after each scenario. I have
deliberately postponed adding scoping of this until I better
understand how people want to define this scoping.

Here is one ideas:

class MySteps < Cucumber::Steps # Same as Story runnner steps_for(:my)
Before do
# This block will run before all scenarios that use steps from
MySteps
# Any scenarios not using any of these will not have this block run
end

Given /bla bla/ do
end
end

I’m open to other ideas too.

In any case, a backwards compatible layer should be added so you can
still use Scenario listeners if you wish.

update them as the feature I was using was deprecated, and I
initially got a lot of warnings, then all my scripts stopped working
after an upgrade.

Don’t worry. We’ll keep it around inside RSpec for a while still. Only
when we are confident that migration to Cucumber is painless will we
take it out. At that point it will probably just be moved to a
separate Git repo so those who wish can maintain it as a separate
project.

Maybe I won’t have to rewrite “thousands” of lines of code, at least
I hope not, if the steps remain the same, I guess I’ll need to
massage them and rewrite many of the scenarios to fit the new format.

My goal is that you won’t have to change anything at all, except
perhaps the parts that deal with running stories (you’ll get a Rake
task and can delete your all.rb file). You shouldn’t have to change
the story/step files.

I have to say my reaction is due to the fact that tests are
something you want to write once and forget, I don’t mind writing
new tests using cucumber moving forward, but to have to revisit old
tests is not a good use of my time. If you were to reword
deprecation to frozen I’d be a lot happier :slight_smile: then I can leave my
old tests alone and use cucumber for new tests.

Words are powerful aren’t they :slight_smile:

Frozen is probably a better word. No more significant development on it.

Thanks a lot for your feedback - looking forward to more concrete ones
so I can make Cucumber better.

Aslak

25

Aslak Hellesøy wrote:

# This block will run before all scenarios that use steps from MySteps
# Any scenarios not using any of these will not have this block run

end

Given /bla bla/ do
end
end

I’m open to other ideas too.

I’d like to have a before story and after story as well as a before
scenario and after scenario.

Right now I do global setup for a story, and cleanup when the story is
done. I also do a setup
before each scenario as well.

It would be nice to have the ability to do both.

Thanks

26

On Aug 20, 2008, at 7:04 AM, Jim M. wrote:

This block will run before all scenarios that use steps from

scenario and after scenario.

Right now I do global setup for a story, and cleanup when the story
is done. I also do a setup before each scenario as well.

It would be nice to have the ability to do both.

Can you give me a code example? (In Cucumber it’s Feature, not Story)

Aslak

27

On Aug 20, 2008, at 1:04 AM, Pat M. wrote:

What do you think is the best thing to do?

I would prefer to merge it into RSpec whenever it’s ready. One of the
strong points about RSpec is that it’s a complete BDD toolkit: example
framework for developer testing, feature runner for customer test, and
the mock objects framework. People can get started with one part of
the toolkit, and gradually use the other pieces as their wants and

I agree. These are the key success factors of Ruby on Rails.

-Which consists of 5 or so gems

Aslak

28

On Aug 20, 2008, at 2:20 AM, Aslak Hellesøy wrote:

(In Cucumber it’s Feature, not Story)

no offense, but while you’re being picky about names, I dont see too
much difference between ‘story’ and ‘feature’
but ‘cucumber’ is a really random meaningless name

29

On Wed, Aug 20, 2008 at 4:20 PM, David C. [email protected]
wrote:

I see them as very different.

User Stories are inputs to a development process and Features are the outputs.

That’s a brilliant description. Much better than these ramblings:
http://www.nabble.com/Re%3A--ANN--Cucumber-p18899320.html

I would add to that that the features, which are the outputs of the
development process, are the inputs to the outcomes. AKA Business
value.
These two posts complement that aspect nicely:

http://www.artima.com/weblogs/viewpost.jsp?thread=183405

Regarding Cucumber - yes, the name is completely meaningless, but a
couple of chicks who attended my presentation about it at Agile 2008
told me afterwards it made their mind drift… That’s good enough for
me.

Aslak

30

On Wed, Aug 20, 2008 at 9:04 AM, Jonathan L.
[email protected] wrote:

On Aug 20, 2008, at 2:20 AM, Aslak Hellesøy wrote:

(In Cucumber it’s Feature, not Story)

no offense, but while you’re being picky about names, I dont see too much
difference between ‘story’ and ‘feature’

I see them as very different.

User Stories are inputs to a development process and Features are the
outputs.

When I’ve worked with FitNesse, we had stories on cards and a suite of
FitNesse tests. Sometimes a Story would come up in an iteration that
was an enhancement of an existing feature. In those cases we did not
add new FitNesse tests, but simply enhanced the existing ones instead.
So the FitNesse test suite grew to represent executable documentation
of an existing system, not a tracking system for stories over the
course of iterations.

Had we grouped all the FitNesse tests by the stories as they came in,
in which iteration, etc, etc, they would have been much more difficult
to navigate and maintain.

Automated scenarios live in the same place that FitNesse tests do.
Ideally, we would group them by Story going into an iteration and then
by Feature coming out, but we don’t really have a good way of doing
that.

Maybe the right approach would be to group them by neither Story nor
Feature, but rather by execution context. For example, right now I’ve
got stories that run one happy path scenario and one error path
scenario that run in-browser, accompanied by a more exhaustive set of
error path scenarios that run in-memory. These live in separate Story
files and are only coupled together by their names.

It would be nice if I could have a suite of in-browser scenarios, a
suite of in-memory scenarios that touch the full stack (in rails), and
possibly a suite of scenarios that touch only a given model. Then each
scenario can be tagged to a Story and a Feature, and the runner could
support running everything by Story, Feature or Execution Context,
thus supporting readability, navigability, etc, from a number of
useful perspectives.

WDYT?

31

On Wed, Aug 20, 2008 at 7:49 AM, aslak hellesoy
[email protected]wrote:

Regarding Cucumber - yes, the name is completely meaningless, but a
couple of chicks who attended my presentation about it at Agile 2008
told me afterwards it made their mind drift… That’s good enough for
me.

Aslak, please feel free to use whatever verbiage you choose. I just
wanted
to mention that for some (including myself), such comments don’t reflect
well on the speaker. Maybe I’m just showing my age. :slight_smile:

///ark

32

aslak hellesoy wrote:

  • How to use with Rails+Webrat
  • How to use with Watir

Please let me know if you’d like to help out documenting Cucumber. One
thing that is missing is a migration guide from RSpec Story runner.

Do you prefer to keep such documentation in the README or would a github
wiki be better?

-Ben

33

On Wed, Aug 20, 2008 at 5:32 PM, Mark W. [email protected] wrote:

well on the speaker. Maybe I’m just showing my age. :slight_smile:

I can live with that.

If you really want to know the origin of the name, check out the bottom
of
http://github.com/aslakhellesoy/cucumber/tree/master/README.textile

Let’s get back on topic.

Aslak

34

On Wed, Aug 20, 2008 at 8:32 AM, Jim M. [email protected] wrote:

Can you give me a code example? (In Cucumber it’s Feature, not Story)

An example of how I would use it or an example of how to do it?

Of how you would use it. Assume that you have something like this for
scenario-level before/after:

class MySteps < Cucumber::Steps # Same as Story runnner steps_for(:my)
Before do

This block will run before all scenarios that use steps from

MySteps

Any scenarios not using any of these will not have this block run

end

Given /bla bla/ do
end
end

Aslak

35

I agree with Pat also. But I think importantly we need a little
tool/guide on converting the old stories to cucumber before any
movement.

I’m currently evaluating moving our stories to cucumber. I’ll see if I
can put something together that might help others (if its not already
available?).

Also the word ‘feature’ resolves so many issues I had with stories and
how they mapped into my testing framework. Thanks Aslak!

–
Joseph W.
http://www.joesniff.co.uk

Josh K. wrote:

On 8/19/08, Pat M. [email protected] wrote:

I would prefer to merge it into RSpec whenever it’s ready.

For whatever its worth I agree with Pat. Most of the short-comings of
the existing story runner that Aslak describes in the Cucumber README
are things that we at weplay have had to hack together. Having some
standardization for running a single scenario, or all scenarios, would
make developing tools on-top of the framework that much easier.

Josh

–
Josh K.
phone: 509-979-1593
email: [email protected]
web: http://joshknowles.com

36

aslak hellesoy wrote:

Given /bla bla/ do
end
end

Aslak

Ok here is an example from one of my current stories… Modified to be
Cucumber-like, although I
see you put the Before and after in the steps, I’d p[refer them to be
global, but could live with
them in the steps class. Of course where would you put the scenarios
before and after? They probably
need to be global?

(Note I am not using Rails,this is an integration test against a Web
Application testing the full
stack using net/http and some helpers to wrap that. It can test remote
servers, It also uses Sequel
to test that the database gets setup properly, again a bunch of helpers
lets me inject data directly
into the database and test afterwards).

setup several users each with their own web context

setup global database accessor

BeforeFeature do

actual user names to login as

user_names= %w(user50 user51 user52)

$umap= {}
$wr= {}
user_names.each_with_index do |e, i|
n= “user#{i+1}”
$umap[n]= e
$wr[n]= WebRequest.new
end

$users= {}
$loggedin= {}

$db= MyDBHelper.new(‘local’, false)
end

logout the users

AfterFeature do
$wr.each_value { |w| w.logout }
end

called before each scenario

to clean up any state conatianing globals used to communicate between

steps

and log out any user that was logged in during the previous Scenario

BeforeScenario do
$wr.each_key do |u|
$wr[u].logout if $loggedin[u]
end

$last_error= nil
$roomid= nil
$roomobjectid= nil
$last_doc= nil
$bagid= nil
end

37

Hello,

I’ve been looking through the cucumber documentation and have a couple
of questions.

I’m curious which of the disadvantages you list would be impossible/very
difficult in the classic story runner. I’m just trying to envisage if
Cucumber and the classic story runner where to co-exist what would the
Cucumber plugin be able to do that the classic story runner could never
realistically hope to achieve.

Also looking at one of your disadvantages:

  • ‘Limited colouring of output’
    I’ve been playing around with patches here and there to improve the
    colour of the classic story runner formatters. Do you still see
    limitations in this as it is in edge?

My final question is about the Rspec book. I’ve no idea when this will
be released or what pressures there are on publish deadlines. How much
would this effect a move to Cucumber (avoiding having redundant story
examples in the book when we all use Cucumber)?

Thanks,

Joseph W.
http://www.joesniff.co.uk
aslak hellesoy wrote:

On Wed, Aug 20, 2008 at 6:24 PM, Ben M. [email protected] wrote:

  • How to use without Rails
    wiki be better?

Good idea, I have moved the README.textile to
GitHub: Let’s build from here · GitHub

Everyone here is welcome to edit it. If you have concrete experience
(or wishes) about how to migrate existing stories to Cucumber this
would be the place to do it.

Aslak

38

On Wed, Aug 20, 2008 at 6:24 PM, Ben M. [email protected] wrote:

  • How to use without Rails
    wiki be better?

Good idea, I have moved the README.textile to

Everyone here is welcome to edit it. If you have concrete experience
(or wishes) about how to migrate existing stories to Cucumber this
would be the place to do it.

Aslak

39

On Aug 20, 2008, at 10:20 AM, David C. wrote:

I see them as very different.

User Stories are inputs to a development process and Features are
the outputs.

I decided to churn on this for a few days before responding. Actually
I was going to let it drop, but kept thinking about it.

Given your observation (above) I see an analogy between Stores ->
Features, and Specs -> Tests, that is, I write specs as input, I
develop the code, and tests are the outputs. Same code, different
purpose.

You (or someone) named this framework rspec perhaps because ‘spec’ is
more descriptive and accurate than ‘test’, but more importantly
because there is a large body of work and history with conventional
QA and testing, and it was important to coin a new term (spec) to
distinguish BDD from conventional testing. However, you can use test
tools to do BDD (e.g. as many people has chosen to continue using
Test Unit and shoulda rather than rspec for whatever reasons).

But with regard to stories vs features, there really isnt a legacy
and baggage about the word ‘story’, so changing the name to ‘feature’
is mostly semantics.

Personally, I do not see a feature as separate from a story, because
a feature means nothing without some context. Stories provide that
context. Scenarios provide the specifics. I might write and organize
my stories by individual feature, but theres other ways as well:
workflows, goals/objectives, different starting setups, user roles,
and so on, all involving one or more sets of features.

I’m not trying to be argumentative, and I’m wholeheartedly
appreciative of the work you and Alask and everyone does on this
project. If anything, I’m just trying to sort all this out for myself.

40

On Fri, Aug 22, 2008 at 12:53 PM, Jonathan L.
[email protected] wrote:

Given your observation (above) I see an analogy between Stores → Features,
and Specs → Tests, that is, I write specs as input, I develop the code,
and tests are the outputs. Same code, different purpose.

You (or someone) named this framework rspec perhaps because ‘spec’ is more
descriptive and accurate than ‘test’, but more importantly because there is
a large body of work and history with conventional QA and testing, and it
was important to coin a new term (spec) to distinguish BDD from conventional
testing.

Steven B. named the framework rspec. I believe his motivation was
Uncle Bob saying “specify, don’t verify” when talking about what TDD
and ATDD is about.

Sadly, “spec” has just as much baggage, if not more, as “test” does.
These days we’re calling these things “code examples,” (tongue
pressing into cheek) so maybe we should change the name to
rcodeexample?

However, you can use test tools to do BDD (e.g. as many people has
chosen to continue using Test Unit and shoulda rather than rspec for
whatever reasons).

Agreed. Tools is tools. Process is process. (boat is boat …)

But with regard to stories vs features, there really isnt a legacy and
baggage about the word ‘story’, so changing the name to ‘feature’ is mostly
semantics.

User Stories have been around since the late 90’s. I’d say that
qualifies them for legacy/baggage.

Personally, I do not see a feature as separate from a story, because a
feature means nothing without some context. Stories provide that context.
Scenarios provide the specifics. I might write and organize my stories by
individual feature, but theres other ways as well: workflows,
goals/objectives, different starting setups, user roles, and so on, all
involving one or more sets of features.

I’m not trying to be argumentative, and I’m wholeheartedly appreciative of
the work you and Alask and everyone does on this project. If anything, I’m
just trying to sort all this out for myself.

This is something that’s bugged me since rbehave first appeared (even
before we merged it into rspec). I’ve discussed this w/ Dan N. a
few times and we’ve agreed on some aspects of this but not all. I
won’t speak for Dan here, but my experience tells me that there is not
a clean mapping of stories (the things that drive development) to
features (the things that exist in the system because they have
already been developed).

For example, imagine we’re working on a conference
organization/registration tool and way back in iteration 1 we had a
story about the organizer being able to see a report of conference
registrations (this may sound familiar if you saw my talk at
RailsConf).

Imagine that one of the conference organizers noted that a given
conference was reporting 100% full even though only 499 of the goal of
500 had registered. So a new story gets added suggesting that it
should not be rounded up to 100%.

So now we have two choices. We can add a new story file with a new
narrative and a single scenario, or we can crack open the existing
story file and add a single scenario.

In terms of the feature (which is the report), I see this as just
another scenario.

In terms of driving development and estimating effort, I see this as a
new User Story.

Does this clarify or further confuse?