I am perplexed by a situation that I am creating for myself, probably
out of ignorance. I have two models connected through a join table
which itself contains information related to the join. I am testing the
creation of this join.
I already have step definitions written for one of the outside models
that include methods to create new records. I say methods because I
have broken down the processing to initializing the model instance,
populating the attributes, and then saving it.
Now, where I think I may be about to commit a grave error is in the step
definitions for the features relating to the join model. Which path
should I take:
Call the existing step definitions in the outer model to create a
test instance? or
Replicate the effect of those existing steps inside the step
definition file for the join model and create test instances of the
outer models completely from within that file.
I tend to avoid calling steps in other steps, as right now it appears
that
cucumber does not tell you which step within the step is pending if you
have
it wrong, but just says that the entire one is pending. I prefer to
separate
them. If it’s something more complex than the above, I might make a
method
that would do the creation, and then call it from both steps. So I guess
that is more like #2 of your question.
I’m sure there are much better ways of doing this though…just my
strategy
so far.
Perhaps the creation of the join is not something that should be
tested by a feature. This sounds to me like an implementation detail
that would be better tested by some sort of unit test. So if your
features want to have things in them mentioning joins, databases and
other such things then you’re probably using the wrong tool. Joseph
wrote a really good blog post about this sort of stuff
It seems to me that there are all sorts of implementation details in
this
story that could make your tests quite brittle. And the feature is
definitiley a programmer writing a test, rather than a customer
specifying
what they want. Putting on my customer hat
Scenario: Add location
Given I have a party
When I set the location
And I view the party
Then I should see the location
Starting with this you can then deal with other customer scenario’s and
work
out what they mean e.g.
These may be complex enought to be new features. allowing you to
explore
meaning and business value e.g. what is the value in prioritising a
location, what happens if there are two main locations etc.
Perhaps the creation of the join is not something that should be
tested by a feature. This sounds to me like an implementation detail
that would be better tested by some sort of unit test. So if your
features want to have things in them mentioning joins, databases and
other such things then you’re probably using the wrong tool. Joseph
wrote a really good blog post about this sort of stuff
Thanks for the reference! I have added a link to it from my own attempt
at describing how to begin testing with cucumber.
I did not show the code because most of it is not written. I am
learning as I go with this exercise and no doubt many of these early
attempts will need to be revised as I become more proficient. The
feature looks like this:
Scenario: Add an initial location for a party
Given I have a party named "My Test Party"
When I add a location for "My Test Party"
And I set the "type" to "MAIN"
And I set the "description" to "Sole Location"
And I set the "effective date" to "1984-11-01"
And I set the "superseded date" to "nil"
And I provide valid site information
And I create the location
Then I should find the "MAIN" location for "My Test Party"
And location "description" should be "sole location"
As you can see, other than the attributes and their expected values, the
only implementation detail exposed is that a site is somehow distinct
from a location.
In this case, the step definitions [When /have a party named “(.*)”/]
and [When /provide valid site/] are effectively factory methods that
provide a valid model instance of the appropriate type. I was debating
with myself where best to locate these methods, either in a
step-definition file relating to the model itself, or as distinctly
named methods within the location_steps.rb file.
Presently, I am proceeding cautiously with the first option of placing
these methods with their models, keeping in mind that this may not be
what I need to do at all. The main reason being is that the factory
methods already contained in the model_steps.rb files are working
without problem.
I am still not content with some of the verbs that I am using in my
features. Particularly those surrounding the process of instantiating a
new row to a table. Add, Create, and Commit all seem to possess
unfortunate inferences when used in a feature step.
I need a bit of instruction. I have spent the day reading online
bloggs, tutorials, and howtos relating to BDD, RSpec and testing. I
have also spent several hours going through the cucumber spec/test
suites in an attempt to absorb some sense of how best to proceed. I am
somewhat nonplussed.
Here is my situation. I am the domain expert (together with a few
others) and I am the business analyst and I am the systems analyst and I
am the coding team. So, I am essentially in the position of having to
interview myself (mostly) to write specifications so that I can design a
system that I will have to code, myself.
I have spent the better part of the last two years learning ruby and
rails, taking related courses, investigating, acquiring, installing and
learning to use various support tools like Subversion, which I have
since replaced with GiT, and Trac project administration system, which I
have since replaced with Redmine. I also beat my head against a brick
wall trying to get some idea on how to use rspec stories to facilitate
my initial foray into TDD which then evolved (for me) into BDD.
Along came cucumber and in the space of a few days I have begun to write
and run actual tests derived from features that I have described in a
cucumber fashion. I have actually written working code driven by
feature requirements. Heavens, I have even “refactored” my initial step
definitions to remove instance variables and create cross feature steps.
I now have rcov reports telling me what tests need to be written for the
code I already have. Things seem good.
Where do I go from here? At issue for me is really how much detail goes
into the features. It was pointed out that the present form of many of
my features betray a “programming” bias. I admit to this. My question
is: Where does one specify the nitty gritty details of what columns go
into what relations that possess what associations with what other
relations? Where does one put the descriptions of business rules and
algorithms?
These are all examples of user driven details to at least some degree.
I mean, not every client management system need distinguish between an
operating name and a full legal style and yet keep both for the same
customer. So this is a detail that I feel should be expressed from the
user pov somewhere. Tax calculations and fee discount structures are
also areas that need user driven expression. Do these not belong in
features too?
I can see that features deal with high level stuff. But there is an
awful lot of low level details that end users have to specify as well.
Even if they do not need to know about database normalization or
attribute naming conventions they nonetheless have to express,
somewhere, their need to retain full addresses, postal codes of varying
format depending on country, telephone numbers, perhaps categorized into
meaningful classes, and the like. Where are these requirements
expressed if not in features? Presumably one must test to see that the
requisite information is retained and retrievable even if exactly how is
not explicitly stated in their requirements.
I had, at one point, the idea that, at least in the beginning, I would
do most of this detail expression in the form of feature steps. The
advice I have received has caused to reflect on whether this is a
desirable course to take. However, I can see no obvious alternative
other than to bury the design requirements in rspec specs or testunit
tests, both of which seems to defeat the idea of having users drive the
project.
It seems to me that there are all sorts of implementation details in
this story that could make your tests quite brittle. And the feature is
definitiley a programmer writing a test, rather than a customer
Guilty as charged.
Putting on my customer hat
Scenario: Add location
Given I have a party
When I set the location
And I view the party
Then I should see the location
Starting with this you can then deal with other customer scenario’s and
work
out what they mean e.g.
These may be complex enough to be new features. allowing you to
explore meaning and business value e.g. what is the value in
prioritising a location, what happens if there are two main locations
etc.
I had, at one point, the idea that, at least in the beginning, I would
do most of this detail expression in the form of feature steps. The
advice I have received has caused to reflect on whether this is a
desirable course to take. However, I can see no obvious alternative
other than to bury the design requirements in rspec specs or testunit
tests, both of which seems to defeat the idea of having users drive
the
project.
Am I missing something obvious here?
You’re not, don’t worry! This is definitely an emerging practice, and
you aren’t going to find any ‘correct answers’. See this thread for a
lengthy discussion on the same topic:
FWIW, my view is becoming: if in doubt, put it in the Acceptance
Tests. These ATs are proving themselves to be less brittle than my
unit tests, and therefore much more valuable.
So back to your original question where does all the detail go? Well
acceptance tests start from the general and go to the specific, so
detail comes further down some sort of heirarchy. Thing is you
haven’t got a hierarchy yet so you don’t know where to put things.
…
As I learn best from examples, let us consider a fairly common feature
of a business application:
Feature: Produce Invoices
In Order to: Bill Chargeable Work
An Invoicing Clerk
Should be able to Produce Invoices
To Increase Revenue
Scenario: Work Order completed
Given work order "X" for client "Y"
And work order "X" is completed
And work order "X" has "N" unbilled charges
And "all" charges have been released for billing
When I view work order "X"
Then I should see "N" unbilled charges
And I should be able to select "all" unbilled charges
And I should add "N" charges to a new invoice
And the new invoice should have a unique transaction number "Q"
And invoice "Q" should have client "Y" as the bill to party
…etc.
Scenario: Work Order open
Given work order "X" for client "Y"
And work order "X" is not complete
And work order "X" has "N" unbilled charges
And "M" charges have been released for billing
When I view the work order
Then I should see "M" unbilled charges
And I should be able to select "M" unbilled charges
... etc.
Now, we have two cases, an open and a closed work order, both producing
new invoices. How does one proceed to decompose this to also cover the
case where one adds the charges to an existing, open, invoice; or the
case where “N” billable charges are listed but only “N”-“M” charges are
to be billed? Do you create more scenarios in the existing feature
file or create new feature files for these scenarios? If the former
then how does one avoid having scenarios that themselves are filled with
compound conditional statements; or. is this OK? If the latter, how
then does one relate high level feature files with their descendants?
Similarly, how much detail goes into the creation of an invoice in this
case? Do you break out invoices features into specifying the bill-to
party, forms of items to add to the invoice body, payment terms, tax
calculations, etc.? Are these individual features?
I realize that this is a styling issue but the spectre of Worf/Sapir
haunts such choices.
Pretty hard for me to comment with so much stuff and so little context.
However
Your telling an epic story involving work orders, charges (unbilled
released
and billed), invoices and transactions. So what are all these things and
what is the business value of them, can we right more focused and
simpler
scenarios. Perhaps if we do stuff to create work orders we will get
steps
we can nest so we can create a step like
Given a work order with outstanding charges
So we get better tools to deal with our more complicated scenarios as we
go
along
Each scenario should be about one thing Work Order Completed has all
sorts
of things going on with it so the scenario needs to be split up into
smaller
ones
Your feature says its about “producing invoices” yet it seems to be all
about work orders
You’ve got lots of assumptions about how things are done, I can almost
see
the legacy application in front of you. Why should you have to even
think
about a work order to produce invoices. You have to do some design and
thinking to produce good features. Challenge assumptions and try and see
things from different view points. For example why can’t I just do
Given outstanding payments
When I visit the make invoices page
I should see a list of invoices I can create
And then do some other little scenarios to create an invoice etc…
I think this is where the art or craft comes in. Features are a great
tool,
but being such a great tool I think its very easy to become tempted to
overuse them. People in software keeps on looking for the silver bullet.
They find something new, overuse it, become frustrated by it, move on to
something else and the cycle begins again. From this comes alot of
innovation, but also alot of frustration and waste.
The second point I’d like to make is that testing is really hard. IMO
its
much harder than writing functionality so you can expect to struggle.
And
its not just about actually getting things tested, but its also about
deciding what to implement
deciding what to test
deciding how to test
organising your tests so they communicate your intention clearly
cover your functionality
… (lots more)
All software projects are entropic. That is if you don’t put a
continuous
input of energy into them they will become chaotic. You can use features
to
counter this chaos to add order to your project or you can just use them
to
test things. If you use them just to test things your missing out though
on
their fundamental purpose which is to express clearly what the intention
of
a project is with a secondary benefit of providing a mechanism to
execute
those intentions and see if they are met.
So back to your original question where does all the detail go? Well
acceptance tests start from the general and go to the specific, so
detail
comes further down some sort of heirarchy. Thing is you haven’t got a
hierarchy yet so you don’t know where to put things. I’m in a similar
situation with the code I’m working on and its real tempting to start
putting in all that detail in the features, but I’d suggest really
trying
hard to avoid that. Basically if you need detail refactor your feature
so
that you don’t and create new scenarios to deal with the detail only
when
you have to. In the end you can have most of the detail in your step
definitions and use the nesting of steps to build more complex stories.
I’ve put my example app on github. It has a refactoring of the
restful-authentication stories following these principles, perhaps it
will
give you some ideas
Pretty hard for me to comment with so much stuff and so little context.
However
My messages are quite long enough as it is I am afraid.
about a work order to produce invoices. You have to do some design and
thinking to produce good features. Challenge assumptions and try and see
things from different view points. For example why can’t I just do
Given outstanding payments
When I visit the make invoices page
I should see a list of invoices I can create
And then do some other little scenarios to create an invoice etc…
HTH
It always helps to have another point of view. I am grappling with this
issue mentally before committing myself to a line of approach. This
project is, as you surmise, a conversion of an existing body of work
into a new form. It is also anticipated that the existing application
will be extended in many areas. The scope of the project encompasses
everything from General Ledger down to edi transmission of tax documents
to the federal government.
I am trying to determine whether the way to control and document this
project while focusing programming activity is best served by using BDD
features or whether those objectives should be satisfied otherwise and
BDD features only used for specific implementation issues. Part of my
difficulty is my limited vocabulary for expressing this and part is my
want of experience with this form of design and development. My
previous experience was with much less sophisticated tools than I am
currently using and I am reluctant to turn a scalpel into a screwdriver
out of ignorance on my part. On the other hand, if a jack hammer is
needed then a scalpel is not of much use.
So reworking my earlier effort:
Feature: Bill chargeable work
In Order To generate income
A Billing Clerk
Should be able to create invoices
To Increase Revenue
Scenario: Manual Invoice Release
Given billable charges
When I visit the create an invoice page
Then I should see a list of all billable charges
Scenario: Manual Invoice Release by Client
Given billable charges
When I visit the create an invoice page
And I search for client “Y”
Then I should see a list of all billable charges only for client “Y”
Is this getting closer to what my features should look like?
Bear with me. I am trying to see how one thing leads to another. Say I
have written:
Scenario: Manual Invoice Release by Client
Given 4 billable charges for client “Y”
When I visit the create an invoice page
And I search for charges belonging to client “Y”
Then I should see a list of 4 billable charges only for client “Y”
This explicitly expresses the idea of a client. Now, I already have an
idea of how I want to treat a client but I want to have the emerging
features drive that expression. So in my step definitions file should I
write something along the lines of:
When /visit the create an invoice page/ do
visits “/invoices/new”
end
When /search for (\d+) charges belonging to client “(.*)”/ do |b,c|
TODO: create a client c
TODO: create b charges for client c
fill_in(“Client Name”, :with => c)
click_button(“Find Charges”)
Well assuming its correct to start simple & general start the feature
without the concept of the client
Scenario: Manual Invoice Release
Given 4 billable charges
When I visit the pending invoices page
Then I should see … (not sure what you should see)
Now think about those charges, who do they come into existence. What are
they. Also think about the name of the page your are visiting - is it
pending invoices?
TIp: When writing the step don’t use - visits “/invoices/new” - use
visits
“invoice_new_path”. Use named routes and let the routing names REST and
resources define where you go, and think about what you want to
implement
If you now deal with your charges, you will probably get to the stage
very
quickly where you can’t avoid clients. So then you do the very simplest
general thing for clients.
Generally all I’m doing here is questioning anything that is not central
to
the scenario and anything that is specific. Each of these things
represents
an assumption. FInding these assumptions and eliminating them will give
you
something thats so simple and general that it becomes obvious (with
practice) that you can start. Then take baby steps adding new scenario’s
or
new whole features to bring back extra things as it becomes imperative
for
you to do so. Because you have already done the simple things you should
have the tools you need (existing steps and object creators) for your
next
feature. Each step you take is an iteration giving feedback and
transforming
information you know from the unknown/implicit to the concrete. So
before
you rush onto the next step think about what you’ve learnt, what you now
know. This will help you take the next step. Finally every time you find
that the baby step you are taking is actually more like a giant leap (or
actually anything other than the smallest of steps) STOP! Then start
again
questioning what is not central and what is not specific.
TIp: When writing the step don’t use - visits “/invoices/new” - use
visits “invoice_new_path”. Use named routes and let the routing names
REST and resources define where you go, and think about what you want
to implement
This advice is really helpful. I was reading the section on routing in
“The Rails Way” last night, and it came to me that perhaps I had
overlooked the essence of what I was really trying to accomplish. That
I was so fixated on the code yet to be produced at the detail level that
I had missed the boat with respect to how the users would treat the
application as a whole. I concluded that I really needed to see the new
application, and the features that I had to write, in terms of
resources. Then, given the resource name, the requisite information
would be exposed. This in turn would expose the requirements needed to
assemble that information from data, and from those assembly
requirements would come the code requirements.
However, I was, until I read your reply, still thinking in terms of
“/application/clients/:myclient/workorders/:id/charges/:billable”
instead of
client_workorder_charges_path(@client,@client.workorders,“billable”) or
however that is supposed to be expressed.
If you now deal with your charges, you will probably get to the stage
very quickly where you can’t avoid clients. So then you do the very
simplest general thing for clients.
It is impossible to undo the past. We have already invested a
considerable amount of energy and time into determining how we wish to
express the concept of a client as a transient role of some persistent
entity rather than as an artifact in itself. But, that said, I am
beginning to see that this can be set aside in the early stages.
Finally every time you find that the baby step you are taking is
actually more like a giant leap (or actually anything other than
the smallest of steps) STOP! Then start again questioning what
is not central and what is not specific.
HTH
Very helpful. I expect that as I gather my wits regarding BDD that this
advice will prove even more valuable.
I tried a little bit of this out on one of the end users and I was
surprised at the responses that I obtained. In consequence, this is
what I contemplate beginning with.
Feature: A web based business application
In Order To: Conduct Business
Any Authorized User
Should Access the Business Application via http
To reduce costs
Scenario: The Application has a home page
Given an application accessible by http
And I am not signed in
When I browse to the url “/” on the application host
Then I should see a welcome message
And I should see a sign on request message
And I should see a sign on link
And I should see a contact us link
And I should see …
Scenario: The Application has a sign on page
Given I am on the home page
And I am not signed in
When I select the sign on link
Then I should see the sign on page
And the protocol should be https
And I should see a text entry for user name
And I should see a text entry for password
And I should see a sign on action selector
Scenario: User Signs On Successfully
Given I am on the sign on page
And I am not signed in
And I have a user named “myuser”
And the user named “myuser” has a password of “myuserpassword”
When I provide a user name “myuser”
And I provide a password “myuserpassword”
And I select the sign on action
Then I should see a sign on success message
And I should see the users home page
Scenario: User Sign On Fails - wrong password
…
Scenario: User Sign On Fails - no such user
…
Scenario: User Sign On Fails - more than 3 attempts in 20 seconds
…
Then we have identified a new feature, the user home page. Also, one
can foresee that additional scenarios are required, like “User Signs in
From a Secured Resource” where following a successful sign in the user
is redirected back to the resource that they originally requested. On
the users home page we discover the resources accessible by that user
are listed. From those resources the next layer of features become
exposed, and so on.
Is this getting closer to the mark or am I moving around the target but
not quite getting it?
Glad to be of some use. I think you’re getting stuff, and your questions
are
helping me get stuff and clarify some thoughts as well so thankyou for
that.
I was so fixated on the code yet to be produced at the detail level that
instead of
express the concept of a client as a transient role of some persistent
Should Access the Business Application via http
And I should see …
This scenario is mixing what should be in a view spec with what should
in a scenario. For example, I think your scenario wants to be
“Anonymous visitors are welcomed to the site”
Given I am an anonymous visitor
When I browse to “/”
Then I should see the welcome message: Howdy hey guy!
All of the other “I should see” steps you have don’t really have
anything to do with the goal of welcoming the visitor to the site.
IMO, these are candidates for view specs since they are ensuring the
lower level details of certain pages are intact.
Scenario: The Application has a sign on page
Given I am on the home page
And I am not signed in
When I select the sign on link
Then I should see the sign on page
And the protocol should be https
And I should see a text entry for user name
And I should see a text entry for password
And I should see a sign on action selector
This scenario has the same problem. If you just have a scenario for
signing on (in) to the site wouldn’t that force you to build a sign on
page for that to work?
And I should see the users home page
How about…
Given there is a user with the login 'myuser' and password
‘mypassword’
When I sign in to site with the login ‘myuser’ and the password
‘mypassword’
Then I should see the success message: You’ve signed in!
It makes it so much clearer IMO. This makes scenario communicate what
is valuable, and leaving out the additional clutter. Leave the lower
things like “filling in text fields” to the step definition
themselves. In your steps, communicate the what you’re doing, and
leave the how you’re doing it to the step definitions.
For example, you’re signing/logging in. That’s the what. The how is
filling in two text fields and clicking a button.
is redirected back to the resource that they originally requested.
I don’t think the “user home page” is a feature, but I bet there is
something on the home page that makes it valuable. What is that? That
is most likely your feature. When writing that feature and its
scenarios don’t focus on the user home page. Focus on what’s really
valuable. If there are things that need to be on the user home page
which are tedious details that don’t really pertain to the feature,
then use a view spec.
For example, in our application we always show a navigation partial
for every page. Based on what part of the site you’re in it will show
a different partial, ie; “fiscal/navigation”, “hierarchy/navigation”,
“operations/navigation”, etc. Using features/scenarios to ensure every
page rendered the right navigation is the wrong place to do it. So we
wrote some shared examples…
shared_examples_for “a page that has fiscal navigation” do
# …
end
Then we’d write view specs that used these shared examples to ensure
the tedious details of the page were taking care of.
Features/scenarios will cover most of the common things like clicking
major navigational links, filling out and submitting forms, etc.
However, webapps usually have a bunch of other components which may
not be linked to “doing” something, or there may be more than one
path, or things may need to show up on more than one page.
When you need to add something like this to the UI that doesn’t fit in
a scenario then write a view spec for it. This allows you to keep the
high level behaviour in the features/scenarios and the lower level
details about the pages in view specs.
However, don’t feel like you need to write a view spec for everything
on the page. If you have form with text fields and a button, and you
have a scenario that uses it, then don’t feel like you have to write a
view spec for those. But if you need to go in and add a “cancel” link
on the form and it’s not doing anything special that requires a
scenario, write a view spec and ensure the cancel link exists (but
feel free to ignore the the things the scenario is already covering).
How does this all sound to you?
On
the users home page we discover the resources accessible by that user
are listed. From those resources the next layer of features become
exposed, and so on.
Is this getting closer to the mark or am I moving around the target but
not quite getting it?
You’re getting there. If anyone asks you are triangulating the target
so you can determine exactly where it lies. To a certain extent, we
all are. =)
You’re getting there. If anyone asks you are triangulating the target
so you can determine exactly where it lies. To a certain extent, we
all are. =)
I think that I am reaching the stage where I am oscillating between over
and under specifying. I am trying this stuff out as I go and no doubt
am leaving artifacts that will be the source of amusement when I finally
do “get it”. It really is not that hard a concept to grasp, in general.
As always, it is in the details of actual employment that the troubles
lie.
Every bit of advice that I have received on this thread, and the two
related ones on this list, is greatly appreciated and valued.
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.
