I have two related questions: What is the best way to express global
requirements, and how does one do it in Cucumber. The first question is
the
one I’m most interested in right now.
By a global requirement I’m talking about requirements like ‘all emails
must
be formatted like this…’ Some people call them constraints, but I’m
focusing on UI or business rules, not technical things. I can think of
two
approaches: put the global constraints in one place and reference these
features in other features. The other point of view is that no, we want
to
see these rules in every feature that uses them, so we (customers, BAs,
developers, testers) can see everything in one place. For example I can
reference the different valid and invalid email scenarios in both the
‘add
new patient’ and the ‘add new doctor’ features. Or I can copy and paste
the
email scenarios directly into the ‘add new patient’ and the ‘add new
doctor’
features.
As a DRY infected developer I prefer the first approach. However I have
not
found a simple way to have one cucumber feature reference or ‘include’
another, and have both features get executed. Perhaps partially for
that
reason, other people on my project want to use the second approach as it
seems more straight forward.
I realize I can reuse steps. But I want to reference the actual feature
in
another. To pick a more important example, in my past life I worked on
a
big leasing application with lots of formulas: to compute the cost of a
lease, the cost of fuel cards, the cost of different levels of
insurance,
etc. I’m going to simply a lot, but say the formula for computing the
sales
tax is ‘subtotal + subtotal * 0.5’. Calculating the sales tax is one
feature. Two other features are ‘compute lease cost’ and ‘compute fuel
card
cost’. They both need to add sales tax. This was before cumber and
stories, but some of us preferred the specifications to look like this:
LEASE COST FORMULA:
a + b + sales tax.
Others, especially the testers, preferred:
a + b + (a + b * 0.5)
In real life the sales (and use) tax calculation was quite complex and
changed. When it changed this broke many manual test plan and
invalidated
many use cases. If we were using cucumber with the second approach it
would
break many features.
Sorry for rambling on so long but I’ve found this is a hard question to
articulate.
I have two related questions: What is the best way to express global
requirements, and how does one do it in Cucumber. The first question is the
one I’m most interested in right now.
By a global requirement I’m talking about requirements like ‘all emails must
be formatted like this…’ Some people call them constraints, but I’m
focusing on UI or business rules, not technical things.
feature that uses them, so we (customers, BAs, developers, testers)
the second approach as it seems more straight forward.
stories, but some of us preferred the specifications to look like
and changed. When it changed this broke many manual test plan and
invalidated many use cases. If we were using cucumber with the
second approach it would break many features.
Sorry for rambling on so long but I’ve found this is a hard question
to articulate.
Steve
I think you might need to try and let go of the idea that your
Cucumber acceptance tests should be a complete specification and
instead focus on making sure that there is just enough specification
so that you can be sure to be alerted by the tests if something is
wrong. Any more is likely to do more harm than good.
I noticed you called yourself a ‘DRY infected developer’. Duplication
is undoubtedly the root of all evil in production code, but there are
times when it pays to allow a little slack in. Check out this article
from Dan N.:
By a global requirement I’m talking about requirements like ‘all emails must
be formatted like this…’ Some people call them constraints, but I’m
focusing on UI or business rules, not technical things.
You say “must.” That’s a programmer’s synonym for “should.” And to
me that feels like a level of detail better handled with RSpec
examples, where “should” is central to the domain language, than
Cucumber features (which are more about stimulus and response). I
know you said “business rules” and that tends to imply Cucumber in a
lot of minds, but a SpecDoc formatted output that lists all the “it
should” text works pretty well for business process description too.
Pat,
Thanks – I misunderstood your original response you sent below. For
some
reason I read it as ‘use rspec specs to validate the date logic’ and
missed
the bit in the second paragraph where you suggested creating an explicit
date feature. Doh! Sorry for not reading carefully.
This sounds like a good approach. We would have to trust that the
developer did indeed call the date validator object. On a large project
developers might not read / remember all the feature docs so this could
happen. Or the customer might have forgotten about the date validation
feature, and this could be an exception where the standard date
validation
should not apply for some reason. But this is probably fixable via a ‘#
see
date_validation.feature’ comment.
From a testing perspective it would be nice if cucumber could actually run
the date validation feature everywhere it applies. But there are
technical
issues as then the date feature would have to be parameterized by uri,
or
whatever varies. More important perhaps this is putting too much
emphasis
on using cucumber as a testing tool when it is primarily a communication
tool. But still if it could be done in a way that was easy to read it
might
be nice.
Anyway you’ve showed me a very workable approach. Thanks!
Steve
P.S. Date validation really isn’t that important in my application;
that’s
just an example. A real example would be the sales tax calculation in
the
leasing app I worked on. That was very important, it was a global
requirement with some important exceptions. But I think your approach
would
have worked there as well.
Thanks Matt I think you’re right. Ironically on a past project
customers
wanted a little drying up in the use cases because that helped their
flow,
and QA wanted more inline expansion because that helped their testing
flow.
The customers wanted to read about the new features, and didn’t care to
see
a regurgitation of all the previously explained features that applied.
A
reference or reminder note was just fine.
But anyway, yes focus on clarity and flow for the reader. Sometimes
that
will mean getting a little DRYer, in other cases it would me getting a
little moister.
Perhaps. But I’m not sure then what the difference is between
requirements
that should (no pun intended) be expressed via RSpec examples versus
features. Lots of features use the word “should” in their then clauses.
Take this example from the ‘Feature Introduction’ of the cucumber wiki:
Scenario: Buy last coffee
Given there are 1 coffees left in the machine
And I have deposited 1$
When I press the coffee button
Then I should be served a coffee
Should this be an rspec example instead? It certainly could be, but I
don’t
think an rspec example would communicate as well with the customer as
the
feature would. And that’s the primary goal here (as Pat and Matt have
reminded me!). Which is not to say that the developers wouldn’t get
benefit
out of a spec example as well – you could do both – just that at least
in
this case a feature is clearer to the customer than a spec example.
Yeah I thought of something like that. Actually we do something like
that
in one step now that I think about it. But I really wanted to execute
the
same exact date feature (for example) doc that the user verified to make
sure nothing got lost in translation. Which I could do if I
programmatically ran the date feature file inside the Given /(.*) date
is
vaild/ step. But all those results would clutter up the report output.
I
like your approach best: simple and doesn’t require a funky technical
solution.
Steve
From a testing perspective it would be nice if cucumber could actually run
the date validation feature everywhere it applies.
Sure, and you can have a step like
Given birth date is valid
Given /(.*) date is valid/ do |field|
TestData.valid_dates.each do |date| @it.send “#{field}_date=”, date @it.should be_valid
end
TestData.invalid_dates.each do |date| @it.send “#{field}_date=”, date @it.should_not be_valid
end
end
That gets you technical validation everywhere you need it. Then you
have another individual feature file that describes date formats.
P.S. Date validation really isn’t that important in my application; that’s
just an example. A real example would be the sales tax calculation in the
leasing app I worked on. That was very important, it was a global
requirement with some important exceptions. But I think your approach would
have worked there as well.
Same ideas apply.
Pat
p.s. I didn’t realize you were also the author of the other thread I
linked to
Lots of features use the word “should” in their then clauses.
Take this example from the ‘Feature Introduction’ of the cucumber wiki:
Scenario: Buy last coffee
Given there are 1 coffees left in the machine
And I have deposited 1$
When I press the coffee button
Then I should be served a coffee
Maybe I’m overly audacious, but I’d call that imprecise wording. A
better clause would be “Then I am served a coffee.”
(Actually, that feature has more than one thing wrong with it. The
“depositing $1” part really ought to be a When, not a Given, since
it’s an action integral to the scenario rather than a starting state.
Unfortunately, this often happens when people make up theoretical
examples instead of presenting real tests for real code. I’ll be this
feature was never actually part of designing a coffee machine…)
Should this be an rspec example instead? It certainly could be, but I don’t
think an rspec example would communicate as well with the customer as the
feature would. And that’s the primary goal here (as Pat and Matt have
reminded me!).
I agree. This is a flawed feature, but it really ought to be a
feature: it describes a stimulus (an action) and a response. But this
example doesn’t have much in common with your original e-mail
validation example. In that case, what’s the action and what’s the
response? And is a feature clearer to the customer to explain e-mail
validation than a spec?
Maybe I’m overly audacious, but I’d call that imprecise wording. A
better clause would be “Then I am served a coffee.”
meh. I use “should” all over the place in my features.
validation than a spec?
hrm…
“[email protected]”.should be_valid_email
“[email protected]”.should be_valid_email
“pat.maddox@gmail”.should_not be_valid_email
Seems pretty clear to me.
Email validation and the lease cost formula are examples of acceptance
tests that are probably best handled with FIT. I know that cucumber
has FIT-inspired tables, but those are still coupled to scenarios,
aren’t they?
Actually, I’d still prefer to write these tests with prose. But it’d
be super simple:
[email protected] is a valid email address
pat.maddox@gmail is an invalid email address
You can achieve this with cucumber already…
Then [email protected] is a valid email address
Then pat.maddox@gmail is an invalid email address
For a more realistic example lets take the leasing application I worked
on.
There were lots of rules and formulas, and exceptions to those rules
and
formulas, and the different pieces were (sometimes) part of larger
calculations. We specified the requirements in the normal declarative
fashion
Another idea I had is to potentially introduce a ValidatedDate class,
and then your “should be a valid date” step checks that the field is
an instance of ValidatedDate. That has the affect of ensuring that
people use your validation code in those spots where you want them to.
How does that sound?
I guess that would work. But a method would probably work too, if I’m
understanding correctly (often a bad assumption!). I.e. /"(.*)" should
be
a valid date/ would call valid_date?(date), or something. Anyway, yes
that
sounds promising.
Steve
Then should be an invalid email address
| email |
| sam@ |
| @bar.com |
I don’t think you need all three steps. Instead of the tables you could
And
them all together if you prefer.
OK, “Then” is a little awkward. Perhaps you could monkey patch cucumber
with your own step name.
I may have picked bad examples, but fundamentally I don’t think this is
an
rspec vs. cucumber issue. In the leasing app I worked on we had lots of
testing scenarios like ‘setup the lease with values x, y, z…, then
press
the calculate button. The result should be xxx’. Obviously these
translate
directly into cucumber scenarios. And then some tests would include
other
tests, as caculcations were often composites of other calculations. I
could
see where sometimes inlining would make sense, other times where that
would
be too much noise and you want a reference.
Steve
Steve
I don’t think you need all 3 steps. OK. “Then” is awkward
Whoops I misunderstood again. You’d validate that it was an instance
ValidDate to make sure the date validation code was executed. Got it
now.
Maybe you could mock out the date validation part of the code and set
an
expectation that it should be called:
mock_date_validator.should_receive?(:validate).with(the_date).return(true)
You’re completely mocking out the date validation result, but you’re
testing
that the date validation routine was indeed called, and if you’ve tested
that routine elsewhere you should be good.
What would work? You top-posted, any way you can inline post to the
spot you’re responding to? Sorry to be an email nazi, but you’re
making me do all of the work for wanting to hopefully participate in
this thread,
Zach
and then your “should be a valid date” step checks that the field is
in one step now that I think about it. But I really wanted to execute
end
leasing app I worked on. That was very important, it was a global
linked to
Another idea I had is to potentially introduce a ValidatedDate class,
and then your “should be a valid date” step checks that the field is
an instance of ValidatedDate. That has the affect of ensuring that
people use your validation code in those spots where you want them to.
How does that sound?
On a more serious note what are the rules/recommendations for posting on
mailing lists to make things as easy as possible for hardened list users.
The only recommendation I have is for people to not remove the portion
of the email they are responding to, and to keep their response close
in approximation to what they are responding to. This helps people
easily piece together a conversation so they can join the discussion.
No one likes to read ugly, messy code. We like clean and easy to read.
Why not clean and easy to read emails?
