Thought on a quality metric for Cucumber test suites

Testing RSpec
1

Writing another post to Zach I had a thought about a quality metric
for Cucumber suites: Ratio of #steps executed / #step matchers

For example, if you can run 100 steps with only 10 step matchers, you
have a nice low ratio of 100 / 10, meaning you are getting loads of
testing done with only a few step matchers to maintain.

WDYT?

Is there a way to pull out the stat of the number of step matchers
defined?

Matt W.

http://blog.mattwynne.net

2

Matt W. wrote:

Writing another post to Zach I had a thought about a quality metric
for Cucumber suites: Ratio of #steps executed / #step matchers

For example, if you can run 100 steps with only 10 step matchers, you
have a nice low ratio of 100 / 10, meaning you are getting loads of
testing done with only a few step matchers to maintain.

WDYT?

I don’t think a metric like that would be enough. That doesn’t take
into account what type of steps these are. If you are aiming for
extremely high reuse you are probably going to end up with extremely
imperative style scenarios… Which could be a bad thing. So I would
be wary of using any such calculation to determine a “quality”. With
something like Cucumber, the quality is really in the eye of the
customer. :slight_smile: (I may be out of context however since I haven’t read the
other thread… I will go do that now…)

If you are trying to get a metric then I would say you would need to
also use the number of scenarios… What if you have 100 steps, 10 step
matchers, and 1 scenario? Would you really want such a scenario in
your app? :stuck_out_tongue:

Is there a way to pull out the stat of the number of step matchers
defined?
Use the usage formatter…

-Ben