Cucumber Scenario Outlines Output Insufficient


#1

I’m having problems knowing what a scenario outline I’ve created is
doing.
In particular I have no idea about what steps are being matched when the
outline is run so when I break something I can’t find out what is wrong.
Is
there any progress on getting better output for scenario outlines.

TIA

Andrew


#2

On Sat, Jan 17, 2009 at 5:08 AM, Andrew P. removed_email_address@domain.invalid
wrote:

I’m having problems knowing what a scenario outline I’ve created is doing.
In particular I have no idea about what steps are being matched when the
outline is run so when I break something I can’t find out what is wrong. Is
there any progress on getting better output for scenario outlines.

It would be immensly helpful if you could give some examples of what
kind of
output you think is better.

Aslak


#3

aslak hellesoy wrote:

I identified a problem with hidden output for pending steps in scenario
outlines in this ticket:

http://rspec.lighthouseapp.com/projects/16211/tickets/57-scenario-templates-to-allow-for-terse-scenario-tables

I’ve create a new ticket for this.

http://rspec.lighthouseapp.com/projects/16211-cucumber/tickets/163-hidden-pending-status-with-output-when-using-scenario-outlines

If this does not fit your problem please as Aslak said give us an
example of the problem.

Thanks

Joseph W.
http://blog.josephwilk.net


#4

My problem doesn’t seem to be covered by these tickets

2009/1/17 Joseph W. removed_email_address@domain.invalid


#5

It took a little time :slight_smile:

2009/1/18 Aslak Hellesøy removed_email_address@domain.invalid


#6

Den 18. jan… 2009 kl. 16.11 skrev Andrew P. removed_email_address@domain.invalid:

My problem doesn’t seem to be covered by these tickets

You still haven’t given us a concrete example of what you think is
useful output. Are you going to?

Aslak


#7

I have output issues both with scenario outlines and using step tables.
I’ll
deal with scenario outline in this mail. This will have to be an html
mail
as I need color to say what I want to say, I’ll also use a real world
example so this will be a bit long

Currently the output of my scenario outline is


Scenario Outline: viewing resources #
features/admin/poop.feature:6
Given an admin user admin exists #
features/admin/poop.feature:7
And there are 4 #
features/admin/poop.feature:8
When I login as admin with adminpass #
features/admin/poop.feature:9
And I visit admin_home page #
features/admin/poop.feature:10
Then I should be at the admin_home page #
features/admin/poop.feature:11
And I should see a resource #
features/admin/poop.feature:12
When I ppope ksdjhfkh jadfh kjh kahjkh ksdfh khsdfa #
features/admin/poop.feature:13
When I follow resource #
features/admin/poop.feature:14
Then PPouay Widgit should see a list of 4 #
features/admin/poop.feature:15
And I should see a new link #
features/admin/poop.feature:16

|resource|
|category|
|product |
|user    |

4 scenarios
15 steps passed
3 steps skipped
12 steps pending (12 with no step definition)

So I’d like more information about which steps are matched which are
skipped
etc.


Scenario Outline: viewing resources #
features/admin/poop.feature:6
Given an admin user admin exists #
features/step_definitions/accounts/creation.steps.rb:66
And there are 4 #
features/step_definitions/general.rb:1
When I login as admin with adminpass #
features/step_definitions/accounts/login.steps.rb:35
And I visit admin_home page #
features/step_definitions/general.rb:19
Then I should be at the admin_home page #
features/step_definitions/general.rb:30
And I should see a resource #
features/admin/poop.feature:12
When I ppope ksdjhfkh jadfh kjh kahjkh ksdfh khsdfa #
features/admin/poop.feature:13
When I follow resource #
features/step_definitions/admin_steps.rb:23
Then PPouay Widgit should see a list of 4 #
features/admin/poop.feature:15
And I should see a new link #
features/admin/poop.feature:16

|resource|
|category|
|product |
|user    |

3 scenarios
5 steps passed 3 times
1 steps skipped 3 times
4 steps pending (4 with no step definition)

Now there are a couple of issues I can think of that might challenge
this
output. This is basically situations where the output varies dependent
on
the resource.

  1. Different step matches a line dependent on resource

And there are 4
And there are 4 category #
features/step_definitions/category.rb:1
And there are 4 product #
features/step_definitions/product.rb:1
And there are 4 user #
features/step_definitions/user.rb:1

  1. Different result of step depending on resource i.e. sometimes is
    matches
    sometime it fails. Here the line color should be of the worst result

And there are 4
And there are 4 category #
features/admin/poop.feature:8
And there are 4 product #
features/step_definitions/product.rb:1
And there are 4 user #
features/step_definitions/user.rb:1


An alternative approach might be to expand the resource display when
things
go wrong, i.e. put problems underneath the resource in which they’re
detected

|resource|
|category|
And there are 4 category #
features/admin/poop.feature:8
|product |
|user |

So here I can tell that this step hasn’t passed

I hope this helps

Andrew

2009/1/17 aslak hellesoy removed_email_address@domain.invalid


#8

rspec-users mailing list
removed_email_address@domain.invalid
http://rubyforge.org/mailman/listinfo/rspec-users


#9

Andrew P. wrote:

You misunderstood I bit of my post, I’ve nested a comment. Its towards
the
end of the post.

2009/1/18 Joseph W. removed_email_address@domain.invalid

Scenario Outline: viewing resources #
features/admin/poop.feature:11


features/step_definitions/general.rb:1
When I follow resource #
|product |
http://rspec.lighthouseapp.com/projects/16211-cucumber/tickets/163-hidden-pending-status-with-output-when-using-scenario-outlines
And there are 4

keen on this solution as I would like to preserve the way the scenarios are
defined (in the feature file) with the way that they are outputted.

And there are 4 category                                      #

features/admin/poop.feature:8
And there are 4 product #
features/step_definitions/product.rb:1

No you’ve misunderstood me. I meant the outline step is colored with the
worst result. Then the details of each run of the step are nested inside
the
outline step showing the output for each resource (entry in the table)

Would you expect to still see the scenario table with status colours?

Do you think it might become difficult to workout which steps are for
which run?
i.e the 5th indented Step broke so lets read through and find each 5th
step to see what the run did.

It does sound like an interesting idea. Would you mind adding the full
output you would expect with this indentation in the ticket please? (If
you want colour as a html attachment).

Thanks,

Joseph W.
http://blog.josephwilk.net


#10

You misunderstood I bit of my post, I’ve nested a comment. Its towards
the
end of the post.

2009/1/18 Joseph W. removed_email_address@domain.invalid

Scenario Outline: viewing resources #
features/admin/poop.feature:11


features/step_definitions/general.rb:1
When I follow resource #
|product |
http://rspec.lighthouseapp.com/projects/16211-cucumber/tickets/163-hidden-pending-status-with-output-when-using-scenario-outlines
And there are 4

keen on this solution as I would like to preserve the way the scenarios are
defined (in the feature file) with the way that they are outputted.

And there are 4 category                                      #

features/admin/poop.feature:8
And there are 4 product #
features/step_definitions/product.rb:1

No you’ve misunderstood me. I meant the outline step is colored with the
worst result. Then the details of each run of the step are nested inside
the
outline step showing the output for each resource (entry in the table)


#11

Comments below

2009/1/19 Joseph W. removed_email_address@domain.invalid


defined (in the feature file) with the way that they are outputted.
worst result. Then the details of each run of the step are nested inside
the
outline step showing the output for each resource (entry in the table)

Would you expect to still see the scenario table with status colours?

Yes it feels that the table would still be useful in this situation to
guide
you back to the nested output to look for things

Do you think it might become difficult to workout which steps are for
which run?
i.e the 5th indented Step broke so lets read through and find each 5th
step to see what the run did.

Hopefully because the nested output contains the expanded template
variable
the navigation will be OK. The idea was that the nesting would only
happen
when there was variation i.e. a different matcher in the first example
or a
different result in the second. If you showed the nesting for every step
regardless then yes I think things could get pretty confusing.

I haven’t really thought about what happens with errors yet. For example
if
you have five things in your table and they all fail generating a
massive
output stack. Would you really want to see five output stacks?

It does sound like an interesting idea. Would you mind adding the full
output you would expect with this indentation in the ticket please? (If
you want colour as a html attachment).

I will do this, but let me think about it a little more

Thanks,

Your welcome