[cucumber] Cucumber and CI

Testing RSpec
21

On Mon, Feb 23, 2009 at 11:44 AM, James B. [email protected]
wrote:

F=31, S=165, PC=100%, TU=30, CU=21, financial services (insurance
claims)

Whathuh? Wow, you really do work for government, don’t you? >8->
That line hurts me to look at and I’m not going to try to put my
answer in that format.

Since I’ve started using Cucumber I’ve mostly been doing smaller
projects, and the only ones I’ve completed or brought near completion
have been personal ones. My last project was a spike that had to be
done inside a week and I abandoned most testing (and it shows), but
apart from that it seems like the projects I consider ‘smallish’ tend
to have about 10-30 features, with usually 5-10 scenarios per feature.

If I had to think about how feature counts scale, I’d say that in the
most general case they scale by significant models. A “significant
model” is an entity that really matters to the business process (as
in, you can’t describe the process without it) and that has a
non-trivial interface. If I’m building a membership registration
system, “Member” is certainly a significant model. “Payment” is
significant. “Phone number” probably isn’t, even if I break it out
into a separate table and Rails model for relational reasons.

A significant model will likely have somewhere from 2-5 features
simply by virtue of CRUD operations. You need to be able to view the
information. That’s a feature. You need to be able to edit it.
That’s a feature too. Whether create/update/delete are separate
features or all one feature varies by how complex or different the
interface needs to be in each case. If it’s all one form and that
form has a “Delete” button with a simple yes/no confirmation, you can
probably cover it with one feature. Sometimes you can’t. …And
sometimes there’s a need for imports or printed reports or whatever,
and those are all features.

So that’s how I gauge this stuff. The driving metric is interface
complexity and the number of different major interactions an actor
could have with the application. I don’t know how to gauge “percent
completion” off of that, or why the nature of the industry would be
important.

And… “Total or concurrent users?” That you’re asking for that
information totally baffles me. How could that possibly make a
difference to the number of features? That’s a scaling issue. An
implementation detail, unrelated to app complexity or business value.
If I wrote an executive information system that only had 15 users, but
those users used it to save millions of dollars, how does the number
“15” help you determine how many features you should write?

–
Have Fun,
Steve E. ([email protected])
ESCAPE POD - The Science Fiction Podcast Magazine
http://www.escapepod.org

22

On Mon, Feb 23, 2009 at 9:09 AM, Stephen E. [email protected] wrote:

On Mon, Feb 23, 2009 at 10:22 AM, Mark W. [email protected] wrote:

Our mantra is ABC: Always Be Committing. So we commit anytime we feel
like it, as long as it doesn’t break the build. This makes life a lot
easier when there is merging to do.

I think your “doesn’t break the build” condition is a lot bigger than
you make it sound. >8-> What’s the definition of “the build” in your
work culture? Do you run all tests every time before committing? Or
just before pushing?

If you’re working in a local branch, you can do whatever you want. :slight_smile:
Personally, I run all specs and features before I commit to my local
topic branch.

When I want to push to the remote respository, I pull into my local
master branch, rebase the topic branch from master, merge the topic
into master, run all specs and features, then push.

///ark

23

James B. wrote:

Just on a side note, how many features / stories have people seen on
their projects and how much of their project was covered by
features/stories?

My current day job is old-school Rails. Some of the tests - written
before I got
here - used a most despicable pattern. Someone would write a test on
some
transactions which pushed the DB into state X. Then, when they needed a
database
in state X to TDD the next transaction, they would call the old test.

This habit - across dozens of business rules - lead to tower-of-jello
test
cases, where any disturbance risks us commenting out the tests, because
we can’t
figure out how to pass them - even by reverting and trying again!

So, in this degenerate case, we have saturation testing for all the
low-level
code methods, but we are missing the high-level view that Cuke ought to
be
providing…

–
Phlip

24

On Mon, Feb 23, 2009 at 3:22 PM, Mark W. [email protected] wrote:

On Mon, Feb 23, 2009 at 9:09 AM, Stephen E. [email protected] wrote:

I think your “doesn’t break the build” condition is a lot bigger than
you make it sound. >8-> What’s the definition of “the build” in your
work culture? Do you run all tests every time before committing? Or
just before pushing?

If you’re working in a local branch, you can do whatever you want. :slight_smile:

Ah, okay, so we are talking about pretty much the same thing. >8->
The way you said “committing” I thought everyone was going into the
same integration branch every time. Mea culpa for assuming.

–
Have Fun,
Steve E. ([email protected])
ESCAPE POD - The Science Fiction Podcast Magazine
http://www.escapepod.org

25

Comment inline

2009/2/23 Mark W. [email protected]

There can be a difference between committing and integrating now we have
distributed vcs.

I’m not sure if you meant that but if you did then I think these
sort of checkins have to be in bigger chunks. The reason is that each
checkin to an intergration server is asking my colleagues to checkout my
code and integrate it into their current work.

Just because I push doesn’t mean my coworkers have to pull.

Yes it does, surely you can’t be saying you can commit to an integration
server without pulling the code from it first. Git wouldn’t let you push
to
the integration server if you were behind.

IMO a complete
scenario is about the smallest size chunk to integrate with, and a
complete
feature about the largest

A refactoring, a new method (and its tests), a new test, a fixed typo

  • these are all appropriate chunks of code to check in.

Agreed but the context was a BDD workflow

I think this is far superior to making massive checkins at the end of
each iteration. We usually fall somewhere in between.

I did say a complete feature was the largest chunk to go to integration,
and
for this to be acceptable for me it should be a very small feature. So
neither of these checkins is massive I’d envisage 30mins to 2 hours work
maybe.

26

Stephen E. wrote:

On Mon, Feb 23, 2009 at 11:44 AM, James B. [email protected]
wrote:

F=31, S=165, PC=100%, TU=30, CU=21, financial services (insurance
claims)

Whathuh? Wow, you really do work for government, don’t you? >8->
That line hurts me to look at and I’m not going to try to put my
answer in that format.

Only in the sense that every taxpayer does. It seemed me a short hand
way of describing the information requested without being particularly
obscure.

Total users vs. concurrent users gives a very good idea of the resources
behind a project, or at least the potential resources, together with an
idea of how important to a business that a project might be. Certainly,
it will not catch the corner cases of say of high value, low volume
processing, and it may give unwarranted weight to a minor project in a
large corporation. On the whole though, I think it will give a good
feel for what size feature/story driven projects are presently. Perhaps
it might be better to ask for the numbers of active developers / client
representatives per project instead.

27

On Mon, Feb 23, 2009 at 5:45 PM, Andrew P. [email protected]
wrote:

Just because I push doesn’t mean my coworkers have to pull.

Yes it does, surely you can’t be saying you can commit to an integration
server without pulling the code from it first

I can’t and I didn’t. :slight_smile:

///ark

28

On Mon, Feb 23, 2009 at 6:57 PM, James B. [email protected]
wrote:

Total users vs. concurrent users gives a very good idea of the resources
behind a project, or at least the potential resources, together with an
idea of how important to a business that a project might be.

If you say so. Personally I don’t grok that relationship at all. My
driving “metaproject” is my organization’s Web site, which has tens of
thousands of total users across different roles (members, academic
institutions, publishers who want to rent our mailing list, employers
and job seekers, etc.) but a concurrent user count of…probably two
digits on an ordinary day. I’ve never bothered to measure it for
certain, but I could do some math with Google Analytics and tell you
that it couldn’t be higher.

What can you learn from that? Taken in isolation, without knowing
anything more, could you compute the value of the Web site to the
organization? Could you tell me what the feature count is likely to
be?

–
Have Fun,
Steve E. ([email protected])
ESCAPE POD - The Science Fiction Podcast Magazine
http://www.escapepod.org

29

On 23 Feb 2009, at 00:09, Yi Wen wrote:

Wait for an hour before I can checkin something is still too long
for me. I’d like to checkin every couple minutes most of time.

But I think to make each step just pending first and then make it
green when I finish implementation for the step makes sense. I
probably will still use unit tests passing as checkin points.

Other people have mentioned this in passing, but here’s a great write-
up of using git branches to manage local work. I’ve been doing it for
a couple of months now and it’s a fantastic way to work:

http://blog.hasmanythrough.com/2008/12/18/agile-git-and-the-story-branch-pattern

See particularly the notes about squashing commits - this allows you
to commit really often in your local branch, then merge these commits
together before you push them into the main source control repository.

You can even use git commit --amend to commit on red (e.g at the end
of the day) and then change that commit later.

Matt W.
http://blog.mattwynne.net

30

On 24 Feb 2009, at 09:30, Rob H. wrote:

What have you gained by commiting a known-bad change set? It does no
harm sure, but I don’t understand the gain.

Good question. It’s not something I do routinely, but when I do, I
have a couple of motives:

I take my laptop home on the bus through central London, so I like to
push my local branch up to the server (using the wonderful
git_remote_branch[1]) just in case some scally-wag takes a shine to it
and decides they want it more than me.

The maturity of the team I’m on means that we have quite a variety of
development environments, so I might again use the remote branch as a
‘shelf’ to pass the failing code from one machine to another if we
decided to move to working on someone else’s machine.

I also do this when I need to urgently switch out of the story branch
to work on something else. In that case I drop everything, commit as
it, and checkout the branch I need to start working on.

It’s also worth mentioning Kent Beck’s advice in the original TDD book
where he advocates leaving one test failing when you go home. Not that
you have to commit it to source control, but it’s a nice idea -
leaving a thread hanging so you know where to start the next day.

[1]GitHub - webmat/git_remote_branch: A tool to simplify working with remote branches

Matt W.
http://blog.mattwynne.net

31

On Tue, Feb 24, 2009 at 9:17 AM, Matt W. [email protected] wrote:

You can even use git commit --amend to commit on red (e.g at the end of the
day) and then change that commit later.

While I think commit --amend is very useful, I’m not sure why you’d
bother to commit at the end of the day, knowing full well you were
going to amend it first thing tomorrow morning.

What have you gained by commiting a known-bad change set? It does no
harm sure, but I don’t understand the gain.

32

David C.
That may be so, but one view of agile is that each iteration is a
mini-waterfall. BDD suggests that we should define all of the
scenarios in the iteration planning meeting because we use them as a
planning tool (how can we estimate a feature at all before we’ve
talked about the acceptance criteria?).

Pat MAddox
I don’t know that it’s bad. At the beginning of an iteration, I have
most of the features & scenarios that I’ll be working on. So I start
off with a big pile of yellows, and as the iteration moves on it
gradually turns green. I’d say we average 8-10 pending features at
the beginning of each iteration maybe.

I really of like this approach of mini-waterfalls, treating them as
short 1-2 week iterations,
writing only the stories for the upcoming iteration, estimating them and
organizing them by priorities.
Pivotal does a much better job than myself of telling us what we can do
depending on our previous velocity.

Regarding commits, the approach I’ve enjoyed the most, was whilst pair
programming.
One of us would make the first step go from pending to passing, and make
the next
step fail. At that point we committed to a branch. The other
programmer,
then pulled from that branch, and followed the same process of
implementing the code
to make the failing step pass, write the definition of the next step so
that it failed and
committed to the branch.

We had to work remotely, so every time we committed to the branch we
also,
disconnected from VNC and connected to the other persons machine.
That way when it was your turn to type you would feel more comfortable
in your
machine whilst the other person is just observing and discussing whilst
connected to your machine.

Usually once we got a single scenario working we would rebase with
master
and push to the integration server. All other scenarios in that feature
were left pending,
to minimize the chances of breaking the build.

I guess this same process can be applied individually, even though,
its hard to keep the discipline, I usually just keep going without
committing until the
whole scenario is passing, and its never as much fun as working with
someone.

Cheers,

Rai

33

Matt W. wrote:

I take my laptop home on the bus through central London

Got WiFi?

–
Phlip

34

On Tue, Feb 24, 2009 at 10:30 AM, Rob H. [email protected]
wrote:

On Tue, Feb 24, 2009 at 9:17 AM, Matt W. [email protected] wrote:

You can even use git commit --amend to commit on red (e.g at the end of the
day) and then change that commit later.

While I think commit --amend is very useful, I’m not sure why you’d
bother to commit at the end of the day, knowing full well you were
going to amend it first thing tomorrow morning.

Because the longer you wait, the more your code will diverge from your
teammates’. If you don’t commit often you rob them of the opportunity
to reduce merge hell.

Aslak

35

Stephen E. wrote:

What can you learn from that? Taken in isolation, without knowing
anything more, could you compute the value of the Web site to the
organization? Could you tell me what the feature count is likely to
be?

I do not know what the actual feature count and scenario count is for
any type of project of any scale at present. Nor is my question
intended to answer what the likely feature/scenario count of any project
is likely to be. However, I do think that getting some initial data
will, in itself, lead to refinements in the approach to the question. I
believe that this concept is essence of agile development is it not?

In addition to the counts I asked for the nature of the project as well.
A social network project with 40 features, 200 scenarios and 100sK/100
total / concurrent users can be considered in a different light than one
with 400 features, 8000 scenarios, 50 total and 15 concurrent users; if
the second one happens to be a manufacturing process control system.

The question of value to the organization is not one that I raise and is
in any case irrelevant. It may be assumed that any project that is
funded represents some value to someone in the sponsoring enterprise.
Whether it actually would provide any material benefit to the
organization as whole is a question which is frequently left unasked and
unanswered in my experience. Which, before you ask, involves computer
systems design and development at several very large multinational
corporations. (Which is the main reason that I now do what I do where I
am.)

To get back to the initial question, I am only looking for a few
primitive metrics regarding scope and scale to get a sense of how
features/scenarios counts relate to specific projects. Doubtless, there
are better questions to ask. Perhaps the number of models, the expected
number of rows, the anticipated number of transactions per day would all
provide better insight. But, other than counting the number of models,
this information requires a deal more effort than counting the number of
features and scenarios one has, estimating how much of ones code base is
covered by them, guessing how many concurrent users you are expected to
support and outlining the basic nature of the project. Or, so I
beleive.

36

On Tue, Feb 24, 2009 at 3:47 PM, Rob H. [email protected]
wrote:

Because the longer you wait, the more your code will diverge from your
teammates’. If you don’t commit often you rob them of the opportunity
to reduce merge hell.

Please note I did say commit, and not push, and I inferred from Matt
he meant commit and not push (although he has explained otherwise
since).

Ok, let’s all be more specific when we talk about scm operations. Not
everyone is using git all the time. (I wish I did, but I often work in
the “enterprise”, so it will take a while for them).

Say “git commit” or “svn commit” or “git push” instead of just
“commit” or “push”.

Aslak

I find pushing last thing at night even more bizarre to be honest :confused:

Completely agree. Ending the day with a git push / svn commit is
verboten where I work. Dave Laribee describes why in biblical form:
http://codebetter.com/blogs/david_laribee/archive/2008/09/22/laribee-s-final-law-of-continuous-integration.aspx

37

Because the longer you wait, the more your code will diverge from your
teammates’. If you don’t commit often you rob them of the opportunity
to reduce merge hell.

Please note I did say commit, and not push, and I inferred from Matt
he meant commit and not push (although he has explained otherwise
since).

I find pushing last thing at night even more bizarre to be honest :confused:
If you’re are going home, it seems reasonable that other people might
be, ergo there won’t be many more changes made (an assumption
granted). Also, if they are going to continue to work and make
changes, why force them to merge a
broken/half-done/possibly-to-be-completely-redone later commit. Makes
no sense to me :confused:

38

On 24 Feb 2009, at 15:23, aslak hellesoy wrote:

I find pushing last thing at night even more bizarre to be honest :confused:

Completely agree. Ending the day with a git push / svn commit is
verboten where I work. Dave Laribee describes why in biblical form:
http://codebetter.com/blogs/david_laribee/archive/2008/09/22/laribee-s-final-law-of-continuous-integration.aspx

I was just looking for that post, and you beat me to it :slight_smile:

Matt W.
http://blog.mattwynne.net

39

I was trying to use intergration and commit to be specific scm terms in
this
sort of discussion, so rephrasing (and hopefully improving on ) Dave
Laribee’s law we get

“If its late in the day save your next integration for the morning”

Now people can go home on time and not fret about a build breaking at
the
end of the day and having to stay late and rush to fix it.

Andrew

2009/2/24 aslak hellesoy [email protected]

40

On Tue, Feb 24, 2009 at 9:47 AM, Rob H. [email protected]
wrote:

I find pushing last thing at night even more bizarre to be honest :confused:
If you’re are going home, it seems reasonable that other people might
be, ergo there won’t be many more changes made (an assumption
granted). Also, if they are going to continue to work and make
changes, why force them to merge a
broken/half-done/possibly-to-be-completely-redone later commit. Makes
no sense to me :confused:

I think that’s because you’re assuming that there’s just one active
thread of code. If you’re not pushing to the integration branch,
you’re not forcing anyone to do anything. You can push your own
in-progress development branch to the server (in SVN, in Git, in
anything that supports branches at all) just to have it someplace
other than your own machine, and that imposes no cost on anyone else.

I do it all the time just to be paranoid. “My laptop might get
stolen” is a perfectly sensible reason to take three seconds before
closing the lid. Or “My place might burn down,” or “I might get hit
by that bus I was waiting for,” or “I might have an epiphany and quit
my job tomorrow morning to become a chess grandmaster,” or even just
“I wonder if my manager would like to look at my functional and
elegant code.” (In some places it might even be “I’d better prove to
my manager that I did something today.”)

In any case: pushing to the team’s main VCS repository may be a
necessary step for integration, but it doesn’t mean every push has to
trigger an integration. Not if you’ve created a consistent and
well-understood culture of branching.

–
Have Fun,
Steve E. ([email protected])
ESCAPE POD - The Science Fiction Podcast Magazine
http://www.escapepod.org