BDD and TDD - What are they for?

Ruby
21

David M. wrote:

I would say that such a project is very much in need of some requirements,
even informal ones.

Here’s a nice post by one Scott Ambler (who I suspect is down with BDD):

He ran a survey comparing projects that called themselves “Agile” to
those
that called themselves “traditional”.

One big difference between them is Agile uses “just in time
requirements”,
while “traditional” uses “big requirements up front”.

Scott then very politely tries to rationalize why the Agile projects
claim a
higher success rate - despite the survey also stated more folks called
their
own process “traditional”. Put another way, the survey was about
traditionalists decrying their own failures - but of course Scott is too
polite to come right out and state that.

The point of TDD is to reduce the risk that your requirements are wrong.
Rapidly delivering small increments of new features, every week,
complete
with literate test cases, helps keep your client “in the loop”.

22

On 18 Aug 2008, at 12:46, paron wrote:

The whole “requirements in front” idea is perfectly simple and
logical; also, typically impossible or wrong.
“If I’d asked people what they wanted, they would have asked for a
faster horse.” – attributed to Henry Ford.

Very true.

As my experience is primarily green-field development I’ve spent my
career arguing with clients who not only didn’t understand their
problem domain but were entirely blinded to that fact by their
preconceptions. This is not to say that the requirements they bring to
the table are completely worthless, but they have to be viewed in
terms of need much more than of wants to develop a cost-effective and
lasting solution.

Taking the horse analogy, a man who wants to buy six horses may be
doing so because he needs to plough three fields in a day and be used
to thinking in terms of three teams all running in parallel, when in
actual fact a tractor may be able to do those same three fields in a
single day by serialising the task and thus do away with the need for
two additional teams. That’s essentially the point of a faster horse.

To convince the man of this you need a working tractor to demonstrate
and the only test that matters is the ploughing, which is a good
argument for prototyping. All other tests that you may frame for the
tractor - or for the horses for that matter - fall under the general
category of robustness/quality and will generally be ignored by the
man at this stage unless the tractor breaks down during the
demonstration.

From what I’ve seen of BDD and TDD in practice, they focus a lot of
effort into making the teams of horses as close a match to those the
man wants as possible so as to achieve incredibly high satisfaction
ratings: they’re thus an excellent marketing technique. However what
the man fundamentally needs is a more effective ploughing solution
that costs him less money, and figuring that out requires a broader
perspective than that provided by just-in-time requirements analysis.

He probably has to put more effort into getting a tractor - as
reflected by a higher capital investment and higher fuel costs - but
the productivity of this solution more than pays for the extra
investment. Would he be satisfied watching the whole tractor
construction project from initial back-of-the-envelope design to real-
world deployment? I very much doubt it. What he’s interested in is
ploughing, not engineering. But what he ends up with is still the
better solution to his actual problem than the wonderfully supportive
development of a six horse ploughing solution even though that’s what
he wanted all along.

I can see that BDD + inspiration could develop the tractor as a
solution to the problem, but could BDD on its own? How flexible is it
in practice and can a project that’s 100k+ SLOC of actual application
(plus whatever overhead in test code) take revolutionary departures?
Does it suit the traditional hacker habit of forking early and forking
often? And what are the implications for all that test code?

Ellie

Eleanor McHugh
Games With Brains
http://slides.games-with-brains.net

raise ArgumentError unless @reality.responds_to? :reason

23

paron wrote:

The whole “requirements in front” idea is perfectly simple and
logical; also, typically impossible or wrong.
“If I’d asked people what they wanted, they would have asked for a
faster horse.” – attributed to Henry Ford.

Tx I am so passing that one to the XP mailing list!

24

On Mon, Aug 18, 2008 at 10:01 AM, Eleanor McHugh
[email protected] wrote:

From what I’ve seen of BDD and TDD in practice, they focus a lot of effort
into making the teams of horses as close a match to those the man wants as
possible so as to achieve incredibly high satisfaction ratings: they’re thus
an excellent marketing technique. However what the man fundamentally needs
is a more effective ploughing solution that costs him less money, and
figuring that out requires a broader perspective than that provided by
just-in-time requirements analysis.

I think you may be conflating two different scenarios. There’s the
scenario where a customer knows approximately what they want, and an
agile team can help them triangulate in on exactly what they need with
a minimum of wasted effort. Then there’s the scenario where someone
who has worked in a particular domain for awhile realizes that things
could be done in a much better way, and starts a project to implement
it. Often, this is someone who has worked on several teams such as
previously described. In this case, the visionary, not the projected
end-user, is the “customer”, at least in initial development. If all
goes well, and they hire a good marketing team, the visionary and her
team then convince their end-users that what they REALLY want is this
revolutionary product. This goes a lot easier if the team in question
is already well known and respected for their evolutionary work in
the same field.

I would submit that when a customer hires you to build them a better
plow, you should build them a better plow. However, once you have
done that a few times you are in an excellent position to go off for
six months and innovate a new hydroponic gardening system and sell it
to your existing customers.

–
Avdi

Home: http://avdi.org
Developer Blog: Avdi Grimm, Code Cleric
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

25

On Mon, Aug 18, 2008 at 4:11 PM, Phlip [email protected] wrote:

paron wrote:

The whole “requirements in front” idea is perfectly simple and
logical; also, typically impossible or wrong.
“If I’d asked people what they wanted, they would have asked for a
faster horse.” – attributed to Henry Ford.

Tx I am so passing that one to the XP mailing list!

Why be offended, our planet might be better off with faster horses, I
actually would like horses running at Warp 10! (No that is not 42
times 10 square times 12 square, 10 is fast enough).
Holding on to them might prove difficult though.
Well did we get off topic here?
R.

26

On Fri, Aug 15, 2008 at 11:12 PM, Phlip [email protected] wrote:

You should run a TDD test suite after the fewest possible edits: 10 at the
most, and hopefully just 1. A coding session should go type-type Test,
type-type Test, type-type Test, in tiny cycles.
Some believe that no editing at all shall be done other than making a
failing test succeed. I quite adhere to this philosophy, I am however
taking the liberty to omit the failing test run before editing as my
experience has shown me that enough failing test runs will follow
anyway :wink:
However I do not code unless I want a new test case to succeed, that
is only when I am doing TDD of course.
Cheers
Robert

27

On Mon, Aug 18, 2008 at 3:54 PM, Clinton D. Judy [email protected]
wrote:

Actually, are there any suggestions for learning rSpec for someone who’s
never done BDD or TDD before? I bought the first rSpec video tutorial
from Peepcode, but haven’t touched it yet.

Peepcode’s pretty awesome, I’d say go with that :slight_smile:

–
Avdi

Home: http://avdi.org
Developer Blog: Avdi Grimm, Code Cleric
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

28

Thanks for the great replies. I’m attempting to learn BDD through rSpec
now.

Actually, are there any suggestions for learning rSpec for someone who’s
never done BDD or TDD before? I bought the first rSpec video tutorial
from Peepcode, but haven’t touched it yet.

29

Clinton D. Judy wrote:

Thanks for the great replies. I’m attempting to learn BDD through rSpec
now.

Actually, are there any suggestions for learning rSpec for someone who’s
never done BDD or TDD before? I bought the first rSpec video tutorial
from Peepcode, but haven’t touched it yet.

Yes - learn TDD, because that’s what most of the tutorials will cover.
And
Test::Unit comes bundled with Ruby. At the entry level, if you try to
learn BDD
first, it’s just TDD with inverted syntax.

Then, also look at non-Ruby TDD tutorials, such as /Test Driven
Development/ by
either Kent Beck or Dave A…

30

On Aug 18, 10:23 am, Avdi G. [email protected] wrote:

team then convince their end-users that what they REALLY want is this

Avdi

Home:http://avdi.org
Developer Blog:Avdi Grimm, Code Cleric
Twitter:http://twitter.com/avdi
Journal:http://avdi.livejournal.com

Yes, there’s a fundamental difference between making incremental
changes and reimagining the problem domain. My background, like
Ellie’s, is in greenfield situations, which colors my thinking. As she
and Avdi point out, BDD is fine for evolutionary changes, but not for
revolutions in thinking. BDD is a caterpillar tractor – controlled
and unstoppable. Real innovation is more like dynamite.

Even a non-working prototype will help your customer make the leap to
see his problem in a different light. It also helps if they are
desperate. If they aren’t desperate, they will stick with various
combinations of horse until they can no longer do so. Then they might
be willing to make the leap – but you still have to show them a
prototype.

Once you have buy-in to a fundamental change or a novel idea, THEN BDD
comes into its own.

31

On Monday 18 August 2008 06:46:46 paron wrote:

“If I’d asked people what they wanted, they would have asked for a
faster horse.” – attributed to Henry Ford.

That’s actually a really good analogy – but you see, Ford’s leap wasn’t
abandoning what his clients wanted altogether. People obviously do want
cars.

But why do they want cars? What’s the benefit of cars?

It would not be very hard to write, up front, a requirement that it be
able to
transport people and some limited amount of cargo, and that it should go
faster than a horse.

More importantly, I don’t think BDD (or TDD) is about specifying this
requirement up-front, nor do they prevent emergent phenomena.

It’s more about specifying it before you code it. Which could be down to
several minutes before you code it.

Unless you really are blindly banging around on your keyboard, to see
what
happens. Which I’ll admit to doing – but I tend to do that in irb, not
in
source code.

32

On Tue, Aug 19, 2008 at 7:01 AM, paron
[email protected] wrote:

Yes, there’s a fundamental difference between making incremental
changes and reimagining the problem domain. My background, like
Ellie’s, is in greenfield situations, which colors my thinking. As she
and Avdi point out, BDD is fine for evolutionary changes, but not for
revolutions in thinking. BDD is a caterpillar tractor – controlled
and unstoppable. Real innovation is more like dynamite.

I think you missed my point. BDD is equally applicable to both
status-quo development and to radical departures and prototyping.
What changes is not the development practices, but the customer who
helps determine what the proper specs should be. When you are
building a typical evolutionary system, the customer is the client;
the end user. When you are developing an innovative new product, the
customer is you - or whoever has the vision for the new product.
BDD/TDD is still equally important, but the person defining the specs
differs.

There is a line of thinking that says that TDD/BDD is inapplicable to
prototypes, because the prototype will just get thrown out anyway. In
my experience, though, this rarely happens - in the real world you
don’t have time to throw out the prototype, and it winds up getting
pressed into service as the basis for the final product. Which then
necessitates a lot of painful retrofitting of test coverage.

Agile methodologies do generally accept creating “spikes” without
testing up front. A spike is basically an exploratory
proof-of-concept for one particular high-risk part of the application

  • e.g. “let’s do a spike to prove that we can process 1000 SOAP
    messages per second”. Spikes are not prototypes, however. Their
    scope is much smaller, usually taking only a few days to a week or two
    to create. As a result they are a lot easier to throw away than a
    working prototype.

Once you have buy-in to a fundamental change or a novel idea, THEN BDD
comes into its own.

In my experience BDD/TDD comes into it’s own as soon as your code
exceeds somewhere around 100 lines of code. Or less.

–
Avdi

Home: http://avdi.org
Developer Blog: Avdi Grimm, Code Cleric
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

33

David M. wrote:

“If I’d asked people what they wanted, they would have asked for a
faster horse.” – attributed to Henry Ford.

That’s actually a really good analogy – but you see, Ford’s leap wasn’t
abandoning what his clients wanted altogether. People obviously do want cars.

The car industry also did it incrementally. The original “horseless
carriages”
were exactly that - a carriage frame with an engine strapped into the
luggage
rack. Model T cars look square and funny because that carriage was still
there,
underneath.

After people stopped thinking they needed faster horses, and after they
participated in reviewing the designs of cars, then the horseless
carriage
morphed, feature by feature, into modern cars.

More importantly, I don’t think BDD (or TDD) is about specifying this
requirement up-front, nor do they prevent emergent phenomena.

TDD allows you to safely change your design and features while keeping
out the
bugs. That makes just-in-time requirements safer.

34

On Tuesday 19 August 2008 06:01:38 paron wrote:

Yes, there’s a fundamental difference between making incremental
changes and reimagining the problem domain.

Yes, there is. But it’s not a difference in testing methodology.

I would use Ruby for any program I could – I find it equally useful for
experimental greenfield apps and aging production apps.

Similarly, I would use BDD as soon as I could, in an experimental
greenfield
app. And I do.

That’s my opinion, of course. I’ll never understand why, but some people
prefer to use Ruby or Python to prototype an app, then throw away the
prototype and start from scratch in Java or C++.

Even a non-working prototype will help your customer make the leap to
see his problem in a different light.

That is true. However, properly-implemented BDD could (sometimes, at
least)
lead to that prototype faster.

All else equal, a working prototype is better than a non-working
prototype.

I’d also argue that a non-working prototype carries several other,
hidden
risks – mainly, the risk of promising too much.

No matter how many times you tell them “This is just a prototype; it’ll
have
to be rewritten from scratch; we can’t actually do this for another six
months…” They’re still going to come out of that presentation
thinking “Man, that’s an awesome prototype! I can’t wait!”

No, you show the awesome-but-nonworking prototype to your boss. Not even
that – if you can, you show him a whiteboard sketch of it.

But even with bosses, and especially with clients, remember the Scotty
philosophy: Underpromise, so you can overdeliver. That’s how you earn a
reputation as a miracle-worker.

It also helps if they are
desperate. If they aren’t desperate, they will stick with various
combinations of horse until they can no longer do so. Then they might
be willing to make the leap – but you still have to show them a
prototype.

If you’ve got all the time it takes them to figure out that various
combinations of horse won’t work, you have the time to do it right.

The first car that Ford showed the world didn’t explode. The Wright
Brothers
built a prototype plane, and it flew. They weren’t complete, but for
some set
of requirements (aka specs/stories), they were working.

35

the counter point:

TDD seems badly suited to really rapid prototyping.

I often use TDD for production code that I need to work long after i
have moved on,
or for production apps that many people need to work on. but its kind
of like red-tape.
if you dont trust yourself or others, then get a contract. if you do,
then word of mouth is fine.

also its an issue of scale - if you are working on sth for > 1 week,
you need tests to keep it all together.
but many proofs are done over a weekend or a hectic all-nighter.

if you are looking for a proof of concept - if something works or not
(today, and you don’t care about tomorrow) then writing iron-clad
tests for everything seems redundant. if you have a level of
confidence, you can fly blind, and i feel get into a much better state
of flow without tests. a week later things may not work as planned
tho.

additionally tests prove things on a very abstract level. if the
wright brothers had spent a bunch of time with RSpec they would have
theoretically proved their model of a cylinder would have enough
horsepower to lift it itself. and the tests would ensure that the next
change to the air system, would be guaranteed to not break the
cylinders. good, solid, mainstream engineering.

but they would never have captured people’s imaginations as much as a
working flying machine.
if it blew up in a future flight, at least it flew first…

/dc

36

On Wed, Aug 20, 2008 at 1:08 PM, d c [email protected] wrote:

if you are looking for a proof of concept - if something works or not
(today, and you don’t care about tomorrow) then writing iron-clad
tests for everything seems redundant. if you have a level of
confidence, you can fly blind, and i feel get into a much better state
of flow without tests. a week later things may not work as planned
tho.

I understand the sentiment, but if you feel the need to write
iron-clad tests up front, you’re probably not going to see most of the
benefits of TDD.
For me, the flow isn’t broken if I limit each test to a single, basic
change that I’m about to make. I don’t even begin to think about edge
cases until I write tests that fail for the bare minimum basics (a
single use of a single function). Then I write that code. Then I
move on to the next thing.

Often times, figuring out what tests I need to write next involves
toying around with a prototype, or writing parts of my function in
IRB. This helps me figure out how my code should work without having
to think it through mentally. Once a get a sense of how that stuff
should work, I might refactor a bit under the green light to make room
for the new behaviour. Then I write the new failing test, then I
write the code.

Most of the time, this doesn’t cause any noticeable decrease in my
velocity, it even increases it sometimes. Purists might say you
shouldn’t write any code at all before your tests, but I think what’s
important is that you shouldn’t integrate any code before you write
the tests. Hacking around with an experiment, so long as it’s tiny
and can easily be followed by a formal test case, is always helpful.

-greg

37

On Wed, Aug 20, 2008 at 1:08 PM, d c [email protected] wrote:

additionally tests prove things on a very abstract level. if the
wright brothers had spent a bunch of time with RSpec they would have
theoretically proved their model of a cylinder would have enough
horsepower to lift it itself. and the tests would ensure that the next
change to the air system, would be guaranteed to not break the
cylinders. good, solid, mainstream engineering.

You mean “Unit Tests”, right? The term “tests” encompasses manual
tests, unit tests, functional tests, integration tests, usability
tests, user acceptance tests, stress tests, fuzz tests, smoke tests,
final acceptance testing, etc.

I don’t know where people get the idea that because you are doing Unit
Tests or BDD, you must not be doing any other kind of testing - for
instance, firing the app up and using it. TDD/BDD is a great way to
drive your coding, but it’s not the only kind of testing in a healthy
project.

See also railspikes.com

–
Avdi

Home: http://avdi.org
Developer Blog: Avdi Grimm, Code Cleric
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

38

On Wed, Aug 20, 2008 at 2:53 PM, d c [email protected] wrote:

i came to ruby cos i like to realize ideas quickly.
premature test cases is like premature optimization.
making a change means fixing a bunch of failing tests…
in the first 24 hours of a project, where you know things are likely
to change a lot, then testing + coding is close to 2X the work.

I don’t have a problem with doing 24 hours worth of exploratory coding
sans tests. But if I decide at the end of that I decide I want to
continue, my best bet is to THROW IT AWAY and rewrite it test-first.

If you and/or your organization have the discipline to a) stop
prototyping early; and b) throw the prototype away, then test-less
prototyping is workable.

–
Avdi

Home: http://avdi.org
Developer Blog: Avdi Grimm, Code Cleric
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

39

On Wednesday 20 August 2008 12:08:01 d c wrote:

the counter point:

TDD seems badly suited to really rapid prototyping.

I thought so, too. And I got quite a lot done, very fast, with this
hobby
project I’m working on…

Then it reached a threshold – not a very high one – where I could no
longer
reasonably develop the app, because of how long it was taking me to test
it,
manually. At this point, I at least needed a testing framework.

And that was short of having the working prototype that I want. It’s
obvious
at this point that from here on (till the working prototype), not having
tests will slow me down, not speed me up.

Now, I suppose it depends on the complexity of the prototype. If you
want to
prototype absurdly fast (a few hours or days), and it’s a reasonably
simple
concept (a couple hundred lines or so), then yes, it’s probably faster
to
just code it.

But the longer you put off writing those tests, the worse it’s going to
be
when you finally start.

40

David M. wrote:

TDD seems badly suited to really rapid prototyping.

Confer “Architectural Spike”. That means a prototype where your
colleagues are
the demonstration’s audience. You should write enough of a new module to
“close
the loop”. The new code will spike thru a view, middleware, back-end, a
little
productization, and a little testage. Y’all will then rewrite the spike
via TDD.

However, if the prototype’s audience is client-side, they might point
here and
there, and say, “okay, keep this here, but change that around. In one
hour.”

If they say that, you are already around the bend. The tests must
already be in
place to sustain the rate-of-change your client now expects of you. This
is why,
in some circles, the ideal is “there are no Prototypes!”

I thought so, too. And I got quite a lot done, very fast, with this hobby
project I’m working on…

Then it reached a threshold – not a very high one – where I could no longer
reasonably develop the app, because of how long it was taking me to test it,
manually. At this point, I at least needed a testing framework.

Right. The bigger the project, the longer it runs, the more you need
tests. They
help you add new features faster than you added old ones. For example,
at work,
last year we added a bad database field that got stale very soon. Back
then, the
code was too fragile to just yank it out. But since then, we have
relentlessly
tested, and cleaned all around that variable, and today we took it out
with no
issues.

Under Brand X, the sooner you take that variable out, the less you
suffer. The
code will ossify around it.