Is BDD with RSpec cheaper?

Testing RSpec
1

This might be a loaded question on this forum, but here goes:
Just had a discussion with a prospective employer, a Ruby On Rails shop.
His
reaction to BDD development on every project was skeptical, saying
something
like: “It depends on the project”. “Some jobs are so short that the
extra
time invested in developing tests doesn’t justify the cost”.
He was insistent that writing tests costs more. After all, you write
twice:
first the tests, then the code (or the other way 'round).
My question is: From hard, practical, cold real world experience, is
that
so? Is BDD development more expensive? Let me qualify that. One could
answer, “no, since the tests save you pain and heartache down the line”.
The
question is whether BDD coding with RSpec is more expensive in the
implementation phase and how much truth there is in the statement that
BDD
isn’t for every project, like quick knock ups for example?

View this message in context:
http://www.nabble.com/Is-BDD-with-RSpec-cheaper--tp18323328p18323328.html
Sent from the rspec-users mailing list archive at Nabble.com.

2

On Mon, Jul 7, 2008 at 2:25 PM, yitzhakbg [email protected] wrote:

answer, “no, since the tests save you pain and heartache down the line”. The
question is whether BDD coding with RSpec is more expensive in the
implementation phase and how much truth there is in the statement that BDD
isn’t for every project, like quick knock ups for example?

I’d say the cutoff-point for me is at around 100 lines of code. I’ve
successfully developed little one-off scripts - either utilities or
throw-away experimental spikes - of that length or less using only
manual testing, which may have taken a little longer had I developed
them test-first. I don’t think something of this size really
qualifies as a “project” though.

With anything larger than 100 lines I know from experience that I’d
just be slowing the project down if I failed to use TDD/BDD.

–
Avdi

Home: http://avdi.org
Developer Blog: Avdi Grimm, Code Cleric
Twitter: http://twitter.com/avdi
Journal: http://avdi.livejournal.com

3

On 7-Jul-08, at 12:25 PM, yitzhakbg wrote:

twice:

View this message in context: http://www.nabble.com/Is-BDD-with-RSpec-cheaper--tp18323328p18323328.html
Sent from the rspec-users mailing list archive at Nabble.com.

rspec-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/rspec-users

This is a TCO issue. IT WILL slow down initial development on a
small, well defined app, and there is no way around that. But over
the life of a medium-to-large application, BDD will greatly reduce
maintenance and enhancement costs.

So, if you are a contracting firm, paid to build sites, but not to
maintain them, that might be a point you want to consider. But if I
was your customer, I wouldn’t likely be coming back to you. If you
are hired to build and maintain an application, I really can’t see a
downside to writing the tests.

Regards,

Lori

4

yitzhakbg wrote:

question is whether BDD coding with RSpec is more expensive in the
implementation phase and how much truth there is in the statement that BDD
isn’t for every project, like quick knock ups for example?

Does TDD/BDD take longer that hacking something out? I would say that
it depends a lot on the project and the developer’s experience in that
domain, BDD, and doing BDD in that domain. That said, it probably isn’t
an unfair generalization to say that quick short jobs can be hacked
together faster without tests. Prototyping, spikes, and exploring new
technology are times when BDD would probably hinder you. As you point
out though you will be building up a technical debt by not having tests
that will come back and bite you at sometime. That is why I generally
throw away any prototype code and start over with a blank slate that I
can start doing BDD with along with my newly gained knowledge from the
prototype.

I have experiences where doing BDD has greatly sped up my development
and has helped me in compartmentalizing a complex problem into smaller
more manageable sub-problems. For a “quick knock up” then I think it it
good to ask how long this software is suppose to live, and whether or
not the developers have enough discipline to start adding tests once it
does become more complex. If the answer to the latter question is no
then forgoing tests in the beginning sounds like a recipe for disaster.
I suppose another question is how important this piece of software is.
If it is just for fun and you just want to see some results and could
care less about maintenance down the line then manual testing is just
fine.

One more thing… you said “The question is whether BDD coding with
RSpec is more expensive in the implementation phase…”.
BDD is not just the implementation phase but is also the design phase as
well. I could certainly throw together some systems faster without
doing BDD and then manually test the system afterwards. Would I have
confidence in the system? Yes. In fact, I would have just as much
confidence in the system as I would if I had a spec/test suite verifying
the behavior. However, if I had created that system with TDD/BDD from
the start I would end up with a greater sense of confidence of the
design. Easily tested code tends to go hand in hand with good design,
where hard to test code tends to carry some code smells resultant of a
poor design. Which is another danger in delaying creating an automated
suite because you may end up with hard to test objects which will be in
dire need of some refactoring.

I don’t know if I really answered your question or if I just added to
your confusion. In the end BDD, just like any other practice, has its
trade offs and should only be applied in the right situations. I think
the “right situations” will vary from developer to developer/team to
team and there may not be one single rule for where the exact cutoff is.

-Ben

5

On Jul 7, 2008, at 2:25 PM, yitzhakbg wrote:

twice:
first the tests, then the code (or the other way 'round).
My question is: From hard, practical, cold real world experience, is
that
so? Is BDD development more expensive? Let me qualify that. One could
answer, “no, since the tests save you pain and heartache down the
line”. The
question is whether BDD coding with RSpec is more expensive in the
implementation phase and how much truth there is in the statement
that BDD
isn’t for every project, like quick knock ups for example?

Yes - of course it will slow you down. There is nothing wrong with
leaving code unspec’ed (just be aware of the consequences).

Almost every app will grow to a size of immense complexity (i.e. every
app in it’s fullest wants to read email). If someone is paying you
for it (and it’s not a prototype), it almost always needs specs. The
real question is: how much is my software worth? How much should it
be worth?

If it’s worth nothing, it needs no specs. If it has any plans to grow
(or is already big), or is ever going to be maintained by someone else
besides the original maintainer, it needs specs. (There is one obvious
exception to this rule: Prototyping needs no specs, and actually
hampers growth).

There have been many times when I’ve been in a “rush” to get a piece
of code done, and so I don’t spec it. I almost always find a bug in
it later on, and always regret not spec’ing it. I’m not a testing
fanatic - I’m human (and make so many tiny mistakes it’s almost
unbelievable).

Dave A. said something a while back to the effect of “Unspec’ed
code sucks”. I’m okay with short scripts and data migrations
sucking. I don’t plan to spend a lot of time on it, and in the end,
I’m going to throw it away. No reason to keep a test suite around for
it. This is why you’ll hear many people say, “if it is under X number
of lines it doesn’t need specs.”

Scott

6

Thanks very much.

–
View this message in context:
http://www.nabble.com/Is-BDD-with-RSpec-cheaper--tp18323328p18326878.html
Sent from the rspec-users mailing list archive at Nabble.com.

7

I met a great analogy the other day - applies to TDD as well as BDD
really.

Unit testing is like double-entry book-keeping.

It takes longer to do the initial work, and yes you have some
repetition - but the reduction in errors saves you time and money
overall.

I’m struggling to think of a project that is so short that it needs no
testing. Anything that you have to run more than once, or that you
might conceivably have to change one day (especially as that change
will probably be done by a different set of developers to the original
code!) is going to benefit from testing.

Ask your employer, “so, would your accountants skip double-entry
bookkeeping on really small expenses?” :slight_smile:

I could see the temptation on a pure prototype mock-up project - but
you’d have to make damn sure that the technical debt of not having
tests was clear and up-front, as far too many prototype mock-up
systems get turned into real production systems, in which case you’d
be spending more effort retro-fitting tests to the prototype to get it
production-ready. And in my experience, it’s much harder to add
testing to existing code than to write the tests as you go.

  • Korny
8

Interesting statement that I have heard several times before, and it
all boils down to how one measures cost. If you measure the cost of
an application only in the time it takes to write the first line of
code to the last line of code for the first and only release, then
yes, you could argue that removing the tests would technically make it
cheaper. However, I’ve never defined the cost of an application like
that. The reason being, the application spends more of its life cycle
in subsequent releases (release 2 through release N). That being
said, the ROI on testing (behavioral or state based) becomes more
apparent in future subsequent release as they take less time to move
through the development life cycle. This is primarily due to the
design being better, as it is more conducive to change (testing
usually produces this side effect IMO), and releases test quicker, as
you have that rich suite of tests to support development and testing
efforts. Just my $0.02.

Anthony Broad-Crawford

9

On 08/07/2008, at 04:25 , yitzhakbg wrote:

Just had a discussion with a prospective employer, a Ruby On Rails
shop. His
reaction to BDD development on every project was skeptical, saying
something
like: “It depends on the project”. “Some jobs are so short that the
extra
time invested in developing tests doesn’t justify the cost”.

If I’m not expected to maintain the code, the project isn’t going to
grow particularly big, and the interface has fewer than 7 major
functions, there’s no real need to spec it out since you can hold the
entire design in your head.

The question is whether BDD coding with RSpec is more expensive in the
implementation phase and how much truth there is in the statement
that BDD
isn’t for every project, like quick knock ups for example?

If you’re trying to specify the design in the implementation phase,
then yes you’re wasting time. The time isn’t being wasted in writing
specs, it’s already been wasted in committing design to paper without
trying out any of the major features to see that they make sense in
code.

There’s one department at the University where I work that insists on
big design up front. They fully specify the software using UML or
whatever the flavour of the month is, then implement from that. Just
one more reason why I don’t ever want to work within a bureaucracy.

Alex