I saw Dave A. talk about this, at Canada on Rails, and initially I
thought it was a bunch of BS. It seemed like the only difference
between BDD and TDD was the words; everything else was just the same
thing with new names. Like the only development involved was a bunch
of alias statements. But I was totally wrong. What you’re really doing
is a more intuitive, smoother version of TDD which produces
documentation as its byproduct. TDD isn’t really about testing – it’s
about design. BDD makes it easier to use TDD for what it’s really for.
Giles B. wrote:
I saw Dave A. talk about this, at Canada on Rails, and initially I
thought it was a bunch of BS. It seemed like the only difference
between BDD and TDD was the words; everything else was just the same
thing with new names.
It’s easy for folk to dismiss RSpec by saying that it’s just semantics
… but when you are dealing with meaning, semantics is everything.
TDD isn’t really about testing – it’s
about design. BDD makes it easier to use TDD for what it’s really for.
Heh, you nailed it!
–
– Jim W.
On 1/23/07, Christian N. [email protected] wrote:
They can contain anything except NUL bytes (the extended symbol
syntax). They are turned into class and method names automatically
(we need some special formatting for that due to Test::Unit
restrictions).Is it possible to nest contexts? Is this a good idea?
test/spec allows them since 0.2, and I use them to group contexts that
go together. Note that nothing is inherited, it’s purely namespacing.
RSpec doesn’t have them, and they aren’t interesting in implementing
them AFAICT.
That’s correct. We’ve really struggled with this because users want a
means of sharing context material, but we believe that nesting
contexts is going to lead to specs that are confusing to look at. We
believe that clarity supercedes DRY in specs, and so we haven’t
supported this yet.
That said, you can include modules and get reuse that way:
module SomeHelperMethodsUsefulAcrossContexts
end
context “some context” do
include SomeHelperMethodsUsefulAcrossContexts
end
I’d be surprised if test/spec doesn’t already support this out of the
box.
Cheers,
David
Is there an equivalent to autotest for rspec (autospec?)? I’ve googled
around, but only came across a Rails plugin, which (perhaps obviously)
is very Rails-centric.
-P
Wayne V. wrote:
I don’t look at this as “add a test to the spec/add a trivial fix to
the code” but instead as “add a test to the spec/do the simplest thing
that could possibly work to get the tests to pass”, and I find
substantial value in this approach.
When I do TDD, I approach it with a Jeckle and Hyde perspective. When I
write a test, I am writing a specification of how the code should work.
When I start writing code, I become the laziest programmer you can
imagine, writing only enough code to pass the specification as it is
written at that moment. If it is not called out in the spec, it doesn’t
get written.
When I switch back to test writing, I cuss out that lazy programmer
(i.e. me just a few moments ago) and write additional specs to force the
code to do exactly what I want.
The tension between Dr. Jeckle (the spec writer), and Mr. Hyde (the
programmer) causes Dr. Jeckle to write better specs and Mr. Hyde to
write lean, correct code.
–
– Jim W.
On 1/23/07, Jim W. [email protected] wrote:
imagine, writing only enough code to pass the specification as it is
written at that moment. If it is not called out in the spec, it doesn’t
get written.When I switch back to test writing, I cuss out that lazy programmer
(i.e. me just a few moments ago) and write additional specs to force the
code to do exactly what I want.The tension between Dr. Jeckle (the spec writer), and Mr. Hyde (the
programmer) causes Dr. Jeckle to write better specs and Mr. Hyde to
write lean, correct code.
With me it’s more like Abbott and Costello!
Seriously, thanks for describing this so well.
Cheers,
David
On 1/19/07, James Edward G. II [email protected] wrote:
On Jan 19, 2007, at 6:55 PM, Sam S. wrote:
The problem is a lot of people don’t do TDD right, and the
semantics of
it raise the barrier to entry.Yes. This has always bothered me about the zentest tool. I love
many of the tools in that package, but the zentest tool seems to
encourage bad testing habits. The video linked to earlier in the
thread explains why.
What’s wrong with zentest, specifically?
Thanks,
Joe
On Jan 23, 2007, at 6:10 PM, Joe Van D. wrote:
thread explains why.
What’s wrong with zentest, specifically?
You will probably understand better by watching the movie, but
zentest encourages you to make a test case for each class and a test
method for each method. BDD teaches that these decisions are pretty
arbitrary.
The true goal of testing is to test the “behaviors” of your object.
If that means interacting with five methods or just one possible call
sequence of a method that has several, that’s what you should really
be testing. What do if you need to work with two classes at one to
test something?
This class and method organization is also encourages you to use so-
so names for your tests at best. If you have a Calendar class and a
TestCalendar, that’s pretty much saying the same thing twice (a
violation of DRY) and it doesn’t tell you much. BDD encourages you
to actually express what you are testing with your naming. This
really helps you focus on the goal of the process and is much less
arbitrary.
There you have it, in my own words. Just to be clear, the above is
my opinion through and through. I do not speak for the zentest or
BDD teams.
James Edward G. II
On Jan 23, 2007, at 16:26, James Edward G. II wrote:
On Jan 23, 2007, at 6:10 PM, Joe Van D. wrote:
What’s wrong with zentest, specifically?
You will probably understand better by watching the movie, but
zentest encourages you to make a test case for each class and a
test method for each method.
Which will match up 1:1 if you factor your code well.
BDD teaches that these decisions are pretty arbitrary.
I don’t use BDD because I already have my tests broken up by units of
behavior. I call them methods.
BDD presents a method of thinking about test design to help you focus
on better object design and object modeling. Unit testing doesn’t
have this method of thinking built-in, you have to discover it.
(Although it makes itself evident if you pay attention to how
difficult it is to test something.)
The true goal of testing is to test the “behaviors” of your
object. If that means interacting with five methods or just one
possible call sequence of a method that has several, that’s what
you should really be testing.
I think there are two true goals of testing. One is to enumerate
your edge cases so each one behaves as you expect. Another is to get
feedback about how well you’ve designed your program. Both are
equally important.
If you want to write specifications to enumerate your edge cases,
great. However, if you’ve factored poorly neither BDD nor TDD will
give you small, concise tests nor a small, concise implementation.
What do if you need to work with two classes at one to test something?
Then your classes are probably coupled too tightly.
This class and method organization is also encourages you to use so-
so names for your tests at best. If you have a Calendar class and
a TestCalendar, that’s pretty much saying the same thing twice (a
violation of DRY) and it doesn’t tell you much. BDD encourages you
to actually express what you are testing with your naming. This
really helps you focus on the goal of the process and is much less
arbitrary.
Mapping tests to classes 1:1 is a sign of proper design. Your
implementation’s names should reflect the behavior implemented
within. If you have to give different names to your tests and your
implementation you’ve probably named your implementation poorly.
Instead you should refactor your implementation so that it can be
named well.
Read the tutorial on the rspec site:
http://rspec.rubyforge.org/tutorials/index.html
The specifications match up with the implementation very well, so the
Stack is probably well-designed. If they didn’t, that’s a good sign
of code smell.
(You get the same code smell out of unit testing, it just smells a
little different.)
–
Eric H. - [email protected] - http://blog.segment7.net
I LIT YOUR GEM ON FIRE!
On 1/23/07, Eric H. [email protected] wrote:
On Jan 23, 2007, at 16:26, James Edward G. II wrote:
On Jan 23, 2007, at 6:10 PM, Joe Van D. wrote:
What’s wrong with zentest, specifically?
You will probably understand better by watching the movie, but
zentest encourages you to make a test case for each class and a
test method for each method.Which will match up 1:1 if you factor your code well.
The point is that you shouldn’t try to shoehorn your tests into this
1:1 approach. You brought up the tutorial as an example of specs that
match up nicely with the code, but even that ends up with a few
contexts. If you just have a StackTest class, you won’t get the same
logical separation. How many programmers are going to write an
EmpyStackTest class, a OneItemStackTest class, a FullStackTest class,
etc?
As your behavior becomes more complex - by interacting with other
objects - you need to be able to drill a bit deeper with your
specifications. I often have one complete context for an individual
method. Not many programmers will create a new test class just to
spec a method. And that doesn’t mean that this method is too tightly
coupled to other objects, it just means that you need more
finely-grained specifications.
BDD presents a method of thinking about test design to help you focus
on better object design and object modeling. Unit testing doesn’t
have this method of thinking built-in, you have to discover it.
The bottom line is that if you’re using either Test::Unit or RSpec the
most effectively, you’re probably doing things nearly the same way
that you would with the other framework. The difference is that RSpec
guides you to the proper way. Unit testing doesn’t have the proper
thinking built-in, as you said. The BDD guys give us a testing tool
with some of the lessons they’ve learned over the years built-in.
It’s just a different path to the same place.
Pat
Eric H. wrote:
end
assert_equal 1, @s.top
endend
Each one of these test methods invokes multiple methods on Stack. You’re
not testing a method, you’re testing the interaction between several
methods – i.o.w. a call sequence. Looking at the contents of each, it’s
not clear that one method really is prominent – i.e. is test_push
testing #push, #empty?, or #top ?
Question – what would you name:
def test_???
@s.push 1
@s.pop
assert @s.empty?
end
or would you merge it in with test_empty_eh ?
Devin
On Jan 23, 2007, at 17:36, Pat M. wrote:
The point is that you shouldn’t try to shoehorn your tests into this
1:1 approach.
Shoehorning is design smell, so I refactor. The 1:1 falls out
because of good design. Its not something I aim for, its something I
refactor to.
You brought up the tutorial as an example of specs that
match up nicely with the code, but even that ends up with a few
contexts. If you just have a StackTest class, you won’t get the same
logical separation. How many programmers are going to write an
EmpyStackTest class, a OneItemStackTest class, a FullStackTest class,
etc?
BDD and TDD are not 1:1 in this manner, so you’ve overdesigned your
unit tests.
Instead, do the simplest thing that works:
require ‘rubygems’
require ‘test/unit’
require ‘test/zentest_assertions’
class TestStack < Test::Unit::TestCase
def setup
@s = Stack.new
end
def test_empty_eh
assert @s.empty?
@s.push 1
deny @s.empty?
end
def test_push
@s.push 1
deny @s.empty?
assert_equal 1, @s.top
end
def test_top
assert_nil @s.top
@s.push 1
assert_equal 1, @s.top
end
end
As your behavior becomes more complex - by interacting with other
objects - you need to be able to drill a bit deeper with your
specifications. I often have one complete context for an individual
method. Not many programmers will create a new test class just to
spec a method. And that doesn’t mean that this method is too tightly
coupled to other objects, it just means that you need more
finely-grained specifications.
This is where I create new methods or new classes on the
implementation side. As the number of edgecases increase I pull out
bits that are resistant to testing by refactoring. Creating multiple
test classes to test one implementation class is design smell.
–
Eric H. - [email protected] - http://blog.segment7.net
I LIT YOUR GEM ON FIRE!
On 1/23/07, Eric H. [email protected] wrote:
On Jan 23, 2007, at 16:26, James Edward G. II wrote:
On Jan 23, 2007, at 6:10 PM, Joe Van D. wrote:
What’s wrong with zentest, specifically?
You will probably understand better by watching the movie, but
zentest encourages you to make a test case for each class and a
test method for each method.Which will match up 1:1 if you factor your code well.
Eric,
I realize that you might have been put on the defensive here, but the
suggestion that 1:1 is the only natural conclusion of well factored
code denies a world of experience.
In my own experience, the ratio of test cases to classes usually
starts off close to 1:1, but as a project evolves it naturally moves
away from that. If you see that a class is growing, for example, it is
common to start refactoring out methods into new classes as those
classes make themselves apparent.
Some people will add new tests for the new class as a matter of
course. If you do that, then you are testing the same code in two
different tests. Multiply that over a large system and you’re
increasing the time it takes your test suites to run exponentially.
Slower test suites get run less often, providing less timely feedback.
Also, 1:1 mapping means that every time you want to refactor you are
burdened with refactoring your tests. 1:1 as a policy has a tendency
to push people towards doing less refactoring and leaving code in a
state that less well factored than they might envision.
Others are satisfied that the new class is already tested through the
classes that are using it. The risk they run is that tests get further
and further away from the code they are testing, and problems become
progressively difficult to isolate.
Which is right? Neither 100% of the time. The skill is in finding the
right balance in each different situation and understanding how to
make good decisions about when new tests resulting from refactoring is
a good idea and when it’s not.
BDD teaches that these decisions are pretty arbitrary.
I don’t use BDD because I already have my tests broken up by units of
behavior. I call them methods.BDD presents a method of thinking about test design to help you focus
on better object design and object modeling. Unit testing doesn’t
have this method of thinking built-in, you have to discover it.
I’m not sure I understand the distinction you’re trying to make here.
TDD/BDD is all about discovery.
feedback about how well you’ve designed your program. Both are
equally important.
Agree.
If you want to write specifications to enumerate your edge cases,
great. However, if you’ve factored poorly neither BDD nor TDD will
give you small, concise tests nor a small, concise implementation.
Agree, though I do believe that writing the tests first helps to push
you in the right direction.
What do if you need to work with two classes at one to test something?
Then your classes are probably coupled too tightly.
Let’s say you have a Money class that handles things like rounding,
math, conversions, etc. This is all very well tested. Now you are
building an Account class that interacts with an instance of Money.
Would you mock that out in every Account test?
This class and method organization is also encourages you to use so-
so names for your tests at best. If you have a Calendar class and
a TestCalendar, that’s pretty much saying the same thing twice (a
violation of DRY) and it doesn’t tell you much. BDD encourages you
to actually express what you are testing with your naming. This
really helps you focus on the goal of the process and is much less
arbitrary.Mapping tests to classes 1:1 is a sign of proper design.
Again, I disagree. 1:1 mapping is a sign that you are good at
following policy, nothing more. If I write a giant class that has 50
methods with a single test case with 50 corresponding test methods
I’ve met 1:1, but I still probably have a shitty design.
If I have some other, looser ratio of tests/classes and every time I
need to add features to my app it takes 10 times what I expect it to,
I probably have a shitty design.
On the flip side, if I have a design that is easy for me or any of my
colleagues to extend as business requirements change, then I have a
good design. For that to happen there is probably good test coverage,
but I really don’t think that 1:1 mapping has much to do with it.
But that’s just me.
Cheers,
David
“David C.” [email protected] writes:
That said, you can include modules and get reuse that way:
module SomeHelperMethodsUsefulAcrossContexts
endcontext “some context” do
include SomeHelperMethodsUsefulAcrossContexts
endI’d be surprised if test/spec doesn’t already support this out of the box.
It works exactly the same in test/spec.
On Jan 23, 2007, at 21:05, David C. wrote:
I realize that you might have been put on the defensive here,
I don’t really care if people use zentest or autotest or unit_diff or
Test::Rails or not, but I will say that you’re missing out if you’re
not testing.
but the suggestion that 1:1 is the only natural conclusion of well
factored
code denies a world of experience.
I’m going to cut and ignore some of your points to respond to this,
which seems to be the core of your message. I think the parts I’ve
ignored are tangent to the central issue.
(If you want to see how I handle testing specific cases you can go
through tests in my various projects. You’ll see I don’t really do
1:1 testing, and some are barely tested at all!)
have this method of thinking built-in, you have to discover it.
I’m not sure I understand the distinction you’re trying to make here.
TDD/BDD is all about discovery.
I added back some context. I believe James’ assertion was that
ZenTest’s approach was different from BDD’s approach. They are the
same, but test/unit has less philosophy standing behind it, and less
guidance about how things should be done.
Generally, implementing a BDD specification would end up as a method
for me. This happens because I use feedback from both the tests and
the implementations to improve my code.
Again, I disagree. 1:1 mapping is a sign that you are good at
following policy, nothing more. If I write a giant class that has 50
methods with a single test case with 50 corresponding test methods
I’ve met 1:1, but I still probably have a shitty design.
Following policy is not listening to your code, which I touched on in
the context below (and hopefully enough above). I listen to my code,
and 50 methods tells me its screaming in pain.
If I have some other, looser ratio of tests/classes and every time I
need to add features to my app it takes 10 times what I expect it to,
I probably have a shitty design.On the flip side, if I have a design that is easy for me or any of my
colleagues to extend as business requirements change, then I have a
good design. For that to happen there is probably good test coverage,
but I really don’t think that 1:1 mapping has much to do with it.But that’s just me.
Following rules sucks, and honestly I really don’t do 1:1 the way you
present it (I probably misrepresented myself due to inexact
expression). It just falls out that way for me because of my style
and the way I listen to my code and my tests.
I don’t want to say, “YOU MUST TEST 1:1!” since that misses out on
half of the information you get from testing. I want to say
something like, “write your tests, listen to your code, listen to
your tests, then refactor and make it simpler. Then do it again and
again and again. Once you’re in the habit you’ll just have to the
first four.”, but even that doesn’t get what I want to convey across.
Oh, and any time your code does something unexpected, write a test.
I still feel that what you test (or specify) should translate very
closely to what you implement, unit to unit (or behavior to
behavior). For me this smells good, feels good, flows well, and is a
pleasure to work with. It took me a long time (at least three years)
to figure out what I was supposed to be doing, and I’ve still got
years to go to get it right. Most of it was from doing unpleasant
things and getting stuck in crappy code. (Basically, the easier it
is, the more I think I’m doing something right.)
We have these marvelous tools (be it test/unit or BDD-style) and it
takes a while to realize we’re probably using only half of what it
can tell us (the tests all pass, done!) and ignoring the half that
can make code beautiful. Not because of ignorance, but because the
information is conveyed in a way that’s unusual at first.
It took effort for me to look at code and say “it would be so much
cleaner if…”, “it would be so much simpler if…”, “I can remove
this whole class by…”.
Now I can look at something and say “oh, eww, I could just do this
and this falls off, then that falls off then …”.
I don’t know how to teach people to see it, but its there, if you
look and listen.
The specifications match up with the implementation very well, so the
Stack is probably well-designed
If they didn’t, that’s a good sign
of code smell.(You get the same code smell out of unit testing, it just smells a
little different.)
–
Eric H. - [email protected] - http://blog.segment7.net
I LIT YOUR GEM ON FIRE!
On 1/24/07, Eric H. [email protected] wrote:
Which will match up 1:1 if you factor your code well.
TDD/BDD is all about discovery.
This class and method organization is also encourages you to use
Again, I disagree. 1:1 mapping is a sign that you are good at
I probably have a shitty design.
expression). It just falls out that way for me because of my style
and the way I listen to my code and my tests.I don’t want to say, “YOU MUST TEST 1:1!” since that misses out on
half of the information you get from testing. I want to say
something like, "write your tests, listen to your code, listen to
your tests, then refactor and make it simpler. Then do it again and
again and again.
Hear, hear!
Once you’re in the habit you’ll just have to the
first four.", but even that doesn’t get what I want to convey across.Oh, and any time your code does something unexpected, write a test.
+1
I still feel that what you test (or specify) should translate very
closely to what you implement, unit to unit (or behavior to
behavior). For me this smells good, feels good, flows well, and is a
pleasure to work with. It took me a long time (at least three years)
to figure out what I was supposed to be doing, and I’ve still got
years to go to get it right. Most of it was from doing unpleasant
things and getting stuck in crappy code. (Basically, the easier it
is, the more I think I’m doing something right.)
I think that’s generally true, with the addition that the “easy”
measurement becomes more meaningful over time. If a month later it’s
easy for me to revisit a place in the code to add or change something,
then I must have been doing something right a month ago.
Not to belittle the flow issue. Flow, how it feels as you’re doing it,
is very, very important as well. I’ve just learned over time that, for
me, not everything that feels right today proves itself out in a month
or later. That said, most things that “feel wrong” today end up being
for a long time.
Now I can look at something and say “oh, eww, I could just do this
and this falls off, then that falls off then …”.I don’t know how to teach people to see it, but its there, if you
look and listen.
Eric,
Thanks for taking the time for such a thoughtful and thorough reply. I
was definitely reacting to two statements you had made:
“Which will match up 1:1 if you factor your code well.”
“Mapping tests to classes 1:1 is a sign of proper design.”
Together, these seemed to advocate a position that 1:1 is good,
anything else is bad. I can see clearly from your response that I
misread your intent, and that we are primarily in agreement.
The overriding message (which I hope I’m reading correctly this time)
with which I agree wholeheartedly is that tools are exactly that:
tools. They can make some things easier. They can make things more
fun. They can shed light on the things you’re interested in lighting.
But they can’t make you produce good code. Only time and experience
and diligent attention to the code itself can get you there.
Cheers,
David
“David C.” [email protected] writes:
end
I’d be surprised if test/spec doesn’t already support this out of the box.
It works exactly the same in test/spec.
Glad to hear it. Do you have any feeling about which approach to reuse
you personally prefer? Nested contexts or included modules?
As said, nesting is purely for organizational and namespacing purposes.
I use modules.
On 1/24/07, Christian N. [email protected] wrote:
I’d be surprised if test/spec doesn’t already support this out of the box.
It works exactly the same in test/spec.
Glad to hear it. Do you have any feeling about which approach to reuse
you personally prefer? Nested contexts or included modules?
On Wed, 24 Jan 2007 22:05:17 +0900
“David C.” [email protected] wrote:
end
I’d be surprised if test/spec doesn’t already support this out of the box.
It works exactly the same in test/spec.
Glad to hear it. Do you have any feeling about which approach to reuse
you personally prefer? Nested contexts or included modules?
I know I’m not the person that this was targeted at, but I would
absolutely
love to see nested contexts included in rspec… I was just wishing I
could
do that yesterday.
On 1/24/07, Christian N. [email protected] wrote:
context “some context” do
As said, nesting is purely for organizational and namespacing purposes.
I use modules.
Interesting. I thought it was for context state sharing like this:
context “a User” do
setup do
user = User.new
end
context “with a valid email address” do
setup do
user.email = “[email protected]”
end
…
end
context “with an invalid email address” do
setup do
user.email = “[email protected]”
end
…
end
end
This leads us quickly down a path in which you have to look up a chain
of several setup methods to understand failures.
If it’s just a matter of namespacing for reporting/organization that’s
kind of interesting.
David