Ruby Forum Ruby > I don't get rspec

I just discovered rspec today, reading the example and tutorials,
besides the doc and coverage features, I can't see how is it different
from unit test. For my rails app, existing testing tools in rails
already fills all my testing needs. So what does rspec do more than
existing testing tools? Thanks.

http://rspec.rubyforge.org/tools/index.html

nicknameoptional wrote:
> I just discovered rspec today, reading the example and tutorials,
> besides the doc and coverage features, I can't see how is it different
> from unit test. For my rails app, existing testing tools in rails
> already fills all my testing needs. So what does rspec do more than
> existing testing tools? Thanks.
> 
> http://rspec.rubyforge.org/tools/index.html
> 

There is a  good Google video about rspec and Behavior Driven
Development by the very well-dressed Dave Astels:

http://video.google.com/videoplay?docid=8135690990081075324&pl=true



--
James Britt

"A language that doesn't affect the way you think about programming is
not worth knowing."
   - A. Perlis

> There is a  good Google video about rspec and Behavior Driven
> Development by the very well-dressed Dave Astels:
>
> http://video.google.com/videoplay?docid=8135690990081075324&pl=true
>
> --
> James Britt
>

I still fail to see the value added by rSpec. In the video, one main
point I got was BDD try to break the 1-1 relation of class and test
class, which he said is useful if refactoring code. but writing rails
app, with convention over configuration ideology, I don't really do
that much refactoring.

and if you look at the example,
"@stack.should_be_empty" is mapped to stack.empty? method,
@stack.top.should_equal "one item" is stack.top == "one item"

besides more English like, how is it better than the assert_equal?

nicknameoptional wrote:
> besides more English like, how is it better than the assert_equal?

It's not. If you're doing TDD right, then BDD is just what you're doing
already, by another name.

The problem is a lot of people don't do TDD right, and the semantics of
it raise the barrier to entry.

BDD is about trying to change those semantics, so TDD
As-It-Was-Meant-To-Be becomes more popular. It's just a bonus that
rSpec reads so nicely and prints out nice looking reports.

So I was in the same boat as you. Then I decided: "You know, they have
a point about people misinterpreting TDD. So is trying to change the
language used to describe it really such an awful thing?"

And that's where I am now.

Is using rSpec in your projects going to increase velocity by 10%? Or
make it easier to communicate with business people? Or allow you to
write better tests? Have fewer bugs? Any of these things? No. Not if
you're already doing TDD well. It might help more developers to get
there though. It might help new team-members integrate faster with less
training on TDD and the way you're doing things. It might just give you
a warm fuzzy. Compared to a lot of the arbitrary and complex choices we
sometimes make, you could do a lot worse I think.

nicknameoptional wrote the following on 20.01.2007 01:20 :
> I still fail to see the value added by rSpec. In the video, one main
>   
assert_equal is ambiguous and as said in the video, many mismatch the
parameters (I've done it several times myself).
@object.method.should_equal is less prone to errors.

This specific comment aside, from what I understood from the video the
main benefit is that it helps those who don't really get the test
process with TDD not being confused by the naming used in Test::Unit. It
keeps them in the right set of mind: "How should objects in some
contexts behave?" and not "what equality/difference/basic property/...
should I test?". This single change of context doesn't seem so helpful
at first glance, but in practice I've seen numerous developers
(including myself) being drowned in tests, adding new useless or
unimportant ones because they lost focus on what they should test:
correct behaviour of the application and its components. So I believe
this approach can help ("the language shapes the thought" as mentioned
in the video is probably true from my point of view).

A good point is that it definitely enhances test readibility too.

Lionel.

> It keeps them in the right set of mind: "How should objects in some
> contexts behave?" and not "what equality/difference/basic property/...
> should I test?".

Each context in rSpec is just a new testcase. For example,
EmptyStackTest's setup() can be written "@stack = Stack.new",
FullStackTest's steup() can be written "@stack = Stack.new;
@stack.fillup...."

so a rSpec file with 5 contexts is same as 5 unit testcase files,
although I almost always only write one testcase per class. if
assert_equal is ambiguous, you always can use assert(object.attr [== |
equal | eql] "expected")

>From my point of view, same testing tasks done in rSpec is not much
easier than using existing tools (Test/unit, rails testing tools).

On Jan 19, 2007, at 6:55 PM, Sam Smoot wrote:

> The problem is a lot of people don't do TDD right, and the  
> semantics of
> it raise the barrier to entry.

Yes.  This has always bothered me about the zentest tool.  I love
many of the tools in that package, but the zentest tool seems to
encourage bad testing habits.  The video linked to earlier in the
thread explains why.

James Edward Gray II

It is worth noting that rSpec is not the only player in the field of 
behavior testing. It simply goes a bit further providing the whole 
framework in place of Test/Unit (it reads much better, for one). I use 
Mocha with Test/Unit for behavior/unit testing an is quite happy with 
the combo. I saw it mentioned somewhere that Mocha comes by default in 
RoR installations, so they have means for behavior testing already ;-).

Gennady.

nicknameoptional wrote:
> I just discovered rspec today, reading the example and tutorials,
> besides the doc and coverage features, I can't see how is it different
> from unit test. For my rails app, existing testing tools in rails
> already fills all my testing needs. So what does rspec do more than
> existing testing tools? Thanks.
>
> http://rspec.rubyforge.org/tools/index.html
>   
Well ... I don't "get" Test::Unit. It just seems so complicated and
intricate. Rspec is a whole lot simpler to use, and it integrates some
good tools, like Rcov. I think it's the perfect way to learn TDD/BDD.

I do have a bone to pick with the tutorial (or maybe I don't understand
TDD/BDD yet). In the stack example, I cringe with horror when I see them
setting a method that just returns "true" simply to get a test to pass.
That grates on me, because I *know* I'm actually going to have to write
the real code eventually. It seems like a waste of time to do the
nitty-gritty "add a test to the spec/add a trivial fix to the code" 
cycle.

Part of that may be the algorithm I'm implementing. It's very well
defined already -- not something I have to "figure out" given a set of
behaviors. But the same is true of a stack, which is the example in the
tutorial. Everybody knows what a stack is supposed to do, everybody
knows what the output from sorting an array of strings is supposed to
be, etc. I think a better example would be something where you didn't
"instantly" know what the code should look like -- a case where you
really had some *behavior* but no code when you started developent, and
had to "create" the objects and methods and classes from the behavior.

--
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blogspot.com/

If God had meant for carrots to be eaten cooked, He would have given 
rabbits fire.

M. Edward (Ed) Borasky wrote:
> I do have a bone to pick with the tutorial (or maybe I don't understand 
> TDD/BDD yet). In the stack example, I cringe with horror when I see them 
> setting a method that just returns "true" simply to get a test to pass. 
> That grates on me, because I *know* I'm actually going to have to write 
> the real code eventually. It seems like a waste of time to do the 
> nitty-gritty "add a test to the spec/add a trivial fix to the code" cycle.

It seems to be part of Kent Beck's definition of TDD. AFAICT, he excuses
it away as, "well, this bogus 'true' method will be fixed when we get to
the refactor stage -- duplication between the 'true' in the code and the
'true' in the test," but I don't really buy that.

Not all of the cases of cheating are going to be duplication -- say,
you're supposed to be implementing the gamma function, but, starting off
only testing natural numbers, you just implement fac first. AFAIK,
implementing factorial is not going to help you implement gamma.
Likewise, saying 'def size; 0 end' may make you feel better, but it's
hardly progress towards an actual implementation of #size.

In any case, if the point is to keep the cycles short and to encourage
the tests to be written, I'd at least like to inject a bit of
pragmatism. We all get interrupted, have short memories, etc. -- at the
very least, the stub implementation should be:
   def empty?
     true # XXX bogus code -- actual implementation pending tests
   end

(Remember, imaginary folks who are the audience of this rant: Bad code
should *look* bad.)

Fondly,
Devin

On 1/19/07, M. Edward (Ed) Borasky <znmeb@cesmail.net> wrote:
> I do have a bone to pick with the tutorial (or maybe I don't understand
> TDD/BDD yet). In the stack example, I cringe with horror when I see them
> setting a method that just returns "true" simply to get a test to pass.
> That grates on me, because I *know* I'm actually going to have to write
> the real code eventually. It seems like a waste of time to do the
> nitty-gritty "add a test to the spec/add a trivial fix to the code" cycle.

First of all, it's just a simple example for people brand new to
RSpec.  It's good to take baby steps.  As you do it more you'll
obviously combine a couple steps into one.  But if you get stuck you
can always take a few steps back and then do the baby steps.

Now for the more important theoretical point...

There are two key parts when doing TDD/BDD.  The first is writing a
spec that fails.  The second is getting it passing.  I do that every
single time, no matter how trivial the code is. After you've been
doing BDD for a while you'll probably write a spec and then write the
code, run the spec, and move on.  That'll work really well for you for
a long time.  Then one day you're going to do that and your spec will
pass, but some part of the behavior still doesn't seem right.  You
spend a lot of time hunting this bug down, and finally you get
frustrated and tear out the new code to start from square one.  And
you run your spec...and it's green.  What the hell?  You just yanked
out your code.  Then you realize that you wrote the wrong spec, and
vow never to skip that trivial red step again.

Pat

On 1/20/07, M. Edward (Ed) Borasky <znmeb@cesmail.net> wrote:
> Well ... I don't "get" Test::Unit. It just seems so complicated and
> intricate. Rspec is a whole lot simpler to use, and it integrates some
> good tools, like Rcov. I think it's the perfect way to learn TDD/BDD.
>
> I do have a bone to pick with the tutorial (or maybe I don't understand
> TDD/BDD yet). In the stack example, I cringe with horror when I see them
> setting a method that just returns "true" simply to get a test to pass.
> That grates on me, because I *know* I'm actually going to have to write
> the real code eventually.


 That is very interesting, I just recently did the same, set up a method 
in
my testee

      def method

      end

and one more test passed I felt like a hero.
Now, maybe I should have felt like a Zero I think this question has a 
lot to
do with abstraction
and I try very hard to abstract. It kind of forces your mind *not to 
think
about implementation*.
So that is the setup I am pretty sure, if it is always the right thing 
to
do? Of course not!

It seems like a waste of time to do the
> had to "create" the objects and methods and classes from the behavior.
Actually that is a really good point it is almost impossible to see a 
stack
abstractly!
  Stacks should be forbidden in tutorials anyway :)

--
> M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
> http://borasky-research.blogspot.com/
>
> If God had meant for carrots to be eaten cooked, He would have given
> rabbits fire.
>
>
> Cheers
Robert

On 1/20/07, Devin Mullins <twifkak@comcast.net> wrote:
> the refactor stage -- duplication between the 'true' in the code and the
> 'true' in the test," but I don't really buy that.
>
> Not all of the cases of cheating are going to be duplication -- say,
> you're supposed to be implementing the gamma function, but, starting off
> only testing natural numbers, you just implement fac first. AFAIK,
> implementing factorial is not going to help you implement gamma.
> Likewise, saying 'def size; 0 end' may make you feel better, but it's
> hardly progress towards an actual implementation of #size.

There's a point that Kent makes I'm not seeing in your description.
Hard coding the return value gets you right to a green bar, which is a
very important part of refactoring. As you refactor towards the real
implementation, the passing tests are an invariant that helps you to
know that you're not doing things that adversely affect not only the
method that you are working on, but everything else that you've
already written.

The value of this is difficult to see in the trivial examples we see
in books, but as an application grows it becomes increasingly
important.

> In any case, if the point is to keep the cycles short and to encourage
> the tests to be written, I'd at least like to inject a bit of
> pragmatism. We all get interrupted, have short memories, etc. -- at the
> very least, the stub implementation should be:
>    def empty?
>      true # XXX bogus code -- actual implementation pending tests
>    end

If that helps you to feel more confidence so that you can progress,
then do it. That's an important point of TDD. To provide YOU with
confidence that you're doing the right thing.

Cheers,
David

On 1/19/07, nicknameoptional <dorrenchen@gmail.com> wrote:
> I just discovered rspec today, reading the example and tutorials,
> besides the doc and coverage features, I can't see how is it different
> from unit test. For my rails app, existing testing tools in rails
> already fills all my testing needs. So what does rspec do more than
> existing testing tools? Thanks.

Disclaimer: I am the lead developer of rspec and freely admit whatever
bias that might bring.

RSpec is a tool which tries to make BDD easier. BDD is an approach to
development that tries to tackle some of the problems that development
teams (customers, developers and testers) run into doing TDD.

For me, BDD addressed some questions that I had at the time I first
heard about it, so I got involved. It has changed the way I approach
development regardless of platform or tools.

RSpec does not set out to do more from a testing perspective than
'test/unit' does. It just tries to help you see the whole process
through a slightly different lens.

If you're not interested in seeing it through a different lens, there
may be no value in it for you. I can tell you that I've heard from
some skeptics who have really given it a shot that it does change the
way they approach testing.

Cheers,
David

On 1/19/07, nicknameoptional <dorrenchen@gmail.com> wrote:
> point I got was BDD try to break the 1-1 relation of class and test
> class, which he said is useful if refactoring code. but writing rails
> app, with convention over configuration ideology, I don't really do
> that much refactoring.

You may not now, but if you have to maintain your app over a long
period of time, with changing requirements, you will likely have to
some restructuring of some kind.

This is an interesting thing about Rails in general right now. It's
still in its infancy and there aren't too many companies besides 37
signals that have had to maintain Rails apps over several years. I
think as that happens that we'll see more and more interest in
refactoring strategies and tools.

>
> and if you look at the example,
> "@stack.should_be_empty" is mapped to stack.empty? method,
> @stack.top.should_equal "one item" is stack.top == "one item"
>
> besides more English like, how is it better than the assert_equal?

It clarifies which comes first, the expected value or the actual value:

assert_equal(5, result) ???
assert_equal(result, 5) ???
result.should == 5

That's pretty easy to get right.

On 1/20/07, M. Edward (Ed) Borasky <znmeb@cesmail.net> wrote:
> intricate. Rspec is a whole lot simpler to use, and it integrates some
> good tools, like Rcov. I think it's the perfect way to learn TDD/BDD.
>
> I do have a bone to pick with the tutorial (or maybe I don't understand
> TDD/BDD yet). In the stack example, I cringe with horror when I see them
> setting a method that just returns "true" simply to get a test to pass.

This dates back to the beginnings of TDD. I just wrote about this in
another thread, but for completeness' sake here: when you're doing TDD
as prescribed, you run all of your unit tests very often. The fact
that they all pass gives you confidence to move quickly rather than
stop and wonder what you might be affecting that's not immediately in
your view. If you make a change that introduces a problem and run the
tests right away, you discover the problem right away and don't have
to start debugging the work you've done over the last
minutes/hours/days.

> That grates on me, because I *know* I'm actually going to have to write
> the real code eventually. It seems like a waste of time to do the
> nitty-gritty "add a test to the spec/add a trivial fix to the code" cycle.

The triviality of the fix is the beauty of the whole thing. Every
problem gets broken down into really tiny bite size problems. You
don't have to hold up a delicate house of cards in your head that
falls down every time the phone rings.

>
> Part of that may be the algorithm I'm implementing. It's very well
> defined already -- not something I have to "figure out" given a set of
> behaviors. But the same is true of a stack, which is the example in the
> tutorial. Everybody knows what a stack is supposed to do, everybody
> knows what the output from sorting an array of strings is supposed to
> be, etc. I think a better example would be something where you didn't
> "instantly" know what the code should look like -- a case where you
> really had some *behavior* but no code when you started developent, and
> had to "create" the objects and methods and classes from the behavior.

I agree. I'll add a new tutorial to the rspec website sometime soon.

David

"nicknameoptional" <dorrenchen@gmail.com> writes:

> I just discovered rspec today, reading the example and tutorials,
> besides the doc and coverage features, I can't see how is it different
> from unit test. For my rails app, existing testing tools in rails
> already fills all my testing needs. So what does rspec do more than
> existing testing tools? Thanks.

I don't know about you, but since I use test/spec[1] (my Test::Unit
based variant of RSpec), I enjoy writing tests and doing test-first a
lot more.  I didn't think it would make such a big difference, but it
really does.

[1]: 
http://chneukirchen.org/blog/archive/2006/10/announcing-test-spec-0-2-a-bdd-interface-for-test-unit.html

(I hope to release test/spec 0.3 this week... speak up now if you wish
for anything.)

Hi Ed,

On 1/19/07, M. Edward (Ed) Borasky <znmeb@cesmail.net> wrote:
> I do have a bone to pick with the tutorial (or maybe I don't understand
> TDD/BDD yet). In the stack example, I cringe with horror when I see them
> setting a method that just returns "true" simply to get a test to pass.
> That grates on me, because I *know* I'm actually going to have to write
> the real code eventually. It seems like a waste of time to do the
> nitty-gritty "add a test to the spec/add a trivial fix to the code" cycle.

I don't look at this as "add a test to the spec/add a trivial fix to
the code" but instead as "add a test to the spec/do the simplest thing
that could possibly work to get the tests to pass", and I find
substantial value in this approach.

A few weeks ago I was doing BDD using RSpec and Francis Hwang's MockFS
library (which mocks the file system).  I needed to add a method to
MockFS to mock File#size.  So I added a spec that called File#size,
and ran it.  As expected, I got some error about File#size not found.
I looked through the MockFS code to see where this error came from,
and I saw that MockFS had an array of symbols which were the names of
the methods that it mocked.  :size wasn't in this array, which caused
the error.  Looking at the MockFS code, I *knew* that I needed to do
two things: add :size to the array, and also implement the method that
mocked File#size.  I remember being strongly tempted to do both these
changes before re-running the spec.  But since I got the error because
:size wasn't in the array, simply adding :size to the array was the
simplest thing that could possibly work.  So I did that, and re-ran
the spec, *knowing* that I was going to get a NoMethodError.  But
instead, ALL MY TESTS PASSED.   I was quite surprised.  Investigating
what had happened, I discovered that Francis implements the mock files
as StringIO objects, so StringIO#size got called, which returned the
size of the string/file, which was correct, so all the tests passed.
(Duck Typing in action).

If I hadn't forced myself to try the simplest thing that could
possibly work, I would have had to implement (and debug) a File#size
mock method.  This would have added unnecessary complexity to the
system.  Worse, my experience is that once code gets implemented and
debugged, it rarely gets removed.  So that code probably would have to
be maintained forever.

By forcing myself to always do the simplest thing that could possibly
work, I find that every day or two I come across a situation just like
this one, where a solution that I would have gladly bet serious money
that it was too simple to really work, actually does work.  In
addition, every hour or two I find a solution that I thought might be
too simple to really work, that does work.  Before I was doing BDD/TDD
I very rarely came up with simple solutions like this.  The cumulative
effect is that my code is noticeably simpler and cleaner, and I find
this to be one of the major benefits I get from BDD/TDD.

Wayne

---
Wayne Vucenic
No Bugs Software
Ruby, C# and Erlang Agile Contract Programming in Silicon Valley

Christian Neukirchen wrote:
> lot more.  I didn't think it would make such a big difference, but it
> really does.
> 
> [1]: http://chneukirchen.org/blog/archive/2006/10/announcing-test-spec-0-2-a-bdd-interface-for-test-unit.html
> 
> (I hope to release test/spec 0.3 this week... speak up now if you wish
> for anything.)

I like the continuity with test/unit. I'd be more likely to use this
than rspec for my older projects.

Questions (from a BDD newbie):

Are the context and specify argument strings purely documentation, or do
they need to be class or method names?

Is it possible to nest contexts? Is this a good idea?

Inside of a context, is there a setup/teardown protocol as there is in
test/unit?

I saw Dave Astels talk about this, at Canada on Rails, and initially I
thought it was a bunch of BS. It seemed like the only difference
between BDD and TDD was the words; everything else was just the same
thing with new names. Like the only development involved was a bunch
of alias statements. But I was totally wrong. What you're really doing
is a more intuitive, smoother version of TDD which produces
documentation as its byproduct. TDD isn't really about testing -- it's
about design. BDD makes it easier to use TDD for what it's really for.

Joel VanderWerf <vjoel@path.berkeley.edu> writes:

> I like the continuity with test/unit. I'd be more likely to use this
> than rspec for my older projects.
>
> Questions (from a BDD newbie):
>
> Are the context and specify argument strings purely documentation, or
> do they need to be class or method names?

They can contain anything except NUL bytes (the extended symbol
syntax).  They are turned into class and method names automatically
(we need some special formatting for that due to Test::Unit
restrictions).

> Is it possible to nest contexts? Is this a good idea?

test/spec allows them since 0.2, and I use them to group contexts that
go together.  Note that nothing is inherited, it's purely namespacing.
RSpec doesn't have them, and they aren't interesting in implementing
them AFAICT.

> Inside of a context, is there a setup/teardown protocol as there is in
> test/unit?

Yes, you can use

    context "Blah" do
      setup do
        @blah = Blah.new
      end

      teardown do
        @blah.destroy!
      end
    end

You can have multiple of these, and they are run in order of
appearance.

Giles Bowkett wrote:
> I saw Dave Astels talk about this, at Canada on Rails, and initially I
> thought it was a bunch of BS. It seemed like the only difference
> between BDD and TDD was the words; everything else was just the same
> thing with new names. 

It's easy for folk to dismiss RSpec by saying that it's just semantics 
... but when you are dealing with meaning, semantics is everything.

> TDD isn't really about testing -- it's
> about design. BDD makes it easier to use TDD for what it's really for.

Heh, you nailed it!

--
-- Jim Weirich

On 1/23/07, Christian Neukirchen <chneukirchen@gmail.com> wrote:
> They can contain anything except NUL bytes (the extended symbol
> syntax).  They are turned into class and method names automatically
> (we need some special formatting for that due to Test::Unit
> restrictions).
>
> > Is it possible to nest contexts? Is this a good idea?
>
> test/spec allows them since 0.2, and I use them to group contexts that
> go together.  Note that nothing is inherited, it's purely namespacing.
> RSpec doesn't have them, and they aren't interesting in implementing
> them AFAICT.

That's correct. We've really struggled with this because users want a
means of sharing context material, but we believe that nesting
contexts is going to lead to specs that are confusing to look at. We
believe that clarity supercedes DRY in specs, and so we haven't
supported this yet.

That said, you can include modules and get reuse that way:

module SomeHelperMethodsUsefulAcrossContexts
end

context "some context" do
  include SomeHelperMethodsUsefulAcrossContexts
end

I'd be surprised if test/spec doesn't already support this out of the 
box.

Cheers,
David

Is there an equivalent to autotest for rspec (autospec?)? I've googled
around, but only came across a Rails plugin, which (perhaps obviously)
is very Rails-centric.

-P

Wayne Vucenic wrote:
> I don't look at this as "add a test to the spec/add a trivial fix to
> the code" but instead as "add a test to the spec/do the simplest thing
> that could possibly work to get the tests to pass", and I find
> substantial value in this approach.

When I do TDD, I approach it with a Jeckle and Hyde perspective.  When I 
write a test, I am writing a specification of how the code should work.

When I start writing code, I become the laziest programmer you can 
imagine, writing only enough code to pass the specification as it is 
written at that moment.  If it is not called out in the spec, it doesn't 
get written.

When I switch back to test writing, I cuss out that lazy programmer 
(i.e. me just a few moments ago) and write additional specs to force the 
code to do exactly what I want.

The tension between Dr. Jeckle (the spec writer), and Mr. Hyde (the 
programmer) causes Dr. Jeckle to write better specs and Mr. Hyde to 
write lean, correct code.

--
-- Jim Weirich

On 1/23/07, Jim Weirich <jim@weirichhouse.org> wrote:
> imagine, writing only enough code to pass the specification as it is
> written at that moment.  If it is not called out in the spec, it doesn't
> get written.
>
> When I switch back to test writing, I cuss out that lazy programmer
> (i.e. me just a few moments ago) and write additional specs to force the
> code to do exactly what I want.
>
> The tension between Dr. Jeckle (the spec writer), and Mr. Hyde (the
> programmer) causes Dr. Jeckle to write better specs and Mr. Hyde to
> write lean, correct code.

With me it's more like Abbott and Costello!

Seriously, thanks for describing this so well.

Cheers,
David

On 1/19/07, James Edward Gray II <james@grayproductions.net> wrote:
> On Jan 19, 2007, at 6:55 PM, Sam Smoot wrote:
>
> > The problem is a lot of people don't do TDD right, and the
> > semantics of
> > it raise the barrier to entry.
>
> Yes.  This has always bothered me about the zentest tool.  I love
> many of the tools in that package, but the zentest tool seems to
> encourage bad testing habits.  The video linked to earlier in the
> thread explains why.

What's wrong with zentest, specifically?

Thanks,
Joe

On Jan 23, 2007, at 6:10 PM, Joe Van Dyk wrote:

>> thread explains why.
>
> What's wrong with zentest, specifically?

You will probably understand better by watching the movie, but
zentest encourages you to make a test case for each class and a test
method for each method.  BDD teaches that these decisions are pretty
arbitrary.

The true goal of testing is to test the "behaviors" of your object.
If that means interacting with five methods or just one possible call
sequence of a method that has several, that's what you should really
be testing.  What do if you need to work with two classes at one to
test something?

This class and method organization is also encourages you to use so-
so names for your tests at best.  If you have a Calendar class and a
TestCalendar, that's pretty much saying the same thing twice (a
violation of DRY) and it doesn't tell you much.  BDD encourages you
to actually express what you are testing with your naming.  This
really helps you focus on the goal of the process and is much less
arbitrary.

There you have it, in my own words.  Just to be clear, the above is
my opinion through and through.  I do not speak for the zentest or
BDD teams.

James Edward Gray II

On Jan 23, 2007, at 16:26, James Edward Gray II wrote:
> On Jan 23, 2007, at 6:10 PM, Joe Van Dyk wrote:
>> What's wrong with zentest, specifically?
>
> You will probably understand better by watching the movie, but  
> zentest encourages you to make a test case for each class and a  
> test method for each method.

Which will match up 1:1 if you factor your code well.

> BDD teaches that these decisions are pretty arbitrary.

I don't use BDD because I already have my tests broken up by units of
behavior.  I call them methods.

BDD presents a method of thinking about test design to help you focus
on better object design and object modeling.  Unit testing doesn't
have this method of thinking built-in, you have to discover it.
(Although it makes itself evident if you pay attention to how
difficult it is to test something.)

> The true goal of testing is to test the "behaviors" of your  
> object.  If that means interacting with five methods or just one  
> possible call sequence of a method that has several, that's what  
> you should really be testing.

I think there are two true goals of testing.  One is to enumerate
your edge cases so each one behaves as you expect.  Another is to get
feedback about how well you've designed your program.  Both are
equally important.

If you want to write specifications to enumerate your edge cases,
great.  However, if you've factored poorly neither BDD nor TDD will
give you small, concise tests nor a small, concise implementation.

> What do if you need to work with two classes at one to test something?

Then your classes are probably coupled too tightly.

> This class and method organization is also encourages you to use so- 
> so names for your tests at best.  If you have a Calendar class and  
> a TestCalendar, that's pretty much saying the same thing twice (a  
> violation of DRY) and it doesn't tell you much.  BDD encourages you  
> to actually express what you are testing with your naming.  This  
> really helps you focus on the goal of the process and is much less  
> arbitrary.

Mapping tests to classes 1:1 is a sign of proper design.  Your
implementation's names should reflect the behavior implemented
within.  If you have to give different names to your tests and your
implementation you've probably named your implementation poorly.
Instead you should refactor your implementation so that it can be
named well.

Read the tutorial on the rspec site:

http://rspec.rubyforge.org/tutorials/index.html

The specifications match up with the implementation very well, so the
Stack is probably well-designed.  If they didn't, that's a good sign
of code smell.

(You get the same code smell out of unit testing, it just smells a
little different.)

--
Eric Hodel - drbrain@segment7.net - http://blog.segment7.net

I LIT YOUR GEM ON FIRE!

On 1/23/07, Eric Hodel <drbrain@segment7.net> wrote:
> On Jan 23, 2007, at 16:26, James Edward Gray II wrote:
> > On Jan 23, 2007, at 6:10 PM, Joe Van Dyk wrote:
> >> What's wrong with zentest, specifically?
> >
> > You will probably understand better by watching the movie, but
> > zentest encourages you to make a test case for each class and a
> > test method for each method.
>
> Which will match up 1:1 if you factor your code well.

The point is that you shouldn't try to shoehorn your tests into this
1:1 approach.  You brought up the tutorial as an example of specs that
match up nicely with the code, but even that ends up with a few
contexts.  If you just have a StackTest class, you won't get the same
logical separation.  How many programmers are going to write an
EmpyStackTest class, a OneItemStackTest class,  a FullStackTest class,
etc?

As your behavior becomes more complex - by interacting with other
objects - you need to be able to drill a bit deeper with your
specifications.  I often have one complete context for an individual
method.  Not many programmers will create a new test class just to
spec a method.  And that doesn't mean that this method is too tightly
coupled to other objects, it just means that you need more
finely-grained specifications.

> BDD presents a method of thinking about test design to help you focus
> on better object design and object modeling.  Unit testing doesn't
> have this method of thinking built-in, you have to discover it.

The bottom line is that if you're using either Test::Unit or RSpec the
most effectively, you're probably doing things nearly the same way
that you would with the other framework.  The difference is that RSpec
guides you to the proper way.  Unit testing doesn't have the proper
thinking built-in, as you said.  The BDD guys give us a testing tool
with some of the lessons they've learned over the years built-in.
It's just a different path to the same place.

Pat

On Jan 23, 2007, at 17:36, Pat Maddox wrote:
>
> The point is that you shouldn't try to shoehorn your tests into this
> 1:1 approach.

Shoehorning is design smell, so I refactor.  The 1:1 falls out
because of good design.  Its not something I aim for, its something I
refactor to.

> You brought up the tutorial as an example of specs that
> match up nicely with the code, but even that ends up with a few
> contexts.  If you just have a StackTest class, you won't get the same
> logical separation.  How many programmers are going to write an
> EmpyStackTest class, a OneItemStackTest class,  a FullStackTest class,
> etc?

BDD and TDD are not 1:1 in this manner, so you've overdesigned your
unit tests.

Instead, do the simplest thing that works:

require 'rubygems'
require 'test/unit'
require 'test/zentest_assertions'

class TestStack < Test::Unit::TestCase

   def setup
     @s = Stack.new
   end

   def test_empty_eh
     assert @s.empty?
     @s.push 1
     deny @s.empty?
   end

   def test_push
     @s.push 1
     deny @s.empty?
     assert_equal 1, @s.top
   end

   def test_top
     assert_nil @s.top
     @s.push 1
     assert_equal 1, @s.top
   end

end

> As your behavior becomes more complex - by interacting with other
> objects - you need to be able to drill a bit deeper with your
> specifications.  I often have one complete context for an individual
> method.  Not many programmers will create a new test class just to
> spec a method.  And that doesn't mean that this method is too tightly
> coupled to other objects, it just means that you need more
> finely-grained specifications.

This is where I create new methods or new classes on the
implementation side.  As the number of edgecases increase I pull out
bits that are resistant to testing by refactoring.  Creating multiple
test classes to test one implementation class is design smell.

--
Eric Hodel - drbrain@segment7.net - http://blog.segment7.net

I LIT YOUR GEM ON FIRE!

Eric Hodel wrote:
>   end
>     assert_equal 1, @s.top
>   end
> 
> end

Each one of these test methods invokes multiple methods on Stack. You're
not testing a method, you're testing the interaction between several
methods -- i.o.w. a call sequence. Looking at the contents of each, it's
not clear that one method really is prominent -- i.e. is test_push
testing #push, #empty?, or #top ?

Question -- what would you name:

   def test_???
     @s.push 1
     @s.pop
     assert @s.empty?
   end

or would you merge it in with test_empty_eh ?

Devin

On 1/23/07, Eric Hodel <drbrain@segment7.net> wrote:
> On Jan 23, 2007, at 16:26, James Edward Gray II wrote:
> > On Jan 23, 2007, at 6:10 PM, Joe Van Dyk wrote:
> >> What's wrong with zentest, specifically?
> >
> > You will probably understand better by watching the movie, but
> > zentest encourages you to make a test case for each class and a
> > test method for each method.
>
> Which will match up 1:1 if you factor your code well.

Eric,

I realize that you might have been put on the defensive here, but the
suggestion that 1:1 is the only natural conclusion of well factored
code denies a world of experience.

In my own experience, the ratio of test cases to classes usually
starts off close to 1:1, but as a project evolves it naturally moves
away from that. If you see that a class is growing, for example, it is
common to start refactoring out methods into new classes as those
classes make themselves apparent.

Some people will add new tests for the new class as a matter of
course. If you do that, then you are testing the same code in two
different tests. Multiply that over a large system and you're
increasing the time it takes your test suites to run exponentially.
Slower test suites get run less often, providing less timely feedback.

Also, 1:1 mapping means that every time you want to refactor you are
burdened with refactoring your tests. 1:1 as a policy has a tendency
to push people towards doing less refactoring and leaving code in a
state that less well factored than they might envision.

Others are satisfied that the new class is already tested through the
classes that are using it. The risk they run is that tests get further
and further away from the code they are testing, and problems become
progressively difficult to isolate.

Which is right? Neither 100% of the time. The skill is in finding the
right balance in each different situation and understanding how to
make good decisions about when new tests resulting from refactoring is
a good idea and when it's not.

>
> > BDD teaches that these decisions are pretty arbitrary.
>
> I don't use BDD because I already have my tests broken up by units of
> behavior.  I call them methods.
>
> BDD presents a method of thinking about test design to help you focus
> on better object design and object modeling.  Unit testing doesn't
> have this method of thinking built-in, you have to discover it.

I'm not sure I understand the distinction you're trying to make here.
TDD/BDD is all about discovery.

> feedback about how well you've designed your program.  Both are
> equally important.

Agree.

> If you want to write specifications to enumerate your edge cases,
> great.  However, if you've factored poorly neither BDD nor TDD will
> give you small, concise tests nor a small, concise implementation.

Agree, though I do believe that writing the tests first helps to push
you in the right direction.

>
> > What do if you need to work with two classes at one to test something?
>
> Then your classes are probably coupled too tightly.

Let's say you have a Money class that handles things like rounding,
math, conversions, etc. This is all very well tested. Now you are
building an Account class that interacts with an instance of Money.
Would you mock that out in every Account test?

>
> > This class and method organization is also encourages you to use so-
> > so names for your tests at best.  If you have a Calendar class and
> > a TestCalendar, that's pretty much saying the same thing twice (a
> > violation of DRY) and it doesn't tell you much.  BDD encourages you
> > to actually express what you are testing with your naming.  This
> > really helps you focus on the goal of the process and is much less
> > arbitrary.
>
> Mapping tests to classes 1:1 is a sign of proper design.

Again, I disagree. 1:1 mapping is a sign that you are good at
following policy, nothing more. If I write a giant class that has 50
methods with a single test case with 50 corresponding test methods
I've met 1:1, but I still probably have a shitty design.

If I have some other, looser ratio of tests/classes and every time I
need to add features to my app it takes 10 times what I expect it to,
I probably have a shitty design.

On the flip side, if I have a design that is easy for me or any of my
colleagues to extend as business requirements change, then I have a
good design. For that to happen there is probably good test coverage,
but I really don't think that 1:1 mapping has much to do with it.

But that's just me.

Cheers,
David

On Jan 23, 2007, at 21:05, David Chelimsky wrote:
>
> I realize that you might have been put on the defensive here,

I don't really care if people use zentest or autotest or unit_diff or
Test::Rails or not, but I will say that you're missing out if you're
not testing.

> but the suggestion that 1:1 is the only natural conclusion of well  
> factored
> code denies a world of experience.

I'm going to cut and ignore some of your points to respond to this,
which seems to be the core of your message.  I think the parts I've
ignored are tangent to the central issue.

(If you want to see how I handle testing specific cases you can go
through tests in my various projects.  You'll see I don't really do
1:1 testing, and some are barely tested at all!)

>> have this method of thinking built-in, you have to discover it.
>
> I'm not sure I understand the distinction you're trying to make here.
> TDD/BDD is all about discovery.

I added back some context.  I believe James' assertion was that
ZenTest's approach was different from BDD's approach.  They are the
same, but test/unit has less philosophy standing behind it, and less
guidance about how things should be done.

Generally, implementing a BDD specification would end up as a method
for me.  This happens because I use feedback from both the tests and
the implementations to improve my code.

>
> Again, I disagree. 1:1 mapping is a sign that you are good at
> following policy, nothing more. If I write a giant class that has 50
> methods with a single test case with 50 corresponding test methods
> I've met 1:1, but I still probably have a shitty design.

Following policy is not listening to your code, which I touched on in
the context below (and hopefully enough above).  I listen to my code,
and 50 methods tells me its screaming in pain.

> If I have some other, looser ratio of tests/classes and every time I
> need to add features to my app it takes 10 times what I expect it to,
> I probably have a shitty design.
>
> On the flip side, if I have a design that is easy for me or any of my
> colleagues to extend as business requirements change, then I have a
> good design. For that to happen there is probably good test coverage,
> but I really don't think that 1:1 mapping has much to do with it.
>
> But that's just me.

Following rules sucks, and honestly I really don't do 1:1 the way you
present it (I probably misrepresented myself due to inexact
expression).  It just falls out that way for me because of my style
and the way I listen to my code and my tests.

I don't want to say, "YOU MUST TEST 1:1!" since that misses out on
half of the information you get from testing.  I want to say
something like, "write your tests, listen to your code, listen to
your tests, then refactor and make it simpler.  Then do it again and
again and again.  Once you're in the habit you'll just have to the
first four.", but even that doesn't get what I want to convey across.

Oh, and any time your code does something unexpected, write a test.

I still feel that what you test (or specify) should translate very
closely to what you implement, unit to unit (or behavior to
behavior).  For me this smells good, feels good, flows well, and is a
pleasure to work with.  It took me a long time (at least three years)
to figure out what I was supposed to be doing, and I've still got
years to go to get it right.  Most of it was from doing unpleasant
things and getting stuck in crappy code.  (Basically, the easier it
is, the more I think I'm doing something right.)

We have these marvelous tools (be it test/unit or BDD-style) and it
takes a while to realize we're probably using only half of what it
can tell us (the tests all pass, done!) and ignoring the half that
can make code beautiful.  Not because of ignorance, but because the
information is conveyed in a way that's unusual at first.

It took effort for me to look at code and say "it would be so much
cleaner if...", "it would be so much simpler if...", "I can remove
this whole class by...".

Now I can look at something and say "oh, eww, I could just do this
and this falls off, then that falls off then ...".

I don't know how to teach people to see it, but its there, if you
look and listen.

>>
>> The specifications match up with the implementation very well, so the
>> Stack is probably well-designed
>> If they didn't, that's a good sign
>> of code smell.
>>
>> (You get the same code smell out of unit testing, it just smells a
>> little different.)

--
Eric Hodel - drbrain@segment7.net - http://blog.segment7.net

I LIT YOUR GEM ON FIRE!

"David Chelimsky" <dchelimsky@gmail.com> writes:

> That said, you can include modules and get reuse that way:
>
> module SomeHelperMethodsUsefulAcrossContexts
> end
>
> context "some context" do
>  include SomeHelperMethodsUsefulAcrossContexts
> end
>
> I'd be surprised if test/spec doesn't already support this out of the box.

It works exactly the same in test/spec.

On 1/24/07, Eric Hodel <drbrain@segment7.net> wrote:
> >> Which will match up 1:1 if you factor your code well.
>
> >>
> > TDD/BDD is all about discovery.
> >> > This class and method organization is also encourages you to use
> > Again, I disagree. 1:1 mapping is a sign that you are good at
> > I probably have a shitty design.
> expression).  It just falls out that way for me because of my style
> and the way I listen to my code and my tests.
>
> I don't want to say, "YOU MUST TEST 1:1!" since that misses out on
> half of the information you get from testing.  I want to say
> something like, "write your tests, listen to your code, listen to
> your tests, then refactor and make it simpler.  Then do it again and
> again and again.

Hear, hear!

> Once you're in the habit you'll just have to the
> first four.", but even that doesn't get what I want to convey across.
>
> Oh, and any time your code does something unexpected, write a test.

+1

> I still feel that what you test (or specify) should translate very
> closely to what you implement, unit to unit (or behavior to
> behavior).  For me this smells good, feels good, flows well, and is a
> pleasure to work with.  It took me a long time (at least three years)
> to figure out what I was supposed to be doing, and I've still got
> years to go to get it right.  Most of it was from doing unpleasant
> things and getting stuck in crappy code.  (Basically, the easier it
> is, the more I think I'm doing something right.)

I think that's generally true, with the addition that the "easy"
measurement becomes more meaningful over time. If a month later it's
easy for me to revisit a place in the code to add or change something,
then I must have been doing something right a month ago.

Not to belittle the flow issue. Flow, how it feels as you're doing it,
is very, very important as well. I've just learned over time that, for
me, not everything that feels right today proves itself out in a month
or later. That said, most things that "feel wrong" today end up being
for a long time.

>
> Now I can look at something and say "oh, eww, I could just do this
> and this falls off, then that falls off then ...".
>
> I don't know how to teach people to see it, but its there, if you
> look and listen.

Eric,

Thanks for taking the time for such a thoughtful and thorough reply. I
was definitely reacting to two statements you had made:

"Which will match up 1:1 if you factor your code well."
"Mapping tests to classes 1:1 is a sign of proper design."

Together, these seemed to advocate a position that 1:1 is good,
anything else is bad. I can see clearly from your response that I
misread your intent, and that we are primarily in agreement.

The overriding message (which I hope I'm reading correctly this time)
with which I agree wholeheartedly is that tools are exactly that:
tools. They can make some things easier. They can make things more
fun. They can shed light on the things you're interested in lighting.

But they can't make you produce good code. Only time and experience
and diligent attention to the code itself can get you there.

Cheers,
David

On 1/24/07, Christian Neukirchen <chneukirchen@gmail.com> wrote:
> >
> > I'd be surprised if test/spec doesn't already support this out of the box.
>
> It works exactly the same in test/spec.

Glad to hear it. Do you have any feeling about which approach to reuse
you personally prefer? Nested contexts or included modules?

"David Chelimsky" <dchelimsky@gmail.com> writes:

>> > end
>> >
>> > I'd be surprised if test/spec doesn't already support this out of the box.
>>
>> It works exactly the same in test/spec.
>
> Glad to hear it. Do you have any feeling about which approach to reuse
> you personally prefer? Nested contexts or included modules?

As said, nesting is purely for organizational and namespacing purposes.
I use modules.

On Wed, 24 Jan 2007 22:05:17 +0900
"David Chelimsky" <dchelimsky@gmail.com> wrote:

> > > end
> > >
> > > I'd be surprised if test/spec doesn't already support this out of the box.
> >
> > It works exactly the same in test/spec.
> 
> Glad to hear it. Do you have any feeling about which approach to reuse
> you personally prefer? Nested contexts or included modules?
> 
I know I'm not the person that this was targeted at, but I would 
absolutely
love to see nested contexts included in rspec... I was just wishing I 
could
do that yesterday.

On 1/24/07, Christian Neukirchen <chneukirchen@gmail.com> wrote:
> >> > context "some context" do
> As said, nesting is purely for organizational and namespacing purposes.
> I use modules.

Interesting. I thought it was for context state sharing like this:

context "a User" do
  setup do
    user = User.new
  end

  context "with a valid email address" do
    setup do
      user.email = "a@b.com"
    end
    ...
  end

  context "with an invalid email address" do
    setup do
      user.email = "this@is.not.correctly.formatted"
    end
    ...
  end
end

This leads us quickly down a path in which you have to look up a chain
of several setup methods to understand failures.

If it's just a matter of namespacing for reporting/organization that's
kind of interesting.

David

On 1/24/07, Luke Ivers <technodolt@gmail.com> wrote:
> > > >
> >
> I know I'm not the person that this was targeted at, but I would absolutely
> love to see nested contexts included in rspec... I was just wishing I could
> do that yesterday.

Google this:

pipermail rspec nested contexts

and you'll see what's been said about it before. Feel free to
resurrect the discussion on the rspec-devel or rspec-users list if you
have some new insights to add to the discussion.

Cheers,
David

"David Chelimsky" <dchelimsky@gmail.com> writes:

>    user = User.new
>    setup do
> kind of interesting.
I'm not sure how I could implement that (and I wont ;-)).  But maybe
includable setups would be useful.

module UserSetup
  setup do  # ???
    @user = User.new
  end
end

context "Users" do
  context "with a valid email address" do
    include UserSetup
    setup { @user.email ... }
  end
  context "with a valid email address" do
    include UserSetup
    setup { @user.email ... }
  end
end

That shouldn't be too hard to do...

On 1/24/07, Christian Neukirchen <chneukirchen@gmail.com> wrote:
>     @user = User.new
>   end
> end


It's actually pretty easy if you don't mind using the Module#included 
hook.

module UserSetup
  def self.included(context)
    context.setup do
      @user = User.new
    end
  end
end

But I'm sure you would have figured that out eventually... :)

context "Users" do
>   context "with a valid email address" do
>     include UserSetup
>     setup { @user.email ... }
>   end
>   context "with a valid email address" do
>     include UserSetup
>     setup { @user.email ... }
>   end
> end


Cheers,
/Nick

"Nick Sieger" <nicksieger@gmail.com> writes:

>>   setup do  # ???
>      @user = User.new
>    end
>  end
> end
>
> But I'm sure you would have figured that out eventually... :)

Nifty, thanks!

M. Edward (Ed) Borasky wrote:
> 
> I do have a bone to pick with the tutorial (or maybe I don't understand 
> TDD/BDD yet). In the stack example, I cringe with horror when I see them 
> setting a method that just returns "true" simply to get a test to pass. 
> That grates on me, because I *know* I'm actually going to have to write 
> the real code eventually. It seems like a waste of time to do the 
> nitty-gritty "add a test to the spec/add a trivial fix to the code" cycle.
> 
The idea behind doing small steps is that you know exactly what went 
wrong
when you run the tests. If you roll several steps into one 
test/development
cycle then you have many possibilities to investigate what went wrong if
they fail.

Sometimes, it is easier to skip the most trivial steps though!

> they fail.
>
> Sometimes, it is easier to skip the most trivial steps though!

Also keep in mind that is just a tutorial example. The goal is just to
communicate the process - spec first, make the spec pass, next step.
It's all about a single point of failure, and a clear mapping between
specs and functionality.

--
Giles Bowkett

Blog: http://gilesbowkett.blogspot.com
Portfolio: http://www.gilesgoatboy.org

On 6/25/07, Giles Bowkett <gilesb@gmail.com> wrote:

> Also keep in mind that is just a tutorial example. The goal is just to
> communicate the process - spec first, make the spec pass, next step.
> It's all about a single point of failure, and a clear mapping between
> specs and functionality.

Actually I believe that the pure process of
test-driven/behavior-driven development is:

1) Write the test/spec
2) Ensure that it FAILS
3) Write the code to make it pass
4) Goto step 1

Step 2 is to debug the spec.

--
Rick DeNatale

My blog on Ruby
http://talklikeaduck.denhaven2.com/

On 6/27/07, Rick DeNatale <rick.denatale@gmail.com> wrote:
> 1) Write the test/spec
> 2) Ensure that it FAILS
> 3) Write the code to make it pass
> 4) Goto step 1
>
> Step 2 is to debug the spec.

The reason that we want to see the test fail is so that we can
confident that when it passes that it's passing because of the code we
wrote in step 3. Otherwise, we could write code that the test doesn't
interact with in any way resulting in a useless test and dead code.

Is that what you mean by "debug the spec?" If not, can you please 
explain?

Thx,
David

On 6/27/07, David Chelimsky <dchelimsky@gmail.com> wrote:
> >
> interact with in any way resulting in a useless test and dead code.
>
> Is that what you mean by "debug the spec?" If not, can you please explain?

Better late than never. Yes

--
Rick DeNatale

My blog on Ruby
http://talklikeaduck.denhaven2.com/

On 7/18/07, Rick DeNatale <rick.denatale@gmail.com> wrote:
> > > test-driven/behavior-driven development is:
> > wrote in step 3. Otherwise, we could write code that the test doesn't
> > interact with in any way resulting in a useless test and dead code.
> >
> > Is that what you mean by "debug the spec?" If not, can you please explain?
>
> Better late than never. Yes

:)

Thanks