Ruby Forum Ruby on Rails > why detest rspec? (was: community poll about testing and spe

On 7/24/07, Nathaniel Brown <nshb@inimit.com> wrote:
>
> I may be biased, but I would really like and have been looking for an
> alternative BDD choice besides RSpec.
>
> What others are there? I have seen a couple by the likes of Ryan, even
> Rick. Are any better? What about a conrast of testing libraries
> highlighting the benefits and opinions of each?
>
> I really want to get into testing beyond Test/Unit, but I detest the
> RSpec library out of spite :)

At the risk of shooting myself in the foot, I'm really curious to know
why you detest RSpec.

I didn't like it for a while because the syntax kept changing.  This
made me rewrite the BDD section of my book twice (three times when I
rewrite to match 1.0).

But now that I've been using the new syntax more, I like it. :P

--Jeremy

On 7/24/07, David Chelimsky <dchelimsky@gmail.com> wrote:
> > I really want to get into testing beyond Test/Unit, but I detest the
> > RSpec library out of spite :)
>
> At the risk of shooting myself in the foot, I'm really curious to know
> why you detest RSpec.
>
> >
>


--
http://www.jeremymcanally.com/

My free Ruby e-book:
http://www.humblelittlerubybook.com/book/

My blogs:
http://www.mrneighborly.com/
http://www.rubyinpractice.com/

David Chelimsky wrote:
>> I may be biased, but I would really like and have been looking for an
>> alternative BDD choice besides RSpec.
>>
>> What others are there? I have seen a couple by the likes of Ryan, even
>> Rick. Are any better? What about a conrast of testing libraries
>> highlighting the benefits and opinions of each?
>>
>> I really want to get into testing beyond Test/Unit, but I detest the
>> RSpec library out of spite :)
> 
> At the risk of shooting myself in the foot, I'm really curious to know
> why you detest RSpec.

I'd like to chime in. I would not at all say that I detest RSpec in any 
way, but I do find it to be rather besides the point. It just never 
caught on to me, you know?

When I first looked at RSpec, I thought it was rather cryptic. This 
boils down to a matter of taste, I know, but all those "recipes" (I 
don't know the term) really aren't all that legible to me. 
"bowling.score.should == 0"? "should" is equal to 0? Is it any more 
expressive than "assert 0, bowling.score"? Again, it's personal 
preference, but this is one DSL that I don't care for. So I dismissed it 
in favor of the Test::Units I have come to cherish.

Then I attended a lecture by Aslak Hellesøy. I mean, with so many people 
around raving about RSpec, I must be missing the point right? But I was 
let down: Alsak demonstrated it to be a wrapper DSL around Test::Unit 
for "people who find it hard to come to gripes with TDD". So it's a sort 
of workaround? I was disappointed by that statement of his.

He says Test::Unit is tightly coupled to the actual code and that that's 
a bad thing. I say I like that coupling, because it keeps a sharp edge 
on the test-before-you-write principle: if you're mucking around code or 
tests that worked before, you'll find yourself pressed to advocate that 
change and express it in an additional or revised test.

Moving from personal preference into the field of computer science, 
there is no argument that I know of favoring either in code coverage or 
testing metrics. (Indeed, "coupling" and "cohesion" have so far only 
referred to the functional components themselves, not their tests.) This 
doesn't prove either Alsak or me wrong or right, but does iterate that 
it's a matter of preference.

"But," some might say, "we're testing *behavior* here. That's different. 
We can clean up methods, throw code around, but as long as the behavior 
remains the same, then that's OK." I see it as the same discussion about 
white box testing and black box testing, but with a twist.

As a black box, the bowling example on the RSpec home page seems to work 
fine. As a white box, it's lacking in every respect because the 
algorithm we assume to be present is, in fact, completely absent. I like 
to test the algorithm, because if I don't prove the behavior of the 
algorithm to be correct then I will need to verify every possible 
outcome instead. That kind of verification isn't very efficient.

So the twist is this: this time around, we're mixing up black box 
testing by dubbing it "behavior" and white box testing by performing 
actual method calls. I've been searching for a rationale, but could only 
find the one by Alsak that I was disappointed by before.

The way I see it, RSpec is just a matter of preference, and a bit of a 
hype at that. Perhaps I'm misunderstanding, and if so, I'd love to see 
the entire mantra clarified. But until then, I'm preferring Test::Unit 
for its readability and code coupling, and saying nay to all those who 
preach that RSpec is a level beyond Test::Unit.

Looking forward to your responses.

--
Roderick van Domburg
http://www.nedforce.nl
Perhaps I'm misunderstanding, and if so I'd certainly appreciate it if

@Roderick:
Wow. That mirrors my thoughts exactly. I'd love for someone to chime in 
and
counter this. Thanks for putting it so well.

> Alsak demonstrated it to be a wrapper DSL around Test::Unit
> for "people who find it hard to come to gripes with TDD". So it's a sort
> of workaround? I was disappointed by that statement of his.

I don't get it. What can you write in RSpec that you can't write,
nearly the same way, in Test::Unit?

> He says Test::Unit is tightly coupled to the actual code and that that's
> a bad thing.

Done right, either one tightly _coheres_ to the actual code.

(Define "couple" as "A must change only because B changed", and define
"cohere" as "A and B share legitimate reasons to change together".)

--
 Phlip
  http://www.oreilly.com/catalog/9780596510657/
 "Test Driven Ajax (on Rails)"
 assert_xpath, assert_javascript, & assert_ajax

Roderick: David Chelimsky might be a little miffed with your selective
quoting which makes it appear as if he doesn't like RSpec...

David Chelimsky wrote:
>> I may be biased, but I would really like and have been looking for an
>> alternative BDD choice besides RSpec.

This was not written by David, but was written in response to an email 
from
him. The only clue to this is the level of indentation of the quoting.
--
James.
http://blog.floehopper.org

On 7/25/07, Roderick van Domburg <rails-mailing-list@andreas-s.net> 
wrote:
> >> RSpec library out of spite :)
> don't know the term) really aren't all that legible to me.
>
> doesn't prove either Alsak or me wrong or right, but does iterate that
> to test the algorithm, because if I don't prove the behavior of the
> the entire mantra clarified. But until then, I'm preferring Test::Unit
> for its readability and code coupling, and saying nay to all those who
> preach that RSpec is a level beyond Test::Unit.

I'm sorry that you see this as preaching and mantras. I have no
intention of converting you. I like RSpec because it expresses the
things I want in a BDD framework. You don't have to like it.

BDD comes from TDD, which comes from XP, which includes Refactoring as
an integral part of the process. Refactoring is more expensive as your
tests are more tightly coupled to the code, so one goal of TDD is to
decouple the tests from the detail of the code (which is what changes
most often). One thing that aids in this decoupling is focusing on
behaviour instead of internal implementation. BDD champions this goal
by putting it front and center. RSpec supports this by trying to use
words like "describe" instead of "TestCase" and "should" instead of
"assert."

It is clear from your arguments that a) your testing goals do not
align with those of RSpec and b) you've only read the cover (your
argument about the example on the Home Page). If you're seriously
interested in learning about RSpec and BDD, then you should try to
actually use it on a small project and see how it feels. If not, then
so be it. I still get to use it, so I am happy.

Cheers,
David

On 7/25/07, Roderick van Domburg <rails-mailing-list@andreas-s.net> 
wrote:
> in favor of the Test::Units I have come to cherish.
Cryptic?  Seriously?
"bowling score should equal 0"
vs
"assert 0 bowling score"

The first statement is parseable by a second grader.  The second
statement only makes sense if you are very familiar with test::unit.

I think the problem is that you're looking at the code and ignoring
the meaning.  You ask "'should' is equal to 0?"  It seems that you're
examining the code the way the interpretter would, and thinking, "the
object returned by the method should is equal to 0."  That's
completely the wrong mindset.

I would say that TDD is a tool to help you solve the problem of
designing and implementing behavior.  Test::Unit works fine in that
regard, but RSpec reduces the semantic distance between the developer
and the problem domain.

Consider a task list you write for yourself.  Again you'll have
statements that look like "bowling score should equal 0" rather than
"assert 0 bowling score."  Throw in a couple dots and you're good to
go :)  RSpec lets you express behavior in code almost as neatly as you
do in your head.  That's very, very powerful.  I've yet to come across
a tool that brings you that close.


> Then I attended a lecture by Aslak Hellesøy. I mean, with so many people
> around raving about RSpec, I must be missing the point right? But I was
> let down: Alsak demonstrated it to be a wrapper DSL around Test::Unit
> for "people who find it hard to come to gripes with TDD". So it's a sort
> of workaround? I was disappointed by that statement of his.

I'm not sure how long ago that lecture was, but to the best of my
knowledge RSpec is no longer just a wrapper around Test::Unit.  On the
other hand, I'm not sure why it matters how it's implemented under the
hood.  We are concerned with behavior here, after all ;)

Anyway, the big BDD guys all basically say that there's nothing
different if you're doing TDD right.  BDD/RSpec isn't a workaround to
TDD, but rather a fast track to the good things about TDD.  Again you
have the benefit of a shorter semantic distance.  By using RSpec you
start off focusing on design and behavior, which is the ultimate goal
of TDD.


> He says Test::Unit is tightly coupled to the actual code and that that's
> a bad thing. I say I like that coupling, because it keeps a sharp edge
> on the test-before-you-write principle: if you're mucking around code or
> tests that worked before, you'll find yourself pressed to advocate that
> change and express it in an additional or revised test.

The concern is that Test::Unit encourages an artificial coupling of
one test class to one production class.  That's a completely different
thing from tests being coupled to the code, which as you pointed out
will/should be the case.  The problem is that a new test class can
feel heavy weight (especially if you've been using JUnit for several
years, you almost certainly have that stigma).  People think, "wow why
should I create a whole new class when it's just a little bit of
behavior?"  Most people know that it's not a good idea to have a class
with just one or two methods, but testing has different rules and the
fact is that a test class with just a few methods is often precisely
what you need.

RSpec helps you avoid this problem by encouraging you to write several
small, focused examples.  Once again it's not something that you
couldn't do with Test::Unit, but the RSpec way is just a lot closer to
the Right Way.


> white box testing and black box testing, but with a twist.
>
> As a black box, the bowling example on the RSpec home page seems to work
> fine. As a white box, it's lacking in every respect because the
> algorithm we assume to be present is, in fact, completely absent. I like
> to test the algorithm, because if I don't prove the behavior of the
> algorithm to be correct then I will need to verify every possible
> outcome instead. That kind of verification isn't very efficient.

I don't think RSpec is meant to be a white-box testing tool.  Again,
it's all about design and behavior.  Despite that, you should still be
able to achieve a sufficient level of white-box testing using mocks.

In fact, sometimes I'll write two sets of specs for the same example.
One will use concrete implementations, allowing me to verify that I
get the desired side-effects (records created, etc).  Another one will
use mocks, allowing me to verify that the proper interactions are
being made.  I've got a good example, but I don't want to add 65 lines
of code to this already lengthy message...but if there's any interest
I'll be happy to post it in a followup message.


> So the twist is this: this time around, we're mixing up black box
> testing by dubbing it "behavior" and white box testing by performing
> actual method calls. I've been searching for a rationale, but could only
> find the one by Alsak that I was disappointed by before.
>
> The way I see it, RSpec is just a matter of preference, and a bit of a
> hype at that. Perhaps I'm misunderstanding, and if so, I'd love to see
> the entire mantra clarified. But until then, I'm preferring Test::Unit
> for its readability and code coupling, and saying nay to all those who
> preach that RSpec is a level beyond Test::Unit.

Sure, just as some people prefer C.  It's certainly more readable -
you have semicolons that tell you where a statement ends!

Pat

Thanks for your replies everyone -- it sure has been improving my 
understanding, plus I love the debate. :-)

Allow me to respond chronologically.

Phlip wrote:
>> Alsak demonstrated it to be a wrapper DSL around Test::Unit
>> for "people who find it hard to come to gripes with TDD". So it's a sort
>> of workaround? I was disappointed by that statement of his.
> 
> I don't get it. What can you write in RSpec that you can't write,
> nearly the same way, in Test::Unit?

That was exactly my point.

Phlip wrote:
>> He says Test::Unit is tightly coupled to the actual code and that that's
>> a bad thing.
> 
> Done right, either one tightly _coheres_ to the actual code.
> 
> (Define "couple" as "A must change only because B changed", and define
> "cohere" as "A and B share legitimate reasons to change together".)

That was me quoting him. Both Alsak and you are right. :-)

Alsak is right in the sense that tight coupling indeed does not improve 
testability or robustness because a network of modules that are 
interrelated are more difficult to test than modules in isolation.

I totally agree with you on the cohesion tab. Cohesion (singleness of 
purposeness of a module) is generally regarded as a good thing to have 
in computer science and is indicative of a well-designed software 
architecture that is reusable and understandable.

In fact, it is often said that a high cohesion correlates with high test 
coverage, because a single test will touch on multiple facets of the 
module, and hence multiple tests will cover the facets multiple times.

But here's the thing: these quality measures are about intra- or 
inter-relationships between functional modules -- not between a 
functional module and its unit tests. When we speak in this context of 
modules and their unit tests, I have yet to find any evidence that 
either refer cohesion or coupling is either desirable or undesirable.

The point I'm trying to make here is this: with "tight coupling" [sic.] 
being listed as one of the reasons to move away from Test::Unit, that 
really just is preference over fact.

James Mead wrote:
> Roderick: David Chelimsky might be a little miffed with your selective
> quoting which makes it appear as if he doesn't like RSpec...
> 
> David Chelimsky wrote:
>>> I may be biased, but I would really like and have been looking for an
>>> alternative BDD choice besides RSpec.
> 
> This was not written by David, but was written in response to an email 
> from him. The only clue to this is the level of indentation of the quoting.

I apologize for my misquoting. It is not my intention to bring either 
discredit or misuse their words.

David Chelimsky wrote:
> I'm sorry that you see this as preaching and mantras. I have no
> intention of converting you. I like RSpec because it expresses the
> things I want in a BDD framework. You don't have to like it.

But maybe I don't know what I'm missing, in which case I want to like 
it! :-)

David Chelimsky wrote:
> BDD comes from TDD, which comes from XP, which includes Refactoring as
> an integral part of the process. Refactoring is more expensive as your
> tests are more tightly coupled to the code, so one goal of TDD is to
> decouple the tests from the detail of the code (which is what changes
> most often). One thing that aids in this decoupling is focusing on
> behaviour instead of internal implementation. BDD champions this goal
> by putting it front and center. RSpec supports this by trying to use
> words like "describe" instead of "TestCase" and "should" instead of
> "assert."

I do acknowledge the design goals, but keep wondering: how can only 
words make the difference between BDD and TDD? I fail to see how 
replacing words and reshuffling parameters (i.e. assert 0, bowling.score 
--> bowling.score.should == 0) can facilitate the decoupling / decrease 
in cohesion.

David Chelimsky wrote:
> It is clear from your arguments that a) your testing goals do not
> align with those of RSpec and b) you've only read the cover (your
> argument about the example on the Home Page). If you're seriously
> interested in learning about RSpec and BDD, then you should try to
> actually use it on a small project and see how it feels. If not, then
> so be it. I still get to use it, so I am happy.

You are absolutely right in this. I should truly give it a whirl and 
haven't yet. I only checked out the examples and attended Alsak's 2007 
lecture. Not trying sure is bad developer practice :-) Thanks for point 
that out to me, it's a good reality check.

Could you tell me though, what is the gap that you perceive between my 
testing goals and those of RSpec? That knowledge would be really helpful 
in completing my mental picture about RSpec.

Pat Maddox wrote:
> On 7/25/07, Roderick van Domburg <rails-mailing-list@andreas-s.net> 
> wrote:
>> in favor of the Test::Units I have come to cherish.
> Cryptic?  Seriously?
> "bowling score should equal 0"
> vs
> "assert 0 bowling score"

Again, thanks! I was reading aloud: "bowling score should is equal to 
0". Never thought of keeping it simpler than that.

Pat Maddox wrote:
>> Then I attended a lecture by Aslak Helles�y. I mean, with so many people
>> around raving about RSpec, I must be missing the point right? But I was
>> let down: Alsak demonstrated it to be a wrapper DSL around Test::Unit
>> for "people who find it hard to come to gripes with TDD". So it's a sort
>> of workaround? I was disappointed by that statement of his.
> 
> I'm not sure how long ago that lecture was, but to the best of my
> knowledge RSpec is no longer just a wrapper around Test::Unit.  On the
> other hand, I'm not sure why it matters how it's implemented under the
> hood.  We are concerned with behavior here, after all ;)

Indeed that was my perception of things, not something he said.

Earlier Philip asked "I don't get it. What can you write in RSpec that 
you can't write, nearly the same way, in Test::Unit?". That's the same 
flow of mind that got me to arrive to this perception.

Pat Maddox wrote:
>> He says Test::Unit is tightly coupled to the actual code and that that's
>> a bad thing. I say I like that coupling, because it keeps a sharp edge
>> on the test-before-you-write principle: if you're mucking around code or
>> tests that worked before, you'll find yourself pressed to advocate that
>> change and express it in an additional or revised test.
> 
> The concern is that Test::Unit encourages an artificial coupling of
> one test class to one production class.  That's a completely different
> thing from tests being coupled to the code, which as you pointed out
> will/should be the case.  The problem is that a new test class can
> feel heavy weight (especially if you've been using JUnit for several
> years, you almost certainly have that stigma).  People think, "wow why
> should I create a whole new class when it's just a little bit of
> behavior?"  Most people know that it's not a good idea to have a class
> with just one or two methods, but testing has different rules and the
> fact is that a test class with just a few methods is often precisely
> what you need.
> 
> RSpec helps you avoid this problem by encouraging you to write several
> small, focused examples.  Once again it's not something that you
> couldn't do with Test::Unit, but the RSpec way is just a lot closer to
> the Right Way.

That did put things into perspective for me. Thanks!

Pat Maddox wrote:
>> As a black box, the bowling example on the RSpec home page seems to work
>> fine. As a white box, it's lacking in every respect because the
>> algorithm we assume to be present is, in fact, completely absent. I like
>> to test the algorithm, because if I don't prove the behavior of the
>> algorithm to be correct then I will need to verify every possible
>> outcome instead. That kind of verification isn't very efficient.
> 
> I don't think RSpec is meant to be a white-box testing tool.  Again,
> it's all about design and behavior.  Despite that, you should still be
> able to achieve a sufficient level of white-box testing using mocks.
> 
> In fact, sometimes I'll write two sets of specs for the same example.
> One will use concrete implementations, allowing me to verify that I
> get the desired side-effects (records created, etc).  Another one will
> use mocks, allowing me to verify that the proper interactions are
> being made.  I've got a good example, but I don't want to add 65 lines
> of code to this already lengthy message...but if there's any interest
> I'll be happy to post it in a followup message.

I am interested! I enjoy the discussion and think it'll be 65 lines well 
spent. :-)

Cheers,

Roderick

Pat Maddox wrote:
> In fact, sometimes I'll write two sets of specs for the same example.
> One will use concrete implementations, allowing me to verify that I
> get the desired side-effects (records created, etc).  Another one will
> use mocks, allowing me to verify that the proper interactions are
> being made.  I've got a good example, but I don't want to add 65 lines
> of code to this already lengthy message...but if there's any interest
> I'll be happy to post it in a followup message.

This sounds very interesting. I would love to see a concrete example
detailing your approach.

--
Cheers,
- Jacob Atzen

On 7/26/07, Jacob Atzen <jacob@jacobatzen.dk> wrote:
> This sounds very interesting. I would love to see a concrete example
> detailing your approach.

Okay, here we go :)

describe VideoService, " when creating a video" do
  fixtures :companies, :sites, :video_formats, :publish_settings

  before(:each) do
    @video = Video.new
    @video.company = companies(:test)
    @video.site = sites(:test)
    sites(:test).stub!(:formats).and_return [ video_formats(:flash) ]

    @origin = mock_model(Origin, :process_source => true,
:transcode_asset => true, :storage => "file")
    Origin.stub!(:find_transcoder).and_return @origin

    @service = VideoService.new
  end

  def create_video
    @service.create_video :video => @video, :signature => "abc123",
:frame => 1, :storage => "file"
  end

  it "should add a new video record" do
    lambda { create_video }.should change(Video, :count).by(1)
  end

  it "should create two assets - one source and one transcoded" do
    lambda { create_video }.should change(Asset, :count).by(2)
    @video.source_asset.should be_source
    @video.should have(2).assets
  end

  it "should create the main asset with the appropriate video format" do
    create_video
    @video.assets.find_by_video_format_id(video_formats(:flash).id).should_not
be_blank
  end
end

describe VideoService, " when creating a video" do
  fixtures :companies, :sites, :video_formats

  before(:each) do
    @video = mock_model(Video, :save => true, :company_id =>
companies(:test).id, :site => sites(:test), :hook => nil)
    sites(:test).stub!(:formats).and_return [ video_formats(:flash) ]

    @source_asset = mock_model(Asset, :video_id= => @video.id, :save => 
true)
    @transcode_asset = mock_model(Asset, :video_id= => @video.id, :save 
=> true)
    Asset.stub!(:new).and_return @source_asset, @transcode_asset
    @video.stub!(:source_asset).and_return @source_asset

    @source_origin = mock_model(Origin, :process_source => true,
:transcode_asset => true, :storage => "file")
    @transcode_origin = mock_model(Origin, :transcode_asset => true,
:storage => "file")
    Origin.stub!(:find_transcoder).and_return @source_origin, 
@transcode_origin

    @service = VideoService.new
  end

  def create_video
    @service.create_video :video => @video, :signature => "abc123",
:frame => 1, :storage => "file"
  end

  it "should find a transcoder" do
    Origin.should_receive(:find_transcoder).with(companies(:test).id,
false, "file").and_return @source_origin
    create_video
  end

  it "should process the source asset" do
    @source_origin.should_receive(:process_source).with("abc123",
@source_asset, 1, "http://test.host:80/")
    create_video
  end

  it "should find an origin for the transcode format" do
    Origin.should_receive(:find_transcoder).with(companies(:test).id,
false, nil).and_return @transcode_origin
    create_video
  end

  it "should transcode the new asset" do
    @transcode_origin.should_receive(:transcode_asset).with("abc123",
@transcode_asset, "http://test.host:80/",
"http://test.host:80/video_formats/#{video_formats(:flash).id}.xml")
    create_video
  end
end



I created the VideoService to wrap the creation of a video in our
system.  At that stage in the code, there were a couple main parts -
creating a new video record, creating a couple asset records, and
notifying our transcode machines about the new assets.

The first example specifies the db behavior I want.  That was my
black-box test (you might say it's grey-box because I'm stubbing some
stuff out, but that wasn't part of the original spec.  It's just there
so the spec can run - Origin makes a network call and it's easier to
stub the whole thing out).

The second example is the white-box test, which verifies that the
internals work.  Certain origins should be found, they need to receive
certain calls with a particular video, etc.

Now I'm not sure if people will like those specs :)  but hopefully it
gives you an idea of how you can approach specifications from both
angles.

Pat

On 7/26/07, Roderick van Domburg <rails-mailing-list@andreas-s.net> 
wrote:
> Could you tell me though, what is the gap that you perceive between my
> testing goals and those of RSpec? That knowledge would be really helpful
> in completing my mental picture about RSpec.

My perception is based on your statement that you find the tight
coupling between tests and the subject code to be a good thing, and
that you like the fact that making changes to code forces you to make
changes to your tests.

Compare this to TDD's goal of supporting inexpensive refactoring
through very loose coupling between tests and code.

I also sensed that you don't find value in focusing the discussion on 
behaviour:

=============================================
"But," some might say, "we're testing *behavior* here."
=============================================

To me, putting the focus on behaviour encourages a subtle, yet
powerful change in how you perceive these things we call
tests/specs/examples, etc. Obviously, RSpec is interested in helping
you to think of tests as executable examples of expected behaviour as
opposed to, well, tests.

This help?

David Chelimsky wrote:
> To me, putting the focus on behaviour encourages a subtle, yet
> powerful change in how you perceive these things we call
> tests/specs/examples, etc. Obviously, RSpec is interested in helping
> you to think of tests as executable examples of expected behaviour as
> opposed to, well, tests.
> 
> This help?

Yes, that is a metaphor that sticks.

All that rests now is how "renaming" assert(x, y) to Object#should truly 
facilitates that idea. Yes there are other niceties and syntactic sugar, 
but isn't providing a humanized DSL really the gist of it all?

In the end, you'll still be comparing method results and variable 
assignments.

--
Roderick van Domburg
http://www.nedforce.nl

Roderick van Domburg wrote:
> 
> All that rests now is how "renaming" assert(x, y) to Object#should truly 
> facilitates that idea. Yes there are other niceties and syntactic sugar, 
> but isn't providing a humanized DSL really the gist of it all?
> 
> In the end, you'll still be comparing method results and variable 
> assignments.

Yes, in the end... you see a result like this. But, I'd like to back
this up to before you write your code.

Let's start with a high-level business rule for an application. We might
have a discussion with our client and determine the following.

A user should not be allowed to create a new account when providing an
existing email address.

This is something that we'd have our clients agree upon before we ever
touched any code. When it comes time to implement this behavior into the
application, we can translate this into RSpec very easily.

describe User, "new account" do

   it "should not be allowed to create a new account when providing an
existing email address."

end

When we run our specs, this will show up as unimplemented. I often add
several of these once a client has signed off on some specifications,
and it results in a TODO-list for me. At this point, I can begin turning
these into full specs.

In a nuthsell, it's not just a DSL for Ruby... it's provides a framework
for discussing requirements with our clients and Interaction Design
team. It's clear English and leaves little room for ambiguity, which I
believe is a good thing! :-)

If you enjoy translating high-level requirements into Test::Unit... and
don't mind the underscore/camelcase madeness... well, good luck with
that. I switched to Ruby/Rails because of how expressive and
English-like it was and think that it's very fitting that RSpec would
evolve out of this community.

Cheers,
Robby


--
Robby Russell
http://www.robbyonrails.com/
http://www.planetargon.com/

Hi --

On Thu, 26 Jul 2007, Robby Russell wrote:

> When we run our specs, this will show up as unimplemented. I often add
> don't mind the underscore/camelcase madeness... well, good luck with
> that. I switched to Ruby/Rails because of how expressive and
> English-like it was and think that it's very fitting that RSpec would
> evolve out of this community.

OK... but, lest we forget, Nathaniel's work on test/unit -- past,
present, and future (I hope) -- has played an incalculable role in
shaping the Ruby programming culture.  Luckily, it's not a zero-sum
game; people can delve into RSpec without having to jettison or scoff
at test/unit.  It's possible for two highly-accomplished test
frameworks to evolve out of the same community :-)

(I'm not trying to get you to like test/unit; I'm just putting in a
word for not having it have to be a winner-take-all kind of
relationship, and also eager to keep the significance of test/unit to
Ruby culture on the radar.  I know I'm showing my age -- in this case,
almost seven :-)


David

--
* Books:
   RAILS ROUTING (new! http://www.awprofessional.com/title/0321509242)
   RUBY FOR RAILS (http://www.manning.com/black)
* Ruby/Rails training
     & consulting:  Ruby Power and Light, LLC (http://www.rubypal.com)

On 7/26/07, Roderick van Domburg <rails-mailing-list@andreas-s.net> 
wrote:
>
> All that rests now is how "renaming" assert(x, y) to Object#should truly
> facilitates that idea.

I think this is one of several aspects of RSpec that help promote this
idea. I do thing that saying "should" instead of "assert" changes the
feel from tests to examples of behaviour. But it's not limited to
that.

The structure/organization using describe/it instead of
TestCase/test_method and, especially, using strings to express the
intent rather than method names both go a long way to support this
shift in focus. The reporting you get helps as well.

David

On 7/27/07, dblack@wobblini.net <dblack@wobblini.net> wrote:
> >
> > for discussing requirements with our clients and Interaction Design
> present, and future (I hope) -- has played an incalculable role in
> shaping the Ruby programming culture.  Luckily, it's not a zero-sum
> game; people can delve into RSpec without having to jettison or scoff
> at test/unit.  It's possible for two highly-accomplished test
> frameworks to evolve out of the same community :-)

More than two. Don't forget watir. And all of the mocking frameworks.
There's a lot of good stuff here.

> (I'm not trying to get you to like test/unit; I'm just putting in a
> word for not having it have to be a winner-take-all kind of
> relationship, and also eager to keep the significance of test/unit to
> Ruby culture on the radar.

A very important point. And now that Ryan has taken over maintenance
of test/unit, I expect to see some great new ideas appearing there as
well.

The thing about RSpec is that it is intended to be a BDD framework,
not a testing framework. It aims to support a behaviour-focused
process in which you, in very granular steps, describe a bit of
behaviour and then implement it.

But BDD is not the end of the testing story on any project. At least I
don't think it _should_ be. Some might even say it's not part of the
testing story at all :)

A perfect example is Matrix Testing, introduced by ZenTest. In
fairness, I haven't done this yet, but I can imagine a process that
goes like this:

1. Develop a component using TDD (and RSpec) to encourage simple
design and usable APIs.
2. Write a matrix test (using ZenTest with test/unit) to ensure that
you've covered all of the permutations you anticipate.
3. If the matrix test exposes any holes, go back and use RSpec to
drive the process of plugging them.

I can hear some saying "but that's doing the same work twice. Why not
just a Matrix Test to begin with and call it a day?"

Because these two tools/approaches solve different problems. RSpec/BDD
is good at expressing behaviour and encouraging simple, flexible
designs. MatrixTesting is good at expressing multiple permutations in
a clean and simple way. Used together, I believe that you'd end up
with a simpler design with better test coverage than you would with
either tool alone.

In XP we do Customer Tests and Developer Tests, though these have a
lot of different names. They are, in the end, exercising the same
code. But they express very different things. And what my experience
tells me is that it's the combination of layers of testing that proves
most effective. So for me, this notion of combining a BDD framework
(for DESIGN) with a testing framework (for TESTING) makes perfect
sense.

> I know I'm showing my age -- in this case,
> almost seven :-)

Happy Birthday (6 year olds only say "almost 7" when it's just a few 
days away).

Cheers,
David

Many thanks for taking the time to write down your thoughts, everyone. 
It's been very helpful to me to put RSpec in perspective. I'll take it 
for a whirl like David suggested, and will revisit this topic if RSpec 
manages to win me over. :-)

--
Roderick van Domburg
http://www.nedforce.nl

This argument is the same one you could use to say "what can you
write in Ruby that you can't write, nearly the same way, in C"? Heck,
they are both TC general purpose computer programming languages.
Ahhhh, but the "feel" is different. That's what I find sets RSpec
apart, and if it doesn't click for you it might not *work* for you.

On Jul 25, 2007, at 11:50 AM, David Chelimsky wrote:
> RSpec supports this by trying to use words like "describe" instead  
> of "TestCase" and "should" instead of "assert."

As an aside, RSpec's use of "should" may be confusing to anyone
coming from the networking world:

http://www.faqs.org/rfcs/rfc2119.html

-faisal

s.ross wrote:

> This argument is the same one you could use to say "what can you write in
> Ruby that you can't write, nearly the same way, in C"?

Uh, in the same volume of code? At the same velocity?

--
 Phlip
 http://www.oreilly.com/catalog/9780596510657/
 ^ assert_xpath
 http://tinyurl.com/23tlu5  <-- assert_raise_message

On 7/25/07, Faisal N Jawdat <faisal@faisal.com> wrote:
>
> On Jul 25, 2007, at 11:50 AM, David Chelimsky wrote:
> > RSpec supports this by trying to use words like "describe" instead
> > of "TestCase" and "should" instead of "assert."
>
> As an aside, RSpec's use of "should" may be confusing to anyone
> coming from the networking world:
>
> http://www.faqs.org/rfcs/rfc2119.html


:(

Can't clarify everything for everybody I guess.

We could debate either side of the issue. I could say "with the same
compactness? with equivalent performance characteristics?" but at the
end of the discussion, it's all about what clicks for you. If you are
happy with asserts (like in C, hence my analogy), then use them. I
believe the BDD people -- and I count David among one of the most
thoughtful of the bunch -- have put a good deal of work into making
the description of the spec as natural as possible. Again, it's what
reads naturally to you. If you don't parse it naturally, then no
prob. Also, make careful note of what David says about the failure
messages. This was one of my biggest complaints with Test::Unit and
the fix is pretty close to free with RSpec. The failures report
pretty much in English and if they don't I probably wrote the example
wrong.

--s

Hi --

On Wed, 25 Jul 2007, Faisal N Jawdat wrote:

>
> On Jul 25, 2007, at 11:50 AM, David Chelimsky wrote:
>> RSpec supports this by trying to use words like "describe" instead
>> of "TestCase" and "should" instead of "assert."
>
> As an aside, RSpec's use of "should" may be confusing to anyone
> coming from the networking world:
>
> http://www.faqs.org/rfcs/rfc2119.html

I don't think that coming from the networking world means one can't
make the leap of imagination required to realize that the entire world
does not speak in compliance with RFC2119 :-)  I wouldn't worry about
it for a second.


David

--
* Books:
   RAILS ROUTING (new! http://www.awprofessional.com/title/0321509242)
   RUBY FOR RAILS (http://www.manning.com/black)
* Ruby/Rails training
     & consulting:  Ruby Power and Light, LLC (http://www.rubypal.com)

On 7/25/07, Phlip <phlip2005@gmail.com> wrote:
>
> s.ross wrote:
>
> > This argument is the same one you could use to say "what can you write in
> > Ruby that you can't write, nearly the same way, in C"?
>
> Uh, in the same volume of code? At the same velocity?

OK, so what about python?

I think the point Steve was making was about "feel", which is what
Ruby and Rails are largely about. To me, there is a huge difference
between this:

team.should have(11).players

  and this

assert_equal 11, team.players.size

And an even bigger difference in the output when these fail:

expected 11 players, got 10

  vs

expected 11, got 10

Implementing these probably takes the same amount of time. A month
later, seeing "expected 11 players" is VERY meaningful, especially
when multiple failures appear due to a change you just made. If I see
10 failures that all say "expected x things, got y", then I have a lot
more information right away about what went wrong than if I just see
"expected x, got y".

FWIW.

David

On 7/25/07, James Mead <jamesmead44@gmail.com> wrote:
> Roderick: David Chelimsky might be a little miffed with your selective
> quoting which makes it appear as if he doesn't like RSpec...
>
> David Chelimsky wrote:
> >> I may be biased, but I would really like and have been looking for an
> >> alternative BDD choice besides RSpec.
>
> This was not written by David, but was written in response to an email from
> him. The only clue to this is the level of indentation of the quoting.

Actually, this makes me laugh more than piss me off. But thanks for
sticking up for me :)

Cheers,
David

@David:

Would you be kind enough to briefly explain what you perceive to be the
advantages to RSpec? I'm really hoping for a good comparison of the two, 
so
I can see if it's worth investigating further. (I am writing a book that
will include testing, and need more info). I've played with RSPec and
haven't really found anything I can't do with test:unit.... so that's 
why
I'm looking for more info. I'd really appreciate some input.  I suspect 
that
it's much like TDD... you have to do it before you really appreciate it.

On Jul 25, 6:34 pm, Faisal N Jawdat <fai...@faisal.com> wrote:
> On Jul 25, 2007, at 11:50 AM, David Chelimsky wrote:
>
> > RSpec supports this by trying to use words like "describe" instead  
> > of "TestCase" and "should" instead of "assert."
>
> As an aside, RSpec's use of "should" may be confusing to anyone  
> coming from the networking world:
>
> http://www.faqs.org/rfcs/rfc2119.html

Fortunately, if you find "should" confusing or misleading you can
write your examples such as this one:

  it "should redirect to the home page after logging out"

like this instead:

  it "must redirect to the home page after logging out"

Or using any other word or language you think is better. The language
you employ in your examples is up to you, although the RSpec community
does tend to use some widespread patterns, including the use of the
word "should" in a way that is closer to the "standard English sense"
rather than the "RFC-2119 sense".

Cheers,
Wincent

s.ross wrote:
> This argument is the same one you could use to say "what can you write 
> in Ruby that you can't write, nearly the same way, in C"? Heck, they are 
> both TC general purpose computer programming languages. Ahhhh, but the 
> "feel" is different. That's what I find sets RSpec apart, and if it 
> doesn't click for you it might not *work* for you.

Agreed. I also consider RSpec to be one of those most exciting things to
pop up in the Ruby community since Rails. Our pure-Ruby libs are being
spec'd (not tested) and we can pretty much run rm -rf test/ in our Rails
applications.

It might not sit well with all, but the syntax is a great example of how
great and expressive Ruby can be.


Robby

--
Robby Russell
http://www.robbyonrails.com/
http://www.planetargon.com/

On 7/26/07, Wincent Colaiuta <win@wincent.com> wrote:
> > http://www.faqs.org/rfcs/rfc2119.html
> Or using any other word or language you think is better. The language
> you employ in your examples is up to you, although the RSpec community
> does tend to use some widespread patterns, including the use of the
> word "should" in a way that is closer to the "standard English sense"
> rather than the "RFC-2119 sense".
>
> Cheers,
> Wincent

I'm pretty sure that he's referring to
foo.should == bar

However I have to say I think it's an absurdly nitty "issue"

Pat

On 7/25/07, Brian Hogan <bphogan@gmail.com> wrote:
> @David:
>
> Would you be kind enough to briefly explain what you perceive to be the
> advantages to RSpec? I'm really hoping for a good comparison of the two, so
> I can see if it's worth investigating further. (I am writing a book that
> will include testing, and need more info). I've played with RSPec and
> haven't really found anything I can't do with test:unit.... so that's why
> I'm looking for more info. I'd really appreciate some input.  I suspect that
> it's much like TDD... you have to do it before you really appreciate it.

I don't know that there is anything that you can't do with test/unit -
just that rspec comes w/ a lot of this stuff out of the box. And I
think it is something experiential. Many people who really enjoy using
RSpec say they were skeptical at first, but have grown to love it.

Here are some specific comparisons:

- dsl for expressing/organizing examples using strings

describe Thing do
  it "should do cool stuff" do
    ...
  end
end

vs

class ThingTest << Test::Unit::TestCase
  def test_should_do_cool_stuff
    ...
  end
end

I find that this encourages better naming and organization.

- dsl for expressing expectations

team.should have(11).players

vs

assert_equal 11, team.players.size

- better failure messages

expected empty? to return true, got false

vs

expected true, got false

- better reporting

Thing
- should do this
- should do that
- should do the other thing (FAILED - 1)
- should do even this (PENDING: Not Yet Implemented)

vs

...............

There is also a nicely formatted HTML report out of the box:

  http://rspec.rubyforge.org/report.html

- rspec_on_rails - component isolation allows you to test views before
controllers or models even exist. This is VERY helpful when you're on
an XP or XP-like team.

I'm sure there's more that I'm not thinking of right now. There's also
the "joy" factor. I find it a more joyful experience to use RSpec, for
the reasons stated above and for reasons that you can only appreciate
by exploring it.

Hope this helps.

Cheers,
David

Brian Hogan wrote:
> @David:
> 
> Would you be kind enough to briefly explain what you perceive to be the
> advantages to RSpec? I'm really hoping for a good comparison of the two, so
> I can see if it's worth investigating further. (I am writing a book that
> will include testing, and need more info). I've played with RSPec and
> haven't really found anything I can't do with test:unit....

And that's as expected. As you'll read in most intros to BDD, it's the
same practice as TDD, but with the focus shifted. Everything you can do
with BDD can be done with TDD.

People seem to be expecting some sort of sea change in development
practices, but that's not going to happen here. And from what I've read,
it was never intended to.

It's a small tweak, to refocus TDD practitioners a slight bit. They
think it produces better specs/tests, not because it's something that
couldn't be done before, but because it's now more obvious what you
should be doing.

If TDD has always worked well for you (as I've read before, "if you've
been doing it right all along"), then great, stick with it.

But the very simple terminology changes are what made the point of TDD
finally click for me.

As for rspec specifically? It does what I need it to, produces
phenomenally legible specs, and then stays out of the way.

Kyle

Pat Maddox wrote:
> > >
> >
> foo.should == bar
Ah, yes, you're almost certainly right.

Well the solution to that is also only a step away (alias "should" and
"should_not" to whatever you want in your spec helper file). Of
course, I wouldn't actually recommend doing so, as it seems to be
going against the grain for no real good reason.

Cheers,
Wincent

@David, @Kyle:

Great stuff. If you were going to expose people to tests, would you 
start
with TDD and test:unit, or would you go right to BDD and RSpec?

> ...............
I first saw RSpec in one of the tutorials at RailsConf this year. The
positive reinforcement from seeing what you were testing each time was
compelling. I also find seeing the list of tested behaviors often
suggests to me the next test/feature that needs to be added to the
project. (Now if I could just get my rspec.conf file to be used so I
don't have to pass '-f specdoc' each time I run my tests.

> - rspec_on_rails - component isolation allows you to test views before
> controllers or models even exist. This is VERY helpful when you're on
> an XP or XP-like team.

David, could you expand on this? Or post a link to docs or a tutorial
or blog post about testing views with RSpec.


--
Cynthia Kiser
cynthia.kiser@gmail.com

On 7/27/07, Cynthia Kiser <cynthia.kiser@gmail.com> wrote:
> David Chelimsky wrote:
> > - rspec_on_rails - component isolation allows you to test views before
> > controllers or models even exist. This is VERY helpful when you're on
> > an XP or XP-like team.
>
> David, could you expand on this? Or post a link to docs or a tutorial
> or blog post about testing views with RSpec.

http://blog.davidchelimsky.net/articles/2006/11/06/view-spec-tutorial
http://rspec.rubyforge.org/documentation/rails/writing/views.html
http://rspec.rubyforge.org/rdoc-rails/index.html

Cheers,
David

On 7/25/07, Brian Hogan <bphogan@gmail.com> wrote:
> @David, @Kyle:
>
> Great stuff. If you were going to expose people to tests, would you start
> with TDD and test:unit, or would you go right to BDD and RSpec?

Green field? i.e. no experience with testing at all? Definitely RSpec.

Beyond that, it depends on myriad factors, mostly boiling down to what
is going to be most pragmatic for the team based on their level of
experience with Ruby, testing in general, other frameworks, etc. I'd
always prefer RSpec because, well, I prefer it. But sometimes going w/
test/unit could be more pragmatic, at least in the short run.

David

Brian Hogan wrote:
> @David, @Kyle:
> 
> Great stuff. If you were going to expose people to tests, would you 
> start with TDD and test:unit, or would you go right to BDD and RSpec?
> 

Our team adopted RSpec about a year ago and we've been through several
projects entirely without Test::Unit. If someone has never had any
experience with TDD, then I would advocate learning through BDD... and
would recommend RSpec as the framework to learn it with.

On a side note. JRuby 1.0 will come with two gems.

Rake and RSpec.

Yes... people are using it in the real world and I can testify on it's
behalf in court. It's elegant. Our developers *LOVE* it. I love it. Even
our Interaction Designers have picked up on the lingo and use
"should"-style language in their interface specifications, which
translates nicely into our implementation process.


RSpec ftw!

Robby

--
Robby Russell
http://www.robbyonrails.com/
http://www.planetargon.com/

Look here:

http://rspec.rubyforge.org/documentation/rails/writing/views.html

The idea is that you can mock enough to populate the view data
allowing you to design from the "outside in." Basically mock up your
UI, do it in RHTML (or whatever markup you like), and write specs for
that.