Fixtures, factories and the standard testing framework

Rails
1

A little background:
I’ve been enjoying rails for about a year now tinkering on meaningless
projects here solely for the purpose of learning the framework… not
because I actually needed to use those apps for anything serious. Now
that I started writing my first serious app, naturally I want to dive
into test driven development (“finally” I know - please, no harassment

  • I’m trying). Now the way I see it, the standard testing framework is
    more than sufficient for my needs and as time goes on it’s only going
    to improve. So while things like RSpec and Cucumber are all well and
    good - they don’t really fit my needs.

With that out of the way - I dove right into creating fixtures and –
halt. Fixtures, while improving - are still cumbersome for nested
models. Take for example my Quote model - it has_many
QuotableOperations and QuotableQuantities. When I test them I want to
have complete associations… Fixtures don’t seem to like assigning to
has_many unless I do it from the belongs_to side (which feels
unnatural) so I gave up on fixtures and started working with Factories
(Factory Girl)… Now, a few days into Factory Girl and it also seems
too cumbersome for the job at hand. I’m not sure, but it seems like
either:

I’m doing it wrong

or

There’s a better way

Can anyone point me towards the light?

Take this very simple test for example:

I have a

Quote(:description => “Some job to quote”)

  • with one QuotableOperation(:operation_name => “Engineering”)
  • and two QuotableQuantities(:quantity => “5”, :rate => “0.21”) &
    QuotableQuantities(:quantity => “10”, :rate => “0.21”)

:description, :quote_operations and :quote_quantities are
validate_presence_of’d on the Quote model.

SO - a simple test like:

test “the description of the quote is not blank” do
test
end

seems to be more trouble than it’s worth because I need a fully
associated Quote object.

A helping hand in the right direction would be more than appreciated.

Thanks,

  • FJM
2

C’mon guys, I know this problem can’t be that hard to tackle. Is there
anyone out there with some helpful insight?

On Aug 11, 1:35 am, “[email protected]” [email protected]

3

[email protected] wrote:
[…]

Now the way I see it, the standard testing framework is
more than sufficient for my needs and as time goes on it’s only going
to improve.

Not necessarily. I don’t use Test::Unit, so I could be wrong (and
someone please correct me if I am), but it seems that all the important
work these days is being done with RSpec and Cucumber, and Test::Unit
has more or less stagnated (with a couple of exceptions, such as Phlip’s
plugins).

So while things like RSpec and Cucumber are all well and
good - they don’t really fit my needs.

Try them out before you say that.

  • The reason I don’t use Test::Unit is that RSpec has a syntax and
    philosophy that I like much better. Test::Unit feels clunky and hard to
    use by comparison.

  • Cucumber, however, is a completely different animal, complementary to
    RSpec/Test::Unit. It takes a little while to figure out how to use it,
    but once you do, you’ll wonder how you lived without it. Please don’t
    ignore it.

With that out of the way - I dove right into creating fixtures and –
halt. Fixtures, while improving - are still cumbersome for nested
models. Take for example my Quote model - it has_many
QuotableOperations and QuotableQuantities. When I test them I want to
have complete associations… Fixtures don’t seem to like assigning to
has_many unless I do it from the belongs_to side (which feels
unnatural)

I don’t find this to be a problem, but I agree that fixtures are clumsy
and hard to work with once you get a serious test suite going. Use
factories.

so I gave up on fixtures and started working with Factories
(Factory Girl)… Now, a few days into Factory Girl and it also seems
too cumbersome for the job at hand. I’m not sure, but it seems like
either:

I’m doing it wrong

or

There’s a better way

Can anyone point me towards the light?

Try Machinist.

Take this very simple test for example:

I have a

Quote(:description => “Some job to quote”)

  • with one QuotableOperation(:operation_name => “Engineering”)
  • and two QuotableQuantities(:quantity => “5”, :rate => “0.21”) &
    QuotableQuantities(:quantity => “10”, :rate => “0.21”)

:description, :quote_operations and :quote_quantities are
validate_presence_of’d on the Quote model.

SO - a simple test like:

test “the description of the quote is not blank” do
test
end

seems to be more trouble than it’s worth because I need a fully
associated Quote object.

With Machinist, once you set up the blueprints (which you only have to
do once), Quote.make will give you a fully associated Quote.

A helping hand in the right direction would be more than appreciated.

Good luck! Please ask if you have further questions.

Thanks,

  • FJM

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

4

I most certainly appreciate any and all replies and with that in mind,
I hope I didn’t come off as coming down hard… Intonation is a hard
thing to communicate in textual media… Anyhow, sorry for sounding
that way. Perhaps an emoticon could have livened it up a bit… Hehe.

Anyway…

I’ve just ripped out all of my Test::Unit stuff am having a go at
starting fresh with RSpec, Cucumber and Machinist… Please understand
that I’ve only been casually programming for the past 10 years as a
way to kill time and challenge myself. I have no formal training,
never wrote anything serious and the fact that I’ve written anything
at all that works is a blessing because before last week - I had never
written a test in my life - let alone put any thought into testing an
entire application. TDD/BDD are both intriguing and believe me - I see
their advantages… I’m just very slow on the uptake right now. It’s
not the way I’m used to programming read that as "I’ve developed a
lot of bad habits on my own and now I want to break them the right
way.

SO.

The one thing that keeps coming back to haunt me is that no Factory
framework seems to let you test complete objects from the has_many
side of the association without doing some serious legwork. I see
examples on the Machinist git page that suggest using helper methods
but that brings me to another road block. Where do I write them? How
are they called? What types of things should I not be testing at that
level?

Now, my own personal itch of a project is still small enough that I
don’t mind ripping entire sections of tests out and rewriting them
with new frameworks just to learn how. 1) I have the time and 2) I’m
interested in learning how to do it right more than I am in doing
it… If that makes sense. The parts of my app work just fine for me
without tests… but knowing my itch, I’m going to want to add
complexity and knowing how Ruby and Rails works - I want to add it the
right way so that it’s fun for me to do…

Anyway - I’m getting off topic again.

Back in my first post I describe a situation that I was trying to
test. Could you tell me how I should be approaching that issue (which
may very well be much different than the way I’ve been approaching
it). Why is testing from the has_many side so taboo? Is there a
legitimate way of doing it from the belongs_to side?

Thank you for your reply and thank you too Marnen.

You have both given me good food for thought, even if I don’t know how
to eat it yet.

Hehe. Thanks,

  • FJM
5

Frank J. Mattia wrote:
[…]

I’ve just ripped out all of my Test::Unit stuff

That may or may not be good, depending on how much you had to rip out.
Then again, you can always use your version control system to go back…

I am having a go at
starting fresh with RSpec, Cucumber and Machinist… Please understand
that I’ve only been casually programming for the past 10 years as a
way to kill time and challenge myself. I have no formal training,

Neither do I.

never wrote anything serious and the fact that I’ve written anything
at all that works is a blessing because before last week - I had never
written a test in my life - let alone put any thought into testing an
entire application.

Hey, don’t feel bad. I worked professionally as a programmer for years
without a systematic testing approach.

TDD/BDD are both intriguing and believe me - I see
their advantages… I’m just very slow on the uptake right now. It’s
not the way I’m used to programming read that as "I’ve developed a
lot of bad habits on my own and now I want to break them the right
way.

OK.

SO.

The one thing that keeps coming back to haunt me is that no Factory
framework seems to let you test complete objects from the has_many
side of the association without doing some serious legwork.

What are you talking about? What do you mean by testing from the
has_many side?

I see
examples on the Machinist git page that suggest using helper methods
but that brings me to another road block. Where do I write them? How
are they called? What types of things should I not be testing at that
level?

If memory serves, the Machinist webpage tells you all you need to know
here.

Now, my own personal itch of a project is still small enough that I
don’t mind ripping entire sections of tests out and rewriting them
with new frameworks just to learn how. 1) I have the time and 2) I’m
interested in learning how to do it right more than I am in doing
it… If that makes sense. The parts of my app work just fine for me
without tests… but knowing my itch, I’m going to want to add
complexity and knowing how Ruby and Rails works - I want to add it the
right way so that it’s fun for me to do…

Great!

Anyway - I’m getting off topic again.

Back in my first post I describe a situation that I was trying to
test. Could you tell me how I should be approaching that issue (which
may very well be much different than the way I’ve been approaching
it). Why is testing from the has_many side so taboo? Is there a
legitimate way of doing it from the belongs_to side?

I think if you just set up your blueprints properly, you’ll get what you
need.

Thank you for your reply and thank you too Marnen.

You have both given me good food for thought, even if I don’t know how
to eat it yet.

Hehe. Thanks,

  • FJM

You’re welcome!

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

6

On Aug 11, 9:39 pm, Marnen Laibow-Koser <rails-mailing-l…@andreas-
s.net> wrote:

starting fresh with RSpec, Cucumber and Machinist… Please understand
Hey, don’t feel bad. I worked professionally as a programmer for years

SO.

The one thing that keeps coming back to haunt me is that no Factory
framework seems to let you test complete objects from the has_many
side of the association without doing some serious legwork.

What are you talking about? What do you mean by testing from the
has_many side?

In retrospect that was a bad choice of words. What I should of said
was, “make complete objects”. Then again, I may not fully understand
what I’m trying to do… In which case, this may be a long, painful
road… err. I mean… labor of love.

From the machinist webpage.

================
Other Associations
For has_many and has_and_belongs_to_many associations, ActiveRecord
insists that the object be saved before any associated objects can be
saved. That means you can’t generate the associated objects from
within the blueprint.

The simplest solution is to write a test helper:

def make_post_with_comments(attributes = {})
post = Post.make(attributes)
3.times { post.comments.make }
post
end
Note here that you can call make on a has_many association. (This
isn’t yet supported for DataMapper.)

So suppose I write the following:

def make_complete_quote_object(attributes = {})
quote = Quote.make(attributes)
5.times {
quote.quotable_operations.make
}
quote.quotable_quantities.make

quote
end

Where does that go and do I just use that in lieu of Quote.make? For
example:

new_quote = make_complete_quote_objects(:description => “blah blah
blah”)

Great!

Hehe. Thanks,

  • FJM

You’re welcome!

Best,

Marnen Laibow-Koserhttp://www.marnen.org
[email protected]

Posted viahttp://www.ruby-forum.com/.

Thank you again (and again, I’m sure).

  • FJM
7

Hello–

I think you are asking a lot of good questions. I would direct you to
the rspec and cukes Google G. for a great in-depth discussion of
all this stuff. Also, Pragmatic Programmers has a beta book called
“The rSpec Book” (IIRC) and you can buy it and read the PDF. The final
print copy (if you buy one) will be delivered when it is complete.
This book covers both rSpec and Cucumber and shows a good division of
responsibility between the two tools.

As I am really sold on rSpec and Cucumber, this is kind of OT for the
Rails list, but my recommendations are:

  • Don’t try to re-test Rails. If there was a bug in
    validates_presence_of or has_many, it would probably be known.
  • Corollary: Test your use of Rails. So (e.g.) if you use
    validates_format_of, make sure your regex is right.
  • Work from the outside in. Describe a feature (Cucumber), make it
    fail, drive the feature out, diving into rSpec where unit tests will
    be of value.
  • [controversial] Don’t write controller tests if you can avoid it.
    Write Cucumber features.
  • [less-controversial] Make your controllers so stupid-simple that you
    can unit-test your models and get good coverage.
  • Re-evaluate your progress as you do the red-green-refactor cycle for
    a while. What works for some won’t work for all.
  • Keep your eyes open for what others are sharing about the tools you
    are using and new tools that also might help.
  • Absolutely, keep the big picture in sight: You aren’t writing specs,
    features, or tests; you’re writing an app.

Good luck… a few more answers below:

On Aug 11, 2009, at 6:52 PM, Frank J. Mattia wrote:

out.

SO.
what I’m trying to do… In which case, this may be a long, painful

end

Where does that go

I often put these in spec/spec_helper.rb. You may want to refactor
them out at some point so you can reuse them in your Cucumber steps.

and do I just use that in lieu of Quote.make? For
example:

new_quote = make_complete_quote_objects(:description => “blah blah
blah”)

Yes. But… the make method is not the only one you will use. You may
find yourself creating partial objects look at make_unsaved and plan.
These will allow you to satisfy certain validation requirements that
simply can’t be gotten from a factory.

8

I think between you and Marnen I’ve really discovered a lot of useful
things.

Now that I’m starting to find myself sold on Cuke and RSpec I’m
looking back on earlier today when I actually bought that book you
mention. I’m a huge fan of the pragprog guys and always find myself
looking towards their new books.

Whats more is, just 15 minutes ago I got my first feature to pass. I
was pulling that code from the machinist git page and after a while,
discovered the error with make and make_unsaved… The tutorial on
their wiki has a mistake it seems.

Feature: Manage Quotes
I want to create and manage quotes

Scenario: Quotes List
Given I have quotes with titles Threadmill, Trepan
When I go to the list of quotes
Then I should see “Threadmill”
And I should see “Trepan”

It only took a wee bit of fighting and determination but it’s most
certainly a start.

Again, your guidance has been very much appreciated.

I’m still a little weak in the knowing-what-to-test when area but “not
testing the framework” seems so obvious now it almost hurts.

So now I have to learn what to test where… for example - looking at
the big picture (feature) then diving into rspec for the unit tests. I
still don’t see the clear line. Also, from what I’ve read - to say its
controversial to leave out controller tests is an understatement of
understatements … heh… It almost feels sinful to read it.

I’m going to start checking the cuke and rspec lists from now on - I
can definitely see how this has gotten way off topic for rails but I
most certainly thank you for the insight provided in light of that.
Looking back I’m surprised I didn’t get any “go to the right group
cause we can’t help you” responses.

Thanks again,

  • FJM
9

On Aug 11, 2009, at 6:58 AM, Frank J. Mattia wrote:

C’mon guys, I know this problem can’t be that hard to tackle. Is there
anyone out there with some helpful insight?

Don’t come down too hard on everyone for not answering right away.
Type “rails test factory” into Google and you can do some independent
research.

I personally use Machinist and Faker (good article:
http://toolmantim.com/articles/fixtureless_datas_with_machinist_and_sham)
. Others swear by Factory Girl (read:
http://www.thoughtbot.com/projects/factory_girl)
and still love Object Mother (read:
Object Mother Testing Pattern in Rails — Jeff Perrin)
. You may even like Fixjour (read:
Elctech.com is for sale | HugeDomains)
.

  • I’m trying). Now the way I see it, the standard testing framework
    is
    more than sufficient for my needs and as time goes on it’s only going
    to improve. So while things like RSpec and Cucumber are all well and
    good - they don’t really fit my needs.

There are far more lively communities surrounding rSpec and Cuke, so
if you want help, you’ll find a lot of people willing to answer
questions. I personally feel the thinking behind spec first makes more
sense than test first. At the execution level it’s the same, but what
you are writing is a spec for code that doesn’t yet exist, not a test.
Cucumber is a whole level of goodness that is orthogonal (in most
ways) to either rSpec or Test::Unit. Acceptance testing can be a
lifesaver. You’d be surprised how often a good set of stories can
either flush out poorly designed features or save your bacon if you
break something.

too cumbersome for the job at hand. I’m not sure, but it seems like
either:

I’m doing it wrong

or

There’s a better way

Most will concede that fixtures that describe associations are brittle
and coupled too tightly to a particular implementation. That’s why the
fixture replacements are so popular. Also consider using a mocking/
stubbing framework. rSpec comes with one baked in, or you can use
Mocha or FlexMock. Best to only mock and stub in unit tests, as
acceptance tests are supposed to exercise the whole stack. Of course,
if your app needs to reach out over the network or something like
that, you might have to use something like FakeWeb or a mock/stub for
the service you don’t want to hit with your test.

10

Hi
And none of you here specified

http://www.thoughtbot.com/projects/shoulda/

 a framework Which extends Test::Unit . So I think not a very big

learning curve needed for a Ruby programmer unlike some other tools you
specified

Sijo

11

Thanks for the link. I hadn’t checked out shoulda as deeply as I could
have. However, in all honesty - I really do believe I was looking for
a paradigm shift from away standard testing. I haven’t ruled out
investigating it in the future, but for now I’m just diggin’ rspec/
cuke/machinist.

And yes, while the learning curve is quite steep - the rails community
has built up a huge amount of “accessible” knowlege (wikis,
screencasts, github). It’s almost feeling like a smooth ride.

12

On Aug 11, 2009, at 9:58 PM, Sijo Kg wrote:

Sijo

If you’re a shoulda fan, check out carlosbrando-remarkable
(GitHub - carlosbrando/remarkable: Simplifying tests!
). It is an implementation of (AFAICT) the shoulda macros for rSpec. I
used remarkable for controller specs, but I’m a far bigger fan of Cuke/
Webrat in place of controller specs (heresy again!). Again, this is
just me, but I put so little logic in the controller that a Cucumber
step can normally cover it.

13

Steve R. wrote:
[…]

I
used remarkable for controller specs, but I’m a far bigger fan of Cuke/
Webrat in place of controller specs (heresy again!).

I’m not sure how heretical this really is. You’re certainly not the
only person who doesn’t like controller specs – I felt guilty about my
poor controller specs until I read someone else (can’t remember who)
talking about the same thing. I now tend to agree with you that, in
many cases, Cucumber features will get you most of the controller test
coverage you need – after all, controllers are really about user-facing
interface stuff, which is exactly what Cucumber tests so well.

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]

14

Steve R. wrote:
[…]

Also consider using a mocking/
stubbing framework. rSpec comes with one baked in, or you can use
Mocha or FlexMock. Best to only mock and stub in unit tests, as
acceptance tests are supposed to exercise the whole stack.

I find that since I started using Machinist, I’ve completely stopped
using mock_model.

Of course,
if your app needs to reach out over the network or something like
that, you might have to use something like FakeWeb or a mock/stub for
the service you don’t want to hit with your test.

Yeah. Phlip (whom I generally consider a testing guru) suggests that’s
that’s the only thing to mock; then again, he also uses fixtures and
dislikes RSpec. :). OTOH, Martin F. (I think), whom I also tend to
trust, says to mock everything that isn’t what you’re directly testing.
Probably the best approach lies somewhere in between, but I don’t know
where…

Best,

Marnen Laibow-Koser
http://www.marnen.org
[email protected]