How do I best setup data for Story Runner?

Just started looking at the Story Runner integration, and am
converting a few Rails integration tests to get a feel for it.

My integration tests relied on fixtures, and since my models have a
significant amount of validation (and hence need valid data unless I
save without validation), this has made setup easy.

With stories however, I’m wondering what the recommended approach is.
Mocks are out I assume, since the point is testing the stack, but are
fixtures out too? Should we be meeting our more complex or repetitive
setup needs by treating it as plain ruby code that needs refactoring
via regular Ruby/OO techniques (Object Mother et al)?

A typical scenario for me might involve a couple of users with
different roles, plus a collection of other collaborating objects.

Thanks

On 9/26/07, Simon Peter N. [email protected] wrote:

setup needs by treating it as plain ruby code that needs refactoring

I don’t use fixtures (the yaml type, anyway) in story runner stories.
I construct objects in the given blocks. I find that that does two
things for me. First, it clearly expresses what a given really means.
So when I say Given “an activated user,” I don’t have to go dig
around some YAML files. If I need to understand it I can just look at
the code and see
u = User.create! :login => “pat”
u.activate

Secondly, constructing your object graph instead of using fixtures
means you’re actually exercising your code. With fixtures you just
instantiate some objects and fill them with data, which may not
necessarily be valid (they require maintenance). Also if you’re using
stuff like before/after create hooks to create child objects, using
fixtures bypasses that.

I look at fixtures as a weird kind of mock. You’re not using the full
implementation, so what’s the point really? I’d rather use a real
object or a proper mock object. In stories I don’t use mocks at all,
so obviously I’ll go for the full implementation.

Pat

On Sep 27, 2007, at 12:41 PM, Pat M. wrote:

fixtures out too? Should we be meeting our more complex or repetitive
http://rubyforge.org/mailman/listinfo/rspec-users
u.activate
object or a proper mock object. In stories I don’t use mocks at all,
so obviously I’ll go for the full implementation.

Pat


rspec-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/rspec-users

+1

I’ve adopted a factory pattern for organizing my tests inspired by:
http://www.dcmanges.com/blog/38

Carl

On Sep 27, 2007, at 6:11 PM, Carl P. wrote:

With stories however, I’m wondering what the recommended approach

things for me. First, it clearly expresses what a given really
instantiate some objects and fill them with data, which may not

Yeah, that post was great, and inspired me to create a plugin, which
has been essential to my workflow. Here is a screencast (and a
shameless plug) for it:

http://railsnewbie.com/files/fixture_replacement_demo.mov

I was/am quite tired (it’s about 3 AM), so just forward through my
stupidity.

Regarding Pat’s post: fixtures aren’t a weird kind of mock, they are
real data. The problem is their large overhead in scaling to a
project of any size (especially the typical one with lots of
associations). Plus, fighting with YAML just sucks. Am I putting in
invalid data, or are those tabs instead of spaces? The truth is that
they are PAINFUL to use, and that’s why no one uses them (and why
“testing” sucks, and is done after a project instead of before).

Personally, I’d love to use mocks in my model specs, but I just
can’t bring myself to do it. It, too, is painful, because I
essentially need to understand the details of how activerecord
works. Why spend an hour figuring out that
User.find_or_create_by_username ‘scott’ should be stubbed out as
User.find(“username” => “scott”) and not User.find(:username =>
“scott”)? Have you read Jay Field’s blog recently? Stubbing
ActiveRecord::Connection::Column ? Give me a break. That’s going to
break the second you upgrade rails.

Mocks are great for external libraries (or objects which you
control), but with rails everything is so tightly woven together that
mocking/stubbing seems to be more pain then it’s worth. The only
positive thing the stub approach has going for it is the speed factor.

I’d be very interested in hearing how you are using mocks
successfully without the overhead of learning tons of plumbing.

Scott

On 9/28/07, Scott T. [email protected] wrote:

significant amount of validation (and hence need valid data unless I

implementation, so what’s the point really? I’d rather use a real

I was/am quite tired (it’s about 3 AM), so just forward through my
stupidity.

Regarding Pat’s post: fixtures aren’t a weird kind of mock, they are
real data.

I had a feeling that comment would get some attention :slight_smile: I hoped I
had clarified it in my post, but apparently I didn’t.

When I say it’s a weird kind of mock, I mean that you’re basically
stubbing data without going through the entire business logic. For
example, if you have a User model such as

class User < ActiveRecord::Base
validates_uniqueness_of :login
end

and a fixture file such as

user1:
id: 1
login: pat
user2:
id: 2
login: pat

Then the fixtures will still load! Of course that’s because there are
no unique constraints in the db…but what bothers me is that Rails
just loads the fixtures like they’re existing records, instead of
something like
User.create attributes_from_fixture_file

So, what I meant was, user stories are intended to exercise the entire
application stack. Fixtures don’t do this, because even though they
preload data, they skip an important part of business logic. In that
sense, they return data without executing business logic, which is why
I consider them a strange kind of stubbed object.

works. Why spend an hour figuring out that
User.find_or_create_by_username ‘scott’ should be stubbed out as
User.find(“username” => “scott”) and not User.find(:username =>
“scott”)? Have you read Jay Field’s blog recently? Stubbing
ActiveRecord::Connection::Column ? Give me a break. That’s going to
break the second you upgrade rails.

You’re right. I’d say this is mostly a consequence of the Active
Record pattern. Basically your AR objects have two primary
responsibilities - business logic and persistence - and any time your
objects have more than one responsibility it becomes difficult to unit
test. One example that I see crop up time and again is automatically
creating associations using callbacks. Eventually you get to a point
where you save an object in a spec, and it blows up because there’s a
null association. We’re saving an object that would normally be
created by the parent, because we need to exercise some behavior that
occurs when the record is saved.

That wouldn’t be an issue if we had separate objects handling the
persistence and domain logic. When you spec your PROs (plain ruby
objects. I thought POROs sucked and I’m really digging PROs) you
might initially use mocks and then swap in real implementations. You
could do this because (a) your specs will still be super quick because
it’s all in memory and (b) you won’t have stuff blow up because you’re
trying to save stuff to the db when you didn’t really want to.

As far as I can tell, there are a couple solutions to this problem:

  1. Change the production code to handle errors that only crop up in
    testing. This sucks for what should be obvious reasons. But it’s
    actually pretty pragmatic when your logic and assocations still aren’t
    very complex. Just be sure to write a comment that says “this sucks
    but what’re ya gonna do”

  2. Refactor your code to partially separate domain logic from
    persistence logic. For example, if you have to create associated
    objects like we discussed above, move all of that into its own method.
    If I had code like this:

class Company < ActiveRecord::Base
has_many :sites
after_create :create_sites

def create_sites
%w(Production Development Testing).each { |n| sites <<
Site.new(:name => n) }
end
end

I would change it to

class Company < ActiveRecord::Base
has_many :sites

def self.create_company(options = {})
c = create options
%w(Production Development Testing).each { |n| c.sites <<
Site.new(:name => n) }
c
end
end

This way you can call Company.create in your model specs and not have
it build all the associations. For such a simple example it doesn’t
really matter much, but once you start getting more complex your specs
will get slow and brittle.

  1. Refactor your code to completely separate the domain logic from
    persistence logic. I’ve never had to do this in the 2.5 years I’ve
    been coding Rails on a daily basis. Until very, very recently, that
    is. I definitely believe in designing code to be testable, but the
    first two options have made it so that I can stick with AR with a
    little bit of smelliness. Now that we’re using the same business
    logic on top of two different persistence mechanisms, it’s time to
    make decouple the business logic and make the persistence logic
    configurable.

Story runner gives us one more option now. You can write a story that
uses the real objects, and then you can mock out associations in your
model specs. You get the benefits of knowing that your objects
integrate well and also enjoy speedy, decoupled unit tests.
Personally, despite how bright I’ve made it sound, I’m not a fan of
this option. I want my unit tests to exercise the interactions
between objects, so I just deal with it. The exception of course is
when I’m using objects from a lower layer, such as controllers
interacting with the model, in which case I always use mocks. Code
from one layer should depend only on the interface of objects in the
lower layer. When you’re dealing with objects in the same layer, it’s
useful to use real implementations because it aids refactoring a great
deal.

Mocks are great for external libraries (or objects which you
control), but with rails everything is so tightly woven together that
mocking/stubbing seems to be more pain then it’s worth. The only
positive thing the stub approach has going for it is the speed factor.

So, you’re right that mocking can be difficult when it comes to AR
associations. As I pointed out, it’s a consequence of coupling
business and persistence logic. There are some things that you can do
to offset that a bit. AR makes it more difficult to write clean
specs, but the tradeoff is that the conceptual weight of AR is
extremely low.

I’d be very interested in hearing how you are using mocks
successfully without the overhead of learning tons of plumbing.

If you could be a bit more specific about what you’re interested in, I
could perhaps shed some light. I heavily rely on mocks in a way that
I would consider successful.

Wow that became a lot longer than I expected. I hope there’s some
value in it instead of it being a steaming pile.

Pat

On 9/27/07, Carl P. [email protected] wrote:

+1

I’ve adopted a factory pattern for organizing my tests inspired by:
http://www.dcmanges.com/blog/38

+1

We have also started doing this. It is freakishly amazing and
maintainable thus far over the course of our app changing,

Scott,
Thanks for the plug-in, http://thmadb.com/public_svn/plugins/
fixture_replacement
I’ve started using it, replacing my ad hoc factory methods
And I like the idea of putting the factory in db/

btw, for the record, while the Dan M. blog was posted Aug 07, the
first time I saw this technique is as a protected create_user method
in the acts_as_authenticated plugin test/user_test.rb (although not
to replace fixtures but to test against them :slight_smile:

Regards
linoj

yes, i appreciate all the smartness built into it.

I wonder if this idea could be extended to generate mocks, eg for my
controller specs

user = mock_user( :email => “[email protected]” )
etc

not sure how far you could go with this, eg stubbing basic stuff
like .find
Just a thought

On Sep 28, 2007, at 1:20 PM, Jonathan L. wrote:

Scott,
Thanks for the plug-in, http://thmadb.com/public_svn/plugins/
fixture_replacement
I’ve started using it, replacing my ad hoc factory methods
And I like the idea of putting the factory in db/

Thanks for trying it out. I’d love to hear any feedback, or anything
that you think should be changed.

btw, for the record, while the Dan M. blog was posted Aug 07, the
first time I saw this technique is as a protected create_user method
in the acts_as_authenticated plugin test/user_test.rb (although not
to replace fixtures but to test against them :slight_smile:

Yeah, I’m sure I’ve seen that too - so maybe I stole one of the ideas
from there, as well. (Actually, I think I got it from topfunky (the
TDD peepcode), who in turn got it from technoweenie - Completely
forgot about til’ now. It’s funny how good ideas can become
unconscious like that…)

The only really cool thing about the plugin is that it uses lambdas
so that extra associations aren’t created. Plus, it dries up
the .create! and merge() that needs to be repeated over and over
again (trust me - this will get tiring if you have something like 30
models).

Regarding the lambdas: With the Dan M. factory, if I do the
following, I end up with two posts (not one):

def create_comment(hash = {})
Comment.create!(
{
:post => create_post
}.merge(hash)
)
end

this yield one associated post (as expected)

comment = create_comment

this yields an associated post, with an extra one which I don’t

care about (and which isn’t associated)
post = create_post
comment = create_comment(:post => post)

And this is because Ruby isn’t lazily evaluated. So if you end up
using the Dan M. method, beware!

Scott

On Sep 29, 2007, at 12:42 AM, Jonathan L. wrote:

Just a thought
Certainly an interesting idea. My guess is that you would need to go
a lot lower then stubbing find, Stubbing the connection parts of
ActiveRecord. (In my ignorance, it ) Looks like a big project to me.

Scott

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs