Testing with activerecord at ThoughtWorks (Stubbing vs real DB)

Testing RSpec
1

I am reading the post by Fowler, ‘Ruby at ThoughtWorks’.
http://martinfowler.com/articl/rubyAtThoughtWorks.html#WasRubyTheRightChoice

He talks about testing with activerecord:

"Right at the beginning of our use of Ruby, there was a debate on how
best to organize testing in the presence of the Active Record database
layer in Rails. The basic problem is that most of the time,
performance of enterprise applications is dominated by database
access. We’ve found that by using a Test Double we can greatly speed
up our tests. Having fast tests is crucial to our test-intensive
development process. Kent Beck recommends a basic commit build of
under ten minutes. Most of our projects manage this these days, and
using a database double is a vital part of achieving it.

The problem with Active Record is that by combining database access
code with business logic, it’s rather harder to create a database
double. The Mingle team’s reaction to this was to accept that Rails
binds the database tightly and thus run all the commit tests against a
real database.

The contrary view was advocated most firmly by the Atlanta and Jersey
teams. Ruby has a powerful feature that allows you to redefine methods
at run-time. You can use this to take an active record class, and
redefine the the database access methods in that class as stubs. The
team started the gem unitrecord to help with this.

In the three years, we’ve not seen a generally accepted victor in this
debate. The Mingle team run a couple of thousand tests against a real
postgres database in around 8 minutes. (They parallelize the tests to
make use of multiple cores.) The Atlanta and Jersey teams consider it
valuable that their commit test runs in 2 minutes with stubs versus 8
minutes without. The trade-off is the simplicity of the direct
database tests versus the faster commit build of the stubbed tests.

While both teams are broadly happy with their positions in this
debate, the use of stubbing has led to another issue for the Atlanta/
Jersey teams. As the teams became familiar with using method stubbing,
they used it more and more - falling into the inevitable over-usage
where unit tests would stub out every method other than the one being
tested. The problem here, as often with using doubles, is brittle
tests. As you change the behavior of the application, you also have to
change lots of doubles that are mimicking the old behavior. This over-
usage has led both teams to move away from stubbed unit tests and to
use more rails-style functional tests with direct database access."

What do you think of the two tactics?
Also, What does it mean and where can I see examples of “running all
the commit tests against a real database” ?

Can I find it in “the rspec book” ?

Thanks!

2

oren wrote:

up our tests. Having fast tests is crucial to our test-intensive
The contrary view was advocated most firmly by the Atlanta and Jersey
minutes without. The trade-off is the simplicity of the direct
usage has led both teams to move away from stubbed unit tests and to
use more rails-style functional tests with direct database access."

What do you think of the two tactics?

I have done both on past projects. I did not use any of the TW gems to
do it. I used Avdi G.'s nice NullDB plugin to do it. IMO it is a
much better and cleaner approach than the ones talked about in the
article. I gave a lightning talk about the tradeoffs of both approaches
and how I used NullDB in my project. Here are the slides for the talk:

The slides go through the benefits and tradeoffs of both approaches.
Let me know if you want me to elaborate on anything else. I should
point out that another, but less extreme, solution to stubbing out the
DB is to use an in-memory DB (like sqllite) to get a speed boost. With
this approach you may still need to switch to your real DB adapter for
DB specific SQL but it all the basic AR stuff with work just fine.

Also, What does it mean and where can I see examples of “running all
the commit tests against a real database” ?

Sounds like they have configured there project to to allow you to hot
swap which strategy to use. So, for development you can use the
stubbing approach and get faster feedback and then swap in the real
database on the CI server to make sure there aren’t any surprises when
it is actually executed against the DB. This is a good approach and can
be done quite easily with NullDB.

Can I find it in “the rspec book” ?

I don’t think so. To be honest, I don’t think it is done that much. I
think most teams just let there model tests hit the DB. (And thus are
not “real” unit tests by some people’s definition…)

-Ben

3

Awesome presentation, thanks for clarifying it Ben.