Basically the Jay Fields style of testing – I want the unit tests to
be run all the time on a continuous integration server, but the
integration/functional and system tests to be run only once a night.
How do I break out the rspec tasks so that they cover this heirarchy?
It looks like they’ll take a pattern, but I don’t want to have to
redefine every rake task just to specify this.
Also, anyone know how hard this would be to maintain? Or if there’s
an easier way to organize this?
David just posted about the new --patterns option in the trunk. His
examples
don’t include changing the directory structure, but I would think that
it
would allow that. If not, that would be a good patch.
Basically the Jay Fields style of testing – I want the unit tests to
be run all the time on a continuous integration server, but the
integration/functional and system tests to be run only once a night.
I would suggest letting autotest run your unit tests every couple
minutes, and have your CI server run the functional/integration tests.
Much tighter feedback loop.
I’d go one further than Pat M.'s point: If your unit specs aren’t
going to touch the database, then you really ought to be running a
tier of more integrated tests on your workstation before every checkin
(and again on the CI server). For obvious reasons, purely mock-based
unit tests/specs can and eventually will lie to you about your objects
actually being ready to work together, so don’t trust them to tell you
about more than what the unit being spec’d does.
I like the idea of running integration tests before every checkin, but
I’ve heard too many horror stories of it taking 30 minutes or more
before a checkin actually happens. We have integration tests that are
going out to external systems, so it’s a single example set can take a
full minute – and this is right now.
I very much want integration tests to be run on the CI server on every
checkin. The last place I set this up has a test suite that last for
90 minutes though, so it’s very much a correctness vs convenience
thing.
Incidentally, when is cruisecontrol.rb going to have a ‘testing’ tab
that shows passed and failed tests like TextMate? The stack trace is ugly.
rails environment, which shaves off a second or two. (That second or
two isn’t a big deal when running rake, but it’s great in TextMate
when running just one spec. It feels pretty much instantaneous.)
That sound really good. I like TextMate’s integration with RSpec.
I’m just using Unitrecord right now and mocking access to db
operations in the unit tests. It’s working fine for me so far. I’m
using fixture_replacement for the integration tests. I read the
Factory pattern that Jay Fields is implementing, but I’m not sure it’s
worth the effort to set up.
One thing I have noticed is that moving the specs from the normal
directory structure means that I have to specify the type directly,
i.e.
describe FooController, 'unit tests: ', :type => :controller do
Will,
I wanted to do something similar. To avoid losing all the tasks and
other goodness Spec::Rails gives you for free, I just created a
separate top-level directory.
/spec
/models
/controllers
…
/unit
/models
…
(It bugs me not having it all contained in one directory, but it’s
consistent with the fact that there’s also a top-level /stories
folder. More importantly, I didn’t have to modify any rspec_on_rails
tasks.)
I’d go one further than Pat M.'s point: If your unit specs aren’t
going to touch the database, then you really ought to be running a
tier of more integrated tests on your workstation before every checkin
(and again on the CI server). For obvious reasons, purely mock-based
unit tests/specs can and eventually will lie to you about your objects
actually being ready to work together, so don’t trust them to tell you
about more than what the unit being spec’d does.
Our commit rake task looks something like this:
svn up
run unit specs
bring the dev database up to date (we blow it away and rebuild from
scratch)
spec (the Spec::Rails task, which depends on db:test:prepare,
cloning the test db from the dev schema)
In the /unit spec_helper, we load the rspec plugin, but not
rspec_on_rails, ARBS in place of ActiveRecord and (for now at least)
require ActiveSupport and the ActionPack but don’t actually load the
rails environment, which shaves off a second or two. (That second or
two isn’t a big deal when running rake, but it’s great in TextMate
when running just one spec. It feels pretty much instantaneous.)
I’ve heard too many horror stories of it taking 30 minutes or more
that shows passed and failed tests like TextMate? The stack trace is ugly.
Are you just talking about the HTML formatted specs that textmate uses?
You can already get that with cruisecontrol.rb… All you have to do is
have tech spec command output a html file.
-Ben
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.