0 errors with 'autotest' BUT 30 errors with 'rake test:units

Rails
1

Hi group,

My test suites don’t behave by the book. I’ve been scratching my head
for hours in vain. I hope you can help me solve this mystery.

1/ With autotest, and
2/ when run manually, class by class

they all pass

BUT

3/ With ‘rake test:units’ + transactional fixture => 25 errors
4/ With ‘rake test:units’ withouth transactional fixture => 33 errors
(not 25)

 130 tests, 232 assertions, 3 failures, 25 errors

and
130 tests, 220 assertions, 4 failures, 33 errors

AND
IN BOTH CASEs (autotest + rake)

5/ tables/models are not empty when they should (in the setup
methods), even when I use no fixtures.

==> this setup code fails

def setup
    assert_equal 0, Production.count
end

6/ I get 2 dozens warnings like:
./test/unit/lib/grid_calendar/…/…/…/test_helper.rb:101:
warning: already initialized constant CANT_BE_BLANK

Any idea? I’m lost.

  • I use Edge Rails + simply_bdd (and a dozen other plugins)
  • All the tables are InnoDB (transactional), and I work with MySql4
  • Locomotive).
    • I can’t tell when this started happening, as I’ve been using
      autotest for weeks.

TIA

Alain

2

[Solved] or rather [Worked around] (but 2 mysteries remain)

What’s rake test:recent do?

Bingo: that’s the sleuthing tool I needed

Debriefing :

So, the problem was:
some code

  • was working fine in the application
  • was working fine in tests when run by ‘autotest’
  • was working fine in tests when run by ‘ruby test/units/…rb’
    , but
  • was failing in tests when run through ‘rake test’

Thanks to
"rake test:recent "
and touching (saving) test classes 1 by 1, I was able to single-out
the problematic code:

class Genre < ActiveRecord::Base
…
MUSIC = find_by_name(‘music’) <<< ---- RAKED TESTS DON’T LIKE
THIS (all they get is a nil)
…
end

Mystery 1:

for some reason, AR-based constants work fine, except when run through
rake tests. Why on earth ??

Mystery 2:

When you use no fixtures, nothing erases the existing data. Use
fixtures, and/or write setup code that zilches everything you
don’t
want to be there.

Shouldn’t any table changes done in the setups and tests be rollbacked
after each tests?

Even though I use
self.use_transactional_fixtures = true

, and purge the tables before launching the test
rake db:test:purge
rake test:units

this test fails:

def setup
end
def test_setup
assert_equal 0, Production.count <<------- TABLE IS NOT EMPTY
end

I still don’t understand this.

3

Alain R. wrote:

My test suites don’t behave by the book. I’ve been scratching my head
for hours in vain. I hope you can help me solve this mystery.

You don’t have isolated test cases. Data from one test case is most
likely leaking into the next one.

1/ With autotest, and
2/ when run manually, class by class

What’s rake test:recent do?

What’s ruby/test/functional/some_controller_test.rb do?

When you get that far, use -n test_my_favorite_test to run them one at a
time.

they all pass

BUT

3/ With ‘rake test:units’ + transactional fixture => 25 errors
4/ With ‘rake test:units’ withouth transactional fixture => 33 errors (not 25)

The unit tests might depend on the data left over from the functional
tests in the last run. Try rake test:functional test:units

5/ tables/models are not empty when they should (in the setup
methods), even when I use no fixtures.

When you use no fixtures, nothing erases the existing data. Use
fixtures, and/or write setup code that zilches everything you don’t
want to be there.

==> this setup code fails

def setup
assert_equal 0, Production.count
end

6/ I get 2 dozens warnings like:
./test/unit/lib/grid_calendar/…/…/…/test_helper.rb:101:
warning: already initialized constant CANT_BE_BLANK

That’s different. Autotest runs each test run within its own, single,
Ruby context. This is a bad design decision (again due to test
isolation concerns), and it should shell to a Ruby with a line like
system(“rake test:recent”). But that’s an internal affair for
Autotest, and I know what will happen if I try to discuss ZenTest with
its authors…

I think you can put unless defined? CANT_BE_BLANK after those lines…

  • I can’t tell when this started happening, as I’ve been using
    autotest for weeks.

Oops. Always run the Grand-Wazoo Test before commits, and always use
test:recent after edits in several areas to make sure their tests can
mix-and-match.

Read this; it has a link back to a simpler (stoopider) autotester that
shells to ruby instead of keeps one in memory:

http://phlip.eblogs.com/growl-driven-development

–
Phlip
http://c2.com/cgi/wiki?ZeekLand ← NOT a blog!!