Testing Issue

Rails
1

I have a weird (at least for me) problem.

$rake test

Units: 50 tests, 155 assertions, 0 failures, 0 errors
Functionals: 51 tests, 122 assertions, 0 failures, 0 errors
Integration: 1 tests, 19 assertions, 0 failures, 0 errors

$rake test:functionals

51 tests, 109 assertions, 3 failures, 0 errors

$ruby test/functional/registrations_controller_test.rb

20 tests, 59 assertions, 3 failures, 0 errors

Why rake test passes everything ok and the other fails?

2

I imagine this could have something to do with test data. Check that
all the require fixtures are included in your controller test. The
data may be being loaded in other tests - hence the passes when you
run the whole suite.

Steve

3

On 7/20/07, Steve B. [email protected] wrote:

I imagine this could have something to do with test data. Check that
all the require fixtures are included in your controller test. The
data may be being loaded in other tests - hence the passes when you
run the whole suite.

Now something weirder happens:

$rake test

everything OK

$ruby test/functional/registrations_controller_test.rb

everything OK

$rake test:functionals

3 failures

$ruby test/functional/registrations_controller_test.rb

3 failures LOL

It’s a cache issue or what is happening here? i think its more like a
bug…

4

On 7/20/07, Emilio T. [email protected] wrote:

$rake test

$ruby test/functional/registrations_controller_test.rb

3 failures LOL

It’s a cache issue or what is happening here? i think its more like a bug…

I solved the problem adding missing fixtures as Steve told me, but
hey! wait a second!, this means that i always have to test my
functional tests after doing a ““complete”” test?

My conclusion is that some fixtures gets cached somewhere after
running my unit tests, that shouldn’t happen.

Is this meant to be this way or it’s a bug?

5

On Jul 20, 2007, at 11:28 AM, Emilio T. wrote:

51 tests, 109 assertions, 3 failures, 0 errors

$ruby test/functional/registrations_controller_test.rb

20 tests, 59 assertions, 3 failures, 0 errors

Why rake test passes everything ok and the other fails?

You are likely using fixtures, right? If you look at the fixtures
loaded for your RegistrationsControllerTest, be sure that all the
fixture expectations are listed in that file. I’ve run into this
very problem before. There are some who will recommend all fixtures
be listed in the test_helper.rb, but I think that’s just an excuse
not to stop and think about which fixtures you really need for a test.

-Rob

Rob B. http://agileconsultingllc.com
[email protected]

6

Emilio T. wrote:

$ruby test/functional/registrations_controller_test.rb

next, try:

$ruby test/functional/registrations_controller_test.rb -n
test_one_specific_test

I solved the problem adding missing fixtures as Steve told me, but
hey! wait a second!, this means that i always have to test my
functional tests after doing a ““complete”” test?

I interpret this to mean “each time I change any test, must I run all
the tests independently to verify they isolate correctly?”

The general answer is no - testing the tests is the road to madness.
Your production code never had a bug here, and tests should fail
more_easily than production code gets bugs.

However, yes, each time you add a fixture to a test case you risk
breaking test isolation with other test cases. A good way to catch
this is test with rake test:recent or rake test:svn_modified, each
time you change the code. That way, as you change different
combinations of test cases, you get to see if they (apparently)
isolate from each other, independent of the group. Many a Rails
application can no longer run a partial test because only the total
test worked when they integrated.

My conclusion is that some fixtures gets cached somewhere after
running my unit tests, that shouldn’t happen.

Yes, in the database. (-;

Is this meant to be this way or it’s a bug?

You are lucky to have test fixtures. Some systems make them so slow,
their programmers must avoid database calls in most tests! So any
residual optimizations here are generally to your benefit. Perfect
test isolation would restart and rebuild the database between each
test run. We rely on less than perfect isolation in exchange for
speed.

–
Phlip
http://www.oreilly.com/catalog/9780596510657/
“Test Driven Ajax (on Rails)”
assert_xpath, assert_javascript, & assert_ajax