Specs pass individually, but fail using autotest

Hi,
I’m trying to run autotest and I’m using ryanb’s nifty authentication
with Rails 3 and Rpec 2. When I run autotest, the user model spec
fails all its validation tests with the “error_on” matcher.

If I run the spec by itself, it passes. If I’m running autotest and
make a trivial change to the spec and save it, so that autotest runs
just the user spec by itself, it passes. All the other tests in the
spec pass. If I hit Ctrl-C to rerun the tests, these tests will fail
again.

Only the ones of the form “new_user(:password => ‘bad’).should
have(1).error_on(:password)” fail.

Below is the output from autotest, including the initial run, and
second run after I make a trivial change to the spec.

Thanks.
Doug

Failures:

  1. User should require username
    Failure/Error: new_user(:username => ‘’).should
    have(1).error_on(:username)
    expected 1 error on :username, got 0

    ./spec/models/user_spec.rb:21:in `block (2 levels) in <top

(required)>’

  1. User should require password
    Failure/Error: new_user(:password => ‘’).should
    have(1).error_on(:password)
    expected 1 error on :password, got 0

    ./spec/models/user_spec.rb:25:in `block (2 levels) in <top

(required)>’

  1. User should require well formed email
    Failure/Error: new_user(:email => ‘[email protected][email protected]’).should
    have(1).error_on(:email)
    expected 1 error on :email, got 0

    ./spec/models/user_spec.rb:29:in `block (2 levels) in <top

(required)>’

  1. User should validate uniqueness of email
    Failure/Error: new_user(:email => ‘[email protected]’).should
    have(1).error_on(:email)
    expected 1 error on :email, got 0

    ./spec/models/user_spec.rb:34:in `block (2 levels) in <top

(required)>’

  1. User should validate uniqueness of username
    Failure/Error: new_user(:username => ‘uniquename’).should
    have(1).error_on(:username)
    expected 1 error on :username, got 0

    ./spec/models/user_spec.rb:39:in `block (2 levels) in <top

(required)>’

  1. User should not allow odd characters in username
    Failure/Error: new_user(:username => ‘odd ^&(@)’).should
    have(1).error_on(:username)
    expected 1 error on :username, got 0

    ./spec/models/user_spec.rb:43:in `block (2 levels) in <top

(required)>’

  1. User should validate password is longer than 3 characters
    Failure/Error: new_user(:password => ‘bad’).should
    have(1).error_on(:password)
    expected 1 error on :password, got 0

    ./spec/models/user_spec.rb:47:in `block (2 levels) in <top

(required)>’

  1. User should require matching password confirmation
    Failure/Error: new_user(:password_confirmation =>
    ‘nonmatching’).should have(1).error_on(:password)
    expected 1 error on :password, got 0

    ./spec/models/user_spec.rb:51:in `block (2 levels) in <top

(required)>’

Finished in 3.46 seconds
111 examples, 8 failures, 4 pending
bundle exec /Users/doug/.rvm/rubies/ruby-1.9.2-p0/bin/ruby -S /Users/
doug/.rvm/gems/[email protected]/gems/rspec-core-2.0.1/bin/
rspec --autotest ‘/Users/doug/dev/hg/class-proto/spec/models/
user_spec.rb’
…Trivial change

Finished in 1.17 seconds
14 examples, 0 failures

On Nov 20, 8:43pm, “Doug E.” [email protected] wrote:

have(1).error_on(:password)

expected 1 error on :username, got 0
7) User should validate password is longer than 3 characters

./spec/models/user_spec.rb:51:in `block (2 levels) in <top

Finished in 1.17 seconds
14 examples, 0 failures


rspec-users mailing list
[email protected]://rubyforge.org/mailman/listinfo/rspec-users

I’ll add that this behavior only occurs with the model User. I set up
some similar tests for other classes (named Sample and
SampleClassification). These do not have the same problem. Their
validation tests pass during batch and single test runs.

I created a new User model from scratch with a single
validates_presence_of call, and got the same unexpected behavior: in
single spec runs they pass, and in batch runs they fail.

Doug

Problem discovered, but not solved. It isn’t an rspec problem. It’s
caused by nifty-authentication’s use of Mocha’s any_instance.
users_controller_spec.rb’s last test calls:

User.any_instance.stubs(:valid?).and_returns(true)

This is fine if the tests are run singly, but with autotest the call
in the controller spec still affects every subsequent User instance,
and so it’s impossible later in the same autotest batch to test the
validity of later user objects. They just return true.

An aside: It seems that mocha should make it possible to undo the
effect of any_instance.stubs(:sym).

On Nov 22, 12:02am, David C. [email protected] wrote:

On Nov 20, 2010, at 9:49 PM, Doug E. wrote:

Problem discovered, but not solved. It isn’t an rspec problem. It’s
caused by nifty-authentication’s use of Mocha’s any_instance.
users_controller_spec.rb’s last test calls:

User.any_instance.stubs(:valid?).and_returns(true)

This is fine if the tests are run singly, but with autotest the call
in the controller spec still affects every subsequent User instance,
and so it’s impossible later in the same autotest batch to test the
validity of later user objects. They just return true.

An aside: It seems that mocha should make it possible to undo the
effect of any_instance.stubs(:sym).

AFAIK, this should happen automatically, assuming your suite is configured to
use mocha. Do you have config.mock_with(:mocha) (usually in
‘spec/spec_helper.rb’)?

Thanks, David. I rewrote the tests to use rspec mocks. All the other
tests already use them.

I’m learning rspec 2, and am trying to stick strictly with rspec tests
and mocks. In this application, I’ve replaced the mocha based specs
created by the nifty scaffold generator with rspec scaffold specs.
Rspec, of course, doesn’t have a generator parallel to the nifty
authentication generator, so I left those tests in place, not knowing
that the two would not play nice together. Now that I understand
what’s happened, I’ve removed mocha and rewritten mocha-based specs.

Doug

On Nov 20, 2010, at 9:49 PM, Doug E. wrote:

An aside: It seems that mocha should make it possible to undo the
effect of any_instance.stubs(:sym).

AFAIK, this should happen automatically, assuming your suite is
configured to use mocha. Do you have config.mock_with(:mocha) (usually
in ‘spec/spec_helper.rb’)?

again.
Failures:
expected 1 error on :password, got 0
4) User should validate uniqueness of email
# ./spec/models/user_spec.rb:39:in `block (2 levels) in <top
Failure/Error: new_user(:password => ‘bad’).should
(required)>’
Finished in 1.17 seconds
I created a new User model from scratch with a single
http://rubyforge.org/mailman/listinfo/rspec-users
Cheers,
David