I have a spec for a model that passes all tests if I run it using
textmate but I run rake some of the otherwise passed tests fail.
I would like to use autotest but I can’t trust the results.
Any ideas?
On Mon, Jul 21, 2008 at 4:33 PM, Macario O. [email protected]
wrote:
I have a spec for a model that passes all tests if I run it using
textmate but I run rake some of the otherwise passed tests fail.
AFAIK, the main difference is that rake copies the development db
structure
to the test db, whereas TextMate does not (and runs faster as a result).
I
don’t know if that explains the behavior you’re seeing.
I would like to use autotest but I can’t trust the results.
autotest also does not copy the database structure. I’ve been using it
for
six months and I’ve never noticed a problem with it. Why don’t you trust
it?
///ark
Mark W. wrote:
On Mon, Jul 21, 2008 at 4:33 PM, Macario O. [email protected]
wrote:I have a spec for a model that passes all tests if I run it using
textmate but I run rake some of the otherwise passed tests fail.AFAIK, the main difference is that rake copies the development db
structure
to the test db, whereas TextMate does not (and runs faster as a result).
I
don’t know if that explains the behavior you’re seeing.I would like to use autotest but I can’t trust the results.
autotest also does not copy the database structure. I’ve been using it
for
six months and I’ve never noticed a problem with it. Why don’t you trust
it?///ark
Well I don’t trust autotest because the same tests pass when i run them
from textmate and they fail when I run autotest.
The dubbious specs are all for a specific model but some of them are
very basic such as testing validates_presence_of in which the model code
is obviously good and yet the spec fails on rake or autotest.
So I’ve been running my specs just on textmate but I would like to use
autotest too.
On Tue, Jul 22, 2008 at 1:56 PM, Macario O. [email protected]
wrote:
Well I don’t trust autotest because the same tests pass when i run them
from textmate and they fail when I run autotest.The dubbious specs are all for a specific model but some of them are
very basic such as testing validates_presence_of in which the model code
is obviously good and yet the spec fails on rake or autotest.
I’ve had this happen before, and it turned out to be a state problem.
Things I did in one spec file were altering the state of the
application, so that spec files run after it would fail in certain
ways. Thus, passing or failing was entirely dependent on the order
the files were run in. Rake runs things in a different order than
autotest, so different tests were failing; and running a single file
by itself always passed.
In my case the state change was stupid and totally unnecessary. At
the time I didn’t really understand mocking and stubbing, so I was
trying to bypass my authentication code in my specs by reopening the
authentication modules in lib/ and overriding the live logged_in?
method to return ‘true’. This worked, but I didn’t realize the
overrides would continue to be in place for all future specs. And
quite naturally, some of the specs for restful_authentication were
failing.
I wasted hours figuring that one out. The moral I learned was, don’t
EVER try to screw with your actual application code in a spec, and
make sure that nothing you do in a spec leaves a permanent change in
memory. I can’t know if that’s your problem, of course, but it’s
something to think about.
–
Have Fun,
Steve E.
Deep Salt Team
On Mon, Jul 21, 2008 at 8:34 PM, Mark W. [email protected]
wrote:
don’t know if that explains the behavior you’re seeing.
I would like to use autotest but I can’t trust the results.
autotest also does not copy the database structure. I’ve been using it for
six months and I’ve never noticed a problem with it. Why don’t you trust it?I doubt that it’s a problem with not copy the database structure from dev
to test. That sync’s the db schema and would only be needed after a
schema
change (i.e. one or more migrations have been run), and it would be much
more likely to cause a failure in Textmate when the code under test
doesn’t
see the expected db schema.
It’s more likely that there are undeclared fixtures affecting the spec,
a
spec which needs particular state in the database might succeed or not
depending on whether or not previous specs either left data behind, or
deleted data needed by the spec in question.
When I’ve seen cases where test/specs behaved differently when run
separately vs. being run in batch (e.g. by Rake) it’s almost always
because
I left out a fixture declaration.
–
Rick DeNatale
My blog on Ruby
http://talklikeaduck.denhaven2.com/
Rick Denatale wrote:
On Mon, Jul 21, 2008 at 8:34 PM, Mark W. [email protected]
wrote:don’t know if that explains the behavior you’re seeing.
I would like to use autotest but I can’t trust the results.
autotest also does not copy the database structure. I’ve been using it for
six months and I’ve never noticed a problem with it. Why don’t you trust it?I doubt that it’s a problem with not copy the database structure from dev
to test. That sync’s the db schema and would only be needed after a
schema
change (i.e. one or more migrations have been run), and it would be much
more likely to cause a failure in Textmate when the code under test
doesn’t
see the expected db schema.It’s more likely that there are undeclared fixtures affecting the spec,
a
spec which needs particular state in the database might succeed or not
depending on whether or not previous specs either left data behind, or
deleted data needed by the spec in question.When I’ve seen cases where test/specs behaved differently when run
separately vs. being run in batch (e.g. by Rake) it’s almost always
because
I left out a fixture declaration.–
Rick DeNataleMy blog on Ruby
http://talklikeaduck.denhaven2.com/
Hi, thanks for the answer today I ran rake several times and all my test
passed (and that made me very happy) but suddenly the tests for this
model started failing again.
Prior to each test I delete all records for the model and I have no
fixture for that particular model.
I am just starting with sdd and I find fixtures to give me more trouble
than they solve so when I need to populate the database I create and
save the models I need thus I have all the dependencies.
What do you mean by undeclared fixture? a fixture with no data?
What is strange is that some validation specs (‘it should require name’)
that don’t depend on database also fail.
On Thu, Jul 24, 2008 at 12:27 PM, Macario O. [email protected]
wrote:
autotest also does not copy the database structure. I’ve been using it for
It’s more likely that there are undeclared fixtures affecting the spec,
–
Rick DeNataleMy blog on Ruby
http://talklikeaduck.denhaven2.com/Hi, thanks for the answer today I ran rake several times and all my test
passed (and that made me very happy) but suddenly the tests for this
model started failing again.
We get failures at times when running specs with rake spec that we
can’t reproduce when running in Textmate. We came to the conclusion
that it may have something to do with transactions not rolling back
before the start of the next test. So even though the models we
create should be unique (we create them with some randomness built
in), we will get validation failures when saving a new record to the
DB.
You may want to try turning off transactions with:
config.use_transactional_fixtures = false
and see if that makes a difference.
We are using Rspec 1.1.4 and PostgreSQL as our db. Let me know if that
helps.
dan
Btw, I am with sqlite3.
Macario O. wrote:
Btw, I am with sqlite3.
Hi, I think I solved the issue, I had mocha in my plugins but I was
using it in just a couple of tests and this line was commented in the
spec_helper.rb
config.mock_with :mocha
I removed mocha from my plugins folder ran rake and this two tests
failed, I fixed them and now I can see about 600 green dots and no F’s
when I run rake 
Dan Herrera wrote:
We get failures at times when running specs with rake spec that we
can’t reproduce when running in Textmate. We came to the conclusion
that it may have something to do with transactions not rolling back
before the start of the next test. So even though the models we
create should be unique (we create them with some randomness built
in), we will get validation failures when saving a new record to the
DB.You may want to try turning off transactions with:
config.use_transactional_fixtures = false
and see if that makes a difference.We are using Rspec 1.1.4 and PostgreSQL as our db. Let me know if that
helps.dan
Thanks, I’ve tried that and I got four additional failures concerning
Attachment Fu not using the expected path to save the images.
I guess the whole rspec system is very complex.