Spec_server requiring absolute paths (and autotest breaking)

Testing RSpec
1

rspec and rspec_on_rails rev 3316, ZenTest 3.9.1

When run through spec_server, running spec fails if the spec file is
not specified with an absolute path:

Probutanol:koulutusweb jarkko$ script/spec --options spec/spec.opts
pwd/spec/models/address_spec.rb
…

Finished in 0.166251 seconds

6 examples, 0 failures
Probutanol:koulutusweb jarkko$ script/spec --options spec/spec.opts
spec/models/address_spec.rb
(druby://localhost:8989) ./…/vendor/plugins/rspec/lib/spec/runner/
options.rb:197:in files_to_load': File or directory not found: spec/ models/address_spec.rb (RuntimeError) from (druby://localhost:8989) ./../vendor/plugins/rspec/lib/spec/ runner/options.rb:189:in each’
from (druby://localhost:8989) ./…/vendor/plugins/rspec/lib/spec/
runner/options.rb:189:in `files_to_load’

In the same way, autotest fails with spec_server:

Probutanol:koulutusweb jarkko$ autotest
loading autotest/rails_rspec
/System/Library/Frameworks/Ruby.framework/Versions/1.8/usr/bin/ruby -S
script/spec -O spec/spec.opts spec/controllers/
activations_controller_spec.rb spec/helpers/clients_helper_spec.rb
[a bunch of files omitted]
spec/models/enrollment_notifier_spec.rb spec/views/clients/
email_form.html.erb_spec.rb
(druby://localhost:8989) ./…/vendor/plugins/rspec/lib/spec/runner/
options.rb:5:in mtime': Not a directory - spec/models/address_spec.rb (Errno::ENOTDIR) from (druby://localhost:8989) ./../vendor/plugins/rspec/lib/spec/ runner/options.rb:5 from (druby://localhost:8989) ./../vendor/plugins/rspec/lib/spec/ runner/options.rb:251:in sort’
from (druby://localhost:8989) ./…/vendor/plugins/rspec/lib/spec/
runner/options.rb:251:in `sorted_files’

Any ideas why spec_server has lost its marbles? Without the --drb
option autotest runs fine.

Cheers,
//jarkko

–
Jarkko L.

http://www.railsecommerce.com
http://odesign.fi

2

On Feb 19, 2008 10:42 AM, Jarkko L. [email protected] wrote:

email_form.html.erb_spec.rb

Any ideas why spec_server has lost its marbles? Without the --drb
option autotest runs fine.

Did this just start happening for you? If so, it could be the patch I
applied in r3310. Would you mind checking out 3309 and seeing if you
still have this problem?

3

On 19.2.2008, at 17.57, David C. wrote:

Did this just start happening for you? If so, it could be the patch I
applied in r3310. Would you mind checking out 3309 and seeing if you
still have this problem?

That’s it. However, with 3309 it seems that autotest doesn’t really
run through drb at all since it runs the same whether or not the
spec_server is running. Yeah, I know that was the reason for the patch.

//jarkko

–
Jarkko L.

http://www.railsecommerce.com
http://odesign.fi

4

On Feb 19, 2008 12:08 PM, Jarkko L. [email protected] wrote:

That said, now that autotest runs via drb, it’s vastly (like 50%)
got: [“Barney Hops doesn’t have the required competency (MBA) to
teach this class”, “Barney Hops doesn’t have the required competency
(MBA) to teach this class”]

Mock ‘Enrollment’ expected :on_waiting_list? with (any args) twice,
but received it 3 times

There are a few drb related tickets in lighthouse. Please add this if
information if/where appropriate or add a new ticket.

Thanks,
David

5

On 19.2.2008, at 17.57, David C. wrote:

Did this just start happening for you? If so, it could be the patch I
applied in r3310. Would you mind checking out 3309 and seeing if you
still have this problem?

Ok, found out the reason. I started spec_server from within the script
directory and it only works correctly if you start it from the root of
your Rails app. Gotcha.

That said, now that autotest runs via drb, it’s vastly (like 50%)
slower than without it, doesn’t respect the --colour option in
spec.opts and causes failures that don’t happen when running just
plain autotest. All of them seem to be related to methods being run
more times than they are supposed to. Examples:

Mock ‘Location_18994’ expected :valid? with (any args) once, but
received it twice

expected: “Barney Hops doesn’t have the required competency (MBA) to
teach this class”,
got: [“Barney Hops doesn’t have the required competency (MBA) to
teach this class”, “Barney Hops doesn’t have the required competency
(MBA) to teach this class”]

Mock ‘Enrollment’ expected :on_waiting_list? with (any args) twice,
but received it 3 times

//jarkko

–
Jarkko L.

http://www.railsecommerce.com
http://odesign.fi

6

On Feb 19, 2008, at 12:08 PM, Jarkko L. wrote:

That said, now that autotest runs via drb, it’s vastly (like 50%)
slower than without it,

That’s the idea.

doesn’t respect the --colour option in spec.opts

That’s been a bug since 0.6 or 0.7 when I started with rspec. I
guess not enough people have used the spec server for it to be
already patched.

I’ll take a look at fixing it this weekend.

and causes failures that don’t happen when running just plain
autotest.

All of them seem to be related to methods being run more times than
they are supposed to. Examples:

Mock ‘Location_18994’ expected :valid? with (any args) once, but
received it twice

I saw this happen recently when the spec server was started twice:
once with the rake task (rake spec:server:start), the other with a
straight script/spec_sever. My hunch is that both servers are
loading the files at the same time.

expected: “Barney Hops doesn’t have the required competency (MBA)
to teach this class”,
got: [“Barney Hops doesn’t have the required competency (MBA)
to teach this class”, “Barney Hops doesn’t have the required
competency (MBA) to teach this class”]

Yeah - this is because rails load’s instead of requires. You’ll also
see this sort of thing if you just call load directly in your specs.
I’ve never dug deeper into how all of the rails magic happens
regarding loading and constants (and the method generation with the
association macros (has_* belongs_to), so if you have any insight,
I’d be happy to hear it.

Scott

7

On 20.2.2008, at 6.55, Scott T. wrote:

That said, now that autotest runs via drb, it’s vastly (like 50%)
slower than without it,

That’s the idea.

Isn’t the idea that specs run faster through drb, rather than slower?
However, in general my specs seem to take a lot longer when run
through autotest in the first place (compared to running just rake
spec:models, e.g.). I haven’t so far been able to track down what
causes this.

Mock ‘Location_18994’ expected :valid? with (any args) once, but
received it twice

I saw this happen recently when the spec server was started twice:
once with the rake task (rake spec:server:start), the other with a
straight script/spec_sever. My hunch is that both servers are
loading the files at the same time.

That would seem logical. However, I just checked that I only have one
spec_server process running and am still able to reproduce the problem.

regarding loading and constants (and the method generation with the
association macros (has_* belongs_to), so if you have any insight,
I’d be happy to hear it.

Yeah, all these failures are certainly related and probably caused by
something like what you describe. Got to try to figure out what
spec_server does differently than normal spec. Or maybe we should just
bite the bullet and make the deeptest spec fork work :slight_smile:

Cheers,
//jarkko

–
Jarkko L.

http://www.railsecommerce.com
http://odesign.fi

8

On Feb 20, 2008, at 1:52 AM, Jarkko L. wrote:

what causes this.
Oops. Misread your post.

Is this true running a single spec (or a single file)? I’ve noticed
that a test suite which takes about 5 seconds normally takes about 7
seconds under drb (according to the numbers reported by rspec). Then
again, running a single file is blazingly fast, as their is no wait
(as though I’m testing against a non-rails project).

one spec_server process running and am still able to reproduce the
problem.

Hmm. Are you reloading classes or using anonymous classes derived
from AR::Base, or anything crazy like that? Are all your models in
app/models? Do you have pending migrations? Are your running with
rake, or with autotest? How about running the file alone?

I’ve never dug deeper into how all of the rails magic happens
regarding loading and constants (and the method generation with the
association macros (has_* belongs_to), so if you have any insight,
I’d be happy to hear it.

Yeah, all these failures are certainly related and probably caused
by something like what you describe. Got to try to figure out what
spec_server does differently than normal spec. Or maybe we should
just bite the bullet and make the deeptest spec fork work :slight_smile:

Yeah - deeptest is great for slow suites, but now that I have a fast
suite (take 5-7 seconds), deeptest probably wouldn’t be much of a
performance gain for me.

Scott

9

On 21.2.2008, at 4.14, Scott T. wrote:

slower? However, in general my specs seem to take a lot longer when
(as though I’m testing against a non-rails project).
If I run a single, small file (like a helper spec), it’s faster in an
absolute sense (using time):

Probutanol:koulutusweb jarkko$ time script/spec -O spec/spec.opts spec/
helpers/date_extension_spec.rb
…

Finished in 2.061605 seconds

4 examples, 0 failures

real 0m2.212s
user 0m0.060s
sys 0m0.024s
Probutanol:koulutusweb jarkko$ time script/spec spec/helpers/
date_extension_spec.rb
…

Finished in 0.102506 seconds

4 examples, 0 failures

real 0m4.682s
user 0m3.738s
sys 0m0.688s

However, as you can see even there the actual running of the specs is
painfully slow (2 vs 0.1 secs) and the only reason drb is faster is
because there is no startup time. This is on 3316. Also, there really
seems to be something wonky going on with the loading. When I run a
model spec right after starting the spec server, it works, but running
it the second time already gives some weird stuff back:

Probutanol:koulutusweb jarkko$ time script/spec -O spec/spec.opts spec/
models/account_spec.rb
…

Finished in 0.371612 seconds

10 examples, 0 failures

real 0m1.647s
user 0m0.068s
sys 0m0.024s
Probutanol:koulutusweb jarkko$ time script/spec -O spec/spec.opts spec/
models/account_spec.rb
.FFF.FFFFF

ActiveRecord::AssociationTypeMismatch in ‘Account users should return
user’
Address expected, got Address

That would seem logical. However, I just checked that I only have
one spec_server process running and am still able to reproduce the
problem.

Hmm. Are you reloading classes or using anonymous classes derived
from AR::Base, or anything crazy like that? Are all your models in
app/models? Do you have pending migrations? Are your running with
rake, or with autotest? How about running the file alone?

Doesn’t matter. Shouldn’t be doing anything super special with models
but got to think about it.

I’ve never dug deeper into how all of the rails magic happens
suite (take 5-7 seconds), deeptest probably wouldn’t be much of a
performance gain for me.

Yeah, the funny thing is, model specs that sometimes need to go to the
db are pretty fast but controller specs where most of the db traffic
is stubbed out seem to become a drag, and I would think deeptest would
help there.

–
Jarkko L.

http://www.railsecommerce.com
http://odesign.fi