Forum: RSpec autospec runs failing specs forever in endless loop

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Ff97ca87af59ee68ceff5877a8365788?d=identicon&s=25 Jarmo Pertman (juuser)
on 2009-05-31 23:01
(Received via mailing list)
I wanted to start using autospec on my little (non-Rails) project. I'm
on Windows XP and have used it some time ago successfully (on the same
PC), although I've updated many gems in the meantime (RSpec for
example among others).

In short, everything is fine when all specs pass, but if some fails,
then autospec won't stop running failing specs again even if no files
are changed. I expect it to run specs again only if some file gets
modified. It's kinda irritating when it doesn't stop and is by no
means helping any development :/

Also, when I fix the failing spec then it will run it only once and
then run whole suit once and stops as expected.

Any ideas what might cause this distracting behaviour?

BR,
Jarmo
5d38ab152e1e3e219512a9859fcd93af?d=identicon&s=25 David Chelimsky (Guest)
on 2009-05-31 23:59
(Received via mailing list)
On Sun, May 31, 2009 at 3:59 PM, Jarmo Pertman <Jarmo.P@gmail.com>
wrote:
>
> Also, when I fix the failing spec then it will run it only once and
> then run whole suit once and stops as expected.
>
> Any ideas what might cause this distracting behaviour?

Do you have "--format failing_examples" in spec/spec.opts? If so,
rspec saves a file with a list of the failing examples in it, which
would trigger autotest to run again unless that file is explicitly
ignored.

HTH,
David
Ff97ca87af59ee68ceff5877a8365788?d=identicon&s=25 Jarmo Pertman (juuser)
on 2009-06-01 23:14
(Received via mailing list)
No, I didn't have that one, but had one similar problem where one of
my specs edited one non-Ruby file, which caused autotest to reload. I
solved that problem by explicitly ignoring it in autotest. I was
surprised that autotest cares about non-mapped files :)

But now I have another problem - it seems that autotest doesn't
understand always when something really failed and it seems to be due
to the fact that autotest.rb has this line:
self.failed_results_re = /^\s+\d+\) (?:Failure|Error):\n(.*?)\((.*?)
\)/

It doesn't match correctly with most of the failing specs so Snarl
(like Growl for Windows) shows that tests are passing when they're
actually not. So, I digged around a little more to see if RSpec
modifies this regular expression somewhere and I found it at lib/
autotest/rspec.rb and there is this class Autotest::Rspec < Autotest,
which overwrites failed_results_re as: self.failed_results_re = /^\d+\)
\n(?:\e\[\d*m)?(?:.*?in )?'([^\n]*)'(?: FAILED)?(?:\e\[\d*m)?\n(.*?)\n
\n/m

This autotest thing seems to be quite strange, because if I added some
puts statements into discover.rb and rspec.rb then they didn't seem to
get executed, BUT if I putsed this failed_results_re in autotest.rb
itself, then it was set correctly as written above.

My .autotest file is as following:
require 'autotest/menu'
require 'autotest/snarl'

Autotest.add_hook :initialize do |at|
  at.add_exception(/\.sqlite/)
    at.add_exception(/^\.git/)
end

And spec.opts is like this:
-r spec/env.rb
-c

And I'm running autospec and seeing this output:
1)
'correct year' FAILED

expected "2008"
     got "2009"

(compared using eql?)

./spec/year_spec.rb:38:

Finished in 5.918 seconds

11 examples, 1 failure

And when I added to autotest.rb line 410 this: p self.files_to_test
Then the resulted hash was empty, e.g. this line sets result to
"green": color = completed &&
self.files_to_test.empty? ? :green : :red

When I break some other test with different error message like this,
then Snarl shows me failing tests:
1)
NoMethodError in 'DatabaseOperations adds options'
undefined method `[]' for nil:NilClass
./spec/database_operations_spec.rb:50:

The difference is that failing reason is shown on one line, instead of
two lines. Also, p elf.files_to_test results in "normal" result:
{"spec/database_operations_spec.rb"=>["DatabaseOperations adds
options", "DatabaseOperations adds new default_options if some are
missing"]}

- Is it possible that this regular expression is little outdated and
can't handle the situation where failing reason is shown on 2 lines?
- Why won't anything puts'ed from autotest/discover.rb or autotest/
rspec.rb like they aren't loaded?
- Any other reasons why autotest behaviour might be broken?

Jarmo
Ff97ca87af59ee68ceff5877a8365788?d=identicon&s=25 Jarmo Pertman (juuser)
on 2009-06-02 18:23
(Received via mailing list)
Ok. I've created failing example.

Just create project directory and spec directory into it and add there
one spec file:

describe "autospec" do

  it "fails" do
    "hi".should eql("hello")
  end

end

It fails, but for autotest it is still green (thus making growl and
snarl to report as everything is passing) and now I'm pretty sure that
it has to do with the fact that "should eql" outputs "expected" and
"got" result on different lines. As soon as you replace the matcher
to: "hi".eql?("hello").should be_true - then output is on single line
and autotest detects it as failing.

Still, problem is with the regular expression provided for autotest?

Jarmo
5d38ab152e1e3e219512a9859fcd93af?d=identicon&s=25 David Chelimsky (Guest)
on 2009-06-02 18:53
(Received via mailing list)
On Tue, Jun 2, 2009 at 11:07 AM, Jarmo Pertman <Jarmo.P@gmail.com>
wrote:
>
> end
>
> It fails, but for autotest it is still green (thus making growl and
> snarl to report as everything is passing) and now I'm pretty sure that
> it has to do with the fact that "should eql" outputs "expected" and
> "got" result on different lines. As soon as you replace the matcher
> to: "hi".eql?("hello").should be_true - then output is on single line
> and autotest detects it as failing.
>
> Still, problem is with the regular expression provided for autotest?

Seems likely. Wanna make a proper patch w/ specs?
Ff97ca87af59ee68ceff5877a8365788?d=identicon&s=25 Jarmo Pertman (juuser)
on 2009-06-02 19:35
(Received via mailing list)
Done at
https://rspec.lighthouseapp.com/projects/5645-rspe...

I started even before you've replied, because that bugged me too
much :)

Jarmo
This topic is locked and can not be replied to.