We are trying to automate the running of our rspec tests for our Rails
app
on a build server using Capistrano. The problem is that Capistrano
seems to
think that the command called to run the model tests failed when in fact
I
believe it succeeded. I believe the problem has to do with the return
code
from the command I am running… The command I am running is:
ruby script/spec spec/models -f html:model_test_results.html
The output file seems to be complete as it has both the opening and
closing
html tags and it has the results of all of our tests. But when I run $?
To
get the return code of the last command it says it was “1” which I
believe
signifies that the command was not successful and is what is causing our
Capistrano task to abort. We are using RSpec 1.1.4 and in case you’re
curious the output from Capistrano looks like this:
- executing “cd /data/nightingalenotes/releases/20080812160703 && ruby
script/
spec spec/models -f
html:public/rspec_test_results/model_test_results.html”
servers: [“buildnexus.champ.net”]
[buildnexus.champ.net] executing command
command finished
*** [deploy:update_code] rolling back
- executing “rm -rf /data/nightingalenotes/releases/20080812160703;
true”
servers: [“buildnexus.champ.net”]
[buildnexus.champ.net] executing command
command finished
command “cd /data/nightingalenotes/releases/20080812160703 && ruby
script/spec s
pec/models -f html:public/rspec_test_results/model_test_results.html”
failed
on
buildnexus.champ.net
Thanks!
Ben F.
On Tue, Aug 12, 2008 at 1:22 PM, Ben F. [email protected]
wrote:
The output file seems to be complete as it has both the opening and closing
html tags and it has the results of all of our tests. But when I run $? To
get the return code of the last command it says it was “1” which I believe
signifies that the command was not successful and is what is causing our
Capistrano task to abort.
I just did this in a Rakefile:
task :models do
ruby script/spec spec/models -f html:model_test_results.html
puts $?
end
And I get this when I run ‘rake models’
$ rake models
(in /Users/david/projects/ruby/mycastory)
0
I’m using the latest from git (naturally).
How are you checking the exit code?
The command I was running to get the return code was “echo $?” After
reading
over my email it seems I left out a bit of rather important information.
You see we expect the command to return a return code of “0” if the
command
was successful in it’s task which was to generate an html file with the
results of running the model tests. In our case the command is
successful
and it generates the desired output and puts it into the HTML file as
expected. However we find that the return code is changed to “1” if any
single test fails. To me this seems incorrect because the result of a
test
should not influence the return code as the return code should be
associated
with whether or not the command executed successfully or not.
Our current situation is a great example of why the return code should
only
represent the success or failure of the command and not the results of
the
command. Whenever a test fails the return code of “1” causes our build
to
break and therefore nobody even knows a test is failing until we notice
that
the build server is no longer updating on regular intervals.
Ben F.