Failing rspec story not causing a CruiseC ontrol.rb build to fail?

Testing RSpec
1

We’ve just added rspec stories to our CruiseControl.rb build. When
there’s a failing scenario we see the failure in the log output, but it
doesn’t cause the CruiseControl build to fail. We’ve also just upgraded
to the latest rspec version, 1.1.4, which fixes rspec bug 228, but the
CC.rb build is still passing.

Has anyone got failing rspec stories making CruiseControl.rb fail?

Thanks,
Sean M…

2

On Tue, May 27, 2008 at 8:41 AM, Sean M. [email protected] wrote:

http://rubyforge.org/mailman/listinfo/rspec-users

I wasn’t aware that CC.rb integrated with story runner yet. Are you
sure it does?

Pat

3

Pat M. wrote:

rspec-users mailing list
rspec-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/rspec-users

To integrate with CC.rb all is that required is that your build
task/process exits with the appropriate code (0 or non-zero for
failing.) I originally made the patch so that we could integrate it
with CC.rb…

Sean, if you run the stories from the command line and it fails what is
the exit code? (run the stories and type ‘echo $?’ to find out.) If
you can verify that the exit code is correct then the problem might be
with your cc.rb rake task. Maybe you could post it? Also, what version
of ruby are you using? I have heard that a recent version of ruby had
an at_exit bug, so try it on different versions of ruby to see if it is
environment specific.

-Ben

4

Thanks! The problem seems to be that the rake spec task is messing up
the exit code of the story spec, but I still can’t see how.

When I run the failing story spec outside of a rake task (ruby
stories/all.rb), I get the correct exit code of 1.

When I run it in rake spec:stories, I get the incorrect exit code of 0.

namespace :spec do

desc “run spec user stories”
Spec::Rake::SpecTask.new(‘stories’) do |t|
t.spec_files = FileList[‘stories/**/*_stories.rb’]
t.rcov = true
output_root = ENV[“CC_BUILD_ARTIFACTS”] || “spec”
t.rcov_dir = “#{output_root}/user_stories_coverage”
t.rcov_opts = RCOV_OPTIONS
end

But when I make one of the non-story specs fail, I get the exit code of
1 both running the spec outside of rake and as part of “rake
spec:coverage”, and the rake SpecTask is defined the same way:

desc “Run all examples with RCov”
Spec::Rake::SpecTask.new(‘coverage’) do |t|
t.spec_files = full_spec_file_list
t.rcov = true
output_root = ENV[“CC_BUILD_ARTIFACTS”] || “spec”
t.rcov_dir = “#{output_root}/rspec_coverage”
t.rcov_opts = RCOV_OPTIONS
end

Any ideas?

Thanks,
Sean.

5

Sean,
This is what I have on a project that is running CC.rb with stories:

desc “Task for cruise Control”
task :cruise do
RAILS_ENV = ENV[‘RAILS_ENV’] = ‘test’ # Without this, it will drop
your production database.
CruiseControl::invoke_rake_task ‘db:reset_all’
CruiseControl::invoke_rake_task ‘cruise_coverage’
CruiseControl::invoke_rake_task ‘stories:all’
sh “rake kablame:git:specs >
#{ENV[‘CC_BUILD_ARTIFACTS’]}/hall_of_shame.txt”
end

namespace :stories do
desc “Run all stories (that are not pending)”
task :all do
ruby “stories/all.rb --format plain”
end
end

I think your problem is that you are using a SpecTask to run the
stories… I don’t know if that is supported. I just open up a new ruby
process to run the stories.
Hope that helps,
Ben