Possible Problem with RSpec and

Testing RSpec
#1

Sorry to be such a pest but I am trying to learn Ruby, Rails and RSpec
all
at one go and it is a bit overwhelming. I have previously completed the
depot tutorial in the Agile Web Dev with rails book and now I am trying
do
do it again using RSpec.

What I would like to know now is why I am getting a rake failure error
at
the end of every spec:models run. Is this the expected behaviour when a
test / expectation fails?

—>

C:\var\RSpec\depot>rake spec:models
(in C:/var/RSpec/depot)
NOTICE: CREATE TABLE will create implicit sequence “products_id_seq”
for
serial
column “products.id”
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index
“products_pkey” f
or table “products”

Product

  • should have a valid image uri (PENDING: Not Yet Implemented)
  • should have a description
  • should have a title (FAILED - 1)

Pending:
Product should have a valid image uri (Not Yet Implemented)

‘Product should have a title’ FAILED
expected valid? to return true, got false
./spec/models/product_spec.rb:11:

Finished in 0.75 seconds

3 examples, 1 failure, 1 pending
rake aborted!
Command ruby -I
“C:/var/RSpec/depot/vendor/plugins/rspec/lib”
“C:/var/RSpec/depot/vendor/plugins/rspec/bin/spec”
“spec/models/product_spec.rb”
–options
“C:/var/RSpec/depot/config/…/spec/spec.opts” failed

(See full trace by running task with --trace)

C:\var\RSpec\depot>rake spec:models --trace
(in C:/var/RSpec/depot)
** Invoke spec:models (first_time)
** Invoke db:test:prepare (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute db:test:prepare
** Invoke db:test:clone (first_time)
** Invoke db:schema:dump (first_time)
** Invoke environment
** Execute db:schema:dump
** Invoke db:test:purge (first_time)
** Invoke environment
** Execute db:test:purge
** Execute db:test:clone
** Invoke db:schema:load (first_time)
** Invoke environment
** Execute db:schema:load
NOTICE: CREATE TABLE will create implicit sequence “products_id_seq”
for
serial column “products.id”
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index
“products_pkey” for table “products”
** Execute spec:models

Product

  • should have a valid image uri (PENDING: Not Yet Implemented)
  • should have a description
  • should have a title (FAILED - 1)

Pending:
Product should have a valid image uri (Not Yet Implemented)

‘Product should have a title’ FAILED
expected valid? to return true, got false
./spec/models/product_spec.rb:11:

Finished in 0.563 seconds

3 examples, 1 failure, 1 pending
rake aborted!
Command ruby -I"C:/var/RSpec/depot/vendor/plugins/rspec/lib"
“C:/var/RSpec/depot/vendor/plugins/rspec/bin/spec” “spec/models/
product_spec.rb” --options “C:/var/RSpec/depot/config/…/spec/spec.opts”
failed
C:/var/RSpec/depot/vendor/plugins/rspec/lib/spec/rake/
spectask.rb:173:in define' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:823:inverbose’
C:/var/RSpec/depot/vendor/plugins/rspec/lib/spec/rake/
spectask.rb:142:in define' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:392:incall’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/
rake.rb:392:in execute' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:392:ineach’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/
rake.rb:392:in execute' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:362:ininvoke’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/
rake.rb:355:in synchronize' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:355:ininvoke’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/
rake.rb:1739:in top_level' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:1739:ineach’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/
rake.rb:1739:in top_level' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:1761:instandard_exception_handling’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/
rake.rb:1733:in top_level' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:1711:inrun’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/
rake.rb:1761:in standard_exception_handling' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:1708:inrun’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/bin/rake:7
C:/usr/local/bin/ruby/bin/rake.bat:20:in `load’
C:/usr/local/bin/ruby/bin/rake.bat:20

C:\var\RSpec\depot>
<—

–
*** E-Mail is NOT a SECURE channel ***
James B. Byrne mailto:[email protected]
Harte & Lyne Limited http://www.harte-lyne.ca
9 Brockley Drive vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada L8E 3C3

#2

On Nov 24, 2007, at 7:48 PM, James B. Byrne wrote:

the end of every spec:models run. Is this the expected behaviour
when a
test / expectation fails?

What database are you using? What verison of: rails, ruby, database
engine (mysql, sqlite…), rspec? What version of windows are you
running?

It would probably also help if we could see your code, as well as
your failing spec.

Scott.

#3

On Nov 24, 2007 9:48 PM, James B. Byrne [email protected] wrote:

Sorry to be such a pest but I am trying to learn Ruby, Rails and RSpec all
at one go and it is a bit overwhelming. I have previously completed the
depot tutorial in the Agile Web Dev with rails book and now I am trying do
do it again using RSpec.

What I would like to know now is why I am getting a rake failure error at
the end of every spec:models run. Is this the expected behaviour when a
test / expectation fails?

You didn’t pasted the code of your specs… only the backtrace…

Please include the spec/models/product_spec.rb code here so we can see
your problem.

Also, it seems your validations aren’t right, since you’re evaluating
“be_valid” for product after title being set, but #valid? is returning
false.

Maybe you’re “asking” the wrong question to your model?
(not tested code)

it “should have a title” do
@product = Product.new(:title => nil)
@product.should_not be_valid
@product.should have_at_least(1).errors_on(:title)
end

also, versions of ruby, rails, rspec and database engine are helpful too
:wink:

–
Luis L.
Multimedia systems

Leaders are made, they are not born. They are made by hard effort,
which is the price which all of us must pay to achieve any goal that
is worthwhile.
Vince Lombardi

#4

On Sat, November 24, 2007 19:48, James B. Byrne wrote:

What I would like to know now is why I am getting a rake failure error at
the end of every spec:models run. Is this the expected behaviour when a
test / expectation fails?

Sorry about my delay in replying. I am a digest subscriber ordinarily
and
did not switch my list preferences when I posted as I usually do.
Thanks
for the suggestions.

The problem was that I left out a column from the product migration that
I
referred to in the spec.

Regards,

–
*** E-Mail is NOT a SECURE channel ***
James B. Byrne mailto:[email protected]
Harte & Lyne Limited http://www.harte-lyne.ca
9 Brockley Drive vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada L8E 3C3

#5

On Sat, November 24, 2007 20:48, James B. Byrne wrote:

The problem was that I left out a column from the product migration that I
referred to in the spec.

Sorry, but I need to revisit this again. If the specs pass then I get
no
rake error. If one spec fails then after the test reports I get a rake
error and back trace. Why?

OS = MS XP pro SP2
Ruby = 1.8.6
Rails = 1.2.5
RSpec = TRUNK d.20071123

product_spec.rb

—>
require File.dirname(FILE) + ‘/…/spec_helper’

module ProductSpecHelper

def valid_product_attributes
{
:code => ‘PROD1234CODE’,
:description => ‘a short description’,
:image_url => ‘http://imageserver.tld.com/images/image.jpg’,
:price => 1234.56,
:title => ‘a test product’
}
end
end

describe Product do

include ProductSpecHelper

before(:each) do
@product = Product.new
end

it “should have a title” do
@product.attributes = valid_product_attributes.except(:title)
@product.should have(1).error_on(:title)
@product.title = ‘a test product’
@product.should be_valid
end

it “should have a description” do
@product.attributes = valid_product_attributes.except(:description)
@product.should have(1).error_on(:description)
@product.description = ‘a short description’
@product.should be_valid
end

it “should have a valid image uri”

end
<—

app/models/product.rb

—>
class Product < ActiveRecord::Base

validates_presence_of :title
validates_presence_of :description

end
<—

Specs pass:
—>
C:\var\RSpec\depot>rake spec:models
(in C:/var/RSpec/depot)
NOTICE: CREATE TABLE will create implicit sequence “products_id_seq”
for
serial column “products.id”
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index
“products_pkey” for table “products”

Product

  • should have a valid image uri (PENDING: Not Yet Implemented)
  • should have a description
  • should have a title

Pending:
Product should have a valid image uri (Not Yet Implemented)

Finished in 0.985 seconds

3 examples, 0 failures, 1 pending
<—

Spec fails: (I changed the .should be_valid to .should_not be_valid)

—>
C:\var\RSpec\depot>rake spec:models --trace
(in C:/var/RSpec/depot)
** Invoke spec:models (first_time)
** Invoke db:test:prepare (first_time)
** Invoke environment (first_time)
** Execute environment
** Execute db:test:prepare
** Invoke db:test:clone (first_time)
** Invoke db:schema:dump (first_time)
** Invoke environment
** Execute db:schema:dump
** Invoke db:test:purge (first_time)
** Invoke environment
** Execute db:test:purge
** Execute db:test:clone
** Invoke db:schema:load (first_time)
** Invoke environment
** Execute db:schema:load
NOTICE: CREATE TABLE will create implicit sequence “products_id_seq”
for
serial column “products.id”
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index
“products_pkey” for table “products”
** Execute spec:models

Product

  • should have a valid image uri (PENDING: Not Yet Implemented)
  • should have a description (FAILED - 1)
  • should have a title

Pending:
Product should have a valid image uri (Not Yet Implemented)

‘Product should have a description’ FAILED
expected valid? to return false, got true
./spec/models/product_spec.rb:35:

Finished in 1.266 seconds

3 examples, 1 failure, 1 pending
rake aborted!
Command ruby -I
“C:/var/RSpec/depot/vendor/plugins/rspec/lib”
“C:/var/RSpec/depot/vendor/plugins/rspec/bin/spec”
“spec/models/product_spec.rb”
–options
“C:/var/RSpec/depot/config/…/spec/spec.opts”
failed

C:/var/RSpec/depot/vendor/plugins/rspec/lib/spec/rake/
spectask.rb:173:in define' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:823:inverbose’
C:/var/RSpec/depot/vendor/plugins/rspec/lib/spec/rake/
spectask.rb:142:in define' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:392:incall’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/
rake.rb:392:in execute' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:392:ineach’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/
rake.rb:392:in execute' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:362:ininvoke’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/
rake.rb:355:in synchronize' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:355:ininvoke’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/
rake.rb:1739:in top_level' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:1739:ineach’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/
rake.rb:1739:in top_level' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:1761:instandard_exception_handling’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/
rake.rb:1733:in top_level' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:1711:inrun’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/
rake.rb:1761:in standard_exception_handling' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:1708:inrun’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/bin/rake:7
C:/usr/local/bin/ruby/bin/rake.bat:20:in `load’
C:/usr/local/bin/ruby/bin/rake.bat:20

C:\var\RSpec\depot>
<—

–
*** E-Mail is NOT a SECURE channel ***
James B. Byrne mailto:[email protected]
Harte & Lyne Limited http://www.harte-lyne.ca
9 Brockley Drive vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada L8E 3C3

#6

On Sat Nov 24 21:54:58 EST 2007, Pat M. pergesu at gmail.com wrote:

Failed expectations are implemented using exceptions, so naturally
if you use --trace then rake will print a backtrace. If you don’t
want it, don’t use --trace.

I do not care about the presence or absence of a trace. What I want to
know is why I get a rake failure if one of the specs fail. Surely a
specification failure is an expected outcome for at last some of the
trials. RSpec should just report that fact and let rake exit cleanly.
I
ran the previous trial with --trace just to avoid having someone write
“please provide a trace”.

—>
C:\var\RSpec\depot>rake spec:models
(in C:/var/RSpec/depot)
NOTICE: CREATE TABLE will create implicit sequence “products_id_seq”
for
serial
column “products.id”
NOTICE: CREATE TABLE / PRIMARY KEY will create implicit index
“products_pkey” f
or table “products”

Product

  • should have a valid image url (FAILED - 1)
  • should have a description
  • should have a title

‘Product should have a valid image url’ FAILED
expected 1 error on :image_url, got 0
./spec/models/product_spec.rb:40:

Finished in 0.922 seconds

3 examples, 1 failure
rake aborted!
Command ruby -I"C:/var/RSpec/depot/vendor/plugins/rspec/lib"
“C:/var/RSpec/depot/vendor/plugins/rspec/bin/spec”
“spec/models/product_spec.rb”
–options “C:/var/RSpec/depot/config/…/spec/spec.opts” failed

(See full trace by running task with --trace)
<—

–
*** E-Mail is NOT a SECURE channel ***
James B. Byrne mailto:[email protected]
Harte & Lyne Limited http://www.harte-lyne.ca
9 Brockley Drive vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada L8E 3C3

#7

On Nov 24, 2007 6:22 PM, James B. Byrne [email protected] wrote:

error and back trace. Why?

end
it “should have a title” do
@product.should be_valid
class Product < ActiveRecord::Base
(in C:/var/RSpec/depot)
Pending:
C:\var\RSpec\depot>rake spec:models --trace
** Invoke db:test:purge (first_time)
** Execute spec:models
‘Product should have a description’ FAILED
“spec/models/product_spec.rb”
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/
rake.rb:355:in synchronize' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:1733:intop_level’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/
rake.rb:1711:in run' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/ rake.rb:1761:instandard_exception_handling’
C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/lib/
rake.rb:1708:in run' C:/usr/local/bin/ruby/lib/ruby/gems/1.8/gems/rake-0.7.3/bin/rake:7 C:/usr/local/bin/ruby/bin/rake.bat:20:inload’
C:/usr/local/bin/ruby/bin/rake.bat:20

Failed expectations are implemented using exceptions, so naturally if
you use --trace then rake will print a backtrace. If you don’t want
it, don’t use --trace.

Pat

#8

On Sat, November 24, 2007 23:15, James B. Byrne wrote:

I do not care about the presence or absence of a trace. What I want to
know is why I get a rake failure if one of the specs fail. Surely a
specification failure is an expected outcome for at last some of the
trials. RSpec should just report that fact and let rake exit cleanly. I
ran the previous trial with --trace just to avoid having someone write
“please provide a trace”.

Anyhow, This is the code that is causing rake to whine at me. So where
does fail_on_error get set? Is this a configurable option?

spectask.rb
—>
171 unless system(cmd)
172 STDERR.puts failure_message if failure_message
173 raise(“Command #{cmd} failed”) if fail_on_error
174 end
<—

–
*** E-Mail is NOT a SECURE channel ***
James B. Byrne mailto:[email protected]
Harte & Lyne Limited http://www.harte-lyne.ca
9 Brockley Drive vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada L8E 3C3

#9

We saw some annoying behavior related to pending tests. Maybe you could
delete it and rerun your specs?

An error typically indicates that you have some sort of error in your
code -
failing tests should not cause rake to abort.

Stefan

2007/11/25, James B. Byrne [email protected]:

#10

On Sat, November 24, 2007 23:24, James B. Byrne wrote:

Anyhow, This is the code that is causing rake to whine at me. So where
does fail_on_error get set? Is this a configurable option?

This entire issue is discussed at length here:

http://rubyforge.org/pipermail/rspec-devel/2007-June/003048.html

Rake’s exit is the expected behaviour if a test fails. I presume this
allows automated build systems to detect a failing test suite since it
provides no evident value for a human observer.

Regards,

–
*** E-Mail is NOT a SECURE channel ***
James B. Byrne mailto:[email protected]
Harte & Lyne Limited http://www.harte-lyne.ca
9 Brockley Drive vox: +1 905 561 1241
Hamilton, Ontario fax: +1 905 561 0757
Canada L8E 3C3

#11

I believe I was a bit quick on this one…

Of course, if you can’t run your tests, the build should fail and return
non-0 as fast as possible.

However, it would be nice to distinguish between errors and failures. I
initially thought rake would output “rake aborted!” only when you have
errors - not just failing tests, and as a ruby newbie I sometimes found
it
hard to know if I had an error or just a failing test.

Does this make more sense?

Stefan

2007/11/28, aslak hellesoy [email protected]:

#12

On 11/28/07, Stefan Magnus Landrø [email protected] wrote:

We saw some annoying behavior related to pending tests. Maybe you could
delete it and rerun your specs?

An error typically indicates that you have some sort of error in your code -
failing tests should not cause rake to abort.

It does, and it should. Failures are just as fatal as errors, and
should make rake exit with a non-0.

Aslak

#13

On Nov 28, 2007 2:38 AM, Stefan Magnus Landrø [email protected]
wrote:

We saw some annoying behavior related to pending tests.

Can you be more specific?

#14

On Nov 28, 2007 7:16 AM, Stefan Magnus Landrø [email protected]
wrote:

However, it would be nice to distinguish between errors and failures. I
initially thought rake would output “rake aborted!” only when you have
errors - not just failing tests, and as a ruby newbie I sometimes found it
hard to know if I had an error or just a failing test.

Does this make more sense?

It does from the perspective of a human being, but keep in mind that
people are not the only consumers of rake.

As for distinguishing between errors and failures, RSpec is admittedly
riding both sides of this fence. I just added a ticket to address
this:

http://rspec.lighthouseapp.com/projects/5645-rspec/tickets/152

Please feel free to comment in the ticket.

Cheers,
David

#15

On Nov 29, 2007 11:23 AM, Stefan Magnus Landrø [email protected]
wrote:

it “should whatever” do
pending “whatever”
end
end

Removing the expectation from the before block, fixed our problem.

Interesting. pending actually raises a special exception that RSpec
catches and reports as a P. But your OtherModel never got the expected
message, so RSpec changes its mind and reports an F instead.

Perhaps we should never report failures for pending examples?

Aslak

#16

It seems like setting up expectations in before blocks, in combination
with
a pending example, makes rake return non-0

We had code similar to this:

describe Model do

before do
OtherModel.should_receive(:some_method)
end

it “should whatever” do
pending “whatever”
end

end

Removing the expectation from the before block, fixed our problem.

Stefan

2007/11/28, David C. [email protected]: