Re: customize description and failure messages when writing rspec matcher dsl

On Nov 24, 2011, at 12:48 AM, Anran Yang wrote:

Dear all,

I recently tried to write a custom matcher using rspec dsl, which I’d
like to use as the following:
:name,
:vector => {},
]

end

===========================================

The purpose is to test the entire model structure(attributes, inheritance
and association).

Then term Behavior Driven Development came into being, in part, to
discourage this approach to testing. We believe testing should be about
behavior, and that testing structures leads to highly coupled tests that
are hard to change.

I’d recommend thinking about the need for each of these attributes, come
up
with examples of how they are used and write those instead. That said,
see
below for more on how matchers work.


It made me thinking a while. But at last I found that I’ve thought about
it, though it may only happen unconsciously. So below is why I need this
kind of spec.
I must admit the Spec is against BDD’s spirit. However, I think BDD may
not
be enough for all kinds of projects in all kinds of circumstance. In my
project, the model itself is also a product, not only the “surface” of
the
software. Maybe the best way to accomplish this is to separate this kind
of
test and the “real” BDD test. But rspec can support them perfectly so
I’m
lazy this time.
Maybe there are some other projects involved in this kind of situation?
I
mean, some projects with a concrete domain model, so to some extent the
development is driven by both behavior and knowledge?

Then I wrote following code to implement it:

========be_structured_as.rb===================

This is a matcher for test hpgc model structure

RSpec::Matchers.define :be_structured_as do |structure|
def check(namespace, klass, desc)
(desc[:has_many] ||= {}).keys.each { |item|
klass.to_s.camelize.constantize.new.should have_many
item.to_s.pluralize.to_sym }
if !desc[:can_be_a] || desc[:can_be_a].empty?
obj = Factory.create(klass)
obj.should have_these_attributes (desc[:has] ||= [])
else
desc[:can_be_a].keys.map { |sub| sub.to_s }.should ==
namespace.subclasses_of(klass.to_s)

To simplify the matcher definition, I used some other machers like
“obj.should have_these_attributes (desc[:has] ||= [])”, this works fine,
thanks to the excellent new matcher dsl syntax. Howerver, I got trouble
when running the test using “rspec spec/models/hpgc_spec.rb -fd”. The
test
passed but print

<Screenshot.png>

instead of something like “Hpgc should be structured as …”. I think it
may due to the other matcher I used because the message is from the
matcher
“have_these_attributes”.

That’s correct. The default behavior is that the block passed to the
“match” method in the matcher DSL is expected to return a boolean. If it
raises an exception instead, that exception bubbles up and becomes the
source of the failure message.

I try to add method “description” but seems it doesn’t work.

“description” is used for the documentation formatter, not for failure
messages. If you run “rspec spec --format documentation” you’ll see
names
for all your examples.


Maybe I failed express my problem clearly, but the question is for pass
message, not failure message. Knowing how rspec raise exception help
little
to get my spec say "Hpgc should be structured as… " when it passes.
The
method “description” failed to present itself even when the spec passed,
as
long as there is an inner “should” clause.

And the failure message is also a problem. I search the web and come
across some solutions, but all of them need manually build the message.
I’ve not come out a clean way to do this(I must add some invasive
clauses
to track the error) and think maybe the best behaviour would be that it
“throw” the failure message of inner matchers or false-value
expressions.
But this time it just told me something like “Hpgc is expected to be
structured as …”.

Instead of using the “match” method in the matcher, you can use
“match_unless_raises”, which returns false if it captures an exception,
letting the matcher control the failure message (which is what I think
you’re looking for):

RSpec::Matchers.define :be_structured_as do |structure|
def check(namespace, klass, desc)

end

match_unless_raises do |namespace|
self.class.send :include, namespace
structure.each { |key, value| check namespace, key, value }
end
end

HTH,
David


Is the method in rspec-expectation? I searched the local doc but didn’t
find it. Also I don’t quite understand how the method can help track the
failing message. It return false, but I still cannot figure out which
line
fails, isn’t it?

Thank you for your inspiring reply and tolerance to my poor English :slight_smile:

Regards,
YANG

On Nov 27, 2011, at 7:22 AM, Anran Yang wrote:

describe Hpgc do
],
:has => [
:has => [
]
Then term Behavior Driven Development came into being, in part, to discourage
this approach to testing. We believe testing should be about behavior, and that
testing structures leads to highly coupled tests that are hard to change.

I’d recommend thinking about the need for each of these attributes, come up with
examples of how they are used and write those instead. That said, see below for
more on how matchers work.


It made me thinking a while. But at last I found that I’ve thought about it,
though it may only happen unconsciously. So below is why I need this kind of spec.
I must admit the Spec is against BDD’s spirit. However, I think BDD may not be
enough for all kinds of projects in all kinds of circumstance. In my project, the
model itself is also a product, not only the “surface” of the software. Maybe the
best way to accomplish this is to separate this kind of test and the “real” BDD
test. But rspec can support them perfectly so I’m lazy this time.
Maybe there are some other projects involved in this kind of situation? I mean,
some projects with a concrete domain model, so to some extent the development is
driven by both behavior and knowledge?

The specific problem is that tests about structure are more highly
coupled to structure than behavior, and therefore more brittle. This is
true whether you wrote the test first or wrote it after, and regardless
of what methodology you are thinking of when you write the tests.

I don’t understand what you mean when you say ‘In my project, the model
itself is also a product, not only the “surface” of the software.’. Can
you elaborate?

  obj = Factory.create(klass)

end
That’s correct. The default behavior is that the block passed to the “match”
method in the matcher DSL is expected to return a boolean. If it raises an
exception instead, that exception bubbles up and becomes the source of the failure
message.

I try to add method “description” but seems it doesn’t work.

“description” is used for the documentation formatter, not for failure messages.
If you run “rspec spec --format documentation” you’ll see names for all your
examples.


Maybe I failed express my problem clearly, but the question is for pass message,
not failure message. Knowing how rspec raise exception help little to get my spec
say "Hpgc should be structured as… " when it passes. The method “description”
failed to present itself even when the spec passed, as long as there is an inner
“should” clause.

You can define your own description method in the same way:

description do |actual|

end

self.class.send :include, namespace
structure.each { |key, value| check namespace, key, value }
end
end

HTH,
David


Is the method in rspec-expectation? I searched the local doc but didn’t find it.

It is in rspec-expectations but it is not documented. I added it to
support matchers in rspec-rails that delegate to minitest assertions
(which raise their own exception), and never took the time to formalize
it as a public API. That said, it should be considered public, will not
change without appropriate warning, and will be documented in the rdoc
for the 2.8 release.

Also I don’t quite understand how the method can help track the failing message.
It return false, but I still cannot figure out which line fails, isn’t it?

It stores the exception from the failure in an instance variable named
@rescued_exception, which you can access through a method named
rescued_exception e.g.

failure_messsage_for_should do |actual|
“Expected … but something
happened\n#{rescued_exception.message}\n#{rescued_exception.backtrace}”
end

Another approach would be to keep track of each thing individually using
the standard matches method:

match do |actual|
@reasons == []
@reasons << “no foo” unless foo
@reasons << “no bar” unless bar
@reasons << “no baz” unless baz
@reasons.empty?
end

That returns false if any conditions fail, but doesn’t raise an
exceptions. Then you can use @reasons in the failure message, e.g.

failure_message_for_should do |actual|
“Failed because #{@reasons.join(’, ')}”
end

Thank you for your inspiring reply and tolerance to my poor English :slight_smile:

Your English is perfectly understandable.

Cheers,
David