Test for a gem/plugin?

Testing RSpec
1

I have decided that for now I am going to use the authlogic gem to
provide authentication for the project I am creating. The question
arises, since authentication is a user feature request and since it is
to be satisfied through the use of a gem (not a plugin) then should one
test for the availability of the gem or not?

Secondly, given that a gem is providing the authentication logic (and is
not my code which therefore should not be tested by me) then what should
one test for with respect to the specific implementation? The existence
of particular attributes? The existence of a users model?

Does one write step definitions (assuming rspec) like:

When /users are authenticated/ do
#TODO: This does not work even when the model class exists.
User.should exist
end

When /authentication by authlogic/ do
#TODO: Find out how to test for a loaded gem within an application
end

2

On 9 Dec 2008, at 19:06, James B. wrote:

should

When /authentication by authlogic/ do
#TODO: Find out how to test for a loaded gem within an application
end

If you’re working at the level of Cucumber features, I would stay well
away from implementation details like the particular technology you’re
using for authentication.

Remember this is called Behaviour Driven Development[1] for a reason.
You’re focussing on specifying the desired behaviour of the system.
How you chose to satisfy the user’s need for that behaviour is an
entirely separate matter.

A large benefit of having Acceptance Tests, IMO, is that you could
radically change your implementation under the hood, such as switching
to a different authentication mechanism, and the features would still
be valid. You might have to code up some of your steps a little
differently, but as long as the users want the same thing from the
system, the features should not have to change in the slightest.

[1]Introducing BDD - Dan North & Associates Ltd

Matt W.
http://blog.mattwynne.net

3

Matt W. wrote:

On 9 Dec 2008, at 19:06, James B. wrote:

should

When /authentication by authlogic/ do
#TODO: Find out how to test for a loaded gem within an application
end

If you’re working at the level of Cucumber features, I would stay well
away from implementation details like the particular technology you’re
using for authentication.

The features for this simply say something along the lines of:

Scenario: Application has known users
Given that the application has users
When the user signs on
Then the user is authenticated

Remember this is called Behaviour Driven Development[1] for a reason.
You’re focussing on specifying the desired behaviour of the system.
How you chose to satisfy the user’s need for that behaviour is an
entirely separate matter.

Nonetheless, in the step definitions I must test the behaviour in a
fashion which first drives and then confirms the implementation details.
If the chosen implementation depends upon a third party plugin or gem
then surely one should test for its provision? What is the alternative?
Write tests to test that gems behaviour? Does that not violate TDD/BDD
standard of only testing your own code?

A large benefit of having Acceptance Tests, IMO, is that you could
radically change your implementation under the hood, such as switching
to a different authentication mechanism, and the features would still
be valid. You might have to code up some of your steps a little
differently, but as long as the users want the same thing from the
system, the features should not have to change in the slightest.

If the implementation changed then the step definitions must change to
suit, must they not? One is, in the end, only testing what one actually
does or wishes done. I do not see how in this case any implementation
change would necessitate revising the features given above. If we were
implementing our own authentication system then clearly more features or
scenarios might be specified but the existing ones need not change.

[1]Introducing BDD - Dan North & Associates Ltd

Read, re-read, and re-re-read. Nonetheless, it is possible to become
dogmatic over such things. There is a considerable, and growing, body
of opinion that so-called acceptance testing is suitable for many,
possibly, all levels of application implementation. That the key to
understanding when to use AT really lies in a flexible interpretation of
the role of user. This is, as has been pointed out to me many times, an
evolving art.

The imaginary end user of any non-trivial application is actually an
assemblage of disparate roles, many of which consciously do not overlap.
Some of those users are the system administrators which tend to the day
to day background maintenance tasks, some of which are automated and
some of which are not. Some of the application users are, in truth, the
implementors themselves.

Not all features are of concern to all users/roles. Why should some
users be denied the use of features and BDD as a tool for their own
concerns simply because they are writing the code? It seems to me an
artificial, and logically insupportable, distinction to make.

While I am still a novice with much of this specific technology
(TDD/BDD/Ruby/Rails) I have had a lot of experience, much of it bad,
with overly rigid, and often misinformed, interpretation and employment
of design methodologies. The quest for the silver bullet has led many
teams down methodological dead ends, whereas a more flexible use of the
same tool might have produced a beneficial outcome.

What I have come up with is this, for now:

When /has known users/ do

If we have known users then they must be stored somewhere

my_user = User.new
end

When /user logs in/ do

If a user signs on then they do it here

visits login_path
end

When /user signs on/ do
When “user logs in”
end

When /user “(.*)” is authenticated/ do |u|

We use Authlogic for authentication so just test that gem is loaded

assert defined?(“Authlogic”)
end

4

James B. wrote:
Of course, the last example should have been:

When /user is authenticated/ do

We use Authlogic for authentication so just test that gem is loaded

assert defined?(“Authlogic”)
end

5

On 10 Dec 2008, at 16:04, James B. wrote:

Nonetheless, in the step definitions I must test the behaviour in a
fashion which first drives and then confirms the implementation
details.

This is not the path to BDD, it’s the path to state-based testing.
Matt has already explained this well:

“How you chose to satisfy the user’s need for that behaviour is an
entirely separate matter. A large benefit of having Acceptance Tests,
IMO, is that you could radically change your implementation under the
hood, such as switching to a different authentication mechanism, and
the features would still be valid.”

I’m guessing the customer for this project didn’t ask for AuthLogic,
they asked for part of the app to be protected. You should only write
acceptance tests for things these users care about.

If the chosen implementation depends upon a third party plugin or gem
then surely one should test for its provision? What is the
alternative?
Write tests to test that gems behaviour? Does that not violate TDD/
BDD
standard of only testing your own code?

Don’t confuse library code with code you write to use this library.
The AuthLogic config in your app is code too, and it adds behaviours
to your app. The fact there are 10, 100 or 10,000 lines of code
behind it is irrelevant.

There’s two attitudes you can take in spec files: write specs that
your code calls some library, or write specs that describe the
behaviour that library provides.

For example (the following is rushed, contrived code but illustrates
the idea):

class Sorter
def initialize(array)
@array = array
end
def sort
array.sort
end
end

now when you spec
@array = [5,9,6,1,4]
@sorter = Sorter.new(@array)

are you going to write
@array.should_receive(:sort)

or
@sorter.sort.should == [1,4,5,6,9]
?

I believe that in this case, the second option is more valuble: you
can change the implementation to sort other collections. In the first
case, you could refactor the code without breaking the existing spec.

This is important: The ability to refactor code without breaking the
specs is one of the major sources of value in the specs. This lets
you reduce technical debt with no risk to your app. If you rely on
the implementation in your spec, you have to change the spec, and
therefore you have no guarantee that the behaviour your users asked
for still works.

The definition of refactoring is changing the implementation of code
without changing its behaviour (and by extension its specs). Without
this constant your code will be fragile.

As a extreme, albeit small, example- a while back I took the code for
a presentation I gave[1] and changed the app framework from Ramaze to
Merb (it only took about 10 mins). Because the specs all ran across
the public HTML interface, there was no visible difference. If the
feature steps relied on the implementation this wouldn’t have been
possible.

I use Celerity[2] instead of Webrat and write all steps across an
app’s public interface for this reason.

The imaginary end user of any non-trivial application is actually an
assemblage of disparate roles, many of which consciously do not
overlap.
Some of those users are the system administrators which tend to the
day
to day background maintenance tasks, some of which are automated and
some of which are not. Some of the application users are, in truth,
the
implementors themselves.

This is fine, but you should still write acceptance tests from the
perspective of the user. If the user is a shell script, drive your
app on the command line. If the user is a human, drive it over the
visible interface. But do you really have any users that access your
app code directly?

Hope this gives another perspective on the subject.

Ashley

[1]
http://aviewfromafar.net/2008/10/2/geekup-sheffield-vi-from-specification-to-success
[2] http://celerity.rubyforge.org/

–
http://www.patchspace.co.uk/

6

Ashley M. wrote:

Don’t confuse library code with code you write to use this library.
The AuthLogic config in your app is code too, and it adds behaviours
to your app. The fact there are 10, 100 or 10,000 lines of code
behind it is irrelevant.

So, if I understand you correctly, then I should write features somewhat
like this:

Scenario: A known user signs in successfully
Given a known user “myuser” with a password “myuserpassword”
When user “myuser” signs on with a password “myuserpassword”
Then user “myuser” is authenticated
And the user sees a sign in success message

Scenario: A known user does not sign in successfully
Given a known user “myuser” with a password “myuserpassword”
When user “myuser” signs in with a password “anotherpassword”
Then user “myuser” is not authenticated
And the user sees a sign in failure message
And the user sees the sign in page

And I should simply not worry about what is provided by authlogic, other
than the requisite behaviour as exhibited in this application.

Hope this gives another perspective on the subject.

I am still grappling with the entire concept. A “perspective” graces
what I as yet possess with too much dignity.

Thanks.

7

I put a starter application with a bunch of features to test
authorisation
code implemented by Restful Authentication on Github a few days ago. In
theory you could apply these features directly to your Authlogic
implementation.

All best

Andrew

2008/12/10 James B. [email protected]

8

On 10 Dec 2008, at 17:40, James B. wrote:

Scenario: A known user does not sign in successfully
Given a known user “myuser” with a password “myuserpassword”
When user “myuser” signs in with a password “anotherpassword”
Then user “myuser” is not authenticated
And the user sees a sign in failure message
And the user sees the sign in page

And I should simply not worry about what is provided by authlogic,
other
than the requisite behaviour as exhibited in this application.

That’s how I (and many others here) tackle it. But there’s no need
for the steps

Then user “myuser” is authenticated

and

Then user “myuser” is not authenticated

because these don’t describe behaviour, they describe state. The
“sign in message” steps are what you really want. And you can
implement these as Matt described. Since you’ve written steps that
create the user (preferably through the app itself!), you could extend
the step like this

Given a known user “Mr Known User” with credentials “myuser” :
“myuserpassword”

Then you already have the name to verify if you want to display a
personalised message. (On the other hand, putting too much data in
the steps gets cumbersome; I tend to write them more like this:

Scenario: A known user signs in successfully
Given I am a registered user
When I sign on with the correct credentials
Then I should see a sign in success message

Even if internally I call out to steps more like the ones you wrote as
examples. Hope that doesn’t muddy the waters though.

Ashley

–
http://www.patchspace.co.uk/

9

On 10 Dec 2008, at 16:26, James B. wrote:

James B. wrote:
Of course, the last example should have been:

When /user is authenticated/ do

We use Authlogic for authentication so just test that gem is loaded

assert defined?(“Authlogic”)
end

To verify that the user has been authenticated, why not try something
like

Then /I should see that I am logged in/ do
user = User.first
response.should include_text “welcome, #{user.username}”
end

This is much less coupled to your implementation, though it does
assume that the ‘I’ referred to in the step is the only User in your
database, or at least happens to be the first one. I’ll leave it as a
exercise to the reader to work around that one.

Matt W.
http://blog.mattwynne.net

10

Ashley M. wrote:

Then you already have the name to verify if you want to display a
personalised message. (On the other hand, putting too much data in
the steps gets cumbersome; I tend to write them more like this:

Scenario: A known user signs in successfully
Given I am a registered user
When I sign on with the correct credentials
Then I should see a sign in success message

Even if internally I call out to steps more like the ones you wrote as
examples. Hope that doesn’t muddy the waters though.

I believe that I am gaining some insight into how this is all meant to
pull together. One of the difficulties I face is that different people
evidently have significantly different philosophies about how and what
to test (surprise, surprise, surprise!). As I am coming at this from
the pov of someone who has had no prior experience with this approach,
and whose training in it to date has been entirely theoretical in
nature, I find this somewhat confusing. At the moment I tend to see
cucumber features as my sole method of testing, an approach that I
realize is not favoured by many.

I began learning cucumber by writing a set of fairly low level
functional tests following the template generated by cucumber. Now,
based on the advice I have received here, I am trying a much smaller
scale approach and generating a few very high level features. However,
in the back of my mind I still consider that eventually I will specify
much of the implementation detail, model attributes, data normalization
routines, input limits, and so forth as feature steps somewhere.
Experience will eventually teach me whether that approach is sustainable
or not.

I presently have in mind two distinct types of tests/features that I
wish to represent. The first is the end user type of feature writing
which conforms generally to the analysis of system requirements. The
second set of features will be more like functional tests, that exercise
the implementation details.

My expression of the second type of features, the functional tests, are
probably what is generating the greatest controversy. Some
practitioners evidently see BDD features more or less as a pure
analytical tool. They expect that functional and unit tests will be
conducted mostly in a “traditional” manner, via test unit or rspec or
similar testing framework. With this approach there is no need, or
desire, that features elaborate great detail regarding implementation
since that is done elsewhere.

The dichotomy between feature steps and step definitions is another
point of confusion for me. Take your reference to “if internally I call
out to steps more like the ones you wrote.” Does this refer to feature
“Steps” or to step definition “Steps”. I suspect the latter. In that
case one can imagine that step definitions become rather more elaborate
structures. Some step definitions perhaps even assume the appearance
and role of rspec specs and have only a remotely dependent relationship
to any feature step.

It is in these obscure details that much of my confusion arises.
Perhaps I have inferred your meaning correctly, perhaps I have totally
misunderstood it. Regardless of which is true, doubt remains.

11

James B. [email protected] writes:

away from implementation details like the particular technology you’re
using for authentication.

The features for this simply say something along the lines of:

Scenario: Application has known users
Given that the application has users
When the user signs on
Then the user is authenticated

WTBV?

(Where’s the business value (I totally made that up, but I’m gonna stick
with it :))

On its own, being authenticated doesn’t have any business value.
Authentication only enhances business value. Consider a feature that
lets me manage finances. That has some business value. Then we ensure
that only authenticated users may manage their finances, which means we
can charge people for the service - enhancing the business value that
already exists.

Remember this is called Behaviour Driven Development[1] for a reason.
You’re focussing on specifying the desired behaviour of the system.
How you chose to satisfy the user’s need for that behaviour is an
entirely separate matter.

Nonetheless, in the step definitions I must test the behaviour in a
fashion which first drives and then confirms the implementation
details.

I don’t understand this. Your features/specs drive the implementation,
but they confirm the behavior.

If the chosen implementation depends upon a third party plugin or gem
then surely one should test for its provision? What is the alternative?
Write tests to test that gems behaviour?

I would specify my app’s behavior, not the gem’s. And I certainly
wouldn’t test that the gem is there. If it’s not, then my code is going
to blow up anyway!

Instead, write features that specify what you want from your
application.

Given a user named ‘padillac’
When I log in as ‘padillac’
Then I should see ‘Hello, padillac!’
And I should see ‘Edit your account’

How you handle this - whether it’s restful_authentication, authlogic, or
roll-your-own - doesn’t matter.

A large benefit of having Acceptance Tests, IMO, is that you could
radically change your implementation under the hood, such as switching
to a different authentication mechanism, and the features would still
be valid. You might have to code up some of your steps a little
differently, but as long as the users want the same thing from the
system, the features should not have to change in the slightest.

If the implementation changed then the step definitions must change to
suit, must they not?

It depends on the level at which you’ve written your step definitions.
If you’ve written them to hit the outer-most interface, for example
using webrat in a rails app, then no, you wouldn’t need to change the
definitions. If you wrote them at a lower level, creating a user and
adding his id to the session, then there’s a chance you would have to
change them.

The key is writing the features themselves at a level of abstraction
that completely hides the implementation. That way you can change
your code and step definitions freely.

What you’re left with is a balancing act, deciding which implementation
details can force changes to your step definitions. If you write the
definitions at a low level, accessing the model directly, you’ll likely
need to change them when the definitions change.

On the other hand, if you write them at a higher level, there’s the real
possibility that minor changes to your app code will necessitate changes
to your step definitions. Consider the following step definition:

When /^I log in as ‘(.*)’$/ do |username|
visits login_url
fills_in ‘Username’, :with => username
fills_in ‘Password’, :with => ‘password’
end

If you change the template to read ‘Login’ instead of ‘Username’ for the
username field, the step definition is going to have to change.

So you have to evaluate each piece of your app, figure out what’s likely
to change, determine the upfront and maintenance costs of writing step
definitions at varying levels of abstraction, and make a decision. Then
be thankful that you’re not building a bridge, because even if you get
it wrong, the cost of fixing your mistake is relatively cheap :slight_smile:

Not all features are of concern to all users/roles. Why should some
users be denied the use of features and BDD as a tool for their own
concerns simply because they are writing the code? It seems to me an
artificial, and logically insupportable, distinction to make.

I don’t understand what you mean by this. Which users are “denied the
use of features and BDD?” Could you please elaborate for me?

When /user “(.*)” is authenticated/ do |u|

We use Authlogic for authentication so just test that gem is loaded

assert defined?(“Authlogic”)
end

WTBV?! :slight_smile:

Pat

12

Pat M. wrote:

Scenario: Application has known users
Given that the application has users
When the user signs on
Then the user is authenticated

WTBV?

(Where’s the business value (I totally made that up, but I’m gonna stick
with it :))

I like it too. That said, regulatory compliance and financial security
are two that come to mind. And, yes, I and my users consider that both
represent real business value.

I am not trying to put forth a design philosophy. I am trying to
discover what techniques experienced BDD practitioners consider useful
and what they do not. My ignorance may lead me into astoundingly poor
conclusions at times but, I am willing to bear the resulting ridicule if
in the process I learn that which I should have realized. Fortunately,
ridicule has been noteworthy by its absence and the knowledge revealed
by respondents quite illuminating.

I am beginning to see that there exists a wide range of acceptable
practices gathered under the rubric of BDD. At the beginning I acquired
the idea that step definitions were tied to feature steps N:1. Now I
gather, perhaps incorrectly, that step definitions might replace
discrete tests/specifications in frameworks like TestUnit and RSpec
without the need for a directly corresponding feature step, N:N where N

= 0

Maybe that is my problem. Perhaps I am trying too hard to use features
for everything. I really do not want to scatter my tests over hell’s
half acre. I do not wish to have some tests under ./test, others under
./spec and still others under ./features. I do not want to have three
different rake tasks to run all the tests. I do not want to integrate
rcov results from one test suite with the results from another. I do
not wish to expend time on integrating test suites with one another.

So, given all of these desires, perhaps I am considering features in a
way that is considerably less than optimal. On the other hand, given
that the rspec and testunit matchers are available in step definitions,
is there any harm in using step definitions in place of specs and unit
tests? In the absence of any other method, one could imagine I suppose
a feature step that simply invokes that portion of the step definitions
test suite that is not tied otherwise to any other feature step.