Forum: RSpec Test for a gem/plugin?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
James B. (Guest)
on 2008-12-09 21:06
I have decided that for now I am going to use the authlogic gem to
provide authentication for the project I am creating.  The question
arises, since authentication is a user feature request and since it is
to be satisfied through the use of a gem (not a plugin) then should one
test for the availability of the gem or not?

Secondly, given that a gem is providing the authentication logic (and is
not my code which therefore should not be tested by me) then what should
one test for with respect to the specific implementation?  The existence
of particular attributes?  The existence of a users model?

Does one write step definitions (assuming rspec) like:

When /users are authenticated/ do
  #TODO: This does not work even when the model class exists.
  User.should exist
end

When /authentication by authlogic/ do
  #TODO: Find out how to test for a loaded gem within an application
end
Matt W. (Guest)
on 2008-12-10 17:06
(Received via mailing list)
On 9 Dec 2008, at 19:06, James B. wrote:

> should
>
> When /authentication by authlogic/ do
>  #TODO: Find out how to test for a loaded gem within an application
> end

If you're working at the level of Cucumber features, I would stay well
away from implementation details like the particular technology you're
using for authentication.

Remember this is called Behaviour Driven Development[1] for a reason.
You're focussing on specifying the desired *behaviour* of the system.
How you chose to satisfy the user's need for that behaviour is an
entirely separate matter.

A large benefit of having Acceptance Tests, IMO, is that you could
radically change your implementation under the hood, such as switching
to a different authentication mechanism, and the features would still
be valid. You might have to code up some of your steps a little
differently, but as long as the users want the same thing from the
system, the features should not have to change in the slightest.

[1]http://dannorth.net/introducing-bdd

Matt W.
http://blog.mattwynne.net
http://www.songkick.com
James B. (Guest)
on 2008-12-10 18:04
Matt W. wrote:
> On 9 Dec 2008, at 19:06, James B. wrote:
>
>> should
>>
>> When /authentication by authlogic/ do
>>  #TODO: Find out how to test for a loaded gem within an application
>> end
>
> If you're working at the level of Cucumber features, I would stay well
> away from implementation details like the particular technology you're
> using for authentication.
>

The features for this simply say something along the lines of:

Scenario: Application has known users
  Given that the application has users
  When the user signs on
  Then the user is authenticated

> Remember this is called Behaviour Driven Development[1] for a reason.
> You're focussing on specifying the desired *behaviour* of the system.
> How you chose to satisfy the user's need for that behaviour is an
> entirely separate matter.

Nonetheless, in the step definitions I must test the behaviour in a
fashion which first drives and then confirms the implementation details.
If the chosen implementation depends upon a third party plugin or gem
then surely one should test for its provision?  What is the alternative?
Write tests to test that gems behaviour?  Does that not violate TDD/BDD
standard of only testing your own code?

>
> A large benefit of having Acceptance Tests, IMO, is that you could
> radically change your implementation under the hood, such as switching
> to a different authentication mechanism, and the features would still
> be valid. You might have to code up some of your steps a little
> differently, but as long as the users want the same thing from the
> system, the features should not have to change in the slightest.
>

If the implementation changed then the step definitions must change to
suit, must they not?  One is, in the end, only testing what one actually
does or wishes done. I do not see how in this case any implementation
change would necessitate revising the features given above.  If we were
implementing our own authentication system then clearly more features or
scenarios might be specified but the existing ones need not change.

> [1]http://dannorth.net/introducing-bdd

Read, re-read, and re-re-read.  Nonetheless, it is possible to become
dogmatic over such things.  There is a considerable, and growing, body
of opinion that so-called acceptance testing is suitable for many,
possibly, all levels of application implementation.  That the key to
understanding when to use AT really lies in a flexible interpretation of
the role of user.  This is, as has been pointed out to me many times, an
evolving art.

The imaginary end user of any non-trivial application is actually an
assemblage of disparate roles, many of which consciously do not overlap.
Some of those users are the system administrators which tend to the day
to day background maintenance tasks, some of which are automated and
some of which are not.  Some of the application users are, in truth, the
implementors themselves.

Not all features are of concern to all users/roles. Why should some
users be denied the use of features and BDD as a tool for their own
concerns simply because they are writing the code? It seems to me an
artificial, and logically insupportable, distinction to make.

While I am still a novice with much of this specific technology
(TDD/BDD/Ruby/Rails) I have had a lot of experience, much of it bad,
with overly rigid, and often misinformed, interpretation and employment
of design methodologies.  The quest for the silver bullet has led many
teams down methodological dead ends, whereas a more flexible use of the
same tool might have produced a beneficial outcome.

What I have come up with is this, for now:

When /has known users/ do
  # If we have known users then they must be stored somewhere
  my_user = User.new
end

When /user logs in/ do
  # If a user signs on then they do it here
  visits login_path
end

When /user signs on/ do
  When "user logs in"
end

When /user "(.*)" is authenticated/ do |u|
  # We use Authlogic for authentication so just test that gem is loaded
  assert defined?("Authlogic")
end
James B. (Guest)
on 2008-12-10 18:26
James B. wrote:
Of course, the last example should have been:

When /user is authenticated/ do
  # We use Authlogic for authentication so just test that gem is loaded
  assert defined?("Authlogic")
end
Ashley M. (Guest)
on 2008-12-10 19:15
(Received via mailing list)
On 10 Dec 2008, at 16:04, James B. wrote:

> Nonetheless, in the step definitions I must test the behaviour in a
> fashion which first drives and then confirms the implementation
> details.

This is not the path to BDD, it's the path to state-based testing.
Matt has already explained this well:

"How you chose to satisfy the user's need for that behaviour is an
entirely separate matter.  A large benefit of having Acceptance Tests,
IMO, is that you could radically change your implementation under the
hood, such as switching to a different authentication mechanism, and
the features would still be valid."

I'm guessing the customer for this project didn't ask for AuthLogic,
they asked for part of the app to be protected.  You should only write
acceptance tests for things these users care about.


> If the chosen implementation depends upon a third party plugin or gem
> then surely one should test for its provision?  What is the
> alternative?
> Write tests to test that gems behaviour?  Does that not violate TDD/
> BDD
> standard of only testing your own code?

Don't confuse library code with code you write to use this library.
The AuthLogic config in your app is code too, and it adds behaviours
to your app.  The fact there are 10, 100 or 10,000 lines of code
behind it is irrelevant.

There's two attitudes you can take in spec files: write specs that
your code calls some library, or write specs that describe the
behaviour that library provides.

For example (the following is rushed, contrived code but illustrates
the idea):

class Sorter
   def initialize(array)
     @array = array
   end
   def sort
     array.sort
   end
end

now when you spec
   @array = [5,9,6,1,4]
   @sorter = Sorter.new(@array)

are you going to write
   @array.should_receive(:sort)

or
   @sorter.sort.should == [1,4,5,6,9]
?

I believe that in this case, the second option is more valuble: you
can change the implementation to sort other collections.  In the first
case, you could refactor the code without breaking the existing spec.

This is important: *The ability to refactor code without breaking the
specs is one of the major sources of value in the specs*.  This lets
you reduce technical debt with no risk to your app.  If you rely on
the implementation in your spec, you have to change the spec, and
therefore you have no guarantee that the behaviour your users *asked
for* still works.

The definition of refactoring is changing the implementation of code
without changing its behaviour (and by extension its specs).  Without
this constant your code will be fragile.

As a extreme, albeit small, example- a while back I took the code for
a presentation I gave[1] and changed the app framework from Ramaze to
Merb (it only took about 10 mins).  Because the specs all ran across
the public HTML interface, there was no visible difference.  If the
feature steps relied on the implementation this wouldn't have been
possible.

I use Celerity[2] instead of Webrat and write all steps across an
app's public interface for this reason.

> The imaginary end user of any non-trivial application is actually an
> assemblage of disparate roles, many of which consciously do not
> overlap.
> Some of those users are the system administrators which tend to the
> day
> to day background maintenance tasks, some of which are automated and
> some of which are not.  Some of the application users are, in truth,
> the
> implementors themselves.

This is fine, but you should still write acceptance tests from the
perspective of the user.  If the user is a shell script, drive your
app on the command line.  If the user is a human, drive it over the
visible interface.  But do you really have any users that access your
app code directly?

Hope this gives another perspective on the subject.

Ashley


[1]
http://aviewfromafar.net/2008/10/2/geekup-sheffiel...
[2] http://celerity.rubyforge.org/

--
http://www.patchspace.co.uk/
http://aviewfromafar.net/
James B. (Guest)
on 2008-12-10 19:40
Ashley M. wrote:

>
> Don't confuse library code with code you write to use this library.
> The AuthLogic config in your app is code too, and it adds behaviours
> to your app.  The fact there are 10, 100 or 10,000 lines of code
> behind it is irrelevant.

So, if I understand you correctly, then I should write features somewhat
like this:

Scenario: A known user signs in successfully
  Given a known user "myuser" with a password "myuserpassword"
  When user "myuser" signs on with a password "myuserpassword"
  Then user "myuser" is authenticated
    And the user sees a sign in success message

Scenario: A known user does not sign in successfully
  Given a known user "myuser" with a password "myuserpassword"
  When user "myuser" signs in with a password "anotherpassword"
  Then user "myuser" is not authenticated
    And the user sees a sign in failure message
    And the user sees the sign in page

And I should simply not worry about what is provided by authlogic, other
than the requisite behaviour as exhibited in this application.


>
> Hope this gives another perspective on the subject.
>

I am still grappling with the entire concept.  A "perspective" graces
what I as yet possess with too much dignity.

Thanks.
Matt W. (Guest)
on 2008-12-10 20:13
(Received via mailing list)
On 10 Dec 2008, at 16:26, James B. wrote:

> James B. wrote:
> Of course, the last example should have been:
>
> When /user is authenticated/ do
>  # We use Authlogic for authentication so just test that gem is loaded
>  assert defined?("Authlogic")
> end

To verify that the user has been authenticated, why not try something
like

Then /I should see that I am logged in/ do
   user = User.first
   response.should include_text "welcome, #{user.username}"
end

This is much less coupled to your implementation, though it does
assume that the 'I' referred to in the step is the only User in your
database, or at least happens to be the first one. I'll leave it as a
exercise to the reader to work around that one.

Matt W.
http://blog.mattwynne.net
http://www.songkick.com
Andrew P. (Guest)
on 2008-12-11 09:54
(Received via mailing list)
I put a starter application with a bunch of features to test
authorisation
code implemented by Restful Authentication on Github a few days ago. In
theory you could apply these features directly to your Authlogic
implementation.
http://github.com/diabolo/fbrp/tree/master


All best

Andrew

2008/12/10 James B. <removed_email_address@domain.invalid>
Ashley M. (Guest)
on 2008-12-11 12:33
(Received via mailing list)
On 10 Dec 2008, at 17:40, James B. wrote:

> Scenario: A known user does not sign in successfully
>  Given a known user "myuser" with a password "myuserpassword"
>  When user "myuser" signs in with a password "anotherpassword"
>  Then user "myuser" is not authenticated
>    And the user sees a sign in failure message
>    And the user sees the sign in page
>
> And I should simply not worry about what is provided by authlogic,
> other
> than the requisite behaviour as exhibited in this application.

That's how I (and many others here) tackle it.  But there's no need
for the steps

   Then user "myuser" is authenticated

and

  Then user "myuser" is not authenticated

because these don't describe behaviour, they describe state.  The
"sign in message" steps are what you really want.  And you can
implement these as Matt described.  Since you've written steps that
create the user (preferably through the app itself!), you could extend
the step like this

   Given a known user "Mr Known User" with credentials "myuser" :
"myuserpassword"

Then you already have the name to verify if you want to display a
personalised message.  (On the other hand, putting too much data in
the steps gets cumbersome; I tend to write them more like this:

   Scenario: A known user signs in successfully
     Given I am a registered user
     When I sign on with the correct credentials
     Then I should see a sign in success message

Even if internally I call out to steps more like the ones you wrote as
examples.  Hope that doesn't muddy the waters though.

Ashley

--
http://www.patchspace.co.uk/
http://aviewfromafar.net/
James B. (Guest)
on 2008-12-11 17:19
Ashley M. wrote:

>
> Then you already have the name to verify if you want to display a
> personalised message.  (On the other hand, putting too much data in
> the steps gets cumbersome; I tend to write them more like this:
>
>    Scenario: A known user signs in successfully
>      Given I am a registered user
>      When I sign on with the correct credentials
>      Then I should see a sign in success message
>
> Even if internally I call out to steps more like the ones you wrote as
> examples.  Hope that doesn't muddy the waters though.

I believe that I am gaining some insight into how this is all meant to
pull together.   One of the difficulties I face is that different people
evidently have significantly different philosophies about how and what
to test (surprise, surprise, surprise!).  As I am coming at this from
the pov of someone who has had no prior experience with this approach,
and whose training in it to date has been entirely theoretical in
nature, I find this somewhat confusing.  At the moment I tend to see
cucumber features as my sole method of testing, an approach that I
realize is not favoured by many.

I began learning cucumber by writing a set of fairly low level
functional tests following the template generated by cucumber.  Now,
based on the advice I have received here, I am trying a much smaller
scale approach and generating a few very high level features.  However,
in the back of my mind I still consider that eventually I will specify
much of the implementation detail, model attributes, data normalization
routines, input limits, and so forth as feature steps somewhere.
Experience will eventually teach me whether that approach is sustainable
or not.

I presently have in mind two distinct types of tests/features that I
wish to represent.  The first is the end user type of feature writing
which conforms generally to the analysis of system requirements.  The
second set of features will be more like functional tests, that exercise
the implementation details.

My expression of the second type of features, the functional tests, are
probably what is generating the greatest controversy.  Some
practitioners evidently see BDD features more or less as a pure
analytical tool.  They expect that functional and unit tests will be
conducted mostly in a "traditional" manner, via test unit or rspec or
similar testing framework.  With this approach there is no need, or
desire, that features elaborate great detail regarding implementation
since that is done elsewhere.

The dichotomy between feature steps and step definitions is another
point of confusion for me.  Take your reference to "if internally I call
out to steps more like the ones you wrote."  Does this refer to feature
"Steps" or to step definition "Steps".  I suspect the latter.  In that
case one can imagine that step definitions become rather more elaborate
structures.  Some step definitions perhaps even assume the appearance
and role of rspec specs and have only a remotely dependent relationship
to any feature step.

It is in these obscure details that much of my confusion arises.
Perhaps I have inferred your meaning correctly, perhaps I have totally
misunderstood it.  Regardless of which is true, doubt remains.
Pat M. (Guest)
on 2008-12-11 20:56
(Received via mailing list)
James B. <removed_email_address@domain.invalid> writes:

>> away from implementation details like the particular technology you're
>> using for authentication.
>>
>
> The features for this simply say something along the lines of:
>
> Scenario: Application has known users
>   Given that the application has users
>   When the user signs on
>   Then the user is authenticated

WTBV?

(Where's the business value (I totally made that up, but I'm gonna stick
with it :))

On its own, being authenticated doesn't have any business value.
Authentication only *enhances* business value.  Consider a feature that
lets me manage finances.  That has some business value.  Then we ensure
that only authenticated users may manage their finances, which means we
can charge people for the service - enhancing the business value that
already exists.


>> Remember this is called Behaviour Driven Development[1] for a reason.
>> You're focussing on specifying the desired *behaviour* of the system.
>> How you chose to satisfy the user's need for that behaviour is an
>> entirely separate matter.
>
> Nonetheless, in the step definitions I must test the behaviour in a
> fashion which first drives and then confirms the implementation
> details.

I don't understand this.  Your features/specs drive the implementation,
but they confirm the *behavior*.


> If the chosen implementation depends upon a third party plugin or gem
> then surely one should test for its provision?  What is the alternative?
> Write tests to test that gems behaviour?

I would specify my app's behavior, not the gem's.  And I certainly
wouldn't test that the gem is there.  If it's not, then my code is going
to blow up anyway!

Instead, write features that specify *what* you want from your
application.

Given a user named 'padillac'
When I log in as 'padillac'
Then I should see 'Hello, padillac!'
And I should see 'Edit your account'

How you handle this - whether it's restful_authentication, authlogic, or
roll-your-own - doesn't matter.


>> A large benefit of having Acceptance Tests, IMO, is that you could
>> radically change your implementation under the hood, such as switching
>> to a different authentication mechanism, and the features would still
>> be valid. You might have to code up some of your steps a little
>> differently, but as long as the users want the same thing from the
>> system, the features should not have to change in the slightest.
>>
>
> If the implementation changed then the step definitions must change to
> suit, must they not?

It depends on the level at which you've written your step definitions.
If you've written them to hit the outer-most interface, for example
using webrat in a rails app, then no, you wouldn't need to change the
definitions.  If you wrote them at a lower level, creating a user and
adding his id to the session, then there's a chance you would have to
change them.

The key is writing the features themselves at a level of abstraction
that *completely* hides the implementation.  That way you can change
your code and step definitions freely.

What you're left with is a balancing act, deciding which implementation
details can force changes to your step definitions.  If you write the
definitions at a low level, accessing the model directly, you'll likely
need to change them when the definitions change.

On the other hand, if you write them at a higher level, there's the real
possibility that minor changes to your app code will necessitate changes
to your step definitions.  Consider the following step definition:

When /^I log in as '(.*)'$/ do |username|
  visits login_url
  fills_in 'Username', :with => username
  fills_in 'Password', :with => 'password'
end

If you change the template to read 'Login' instead of 'Username' for the
username field, the step definition is going to have to change.

So you have to evaluate each piece of your app, figure out what's likely
to change, determine the upfront and maintenance costs of writing step
definitions at varying levels of abstraction, and make a decision.  Then
be thankful that you're not building a bridge, because even if you get
it wrong, the cost of fixing your mistake is relatively cheap :)


> Not all features are of concern to all users/roles. Why should some
> users be denied the use of features and BDD as a tool for their own
> concerns simply because they are writing the code? It seems to me an
> artificial, and logically insupportable, distinction to make.

I don't understand what you mean by this.  Which users are "denied the
use of features and BDD?"  Could you please elaborate for me?


> When /user "(.*)" is authenticated/ do |u|
>   # We use Authlogic for authentication so just test that gem is loaded
>   assert defined?("Authlogic")
> end

WTBV?! :)

Pat
James B. (Guest)
on 2008-12-11 22:12
Pat M. wrote:
>>
>> Scenario: Application has known users
>>   Given that the application has users
>>   When the user signs on
>>   Then the user is authenticated
>
> WTBV?
>
> (Where's the business value (I totally made that up, but I'm gonna stick
> with it :))

I like it too.  That said, regulatory compliance and financial security
are two that come to mind.  And, yes, I and my users consider that both
represent real business value.

I am not trying to put forth a design philosophy.  I am trying to
discover what techniques experienced BDD practitioners consider useful
and what they do not.  My ignorance may lead me into astoundingly poor
conclusions at times but, I am willing to bear the resulting ridicule if
in the process I learn that which I should have realized.  Fortunately,
ridicule has been noteworthy by its absence and the knowledge revealed
by respondents quite illuminating.

I am beginning to see that there exists a wide range of acceptable
practices gathered under the rubric of BDD.  At the beginning I acquired
the idea that step definitions were tied to feature steps N:1.  Now I
gather, perhaps incorrectly, that step definitions might replace
discrete tests/specifications in frameworks like TestUnit and RSpec
without the need for a directly corresponding feature step, N:N where N
>= 0

Maybe that is my problem.  Perhaps I am trying too hard to use features
for everything.  I really do not want to scatter my tests over hell's
half acre.  I do not wish to have some tests under ./test, others under
./spec and still others under ./features.  I do not want to have three
different rake tasks to run all the tests.  I do not want to integrate
rcov results from one test suite with the results from another.  I do
not wish to expend time on integrating test suites with one another.

So, given all of these desires, perhaps I am considering features in a
way that is considerably less than optimal.  On the other hand, given
that the rspec and testunit matchers are available in step definitions,
is there any harm in using step definitions in place of specs and unit
tests?  In the absence of any other method, one could imagine I suppose
a feature step that simply invokes that portion of the step definitions
test suite that is not tied otherwise to any other feature step.
This topic is locked and can not be replied to.