Forum: Ruby Unit test setup

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Eustáquio R. (Guest)
on 2006-05-02 17:35
Hi there.

I noticed that when using unit testing the setup and destroy methods
works for every test executed. Is there a way for use general
"constructor" and "destructor"  methods, to, for example, open a socket
on the "constructor", execute all the tests and close it on the
"destructor", executing some operations that do need to be a sequence on
the same opened socket?

Thanks!
Eric H. (Guest)
on 2006-05-03 21:00
(Received via mailing list)
On May 2, 2006, at 6:35 AM, Eustáquio Rangel wrote:

> Hi there.
>
> I noticed that when using unit testing the setup and destroy
> methods works for every test executed. Is there a way for use
> general "constructor" and "destructor"  methods, to, for example,
> open a socket on the "constructor", execute all the tests and close
> it on the "destructor", executing some operations that do need to
> be a sequence on the same opened socket?

There isn't.  You don't need it and you don't want it.

--
Eric H. - removed_email_address@domain.invalid - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com
John W. (Guest)
on 2006-05-03 21:02
(Received via mailing list)
On 5/2/06, Eric H. <removed_email_address@domain.invalid> wrote:
> On May 2, 2006, at 6:35 AM, Eustáquio Rangel wrote:
> > Is there a way for use
> > general "constructor" and "destructor"  methods, to, for example,
> > open a socket on the "constructor", execute all the tests and close
> > it on the "destructor"

> There isn't.

Only half correct. You can override the #initialize method in the test
case:

    class TestTheTestsTest < Test::Unit::TestCase
      def initialize( *args )
        super
        @foo = :bar
      end

      def test_initialize
        assert_equal :bar, @foo
      end
    end

However, Ruby doesn't have "deconstructors" for classes (AFAIK).

> You don't need it and you don't want it.

While I would agree with this in almost 100% of cases, there _are_
edge cases where this may not be true.

The reason you probably don't want it is that you want each test to
run in a 'clean' environment. Basically, you want to make sure you're
not introducing dependencies between the tests within a test case. If
you do, it makes refactoring that much harder. Also, there's no
guarantee about the order in which the individual tests are run within
a test case. (I believe they are run in lexical order with the current
Test::Unit implementation, but that's a coincedence, not a
requirement.)

However, one of the rules of agile unit testing is also that the tests
must be able to run quickly (so that people will actually bother to
run them). Therefore, it is sometimes (though rarely, if you're doing
things right) helpful to make sure expensive operations happen only
when absolutely needed -- but you should only make this compromise if
you can be absolutely sure that you're not introducing state
dependencies into your tests.

In the original example given by Eustáquio, I would bet that Eric is
correct -- you probably don't want to do this.

--
Regards,
John W.
http://johnwilger.com
Eric H. (Guest)
on 2006-05-03 21:02
(Received via mailing list)
On May 2, 2006, at 1:38 PM, John W. wrote:

>
>    class TestTheTestsTest < Test::Unit::TestCase
>      def initialize( *args )
>        super
>        @foo = :bar
>      end

Eww, yuck.

> However, Ruby doesn't have "deconstructors" for classes (AFAIK).

Ruby doesn't have deconstructors at all.

> things right) helpful to make sure expensive operations happen only
> when absolutely needed -- but you should only make this compromise if
> you can be absolutely sure that you're not introducing state
> dependencies into your tests.

When I require an expensive setup per test invocation my code is
telling me it is not easily testable, so instead of hacking
Test::Unit I refactor.

--
Eric H. - removed_email_address@domain.invalid - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com
Stefano T. (Guest)
on 2006-05-03 21:05
(Received via mailing list)
Actually, a test case object is created for each test_something method
(see test/unit/testcase.rb:51 in ruby 8.4).

This means that overriding the initializer does not bring anything
more than overriding setup. In fact, it makes things worse, as
exceptions in setup and teardown are caught by the Test::Unit
framework (and hence correctly reported), whereas those in the test
case initializer are not.

Ciao
Stefano

On 02/05/06, John W. <removed_email_address@domain.invalid> wrote:
[...]
>       end
>     end
[...]
John W. (Guest)
on 2006-05-03 21:05
(Received via mailing list)
On 5/2/06, Stefano T. <removed_email_address@domain.invalid> wrote:
> Actually, a test case object is created for each test_something method
> (see test/unit/testcase.rb:51 in ruby 8.4).
>
> This means that overriding the initializer does not bring anything
> more than overriding setup.

Interesting. I wasn't aware of that. I guess that should at least earn
me some points with Eric, as it should now be apparent that I've never
_actually_ relied on the method I described earlier. ;-)

--
Regards,
John W.
http://johnwilger.com
John W. (Guest)
on 2006-05-03 21:05
(Received via mailing list)
On 5/2/06, Eric H. <removed_email_address@domain.invalid> wrote:
> On May 2, 2006, at 1:38 PM, John W. wrote:
> >    class TestTheTestsTest < Test::Unit::TestCase
> >      def initialize( *args )
> >        super
> >        @foo = :bar
> >      end
>
> Eww, yuck.

I didn't say it was pretty, just that it was possible. "Can't" and
"shouldn't" are two different things, after all. (Of course, Stefano
pointed out that I'm wrong about it being possible, too -- so maybe
the only thing I've proven is that I'm good at talking out of my ass!
:-D)

> When I require an expensive setup per test invocation my code is
> telling me it is not easily testable, so instead of hacking
> Test::Unit I refactor.

100% agreed.

--
Regards,
John W.
http://johnwilger.com
Stefano T. (Guest)
on 2006-05-03 21:06
(Received via mailing list)
The easiest way to achieve a "global" test setup is by invoking
explicitly the test runner:

begin
  # do your setup
  Test::Unit::AutoRunner.run
ensure
  # clean up
end

Ciao
Stefano
Eustáquio R. (Guest)
on 2006-05-03 21:45
Hi.

> You don't need it and you don't want it.

You can't be so sure about that all over the time. ;-)

> Ruby doesn't have deconstructors at all.

Yes, I know, I was trying to figure out some global method like the
"destroy" method on the unit testing class. I mean, a global "destroy"
like the global "setup". Not really C++ destroy stuff.

> The easiest way to achieve a "global" test setup is by invoking
> explicitly the test runner:

That worked, thanks! Btw, it was the only way I thought to test the
socket operations. Even if most tests are not on the correct sequence (I
just need step 2 below are the first one - ok, it's requirement), I can
make things like:

1 - Connect on the socket (the "global" setup).
2 - Send a hello and get an id.
3 - List all the other users there.
4 - Send a ping to make sure we're still there.
5 - Send a quit, goodbye.
6 - Close the socket.

All the operations from 3 and below needs the id from 2, where the
server authorize the client with a password and return its id. So if I
open/close the socket on every test I lost the authorization and the id.
I use the asserts to test if the authentication worked and returned a
valid id, if returns the user list, and so on.

But I accept suggestions for improvements. :-)

Thanks!
Eric H. (Guest)
on 2006-05-03 22:55
(Received via mailing list)
On May 3, 2006, at 10:45 AM, Eustáquio Rangel wrote:

> Hi.

You're mixing up responses among different people here, and that is
very poor mailing list etiquette.

>> You don't need it and you don't want it.
>
> You can't be so sure about that all over the time. ;-)

I'm 100% certain of that.  If you think you need it your code isn't
easily testable.  You should refactor instead.

> socket operations. Even if most tests are not on the correct
> sequence (I
> just need step 2 below are the first one - ok, it's requirement), I
> can
> make things like:
>
> 1 - Connect on the socket (the "global" setup).

You should be using a stub instead.  Here's what I've got for MogileFS:

class FakeSocket

   def initialize
     @closed = false
   end

   def closed?
     @closed
   end

   def close
     @closed = true
     return nil
   end

end

That's all I need, so add methods to the stub as appropriate.

> 2 - Send a hello and get an id.
> 3 - List all the other users there.
> 4 - Send a ping to make sure we're still there.
> 5 - Send a quit, goodbye.
> 6 - Close the socket.
>
> All the operations from 3 and below needs the id from 2, where the
> server authorize the client with a password and return its id.

You shouldn't be connecting to anything during a unit test.  If you
are it isn't really a test of a unit.

> So if I open/close the socket on every test I lost the
> authorization and the id. I use the asserts to test if the
> authentication worked and returned a
> valid id, if returns the user list, and so on.

You shouldn't have to do all the setup for every test, just the setup
for the test.

You should be able to test sending a ping without fetching an id.
You should be able to test sending a ping without an id.  You may
need to refactor your code.

To help you on your way I've got another trick for you.  I wrote a
flickr library using open-uri.  open-uri overrides Kernel#open, but I
don't really want to connect to flickr to run my tests.  Instead I
just insert an open into my flickr class while testing that pretends
to be flickr.

You can do the same with a socket stub, just have the socket pretend
it is already initialized and have it return the proper responses.

class Flickr

   attr_accessor :responses, :uris

   def open(uri)
     @uris << uri
     yield StringIO.new(@responses.shift)
   end

end

class FlickrTest < Test::Unit::TestCase

   def setup
     @flickr = Flickr.new 'API_KEY'
     @flickr.responses = []
     @flickr.uris = []
   end

   def test_search_multipage
     @flickr.responses << <<-EOF
<?xml version="1.0" encoding="utf-8" ?>
<rsp stat="ok">
[...]
</rsp>
     EOF

     @flickr.responses << <<-EOF
<?xml version="1.0" encoding="utf-8" ?>
<rsp stat="ok">
[...]
</rsp>
     EOF

     photos = @flickr.search :user_id => '50178138@N00',
                             :min_taken_date => Time.parse
('2005-10-21'),
                             :per_page => 2

     assert_equal 2, @flickr.uris.length
     assert_equal 'http://flickr.com/services/rest/...',
                  @flickr.uris.first
     assert_equal 'http://flickr.com/services/rest/...',
                  @flickr.uris.last

     assert_equal 3, photos.length
     assert_equal [59864477, 55457289, 55457233], photos.map { |p|
p.photo_id }
   end

end

--
Eric H. - removed_email_address@domain.invalid - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com
Eustáquio R. (Guest)
on 2006-05-03 23:15
Hi.

> You're mixing up responses among different people here, and that is
> very poor mailing list etiquette.

Sorry if you didn't like that. I was just trying to be shorter on the
same answer.

>> 1 - Connect on the socket (the "global" setup).
> You should be using a stub instead.
> You shouldn't be connecting to anything during a unit test.
> You shouldn't have to do all the setup for every test, just the setup
> for the test.
> You should be able to test sending a ping without fetching an id.
> You should be able to test sending a ping without an id.  You may
> need to refactor your code.
> To help you on your way I've got another trick for you.

Thanks for the code/tips. :-)
Stefano T. (Guest)
on 2006-05-04 16:47
(Received via mailing list)
On 02/05/06, John W. <removed_email_address@domain.invalid> wrote:
[...]
> However, one of the rules of agile unit testing is also that the tests
> must be able to run quickly (so that people will actually bother to
> run them). Therefore, it is sometimes (though rarely, if you're doing
> things right) helpful to make sure expensive operations happen only
> when absolutely needed -- but you should only make this compromise if
> you can be absolutely sure that you're not introducing state
> dependencies into your tests.
[...]

You are absolutely right. In fact, this is exactly how it is done in
rails when using transactional fixtures (which is the default
setting):

TestCase#setup uses the class variable @@already_loaded_fixtures to
make it sure that the fixtures are loaded just once, whereas the
rollback mechanism in transactional fixtures prevents the state
dependencies that you mentioned.

Ciao,
Stefano
John W. (Guest)
on 2006-05-04 19:56
(Received via mailing list)
On 5/4/06, Stefano T. <removed_email_address@domain.invalid> wrote:
> On 02/05/06, John W. <removed_email_address@domain.invalid> wrote:
> > However, one of the rules of agile unit testing is also that the tests
> > must be able to run quickly (so that people will actually bother to
> > run them).
>
> You are absolutely right. In fact, this is exactly how it is done in
> rails when using transactional fixtures (which is the default
> setting):

Then again, Eric's point holds true here as well. My biggest complaint
with Rails is that you can't truly _unit_ test your model classes. Not
only is it necessary to have a database connection, but it's also very
difficult to inject mocks in place of the other model classes that the
class under test is interacting with. In a better design, you wouldn't
_need_ transactional fixtures, because you wouldn't be talking to a
real database connection at all. This is one reason why I was
disappointed that Jamis B.'s ideas for using dependency injection in
Rails never made it into the framework. Ruby itself doesn't really
_need_ dependency injection, but I think it would have improved this
aspect of the Rails framework.

--
Regards,
John W.
http://johnwilger.com
This topic is locked and can not be replied to.