Unit test setup


#1

Hi there.

I noticed that when using unit testing the setup and destroy methods
works for every test executed. Is there a way for use general
“constructor” and “destructor” methods, to, for example, open a socket
on the “constructor”, execute all the tests and close it on the
“destructor”, executing some operations that do need to be a sequence on
the same opened socket?

Thanks!


#2

On 5/2/06, Eric H. removed_email_address@domain.invalid wrote:

On May 2, 2006, at 6:35 AM, Eustáquio Rangel wrote:

Is there a way for use
general “constructor” and “destructor” methods, to, for example,
open a socket on the “constructor”, execute all the tests and close
it on the “destructor”

There isn’t.

Only half correct. You can override the #initialize method in the test
case:

class TestTheTestsTest < Test::Unit::TestCase
  def initialize( *args )
    super
    @foo = :bar
  end

  def test_initialize
    assert_equal :bar, @foo
  end
end

However, Ruby doesn’t have “deconstructors” for classes (AFAIK).

You don’t need it and you don’t want it.

While I would agree with this in almost 100% of cases, there are
edge cases where this may not be true.

The reason you probably don’t want it is that you want each test to
run in a ‘clean’ environment. Basically, you want to make sure you’re
not introducing dependencies between the tests within a test case. If
you do, it makes refactoring that much harder. Also, there’s no
guarantee about the order in which the individual tests are run within
a test case. (I believe they are run in lexical order with the current
Test::Unit implementation, but that’s a coincedence, not a
requirement.)

However, one of the rules of agile unit testing is also that the tests
must be able to run quickly (so that people will actually bother to
run them). Therefore, it is sometimes (though rarely, if you’re doing
things right) helpful to make sure expensive operations happen only
when absolutely needed – but you should only make this compromise if
you can be absolutely sure that you’re not introducing state
dependencies into your tests.

In the original example given by Eustáquio, I would bet that Eric is
correct – you probably don’t want to do this.


Regards,
John W.
http://johnwilger.com


#3

On May 2, 2006, at 1:38 PM, John W. wrote:

class TestTheTestsTest < Test::Unit::TestCase
def initialize( *args )
super
@foo = :bar
end

Eww, yuck.

However, Ruby doesn’t have “deconstructors” for classes (AFAIK).

Ruby doesn’t have deconstructors at all.

things right) helpful to make sure expensive operations happen only
when absolutely needed – but you should only make this compromise if
you can be absolutely sure that you’re not introducing state
dependencies into your tests.

When I require an expensive setup per test invocation my code is
telling me it is not easily testable, so instead of hacking
Test::Unit I refactor.


Eric H. - removed_email_address@domain.invalid - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com


#4

Actually, a test case object is created for each test_something method
(see test/unit/testcase.rb:51 in ruby 8.4).

This means that overriding the initializer does not bring anything
more than overriding setup. In fact, it makes things worse, as
exceptions in setup and teardown are caught by the Test::Unit
framework (and hence correctly reported), whereas those in the test
case initializer are not.

Ciao
Stefano

On 02/05/06, John W. removed_email_address@domain.invalid wrote:
[…]

  end
end

[…]


#5

On May 2, 2006, at 6:35 AM, Eustáquio Rangel wrote:

Hi there.

I noticed that when using unit testing the setup and destroy
methods works for every test executed. Is there a way for use
general “constructor” and “destructor” methods, to, for example,
open a socket on the “constructor”, execute all the tests and close
it on the “destructor”, executing some operations that do need to
be a sequence on the same opened socket?

There isn’t. You don’t need it and you don’t want it.


Eric H. - removed_email_address@domain.invalid - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com


#6

On 5/2/06, Eric H. removed_email_address@domain.invalid wrote:

On May 2, 2006, at 1:38 PM, John W. wrote:

class TestTheTestsTest < Test::Unit::TestCase
def initialize( *args )
super
@foo = :bar
end

Eww, yuck.

I didn’t say it was pretty, just that it was possible. “Can’t” and
“shouldn’t” are two different things, after all. (Of course, Stefano
pointed out that I’m wrong about it being possible, too – so maybe
the only thing I’ve proven is that I’m good at talking out of my ass!
:-D)

When I require an expensive setup per test invocation my code is
telling me it is not easily testable, so instead of hacking
Test::Unit I refactor.

100% agreed.


Regards,
John W.
http://johnwilger.com


#7

On 5/2/06, Stefano T. removed_email_address@domain.invalid wrote:

Actually, a test case object is created for each test_something method
(see test/unit/testcase.rb:51 in ruby 8.4).

This means that overriding the initializer does not bring anything
more than overriding setup.

Interesting. I wasn’t aware of that. I guess that should at least earn
me some points with Eric, as it should now be apparent that I’ve never
actually relied on the method I described earlier. :wink:


Regards,
John W.
http://johnwilger.com


#8

The easiest way to achieve a “global” test setup is by invoking
explicitly the test runner:

begin

do your setup

Test::Unit::AutoRunner.run
ensure

clean up

end

Ciao
Stefano


#9

Hi.

You don’t need it and you don’t want it.

You can’t be so sure about that all over the time. :wink:

Ruby doesn’t have deconstructors at all.

Yes, I know, I was trying to figure out some global method like the
“destroy” method on the unit testing class. I mean, a global “destroy”
like the global “setup”. Not really C++ destroy stuff.

The easiest way to achieve a “global” test setup is by invoking
explicitly the test runner:

That worked, thanks! Btw, it was the only way I thought to test the
socket operations. Even if most tests are not on the correct sequence (I
just need step 2 below are the first one - ok, it’s requirement), I can
make things like:

1 - Connect on the socket (the “global” setup).
2 - Send a hello and get an id.
3 - List all the other users there.
4 - Send a ping to make sure we’re still there.
5 - Send a quit, goodbye.
6 - Close the socket.

All the operations from 3 and below needs the id from 2, where the
server authorize the client with a password and return its id. So if I
open/close the socket on every test I lost the authorization and the id.
I use the asserts to test if the authentication worked and returned a
valid id, if returns the user list, and so on.

But I accept suggestions for improvements. :slight_smile:

Thanks!


#10

Hi.

You’re mixing up responses among different people here, and that is
very poor mailing list etiquette.

Sorry if you didn’t like that. I was just trying to be shorter on the
same answer.

1 - Connect on the socket (the “global” setup).
You should be using a stub instead.
You shouldn’t be connecting to anything during a unit test.
You shouldn’t have to do all the setup for every test, just the setup
for the test.
You should be able to test sending a ping without fetching an id.
You should be able to test sending a ping without an id. You may
need to refactor your code.
To help you on your way I’ve got another trick for you.

Thanks for the code/tips. :slight_smile:


#11

On May 3, 2006, at 10:45 AM, Eustáquio Rangel wrote:

Hi.

You’re mixing up responses among different people here, and that is
very poor mailing list etiquette.

You don’t need it and you don’t want it.

You can’t be so sure about that all over the time. :wink:

I’m 100% certain of that. If you think you need it your code isn’t
easily testable. You should refactor instead.

socket operations. Even if most tests are not on the correct
sequence (I
just need step 2 below are the first one - ok, it’s requirement), I
can
make things like:

1 - Connect on the socket (the “global” setup).

You should be using a stub instead. Here’s what I’ve got for MogileFS:

class FakeSocket

def initialize
@closed = false
end

def closed?
@closed
end

def close
@closed = true
return nil
end

end

That’s all I need, so add methods to the stub as appropriate.

2 - Send a hello and get an id.
3 - List all the other users there.
4 - Send a ping to make sure we’re still there.
5 - Send a quit, goodbye.
6 - Close the socket.

All the operations from 3 and below needs the id from 2, where the
server authorize the client with a password and return its id.

You shouldn’t be connecting to anything during a unit test. If you
are it isn’t really a test of a unit.

So if I open/close the socket on every test I lost the
authorization and the id. I use the asserts to test if the
authentication worked and returned a
valid id, if returns the user list, and so on.

You shouldn’t have to do all the setup for every test, just the setup
for the test.

You should be able to test sending a ping without fetching an id.
You should be able to test sending a ping without an id. You may
need to refactor your code.

To help you on your way I’ve got another trick for you. I wrote a
flickr library using open-uri. open-uri overrides Kernel#open, but I
don’t really want to connect to flickr to run my tests. Instead I
just insert an open into my flickr class while testing that pretends
to be flickr.

You can do the same with a socket stub, just have the socket pretend
it is already initialized and have it return the proper responses.

class Flickr

attr_accessor :responses, :uris

def open(uri)
@uris << uri
yield StringIO.new(@responses.shift)
end

end

class FlickrTest < Test::Unit::TestCase

def setup
@flickr = Flickr.new ‘API_KEY’
@flickr.responses = []
@flickr.uris = []
end

def test_search_multipage
@flickr.responses << <<-EOF

<?xml version="1.0" encoding="utf-8" ?> [...] EOF
 @flickr.responses << <<-EOF
<?xml version="1.0" encoding="utf-8" ?> [...] EOF
 photos = @flickr.search :user_id => '50178138@N00',
                         :min_taken_date => Time.parse

(‘2005-10-21’),
:per_page => 2

 assert_equal 2, @flickr.uris.length
 assert_equal 'http://flickr.com/services/rest/...',
              @flickr.uris.first
 assert_equal 'http://flickr.com/services/rest/...',
              @flickr.uris.last

 assert_equal 3, photos.length
 assert_equal [59864477, 55457289, 55457233], photos.map { |p|

p.photo_id }
end

end


Eric H. - removed_email_address@domain.invalid - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com


#12

On 5/4/06, Stefano T. removed_email_address@domain.invalid wrote:

On 02/05/06, John W. removed_email_address@domain.invalid wrote:

However, one of the rules of agile unit testing is also that the tests
must be able to run quickly (so that people will actually bother to
run them).

You are absolutely right. In fact, this is exactly how it is done in
rails when using transactional fixtures (which is the default
setting):

Then again, Eric’s point holds true here as well. My biggest complaint
with Rails is that you can’t truly unit test your model classes. Not
only is it necessary to have a database connection, but it’s also very
difficult to inject mocks in place of the other model classes that the
class under test is interacting with. In a better design, you wouldn’t
need transactional fixtures, because you wouldn’t be talking to a
real database connection at all. This is one reason why I was
disappointed that Jamis B.'s ideas for using dependency injection in
Rails never made it into the framework. Ruby itself doesn’t really
need dependency injection, but I think it would have improved this
aspect of the Rails framework.


Regards,
John W.
http://johnwilger.com


#13

On 02/05/06, John W. removed_email_address@domain.invalid wrote:
[…]

However, one of the rules of agile unit testing is also that the tests
must be able to run quickly (so that people will actually bother to
run them). Therefore, it is sometimes (though rarely, if you’re doing
things right) helpful to make sure expensive operations happen only
when absolutely needed – but you should only make this compromise if
you can be absolutely sure that you’re not introducing state
dependencies into your tests.
[…]

You are absolutely right. In fact, this is exactly how it is done in
rails when using transactional fixtures (which is the default
setting):

TestCase#setup uses the class variable @@already_loaded_fixtures to
make it sure that the fixtures are loaded just once, whereas the
rollback mechanism in transactional fixtures prevents the state
dependencies that you mentioned.

Ciao,
Stefano