Problem in Unit Testing Methods that start new threads

Ruby
1

I have a bit of doubt, in Unit Testing Programs that start new threads.
Please have a look at the code below:

class Foobar

def hello_world
p “Hello World”
@thread_status = false
end

def new_thread_start
Thread.new do
sleep(100)
@thread_status = true
end
end

end

here goes the lame test case

require “test/unit”

module Test::Unit::Assertions
def assert_false(t_object,message=nil)
boolean = !t_object
full_message = build_message(message,’<?> Object is not
false’,boolean)
assert_block(full_message) { boolean}
end
end

class TestFoobar < Test::Unit::TestCase
def setup
@foo = Foobar.new
class << @foo
def ivar var
instance_variable_get(:"@#{var}")
end
end
end

def test_hello_world
@foo.hello_world
assert_false @foo.ivar(:thread_status)
end

def test_new_thread
# sorry for a bit of not so DRY thingy
@foo.hello_world
assert_false @foo.ivar(:thread_status)
@foo.new_thread_start

# next assert is true because method was started in a new thread
# and control came back immediately, what i would probably want is
# to wait here so that i can have proper check on the method and
# state of the program, but i am not sure, if that's exactly
# approach i should take.

assert_false @foo.ivar(:thread_status)

end
end

Now, as someone suggested on IRC, I can do a join and wait for the
thread to finish. But the problem is, I don’t exactly have an instance
to the thread, because its managed by a plugin and i am just using the
plugin to do stuff.

Any ideas/suggestions are more than welcome.

gnufied

2

On Wed, 2007-01-03 at 22:30 +0900, Hemant K. wrote:

require “test/unit”
class TestFoobar < Test::Unit::TestCase
@foo.hello_world
# next assert is true because method was started in a new thread
Now, as someone suggested on IRC, I can do a join and wait for the
thread to finish. But the problem is, I don’t exactly have an instance
to the thread, because its managed by a plugin and i am just using the
plugin to do stuff.

Any ideas/suggestions are more than welcome.

Or is it this a good idea?

Thread.list.each { |t| t.join }

3

On Jan 3, 2007, at 8:30 AM, Hemant K. wrote:

end

here goes the lame test case

def test_hello_world

Now, as someone suggested on IRC, I can do a join and wait for the
thread to finish. But the problem is, I don’t exactly have an instance
to the thread, because its managed by a plugin and i am just using the
plugin to do stuff.

Any ideas/suggestions are more than welcome.

Ara’s right on this one. It’s a race condition. But it looks like
you’re interested in checking that the thread has indeed started. If
I were to write something like this I would probably have the unit
test run @foo.new_thread_start and immediately sleep waiting for a
signal from the child thread. The child thread would then signal
back to the any waiting threads that it had set it’s thread status to
true. Altough the more I ponder it, the more I feel like I’ve just
moved the race condition to the sleep call. I’ll have to review my
concurrent programming texts a bit. Seems like you should do the same.

Either way, check out Monitor for some insight:
http://www.ruby-doc.org/stdlib/libdoc/monitor/rdoc/index.html

-Mat

4

On Thu, 4 Jan 2007, Mat S. wrote:

Either way, check out Monitor for some insight:
http://www.ruby-doc.org/stdlib/libdoc/monitor/rdoc/index.html

-Mat

good idea mat. it’s easy in ruby too:

require ‘thread’

q = Queue.new

Thread.new{
q.push :running
# run
}

q.pop # wait for thread to start…

regards.

-a

5

On Wed, 3 Jan 2007, Hemant K. wrote:

def new_thread_start

def setup
assert_false @foo.ivar(:thread_status)

and control came back immediately, what i would probably want is

thread to finish. But the problem is, I don’t exactly have an instance
to the thread, because its managed by a plugin and i am just using the
plugin to do stuff.

Any ideas/suggestions are more than welcome.

it seems that all you’ve managed to do is write a very long race
condition.
i’m not one of those people who think the mere mention of the word
‘unit-testing’ bestows any sort of robustness on code. for example, the
testing of ‘thread_status’ in your test means nothing, as it’s the
source of
the race condition: you need to wrap setting and reading this var with a
semaphore. this shows why:

harp:~ > cat a.rb
class C
attr :thread
def initialize
@thread = nil
end
def new_thread
Thread.new{ @thread = Thread.current }
end
end

4242.times{|i| raise “race condition @ loopno #{ i }!” unless((c =
C.new) and (Thread === c.new_thread) and c.thread) }

harp:~ > ruby a.rb
a.rb:11: race condition @ loopno 942! (RuntimeError)
from a.rb:11

regarding your specific question though, you need to verify that a
thread is
created and that it’s status is running. even if you don’t have a
handle on
the thread you can set things up in your unit test to get one.
something like

harp:~ > cat a.rb
def tracking_threads &b
before = Thread.list
yield
after = Thread.list
return after - before
end

threads = tracking_threads{ 2.times{ Thread.new{ sleep } } }

threads.each{|t| p t.status}

harp:~ > ruby a.rb
“sleep”
“sleep”

cool. now we’ve managed to shows that Thread.new works! :wink:

i wouldn’t bother with this at all unless i could also test that the
created
thread did the right thing.

my 2 cts.

kind regards.

-a

6

Hemant K. schrieb:

I have a bit of doubt, in Unit Testing Programs that start new threads.
Please have a look at the code below:
(…)

Hemant, in addition to what the others said, it’s not clear to me what
you really want to test: whether a new thread is started, whether the
instance variable changed after a certain amount of time, whether the
instance variable changed at the end of the new thread, …

I’m a fan of black box unit tests, so for me, the tests should specify
what you expect from the interface of the object under test. I
wouldn’t test for values of certain instance variables or for threads
created internally, unless those are part of the desired interface of
your objects.

Regards,
Pit

7

On Thu, 2007-01-04 at 01:04 +0900, [email protected] wrote:

testing of ‘thread_status’ in your test means nothing, as it’s the source of
Thread.new{ @thread = Thread.current }

Thanks for the insight Ara, although my class is singleton so i
shouldn’t get the problem you have described above.

However can you show me a code sample, that doesn’t cause race condition
in above code. I tried using Mutex on above code,

require “thread”
class C
attr :thread
def initialize
mutex = Mutex.new
mutex.lock
@thread = nil
mutex.unlock
end

def new_thread
Thread.new{
mutex = Mutex.new
mutex.lock
@thread = Thread.current
mutex.unlock
}
end

end

4242.times{|i| raise “race condition @ loopno #{ i }!” unless((c =
C.new) and (Thread === c.new_thread) and c.thread) }

And I am still getting a race condition. Using MonitorMixin to signal
execution seems like an overkill to me in above code.

So can you show me a way of making above code not race.

end

Cool thanks.

8

On Thu, 2007-01-04 at 06:57 +0900, Pit C. wrote:

I’m a fan of black box unit tests, so for me, the tests should specify
what you expect from the interface of the object under test. I
wouldn’t test for values of certain instance variables or for threads
created internally, unless those are part of the desired interface of
your objects.

Regards,
Pit

Agreed Pit, but basically I want to test state of my program through
Unit Tests and whether each method modifies state of program as it was
intended?

Now, unit testing methods that return something on invocation is easy
and i don’t need to go around running asserts on instance variables.

But I have some doubt regarding testing methods that don’t return
anything explicitly and rather update instance variables.

Earlier, I was writing Unit Tests for a networking application which i
wrote using EventMachine. Now since, EM is completely based on
callbacks, you can’t do a check on return values of methods. Hence I had
to rely on doing asserts on instance variables.

I would love to know, how do i go about unit testing in such cases. I
can’t yet grasp concept of code that can be unit tested easily i guess.

gnufied

9

Hemant K. wrote:

‘unit-testing’ bestows any sort of robustness on code. for example, the
def new_thread

attr :thread
mutex.lock
And I am still getting a race condition. Using MonitorMixin to signal

def tracking_threads &b

harp:~ > ruby a.rb
“sleep”
“sleep”

Cool thanks.

Try removing the “mutex = Mutex.new” from “new_thread” That’s causing a
thread local mutex to be created that’s only being used by that thread
and none of the others, because they have their own mutex. The threads
need to share the SAME mutex in order to enable mutual exclusion
between them, ie, the mutex you create in initialize(). That said, us
better understanding what you actually want to unit test would help us
to help you.

Cheers,

Ken

10

On Thu, 4 Jan 2007, Hemant K. wrote:

Earlier, I was writing Unit Tests for a networking application which i
wrote using EventMachine. Now since, EM is completely based on
callbacks, you can’t do a check on return values of methods. Hence I had
to rely on doing asserts on instance variables.

I would love to know, how do i go about unit testing in such cases. I
can’t yet grasp concept of code that can be unit tested easily i guess.

often methods should do one of two things:

  • succeed or throw an exception

  • return a value indicating success

in your case the former should be sufficient - you can assert that
nothing is
raised and move on. if that’s not sufficient consider changing the way
your
method works: testing internal state is, at least, going to make
maintaining
your tests very very hard since they’re so cozy with your impl.

cheers.

-a

11

On Thu, 2007-01-04 at 07:15 +0900, Kenosis wrote:

i’m not one of those people who think the mere mention of the word
end
from a.rb:11
class C
mutex = Mutex.new

harp:~ > cat a.rb

thread local mutex to be created that’s only being used by that thread
and none of the others, because they have their own mutex. The threads
need to share the SAME mutex in order to enable mutual exclusion
between them, ie, the mutex you create in initialize(). That said, us
better understanding what you actually want to unit test would help us
to help you.

Cheers,

Ken

Since We are calling new each time, even using instance variables won’t
solve the race condition. I have even tried using class variables, and I
am still getting race condition.

Ara, please demonstrate how you would solve the race condition in above
shown code of yours?

12

On Thu, 4 Jan 2007, Hemant K. wrote:

Since We are calling new each time, even using instance variables won’t
solve the race condition. I have even tried using class variables, and I am
still getting race condition.

Ara, please demonstrate how you would solve the race condition in above
shown code of yours?

hi hemant-

without thinking too hard about it i’d probably do something like this:

harp:~ > cat a.rb
require ‘sync’
require ‘thread’

class Module
def tattr a
module_eval <<-code
def #{ a }= arg
synchronize(:EX){ @#{ a } = arg }
end
def #{ a }
synchronize(:SH){ @#{ a } }
end
code
end
end

class C
include Sync_m

 tattr :thread

 def initialize
   @thread = nil
   sync_initialize
 end

 def new_thread
   q = Queue.new
   Thread.new{ q.push( self.thread = Thread.current ) }
 ensure
   q.pop
 end

end

4242.times{|i| raise “race @ loop #{ i } condition!” unless((c =
C.new) and (Thread === c.new_thread) and c.thread) }

p 42

harp:~ > ruby a.rb
42

harp:~ > ruby a.rb
42

harp:~ > ruby a.rb
42

harp:~ > ruby a.rb
42

the methodology of waiting for the ‘q.push’ is useful so your method
only
returns after the thread has ‘started’ running.

kind regards.

-a

13

Hemant K. schrieb:

(…)
Earlier, I was writing Unit Tests for a networking application which i
wrote using EventMachine. Now since, EM is completely based on
callbacks, you can’t do a check on return values of methods. Hence I had
to rely on doing asserts on instance variables.

I would love to know, how do i go about unit testing in such cases. I
can’t yet grasp concept of code that can be unit tested easily i guess.

Hemant, in this case I’d try to use a framework like Mocha to create a
mock for EventMachine. Then you can invoke the callbacks from the test
code and verify that your application reacts as expected. I’m sure the
results of invoking the callbacks can be examined from the outside,
without having to look at instance variables.

Regards,
Pit