Forum: Ruby Problem in Unit Testing Methods that start new threads

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
87e41d0d468ad56a3b07d9a6482fd6d5?d=identicon&s=25 Hemant Kumar (gnufied)
on 2007-01-03 14:32
(Received via mailing list)
I have a bit of doubt, in Unit Testing Programs that start new threads.
Please have a look at the code below:


class Foobar

  def hello_world
    p "Hello World"
    @thread_status = false
  end

  def new_thread_start
    Thread.new do
      sleep(100)
      @thread_status = true
    end
  end

end

# here goes the lame test case
require "test/unit"

module Test::Unit::Assertions
  def assert_false(t_object,message=nil)
    boolean = !t_object
    full_message = build_message(message,'<?> Object is not
false',boolean)
    assert_block(full_message) { boolean}
  end
end

class TestFoobar < Test::Unit::TestCase
  def setup
    @foo = Foobar.new
    class << @foo
      def ivar var
        instance_variable_get(:"@#{var}")
      end
    end
  end

  def test_hello_world
    @foo.hello_world
    assert_false @foo.ivar(:thread_status)
  end


  def test_new_thread
    # sorry for a bit of not so DRY thingy
    @foo.hello_world
    assert_false @foo.ivar(:thread_status)
    @foo.new_thread_start

    # next assert is true because method was started in a new thread
    # and control came back immediately, what i would probably want is
    # to wait here so that i can have proper check on the method and
    # state of the program, but i am not sure, if that's exactly
    # approach i should take.

    assert_false @foo.ivar(:thread_status)
  end
end


Now, as someone suggested on IRC, I can do a join and wait for the
thread to finish. But the problem is, I don't exactly have an instance
to the thread, because its managed by a plugin and i am just using the
plugin to do stuff.

Any ideas/suggestions are more than welcome.


gnufied
87e41d0d468ad56a3b07d9a6482fd6d5?d=identicon&s=25 Hemant Kumar (gnufied)
on 2007-01-03 14:52
(Received via mailing list)
On Wed, 2007-01-03 at 22:30 +0900, Hemant Kumar wrote:
>
> require "test/unit"
> class TestFoobar < Test::Unit::TestCase
>     @foo.hello_world
>     # next assert is true because method was started in a new thread
> Now, as someone suggested on IRC, I can do a join and wait for the
> thread to finish. But the problem is, I don't exactly have an instance
> to the thread, because its managed by a plugin and i am just using the
> plugin to do stuff.
>
> Any ideas/suggestions are more than welcome.

Or is it this a good idea?

 Thread.list.each { |t| t.join }
Cb48ca5059faf7409a5ab3745a964696?d=identicon&s=25 unknown (Guest)
on 2007-01-03 17:05
(Received via mailing list)
On Wed, 3 Jan 2007, Hemant Kumar wrote:

>  def new_thread_start
>
>  def setup
>    assert_false @foo.ivar(:thread_status)
>    # and control came back immediately, what i would probably want is
> thread to finish. But the problem is, I don't exactly have an instance
> to the thread, because its managed by a plugin and i am just using the
> plugin to do stuff.
>
> Any ideas/suggestions are more than welcome.

it seems that all you've managed to do is write a very long race
condition.
i'm not one of those people who think the mere mention of the word
'unit-testing' bestows any sort of robustness on code.  for example, the
testing of 'thread_status' in your test means nothing, as it's the
source of
the race condition: you need to wrap setting and reading this var with a
semaphore.  this shows why:

   harp:~ > cat a.rb
   class C
     attr :thread
     def initialize
       @thread = nil
     end
     def new_thread
       Thread.new{ @thread = Thread.current }
     end
   end

   4242.times{|i| raise "race condition @ loopno #{ i }!" unless((c =
C.new) and (Thread === c.new_thread) and c.thread) }


   harp:~ > ruby a.rb
   a.rb:11: race condition @ loopno 942! (RuntimeError)
           from a.rb:11


regarding your specific question though, you need to verify that a
thread is
created and that it's status is running.  even if you don't have a
handle on
the thread you can set things up in your unit test to get one.
something like

   harp:~ > cat a.rb
   def tracking_threads &b
     before = Thread.list
     yield
     after = Thread.list
     return after - before
   end

   threads = tracking_threads{ 2.times{ Thread.new{ sleep } } }

   threads.each{|t| p t.status}


   harp:~ > ruby a.rb
   "sleep"
   "sleep"


cool.  now we've managed to shows that Thread.new works!  ;-)

i wouldn't bother with this at all unless i could also test that the
created
thread did the right thing.

my 2 cts.

kind regards.

-a
Fc784eadb3b54531fdc3d2053db6f83f?d=identicon&s=25 Mat Schaffer (Guest)
on 2007-01-03 17:35
(Received via mailing list)
On Jan 3, 2007, at 8:30 AM, Hemant Kumar wrote:

>   end
> # here goes the lame test case
>
>   def test_hello_world
>
>
> Now, as someone suggested on IRC, I can do a join and wait for the
> thread to finish. But the problem is, I don't exactly have an instance
> to the thread, because its managed by a plugin and i am just using the
> plugin to do stuff.
>
> Any ideas/suggestions are more than welcome.


Ara's right on this one.  It's a race condition.  But it looks like
you're interested in checking that the thread has indeed started.  If
I were to write something like this I would probably have the unit
test run @foo.new_thread_start and immediately sleep waiting for a
signal from the child thread.  The child thread would then signal
back to the any waiting threads that it had set it's thread status to
true.  Altough the more I ponder it, the more I feel like I've just
moved the race condition to the sleep call.  I'll have to review my
concurrent programming texts a bit.  Seems like you should do the same.

Either way, check out Monitor for some insight:
http://www.ruby-doc.org/stdlib/libdoc/monitor/rdoc...

-Mat
Cb48ca5059faf7409a5ab3745a964696?d=identicon&s=25 unknown (Guest)
on 2007-01-03 17:41
(Received via mailing list)
On Thu, 4 Jan 2007, Mat Schaffer wrote:

>
> Either way, check out Monitor for some insight:
> http://www.ruby-doc.org/stdlib/libdoc/monitor/rdoc...
>
> -Mat

good idea mat.  it's easy in ruby too:

   require 'thread'

   q = Queue.new

   Thread.new{
     q.push :running
     # run
   }

   q.pop # wait for thread to start...

regards.

-a
87e41d0d468ad56a3b07d9a6482fd6d5?d=identicon&s=25 Hemant Kumar (gnufied)
on 2007-01-03 18:23
(Received via mailing list)
On Thu, 2007-01-04 at 01:04 +0900, ara.t.howard@noaa.gov wrote:
> testing of 'thread_status' in your test means nothing, as it's the source of
>        Thread.new{ @thread = Thread.current }
>
Thanks for the insight Ara, although my class is singleton so i
shouldn't get the problem you have described above.

However can you show me a code sample, that doesn't cause race condition
in above code. I tried using Mutex on above code,

require "thread"
class C
  attr :thread
  def initialize
    mutex = Mutex.new
    mutex.lock
    @thread = nil
    mutex.unlock
  end

  def new_thread
    Thread.new{
      mutex = Mutex.new
      mutex.lock
      @thread = Thread.current
      mutex.unlock
    }
  end

end

4242.times{|i| raise "race condition @ loopno #{ i }!" unless((c =
C.new) and (Thread === c.new_thread) and c.thread) }

And I am still getting a race condition. Using MonitorMixin to signal
execution seems like an overkill to me in above code.

So can you show me a way of making above code not race.


>    end
>
Cool thanks.
93d566cc26b230c553c197c4cd8ac6e4?d=identicon&s=25 Pit Capitain (Guest)
on 2007-01-03 22:58
(Received via mailing list)
Hemant Kumar schrieb:
> I have a bit of doubt, in Unit Testing Programs that start new threads.
> Please have a look at the code below:
> (...)

Hemant, in addition to what the others said, it's not clear to me what
you really want to test: whether a new thread is started, whether the
instance variable changed after a certain amount of time, whether the
instance variable changed at the end of the new thread, ...

I'm a fan of black box unit tests, so for me, the tests should specify
what you expect from the *interface* of the object under test. I
wouldn't test for values of certain instance variables or for threads
created internally, unless those are part of the desired interface of
your objects.

Regards,
Pit
5ecfb4046c8d8333bdd13464b3d123c0?d=identicon&s=25 Kenosis (Guest)
on 2007-01-03 23:16
(Received via mailing list)
Hemant Kumar wrote:
> > 'unit-testing' bestows any sort of robustness on code.  for example, the
> >      def new_thread
> >
>   attr :thread
>       mutex.lock
> And I am still getting a race condition. Using MonitorMixin to signal
> >    def tracking_threads &b
> >
> >    harp:~ > ruby a.rb
> >    "sleep"
> >    "sleep"
> >
> >
>
> Cool thanks.

Try removing the "mutex = Mutex.new" from "new_thread" That's causing a
thread local mutex to be created that's only being used by that thread
and none of the others, because they have their own mutex.  The threads
need to share the SAME mutex in order to enable mutual exclusion
between them, ie, the mutex you create in initialize().  That said, us
better understanding what you actually want to unit test would help us
to help you.

Cheers,

Ken
87e41d0d468ad56a3b07d9a6482fd6d5?d=identicon&s=25 Hemant Kumar (gnufied)
on 2007-01-04 02:09
(Received via mailing list)
On Thu, 2007-01-04 at 06:57 +0900, Pit Capitain wrote:
> I'm a fan of black box unit tests, so for me, the tests should specify
> what you expect from the *interface* of the object under test. I
> wouldn't test for values of certain instance variables or for threads
> created internally, unless those are part of the desired interface of
> your objects.
>
> Regards,
> Pit
>

Agreed Pit, but basically I want to test state of my program through
Unit Tests and whether each method modifies state of program as it was
intended?

Now, unit testing methods that return something on invocation is easy
and i don't need to go around running asserts on instance variables.

But I have some doubt regarding testing methods that don't return
anything explicitly and rather update instance variables.

Earlier, I was writing Unit Tests for a networking application which i
wrote using EventMachine. Now since, EM is completely based on
callbacks, you can't do a check on return values of methods. Hence I had
to rely on doing asserts on instance variables.

I would love to know, how do i go about unit testing in such cases. I
can't yet grasp concept of code that can be unit tested easily i guess.



gnufied
Cb48ca5059faf7409a5ab3745a964696?d=identicon&s=25 unknown (Guest)
on 2007-01-04 02:17
(Received via mailing list)
On Thu, 4 Jan 2007, Hemant Kumar wrote:

>
> Earlier, I was writing Unit Tests for a networking application which i
> wrote using EventMachine. Now since, EM is completely based on
> callbacks, you can't do a check on return values of methods. Hence I had
> to rely on doing asserts on instance variables.
>
> I would love to know, how do i go about unit testing in such cases. I
> can't yet grasp concept of code that can be unit tested easily i guess.
>

often methods should do one of two things:

   - succeed or throw an exception

   - return a value indicating success

in your case the former should be sufficient - you can assert that
nothing is
raised and move on.  if that's not sufficient consider changing the way
your
method works: testing internal state is, at least, going to make
maintaining
your tests very very hard since they're so cozy with your impl.

cheers.

-a
87e41d0d468ad56a3b07d9a6482fd6d5?d=identicon&s=25 Hemant Kumar (gnufied)
on 2007-01-04 04:49
(Received via mailing list)
On Thu, 2007-01-04 at 07:15 +0900, Kenosis wrote:
> > > i'm not one of those people who think the mere mention of the word
> > >      end
> > >            from a.rb:11
> > class C
> >       mutex = Mutex.new
> >
> > >    harp:~ > cat a.rb
> > >
> thread local mutex to be created that's only being used by that thread
> and none of the others, because they have their own mutex.  The threads
> need to share the SAME mutex in order to enable mutual exclusion
> between them, ie, the mutex you create in initialize().  That said, us
> better understanding what you actually want to unit test would help us
> to help you.
>
> Cheers,
>
> Ken
>

Since We are calling new each time, even using instance variables won't
solve the race condition. I have even tried using class variables, and I
am still getting race condition.

Ara, please demonstrate how you would solve the race condition in above
shown code of yours?
Cb48ca5059faf7409a5ab3745a964696?d=identicon&s=25 unknown (Guest)
on 2007-01-04 05:13
(Received via mailing list)
On Thu, 4 Jan 2007, Hemant Kumar wrote:

>
> Since We are calling new each time, even using instance variables won't
> solve the race condition. I have even tried using class variables, and I am
> still getting race condition.
>
> Ara, please demonstrate how you would solve the race condition in above
> shown code of yours?
>

hi hemant-

without thinking too hard about it i'd probably do something like this:


   harp:~ > cat a.rb
   require 'sync'
   require 'thread'

   class Module
     def tattr a
       module_eval <<-code
         def #{ a }= arg
           synchronize(:EX){ @#{ a } = arg }
         end
         def #{ a }
           synchronize(:SH){ @#{ a } }
         end
       code
     end
   end

   class C
     include Sync_m

     tattr :thread

     def initialize
       @thread = nil
       sync_initialize
     end

     def new_thread
       q = Queue.new
       Thread.new{ q.push( self.thread = Thread.current ) }
     ensure
       q.pop
     end
   end

   4242.times{|i| raise "race @ loop #{ i } condition!" unless((c =
C.new) and (Thread === c.new_thread) and c.thread) }

   p 42

   harp:~ > ruby a.rb
   42

   harp:~ > ruby a.rb
   42

   harp:~ > ruby a.rb
   42

   harp:~ > ruby a.rb
   42


the methodology of waiting for the 'q.push' is useful so your method
only
returns after the thread has 'started' running.

kind regards.

-a
93d566cc26b230c553c197c4cd8ac6e4?d=identicon&s=25 Pit Capitain (Guest)
on 2007-01-06 19:30
(Received via mailing list)
Hemant Kumar schrieb:
> (...)
> Earlier, I was writing Unit Tests for a networking application which i
> wrote using EventMachine. Now since, EM is completely based on
> callbacks, you can't do a check on return values of methods. Hence I had
> to rely on doing asserts on instance variables.
>
> I would love to know, how do i go about unit testing in such cases. I
> can't yet grasp concept of code that can be unit tested easily i guess.

Hemant, in this case I'd try to use a framework like Mocha to create a
mock for EventMachine. Then you can invoke the callbacks from the test
code and verify that your application reacts as expected. I'm sure the
results of invoking the callbacks can be examined from the outside,
without having to look at instance variables.

Regards,
Pit
This topic is locked and can not be replied to.