Forum: Ruby threading sync

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Mage (Guest)
on 2006-01-25 13:45
(Received via mailing list)
Hello,

I was thinking of writing my first threaded app in Ruby. One thread
should fetch data from the database, another should write it to a file.

I would like to use an array, which has hash elements. I append new
elements to the end, and I write to the file from the beginning (FIFO).

My question is that do I really need mutex? Is appending a hash to the
end of an array atomic, or not? How can I know which method is atomic?

       Mage
Robert K. (Guest)
on 2006-01-25 14:06
(Received via mailing list)
Mage wrote:
> My question is that do I really need mutex? Is appending a hash to the
> end of an array atomic, or not? How can I know which method is atomic?

In your case you probably better use a Queue which saves you the
synchronization overhead (the Queue does it internally). Something along
the lines of this (untested):

require 'thread'

QUEUE = Queue.new

threads = [
  Thread.new(QUEUE) do |queue|
    File.open("foo", "w") do |io|
      while x = queue.deq
        io.puts x
      end
    end
  end,
  Thread.new(QUEUE) do |queue|
    while rec = fetch_from_db()
      queue.enq rec
    end
    queue.enc false
  end
]

thread.each {|th|th.join}

Note, if you like you can increase the number of DB writer and / or file
reader threads. You can as well leave the db reader in the main loop if
you need only one. Depending on your situation you may need a different
end of processing detection.

Kind regards

    robert
Mage (Guest)
on 2006-01-25 14:55
(Received via mailing list)
Robert K. wrote:

>
>In your case you probably better use a Queue which saves you the
>synchronization overhead (the Queue does it internally). Something along
>the lines of this (untested):
>
>
Thank you. I see.

However, in cases when there is no existing solution, should I write a
tester unit that determines about a method its atomicness?

       Mage
Robert K. (Guest)
on 2006-01-25 15:10
(Received via mailing list)
Mage wrote:
> However, in cases when there is no existing solution, should I write a
> tester unit that determines about a method its atomicness?

IMHO it's generally a bad idea to determine thread safety properties of
code by testing. The test may fail, in which case you got a definitve
answer (it's unsafe). But the test may succeed and then you don't know
whether it was by accident (timing) or by design / implementation.

You could look at the source code of Ruby but on one hand that may
change
and on the other hand there will be definitively issues when Ruby 2.0
comes out and supports native threads.

If you build a MT application you should design MT right into it, which
also means that you have to think about which resources (aka objects)
are
accessed from multiple threads and how you synchronize access to them.

Kind regards

    robert
James G. (Guest)
on 2006-01-25 15:52
(Received via mailing list)
On Jan 25, 2006, at 5:41 AM, Mage wrote:

> Is appending a hash to the end of an array atomic, or not? How can
> I know which method is atomic?

In general, assume nothing is atomic in Ruby.  Even a simple
expression like:

my_val = 5

Is many steps inside the interpreter.  If you're going to thread,
handle all shared resources accordingly.

James Edward G. II
Erik V. (Guest)
on 2006-01-25 16:13
(Received via mailing list)
> My question is that do I really need mutex?

No, you don't.

You could as well use Thread.exclusive. This prevents other,
excisting threads from being run. If the given block can't get
blocked (e.g. by IO), it's much faster then Mutex.

gegroet,
Erik V. - http://www.erikveen.dds.nl/
Eric H. (Guest)
on 2006-01-26 21:31
(Received via mailing list)
On Jan 25, 2006, at 5:28 AM, Erik V. wrote:

>> My question is that do I really need mutex?
>
> No, you don't.
>
> You could as well use Thread.exclusive. This prevents other,
> excisting threads from being run. If the given block can't get
> blocked (e.g. by IO), it's much faster then Mutex.

No, Thread.exclusive (which uses Thread.critical) does not prevent
other threads from being run.  It prevents them from being scheduled
automatically.  You can still switch threads manually while
Thread.critical is true.

$ cat exclusive.rb
require 'thread'

Thread.exclusive do
   Thread.new { sleep 10 }
   puts 'should be instant'
end

$ time ruby exclusive.rb
should be instant

real    0m10.061s
user    0m0.016s
sys     0m0.013s
$ cat mutex.rb
require 'thread'

m = Mutex.new

m.synchronize do
   Thread.new do
     m.synchronize do sleep 10 end
   end
   puts 'should be instant'
end

$ time ruby mutex.rb
should be instant

real    0m0.060s
user    0m0.016s
sys     0m0.014s


--
Eric H. - removed_email_address@domain.invalid - http://segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com
This topic is locked and can not be replied to.