Forum: Ruby Ruby Threading question Creating 4 threads

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Ddfa87f4f2e9cafd84fa804d2ce26bd9?d=identicon&s=25 Tristin Davis (tristincolby)
on 2009-02-16 22:52
(Received via mailing list)
Newbie Question:

I have some code where I create 4 threads that pop elements off an
array and process them.  The problem is that each thread only pops off
one element, processes it, then returns; they don't continue to
process until the entire array is consumed.

<code>
$jobs = %w[a b c d e f g]
def simulate_cleanup

  puts "Checking for #{$jobs.length} jobs in /mnt/gen1/focal/temp"
  threads = []

  mu = Mutex.new
  total = 0

  1.upto(4) do |c|
     threads << Thread.new do
      puts "Thread #{c} created"
      job = nil

      mu.synchronize do
        job = $jobs.pop
        puts "Checking for #{job}"
        #Exit our thread if we have no more jobs to check
        puts "No more jobs" and return if job.nil?
      end

      t = count_files(job)
      mu.synchronize { total += t }
    end
  end
  threads.each { |t| t.join }

  puts "Reduced by #{total} files"
end

Example of output:
Thread 1 created
Thread 2 created
Thread 3 created
Thread 4 created
Checking for a
Checking for b
Checking for c
Checking for d
54185df1d348bbd34587fcd4f8e4779b?d=identicon&s=25 Louis-Philippe (Guest)
on 2009-02-16 23:11
(Received via mailing list)
what if you put your thread body in an infinite loop, something like:

Thread.new do
  loop do
    puts "Thread #{c} created"
     job = nil

     mu.synchronize do
       job = $jobs.pop
       puts "Checking for #{job}"
       #Exit our thread if we have no more jobs to check
       puts "No more jobs" and Thread.exit if job.nil?
     end

     t = count_files(job)
     mu.synchronize { total += t }
  end
end

and as you may have noticed, I replaced the 'return' statement inside
your
thread with a 'Thread.exit' method.
I think it would be the right way to do it...

2009/2/16 <tristin.colby@gmail.com>
54185df1d348bbd34587fcd4f8e4779b?d=identicon&s=25 Louis-Philippe (Guest)
on 2009-02-16 23:20
(Received via mailing list)
and looking at it more, I think it would be better to leave the less
stuff
in the mutex block as possible,
making it something something like:

mu.synchronize do
       job = $jobs.pop
end

puts "Checking for #{job}"
#Exit our thread if we have no more jobs to check
puts "No more jobs" and Thread.exit if job.nil?

I guess the Thread.exit could have caused a deadlock if inside the
mutex,
and since the mutex locks your other threads you better keep it as small
as
possible for efficiency sake.

2009/2/16 Louis-Philippe <default@spiralix.org>
3131fcea0a711e5ad89c8d49cc9253b4?d=identicon&s=25 Julian Leviston (Guest)
on 2009-02-17 04:39
(Received via mailing list)
Put a while around the pop and block of code to run.

Blog: http://random8.zenunit.com/
Learn rails: http://sensei.zenunit.com/
3131fcea0a711e5ad89c8d49cc9253b4?d=identicon&s=25 Julian Leviston (Guest)
on 2009-02-17 04:41
(Received via mailing list)
That won't ever exit the thread because puts returns nil.

Blog: http://random8.zenunit.com/
Learn rails: http://sensei.zenunit.com/
E0d864d9677f3c1482a20152b7cac0e2?d=identicon&s=25 Robert Klemme (Guest)
on 2009-02-17 08:25
(Received via mailing list)
On 17.02.2009 04:38, Julian Leviston wrote:
> Put a while around the pop and block of code to run.

Also class Queue has thread safety built in already.

Kind regards

  robert
54404bcac0f45bf1c8e8b827cd9bb709?d=identicon&s=25 7stud -- (7stud)
on 2009-02-17 15:14
Robert Klemme wrote:
> On 17.02.2009 04:38, Julian Leviston wrote:
>> Put a while around the pop and block of code to run.
>
> Also class Queue has thread safety built in already.
>
> Kind regards
>
>   robert

...an example of using a Queue:

require 'thread'

arr = %w[a b c d e f g h i j]
num_threads = 4
q = Queue.new

workers = []

num_threads.times do |i|
  workers << Thread.new("worker#{i+1}") do |name|
    loop do
      data = q.pop    #pops data off bottom of q, blocks if no data
      puts "#{name} retrieved data: #{data}"
      break if data == "*END_OF_DATA_FLAG*"
    end
  end
end

arr.each {|item| q.push(item)}
num_threads.times do
  q.push("*END_OF_DATA_FLAG*")
end

workers.each {|thr| thr.join}
puts "Workers are all done working."


--output:--
worker1 retrieved data: a
worker2 retrieved data: b
worker3 retrieved data: c
worker4 retrieved data: d
worker1 retrieved data: e
worker2 retrieved data: f
worker3 retrieved data: g
worker4 retrieved data: h
worker1 retrieved data: i
worker2 retrieved data: j
worker3 retrieved data: *END_OF_DATA_FLAG*
worker4 retrieved data: *END_OF_DATA_FLAG*
worker1 retrieved data: *END_OF_DATA_FLAG*
worker2 retrieved data: *END_OF_DATA_FLAG*
Workers are all done working.
This topic is locked and can not be replied to.