Ruby Threading question Creating 4 threads

Ruby
1

Newbie Question:

I have some code where I create 4 threads that pop elements off an
array and process them. The problem is that each thread only pops off
one element, processes it, then returns; they don’t continue to
process until the entire array is consumed.

$jobs = %w[a b c d e f g] def simulate_cleanup

puts “Checking for #{$jobs.length} jobs in /mnt/gen1/focal/temp”
threads = []

mu = Mutex.new
total = 0

1.upto(4) do |c|
threads << Thread.new do
puts “Thread #{c} created”
job = nil

  mu.synchronize do
    job = $jobs.pop
    puts "Checking for #{job}"
    #Exit our thread if we have no more jobs to check
    puts "No more jobs" and return if job.nil?
  end

  t = count_files(job)
  mu.synchronize { total += t }
end

end
threads.each { |t| t.join }

puts “Reduced by #{total} files”
end

Example of output:
Thread 1 created
Thread 2 created
Thread 3 created
Thread 4 created
Checking for a
Checking for b
Checking for c
Checking for d
2

what if you put your thread body in an infinite loop, something like:

Thread.new do
loop do
puts “Thread #{c} created”
job = nil

 mu.synchronize do
   job = $jobs.pop
   puts "Checking for #{job}"
   #Exit our thread if we have no more jobs to check
   puts "No more jobs" and Thread.exit if job.nil?
 end

 t = count_files(job)
 mu.synchronize { total += t }

end
end

and as you may have noticed, I replaced the ‘return’ statement inside
your
thread with a ‘Thread.exit’ method.
I think it would be the right way to do it…

2009/2/16 [email protected]

3

and looking at it more, I think it would be better to leave the less
stuff
in the mutex block as possible,
making it something something like:

mu.synchronize do
job = $jobs.pop
end

puts “Checking for #{job}”
#Exit our thread if we have no more jobs to check
puts “No more jobs” and Thread.exit if job.nil?

I guess the Thread.exit could have caused a deadlock if inside the
mutex,
and since the mutex locks your other threads you better keep it as small
as
possible for efficiency sake.

2009/2/16 Louis-Philippe [email protected]

4

Put a while around the pop and block of code to run.

Blog: http://random8.zenunit.com/
Learn rails: http://sensei.zenunit.com/

5

On 17.02.2009 04:38, Julian L. wrote:

Put a while around the pop and block of code to run.

Also class Queue has thread safety built in already.

Kind regards

robert

6

That won’t ever exit the thread because puts returns nil.

Blog: http://random8.zenunit.com/
Learn rails: http://sensei.zenunit.com/

7

Robert K. wrote:

On 17.02.2009 04:38, Julian L. wrote:

Put a while around the pop and block of code to run.

Also class Queue has thread safety built in already.

Kind regards

robert

…an example of using a Queue:

require ‘thread’

arr = %w[a b c d e f g h i j]
num_threads = 4
q = Queue.new

workers = []

num_threads.times do |i|
workers << Thread.new(“worker#{i+1}”) do |name|
loop do
data = q.pop #pops data off bottom of q, blocks if no data
puts “#{name} retrieved data: #{data}”
break if data == “END_OF_DATA_FLAG”
end
end
end

arr.each {|item| q.push(item)}
num_threads.times do
q.push(“END_OF_DATA_FLAG”)
end

workers.each {|thr| thr.join}
puts “Workers are all done working.”

–output:–
worker1 retrieved data: a
worker2 retrieved data: b
worker3 retrieved data: c
worker4 retrieved data: d
worker1 retrieved data: e
worker2 retrieved data: f
worker3 retrieved data: g
worker4 retrieved data: h
worker1 retrieved data: i
worker2 retrieved data: j
worker3 retrieved data: END_OF_DATA_FLAG
worker4 retrieved data: END_OF_DATA_FLAG
worker1 retrieved data: END_OF_DATA_FLAG
worker2 retrieved data: END_OF_DATA_FLAG
Workers are all done working.