Multitasking and collecting results

Ruby
1

(new to ruby)

I have a routine that makes a web call then does some work. I need to do
many of these, and because of latency and all that, I want to go ahead
and spawn several of these in async “tasks” (trying to avoid key words
like processes and threads). I need to collect the results of each task.

So, generically, let’s say I have a TaskManager and a Task. TaskManager
is initialized with an array of elements to process. Each element is
passed to a Task for processing. I want several Tasks processing in
parallel, and the result from each Task must get reported back to
TaskManager.

I see Ruby threads is one way, and despite the disparaging commens I see
about them, they’re probably fine for me to get started with to figure
out the “feedback” mechanisms, but eventually I do need a method which
is capable of leveraging multi-core machines.

Using the pickaxe book Multithreading chapter, I’ve put together this
little outline, but the results always come back in order when I believe
useing sleep should randomize them, so I’m suspicioius that I’m not
really getting async processing out of my code.

#! /usr/local/bin/ruby

class TaskManager

attr_reader :results

def manageTasks
taskList = [‘one’, ‘two’, ‘three’]
threads = []
@results = []
indx = 0

taskList.each do |taskParam|
  threads[indx] = Thread.new do
    thisTask = Task.new
    @results[indx] = thisTask.doStuff(taskParam)
  end
  indx += 1
end

threads.each {|thred| thred.join}

end
end

class Task
def doStuff(taskParam)
pause = ((rand(0.1))*4)
sleep pause
return ('handing back ’ + taskParam + ', ’ + pause.to_s)
end
end

taskMngr = TaskManager.new
taskMngr.manageTasks
puts taskMngr.results

Looking for hints as to what I am doing wrong.

Can anyone point me to a tutorial using a technique that will use
multi-core CPUs?

Many thanks.

– gw

2

Greg W. wrote:

I’ve put together this
little outline, but the results always come back in order when I believe
useing sleep should randomize them, so I’m suspicioius that I’m not
really getting async processing out of my code.

Oh wait, they’re in order because that’s the order I have them inserted
into @results. Duh.

I confirmed async by putting a timer wrapper around the whole thing, and
indeed it completes in less time than the sum of each task’s pause time.

– gw

3

Greg W. wrote:

Greg W. wrote:

I’ve put together this
little outline, but the results always come back in order when I believe
useing sleep should randomize them, so I’m suspicioius that I’m not
really getting async processing out of my code.

Oh wait, they’re in order because that’s the order I have them inserted
into @results. Duh.

I confirmed async by putting a timer wrapper around the whole thing, and
indeed it completes in less time than the sum of each task’s pause time.

Try the following to see some random order in the results. The threads
in the code enter their data in a
Queue (located in the ‘thread’ module in the Standard Library). A Queue
enables you to pass data between threads in a “thread-safe manner”. A
“thread safe manner” means that only one thread can access the Queue at
the same time. A Queue guarantees that two threads can’t access the
Queue at the same time, so the data cannot get scrambled.

require ‘thread’

class TaskManager

attr_reader :results

def initialize(*task_params)
@task_params = task_params
@results = Queue.new
end

def start_producers
threads = []

@task_params.each_with_index do |param, indx|
  threads[indx] = Thread.new do
    @results << Task.new.do_stuff(param)
  end
end

threads.each {|thred| thred.join}

end
end

class Task
def do_stuff(x)
pause = rand() * 4
sleep pause

"handing back #{x}, #{pause}"

end
end

task_mngr = TaskManager.new(‘one’, ‘two’, ‘three’)
task_mngr.start_producers
num_results = task_mngr.results.length
num_results.times {puts task_mngr.results.pop}

Note that rand(.1) is the same as rand(), so I don’t know why pickaxe2
uses rand(.1).

4

Greg W. wrote:

I see Ruby threads is one way, and despite the disparaging commens I see

Ruby threads are a good thing - I wouldn’t disparage them - but beware
that some Windows versions have broken synchronization primitives.

Can anyone point me to a tutorial using a technique that will use
multi-core CPUs?

Yes, but not using the current Ruby interpreter, which uses a single
operating-system thread, so cannot parallelize across a multi-core
system. If you need this with Ruby, you’ll have to look at one of the
new interpreters.

Clifford H…