Options for running code in parallel

Our testing department wants a small tool that can make a number of
http calls simultaneously to the same server, the basic logic being

-----------------------------------------------------------------

calls = [“call1”…“calln”].map {|x| “http://server/#{x}”}

calls.each do |x|
OnSignal(START) do
http_send call(x)
log_result
end
end

SendSignal(START)

-----------------------------------------------------------------

What are the options for doing this cleanly and robustly? I haven’t
been keeping up with the recent work people have been doing on
distributed/parallel code in ruby, but I get the general impression
there’s been a lot of activity in this area.

martin

Martin DeMello wrote:

Our testing department wants a small tool that can make a number of
http calls simultaneously to the same server

Before writing your own, you may want to take a look at existing tools
like:

If you still want to build your own, please review the last couple days
of ruby-talk mailing list archives because there’s been some spirited
debate about fork/thread that may be useful.

-igal

On Jul 1, 2008, at 11:52 AM, Martin DeMello wrote:

end
martin

give this a whack:

cfp:~> cat a.rb
module Threadify
require ‘thread’

@threads = 8
@abort_on_exception = true

class << self
attr_accessor :threads
attr_accessor :abort_on_exception
end
end

module Enumerable
def threadify opts = {}, &block
opts = {:threads => opts} if Numeric === opts
threads = Integer(opts[:threads] || opts[‘threads’] ||
Threadify.threads)
done = Object.new.freeze
jobs = Queue.new

 each_with_index{|elem, i| jobs.push [elem, i]}
 threads.times{ jobs.push done} # mark the end

 consumers = Array.new threads

 threads.times do |i|
   consumers[i] = Thread.new do
     this = Thread.current
     this.abort_on_exception = Threadify.abort_on_exception
     loop{
       job = jobs.pop
       this.exit if job == done
       args = job.first
       jobs << (job << block.call(*args))
     }
   end
 end

 consumers.map{|t| t.join}
 jobs.push done

 ret = []
 while((job = jobs.pop) != done)
   elem, i, value = job
   ret[i] = value
 end
 ret

end
end

class Thread
def Thread.ify enumerable, *args, &block
enumerable.send :threadify, *args, &block
end
end

if FILE == $0
require ‘open-uri’
require ‘yaml’

uris = %w( http://google.com http://yahoo.com http://rubyforge.org/
http://ruby-lang.org
)

Thread.ify uris, :threads => 3 do |uri|
body = open(uri){|pipe| pipe.read}
y uri => body.size
end
end

cfp:~> ruby a.rb

http://yahoo.com: 9562

http://google.com: 6290

http://rubyforge.org/: 22352

http://ruby-lang.org: 9984

soon to be released as a gem.

a @ http://codeforpeople.com/

On Jul 1, 2008, at 2:24 PM, Martin DeMello wrote:

Thanks, will take it for a spin!

just threw a gem up too - grab it from rubyforge by hand until it
propagates…

cheers.

a @ http://codeforpeople.com/

On Tue, Jul 1, 2008 at 11:50 AM, ara.t.howard [email protected]
wrote:

give this a whack:

Thanks, will take it for a spin!

martin

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs