# Options for running code in parallel

**URL:** <https://www.ruby-forum.com/t/options-for-running-code-in-parallel/141927>\
**Category:** Ruby\
**Created:** [July 1, 2008, 8:35pm UTC](https://www.ruby-forum.com/t/options-for-running-code-in-parallel/141927 "2008-07-01T20:35:33Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Martin\_DeMello](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/f17d59/32.png) [@Martin\_DeMello](https://www.ruby-forum.com/u/Martin_DeMello)\
**Post date:** [July 1, 2008, 8:35pm UTC](https://www.ruby-forum.com/t/options-for-running-code-in-parallel/141927/1 "2008-07-01T20:35:33Z")

</div>

Our testing department wants a small tool that can make a number of  
http calls simultaneously to the same server, the basic logic being

# -----------------------------------------------------------------

calls = [“call1”…“calln”].map {|x| “[http://server/#{x}](http://server/#%7Bx%7D)”}

calls.each do |x|  
OnSignal(START) do  
http\_send call(x)  
log\_result  
end  
end

SendSignal(START)

# -----------------------------------------------------------------

What are the options for doing this cleanly and robustly? I haven’t  
been keeping up with the recent work people have been doing on  
distributed/parallel code in ruby, but I get the general impression  
there’s been a lot of activity in this area.

martin

---

<div class="post-metadata">

**Author:** ![Martin\_DeMello](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/f17d59/32.png) [@Martin\_DeMello](https://www.ruby-forum.com/u/Martin_DeMello)\
**Post date:** [July 1, 2008, 9:06pm UTC](https://www.ruby-forum.com/t/options-for-running-code-in-parallel/141927/2 "2008-07-01T21:06:02Z")

</div>

Martin DeMello wrote:

> Our testing department wants a small tool that can make a number of  
> http calls simultaneously to the same server

Before writing your own, you may want to take a look at existing tools  
like:

- [http://www.hpl.hp.com/research/linux/httperf/](http://www.hpl.hp.com/research/linux/httperf/)
- [ab - Apache HTTP server benchmarking tool - Apache HTTP Server](http://httpd.apache.org/docs/2.0/programs/ab.html)

If you still want to build your own, please review the last couple days  
of ruby-talk mailing list archives because there’s been some spirited  
debate about fork/thread that may be useful.

-igal

---

<div class="post-metadata">

**Author:** ![Martin\_DeMello](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/f17d59/32.png) [@Martin\_DeMello](https://www.ruby-forum.com/u/Martin_DeMello)\
**Post date:** [July 1, 2008, 8:54pm UTC](https://www.ruby-forum.com/t/options-for-running-code-in-parallel/141927/3 "2008-07-01T20:54:54Z")

</div>

On Jul 1, 2008, at 11:52 AM, Martin DeMello wrote:

> end  
> martin

give this a whack:

cfp:~\> cat a.rb  
module Threadify  
require ‘thread’

@threads = 8  
@abort\_on\_exception = true

class \<\< self  
attr\_accessor :threads  
attr\_accessor :abort\_on\_exception  
end  
end

module Enumerable  
def threadify opts = {}, &block  
opts = {:threads =\> opts} if Numeric === opts  
threads = Integer(opts[:threads] || opts[‘threads’] ||  
Threadify.threads)  
done = Object.new.freeze  
jobs = Queue.new

```
 each_with_index{|elem, i| jobs.push [elem, i]}
 threads.times{ jobs.push done} # mark the end

 consumers = Array.new threads

 threads.times do |i|
   consumers[i] = Thread.new do
     this = Thread.current
     this.abort_on_exception = Threadify.abort_on_exception
     loop{
       job = jobs.pop
       this.exit if job == done
       args = job.first
       jobs << (job << block.call(*args))
     }
   end
 end

 consumers.map{|t| t.join}
 jobs.push done

 ret = []
 while((job = jobs.pop) != done)
   elem, i, value = job
   ret[i] = value
 end
 ret

```

end  
end

class Thread  
def Thread.ify enumerable, \*args, &block  
enumerable.send :threadify, \*args, &block  
end  
end

if **FILE** == $0  
require ‘open-uri’  
require ‘yaml’

uris = %w( [http://google.com](http://google.com) [http://yahoo.com](http://yahoo.com) [http://rubyforge.org/](http://rubyforge.org/)  
[http://ruby-lang.org](http://ruby-lang.org)  
)

Thread.ify uris, :threads =\> 3 do |uri|  
body = open(uri){|pipe| pipe.read}  
y uri =\> body.size  
end  
end

## cfp:~\> ruby a.rb

## [http://yahoo.com](http://yahoo.com): 9562

## [http://google.com](http://google.com): 6290

## [http://rubyforge.org/:](http://rubyforge.org/:) 22352

[http://ruby-lang.org](http://ruby-lang.org): 9984

soon to be released as a gem.

a @ [http://codeforpeople.com/](http://codeforpeople.com/)

---

<div class="post-metadata">

**Author:** ![Martin\_DeMello](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/f17d59/32.png) [@Martin\_DeMello](https://www.ruby-forum.com/u/Martin_DeMello)\
**Post date:** [July 1, 2008, 11:18pm UTC](https://www.ruby-forum.com/t/options-for-running-code-in-parallel/141927/4 "2008-07-01T23:18:06Z")

</div>

On Jul 1, 2008, at 2:24 PM, Martin DeMello wrote:

> Thanks, will take it for a spin!

just threw a gem up too - grab it from rubyforge by hand until it  
propagates…

cheers.

a @ [http://codeforpeople.com/](http://codeforpeople.com/)

---

<div class="post-metadata">

**Author:** ![Martin\_DeMello](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/f17d59/32.png) [@Martin\_DeMello](https://www.ruby-forum.com/u/Martin_DeMello)\
**Post date:** [July 1, 2008, 11:03pm UTC](https://www.ruby-forum.com/t/options-for-running-code-in-parallel/141927/5 "2008-07-01T23:03:27Z")

</div>

On Tue, Jul 1, 2008 at 11:50 AM, ara.t.howard [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
wrote:

> give this a whack:

Thanks, will take it for a spin!

martin