# Forkoff-1.1.0 **URL:** \ **Category:** Ruby\ **Created:** [October 12, 2009, 5:59am UTC](https://www.ruby-forum.com/t/forkoff-1-1-0/177127 "2009-10-12T05:59:03Z")\ **Posts on this page:** 1\ **Page:** 1
**Author:** ![ahoward](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/779978/32.png) [@ahoward](https://www.ruby-forum.com/u/ahoward)\ **Post date:** [October 12, 2009, 5:59am UTC](https://www.ruby-forum.com/t/forkoff-1-1-0/177127/1 "2009-10-12T05:59:03Z")
NAME forkoff SYNOPSIS brain-dead simple parallel processing for ruby URI [http://rubyforge.org/projects/codeforpeople](http://rubyforge.org/projects/codeforpeople) [GitHub - ahoward/forkoff: brain-dead simple parallel processing for ruby](http://github.com/ahoward/forkoff) INSTALL gem install forkoff DESCRIPTION forkoff works for any enumerable object, iterating a code block to run in a child process and collecting the results. forkoff can limit the number of child processes which is, by default, 2. SAMPLES \<========\< samples/a.rb \>========\> ~ \> cat samples/a.rb ``` # forkoff makes it trivial to do parallel processing with ruby, ``` the following # prints out each word in a separate process # ``` require 'forkoff' %w( hey you ).forkoff!{|word| puts "#{ word } from # ``` { Process.pid }"} ~ \> ruby samples/a.rb ``` hey from 7907 you from 7908 ``` \<========\< samples/b.rb \>========\> ~ \> cat samples/b.rb ``` # for example, this takes only 4 seconds or so to complete (8 ``` iterations # running in two processes = twice as fast) # ``` require 'forkoff' a = Time.now.to_f results = (0..7).forkoff do |i| sleep 1 i ** 2 end b = Time.now.to_f elapsed = b - a puts "elapsed: #{ elapsed }" puts "results: #{ results.inspect }" ``` ~ \> ruby samples/b.rb ``` elapsed: 4.19184589385986 results: [0, 1, 4, 9, 16, 25, 36, 49] ``` \<========\< samples/c.rb \>========\> ~ \> cat samples/c.rb ``` # forkoff does *NOT* spawn processes in batches, waiting for each ``` batch to # complete. rather, it keeps a certain number of processes busy until all # results have been gathered. in otherwords the following will ensure that 3 # processes are running at all times, until the list is complete. note that # the following will take about 3 seconds to run (3 sets of 3 @ 1 second). # ``` require 'forkoff' pid = Process.pid a = Time.now.to_f pstrees = %w( a b c d e f g h i ).forkoff! :processes => 3 do |letter| sleep 1 { letter => ` pstree -l 2 #{ pid } ` } end b = Time.now.to_f puts puts "pid: #{ pid }" puts "elapsed: #{ b - a }" puts require 'yaml' pstrees.each do |pstree| y pstree end ``` ~ \> ruby samples/c.rb ``` pid: 7922 elapsed: 3.37899208068848 --- a: | -+- 07922 ahoward ruby -Ilib samples/c.rb |-+- 07923 ahoward ruby -Ilib samples/c.rb |-+- 07924 ahoward (ruby) \-+- 07925 ahoward ruby -Ilib samples/c.rb --- b: | -+- 07922 ahoward ruby -Ilib samples/c.rb |-+- 07923 ahoward ruby -Ilib samples/c.rb |-+- 07924 ahoward ruby -Ilib samples/c.rb \-+- 07925 ahoward ruby -Ilib samples/c.rb --- c: | -+- 07922 ahoward ruby -Ilib samples/c.rb |-+- 07923 ahoward ruby -Ilib samples/c.rb |-+- 07924 ahoward (ruby) \-+- 07925 ahoward ruby -Ilib samples/c.rb --- d: | -+- 07922 ahoward ruby -Ilib samples/c.rb |-+- 07932 ahoward ruby -Ilib samples/c.rb |--- 07933 ahoward ruby -Ilib samples/c.rb \--- 07934 ahoward ruby -Ilib samples/c.rb --- e: | -+- 07922 ahoward ruby -Ilib samples/c.rb |--- 07932 ahoward (ruby) |-+- 07933 ahoward ruby -Ilib samples/c.rb \-+- 07934 ahoward (ruby) --- f: | -+- 07922 ahoward ruby -Ilib samples/c.rb |--- 07932 ahoward (ruby) |-+- 07933 ahoward ruby -Ilib samples/c.rb \-+- 07934 ahoward ruby -Ilib samples/c.rb --- g: | -+- 07922 ahoward ruby -Ilib samples/c.rb |-+- 07941 ahoward ruby -Ilib samples/c.rb |--- 07942 ahoward ruby -Ilib samples/c.rb \--- 07943 ahoward ruby -Ilib samples/c.rb --- h: | -+- 07922 ahoward ruby -Ilib samples/c.rb |-+- 07941 ahoward (ruby) |-+- 07942 ahoward ruby -Ilib samples/c.rb \--- 07943 ahoward ruby -Ilib samples/c.rb --- i: | -+- 07922 ahoward ruby -Ilib samples/c.rb |--- 07942 ahoward (ruby) \-+- 07943 ahoward ruby -Ilib samples/c.rb ``` \<========\< samples/d.rb \>========\> ~ \> cat samples/d.rb ``` # forkoff supports two strategies of reading the result from the ``` child: via # pipe (the default) or via file. you can select which to use using the # :strategy option. # ``` require 'forkoff' %w( hey you guys ).forkoff :strategy => :file do |word| puts "#{ word } from #{ Process.pid }" end ``` ~ \> ruby samples/d.rb ``` hey from 7953 you from 7954 guys from 7955 ``` HISTORY 1.1.0 - move to a model with one work queue and signals sent from consumers to producer to noitify ready state. this let’s smaller jobs race through a single process even while a larger job may have one sub-process bound up. incorporates a fix from [GitHub - fredrikj/forkoff: brain-dead simple parallel processing for ruby](http://github.com/fredrikj/forkoff) which meant some processes would lag behind when jobs didn’t have similar execution times. 1.0.0 - move to github 0.0.4 - code re-org - add :strategy option - default number of processes is 2, not 8 0.0.1 ``` - updated to use producer threds pushing onto a SizedQueue for ``` each consumer channel. in this way the producers do not build up a massize parllel data structure but provide data to the consumers only as fast as they can fork and proccess it. basically for a 4 process run you’ll end up with 4 channels of size 1 between 4 produces and 4 consumers, each consumer is a thread popping of jobs, forking, and yielding results. ``` - removed use of Queue for capturing the output. now it's simply ``` an array of arrays which removed some sync overhead. ``` - you can configure the number of processes globally with Forkoff.default['proccess'] = 4 - you can now pass either an options hash forkoff( :processes => 2 ) ... or plain vanilla number forkoff( 2 ) ... to the forkoff call - default number of processes is 8, not 2 ``` 0.0.0 ``` initial version ``` enjoy.