There doesn’t seem to be any EASY way of doing a parallel computation
in Ruby.
I would like to do something like this :
array.map do |i|
fork do
i + 1
end
end
Process.waitall
wich would give back the array with one added to each element in an
array, and it would perform this “calculation” in parallel. However,
this doesn’t work since fork runs a subprocess which is another Ruby
interpreter and I can’t get anything back from that black hole, except
some exit status.
Actually, it would be really nice if there was a ‘forkmap’ method that
could do this:
Fredrik wrote:
| There doesn’t seem to be any EASY way of doing a parallel computation
| in Ruby.
| I would like to do something like this :
|
| array.map do |i|
| fork do
| i + 1
| end
| end
| Process.waitall
|
| wich would give back the array with one added to each element in an
| array, and it would perform this “calculation” in parallel. However,
| this doesn’t work since fork runs a subprocess which is another Ruby
| interpreter and I can’t get anything back from that black hole, except
| some exit status.
|
| Actually, it would be really nice if there was a ‘forkmap’ method that
| could do this:
|
| array.forkmap do |i|
| i + 1
| end
|
| But there isn’t, right?
|
|
Ruby uses green threads. All your threads would run within the Ruby
process, and aren’t running parallel in the sense you seem to imply. If
you want to use threads, maybe JRuby and Java are what you seek.
JRuby uses Java threads, which are OS threads.
If I’m on the wrong tangent, just ignore this reply.
You thought I was taking your woman away from you. You’re jealous.
You tried to kill me with your bare hands. Would a Kelvan do that?
Would he have to? You’re reacting with the emotions of a human.
You are human.
~ – Kirk, “By Any Other Name,” stardate 4657.5
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
~ “But the important thing is persistence.” -Calvin trying to juggle
eggs
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
Charles Oliver N. wrote:
| Phillip G. wrote:
|> -----BEGIN PGP SIGNED MESSAGE-----
|> Hash: SHA1
|>
|> Fredrik wrote:
|>
|> |
|> | Thanks! This code is just what I am looking for!
|> | Peach for JRuby seems nice too, but I don’t have JRuby
|>
|> It’s just a download away.
|> jruby.codehouse.org
|>
|> However, you’ll need a JVM that is compatible (IIRC, JRE 1.4.2 and
|> newer).
|
| www.jruby.org will get you there,
Right, codehaus, not house. facepalm
| and JRuby 1.1 requires Java 1.5 or
| higher.
Oops, my mistake (didn’t the 1.0 series require only JRE 1.4.2, or so?).
~ - You know you’ve been hacking too long when…
…you see a flock of birds and try to figure out the algorithms that
determine their movement.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.8 (MingW32)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org
Actually, I’ll change it a bit. I added Process.waitall since there
are otherwise some dead(?) processes left.
module Enumerable
def fmap &b
result = map do |*a|
r, w = IO.pipe
fork do
r.close
w.write( Marshal.dump( b.call(*a) ) )
end
[ w.close, r ].last
end
Process.waitall
result.map{|r| Marshal.load [ r.read, r.close ].first}
end
end
it’s damn tricky - it excepts get get an exception marshaled up a
dedicated pipe, if this does not occur we know the child process
started successfully. you can adapt.
Oops, my mistake (didn’t the 1.0 series require only JRE 1.4.2, or so?).
Yes, JRuby 1.0 worked on Java 1.4.2, but there were too many benefits
moving to Java 5 to keep it that way, especially availability of
annotations and the concurrency APIs.
I added an argument to limit the number of concurrent processes (my
workstation practically died when I ran all the processes I wanted to
run):
module Enumerable
def forkmap n, &b
result = map do |*a|
nproc = 0
r, w = IO.pipe
fork do
r.close
w.write( Marshal.dump( b.call(*a) ) )
end
if (nproc+=1) >= n
Process.wait ; nproc -= 1
end
[ w.close, r ].last
end
Process.waitall
result.map{|r| Marshal.load [ r.read, r.close ].first}
end
end
It’s really great. I just see one thing to improve:
The new “n” parameter is mandatory since it’s the first parameter. It
would be
nice if it could be not defined (so = infinite):
forkmap(4) { code } --> max 4 process
forkmap { code } --> max infinite
Do you think your code can be feasible for production enviroments? maybe
it
envolves some danger or risk? If not I suggest you to publish it in any
way
since it’s really cool and a missing feature of Ruby.
Do you think your code can be feasible for production enviroments?
maybe it
envolves some danger or risk? If not I suggest you to publish it in
any way
since it’s really cool and a missing feature of Ruby.
i’ve got something close to gem’ing… there is nothing wrong with the
concept - this is precisly how objects are returned from drb:
marshaled data over a socket/pipe.
The new “n” parameter is mandatory since it’s the first parameter. It would be
nice if it could be not defined (so = infinite):
forkmap(4) { code } --> max 4 process
forkmap { code } --> max infinite
I was thinking about that too, but as far as I understand it Ruby only
allows optional arguments to be the last arguments - i.e. the “n”
parameter would have to appear after the code block. And that would
look strange : forkmap{ code }(4).
