Any HPC on Ruby?

Ruby
1

Hello,
I was wondering if someone here has had any experience in programming
for HPC with Ruby, or it’s just not feasible.

Usually many scientific codes, for example, spend most of their time in
only a few functions, which (from what I understood) could be
programmed directly in C, letting you use Ruby for the remaining 90% of
the code.

That’s why I thought it was feasible, or am I on the wrong track?

And if someone has tried this, how was the experience?

Diego

2

Diego V. wrote:

And if someone has tried this, how was the experience?

Diego

There are people doing this – head over to

http://sciruby.codeforpeople.com/sr.cgi/FrontPage

The best approach for a “typical” HPC number crunching problem is most
likely some form of distributed Ruby on a cluster, with the individuals
nodes running processes built around the Ruby “narray” library. I don’t
know much about non-numeric HPC problems, but I’d be willing to bet
someone has implemented “MapReduce” in distributed Ruby for stuff like
that.

http://labs.google.com/papers/mapreduce.html

–
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blogspot.com/

If God had meant for carrots to be eaten cooked, He would have given
rabbits fire.

3

Diego V. wrote:

Hello,
I was wondering if someone here has had any experience in programming
for HPC with Ruby, or it’s just not feasible.

I am trying to port mpi-ruby to bluegene. I do combinatorial
optimization so it makes sense for me (cpu intensive, low
communication). You still have to write your computation kernels in C,
but it makes communication and high level operations way faster to code.

Usually many scientific codes, for example, spend most of their time in
only a few functions, which (from what I understood) could be
programmed directly in C, letting you use Ruby for the remaining 90% of
the code.
Exactly.

That’s why I thought it was feasible, or am I on the wrong track?

And if someone has tried this, how was the experience?
I see no slowdown, but then I do low communication high CPU codes. If
you have to pass a lot of data around, or need every megabyte of memory
ruby might not be the right choice for your application. You can always
write your app in ruby as a prototype, then re-write it in C chunk by
chunk until you get acceptable preformance.

4

On Mon, Nov 20, 2006 at 05:07:09AM +0900, M. Edward (Ed) Borasky wrote:

I don’t
know much about non-numeric HPC problems, but I’d be willing to bet
someone has implemented “MapReduce” in distributed Ruby for stuff like that.

http://labs.google.com/papers/mapreduce.html

Yep
http://www.rufy.com/starfish/doc/

5

Logan C. wrote:

http://www.rufy.com/starfish/doc/

Hmmmm … someone a while back was looking for a way to deal with 60 GB
of traceroute data in Ruby. If he’s still here, this might be what he
needs!

–
M. Edward (Ed) Borasky, FBG, AB, PTA, PGS, MS, MNLP, NST, ACMC(P)
http://borasky-research.blogspot.com/

If God had meant for carrots to be eaten cooked, He would have given
rabbits fire.

6

Diego V. wrote:

And if someone has tried this, how was the experience?

This is a good approach. You can do all the mundane stuff (files, ui,
config) in ruby, and the interesting stuff in C. Also, ruby is very good
for writing domain-specific languages (DSLs), which can make it easier
to use your C library. For example, the DSL could specify your system
(whatever it is), and that specification would be the input to your C
functions. Or the specification could even be used to generate C code
dynamically.

Can you tell us a little more about the area you are working in, the
problems you want to solve, etc?

7

http://sciruby.codeforpeople.com/sr.cgi/FrontPage

Thanks: I’ve checked it and it looks like a pretty large site. I’ll
have something to read for a while. :slight_smile:

See you

Diego

8

Can you tell us a little more about the area you are working in, the
problems you want to solve, etc?

I am interested in Computational science, but so far all my experience
was on desktop computers. Now I’ve started to study HPC stuff… which
arised the question.

I do not have a current specific need to fill, though, it was more of a
general curiosity: something to keep in mind for the future program
designs. If I know people have tried it and with success then I might
attempt to do so myself. (I should choose a dissitation soon, so this
might be a possibility).

Thank you by the way. I had totally forgotten about the DSL potential
of Ruby (I am still pretty much a Ruby newbie :slight_smile: ).

Diego