I’m putting this at the top of my post because I think the basic problem
here may be intensive numeric calculations, and - even more so - disk
and) output of about 16 MiB x N bytes of data, where N is 8 bytes (? for
Floating point numbers), so about 128 MiB in total, and other people
have a better knowledge of some possibly useful links.
On Sun, Jul 25, 2010 at 11:15 PM, Junhui L.
Actually, I developed two versions of C++ script.
One is opening 4096 files at the same time. This cost 3 hours.
Another version is saving all of the data in a big vector,
then scanning the vector to pick the right items to write
in files. This cost 2 hours and 45 minutes. :-).
Sorry - in my post I misunderstood what you meant by “cost”. I think it
(very?) unlikely that any Ruby (or Perl or Python, etc?) program will
faster than your C++ scripts. Where Ruby (or Python - I’m not so sure
Perl, I haven’t used it) does have an advantage is that I think
may be quicker. So there are trade-offs. (Incidentally, I’m not an
but those timings suggest to me that the major processing cost may be in
writing the results out to disk, so changing the language for all or
the processing is unlikely to make a large difference?)
But I’m open to correction: there are people who have used Ruby for
intensive large data sets processing, but my understanding is that they
a mixture of Ruby as “glue” with any intensive calculations in C, etc.
example, from some limited experience I have the speed of Ruby reading
strings of bytes in from files is similar to the speed of Java or
Pascal, but for calculating CRCs of files the speed of pure Ruby
the CRCs once the bytes had read in was much slower than Java or
Pascal: so I used Ruby (or rather JRuby) to read in the strings of bytes
from the files, and then called Java code from Ruby to calculate the CRC
from the bytes. Overall the speed of this was similar to a pure Java or
compiled Pascal program.
Piet Hut and Jun Makino have been using Ruby to model dense star
(Note that this is something I know nothing about! I’m just intrigued by
underlying principle of using Ruby for intensive numerical calculations
developing in Ruby without worrying about speed by using smaller
models, and then using more realistic models by translating part (or
of the Ruby code to a faster language.)
…MODEST is the new name for the Stellar Dynamics workshop. It stands
MOdeling DEnse STellar systems
The basic idea is to start a kind of N-body wikipedia, as a group’s
It should be self contained, a place to gather all the basic information
that is currently missing from the literature. Up till now, if you want
write a decent N-body code from scratch, you have to somehow catch the
knowledge that is floating around in the community, most of which has
been written down. We want to change that.
One place to start is the open-source introductory text written over the
last several years by Piet Hut and Jun Makino, “Moving Stars Around”;
http://www.artcompsci.org/#msa where we give pointers to two versions.
older one is written using C++ and the newer one, with even more
background, uses Ruby. We propose to use the basic text, and to
the (rather short) pieces of code into other languages, starting with
the Linden Scripting Language, and then to move on to OpenSim. …
Dan: This confirms our earlier conclusions. At least on this particular
computer, that we are now using to do some speed tests, the unoptimized
version takes 50% more time than the optimized version, the simplest
version takes about 50 times more time, the Ruby array version about 100
times more, and finally the Ruby vector version takes more than 250
more time than the optimized C version.
Carol: But even so, for short calculations, who cares if a run takes ten
millisecond or a few seconds? I certainly like the power of Ruby in
us vector classes, and a lot more goodies. We have barely scratched the
surface of all the power that Ruby can give us. You should see what we
do when we really start to pass blocks to methods and . . .
Dan: . . . and then we will start drinking a lot of coffee, while
for results when we begin to run 100-body experiments! Is there no way
speed up Ruby calculations?
Carol: There is. By the time we use 100 particles, we are talking about
- 10^2 = 10^4 force calculations for every time step. This means that
calculation of the mutually accelerations will take up almost all of the
computer time. What we can do is write a short C code for computing the
accelerations. It is possible to invoke such a C code from within a Ruby
code. In that way, we can leave most of the Ruby code unchanged, while
gaining most of the C speed.
Erica: I certainly like the flexibility of a high-level language like
at least for writing a few trial versions of a new code. In order to
around, Ruby is a lot more fun and a lot easier to use than C or C++ or
Fortran. After we have constructed a general N-body code that we are
happy with, we can always translate part of it into C, as Carol just
suggested. Or, if really needed to gain speed, we could even translate
whole code into C. Translating a code will always take far less time
developing a code in the first place. And is seems pretty clear to me
development will be faster in Ruby.
Dan: I’m not so sure about all that. In any case, we got started now
Ruby, so let us see how far we get. But if and when we really get bogged
down by the lack of speed of Ruby, we should not hesitate to switch to a
more efficient language.
Thanks a lot for your encourage,
I tried to read something on ruby already.
Since this language is very simple and beautiful,
no matter it works for my case or not.
(But I hope it could be.)
To quote something Damon Runyon quoted/adapted: The race is not always
the swift nor the battle to the strong - but thatâ€™s the way to bet.
Or to continue that analogy, “horses for courses”!
(Translation: for parts of a task that are heavy numerical processing a
“lower level” language is likely to be faster. That said, sometimes the
slower higher level language is fast enough: I was doing some actuarial
programming in JRuby, and intended to have some intensive numerical
calculations in Java. It turned out that JRuby was actually quite fast
enough for those particular calculations, although (for portability
I will be translating that numerical code from Ruby to Java.)
Maybe it partly depends on what standard methods/functions are
available: for example, in Ruby you can read a line from a file
into a String, and then use a builtin method on the String
to split it into an array of values using a specified delimiter,
so in your case a space character?
I need this kind of comment seriously, saying, what are the knowledge
which is necessary and enough to do my job. If there are some special
and powerful methods or stances to do this kind of stuff.
Or be better, give a example just very close my case.
I can get the detailed by reading book(s) or googling.
For your particular problem you’ve already written the C++ code to do
splitting and the processing, so (ignoring maintainability issues) I
think Ruby will speed up the development, and - for the reasons above -
think it is unlikely to result in faster processing. But you might find
worth looking at integrating Ruby (or Python or Perl) with C, C++ or
for other processing problems. I don’t have any experience with C or
but I have found integrating Ruby with Java fairly easily. (Once I found
way round some initial misunderstandings I had with how to package Java
be used by Ruby.)
Anyway, good luck with your task(s)!