Question: if I made the statement “Ruby is slower than some other
interpreted languages” almost anyone could agree with it (slower
computationally, not programmer time-wise).
So my question is…where is the bottleneck? Is it in method lookup?
Is it that method lookup is more frequent, since everything is an
object? Is it in parsing? Storage? Just wondering. Thanks!
-Roger
On Thu, 6 Sep 2007, Roger P. wrote:
Question: if I made the statement “Ruby is slower than some other
interpreted languages” almost anyone could agree with it (slower
computationally, not programmer time-wise).
So my question is…where is the bottleneck? Is it in method lookup?
Is it that method lookup is more frequent, since everything is an
object? Is it in parsing? Storage? Just wondering. Thanks!
The dynamic nature of the method dispatch. The overall implementation
both the tree walker versus a vm, and some of the implementation
details.
The GC can also be a bottle neck. It stops the world when it runs, and
it
has to touch every object when it runs, so as the program gets bigger
(in
object count), the GC burden gets higher.
Kirk H.
On 9/27/07, Roger P. [email protected] wrote:
method dispatch and a slow GC or something? Hmm.
has to touch every object when it runs, so as the program gets bigger
(in
object count), the GC burden gets higher.–
Posted via http://www.ruby-forum.com/.
I agree with Kirk. Ruby’s inherent performance bottleneck is also its
most
distinctive feature: the open classes and data structures. There’s
always
going to be a limit to how much faster you can do name-resolution at
runtime, since it’s costly to begin with, and you can’t
deterministically
predict it. (There are approaches to adaptively predict name-bindings at
runtime, notably from Smalltalk, and I expect JRuby will be able to
leverage
them. But they can ameliorate, not solve, the fundamental issue.)
Garbage collection and poor memory usage, on the other hand, are
problems
that can and should be solved.
I used to believe that large Ruby programs could be constructed as
independent coarse-grained modules running in their own processes,
communicating through message-passing. Having done quite a bit of this
now,
I think the approach helps quite a bit, but still suffers from memory
usage.
It’s hard to get the memory-usage of even a small Ruby program down to a
level that would permit (perhaps) hundreds of cooperative processes to
be
running at once.
Bottom line: using Ruby will always be characterized by a tradeoff
between
performance and programmer productivity. This is not a criticism of Ruby
in
any way, shape or form! Productivity is a fundamental engineering value,
and
time-to-market is a fundamental quality dimension. Ruby therefore has,
and
will continue to have, a unique value proposition.
Thanks Kirk.
Follow-up question: it seems like there have been a lot of ‘replacement
attempts’ of the Ruby interpreter, but none of them seem to have really
really sped it up (jruby, rubinius, etc.) They do add cool features like
thread safeness or what not, and in certain instances are faster, but
not like ‘we’ve overcome the bottleneck!’ faster.
Does this mean that they suffer from the same bottlenecks? Is Ruby just
a hard language to optimize for? Do they all tree walk and have slow
method dispatch and a slow GC or something? Hmm.
Stay well.
-Roger
So my question is…where is the bottleneck? Is it in method lookup?
The dynamic nature of the method dispatch. The overall implementation
–
both the tree walker versus a vm, and some of the implementation
details.
The GC can also be a bottle neck. It stops the world when it runs, and
it
has to touch every object when it runs, so as the program gets bigger
(in
object count), the GC burden gets higher.
Hi,
In message “Re: question on bottleneck of ruby”
on Fri, 28 Sep 2007 01:24:02 +0900, “Francis C.”
[email protected] writes:
|Bottom line: using Ruby will always be characterized by a tradeoff between
|performance and programmer productivity. This is not a criticism of Ruby in
|any way, shape or form! Productivity is a fundamental engineering value, and
|time-to-market is a fundamental quality dimension. Ruby therefore has, and
|will continue to have, a unique value proposition.
Thank you for kind words. I have been sick of slowness claims based
on Shootout.
Regarding Shootout, Ruby 1.9 runs as fast as (or even faster than)
Python or Perl. Praise should go to Koichi, the YARV originator.
matz.
On Fri, 28 Sep 2007 01:59:36 +0900
“Richard C.” [email protected] wrote:
MRI for Ruby 2.0; in the meantime YARV (1.9) is the Ruby Interpreter
You should definitely keep up to date with the JRuby guy’s blogs (Ola
Bini and Charles Oliver N.). They have been quite detailed on
this topic over the last few months. While their solutions for
performance are to make JRuby very compatible with the JVM, they
articulate the particular reasons for Ruby’s weak performance.Basically, Ruby is hard to optimise for. It’s super dynamic nature
(Open Classes) mean that traditional compiler/interpreter tricks
cannot be used.
So why not add a “class freeze method”, so that these traditional
compiler/interpreter tricks can be used?
During development you just not activate this “frozen class state”, for
releases you can freeze some “performance penalty” classes.
So we not loose ruby’s super-dynamic character (during development),
and got the benefits from more agressive optimisations.
On 9/27/07, Roger P. [email protected] wrote:
Thanks Kirk.
Follow-up question: it seems like there have been a lot of ‘replacement
attempts’ of the Ruby interpreter, but none of them seem to have really
really sped it up (jruby, rubinius, etc.)
I don’t think that the developers of the alternatives would consider
themselves
as ‘replacements’ or ‘competitors’. With the possible exception of
Rubinius.
But that’s its intent - to be the replacement MRI for Ruby 2.0; in the
meantime
YARV (1.9) is the Ruby Interpreter based on the current code base. There
have
been a number of synthetic benchmarks that show interesting performance
improvements in YARV currently. I am not sure how it stacks up in real
world use though. (Not being critical here, real-world benchmarks are
notoriously
difficult to do right).
Does this mean that they suffer from the same bottlenecks? Is Ruby just
a hard language to optimize for? Do they all tree walk and have slow
method dispatch and a slow GC or something? Hmm.
You should definitely keep up to date with the JRuby guy’s blogs (Ola
Bini
and Charles Oliver N.). They have been quite detailed on this topic
over the last few months. While their solutions for performance are to
make
JRuby very compatible with the JVM, they articulate the particular
reasons
for Ruby’s weak performance.
Basically, Ruby is hard to optimise for. It’s super dynamic nature (Open
Classes) mean that traditional compiler/interpreter tricks cannot be
used.
Francis C. wrote:
I agree with Kirk. Ruby’s inherent performance bottleneck is also its most
distinctive feature: the open classes and data structures. There’s always
going to be a limit to how much faster you can do name-resolution at
runtime, since it’s costly to begin with, and you can’t deterministically
predict it. (There are approaches to adaptively predict name-bindings at
runtime, notably from Smalltalk, and I expect JRuby will be able to leverage
them. But they can ameliorate, not solve, the fundamental issue.)
The fact that classes are open doesn’t really impact Ruby performance
all that much. Ultimately the largest reasons for poor performance are
the fact that methods, instance variables, and constants can’t be bound
during a compile phase and must be looked up at runtime. But the same
techniques used to speed up other dynamic languages can apply equally
well here.
Garbage collection and poor memory usage, on the other hand, are problems
that can and should be solved.
Very true, and I hope these are solved some time in future 1.9.1
releases, because I worry that with YARV running faster and generating
garbage faster, the GC bottlenecks are going to become more pronounced.
- Charlie
Hi,
In message “Re: question on bottleneck of ruby”
on Fri, 28 Sep 2007 09:18:33 +0900, Charles Oliver N.
[email protected] writes:
|> Garbage collection and poor memory usage, on the other hand, are problems
|> that can and should be solved.
|
|Very true, and I hope these are solved some time in future 1.9.1
|releases, because I worry that with YARV running faster and generating
|garbage faster, the GC bottlenecks are going to become more pronounced.
Although I have my own opinion, first, can you elaborate your opinion
on GC problems and possible solutions?
Note that we already implemented generational GC, which happened to
have no positive effect on performance due to write barrier cost.
matz.
On 9/27/07, Markus S. [email protected] wrote:
On Fri, 28 Sep 2007 01:59:36 +0900
“Richard C.” [email protected] wrote:Basically, Ruby is hard to optimise for. It’s super dynamic nature
(Open Classes) mean that traditional compiler/interpreter tricks
cannot be used.So why not add a “class freeze method”, so that these traditional
compiler/interpreter tricks can be used?
IIRC you can set how open your classes are at runtime. Though
we like the fact that Ruby classes are open at runtime. Its not
a development feature by any means - Rails makes huge use of
this.
Interpreter writers know this and design accordingly. So they use
non-traditional tricks instead.
Mostly I just don’t see Ruby performance as a significant issue.
Memory management and GC would be higher on my priority list,
and are arguably easier to solve anyway.
JRuby now adds another option for dropping down to system
language performance for specific cases. .NET users will get
similar treatment eventually, and yet another system language
option will be added to the mix (and .NET users will get a
proper scripting option on the CLR as well 
)
The only area where I have noticed very poor Ruby performance
is in REXML. Though REXML is very much an entry-level option,
and the authors themselves would redirect you to alternatives if
your needs are greater. My observations were probably multiplied
by my absolutely rubbish XML skills too…
Charles Oliver N. wrote:
The fact that classes are open doesn’t really impact Ruby performance
all that much.
That’s true for single-process benchmarks where you don’t implement
sharing of compiled modules. In a compiler-interpreter I once worked
on, we serialised the bytecode to a memory-mappable file for each
cluster of modules, which did a lot to improve startup performance,
especially for subsequent instances. Not possible for Ruby however,
doe to the open classes - QED.
The other performance factor (related to a different discussion) that
makes byte-code interpretation faster than AST-interpretation is that
with byte-code, you get much better locality of reference, so your
cache+memory system works much better. This is a very significant
factor that justifies some recent complaints. It’s also quite possible
for Ruby interpreters to implement, despite open classes.
Clifford H…
Yukihiro M. wrote:
Thank you for kind words. I have been sick of slowness claims based
on Shootout.Regarding Shootout, Ruby 1.9 runs as fast as (or even faster than)
Python or Perl. Praise should go to Koichi, the YARV originator.
Yes indeed!
Yukihiro M. wrote:
|garbage faster, the GC bottlenecks are going to become more pronounced.
Although I have my own opinion, first, can you elaborate your opinion
on GC problems and possible solutions?Note that we already implemented generational GC, which happened to
have no positive effect on performance due to write barrier cost.
I think part of the problem, as I understand it, is the “stop the world”
nature of the GC. Because the GC can’t run in parallel, it’s more
difficult to use Ruby for pauseless applications.
Unfortunately I can’t really speak definitively on GC techniques, I’ve
largely just heard stories from others about problems with Ruby’s GC and
issues with large or long-running apps. Can anyone else elaborate?
- Charlie
Clifford H. wrote:
Charles Oliver N. wrote:
The fact that classes are open doesn’t really impact Ruby performance
all that much.That’s true for single-process benchmarks where you don’t implement
sharing of compiled modules. In a compiler-interpreter I once worked
on, we serialised the bytecode to a memory-mappable file for each
cluster of modules, which did a lot to improve startup performance,
especially for subsequent instances. Not possible for Ruby however,
doe to the open classes - QED.
Startup time does not general performance make. Just ask Java 
The other performance factor (related to a different discussion) that
makes byte-code interpretation faster than AST-interpretation is that
with byte-code, you get much better locality of reference, so your
cache+memory system works much better. This is a very significant
factor that justifies some recent complaints. It’s also quite possible
for Ruby interpreters to implement, despite open classes.
JRuby has successfully implemented a full Ruby 1.8 to Java bytecode
compiler (as of about ten minutes ago), so it is indeed possible…and
nicely fast.
- Charlie
Francis C. wrote:
I agree with Kirk. Ruby’s inherent performance bottleneck is also its most
distinctive feature: the open classes and data structures. There’s always
going to be a limit to how much faster you can do name-resolution at
runtime, since it’s costly to begin with, and you can’t deterministically
predict it. (There are approaches to adaptively predict name-bindings at
runtime, notably from Smalltalk, and I expect JRuby will be able to leverage
them. But they can ameliorate, not solve, the fundamental issue.)
Which is why they teach data structures in computer science class. It’s
all about fast search, I think. That’s one of the big gripes I have with
“lazy” interpretation. If you don’t do stuff until you have to do it, it
only pays off if you end up never having to do it.
[snip]
I used to believe that large Ruby programs could be constructed as
independent coarse-grained modules running in their own processes,
communicating through message-passing. Having done quite a bit of this now,
I think the approach helps quite a bit, but still suffers from memory usage.
It’s hard to get the memory-usage of even a small Ruby program down to a
level that would permit (perhaps) hundreds of cooperative processes to be
running at once.
And unless the Ruby “inner interpreter” is highly optimized, even if you
have only one copy of the text segment and only one copy of all the
libraries in RAM, you’re really unlikely to have all the “good stuff”
in the tiny caches processors have.
Bottom line: using Ruby will always be characterized by a tradeoff between
performance and programmer productivity. This is not a criticism of Ruby in
any way, shape or form! Productivity is a fundamental engineering value, and
time-to-market is a fundamental quality dimension. Ruby therefore has, and
will continue to have, a unique value proposition.
I’m not sure this is a valid tradeoff. The economics of development
and the economics of operating a large code are two entirely different
subjects. People have “always” prototyped in “slow but productive”
languages, like Lisp, Perl, PHP and Ruby, and then reached a point where
the economics dictated a complete rewrite for speed into C, C++ or Java.
I can think of more examples of this than I can of something that was
developed and prototyped rapidly and then grew by “just throwing more
hardware at inefficient software.”
So … just like a startup should plan for the day when a big company
offers them the choice of selling out or being crushed like a bug, when
you implement a great idea in some rapid prototyping framework like
Rails, plan for the day when you are offered the choice of rewriting
completely in a compiled language or going bankrupt buying hardware.
M. Edward (Ed) Borasky wrote:
offers them the choice of selling out or being crushed like a bug, when
you implement a great idea in some rapid prototyping framework like
Rails, plan for the day when you are offered the choice of rewriting
completely in a compiled language or going bankrupt buying hardware.
OK … so … is Twitter in trouble?
http://www.scripting.com/stories/2007/09/27/twitterIsTakingAShowerToni.html
“I have seen the future, and it’s just like the present, only longer.”
– Kehlog Albran
Charles Oliver N. wrote:
Startup time does not general performance make. Just ask Java
True, but lack of memory pressure enables you to keep the whole
working set in memory, and that does affect runtime.
Plus there is a large class of applications for which startup time
dominates, otherwise we’d be writing /bin/cat in Java :-).
Clifford H…
On 9/27/07, M. Edward (Ed) Borasky [email protected] wrote:
Which is why they teach data structures in computer science class. It’s
all about fast search, I think. That’s one of the big gripes I have with
“lazy” interpretation. If you don’t do stuff until you have to do it, it
only pays off if you end up never having to do it.
I meant something different. Ruby has to bind most of the names it uses
at
runtime, precisely because the language makes it possible (and extremely
useful) to do things which invalidate them. Ruby does this quite
efficiently
(so it’s not the case that Ruby’s data structures were written
incompetently, as you seem to be suggesting), but the fact is that you
simply have to do it every time you access a method or variable.
I’ve nearly given up on extensively profiling my Ruby programs. Over the
years I’ve gotten a pretty good feel for what generates hot spots so I
can
avoid them upfront. What I always end up with is long lists of “warm”
spots
that are basically irreducible (Things like calls to === and []). Ruby’s
basic behavior at runtime just seems to generate a lot of “background
noise.” We’ll see if this improves in the new runtimes.
and the economics of operating a large code are two entirely different
Rails, plan for the day when you are offered the choice of rewriting
completely in a compiled language or going bankrupt buying hardware.
It sounds like your experience has been largely with systems in which
development activities have a long “tail,” extending well into the
production cycle. It’s certainly been my experience that systems written
in
C/C++ and Java work this way. It’s almost as if the high investment in
initial development necessitates a continuing commitment to that code
base,
along with the business assumptions and engineering decisions it
originally
embodied.
But over the last four years of working with Ruby, I’ve found something
like
the opposite. Ruby programs largely have a “write it and forget it”
quality
to them. The very first Ruby program I ever wrote, on the day that I
learned
Ruby, was a RADIUS server that worked off an LDAP datastore. It took
four
hours to write (including the time spent learning the language), went
into
live production the next day (the client was as adventurous as I was),
and
has been running ever since. It was only modified once, to add a new
feature.
Since them, my teams have written innumerable Ruby programs that are
usually
developed as small gems with full unit-test suites, and they find their
way
into one (or, infrequently, more than one) much larger project.
How does it change the economics? Because of the time-to-market
dimension. I
believe that a lot of business issues that can be addressed with the
creation of some software (and that usually involve increasing the
“surface
area” (or timely accessibility) of generally small bodies of
information)
only make sense if the development cycle is very short, and the
user-acceptance period is quite nearly zero. (Meaning, the software
works as
expected the first time out.)
I’ve found in my businesses that such fleeting opportunities come up all
the
time. If you’re a person who believes in proper software engineering,
and
well-controlled processes, you’re probably climbing the walls and
getting
ready to throw things at me right now! But this is why I was talking
about a
value-tradeoff. If I’m right, then there are a lot of opportunities to
create capturable business value that traditional methodologies
(including
fast-prototype-followed-by-extensive-rewrite) simply can’t touch.
For these cases, Ruby is uniquely valuable.
So am I the person who is creating the boatloads of crapware that no one
else can understand but that can’t be gotten rid of, and eventually
drives
purchases of larger hardware?
In some respects, guilty as charged. (I deny the point about
non-understandability. If you’re going to develop like this, then
writing
documentation and unit tests must dominate the development effort,
perhaps
by 10-to-1. Otherwise, you end up with nothing usable.)
Larger hardware: the trend I see in every single enterprise client is
toward
virtualization. We’re entering a long cycle in which fewer hardware
resources will be available for the typical program rather than more.
I’m
already expecting Ruby to suffer as a result. Java, with its enormous
memory
footprint, is hardly the solution to this problem.
On Fri, 2007-09-28 at 19:17 +0900, Francis C. wrote:
On 9/27/07, M. Edward (Ed) Borasky [email protected] wrote:
Which is why they teach data structures in computer science class. It’s
all about fast search, I think. That’s one of the big gripes I have with
“lazy” interpretation. If you don’t do stuff until you have to do it, it
only pays off if you end up never having to do it.
[…snip…]
I’ve found in my businesses that such fleeting opportunities come up all the
time. If you’re a person who believes in proper software engineering, and
well-controlled processes, you’re probably climbing the walls and getting
ready to throw things at me right now! But this is why I was talking about a
value-tradeoff. If I’m right, then there are a lot of opportunities to
create capturable business value that traditional methodologies (including
fast-prototype-followed-by-extensive-rewrite) simply can’t touch.For these cases, Ruby is uniquely valuable.
It seems like you are angry. I can feel that you like Ruby very much.
[…snip…]
On Fri, Sep 28, 2007 at 02:20:04PM +0900, Clifford H. wrote:
Charles Oliver N. wrote:
Startup time does not general performance make. Just ask Java
Plus there is a large class of applications for which startup time
dominates, otherwise we’d be writing /bin/cat in Java :-).
I second the motion.
For most of what I write in Ruby, startup time is much more important
than long-running performance. I’d prefer to avoid having to go back to
Perl for some of that, to avoid extremely long waits (for some
definition
of “extremely long”) for basic sysadmin utilities.