Forum: Ruby Zed and Luis drop the bomb on Ruby's poor performance

The O'Reilly blog has two great interviews posted from the last week
or so. The first [1] is with Zed Shaw, the author of Mongrel (and a
bunch of other Ruby frameworks and utilities). A follow-up interview
with one of the contributors on the Mongrel project, Luis Lavena [2],
covered some similar ground.

What I found most striking about both interviews was their mention of
YARV and Ruby's poor computational performance. Zed said, and I quote:

> I?ll be honest right away though and say that Ruby is slow. The  
> Ruby community has been ignoring the huge ?performance? elephant  
> standing in the room and they need to start talking about it so it  
> goes away. Elephants hate being talked about. There are a few  
> efforts to make Ruby faster, but I see a lot less action than is  
> needed to solve the problem. One solution in the works is a real  
> virtual machine called Rite (or YARV depending on who you talk to)  
> which is showing some real promise and seems to be speed  
> competitive with the fastest Java implementations.

I tend to agree with this. Whenever the subject of computational
performance comes up, quite a few posts end up in the thread
questioning the meaning of the word performance and turn it into an
argument over semantics. Is it coding speed, clarity of thought,
readability, etc. To me that kind of crazy semantic dodge doesn't
serve us well. Those discussions should be remain on computational
performance since the language has been (mostly) stable in syntax for
years.

So what is being done to get YARV the attention it deserves? Why
aren't we talking about it all the time? What can I personally do to
help? Are there valid excuses for not focusing on improving Ruby's
poor computational performance?

cr

[1] http://www.oreillynet.com/ruby/blog/2006/05/post.html
[2] http://www.oreillynet.com/ruby/blog/2006/05/
interview_with_luis_lavena_2.html

One question I have to ask is "Wouldn't it make more sense to piggy back
on parrot (the Perl v6) engine?" and get all the benefit of the
performance improvements that parrot will pick up and the fact that
there are many, many very clever people working on parrot whose work we
could make use of.

Is there a reason why we are reinventing the wheel?

unknown wrote:
> So what is being done to get YARV the attention it deserves? 
AFAIK YARV will be the default VM when ruby 2.0 comes out. And I think 
2.0 will be the next release of ruby. Becoming the default VM seems like 
as much attention as you can get.

> Why aren't we talking about it all the time? 
It will be in the next major release, what more is there to talk about? 
:) At the moment it is still not 100% compatible with ruby 1.8 (and 
might never be), but it works for a lot of ruby scripts/programs.

> What can I personally do to help?
Check out their site:
http://www.atdot.net/yarv/#i-1

Edwin

On 5/22/06, Peter Hickman <peter@semantico.com> wrote:
> One question I have to ask is "Wouldn't it make more sense to piggy back
> on parrot (the Perl v6) engine?" and get all the benefit of the
> performance improvements that parrot will pick up and the fact that
> there are many, many very clever people working on parrot whose work we
> could make use of.
>
> Is there a reason why we are reinventing the wheel?

http://wiki.rubygarden.org/Ruby/page/show/VirtualMachineOptions

Cardinal
A re-implementation of Ruby on top of the [Parrot] virtual machine
This is the virtual machine that's being developed to support Perl 6
and other languages.

Availability
Parrot is currently in development. The Cardinal project has it's
homepage at [RubyForge]. However it has been dormant since August
2004.

In [languages/ruby] of the Parrot SVN repository there is an earlier
attempt at targetting Ruby on Parrot. The last real update of that was
in August 2002.

Advantages
   1. Access to large Perl development community and resources
   2. Will be able to use modules developed in and/or for Perl and
other Parrot-based languages
   3. The Ruby interpreter could be written in Ruby
   4. Native threads
   5. Precompiled bytecode
   6. JIT
   7. Possible speed improvement (but note that Parrot and Cardinal
are still in development)

Disadvantages
   1. Different C interface (But Parrot's C interface is also clean)
   2. Relies on Parrot, a large work-in-progress

Notes
   1. Not an "official" VM; when asked about VM Matz himself answered
he prefers to develop his own VM

*snip*
> So what is being done to get YARV the attention it deserves? Why
> aren't we talking about it all the time? What can I personally do to
> help? Are there valid excuses for not focusing on improving Ruby's
> poor computational performance?

There vertainly are perfectly good "excuses" (I didn't know I needed
one! :) ) for not focussing on Ruby's performance - it's performance is
only a concern for one (poor or otherwise) if it's actually causing one
a problem.

In my previous incarnation as a signal processing and volume rendering
coder, I imagine Ruby's performance would have been unnaceptable. In my
current role as a testing framework programmer, it's plenty fast thanks,
and jolly expressive. I've no reason (at the moment) to require more
performance from it. When I switch back to doing graphics again, I'll be
generating my c, c++ or assembly with Ruby anyway, rather than hand
coding it, and it'll probably be fast enough for that too.

All that said - I'll crack open a drink to Ruby's performance improving.
I just want to point out why it doesn't (pragmatically) matter to me
personally atm.

Cheers,
  Benjohn

On 5/22/06, benjohn@fysh.org <benjohn@fysh.org> wrote:
> I just want to point out why it doesn't (pragmatically) matter to me
> personally atm.
>

Unfortunately, not all of us are in your boat. I really like Ruby, but
if performance improvement is not going to be in Ruby's near future,
my management is not going to buy into Ruby!

In most of the corporate environments I have worked in, many managers
tend to take IT decisions based on 'feelings'; 'speed' is a word that
creates good 'feelings'. So, when the issue of performance is bought
up, Ruby doesn't win (doesn't matter if the application requires it or
not).

IMO, if Ruby's vision is to penetrate the corporate environment, Ruby
should address its performance issues among other things (e.g. support
for apps -- Ruby on Rails).

-Madan.

On 5/22/06, Madan Manoharan <madan.manoharan@gmail.com> wrote:
>
> IMO, if Ruby's vision is to penetrate the corporate environment, Ruby
> should address its performance issues among other things (e.g. support
> for apps -- Ruby on Rails).

I don't think it is Ruby's (or Matz's) vision for Ruby to penetrate
the corporate environment. Keep in mind that unlike Java or C#, Ruby
is not a commercial language, and there isn't really any benefit to
massive corporate uptake of Ruby. For those of us that enjoy the
language it certainly would be nice for our choice to be "validated"
by getting more market share, but in general myself and probably most
other Rubyists don't put much value on corporate use being an
indicator of language popularity.

Still, performance is important, and YARV will be a great improvement
in that area.

Ryan

Madan, can you elaborate on this:
"(e.g. support for apps -- Ruby on Rails)"

thanks

Pretty much the same for me. If Ruby is not fast enough then I convert
it to C, nothing less than C seems to have the performance if
performance is an issue (ok in extreme cases there is assembler but
nothing I have ever written *needed* to be written in assembler -
besides it's not as much fun since the m68k went south). The pain of
going from Ruby to Java is not going to realise the performance
improvement of going from Ruby to C for, what appears to me to be, not
quite as much pain.

On Monday 22 May 2006 9:25 am, Madan Manoharan wrote:
> On 5/22/06, benjohn@fysh.org <benjohn@fysh.org> wrote:
> > I just want to point out why it doesn't (pragmatically) matter to me
> > personally atm.
>
> Unfortunately, not all of us are in your boat. I really like Ruby, but
> if performance improvement is not going to be in Ruby's near future,
> my management is not going to buy into Ruby!

The thing that always stands out to me in these performance discussions 
is
that people argue that Ruby needs a performance improvement, but there 
are
rarely any specifics attached to that analysis.

How much of speed improvement?  How fast should Ruby be?  For what?

I like speed in my applications.  It gives me a warm, fuzzy feeling. 
However
Ruby's speed has never been a problem for me, and I have been using Ruby 
in
enterprise applications, for at least some definitions of enterprise, 
for 4
years.

On really very modest hardware it is no problem for me to dynamically 
generate
web pages for sites/applications fast enough that it could handle 
_millions_
of hits a day.  That is, if I can optimize away the cost of most 
database
interactions through caching or other mechanisms.  Ruby could be 10x 
faster,
and if I have to make database queries with each request, it's not going 
to
matter, because Ruby isn't the bottleneck, most of the time.

I do a lot of data/file generation/handling/transformation/transport 
duties
with Ruby, and Ruby is never the bottleneck.

I do email build/send tasks with Ruby, and Ruby isn't the bottleneck. 
In
fact, I have a number of years experience in the dynamic, targetted 
email
business, and my relatively simple Ruby based email build platform is 
more
powerful, easier to use, and faster than the quite expensive (in 
developer
hours spent to build/optimize it) Java based system that my last 
employer
used.

Ruby can be a bottleneck, but for the vast majority of Ruby users and 
the vast
majority of tasks people use if for, it is not the bottleneck.

It is not an elephant in the living room that everyone is ignoring if 
the
performance simply isn't a limiting factor.  It's an elephant out in the
paddock, where it belongs in those cases.  And those cases are the 
majority
of cases.


Kirk Haines

On May 22, 2006, at 11:25 AM, Madan Manoharan wrote:
> In most of the corporate environments I have worked in, many managers
> tend to take IT decisions based on 'feelings'; 'speed' is a word that
> creates good 'feelings'. So, when the issue of performance is bought
> up, Ruby doesn't win (doesn't matter if the application requires it or
> not).


Personally, I like it just fine when companies make decisions based on
feelings rather than logic. This is just one reason why there is
always room in the market for a new competitor.

Gary Wright

On May 22, 2006, at 10:07 AM, benjohn@fysh.org wrote:

> one
> generating my c, c++ or assembly with Ruby anyway, rather than hand
> coding it, and it'll probably be fast enough for that too.

Not to single you out (though I guess I am) but you are the first one
in the thread to write-in with "it's good enough for me because of
<insert reasons having nothing to do with computation speed here>."

I think we all know this. Most of us on this list are here because we
find the language expressive, clean, fast enough for whatever, yada
yada yada.

This thread is *specifically* about getting more computational
performance for Ruby. Let's not throw it off track by reiterating all
the *other* reasons the language is "good enough" for us.

> All that said - I'll crack open a drink to Ruby's performance  
> improving.

Good! Me too!

> I just want to point out why it doesn't (pragmatically) matter to me
> personally atm.

Oops! Wrong thread! :-)

cr

The problem is that Zed and Luis make out that performance is something
that the whole Ruby community is deliberately ignoring. This is not the
case, there are plenty of people who have no issue with the performance
of Ruby. The truth is the performance is an issue for Zed and Luis and
they are trying to make out that it is not their personal bugbear but
some sort of conspiracy so they don't look like whiners. I have never
though "damn Ruby's dismal performance, it should be able to do this
faster" and I probably never will.

Yes we wouldn't say no to better performance but we (the Ruby community)
are not being held back by the current level of performance.

On Tue, 23 May 2006 cremes.devlist@mac.com wrote:

>
>> All that said - I'll crack open a drink to Ruby's performance improving.
>
> Good! Me too!
>
>> I just want to point out why it doesn't (pragmatically) matter to me
>> personally atm.
>
> Oops! Wrong thread! :-)
>
> cr

you make good points - but let me just throw in my 2 cts:

virtual machines are stupid.  java is slow.  c++ is slow.  python is 
slow.
ruby is slow.  so what.  if you want fast you need c or fortan or 
assembler.
all three are very easy to call from ruby.  i use mmap, narray, gsl, 
ruby/dl,
ruby queue to cluster and other combos to process GB sized images on a 
routine
basis.  people are doing real-time video processing with ruby using a 
similar
approach.  the key to speed is c, not a vm and, if you ask me, that 
quest is
the white elephant - not ruby's speed.  projects like ruby2c, 
ruby-inline,
ruby/dl, swig - those are the way to speed.  putting a vm on top of ruby 
and
trying to make it fast is like chopping and dropping your mini-van and 
heading
to the track : you are going to get killed.  i think it's right to 
demand a
specfic need for speed because, if people do, i think it will become 
apparent
that any vm will have a hard time competing with a little thinking, a 
lot of
ruby, and a little c via some of the nice projects that let us it.  and, 
if we
stick to gcc, we also get the best portability of any code generator out
there.  lastly, this approach is here __now__ - we don't have to wait.

just my thoughts.

-a

> This thread is *specifically* about getting more computational
> performance for Ruby. Let's not throw it off track by reiterating all
> the *other* reasons the language is "good enough" for us.

How much more? Ruby will never be as fast as hand written assembly
code, well written C, etc. The problem with the performance debate is
premature optimization. When things are too slow, optimize the 5% of
your code that is too slow (after careful profiling to locate that 5%)
-- if necessary as a C extension.

I am all for a fast Ruby, YARV will likely deliver when it is ready.
Even with YARV and a some future JIT compiler, there will be times
when your application is too slow and will require optimization.
Making Ruby faster is not going to help in many cases, you still need
to profile and optimize.

Can Ruby's performance be increased? Yes, but the only "elephant" in
the room is the astral projection of Sir Charles Antony Richard Hoare
muttering that the root of all evil is premature optimization...

pth

Hi cr,

You stated that this kind of comment is off the thread topic:

>>I just want to point out why it doesn't (pragmatically) matter to me
>>personally atm.
>
>Oops! Wrong thread! :-)
>
>cr

But when you started the thread one of the questions you asked was:

  "Why aren't we talking about it all the time?"

"it" being the performance issue. If this was not one of the topics you 
wished to discuss in this thread it would be better not to specifically 
ask about it.

----

I think we aren't talking about it all the time because most of us don't 
have the expertise to contribute to the design and programming of a Ruby 
byte-code compiler however I am closely following the progress of YARV.

Here's a concrete example of something I am doing in Ruby that I would 
like to be faster:

I am using REXML and ruby-spreadsheet to read XML archives of Blackboard 
courses (an e-learning platform) and to process the forum discussions to 
produce interesting (at least to the education researchers I work with) 
statistics of all kinds about the conversations learners have in our 
on-line courses.

Right now it takes about 30s to process a BB archive and produce the 
excel report. I am VERY happy programming using the native ruby tools 
for parsing the BB XML and writing my object representations back out as 
excel worksheets. These libraries are almost trivial to acquire (gem for 
the spreadsheet) and to use. However if I was to instead make an 
interactive application I'd like the process to be sped up by about 10x. 
I could use the ruby bindings for the C libxml library if I need to but 
I'd rather stay totally in Ruby. Until Ruby 2.0 is available I am quite 
happy that if I need the performance I can use a C library to speed up 
the slow parts.

Something practical: I'll test my relatively simple app with the latest 
YARV and post here and to the YARV folks my results.

FYI: Downloading and installing YARV: http://www.atdot.net/yarv/#i-4

Kirk Haines wrote:

> 
> 
> The thing that always stands out to me in these performance discussions is 
> that people argue that Ruby needs a performance improvement, but there are 
> rarely any specifics attached to that analysis.

Very good point.

> ...
> 
> Ruby can be a bottleneck, but for the vast majority of Ruby users and the vast 
> majority of tasks people use if for, it is not the bottleneck.
> 
> It is not an elephant in the living room that everyone is ignoring if the 
> performance simply isn't a limiting factor.  It's an elephant out in the 
> paddock, where it belongs in those cases.  And those cases are the majority 
> of cases.

Or it's the hamster in the living room.  People don't comment on it
because, for the most part, they don't see it.

Those people having speed issues: Is it not a reasonable option to
replace the bottleneck Ruby code with custom compiled C libraries?

I tend to offer that as the general speed solution, but don't have all
that much experience with it.  I've written Ruby modules in C as an
educational exercise, and found it to be as easy as people say it is,
but I've not had to do it in Real Life.


I'm curious if this is as good an option as I've been inclined to 
believe.

--
James Britt

"You harmonize; then you customize."
  - Wilson Pickett

From: ara.t.howard@noaa.gov [mailto:ara.t.howard@noaa.gov]

> similar
> approach.  the key to speed is c, not a vm and, if you ask me, that quest
> is
> the white elephant - not ruby's speed.  projects like ruby2c, ruby-inline,
> ruby/dl, swig - those are the way to speed.

Makes sense. But.
I'd like to think about things like "reasonable slow" and "unreasonable
slow"; and Ruby sometimes is just __unreasonable__ slow. Here are some
examples:

* I'm working with custom UI library, which is based on HTML (namely, on
terrainformatica.com/htmlayout). There are a large amount of objects 
created
corresponding to DOM tree nodes ("large" means thousands) and profiler 
says,
that one of the slowest operation is Class#new - is it normal?

* 5000 objects is loaded from database (by SELECT'ing and setting 
fields).
The major slower is that each object has Date field (Integer#gcd is very
slow). Or Kernel#send (I use it because of my DSL created) and so on.

Summarizing, after large profilings and optimizations, there are oftenly
some core methods (even Fixnum#+) show themselves as major speed 
problems.

This situation I call "unreasonable slow".

> -a

Victor.

Peter Hickman wrote:
> [...] The truth is the performance is an issue for Zed and Luis and
> they are trying to make out that it is not their personal bugbear but
> some sort of conspiracy so they don't look like whiners. [...]

Nice comment Peter!

My comments aren't "look, I cannot code a good program... blame ruby for 
its poor performance"

Ruby do his job, and do it well (as I said in the interview)

I work with special MPEG2 playback hardware, some of them analog only, 
others with digital interface (ASI-DVB)

Even that I like ruby, cannot use to interface that hardware, mostly 
because the quantity of computation needed in the results turns ruby in 
a slow, memory comsuming pig.

I don't say that YARV and later a JIT compiler will solve this, but will 
provide  better performance in general.

I came from a Python background, and ruby was "slow" compared to it 
(http://shootout.alioth.debian.org/gp4sandbox/benchmark.php?test=all&lang=ruby&lang2=python)

But I when come to programing speed (or performance) Ruby wins.

> Yes we wouldn't say no to better performance but we (the Ruby community)
> are not being held back by the current level of performance.

As another community member, I agree with you... thats why I keep 
working in Ruby :-)

Later,

Luis

ara.t.howard@noaa.gov wrote:

> ... 
> you make good points - but let me just throw in my 2 cts:
> 
> virtual machines are stupid.  java is slow.  c++ is slow.  python is slow.
> ruby is slow.  so what.  if you want fast you need c or fortan or 
> assembler.


Ara, do you not see value in a Ruby VM, or is just that speed is not, in
itself, a justification for a VM?

(The speed-related advantage to a VM is that it does not require one to
know C, etc.,  or have a compiler for all target platforms.  Maybe not
as fast as tight compiled C, but also not as much trouble.  It's a 
trade.)


--
James Britt

"You harmonize; then you customize."
  - Wilson Pickett

On 5/22/06, cremes.devlist@mac.com <cremes.devlist@mac.com> wrote:
> The O'Reilly blog has two great interviews posted from the last week
> or so. The first [1] is with Zed Shaw, the author of Mongrel (and a
> bunch of other Ruby frameworks and utilities). A follow-up interview
> with one of the contributors on the Mongrel project, Luis Lavena [2],
> covered some similar ground.

First, thanks for reading and pointing out the interviews.  I'm glad 
people
are enjoying them.   I'm hoping to do some more in the coming 
weeks/months.


> > virtual machine called Rite (or YARV depending on who you talk to)
> > which is showing some real promise and seems to be speed
> > competitive with the fastest Java implementations.

I think you're quoting selectively here.  There are some counter points 
in
both interviews.  Luis said:
  "For us, Ruby on Rails (RoR) lets us build *fast*, clean
  (especially clean) prototypes within hours or days, not
  weeks or months. Then we can move from prototype to
  final product within weeks, even for big projects (like
  migrating a whole application from Zope to RoR)."
(emphasis added)

Zed said:
  "Ruby's advantage though is not in it's blazing
  execution speed but it's blazing coding speed.
  I'll put it to you this way: I wrote mongrel in about
  3 months. That's a full featured stable web server
  that can run four Ruby web application frameworks
  and is already powering many Ruby web sites. This
  wouldn't be possible without Ruby the language."
He also wrote:
  "[A] Rails application many times can outperform
  similar applications in Java or PHP."

This doesn't mean Ruby is a speed demon, or that Zed and Luis
don't want to see general speed increases (we all do, and they
seem to be on the horizon).

For most users (as has been repeated on this thread a few times),
Ruby is fast enough, and easy to optimize when it's not (RubyInline
for pure Ruby, cacheing for Rails).

>
> I tend to agree with this. Whenever the subject of computational
> performance comes up, quite a few posts end up in the thread
> questioning the meaning of the word performance and turn it into an
> argument over semantics. Is it coding speed, clarity of thought,
> readability, etc. To me that kind of crazy semantic dodge doesn't
> serve us well. Those discussions should be remain on computational
> performance since the language has been (mostly) stable in syntax for
> years.

I think the main argument you will run into (over and over again) is, 
have
you profiled your code?  Do you know what's running slowly?  If not, you
may just be moving through a bottleneck a little faster once YARV gets
here.  Really, I'm all for improvements to Ruby in general, but 
profiling
and optimizing your bottlenecks by writing better code will help you out
of a number of tight spots.

>
> So what is being done to get YARV the attention it deserves? Why
> aren't we talking about it all the time? What can I personally do to
> help? Are there valid excuses for not focusing on improving Ruby's
> poor computational performance?

Are you on the Ruby Core mailing list?  Are you posting bugs and
feature requests to the trackers?  Have you downloaded and built
YARV so that you can test it on your system?  All of these are ways
you can get involved and help out.

On 5/22/06, Kirk Haines <khaines@enigo.com> wrote:
>
> The thing that always stands out to me in these performance discussions is
> that people argue that Ruby needs a performance improvement, but there are
> rarely any specifics attached to that analysis.
>
> How much of speed improvement?  How fast should Ruby be?  For what?
>

Please don't get me wrong, I really like Ruby. However, just because
*I* like Ruby doesn't mean that I can start using it to develop
applications within our corporation (I work for a large corporation).
When I brought up using Ruby for application development with my
senior IT manager, he was concerned about:

(1) Performance: Here, the question is NOT about 'how fast should Ruby
be', but is about what people who have the capacity to make
'significant' corporate decisions are 'being told' about Ruby's
performance by consultants. I have seen presentations by
'well-established' consulting companies that show Ruby's performance
to be an order of magnitude slower than Java for web apps. I don't
have time to develop benchmark tests comparing the two languages, but
have to depend on Ruby app. developers to produce those benchmarks;
unfortunately I cannot find them easily. Now, Zed (I would consider
him an authority on Ruby) comes out and talks about performance issues
with Ruby (and YARV will solve it in the future). How am I now to
convince my manager that Ruby is 'better' (within the context of
performance)?

The reason why IT managers want 'performance' even if the application
*really* doesn't require it, is to cover their behinds, among other
thing. For example, if the website is slow, they don't want it to be
attributed to the language, even if that is not the case (could be a
network problem, but the network group would more willingly point
their fingers at the app. development group for their choice of
language than to accept their problem); has happened before and will
happen again.

So, here 'performance' is not for the sake of performance demanded by
the application, but for the sake of 'not wanting to be talked to by
the boss'. That is the reason I said that many IT managers make
'performance' decisions based on 'feelings', not necessarily had data.

(2) Using two different languages for different performance needs:
Some fellow Rubyists have said that they would switch to C if their
app. needed performance boosts. This will work if it is a personal or
small project. But when in an large corporation, switching between
languages to get performance gains will result in maintenance
nightmares. Many IT managers understand this and have experienced
this. Combined that with the mantra 'Java is fast' preached by many
consultants, and the decision for the senior IT manager becomes easy:
one language that can be used for apps that require *real* performance
and for ones that don't: Java.

(3) Survival of Language: This is another bigee. If the IT manager OKs
a language that happens to be a fad, then he might as well bid goodbye
to his job (lost productivity, cost, etc). So, they want to be sure
that the language is going to be in the 'main stream' for a long time.
In their mind, the only way they can be assured of that is:
       (a) if a large corporation is going to support it -- not
necessarily the language, but any application build using the
language. The thinking goes that if a large corporation is going to
put its weight behind the language, then that corporation will not let
the language die. This is what I meant when I said 'lack of corporate
support for Ruby on Rails'; I don't mean to belittle the corporations
that are offering support for Ruby on Rails, but I am talking from a
large corporation's point of view. A good case study would be on how
Linux got its foot hold into the corporate world.

Once again, if Ruby's vision is *not* to penetrate the corporate
environment as Ryan pointed out (but would be a nice validation if it
indeed penetrate the corporate world by *some* means), then this
discussion is moot. Knowing that, I wouldn't try to push Ruby to my
managers; I will use it for personal projects.

On the other hand, if Ruby has a dream of penetrating the corporate
world, then it needs a marketing machine that would deal with the real
and, more importantly, the 'illusionary' problems created in the
corporate world -- just like Firefox's or Linux's marketing machine.
Unfortunately, penetrating the corporate world by *some* means is
never going to happen, unless an concerted effort is made.

Just some thoughts from a guy who has been in the IT hot seat before,
and is glad he is no longer there!

Sorry for this long post.

-Madan.

On Tue, 23 May 2006, Victor Shepelev wrote:

> Makes sense. But.
> I'd like to think about things like "reasonable slow" and "unreasonable
> slow"; and Ruby sometimes is just __unreasonable__ slow. Here are some
> examples:
>
> * I'm working with custom UI library, which is based on HTML (namely, on
> terrainformatica.com/htmlayout). There are a large amount of objects created
> corresponding to DOM tree nodes ("large" means thousands) and profiler says,
> that one of the slowest operation is Class#new - is it normal?

devil's advocate: yes it makes sense that 'new' is slow.  it's generally 
slow
in all langs, including c++ (where stl insertion causes 3 of ctors and 
the
profiler will show this) and perl, which is claimed faster than ruby 
but, when
objects are used, crawls compared to ruby because object creation is so 
bad in
it.  the fact that object creation is so slow is why people historically 
don't
represent chars as objects in text editors.  that said, perhaps it can 
be sped
up - at least this is a good, concrete example!

> * 5000 objects is loaded from database (by SELECT'ing and setting fields).
> The major slower is that each object has Date field (Integer#gcd is very
> slow). Or Kernel#send (I use it because of my DSL created) and so on.

probably this could be sped up too - but your point is taken.

> Summarizing, after large profilings and optimizations, there are oftenly
> some core methods (even Fixnum#+) show themselves as major speed problems.
>
> This situation I call "unreasonable slow".

i like where you are going with this - let's see if a critical analysis 
of
some 'real' issues sheds light on whether they point to the need for a 
vm (my
feelings are that it will not, but i'd be happy to be proven wrong).  i 
can't
think of a better way than to post some specific code and see what 
people can
do with it.  history has shown that posting such challenges to the list 
yields
at least an order of magnitude speed-up, if it doesn't i'd say you are
correct - anyone have an example less than 100 lines or so we can play 
with?
without an example i'm afraid this discussion will be little more than 
hot
air.  let's get something real to work with, prove that it is 
'unreasonably'
slow, and then discuss.

kind regards.

-a

On Tue, 23 May 2006, James Britt wrote:

> itself, a justification for a VM?
i'm saying that gcc kills any vm both on speed and portability, 
including
windows, and has decades of hard work put in.

> (The speed-related advantage to a VM is that it does not require one to know 
> C, etc.,  or have a compiler for all target platforms.  Maybe not as fast as 
> tight compiled C, but also not as much trouble.  It's a trade.)

hmmm.  i think sun's experience with the java vm shows otherwise - 
remember
when you couldn't even 'stop' threads!  i think making a fast, protable, 
vm is
a lions job.  however, i'd be quite happy to be wrong, it just seems
un-likely.  consider how long has parrot been 'almost', for example...

-a

From: ara.t.howard@noaa.gov [mailto:ara.t.howard@noaa.gov]
> > that one of the slowest operation is Class#new - is it normal?
> represent chars as objects in text editors.  that said, perhaps it can be
> sped
> up - at least this is a good, concrete example!

Caugh... I've thought, that the main reason not to have an object for 
each
char is memory reason... But I can miss.
BTW, I was the C++ guy before Ruby, and I've never seen object creation 
took
a noticeable amount of performance. As well as other "simple core"
operations, like working with numbers and the same stuff.
Hence my astonishement, when after optimizing, optimizing, optimizing my
program with large UI and DB operations I see the top time-eating 
methods is
not UI, not DB, not sorting or filtering large arrays, but dumb stuff 
like
Fixunum#+ or Class#new.

> 'unreasonably'
> slow, and then discuss.

OK, maybe later. Right now I haven't any "small function which is too 
slow",
but entire program works with unacceptable speed.
On my old Celeron 900 I can hardly debug it (but even on more modern 
1500+
and 2000+ machines, which are normal in Ukraine, it also works slow).

> -a

Victor.

On 5/22/06, Madan Manoharan <madan.manoharan@gmail.com> wrote:
> *I* like Ruby doesn't mean that I can start using it to develop
> have time to develop benchmark tests comparing the two languages, but
> have to depend on Ruby app. developers to produce those benchmarks;
> unfortunately I cannot find them easily. Now, Zed (I would consider
> him an authority on Ruby) comes out and talks about performance issues
> with Ruby (and YARV will solve it in the future). How am I now to
> convince my manager that Ruby is 'better' (within the context of
> performance)?

Read what Zed said about Rails app performance though.  Even though
Ruby can be slow, he said that with its built in caching, Rails apps are 
often
faster than Java Web apps.


>
> The reason why IT managers want 'performance' even if the application
> *really* doesn't require it, is to cover their behinds, among other
> thing. For example, if the website is slow, they don't want it to be
> attributed to the language, even if that is not the case (could be a
> network problem, but the network group would more willingly point
> their fingers at the app. development group for their choice of
> language than to accept their problem); has happened before and will
> happen again.
>

sorry, but if you don't have the logging/instrumentation to show where 
the
slowdown is, it doesn't matter what language you've written in.  Using
a languages because its marketing is better in order to "protect" 
yourself
is a good way to end up in bigger trouble down the line.  Profile, 
Optimize,
Log.   Hard, verifiable numbers will win everytime -- if they don't 
you've got
another problem.


> So, here 'performance' is not for the sake of performance demanded by
> the application, but for the sake of 'not wanting to be talked to by
> the boss'. That is the reason I said that many IT managers make
> 'performance' decisions based on 'feelings', not necessarily had data.
>

see above.

>
you don't need to switch out the whole language.  Ruby uses C extensions
quite readily and they're not too hard to write (RubyInline makes it 
even
easier.)  You still get something that looks and feels like Ruby from 
the
high level view, only the bottleneck(s) have been replaced.



The remainder of this email is off topic to the general performance 
thread,
but I find it to be wrong-headed as well.  If need be, maybe we should 
start
another email thread to discuss Ruby and it's commercial support or lack
thereof.

On May 22, 2006, at 11:59 AM, pat eyler wrote:

>> > needed to solve the problem. One solution in the works is a real
>> > virtual machine called Rite (or YARV depending on who you talk to)
>> > which is showing some real promise and seems to be speed
>> > competitive with the fastest Java implementations.
>
> I think you're quoting selectively here.  There are some counter  
> points in
> both interviews.

Of course I engaged in selective quoting. That's 90% of the fun! :-)

>> So what is being done to get YARV the attention it deserves? Why
>> aren't we talking about it all the time? What can I personally do to
>> help? Are there valid excuses for not focusing on improving Ruby's
>> poor computational performance?
>
> Are you on the Ruby Core mailing list?  Are you posting bugs and
> feature requests to the trackers?  Have you downloaded and built
> YARV so that you can test it on your system?  All of these are ways
> you can get involved and help out.

I am on ruby-core but mostly just so I can bask in the glow of the
smart people on that list. Plus, I *have* submitted at least one bug
report to that least (only to have it turn out to be clumsy handling
if IPv4 and IPv6 on OSX).

I have not yet downloaded and built YARV, but I will in the very near
future. I am not a great programmer, but as a result I do find neat
little test cases that can try the patience of even the most liberal
parser & compiler!

cr

Madan Manoharan wrote:

> corporate world -- just like Firefox's or Linux's marketing machine.
> Unfortunately, penetrating the corporate world by *some* means is
> never going to happen, unless an concerted effort is made.
> 
Ruby doesn't have a vision or a dream; Rubyists have dreams and visions.

Ruby is what you make it.



--
James Britt

"You harmonize; then you customize."
  - Wilson Pickett

On Tue, 2006-05-23 at 01:32 +0900, Peter Hickman wrote:
> The problem is that Zed and Luis make out that performance is something 
> that the whole Ruby community is deliberately ignoring. This is not the 
> case, there are plenty of people who have no issue with the performance 
> of Ruby. The truth is the performance is an issue for Zed and Luis and 
> they are trying to make out that it is not their personal bugbear but 
> some sort of conspiracy so they don't look like whiners. I have never 
> though "damn Ruby's dismal performance, it should be able to do this 
> faster" and I probably never will.

Hey Peter, thanks for deliberately ignoring my whining.  Now, if you'll
excuse me I have to go and continue to security audit the C extensions
in Mongrel so that I can be sure there's no buffer overflows.

Man, if only Ruby were faster.  Then I wouldn't have to write so much
stuff in C and could just worry about crafting beautiful code.  Oh well,
back to work on my C .... I mean Ruby ... program.

On 5/22/06, James Britt <james_b@neurogami.com> wrote:
>
> Ruby doesn't have a vision or a dream; Rubyists have dreams and visions.
>
> Ruby is what you make it.
>
>

Thanks for catching my faux pas! I should have written: 'If Rubyists
have a vision to ....' instead of 'If Ruby's vision ...'

-Madan.

On 5/22/06, pat eyler <pat.eyler@gmail.com> wrote:
> On 5/22/06, Madan Manoharan <madan.manoharan@gmail.com> wrote:
> > On 5/22/06, Kirk Haines <khaines@enigo.com> wrote:
> > >
>
> Read what Zed said about Rails app performance though.  Even though
> Ruby can be slow, he said that with its built in caching, Rails apps are often
> faster than Java Web apps.
>
>

Pat, that is exactly the point: Ruby is slow; so you have to create
work arounds! Not something I would like to hear, if I were evaluating
a language.

We need a version of Ruby that is *comparable* in execution speed to
Python or Java. If it is YARV, then let us roll it out fast (no pun
intended)

-Madan.

Re: Zed and Luis drop the bomb on Ruby's poor performance

On Tue, 23 May 2006, Victor Shepelev wrote:

> Caugh... I've thought, that the main reason not to have an object for each
> char is memory reason... But I can miss.

easy to avoid with flyweight pattern like fixnums in ruby.  then one is 
left
with performance of object lists vs. sequential memory strings.

> BTW, I was the C++ guy before Ruby, and I've never seen object creation took
> a noticeable amount of performance. As well as other "simple core"
> operations, like working with numbers and the same stuff.  Hence my
> astonishement, when after optimizing, optimizing, optimizing my program with
> large UI and DB operations I see the top time-eating methods is not UI, not
> DB, not sorting or filtering large arrays, but dumb stuff like Fixunum#+ or
> Class#new.

we are obiviously working in different programming domains - but it does 
sound
like you have real issues, just ones i've never experienced...

> OK, maybe later. Right now I haven't any "small function which is too slow",
> but entire program works with unacceptable speed.  On my old Celeron 900 I
> can hardly debug it (but even on more modern 1500+ and 2000+ machines, which
> are normal in Ukraine, it also works slow).

ok then - your contribution, whenever you have time, of code will 
certainly
help the cause to speed-up ruby or, on the contrary, to decide there is 
no
problem.  we really need examples to spend our energy productively i 
think.

cheers.

-a

I've always had a sense that some of what makes Ruby so beautiful is
_precisely_ what makes it slow. Some of this is a matter of
implementation (if anyone remembers the early implementations of ML,
you know what good tail-recursion can do for you. Plus Ruby seems to
get super-linearly worse as its working set grows), but some of it is
fundamental. Metaprogramming is very hard to do without depending on a
lot of string comparing at runtime. I've got a language design
background (I made a lot of money writing compilers for scripting
languages), and I've had to deal with this problem in other languages.
YARV looks like beautiful work, but I have a gut-feeling that it won't
solve the "speed" problem all by itself.

I won't wade into the whole "is Ruby fast enough" question because
defining "enough" really deserves its own thread. It's an important
question though, especially as regards Ruby in corporate IT.

Quoting james_b@neurogami.com, on Tue, May 23, 2006 at 01:50:34AM +0900:
> Those people having speed issues: Is it not a reasonable option to 
> replace the bottleneck Ruby code with custom compiled C libraries?

Its not obvious that this can always help, though it might in some 
cases.

I've been benchmarking vpim's icalendar decoding, and haven't found any
way to get it significantly faster yet.

Its a bit of an issue, a utility I use a lot to print out events and
todos for the next week usually takes 5-10 seconds to run. This is long
enough to notice!

When I tried running profile, and it told me 40% of the time was spent
in String case-insensitive comparisons, so I reworked the internals to
always uppercase strings so that wasn't necessary. Got about 5%.

I even tried using symbols instead of strings, I expected from lisp
experience that symbol comparisons would be really fast, but they
are only slightly faster (for strings of the size I'm working with
anyway). **

I ran out of time at that point, but its not clear to me what kind
of algorithmic changes I can make to speed things up, and I'm not too
sure where the time is being spent, other than in string operations.

There definitely isn't one part of the app I can point to and say
"recode that in C and my problems will go away". Also, you might have
noticed but ruby libraries that require extensions to be compiled tend
to not get a lot of traction in the ruby community.

Anyhow, I just spent a weekend on performance tuning, I might still find
out some clever way to restructure my code so that it gets twice as
fast, but a 2-3 times speed improvement would make a big difference to
my library. The difference between 6 seconds and under 2 seconds is huge
in human interface terms.

Cheers,
Sam


**

                          user     system      total        real
noop                  0.700000   0.000000   0.700000 (  0.698633)
String#==             2.410000   0.010000   2.420000 (  2.445288)
String#casecmp        2.650000   0.000000   2.650000 (  2.652620)
Symbol#==             2.280000   0.000000   2.280000 (  2.274505)
String#to_sym         2.460000   0.000000   2.460000 (  2.449362)
String#up             4.730000   0.000000   4.730000 (  4.721140)
String#up#to_sym      6.030000   0.000000   6.030000 (  5.986897)


-- bench.rb --
require 'benchmark'
require 'pp'

str0 = "a" * 20
str1 = str0.upcase
sym0 = str0.to_sym
sym1 = str1.to_sym

$N = 1000000

Benchmark.bm(20) do |x|

	x.report("noop")  do
    $N.times do
    end
  end

	x.report("String#==")  do
    $N.times do
      str0 == str1
    end
  end

	x.report("String#casecmp")  do
    $N.times do
      str0.casecmp(str1)
    end
  end

	x.report("Symbol#==")  do
    $N.times do
      sym0 == sym1
    end
  end

	x.report("String#to_sym")  do
    $N.times do
      str1.to_sym
    end
  end

	x.report("String#up")  do
    $N.times do
      str1.upcase
    end
  end

	x.report("String#up#to_sym")  do
    $N.times do
      str1.upcase.to_sym
    end
  end

end

Hi Sam!
                          user     system      total        real
> noop                  0.700000   0.000000   0.700000 (  0.698633)
> String#==             2.410000   0.010000   2.420000 (  2.445288)
> String#casecmp        2.650000   0.000000   2.650000 (  2.652620)
> Symbol#==             2.280000   0.000000   2.280000 (  2.274505)
> String#to_sym         2.460000   0.000000   2.460000 (  2.449362)
> String#up             4.730000   0.000000   4.730000 (  4.721140)
> String#up#to_sym      6.030000   0.000000   6.030000 (  5.986897)

[...]
  x.report("Fixnum#==")  do
    $N.times do
      0 == 1
    end
  end
[...]
  x.report("String#up!")  do
    $N.times do
      str1.upcase!
    end
  end
[...]

                          user     system      total        real
noop                  0.157000   0.000000   0.157000 (  0.156000)
Fixnum#==             0.390000   0.000000   0.390000 (  0.391000)
String#==             0.438000   0.000000   0.438000 (  0.437000)
String#casecmp        0.578000   0.000000   0.578000 (  0.594000)
Symbol#==             0.390000   0.000000   0.390000 (  0.390000)
String#to_sym         0.563000   0.015000   0.578000 (  0.579000)
String#up             1.437000   0.016000   1.453000 (  1.484000)
String#up!            0.532000   0.000000   0.532000 (  0.531000)
String#up#to_sym      1.890000   0.000000   1.890000 (  1.922000)


Well i think in this trivial cases the problem isn't the string
compare or whatever C algorithm is called - the problem is the call
itself. (two trivial fixnum compares are already more expensive
than a single String compare)

String#upcase constructs a new object (using the bang method is
way faster)

Conclusion: method calls in general and object creation are the
bottlenecks in this (very, very) simple cases.

cheers

Simon

On May 22, 2006, at 3:49 PM, Sam Roberts wrote:

> I even tried using symbols instead of strings, I expected from lisp
> experience that symbol comparisons would be really fast, but they
> are only slightly faster (for strings of the size I'm working with
> anyway). **

Don't forget that string comparison is pretty quick if the strings
differ in the first character. If you try comparing to the same
string and make the string a lot longer (say 2000 characters) you'll
get a situation where symbol comparisons are about three times faster
than string comparisons.

You also need a better noop. If you call a method in there that does
nothing then I'm getting:

                           user     system      total        real
noop                  0.490000   0.000000   0.490000 (  0.664154)
do nothing method     1.650000   0.030000   1.680000 (  2.501031)
String#==             3.030000   0.040000   3.070000 (  4.191784)
String#eql?           2.930000   0.030000   2.960000 (  4.237489)
String#equal?         1.300000   0.010000   1.310000 (  1.858182)
Symbol#==             1.310000   0.020000   1.330000 (  1.979673)
Symbol#eql?           1.300000   0.010000   1.310000 (  2.036749)
Symbol#equal?         1.310000   0.010000   1.320000 (  1.727041)

(this is on OS X... ignore the real time since it is pretty much
random on OS X)


Have you tried profiling your code?

----
Bob Hutchison                  -- blogs at <http://www.recursive.ca/
hutch/>
Recursive Design Inc.          -- <http://www.recursive.ca/>
Raconteur                      -- <http://www.raconteur.info/>
xampl for Ruby                 -- <http://rubyforge.org/projects/xampl/>

Quoting SimonKroeger@gmx.de, on Tue, May 23, 2006 at 05:12:14AM +0900:
> Hi Sam!

Hi Simon,

> Conclusion: method calls in general and object creation are the
> bottlenecks in this (very, very) simple cases.

My conclusion as well.

My real app basically runs over all the lines in a set of files using
IO#each, runs a regex, turns the matched strings into objects, so is
pretty much dominated by these simple cases (I think, and profile
suggests).

Your !-optimizations are interesting. I'll look back at my low-level
code. Maybe by changing my "name = $2.upcase" statements to "name = $2;
name.upcase!" I can get some good speedups. Depends on whether I need to
keep both versions around, I'll have to construct some test cases and
profile them.

Thanks,
Sam

Sam Roberts wrote:
> My real app basically runs over all the lines in a set of files using
> Thanks,
> Sam

There were some performance tips posted here some time ago, keeping the
number of objects low is definitely a good start. I just don't like the
bang methods because you can't method-chain with them. (a lot of the
return nil if there is no change)

If the files aren't that big try to read them in one big read - should
be way faster.

Another one of the tips posted before was: use sqlite - and i can
wholeheartedly support that. If you have the chance to change your data
format to a sqlite database you can run a select on thousands of
lines in virtually no time. (and it's really, really painless to use)

cheers

Simon

The elephant in the room is that programmers can't put the importance
of execution speed into perspective :-)

There isn't a lot of discussion on the Ruby list because execution
speed isn't the biggest issue for most people.

However, when execution speed *is* an issue it can be a bad bad thing.

---

Anyhow, here are a few random thoughts (with some real numbers)
around the issue.

---

First off. Premature optimisation. We've all heard of it, even heard
that's it is bad. Okay. Quick quiz. Which loop is faster?

#1
for i in 1..(a.length - 1) do
   ... do something to a[i]
end

or

#2
a.each { | thing | ... do something to thing }

or

#3
for thing in a do
   ... do something to thing
end

??

In Ruby 1.6.x #1 was a bit faster than #3 which was a lot faster than #2

(When I replaced the 'each' loop form in xampl-pp -- an XML pull
parser that I wrote -- with the 'for ... in' loop form the throughput
of the parser was increased by 2-4 times).

In Ruby 1.8.4, #3 is about 15% faster than #2 which is quite a lot
faster than #1

(So, I know how to speed up my parser, but I have not done this
yet... it is working well enough and because the real speed up is to
go to libxml or something).

So. What happened? What we consider to be idiomatic Ruby is now only
a little slower than non-idiomatic Ruby. What would have happened if
the community had *really* regarded execution speed as a primary
criteria? We'd have a different idea of 'idiomatic' is what.

Ruby's focus is on something other than performance.

---

So what happens when speed is important and you've already optimised
your algorithms? These days you move out of Ruby. That's your only
option. I thank Zed for doing that with Mongrel -- it makes a big
difference.

Would a VM have helped? Sure. Sufficiently? I doubt it. Java is what,
ten times faster than Ruby? Lets say. Is that sufficient to actually
solve the problem?

Not for a lot of the examples that come up.

Definitely not for me.

---

Here's an example. I've got a system written in Rails that is a kind
of multi-user authoring tool that runs as a webapp. My customers
build multiple inter-dependent websites using it. Publishing takes
anywhere from 7 to 45 seconds. This is Rails we are talking about
here... nobody gets to do anything while that publishing is going on.
There are a number of ways to alleviate the problem (in particular
async publishing using DRb or something like that).

Thing is, I got bored last weekend and decided to re-write the main
culprit in Common Lisp. I'm using the liquid templating system
because it is simple, my customers like it, and you can't change any
data when using it. But it is slow (with respect to my applications
requirements) -- I *still* *recommend* it. Common Lisp is roughly the
speed of C++, maybe a tad faster these days (actually, on OS X with
that obnoxious gcc 4.x error that prevents compiling at higher
optimisation levels than O1, CL is quite a bit faster than C even --
but lets not go there). The CL version of liquid was more than 120
times faster than Ruby -- and I did *nothing* to optimise the code.
That's publishing in less than 0.4 seconds in CL, 45 seconds in Ruby,
and lets be generous, 4.5 seconds in Java. Java doesn't solve my
problem. I doubt a VM for Ruby would either.

What am I going to do? Well, first off, async publishing using DRb.
If I still have a problem I'll rig up something with CL. No, I'm not
going to jump straight to CL (and just to be crystal clear here, I
like CL just fine, and I like Ruby just fine -- I've just got other
higher priority things than raw execution speed to consider).

---

Lets talk a little more about webservers. I've written a lot of Java
based web applications over the years. I had a new webapp I had to
write and I was more than a little concerned with performance in
Java. I had prototyped something but it was horribly complex trying
to minimise page generation (it was a completely dynamic site, also
involving the construction of multi-page websites). So I wrote the
thing in Common Lisp in the end. Two things: first, it generated the
pages so quickly I didn't have to optimise anything, it was quicker
to just re-generate; second, I used TBNL (slightly modified lisp
framework) to serve the web-pages -- it was able to sustain about 70M/
s in CL vs about 15k/s in Java (I thought I was going to have a heart
attack, that's more than 4500 times higher throughput).

What's the point? Well, that how you do things can be pretty
important. RoR is roughly the same throughput as my java applications
have been -- even using Webrick, and Mongrel is faster still --
assuming no long-running operations. Also, I still don't do
everything in CL.

---

The trouble with Ruby is that there are situations where some low-
level operation is being executed so often that it becomes the
problem. For example, my XML parser string manipulation is the main
consumer of time -- lots and lots of calls to pretty quick methods
(Sam Roberts' example in this thread of string compares shows the
problem, string comparison can be faster than a method call, yet
still be responsible for most of your execution time). I don't have a
solution for this. *Maybe* a VM will help with this but I'm not
really sure of that.

I also have my suspicions about garbage collection in Ruby. I may be
wrong to suspect GC, but you won't need a VM to do something to speed
it up.

---

There are trade-offs all over the place here. And aside from using
Mongrel, I don't think I've ever chosen a route (in Ruby) where
native code was involved. And that includes databases -- I use a pure
Ruby thing, xampl (see link below), and don't regret it.

Cheers,
Bob

----
Bob Hutchison                  -- blogs at <http://www.recursive.ca/
hutch/>
Recursive Design Inc.          -- <http://www.recursive.ca/>
Raconteur                      -- <http://www.raconteur.info/>
xampl for Ruby                 -- <http://rubyforge.org/projects/xampl/>

On Tue, 23 May 2006, Bob Hutchison wrote:

> Ruby's focus is on something other than performance.

exactly.  write poetry in french - scream insults at hockey games in 
english.


> Not for a lot of the examples that come up.
>
> Definitely not for me.

my stance exactly.


nice summary btw.

-a

Quoting hutch@recursive.ca, on Tue, May 23, 2006 at 05:36:28AM +0900:
> Don't forget that string comparison is pretty quick if the strings  
> differ in the first character. If you try comparing to the same  
> string and make the string a lot longer (say 2000 characters) you'll  
> get a situation where symbol comparisons are about three times faster  
> than string comparisons.

But the strings in my app are short, about 20 chars.

> Have you tried profiling your code?

Yes, performance was dominated by string comparisons, I wrote the
benchmark to see if changing to symbols would help.

Sam

> > It is not an elephant in the living room that everyone is ignoring if the
> > performance simply isn't a limiting factor.  It's an elephant out in the
> > paddock, where it belongs in those cases.  And those cases are the majority
> > of cases.
>
> Or it's the hamster in the living room.  People don't comment on it
> because, for the most part, they don't see it.

No! It's the hamster in the circus! Or no! It's the elephant in the
garage! It's the midget on top of your car! It's the rabbit in the
moon!

Sorry, I'm having issues taking this seriously...I saw the subject
header and I was like, "Somebody set us up the bomb!"

ara.t.howard@noaa.gov wrote:
> On Tue, 23 May 2006, Bob Hutchison wrote:
> 
>> Ruby's focus is on something other than performance.
> 
> 
> exactly.  write poetry in french - scream insults at hockey games in 
> english.

Unless they're French-Canadian.



--
James Britt

"Programs must be written for people to read, and only incidentally
  for machines to execute."
   - H. Abelson and G. Sussman
   (in "The Structure and Interpretation of Computer Programs)

On May 22, 2006, at 9:32 PM, James Britt wrote:

> ara.t.howard@noaa.gov wrote:
>> On Tue, 23 May 2006, Bob Hutchison wrote:
>>> Ruby's focus is on something other than performance.
>> exactly.  write poetry in french - scream insults at hockey games  
>> in english.
>
> Unless they're French-Canadian.

Hmmm? French-Canadians are pretty good at poetry.... Oh, you mean the
insults... well of course it's in english... the english in Canada
don't speak french... not a lot of fun if nobody understands your
insults :-)

Cheers,
Bob


>
----
Bob Hutchison                  -- blogs at <http://www.recursive.ca/
hutch/>
Recursive Design Inc.          -- <http://www.recursive.ca/>
Raconteur                      -- <http://www.raconteur.info/>
xampl for Ruby                 -- <http://rubyforge.org/projects/xampl/>

On 5/22/06, Peter Hickman <peter@semantico.com> wrote:
> One question I have to ask is "Wouldn't it make more sense to piggy back
> on parrot (the Perl v6) engine?" and get all the benefit of the
> performance improvements that parrot will pick up and the fact that
> there are many, many very clever people working on parrot whose work we
> could make use of.

Are there? Parrot has always seemed like a dead Parrot, to me, with
far too much trying to be done in too poorly a way.

> Is there a reason why we are reinventing the wheel?

Yes. Ask ko1 and matz why for specifics, but essentially Ruby is more
dynamic than other VMs expect it to be.

-austin

On 5/22/06, Madan Manoharan <madan.manoharan@gmail.com> wrote:
>   performance issues with Ruby (and YARV will solve it in the future).
>   How am I now to convince my manager that Ruby is 'better' (within
>   the context of performance)?

Zed is but one of many "authorities" on Ruby. He's got certain
experiences, but others -- and there's a lot of them -- have different
experiences.

Why should we develop *your* benchmarks to challenge something that
consulting companies are lying about? If someone gives you benchmark
results without publishing the source code for review and having
independent verification of those results with properly tuned algorithms
in both cases -- they are lying. It's sort of like the Alioth shootout
(that someone has already posted about). There's not a lick of worthhile
truth on that site, because there are implementations of algorithms that
are unmaintainable (and other implementations that require esoteric
hacks to get working).

> The reason why IT managers want 'performance' even if the application
> *really* doesn't require it, is to cover their behinds, among other
> thing. For example, if the website is slow, they don't want it to be
> attributed to the language, even if that is not the case (could be a
> network problem, but the network group would more willingly point
> their fingers at the app. development group for their choice of
> language than to accept their problem); has happened before and will
> happen again.

Then you need to have stronger IT managers. The development group should
be familiar with various profiling tools if they have an asinine network
group who points fingers without absolute proof.

Sorry, but it's true. Performance of a website/application is the
responsibility of the entire company.

> So, here 'performance' is not for the sake of performance demanded by
> the application, but for the sake of 'not wanting to be talked to by
> the boss'. That is the reason I said that many IT managers make
> 'performance' decisions based on 'feelings', not necessarily had data.

It is usually bad data, though.

> (2) Using two different languages for different performance needs:
> Some fellow Rubyists have said that they would switch to C if their
> app. needed performance boosts. This will work if it is a personal or
> small project. But when in an large corporation, switching between
> languages to get performance gains will result in maintenance
> nightmares. Many IT managers understand this and have experienced
> this. Combined that with the mantra 'Java is fast' preached by many
> consultants, and the decision for the senior IT manager becomes easy:
> one language that can be used for apps that require *real* performance
> and for ones that don't: Java.

This is not correct. I have worked in several companies and have used,
over the years, no fewer than fifteen languages. We use the language
that is most appropriate to the task at hand. Now, switching between
Java and C is a maintenance nightmare because JNI sucks so badly.

(Remember: those same consultants who are now preaching "Java is fast,
Ruby is slow" were the jackasses a decade ago who were preaching "C++ is
fast, Java is slow".)

> (3) Survival of Language: This is another bigee. If the IT manager OKs
> a language that happens to be a fad, then he might as well bid goodbye
> to his job (lost productivity, cost, etc). So, they want to be sure
> that the language is going to be in the 'main stream' for a long time.
> In their mind, the only way they can be assured of that is:

This is the least worry. Ruby will survive with or without "enterprise"
support.

> A good case study would be on how Linux got its foot hold into the
> corporate world.

Usually? Through the back door.

> Once again, if Ruby's vision is *not* to penetrate the corporate
> environment as Ryan pointed out (but would be a nice validation if it
> indeed penetrate the corporate world by *some* means), then this
> discussion is moot. Knowing that, I wouldn't try to push Ruby to my
> managers; I will use it for personal projects.

Then that would be your mistake. Know the battles you can fight. Prove
Ruby useful and its use will grow.

> On the other hand, if Ruby has a dream of penetrating the corporate
> world, then it needs a marketing machine that would deal with the real
> and, more importantly, the 'illusionary' problems created in the
> corporate world -- just like Firefox's or Linux's marketing machine.
> Unfortunately, penetrating the corporate world by *some* means is
> never going to happen, unless an concerted effort is made.

Fortunately, Ruby doesn't need this.

-austin

On 5/22/06, Madan Manoharan <madan.manoharan@gmail.com> wrote:
> On 5/22/06, pat eyler <pat.eyler@gmail.com> wrote:
> > On 5/22/06, Madan Manoharan <madan.manoharan@gmail.com> wrote:
> > > On 5/22/06, Kirk Haines <khaines@enigo.com> wrote:
> > Read what Zed said about Rails app performance though.  Even though
> > Ruby can be slow, he said that with its built in caching, Rails apps are often
> > faster than Java Web apps.
> Pat, that is exactly the point: Ruby is slow; so you have to create
> work arounds! Not something I would like to hear, if I were evaluating
> a language.

No, that's not the point. That's you missing the point. Caching is
usually better than recomputing for certain classes of applications in
any case, even if it's partial caching. There's a reason that you use
templates wherever possible in output -- it's partial caching.

> We need a version of Ruby that is *comparable* in execution speed to
> Python or Java. If it is YARV, then let us roll it out fast (no pun
> intended)

It will be rolled out in its time.

And, to be honest, Ruby *is comparable* to the others in execution.
Numbers, Madan, numbers. Hard numbers will run every time, not
consultant feelings.

-austin

On Tue, May 23, 2006 at 03:39:37AM +0900, Madan Manoharan wrote:
> work arounds! Not something I would like to hear, if I were evaluating
> a language.

That is *exactly* the *other* point. Rails has superior built in
caching that makes up for any perceived or true interpreter slowness.
Caching is not a workaround, it is the solution to a problem, already
solved for you by the rails team, and easily reuseable.

If said web apps are faster than J2EE already, will your manager care
if the interpreter itself is slow? Will he even understand?

Jürgen

Peter Hickman wrote:
> One question I have to ask is "Wouldn't it make more sense to piggy back 
> on parrot (the Perl v6) engine?" and get all the benefit of the 
> performance improvements that parrot will pick up and the fact that 
> there are many, many very clever people working on parrot whose work we 
> could make use of.
> 
> Is there a reason why we are reinventing the wheel?
> 

There is indeed a project to piggyback on Parrot (called Cardinal),
probably a little less mature than Parrot itself, which is saying
something.

The wheel is being reinvented because, while Ruby can be written
for Parrot, Parrot isn't written (entirely) for Ruby.

We want a nice, clean impedance match between the language and
the bytecode compiler. Also complete control over its specification
is handy.

Finally, I wouldn't be surprised if YARV ends up outracing Parrot
(both in maturity and in running speed). But we'll see.


Hal

Peter Hickman wrote:
> are not being held back by the current level of performance.
Precisely. I'm not pretending there's no elephant in the room. My room
actually has no elephant in it. If yours does, well, my sympathies.

Hal

On 5/22/06, Madan Manoharan <madan.manoharan@gmail.com> wrote:
> not).
>
> IMO, if Ruby's vision is to penetrate the corporate environment, Ruby
> should address its performance issues among other things (e.g. support
> for apps -- Ruby on Rails).

This is exactly - but *exactly* - the sort of thing that was said
about Java at its onset. Especially in environments where it was
evaluated against a C++ backdrop. And guess what?  Eventually, it
didn't matter!  Java did get widely adopted because it was a better
language and offered an improved development experience.

In fact IIRC, Java was relatively much slower than Ruby when it
started gaining momentum. Certainly if you consider the computer speed
back then. How many project managers are saying "well, Java would give
us a much faster development process, and with the superior
maintainability we should not only beat everyone else to the market
but also keep ahead of the competition, but let's write the thing in
C++ to save those few extra CPU cycles"?.  Now replace C++ with C,
Fortran or Assembly. How much software is written today in Assembly to
truly squeeze the last drops of performance out of the machine? ...
Exactly.

In fact, most of C++ (and I'd arge - any lanauge lower-level than
Ruby, including Java)  use is *misuse*. Most developers using C++
today are doing it out of inertia, habit, or at most for legacy
reason. Even without the insanely powerful machines considered "norm"
on our desktop today, C++ development is 99% premature optimization.
And the funny fact is that most of those 99% won't even get the
performance they wasted 50%-90% of their development sweat (and
tears!) upon. Since contrary to popular myth, writing your application
in C++ won't make it magically faster. The developer would have to be
pretty smart for that. And for those 99%, being smart means they would
have been using Java. If they were even smarter, they'd be using Ruby
:P

James Britt wrote:
> 
> Ara, do you not see value in a Ruby VM, or is just that speed is not, in 
> itself, a justification for a VM?
> 
> (The speed-related advantage to a VM is that it does not require one to 
> know C, etc.,  or have a compiler for all target platforms.  Maybe not 
> as fast as tight compiled C, but also not as much trouble.  It's a trade.)

I thought he was saying it's not that significant a speed increase.

I agree with you, if I understand you. Blazing speed requires C or
some such. If you just want overall reasonable speed, a VM is OK;
it gives you some speed without needing C.


Hal

2006/5/22, ara.t.howard@noaa.gov <ara.t.howard@noaa.gov>:
> >
> > Ara, do you not see value in a Ruby VM, or is just that speed is not, in
> > itself, a justification for a VM?
>
> i'm saying that gcc kills any vm both on speed and portability, including
> windows, and has decades of hard work put in.

I wouldn't be so sure of that. I work with Java for several years now
and we do have performance critical parts in our application. First, I
doubt that a C++ version will show *significant* performance
improvements if at all (because there's a lot IO going on but also
because a JVM is neither slow nor dumb). Second, with a JVM you get a
lot of smartness with regard to memory handling etc., runtime
optimization and instrumentation. Current JVMs' instrumentation
interfaces give you a lot of options in analyzing runtime behavior of
your application.

<OT>Personally I prefer Java over C++ because you get a lot things for
free (thread handling, memory management) that you have to use non
standard libs in C++ for or have to do it on your own. Plus, you still
get a relatively clean OO programming model. From what I hear from my
colleagues the portability of C++ apps is an issue in practice as soon
as you do not limit yourself to Unixes.</OT>

> > (The speed-related advantage to a VM is that it does not require one to know
> > C, etc.,  or have a compiler for all target platforms.  Maybe not as fast as
> > tight compiled C, but also not as much trouble.  It's a trade.)
>
> hmmm.  i think sun's experience with the java vm shows otherwise - remember
> when you couldn't even 'stop' threads!  i think making a fast, protable, vm is
> a lions job.  however, i'd be quite happy to be wrong, it just seems
> un-likely.  consider how long has parrot been 'almost', for example...

Again, if you look at and use modern JVM's from Sun you'll see that
it's quite fast and stable. I know it's a lion's job but Sun and
others have invested serious amount of work into this.  If you want to
get an impression of this I can dig up some links on JVM internals
that will show you how smart and complex those beasts have become.
I'd certainly love to see Ruby use a VM with that much power. (And
yes, I don't have performance issues with Ruby, just to state it. :-))

Kind regards

robert

Francis Cianfrocca wrote:
> solve the "speed" problem all by itself.
Maybe you're the one to ask then.  With all the talk of VMs, everyone
seems to make the comparison to Java and Python.  Surely Lisp is a
better comparison?  Lisps can be stupid fast (as mentioned in another
response to this thread), so what does Lisp do right that Ruby doesn't?

On May 23, 2006, at 12:15 AM, Alex Young wrote:

>> languages), and I've had to deal with this problem in other  
>> languages.
>> YARV looks like beautiful work, but I have a gut-feeling that it  
>> won't
>> solve the "speed" problem all by itself.
>
> Maybe you're the one to ask then.  With all the talk of VMs,  
> everyone seems to make the comparison to Java and Python.  Surely  
> Lisp is a better comparison?  Lisps can be stupid fast (as  
> mentioned in another response to this thread), so what does Lisp do  
> right that Ruby doesn't?

Lisp has a tiny syntax.
Lisp has uncountable years of development.
Lisp had lots of people who wanted it to go fast, so it goes fast.

Ruby has a huge syntax.
Comparably, Ruby has a thimbleful of years of development.
Ruby has lots of people complaining about it being slow but only a
handful of people doing something about it.  (No, I don't consider
myself someone who is doing something about it.)

Its harder to make ruby go fast because its a bigger thing to
optimize and there are less people willing to attempt it it.  Look at
Squeak for inspiration, though.

Mostly you need enough people with enough brains and enough desire to
make ruby go faster.  (But really, you don't need the brains, they'll
grow as you go.)

--
Eric Hodel - drbrain@segment7.net - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

On 22 May 2006, at 18:10, Madan Manoharan wrote:
[snip]
> (1) Performance: Here, the question is NOT about 'how fast should Ruby
> be', but is about what people who have the capacity to make
> 'significant' corporate decisions are 'being told' about Ruby's
> performance by consultants. I have seen presentations by
> 'well-established' consulting companies that show Ruby's performance
> to be an order of magnitude slower than Java for web apps.

I don't thing Ruby is an order of magnitude slower than Java in real
world usage for web applications. It certainly doesn't match my
experiences. Slower maybe, but not by an order of magnitude.

If somebody said that to me I'd want to see more detail on the
architecture, benchmark process, etc.

[snip]
> I don't have time to develop benchmark tests comparing the two  
> languages, but have to depend on Ruby app. developers to produce  
> those benchmarks; unfortunately I cannot find them easily. Now, Zed  
> (I would consider him an authority on Ruby) comes out and talks  
> about performance issues with Ruby (and YARV will solve it in the  
> future). How am I now to convince my manager that Ruby is  
> 'better' (within the context of performance)?

You can't without more work I'm afraid. Because "performance" without
context is meaningless.

Remember when Java was introduced?

"A virtual machine? Garbage collection? All this useless academic
object-oriented nonsense introducing tons of wasteful indirection.
Good grief it runs an order of magnitude slower than my finely
crafted C code!"

Guess what. Most Java is /still/ significantly slower than than
finely crafted C code.

Of course nobody cares :-)

Nobody cares because for a significant number of domains:
* you can develop applications of better quality more quickly and
easily in Java
* raw language speed doesn't matter because it's masked by issues
like blocking for IO
* hardware is cheaper than programmers

And for those domains where raw speed is still an issue - there is
still C.

> The reason why IT managers want 'performance' even if the application
> *really* doesn't require it, is to cover their behinds, among other
> thing.
[snip]
> That is the reason I said that many IT managers make
> 'performance' decisions based on 'feelings', not necessarily had data.
[snip]

Yeah - people who don't make decisions based on reality are a pain.
Your options are:
* ignore / work around them
* get them relocated :-)
* wait until everybody else is in the far end of the adoption curve

Because waiting until everybody else proves that a decision is "safe"
is the only way to convince some people. Sorry :-(

[snip]
> But when in an large corporation, switching between
> languages to get performance gains will result in maintenance
> nightmares. Many IT managers understand this and have experienced
> this.
[snip]

Yet many other IT managers have experience of multi-language
applications that work jolly well and are have been maintained
successfully for years. I know I have. Odd that...

> A good case study would be on how
> Linux got its foot hold into the corporate world.
[snip]

In exactly the same way Ruby is. By doing some things better and
cheaper than the alternatives. By there being people who spotted this
and choosing the better alternative despite the wisdom passed down by
"the consultants". By people seeing it working and going "hey - that
could well work for me".   By this group getting slowly larger over
time until larger organisations can't ignore it and start adopting
Linux themselves.

> Once again, if Ruby's vision is *not* to penetrate the corporate
> environment as Ryan pointed out (but would be a nice validation if it
> indeed penetrate the corporate world by *some* means), then this
> discussion is moot. Knowing that, I wouldn't try to push Ruby to my
> managers; I will use it for personal projects.

Suit yourself :-)

Personally my vision is to do the best job I possibly can for my own
self respect, and to make the company money (and me some cash too -
indirectly). That means I'm going to be picking the most effective
solutions I can find. This means I am going to be looking hard at
Ruby for use in the places we are using Perl and Java now.

In fact, now I think of it, please don't push Ruby too hard. You
might be a competitor :-)

Cheers,

Adrian

Zed Shaw wrote:
> Hey Peter, thanks for deliberately ignoring my whining.  Now, if you'll
> excuse me I have to go and continue to security audit the C extensions
> in Mongrel so that I can be sure there's no buffer overflows.
>
> Man, if only Ruby were faster.  Then I wouldn't have to write so much
> stuff in C and could just worry about crafting beautiful code.  Oh well,
> back to work on my C .... I mean Ruby ... program.
>   
Just so that you don't go to bed all grumpy I will just make it clear
that my post was in no way a dig at your coding skills or belittling you
contribution to the Ruby community. However to say that the Ruby
community is ignoring performance is simply not true. For what you are
doing it clearly is an issue, for others it is not. Perhaps if I had
used the word 'bitching' rather than 'whining' it might have read
better. Who knows, is 'bitching' less negative? More masculine? God 
knows.

As to converting things to C. I've had to convert my graphics tools from
Ruby to C to get the performance I required. But coding them in Ruby
allowed me to play about with various algorithms
to find the best before committing my time to crafting the C code. I
never expected Ruby (or anything but C) to have been fast enough to
process more than 400,000 images on the kit that I had. Same with the
database tools. I do not see this as a problem with Ruby. It would have
been really nice, in terms of writing code, to be able to do everything
in Ruby but there is a trade off. As you move away from assembler you
start trade finely honed code for faster development. Even C is a trade 
off.

My experience has been that Ruby programs run just fine as long as their
working sets are relatively small. (In the most important measure of
performance, the subjective one, they are "fast enough.") As they have
larger and larger sets of data to work on, they get slower and slower, 
at an
increasing rate. With programs that have large working sets (notice, I
didn't say "large programs" - this has nothing to do with line count), a
point comes at which Ruby isn't just too slow- it's too slow by a vast
amount. I don't think this has anything to do with GC, but it may have 
to do
with all the runtime hashing that Ruby has to do in order to be as 
dynamic
as it is.

If I'm right about this, then you might be able to address it in some 
cases
by partitioning working sets across multiple processes or even multiple
process spaces.

On 5/23/06, Robert Klemme <shortcutter@googlemail.com> wrote:
> optimization and instrumentation. Current JVMs' instrumentation
> interfaces give you a lot of options in analyzing runtime behavior of
> your application.

And how well will the Java app run across (list taken from
http://gcc.gnu.org/install/specific.html) :
    * alpha*-*-*
    * alpha*-dec-osf*
    * alphaev5-cray-unicosmk*
    * arc-*-elf
    * arm-*-elf arm-*-coff arm-*-aout
    * xscale-*-*
    * avr
    * Blackfin
    * c4x
    * DOS
    * *-*-freebsd*
    * h8300-hms
    * hppa*-hp-hpux*
    * hppa*-hp-hpux10
    * hppa*-hp-hpux11
    * *-*-linux-gnu
    * i?86-*-linux*aout
    * i?86-*-linux*
    * i?86-*-sco3.2v5*
    * i?86-*-solaris2.10
    * i?86-*-udk
    * ia64-*-linux
    * ia64-*-hpux*
    * *-ibm-aix*
    * iq2000-*-elf
    * m32c-*-elf
    * m32r-*-elf
    * m6811-elf
    * m6812-elf
    * m68k-hp-hpux
    * mips-*-*
    * mips-sgi-irix5
    * mips-sgi-irix6
    * powerpc*-*-* powerpc-*-sysv4
    * powerpc-*-darwin*
    * powerpc-*-elf powerpc-*-sysv4
    * powerpc*-*-linux-gnu*
    * powerpc-*-netbsd*
    * powerpc-*-eabisim
    * powerpc-*-eabi
    * powerpcle-*-elf powerpcle-*-sysv4
    * powerpcle-*-eabisim
    * powerpcle-*-eabi
    * s390-*-linux*
    * s390x-*-linux*
    * s390x-ibm-tpf*
    * *-*-solaris2*
    * sparc-sun-solaris2*
    * sparc-sun-solaris2.7
    * sparc-*-linux*
    * sparc64-*-solaris2*
    * sparcv9-*-solaris2*
    * *-*-sysv*
    * vax-dec-ultrix
    * *-*-vxworks*
    * x86_64-*-* amd64-*-*
    * xtensa-*-elf
    * xtensa-*-linux*
    * Microsoft Windows
    * OS/2
Note that this list of install notes is not a list of supported hosts
or targets. Not all supported hosts and targets are listed here, only
the ones that require host-specific or target-specific information
are.

Ara did say speed *and* portability ;)

2006/5/23, pat eyler <pat.eyler@gmail.com>:
> > lot of smartness with regard to memory handling etc., runtime
>     * arm-*-elf arm-*-coff arm-*-aout
>     * *-*-linux-gnu
>     * m32r-*-elf
>     * powerpc-*-netbsd*
>     * sparc-sun-solaris2.7
>     * OS/2
> Note that this list of install notes is not a list of supported hosts
> or targets. Not all supported hosts and targets are listed here, only
> the ones that require host-specific or target-specific information
> are.

Impressive list.

> Ara did say speed *and* portability ;)

You said gcc will beat on speed and portability. And I was arguing
that it does not necessarily beat Java on speed.  My own Java
portability experience extends to various versions of Windows, Linux
(x86) and Solaris (sparc).  No problems so far but of course the list
is smaller than the one you showed off with. :-)

Kind regards

robert

> Lisp has a tiny syntax.
> Lisp has uncountable years of development.
> Lisp had lots of people who wanted it to go fast, so it goes fast.
...
> Ruby has lots of people complaining about it being slow but only a
> handful of people doing something about it.

Lisp does not have uncountable years of development. unless you're
speaking in terms of man-years, which would indeed be very difficult
to calculate, Lisp has 48 years of development (not counting
brainstorming or experimentation prior to its original specification
in 1958). I can count to 48 and I'm sure this ability is not unique to
me.

however, I think your point is actually a very good point. if the
things I've read have been true, compiled Lisp has performance
competitive with C and superior to Java. it started out slow and
became fast because people wanted it to do that. if people want Ruby
to do the same thing, the work is there for them to do.

I'm still skeptical about the whole question, though. if **other**
things I've read have also been true, you can get much better
performance improvements from putting your database in RAM than you
can from any language change. in fact, don't quote me on this, but my
recollection is that the difference was measured in orders of
magnitude.

On Monday 22 May 2006 05:45 pm, Sam Roberts wrote:
>
> Yes, performance was dominated by string comparisons, I wrote the
> benchmark to see if changing to symbols would help.

How long is your program (lines, for example)?  Is it small enough to 
either
post to the list or make available to those who might be interested.

It would be interesting to see what others might suggest to improve the
performance of your code.  (For example, maybe there are ways to use 
fewer
string comparisons?)

If your program is (to pick an arbitrary number) 60 lines or less, post 
it.
(And you're willing to ;-)  If longer, post with the length of the
program--maybe some people will be interested enough to request a copy 
to
look at (with an eye to optimizing it).

Randy Kramer

Eric Hodel wrote:
> Lisp has a tiny syntax.
> Lisp has uncountable years of development.
> Lisp had lots of people who wanted it to go fast, so it goes fast.

The Lisp compiler can optimize in ways that the ruby interpreter cannot.
(Optional type declarations, for example. Also, list manipulations are
not method calls.)

On 5/22/06, Alder Green <alder.green@gmail.com> wrote:
> > up, Ruby doesn't win (doesn't matter if the application requires it or
> language and offered an improved development experience.
> Exactly.
> pretty smart for that. And for those 99%, being smart means they would
> have been using Java. If they were even smarter, they'd be using Ruby
> :P

A lot of people write a lot of software that don't run on desktop
machines.  A lot of people write a lot of realtime or embedded
software.    In those application domains, C++ works pretty well
(maybe the best?).

(although, I'm not sure why you'd use C++ for anything not embedded or
non-realtime.  That's my experience, anyways.)

On May 23, 2006, at 1:52 PM, Joel VanderWerf wrote:

> -- 
>       vjoel : Joel VanderWerf : path berkeley edu : 510 665 3407
>

How about we rewrite the ruby runtime as a combination of macros and
functions in Lisp, use ParseTree to get the S-Exp representation of a
ruby program and then feed that into our Lisp compiler? Instant
speedy ruby! ;)

> How about we rewrite the ruby runtime as a combination of macros and
> functions in Lisp, use ParseTree to get the S-Exp representation of a
> ruby program and then feed that into our Lisp compiler? Instant
> speedy ruby! ;)

I don't know what you're saying but it sounds like the work of the 
devil.

Go for it.

2006/5/24, Logan Capaldo <logancapaldo@gmail.com>:
>
> How about we rewrite the ruby runtime as a combination of macros and
> functions in Lisp, use ParseTree to get the S-Exp representation of a
> ruby program and then feed that into our Lisp compiler? Instant
> speedy ruby! ;)
>

With an install of ParseTree:

$ cat example.rb
#!/usr/local/bin/ruby
def sum(a, b)
  return a+b
end

a = 3
b = 4
c = pythag(a, b)
puts c

$ cat lispify.rb
#!/usr/local/bin/ruby

def lispify(code)
  code.gsub!(/[:,]/, '')
  code.gsub!(/\[/, '(')
  code.gsub!(/\]/, ')')
end

code = $stdin.read

puts lispify(code)

$ cat example.rb | parse_tree_show -f | ./lispify.rb
((defn
  sum
  (scope
   (block
    (args a b)
    (return (call (lvar a) + (array (lvar b)))))))
 (lasgn a (lit 3))
 (lasgn b (lit 4))
 (lasgn c (fcall pythag (array (lvar a) (lvar b))))
 (fcall puts (array (lvar c))))

So, it looks like Lisp to me! Now, some macros?

Douglas

On May 23, 2006, at 10:24 PM, Douglas Livingstone wrote:

> $ cat example.rb
> $ cat lispify.rb
> puts lispify(code)
> (lasgn c (fcall pythag (array (lvar a) (lvar b))))
> (fcall puts (array (lvar c))))
>
> So, it looks like Lisp to me! Now, some macros?
>
> Douglas
>

What scares (excites?) me is that this looks almost close enough to
work already.

Douglas Livingstone wrote:
> 
> 
> $ cat lispify.rb
> #!/usr/local/bin/ruby
> 
> def lispify(code)
>  code.gsub!(/[:,]/, '')
>  code.gsub!(/\[/, '(')
>  code.gsub!(/\]/, ')')
> end

Topher's article[0] at RC&S might be of interest, too.


[0] http://www.artima.com/rubycs/articles/patterns_sexp_dsls.html

--
James Britt

"People want simple stories."

ara.t.howard@noaa.gov wrote:
> i think making a fast, protable, vm is
> a lions job.  however, i'd be quite happy to be wrong, it just seems
> un-likely.  consider how long has parrot been 'almost', for example...
Well ... I guess my point of view is that Forth is the ultimate "virtual
machine", and that the good folks who created GForth exploited a feature
in GCC that's outside the ANSI C standard to create a "fast, portable,
virtual machine." In passing, I found it interesting that at least one
of the Ruby VMs posted on the link a couple of messages back uses the
same "virtual machine generation" concept that GForth uses.

So ... does Parrot use this? I'm pretty sure YARV doesn't. Meanwhile,
have a look at

https://www.cs.tcd.ie/David.Gregg/papers/ertl-europar01.pdf

--
M. Edward (Ed) Borasky

http://linuxcapacityplanning.com

On May 23, 2006, at 11:15 PM, James Britt wrote:

> Topher's article[0] at RC&S might be of interest, too.
>
>
> [0] http://www.artima.com/rubycs/articles/patterns_sexp_dsls.html
>

I was actually reading this earlier today, which brought it to mind.

2006/5/24, James Britt <james_b@neurogami.com>:
> Topher's article[0] at RC&S might be of interest, too.
>
>

gemp install sexp

Nice link. Doing the hack job solution, taking parse_tree_show and
appending sexp to it, gives:

$ diff parse_tree_show parse_tree_lisp
44,48c44,48
< unless defined? $q then
<   pp result
< else
<   p result
< end
---
> class Object; def to_sexp; inspect(); end; end
> class Symbol; def to_sexp; id2name(); end; end
> class Array;  def to_sexp; "(#{map{|x| x.to_sexp }.join(' ')})"; end; end
>
> puts result.to_sexp

Resulting in the probably-more-reliable-than-gsub result of:

((defn sum (scope (block (args a b) (return (call (lvar a) + (array
(lvar b))))))) (lasgn a (lit 3)) (lasgn b (lit 4)) (lasgn c (fcall sum
(array (lvar a) (lvar b)))) (fcall puts (array (lvar c))))

Douglas

On 5/23/06, Douglas Livingstone <rampant@gmail.com> wrote:
> ((defn sum (scope (block (args a b) (return (call (lvar a) + (array
> (lvar b))))))) (lasgn a (lit 3)) (lasgn b (lit 4)) (lasgn c (fcall sum
> (array (lvar a) (lvar b)))) (fcall puts (array (lvar c))))

Odd, the output I get for lispifying:

  $ cat example.rb
  def sum(a, b)
    a + b
  end

  a = 3
  b = 4
  c = sum(a, b)
  puts c

is:

  ((class Example Object (defn example (scope (block (args) (defn sum
  (scope (block (args a b) (call (lvar a) + (array (lvar b)))))) (lasgn 
a
  (lit 3)) (lasgn b (lit 4)) (lasgn c (fcall sum (array (lvar a) (lvar
  b)))) (fcall puts (array (lvar c))))))))

The interior (starting from "(defn sum...") is the same as you've got
(minus the return statement), but I've got a lot of extra stuff
wrapped around it. I've double checked the my sexp and parsetree gems
are the latest (sexp-0.1, ParseTree-1.4.1).

Jacob Fugal

2006/5/24, Jacob Fugal <lukfugl@gmail.com>:
>
I think you've deleted an extra line, try putting this in after your
result=ParseTree:

result = result[0][3][2][1][2..-1]

Or it might just be how you're running the script, I'm doing:

$ cat example.rb | ./parse.rb -f

hth,
Douglas

On May 23, 2006, at 7:24 PM, Douglas Livingstone wrote:

> (fcall puts (array (lvar c))))
It'll look better if you run it through Rewriter first.  Rewriter
removes a few of the AST optimizations ruby makes in exchange for
more-readable sexps.

--
Eric Hodel - drbrain@segment7.net - http://blog.segment7.net
This implementation is HODEL-HASH-9600 compliant

http://trackmap.robotcoop.com

On 5/24/06, Douglas Livingstone <rampant@gmail.com> wrote:
> 2006/5/24, Jacob Fugal <lukfugl@gmail.com>:
> > The interior (starting from "(defn sum...") is the same as you've got
> > (minus the return statement), but I've got a lot of extra stuff
> > wrapped around it. I've double checked the my sexp and parsetree gems
> > are the latest (sexp-0.1, ParseTree-1.4.1).
>
> I think you've deleted an extra line, try putting this in after your
> result=ParseTree:
>
> result = result[0][3][2][1][2..-1]

Odd. Adding that line does indeed "fix" it (make the output match),
but my original copy of parse_tree_show (copied right out of the gem
directory) didn't have that line to start with. I don't care much, it
just struck me as interesting.

Jacob Fugal

"Douglas Livingstone" <rampant@gmail.com> writes:

>> How about we rewrite the ruby runtime as a combination of macros and
>> functions in Lisp, use ParseTree to get the S-Exp representation of a
>> ruby program and then feed that into our Lisp compiler? Instant
>> speedy ruby! ;)

>
> So, it looks like Lisp to me! Now, some macros?

Now try to port the Ruby object system to CL so it will have
reasonable runtime performance.

Good luck.

On May 24, 2006, at 5:08 PM, Christian Neukirchen wrote:

>>  (scope
> Now try to port the Ruby object system to CL so it will have
> reasonable runtime performance.
>
> Good luck.
>

I'm pretty sure CLOS is a super-set of the ruby object system. Only
thing I'm not sure of is Mixins and singleton methods, but I don't
imagine Mixins to be too hard to add if necessary. One could just
define a module, and then when one included it in a class just have a
macro go down the list of methods in the Module and define them for
the class (basically #append_features). Singleton methods are harder,
but one could probably do what is done in ruby, (construct a class on
the fly that inherits from an objects real class and change the
objects class to this new class.). We already get handling all the
inheritance stuff for free (and arrays and strings, numbers etc.).

Christian Neukirchen wrote:
>>   (block
> reasonable runtime performance.
> 
> Good luck.

Yup. I'm afraid all this lovely syntactical work is not going to get
past the basic facts that (a) method calls are pervasive in ruby and (b)
we depend on (a) for all of our favorite tricks. It's hard to optimize a
language in which so much meaning is deferred until the moment of 
execution.

What's special about ruby (and languages that share the smalltalk
heritage) is that this deferment of meaning (aka late binding) is
enforced so pervasively that it becomes possible to reuse libraries in
ways that were not originally intended (duck typing). Duck typing in CL
is probably much harder, because library writers may make more type
decisions in the interest of "efficiency" (but that's just a wild guess
from someone who hasn't used Lisp seriously in 10 years, so I'll pipe
down about that now). A good example in ruby is the #to_s method invoked
by the #{ } construct. Another example is #each.

On Thu, May 25, 2006 at 06:29:39AM +0900, Logan Capaldo wrote:
> inheritance stuff for free (and arrays and strings, numbers etc.).
> 

From what I remember CLOS has distinct class and method hierarchies,
and multi-dispatch of methods on all parameters.  You can specialize
functions depending on almost anything, not just class of the first
parameter, which is the implicit receiver in message based OO systems.

So, for singleton methods, it should suffice to specialize a generic
function on the first paramter beeing some specific instance.

CLOS classes do have a fixed and predeclared number of attributes
called slots, and Ruby's dynamic attributes would have to be assigned
to a slot holding a list/array/hash of attributes.

Alas, my Lisp times are gone, short as they were in academics. But
it's itching.

Jürgen

I don't have ParseTree installed on my (dooky) work machine, I think
I'll try this at home tho.

But I have a question, apologies if it's obvious -- is that a
superficial result or a meaningful result? It looks as if the magic
missing link here is ParseTree. Can you just drop that output into a
Lisp interpreter, badda bing badda boom?

> From what I remember CLOS has distinct class and method hierarchies,
> and multi-dispatch of methods on all parameters.  You can specialize
> functions depending on almost anything, not just class of the first
> parameter, which is the implicit receiver in message based OO systems.

I believe this is correct, and if I recall correctly it means that
CLOS is a superset of all normal OO systems. But I could be wrong.

On May 25, 2006, at 10:04 AM, Giles Bowkett wrote:

> But I have a question, apologies if it's obvious -- is that a
> superficial result or a meaningful result? It looks as if the magic
> missing link here is ParseTree. Can you just drop that output into a
> Lisp interpreter, badda bing badda boom?

Absolutely not. The sexp generated by ParseTree is ruby specific and
has nothing to do with lisp.

--
_why: zenspider's most intense moments of solice are immediately
following the slaughter [...]
_why: that topknot's the only thing keeping a lid on the righteous anger
bricolage: yeah, that and his flagrant obsession with dvorak

On 22 May 2006, at 17:18, cremes.devlist@mac.com wrote:

>> one! :) ) for not focussing on Ruby's performance - it's  
>> thanks,
>> and jolly expressive. I've no reason (at the moment) to require more
>> performance from it. When I switch back to doing graphics again,  
>> I'll be
>> generating my c, c++ or assembly with Ruby anyway, rather than hand
>> coding it, and it'll probably be fast enough for that too.
>
> Not to single you out (though I guess I am) but you are the first  
> one in the thread to write-in with "it's good enough for me because  
> of <insert reasons having nothing to do with computation speed here>."

:) Woah! Now hang on there.

I was giving a response to:
	"Are there valid excuses for not focusing on improving Ruby's poor
computational performance?"

And to paraphrase myself, the answer is:
	Yes

However, as I said, I'll be as happy as the next about any
performance improvements.

> > But I have a question, apologies if it's obvious -- is that a
> > superficial result or a meaningful result? It looks as if the magic
> > missing link here is ParseTree. Can you just drop that output into a
> > Lisp interpreter, badda bing badda boom?
>
> Absolutely not. The sexp generated by ParseTree is ruby specific and
> has nothing to do with lisp.

so they didn't add "puts" to Lisp while I wasn't looking then. well,
that's a bummer.

coming back to a brief moment of sanity, it'd probably be more useful
to write a Ruby VM in Lisp rather than a Lisp interpreter in Ruby
anyway. I mean if we're still talking performance here.

ruby -> lisp (for those who have not seen the news-group):

http://groups.google.com/group/comp.lang.ruby/browse_thread/thread/5d04fe293afcbcb9/771972335aae65cc?q=lisp&rnum=19#771972335aae65cc

2006/5/25, Giles Bowkett <gilesb@gmail.com>:
> I don't have ParseTree installed on my (dooky) work machine, I think
> I'll try this at home tho.
>
> But I have a question, apologies if it's obvious -- is that a
> superficial result or a meaningful result? It looks as if the magic
> missing link here is ParseTree. Can you just drop that output into a
> Lisp interpreter, badda bing badda boom?
>

You can drop it into Lisp and it'll say, "that's a nice list, but what
am I supposed to do with it?"  What needs done it to write a set of
macros to expand the Ruby ParseTree output into Lisp code which does
much the same thing. It would be my first Lisp project, but with
people shouting about how easy Lisp syntax is to manipulate, and how
it is a superset of everything already, it shouldn't be impossible to
do.

I guess the first thing to try and work out, is, which Lisp? I don't
know them well enough to tell them apart, so my pick would be GNU
CLisp which I happen to have lying around anyway. Good choice?

Douglas

Unfortunately I'm a total Lisp n00b, but what I have on my machine @
home is a variant of CLISP. I don't think it's Gnu, but it is Common
Lisp, and as far as I can tell that is the most popular variant
around.

That thread in comp.lang.ruby is insane! In a good way.

On May 25, 2006, at 6:21 PM, Douglas Livingstone wrote:

> I guess the first thing to try and work out, is, which Lisp? I don't
> know them well enough to tell them apart, so my pick would be GNU
> CLisp which I happen to have lying around anyway. Good choice?

Yes, it is a good choice I think. CLisp is very fast at 'compiling'
new code so development is quick, but it does not have great
execution speed characteristics (so any benchmarks you do in CLisp
will be easily blown away by other implementations). Unless you get
into some dark corners of CL you can easily move to a faster lisp
later. The CLisp community is big and helpful (not to say the others
aren't). Development environment might be a bit of a shock, you are
going to be moving right into the middle of the emacs world -- so you
are looking at emacs and SLIME as your best option (if you are
already using vi/vim then you can get along but you'll not have SLIME
(there's a project underway to get SLIME and vim working together)).

Cheers,
Bob

----
Bob Hutchison                  -- blogs at <http://www.recursive.ca/
hutch/>
Recursive Design Inc.          -- <http://www.recursive.ca/>
Raconteur                      -- <http://www.raconteur.info/>
xampl for Ruby                 -- <http://rubyforge.org/projects/xampl/>

Logan Capaldo <logancapaldo@gmail.com> writes:

> macro go down the list of methods in the Module and define them for
> the class (basically #append_features). Singleton methods are harder,
> but one could probably do what is done in ruby, (construct a class on
> the fly that inherits from an objects real class and change the
> objects class to this new class.). We already get handling all the
> inheritance stuff for free (and arrays and strings, numbers etc.).

How do you handle method_missing?

On May 27, 2006, at 1:08 PM, Christian Neukirchen wrote:

>> imagine Mixins to be too hard to add if necessary. One could just
>>>> Douglas
> --  
> Christian Neukirchen  <chneukirchen@gmail.com>  http:// 
> chneukirchen.org
>

Custom condition handler.

On May 22, 2006, at 9:50, James Britt wrote:

> Kirk Haines wrote:

> Or it's the hamster in the living room.  People don't comment on it 
> because, for the most part, they don't see it.
>
> Those people having speed issues: Is it not a reasonable option to 
> replace the bottleneck Ruby code with custom compiled C libraries?

In a word, "no." Not for me.

I'm a hobbyist programmer, not a professional. I program in order to
accomplish something else, or to save time, and I don't spend more
than, oh, 4-5 hours a week, on average, programming. I cannot *imagine*
what I would *ever* program that would justify learning a whole new
language, when I haven't even finished learning this one yet. (I know,
one never 'finishes' learning a language. But I still have to look up
"Array" and "Enumerate" methods, and probably always will, because I
haven't memorized which one does what yet.)

On the other hand, it's also amusingly irrelevant for me. I have
replaced some of my older code with newer Ruby versions in order to
make them *faster.* Ruby's way faster than AppleScript, which is what
most of my older programs are written in.

On the third hand, I hope the next time this thread appears, nobody
will waste time advancing the nearly nonsensical argument that you can
always speed up ruby by replacing parts with custom compiled C
libraries. Or, to put it another way "You can make Ruby faster by . . .
not using Ruby."

That might make my *program* run faster, but it didn't do diddly-squat
for making *Ruby* faster. And a lot of this thread was occupied by
people who were distracted by this right answer to the wrong question.
Of course IT departments looking (rightly or wrongly) at the question
"How fast is Ruby?" will find this answer inadequate. Of course Zed is
underwhelmed by the joy of having to cross-code more of Mongrel from
Ruby to C. Of course non-native gems are generally less popular than
native ones. None of these 'solutions' make *Ruby* one iota faster.

If I use a non-Ruby library, I might be able to make my program go
faster, but I'm also incorporating into my program a chunk of code I
can't rewrite, or even read. It also makes my program grossly less
portable. I can easily transport a 100% Ruby program to another system,
and even another platform, as long as they've got the right version of
Ruby installed.

Alas, the one program I currently need to move to other systems is
using a non-Ruby database interface. Fortunately, I only have to move
it to other binary-compatible systems, so I can embed the library in my
code. For a binary-incompatible system, I'm completely screwed, since I
cannot reasonably expect an end-user to install Ruby, Rubygems, this
particular gem, AND PostgreSQL on their desktop merely to use this cute
little program. The program requires the db driver, compiling the
driver requires pglib, and getting pglib involves installing/compiling
Postgres. (Or stealing it off some other binary-compatible system,
although maybe I just got lucky when I did that. {shrug})

So I encourage people to be mindful that speeding up a _program_ that's
written, in whole or in part, with Ruby is not the same as speeding up
Ruby itself. It's great to have that option, and sometimes it's the
optimal solution, but it's not the same thing.

1. YARV/Rite/whatever appears to be an impending scheme that will speed
up Ruby itself. Continuing to refine and improve core Ruby code
(whether that core code itself is written in Ruby or C or whatever) is
another avenue to speed up Ruby.

2. Effective, easy to use profilers/benchmarkers/analyzers, good/better
documentation for same, and/or more/better signs pointing to such tools
will help programmers make their own Ruby code go faster. Even if
Ruby-the-language didn't get any faster, if the average program, by the
average programmer, is more likely to have been optimized (for speed or
size or whatever that programmer cares about), that's nearly the same
thing: native Ruby code running faster than it used to.

3. Touting or enhancing Ruby's low-friction interfaces to other
languages might make programs that incorporate some amount of Ruby run
faster, but it also is effectively an admission of guilt. "Yes, Ruby
really is as slow as you think it is; my advice is to use something
else if speed matters to you." That's fine if/when Ruby really is too
slow, but it looks to me (e.g. the counter-example of Rails) that
there's lots of 'slow Ruby' that could be faster if the code were
slightly rewritten. This seems to be especially true when the specific
issue is not "Ruby is too slow to do X" but rather "Ruby is slower than
[put name of high-level language, not 'C', here]"

Based on this thread, and other messages I've read, Ruby's speed
'issue' sure doesn't sound like an elephant to me. I think it's a cat.
For me, Ruby is soft, cuddly, and friendly, and more than rewards the
occasional scratches, hair balls, and litterbox scooping. But there are
some people with really nice furniture, allergies, and/or a small
apartment, and if more practical/useful profilers, or YARV, or whatever
can deodorize and automatically scoop the catbox, train Ruby not to
scratch the furniture, and cut down on the shedding, then I'm sure
they'll be thrilled.

Dave Howell wrote:
> accomplish something else, or to save time, and I don't spend more than, 
> oh, 4-5 hours a week, on average, programming. I cannot *imagine* what I 
> would *ever* program that would justify learning a whole new language, 
> when I haven't even finished learning this one yet. (I know, one never 
> 'finishes' learning a language. But I still have to look up "Array" and 
> "Enumerate" methods, and probably always will, because I haven't 
> memorized which one does what yet.)
> 

I would posit, then, that the discussion of speed here is probably not
the same for you as it is some others in the thread.


> On the other hand, it's also amusingly irrelevant for me. I have 
> replaced some of my older code with newer Ruby versions in order to make 
> them *faster.* Ruby's way faster than AppleScript, which is what most of 
> my older programs are written in.
> 
> On the third hand, I hope the next time this thread appears, nobody will 
> waste time advancing the nearly nonsensical argument that you can always 
> speed up ruby by replacing parts with custom compiled C libraries. Or, 
> to put it another way "You can make Ruby faster by . . . not using Ruby."
> 

Interesting.  You assert this to be a nearly nonsensical argument, yet
you describe yourself as hobbyist programmer, and say that, for you, the
speed concerns are amusingly irrelevant.

> That might make my *program* run faster, but it didn't do diddly-squat 
> for making *Ruby* faster. 

Do you use the YAML parser in Ruby?  Or the String class?  They're C.
Ruby and C are old friends.  If you really want to know Ruby, you need
to know at least some amount of C.    One of the great virtues of Ruby
is the way it hooks into C code.  Suggesting that augmenting a Ruby
program with a C-based library is somehow un-Rubyish strikes me as,
well, nearly nonsensical.

Most of what I've seen in this thread are concerns that, at certain
times, for certain programs, certain people have certain speed issues.

I don't think anyone said that Ruby itself was intrinsically slow for
every task, so for most of the people concerned with speed (people whose
concerns tend to arise from professional or academic circumstances),
suggesting the use of C sounds more than a little reasonable.  It's
pragmatic and professional.



--
James Britt

"The use of anthropomorphic terminology when dealing with
computing systems is a symptom of professional immaturity."

  - Edsger W. Dijkstra

On Jun 1, 2006, at 14:59, James Britt wrote:

> Interesting.  You assert this to be a nearly nonsensical argument, yet 
> you describe yourself as hobbyist programmer, and say that, for you, 
> the speed concerns are amusingly irrelevant.

That's correct. I'm sure it's an error on my part to assume that you're
implying that the fact I don't find Ruby too slow for my own needs
somehow makes me incapable of logical thought.

>
>> That might make my *program* run faster, but it didn't do 
>> diddly-squat for making *Ruby* faster.
>
> Do you use the YAML parser in Ruby?

No.

> Or the String class?

Yes.

>  They're C.

I find that assertion to be quite ludicrous. A C string is a
null-terminated byte sequence. I've happily never had to use to any
great extent any other language that didn't effectively insulate me
from any need to track the terminal null (the vast majority don't even
*use* that, but rather store the length as an integer at the front of
the string), or that allowed my code to inadvertantly overwrite the
terminal null, thus turning a string into an object of indefinite
length, sucking down whatever random bytes lay beyond the end of the
actual string bytes until it found some other "00x0" in memory.

The String class is not only nothing like a C string, it's not a string
at all, in the sense that any/all other languages I've met use the
term. It's an Object, and that makes it act in ways fundamentally
different than the strings I've met in SQL, AppleScript, Modula-2,
Pascal, or Basic (various dialects, all with line numbers), to name a
few.

> Ruby and C are old friends.  If you really want to know Ruby, you need 
> to know at least some amount of C.

You're going to have to provide some rather compelling evidence before
I give that opinion much credence. Ruby and Perl are "old friends,"
too. Nevertheless, if there are aspects of Ruby that came from these
languages, and I cannot learn them directly and exclusively via Ruby,
that just says to me that the Ruby documentation still needs work. When
I decided that AppleScript (XCode and AppleScript Studio
notwithstanding) were inherently inadequate for the programming I
wanted to do, I selected Ruby in part because of how much NOT like Perl
it is.

Or perhaps I *already* know enough C. I spent part of a summer learning
C, and the most important thing I learned was how much I disliked the
decisions the language designers had made.

>    One of the great virtues of Ruby is the way it hooks into C code.  
> Suggesting that augmenting a Ruby program with a C-based library is 
> somehow un-Rubyish strikes me as, well, nearly nonsensical.

"Un-Rubyish?" I don't believe I said any such thing. I was quite
specific. Claiming you can make Ruby faster by not using it, is
nonsensical. Claiming you can make a *program* written in Ruby by
replacing some part of it with another language is perfectly
reasonable. Confusing the two is sloppy reasoning.

For the record, although it's irrelevant to my position, I agree that
Ruby's ability to hook into C code is a strength of the language.

> I don't think anyone said that Ruby itself was intrinsically slow for 
> every task,

Let's see. What was that quote that started this thread? Oh, yes. "I?ll
be honest right away though and say that Ruby is slow."

Note the lack of qualifiers.

Happily, most people (including, as stated earlier, myself) find Ruby's
speed, however it might compare to alternatives, more than adequate.

>  so for most of the people concerned with speed (people whose concerns 
> tend to arise from professional or academic circumstances), suggesting 
> the use of C sounds more than a little reasonable.  It's pragmatic and 
> professional.

I quite agree. Nor have I disagreed with this in any way. I will point
out, superfluously I hope, that the merits of extending Ruby's
abilities, whether for speed or for some other reason, are quite
unrelated to my precis, as summarized in the subject line.

On 6/2/06, Dave Howell <groups@grandfenwick.net> wrote:
> I find that assertion to be quite ludicrous. A C string is a
> at all, in the sense that any/all other languages I've met use the
> term. It's an Object, and that makes it act in ways fundamentally
> different than the strings I've met in SQL, AppleScript, Modula-2,
> Pascal, or Basic (various dialects, all with line numbers), to name a
> few.

They're written in C. Or, rather, in the standard implementation of
Ruby, they're written in C. The statement makes perfect sense. Whether
a String is represented as an Object or a primitive or a green baboon
is irrelevant.

> > Ruby and C are old friends.  If you really want to know Ruby, you need
> > to know at least some amount of C.
>
> You're going to have to provide some rather compelling evidence before
> I give that opinion much credence. Ruby and Perl are "old friends,"
> too. Nevertheless, if there are aspects of Ruby that came from these
> languages, and I cannot learn them directly and exclusively via Ruby,
> that just says to me that the Ruby documentation still needs work.

You don't need to know C to know Ruby. But to know how the standard C
implementation, the one you can download from ruby-lang.org, works,
then you're gonna have to know some C.

Ruby is a computer language. In theory, it is indenpendent from the
implementation, and that's an important distinction. There's JRuby,
for example. Bt realistically, most people use the C Ruby
implementation, and you can't ignore that.

On Jun 1, 2006, at 6:56 PM, Dave Howell wrote:

> I find that assertion to be quite ludicrous. A C string is a null- 
> terminated byte sequence. I've happily never had to use to any  
> great extent any other language that didn't effectively insulate me  
> from any need to track the terminal null (the vast majority don't  
> even *use* that, but rather store the length as an integer at the  
> front of the string), or that allowed my code to inadvertantly  
> overwrite the terminal null, thus turning a string into an object  
> of indefinite length, sucking down whatever random bytes lay beyond  
> the end of the actual string bytes until it found some other "00x0"  
> in memory.

Um, he was saying how they are _implemented_ in C.


> You're going to have to provide some rather compelling evidence  
> before I give that opinion much credence. Ruby and Perl are "old  
> friends," too. Nevertheless, if there are aspects of Ruby that came  
> from these languages, and I cannot learn them directly and  
> exclusively via Ruby, that just says to me that the Ruby  
> documentation still needs work. When I decided that AppleScript  
> (XCode and AppleScript Studio notwithstanding) were inherently  
> inadequate for the programming I wanted to do, I selected Ruby in  
> part because of how much NOT like Perl it is.
>

He's not saying ruby was influenced by C (like it was by perl) he is
saying C is the platform upon which ruby is built.

I think Dave's point is that it's irrelevant what Ruby itself is
implemented in.  The problem is that whenever performance dicussions
come up, instead of discussing how to imrpove the speed of
Ruby-the-language, people instead talk about how easy it is to use
languages other than Ruby for programs that are supposed to be fast.

Dave wants to talk about how to make fast programs that are completely
written in Ruby.  And I do too, though I have little to contribute
except to perhaps bring the discussion back in line.  Maybe we could
talk about improving the speed of the Ruby implementation, or about
improving our use of the language.  But can we have a discussion where
we pretend for a moment that we don't all want to run out and
implement things in C?

-RYaN

On Fri, Jun 02, 2006 at 07:56:10 +0900, Dave Howell wrote:

> 
>> They're C.
> 
> I find that assertion to be quite ludicrous. A C string is a 
> null-terminated byte sequence. 

..snip..

> The String class is not only nothing like a C string, it's not a string at
> all, in the sense that any/all other languages I've met use the term.

This is the point you should just let this one go I think. Ruby is 
written
in C and the String object wraps one of these (as defined in ruby.h):

    struct RString {
        struct RBasic basic;
        long len;
        char *ptr;
        union {
            long capa;
            VALUE shared;
        } aux;
    };

What this boils down to is that a Ruby String *is* a C string with fancy
clothes on.

Cheers,
Phil

On 6/3/06, Ryan Williams <mr.cruft@gmail.com> wrote:
> improving our use of the language.  But can we have a discussion where
> we pretend for a moment that we don't all want to run out and
> implement things in C?
>
> -RYaN

It's difficult to talk about the implementation of Ruby without
talking about C because the (most common) implementation is in C. The
work Koichi san is doing on YARV is great and I'm really looking
forward to Ruby 2.0 but the answer to many performance problems in
Ruby is still going to be to write C extensions. I'll give the work
I'm doing on Ferret (port of Apache Lucene search library) as an
example. Search can be a performance critical portion of many
applications. Now the pure Ruby version of Ferret is about 100 times
slower than the version with the C extension. Now along comes YARV.
From what I've been hearing there is a 2-4 times speed up for some
applications. That still leaves an order of magnitude difference.

Now I'm not saying that writing C extensions is the answer to all your
Ruby performance woes. Most of the time, a better algorithm will do
the job. A faster implementation will help too. I will say however
that Ruby and C together make a powerful combination and I think C is
well worth learning. Not only will you be able to write C extensions
when you need to, but you'll have a better understanding of the Ruby
internals which will in turn help you write faster Ruby code. And it's
really not that difficult a language to learn.

C is not a dirty word,
Dave

Ryan Williams wrote:
> I think Dave's point is that it's irrelevant what Ruby itself is
> implemented in.  The problem is that whenever performance dicussions
> come up, instead of discussing how to imrpove the speed of
> Ruby-the-language, people instead talk about how easy it is to use
> languages other than Ruby for programs that are supposed to be fast.
> 
> Dave wants to talk about how to make fast programs that are completely
> written in Ruby.  And I do too, though I have little to contribute
> except to perhaps bring the discussion back in line.  Maybe we could
> talk about improving the speed of the Ruby implementation, or about
> improving our use of the language.  But can we have a discussion where
> we pretend for a moment that we don't all want to run out and
> implement things in C?
> 
> -RYaN

I agree with Dave and Ryan.

This discussion should be about how to make "RUBY's" performance faster 
when you are using "RUBY", not C.

I'm very well versed in the C/C++ language and I think C is a great way 
to "extend" the language of Ruby, but C should not be a requirement to 
get great performance from Ruby, this should already be available.

So let's talk about what can be changed about Ruby to make it faster 
out-of-the-box?

I don't know the internals of Ruby (and I shouldn't have to...yet), but 
how about we start with the engine, rubyw.exe, are there some area's 
here that can be improved?


Thanks

On 03/06/06, Phil Jackson <phil@shellarchive.co.uk> wrote:
> On Fri, Jun 02, 2006 at 07:56:10 +0900, Dave Howell wrote:

>         union {
>             long capa;
>             VALUE shared;
>         } aux;
>     };
>
> What this boils down to is that a Ruby String *is* a C string with fancy
> clothes on.

That's the implementation. By that argument a Java XXX is just a C 
struct,
because the JVM is written in C (actually C++, I think).

Rasputin :: Jack of All Trades - Master of Nuns
http://number9.hellooperator.net/

> YARV and Ruby's poor computational performance. Zed said, and I quote:
> 
>> I?ll be honest right away though and say that Ruby is slow. The  
>> Ruby community has been ignoring the huge ?performance? elephant  
>> standing in the room and they need to start talking about it so it  
>> goes away. Elephants hate being talked about. There are a few  
>> efforts to make Ruby faster, but I see a lot less action than is  
>> needed to solve the problem. One solution in the works is a real  
>> virtual machine called Rite (or YARV depending on who you talk to)  
>> which is showing some real promise and seems to be speed  
>> competitive with the fastest Java implementations.

Yeah I have come to the sad conclusion that, at least with the 1.8.x 
world, if speedy execution is your goal you should probably choose 
another scripting language [python+psycho comes to mind].  Especially 
when considered in conjunction with rails.  There I have mentioned the 
elephant :)
That being said, thankfully 95% of web pages and scripts don't really 
care about getting hammered, since they never will be.  But the other 5% 
will suffer until this gets figured out.  And I'm not volunteering. 
Hopefully this will improve in the near future.  Until then back to my 
coding of some poorly performing, elegantly written web pages.
-R

On Mon, Jun 23, 2008 at 7:09 AM, Roger Pack <rogerpack2005@gmail.com> 
wrote:
> -R
> --
> Posted via http://www.ruby-forum.com/.

Agreed, but hasn't this dead horse been beaten enough?

Roger Pack wrote:
>>> competitive with the fastest Java implementations.
> coding of some poorly performing, elegantly written web pages.
> -R

Well, Zed's rant *was* published some time ago -- before the 1.9.0
release, I think. I know very little has changed in the MRI performance
arena, but "Rite" and "YARV" are the same thing as far as I know and
are/is indeed faster at the core than MRI.

As far as Rails is concerned, though, quite a bit of work is available
on the web on hacks for tuning it, things to avoid in your Ruby code,
etc. I suspect there is more that *isn't* publicly available on tuning
Rails.

The way I interpret open source licenses, if you take an open source
toolset, squeeze all the major bottlenecks out of it and put it into
production in a server, there's no requirement for you to release the
source of your hacks as long as you don't attempt to distribute the
resulting binaries. :)

I've speculated a bit on what I think might happen in my blog.

http://ruby-perspectives.blogspot.com/2008/06/ruby-rails-and-life-on-edge-of-chaos.html

On Jun 23, 2008, at 8:24 AM, M. Edward (Ed) Borasky wrote:

>
> The way I interpret open source licenses, if you take an open source  
> toolset, squeeze all the major bottlenecks out of it and put it into  
> production in a server, there's no requirement for you to release  
> the source of your hacks as long as you don't attempt to distribute  
> the resulting binaries. :)

though clearly you would accumulate some mighty bad karma if you'd
done so ;-)

a @ http://codeforpeople.com/

> put it into production in a server [...]
> as long as you don't attempt to distribute the
> resulting binaries. :)

Your server or the server of a client?

ThoML wrote:
>> put it into production in a server [...]
>> as long as you don't attempt to distribute the
>> resulting binaries. :)
> 
> Your server or the server of a client?
> 
> 

Good question. :) I am not an attorney. I'm guessing "your server" is
OK, "a client's server" is not.

On Jun 23, 7:24 am, "M. Edward (Ed) Borasky" <zn...@cesmail.net>
wrote:
> >>> which is showing some real promise and seems to be speed  
> > Hopefully this will improve in the near future.  Until then back to my
> etc. I suspect there is more that *isn't* publicly available on tuning
> Rails.
>
> The way I interpret open source licenses, if you take an open source
> toolset, squeeze all the major bottlenecks out of it and put it into
> production in a server, there's no requirement for you to release the
> source of your hacks as long as you don't attempt to distribute the
> resulting binaries. :)
>
> I've speculated a bit on what I think might happen in my blog.

I could be wrong, but I believe that only applies to the BSD license.
Not the GNU one.

Chad

On Mon, Jun 23, 2008 at 4:57 PM, M. Edward (Ed) Borasky
<znmeb@cesmail.net> wrote:
> Good question. :) I am not an attorney. I'm guessing "your server" is OK, "a
> client's server" is not.
>
>
but he explicitly said keeping the binaries, so it cannot be client's
server, right?
Robert

---
Les mêmes questions qu'on se pose
On part vers où et vers qui
Et comme indice pas grand-chose
Des roses et des orties.
-
Francis Cabrel

On Jun 23, 2008, at 9:42 AM, Robert Dober wrote:

> but he explicitly said keeping the binaries, so it cannot be client's
> server, right?

it depends.  you can easily server the binary such as allowing a
client to run and applet from a webserver.  things would get muddy
then.  it all depends on the exact licenses involved i think.  that
and religion.

a @ http://codeforpeople.com/

James Britt wrote:
> ara.t.howard@noaa.gov wrote:
>> On Tue, 23 May 2006, Bob Hutchison wrote:
>> 
>>> Ruby's focus is on something other than performance.
>> 
>> 
>> exactly.  write poetry in french - scream insults at hockey games in 
>> english.
> 
> Unless they're French-Canadian.
>
Wasn't there a comment on how great French was for cursing in a sequel 
to "The Matrix"?

On Jun 23, 2008, at 12:44 PM, Lloyd Linklater wrote:

> Wasn't there a comment on how great French was for cursing in a sequel
> to "The Matrix"?

afaik they have more 'four letter' words than any language in the
world - there is a great french dictionary dedicated to that which
googling might turn up...

a @ http://codeforpeople.com/

Kyle Schmitt wrote:
> On Mon, Jun 23, 2008 at 7:09 AM, Roger Pack <rogerpack2005@gmail.com> 
> wrote:
>> -R
>> --
>> Posted via http://www.ruby-forum.com/.
> 
> Agreed, but hasn't this dead horse been beaten enough?

I would call them a sadistic necro-equine but that IS beating a dead 
horse.

On Mon, Jun 23, 2008 at 8:51 PM, ara.t.howard <ara.t.howard@gmail.com> 
wrote:
>
> On Jun 23, 2008, at 12:44 PM, Lloyd Linklater wrote:
>
>> Wasn't there a comment on how great French was for cursing in a sequel
>> to "The Matrix"?
>
> afaik they have more 'four letter' words than any language in the world -
> there is a great french dictionary dedicated to that which googling might
> turn up...
you mean more than in Italian, ridicolous, believe me I lived in both
countries ;). But I have heared, honestly, that Hungarian beats all
records.
I could not withstand the French quote for its beauty(1) I will very
soon switch back to English quotes as for respect for the community.

(1) Which lies in the ears of the beholder, but so far I have not yet
found a beholder who disliked it :)
R.

grocery_stocker:

> On Jun 23, 7:24 am, "M. Edward (Ed)
> Borasky" <zn...@cesmail.net> wrote:

>> The way I interpret open source licenses, if you take an open source
>> toolset, squeeze all the major bottlenecks out of it and put it into
>> production in a server, there's no requirement for you to release the
>> source of your hacks as long as you don't attempt to distribute the
>> resulting binaries. :)

> I could be wrong, but I believe that only
> applies to the BSD license. Not the GNU one.

It applies both to BSD and GPL, but not to AGPL – which is basically GPL
plus the clause that the requirement of sharing the code also applies
when the code is distributed over a network, not just as a ‘binary’.

Or, to say it differently: in general, releasing a webapp under the
GPL is a bit pointless, and either BSD or AGPL should be used instead.
(The only case I can think of is the situation when you want a BSD-like
licence, but you incorporate some third-party code that’s GPL-licenced,
so you can’t BSD-licence it.)

-- Shot

On Mon, Jun 23, 2008 at 10:58 PM, grocery_stocker <cdalten@gmail.com> 
wrote:

>
> I could be wrong, but I believe that only applies to the BSD license.
> Not the GNU one.
>

And yes, you're wrong there. It's about not (Shot's made the point about 
the
AGPL, but I havent taken a look at that) punishing others with your code 
;).
If you decide to punish others with the results of your code (binaries), 
you
*have* to distribute (or make available some means of getting the code) 
your
code to them.

http://www.gnu.org/philosophy/free-sw.html: freedom 3 - "The freedom to
improve the program, and release your improvements to the public, so 
that
the whole community benefits (freedom 3). Access to the source code is a
precondition for this." Notice how it says "freedom"? Not "requirement"

-jf

--
In the meantime, here is your PSA:
"It's so hard to write a graphics driver that open-sourcing it would not
help."
-- Andrew Fear, Software Product Manager, NVIDIA Corporation
http://kerneltrap.org/node/7228