Forum: IronRuby Will the performance catch up be next milestone?

IBM Engineer completed a performance benchmark for rubys, seems ir does
not well done in the performance.


http://programmingzen.com/2010/07/19/the-great-ruby-shootout-july-2010/

Quoting the web site:

*Disclaimer*

Synthetic benchmarks cannot predict how fast your programs will be when
dealing with a particular implementation. They provide an (entertaining)
educated guess, but you shouldn’t draw overly definitive conclusions 
from
them. The values reported here should be assumed to be characteristic of
server-side — and long running — processes; they should be taken with a
grain of salt.

If your primary concern is speed, you should probably look to write your 
app
in C or Assembly.

--
Will Green
http://hotgazpacho.org/

It's probably not intentional but his benchmark graphs are misleading.

Because Mono is not nearly as fast or as mature as Microsoft's .NET, the
performance of IronRuby on mono is much worse. Unfortunately all his 
graphs
show Mono performance only, which makes IronRuby appear very slow.

If you look at the numbers directly (there is a table further down 
comparing
IronRuby on mono vs IronRuby on .net), IronRuby is much much faster. It
appears to me that IronRuby on windows (.NET) is faster than MRI 1.9.2
("regular" ruby) on windows!

It's still not as fast as MRI 1.9.2 on linux, but it's not that far 
behind
either.

Yeah, but who wants to *deploy* Ruby code on Windows? Develop,
sure...but then performance doesn't matter. If IronRuby is aiming only
to be a windows centric technology, I can't imagine what future it
really has in store for it, that is, standing alone on its own two feet
as a Ruby implementation. Integrated into .NET software is a different
story irrelevant to the benchmarks being discussed, but I don't think
the benchmarks are misleading as far as the Ruby community at large is
concerned, because for that group, I don't imagine Windows is a viable
deployment target ( why would it be?) so benchmarking on Linux is
probably the most realistic kind of benchmarking you can do when
comparing ruby interpreters for that particular audience. I suppose that
is one of the things that makes the IronRuby project an enigma to me -
in my mind Ruby is a finger pointing to Linux, so it seems an odd one
for Microsoft to extend.



From: ironruby-core-bounces@rubyforge.org
[mailto:ironruby-core-bounces@rubyforge.org] On Behalf Of Orion Edwards
Sent: Wednesday, July 28, 2010 8:59 PM
To: ironruby-core@rubyforge.org
Subject: Re: [Ironruby-core] Will the performance catch up be next
milestone?



It's probably not intentional but his benchmark graphs are misleading.



Because Mono is not nearly as fast or as mature as Microsoft's .NET, the
performance of IronRuby on mono is much worse. Unfortunately all his
graphs show Mono performance only, which makes IronRuby appear very
slow.



If you look at the numbers directly (there is a table further down
comparing IronRuby on mono vs IronRuby on .net), IronRuby is much much
faster. It appears to me that IronRuby on windows (.NET) is faster than
MRI 1.9.2 ("regular" ruby) on windows!



It's still not as fast as MRI 1.9.2 on linux, but it's not that far
behind either.



On Thu, Jul 29, 2010 at 2:58 PM, Ray Linn <lists@ruby-forum.com> wrote:

IBM Engineer completed a performance benchmark for rubys, seems ir does
not well done in the performance.


http://programmingzen.com/2010/07/19/the-great-ruby-shootout-july-2010/
--
Posted via http://www.ruby-forum.com/.

IronRuby is exciting because it brings Windows, Ruby and .NET together 
as
one.  There are plenty of cases where deploying to Windows would either 
be
ideal, or in some cases, a must.  Imagine developing a Ruby app that 
needs
to integrate with Sharepoint, or Great Plains; how would you go about 
doing
so?  You could implement web services in C++, or perhaps C#... but then 
you
have several languages to deal with and multiple code bases to manage, 
as
opposed to a single Ruby app.

Enter IronRuby.  Now you have the ability to harness the all of the 
Windows
oriented .NET libraries *from* Ruby.  You could use DRb, or perhaps 
RESTful
services using Sinatra, and then consume those services from a Rails 
app.
So, while I agree that I'd rather deploy my main app to a linux box, 
it's
great to know that I can use IronRuby on Windows to seamlessly integrate
with Microsoft solutions.  I would imagine that it's these sorts of
situations that Microsoft is banking on (as well as scripting scenarios 
and
such).

-Charles

On Wed, Jul 28, 2010 at 11:40 PM, Nathan Stults 
<Nathan_Stults@hsihealth.com

I think Ruby on Windows has it's place. There are many companies with
investment in MS servers, Applications and infrastructure so don't
completely discard IronRuby for Windows.
Also there're other scenarios that don't involve Ruby on the server like
Silverlight, standalone WPF apps, Excel Interop, Plugins, Scripting 
Apps.

PS. There's a thread about IronRuby usage on this mailing list. I've not 
had
a chance to read it, but that could give you an idea of the way people 
is
currently using it. I've not had a chance to read it, so it might just 
prove
you're right...

Regards
Miguel



There was recently a thread about adopti

On Thu, Jul 29, 2010 at 2:40 PM, Nathan Stults

This benchmark was done against mono 2.4.4 which is significantly old. I
guess 2.6 shouldn't bring much performance
improvements but the upcoming 2.8 release with the new GC can make huge
differences specially on GC bound
benchmarks such as binary_trees.

On Thu, Jul 29, 2010 at 1:40 AM, Nathan Stults

On 29/07/2010, at 4:40 PM, Nathan Stults wrote:

> Yeah, but who wants to *deploy* Ruby code on Windows?

Personally, I wouldn't run a public facing internet site on windows, but 
that's only because I spent 2 years developing rails apps on FreeBSD for 
another company and picked up a lot of skills from it. Otherwise, I'd 
run a windows server. I also know a lot of people who run windows 
servers. From what I've seen the penetration of linux servers inside 
small-to-medium companies here is approaching 0%.
IronRuby (particularly if it ever gets good userfriendly IIS 
integration) means I can develop a rails app and get the local microsoft 
sysadmin to deploy it in a couple of clicks.

> Develop, sure…but then performance doesn’t matter.

Really? I become rather unhappy if I'm having to dev something where the 
performance sucks (compiling C++ apps really really gets my goat). This 
is the primary reason why I continually wish they'd improve the IronRuby 
startup perf. **scalability** doesn't matter for development, but 
scalability and performance are two different things, and the benchmark 
certainly doesn't measure scalability

> If IronRuby is aiming only to be a windows centric technology, I can’t imagine what future it really has in store for it, that is, standing alone on its own two feet as a Ruby implementation. Integrated into .NET software is a different story irrelevant to the benchmarks being discussed, but I don’t think the benchmarks are misleading as far as the Ruby community at large is concerned, because for that group, I don’t imagine Windows is a viable deployment target ( why would it be?)

Umm... Why wouldn't it be? Windows server pricing is actually not too 
bad, so long as you stay away from the clusterf*** that is SQL server 
(it's great tech but the licensing and pricing are crazy). Again, I've 
seen plenty of windows servers run by company sysadmins and I'd love to 
be able to deploy some rails apps onto them (MRI is worse than terrible 
for deploying to IIS, so again, IronRuby could stand to do really well).

You could argue that the "target audience" for that article was 23 year 
olds creating the next FaceTube because it was posted to Hacker News, 
however Antonio Cangiano is an Evangelist for IBM. I wouldn't equate 
"IBM" with internet startups either... It seemed like the target 
audience was simply  "people who are interested in ruby performance"....

> so benchmarking on Linux is probably the most realistic kind of benchmarking you can do when comparing ruby interpreters for that particular audience. I suppose that is one of the things that makes the IronRuby project an enigma to me – in my mind Ruby is a finger pointing to Linux, so it seems an odd one for Microsoft to extend.

I agree with your point of view, however I draw the opposite conclusion. 
Looking at MRI and JRuby, Linux (or some BSD/solaris/etc) really is the 
go-to platform if you want to develop a ruby application. The 
performance of MRI in particular is clearly a lot better on *nix. This 
is then coupled with the fact that Rails is an outstanding platform for 
getting web apps up and running, and web apps are pretty hot these days.

From that point of view, it makes perfect sense for Microsoft to build 
IronRuby as a top notch Ruby implementation for windows. If they pull 
this off, and couple it with some other windows integration type stuff 
(an IIS admin plugin to administer rails sites would be awesome) then in 
theory two things can happen:

1) People that were familiar with windows but are considering switching 
away because of a ruby/rails app might stay on windows (and then, pay 
microsoft for windows server and maybe even SQL server eventually)

2) If they do a REALLY good job, people running rails apps on linux 
might switch to windows!

I'd love to see #2 happen, but unfortunately it's a long way off. The 
IronRuby team seems to be really awesome, but there's only like 3 of 
you, right? :-(

Oh, and RavenDB, which I have successfully deployed as an Azure worker 
role.

Mark

Sent from my iPhone

When IR achieves a compatibility that will run Rails 3, I'm planning on 
getting it running thru an ASP.NET site which will enable deployment to 
the Azure platform. I believe that will be a very compelling solution 
for many people, especially with a mapper adapter to Table Storage.

Mark

Sent from my iPhone

On Fri, Jul 30, 2010 at 5:27 PM, Orion Edwards <orion.edwards@gmail.com> 
wrote:
> I agree with your point of view, however I draw the opposite conclusion.
> Looking at MRI and JRuby, Linux (or some BSD/solaris/etc) really is the
> go-to platform if you want to develop a ruby application. The performance of
> MRI in particular is clearly a lot better on *nix. This is then coupled with
> the fact that Rails is an outstanding platform for getting web apps up and
> running, and web apps are pretty hot these days.

FWIW, we do a *lot* of work to make sure JRuby behaves properly on
Windows...in many cases more work than even Ruby core can put in.
We've recognized that Ruby needs better representation on Windows, and
until IronRuby can take over we've tried to make JRuby at least a
better option than MRI.

Yes, there's the unfortunate truth that almost none of the JRuby core
devs use Windows, but we are actively trying to avoid Windows being
treated as a second-class platform. We even have a Windows installer,
executable, native library support (including FFI) and maintain all of
it for our Windows users. So while it's probably fair to say *nix
platforms are better supported by JRuby, I think it's also important
to point out that JRuby is currently a better (more consistent, more
stable, faster) option for Ruby on Windows than MRI.

- Charlie

Chiming in a bit late, hoping it will be useful though!

 > Yeah, but who wants to **deploy** Ruby code on Windows?

I do deploy Ruby code on Windows (production), just not for public 
facing
web apps.

For instance, I have MRI 1.8.6 data crunching (datawarehouse) running on
Windows.

I also run JRuby on Windows (see
http://blog.logeek.fr/2010/8/2/on-jruby-resque-and-windows) to interact 
with
APIs that run only on Windows and to generate PDFs (the web front-end is
itself running on Ubuntu).

I have another client who has been able to automate an amazing amount of
work in their company, thanks to relying on Ruby, OLE or other APIs 
(they
work with many tools such as SAS, Excel, Access).

I am for one *very* happy by the work put by both the JRuby team and the
RubyInstaller project to ensure things work properly there (and 
obviously,
by the IronRuby team too :-).

Just to add a few data points!

-- Thibaut

Sorry Charles, I didn't mean to infer that the JRuby team wasn't doing a 
great job on windows (you certainly are - you're also doing a lot better 
job than the core ruby guys for windows as well. Are they still 
compiling with MSVC6?), my point was more about perception.

I'm not sure if this is true internationally, but from what I've seen of 
the industry here and in australia, most development shops here seem to 
fall into 3 categories:

1: Web shops (PHP, etc) which have nothing to do with either Java or 
.NET
2: Microsoft shops (asp.net, windows client apps, deploying on windows 
server)
3: Java shops.

This last lot of Java shops tends to be building software for larger 
companies (banks, etc), and seems to near-universally deploy on a unix 
of some kind. I'm not sure if this is because they think Java itself 
runs better on a non-windows OS, or if they just prefer it. (I've never 
professionally employed as a Java developer so I haven't had any direct 
interaction with these companies)

At any rate, JRuby (at least from a perception/mindshare point) gets 
lumped along with the rest of Java. The perception is that JRuby is 
Java, and Java is better on unix, so therefore you should run JRuby on 
Unix. I don't think this would change even if 100% of the JRuby core 
devs used windows all day long :-)