JRuby benchmarks way slow... something smells wrong here


#1

http://www.cs.purdue.edu/homes/sbarakat/cs456/Scripting.pdf

JRuby’s performance is abysmal in these slides. I’m no expert (nay, not
even a well informed amateur), but something about this whole
presentation, with respect to jruby anyway, smells way off.

Thoughts?


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

#2

Michael C. wrote:

http://www.cs.purdue.edu/homes/sbarakat/cs456/Scripting.pdf

JRuby’s performance is abysmal in these slides. I’m no expert (nay,
not even a well informed amateur), but something about this whole
presentation, with respect to jruby anyway, smells way off.

Thoughts?

Argh, maybe this is because it’s from 2007? Damn, sorry about the
false alarm. =(


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

#3

Well aside from that, there’s also the fact that it isn’t entirely
clear what he’s doing to benchmark besides
measuring the time it takes to do ‘hello world’.

JRuby can generally be beaten by MRI if whatever you’re doing isn’t
taking very long. In these cases
the speed difference is negligible to most users.

Where JRuby excels is in longer running processes. If you saw the post
I made yesterday about running JRuby,
JRuby --server & MRI on a set of scripts I use to process some
bioinformatics data, there is significant
speed up when I use JRuby --server. If you let the JVM warm up,
especially with --server, you get some
really nice performance gains.

Either way, I’m most bothered by the fact that that guy never really
clearly said how he was benchmarking
them but that he was just benchmarking them. Pretty useless as the
only useful benchmarks are testing
real performance in real applications :).

-Zac

On Apr 15, 2009, at 8:16 AM, Michael C. wrote:

false alarm. =(


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

#4

Zachary Brown said…

JRuby --server & MRI on a set of scripts I use to process some
bioinformatics data, there is significant
speed up when I use JRuby --server. If you let the JVM warm up,
especially with --server, you get some
really nice performance gains.

Either way, I’m most bothered by the fact that that guy never really
clearly said how he was benchmarking
them but that he was just benchmarking them. Pretty useless as the
only useful benchmarks are testing
real performance in real applications :).

I enjoy running jruby from within vim. e.g. to reformat xml

:% !jruby ~/xmlformat.rb

jruby --server seems like overkill here!

I expect Charlie along any minute with some speed-up tips :slight_smile:


Cheers,
Marc


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

#5

On Apr 15, 2009, at 9:48 AM, marc wrote:

post
only useful benchmarks are testing
real performance in real applications :).

I enjoy running jruby from within vim. e.g. to reformat xml

:% !jruby ~/xmlformat.rb

jruby --server seems like overkill here!

I expect Charlie along any minute with some speed-up tips :slight_smile:

Well of course --server is overkill there, my point was more that
JRuby these days
isn’t significantly slower than MRI. However, its real shining area is
–server
when you’re doing longer running processes. JRuby’s performance is now
to the point
where I can run it faster than any other scripting language I’ve tried
for
biological models. Its still not as fast as Java or Haskell but its
fast enough that
for smaller models, I can enjoy the flexibility of the language.
(We’re talking
days of running time for certain simulations). In MRI… well the
results would be
obsolete by the time I got them :slight_smile:

-Zac


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

#6

talking
days of running time for certain simulations). In MRI… well the results
would be
obsolete by the time I got them :slight_smile:

Scuse me for my ignorance, I’m new on jruby world.
What is jruby --server?


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

#7

On Apr 15, 2009, at 11:16 AM, Mauro wrote:

tried for

Scuse me for my ignorance, I’m new on jruby world.
What is jruby --server?

If you pass the ‘–server’ flag to JRuby, it causes JRuby to invoke
the JVM
with -server. This is an option only available on Mac, Linux or
Solaris and
is includes higher optimization and (?)inlining done on the code. I
forget
the specifics but it will noticeably improve performance in both java
use
and JRuby. Its not as important to use on short lived scripts and such,
but if you have some serious crunching being done, --server is generally
a good idea.

Its best to look up the HotSpot VM and find the specifics as I’ve really
glossed over the details on it. I do know for a fact it significantly
improves
performance of any type of multi-threaded code as I write a lot of
multi-threaded
java.

-Zac


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

#8

I enjoy running jruby from within vim. e.g. to reformat xml

:% !jruby ~/xmlformat.rb

jruby --server seems like overkill here!

I expect Charlie along any minute with some speed-up tips :slight_smile:

http://kenai.com/projects/jruby/pages/Benchmarks

It’s got some info on benchmarking tips, as well as a list of valid
and invalid benchmarks.


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

#9

On Apr 15, 2009, at 8:16 AM, Mauro wrote:

tried for

Scuse me for my ignorance, I’m new on jruby world.
What is jruby --server?

jruby --server is the same as passing --server to java. Under Sun
VM’s it uses the “server” hotspot vs the “client” VM. The server VM
trades startup time for more optimization. The client VM starts up
faster but does much less optimization. JRuby benefits greatly from
all the optz the server VM can provide so outside of small scripts,
rake tasks, etc., it’s generally suggested to run using --server.

David K.


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

#10

Charles Oliver N. wrote:

Argh, maybe this is because it’s from 2007? Damn, sorry about the
false alarm. =(

Oops, I already emailed the guy to see if I can help him improve those
numbers…before I realized it’s an old presentation. Well, I guess it
can’t hurt anything, and I know that presentation is getting around a
bit.

My fault; I should have paid more attention initially. Sorry 'bout
that.


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

#11

2009/4/15 David K. removed_email_address@domain.invalid:

jruby --server is the same as passing --server to java. Under Sun VM’s it
uses the “server” hotspot vs the “client” VM. The server VM trades startup
time for more optimization. The client VM starts up faster but does much
less optimization. JRuby benefits greatly from all the optz the server VM
can provide so outside of small scripts, rake tasks, etc., it’s generally
suggested to run using --server.

Thank you for your answers.
I have an application written in rails and using jruby.
Then I create the war with warbler and deploy under a servlet container.
When I must pass the --server?


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

#12

Mauro wrote:

Then I create the war with warbler and deploy under a servlet container.
When I must pass the --server?

Usually the servlet container will run under --server by default, but
you’ll need to check the container’s startup params for it. I know GF by
default does not start in server mode, for example.

  • Charlie

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

#13

Michael C. wrote:

false alarm. =(
Oops, I already emailed the guy to see if I can help him improve those
numbers…before I realized it’s an old presentation. Well, I guess it
can’t hurt anything, and I know that presentation is getting around a
bit.

  • Charlie

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

#14

This might help: http://docs.sun.com/app/docs/doc/820-4343/abeib?a=view

Ikai L. wrote:

jruby --server is the same as passing --server to java. Under Sun VM’s it
you’ll need to check the container’s startup params for it. I know GF by


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

#15

Wow, wish I knew that. Does anyone have a link to instructions for
running
Glassfish in server mode?

Ikai

On 4/15/09 12:59 PM, “Charles Oliver N.” removed_email_address@domain.invalid
wrote:

I have an application written in rails and using jruby.
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

#16

Yup … Nuts, our prod applications have the -client flag. We’ll have to
swap this around our next maintenance downtime.

Ikai

On 4/15/09 3:19 PM, “Charles Oliver N.” removed_email_address@domain.invalid
wrote:

I have an application written in rails and using jruby.


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

#17

This is actually a surprisingly common problem: we run without --server
so that developers get faster startup times, and then forget to tell
people to turn that off when they want top performance. I wonder if the
act of spinning up a Jruby runtime takes long enough to justify --server
automatically? Probably not, or we would already do that.

Anyway, to get Glassfish to run in server mode, open up your
glassfish/domains/{domain name}/config/domain.xml and find the
section. Somewhere in there there should be a line like
-client, which you should change to
-server. That’s assuming that you are
launching Glassfish using asadmin. If you’ve been running from the
command line (java -jar glassfish.jar), you may already be running in
server mode. Since 1.5, the JVM tries to figure out if it’s running on a
“server-class” machine and select the appropriate option. Details of
that can be found at
http://java.sun.com/j2se/1.5.0/docs/guide/vm/server-class.html

Ikai L. wrote:

I have an application written in rails and using jruby.
To unsubscribe from this list, please visit:
http://xircles.codehaus.org/manage_email


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email