JRuby vs Java performance

Hi,

New to JRuby and Java, and wondering about ways to improve performance.
Primarily, building a specific class in java rather than jruby and using
javac + jar to create a package.

I did a quick test with a loop n times with some complicated operation
occurring in the middle, and it seemed java performed a lot better (both
were doing the complicated operation in java, so the only thing
different was that the loop was done by jruby or by java).

However, I’ve moved some other code over and I don’t seem to have gained
much performance improvement. It could be that the operations I have
moved to java are not really actually used very often.

What I wanted to know was this: is coding something in java (and using
javac + jar to package it) and running it in jruby usually faster than
pure (j)ruby, or is jruby coded well so that frequently there’s no
difference?

Hoping to learn what the conventional wisdom on this question is, rather
than doing my own experiments for now.

(and I did do a search on forums to see if this has been answered, but
did not yield any obvious threads)

Thanks!

Na Na -

JRuby does tend to be slower than Java. In a great deal of cases, this
doesn’t matter, because the bottleneck is I/O of some kind (file,
network, etc.), or data base speed, or user interaction. Put
differently, if a user has to wait 10 msec in JRuby rather than 1 msec
in Java, it’s not a problem.

However, there will of course be some cases where saving CPU time is
important. That’s where I would look into implementing that part in
Java.

By the way, you probably know this, but whether you use a jar file, or
just use the javac-generated .class files doesn’t really affect the
runtime speed; it just a matter of packaging convenience.

The fact that JRuby is slower than Java doesn’t at all mean that JRuby
isn’t coded well – quite the contrary, shoehorning a flexible and
dynamic language like Ruby into the JVM is quite a technical feat, in my
opinion. There’s a lot of adapting of Ruby to the JVM that needs to be
done at runtime.

I recommend the book “Using JRuby”
(Search). It was a great help for
me in understanding how to get around in JRuby.

Cheers, and welcome to JRuby,
Keith


Keith R. Bennett

Work Status: Available

Thanks for reply!

Keith B. wrote in post #1092905:

However, there will of course be some cases where saving CPU time is
important. That’s where I would look into implementing that part in
Java.

My program will definitely have periods of cpu-intensive tasks where it
will be important for me to improve those.

By the way, you probably know this, but whether you use a jar file, or
just use the javac-generated .class files doesn’t really affect the
runtime speed; it just a matter of packaging convenience.

I did not know this :slight_smile: I was, in fact, mentioning that I use javac just
so someone could tell me if I was doing something wrong or there are
better ways.

Be careful doing performance benchmarks.

Both JRuby and the underlying JVM will improve the performance of code
as the code gets run repeatedly. Make sure to run several hundred
repetitions of the code you want to measure, before you start measuring.
And, of course, make sure to take your measurements without an
intervening JVM restart.

You may find the Ruby code, running in JRuby, to be faster than your
initial measurements indicate.

  • Bruce

Na Na -

[Forward Reference Note: JRuby/Java Bridge Benchmark code and results at
https://gist.github.com/4580344.]

I should have mentioned, there may be ways you can code your JRuby to be
faster too, so you can minimize the need to use Java. Having more JRuby
and less Java would, in my opinion, be a good thing, for all the reasons
that we choose to code in JRuby.

As one example, when JRuby calls a Java function and passes Ruby objects
to it, JRuby may need to create Java objects for those parameters (e.g.
strings and numbers). If the return value is a Java object, JRuby may
need to create a Ruby object with its value.

Consider this JRuby irb session:

jruby-1.7.2 :011 > 2.class
=> Fixnum
jruby-1.7.2 :001 > java.lang.Math.max(2, 3)
(irb):2 warning: ambiguous Java methods found, using max(long,long)
=> 3
jruby-1.7.2 :002 > x = java.lang.Math.max(2, 3)
=> 3
jruby-1.7.2 :003 > x.class
=> Fixnum # a Ruby object, not the Java object actually returned by
java.lang.Math.max

A lot is happening here to bridge JRuby with Java:

  1. In Ruby, the numeric value 2 would be represented as an instance of
    the Fixnum class. When we pass it to a Java function, JRuby needs to
    create a Java object or primitive whose value is 2.

  2. When JRuby receives the return value from the called function, it
    will create a Ruby Fixnum that corresponds to it, and substitute that.

This can be avoided by avoiding crossing the Ruby/Java bridge, by doing
it all in either one language or the other. Ruby has min max built into
the Enumerable module, so you could do this:

jruby-1.7.2 :014 > [3,4].max
=> 4

but that requires instantiating an array and initializing it with the
values, which consumes memory and CPU. Better, I think, to just define
a function and call it, so that Java isn’t needed at all:

jruby-1.7.2 :015 > def max(a,b)
jruby-1.7.2 :016?> a >= b ? a : b
jruby-1.7.2 :017?> end
=> nil
jruby-1.7.2 :018 > max(7,8)
=> 8

Of course, this is a contrived example, since one wouldn’t normally
consider it necessary to use Java to get a maximum, but it does
illustrate the bridge overhead, and also the fact that using Java can be
slower sometimes because of it.

I just spent some time playing with benchmarking various approaches. If
you’re interested, check out Shows the performance effects of crossing the JRuby <--> Java bridge, and using JRuby's java_method. · GitHub. Among
other things, it shows that even crossing the JRuby/Java bridge, you can
improve performance ten times by using JRuby’s java_method to specify
which overload to use.

In any case, if you have any cases that can be distilled down to a few
lines of code, you could ask here.

Regards,
Keith


Keith R. Bennett

Work Status: Available

Bruce -

[By the way, sorry for the bad link before it’s:
Shows the performance effects of crossing the JRuby <--> Java bridge, and using JRuby's java_method. · GitHub ]

You’re right, per iteration duration decreases, even with very large
iteration counts. I parameterized the iteration count and tested it
with some larger values, and this is what I got:


ruby max_example.rb 100_000 2>/dev/null
Performing 100,000 iterations for tests.
Actual X / Min
Using Java: 1.1070 23.0626
Using Java w/alias: 0.1210 2.5208
Using Ruby inline: 0.0480 1.0000
Using Ruby w/function: 0.0510 1.0625

[0] 15:32 mbp-15 /Users/keithb/work/examples

ruby max_example.rb 2_000_000 2>/dev/null
Performing 2,000,000 iterations for tests.
Actual X / Min
Using Java: 3.2630 6.2390
Using Java w/alias: 2.2610 4.3231
Using Ruby inline: 0.5230 1.0000
Using Ruby w/function: 0.5250 1.0038

[0] 15:33 mbp-15 /Users/keithb/work/examples

ruby max_example.rb 100_000_000 2>/dev/null
Performing 100,000,000 iterations for tests.
Actual X / Min
Using Java: 108.9210 4.3728
Using Java w/alias: 110.6450 4.4420
Using Ruby inline: 24.9090 1.0000
Using Ruby w/function: 25.2000 1.0117


That said, depending on the real world use case, it may or may not be
legitimate to discard the first n repetitions and consider only the
later, faster ones. The significant measure is the total duration for
however many are executed in real world use. Per JVM, this penalty is
incurred only once, so if the code in question will be run several or
many times in one JVM lifetime, then I agree, it’s probably appropriate
to discard the warmup penalty.

The 100 million test is interesting. (I’ve added results on a Linux
machine to the gist.) It shows that at large iteration counts:

  1. surprisingly, using java_method is slightly slower than not using
    it
  2. inline JRuby becomes slightly faster than JRuby calling another JRuby
    function
  3. crossing the JRuby/Java bridge results in roughly 4 times slower
    execution
  • Keith

Keith R. Bennett

Work Status: Available

One other note is that package method chain is most of your test_java
cost:

JMath = java.lang.Math

def java_test2
test { |x,y| JMath.max(x, y) }
end

It does not have the indirection cost that an java_method has even
though it is doing a check to make sure the signature has not changed
from the last call. This runs about 2x faster.

On Sun, Jan 20, 2013 at 3:17 PM, Keith B. [email protected]
wrote:

ruby max_example.rb 100_000 2>/dev/null
Actual X / Min
Using Java w/alias: 110.6450 4.4420
JVM lifetime, then I agree, it’s probably appropriate to discard the warmup

On 01/19/2013 11:18 PM, Na Na wrote:
different was that the loop was done by jruby or by java).
Hoping to learn what the conventional wisdom on this question is, rather
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email


blog: http://blog.enebo.com twitter: tom_enebo
mail: [email protected]

On Mon Jan 21, 2013 at 09:50:09AM -0600, Thomas E Enebo wrote:

A basic consideration when using Java Integration (JI) is minimizing
crossing the boundary between Java and Ruby as much as possible. Even
if Ruby code executes as fast as Java (which it does not), you still
have a bunch of code getting executed going over this boundary to deal
with having two type systems (Ruby and Java) in play. Even if we do
no actual conversion between types we are still checking.

A really old example of poor JI performance was people who were
getting a big Java HashMap back from a Java call and then manipulating
every key/value in the hash on the Ruby side. Every key in
java_hash_map[key] was getting converted from a Ruby String to a Java
String so the Java map could try and find a value. Once it found the
value then the value would need to be checked to see if any type
coercion was required to pass it back into Ruby.

Once the person put the hash processing part into Java it sped up
orders of magnitude (this was a long time ago and I am sure this gap
has gotten smaller).

Sometimes the above example is what you desire and the extra cost is
well worth it. Other times you need to do extra planning (like Keith
was saying) where you should consider whether you want most processing
to be done more in Ruby or more in Java to keep the boundary jumping
cost down. In essence, this is a modelling issue.

-Tom

On Sat, Jan 19, 2013 at 10:18 PM, Na Na [email protected] wrote:

than doing my own experiments for now.
To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email


blog: http://blog.enebo.com twitter: tom_enebo
mail: [email protected]

Tom -

I just added the issue you raised to the tests. Using a constant as you
described was faster for 100,000 iterations but slower for 2,000,000 and
100,000,000 (see test results at bottom of
jruby_java_benchmark_2013-121-1312.rb · GitHub ).

I also tried it with a local variable (jMath = java.lang.Math). That
was faster for the 100,000 and 2,000,000 runs, but slower for
100,000,000 iterations.

I tested on both Mac OS running Java 6 and Linux running Java 7.

Strange, eh?

  • Keith

Keith R. Bennett

Tom -

I see your point.

This performance benchmarking and optimization stuff is pretty
complicated.

I can see that one has to have a pretty good understanding of JVM
internals (certainly better than mine) to do it well. Without that,
intuition and common sense are unreliable.

  • Keith

In this particular case, I knew that Charlie had added an optimization
to our generated invokedynamic bytecode which essentially made
constants free if they do not change. So knowing internals helps for
sure.

Had this been running on Android then the optimization would not work
(no invokedynamic there yet) so then knowing the underlying VM
capabilities is also useful.

That said, I think running on actual environment with actual program
is really the only way to go. As implementers we do look at reported
problems and make something artificial, but it is just to get a handle
on the problem as best we can. People evaluating an impl for
performance is tricky without a real test case.

In the case of this thread I think it was useful that you made a
benchmark to show that there is a fairly obvious cost to crossing the
Ruby<->Java boundary. You also showed that not all paths across that
boundary are the same cost.

-Tom

On Wed, Jan 23, 2013 at 7:24 PM, Keith B. [email protected]
wrote:

On Jan 21, 2013, at 3:17 PM, Thomas E Enebo [email protected] wrote:

Using Java w/alias: 69.7490 3.5120
of. If I ran this on my machine and these microbenches reflected what

I also tried it with a local variable (jMath = java.lang.Math). That was
faster for the 100,000 and 2,000,000 runs, but slower for 100,000,000 iterations.

It does not have the indirection cost that an java_method has even

iteration counts. I parameterized the iteration count and tested it with

ruby max_example.rb 100_000_000 2>/dev/null
legitimate to discard the first n repetitions and consider only the later,

  1. inline JRuby becomes slightly faster than JRuby calling another JRuby

measurements indicate.

javac + jar to package it) and running it in jruby usually faster than


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email


blog: http://blog.enebo.com twitter: tom_enebo
mail: [email protected]

Your example does bring up another excellent point about benchmarking
(and even more so when micro-benchmarking): Individual environments
have various variables which can make a big difference on results. In
JRuby, with the appropriate Java 7 invokedynamic patches constant
access is nearly free. Here are my results for 100_000_000 accesses
(MPB):
Actual X / Min
Using Java: 68.0710 3.4275
Using Java (constant): 31.6100 1.5916
Using Java w/alias: 69.7490 3.5120
Using Ruby inline: 19.8600 1.0000
Using Ruby w/function: 20.2150 1.0179

With Java 6 and I suspect the Java 7 (pre good indy support) you see
some optimization finally happen in Java w/ alias not happening to the
constant accessor. With invokedynamic on and working well we see a
substantial new performance improvement.

I guess the conclusion is: test on the environment you plan on using
and really run something close to what you want good performance out
of. If I ran this on my machine and these microbenches reflected what
I was doing…AND THEN…I deployed on Java 6 I would be pretty
surprised at the drop in performance.

-Tom

On Mon, Jan 21, 2013 at 12:22 PM, Keith B.
[email protected] wrote:

  • Keith
                         Actual     X / Min

Using Java w/alias: 2.2610 4.3231
Using Ruby w/function: 25.2000 1.0117

the code gets run repeatedly. Make sure to run several hundred repetitions
Hi,
However, I’ve moved some other code over and I don’t seem to have gained

http://xircles.codehaus.org/manage_email


http://xircles.codehaus.org/manage_email


blog: http://blog.enebo.com twitter: tom_enebo
mail: [email protected]