JRuby invocation through JSR223 aprox 17x faster than BSF

I put together two very simple test scripts for benchmarking running
a short JRuby script 500 times with both BSF and JSR223.

The JSR223 implementation was approximately 17 times faster on
average and the initial invocation was about 9 times faster.

More details on the wiki:

http://wiki.jruby.org/wiki/Java_Integration#Comparing_the_performance_of_JRuby_script_invocation_in_BSF_with_JSR223

In the graphs at these links below you can easily see the periodic
slowdown due to garbage collection:

BSF: http://wiki.jruby.org/wiki/images/5/5c/Jruby_bsf_invocation.png

JSR223:
http://wiki.jruby.org/wiki/images/a/af/Jruby_jsr223_invocation.png

These graphs are on the wikipage as links – they aren’t appearing as
embedded images for some reason ??

I was doing this work trying to track down a memory leak in our use
of jruby through BSF.


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

Hi Stephen,

On Sat, Nov 15, 2008 at 6:52 PM, Stephen B.
[email protected] wrote:

I put together two very simple test scripts for benchmarking running a short
JRuby script 500 times with both BSF and JSR223.

The JSR223 implementation was approximately 17 times faster on average and
the initial invocation was about 9 times faster.

More details on the wiki:

http://wiki.jruby.org/wiki/Java_Integration#Comparing_the_performance_of_JRuby_script_invocation_in_BSF_with_JSR223

Surprisingly, a great differece.
JRuby engine uses Ruby.runNormally method by default, which inovkes
tryCompile method internally, while BSF uses Node.interpret method,
which doesn’t compile scripts. This benchmark proofs that JRuby’s
compiler works effectively. If you turn
com.sun.script.jruby.terminator switch off, JRuby engine will use the
same method as BSF. Consequently, the difference would be very small,
or JRuby engine might be slower than BSF because JRuby engine does
more in every eval invocation to fulfill the API requirements.

-Yoko

I was doing this work trying to track down a memory leak in our use of jruby
through BSF.


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

More details on the wiki:
same method as BSF. Consequently, the difference would be very small,
or JRuby engine might be slower than BSF because JRuby engine does
more in every eval invocation to fulfill the API requirements.

Hi Yoko,

I looked into this difference further and discovered an error in my
analysis.

Inocation using JSR223 is much slower than using BSF which is slower
than using JRuby directly. At least part of this is due to the fact
that using jruby-engine.1.1.6 and creating a new
JRubyScriptEngineManager actually creates two separate JRuby
instances. Running this:

JRubyScriptEngineManager manager = new JRubyScriptEngineManager();

Sepaate JRuby instances are in the manager.nameMap, one keyed to
‘jruby’ and one to ‘ruby’.

This means that invoking a JRuby instance with
JRubyScriptEngineManager takes at least twice a long as it should.

If it is intentional that there are two entries in the map (‘jruby’
and ‘ruby’) could the creation be optimized so that they each point
to the same jruby instance?

I set up these tests because I have a separate program that has a
memory leak when I instantiate about 15 BSF jruby invocations
sequentially (the leak does not yet show up in these tests).

In my test I intended to setup a loop where a ruby instance was
created and evaluated each time through the loop – the GC getting
rid of the previous Ruby instance.

These two statements that were in my loop in the JSR223 test use the
same JRuby instance each time but the manager.getEngineByName
statement does not create a new JRub instance – it just uses the one
that was already created.

ScriptEngine engine = manager.getEngineByName(“jruby”);
engine.eval(“Array.new(16) { rand(256)
}.pack(‘C*’).unpack(‘H*’).first”);

There is nothing wrong of course with using a JRuby instance over and
over but that is not what I was intending to test.

The code for all he following tests is located here:

http://svn.concord.org/svn/projects/trunk/common/java/otrunk/ot-script/ot-jruby/src/test/java/org/concord/otrunk/script/test

Here are the results when testing JSR223 and the creation of a new
manager using JRubyScriptEngineManager is inside the loop

JRuby JSR223 invocation test
Iterations: 500
Total Elapsed Time (ms): 179205
Initial Invocation (ms): 1813
Average Invocation (ms): 355.4
Minimum Invocation (ms): 119
Maximum Invocation (ms): 1143

Here the comparable data for BSF:

JRuby BSF invocation test
Iterations: 500
Total Elapsed Time (ms): 61665
Initial Invocation (ms): 1718
Average Invocation (ms): 120.0
Minimum Invocation (ms): 33
Maximum Invocation (ms): 1603

And the equivalent data for direct JRuby invocation:

JRuby Direct Embedding invocation test
Iterations: 500
Total Elapsed Time (ms): 41343
Initial Invocation (ms): 1717
Average Invocation (ms): 79.3
Minimum Invocation (ms): 53
Maximum Invocation (ms): 199

FYI: It would be much easier for me to send you a patch if the
jruby-engine code was in a newer SCM repository (svn, git, hg) than
in CVS.


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

Stephen B. wrote:

FYI: It would be much easier for me to send you a patch if the
jruby-engine code was in a newer SCM repository (svn, git, hg) than in CVS.

FYI, we would really, really like to move the 223 engine code directly
into JRuby proper and just start shipping it. It would allow us to make
changes more easily and would make it easier to ensure the engine stays
in sync with JRuby proper. Is this possible? What’s the licensing
considerations? Would we need to reimplement our own to annex it?

  • Charlie

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

Hi Charles,

On Mon, Nov 17, 2008 at 7:10 AM, Charles Oliver N.
[email protected] wrote:

Stephen B. wrote:

FYI: It would be much easier for me to send you a patch if the
jruby-engine code was in a newer SCM repository (svn, git, hg) than in CVS.

FYI, we would really, really like to move the 223 engine code directly
into JRuby proper and just start shipping it. It would allow us to make
changes more easily and would make it easier to ensure the engine stays in
sync with JRuby proper. Is this possible? What’s the licensing
considerations? Would we need to reimplement our own to annex it?

It would be great for all JRuby engine users!

However, I’m not a person to answer the question about licensing, I’ll
post this to scripting API’s ML.
I think reimplementation doesn’t have any obstacle since some of
implementations are not released from scripting.dev.java.net so far.
Besides, I think JRuby engine should have refactoring by using JRuby’s
API more and more to improve its performance and for easy maintenance.
It might be another implementation.

-Yoko

  • Charlie

To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email