Forum: JRuby JRuby performance

36498a2ecff4b366792d7f97ab923cdc?d=identicon&s=25 DJ (Guest)
on 2013-11-13 21:52
(Received via mailing list)
I just found out about Web Framework Benchmark at
http://www.techempower.com/benchmarks/
I found it interesting to find a lot of their JRuby benchmarks performed
worse than MRI one. I read their test methodology, they do allow some
warm
up period to make sure JIT based platform has enough time to kick in.

Any thought why JRuby performs worse, when supposedly better than MRI
one?
2c0c4cf3ccc8da22f7c3b9586ce1cd70?d=identicon&s=25 Christian MICHON (Guest)
on 2013-11-13 22:25
(Received via mailing list)
Had a look at their github repo... Gemfile for jruby contains
jruby-jars...
So it might be jruby running within MRI, not jruby running in pure java.

So far over more than 2 years moving from MRI to jruby, I never found a
case where jruby would perform so badly, and I am on Windows...

Using a fully expanded or installed jruby is faster than jruby-jars or
jruby-complete, especially with lots of gems.
13313ac2ec7ba7c43b1b952db034ff3b?d=identicon&s=25 Thomas E Enebo (Guest)
on 2013-11-13 23:01
(Received via mailing list)
We only perform worse on the rack benchmarks for all others we perform
better.  Perhaps there is something in jruby-rack which is not
performing
well?  I would think if a full Rails app warms up enough to perform
better
than MRI a smaller app would warmup faster?

-Tom
6a22563bd5790a15adc54c8ca772eb22?d=identicon&s=25 Ben Browning (Guest)
on 2013-11-13 23:13
(Received via mailing list)
I've actually been looking at these benchmarks the past week in
prototyping the next generation of TorqueBox. So far I've been looking
mainly at the JSON serialization test and the plaintext test to get a
benchmark.

All the JRuby tests are run on top of Resin. Why Resin was chosen
instead of Puma, Trinidad, TorqueBox / torquebox-lite, or something else
more sane I'm not sure. That's the biggest problem with at least the
JSON and plaintext tests.

Techempower gets 69k req/s on their i7 hardware for the 'rack-ruby' JSON
test that uses Ruby 2.0, Unicorn, and Nginx.

As a baseline, on my local laptop I get 32k req/s for that same
rack-ruby test with the same versions of MRI Ruby, Unicorn, and Nginx.

On that same local laptop with torquebox-lite, I get 46k req/s. A nice
improvement for sure, but not anything amazing.

Still on the same laptop, using some unpushed prototype code that will
replace torquebox-lite, I get 71k req/s. This prototype is based on
JBoss Undertow, which you'll notice is on top or near the top of most of
these benchmarks. Perhaps it's time to put a bow on this server and get
it out there to let people play with?

So, we should definitely work up a pull request to use a better JRuby
server option. And it doesn't have to be TorqueBox by any means - these
are just the simple examples I had locally.

One optimization in all my local JRuby tests is I set the JRuby option
"-Xjruby.ji.objectProxyCache=false" which will be the default in JRuby
9k. At lower throughput levels it doesn't really make a difference but
once you start getting over 50k req/s or so the overhead from using the
object proxy cache really starts to show up.

Ben
6a22563bd5790a15adc54c8ca772eb22?d=identicon&s=25 Ben Browning (Guest)
on 2013-11-13 23:21
(Received via mailing list)
It's likely that the tests aren't CPU-bound. These tiny Rack apps are
basically a test of how much the web server, language, and framework can
get out of the way. And my guess is there's either something in
JRuby-Rack or Resin that is getting in the way - excessive object
allocation, synchronization in the request path, etc.

Ben
8bc8621ef6a6d981a4a41c5069f530cf?d=identicon&s=25 Steven McCraw (Guest)
on 2013-11-14 01:36
(Received via mailing list)
I wonder also what templating system they were using for rendering
responses. When I first started using JRuby, my existing Rails apps were
10
times slower using JRuby 6, and I couldn't understand why, but I finally
narrowed it down to anything using haml being terrible (and I was using
a
lot of haml). Erb templates rendered much faster, but still way slower
than
MRI. JRuby 7 got haml down to only 2 or 3 times slower, and Erb was very
close, but like these benchmarks, still a little slower. This makes
decent
sense. It's really hard to see how Java is going to be faster at
something
like String manipulation than C. But unless an application is really
template/view heavy, I think you'll make it up in other ways.

Sent from my mobile device. Apologies for brevity.

On Nov 13, 2013, at 3:51 PM, DJ <wirianto.djunaidi@gmail.com> wrote:

I just found out about Web Framework Benchmark at
http://www.techempower.com/benchmarks/
I found it interesting to find a lot of their JRuby benchmarks performed
worse than MRI one. I read their test methodology, they do allow some
warm
up period to make sure JIT based platform has enough time to kick in.

Any thought why JRuby performs worse, when supposedly better than MRI
one?
6a22563bd5790a15adc54c8ca772eb22?d=identicon&s=25 Ben Browning (Guest)
on 2013-11-15 04:27
(Received via mailing list)
These particular tests weren't using any templating - just strings
returned directly from Rack applications.
6a22563bd5790a15adc54c8ca772eb22?d=identicon&s=25 Ben Browning (Guest)
on 2013-11-15 04:35
(Received via mailing list)
If anyone else is interested, I've released version 0.1.0 of the
'torqbox' gem, which scores drastically higher on the TechEmpower JSON
and plaintext benchmarks on my machine. I'd like to get a couple more
reports of its performance before submitting a pull request to the
TechEmpower repo to replace its usage of Resin with torqbox. This should
outperform any other JRuby web server, and I'd like to hear reports from
anyone that can find an example showing otherwise.

If you're curious, 'torqbox' is the codename for a future TorqueBox
version. The funky gem name is just so we can cut releases of both
without confusing current 'torquebox' gem users.

Read more at https://github.com/torquebox/torquebox/tree/5x-dev
Caa2df9372ffa0a9e95b2bab1e8fea34?d=identicon&s=25 Karol Bucek (Guest)
on 2013-11-15 13:21
(Received via mailing list)
As for the JRuby-Rack part ... yes there might be some "redundant" (as
Tom
mentioned) work in JRuby-Rack's Request/Response lifecycle.

It was not an issue (at least for now) - it simply outperformed MRI as
much
as I've seen with Trinidad and some of the standalone servers - Warble-d
(including JBoss).
I have started some work on this locally ... e.g. rewriting
JRuby::Rack::Response to "plain" Java, but time is a precious commodity
managing a few OSS projects :)

So we'll see, hopefully some of it gets out this year ... at least in a
1.2.0 pre-release.

The other thing is, I've heard quite a lot of "praise buzz" about Resin
but
it's somehow one of the few servers I never actually seen at work even
through my Java days.

K.
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.