JRuby on Rails experience

Just wanted to share my first JRuby experience: A few months earlier,
after all the deliberation, we chose JRuby over vanilla Ruby for a heavy
traffic application…JRuby seemed very promising, and so far so good,
I am pleased to say that JRuby has kept all its promises!
Recently, we successfully deployed our first JRuby on Rails application
in production (JRuby 1.5.0, Rails 2.3.5, Apache, Tomcat 6, PostgreSQL
8.4). Initially faced a lot of challanges to scale it for 2000+
concurrent users, but with excellent help from this forum, the JRuby
cookbook and other JRuby sources on internet, we were able to tune the
application and make those 2000+ concurrent users happy.

The most important learning for us has been the jruby.runtimes tuning.
The
configuration that scaled the best was (enable config.threadsafe! in
Rails) + (jruby min and max runtimes = 1). We faced performance issues
with JRuby to Java calls at higher concurrency and had to switch to
ServletFilter for most of the Java calls.

Overall, JRuby is superb!

Our special thanks to entire JRuby development team (Nick, Charles et
al) for giving us this fantastic bridge between Java and Ruby! It just
works!

Looking forward to further improvements in JRuby libraries…

Keep going JRuby team…!

-Darshan.

Thanks for sharing your experience with us! It’s always reassuring to
hear a success story. Let us know if there’s anything you would like to
see improved.

/Nick

On Sat, Feb 19, 2011 at 1:55 AM, Darshan Karandikar
[email protected] wrote:

The most important learning for us has been the jruby.runtimes tuning.
The
configuration that scaled the best was (enable config.threadsafe! in
Rails) + (jruby min and max runtimes = 1). We faced performance issues
with JRuby to Java calls at higher concurrency and had to switch to
ServletFilter for most of the Java calls.

I’m very interested in hearing more specifics about these performance
issues. At what levels of parallel execution did you notice
performance degradation, and what exactly did you observe?

Anthony J. wrote in post #982645:

On Sat, Feb 19, 2011 at 1:55 AM, Darshan Karandikar
[email protected] wrote:

The most important learning for us has been the jruby.runtimes tuning.
The
configuration that scaled the best was (enable config.threadsafe! in
Rails) + (jruby min and max runtimes = 1). We faced performance issues
with JRuby to Java calls at higher concurrency and had to switch to
ServletFilter for most of the Java calls.

I’m very interested in hearing more specifics about these performance
issues. At what levels of parallel execution did you notice
performance degradation, and what exactly did you observe?

Hi Nick, Anthony,

Summarizing our observations during performance testing with 1500
concurrent users into the system, along with :

  • comments on areas we would like to see improved in JRuby
  • details of “JRuby to Java call” performance issue
  1. With more number of jruby runtimes, the number of classes loaded and
    JVM heap memory utilization grows substantially (analyzed using
    jvisualvm). This is expected JRuby 1.5.0 behavior I suppose.
  • We started with 5 jruby min runtimes and max 25. As more users
    entered the system, we noticed considerable stress on CPU and JVM heap
    memory whenever a new runtime had to be created to serve those users
    (new runtime creation was indicated by increase in number of classes
    loaded, noticed using jvisualvm). We concluded that the resources
    required to create a jruby runtime was a bit high and slows down things
    under load. So we configured jruby.runtime min=max=25, so that they get
    created during application deployment itself. This improved response
    time & resources utilization during user ramp up, but slowed down
    application deployment (which was ok for us).
  • With jruby min runtimes = 5 & max = 25, (with jvisualvm) we noticed
    that the classes created for 25 runtimes at peak were not unloaded (&
    the memory utilization didn’t reduce) even after the application usage
    dropped down to a few users. I suppose this is known issue, and higher
    JRuby version might have fixed it already (if yes, would like to know
    which JRuby version to upgrade to for the fix?).
  • Around 25 jruby runtimes (required to keep response time within
    acceptable limit) consumed approx 40-50% of JVM heap memory, and with
    approx around 1000 concurrent users in, the free heap memory reduced to
    a few MBs, thus triggering GC quite often. It is desirable if the heap
    memory consumption by a jruby runtime can be reduced further so that we
    can accomodate more runtimes in less heap space, leaving enough heap
    space for users’ session data.
  • Looking at the foreseen user growth & constraints on h/w we have, we
    decided to reduce resource consumption by jruby runtimes and give those
    resources for application functionality (mainly memory for session
    data). The best option for this was to ensure we have thread-safe code
    in place (= efforts[code review]), then enable config.threadsafe! in
    Rails and set jruby.runtime.min=max=1. We got rid of some Rails plugins
    as we weren’t sure if they are thread-safe or not. This configuration
    worked well and spared much more memory for usage by application logic.
    We would like this to be default JRuby+Rails behavior/configuration
    (like default Servlet behavior in JEE), though we understand this might
    also depend on the way Ruby+Rails handles threads/requests by default.
  1. JRuby to Java calls: The code had a few simple Java calls (create a
    Java object and call a method with some input params) from JRuby in one
    functionality. Everything worked ok till around 1100 concurrent users
    used that functionality, but beyond that concurrency application server
    CPU utilization started increasing non-linearly, and response time
    increased beyond acceptable limit. We started RCA to get to root cause
    by commenting code in that particular functionality one by one, and
    noticed that “without” the JRuby to Java calls, everything worked fine
    even with 1500 concurrent users. Hence, we removed these calls, and
    placed them in Servlet Filter. This change fixed the CPU utilization
    issue under high load. This behavior has led us to conclude that the
    JRuby to Java calls was the root cause of slow response and high CPU
    utilization at high load. Not sure if this observation has anything to
    do with this discussion:
    http://www.mail-archive.com/[email protected]/msg06990.html

  2. After the test, when we stop the Tomcat server, we get following
    errors in the Tomcat logs (not sure the impact of these during heavy
    load, but still would like to see these gone):

SEVERE: A web application created a ThreadLocal with key of type [null]
(value [[email protected]]) and a value of
type [java.lang.ref.SoftReference] (value
[[email protected]]) but failed to remove it when the
web application was stopped. To prevent a memory leak, the ThreadLocal
has been forcibly removed.
xxxx yyyy PM org.apache.catalina.loader.WebappClassLoader
clearThreadLocalMap
SEVERE: A web application created a ThreadLocal with key of type [null]
(value [[email protected]]) and a value of type
[java.lang.ref.WeakReference] (value
[[email protected]]) but failed to remove it when the
web application was stopped. To prevent a memory leak, the ThreadLocal
has been forcibly removed.
xxxx yyyy PM org.apache.catalina.loader.WebappClassLoader
clearThreadLocalMap

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs