Forum: JRuby Min/Max Runtime Configuration

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Ryan H. (Guest)
on 2009-04-17 21:00
(Received via mailing list)
Hello,

We currently have the following configuration in our warbler.rb:

  config.webxml.jruby.min.runtimes = 2  config.webxml.jruby.max.runtimes
= 6

Upon starting Tomcat, we receive the following messages in our localhost
log file:

INFO: Info: using runtime pool timeout of 30 seconds
INFO: Info: received min runtimes = 2
INFO: Info: received max runtimes = 6
INFO: Info: add application to the pool. size now = 1
INFO: Info: add application to the pool. size now = 2


What specifically is it that will cause the pool size to grow and will
we receive any indication that it has grown? I've looked for messages
indicating applications have been added to the pool and run a variety of
tests trying to cause applications to get added to the pool and have
never seen additional applications added or don't know how to check the
current pool size if a message is not printed to the localhost log.

The reason we've been looking into this is we receive the following
stack trace periodically which I am attributing to more traffic than we
are able/configured to handle:

SEVERE: Exception caught
org.jruby.rack.RackInitializationException: timeout: all listeners busy
        at
org.jruby.rack.PoolingRackApplicationFactory.getApplication(PoolingRackApplicationFactory.java:87)
        at
org.jruby.rack.DefaultRackDispatcher.process(DefaultRackDispatcher.java:31)
        at org.jruby.rack.RackFilter.doFilter(RackFilter.java:51)
        at
org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:235)
        at
org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:206)
        at
org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:233)
        at
org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:191)
        at
org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
        at
org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:102)
        at
org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
        at
org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:286)
        at
org.apache.coyote.http11.Http11Processor.process(Http11Processor.java:845)
        at
org.apache.coyote.http11.Http11Protocol$Http11ConnectionHandler.process(Http11Protocol.java:583)
        at
org.apache.tomcat.util.net.JIoEndpoint$Worker.run(JIoEndpoint.java:447)
        at java.lang.Thread.run(Thread.java:637)
Caused by: java.lang.InterruptedException
        ... 15 more


We are using jruby 1.2.0 and jruby-rack 0.9.3.

Thanks
Ryan
AD (Guest)
on 2009-04-17 21:10
(Received via mailing list)
Ryan,

 I went through a bunch of this pain prior to rails thread-safety
(which trust me comes with its own pain).  I recommend using JConsole
to monitor the active threads and see if there is blocking anywhere.
You can also get a thread dump by doing a kill -QUIT on the java pid.
Reading these is by no means easy but may help see if you are blocking
all the runtimes and not releasing them for some reason.

 Are you integrating with any other components, like JMS, Memcache,
long file uploads, external HTTP connections that could be blocking
the thread ?

 Is this for rails?  If so you can look at enabling config.threadsafe!
and setting min/max to 1/1 which auto turns on threading.

Adam

On Fri, Apr 17, 2009 at 12:59 PM, Howe, Ryan 
<removed_email_address@domain.invalid>
wrote:
> INFO: Info: received min runtimes = 2
> pool size if a message is not printed to the localhost log.
> org.jruby.rack.DefaultRackDispatcher.process(DefaultRackDispatcher.java:31)
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
>         at
>
---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Ryan H. (Guest)
on 2009-04-17 21:24
(Received via mailing list)
Yes, this is for rails. We are currently using 2.1.2 and as part of this
I've done some testing getting it running under 2.2.2 and in general
tried some different min/max configurations and also turned on
config.threadsafe! The config.threadsafe! seemed to alleviate this
specific issue but I'm concerned about other possible race condition
type issues as once I turned it on for my tests I had to research
memcache related threadsafe configurations. We are stable under the
current configuration but have been playing with different numbers of
runtimes and I'm interested in knowing if we are indeed growing to 6
runtimes with our current configuration or staying at the minimum with
2.

For the other questions, we are definitely using memcache and making
external HTTP connections in several places. These external connections
are indeed probably the root cause of our problems but at the same time,
the customer requires that we pull this data in to the pages. We use
timeouts but that is another problematic area from the research we've
done.


On 4/17/09 12:09 PM, "AD" <removed_email_address@domain.invalid> wrote:

Ryan,

 I went through a bunch of this pain prior to rails thread-safety
(which trust me comes with its own pain).  I recommend using JConsole
to monitor the active threads and see if there is blocking anywhere.
You can also get a thread dump by doing a kill -QUIT on the java pid.
Reading these is by no means easy but may help see if you are blocking
all the runtimes and not releasing them for some reason.

 Are you integrating with any other components, like JMS, Memcache,
long file uploads, external HTTP connections that could be blocking
the thread ?

 Is this for rails?  If so you can look at enabling config.threadsafe!
and setting min/max to 1/1 which auto turns on threading.

Adam

On Fri, Apr 17, 2009 at 12:59 PM, Howe, Ryan 
<removed_email_address@domain.invalid>
wrote:
> INFO: Info: received min runtimes = 2
> pool size if a message is not printed to the localhost log.
> org.jruby.rack.DefaultRackDispatcher.process(DefaultRackDispatcher.java:31)
> org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:128)
>         at
>
---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
AD (Guest)
on 2009-04-17 22:04
(Received via mailing list)
yea your best bet right now is JConsole to see what the threads are
doing and if they are blocking on something.

I wish i was better at this and could give you better advice, but if
you do find a blocking thread, post it here someone will certainly be
able to help.

Adam

On Fri, Apr 17, 2009 at 1:23 PM, Howe, Ryan 
<removed_email_address@domain.invalid>
wrote:
>
>
>
>>
>> INFO: Info: add application to the pool. size now = 1
>> The reason we’ve been looking into this is we receive the following stack
>> org.jruby.rack.DefaultRackDispatcher.process(DefaultRackDispatcher.java:31)
>>         at
>> org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:109)
>>         at java.lang.Thread.run(Thread.java:637)
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>     http://xircles.codehaus.org/manage_email
>
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Nick S. (Guest)
on 2009-04-18 00:02
(Received via mailing list)
On Fri, Apr 17, 2009 at 11:59 AM, Howe, Ryan 
<removed_email_address@domain.invalid>
wrote:
> INFO: Info: received min runtimes = 2
> pool size if a message is not printed to the localhost log.
>
> The reason we’ve been looking into this is we receive the following stack
> trace periodically which I am attributing to more traffic than we are
> able/configured to handle:

JRuby-Rack should initialize a new runtime on demand if there are none
available when a request comes in, up to the max runtimes. Do you see
messages in the log file like this, indicating you've reached the max?

add application to the pool. size now = 6

/Nick

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Ryan H. (Guest)
on 2009-04-18 00:08
(Received via mailing list)
No, that is just it. I have never seen any message (grepping the log for
the word "pool") other than the entries at initialization adding the
"min" number to the pool.

I've also tried in my development environment to hammer a tomcat
instance using httperf and never successfully gotten a message that the
pool grew.



On 4/17/09 3:01 PM, "Nick S." <removed_email_address@domain.invalid> wrote:

On Fri, Apr 17, 2009 at 11:59 AM, Howe, Ryan 
<removed_email_address@domain.invalid>
wrote:
> INFO: Info: received min runtimes = 2
> pool size if a message is not printed to the localhost log.
>
> The reason we've been looking into this is we receive the following stack
> trace periodically which I am attributing to more traffic than we are
> able/configured to handle:

JRuby-Rack should initialize a new runtime on demand if there are none
available when a request comes in, up to the max runtimes. Do you see
messages in the log file like this, indicating you've reached the max?

add application to the pool. size now = 6

/Nick

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Nick S. (Guest)
on 2009-04-18 00:11
(Received via mailing list)
For your production config, I'd suggest setting min = max. Maybe try
with both set to 6 and see how you handle the requests?

/Nick

On Fri, Apr 17, 2009 at 3:07 PM, Howe, Ryan 
<removed_email_address@domain.invalid>
wrote:
>
>> log
>> receive any indication that it has grown? I’ve looked for messages
> available when a request comes in, up to the max runtimes. Do you see
>
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email
Keshav P. (Guest)
on 2011-03-26 00:58
My configuration in warble.rb is as follows

  config.webxml.jruby.min.runtimes = 2
  config.webxml.jruby.max.runtimes = 4


and under production.rb

#config.threadsafe!


Error :
org.jruby.rack.RackInitializationException: timeout: all listeners busy

I am suddenly getting errors as above  and when I searched through the
logs. This was what I found.

Mar 24, 2011 10:50:46 PM org.apache.catalina.core.ApplicationContext log
INFO: Info: add application to the pool. size now = 2


Any suggestions much appreciated.

Thanks for the help in advance,
Kesh.
Adam D. (Guest)
on 2012-09-04 12:24
Keshav P. wrote in post #989324:
> My configuration in warble.rb is as follows
>
>   config.webxml.jruby.min.runtimes = 2
>   config.webxml.jruby.max.runtimes = 4
>
>
> and under production.rb
>
> #config.threadsafe!
>
>
> Error :
> org.jruby.rack.RackInitializationException: timeout: all listeners busy
>
> I am suddenly getting errors as above  and when I searched through the
> logs. This was what I found.
>
> Mar 24, 2011 10:50:46 PM org.apache.catalina.core.ApplicationContext log
> INFO: Info: add application to the pool. size now = 2
>
>
> Any suggestions much appreciated.
>
> Thanks for the help in advance,
> Kesh.


Did you find an answer to this? I have the same problem.
many thanks,
Adam.
Craig W. (Guest)
on 2012-09-20 17:15
I also ran into this using Rails 3.2.3 on JRuby 1.6.7 (ruby 1.9)
deployed to Tomcat 7.0.x.

I had min=2, max=4 and threadsafe was not enabled (it was actually
commented out). I've changed min/max to equal 1 and I turned threadsafe
on...so far no problems.

Although I'm still wondering what the "proper" number of runtimes is if
there is one.
Jamie Jackson (Guest)
on 2012-09-20 18:36
(Received via mailing list)
Nick S. (Warbler creator) suggests min=max for production.

Also, a while back, I remember having read (though I don't remember
where)
a suggestion to set them to 6, which I did.

  config.webxml.jruby.min.runtimes = 6
  config.webxml.jruby.max.runtimes = 6

I could perform load testing or tuning to determine the perfect
settings,
but I haven't bothered.

Thanks,
Jamie
This topic is locked and can not be replied to.