Jruby and Tomcat Redeployment

Hi all, I’m running a site using Jruby on Rails and Tomcat 5.5. I
deploy to
the site using warbler to war the project. This is all working great,
with
one exception: when deploying new versions, old code does not get
unloaded
by Tomcat. So, when I place a new war in the webapps directory, Tomcat
detects it and explodes the war properly (or so it seems), however, if
you
visit the site all the old code appears to still be cached. Has anyone
seen
this and/or know of a way to force Tomcat to unload and reload
everything so
I don’t have to restart Tomcat to get it to pick up the changes? Thanks
in
advance!

tony

On Tue, Dec 8, 2009 at 11:57 AM, Tony C. [email protected]
wrote:

Hi all, I’m running a site using Jruby on Rails and Tomcat 5.5.

A little skimpy on real information – platform? actual versions?

And exactly how are you redeploying? Are there any errors in the
logs relating to the redeployment?


Hassan S. ------------------------ [email protected]
twitter: @hassan


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email

Hassan,

Thanks for the reply. Wasn’t sure if anyone had ever hit something like
this so was testing the waters out first.

Here’s some more info: Tomcat 5.5.17 running on redhat enterprise 5.4.
This
is JRuby 1.3.1 and Rails 2.3.5 (happened also on 2.3.4). I’m deploying
the
war by dropping it into tomcat’s webapps directory, copying it over the
previous version. Tomcat detects the new war and starts to unpackage
it.
It deploys the code as any files in the exploded directory are updated
(e.g.
the css gets updated and new css files are served).

I’ve copied the log output during a redeploy below. Some things to
note:
First, I’m using a gem called Rufus Scheduler to do a simple nightly
background task. Which is where the shutdown exception comes from. The
redeployment problem has been around longer than I’ve been using that
gem,
but I’ll pull it out tomorrow and see if I can get a log without that
exception in it. Second, the exclamation points (one per line) appear
in
the log for hundreds and hundreds of lines after I’ve cut the log off.
Those seem to appear on almost every request and I’m not sure where they
are
actually coming from as I don’t log to catalina.out from my application.

Thanks for the look! Log below, let me know if I there’s more I can
provide.

tony

Dec 9, 2009 5:27:13 AM org.apache.catalina.startup.HostConfig
checkResources
INFO: Undeploying context [/gsrails]
Exception in thread “Thread-23” java.lang.NullPointerException
at
org.joda.time.DateTimeUtils.currentTimeMillis(DateTimeUtils.java:53)
at org.joda.time.base.BaseDateTime.(BaseDateTime.java:73)
at org.joda.time.DateTime.(DateTime.java:96)
at org.jruby.RubyTime$1.allocate(RubyTime.java:170)
at org.jruby.RubyClass.allocate(RubyClass.java:129)
at org.jruby.RubyTime.newInstance(RubyTime.java:626)
at
org.jruby.RubyTime$s_method_0_0$RUBYINVOKER$newInstance.call(org/jruby/RubyTime$s_method_0_0$RUBYINVOKER$newInstance.gen)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:75)
at
ruby.jit.ruby.opt.apache_minus_tomcat_minus_5_dot_5_dot_17.webapps.gsrails.WEB_minus_INF.gems.gems.rufus_minus_scheduler_minus_2_dot_0_dot_1.lib.rufus.sc.scheduler.cron_step15497230_3751317.file(scheduler.rb:274)
at
org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:111)
at
org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:138)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:75)
at
ruby.jit.ruby.opt.apache_minus_tomcat_minus_5_dot_5_dot_17.webapps.gsrails.WEB_minus_INF.gems.gems.rufus_minus_scheduler_minus_2_dot_0_dot_1.lib.rufus.sc.scheduler.step7113595_3751317.file(scheduler.rb:266)
at
org.jruby.internal.runtime.methods.JittedMethod.call(JittedMethod.java:111)
at
org.jruby.internal.runtime.methods.DefaultMethod.call(DefaultMethod.java:138)
at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:75)
at org.jruby.ast.CallNoArgNode.interpret(CallNoArgNode.java:61)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at org.jruby.ast.BlockNode.interpret(BlockNode.java:71)
at
org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:218)
at
org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:169)
at org.jruby.runtime.Block.yield(Block.java:194)
at org.jruby.RubyKernel.loop(RubyKernel.java:1147)
at
org.jruby.RubyKernel$s_method_0_0$RUBYFRAMEDINVOKER$loop.call(org/jruby/RubyKernel$s_method_0_0$RUBYFRAMEDINVOKER$loop.gen)
at
org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:268)
at
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:86)
at
org.jruby.runtime.callsite.CachingCallSite.callIter(CachingCallSite.java:101)
at
org.jruby.ast.FCallNoArgBlockNode.interpret(FCallNoArgBlockNode.java:32)
at org.jruby.ast.NewlineNode.interpret(NewlineNode.java:104)
at
org.jruby.runtime.InterpretedBlock.evalBlockBody(InterpretedBlock.java:218)
at
org.jruby.runtime.InterpretedBlock.yield(InterpretedBlock.java:206)
at org.jruby.runtime.BlockBody.call(BlockBody.java:72)
at org.jruby.runtime.BlockBody.call(BlockBody.java:78)
at org.jruby.runtime.Block.call(Block.java:89)
at org.jruby.RubyProc.call(RubyProc.java:221)
at org.jruby.RubyProc.call(RubyProc.java:204)
at org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:92)
at java.lang.Thread.run(Thread.java:595)
Dec 9, 2009 5:27:15 AM org.apache.catalina.startup.HostConfig deployWAR
INFO: Deploying web application archive gsrails.war
JRuby limited openssl loaded. gem install jruby-openssl for full
support.
http://wiki.jruby.org/wiki/JRuby_Builtin_OpenSSL
!
!
!
!
!
!
!

On Tue, Dec 8, 2009 at 5:55 PM, Hassan S.
<[email protected]

On Wed, Dec 9, 2009 at 8:10 AM, Hassan S.
<[email protected]

wrote:

I vaguely recall some apparent caching problems with Tomcat, but IIRC
it involved accidentally starting a second instance, which wasn’t actually
listening to anything because the first had the port(s) already. In any
case…

Hmmm, this could definitely be something. It didn’t even cross my mind
that
this might be the issue, but the background task that I have set up
tends to
execute twice in a row. I’ve looked around trying to find what might be
causing it but haven’t had any luck. Any thoughts on where dig?

Here’s some more info: Tomcat 5.5.17 running on redhat enterprise 5.4.

5.5.17 is very old - you should definitely upgrade to the latest 5.5.x or
6.0 version (for security reasons as well). Also, is this a Tomcat from
tomcat.apache.org or an RH package-mangled version? If the latter,
they’re almost always broken. And what JVM+version are you using?

Yeah, I know. Unfortunately, I don’t have control over the software on
the
servers. I’ll lobby for a more recent version. I also don’t know what
package it was installed with. I’ll check and see what I can find out.
Oh,
and we’re using java 1.5.0_21. Thanks for the pointers.

is JRuby 1.3.1 and Rails 2.3.5 (happened also on 2.3.4). I’m deploying
the
war by dropping it into tomcat’s webapps directory, copying it over the
previous version.

You might want to try an explicit undeploy first and watch the log files
to make sure that’s clean.

When you explicit undeploy do you mean through the tomcat manager? I’ve
attempted that with similar results to just dropping that war,
unfortunately.

are

actually coming from as I don’t log to catalina.out from my application.

  1. yeah, it would be good to make sure you can get an absolutely
    clean Tomcat startup/shutdown. What other applications are on
    this server? Presumably at least something deployed as ROOT,
    yes?

Yes, there are multiple other applications deployed in the webapps
directory
(all wars are in the same directory). The Rails site is the application
deployed at the context’s root (is that what you mean?). All the other
apps
have context paths matching their war. Sorry if I’m not understanding,
I’m
no expert on Tomcat deployment.

  1. All those ! are strange, for sure. I would check the other deployed
    apps.

    Also, if you’re not already using it, install Lambda Probe
    http://www.lambdaprobe.org/ to get a bit more visibility into what
    your server and apps are doing.

Didn’t know about lambdaprobe. That looks great. Thanks! The
exclamation
points are definitely coming from the rails app (or at least they only
occur
when that app is hit/active), but they’ve got me completely stumped.
I’ll
try pulling the other wars and attempt to get a clean startup/shutdown
on
tomcat with just the jruby stuff running to see if I can get a clean
environment to try and figure this out.

That’s a start, at least :slight_smile:

Thanks again! Might take me a few days to get through all this as this
isn’t my primary project at the moment, but I’ll report back with what I
find.

On Tue, Dec 8, 2009 at 11:01 PM, Tony C. [email protected]
wrote:

I vaguely recall some apparent caching problems with Tomcat, but IIRC
it involved accidentally starting a second instance, which wasn’t
actually
listening to anything because the first had the port(s) already. In any
case…

Here’s some more info: Tomcat 5.5.17 running on redhat enterprise 5.4.

5.5.17 is very old - you should definitely upgrade to the latest 5.5.x
or
6.0 version (for security reasons as well). Also, is this a Tomcat from
tomcat.apache.org or an RH package-mangled version? If the latter,
they’re almost always broken. And what JVM+version are you using?

is JRuby 1.3.1 and Rails 2.3.5 (happened also on 2.3.4). I’m deploying the
war by dropping it into tomcat’s webapps directory, copying it over the
previous version.

You might want to try an explicit undeploy first and watch the log files
to make sure that’s clean.

I’ve copied the log output during a redeploy below. Some things to note:
First, I’m using a gem called Rufus Scheduler to do a simple nightly
background task. Which is where the shutdown exception comes from. The
redeployment problem has been around longer than I’ve been using that gem,
but I’ll pull it out tomorrow and see if I can get a log without that
exception in it. Second, the exclamation points (one per line) appear in
the log for hundreds and hundreds of lines after I’ve cut the log off.
Those seem to appear on almost every request and I’m not sure where they are
actually coming from as I don’t log to catalina.out from my application.

  1. yeah, it would be good to make sure you can get an absolutely
    clean Tomcat startup/shutdown. What other applications are on
    this server? Presumably at least something deployed as ROOT,
    yes?

  2. All those ! are strange, for sure. I would check the other deployed
    apps.

    Also, if you’re not already using it, install Lambda Probe
    http://www.lambdaprobe.org/ to get a bit more visibility into what
    your server and apps are doing.

That’s a start, at least :slight_smile:


Hassan S. ------------------------ [email protected]
twitter: @hassan


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email