Forum: JRuby Start a script/console only with the contents of the warbled .war file?

Hi,

I'm about to deploy a JRuby on Rails app with warbler (and bundler).
Inside a Servlet container it's running fine, but now I ask myself if or
how it's possible to start a rails console or do a "rake db:migrate"
using only the files that are inside the .war file.
JRuby is there, all the gems are there, so theoretically it should be
possible, esp. if the .war file is unzipped. I guess, then it's only a
question of how to setup alle the paths to jruby and the gems. But how
to do that (with bundler in this case) ?
It would be a lot easier and consistent to maintain the production
system only with the stuff in the .war file and don't have to install
another system-global ruby environment.
Did anybody try that before or have any jump-start ideas on how to do 
that?

thanks, Heiko Seebach


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Hi all,

I made good progress on this:

in your app:
add the bundler gem itself as a requirement to the Gemfile

preparation:
1. export WEBAPP_HOME=<the location of your webapp>
2. unzip the .war
3. unzip jruby-core-1.5.1.jar and jruby-stdlib-1.5.1.jar
4. ln -s $WEBAPP_HOME/WEB-INF/lib/META-INF/jruby.home/bin /tmp/bin
# this is necessary, because it couldn't make it to set the RUBY
EXECUTABLE to another path. jgem env always shows "RUBY EXECUTABLE:
/tmp/bin/jruby"
5. ln -s  $WEBAPP_HOME/WEB-INF/lib/META-INF/jruby.home/lib /tmp/lib

then you can call the console like this:
6. export GEM_HOME=$WEBAPP_HOME/WEB-INF/lib/META-INF/jruby.home/
7. export GEM_PATH=$WEBAPP_HOME/WEB-INF/gems/
8. cd $WEBAPP_HOME/WEB-INF/
9. java  -cp lib/jruby-stdlib-1.5.1.jar  -jar lib/jruby-core-1.5.1.jar
-I $WEBAPP_HOME/WEB-INF/lib/META-INF/jruby.home/lib/ruby/1.8 -I
$WEBAPP_HOME/WEB-INF/lib/META-INF/jruby.home/lib/ruby/gems/1.8/ -I
$WEBAPP_HOME/WEB-INF/gems/ -I
$WEBAPP_HOME/WEB-INF/lib/META-INF/jruby.home/lib/ruby/site_ruby/1.8
-S  script/console production

and really, this will call the console!

But:  it's extremely slow in loading the source files.
Starting up the console on a modern machine takes about 5 minutes
a "kill -3" on the VM shows file system access stuff like this:
     at java.io.UnixFileSystem.getBooleanAttributes0(Native Method)
     at 
java.io.UnixFileSystem.getBooleanAttributes(UnixFileSystem.java:243)
     at java.io.File.exists(File.java:750)
     at 
org.jruby.ext.posix.JavaSecuredFile.exists(JavaSecuredFile.java:119)
     at org.jruby.RubyFileTest.file_p(RubyFileTest.java:137)
     at
org.jruby.RubyFileTest$s_method_1_0$RUBYINVOKER$file_p.call(org/jruby/RubyFileTest$s_method_1_0$RUBYINVOKER$file_p.gen:65535)
     at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:146)
     at org.jruby.ast.CallOneArgNode.interpret(CallOneArgNode.java:57)
or like that:
     at java.io.UnixFileSystem.checkAccess(Native Method)
     at java.io.File.canRead(File.java:708)
     at
org.jruby.ext.posix.JavaSecuredFile.canRead(JavaSecuredFile.java:101)
     at
org.jruby.runtime.load.LoadService.tryResourceFromLoadPath(LoadService.java:1086)
     at
org.jruby.runtime.load.LoadService.tryResourceFromLoadPathOrURL(LoadService.java:993)
     at
org.jruby.runtime.load.LoadService.findLibraryWithoutCWD(LoadService.java:766)
     at
org.jruby.runtime.load.LoadService$NormalSearcher.trySearch(LoadService.java:491)
     at
org.jruby.runtime.load.LoadService.findFileForLoad(LoadService.java:268)
     at 
org.jruby.runtime.load.LoadService.smartLoad(LoadService.java:313)
     at org.jruby.runtime.load.LoadService.require(LoadService.java:357)
     at
org.jruby.runtime.load.LoadService.lockAndRequire(LoadService.java:289)
     - locked <0x00002aaab04902b8> (a java.lang.Object)
     at org.jruby.RubyKernel.requireCommon(RubyKernel.java:984)
     at org.jruby.RubyKernel.require(RubyKernel.java:967)
     at
org.jruby.RubyKernel$s_method_1_0$RUBYFRAMEDINVOKER$require.call(org/jruby/RubyKernel$s_method_1_0$RUBYFRAMEDINVOKER$require.gen:65535)
     at
org.jruby.internal.runtime.methods.JavaMethod$JavaMethodOneOrNBlock.call(JavaMethod.java:319)
I analyzed this closely and found out, that this happens only, if the
two jruby jar files are unzipped. It takes also 5 minutes to start up
the app in tomcat. Whenever it's not unzipped, in tomcat it starts up
within about 15 seconds, but then of course I can't start the console 
again.

I used -Djruby.debug.loadService.timing=true and found out that the
*normal jruby boot process is ok*, but it looks like when the system
starts loading active_support, it slows down:

   -> script/../config/boot
    ...
<- script/../config/boot - 6545ms
   -> commands/runner
     -> optparse
<- optparse - 0ms
     -> /wepapp/releases/20100821-2204/WEB-INF/config/environment
       -> /wepapp/releases/20100821-2204/WEB-INF/config/boot
<- /wepapp/releases/20100821-2204/WEB-INF/config/boot - 3ms
       -> ruby_version_check
<- ruby_version_check - 8ms
       -> jruby/util
<- jruby/util - 0ms
       -> rubygems
<- rubygems - 0ms
       -> active_record
         -> active_support
           -> active_support/vendor
           ....
<- active_support/vendor - 956ms
and this continues many minutes. Here's another excerpt from the output:
             -> active_support/core_ext/range
               -> active_support/core_ext/range/conversions
<- active_support/core_ext/range/conversions - 951ms
               -> active_support/core_ext/range/overlaps
<- active_support/core_ext/range/overlaps - 959ms
               -> active_support/core_ext/range/include_range
<- active_support/core_ext/range/include_range - 922ms
               -> active_support/core_ext/range/blockless_step
<- active_support/core_ext/range/blockless_step - 905ms
<- active_support/core_ext/range - 4658ms

Any ideas about these loading times probs?

thanks, Heiko Seebach


On 22.08.2010 01:00, Heiko Seebach wrote:
> It would be a lot easier and consistent to maintain the production 
>
>    http://xircles.codehaus.org/manage_email
>
>

--
Heiko Seebach

Dr. Alfred-Neff-Str. 15
75015 Bretten

Telefon: +49 7252/97 54 07
Mobil:   +49 179/836 60 10
Email:   heiko.seebach@web.de

Hi again,

I moved further on and with strace I found, that the system hangs a lot
of "/lib" path entries inside the load calls and tries to detect files
like crazy.
There are zillions of strace output like this:
stat("/webapp/releases/20100822-1655/WEB-INF/lib/META-INF/jruby.home/lib/lib/lib/lib/lib/lib/lib/lib/lib/lib/lib/ruby/1.8/test/unit/util/active_record.class",
0x41110520) = -1 ENOENT (No such file or directory)

After some more searching I found, that with a manual install of a
system-global JRuby instance with a system-global bundler installation,
I could see this behaviour also.

But after a fresh install, a reboot and cleaning up the environent
variables I had it running successfully!

So I can confirm, that the console definitely comes up with this:

--------------------
export WEBAPP_HOME=...
export GEM_HOME=$WEBAPP_HOME/WEB-INF/lib/META-INF/jruby.home/
ln -s $WEBAPP_HOME/WEB-INF/lib/META-INF/jruby.home/bin /tmp/bin #gem env
shows "RUBY EXECUTABLE: /tmp/bin/jruby"
ln -s  $WEBAPP_HOME/WEB-INF/lib/META-INF/jruby.home/lib /tmp/lib
export GEM_PATH=$WEBAPP_HOME/WEB-INF/gems/
cd $WEBAPP_HOME/WEB-INF/
java  -cp lib/jruby-stdlib-1.5.1.jar  -jar lib/jruby-core-1.5.1.jar -I
$WEBAPP_HOME/WEB-INF/lib/META-INF/jruby.home/lib/ruby/1.8 -I
$WEBAPP_HOME/WEB-INF/lib/META-INF/jruby.home/lib/ruby/gems/1.8/ -I
$WEBAPP_HOME/WEB-INF/gems/ -I
$WEBAPP_HOME/WEB-INF/lib/META-INF/jruby.home/lib/ruby/site_ruby/1.8
-S  script/console production
--------------------

really cool...

There are two question remaining:
1. How to set the RUBY EXECUTABLE path to a correct value, so that the
soft links in /tmp/ can be avoided?
2. Is it necessary to unzip the jruby jars or is there a way to make
jruby find executables like jirb or rake even if they're inside the jar?

thanks, Heiko Seebach


On 22.08.2010 16:21, Heiko Seebach wrote:...
> But:  it's extremely slow in loading the source files.
> Starting up the console on a modern machine takes about 5 minutes
>
> Any ideas about these loading times probs?


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Heiko,

You've done a lot of good sleuthing here, and I'm glad to see you were
able to get close to something working.

There's definitely something wrong with the strace output here as well
as the ActiveSupport loading slowness. If you could file bugs for
those at http://bugs.jruby.org that would be great!

I also have ideas for making this sort of activity even easier than
what you've done here, so that you wouldn't even need to unpack files
or set environment variables. Watch for Warbler 1.3 to do something
along these lines.

Cheers,
/Nick

On Sun, Aug 22, 2010 at 1:21 PM, Heiko Seebach <heiko.seebach@web.de> 
wrote:
> system-global JRuby instance with a system-global bundler installation, I
> ln -s $WEBAPP_HOME/WEB-INF/lib/META-INF/jruby.home/bin /tmp/bin #gem env
> --------------------
>
> To unsubscribe from this list, please visit:
>
>   http://xircles.codehaus.org/manage_email
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Nick,

ok, here it is with all the available infos:
http://jira.codehaus.org/browse/JRUBY-5034

Looking forward to see support for something like this in Warbler 1.3!


thanks, Heiko


On 23.08.2010 07:20, Nick Sieger wrote:
> what you've done here, so that you wouldn't even need to unpack files
> or set environment variables. Watch for Warbler 1.3 to do something
> along these lines.
>
> Cheers,
> /Nick
>    


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Hi guys,

This integration sounds really great. And what about trying to execute 
some
rake tasks, like db:migrate, in this mode ? As Heiko says, this should 
be
possible.

I made some similar experiments with a small script like "require
'rubygems'; require 'rake'; Rake::Task["db:migrate"].invoke" etc. But I 
am
not sure whether this is the right way to go. What could be the ways to
achieve this ?
Maybe launching a new Ruby runtime (from Java) with proper 
configurations,
and executing 'evalscriptlet' ? Or sticking to the above idea: call the 
rake
tasks programatically ?

It would be great to have some way to execute rake tasks from within the 
web
application itself - hence the first idea. Running a new process "java 
-cp
-jar ..." via System.exec from Java would probably do, but a more lean
solution would be much better.

Kindest Regards,
Krum.

We do something similar and haven't had any of the issues you
describe.  Our war is created with warbler.  We drop a
jruby-complete.jar in lib, then have the following script on our
tomcat servers.  Works fine.

Chris

-----------------------------------------------------------------------------------
#!/bin/bash
export GEM_PATH=/home/tomcat/marvel/tomcat_root/WEB-INF/gems
export RAILS_ENV=$1

if [ "$2" = "console" ]
then
   cd /home/tomcat/marvel/tomcat_root/WEB-INF
   java -jar 
/home/tomcat/marvel/tomcat_root/WEB-INF/lib/jruby-complete.jar
script/rails console
fi

if [ "$2" = "migrate" ]
then
   cd /home/tomcat/marvel/tomcat_root/WEB-INF
   java -jar 
/home/tomcat/marvel/tomcat_root/WEB-INF/lib/jruby-complete.jar
-S rake database:migrate_all
fi

Thanks, Chris. That's great!

I was more wondering whether I could call the rake task 
programmatically,
i.e.within Java/Ruby, so that I do not have to start a new process on 
the
Tomcat machine just for the sake of a rake task. Of course, solutions 
like
System.execute("java -jar.."); are obviously fake. Maybe I could  launch 
a
new JRuby runtime, supply it with the appropriate configurations, and
execute the task, or smth else..

Kindest Regards,
Krum.