Anything wrong with shelling out to jruby?

Ruby JRuby
#1

I have an application where I would like to shell out to jruby. A very
simple script that essentially does this a dozen times serially:

begin
rails runner scripts/foo.rb
rescue

… report error

end

I remember hearing about problems invoking jruby from jruby. Some
initial
basic tests in my dev environment don’t show any problems.

My guess at what the problem is: if jruby is configured to take an
application’s worth of RAM (say 512M), then the small out script will
take
that and then each individual sub process will take that as well.

I wonder if in practice this won’t matter much, because the OS will move
most of the outer instance’s memory into VM and just keep it there?

Thanks for any insights or pointers to other info about this!

John

#2

I don’t know of any problems shelling out to JRuby, but it will require
the memory and startup time of starting a whole new JVM for each time
you do. I would not be comforted by the fact that the OS might save
your first JVM’s RAM to disk and restore it later; that would take time.
Is speed not important in your case?

If I were you, I’d look at minimum to shelling out once rather than a
dozen times. Why do you want to shell out so many times?

Also, your script looks like it shells out only once, no?

  • Keith
#3

There’s a section about this in the jruby wiki:
https://github.com/jruby/jruby/wiki/Improving-startup-time#running-sub-rubies-in-the-same-jvm

On Tue, Mar 17, 2015 at 9:07 AM, Keith B. [email protected]

#4

Ted -

That text mentions Nailgun; is Nailgun use still supported/recommended?
I thought no.

Thanks,
Keith

#5

Some simple tests at the heroku console show that this feature doesn’t
help
performance at all:

JRuby.runtime.instance_config.run_ruby_in_process = false
=> false
Benchmark.measure{ruby -e"puts 'foo'"}
=> 0.020000 0.010000 14.980000 ( 17.739000)

JRuby.runtime.instance_config.run_ruby_in_process = true
=> true
Benchmark.measure{ruby -e"puts 'foo'"}
=> 0.010000 0.000000 14.740000 ( 17.536000)

I wonder if the in-process feature isn’t being used for some reason? Any
way to inspect this in either the inner or outer process?

#6

On Tue, Mar 17, 2015 at 9:07 AM, Keith B. [email protected]
wrote:

Is speed not important in your case?

It’s not important. These are batch jobs that run nightly. This master
script runs a few sub scripts, each of which do a few calculations and
then
queue up many (more efficient and well-structured) jobs in my background
job system (sidekiq). Eventually I’d like to make the work in the sub
scripts be done in workers as well, but for the time being I’d like to
just
keep things as-is and bring all the invocations into one place, and then
start refactoring.

If I were you, I’d look at minimum to shelling out once rather than a
dozen

times. Why do you want to shell out so many times?

As described above, I have many sub scripts which work and I don’t want
to
refactor them in this first phase.

Also, your script looks like it shells out only once, no?

That’s an example of the code that I’ll be doing about a dozen times in
the
script.

On Tue, Mar 17, 2015 at 9:10 AM, Ted H. [email protected] wrote:

There’s a section about this in the jruby wiki:

https://github.com/jruby/jruby/wiki/Improving-startup-time#running-sub-rubies-in-the-same-jvm

“we will attempt to run it in the same JVM using a new JRuby instance”

Great news, thanks! FYI if others are interested, this feature can be
set
at the commandline[1]:

JRUBY_OPTS="-J-Djruby.launch.inproc=true" rails runner

scripts/foo.rb

Or within your ruby code[2]:

# scripts/foo.rb
require 'jruby'
JRuby.runtime.instance_config.run_ruby_in_process = true

[1]
http://developers.mobilesystem7.com/blog/post/speeding-up-rails-on-virtualbox/
[2]
https://github.com/jruby/jruby/wiki/FAQs#how-can-i-increase-the-heapmemory-size-when-launching-a-sub-jruby

#7

I made some more observations. I did an experiment similar to my March
17
email, but this time with ruby -e"sleep 1-", and then I observed my
process list to see if a new “java” was created and how much memory it
took.

with JRuby.runtime.instance_config.run_ruby_in_process set to true or
false, a second java instance was always created. The outer Java
instance
(running pry) took about 180 megs, and the inner about 80 megs.

Is this behavior expected? Let me know if there is more info I can
provide.

John

#8

I also tried invoking pry like so:

JRUBY_OPTS="-J-Djruby.launch.inproc=true" pry

Same behavior.

#9

On Sat, Mar 21, 2015 at 1:46 PM, John Joseph B. <
[email protected]> wrote:

ruby -e"sleep 1-"

whoops, i mean ruby -e"sleep 10"

#10

Any word from the Jruby maintainers about this?

It seems to me that it could be a huge win for jruby if a new java
instance
was not launched both for speed and memory utilization.

On Sun, Mar 22, 2015 at 6:54 AM, John Joseph B. <

#11

Regarding pry debugging, you should use pry-remote instead of pry.
Indeed,
you may have launched the jruby job as a service without a console
attached
to it.

#12

On Mon, Mar 23, 2015 at 3:02 AM, Christian MICHON <
[email protected]> wrote:

Regarding pry debugging, you should use pry-remote instead of pry. Indeed,
you may have launched the jruby job as a service without a console attached
to it.

Good thinking. I did another test, but am still finding a new JVM being
created.

I wrote a script, shell-out-test.rb:

puts “starting script and sleeping for 5 seconds”
sleep 5
puts “shelling out for 10 seconds”
ruby -e 'sleep 10'
puts “done shelling out, sleeping another 5 seconds”
sleep 5

Then opened two terminals side-by-side. In one I ran:

watch ‘ps aux |grep [j]ava’

In the other I ran my script:

ruby shell-out-test.rb

And I observed two java processes running while shelling out.

#13

Okay, in the IRC channel headius requested I make a ticket for this:

https://github.com/jruby/jruby/issues/2754

So I suppose it’s considered broken for the time being.