Forum: JRuby Anything wrong with shelling out to jruby?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
58c538d6f6e8fbf54779c5607e567bff?d=identicon&s=25 John B. (john_b)
on 2015-03-16 23:47
(Received via mailing list)
I have an application where I would like to shell out to jruby. A very
simple script that essentially does this a dozen times serially:

begin
  `rails runner scripts/foo.rb`
rescue
  # .. report error
end

I remember hearing about problems invoking jruby from jruby. Some
initial
basic tests in my dev environment don't show any problems.

My guess at what the problem is: if jruby is configured to take an
application's worth of RAM (say 512M), then the small out script will
take
that and then each individual sub process will take that as well.

I wonder if in practice this won't matter much, because the OS will move
most of the outer instance's memory into VM and just keep it there?

Thanks for any insights or pointers to other info about this!

John
486ca04f06d968004643ce5b47376ded?d=identicon&s=25 Keith B. (keith_b)
on 2015-03-17 14:09
(Received via mailing list)
I don’t know of any problems shelling out to JRuby, but it will require
the memory and startup time of starting a whole new JVM for each time
you do.  I would not be comforted by the fact that the OS might save
your first JVM’s RAM to disk and restore it later; that would take time.
Is speed not important in your case?

If I were you, I’d look at minimum to shelling out once rather than a
dozen times.  Why do you want to shell out so many times?

Also, your script looks like it shells out only once, no?

- Keith
1ed56a3c8445bfafc7a545d9a63175ce?d=identicon&s=25 Ted Han (Guest)
on 2015-03-17 14:12
(Received via mailing list)
There's a section about this in the jruby wiki:
https://github.com/jruby/jruby/wiki/Improving-star...

On Tue, Mar 17, 2015 at 9:07 AM, Keith Bennett <keithrbennett@gmail.com>
486ca04f06d968004643ce5b47376ded?d=identicon&s=25 Keith B. (keith_b)
on 2015-03-17 14:15
(Received via mailing list)
Ted -

That text mentions Nailgun; is Nailgun use still supported/recommended?
I thought no.

Thanks,
Keith
58c538d6f6e8fbf54779c5607e567bff?d=identicon&s=25 John B. (john_b)
on 2015-03-17 18:17
(Received via mailing list)
On Tue, Mar 17, 2015 at 9:07 AM, Keith Bennett <keithrbennett@gmail.com>
 wrote:

> Is speed not important in your case?
>

It's not important. These are batch jobs that run nightly. This master
script runs a few sub scripts, each of which do a few calculations and
then
queue up many (more efficient and well-structured) jobs in my background
job system (sidekiq). Eventually I'd like to make the work in the sub
scripts be done in workers as well, but for the time being I'd like to
just
keep things as-is and bring all the invocations into one place, and then
start refactoring.


If I were you, I’d look at minimum to shelling out once rather than a
dozen
> times.  Why do you want to shell out so many times?
>

As described above, I have many sub scripts which work and I don't want
to
refactor them in this first phase.



> Also, your script looks like it shells out only once, no?
>

That's an example of the code that I'll be doing about a dozen times in
the
script.



On Tue, Mar 17, 2015 at 9:10 AM, Ted Han <ted@knowtheory.net> wrote:

> There's a section about this in the jruby wiki:
>
https://github.com/jruby/jruby/wiki/Improving-star...
>

“we will attempt to run it in the same JVM using a new JRuby instance”

Great news, thanks! FYI if others are interested, this feature can be
set
at the commandline[1]:

    JRUBY_OPTS="-J-Djruby.launch.inproc=true" rails runner
scripts/foo.rb

Or within your ruby code[2]:

    # scripts/foo.rb
    require 'jruby'
    JRuby.runtime.instance_config.run_ruby_in_process = true


[1]
http://developers.mobilesystem7.com/blog/post/spee...
[2]
https://github.com/jruby/jruby/wiki/FAQs#how-can-i...
58c538d6f6e8fbf54779c5607e567bff?d=identicon&s=25 John B. (john_b)
on 2015-03-17 20:04
(Received via mailing list)
Some simple tests at the heroku console show that this feature doesn't
help
performance at all:

> JRuby.runtime.instance_config.run_ruby_in_process = false
=> false
> Benchmark.measure{`ruby -e"puts 'foo'"`}
=>   0.020000   0.010000  14.980000 ( 17.739000)

> JRuby.runtime.instance_config.run_ruby_in_process = true
=> true
> Benchmark.measure{`ruby -e"puts 'foo'"`}
=>   0.010000   0.000000  14.740000 ( 17.536000)


I wonder if the in-process feature isn't being used for some reason? Any
way to inspect this in either the inner or outer process?
58c538d6f6e8fbf54779c5607e567bff?d=identicon&s=25 John B. (john_b)
on 2015-03-21 18:48
(Received via mailing list)
I made some more observations. I did an experiment similar to my March
17
email, but this time with `ruby -e"sleep 1-"`, and then I observed my
process list to see if a new "java" was created and how much memory it
took.

with JRuby.runtime.instance_config.run_ruby_in_process set to true or
false, a second java instance was always created. The outer Java
instance
(running pry) took about 180 megs, and the inner about 80 megs.

Is this behavior expected? Let me know if there is more info I can
provide.

John
58c538d6f6e8fbf54779c5607e567bff?d=identicon&s=25 John B. (john_b)
on 2015-03-21 18:50
(Received via mailing list)
On Sat, Mar 21, 2015 at 1:46 PM, John Joseph Bachir <
johnjosephbachir@gmail.com> wrote:

> `ruby -e"sleep 1-"`


whoops, i mean  `ruby -e"sleep 10"`
58c538d6f6e8fbf54779c5607e567bff?d=identicon&s=25 John B. (john_b)
on 2015-03-21 18:57
(Received via mailing list)
I also tried invoking pry like so:

  JRUBY_OPTS="-J-Djruby.launch.inproc=true" pry

Same behavior.
C0cb3aadb9a66d00df16686f4a1fc312?d=identicon&s=25 Tim Uckun (Guest)
on 2015-03-22 21:51
(Received via mailing list)
Any word from the Jruby maintainers about this?

It seems to me that it could be a huge win for jruby if a new java
instance
was not launched both for speed and memory utilization.

On Sun, Mar 22, 2015 at 6:54 AM, John Joseph Bachir <
2c0c4cf3ccc8da22f7c3b9586ce1cd70?d=identicon&s=25 Christian MICHON (Guest)
on 2015-03-23 08:03
(Received via mailing list)
Regarding pry debugging, you should use pry-remote instead of pry.
Indeed,
you may have launched the jruby job as a service without a console
attached
to it.
58c538d6f6e8fbf54779c5607e567bff?d=identicon&s=25 John B. (john_b)
on 2015-03-23 14:39
(Received via mailing list)
On Mon, Mar 23, 2015 at 3:02 AM, Christian MICHON <
christian.michon@gmail.com> wrote:

> Regarding pry debugging, you should use pry-remote instead of pry. Indeed,
> you may have launched the jruby job as a service without a console attached
> to it.
>

Good thinking. I did another test, but am still finding a new JVM being
created.

I wrote a script, shell-out-test.rb:

puts "starting script and sleeping for 5 seconds"
sleep 5
puts "shelling out for 10 seconds"
`ruby -e 'sleep 10'`
puts "done shelling out, sleeping another 5 seconds"
sleep 5


Then opened two terminals side-by-side. In one I ran:

  watch 'ps aux |grep [j]ava'

In the other I ran my script:

  ruby shell-out-test.rb

And I observed two java processes running while shelling out.
58c538d6f6e8fbf54779c5607e567bff?d=identicon&s=25 John B. (john_b)
on 2015-03-24 21:21
(Received via mailing list)
Okay, in the IRC channel headius requested I make a ticket for this:

  https://github.com/jruby/jruby/issues/2754

So I suppose it's considered broken for the time being.
This topic is locked and can not be replied to.