How to create run multiple scripts isolated in the same JRuby runtime?

Hi there

In Vert.x we want to run multiple JRuby scripts (using
ScriptingContainer) concurrently in the same JVM instance such that
they’re isolated - at the very least I don’t want them to be able to see
any top level variables or methods from other instances.

Up until now we’ve done this by each script running in its own
ScriptingContainer instance, e.g.:

ScriptingContainer container1 = new
ScriptingContainer(LocalContextScope.SINGLE_THREAD);
container1.runScriptlet(“script1.rb”);

ScriptingContainer container2 = new
ScriptingContainer(LocalContextScope.SINGLE_THREAD);
container2.runScriptlet(“script2.rb”);

This works fine, but we end up with each container having its own JRuby
runtime so there’s a big memory overhead which results in us not being
able to have more than a few hundred containers running at the same time
in the JVM.

I would like to somehow share a ScriptingContainer between the various
script instances so as to reduce the memory overhead.

I.e. do something like:

ScriptingContainer container = new ScriptingContainer();
container.runScriptlet(“script1.rb”);
container.runScriptlet(“script2.rb”);

I looked at the various context instance types here

The SINGLETON type is obviously not appropriate since there is only one
variable map and we want each script to be isolated.

The THREAD_SAFE type is not appropriate since it seems to tie the
runtime instance to the thread its being executed on - this means any
two scripts run with the same thread end up with the same variable map,
which breaks isolation.

The SINGLE_THREAD is not appropriate since container.runScriptlet might
be called from different threads.

The CONCURRENT type is not appropriate since it also seems to map the
scope to the thread its being executed on (similarly to THREAD_SAFE)

Basically I want to get an isolated variable map for every time I call
runScriptlet, but share a ruby runtime between them, but I can’t see any
way to do this using the API.

Any help greatly appreciated!

Tim,

I don’t think what you are asking for is possible (even if it was
possible I think it would be prone to all the same issues I mention
below w/ regards to side-effects), but I think we might be able to
figure something out. I guess I need to know more about your use-case
to recommend anything. So from what I understand you want n tasks to
use 1 runtime to cut down on memory requirements. You also want some
level of isolation. Can you explain what level of isolation
protection you require? Like:

  1. Can artibitrary constants get made (e.g. Foo or worse ::Foo)?
  2. Are you depending on local variable values to persist between
    executions?

When I have looked at this sort of isolation issue in the past I did
not need to worry about security or rogue modification of my env. In
that case I would construct and instance of an object (e.g. some
object which can be my ‘context’ object) return that back to the Java
side and then invoke a method on that context per thread task. Since
everything was executing in the context of a context object I really
only needed to worry about broader side-effects (like overriding
method defs, new constants) leading to isolation issues.

-Tom

On Fri, Jan 18, 2013 at 6:13 AM, Tim F. [email protected] wrote:

ScriptingContainer container1 = new
JVM.
I looked at the various context instance types here
The SINGLE_THREAD is not appropriate since container.runScriptlet might be


blog: http://blog.enebo.com twitter: tom_enebo
mail: [email protected]

Tom -

Might something like Nailgun be helpful for this? You might need to
start each script as its own process, though.

  • Keith

Keith R. Bennett

On Jan 18, 2013, at 6:13 AM, Tim F. wrote:

This works fine, but we end up with each container having its own JRuby runtime
so there’s a big memory overhead which results in us not being able to have more
than a few hundred containers running at the same time in the JVM.

I’m not real familiar with this code but I was wondering if something
could be done by directing things to different class loaders?
The ScriptingContainer would be instantiated in one ClassLoader.
Tracking down a little of the source it looks like then the

container.runScriptlet(“script1.rb”);

creates some sort of embed EvalUnit that is what is actually executed.
If that EvalUnit could be produced from a unique ClassLoader that has
the ScriptingContainer one as it’s parent ClassLoader you might be able
to share the ScriptingContainer runtime and get the isolation you want
from having actual execution in a unique child ClassLoader?

Michael H.

trz nio.2 for OS X http://www195.pair.com/mik3hall/index.html#trz

HalfPipe Java 6/7 shell app
http://www195.pair.com/mik3hall/index.html#halfpipe

AppConverter convert Apple jvm to openjdk apps
http://www195.pair.com/mik3hall/index.html#appconverter

On Jan 19, 2013, at 12:43 PM, Michael H. wrote:

On Jan 18, 2013, at 6:13 AM, Tim F. wrote:

This works fine, but we end up with each container having its own JRuby runtime
so there’s a big memory overhead which results in us not being able to have more
than a few hundred containers running at the same time in the JVM.

I’m not real familiar with this code but I was wondering if something could be
done by directing things to different class loaders?
The ScriptingContainer would be instantiated in one ClassLoader.
Tracking down a little of the source it looks like then the

Took a quick try at this out of curiosity.
Seem to be stuck now though.
I set up a delegating URLClassLoader subclass. To try and class loader
manipulate all of this, jruby had to be taken out of the class path or
java will preferentially find what it wants there and not the custom
class loaders.
The delegate was supposed to pass off ScriptingContainer load requests
to it’s own loader, the eval unit request to each their own.
So parent hierarchy like
DelegatingClassLoader
JRBSplitterClassLoader (for ScriptingContainer)
JRBSplitterClassLoader (eval unit) ----sibling—
JRBClassLoader (eval unit) — sibling — JRBClassLoader (eval unit)

Seemed to have some difficulties with the URLClassLoader set to
jruby.jar so went with jruby-complete-1.7.2.jar, still no joy.
First,

java.lang.ExceptionInInitializerError
at org.jruby.RubyString.(RubyString.java:123)
at org.jruby.Ruby.initCore(Ruby.java:1272)
at org.jruby.Ruby.bootstrap(Ruby.java:1199)
at org.jruby.Ruby.init(Ruby.java:1143)
at org.jruby.Ruby.newInstance(Ruby.java:281)
at
org.jruby.embed.internal.SingletonLocalContextProvider.getRuntime(SingletonLocalContextProvider.java:95)
at
org.jruby.embed.internal.EmbedRubyRuntimeAdapterImpl.parse(EmbedRubyRuntimeAdapterImpl.java:127)
at
org.jruby.embed.ScriptingContainer.parse(ScriptingContainer.java:1227)
at
org.jruby.embed.ScriptingContainer.runScriptlet(ScriptingContainer.java:1307)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at
us.hall.jrbsplitter.ScriptingContainerProxy.invoke(JRBSplitter.java:89)
at
us.hall.jrbsplitter.ScriptingContainerProxy.runScriptlet(JRBSplitter.java:75)
at us.hall.jrbsplitter.JRBSplitter.(JRBSplitter.java:44)
at us.hall.jrbsplitter.JRBSplitter.main(JRBSplitter.java:29)
Caused by: org.jcodings.exception.InternalException: entry:
/tables/CaseFold_From.bin not found
at org.jcodings.util.ArrayReader.openStream(ArrayReader.java:33)
at org.jcodings.util.ArrayReader.readIntArray(ArrayReader.java:54)
at
org.jcodings.unicode.UnicodeEncoding.(UnicodeEncoding.java:480)
… 17 more

followed by one of these for every attempted runScriptlet

java.lang.NoClassDefFoundError: Could not initialize class
org.jruby.RubyString
at org.jruby.Ruby.initCore(Ruby.java:1272)
at org.jruby.Ruby.bootstrap(Ruby.java:1199)
at org.jruby.Ruby.init(Ruby.java:1143)
at org.jruby.Ruby.newInstance(Ruby.java:281)

Too bad, sorry about replying to myself and length.

Michael H.

trz nio.2 for OS X http://www195.pair.com/mik3hall/index.html#trz

HalfPipe Java 6/7 shell app
http://www195.pair.com/mik3hall/index.html#halfpipe

AppConverter convert Apple jvm to openjdk apps
http://www195.pair.com/mik3hall/index.html#appconverter

I think it will violate his first issue of having more memory in play
than he wants.

-Tom

On Fri, Jan 18, 2013 at 2:20 PM, Keith B. [email protected]
wrote:

On Jan 18, 2013, at 2:53 PM, Thomas E Enebo [email protected] wrote:

protection you require? Like:
only needed to worry about broader side-effects (like overriding

methods from other instances.
container2.runScriptlet(“script2.rb”);

The THREAD_SAFE type is not appropriate since it seems to tie the runtime
Basically I want to get an isolated variable map for every time I call
blog: http://blog.enebo.com twitter: tom_enebo


To unsubscribe from this list, please visit:

http://xircles.codehaus.org/manage_email


blog: http://blog.enebo.com twitter: tom_enebo
mail: [email protected]