JRuby complete, multi-threading, and java.lang.NoClassDefFoundError


#1

I am having problems running a JRuby scripting container accessed by
multiple threads. Here’s my declaration:

protected static ScriptingContainer container = new
ScriptingContainer(LocalContextScope.THREADSAFE);

When I run my application which has multiple threads utilizing this
container, I often get java.lang.NoClassDefFoundError or other similar
exceptions soon after starting up. Not always, but often. And not always
the same class. Here are four different stack traces that I have seen:

==> stack-1.txt <==
org.jruby.embed.InvokeFailedException: (NameError) uninitialized
constant
Krypt::Provider::JDK
at
org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.call(EmbedRubyObjectAdapterImpl.java:317)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.callMethod(EmbedRubyObjectAdapterImpl.java:169)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.jruby.embed.ScriptingContainer.callMethod(ScriptingContainer.java:1508)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at com.needco.scrapers.JRubyScraper.init(JRubyScraper.java:43)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at com.needco.scrapers.Scraper.scrape(Scraper.java:90)
[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
com.needco.scrapers.ScraperProcessor.run(ScraperProcessor.java:108)
[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[?:1.7.0_71]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[?:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [?:1.7.0_71]
Caused by: org.jruby.exceptions.RaiseException: (NameError)
uninitialized
constant Krypt::Provider::JDK
at
org.jruby.RubyModule.const_missing(org/jruby/RubyModule.java:2726)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
RUBY.(root)(/home/scraper/scrapers/nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar!/META-INF/jruby.home/lib/ruby/shared/krypt/core.rb:41)
~[?:?]
at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1071)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
Kernel.require(/home/scraper/scrapers/nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:73)
~[?:?]

==> stack-2.txt <==
org.jruby.embed.InvokeFailedException: (LoadError) load error: kryptcore

java.lang.NoClassDefFoundError: org/jruby/ext/krypt/KryptCoreService
at
org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.call(EmbedRubyObjectAdapterImpl.java:317)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.callMethod(EmbedRubyObjectAdapterImpl.java:169)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.jruby.embed.ScriptingContainer.callMethod(ScriptingContainer.java:1508)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at com.needco.scrapers.JRubyScraper.init(JRubyScraper.java:43)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at com.needco.scrapers.Scraper.scrape(Scraper.java:90)
[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
com.needco.scrapers.ScraperProcessor.run(ScraperProcessor.java:108)
[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[?:1.7.0_71]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[?:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [?:1.7.0_71]
Caused by: org.jruby.exceptions.RaiseException: (LoadError) load error:
kryptcore – java.lang.NoClassDefFoundError:
org/jruby/ext/krypt/KryptCoreService
at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1071)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
Kernel.require(/home/scraper/scrapers/nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:55)
~[?:?]
at
Kernel.require(/home/scraper/scrapers/nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:53)
~[?:?]
at
RUBY.(root)(/home/scraper/scrapers/nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar!/META-INF/jruby.home/lib/ruby/shared/krypt/core.rb:37)
~[?:?]

==> stack-3.txt <==
org.jruby.embed.InvokeFailedException: (LoadError) load error: kryptcore

java.lang.NoClassDefFoundError: org/jruby/ext/krypt/helper/RubyHelper
at
org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.call(EmbedRubyObjectAdapterImpl.java:317)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.callMethod(EmbedRubyObjectAdapterImpl.java:169)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.jruby.embed.ScriptingContainer.callMethod(ScriptingContainer.java:1508)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at com.needco.scrapers.JRubyScraper.init(JRubyScraper.java:43)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at com.needco.scrapers.Scraper.scrape(Scraper.java:90)
[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
com.needco.scrapers.ScraperProcessor.run(ScraperProcessor.java:108)
[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[?:1.7.0_71]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[?:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [?:1.7.0_71]
Caused by: org.jruby.exceptions.RaiseException: (LoadError) load error:
kryptcore – java.lang.NoClassDefFoundError:
org/jruby/ext/krypt/helper/RubyHelper
at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1071)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
Kernel.require(/home/scraper/scrapers/nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:126)
~[?:?]
at
Kernel.require(/home/scraper/scrapers/nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:124)
~[?:?]
at
RUBY.(root)(/home/scraper/scrapers/nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar!/META-INF/jruby.home/lib/ruby/shared/krypt/core.rb:37)
~[?:?]

==> stack-4.txt <==
org.jruby.embed.InvokeFailedException: (LoadError) load error:
jopenssl/load – java.lang.NoClassDefFoundError:
org/bouncycastle/asn1/DERBoolean
at
org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.call(EmbedRubyObjectAdapterImpl.java:317)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.jruby.embed.internal.EmbedRubyObjectAdapterImpl.callMethod(EmbedRubyObjectAdapterImpl.java:169)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
org.jruby.embed.ScriptingContainer.callMethod(ScriptingContainer.java:1508)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at com.needco.scrapers.JRubyScraper.init(JRubyScraper.java:43)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at com.needco.scrapers.Scraper.scrape(Scraper.java:90)
[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
com.needco.scrapers.ScraperProcessor.run(ScraperProcessor.java:108)
[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
[?:1.7.0_71]
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
[?:1.7.0_71]
at java.lang.Thread.run(Thread.java:745) [?:1.7.0_71]
Caused by: org.jruby.exceptions.RaiseException: (LoadError) load error:
jopenssl/load – java.lang.NoClassDefFoundError:
org/bouncycastle/asn1/DERBoolean
at org.jruby.RubyKernel.require(org/jruby/RubyKernel.java:1071)
~[nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar:?]
at
Kernel.require(/home/scraper/scrapers/nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:126)
~[?:?]
at
Kernel.require(/home/scraper/scrapers/nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar!/META-INF/jruby.home/lib/ruby/shared/rubygems/core_ext/kernel_require.rb:124)
~[?:?]
at
RUBY.(root)(/home/scraper/scrapers/nc-scrapers-0.0.1-SNAPSHOT-jar-with-dependencies.jar!/META-INF/jruby.home/lib/ruby/shared/openssl.rb:1)
~[?:?]

I have tried setting global.require.lock=true and
.compile.invokedynamic=false as they had been suggested to others that
seemed to have similar issues, but that has not helped.

I am using jruby-complete 1.7.18 and Java 1.7.0_71.

Can anyone help?

Thanks!


#2

could you set

System.setProperty(“jruby.debug.loadService”, “true” );

just before you instantiate the ScriptingContainer. it produces a lot of
output but could help to understand what is going on.

all those errors are with one or the other default gems and their jars
from
those default gems.


#3

I passed that option in with -Djruby.debug.loadService=true and got some
output. I’m not sure what to do with the output thought. I’ve
compressed
and attached it here.

Does it give you any insight?

Thanks.


#4

well, it shows what was almost clear. you start kind of the jruby
runtimes
at the “same” time and with loading the jar into jruby-classloader of
the
runtime something goes wrong. LocalContextScope.THREADSAFE should do the
trick but apparently it does NOT :frowning:


#5

So any suggestions? Is my only option to use
LocalContextScope.SINGLETON
and have each thread use it’s own ruby runtime?


#6

sorry can not help more here but I filed an issue:
https://github.com/jruby/jruby/issues/2561


#7

I looked into the code and yes those runtimes do share the
JRubyClassLoader
which is a bug IMO

somehow you need to serialize the startup of the runtime. the crucial
thing
is “require ‘openssl’” which loads a few jars. once these jars are
loading
things probably work OK.

not sure how to do this.


#8

Thanks, Christian.

I worked around this by using some static code in my Java class to force
the Ruby runtime to load first before any of the thread start. Ugly, but
it
appears to be effective.

I’m surprised that nobody else has hit this yet.