Forum: JRuby Problem reading SSL PEM file with jruby-1.7.10

1272e095e145697bb0371230688bd2c3?d=identicon&s=25 Michael Pitman (mcp)
on 2014-02-24 07:32
(Received via mailing list)
Hi,

We're having problems reading a PEM file with jruby-1.7.10, that was
able
to be read successfully in jruby-1.7.9.

A colleague of mine is trying to write up a test case to send (which
doesn't use the actual PEM file we're failing on, since that has a
confidential key), but I thought I'd see if anyone else had encountered
this error and determined a solution:

#<OpenSSL::PKey::RSAError: Neither PUB key nor PRIV key:>
["org/jruby/ext/openssl/PKeyRSA.java:289:in `initialize'",
...


Here's the result of the investigation so far:
- We can read the file in MRI, jruby-1.7.9 and earlier jruby (back to
1.7.4)
- We can't read it with 1.7.10, 1.7.11-SNAPSHOT and 9000.dev.
- If we include bcprov-jdk15on-1.49.jar and bcpkix-jdk15on-1.49.jar in
the
class path before the jruby jars (which look like 1.47), the file can be
read with 1.7.10 and 1.7.11-SNAPSHOT, but still not with 9000.dev.
- If we include bcprov-jdk15on-1.50.jar and bcpkix-jdk15on-1.50.jar in
the
class path, things go haywire.

For our purposes, we'll probably revert to 1.7.9, but I'm happy to test
out
any potential fixes if that would help.

Thanks,

Michael
B05d3cbc64b0031a24c2887fb6ddc173?d=identicon&s=25 christian (Guest)
on 2014-02-24 10:27
(Received via mailing list)
hello,

any reason that you use 1.49 and not the 1.47 to add to classpath ?

which OS are you using ?


On Mon, Feb 24, 2014 at 6:30 AM, Michael Pitman
<michaelp@ibsglobalweb.com>wrote:

> Here's the result of the investigation so far:
> - If we include bcprov-jdk15on-1.49.jar and bcpkix-jdk15on-1.49.jar in the
> class path before the jruby jars (which look like 1.47), the file can be
> read with 1.7.10 and 1.7.11-SNAPSHOT, but still not with 9000.dev.
>

with 9000.dev you need to require the bc*jar for the ruby part to see

- If we include bcprov-jdk15on-1.50.jar and bcpkix-jdk15on-1.50.jar in
the
> class path, things go haywire.
>

look like the BC API changed a bit here



>
>
For our purposes, we'll probably revert to 1.7.9, but I'm happy to test
out
1272e095e145697bb0371230688bd2c3?d=identicon&s=25 Michael Pitman (mcp)
on 2014-02-24 22:21
(Received via mailing list)
Hi Christian,

It does seem to work with 1.47 on the classpath also. I had prematurely
concluded that it was a version issue with bouncy-castle, because
including
the bouncy-castle-java gem did not seem to fix the problem, but those
1.47
jars do fix it.

We're trying on Mac OS X, Linux (Ubuntu 12.04) and Windows Server 2012.

I guess it now feels like the issue may be around the inclusion of the
bc
jars from that gem into the classpath.

Any ideas? Do we need to adopt the same approach as you've suggested for
9000.dev (ie an explicit require)? That latter is a pain, because all
our
systems are made to be deployed to either MRI or jruby, so we try to
avoid
any java specific code.

Thanks,

Michael
B05d3cbc64b0031a24c2887fb6ddc173?d=identicon&s=25 christian (Guest)
on 2014-02-25 10:41
(Received via mailing list)
hi Michael,

actually the bouncy-castle jars are part of jruby (either
jruby-complete.jar, jruby-jars.gem or the distribution tarball) and we
see
this problem of not loading those jars on various issues but still
unclear
why. since I can no reproduce the problem so far (not tried the last
hint I
got) it would great if you could try out the latest idea to fix it:

https://github.com/jruby/jruby/commit/d08c00e42354...

so basically not using a glob to find the bouncy castle jars.

thanx + regards,
christian



On Mon, Feb 24, 2014 at 9:19 PM, Michael Pitman
1272e095e145697bb0371230688bd2c3?d=identicon&s=25 Michael Pitman (mcp)
on 2014-02-25 22:43
(Received via mailing list)
Hi Christian,

Thanks for that, but no luck, unfortunately. I applied this patch to
master
(9000.dev), and it is still not able to extract the key from the pem
file.

Thanks,

Michael
B05d3cbc64b0031a24c2887fb6ddc173?d=identicon&s=25 christian (Guest)
on 2014-02-25 22:54
(Received via mailing list)
ok, then I need to ask how you deploy the app with jruby ? is it a
webapp ?
or something else.

- christian
1272e095e145697bb0371230688bd2c3?d=identicon&s=25 Michael Pitman (mcp)
on 2014-02-26 00:51
(Received via mailing list)
We actually invoke jruby in a few different ways.

In development & test, we use jruby-complete.jar, which is how I'm
running
the tests that expose the problem. We also have the exact same
applications
running under both jruby and MRI.

In production, we deploy the applications as web applications (built
with
warbler) and as stand-alone executables (using runnable war files).

We send our clients war files to run, and those run on application
servers
running other applications. Everything we need on the target machine has
to
be packaged in the war. That's why it's not very workable to have to set
a
class path on deployment.

Thanks,

Michael
B05d3cbc64b0031a24c2887fb6ddc173?d=identicon&s=25 christian (Guest)
on 2014-02-26 11:50
(Received via mailing list)
so the warbler issue with bouncy-castle I found but I am not it applies
to
your setup with jruby-complete during testing/development.

the warbler issue is about having jruby-openssl in the Gemfile, removing
it
form there is workaround.

if you could provide a simple test case using jruby-complete.jar that
would
be very helpful.

- christian
1272e095e145697bb0371230688bd2c3?d=identicon&s=25 Michael Pitman (mcp)
on 2014-03-05 23:24
(Received via mailing list)
Hi Christian,

We've come up with a pretty minimal example that shows the change in
behaviour from jruby 1.7.9 and 1.7.10. We're using the jruby-complete
jars
unaltered from jruby.org/download

First, create a dummy key and certificate. Use password "TestPW" when
prompted:

openssl req -x509 -newkey rsa:2048 -keyout mykey.pem -out mycert.pem

Then we have the following ruby script to run to test out the
extraction.
It's small enough to include in line:

gem 'jruby-openssl'
require 'openssl'

begin
  file_name = 'mykey.pem'
  x = OpenSSL::PKey::RSA.new(File.read(file_name), 'TestPW')
  puts x.inspect
rescue Exception => e
  puts "e.message = #{e.message}"
  puts "obj = " + e.backtrace.join("\n")
end


Now, we install the gems required for jruby-1.7.9, but not, apparently,
for
1.7.10:

GEM_HOME=./testrsagem GEM_PATH=./testrsagem java -jar
jruby-complete-1.7.9.jar -S jgem install jruby-openssl
bouncy-castle-java

Finally we try:

GEM_HOME=./testrsagem GEM_PATH=./testrsagem java -jar
jruby-complete-1.7.9.jar ./rsa_from_file.rb

That works, and extracts the key, but:

GEM_HOME=./testrsagem GEM_PATH=./testrsagem java -jar
jruby-complete-1.7.10.jar ./rsa_from_file.rb

Fails with:

e.message = Neither PUB key nor PRIV key:
obj = org/jruby/ext/openssl/PKeyRSA.java:283:in `initialize'
./rsa_from_file.rb:6:in `(root)'


I hope that you get similar results and it provides some insight into
the
problem :)

Thanks,

Michael
B05d3cbc64b0031a24c2887fb6ddc173?d=identicon&s=25 christian (Guest)
on 2014-03-05 23:30
(Received via mailing list)
cool, I give it some time tomorrow !

-christian


On Wed, Mar 5, 2014 at 10:23 PM, Michael Pitman
B05d3cbc64b0031a24c2887fb6ddc173?d=identicon&s=25 christian (Guest)
on 2014-03-06 10:36
(Received via mailing list)
well, I do not see any errors which ever jruby version I used: 1.7.4,
1.7.9, 1.7.10, 1.7.11 or 1.7.12-SNAPSHOT

I also switch jvm version from openjdk7 to oracle-java6, oracle-java7,
oracle-java8 and they all worked with 1.7.10

double checked the md5sum of my jruby-complete-1.7.10.jar and even
downloaded afresh from jruby.org

anything I can suggest now you probably already did. maybe try 1.7.11 ;)

hmm, I am still willing to help, so I am open to any new ideas ;)

regards,
christian
1272e095e145697bb0371230688bd2c3?d=identicon&s=25 Michael Pitman (mcp)
on 2014-03-06 22:52
(Received via mailing list)
How odd. I wonder if the bouncy castle jars are in your classpath
somehow.
Could you try doing:

GEM_HOME=./testrsagem GEM_PATH=./testrsagem java -cp
jruby-complete-1.7.10.jar org.jruby.Main ./rsa_from_file.rb

We are trying in development on Mac OS and Linux. We see the same error
in
production on Windows. I've tried with Java 7 and 8.

Thanks very much,

Michael
B05d3cbc64b0031a24c2887fb6ddc173?d=identicon&s=25 christian (Guest)
on 2014-03-06 23:13
(Received via mailing list)
hi Michael,

could you add
  Gem.loaded_specs.each do |k,v|
    p "#{k} #{v.gem_dir}"
  end

to your script and let me know what gems you are actually using ?

actually I see only the default gems from jruby-complete itself, so I am
probably doing something different then you.

- christian
1272e095e145697bb0371230688bd2c3?d=identicon&s=25 Michael Pitman (mcp)
on 2014-03-10 09:29
(Received via mailing list)
Hi Christian,

These are certainly returning different results:

jruby-1.7.9:

bouncy-castle-java
/Volumes/Code/testbc/testrsagem/gems/bouncy-castle-java-1.5.0147
jruby-openssl /Volumes/Code/testbc/testrsagem/gems/jruby-openssl-0.9.4
-----BEGIN RSA PRIVATE KEY-----
...


jruby-1.7.10:

bouncy-castle-java
file:/Volumes/Code/testbc/jruby-complete-1.7.10.jar!/META-INF/jruby.home/lib/ruby/gems/shared/gems/bouncy-castle-java-1.5.0147
jruby-openssl /Volumes/Code/testbc/testrsagem/gems/jruby-openssl-0.9.4
krypt-provider-jdk
file:/Volumes/Code/testbc/jruby-complete-1.7.10.jar!/META-INF/jruby.home/lib/ruby/gems/shared/gems/krypt-provider-jdk-0.0.1
krypt-core
file:/Volumes/Code/testbc/jruby-complete-1.7.10.jar!/META-INF/jruby.home/lib/ruby/gems/shared/gems/krypt-core-0.0.1-universal-java
krypt
file:/Volumes/Code/testbc/jruby-complete-1.7.10.jar!/META-INF/jruby.home/lib/ruby/gems/shared/gems/krypt-0.0.1
e.message = Neither PUB key nor PRIV key:
obj = org/jruby/ext/openssl/PKeyRSA.java:283:in `initialize'
./rsa_from_file.rb:11:in `(root)'



And if I don't use any extra gems (ie just using the built-in
jruby-openssl) with 1.7.10:

bouncy-castle-java
file:/Volumes/Code/testbc/jruby-complete-1.7.10.jar!/META-INF/jruby.home/lib/ruby/gems/shared/gems/bouncy-castle-java-1.5.0147
jruby-openssl
file:/Volumes/Code/testbc/jruby-complete-1.7.10.jar!/META-INF/jruby.home/lib/ruby/gems/shared/gems/jruby-openssl-0.9.3
krypt-provider-jdk
file:/Volumes/Code/testbc/jruby-complete-1.7.10.jar!/META-INF/jruby.home/lib/ruby/gems/shared/gems/krypt-provider-jdk-0.0.1
krypt-core
file:/Volumes/Code/testbc/jruby-complete-1.7.10.jar!/META-INF/jruby.home/lib/ruby/gems/shared/gems/krypt-core-0.0.1-universal-java
krypt
file:/Volumes/Code/testbc/jruby-complete-1.7.10.jar!/META-INF/jruby.home/lib/ruby/gems/shared/gems/krypt-0.0.1
e.message = Neither PUB key nor PRIV key:
obj = org/jruby/ext/openssl/PKeyRSA.java:289:in `initialize'
./rsa_from_file.rb:11:in `(root)'



Thanks,

Michael
1272e095e145697bb0371230688bd2c3?d=identicon&s=25 Michael Pitman (mcp)
on 2014-03-13 22:49
(Received via mailing list)
Hi Christian,

I've been some more digging, and it looks like the issue is that since
jruby-1.7.10, if we require a gem that is already contained in the
jruby-complete jar as well as outside it, it will prefer to use the
internal one. That looks to be a problem when the gem contains a jar,
because then it puts a jar inside a jar in the classpath.

Here's some small example code (I've called the script
test_classpath.rb):

gem 'bouncy-castle-java'
require 'openssl'
puts 'Class path:'
$CLASSPATH.each{|path| puts path}


After having installed the bouncy-castle-java and jruby-openssl gems in
the
local directory "testgem", my directory structure looks like:

jruby-complete-1.7.10.jar  jruby-complete-1.7.9.jar
 jruby-complete-9000.dev.jar  test_classpath.rb  testgem/


and the directory testgem/gems has only two gems installed:

bouncy-castle-java-1.5.0147/  jruby-openssl-0.9.4/


Now, the results for jruby 1.7.9:
] PATH=. CLASSPATH=. GEM_HOME=./testgem GEM_PATH=./testgem
$JAVA_HOME/bin/java -cp jruby-complete-1.7.9.jar org.jruby.Main
./test_classpath.rb
Class path:
file:/Volumes/Code/testbc/testgem/gems/bouncy-castle-java-1.5.0147/lib/bcpkix-jdk15on-147.jar
file:/Volumes/Code/testbc/testgem/gems/bouncy-castle-java-1.5.0147/lib/bcprov-jdk15on-147.jar
jar:file:/Volumes/Code/testbc/jruby-complete-1.7.9.jar!/META-INF/jruby.home/lib/ruby/shared/jopenssl.jar
jar:file:/Volumes/Code/testbc/jruby-complete-1.7.9.jar!/META-INF/jruby.home/lib/ruby/shared/kryptcore.jar
jar:file:/Volumes/Code/testbc/jruby-complete-1.7.9.jar!/META-INF/jruby.home/lib/ruby/shared/kryptproviderjdk.jar


For jruby 1.7.10:

] PATH=. CLASSPATH=. GEM_HOME=./testgem GEM_PATH=./testgem
$JAVA_HOME/bin/java -cp jruby-complete-1.7.10.jar org.jruby.Main
./test_classpath.rb
Class path:
jar:file:/Volumes/Code/testbc/jruby-complete-1.7.10.jar!/META-INF/jruby.home/lib/ruby/shared/bcpkix-jdk15on-1.47.jar
jar:file:/Volumes/Code/testbc/jruby-complete-1.7.10.jar!/META-INF/jruby.home/lib/ruby/shared/bcprov-jdk15on-1.47.jar
file:/Volumes/Code/testbc/testgem/gems/jruby-openssl-0.9.4/lib/jopenssl.jar
jar:file:/Volumes/Code/testbc/jruby-complete-1.7.10.jar!/META-INF/jruby.home/lib/ruby/shared/kryptcore.jar
jar:file:/Volumes/Code/testbc/jruby-complete-1.7.10.jar!/META-INF/jruby.home/lib/ruby/shared/kryptproviderjdk.jar


And jruby 9000.dev:

] PATH=. CLASSPATH=. GEM_HOME=./testgem GEM_PATH=./testgem
$JAVA_HOME/bin/java -cp jruby-complete-9000.dev.jar org.jruby.Main
./test_classpath.rb
Class path:
jar:file:/Volumes/Code/testbc/jruby-complete-9000.dev.jar!/META-INF/jruby.home/lib/ruby/shared/bcpkix-jdk15on-1.47.jar
jar:file:/Volumes/Code/testbc/jruby-complete-9000.dev.jar!/META-INF/jruby.home/lib/ruby/shared/bcprov-jdk15on-1.47.jar
jar:file:/Volumes/Code/testbc/jruby-complete-9000.dev.jar!/META-INF/jruby.home/lib/ruby/shared/jopenssl.jar
jar:file:/Volumes/Code/testbc/jruby-complete-9000.dev.jar!/META-INF/jruby.home/lib/ruby/shared/kryptcore.jar
jar:file:/Volumes/Code/testbc/jruby-complete-9000.dev.jar!/META-INF/jruby.home/lib/ruby/shared/kryptproviderjdk.jar



So I guess this comes down to the question of whether we are using a
classloader in jruby that can load classes from a jar inside a jar, Or
is
this just a red herring?

Thanks,

Michael
B05d3cbc64b0031a24c2887fb6ddc173?d=identicon&s=25 christian (Guest)
on 2014-03-14 01:26
(Received via mailing list)
it is already bedtime for me - but yes I made the same observation that
the
jar from within the jruby jar get preference which is a problem when you
install a gem and want to use jars from the gem instead.

actually if I look at all three 1.7.9, 1.7.10 and 9000.dev they all look
wrong to me, I would expect them to be:
file:/Volumes/Code/testbc/testgem/gems/bouncy-castle-java-1.5.0147/lib/bcpkix-jdk15on-147.jar
file:/Volumes/Code/testbc/testgem/gems/bouncy-castle-java-1.5.0147/lib/bcprov-jdk15on-147.jar

file:/Volumes/Code/testbc/testgem/gems/jruby-openssl-0.9.4/lib/jopenssl.jar
jar:file:/Volumes/Code/testbc/jruby-complete-1.7.9.jar!/META-INF/jruby.home/lib/ruby/shared/kryptcore.jar

jar:file:/Volumes/Code/testbc/jruby-complete-1.7.9.jar!/META-INF/jruby.home/lib/ruby/shared/kryptproviderjdk.jar

the jar within a jar in general works BUT there be cases where is does
not.
that is why I unpack the jruby-stdlib.jar for the war-file in my
personal
project:
https://github.com/mkristian/war-pack
I also added a testcase to jruby itself which verifies that the
jopenssl.jar gets loaded (to late to find serach for commits now).

what I found with getting websphere to work with a simple rails app that
a
gem 'jruby-openssl'
gem 'bouncy-castle-java'
helped to find those jars from the gems (websphere can not load those
embedded jars at all).

unfortunately I was still not able to reproduce your failing PEM. but
the
default gems should be just gems and not given preference IMO. I will
double-check what the other jruby folks think, about that.

thanx for digging further.

regards,
christian

PS did not know about the $CLASSPATH varibale - nice to have ;)
B05d3cbc64b0031a24c2887fb6ddc173?d=identicon&s=25 christian (Guest)
on 2014-03-17 15:39
(Received via mailing list)
OK my last email was not really helpful - wasn't my day.

loading jars from within jruby-complete.jar works even for you since you
get some errors from within the jopenssl.jar (when I understood one of
the
many remarks right).

the problem with this piece of code is that the actual error gets
interpreted as "it can not load the PEM" which makes debugging really
hard.
for this extracted the piece of code of jopenssl.jar into that
rsa_from_file.rb script:

require 'openssl'
file_name = 'mykey.pem'
pwd = 'TestPW'
begin
  x = OpenSSL::PKey::RSA.new(File.read(file_name), pwd)
  puts x.inspect
  p x.class
rescue Exception => e
  puts "e.message = #{e.message}"
  puts "obj = " + e.backtrace.join("\n")
end

begin
  p
org.jruby.ext.openssl.x509store.PEMInputOutput.readPrivateKey(java.io.StringReader.new(File.read(file_name).to_java()),
pwd.to_java.to_char_array)
rescue => e
  puts "e.message = #{e.message}"
  puts "obj = " + e.backtrace.join("\n")
end


that PEMInputOutput.readPrivateKey is the one which reads the PEM file
correctly on my machine and I assume that method has a problem on your
machine. maybe the stacktrace helps to see the problem.

I also made an issue on github to clearify the loading of gems, default
gems, etc
https://github.com/jruby/jruby/issues/1566

- christian
1272e095e145697bb0371230688bd2c3?d=identicon&s=25 Michael Pitman (mcp)
on 2014-03-17 22:27
(Received via mailing list)
Thanks for following up on that - was trying some other things myself,
and
found the explicitly requiring the bouncy castle jar files using an
absolute path, before requiring openssl got things to work, but it's not
a
very satisfying work-around.

Here's the result of that script (I explicitly added the gems at the top
to
get it to still work with 1.7.9, so the line number will be off a bit
from
your script, but the error messages don't change):

Under 1.7.10:

e.message = problem creating RSA private key:
java.security.InvalidKeyException: Wrong algorithm: DESede or TripleDES
required
obj =
org.jruby.ext.openssl.x509store.PEMInputOutput.readPrivateKey(PEMInputOutput.java:291)
sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
java.lang.reflect.Method.invoke(Method.java:606)
org.jruby.javasupport.JavaMethod.invokeDirectWithExceptionHandling(JavaMethod.java:470)
org.jruby.javasupport.JavaMethod.invokeStaticDirect(JavaMethod.java:372)
org.jruby.java.invokers.StaticMethodInvoker.call(StaticMethodInvoker.java:70)
org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:346)
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:204)
$_dot_.rsa_test2.chained_2_rescue_2$RUBY$SYNTHETIC__file__(./rsa_test2.rb:18)
$_dot_.rsa_test2.__file__(./rsa_test2.rb:17)
$_dot_.rsa_test2.load(./rsa_test2.rb)
org.jruby.Ruby.runScript(Ruby.java:811)
org.jruby.Ruby.runScript(Ruby.java:804)
org.jruby.Ruby.runNormally(Ruby.java:673)
org.jruby.Ruby.runFromMain(Ruby.java:522)
org.jruby.Main.doRunFromMain(Main.java:395)
org.jruby.Main.internalRun(Main.java:290)
org.jruby.Main.run(Main.java:217)
org.jruby.Main.main(Main.java:197)


And jruby-9000:

e.message = problem creating RSA private key:
java.security.InvalidKeyException: Wrong algorithm: DESede or TripleDES
required
obj = java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:606)
$_dot_.rsa_test2.(root)(./rsa_test2.rb:18)
$_dot_.rsa_test2.(root)($_dot_/./rsa_test2.rb:18)
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.