Out of memory problem with JRuby-Complete

Hi,
I am working on a windows desktop application where I use JRuby for
developing the application and JRuby-Complete to package the application
in a jar. The application uses a fair number of external libraries -
including Qt for the user interface - but so far I have had no trouble
in running the packaged version once the JRuby version worked.
Recently I added functionality to the application to interoperate with
Microsoft Word using WIN32OLE. This works very nicely under JRuby using
the jruby-win32ole-0.8.5 gem, but when I run the same program using
JRuby-Complete, I consistently get the errormessage :

“Can’t QI object for IDispatch”

Inspecting the error condition that leads to this message, it appears
that the interfacing software in the gem runs out of memory ( Windows
error: 0x8007000E ; E_OUTOFMEMORY).
I don’t know if there is a difference in memory allocation policy
between the two JRuby implementations. I have experimented with
explicitly setting heap and stacksizes for JRuby-Complete e.g.

java -Xms256m -Xmx500m -Xss1024k -cp
lib\java\jruby-complete-1.7.13.jar;(other jars) org.jruby.Main
src\jruby_bootstrap.rb

But that made no difference to the error.

Any suggestions how I may solve this problem are very welcome

Cees Z.

Hi,

so now your application is not portable: it’s Windows specific, because
of
win32ole. Worse: you need the software MS word to be installed. This
reduces the number of people who might help you out in this community.

If you still wish to keep this way forward, please have a look at the
memory availabe on your machine, and how pagefile.sys is configured on
your
windows OS. I suggest to contact the win32ole maintainers.

Now there’s real hope: make your application portable on JVM. Try Apache
POI and stay in the java world.

You might still need to increase your heap, and if you’re using jvm 7 or
8
(please use JDK not JRE), you could use java control monitor to have a
look
at your jruby memory allocation.

Good luck!

On Tue, Aug 19, 2014 at 1:12 PM, Cees Z. [email protected]

Christian MICHON wrote in post #1155490:

Hi,

so now your application is not portable: it’s Windows specific, because
of
win32ole. Worse: you need the software MS word to be installed. This
reduces the number of people who might help you out in this community.

If you still wish to keep this way forward, please have a look at the
memory availabe on your machine, and how pagefile.sys is configured on
your
windows OS. I suggest to contact the win32ole maintainers.

Now there’s real hope: make your application portable on JVM. Try Apache
POI and stay in the java world.

You might still need to increase your heap, and if you’re using jvm 7 or
8
(please use JDK not JRE), you could use java control monitor to have a
look
at your jruby memory allocation.

Good luck!

Many thanks for your suggestions.
I did have a look at Apache POI, but it doesn’t quite do what I want; I
need to drag references and notes from the Qt application into existing
Word documents, so positioning, layout and interaction is important.
With WIN32OLE it was relatively easy to implement. The rest of the
application is indeed portable and at a later stage I may add support
for other wordprocessors/platforms. For the moment I would first like to
find out if I can get the JRuby-Complete version to work.

Cees.

On Tue, Aug 19, 2014 at 1:12 PM, Cees Z. [email protected]

I wonder if this issue is related to
native memory leak when reading files from inside a .jar · Issue #1888 · jruby/jruby · GitHub. Basically, there is a memory
leak in JRuby 1.7.13 triggered by loading files from a jar, which
happens quite a lot when you use jruby-complete.jar to run an
application.

If you’re up for building from source, the “jruby-1_7” branch in git has
a fix for this issue.

Ben

Ben B. wrote in post #1155501:

I wonder if this issue is related to
native memory leak when reading files from inside a .jar · Issue #1888 · jruby/jruby · GitHub. Basically, there is a memory
leak in JRuby 1.7.13 triggered by loading files from a jar, which
happens quite a lot when you use jruby-complete.jar to run an
application.

If you’re up for building from source, the “jruby-1_7” branch in git has
a fix for this issue.

Ben
Thank you for pointing this out.
I just downloaded jruby-complete-1.7.12 (which did not have the memory
leak) and tried it, but it has the same problem with .

Cees

Thomas E Enebo wrote in post #1155520:

E_OUTOFMEMORY looks more like a Windows error (via whatever is happening
in
win32ole) than a Java one but you may still want to use much more than
500m
for max heap as a test. I believe we actually specifiy more than that
from
jruby command-line.

Also if you are on 32 bits you are limited to 3.2GB for a whole process
(I
think?). If so then you might want to make sure something is not
leaking?

-Tom

On Tue, Aug 19, 2014 at 6:12 AM, Cees Z. [email protected]
I tried with much larger heapsize (up till getting at -Xmx2048m the
errormessage “Could not reserve enough space for object heap”) with no
success. Somehow it seems to me that the amount of memory available for
“native processes” within WIN32OLE does not depend on the heapsize. If
it was caused by a major memory leak I would expect the same behavior
under JRuby as when using JRuby-Complete

=Cees

Yeah sorry I missed the part where you say this works with a regular
install of JRuby but not with jruby-complete. I agree native memory
will
be competing with the heap for overall process space. I was just
unclear
whether this was really a native memory error or not (although it looked
like it).

Does it work for a while and then give you this error or is it right
away?

-Tom

On Tue, Aug 19, 2014 at 1:22 PM, Cees Z. [email protected]

Hi again,

so you’re using a 32 bits version of the JVM. If you have the
possibility
to use the 64 bits, please do so. Otherwise, you’ll be actually limited
to
~1.5Gb max.

I suggest -Xmx1400m or -Xmx1500m. You should use jvisualvm or jmc
depending
on your JVM version, just to rule out java memory issues and maybe
compare
possibly different behavior between jruby and jruby-complete…

On Tue, Aug 19, 2014 at 8:22 PM, Cees Z. [email protected]

E_OUTOFMEMORY looks more like a Windows error (via whatever is happening
in
win32ole) than a Java one but you may still want to use much more than
500m
for max heap as a test. I believe we actually specifiy more than that
from
jruby command-line.

Also if you are on 32 bits you are limited to 3.2GB for a whole process
(I
think?). If so then you might want to make sure something is not
leaking?

-Tom

On Tue, Aug 19, 2014 at 6:12 AM, Cees Z. [email protected]

Thomas E Enebo wrote in post #1155533:

Yeah sorry I missed the part where you say this works with a regular
install of JRuby but not with jruby-complete. I agree native memory
will
be competing with the heap for overall process space. I was just
unclear
whether this was really a native memory error or not (although it looked
like it).

Does it work for a while and then give you this error or is it right
away?

-Tom

On Tue, Aug 19, 2014 at 1:22 PM, Cees Z. [email protected]
Hi Tom, thanks for spending your time on this.
It fails the very first time (on WIN32OLE.new(‘Word.Application’)) in
org.racob.com.Dispatch.createInstanceNative
it successfully executes a ‘CoCreateInstance’ and then fails in
‘QueryInterface’ returning with hr=0x8007000e (E_OUTOFMEMORY)
Interesting enough, when I look with the Windows Task Manager, each try
creates an instance of WINWORD.EXE*32 which can only be removed by
ending the process with the Window Manager.

-Cees

Christian MICHON wrote in post #1155570:

Hi again,

so you’re using a 32 bits version of the JVM. If you have the
possibility
to use the 64 bits, please do so. Otherwise, you’ll be actually limited
to
~1.5Gb max.

I suggest -Xmx1400m or -Xmx1500m. You should use jvisualvm or jmc
depending
on your JVM version, just to rule out java memory issues and maybe
compare
possibly different behavior between jruby and jruby-complete…

On Tue, Aug 19, 2014 at 8:22 PM, Cees Z. [email protected]
Thanks again.
There was a reason sometime ago why I switched from the 64 bits version
to the 32 bits version. Probably to do with a bug report. I think it is
important anyhow that the distribution set works with the 32 bits
version. I will look into jvisualvm/jmc, but I have little experience
with the Java world.

-Cees

I guess I am not surprised if you run out of memory the win32ole stuff
is
not cleaning up the cocreated process. The fact that this works with
jruby
but fails with jruby-complete is the big clue. For the life of me I
don’t
know what the difference could be however…

I don’t have 32 bit windows nor MS Word so this is very diificult for me
to
try stuff out…

-Tom

On Wed, Aug 20, 2014 at 3:12 AM, Cees Z. [email protected]

After some more trials, it is clear that the E_OUTOFMEMORY problem
depends on how JRuby-Complete is activated:

Calling with a programfile will cause the problem:

java -cp lib\java\jruby-complete-1.7.13.jar;(other jars) org.jruby.Main
src\jruby_bootstrap.rb

While calling it with one-line of script avoids the problem:

java -cp lib\java\jruby-complete-1.7.13.jar;(other jars) org.jruby.Main
-e “require ‘.\src\jruby_bootstrap.rb’”

alternatively, in stead of using org.jruby.Main (the JRuby compiler) the
Ruby program can be started with a small Java program that uses the
org.jruby.embed.ScriptingContainer class. This will also avoid the
problem.

In most cases either method will do to run your Ruby program with
JRuby-Complete. The difference is the amount of native memory available
to your program.

-Cees

Thomas E Enebo wrote in post #1155603:

I guess I am not surprised if you run out of memory the win32ole stuff
is
not cleaning up the cocreated process. The fact that this works with
jruby
but fails with jruby-complete is the big clue. For the life of me I
don’t
know what the difference could be however…

I don’t have 32 bit windows nor MS Word so this is very diificult for me
to
try stuff out…

-Tom

On Wed, Aug 20, 2014 at 3:12 AM, Cees Z. [email protected]

I agree with you that the different behavior between JRuby and
JRuby-Complete must be the big clue, but I ran out of ideas of things to
try. I think I will give this up for the moment and try to find a
work-around.
Thanks again for trying to help find a solution.

N.B. I tried even reducing maximum and initial heapsize with the idea
that it competes with memory allocation for native processes. The main
program runs fine (so it doesn’t need an excessive amount of memory),
but I still run into the same E_OUTOFMEMORY problem when I start a
WIN32OLE session.

-cees