Forum: JRuby JRuby PermGen. Out of memory. Heap space memory leaks

Cd9ea4356e673d8ceb9f3ed943e374c3?d=identicon&s=25 Veniamin Z. (veniamin_z)
on 2012-06-22 15:47
We are working on a big project. During our development we faced with a
big a problem related to Heap and PermGen Space. Here our error message:
Current error:  java.lang.OutOfMemoryError: PermGen space.


Ways to solution
1.  Increase PermGen space
PermGen space was increased to 4Gb. Soon it became full too.

Conclusion: This mean that just increasing available space isn’t a
solution of this hot problem.



2.  Using Red Bridge (JRuby Embed) Core, JSR223, BSF
We had try to use different types of Red Bridge (Embed JRuby): Core,
JSR223, BSF. But, actually, there wasn’t any affect to prevent
increasing PermGen space. Here results of our investigation:



….
System.out.println("Initialazing..");
//Spring applicaton context
WebApplicationContext wac = (WebApplicationContext)
AppContext.getApplicationContext();
// prepare path to internal ruby
String scriptsPath = wac.getServletContext().getRealPath(RUBY_PATH);
              String jrubyHome =
wac.getServletContext().getRealPath("WEB-INF" + File.separator +
"jruby");
// Initializing Scripting container
ScriptingContainer container = new ScriptingContainer(isShared ?
LocalContextScope.SINGLETHREAD
                    : LocalContextScope.THREADSAFE,
LocalVariableBehavior.PERSISTENT);
// Configuring scriptingcontainer to avoid memory leaks
              container.setCompileMode(RubyInstanceConfig.CompileMode.OFF);
System.setProperty("org.jruby.embed.compilemode", "OFF");
System.setProperty("jruby.compile.mode", "OFF");
// Setup ruby version
container.setCompatVersion(CompatVersion.RUBY1_9);
// Set jruby home
              container.getProvider().getRubyInstanceConfig().setJRubyHome(jrubyHome);
List<String> loadPaths = new ArrayList<String>();
// load path
loadPaths.add(scriptsPath);
container.getProvider().setLoadPaths(loadPaths);
// ruby dispatcher initializing and run in simple mood
String fileName = scriptsPath + File.separator + "dispatcher_fake.rb";
// run scriplet
container.runScriptlet(PathType.ABSOLUTE, fileName);
// terminate container to cleanup memory without any effects
container.terminate();
container=null;

Description: above code create and configure Scripting container. This
methods running in separate thread. We use 4 threads of ruby runs. If we
use the same scripting container and call internal scriptlets (call
internall methods in java  threading) we get issues with ruby variables
because it is visible cross threading.

Main issue with JRuby are: growing heap memory space and perm gen memory
space. We can`t call any system`s garbage collection in the ruby code.


Bellow you can found simple parts of our scriptlet:
Ruby:

ENV['GEM_PATH'] = File.expand_path('../../jruby/1.9', __FILE__)
ENV['GEM_HOME'] = File.expand_path('../../jruby/1.9', __FILE__)
ENV['BUNDLE_BIN_PATH'] =
File.expand_path('../../jruby/1.9/gems/bundler-1.0.18/bin/bundle',
__FILE__)

require 'java'
require 'rubygems'
require "bundler/setup"
require 'yaml'
require 'mechanize'
require 'spreadsheet'
require 'json'
require 'rest-client'
require 'active_support/all'
require 'awesome_print'
require 'csv'
require 'builder'
require 'soap/wsdlDriver' rescue nil
ROOT_DIR = File.dirname(__FILE__)
require File.join(ROOT_DIR, "base", "xsi.rb")


import org.jsoup.Jsoup
import org.jsoup.nodes.Document
import org.jsoup.nodes.Element


module JavaListing
  include_package "com.homeaway.psplatform.util.listing"
end

class A
  include JavaListing
  def run
    1000.times do |index|
      puts "iterating #{index}"
      prop = JavaListing::Property.new
      prop.proNo = 111
      prop.remoteID = "1111"
      prop.ownerID = "1111"
      prop.advertiserID = "1111"
      prop.title = "Atite"
      prop.summary = "Asummury"
      prop.description = "Adescription"
      # prop.images << JavaListing::Image.new("111", "Acaption")
      prop.lat = 12.23
      prop.lng = 13.21
      #prop.address = JavaListing::Address.new("Acity", "Acountry")
      prop.location = "Alocation"
      prop.policy = JavaListing::Policy.new("AcheckinAt", "AcheckoutAt")
      prop.surfaceArea = "Asurfscearea"
      prop.notes[index] = JavaListing::Note.new("Atitle", "Atext")
      prop.order = "Aorder"
      prop.map = JavaListing::Map.new(true, 14)
      prop.units[index] = JavaListing::Unit.new("Aproptype", 2)
      obj = Nokogiri::XML "<root><elements><element>Application Error
#{index}  </element></elements></root>"

    end
  end
end

A.new.run



-  Create properties using JSR223

ScripHelperBase.java


ScriptEngineManager manager = new ScriptEngineManager();
ScriptEngine engine = manager.getEngineByName("jruby");
Reader reader = null;
String fileName = scriptsPath + File.separator + "dispatcher_java.rb";
try {
reader = new FileReader(fileName);
} catch (FileNotFoundException ex) {
Logger.getLogger(ScriptHelperBase.class.getName()).log(Level.SEVERE,
null, ex);
 }
engine.eval(reader);


-  Create properties using BSF

String fileName = scriptsPath + File.separator +
"dispatcher_fake_ruby.rb";
String jrubyhome = "WEB-INF" + File.separator + "jruby";

BSFManager.registerScriptingEngine("jruby",
"org.jruby.embed.bsf.JRubyEngine", new String[]{"rb"});
BSFManager manager = new BSFManager();
manager.setClassPath(jrubyhome);

try {
   manager.exec("jruby", fileName, 0, 0, PathType.ABSOLUTE);
} catch (BSFException ex) {
             Logger.getLogger(ScriptHelperBase.class.getName()).log(Level.SEVERE,
null, ex);
}


Conclusion: The method described above don’t allow to clear needed
memory to original state. It means that every additional script running
still increase filled PermGen space.


3.  Running system using CompileMode=OFF
container.setCompileMode(RubyInstanceConfig.CompileMode.OFF);
System.setProperty("org.jruby.embed.compilemode", "OFF");
System.setProperty("jruby.compile.mode", "OFF");


Conclusion: at first sight seems that PermGen doesn’t overfill, but as a
result after set a bigger loading PermGen error occurs.
Bec38d63650c8912b6ba9b557fb953b9?d=identicon&s=25 Roger Pack (rogerdpack)
on 2012-06-22 19:09
Have you tried analyzing it using tools like jhat to see where the leaks
are occurring?
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.