Forum: JRuby too many open files with imap connections

Running into an issue with jruby throwing too many open files when
opening imap connections.  I am not sure, but I have a hunch it's an
issue specifically with ssl connections.  Able to reproduce with this
script:

require 'net/imap'
10000.times.each do |i|
   puts "connection #{i}"
   imap = Net::IMAP.new('imap.gmail.com', 993, true)
   imap.disconnect
end

Depending on OS settings, eventually java raises:

OpenSSL::SSL::SSLError: Too many open files
  from /home/ubuntu/.rvm/rubies/jruby-1.5.1/lib/ruby/1.8/net/imap.rb:
904:in `initialize'
  from (irb):4:in `new'
  from (irb):4

I am running

jruby 1.5.1
java version "1.6.0_18"
OpenJDK Runtime Environment (IcedTea6 1.8) (6b18-1.8-0ubuntu1)
OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)

Thx.

-karl

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

On Aug 15, 2010, at 9:15 PM, Karl Baum wrote:

> 
> OpenJDK 64-Bit Server VM (build 14.0-b16, mixed mode)
> 
> Thx.
> 
> -karl
> 
> 

Karl,

This should not be a surprise. All computers have finite resources, and 
if you exhaust them, you'll run into problems.

Specifically, the "problem" (in quotation marks) manifests itself with 
MRI as well.

Cheers.
---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

beside the limit of resources net-ldap indeed does not close the
connections which is patch on svn head:

http://rubyforge.org/tracker/index.php?func=detail&aid=24178&group_id=143&atid=633

regards Kristian


On Mon, Aug 16, 2010 at 9:13 AM, Hirotsugu Asari <asari.ruby@gmail.com> 
wrote:
>> end
>> jruby 1.5.1
> Karl,
>
>
>

---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

For kicks, I modified your script as follows:

10000.times.each do |i|
  puts "connection #{i}"
  imap = Net::IMAP.new('imap.gmail.com', 993, true)
  imap.disconnect
  `lsof > imap-#{i}.txt` if (i % 1000) == 0
end

With my Mac OS 10.6, I could not reproduce the problem. Did you see this
problem with a Sun JVM ?

However, the lsof output shows a lot of pipes left open. I can attach 
them
to a bug if you open one.

Thanks,

Antoine

Tried this with ruby enterprise 1.8.7 and I did not run into the
issue.  Will also give it a go with MRI later today.

In regards to it not being a surprise, I'm not sure I understand what
you are saying.  Since we are opening and immediately closing the
connection, I would think things would be getting cleaned up.

thx

-karl

On Aug 15, 2010, at 11:43 PM, Hirotsugu Asari wrote:

>> puts "connection #{i}"
>>   from (irb):4
>> -karl
>
> Cheers.
> ---------------------------------------------------------------------
> To unsubscribe from this list, please visit:
>
>    http://xircles.codehaus.org/manage_email
>
>


---------------------------------------------------------------------
To unsubscribe from this list, please visit:

    http://xircles.codehaus.org/manage_email

Installed the sun jdk on ubuntu and ran into the same issue.  Will try
on OSX later on this morning.

Thanks!

Ran into the same issue on OSX:

Mac OSX 10.5.8
Jruby 1.5.1
java version "1.6.0_17"
Java(TM) SE Runtime Environment (build 1.6.0_17-b04-248-9M3125)
Java HotSpot(TM) 64-Bit Server VM (build 14.3-b01-101, mixed mode)

On ruby 1.8.7 I did not have the issue: ruby 1.8.7 (2010-01-10
patchlevel 249) [i686-darwin9.8.0])

Antoine, I am surprised that you did not run into the issue on OSX.
My only guess is that I am not yet running OSX 10.6 as i have been
lazy about upgrading.  How long did you let it run for?  When you run
"lsof -p <pid> | wc -l" do you see the open file descriptors actually
go back down?  I will get one of my colleagues with OSX 10.6 to try
and reproduce.

thx.

It took at least 10 minutes. In my script, I captured the lsof output 
every
1000 connections. The result shows exponentially more imap connections
opened.

I fiddled with my mac quite a bit and I might have managed to change the
soft limit of open files, or I was just lucky not to crash. In any case,
this is a bug for sure, and I can show some lsof output to prove it.

Thanks,

Antoine

Should I open up a jruby ticket for this?

Also, any ideas of ways i can work around this in the time being?

Thanks!

-karl

Please do. Not sure you can work around it for now.

Created a ticket:

http://jira.codehaus.org/browse/JRUBY-5015

thx.

-karl