Forum: JRuby out of memory error with file IO expected?

Bec38d63650c8912b6ba9b557fb953b9?d=identicon&s=25 Roger Pack (rogerdpack)
on 2013-07-19 19:31
Hello, I noticed when running this code, with 1.7.4:

 100.times { Thread.new { loop { File.open('big', 'w') do |f| f.seek
10_000_000_000; f.puts 'a'; end}}}

after awhile, I got this failure in all threads.  Perhaps the
"directBuffer" is not being freed on close or something, and could be?

http://rxr.whitequark.org/jruby/source/src/org/jru...


 Exception in thread "RubyThread-58: (irb):1"
java.lang.OutOfMemoryError: Direct buffer memory
        at java.nio.Bits.reserveMemory(Bits.java:633)
        at java.nio.DirectByteBuffer.<init>(DirectByteBuffer.java:95)
        at java.nio.ByteBuffer.allocateDirect(ByteBuffer.java:288)
        at
org.jruby.util.io.ChannelDescriptor.<init>(ChannelDescriptor.java:663)
        at
org.jruby.util.io.ChannelDescriptor.<init>(ChannelDescriptor.java:205)
        at
org.jruby.util.io.ChannelDescriptor.open(ChannelDescriptor.java:915)
        at org.jruby.RubyFile.sysopen(RubyFile.java:1242)
        at org.jruby.RubyFile.sysopenInternal(RubyFile.java:1199)
        at org.jruby.RubyFile.openFile19(RubyFile.java:1150)
        at org.jruby.RubyFile.initialize19(RubyFile.java:334)
        at
org.jruby.RubyFile$INVOKER$i$0$2$initialize19.call(RubyFile$INVOKER$i$0$2$initialize19.gen)
        at
org.jruby.runtime.callsite.CachingCallSite.callBlock(CachingCallSite.java:79)
        at
org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:85)
        at org.jruby.RubyClass.newInstance(RubyClass.java:876)
        at org.jruby.RubyIO.open(RubyIO.java:1144)
F1d37642fdaa1662ff46e4c65731e9ab?d=identicon&s=25 Charles Nutter (headius)
on 2013-07-19 21:59
(Received via mailing list)
I believe there was a bug reported where we were not freeing some direct
byte buffers that appears to be fixed in 1.7.5. Can you try master?
486ca04f06d968004643ce5b47376ded?d=identicon&s=25 Keith B. (keith_b)
on 2013-07-19 22:23
(Received via mailing list)
I would be surprised if that worked fine. ;)

You're opening the same file in 100 different threads, and writing to it
simultaneously.  There's no guarantee that any given thread's operation
will not be interrupted.

You'd normally want to synchronize access to a resource like that, by
having the many threads post something to a queue, and having 1 thread
reading from the queue and doing the writes.

- Keith
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.