Forum: Ruby JRuby, ffi, bad file descriptor

Hi,

JRuby 1.3.1
OS X 10.4.9

The following code is a port of a tempfile library I wrote. Ruby 1.8.6
and 1.9.x work alright, but with JRuby I'm getting a bad file descriptor
error.

Any ideas? Here's the code:

require 'ffi'

class FileTemp < File
    extend FFI::Library

    attach_function 'fileno',  [:pointer], :int
    attach_function 'fclose',  [:pointer], :int
    attach_function 'mkstemp', [:string],  :int
    attach_function 'tmpfile', [],         :pointer
    attach_function 'tmpnam',  [:string],  :string
    attach_function 'umask',   [:int],     :int

    TMPDIR = ENV['TEMP'] || ENV['TMP'] || '/tmp'

    def initialize(delete = true, template = 'rb_file_temp_XXXXXX')
       @fptr = nil

       if delete
          @fptr = tmpfile()
          fd = fileno(@fptr)
       else
          begin
             omask = umask(077)
             fd = mkstemp(template)
             raise SystemCallError, 'mkstemp()' if fd < 0
          ensure
             umask(omask)
          end
       end

       super(fd, 'wb+')
    end

    def close
       super
       fclose(@fptr) if @fptr
    end

    def self.temp_name
       TMPDIR + tmpnam(nil) << '.tmp'
    end
end

if $0 == __FILE__
    fh = FileTemp.new
    fh.print 'hello'
    fh.close
end

Result:

temp.rb:42:in `initialize': Bad file descriptor - Bad file descriptor
(Errno::EBADF)
         from temp.rb:56:in `new'
         from temp.rb:56

Regards,

Dan

Hmm, did you post this to JRuby dev list or ruby-ffi list yet? Could
be something simple...

You might also try getting a raw Java backtrace by passing this flag to 
JRuby:

-J-Djruby.backtrace.style=raw

That will show us if it's happening in JRuby's IO code or in FFI 
somewhere.

- Charlie

On Aug 2, 2:32 am, Charles Oliver Nutter <head...@headius.com> wrote:
> Hmm, did you post this to JRuby dev list or ruby-ffi list yet? Could
> be something simple...
>
> You might also try getting a raw Java backtrace by passing this flag to JRuby:
>
> -J-Djruby.backtrace.style=raw
>
> That will show us if it's happening in JRuby's IO code or in FFI somewhere.

I tried to post to jruby-dev, but I guess you can't post directly to
the list via google groups.

Here's the output using the flag you suggested:

daniel-bergers-computer:~/Documents/workspace/file-temp/lib/file
djberge$ jruby -J-Djruby.backtrace.style=raw temp.rb
Thread.java:1409:in `getStackTrace': Bad file descriptor - Bad file
descriptor (Errno::EBADF)
        from RubyException.java:139:in `setBacktraceFrames'
        from RaiseException.java:160:in `setException'
        from RaiseException.java:72:in `<init>'
        from Ruby.java:3033:in `newRaiseException'
        from Ruby.java:2789:in `newErrnoEBADFError'
        from RubyIO.java:895:in `initialize'
        from RubyFile.java:405:in `initialize'
        from org/jruby/RubyFile$i_method_0_2$RUBYFRAMEDINVOKER
$initialize.gen:-1:in `call'
        from DynamicMethod.java:176:in `call'
        from SuperCallSite.java:344:in `cacheAndCall'
        from SuperCallSite.java:188:in `callBlock'
        from SuperCallSite.java:193:in `call'
        from temp.rb:31:in `method__1$RUBY$initialize'
        from temp#initialize:-1:in `call'
        from CachingCallSite.java:238:in `cacheAndCall'
        from CachingCallSite.java:46:in `callBlock'
        from CachingCallSite.java:51:in `call'
        from RubyClass.java:593:in `newInstance'
        from RubyIO.java:822:in `newInstance'
        from org/jruby/RubyIO$s_method_0_0$RUBYFRAMEDINVOKER
$newInstance.gen:-1:in `call'
        from DynamicMethod.java:160:in `call'
        from DynamicMethod.java:156:in `call'
        from CachingCallSite.java:258:in `cacheAndCall'
        from CachingCallSite.java:77:in `call'
        from temp.rb:45:in `__file__'
        from temp.rb:-1:in `load'
        from Ruby.java:592:in `runScript'
        from Ruby.java:514:in `runNormally'
        from Ruby.java:360:in `runFromMain'
        from Main.java:268:in `run'
        from Main.java:113:in `run'
        from Main.java:97:in `main'

Regards,

Dan

Got the source of that temp.rb handy? It looks like it's passing a
bad/closed descriptor to IO.new or something...

On Aug 2, 10:34 pm, Charles Oliver Nutter <head...@headius.com> wrote:
> Got the source of that temp.rb handy? It looks like it's passing a
> bad/closed descriptor to IO.new or something...

require 'ffi'

class FileTemp < File
   extend FFI::Library

   attach_function 'fileno',  [:pointer], :int
   attach_function 'fclose',  [:pointer], :int
   attach_function 'mkstemp', [:string],  :int
   attach_function 'tmpfile', [],         :pointer
   attach_function 'tmpnam',  [:string],  :string
   attach_function 'umask',   [:int],     :int

   TMPDIR = ENV['TEMP'] || ENV['TMP'] || '/tmp'

   def initialize(delete = true, template = 'rb_file_temp_XXXXXX')
      @fptr = nil

      if delete
         @fptr = tmpfile()
         fd = fileno(@fptr)
      else
         begin
            omask = umask(077)
            fd = mkstemp(template)
            raise SystemCallError, 'mkstemp()' if fd < 0
         ensure
            umask(omask)
         end
      end

      super(fd, 'wb+')
   end

   def close
      super
      fclose(@fptr) if @fptr
   end

   def self.temp_name
      TMPDIR + tmpnam(nil) << '.tmp'
   end
end

if $0 == __FILE__
   fh = FileTemp.new
   fh.print 'hello'
   fh.close
end

Ahhh. Yeah I see the problem.

In JRuby, because we don't normally have access to the "real" file
descriptor for any IO channel, all our logic for fileno is basically
fake. We keep an artificial list of numbers that map to IO channels
and use that as our file descriptor table. In this case, you're
pulling in a real file descriptor from the system, which does not
exist in our table, so we raise an error.

In order for us to support arbitrary descriptors in our IO we'd
probably need an implementation of IO that used all low-level C APIs
rather than Java IO APIs. There's currently no (public) way to get
from a file descriptor to an IO channel in Java APIs.

So I'll divert this issue by asking: what does this get you that our
built-in tempfile support does not? We do not use MRI's tempfile.rb;
we've implemented our own on top of Java's tempfile support that
performs quite a bit better. Perhaps there's a missing feature we can
add.

It would also be worth discussing this with Wayne Meissner. It would
certainly be *nice* if we could transparently support real file
descriptors, and he may know a way to do it.

On 3 Aug 2009, at 14:06, Charles Oliver Nutter wrote:
> probably need an implementation of IO that used all low-level C APIs
> certainly be *nice* if we could transparently support real file
> descriptors, and he may know a way to do it.

I'd be interested in support for that as well if it's possible: a lot
of fun tricks on Unix platforms rely on misappropriating file
descriptors and injecting them into IO objects.


Ellie

Eleanor McHugh
Games With Brains
http://slides.games-with-brains.net
----
raise ArgumentError unless @reality.responds_to? :reason

On Aug 3, 7:06 am, Charles Oliver Nutter <head...@headius.com> wrote:
> probably need an implementation of IO that used all low-level C APIs
> rather than Java IO APIs. There's currently no (public) way to get
> from a file descriptor to an IO channel in Java APIs.

Ok, that's unfortunate.

> So I'll divert this issue by asking: what does this get you that our
> built-in tempfile support does not? We do not use MRI's tempfile.rb;
> we've implemented our own on top of Java's tempfile support that
> performs quite a bit better. Perhaps there's a missing feature we can
> add.

Where is it? I looked through the source but didn't see it.

> It would also be worth discussing this with Wayne Meissner. It would
> certainly be *nice* if we could transparently support real file
> descriptors, and he may know a way to do it.

That would be good. :)

Regards,

Dan

Daniel Berger wrote:
> On Aug 3, 7:06 am, Charles Oliver Nutter <head...@headius.com> wrote:
>> So I'll divert this issue by asking: what does this get you that our
>> built-in tempfile support does not? We do not use MRI's tempfile.rb;
>> we've implemented our own on top of Java's tempfile support that
>> performs quite a bit better. Perhaps there's a missing feature we can
>> add.
> Where is it? I looked through the source but didn't see it.

The mapping for Ruby Classes in JRuby is Foo -> org.jruby.RubyFoo, so
Tempfile is in src/org/jruby/RubyTempfile.java:

  https://GitHub.Com/JRuby/JRuby/blob/master/src/org/jruby/RubyTempfile.java

I find it a bit hard to get around the project structure, too,
especially since I'm not used to navigating big Java projects. (Also,
the fact that the directory structure has "organically" grown over,
what, 9(?) years now doesn't help, although it's still not 10% as bad
as YARV.)

jwm