FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7 and 1.9


#1

The JRuby team is proud to announce the release of FFI for Ruby 1.8.6/7
and 1.9!

FFI (gem install ffi) is a library for programmatically loading dynamic
libraries, binding functions within them, and calling those functions
from Ruby code. Here’s a quick sample of binding and calling the getpid
C library function:

require ‘ffi’

module GetPid
extend FFI::Library

attach_function :getpid, [], :uint
end

puts GetPid.getpid

Here’s another, calling qsort and passing a Ruby block as a C callback:

require ‘ffi’

module LibC
extend FFI::Library
callback :qsort_cmp, [ :pointer, :pointer ], :int
attach_function :qsort, [ :pointer, :int, :int, :qsort_cmp ], :int
end

p = MemoryPointer.new(:int, 2)
p.put_array_of_int32(0, [ 2, 1 ])
puts “Before qsort #{p.get_array_of_int32(0, 2).join(’, ')}”
LibC.qsort(p, 2, 4) do |p1, p2|
i1 = p1.get_int32(0)
i2 = p2.get_int32(0)
i1 < i2 ? -1 : i1 > i2 ? 1 : 0
end
puts “After qsort #{p.get_array_of_int32(0, 2).join(’, ')}”

I posted a blog entry with a longer description of the library,
additional examples, and links to some other documentation and posts.
Docs are a little slim at this point, so feel free to experiment and
update the JRuby wiki page:

http://wiki.jruby.org/wiki/Calling_C_from_JRuby

I’m sure docs from here will filter back into the library and out into
the general cosmos.

Finally, there’s no need to write a C extension to call C libraries, and
the same FFI code will work in Ruby 1.8.6/7, Ruby 1.9, JRuby 1.1.4+, and
Rubinius (though Rubinius has no callback support yet).

Don’t be an extension stooge! Use FFI!

  • Charlie

#2
  • Charles Oliver N. (removed_email_address@domain.invalid) wrote:

The JRuby team is proud to announce the release of FFI for Ruby
1.8.6/7 and 1.9!

Finally, there’s no need to write a C extension to call C libraries,
and the same FFI code will work in Ruby 1.8.6/7, Ruby 1.9, JRuby
1.1.4+, and Rubinius (though Rubinius has no callback support yet).

Awesome. I’ve used DL to link up some custom libs to a Ruby service,
will give FFI a go and see how it compares :slight_smile:

Are things like structs likely to be supported in future, ala Python
ctypes?


#3

Thomas H. wrote:

will give FFI a go and see how it compares :slight_smile:

Are things like structs likely to be supported in future, ala Python
ctypes?

Actually structs are already supported! See the blog post, and I believe
there’s some examples shipped with the gem. There needs to be more docs,
certainly, and hopefully they’ll get some TLC soon.

Also, I forgot to call out Evan P. for coming up with the API and
initial design, and he or someone else on Rubinus wrote up the
templating/header-file-processing stuff as well. And of course a huge
thanks to Wayne M. for implementing FFI not just once (for JRuby)
but twice (for C Ruby). His work will mean a huge leap forward in
cross-impl portability.

  • Charlie

#4

On Sat, Nov 1, 2008 at 6:40 AM, Charles Oliver N.
removed_email_address@domain.invalid wrote:

Actually structs are already supported! See the blog post, and I believe
there’s some examples shipped with the gem.

Your blog is preaty known but for clarity:
http://blog.headius.com/2008/10/ffi-for-ruby-now-available.html
:slight_smile:


Rados³aw Bu³at

http://radarek.jogger.pl - mój blog


#5

Radosław Bułat wrote:

On Sat, Nov 1, 2008 at 6:40 AM, Charles Oliver N.
removed_email_address@domain.invalid wrote:

Actually structs are already supported! See the blog post, and I believe
there’s some examples shipped with the gem.

Your blog is preaty known but for clarity:
http://blog.headius.com/2008/10/ffi-for-ruby-now-available.html
:slight_smile:

Oh right, a link would have been useful. Thank you :slight_smile:

  • Charlie

#6

Charles Oliver N. wrote:

I posted a blog entry with a longer description of the library,
additional examples, and links to some other documentation and posts.

And then I completely forgot to include the blog post URL…

http://blog.headius.com/2008/10/ffi-for-ruby-now-available.html

  • Charlie

#7

Luc H. wrote:

through FFI as opposed to using a compiled extension?
Wayne answers the latter, sorta, on his followup blog post:

http://blog.headius.com/2008/10/ffi-for-ruby-now-available.html

He doesn’t have specific numbers for performance at the moment, but the
short story is that FFI introduces a bit of overhead; ultimately I
believe that the overhead gets lost in the flow of a Ruby application,
especially when you’re tossing units of work across like SQL queries or
arrays. Wayne probably can fill in more details on what the actual
overhead is like.

And I’d also expect that any small amount of overhead is vastly
outweighed by the ability to write an FFI-based library once and use it
across implementations.

  • Charlie

#8

On 1 nov. 08, at 12:16, Charles Oliver N. wrote:

He doesn’t have specific numbers for performance at the moment, but
the short story is that FFI introduces a bit of overhead; ultimately
I believe that the overhead gets lost in the flow of a Ruby
application, especially when you’re tossing units of work across
like SQL queries or arrays.

— [begin unscientific test] --------

require ‘rubygems’
require ‘benchmark’
require ‘zlib’
require ‘ffi’
require ‘dl/import’

module Zlib_ffi
extend FFI::Library
attach_function :zlib_version, :zlibVersion, [], :string
end

module Zlib_dl
extend DL::Importable
dlload “libz.dylib”
extern “const char* zlibVersion()”
end

puts Zlib.zlib_version
puts Zlib_ffi.zlib_version
puts Zlib_dl.zlibVersion

Benchmark.bm(3) do |bm|
bm.report(“ext”) { 500_000.times { Zlib.zlib_version } }
bm.report(“ffi”) { 500_000.times { Zlib_ffi.zlib_version } }
bm.report(“dl”) { 500_000.times { Zlib_dl.zlibVersion } }
end

— [end unscientific test] --------

This gives the following results:

1.2.3
1.2.3
1.2.3
user system total real
ext 1.050000 0.320000 1.370000 ( 1.373800)
ffi 2.160000 0.660000 2.820000 ( 2.818966)
dl 3.500000 1.060000 4.560000 ( 4.552789)

All this using MacPorts MRI 1.8.7-p72 under OS X 10.5.5. The observed
overhead is slightly over 2x for ffi, probably not a big deal unless
ffi calls are used in tight loops I guess.

PS: haven’t seen any trace of variadic function support in the code.


#9

On 1 nov. 08, at 11:19, Charles Oliver N. wrote:

And then I completely forgot to include the blog post URL…

Looks cool, great work.

Two questions:

  • Are variadic functions supported?
  • Do you have any idea or measurements of the overhead of calling
    through FFI as opposed to using a compiled extension?

#10

Where can I file an issue? I have trouble to build in with ruby1.9 on
64bit.


Rados³aw Bu³at

http://radarek.jogger.pl - mój blog


#11

Luc H. wrote:

     user     system      total        real

ext 1.050000 0.320000 1.370000 ( 1.373800)
ffi 2.160000 0.660000 2.820000 ( 2.818966)
dl 3.500000 1.060000 4.560000 ( 4.552789)

All this using MacPorts MRI 1.8.7-p72 under OS X 10.5.5. The observed
overhead is slightly over 2x for ffi, probably not a big deal unless ffi
calls are used in tight loops I guess.

Seems like that pretty well seals the deal for ffi over dl, at the very
least. I’m also glad to see FFI wasn’t even that bad, especially
considering it hasn’t received any optimization.

  • Charlie

#12

This gives the following results:

1.2.3
1.2.3
1.2.3
user system total real
ext 1.050000 0.320000 1.370000 ( 1.373800)
ffi 2.160000 0.660000 2.820000 ( 2.818966)
dl 3.500000 1.060000 4.560000 ( 4.552789)

Ubuntu 8.10 64bit

$ ruby --version && ruby ffi_bench.rb
ruby 1.8.7 (2008-08-11 patchlevel 72) [x86_64-linux]
1.2.3.3
1.2.3.3
1.2.3.3
user system total real
ext 0.320000 0.070000 0.390000 ( 0.396774)
ffi 0.770000 0.120000 0.890000 ( 0.895093)
dl 2.090000 0.270000 2.360000 ( 2.365029)


Rados³aw Bu³at

http://radarek.jogger.pl - mój blog


#13

Radosław Bułat wrote:

dl 2.090000 0.270000 2.360000 ( 2.365029)
Seems like about the same ratio…probably can be improved too!

  • Charlie

#14

Radosław Bułat wrote:

Where can I file an issue? I have trouble to build in with ruby1.9 on 64bit.

Hop on the ruby-ffi project mailing lists, and I guess you can file a
bug in the tracker there too:

http://kenai.com/projects/ruby-ffi

  • Charlie

#15

Sylvain J. wrote:

using dyncall to do the actual interfacing work (http://www.dyncall.org/) in a
DL-replacement library, but my problem is that dyncall does not like Linux-PPC.
What are you using on your side ?

Ruby FFI uses libffi, as does JNA which ships with JRuby. I’m not
certain about libffi specifically. but JNA claims to support OSX (ppc,
x86, x86_64), linux (x86, amd64), FreeBSD/OpenBSD (x86, amd64), Solaris
(x86, amd64, sparc, sparcv9) and Windows (x86, amd64).

I’m not sure if linux-ppc support is not provided because it’s not
supported or because nobody has a linux-ppc machine to build on. The
latter has been the case for several entries on the list; I myself was
the build monkey for Solaris/AMD64 and Linux/AMD64 for a short time,
before which there was no shipped support.

There’s certainly one way to find out…gem install ffi. Report back
here or on ruby-ffi mailing lists what you learn :slight_smile:

http://kenai.com/projects/ruby-ffi

  • Charlie

#16

On Sat, Nov 01, 2008 at 08:16:39AM +0900, Charles Oliver N. wrote:

The JRuby team is proud to announce the release of FFI for Ruby 1.8.6/7
and 1.9!

FFI (gem install ffi) is a library for programmatically loading dynamic
libraries, binding functions within them, and calling those functions
from Ruby code. Here’s a quick sample of binding and calling the getpid
C library function:

Interesting. On what kind of architectures is the binding part working ?
I’m
using dyncall to do the actual interfacing work
(http://www.dyncall.org/) in a
DL-replacement library, but my problem is that dyncall does not like
Linux-PPC.
What are you using on your side ?

Sylvain


#17

Interesting. On what kind of architectures is the binding part working ? I’m
using dyncall to do the actual interfacing work (http://www.dyncall.org/) in a
DL-replacement library, but my problem is that dyncall does not like Linux-PPC.
What are you using on your side ?

Ruby FFI uses libffi, as does JNA which ships with JRuby. I’m not
certain about libffi specifically. but JNA claims to support OSX (ppc,
x86, x86_64), linux (x86, amd64), FreeBSD/OpenBSD (x86, amd64), Solaris
(x86, amd64, sparc, sparcv9) and Windows (x86, amd64).
I did not know about libffi … I’ll have to look if I should not
replace
dyncall by libffi then. Thanks for the info.

I’m not sure if linux-ppc support is not provided because it’s not
supported or because nobody has a linux-ppc machine to build on.
I guess it is a bit of both. Given that libffi is supported on
Linux/PPC64, I
guess having it on Linux/PPC should not be that much of a problem – but
still,
I think the calling convention can be slightly different between the two
architectures.

Sylvain


#18

On Mon, 3 Nov 2008 05:01:11 -0500, Charles Oliver N. wrote:

I’m not sure if linux-ppc support is not provided because it’s not
supported or because nobody has a linux-ppc machine to build on. The
latter has been the case for several entries on the list; I myself was
the build monkey for Solaris/AMD64 and Linux/AMD64 for a short time,
before which there was no shipped support.

There’s certainly one way to find out…gem install ffi. Report back
here or on ruby-ffi mailing lists what you learn :slight_smile:

Ubuntu 8.10 provides libffi on ppc. The gem builds, but a simple ruby
program hangs (rather uses 100% CPU).

I can investigate further (e.g building my own libffi), but that may
not happen immediately.

-jh


#19

We (JRuby and Rubinius contributors) designed in the ability to
specify calling convention for a given library bound through FFI.
Largely this was needed to support Win32’s stdcall, but if Linux/PPC
has a different convention and libffi supports it, then FFi will too
(though we may have to add it to the list of accepted conventions).

I’ll check on the syntax we’re supporting and get back to ya.

  • Charlie (mobile)

On Nov 3, 2008, at 12:56, Sylvain J.
<removed_email_address@domain.invalid


#20

Charles Oliver N. wrote:

We (JRuby and Rubinius contributors) designed in the ability to specify
calling convention for a given library bound through FFI. Largely this
was needed to support Win32’s stdcall, but if Linux/PPC has a different
convention and libffi supports it, then FFi will too (though we may have
to add it to the list of accepted conventions).

I’ll check on the syntax we’re supporting and get back to ya.

In the body of the module you’re binding:

ffi_convention :stdcall # :default is for normal C convention

We could add whatever convention Linux/PPC needs.

  • Charlie