FFI 0.1.1 (Foreign Function Interface) for Ruby 1.8.6/7 and 1.9

Charles Oliver N. wrote:

ffi_convention :stdcall # :default is for normal C convention

We could add whatever convention Linux/PPC needs.

Or, duh, have FFI use the appropriate default convention on the target
platform. Either way, it should be simple to deal with.

  • Charlie

On Tue, Nov 04, 2008 at 04:54:35AM +0900, Charles Oliver N. wrote:

In the body of the module you’re binding:

ffi_convention :stdcall # :default is for normal C convention

We could add whatever convention Linux/PPC needs.

Or, duh, have FFI use the appropriate default convention on the target
platform. Either way, it should be simple to deal with.

Thanks Charlie, but I have already my own library which does more than
Ruby-FFI
(typelib.sf.net and typelib.sf.net/html/ruby) so I’ll probably be using
libffi
directly.

Sylvain

[email protected] wrote:

Ubuntu 8.10 provides libffi on ppc. The gem builds, but a simple ruby
program hangs (rather uses 100% CPU).

I can investigate further (e.g building my own libffi), but that may
not happen immediately.

Hey, please do. Sounds like it should work, at least. I sure don’t
have a PPC box to test on and I don’t think anyone else involved in Ruby
FFI does either.

  • Charlie

On Tue, Nov 04, 2008 at 06:45:27AM +0900, Charles Oliver N. wrote:

(typelib.sf.net and typelib.sf.net/html/ruby) so I’ll probably be using libffi
directly.

Is your library compatible with JRuby? I think that’s a key point for
Ruby FFI.
It is not – it is mainly C++ – but given that most of the
functionality is
written in C++ and I use that C++ interface in other C++ programs,
moving to
Ruby-FFI is not practical for me. So …

Sylvain

On Fri, 31 Oct 2008 18:16:39 -0500, Charles Oliver N. wrote:

p.put_array_of_int32(0, [ 2, 1 ])
additional examples, and links to some other documentation and posts.
Rubinius (though Rubinius has no callback support yet).

Don’t be an extension stooge! Use FFI!

  • Charlie

Based on all of the examples I’ve seen, there’s something that’s still
not clear to me. How does ruby-ffi know which library to load to find
the
specified function?

–Ken

On Tue, Nov 4, 2008 at 2:38 PM, Ken B. [email protected] wrote:

Based on all of the examples I’ve seen, there’s something that’s still
not clear to me. How does ruby-ffi know which library to load to find the
specified function?

–Ken

Use ffi_lib in the extended module, e.g.

require ‘rubygems’
require ‘ffi’

module NCursesFFI
extend FFI::Library
ffi_lib ‘ncurses’
attach_function ‘clear’, [], :int
attach_function ‘endwin’, [], :int
attach_function ‘getch’, [], :int
attach_function ‘initscr’, [], :int
attach_function ‘printw’, [ :string ], :int
attach_function ‘refresh’, [], :int
end

NCursesFFI.initscr
NCursesFFI.printw(“Hello again”)
NCursesFFI.refresh
NCursesFFI.getch
NCursesFFI.endwin

Regards,
Sean

Sylvain J. wrote:

In the body of the module you’re binding:

ffi_convention :stdcall # :default is for normal C convention

We could add whatever convention Linux/PPC needs.
Or, duh, have FFI use the appropriate default convention on the target
platform. Either way, it should be simple to deal with.

Thanks Charlie, but I have already my own library which does more than Ruby-FFI
(typelib.sf.net and typelib.sf.net/html/ruby) so I’ll probably be using libffi
directly.

Is your library compatible with JRuby? I think that’s a key point for
Ruby FFI.

  • Charlie

Sean O’halpin wrote:

On Tue, Nov 4, 2008 at 2:38 PM, Ken B. [email protected] wrote:

module NCursesFFI
extend FFI::Library
ffi_lib ‘ncurses’
attach_function ‘clear’, [], :int
attach_function ‘endwin’, [], :int
attach_function ‘getch’, [], :int
attach_function ‘initscr’, [], :int
attach_function ‘printw’, [ :string ], :int
attach_function ‘refresh’, [], :int
end

Sean,
Going by the above, and the samples of structs on the blog, for a medium
sized library such as ncurses, there can be some effort in getting this
module ready. Will there be some kind of repository of such modules, so
we are not duplicating each others effort ?

I’d like to also add that the above sample is working on osx
10.5.5/darwin/ppc after a small modification in platform.rb above line
28.

ARCH = case CPU.downcase
when /i?86|x86|i86pc|powerpc/
  "i386"
when /amd64|x86_64/
  "x86_64"

I have added “powerpc” to the first “when”. btw, i had a clean gem
install.
Thanks.

On Tue, Nov 4, 2008 at 3:10 PM, Nit K. [email protected] wrote:

attach_function ‘initscr’, [], :int
attach_function ‘printw’, [ :string ], :int
attach_function ‘refresh’, [], :int
end

Sean,
Going by the above, and the samples of structs on the blog, for a medium
sized library such as ncurses, there can be some effort in getting this
module ready. Will there be some kind of repository of such modules, so
we are not duplicating each others effort ?

It’s a fairly tedious (and largely thankless) task to convert a lib
like ncurses. However, I’m in the mood right now so expect a github
repo in the near future :slight_smile: Your point about avoiding duplication of
effort is a good one. The main problem in tackling something like this
is fatigue setting in when you’ve got enough for your immediate
requirement and so ending up with a half-baked conversion (I speak
from experience). I believe another pitfall is trying to rubyify the
interface. I think it’s better to provide a thin bridge and then
abstract on top of that. Then you get the benefit of being able to use
existing experience and documentation (with the usual translations)
and others can take or leave your abstraction as they see fit. (I do
think Ruby needs a portable console library but I’d rather build that
on top of ruby versions of ncurses and Win32 console than try to make
ncurses work on Windows. I’m hoping FFI will make that possible.)

I’m wondering if it would be useful to have these FFI-enabled libs
under an ffi/ namespace, e.g. you’d require ‘ffi/ncurses’ and possibly
include FFI::NCurses. We could put shared structs, constants, etc.
under ffi/include. Any thoughts anyone?

I have added “powerpc” to the first “when”. btw, i had a clean gem
install.

Excellent news.

Regards,
Sean

On windows, I can’t get it to work for jruby or ruby

C:\jruby-1.1.5>jruby -v
jruby 1.1.5 (ruby 1.8.6 patchlevel 114) (2008-11-03 rev 7996) [x86-java]

C:\jruby-1.1.5>cd samples\ffi

C:\jruby-1.1.5\samples\ffi>dir
Volume in drive C has no label.
Volume Serial Number is F4EA-F50A

Directory of C:\jruby-1.1.5\samples\ffi

11/03/2008 05:04 PM .
11/03/2008 05:04 PM …
11/03/2008 04:42 PM 301 ffi.rb
11/03/2008 04:42 PM 367 gettimeofday.rb
11/03/2008 04:42 PM 2,340 pty.rb
11/03/2008 04:42 PM 476 qsort.rb
11/03/2008 04:42 PM 2,078 win32api.rb
5 File(s) 5,562 bytes
2 Dir(s) 18,840,158,208 bytes free

C:\jruby-1.1.5\samples\ffi>jruby win32api.rb
null:1:in const_missing': uninitialized constant POSIX::FFI (NameError) from ffi.rb:4 from ffi.rb:1:inrequire’
from win32api.rb:1

C:\jruby-1.1.5\samples\ffi>

For MRI, it tries to compile

C:\jruby-1.1.5\samples\ffi>gem install ffi
Building native extensions. This could take a while…
ERROR: Error installing ffi:
ERROR: Failed to build gem native extension.

c:/ruby/bin/ruby.exe extconf.rb install ffi
creating Makefile

nmake
‘nmake’ is not recognized as an internal or external command,
operable program or batch file.

Gem files will remain installed in
c:/ruby/lib/ruby/gems/1.8/gems/ffi-0.1.1
for inspection.
Results logged to
c:/ruby/lib/ruby/gems/1.8/gems/ffi-0.1.1/ext/gem_make.out

C:\jruby-1.1.5\samples\ffi>

Sean O’halpin wrote:

On Tue, Nov 4, 2008 at 3:10 PM, Nit K. [email protected] wrote:

attach_function ‘initscr’, [], :int
attach_function ‘printw’, [ :string ], :int
attach_function ‘refresh’, [], :int
end

Sean,
Going by the above, and the samples of structs on the blog, for a medium
sized library such as ncurses, there can be some effort in getting this
module ready. Will there be some kind of repository of such modules, so
we are not duplicating each others effort ?

It’s a fairly tedious (and largely thankless) task to convert a lib
like ncurses. However, I’m in the mood right now so expect a github
repo in the near future :slight_smile: Your point about avoiding duplication of
effort is a good one. The main problem in tackling something like this
is fatigue setting in when you’ve got enough for your immediate
requirement and so ending up with a half-baked conversion (I speak
from experience). I believe another pitfall is trying to rubyify the
interface.

I suppose some of these conversions/wrappers would be “approved” and
sort of become a standard, so we don’t have a plethora of them. Then
they could be placed in one repo (after some sort of review). Just a
thought.

Would hate to have find 5 different ncurses wrappers and have to study
each to figure out which to use.

Nit K. wrote:

attach_function ‘printw’, [ :string ], :int
attach_function ‘refresh’, [], :int
end

Sean,
Going by the above, and the samples of structs on the blog, for a medium
sized library such as ncurses, there can be some effort in getting this
module ready. Will there be some kind of repository of such modules, so
we are not duplicating each others effort ?

I’d recommend folks start releasing FFI-based “thin wrappers” for each
library they come across, and then depend on that gem. Don’t go over the
top…just get the basic bindings in there, structs, tests to make sure
it’s all working, and release a gem. Then build nice APIs on top of
that, separately.

One of my biggest gripes about C extensions in Ruby is that everyone
feels the need to release their own wrapper PLUS Ruby-like API in a
single gem, when just releasing the wrapper would allow a lot more
experimentation. So you end up with multiple people writing their own
extensions with their own APIs and little sharing.

If we just get a whole ton of, simple, clean wrappers out there, the
APIs will surely follow.

  • Charlie

On Nov 4, 12:51 pm, Sean O’Halpin [email protected] wrote:

attach_function ‘initscr’, [], :int
It’s a fairly tedious (and largely thankless) task to convert a lib
think Ruby needs a portable console library but I’d rather build that
on top of ruby versions of ncurses and Win32 console than try to make
ncurses work on Windows. I’m hoping FFI will make that possible.)

Yeah, first layer of implementation and then abstraction (Rubify) is
the way to go :slight_smile:

I’m wondering if it would be useful to have these FFI-enabled libs
under an ffi/ namespace, e.g. you’d require ‘ffi/ncurses’ and possibly
include FFI::NCurses. We could put shared structs, constants, etc.
under ffi/include. Any thoughts anyone?

Something like the win32utils team does:

gem: win32-service
gem win32-eventlog

require ‘win32/service’
require ‘win32/eventlog’

All these live inside Win32 namespace

I think these could be named ffi-ncourses but the name implies “FFI
implementation of ncurses” where I will love to have “cross platform
curses” (note the lack of ‘n’).

These should depend on ruby-ffi gem and we are set to go :slight_smile:

My to cents

Charles Oliver N. wrote:

I posted a blog entry with a longer description of the library,
additional examples, and links to some other documentation and posts.
Docs are a little slim at this point, so feel free to experiment and
update the JRuby wiki page:

http://wiki.jruby.org/wiki/Calling_C_from_JRuby

I’m sure docs from here will filter back into the library and out into
the general cosmos.

FYI, I’ve moved the content of the page above to the Ruby FFI wiki here:

http://kenai.com/projects/ruby-ffi/pages/Home

Please make updates and additions to that copy. I’ll modify the JRuby
version to link to Ruby FFI’s wiki.

  • Charlie

Dominic S. wrote:

C:\jruby-1.1.5\samples\ffi>jruby win32api.rb
null:1:in const_missing': uninitialized constant POSIX::FFI (NameError) from ffi.rb:4 from ffi.rb:1:inrequire’
from win32api.rb:1

That sample could be behind the times…play with it a bit and see what
you can see. None of the core JRuby devs use Windows on a regular
basis…I think this was just a quick experiment created early in FFI
dev cycle.

FWIW, I believe someone (Dan B., probably) is working on a full FFI
win32api impl. He could use some help.

nmake
‘nmake’ is not recognized as an internal or external command,
operable program or batch file.

We need to get a win32 binary FFI gem released. I think one or two
people are working on that. Jump on Ruby FFI mailing lists if you are
interested in helping.

  • Charlie

On Tue, Nov 4, 2008 at 7:02 PM, Charles Oliver N.
[email protected] wrote:

APIs and little sharing.

If we just get a whole ton of, simple, clean wrappers out there, the APIs
will surely follow.

  • Charlie

Maybe I wasn’t clear - what you’re proposing is exactly what I
intended to convey. Thin wrapper in a single gem. Any abstractions
should be separate gems.

Regards,
Sean

On Tue, 04 Nov 2008 14:02:50 -0500, Charles Oliver N. wrote:

attach_function ‘initscr’, [], :int

extensions with their own APIs and little sharing.

If we just get a whole ton of, simple, clean wrappers out there, the
APIs will surely follow.

  • Charlie

The good news is that unlike C libraries, if someone bundles their own
wrapper with their own API, you can still get at the wrapper, by finding
the module that imports FFI and just using that.

Sean O’Halpin wrote:

Maybe I wasn’t clear - what you’re proposing is exactly what I
intended to convey. Thin wrapper in a single gem. Any abstractions
should be separate gems.

Perfect then :slight_smile:

Tom Enebo actually suggested just naming them the library name, as in
libncurses, libtommath, whatever.

  • Charlie

On Wed, Nov 5, 2008 at 4:34 AM, Charles Oliver N.
[email protected] wrote:

Tom Enebo actually suggested just naming them the library name, as in
libncurses, libtommath, whatever.

That’s a little *nix-centric, no? If we had everything under ffi/
you’d be able to find everything in one place (e.g. easier to find
examples to copy from, easier to set up conventions for where to put
shared structs, constants, etc.). As for OS-specific libs, there could
be ffi/win32/ ffi/unix, where appropriate.

It would be useful to have conventions on how to implement thin layer
ffis (e.g. how to handle return values in *ptrs, how to implement
functons defined as macros, namespacing constants, etc.). This would
help POLS. Also, guidelines on how to portably implement layers for
32-bit/64-bit and endian-ness would be handy (I know I’d like some
tips).

While there is no general solution to converting header files,
projects such as SWIG have shown that you can get a lot done
automatically. It would be worthwhile to work on tools to help with
this - even an 80% solution removes a lot of the donkey work (as I’m
finding with my ncurses port).

My general opinion is that the more boring, mechanical, choice-free
and thoughtless we can make this work, the easier it will be to do and
so the more likely to get done.

Your cross-implementation FFI initiative is very welcome - it’s a
liberating move. Now we can get on with the mindless (but worthy)
drudgery :slight_smile:

Regards,
Sean

Sean O’halpin wrote:

On Wed, Nov 5, 2008 at 4:34 AM, Charles Oliver N.
[email protected] wrote:

automatically. It would be worthwhile to work on tools to help with
this - even an 80% solution removes a lot of the donkey work (as I’m
finding with my ncurses port).

Regards,
Sean

Eagerly awaiting your ncurses port…

Is it possible for you to document the procedure, how you go about doing
it, so one can follow that for another port, rather than be totally
lost.