Forum: Ruby Q: FFI and C++?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Jeremy H. (Guest)
on 2008-12-11 17:11
(Received via mailing list)
If I want to  wrap a C++ library using FFI, can  it cope with the name
mangling?  I've got a working example  where I compile a C wrapper for
the C++ and call the wrapper from FFI.  That's rather clunky, I'd hope
there is  a cleaner way.   If it does  come to writing a  wrapper, are
there any packages for  auto-generating the wrapper code?  (Maybe SWIG
can do something?)

Regards,

Jeremy H.
Jason R. (Guest)
on 2008-12-11 17:21
(Received via mailing list)
Because of the complexity that C++ adds, I doubt that any feasible FFI
library can be made that works reliably against C++ libraries, though
Charles will probably be a better voice for whether this is truely
possible or not.

For now, there are a few libraries now out that will definitely help
you out (and are IMO a ton better than trying to deal with SWIG, but
I'm slightly biased). They are:

Rice - A C++ development library that abstracts out the internals of
Ruby's object model to make wrapping C++ libraries as easy as
possible.
  http://rice.rubyforge.org/
  http://github.com/cout/rice

Rb++ - Using GCCXML and Rice to auto-generate wrappers of the C++ code
you are wanting to wrap
  http://rbplusplus.rubyforge.org
  http://github.com/jameskilton/rbplusplus


Jason
Charles Oliver N. (Guest)
on 2008-12-11 17:28
(Received via mailing list)
Jason R. wrote:
> Because of the complexity that C++ adds, I doubt that any feasible FFI
> library can be made that works reliably against C++ libraries, though
> Charles will probably be a better voice for whether this is truely
> possible or not.

Wayne and I talked about this a bit, and largely the problem is the
name-mangling involved. So there would always need to be a compile phase
for the C++ stuff to be callable from an FFI-based library.

Wayne did propose adding an inline C syntax similar to RubyInline but
which simple created a shim library that would be found through FFI
rather than the more complex MRI-specific mechanisms RubyInline uses.
this would make it possible to bind C++ code through the shim into an
FFI-based library, and potentially recomposed back into an Object-like
structure.

I don't believe he's done any work on this. I also do not know how any
of the other C++ options work, but I assume they all have a compilation
component.

- Charlie
Luc H. (Guest)
on 2008-12-11 17:29
(Received via mailing list)
On 11 déc. 08, at 16:13, Jason R. wrote:

> Because of the complexity that C++ adds, I doubt that any feasible FFI
> library can be made that works reliably against C++ libraries

The problem in the case of C++ is its non-standardized ABI which
prevents binding to libraries without knowing the name mangling
scheme. Pretty much the same reason why bitfield structs aren't
supported in libffi: because their layout is not standardized and thus
compiler dependent.
Jeremy H. (Guest)
on 2008-12-11 18:35
(Received via mailing list)
On 2008-12-11, Jason R. <removed_email_address@domain.invalid> wrote:

> Because of the  complexity that C++ adds, I  doubt that any feasible
> FFI library can be made that works reliably against C++ libraries,

I was afraid of that.

> Rice - ...
> Rb++ - ...

I looked at  those but I thought they were  targeted at generating MRI
bindings.  The point of using FFI is to eliminate MRI-specific C code.
I just need to wrap C++ with C to get functions that FFI can call, eg.

    extern "C" {

    void *thing_new() { return new Thing(); }
    void thing_delete(void *vt) { delete (Thing *) vt; }
    void *thing_bar(void *vt) { return ((Thing *) vt)->bar(); }

    // etc. etc. etc.
    }

I already know this works, now I'm trying to save effort, particularly
with parsing C++  and autogenerating wrapper code.  Maybe  I could use
rbgccxml and  add my  own XML  -> wrapper code  backend.  It  looks as
though rbgccxml is  precisely the bit of rbplusplus  that I would want
to reuse.

Thanks for your help,

Jeremy H.
Alex F. (Guest)
on 2008-12-11 18:55
(Received via mailing list)
Jeremy H. wrote:
> I looked at  those but I thought they were  targeted at generating MRI
> bindings.  The point of using FFI is to eliminate MRI-specific C code.
> I just need to wrap C++ with C to get functions that FFI can call, eg.

Latest SWIG does include C as a target language. In other words it can
generate ANSI C wrapper code around C++ classes / libraries.

This was a Google Summer of Code project from this year and I believe
it's only available from SVN, not in a released version.

http://swig.svn.sourceforge.net/viewvc/swig/branch...


> I already know this works, now I'm trying to save effort, particularly
> with parsing C++  and autogenerating wrapper code.  Maybe  I could use
> rbgccxml and  add my  own XML  -> wrapper code  backend.  It  looks as
> though rbgccxml is  precisely the bit of rbplusplus  that I would want
> to reuse.

SWIG can also output a parse of C++ classes and methods to XML. But if
this is all you want to do, probably the Rb++ route is better as it has
existing Ruby classes for making use of the parse. I guess in theory you
might also be able to generate the FFI code from this at the same time.

alex
Jason R. (Guest)
on 2008-12-11 18:59
(Received via mailing list)
On Thu, Dec 11, 2008 at 11:28 AM, Jeremy H. 
<removed_email_address@domain.invalid>
wrote:
> I looked at  those but I thought they were  targeted at generating MRI
>    }
>
>

I'd ask that if you already need to write and compile a wrapper to
make a library work with FFI, then why bother with FFI in the first
place? Rice / Rb++ will get you a lot farther in less time than trying
to hack your way into using the FFI library, besides the fact that
your sample wrapper code will wreck complete havok with your C++ <->
Ruby type mapping, requiring a lot more code to handle converting
types to and fro as needed. Rice gives you all this automatically.

In short, Rice isn't about generating MRI bindings for you but more
about exposing an OO wrapper around the MRI bindings, so you don't
have to get into the nitty-gritty details of Ruby's C API, and then
Rb++ sits on top of that to auto-generate as much Rice code as
possible so you only have to work in Ruby.

Jason
Sean O. (Guest)
on 2008-12-11 19:20
(Received via mailing list)
On Thu, Dec 11, 2008 at 3:21 PM, Charles Oliver N.
<removed_email_address@domain.invalid> wrote:
> Jason R. wrote:
>>
>> Because of the complexity that C++ adds, I doubt that any feasible FFI
>> library can be made that works reliably against C++ libraries, though
>> Charles will probably be a better voice for whether this is truely
>> possible or not.
>
> Wayne and I talked about this a bit, and largely the problem is the
> name-mangling involved. So there would always need to be a compile phase for
> the C++ stuff to be callable from an FFI-based library.

The compile step may not be necessary. I'm looking at using gccxml to
generate function signatures. It provides the mangled names in the
output (and emulates a number of compilers including MSVC versions 6
to 8). I'll report back as I find out more.

Regards,
Sean
Jeremy H. (Guest)
on 2008-12-11 19:36
(Received via mailing list)
On 2008-12-11, Jason R. <removed_email_address@domain.invalid> wrote:

> I'd ask that  if you already need to write and  compile a wrapper to
> make a library work with FFI,  then why bother with FFI in the first
> place?

To  get portability  across different  Ruby implementations.   This is
really my way of contributing to  FFI: it's really cool but it seems a
little short of  actual applications.  I doubt I can  make it any more
cool than  it already  is, but I  can do  something about the  lack of
applications.

> Rice / Rb++ will  get you a lot farther in less  time than trying to
> hack your way into using the FFI library,

I already have a  working toy FFI/C++ application, including callbacks
that  hook  virtual C++  methods  back into  Ruby  and  it was  almost
disappointingly  straightforward to  do.  I  suspect it  will  be less
effort than  you make  out.  And if  I'm wrong  about that, I'll  be a
sadder and wiser man.  Worse things happen at C, I mean sea.

Thanks again for your help,

Jeremy H.
Jason R. (Guest)
on 2008-12-11 19:45
(Received via mailing list)
On Thu, Dec 11, 2008 at 12:28 PM, Jeremy H. 
<removed_email_address@domain.invalid>
wrote:
> applications.
> Thanks again for your help,
>
> Jeremy H.
>
>

Makes sense. Looking forward to see what you get working, and if it's
as easy as you're saying, rb++ could be updated to also use this way
of wrapping.

More ways of making Ruby extensions easier to build is never a bad
thing.

Good luck in your endeavors.

Jason
Charles Oliver N. (Guest)
on 2008-12-11 19:47
(Received via mailing list)
Sean O'Halpin wrote:
>
> The compile step may not be necessary. I'm looking at using gccxml to
> generate function signatures. It provides the mangled names in the
> output (and emulates a number of compilers including MSVC versions 6
> to 8). I'll report back as I find out more.

That would be interesting...if it works, FFI could potentially bind an
entire C++ class too.

- Charlie
Jeremy H. (Guest)
on 2008-12-12 12:55
(Received via mailing list)
On 2008-12-11, Alex F. <removed_email_address@domain.invalid> wrote:

> Latest SWIG does  include C as a target language.  In other words it
> can generate ANSI C wrapper code around C++ classes / libraries.

Very interesting!  Thanks.

> SWIG can also output a parse  of C++ classes and methods to XML. But
> if this is all you want to  do, probably the Rb++ route is better as
> it has existing Ruby classes for making use of the parse. I guess in
> theory you might also be able  to generate the FFI code from this at
> the same time.

That's my plan, but SWIG is an interesting option too.

Jeremy H.
Alex F. (Guest)
on 2008-12-12 16:35
(Received via mailing list)
Jeremy H. wrote:
>> theory you might also be able  to generate the FFI code from this at
>> the same time.
>
> That's my plan, but SWIG is an interesting option too.


Once you've had a chance to try things out, I'd be very interested in
hearing about your experiences. I work on the bindings for a large C++
library (wxWidgets) and I'm looking at options for the next version.

We use SWIG at the moment to generate a (MRI-only) wrapper, but this
means that hacking on the core library is a daunting prospect (SWIG is
powerful but complex, C++ is hard and requires a compiler).

I looked a Rb++ for the whole work but there didn't seem to be enough
gain for the pain of switching over a large code base. SWIG deals with
several C++ features that aren't supported in Rb++ as far as I can tell:
overloaded methods, implementing virtual methods in Ruby, keeping a
one-to-one link between Ruby instances and C++ objects.

So I'm keen to find out if there's a way of reducing the complexity of
the SWIG/C++ code and doing more of the wrapping work in Ruby - as well
as being able to potentially run wxRuby on JRuby.

a
Jeremy H. (Guest)
on 2008-12-15 11:05
(Received via mailing list)
On 2008-12-12, Alex F. <removed_email_address@domain.invalid> wrote:

> Once you've had  a chance to try things out,  I'd be very interested
> in  hearing about your  experiences. I  work on  the bindings  for a
> large C++  library (wxWidgets)  and I'm looking  at options  for the
> next version.

Aha!  I'm looking at binding FTLK1.  Stay tuned for the ffi-fltk1 gem.
The first  version will probably hand-code  the wrapper so  that I can
deliver something while I learn what the wrapper has to look like, but
later versions will auto-generate as much as is feasible.

Regards,

Jeremy H.
This topic is locked and can not be replied to.