CplusRuby - Gluing C and Ruby

Hi all,

I am proud to announce my latest project called CplusRuby.
Below I pasted it’s README for further explanation.
Alternatively read the following blog article:

http://www.ntecs.de/blog/articles/2007/09/21/cplusruby-gluing-c-and-ruby


CplusRuby - Glue C and Ruby together (OO-like)

COPYRIGHT

Copyright (c) 2007 by Michael N. ([email protected]).
All rights reserved.

LICENSE

Ruby License.

ABOUT

With CplusRuby you can define custom C structures from within Ruby
and
wrap them easily in a Ruby class. You can as well define C functions
that can access this structure easily. CplusRuby generates
setter/getter
methods for every property and wrapper methods for the C functions.

The purpose is speed! The C functions can access the C-structure,
which is much faster than accessing instance variables. Also, the C
functions can call each other directly. This is much faster than
invoking a method in Ruby. As wrappers are generated, the Ruby-side
can access all C-structures and functions as well.

I started to write CplusRuby to implement a high-performance pulsed
neural network simulator. My existing C++ implementation suffered
from - well - C++ :). This enables me to write the core algorithms
in C or C++ and do all the other non performance-critical tasks in
Ruby.

EXAMPLE

Take a look at the following example. You should also take a look
at the generated C source file (inspire.c). Note that properties
are actually members of a C-struct, not instance variables, and as
such, their access from C is very fast. As calling a method is quite
slow in Ruby, method defined in C (method_c) can be called directly
from C, which again is very fast!

 # example.rb
 require 'cplusruby'

 class NeuralEntity < CplusRuby
   property :id
 end

 class Neuron < NeuralEntity
   property :potential,        :float
   property :last_spike_time,  :float
   property :pre_synapses,     :value

   method_c :stimulate, %(float at, float weight), %{
     // this is C code
     selfc->potential += at*weight;
   }

   def initialize
     self.pre_synapses = []
   end
 end

 # generate C file, compile it and load the .so
 CplusRuby.evaluate("inspire.cc", "-O3", "-lstdc++")

 if __FILE__ == $0
   n = Neuron.new
   n.id = "n1"
   n.potential = 1.0
   n.stimulate(1.0, 2.0)
   p n.potential # => 3.0
 end

END

On Sep 21, 1:01 pm, Michael N. [email protected] wrote:

Hi all,

I am proud to announce my latest project called CplusRuby.
Below I pasted it’s README for further explanation.
Alternatively read the following blog article:

http://www.ntecs.de/blog/articles/2007/09/21/cplusruby-gluing-c-and-ruby

First patch!

C:\Documents and Settings\djberge\My Documents\My Downloads\Ruby>diff -
u cplusruby.orig cplusruby.rb
— cplusruby.orig Fri Sep 21 15:50:38 2007
+++ cplusruby.rb Fri Sep 21 15:49:04 2007
@@ -11,6 +11,10 @@

Implements a simple ordered Hash

+require ‘rbconfig’
+require ‘win32/process’ if RUBY_PLATFORM.match(‘mswin’)
+include Config
+
class OHash < Hash
def []=(k, v)
@order ||= []
@@ -146,20 +150,28 @@
dir = File.dirname(file)
mod, ext = base.split(“.”)

  • make = RUBY_PLATFORM.match(‘mswin’) ? ‘nmake’ : ‘make’
  • File.open(file, ‘w+’) {|f| f << self.generate_code(mod) }
    Dir.chdir(dir) do
  •  system("make clean")
    
  •  system("#{make} clean")
    
  •  pid = fork do
       require 'mkmf'
       $CFLAGS = cflags
       $LIBS << (" " + libs)
       create_makefile(mod)
    
  •    exec "make"
    
  •    exec "#{make}"
     end
     _, status = Process.waitpid2(pid)
    
  •  raise if status.exitstatus != 0
    
  •  if RUBY_PLATFORM.match('mswin')
    
  •     raise if status != 0
    
  •  else
    
  •     raise if status.exitstatus != 0
    
  •  end
    
    end
  • require “#{dir}/#{mod}.so”
  • require “#{dir}/#{mod}.” + CONFIG[‘DLEXT’]
    end

def self.generate_code_for_class

Mostly fixes MS Windows issues, but the DLEXT bit also fixes platforms
that don’t generate .so files (OS X, HP-UX).

Regards,

Dan

Quoth Daniel B.:

#
     $LIBS << (" " + libs)
  •     raise if status.exitstatus != 0
    

Regards,

Dan

Does the C->ruby mapping map between char * and Strings?

Thanks,

Quoth Konrad M.:

Implements a simple ordered Hash

 mod, ext = base.split(".")
     $CFLAGS = cflags
  •  else
    

that don’t generate .so files (OS X, HP-UX).

Regards,

Dan

Does the C->ruby mapping map between char * and Strings?

Thanks,

Konrad M. [email protected] http://konrad.sobertillnoon.com/

Okay, so I have added the following to DEFAULT_OPTIONS:

:int => {
  :default => '0',
  :ctype => 'int %s',
  :ruby2c => '(int)NUM2INT(%s)',
  :c2ruby => 'INT2NUM(%s)'},

:char_p => {
  :default => '""',
  :ctype => 'char *%s',
  :ruby2c => 'StringValuePtr(%s)',
  :c2ruby => 'rb_str_new2(%s)'}

My code is like this:

#---------------------start

require ‘cplusruby’

class BankAccount < CplusRuby
property :balance, :int
property :name, :char_p

method_c :withdraw, %{int dollars}, %{
if (dollars <= selfc->balance){
selfc->balance -= dollars;
return true;
}
}

def initialize
end
end

CplusRuby.evaluate(“foo.c”, “-Os”, “”)

n = BankAccount.new
puts n.balance
puts n.name

#---------------------end

When I run it with ruby I get a whole bunch of errors stemming from
the fact that it generates C code that uses a type called
“BankAccount” without defining it anywhere (struct BankAccount is
defined though).

Ideas, suggestions, questions?

Thanks,

Konrad M. schrieb:

#
     $LIBS << (" " + libs)
  •     raise if status.exitstatus != 0
    
  :c2ruby => 'rb_str_new2(%s)'} 

method_c :withdraw, %{int dollars}, %{
if (dollars <= selfc->balance){
selfc->balance -= dollars;
return true;

I think you shouldn’t return “true” here. At least it’s
not indended, as the withdraw function is “void”.

puts n.balance
puts n.name

#---------------------end

When I run it with ruby I get a whole bunch of errors stemming from
the fact that it generates C code that uses a type called
“BankAccount” without defining it anywhere (struct BankAccount is
defined though).

I think C does not yet work. Try C++ :slight_smile:

CplusRuby.evaluate(“foo.cc” …)

But it should be easy to fix, by adding a “typedef struct”, and
“VALUE()()" instead of "VALUE ()(…)”.

Ah, and returns values are not yet implemented. Nevertheless you can
assign to a property and use it as a return value.

Thanks,

Michael

Quoth Konrad M.:

Implements a simple ordered Hash

 mod, ext = base.split(".")
     $CFLAGS = cflags
  •  else
    

that don’t generate .so files (OS X, HP-UX).

Regards,

Dan

Does the C->ruby mapping map between char * and Strings?

Thanks,

Konrad M. [email protected] http://konrad.sobertillnoon.com/

Also, is there a way to define C functions from ruby that return a C
type?

Daniel B. schrieb:

First patch!

Thanks! Applied!

I didn’t know that adding Windows support is that easy. Cool!

BTW, I am using Mercurial as repository, so it should be easy for
everyone to keep track of changes. To get the repository:

hg clone static-http://ntecs.de/hg-projects/cplusruby

Regards,

Michael

Hi,

At Sat, 22 Sep 2007 06:54:13 +0900,
Daniel B. wrote in [ruby-talk:270261]:

  •  if RUBY_PLATFORM.match('mswin')
    
  •     raise if status != 0
    
  •  else
    
  •     raise if status.exitstatus != 0
    
  •  end
    

I’m curious why win32/process doesn’t return Process::Status.

Konrad M. schrieb:

  :default => '""',

class BankAccount < CplusRuby
def initialize
end
end

CplusRuby.evaluate(“foo.c”, “-Os”, “”)

n = BankAccount.new
puts n.balance
puts n.name

I’ve applied your patch. Plus, generating C code is now possible. The
example above should now just work. Alternatively “foo.cc” :slight_smile:

Regards,

Michael

Nobuyoshi N. wrote:

I’m curious why win32/process doesn’t return Process::Status.
We had to reimplement waitpid and waitpid2 because of our custom fork
implementation. But, we weren’t sure how to set $? manually to a
Process::Status object in pure Ruby. Do you have any suggestions?

Thanks,

Dan

Michael N. wrote:


It’s been done, but I’m glad to see it done again and probably with new
ideas :slight_smile:

For comparison:

require ‘cgen/cshadow’ # from cgen on RAA

class NeuralEntity
include CShadow
attr_accessor :id
end

class Neuron < NeuralEntity
shadow_attr_accessor :potential => “double potential”
shadow_attr_accessor :last_spike_time => “double last_spike_time”
shadow_attr_accessor :pre_synapses => Object

 define_c_method :stimulate do
   arguments :at, :weight
   body %{
     shadow->potential += NUM2DBL(at)*NUM2DBL(weight);
   }
 end

 def initialize
   self.pre_synapses = []
 end

end

NeuralEntity.commit # generate and load the shared lib

if FILE == $0
n = Neuron.new
n.id = “n1”
n.potential = 1.0
n.stimulate(1.0, 2.0)
p n.potential # => 3.0
end

Cgen is middle aged, as ruby libraries go (written mostly in 2001), and
it probably isn’t up to community standards in a number of ways. Still,
I’ve used it intensively in numerical and hybrid simulation since then.
It’s the foundation for redshift, which is a DSL for simulation of
hybrid automata (networked state machines with continuous as well as
discrete dynamics).

Cgen/cshadow handles a number of issues for you:

  • creating accessors (r, w, rw)
  • type checking and conversion in the write accessor
  • mark and free functions
  • attr initialization
  • persistence via marshal and yaml (or an attr may be specified
    non-persistent)
  • extensible attribute classes for defining new attr types, other than
    the basic numeric, string, and Ruby object types
  • inheritance (a subclass of a shadow class inherits the parent’s shadow
    attrs in its own C struct; struct may be safely cast to parent type)
  • tools for managing how code is distributed among files (very important
    with a large project, or else recompilation time gets painful).
  • tools for managing declarations and preprocessor includes
  • DSL for rb_scan_args()

This is pretty well documented, though it is in the old rd format,
rather than rdoc.

Michael N. wrote:

In the example you give I only see one minor difference. In CplusRuby,
method :stimulate can be called directly from the C-side, without
NUM2DBL conversions of Ruby-types. That’s very important for my
usage scenario.
But I think you can do the same with cgen.

Actually, I don’t think cgen has that feature… one of the things I’ll
have to er, borrow, from cplusruby. Currently the only way to do this is
to define a separate C function that the method calls.

Reading the cplusruby .c output, it looks like there is an outer
function (Neuron_call__stimulate) that accepts ruby value arguments, and
then converts them and calls an inner function (Neuron__stimulate),
which expects C-typed arguments. You could do this in cgen by using
define_c_method to define the outer function and define_c_function to
define the inner function.

I guess I was used to performing the conversion manually, so I never
added that feature as a convenience. I can see that it would be more of
a necessity than a convenience if the function needed to be called from
C as well as ruby.

The selfc->stimulate function pointer is another difference, and
necessary for the same reason, I guess. In a cshadow struct, this struct
member doesn’t exist, but there is reference (“VALUE self;”) back to the
original object, on which you can call methods (with the additional
friction of type conversions, if the caller is native C code).

The fact that you have pointers to functions that accept C-typed args
makes access from C easy and efficient, but it also uses a lot of space
in the struct with a pointer to each function that implements a method.
What about having a unique struct for each class that keeps these
pointers, and then the struct only needs a pointer to this “singleton”?

Joel VanderWerf schrieb:

CplusRuby - Glue C and Ruby together (OO-like)

It’s been done, but I’m glad to see it done again and probably with new
ideas :slight_smile:

Wow, cool! I knew about cgen, but then didn’t knew that it’s that easy
to use.

In the example you give I only see one minor difference. In CplusRuby,
method :stimulate can be called directly from the C-side, without
NUM2DBL conversions of Ruby-types. That’s very important for my
usage scenario.
But I think you can do the same with cgen.

Regards,

Michael

Daniel B. wrote:

  • require “#{dir}/#{mod}.so”
  • require “#{dir}/#{mod}.” + CONFIG[‘DLEXT’]
    end

def self.generate_code_for_class

Mostly fixes MS Windows issues, but the DLEXT bit also fixes platforms
that don’t generate .so files (OS X, HP-UX).

Why not just

  • require “#{dir}/#{mod}”

which should work if there are no stray .rb files in the dir?

On Sep 21, 2007, at 12:01, Michael N. wrote:

I am proud to announce my latest project called CplusRuby.
Below I pasted it’s README for further explanation.
Alternatively read the following blog article:

http://www.ntecs.de/blog/articles/2007/09/21/cplusruby-gluing-c-and-
ruby

[…]
system(“#{make} clean”)

Why not use RubyInline?

You’ll get more than twice the utility and drop a bunch of code.

One comment on inheritance in cplusruby:

def self.inherited(klass)
# inherit properties and c_methods
self.properties.each {|k,v| klass.properties[k] = v}
CplusRuby.registered_classes << klass
end

Re-opening a parent class after a child class has been defined and
adding more properties will not propagate them to the child. And you
can’t simply add such properties to the end of the parent and child
struct definitions, since that will make them incompatible (in the sense
of casting).

Eric H. schrieb:

Why not use RubyInline?

You’ll get more than twice the utility and drop a bunch of code.

Drop less than 20 lines of code? It really takes longer to get into
RubyInline than to write those 20 lines :slight_smile:

Regards,

Michael

Joel VanderWerf schrieb:

One comment on inheritance in cplusruby:

def self.inherited(klass)

inherit properties and c_methods

self.properties.each {|k,v| klass.properties[k] = v}
CplusRuby.registered_classes << klass
end

Re-opening a parent class after a child class has been defined and
adding more properties will not propagate them to the child. And you
can’t simply add such properties to the end of the parent and child
struct definitions, since that will make them incompatible (in the
sense of casting).

Ah thanks. Well, I should definitively refactor cplusruby a bit :).

A quick C question about casting:

struct A {
int a;
}

struct B {
int a;
int b;
}

Is it always safe (even with -O3) to cast a pointer of B to A?
Or do I have to do:

struct B {
A x;
int b;
}

which is very ugly IMHO. Are there pramgas like “__packed” for this?

C as well as ruby.

The selfc->stimulate function pointer is another difference, and
necessary for the same reason, I guess. In a cshadow struct, this struct
member doesn’t exist, but there is reference (“VALUE self;”) back to the
original object, on which you can call methods (with the additional
friction of type conversions, if the caller is native C code).

The “VALUE self” in the C structure is a feature I will borrow. It’s a
great idea!

The fact that you have pointers to functions that accept C-typed args
makes access from C easy and efficient, but it also uses a lot of space
in the struct with a pointer to each function that implements a method.
What about having a unique struct for each class that keeps these
pointers, and then the struct only needs a pointer to this “singleton”?

I plan to make methods by default “non-virtual” in the C++ sense. That
is, they will by default not be included as function pointers in the
struct. If you want to have virtual methods, you’d then need to add:

property :a_virtual_method, :virtual_method

I also plan to support templates, which is a feature I really need for
some data types like priority queues (I noticed that using templates
they are a lot faster than a generic version):

class ImplicitBinaryHeap
include CplusRuby::Template

 property :size, :int
 property :capacity, :int
 property :elements, "%ELEMENT_TYPE%*"

 method :initialize do
   arguments [:initial_capacity, :int]
   body %{
     @capacity = initial_capacity;
     @size = 0;
     @elements = ALLOC_N(%ELEMENT_TYPE%, @capacity+1);
   }
 end

 method :push do
   arguments [:element, "%ELEMENT_TYPE%*"]
   body %{
     @size += 1;
     if (@size > @capacity) $resize(self, 2*@capacity + 1);
     @elements[@size] = *element;
     $propagate_up(self, @size);
   }
 end

 #...

end

MyPQ = ImplicitBinaryHeap.bind(:ELEMENT_TYPE => 'int)

now MyPQ is similar ImplicitBinaryHeap

self is now a pointer to a C struct (thanks to the “VALUE self”
reference). “@” is replaced by “self->”, and $ is replaced by
“classname__”, so that you don’t have to hardcode the prefix of methods
(only works for non-virtual methods of course).

The good thing is, that you can use the structure without associating it
to a Ruby object (so you get a poor mans or not so poor mans “C with
classes”):

MyPQ mypq;
MyPQ__initialize(&mypq, 31);
for (int i = 0; i<10; i++)
{
MyPQ__push(&mypq, &i);
}

It’s not that nice as the equivalent in C++, but it doesn’t need a C++
compiler at all.

Regards,

Michael

Quoth Paul B.:

Paul

Yes, I believe that’s the problem I ran into.

:char_p => {
:default => ‘""’,
:ctype => ‘char* %s’,
:ruby2c => ‘StringValuePtr(%s)’,
:c2ruby => ‘rb_str_new2(%s)’}

I think you’ll want to store either a copy of the string or a reference
to the string object (which would need to be marked). If you don’t, the
string could be destroyed before the containing struct, which would
leave you with a dangling pointer.

Paul