Forum: Ruby rocaml: Ruby extensions in Objective Caml

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Dd76a12d66f843de5c5f8782668e7127?d=identicon&s=25 Mauricio Fernandez (Guest)
on 2007-06-22 13:37
(Received via mailing list)
rocaml allows you to write Ruby extensions in Objective Caml.

I never seem to manage to release things when I should, so here's a
pre-release announcement to let you know about this so you can play with
it
before the actual release, which could take longer than necessary.

    http://eigenclass.org/repos/rocaml/head/

Young as it is, rocaml is very usable and the generated extensions are
reliable, since they enforce type safety and handle exceptions both
in Ruby and OCaml (OCaml exceptions are passed to Ruby).

Developing Ruby extensions with rocaml is easier and more convenient
than
writing a plain old C extension since rocaml performs Ruby<->OCaml
conversions
for a wide range of types, including abstract types and arrays, tuples,
variants and records of values of any supported type (e.g. arrays of
arrays of
variants of tuples of ...).

Making an extension with rocaml involves two steps:
* implementing the desired functionality in Objective Caml, and
registering
  the functions to be exported
  (using Callback.register : string -> 'a -> unit)
* creating the extconf.rb file (just modify the sample extconf.rb
distributed
  with rocaml) defining the interface of your Objective Caml code.

** At no point is there any need to write a single line of C code when
**
** using rocaml.
**

The mandatory trivial example
=============================

Let's create an extension with a 'fib' function.
Here's the OCaml code:

 let rec fib n = if n < 2 then 1 else fib (n-1) + fib (n-2)

 let _ = Callback.register "Fib.fib" fib

Here's the interface declaration in your extconf.rb:

 Interface.generate("fib") do
   def_module("Fib") do
     fun "fib", INT => INT
   end
 end

That's it. Running extconf.rb will generate all the required wrappers
and
make will link them against your ml code, creating a normal Ruby
extension
that can be used simply with
  require 'fib'
  p Fib.fib 10

Set of strings using a RB tree
==============================
Here's a simple set based on an RB tree, specialized for strings (see
examples/tree for how to create several classes from a single
polymorphic
structure). The (unoptimized) RB tree takes only ~30LoCs, but lookup is
3X
faster than with RBTree, which takes >3000 lines and over ~250 lines for
the
equivalent functionality, without counting the *manually written*
wrappers for
the underlying C data structure.

This shows how rocaml handles complex types, including variant and
recursive
types.

Given this interface definition:

Interface.generate("tree") do
  string_tree_t = sym_variant("string_tree_t") do |t|
    constant :Empty
    non_constant :Node, TUPLE(t, type, t)
  end

  def_class("StringRBSet") do |c|
    t = c.abstract_type
    fun "empty", UNIT => t, :aliased_as => "new"
    fun "make", string_tree_t => t

    method "add", [t, STRING] => t
    method "mem", [t, STRING] => BOOL, :aliased_as => "include?"
    method "dump", t => string_tree_t
    method "iter", t => t, :aliased_as => "each", :yield => [STRING,
UNIT]
  end
end

You can use the generated extension as follows (you can find the OCaml
code
below):

 require 'tree'
 set = StringRBSet.new
 set2 = s.add "foo"  # the RB set is a functional, i.e. persistant
                     # data structure
 # see how rocaml handles conversions for recursive variant types
 p s.add("foo").dump
 p s.add("foo").add("bar").dump

The above will print
 [:Node, [:B, :Empty, "foo", :Empty]]
 [:Node, [:B, [:Node, [:R, :Empty, "bar", :Empty]], "foo", :Empty]]

showing you the structure of the RB tree.


That's it for now, enjoy.
Further updates on eigenclass.org.

PS:
For the sake of completeness, here's the OCaml code. You can find the
full
example in examples/tree.

exception Found

module RBSet =
struct
  type color = R | B
  type 'a t = Empty | Node of color * 'a t * 'a * 'a t

  let empty = Empty

  let rec mem x = function
      Empty -> false
    | Node(_, l, y, r) ->
        if y < x then mem x l else if y > x then mem x r else true

  let balance = function
      B, Node(R, Node(R, a, x, b), y, c), z, d
    | B, Node(R, a, x, Node(R, b, y, c)), z, d
    | B, a, x, Node(R, Node(R, b, y, c), z, d)
    | B, a, x, Node(R, b, y, Node(R, c, z, d)) -> Node(R, Node(B, a, x,
b), y, Node(B, c, z, d))
    | (c, a, x, b) -> Node (c, a, x, b)

  let add x t =
    let rec ins = function
        Empty -> Node(R, Empty, x, Empty)
      | Node(color, a, y, b) ->
          if x < y then balance (color, ins a, y, b)
          else if x > y then balance (color, a, y, ins b)
          else raise Found
    in try match ins t with
        Node (_, a, y, b) ->  Node(B, a, y, b)
      | Empty -> assert false (* ins always returns Node _ *)
    with Found -> t

  let rec iter f = function
      Empty -> ()
    | Node(_, l, x, r) -> iter f l; f x; iter f r
end


external intset_yield : int -> unit = "IntRBSet_iter_yield"
external stringset_yield : int -> unit = "StringRBSet_iter_yield"

let identity x = x

open Callback
let _ =
  let def_set t =
    let r name f = register (t ^ "RBSet" ^ "." ^ name) f in
      r "empty" (fun () -> RBSet.empty);
      r "add" (fun t x -> RBSet.add x t);
      r "mem" (fun t x -> RBSet.mem x t);
      r "dump" identity;
      r "make" identity;
  in
    List.iter def_set ["Int"; "String"];
    register "IntRBSet.iter" (RBSet.iter intset_yield);
    register "StringRBSet.iter" (RBSet.iter stringset_yield);
Ffcb418e17cac2873d611c2b8d8d891c?d=identicon&s=25 unknown (Guest)
on 2007-06-22 14:32
(Received via mailing list)
Hi Mauricio,

a quick thought that I'm pretty sure you've already had: I notice that
you define the interface in Ruby, and also define what should be
exported in OCaml. Couldn't you, say, just define it in OCaml (where the
type sigantures will be know completely, I guess?), and have the Ruby
interface generated from this?

I've been meaning to learn oCaml; if i do, I'll definitely give this a
look.

Thanks,
  Benjohn
Dd76a12d66f843de5c5f8782668e7127?d=identicon&s=25 Mauricio Fernandez (Guest)
on 2007-06-23 18:05
(Received via mailing list)
On Fri, Jun 22, 2007 at 09:31:12PM +0900, benjohn@fysh.org wrote:
> Hi Mauricio,
>
> a quick thought that I'm pretty sure you've already had: I notice that
> you define the interface in Ruby, and also define what should be
> exported in OCaml. Couldn't you, say, just define it in OCaml (where the
> type sigantures will be know completely, I guess?), and have the Ruby
> interface generated from this?

Even though the types are known by the compiler, human intervention is
needed
at some point because:
* we have to define what needs to be exported
* the naming and parameter passing conventions might differ (e.g. the
  data structure often being given after the element to operate with in
  functional data structures)
* polymorphic functions have too broad a type, and the concrete
  type(s) you want must be specified. For instance, in the RB tree
example,
  two classes are generated for sets of strings and ints. Their instance
  methods correspond to the same polymorphic OCaml functions, but
providing
  the desired concrete types allows the wrapper generated by OCaml to
perform
  the necessary type checking and Ruby->OCaml conversions. The
alternative
  would be wrapping Ruby values in OCaml with an abstract universal
type, but
  introducing dynamic typing defeats the purpose of writing an extension
in
  OCaml to some extent (it'll be slower and the type system will not
help you
  that much).

That said, it would be possible to encode all that information in a .ml
file,
instead of splitting it into an OCaml part (which functions are to be
exported) and another in Ruby, in extconf.rb (how that functionality is
accessible from Ruby, and the method signatures). It's a bit harder to
implement though, as building the extension would involve an extra stage
to
compile the file holding that information and extract it in order to
generate
the wrapper. Going the other way around, specifying it all in extconf.rb
and
generating the .ml code that registers the functions from it would be
very
easy to implement, but would force one to use named functions(1).

In the meantime, I don't find the need to register the functions in
OCaml too
onerous, as it's at most one line per method, and the extra degree of
freedom
in the OCaml -> Ruby mapping is quite convenient (I can do e.g.
parameter
reordering with an anonymous function).

(1) another benefit from that would be the possibility to check that the
specified types are included in those from the .cmi (a file generated by
OCaml
with interface information)
Ae16cb4f6d78e485b04ce1e821592ae5?d=identicon&s=25 Martin DeMello (Guest)
on 2007-06-26 10:12
(Received via mailing list)
On 6/22/07, Mauricio Fernandez <mfp@acm.org> wrote:
> rocaml allows you to write Ruby extensions in Objective Caml.

exciting stuff! looking forward to playing with this.

martin
Ffcb418e17cac2873d611c2b8d8d891c?d=identicon&s=25 unknown (Guest)
on 2007-06-26 11:59
(Received via mailing list)
Mauricio:
*snip*
> In the meantime, I don't find the need to register the functions in
> OCaml too
> onerous, as it's at most one line per method, and the extra degree of
> freedom
> in the OCaml -> Ruby mapping is quite convenient (I can do e.g.
> parameter
> reordering with an anonymous function).

*snip*

:-) Thanks for the reasoned reply. I've got ocaml installed, and I've
started reading the manual. It feels somewhat like it is to functional
programming as c is to imperative programming at the moment, which is
interesting - but could be a useless thought ;-) I don't like the syntax
a lot at the moment, but that's just lack of experience. I'm very
intregued by the open GL bindings.
D3944ff4e8bc05067a615579b6ef599b?d=identicon&s=25 Phil Tomson (Guest)
on 2007-06-26 18:32
(Received via mailing list)
On 6/22/07, Mauricio Fernandez <mfp@acm.org> wrote:
> in Ruby and OCaml (OCaml exceptions are passed to Ruby).
>

Nice.  I've been learning OCaml for the last few months.  I'll
definitely give this a try.

Phil
Dd76a12d66f843de5c5f8782668e7127?d=identicon&s=25 Mauricio Fernandez (Guest)
on 2007-06-27 12:01
(Received via mailing list)
On Tue, Jun 26, 2007 at 06:58:03PM +0900, benjohn@fysh.org wrote:
> started reading the manual. It feels somewhat like it is to functional
> programming as c is to imperative programming at the moment, which is
> interesting - but could be a useless thought ;-)

If you mean by this that performance is easy to predict, you're very
right :)

Allow me expand a bit on this. One of the distinct advantages of OCaml
is that
the compiler doesn't perform any deep magic (for instance, it doesn't do
loop-invariant code motion, IIRC). How can this be a good thing? It
means that
it's very easy to get from e.g. guessing why so much time is spent in
the GC
to giving the finishing touches to your code, because you can predict
the
effect of your modifications quite easily (and OCaml also has nice
profiling
tools for both native code and bytecode). OCaml's excellent performance
is a
testament to the effectiveness of a good basic compilation strategy.  So
you
can reword the initial statement in a less enigmatic way: Objective Caml
*doesn't need* deep magic in the compiler to yield good performance.

This is especially true when you compare OCaml to languages with a lazy
evaluation discipline like Haskell. I don't have any sizeable experience
with
the latter, but I've often heard about the difficulty of predicting the
performance of code involving lazy evaluation, even for seasoned
programmers,
like a single line change *mysteriously* turning some O(n) code into
O(exp(n))
or the other way around. This caught my attention when I read it in
Okasaki's
book[1], which I can't recommend enough:

 "Historically, the most common technique for analyzing lazy programs
has been
  pretending that they are actually strict".

Okasaki introduces a basic framework to perform such analyzes, but the
fact
remains that eager evaluation is easier to understand in that regard.

> I don't like the syntax a lot at the moment, but that's just lack of
> experience. I'm very intregued by the open GL bindings.

There are a few problems with the original syntax, but nothing that
cannot be
solved with a couple parentheses here and there (for instance, with
nested
pattern matching expressions). There is an alternative syntax ("revised
syntax") which addresses these "ambiguities" by adding some punctuation
and
differentiating constructions that are arguably too hard to tell apart
in the
original syntax, but it doesn't seem to be widely used (I prefer the
original
one myself, but it's probably because it's the first one I was exposed
to).
camlp4 can take code in one syntax and rewrite it using the other, so
you
don't have to decide upfront what you will be using in the end.

[1]
Purely Functional Data Structures by Chris Okasaki, Cambridge University
Press, 1998.

You can find the PhD. thesis on which that book was based at
http://www.cs.cmu.edu/~rwh/theses/okasaki.pdf
Aee77dba395ece0a04c688b05b07cd63?d=identicon&s=25 Daniel Berger (Guest)
on 2007-06-29 20:03
(Received via mailing list)
On Jun 22, 5:35 am, Mauricio Fernandez <m...@acm.org> wrote:
> rocamlallows you to write Ruby extensions in Objective Caml

Why go half way? How about a Ruby interpreter implemented in OCaml?

ORuby now! :)

Dan
D3944ff4e8bc05067a615579b6ef599b?d=identicon&s=25 Phil Tomson (Guest)
on 2007-06-30 08:52
(Received via mailing list)
On 6/29/07, Daniel Berger <djberg96@gmail.com> wrote:
> On Jun 22, 5:35 am, Mauricio Fernandez <m...@acm.org> wrote:
> > rocamlallows you to write Ruby extensions in Objective Caml
>
> Why go half way? How about a Ruby interpreter implemented in OCaml?
>
> ORuby now! :)

I've been thinking along similar lines.
Here's a Google tech talk about compiling Python to OCaml:
http://video.google.com/videoplay?docid=-207775537...

It should be possible to do something similar with Ruby.
...yet another Ruby implementation.

Phil
This topic is locked and can not be replied to.