Forum: Ruby embedding ruby

Hello,

After reading a little, I found out that ruby was not designed with
embedding in mind. However, I really love Ruby and I would like to embed
it into my application.

I would like to know if the problem with embedding ruby is that there
are global variables and also because of the stack.

If this is the case, can I assume that it is safe to use the ruby engine
if I initialize it in the main process, before any thread is launched in
my C++ application (so the stack is in the lower position) and I use
mutex to prevent more than one run at any time (for the global
variables)?

Thank you very much in advance.
Josep.

Josep Pujol wrote:
> 
> I would like to know if the problem with embedding ruby is that there
> are global variables and also because of the stack.
> 
> If this is the case, can I assume that it is safe to use the ruby engine
> if I initialize it in the main process, before any thread is launched in
> my C++ application (so the stack is in the lower position) and I use
> mutex to prevent more than one run at any time (for the global
> variables)?

We embed ruby 1.8.x in a C++ app on Windows / OS X.

In our case, we run Ruby in a separate thread.  Only trick was that
the stack size of threads other than the main thread may be small by
default.  And we use boost threads.  So I had to modify boost to
create the thread with a large enough stack:

// boost/libs/thread/src/thread/cpp
#if defined(BOOST_HAS_PTHREADS)
    const size_t THREAD_STACK_SIZE_BYTES = 8 * 1024 * 1024;
    pthread_attr_t tattr;
    int res = 0;
    res = pthread_attr_init(&tattr);
    if (res != 0)
        throw thread_resource_error();
    res = pthread_attr_setstacksize(&tattr, THREAD_STACK_SIZE_BYTES);
    if (res != 0)
        throw thread_resource_error();
    res = pthread_create(&m_thread, &tattr, &thread_proxy, &param);
    (void) pthread_attr_destroy(&tattr);
    if (res != 0)
        throw thread_resource_error();



When Ruby code wants to call into the C++ side of the app, it's easy,
we just wrap whatever functionality from the app in a Ruby class, as
one would do with any ruby 'C' extension.

In the other direction, when an arbitrary C++ thread wants to talk to
Ruby, it's a little more complicated.

I've set up a queueing approach, where a C++ thread can create a
"RubyContext" object, and throw eval's at it.  Like:


   RubyContext r;

   std::string result = r.eval("...some_ruby_code...");


It works out pretty well.

I haven't tried embedding ruby 1.9.x yet.


Regards,

Bill

Thank you. I'll try it.

I plan to use ruby in several threads (it's a gui application and
each menu can trigger some ruby code). Is it safe enough to use
a mutex or I must initialize something (the stack?) for each
thread?

Regards,
Josep.

Bill Kelly wrote:
> 
> We embed ruby 1.8.x in a C++ app on Windows / OS X.
> 
> In our case, we run Ruby in a separate thread.  Only trick was that
> the stack size of threads other than the main thread may be small by
> default.  And we use boost threads.  So I had to modify boost to
> create the thread with a large enough stack:
> [...]
> It works out pretty well.
> 
> I haven't tried embedding ruby 1.9.x yet.
> 
> 
> Regards,
> 
> Bill

Josep Pujol wrote:
> After reading a little, I found out that ruby was not designed with
> embedding in mind. However, I really love Ruby and I would like to embed
> it into my application.

It is true that Ruby wasn't designed with embedding in mind. However,
this ...

> I would like to know if the problem with embedding ruby is that there
> are global variables and also because of the stack.

... has absolutely nothing to do with that. What you are asking about
has nothing to do with Ruby, this kind of behavior is specific to the
particular implementation you are using.

Rubinius was very much designed for embedding. It doesn't have any
global variables or other sort of global state and it minimizes its
use of the C stack as far as possible. It also has a well-defined
embedding API. (In earlier versions, it didn't use the C stack *at
all*, but that was changed for performance and interoperability
reasons.)

JRuby also is easy to embed, and it also has a well-defined embedding
API. It supports the scripting API which is part of the Java Platform
Specification, and also its own scripting API (RedBridge).

IronRuby supports the standard DLR embedding API and can be easily
embedded.

If you look at the actual commandline interpreter binaries for JRuby
and IronRuby, you will find that they are just pretty simple
commandline wrappers around the engines, using the respective
embedding APIs.

Personally, I would look at Rubinius, JRuby or IronRuby for embedding,
rather than MRI or YARV. But really, I would look at Lua, because
while you can find a Ruby *implementation* that is specifically
designed for embedding, this doesn't change the fact that the Ruby
*language* isn't.

jwm

Thanks for your answer.

Jörg W Mittag wrote:
> Personally, I would look at Rubinius, JRuby or IronRuby for embedding,
> rather than MRI or YARV. But really, I would look at Lua, because
> while you can find a Ruby *implementation* that is specifically
> designed for embedding, this doesn't change the fact that the Ruby
> *language* isn't.
> 
> jwm

I've just downloaded the rubinius code and I read this statement in the 
README file.

5. Goals

* Thread safety. Rubinius intends to be thread-safe so you could embed 
more
  than one interpreter in a single application. It does not currently 
meet
  this goal due to some components borrowed from the mainline Ruby 
interpreter.

Is this true also for the 1.0 version? (or this is an old statement that 
hasn't been removed)

About Lua, I've seen it, but my blind love about ruby is too strong... 
;-)

By the way, I've ruled out JRuby and IronRuby because they need extra 
components (java and mono).

Jörg W Mittag wrote:
> [...] But really, I would look at Lua, because
> while you can find a Ruby *implementation* that is specifically
> designed for embedding, this doesn't change the fact that the Ruby
> *language* isn't.

What do you mean? What is there in the language itself that hinders 
embedding?

Josep Pujol wrote:
> each menu can trigger some ruby code). Is it safe enough to use
> a mutex or I must initialize something (the stack?) for each
> thread?

Well again, my experience is only embedding MRI Ruby 1.8.x.  In this
case, the ruby interpreter itself runs in a single thread.

So on the Ruby side, all Ruby-threads are green threads running in
the single native Ruby thread.

We do indeed use Ruby threads to create GUI elements.  And in the
reverse direction we do forward GUI events from the C++ GUI event
thread back to Ruby.

What we never do is allow arbitrary C++ threads to call directly
into Ruby.

So in a situation where C++ makes a call like this:

  ruby_context.eval("GUI.forward_event(:click,"+control_id+")");

This 'eval' doesn't call ruby directly, but queues the command and
waits for the Ruby native thread to process the command.

So there is indeed a mutex (and a condition variable) involved here,
but its purpose is to implement the queuing mechanism between C++
threads and the Ruby interpreter thread.



For what it's worth, in the latest app I'm developing, I've changed
this so that Ruby runs in an entirely separate process.  And the C++
part of th app just acts like a "window server", which the Ruby
process connects to, and uses to create the GUI.


Regards,

Bill

On 2010-02-07, Albert Schlef <albertschlef@gmail.com> wrote:
> Jörg W Mittag wrote:
>> [...] But really, I would look at Lua, because
>> while you can find a Ruby *implementation* that is specifically
>> designed for embedding, this doesn't change the fact that the Ruby
>> *language* isn't.

> What do you mean? What is there in the language itself that hinders 
> embedding?

It's not so much that it "hinders" embedding is that embedding isn't the
explicit design goal.  Lua is *designed* with the intent that the 
primary
use would be embedding it in other programs.

-s

Albert Schlef wrote:
> Jörg W Mittag wrote:
>> [...] But really, I would look at Lua, because
>> while you can find a Ruby *implementation* that is specifically
>> designed for embedding, this doesn't change the fact that the Ruby
>> *language* isn't.
> What do you mean? What is there in the language itself that hinders 
> embedding?

Everything that you don't need.

That's kind of a dick answer. What I mean by that is that when you
embed a language into an application, this is usually a very special
purpose deal. Which means that all the stuff which makes Ruby
*brilliant* as a general purpose language, can get in the way. Do you
really need an almost Turing-complete Regexp implementation in a CAD
program? Or text processing? Do you need Database I/O in a game AI?
Arbitrary precision integers in a text editor?

Lua is often criticized for its small (to almost non-existent)
standard library (compared to Python, Ruby, Java, .NET), but when the
main purpose is as a special purpose embedded language, then there
simply isn't that much "standard" functionality that you could put in
a standard library. What do Adobe Lightroom, World of Warcraft and
NginX have in common that you could put there?

Lua is designed in such a way that the embedding program *is* the
"standard" library (or more specifically *provides* the library). Ruby
comes with "batteries included" which is really great if you want to
run it autonomous, but is just annoying when you want to hook it up to
your application's power supply.

jwm

Josep Pujol wrote:
> 
> 5. Goals
> 
> * Thread safety. Rubinius intends to be thread-safe so you could embed 
>   more than one interpreter in a single application. It does not currently 
>   meet this goal due to some components borrowed from the mainline Ruby 
>   interpreter.
> 
> Is this true also for the 1.0 version? (or this is an old statement that 
> hasn't been removed)

I'm not sure. I haven't followed Rubinius recently, and pretty much
*everything* has changed since I last followed it closely: a new VM,
three new parsers, a new compiler, a new garbage collector, a new
threading model. Literally *nothing* is like when I last looked at it.

However, please note that this snippet only talks about embedding
*multiple* Rubinius interpreters into a *single process*. Also, it
specifically mentions that the problematic parts are borrowed from
MRI/YARV, which means that if you use one of those, you will have to
deal with the exact same problems.

jwm

On Feb 12, 2010, at 4:05 PM, Jörg W Mittag wrote:

>> README file.
> 
> I'm not sure. I haven't followed Rubinius recently, and pretty much
> *everything* has changed since I last followed it closely: a new VM,
> three new parsers, a new compiler, a new garbage collector, a new
> threading model. Literally *nothing* is like when I last looked at it.
> 
> However, please note that this snippet only talks about embedding
> *multiple* Rubinius interpreters into a *single process*. Also, it
> specifically mentions that the problematic parts are borrowed from
> MRI/YARV, which means that if you use one of those, you will have to
> deal with the exact same problems.



  You can currently run multiple rubinius VM's in one process, each VM 
will run on it's own native thread in parallel and there is a simple 
message passing interface to pass messages between running VM's.

Cheers-
Ezra Zygmuntowicz
ez@engineyard.com