Celluloid 0.1.0: a concurrent object framework for Ruby

dubstep · June 21, 2011, 10:20pm

Celluloid is a concurrent object framework for Ruby inspired by Erlang
and
the Actor Model:

This is the second release of Celluloid, and introduces a new reentrant
method dispatch system based on fibers. More information about this new
system, the problems it solves, and how Celluloid uses fibers is
available
in this commit message:

github.com/tarcieri/celluloid

Wrap synchronous method dispatch in a fiber. Celluloid now requires 1.9

committed 03:17AM - 20 Jun 11 UTC

tarcieri

+25 -6

The initial release of Celluloid suffered from a pretty nasty problem/bug. Synch…ronous calls work fine so long as the actors in a call chain are all unique (e.g. A calls B calls C). However, if the call chain ever loops back around to the same actor (e.g. A calls B calls A) a deadlock occurs. A is blocked waiting for a response from B, so it never realizes B called it. Meanwhile B has asked A to handle a call and starts blocking waiting from a response from A. DEADLOCK! How can this be fixed? The only solution is to allow A to continue to service method calls even though the method it is currently dispatching is blocked waiting for a call response. If A could take another request while it's waiting for a call response from B, it could in turn service the new request B just made, and unblock the entire call chain. To accomplish this, we need to wrap the call in a fiber. Whenever an actor makes a synchronous call to another actor, the fiber can yield, and the actor can wait for the call response as it processes additional incoming method calls. As soon as the call response is received, the fiber is resumed, and method execution continues. Actors remain sequential and only process one method at a time, but switch back and forth between running methods whenever synchronous calls are made to other actors. With this approach in Celluloid, you no longer have any control of the order methods are executed in. Several actors (each running in their own thread) may be making method calls at the same time. The receivers process them in a first come, first served manner, but if the receiver makes another outgoing call, the actor suspends the method it's processing and moves on to the next request. The end result is you may just need to be a little more vigilant when working with instance variables. Other methods need to work with that state too, so keep it consistent. Don't mutate an object into an inconsistent state then make a method call. In general, try to preserve a class invariant. Don't write sloppy code with implementation logic coupled together between methods through instance variables. Treat instance variables as the state of the object, not a temporary space for passing state between method calls. That's what the stack is for. If you do that, which I hope you're doing anyway, then everything should be fine and concurrent objects Celluloid's actors provide should continue functioning as plain old Ruby objects with awesome concurrent and asynchronous capabilities.

Celluloid’s concurrent objects should now function just like Ruby
objects
whenever synchronous calls are used (the previous release suffered
deadlocks
in certain cases). Celluloid objects also support asynchronous method
calls
which allow them to run methods in the background without blocking the
caller.

Because this release uses fibers, it no longer works on 1.8.7 and
requires
you use 1.9 mode on JRuby (see README for instructions). Rubinius is
still
supported in 1.8 mode and Celluloid will work on Rubinius 2.0.0pre.

murfhy · June 22, 2011, 12:23am

Tony A. [email protected] wrote:

This is the second release of Celluloid, and introduces a new reentrant
method dispatch system based on fibers. More information about this new
system, the problems it solves, and how Celluloid uses fibers is available
in this commit message:

github.com/tarcieri/celluloid

Wrap synchronous method dispatch in a fiber. Celluloid now requires 1.9

committed 03:17AM - 20 Jun 11 UTC

tarcieri

+25 -6

The initial release of Celluloid suffered from a pretty nasty problem/bug. Synch…ronous calls work fine so long as the actors in a call chain are all unique (e.g. A calls B calls C). However, if the call chain ever loops back around to the same actor (e.g. A calls B calls A) a deadlock occurs. A is blocked waiting for a response from B, so it never realizes B called it. Meanwhile B has asked A to handle a call and starts blocking waiting from a response from A. DEADLOCK! How can this be fixed? The only solution is to allow A to continue to service method calls even though the method it is currently dispatching is blocked waiting for a call response. If A could take another request while it's waiting for a call response from B, it could in turn service the new request B just made, and unblock the entire call chain. To accomplish this, we need to wrap the call in a fiber. Whenever an actor makes a synchronous call to another actor, the fiber can yield, and the actor can wait for the call response as it processes additional incoming method calls. As soon as the call response is received, the fiber is resumed, and method execution continues. Actors remain sequential and only process one method at a time, but switch back and forth between running methods whenever synchronous calls are made to other actors. With this approach in Celluloid, you no longer have any control of the order methods are executed in. Several actors (each running in their own thread) may be making method calls at the same time. The receivers process them in a first come, first served manner, but if the receiver makes another outgoing call, the actor suspends the method it's processing and moves on to the next request. The end result is you may just need to be a little more vigilant when working with instance variables. Other methods need to work with that state too, so keep it consistent. Don't mutate an object into an inconsistent state then make a method call. In general, try to preserve a class invariant. Don't write sloppy code with implementation logic coupled together between methods through instance variables. Treat instance variables as the state of the object, not a temporary space for passing state between method calls. That's what the stack is for. If you do that, which I hope you're doing anyway, then everything should be fine and concurrent objects Celluloid's actors provide should continue functioning as plain old Ruby objects with awesome concurrent and asynchronous capabilities.

I initially had one question/concern with this release I answered for
myself by reading the code:

Q: Did it switch to using fibers (and drop threads), so it becomes like
Revactor uses and thus need its own scheduler?

A: No, it still uses threads in addition to fibers. It’ll still work
with
existing, thread-safe/aware code and not require rearchitecting your
application like Revactor did.

murfhy · June 23, 2011, 8:08am

Hi,

Thanks for this useful framework with its nice and Ruby like API.

I’m wondering whether you intend to support asynchronous call with
return value(s). Such call would return a future, then the caller would
be able to retrieve or try to retrieve the value(s) at any time.
I know that there is Celluloid::Future but this is not really
“integrated” with the asynchronous call feature. So the nice API is not
complete regarding this aspect.

Keep on the good work.
Chauk-Mean

PS : The readme still indicates that ‘new’ creates a normal object, yet
new is now an alias for spawn.

Tony A. wrote in post #1006735:

Celluloid is a concurrent object framework for Ruby inspired by Erlang
and
the Actor Model:

Github: GitHub - tarcieri/celluloid: My personal fork of Celluloid. Please don't open PRs or issues here.

RDoc: http://celluloid.github.com/

murfhy · June 29, 2011, 1:28am

On Tue, Jun 28, 2011 at 2:54 AM, Chauk-Mean P. [email protected]
wrote:

Hi,

I’ve posted a reply to your announcement to the Ruby Forum.
Celluloid 0.1.0: a concurrent object framework for Ruby - Ruby - Ruby-Forum

Maybe you have not received it.

Hey, not sure why but I didn’t get that email. Are you posting through
Usenet or something?

Thanks for this useful framework with its nice and Ruby like API.

I’m wondering whether you intend to support asynchronous call with
return value(s). Such call would return a future, then the caller would
be able to retrieve or try to retrieve the value(s) at any time.
I know that there is Celluloid::Future but this is not really
“integrated” with the asynchronous call feature. So the nice API is not
complete regarding this aspect.

Yeah, I wanted to do this sort of thing in Reia. I was going to do
obj<-futurized_method_call(…) for that, which would return a future.
The
cooresponding obj->methoI’m not sure what syntax would be appropriate
for
Celluloid. Perhaps obj.future(:method, *args, &block) ?

PS : The readme still indicates that ‘new’ creates a normal object, yet
new is now an alias for spawn.

Oops, I changed that in the most recent release and didn’t update the
README. Thanks!