C ext / threading / callback issue, ruby-1.9.1p129

Hi,

I’ve never written a C extension for Ruby before, and I’m struggling
with
some demo code which uses C and/or Ruby pthreads, and a C or Ruby
callback
(address passed as an arg into the C lib).

There’s a watcher thread (in C) that waits for an event (the creation of
a file), then triggers whichever callback is specified.

Triggering the Ruby callback causes MRI to crash. The C callback
works…
but when it completes, all Ruby threads appear to suspend (control is
never returned to Ruby / the threads are never rejoined?)

I’m guessing that the MRI crash is a dereferencing bug?? I haven’t
figured
that out obviously, but I have a feeling that the second problem
(threads
never rejoin?) will show up after the crash is resolved.

I put the code up on github: http://github.com/reynhout/callme/ . There
is a bunch more description there, too.

There are two versions, C_ext (described above), and FFI_lib which uses
ruby-ffi to achieve the same goals … but is also failing similarly.
I worked on the FFI_lib one first, then backed down to a straight C ext
trying to simplify it and make the bugs easier to find.

I’m pretty sure that if I can get the C_ext version working, the FFI_lib
one will fall into place. The ultimate goal is a Ruby library for
Apache
ZooKeeper. The rest of the code is going great, but many ZK functions
depend on this sort of mechanism…

I hope the bugs are obvious to anyone who has written a C ext before.
I’d greatly appreciate any ideas or pointers (no pun intended :)…

Thank you!
Andrew

There’s a watcher thread (in C) that waits for an event (the creation of
a file), then triggers whichever callback is specified.

Hmm. MRI isn’t “used” to other threads running–are you running 1.8.6
or something else?
=r

On Tue, Jul 14, 2009 at 10:57:07AM +0900, Roger P. wrote:

There’s a watcher thread (in C) that waits for an event (the creation of
a file), then triggers whichever callback is specified.

Hmm. MRI isn’t “used” to other threads running–are you running 1.8.6
or something else?

I’m using 1.9.1p129, but I tried the FFI version under 1.8.6 too, and
got similar MRI crashes and/or stuck threads. I did wrap the callback
invocation inside rb_thread_blocking_region, though I’m not sure it’s
necessary for this sample code.

I haven’t written a 1.8.6 version of the C ext (the syntax/semantics
are slightly different than 1.9), but I’d be very surprised to see
anything change – these failures are consistent with MRI through
multiple versions and approaches.

This code DOES work correctly under JRuby. JRuby has a different
threading model, of course, but I don’t understand it well enough to
determine why it fails under MRI.

Sample code and details at: http://github.com/reynhout/callme/

Thanks again,
Andrew

Roger P. wrote:

Does the callback use ruby code from within the thread blocking region?

Yes, e.g.:

rb_thread_blocking_region((rb_blocking_function_t *)w->callback, tmp_x,
RUBY_UBF_IO, 0);

w->callback is the address of the Ruby proc. The address is correct
afaict, but note that this does crash MRI before the callback appears to
execute (dereferencing failure?). When w->callback is the address of
the C function, the callback runs, but the Ruby threads apparently
suspend.

Both callbacks in this sample code are very simple – they just print
the int value of tmp_x, which is the number of seconds since the watcher
thread was started.

Thanks,
Andrew

On Jul 14, 10:48 am, Andrew R. [email protected] wrote:

execute (dereferencing failure?). When w->callback is the address of
the C function, the callback runs, but the Ruby threads apparently
suspend.

Both callbacks in this sample code are very simple – they just print
the int value of tmp_x, which is the number of seconds since the watcher
thread was started.

Ruby-FFI issues, more related to callbacks can be asked at Ruby-ffi
mailing list:

http://kenai.com/projects/ruby-ffi/lists

If memory plays well, I saw something related to callbacks not long
ago there:

http://kenai.com/projects/ruby-ffi/lists/dev/archive

Sometimes the API hasn’t been updated across the implementations.

Luis L. wrote:

Ruby-FFI issues, more related to callbacks can be asked at Ruby-ffi
mailing list:

Sorry, I didn’t mean to confuse the issue by mentioning FFI. I did try
it with FFI initially (and have been discussing the problem on the
ruby-ffi list), but I’ve completely removed FFI from the current version
of the code. It’s a straight C ext with no dependencies except gcc,
MRI, and mkmf.

Thanks,
Andrew

I’m using 1.9.1p129, but I tried the FFI version under 1.8.6 too, and
got similar MRI crashes and/or stuck threads. I did wrap the callback
invocation inside rb_thread_blocking_region, though I’m not sure it’s
necessary for this sample code.

Does the callback use ruby code from within the thread blocking region?
=r

rb_thread_blocking_region((rb_blocking_function_t *)w->callback, tmp_x,
RUBY_UBF_IO, 0);

Hmm. This question might be more easily answered by the core folk, but
perhaps you’re assigning the value of w->callback to a ruby proc object,
when what you wanted was perhaps a pointer to a C method.

So the question is how to call proc objects in C?
I’ve never done it, but glancing at README.EXT appears to be something
like rb_iterate or rb_block_call (the latter for 1.9 esp.)

Not sure of that’s right but it’s about as far as my know-how goes.
GL.
=r

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs