# Rb\_funcall() Ruby code callback invoked from within a native **URL:** \ **Category:** Ruby\ **Created:** [October 4, 2006, 7:29am UTC](https://www.ruby-forum.com/t/rb-funcall-ruby-code-callback-invoked-from-within-a-native/70950 "2006-10-04T07:29:38Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![ru551an](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e9a140/32.png) [@ru551an](https://www.ruby-forum.com/u/ru551an)\ **Post date:** [October 4, 2006, 7:29am UTC](https://www.ruby-forum.com/t/rb-funcall-ruby-code-callback-invoked-from-within-a-native/70950/1 "2006-10-04T07:29:38Z")
Dear All, I have created a Ruby extension that runs under both Windows (Ruby 1.8.4) and Linux (Ruby 1.8.5), using SWIG. The extension is written in C and is multi-threaded, thanks to PThreads. Several C functions are exported to Ruby to perform various operations. One of the C functions is a callback notifier whose usage is as follows: 1. The main Ruby program must first invoke a C native function: Libext.ext\_notifier\_add(dhtid, proc, “method@class”, 0) where the “method@class” is a dummy used normally for the Java version of the extension, and the main parameter is the proc address corresponding to a Ruby block like: proc = Proc.new { | value | Libext.ext\_put(id, key, value, index) } . This registers a Ruby callback with the C extension. 2. The main Ruby program will also invoke C native functions that will result in PThreads being created to receive and process UDP messages. On average, some 50 PThreads are active at any time inside the C extension. There are many separate instances of the Ruby program (processes) running simultaneously on the same or different (networked) computers. 3. When a Ruby program calls Libext.ext\_put(id, key, value, index) a message will be sent to another node. On the destination node upon receiving the message some parameters are evaluated and if applicable the registered Ruby callback is invoked with: rb\_thread\_critical = Qtrue; res = rb\_funcall(proc, rb\_intern(“call”), 4, rb\_str\_new2(triggerstr), rb\_str\_new2(evalstr), INT2NUM(index), LONG2NUM(state-\>id)); rb\_thread\_critical = 0; 1. The callback will normally contain only fairly simple statements but may as well need to invoke C native functions (as in the example above). So far I have seen the following problems with this solution: a) cross-thread violation on rb\_thread\_schedule() : this happens on the Windows version of the C extension when NO native function are invoked from the callback and rb\_thread\_critical = Qtrue; is not used. Under heavy load of incoming UDP messages from other nodes, the callback execution will produce that error. It seems that the Ruby scheduler is preempting the execution of the Ruby callback and determines that a native thread has invoked it (a bad thing it seems). Adding rb\_thread\_critical = Qtrue; makes it impossible for the scheduler to change the thread of execution and eliminates the error except for once every several hundred callbacks under sustained load. b) SystemStackError: stack level too deep : this happens exclusively on Linux (Debian) after only a few callbacks when NO native function are invoked and rb\_thread\_critical = Qtrue; is used. The C extension uses a fairly deeply nested function calls (with several layers of callbacks). I assume that Ruby somehow (how?) detects that the stack has grown past permitted levels (which are?) and throw that error message. Now coming to the main issue… I would like to get rid of problem b) and one possible solution that occurred to me is to create a native PThread that will perform the Ruby callback and exit when all Ruby and native method/function calls have completed. This seems possible under Java JNI through the use of AttachCurrentThread() that converts temporarily a native thread into a Java thread. However, I did not discover any similar function in Ruby. Besides, I read plenty of comments that mixing native threads with the Ruby interpreter is a bad idea. As a sideline, all the testing has been done from the command line using ruby.exe, although under Windows I will soon have to migrate to Active Ruby. Being new to all this Ruby internal coding I would like to sollicit your feeback and advice on how I could tackle this problem. I hope I have overlooked something that will make my life easier 🙂 Overall, Ruby has proven a fantastic language to develop on but I fear that compared to JNI the support for native threads is severely lacking ☹ Cheers, Serge K. CTO - Simitel ---
**Author:** ![ru551an](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e9a140/32.png) [@ru551an](https://www.ruby-forum.com/u/ru551an)\ **Post date:** [October 4, 2006, 7:51am UTC](https://www.ruby-forum.com/t/rb-funcall-ruby-code-callback-invoked-from-within-a-native/70950/2 "2006-10-04T07:51:40Z")
On 10/4/06, Serge K. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Dear All, > > I have created a Ruby extension that runs under both Windows (Ruby > 1.8.4) and Linux (Ruby 1.8.5), using SWIG. The extension is written in > C and is multi-threaded, thanks to PThreads. Several C functions are > exported to Ruby to perform various operations. > > One of the C functions is a callback notifier whose usage is as > follows: You’re making some assumptions here which I think are challenging. Whenever your Ruby code calls native code, Ruby will block until the native call returns. If the native call makes calls back into Ruby, you’re generally safe. But if you spin native threads that call Ruby code (as you suggest here) then you will have no way to completely synchronize the Ruby code that is now running on multiple native threads. There have been several discussions on this list about how to send events back and forth between Ruby and native code running on native threads. The ideas which seem to work generally involve sending messages on socketpairs or pipes. A look through the archives may help you. If you have the freedom to re-architect your application, look at the EventMachine library. It was designed to enable applications like yours to be implemented without threads.