Ruby 1.9.2-p0: segmenfault in multithreading when using a custom C extension

Ruby
1

Hi, I’ve coded a SIP protocol parser (very close to the Ragel HTTP
parser) as a Ruby C extension. It works nice (even with very high
traffic)… until now when I’ve tryed to add multithread logic to
handle the parser SIP message. It seems complex but I’ve been able to
create a very simple test script that generates the segmentfault.

To summarize, the following code DOES NOT crash:

data = “INVITE sip:[email protected] SIP/2.0\r\n\r\n”
parser = SIP::MessageParser.new

100000.times do
msg = SIP::Message.new
puts “msg.object_id = #{msg.object_id}”
parser.execute(msg, data.to_str, 0)
parser.reset

t = Thread.new { }
end

but the following code DOES crash:

threads = [ ]
data = “INVITE sip:[email protected] SIP/2.0\r\n\r\n”
parser = SIP::MessageParser.new

100000.times do
msg = SIP::Message.new
puts “msg.object_id = #{msg.object_id}”
parser.execute(msg, data.to_str, 0)
parser.reset

threads << Thread.new { }
end

The segmentfault occurs randomly in any iteration of the loop (usually
after 4900 iterations).
As you can see I do nothing within the thread. The only difference
between both cases is the fact that in the second case (the crashing
one) the thread is stored in an array.

And worse: I get segmentfault for MANY reasons (randomly too):

a) In line: threads << Thread.new { }
– control frame ----------
c:0008 p:---- s:0032 b:0032 l:000031 d:000031 CFUNC :initialize
c:0007 p:---- s:0030 b:0030 l:000029 d:000029 CFUNC :new

b) In line: parser.execute(msg, data.to_str, 0)

c) When Message#initialize() executes “@timestamp = Time.now”:
– control frame ----------
c:0011 p:---- s:0039 b:0039 l:000038 d:000038 CFUNC :(null)
c:0010 p:---- s:0037 b:0037 l:000036 d:000036 CFUNC :now
c:0009 p:0017 s:0034 b:0034 l:000033 d:000033 METHOD /xxxx/message.rb:66

d) In line: puts “msg.object_id = #{msg.object_id}”

In other server with ruby 1.9.1p376 I’ve got, just once, a different
error (not a segmentfault in fact):

in execute': methodhash’ called on terminated
object (0x00000017c48ac8) (NotImplementedError)

Any help please? I cannot imagine the reason of this issue, neither
I’m sure that it’s caused by my C extension.

Thanks a lot.

2

2011/1/11 Iñaki Baz C. [email protected]:

Hi, I’ve coded a SIP protocol parser (very close to the Ragel HTTP
parser) as a Ruby C extension. It works nice (even with very high
traffic)… until now when I’ve tryed to add multithread logic to
handle the parser SIP message. It seems complex but I’ve been able to
create a very simple test script that generates the segmentfault.

The segment fault also occurs with Ruby 1.9.2-p136.

3

2011/1/12 Iñaki Baz C. [email protected]:

2011/1/11 Iñaki Baz C. [email protected]:

Hi, I’ve coded a SIP protocol parser (very close to the Ragel HTTP
parser) as a Ruby C extension. It works nice (even with very high
traffic)… until now when I’ve tryed to add multithread logic to
handle the parser SIP message. It seems complex but I’ve been able to
create a very simple test script that generates the segmentfault.

The segment fault also occurs with Ruby 1.9.2-p136.

I’ve repeated the same test but using the Ragel HTTP parser (in
em-http-request gem) and it doesn’t crash, so I expect some bug in my
C code :frowning:

4

2011/1/12 Iñaki Baz C. [email protected]:

I’ve repeated the same test but using the Ragel HTTP parser (in
em-http-request gem) and it doesn’t crash, so I expect some bug in my
C code :frowning:

I think I’ve found the problem! and it could be related to a bug in
Ruby:

In my SIP parser (Ruby C extension like Ragel HTTP parser) I generate
Ruby string encoded in UTF-8 using:

rb_enc_str_new(s, len, rb_utf8_encoding());

But if I change it to:

rb_str_new(s, len)

then the segment fault does NEVER occur!

5

Hello Iñaki,

“Iñaki Baz C.” [email protected] wrote in post #974123:

2011/1/12 Iñaki Baz C. [email protected]:
In order to demostrate the problem I’ve reported the bug in Ruby
tracker by providing a code that generates the segmentfault just when
using rb_enc_str_new:

http://redmine.ruby-lang.org/issues/show/4272

Thanks for taking the time to file that issue, along with instructions
on how to reproduce the problem.

In my experience, C extension problems can be disheartening (especially
when nobody seems to care on the ruby-(talk|core) mailing lists) and I
know it takes a lot of energy to continue to solve or workaround those
problems.

I just wanted to say that I appreciate your efforts and I encourage you
to continue following up on the Ruby Issue Tracking System periodically.
It may take months, even a year, before you get any response, but do not
lose heart, for your efforts shows true determination and spirit.

Cheers!

6

2011/1/12 Iñaki Baz C. [email protected]:

then the segment fault does NEVER occur!

In order to demostrate the problem I’ve reported the bug in Ruby
tracker by providing a code that generates the segmentfault just when
using rb_enc_str_new:

http://redmine.ruby-lang.org/issues/show/4272

7

2011/1/12 Suraj K. [email protected]:

In my experience, C extension problems can be disheartening (especially
when nobody seems to care on the ruby-(talk|core) mailing lists) and I
know it takes a lot of energy to continue to solve or workaround those
problems.

I just wanted to say that I appreciate your efforts and I encourage you
to continue following up on the Ruby Issue Tracking System periodically.
It may take months, even a year, before you get any response, but do not
lose heart, for your efforts shows true determination and spirit.

Hi Suraj, thanks a lot for your response :slight_smile:

BTW I’ve tryed the test script with ruby-1.9.1-p376 and it DOES NOT
crash!