Strange bug in irb1.9


#1

Hi.

I am developing bindings for libconfig and discovered a strange bug in
irb1.9 (ruby 1.9.0 (2008-06-20 revision 17482) [i486-linux], irb
0.9.5(05/04/13); Debian GNU/Linux 5.0/lenny): ruby1.9 executes my
script nice, but irb crashes at some random place. Crash is affected
by most theoretically unlinked commands and may disappear even after
removing a comment (sic).
You can download my extension sources here
(http://files.whitequark.ru/libconfig-ruby.tbz2).

Crashing script:
— BEGIN —
require ‘rconfig’
c = Config.new

c.read(‘test.cfg’)

=> false

c.append ‘fixnum’, Config::Fixnum.new(150)

#Config::Fixnum...

f1 = Config::Fixnum.new(1)
c.append ‘another_fixnum’, f1

f2 = Config::Fixnum.new(256)
c.append ‘next_fixnum’, f2

p c.size

=> 2

c.delete(f1) # by element
c.delete(0) # by index
c.delete(‘next_fixnum’) # by name

note: (at now) you cannot delete nested elements by Config#delete

you can do

c[‘nested.element’].parent.delete(c[‘nested.element’])

p c.size

=> 0

l = Config::List.new
c.append ‘the_list’, l

l.append Config::String.new(“abcdef”)
l << Config::Float.new(3.14)
— END —

Sample log:
— BEGIN —
whitequark@hell:~$ tar xf libconfig-ruby.tbz2
whitequark@hell:~$ cd libconfig-ruby/ext/
whitequark@hell:~/libconfig-ruby/ext$ ruby1.9 extconf.rb
checking for rb_block_call() in ruby/ruby.h… yes
creating Makefile
whitequark@hell:~/libconfig-ruby/ext$ make
cc -I. -I/usr/include/ruby-1.9.0/i486-linux -I/usr/include/ruby-1.9.0
-I. -DHAVE_RB_BLOCK_CALL -D_FILE_OFFSET_BITS=64 -fPIC
-fno-strict-aliasing -g -g -O2 -O2 -g -Wall -Wno-parentheses -fPIC
-o rconfig.o -c rconfig.c
cc -shared -o rconfig.so rconfig.o -L. -L/usr/lib -L. -rdynamic
-Wl,-export-dynamic -lruby1.9 -lconfig -lpthread -lrt -ldl
-lcrypt -lm -lc
whitequark@hell:~/libconfig-ruby/ext$ ruby1.9 crash.rb
3

whitequark@hell:~/libconfig-ruby/ext$ irb1.9 crash.rb
crash.rb(main):001:0> require ‘rconfig’
=> true
crash.rb(main):002:0> c = Config.new
=> #<Config:0x9dd76d4 @config=#Object:0x9dd76c0>
crash.rb(main):003:0>
crash.rb(main):004:0* c.read(‘test.cfg’)
=> false
crash.rb(main):005:0> # => false
crash.rb(main):006:0*
crash.rb(main):007:0* c.append ‘fixnum’, Config::Fixnum.new(150)
=> #<Config::Fixnum:0x9dc0dd0 @setting=#Object:0x9dc0d30,
@value=150, @format=0>
crash.rb(main):008:0> # #Config::Fixnum...
crash.rb(main):009:0*
crash.rb(main):010:0* f1 = Config::Fixnum.new(1)
=> #<Config::Fixnum:0x9db6a9c @setting=nil, @value=1, @format=0>
crash.rb(main):011:0> c.append ‘another_fixnum’, f1
=> #<Config::Fixnum:0x9db6a9c @setting=#Object:0x9db11f0, @value=1,
@format=0>
crash.rb(main):012:0>
crash.rb(main):013:0* f2 = Config::Fixnum.new(256)
=> #<Config::Fixnum:0x9daa828 @setting=nil, @value=256, @format=0>
crash.rb(main):014:0> c.append ‘next_fixnum’, f2
=> #<Config::Fixnum:0x9daa828 @setting=#Object:0x9da86f4,
@value=256, @format=0>
crash.rb(main):015:0>
crash.rb(main):016:0* p c.size
3
=> 3
crash.rb(main):017:0> # => 2
crash.rb(main):018:0*
crash.rb(main):019:0* c.delete(f1) # by element
=> nil
crash.rb(main):020:0> c.delete(0) # by index
=> nil
crash.rb(main):021:0> c.delete(‘next_fixnum’) # by name
=> nil
crash.rb(main):022:0> # note: (at now) you cannot delete nested
elements by Config#delete
crash.rb(main):023:0* # you can do
c[‘nested.element’].parent.delete(c[‘nested.element’])
crash.rb(main):024:0*
crash.rb(main):025:0* p c.size

=> 0
crash.rb(main):026:0> # => 0
crash.rb(main):027:0*
crash.rb(main):028:0* l = Config::List.new
=> #<Config::List:0x9ccb128 @list=[], @setting=nil>
crash.rb(main):029:0> c.append ‘the_list’, l
=> #<Config::List:0x9ccb128 @list=[], @setting=#Object:0x9cc91fc>
crash.rb(main):030:0>
crash.rb(main):031:0* l.append Config::String.new(“abcdef”)
=> #<Config::String:0x9c7e92c @setting=#Object:0x9c7e8c8,
@value=“abcdef”, @format=0>
crash.rb(main):032:0> l << Config::Float.new(3.14)
crash.rb:32: [BUG] Segmentation fault
ruby 1.9.0 (2008-06-20 revision 17482) [i486-linux]

– control frame ----------
c:0024 p:---- s:0080 b:0080 l:000079 d:000079 CFUNC :<<
c:0023 p:0021 s:0076 b:0076 l:001f74 d:0002fc EVAL crash.rb:32
c:0022 p:---- s:0074 b:0074 l:000073 d:000073 FINISH :empty?
c:0021 p:---- s:0072 b:0072 l:000071 d:000071 CFUNC :eval
c:0020 p:0023 s:0065 b:0065 l:000064 d:000064 METHOD
/usr/lib/ruby/1.9.0/irb/workspace.rb:80
c:0019 p:0025 s:0058 b:0057 l:000056 d:000056 METHOD
/usr/lib/ruby/1.9.0/irb/context.rb:218
c:0018 p:0024 s:0052 b:0052 l:0007bc d:000051 BLOCK
/usr/lib/ruby/1.9.0/irb.rb:149
c:0017 p:0025 s:0044 b:0044 l:000043 d:000043 METHOD
/usr/lib/ruby/1.9.0/irb.rb:263
c:0016 p:0009 s:0039 b:0039 l:0007bc d:000038 BLOCK
/usr/lib/ruby/1.9.0/irb.rb:146
c:0015 p:0093 s:0035 b:0035 l:000022 d:000034 BLOCK
/usr/lib/ruby/1.9.0/irb/ruby-lex.rb:243
c:0014 p:---- s:0034 b:0034 l:000033 d:000033 FINISH :block_given?
c:0013 p:---- s:0032 b:0032 l:000031 d:000031 CFUNC :loop
c:0012 p:0007 s:0029 b:0029 l:000022 d:000028 BLOCK
/usr/lib/ruby/1.9.0/irb/ruby-lex.rb:230
c:0011 p:---- s:0029 b:0029 l:000028 d:000028 FINISH :each
c:0010 p:---- s:0027 b:0027 l:000026 d:000026 CFUNC :catch
c:0009 p:0017 s:0023 b:0023 l:000022 d:000022 METHOD
/usr/lib/ruby/1.9.0/irb/ruby-lex.rb:229
c:0008 p:0034 s:0020 b:0020 l:0007bc d:0007bc METHOD
/usr/lib/ruby/1.9.0/irb.rb:145
c:0007 p:0009 s:0017 b:0017 l:001d6c d:000016 BLOCK
/usr/lib/ruby/1.9.0/irb.rb:69
c:0006 p:---- s:0017 b:0017 l:000016 d:000016 FINISH :(null)
c:0005 p:---- s:0015 b:0015 l:000014 d:000014 CFUNC :catch
c:0004 p:0152 s:0011 b:0011 l:001d6c d:001d6c METHOD
/usr/lib/ruby/1.9.0/irb.rb:68
c:0003 p:0033 s:0006 b:0006 l:000005 d:000005 TOP /usr/bin/irb1.9:12
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
:private_class_method
c:0001 p:0000 s:0002 b:0002 l:000001 d:000001 TOP :17

DBG> : “crash.rb:32:in <<'" DBG> : "crash.rb:32:inirb_binding’”
DBG> : “/usr/lib/ruby/1.9.0/irb/workspace.rb:80:in eval'" DBG> : "/usr/lib/ruby/1.9.0/irb/workspace.rb:80:inevaluate’”
DBG> : “/usr/lib/ruby/1.9.0/irb/context.rb:218:in evaluate'" DBG> : "/usr/lib/ruby/1.9.0/irb.rb:149:inblock (2 levels) in
eval_input’”
DBG> : “/usr/lib/ruby/1.9.0/irb.rb:263:in signal_status'" DBG> : "/usr/lib/ruby/1.9.0/irb.rb:146:inblock in eval_input’”
DBG> : “/usr/lib/ruby/1.9.0/irb/ruby-lex.rb:243:in block (2 levels) in each_top_level_statement'" DBG> : "/usr/lib/ruby/1.9.0/irb/ruby-lex.rb:230:inloop’”
DBG> : “/usr/lib/ruby/1.9.0/irb/ruby-lex.rb:230:in block in each_top_level_statement'" DBG> : "/usr/lib/ruby/1.9.0/irb/ruby-lex.rb:229:incatch’”
DBG> : “/usr/lib/ruby/1.9.0/irb/ruby-lex.rb:229:in
each_top_level_statement'" DBG> : "/usr/lib/ruby/1.9.0/irb.rb:145:ineval_input’”
DBG> : “/usr/lib/ruby/1.9.0/irb.rb:69:in block in start'" DBG> : "/usr/lib/ruby/1.9.0/irb.rb:68:incatch’”
DBG> : “/usr/lib/ruby/1.9.0/irb.rb:68:in start'" DBG> : "/usr/bin/irb1.9:12:in'”
– backtrace of native function call (Use addr2line) –
0xb7efac46
0xb7e243a9
0xb7e2441a
0xb7eaba56
0xb7f6e40c
0xb7ee5980
0xb7ef2b07
0xb7ef3a12
0xb7e56f2a
0xb7e013e5
0xb7f61434
0xb7ee6bfb
0xb7ee9b55
0xb7eec59e
0xb7ef1a84
0xb7ef2613
0xb7ef2a4e
0xb7ee6c30
0xb7ee9b55
0xb7eeb8b4
0xb7ef1a84
0xb7ef6add
0xb7e263c1
0xb7ee872c
0xb7ee6c12
0xb7ee9b55
0xb7eeb8b4
0xb7ef1a84
0xb7ef64a9
0xb7ee6c30
0xb7ee9b55
0xb7eeb8b4
0xb7ef1a84
0xb7ef64a9
0xb7ee6c30
0xb7ee9b55
0xb7eeb8b4
0xb7ef1a84
0xb7ef1cad
0xb7e26bd7
0xb7e28156
0x8048794
0xb7c17775
0x8048681


#2

On 24.03.2009 14:36, Peter Z. wrote:

I am developing bindings for libconfig and discovered a strange bug in
irb1.9 (ruby 1.9.0 (2008-06-20 revision 17482) [i486-linux], irb
0.9.5(05/04/13); Debian GNU/Linux 5.0/lenny): ruby1.9 executes my
script nice, but irb crashes at some random place. Crash is affected
by most theoretically unlinked commands and may disappear even after
removing a comment (sic).

Spontaneously two things come to mind: garbage collection (because of
the seemingless random timing) and real threads (which are introduced in
1.9). Maybe your cleanup code is not thread safe or does not play well
with concurrent GC (not sure whether the 1.9 runtime really does this
but it uses native threads IIRC).

Kind regards

robert


#3

Quoting “Robert K.” removed_email_address@domain.invalid:

Spontaneously two things come to mind: garbage collection (because
of the seemingless random timing) and real threads (which are
introduced in 1.9). Maybe your cleanup code is not thread safe or
does not play well with concurrent GC (not sure whether the 1.9
runtime really does this but it uses native threads IIRC).

Disabling GC solved this problem. I think only this
Data_Wrap_Struct(rb_cObject, 0, 0, setting)
place of code can be buggy.
I passed NULL to “free” parameter because I do not allocate memory for
setting structure and do not free it (not sure what will be if it is
freed by library, probably I need to make some checks). What do I need
to pass as destructor in this case?


#4

On 24.03.2009 19:09, Peter Z. wrote:

I passed NULL to “free” parameter because I do not allocate memory for
setting structure and do not free it (not sure what will be if it is
freed by library, probably I need to make some checks). What do I need
to pass as destructor in this case?

I have to create a C extension yet, but you certainly need to clearly
separate memory that you allocate in the extension and memory that you
allocate via Ruby’s core. The first type needs to be manually freed in
the extension and the latter is probably subject to GC. But someone
else will be able to answer this much better than I can. You could as
well look into another extension and see how they do it there.

HTH

robert


#5

On 24.03.2009 20:07, Peter Z. wrote:

Data_Wrap_Struct(rb_cObject, 0, 0, setting)
better than I can. You could as well look into another extension
and see how they do it there.

The problem is that I do not allocate that memory in extension. It
is allocated by wrapped library, not me. Okay, I added hooks that
notify Ruby objects that setting was freed by library, and changed 0
in “free” parameter of Data_Wrap_Struct to empty destructor.
Nothing happened, same segfault is in same place.

I consider the lib as part of the extension for this example. The
difference is really which malloc is used.

Cheers

robert


#6

Quoting “Robert K.” removed_email_address@domain.invalid:

place of code can be buggy.
better than I can. You could as well look into another extension
and see how they do it there.

The problem is that I do not allocate that memory in extension. It
is allocated by wrapped library, not me. Okay, I added hooks that
notify Ruby objects that setting was freed by library, and changed 0
in “free” parameter of Data_Wrap_Struct to empty destructor.
Nothing happened, same segfault is in same place.


#7

Peter, this is bug in your code, not ext lib or ruby-1.9

Actually, your problem is that in your code (rconfig.c) you create
some intermediate objects that can be garbage collected in IRB
session (and will be under some conditions, as crash.rb shows):

—8<----
aConfigScalars = rb_ary_new3(5, cConfigFixnum, cConfigBignum,
cConfigFloat, cConfigBoolean, cConfigString);
aConfigAggregates = rb_ary_new3(3, cConfigGroup, cConfigArray,
cConfigList);
aConfigSettings = rb_ary_plus(aConfigScalars, aConfigAggregates);
—8<—

as these values are never references from other objects, they die in
GC, receive junk data, and lead to crash:

—8<—
#0 0x081149e0 in search_method (klass=105, id=325, klassp=0x0) at
vm_method.c:229
#1 0x08114a43 in rb_get_method_body (klass=105, id=325,
idp=0xbfffce68) at vm_method.c:256
#2 0x0812074b in rb_call0 (klass=105, recv=138149640, mid=325,
argc=1, argv=0xbfffce90, scope=1, self=6) at vm_eval.c:205
#3 0x08120a3f in rb_call () at vm_eval.c:255
#4 rb_funcall (recv=138149640, mid=325, n=1) at vm_eval.c:406
#5 0x0808247d in rb_equal (obj1=325, obj2=137879300) at object.c:51
#6 0x08132e3d in rb_ary_includes (ary=137877820, item=137879300) at
array.c:2770
#7 0x005e02b4 in rbConfigAggregate_append (self=137015820,
target=138267160) at ext/rconfig.c:399
—8<—

usually, in 1.8.x C extensions people overcome this by declaring
global variables, or constants under class (there might be other
options, like freezing, or pure-c managing of consts, but
aforementioned two are most common)

e.g., with such a quick-and-dirty-hack patch, i could not reproduce bug
anymore:
—8<—
— libconfig-ruby/ext/rconfig.c
+++ libconfig-ruby-mod/ext/rconfig.c
@@ -643,6 +642,9 @@
aConfigScalars = rb_ary_new3(5, cConfigFixnum, cConfigBignum,
cConfigFloat, cConfigBoolean, cConfigString);
aConfigAggregates = rb_ary_new3(3, cConfigGroup, cConfigArray,
cConfigList);
aConfigSettings = rb_ary_plus(aConfigScalars, aConfigAggregates);

  • rb_define_variable("$RConfigSetting", &aConfigSettings);

  • rb_define_variable("$RConfigScalars", &aConfigScalars);

  • rb_define_variable("$RConfigAggregates", &aConfigAggregates);

    char* settingNameRegexp = “^[A-Za-z*][A-Za-z\-_]$”;
    rSettingNameRegexp = rb_reg_new(settingNameRegexp,
    strlen(settingNameRegexp), 0);
    —8<—

two side notes:

  1. in ruby-1.9.1 some macros changed (see
    http://pennysmalls.com/2009/03/24/ferret-on-ruby-191/ for example):
    —8<—
    int i;
  •  for(i = 0; i < RARRAY(children)->len; i++) {
    
  •    VALUE key = RARRAY(children)->ptr[i];
    
  •  for(i = 0; i < RARRAY_LEN(children); i++) {
    
  •    VALUE key = RARRAY_PTR(children)[i];
       rconfig_do_append(new_setting, rb_hash_aref(hash, key), key);
     }
    
    }
    —8<—
  1. just curious, why C extension, and not FFI, as you use quite a
    bunch of Ruby methods and few libconfig structs/calls? (see
    http://kenai.com/projects/ruby-ffi )

Thx for this quiz anyway! :slight_smile:


#8

Quoting “Dmitry S.” removed_email_address@domain.invalid:

Thanks, fixed everything. Added these arrays as constants.

  1. just curious, why C extension, and not FFI, as you use quite a
    bunch of Ruby methods and few libconfig structs/calls? (see
    http://kenai.com/projects/ruby-ffi )

Libconfig is a bit Cxx-specific, so I decided to make binding
behaviour more rubyish, for example added value caching to allow
creation of unattached nodes or deletion of existing (how can I force
destroying of Ruby object? :). Also, as I seen, FFI doesn’t work on
Windows, and that isn’t good for me.

If anyone is interested, you can view my final version in repository at

http://git.whitequark.ru/libconfig-ruby.git