[BUG: trunk] GC mark bug

[email protected]$G$9!%e(B

e$B!!0J2<$N$h$&$K!$e(BGC.stress = true e$B$H$7$Fe(B fileutils e$B$re(B
require e$B$9$k$He(B SEGV
e$B$7$^$9!%B?J,!$e(Bmark
e$B%_%9$J$s$8$c$J$$$+$H;W$$$^$9!%$h$/$o$+$i$J$$$N$G!$$He(B
e$B$j$"$($:EPO?$7$F$*$-$^$9!%e(B

$ gdb -x run.gdb --args ./miniruby -ve ‘GC.stress = true; require
“fileutils”’
GNU gdb 6.8-debian
Copyright © 2008 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later
http://gnu.org/licenses/gpl.html
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law. Type “show
copying”
and “show warranty” for details.
This GDB was configured as “i486-linux-gnu”…
Function “iseq_set_local_table” not defined.
Make breakpoint pending on future shared library load? (y or [n])
[answered N; input not from terminal]
Breakpoint 1 at 0x8191680: file …/trunk/debug.c, line 137.
[Thread debugging using libthread_db enabled]
[New Thread 0xb7d9d6b0 (LWP 1797)]
[New Thread 0xb7c14b90 (LWP 1800)]
ruby 1.9.2dev (2009-10-10 trunk 25288) [i686-linux]
/tmp/ruby-trunk/lib/ruby/1.9.1/fileutils.rb:1528: [BUG] rb_gc_mark():
unknown data type 0x0(0xb7ef7190) non object
ruby 1.9.2dev (2009-10-10 trunk 25288) [i686-linux]

– control frame ----------
c:0012 p:---- s:0032 b:0032 l:000031 d:000031 CFUNC :module_eval
c:0011 p:0033 s:0026 b:0026 l:000017 d:000025 BLOCK
/tmp/ruby-trunk/lib/ruby/1.9.1/fileutils.rb:1528
c:0010 p:---- s:0023 b:0023 l:000022 d:000022 FINISH
c:0009 p:---- s:0021 b:0021 l:000020 d:000020 CFUNC :each
c:0008 p:0059 s:0018 b:0018 l:000017 d:000017 CLASS
/tmp/ruby-trunk/lib/ruby/1.9.1/fileutils.rb:1527
c:0007 p:2527 s:0016 b:0016 l:000015 d:000015 CLASS
/tmp/ruby-trunk/lib/ruby/1.9.1/fileutils.rb:1523
c:0006 p:0009 s:0014 b:0014 l:000013 d:000013 TOP
/tmp/ruby-trunk/lib/ruby/1.9.1/fileutils.rb:85
c:0005 p:---- s:0012 b:0012 l:000011 d:000011 FINISH
c:0004 p:---- s:0010 b:0010 l:000009 d:000009 CFUNC :require
c:0003 p:0029 s:0006 b:0006 l:0005fc d:000cec EVAL -e:1
c:0002 p:---- s:0004 b:0004 l:000003 d:000003 FINISH
c:0001 p:0000 s:0002 b:0002 l:0005fc d:0005fc TOP

-e:1:in <main>' -e:1:inrequire’
/tmp/ruby-trunk/lib/ruby/1.9.1/fileutils.rb:85:in <top (required)>' /tmp/ruby-trunk/lib/ruby/1.9.1/fileutils.rb:1523:inmodule:FileUtils
/tmp/ruby-trunk/lib/ruby/1.9.1/fileutils.rb:1527:in <module:Verbose>' /tmp/ruby-trunk/lib/ruby/1.9.1/fileutils.rb:1527:ineach’
/tmp/ruby-trunk/lib/ruby/1.9.1/fileutils.rb:1528:in block in <module:Verbose>' /tmp/ruby-trunk/lib/ruby/1.9.1/fileutils.rb:1528:inmodule_eval’

– C level backtrace information

/home/ko1/build/ruby/build/miniruby(rb_vm_bugreport+0xb6) [0x81ac4d6]
[0x6275722f]

[NOTE]
You may have encountered a bug in the Ruby interpreter or extension
libraries.
Bug reports are welcome.
For details: http://www.ruby-lang.org/bugreport.html

Program received signal SIGABRT, Aborted.
[Switching to Thread 0xb7d9d6b0 (LWP 1797)]
0xb7f80416 in __kernel_vsyscall ()
(gdb) bt
#0 0xb7f80416 in __kernel_vsyscall ()
#1 0xb7dca640 in raise () from /lib/i686/cmov/libc.so.6
#2 0xb7dcc018 in abort () from /lib/i686/cmov/libc.so.6
#3 0x08091a5e in rb_bug (
fmt=0x81bbad0 “rb_gc_mark(): unknown data type 0x%x(%p) %s”)
at …/trunk/error.c:235
#4 0x080a8584 in gc_mark_children (objspace=0x9bf7d20, ptr=3085922704,
lev=14)
at …/trunk/gc.c:1779
#5 0x080aa321 in mark_method_entry_i (key=11568, me=0x9cc7508,
data=3215562828) at …/trunk/gc.c:1507
#6 0x08145928 in st_foreach (table=0x9cc6300,
func=0x80aa240 <mark_method_entry_i>, arg=3215562828) at
…/trunk/st.c:775
#7 0x080a8b17 in gc_mark_children (objspace=0x9bf7d20, ptr=164323380,
lev=13)
at …/trunk/gc.c:1451
#8 0x080a9314 in mark_entry (key=12605, value=164323380,
data=3215563020)
at …/trunk/gc.c:1507
#9 0x08145928 in st_foreach (table=0x9c8f018, func=0x80a9250
<mark_entry>,
arg=3215563020) at …/trunk/st.c:775
#10 0x080a8b49 in gc_mark_children (objspace=0x9bf7d20, ptr=164338460,
lev=12)
at …/trunk/gc.c:1359
#11 0x080a9314 in mark_entry (key=11413, value=164338460,
data=3215563212)
at …/trunk/gc.c:1507
#12 0x08145928 in st_foreach (table=0x9c2bb08, func=0x80a9250
<mark_entry>,
arg=3215563212) at …/trunk/st.c:775
#13 0x080a8b49 in gc_mark_children (objspace=0x9bf7d20, ptr=163744820,
lev=11)
at …/trunk/gc.c:1359
#14 0x080a9314 in mark_entry (key=504, value=163744820, data=3215563404)
at …/trunk/gc.c:1507
#15 0x081458ce in st_foreach (table=0x9c2bfd8, func=0x80a9250
<mark_entry>,
—Type to continue, or q to quit—
arg=3215563404) at …/trunk/st.c:745
#16 0x080a8b49 in gc_mark_children (objspace=0x9bf7d20, ptr=163744640,
lev=10)
at …/trunk/gc.c:1359
#17 0x080a9314 in mark_entry (key=504, value=163744640, data=3215563596)
at …/trunk/gc.c:1507
#18 0x081458ce in st_foreach (table=0x9c2c298, func=0x80a9250
<mark_entry>,
arg=3215563596) at …/trunk/st.c:745
#19 0x080a8b49 in gc_mark_children (objspace=0x9bf7d20, ptr=163744560,
lev=9)
at …/trunk/gc.c:1359
#20 0x080a86b1 in gc_mark_children (objspace=0x9bf7d20, ptr=163744660,
lev=8)
at …/trunk/gc.c:1507
#21 0x080a86b1 in gc_mark_children (objspace=0x9bf7d20, ptr=163744780,
lev=7)
at …/trunk/gc.c:1507
#22 0x080a9314 in mark_entry (key=504, value=163744740, data=3215564044)
at …/trunk/gc.c:1507
#23 0x081458ce in st_foreach (table=0x9c2be78, func=0x80a9250
<mark_entry>,
arg=3215564044) at …/trunk/st.c:745
#24 0x080a8b49 in gc_mark_children (objspace=0x9bf7d20, ptr=163744680,
lev=6)
at …/trunk/gc.c:1359
#25 0x080a9314 in mark_entry (key=504, value=163744680, data=3215564236)
at …/trunk/gc.c:1507
#26 0x081458ce in st_foreach (table=0x9c2c138, func=0x80a9250
<mark_entry>,
arg=3215564236) at …/trunk/st.c:745
#27 0x080a8b49 in gc_mark_children (objspace=0x9bf7d20, ptr=163744600,
lev=5)
at …/trunk/gc.c:1359
#28 0x080a86b1 in gc_mark_children (objspace=0x9bf7d20, ptr=163708920,
lev=4)
at …/trunk/gc.c:1507
#29 0x080a86b1 in gc_mark_children (objspace=0x9bf7d20, ptr=163708940,
lev=3)
at …/trunk/gc.c:1507
—Type to continue, or q to quit—
#30 0x080a86b1 in gc_mark_children (objspace=0x9bf7d20, ptr=163708960,
lev=2)
at …/trunk/gc.c:1507
#31 0x080a86b1 in gc_mark_children (objspace=0x9bf7d20, ptr=163706700,
lev=1)
at …/trunk/gc.c:1507
#32 0x080ab28b in garbage_collect (objspace=0x9bf7d20) at
…/trunk/gc.c:1507
#33 0x080abcdf in vm_xmalloc (objspace=0x9bf7d20, size=116)
at …/trunk/gc.c:622
#34 0x08183ec9 in iseq_setup (iseq=0x9cc64c0, anchor=0xbfa9a1fc)
at …/trunk/compile.c:1315
#35 0x0818ddec in rb_iseq_compile_node (self=164322720, node=0x9cb5d3c)
at …/trunk/compile.c:505
#36 0x08193dc1 in rb_iseq_new_with_bopt_and_opt (node=0x9cb5d3c,
name=164325860, filename=164323180, line_no=3059, parent=164325840,
type=15, bopt=0, option=0xbfa9a288) at …/trunk/iseq.c:396
#37 0x08194105 in rb_iseq_compile_with_option (src=164323140,
file=164323180,
line=3059, opt=4) at …/trunk/iseq.c:407
#38 0x08194287 in rb_iseq_compile (src=164323140, file=164323180,
line=3059)
at …/trunk/iseq.c:556
#39 0x081a3adc in eval_string_with_cref (self=164323380, src=164323140,
scope=4, cref=0x9cb5f80,
file=0x9c8e860 “/tmp/ruby-trunk/lib/ruby/1.9.1/fileutils.rb”,
line=1529)
at …/trunk/vm_eval.c:831
#40 0x081a4ebd in specific_eval (argc=,
argv=, klass=164323380, self=164323380)
at …/trunk/vm_eval.c:1057
#41 0x08197875 in call_cfunc (func=0x81a4ff0 <rb_mod_module_eval>,
recv=164323380, len=1797, argc=6, argv=0xb7c15074)
at …/trunk/vm_insnhelper.c:286
#42 0x081979cc in vm_call_cfunc (th=0x9bf7ac0, reg_cfp=0xb7c94e24,
num=3,
—Type to continue, or q to quit—
recv=164323380, blockptr=0x0, flag=8, me=0x9c3cad0)
at …/trunk/vm_insnhelper.c:386
#43 0x081a7885 in vm_call_method (th=0x9bf7ac0, cfp=0xb7c94e24, num=3,
blockptr=0x0, flag=8, id=2832, me=0x9c3cad0, recv=164323380)
at …/trunk/vm_insnhelper.c:511
#44 0x0819dfeb in vm_exec_core (th=0x9bf7ac0, initial=)
at …/trunk/insns.def:994
#45 0x081a2d67 in vm_exec (th=0x9bf7ac0) at …/trunk/vm.c:1117
#46 0x081a359a in invoke_block_from_c (th=0x9bf7ac0, block=0xb7c94ebc,
self=164323380, argc=1, argv=0xbfa9ab70, blockptr=0x0, cref=0x0)
at …/trunk/vm.c:554
#47 0x081a432f in rb_yield (val=164324900) at …/trunk/vm.c:584
#48 0x08062a61 in rb_ary_each (ary=164323260) at …/trunk/array.c:1405
#49 0x081979cc in vm_call_cfunc (th=0x9bf7ac0, reg_cfp=0xb7c94ea8,
num=0,
recv=164323260, blockptr=0xb7c94ebc, flag=0, me=0x9c525b0)
at …/trunk/vm_insnhelper.c:386
#50 0x081a7885 in vm_call_method (th=0x9bf7ac0, cfp=0xb7c94ea8, num=0,
blockptr=0xb7c94ebc, flag=0, id=424, me=0x9c525b0, recv=164323260)
at …/trunk/vm_insnhelper.c:511
#51 0x0819dfeb in vm_exec_core (th=0x9bf7ac0, initial=)
at …/trunk/insns.def:994
#52 0x081a2d67 in vm_exec (th=0x9bf7ac0) at …/trunk/vm.c:1117
#53 0x081a316a in rb_iseq_eval (iseqval=164338560) at …/trunk/vm.c:1327
#54 0x0809871d in rb_load_internal (fname=163703580,
wrap=) at …/trunk/load.c:293
#55 0x080999ce in rb_require_safe (fname=163706760, safe=0)
at …/trunk/load.c:582
#56 0x081979cc in vm_call_cfunc (th=0x9bf7ac0, reg_cfp=0xb7c94f84,
num=1,
recv=163742100, blockptr=0x0, flag=8, me=0x9c75888)
—Type to continue, or q to quit—
at …/trunk/vm_insnhelper.c:386
#57 0x081a7885 in vm_call_method (th=0x9bf7ac0, cfp=0xb7c94f84, num=1,
blockptr=0x0, flag=8, id=8456, me=0x9c75888, recv=163742100)
at …/trunk/vm_insnhelper.c:511
#58 0x0819dfeb in vm_exec_core (th=0x9bf7ac0, initial=)
at …/trunk/insns.def:994
#59 0x081a2d67 in vm_exec (th=0x9bf7ac0) at …/trunk/vm.c:1117
#60 0x081a305c in rb_iseq_eval_main (iseqval=163693160) at
…/trunk/vm.c:1341
#61 0x080959ac in ruby_exec_internal (n=0x9c1c268) at
…/trunk/eval.c:203
#62 0x080973b5 in ruby_run_node (n=0x9c1c268) at …/trunk/eval.c:237
#63 0x0805d11e in main (argc=3, argv=0xbfa9bb94) at …/trunk/main.c:35
(gdb) quit
The program is running. Exit anyway? (y or n) EOF [assumed Y]

e$B%o%J%Y$H?=$7$^$9!#e(B
redmine e$B$G$&$^$/%3%a%s%H$G$-$J$+$C$?$h$&$J$N$G%a!<%kCW$7$7$^$9!#e(B
e$B$b$7Fs=E$KAw$i$l$?$i?=$7Lu$"$j$^$;$s!#e(B

2009e$BG/e(B10e$B7ne(B10e$BF|e(B22:59 SASADA Koichi [email protected]:

e$B0J2<$N$h$&$K!$e(BGC.stress = true e$B$H$7$Fe(B fileutils e$B$re(B require e$B$9$k$He(B SEGV
e$B$7$^$9!%B?J,!$e(Bmark e$B%_%9$J$s$8$c$J$$$+$H;W$$$^$9!%$h$/$o$+$i$J$$$N$G!$$He(B
e$B$j$"$($:EPO?$7$F$*$-$^$9!%e(B

eval e$B$5$l$?e(B ISeq e$B$,GK4~$5$l$k$H$-!“8=:_$Ne(B iseq_free
e$B$G$Oe(B
e$B;Re(B ISeq e$B$N%^!<%/>uBV$K4X$o$i$:@hF,$Ne(B iseq->iseq
e$B$+$i$9$Y$Fe(B
e$BGK4~$7$F$$$k$+$i$G$O$J$$$G$7$g$&$+!#e(B
e$B0J2<$N$h$&$K!”?Fe(B ISeq e$B$,%^!<%/$5$l$k>l9g$OMn$A$^$;$s$G$7$?!#e(B

$ cat test.rb
GC.stress = true
a = []
(“A”…“Z”).each do |mod|
print mod
(“a”…“k”).each do |meth|
iseq = RubyVM::InstructionSequence.compile(<<-EOS)
module #{mod}
def #{meth}
end
end
EOS
a.push iseq if ARGV[0]
iseq.eval
end
o = Object.new
o.extend(Kernel.const_get(mod))
(“a”…“k”).each do |meth|
o.send(meth)
end
end

$ ./ruby -v test.rb 1
ruby 1.9.2dev (2009-10-14 trunk 25335) [i386-mingw32]
ABCDEFGHIJKLMNOPQRSTUVWXYZ

$ ./ruby test.rb
ABCDEFGtest.rb:19: [BUG] Segmentation fault
ruby 1.9.2dev (2009-10-14 trunk 25335) [i386-mingw32]

– control frame ----------
c:td p:td s:td b:td l:tx d:td BLOCK test.rb:19
c:td p:---- s:td b:td l:td d:td FINISH
c:td p:---- s:td b:td l:td d:td IFUNC :respond_to_missing?
c:td p:---- s:td b:td l:td d:td CFUNC :upto
c:td p:---- s:td b:td l:td d:td CFUNC :each
c:td p:td s:td b:td l:tx d:td BLOCK test.rb:18
c:td p:---- s:td b:td l:td d:td FINISH
c:td p:---- s:td b:td l:td d:td IFUNC :==
c:td p:---- s:td b:td l:td d:td CFUNC :upto
c:td p:---- s:td b:td l:td d:td CFUNC :each
c:td p:td s:td b:td l:tx d:tx EVAL test.rb:4
c:td p:---- s:td b:td l:td d:td FINISH
c:td p:td s:td b:td l:tx d:tx TOP

test.rb:4:in <main>' test.rb:4:ineach’
test.rb:4:in upto' test.rb:18:inblock in ’
test.rb:18:in each' test.rb:18:inupto’
test.rb:19:in `block (2 levels) in ’

[NOTE]
You may have encountered a bug in the Ruby interpreter or extension
libraries.
Bug reports are welcome.
For details: http://www.ruby-lang.org/bugreport.html

This application has requested the Runtime to terminate it in an unusual
way.
Please contact the application’s support team for more information.

e$B%o%J%Y$G$9!#e(B

2009e$BG/e(B10e$B7ne(B15e$BF|e(B6:34 wanabe [email protected]:

e$B%o%J%Y$H?=$7$^$9!#e(B
eval e$B$5$l$?e(B ISeq e$B$,GK4~$5$l$k$H$-!"8=:_$Ne(B iseq_free e$B$G$Oe(B
e$B;Re(B ISeq e$B$N%^!<%/>uBV$K4X$o$i$:@hF,$Ne(B iseq->iseq e$B$+$i$9$Y$Fe(B
e$BGK4~$7$F$$$k$+$i$G$O$J$$$G$7$g$&$+!#e(B

e$B$9$_$^$;$s!"$h$/D4$Y$:$KE*[email protected]$C$F$$$^$7$?!#e(B

e$B$b$7$+$7$F!"%b%8%e!<%k$N%a%=%C%I%(%s%H%j$N%F!<%V%k$,e(B
unpack_entries e$B$G:F9=C[$5$l$k$H$-$K!"$=$NCf$Ne(B st_insert e$B$,e(B
vm_xmalloc e$B$r8F$S=P$7$Fe(B GC
e$B$,;O$^$C$F$7$^$&!"$H$$$&$3$H$O$J$$$G$7$g$&$+!#e(B
[email protected]$H$9$k$H!"$b$7$+$7$FNc$($P0J2<$N$h$&$K$9$k$HMn$A$J$/$J$C$?$j$7$J$$$G$7$g$&$+!#e(B

Index: st.c

— st.c (revision 25371)
+++ st.c (working copy)
@@ -418,15 +418,17 @@
{
int i;
struct st_table_entry packed_bins[MAX_PACKED_NUMHASH2];

  • int num_entries = table->num_entries;
  • st_table tmp_table = *table;
  • memcpy(packed_bins, table->bins, sizeof(struct st_table_entry *)
  • num_entries*2);
  • table->entries_packed = 0;
  • table->num_entries = 0;
  • memset(table->bins, 0, sizeof(struct st_table_entry *) *
    table->num_bins);
  • for (i = 0; i < num_entries; i++) {
  •    st_insert(table, (st_data_t)packed_bins[i*2],
    

(st_data_t)packed_bins[i*2+1]);

  • memcpy(packed_bins, table->bins, sizeof(struct st_table_entry *)
  • table->num_entries*2);
  • table->bins = packed_bins;
  • tmp_table.entries_packed = 0;
  • tmp_table.num_entries = 0;
  • memset(tmp_table.bins, 0, sizeof(struct st_table_entry *) *
    tmp_table.num_bins);
  • for (i = 0; i < table->num_entries; i++) {
  •    st_insert(&tmp_table, (st_data_t)packed_bins[i*2],
    

(st_data_t)packed_bins[i*2+1]);
}

  • *table = tmp_table;
    }

int

e$B$^$D$b$He(B e$B$f$-$R$m$G$9e(B

In message “Re: [ruby-dev:39493] Re: [BUG: trunk] GC mark bug”
on Fri, 16 Oct 2009 21:05:24 +0900, wanabe [email protected]
writes:

|e$B$b$7$+$7$F!"%b%8%e!<%k$N%a%=%C%I%(%s%H%j$N%F!<%V%k$,e(B
|unpack_entries e$B$G:F9=C[$5$l$k$H$-$K!"$=$NCf$Ne(B st_insert e$B$,e(B
|vm_xmalloc e$B$r8F$S=P$7$Fe(B GC e$B$,;O$^$C$F$7$^$&!"$H$$$&$3$H$O$J$$$G$7$g$&$+!#e(B
|[email protected]$H$9$k$H!"$b$7$+$7$FNc$($P0J2<$N$h$&$K$9$k$HMn$A$J$/$J$C$?$j$7$J$$$G$7$g$&$+!#e(B

e$B%3%_%C%H$7$F$/[email protected]$5$$!#e(B

[email protected]$G$9!%e(B

wanabe wrote::

e$B$b$7$+$7$F!"%b%8%e!<%k$N%a%=%C%I%(%s%H%j$N%F!<%V%k$,e(B
unpack_entries e$B$G:F9=C[$5$l$k$H$-$K!"$=$NCf$Ne(B st_insert e$B$,e(B
vm_xmalloc e$B$r8F$S=P$7$Fe(B GC e$B$,;O$^$C$F$7$^$&!"$H$$$&$3$H$O$J$$$G$7$g$&$+!#e(B
[email protected]$H$9$k$H!"$b$7$+$7$FNc$($P0J2<$N$h$&$K$9$k$HMn$A$J$/$J$C$?$j$7$J$$$G$7$g$&$+!#e(B

e$B!!$*$)[email protected]$i$7$$!%e(Bst.c e$B$O;W$$$D$-$^$;$s$G$7$?!%e(B

e$B%A%1%C%He(B #2196 e$B$,99?7$5$l$^$7$?!#e(B (by _ wanabe)

e$B%9%F!<%?%9e(B Opene$B$+$ie(BClosede$B$KJQ99e(B
e$B?JD=e(B % 0e$B$+$ie(B100e$B$KJQ99e(B

This issue was solved with changeset r25377.
Koichi, thank you for your reporting of the issue.
You have greatfully contributed toward Ruby.
May Ruby be with you.


http://redmine.ruby-lang.org/issues/show/2196

e$B1sF#$G$9!#e(B

2009e$BG/e(B10e$B7ne(B16e$BF|e(B21:05 wanabe [email protected]:

[email protected]$H$9$k$H!"$b$7$+$7$FNc$($P0J2<$N$h$&$K$9$k$HMn$A$J$/$J$C$?$j$7$J$$$G$7$g$&$+!#e(B

e$B3N$+$KMn$A$J$/$J$k$3$H$r3NG’$7$^$7$?!#e(B

unpack e$BCf$Ne(B st_table e$B$re(B st_foreach
e$B$7$?>l9g!“FbMFJ*$NNs5sO3$l$,H/@8e(B
e$B$9$k!”$H$$$&%P%0$C$F$3$H$G$9$M!#e(B
e$BFbMFJ*$O0l;~E*$K%9%?%C%/$K:$C$F$$$k$N$G!":#$^$G$Oe(B conservative GC
e$B$K$h$C$Fe(B mark e$BO3$l$OKI$,$l$F$$$^$7$?$,!“e(Br24128
e$B$G%a%=%C%I%F!<%V%k$Ne(B
e$BFbMFJ*$,e(B (GC e$B4IM}BP>]$G$”$ke(B) NODE e$B$+$ie(B (GC
e$B4IM}BP>]30$Ne(B) rb_method_
entry_t e$B$KJQ$o$C$?$?$ae(B mark
e$BO3$l$,H/>I$9$k$h$&$K$J$C$?!"$HM}2r$7$^e(B
e$B$7$?!#e(B
e$BN"$E$1$H$7$F!"$3$NLdBj$Oe(B r24128
e$B$+$iH/>I$9$k$h$&$K$J$C$F$$$^$9!#e(B

e$B$=$&$$$&$o$1$G!"$3$N%Q%C%A$K;[email protected]$G$9!#e(B

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs