Marshal::dump error on PowerBook G4

Ruby
1

I’ve written code for a Splay tree in Ruby, and I’m trying to
serialize it using Marshal::dump.

If I create a tree with 1,000 or 10,000 nodes, Marshal::dump works
perfectly. When I get up to 100,000 nodes, Marshal::dump causes the
irb process to die with the error “Illegal Instruction”.

Is this a known problem? I tried searching for “Marshal::dump” and
“Illegal Instruction” it but I didn’t turn up anything. I’m on a
PowerBook G4, running Ruby version 1.8.5 and IRb version 0.9.5. I’ve
attached the code for the Splay tree; to reproduce, open irb and run:

require ‘splay’
t = SplayTree.new
100000.times {|i| t[i] = i}
Marshal::dump t

Thanks for any help,
Emmett

2

On 12/8/06, Jamey C. [email protected] wrote:

PowerBook G4, running Ruby version 1.8.5 and IRb version 0.9.5. I’ve
If I tried to serialize the entire SkipList and it had too many keys, I
Jamey C.

Confidentiality Notice: This email message, including any attachments, is for the sole use of the intended recipient(s) and may contain confidential and/or privileged information. If you are not the intended recipient(s), you are hereby notified that any dissemination, unauthorized review, use, disclosure or distribution of this email and any materials contained in any attachments is prohibited. If you receive this message in error, or are not the intended recipient(s), please immediately notify the sender by email and destroy all copies of the original message, including attachments.

Thanks, I tried the to_hash workaround and it worked great. It also
reduced the serialized size by a large margin. I’m still slightly
disturbed by Marshal::dump dying so badly, but that definitely solved
the problem.

Emmett

3

Emmett S. wrote:

attached the code for the Splay tree; to reproduce, open irb and run:

require ‘splay’
t = SplayTree.new
100000.times {|i| t[i] = i}
Marshal::dump t

I remember running into something similar when I started working on
Mongoose, which is a ruby rdbms that uses SkipLists for it’s indexing.

If I tried to serialize the entire SkipList and it had too many keys, I
would get an error. I ended adding a method that converts the SkipList
to a hash, then feeds the hash to Marshal.dump, and it worked. I also
have another method that loads the Marshalled hash and turns it back
into a SkipList.

Another thing you could do would be to serialize each SplayNode, instead
of the whole tree.

HTH,

Jamey C.

Confidentiality Notice: This email message, including any attachments,
is for the sole use of the intended recipient(s) and may contain
confidential and/or privileged information. If you are not the intended
recipient(s), you are hereby notified that any dissemination,
unauthorized review, use, disclosure or distribution of this email and
any materials contained in any attachments is prohibited. If you receive
this message in error, or are not the intended recipient(s), please
immediately notify the sender by email and destroy all copies of the
original message, including attachments.

4

Hi,

In message “Re: Marshal::dump error on PowerBook G4”
on Sat, 9 Dec 2006 03:21:51 +0900, “Emmett S.”
[email protected] writes:

|I’ve written code for a Splay tree in Ruby, and I’m trying to
|serialize it using Marshal::dump.
|
|If I create a tree with 1,000 or 10,000 nodes, Marshal::dump works
|perfectly. When I get up to 100,000 nodes, Marshal::dump causes the
|irb process to die with the error “Illegal Instruction”.

Marshal uses recursion to traverse objects and OSX has far smaller
stack rlimits than other platforms. As a result of combination of
above, marshaling an object with long reference chain may cause
unexpected process termination. It is pretty diffucult to fix.

						matz.
5

On Dec 8, 2006, at 10:21 , Emmett S. wrote:

attached the code for the Splay tree; to reproduce, open irb and run:

require ‘splay’
t = SplayTree.new
100000.times {|i| t[i] = i}
Marshal::dump t

I think you ran out of something because your C stack is huge.

(gdb) run splay.rb
Starting program: /usr/local/bin/ruby splay.rb
Reading symbols for shared libraries … done

Program received signal EXC_BAD_ACCESS, Could not access memory.
Reason: KERN_INVALID_ADDRESS at address: 0xbf7fffc0
0x00064bbc in str_independent (str=1854980) at string.c:474
474 {
(gdb) bt
#0 0x00064bbc in str_independent (str=1854980) at string.c:474
#1 0x00064d34 in rb_str_modify (str=1854980) at string.c:505
#2 0x0006540c in rb_str_buf_cat (str=1854980, ptr=0xbf800178 “\b??
T”, len=1) at string.c:706
#3 0x0008bf28 in w_nbyte (s=0x64d34 “/?”, n=1, arg=0xbfffe854) at
marshal.c:128
#4 0x0008bfb4 in w_byte (c=8 ‘\b’, arg=0x1) at marshal.c:141
#5 0x0008c088 in w_long (x=1, arg=0xbfffe854) at marshal.c:183
#6 0x0008c5b8 in w_symbol (id=10154, arg=0xbfffe854) at marshal.c:336
#7 0x0008c9b8 in w_obj_each (id=1854980, value=171689,
arg=0xbf800428) at marshal.c:428
#8 0x00056394 in foreach_safe_i (key=412980, value=3212837240,
arg=0x1) at hash.c:131
#9 0x0007756c in st_foreach (table=0x2b9c200, func=0x56360
<foreach_safe_i>, arg=3212837752) at st.c:487
#10 0x000563e4 in st_foreach_safe (table=0x1c4e04, func=0xbf800178,
a=1) at hash.c:149
#11 0x0008ca24 in w_ivar (tbl=0x2b9c200, arg=0xbf800428) at marshal.c:
440
#12 0x0008d160 in w_object (obj=23661220, arg=0xbfffe854, limit=1) at
marshal.c:648
#13 0x0008c9c8 in w_obj_each (id=1854980, value=23661220,
arg=0xbf800678) at marshal.c:429
#14 0x00056394 in foreach_safe_i (key=412980, value=3212837240,
arg=0x1) at hash.c:131
…

I gave up at:

#3525 0x0007756c in st_foreach (table=0x2bae700, func=0x56360
<foreach_safe_i>, arg=3213184664) at st.c:487

–
Eric H. - [email protected] - http://blog.segment7.net

I LIT YOUR GEM ON FIRE!

6

Tom P. wrote:

Couldn’t you just use setrlimit() to bump up the maximum stack size?
At my day job, we’ve had to resort to that on a number of occasions on
different platforms.

We’ve once discussed on it([ruby-dev:24405]) and concluded that it’s
not a well-doing for ignoring user’s purposely-specified stack limit.

If you really want to, Ruby has Process.setrlimit on both 1.8 and 1.9.
Use them at your own risk.

7

On Dec 8, 2006, at 11:29 PM, Yukihiro M. wrote:

Marshal uses recursion to traverse objects and OSX has far smaller
stack rlimits than other platforms. As a result of combination of
above, marshaling an object with long reference chain may cause
unexpected process termination. It is pretty diffucult to fix.

Couldn’t you just use setrlimit() to bump up the maximum stack size?
At my day job, we’ve had to resort to that on a number of occasions
on different platforms.

Cheers,

TomP