Forum: Ruby Excessive Memory Usage with libxml

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Daniel -. (Guest)
on 2007-05-22 21:28
(Received via mailing list)
Hi,

I hope this is the right place for this.

I'm writing a markup mangler that I'm using libxml for.  Primarily for
speed reasons.

I've got some basic functionality going, it's not finished, and when
I'm doing a benchmark I get some good figures.

I'm using a recursive function to process my data.  The general gist
of it is as follows

http://pastie.caboo.se/63584


But...

When I run the benchmark I get weird things happening.

All benchmarks exhibit steadily increasing memory usage until the end
of the benchmark
10000 iterations were fine
very much above this and I started to get

malloc errors
or
segmentation fault


I changed
doc.root.to_s.gsub( /\<\/?#{artificial_root_tag}\>/, "" )

to
doc.root.to_a.join.gsub( /\<\/?#{artificial_root_tag}\>/, "" )

taking a hit on performance, but it allowed me to get through 100,000
iterations.
The memory usage though was stupid at 235Mb Real and 233Mb Virtual

The HTML I am parsing is a fairly small snippet

I'm running on Mac OsX.

Any ideas what and why?
thankyou

Daniel
Daniel -. (Guest)
on 2007-05-22 21:44
(Received via mailing list)
On 5/23/07, Daniel N <removed_email_address@domain.invalid> wrote:
> I'm using a recursive function to process my data.  The general gist
> of the benchmark
>
>
> Any ideas what and why?
> thankyou
>
> Daniel
>
>

I've managed to reproduce the error by changing to Benchmark.bmbm.  It
is

ruby(3775) malloc: ***  Deallocation of a pointer not malloced:
0x3903a60; This could be a double free(), or free() called with the
middle of an allocated block; Try setting environment variable
MallocHelp to see tools to help debug
ruby(3775) malloc: ***  Deallocation of a pointer not malloced:
0x75b0750; This could be a double free(), or free() called with the
middle of an allocated block; Try setting environment variable
MallocHelp to see tools to help debug
ruby(3775) malloc: ***  Deallocation of a pointer not malloced:
0x75c4950; This could be a double free(), or free() called with the
middle of an allocated block; Try setting environment variable
MallocHelp to see tools to help debug
./markup_mangler_benchmark.rb:79: [BUG] Segmentation fault
This topic is locked and can not be replied to.