Excessive Memory Usage with libxml


#1

Hi,

I hope this is the right place for this.

I’m writing a markup mangler that I’m using libxml for. Primarily for
speed reasons.

I’ve got some basic functionality going, it’s not finished, and when
I’m doing a benchmark I get some good figures.

I’m using a recursive function to process my data. The general gist
of it is as follows

http://pastie.caboo.se/63584

But…

When I run the benchmark I get weird things happening.

All benchmarks exhibit steadily increasing memory usage until the end
of the benchmark
10000 iterations were fine
very much above this and I started to get

malloc errors
or
segmentation fault

I changed
doc.root.to_s.gsub( /</?#{artificial_root_tag}>/, “” )

to
doc.root.to_a.join.gsub( /</?#{artificial_root_tag}>/, “” )

taking a hit on performance, but it allowed me to get through 100,000
iterations.
The memory usage though was stupid at 235Mb Real and 233Mb Virtual

The HTML I am parsing is a fairly small snippet

I’m running on Mac OsX.

Any ideas what and why?
thankyou

Daniel


#2

On 5/23/07, Daniel N removed_email_address@domain.invalid wrote:

I’m using a recursive function to process my data. The general gist
of the benchmark

Any ideas what and why?
thankyou

Daniel

I’ve managed to reproduce the error by changing to Benchmark.bmbm. It
is

ruby(3775) malloc: *** Deallocation of a pointer not malloced:
0x3903a60; This could be a double free(), or free() called with the
middle of an allocated block; Try setting environment variable
MallocHelp to see tools to help debug
ruby(3775) malloc: *** Deallocation of a pointer not malloced:
0x75b0750; This could be a double free(), or free() called with the
middle of an allocated block; Try setting environment variable
MallocHelp to see tools to help debug
ruby(3775) malloc: *** Deallocation of a pointer not malloced:
0x75c4950; This could be a double free(), or free() called with the
middle of an allocated block; Try setting environment variable
MallocHelp to see tools to help debug
./markup_mangler_benchmark.rb:79: [BUG] Segmentation fault