Memory Usage


#1

I have a ruby script that reads data from a database, processes the data
(long
texts) and saves the results back to the database. I works great and at
good
speed.

The problem is that I ran the script on friday and left it until monday
when I
saw some memory related errors in the console (Insufficient Memory
something…). I ran the script again and fired up the windows task
manager
and then I saw that for every 10~15 records in the DB the script
processes
the memory goes up 1MB and there are no signs of it going down.

I started the script this morning and the PC was at 400MB ram usage, now
is
night and the PC is at 1.05GB ram usage.

Is this a memory leak?? is it caused by my scritpt? or is a Ruby thing??

I use Ruby 1.8.2 and ActiveRecord 1.13.1 in a WinXP machine.

any comments on how to find out where the memory is going are welcome.

regards,
Horacio


#2

On Mon, Dec 12, 2005 at 07:10:27PM +0900, Horacio S. wrote:

I started the script this morning and the PC was at 400MB ram usage, now is
night and the PC is at 1.05GB ram usage.

Is this a memory leak?? is it caused by my scritpt? or is a Ruby thing??

I use Ruby 1.8.2 and ActiveRecord 1.13.1 in a WinXP machine.

any comments on how to find out where the memory is going are welcome.

Are you making heavy use of symbols? They don’t relinquish RAM
unless/until the script exits, as far as I’m aware.


Chad P. [ CCD CopyWrite | http://ccd.apotheon.org ]

unix virus: If you’re using a unixlike OS, please forward
this to 20 others and erase your system partition.


#3

I’ve had a problem like this using the flickr api and net/http. All it
did was download images from Flickr, it ran pretty much 24/7 but some
days it would just eat massive amounts of virtual ram, 900Mb plus.

I’ve wrapped the whole thing in a shell script now to get around the
problem. Any idea how to track where the memory is going on a live
system?


#4

Horacio S. wrote:

I started the script this morning and the PC was at 400MB ram usage, now is
night and the PC is at 1.05GB ram usage.

Is this a memory leak?? is it caused by my scritpt? or is a Ruby thing??

I use Ruby 1.8.2 and ActiveRecord 1.13.1 in a WinXP machine.

any comments on how to find out where the memory is going are welcome.

regards,
Horacio

a couple recent threads on digging into ruby’c GC:
http://groups.google.com/group/comp.lang.ruby/browse_thread/thread/4e77925c75949460

and John C.'s thing to dump out your objects and (linux) processes
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/139987


#5

Thanks,

I already ran the Robert K.'s code to see the object count of each
class.
I ran the code every time my script finished processing each record of
the
database and the numbers are like this:

String: 42812
Array: 16651
WIN32OLE: 3184
Hash: 2065

These are the biggest entries but they do not increase all the time,
they
fluctuate up and down. Strings goes up to 60,000+ objects and then
drops
down back to 20,000+ objects. Then starts growing again. This happens
with
all objects. Other classes are below 100 or are even 0.

I do not use symbols apart from those needed by ActiveRecord.

going to test now the MemoryProfiler of John C…

BTW, are these numbers normal??

thanks for the tips.
Horacio

Tuesday 13 December 2005 01:17ã?Gene T. ã?ã??はæ?¸ãã¾ã?ã?:


#6

Horacio S. wrote:

BTW, are these numbers normal??

and John C.'s thing to dump out your objects and (linux) processes
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/139987

i just found this, tagged by 1 del.icio.us user (thx whoeve you are!),
but it really talks about the same things.

http://theexciter.com/articles/finding-leaking-ruby-objects?commented=1#c000092