Forum: Ruby Memory Usage

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Horacio S. (Guest)
on 2005-12-12 12:11
(Received via mailing list)
I have a ruby script that reads data from a database, processes the data
(long
texts) and saves the results back to the database. I works great and at
good
speed.

The problem is that I ran the script on friday and left it until monday
when I
saw some memory related errors in the console (Insufficient Memory
something...). I ran the script again and fired up the windows task
manager
and then I saw that for every 10~15 records in the DB the script
processes
the memory goes up 1MB and there are no signs of it going down.

I started the script this morning and the PC was at 400MB ram usage, now
is
night and the PC is at 1.05GB ram usage.

Is this a memory leak?? is it caused by my scritpt? or is a Ruby thing??

I use Ruby 1.8.2 and ActiveRecord 1.13.1 in a WinXP machine.

any comments on how to find out where the memory is going are welcome.

regards,
Horacio
Chad P. (Guest)
on 2005-12-12 14:11
(Received via mailing list)
On Mon, Dec 12, 2005 at 07:10:27PM +0900, Horacio S. wrote:
>
> I started the script this morning and the PC was at 400MB ram usage, now is
> night and the PC is at 1.05GB ram usage.
>
> Is this a memory leak?? is it caused by my scritpt? or is a Ruby thing??
>
> I use Ruby 1.8.2 and ActiveRecord 1.13.1 in a WinXP machine.
>
> any comments on how to find out where the memory is going are welcome.

Are you making heavy use of symbols?  They don't relinquish RAM
unless/until the script exits, as far as I'm aware.

--
Chad P. [ CCD CopyWrite | http://ccd.apotheon.org ]

unix virus: If you're using a unixlike OS, please forward
this to 20 others and erase your system partition.
Peter H. (Guest)
on 2005-12-12 14:17
(Received via mailing list)
I've had a problem like this using the flickr api and net/http. All it
did was download images from Flickr, it ran pretty much 24/7 but some
days it would just eat massive amounts of virtual ram, 900Mb plus.

I've wrapped the whole thing in a shell script now to get around the
problem. Any idea how to track where the memory is going on a live
system?
Gene T. (Guest)
on 2005-12-12 18:18
(Received via mailing list)
Horacio S. wrote:
> I started the script this morning and the PC was at 400MB ram usage, now is
> night and the PC is at 1.05GB ram usage.
>
> Is this a memory leak?? is it caused by my scritpt? or is a Ruby thing??
>
> I use Ruby 1.8.2 and ActiveRecord 1.13.1 in a WinXP machine.
>
> any comments on how to find out where the memory is going are welcome.
>
> regards,
> Horacio

a couple recent threads on digging into ruby'c GC:
http://groups.google.com/group/comp.lang.ruby/brow...

and John C.'s thing to dump out your objects and (linux) processes
http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/...
Horacio S. (Guest)
on 2005-12-13 02:59
(Received via mailing list)
Thanks,

I already ran the Robert K.'s code to see the object count of each
class.
I ran the code every time my script finished  processing each record of
the
database and the numbers are like this:

String:    42812
Array:    16651
WIN32OLE:	3184
Hash:	2065


These are the biggest entries but they do not increase all the time,
they
fluctuate up and down.  Strings goes up to 60,000+ objects and then
drops
down back to 20,000+ objects. Then starts growing again. This happens
with
all objects.  Other classes are below 100 or are even 0.

I do not use symbols apart from those needed by ActiveRecord.

going to test now the MemoryProfiler of John C..

BTW, are these numbers normal??


thanks for the tips.
Horacio

Tuesday 13 December 2005 01:17ã?Gene T. ã?ã??はæ?¸ãã¾ã?ã?:
Gene T. (Guest)
on 2005-12-13 05:05
(Received via mailing list)
Horacio S. wrote:
>
> BTW, are these numbers normal??
> > >
> > >
> >
> > and John C.'s thing to dump out your objects and (linux) processes
> > http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/...

i just found this, tagged by 1 del.icio.us user (thx whoeve you are!),
but it really talks about the same things.

http://theexciter.com/articles/finding-leaking-rub...
This topic is locked and can not be replied to.