Newbie question regarding memory leak

Hi, Recently, I have written a ruby program for web-scraping. But its a
bit slow and consumes enormous memory due to a possible memory leak. I
am very new to these memory leak issues. Please find my code in the
attachment.

Can anyone patiently go through my code and figure out where the
possible mem leak occurs? Any help/suggestion greatly appreciated…

thanks & regards,
Venkat B.

On Tue, Mar 11, 2008 at 1:33 AM, Venkat B. [email protected]
wrote:

Attachments:
http://www.ruby-forum.com/attachment/1561/test4.rb


Posted via http://www.ruby-forum.com/.

Where do you think your memory leak is? (Also, why don’t you think
Ruby’s garbage collecting is handling it? Maybe you have references
to things you’re not using anymore?)

– Ben (aka divokz)

Ben Oakes wrote:

Where do you think your memory leak is? (Also, why don’t you think
Ruby’s garbage collecting is handling it? Maybe you have references
to things you’re not using anymore?)

I have assigned the useless object references to nil. You can observe, I
assigned

page = nil
page_links = nil
doc = nil
file_name = nil at some point of the program, where I no longer needed
them. Doesn’t assigning a ref to nil, completely free its object memory?
Isn’t nil an empty object? Does nil also claims memory? I am very sorry
if my questions doesn’t make any sense !!!

On Tue, Mar 11, 2008 at 7:51 AM, Venkat B. [email protected]
wrote:

page_links = nil

I haven’t had much time to review the code you’ve posted, but
regarding your questions here, nil does take up memory, however,
there is only one instance of nil (or NilClass, rather), if I
remember correctly. Whenever you use “nil,” you are using the
reference to that class. (That is, every nil points to the same
place.) If you believe that your objects are still resident after
garbage collecting, there’s a chance that you still have references to
them somewhere in memory. (For example, you may set a variable that
once held an array to nil, but then if that array had references to
other objects, there would still be a reference count above 0, so the
objects would stick around. I’m not sure if that example would hold
true, but I believe that’s what the general idea is.)

I hope this helps,

– Ben

Atlast I figured it out. Rather than declaring @agent as an instance
variable, i tried declaring it as a local variable and that does the
trick. Previously it used to eat up 400-500 MB of memory but now, it
uses a max of 15 MB.

I modified my code from

def initialize
@agent = WWW::Mechanize.new
end

def fetch_links(start, finish, count)
page = WWW::Mechanize::Page


end

to look like below

def initialize

end

def fetch_links(start, finish, count)
agent = WWW::Mechanize.new
page = WWW::Mechanize::Page
end

Any interpretations from this?

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs