Leaking weirdness

Previously posted on ruby talk. Response…silence :slight_smile:


Perhaps someone out there can help give me a clue about the following
situation:

My mongrel processes seemed to be leaking.
As an example of this, if I added this code to the bottom of
environment.rb
Thread.new {loop do; print ‘w’; end}
it didn’t affect memory. At first. However after the web server got
its
first hit its memory consumption would sky-rocket. [i.e. it would monotonically grow by about 50MB/s]

Then I ran the same scenario using webrick, instead of mongrel [1.1.3].
No leak.

Then I’m thinking ‘hmm maybe it’s mongrel and the mongrel guys have
fixed this.’

Update my gem to 1.1.4 from 1.1.3.
Using mongrel this time.
No leak! Yea!

Now this is the weird part.
Attempted to recreate the bug.
Uninstalled mongrel 1.1.4, reinstalled 1.1.3
uninstalled daemons [its dependency] 1.0.10, reinstalled 1.0.9 [what I
had before].

Ran it with Mongrel again [which is where it used to leak].
No leak.
I am so confused!
Perhaps rubygems was updated and so it “compiled it right this time”?
Note that there don’t appear to be significant code changes in mongrel
from 1.1.3 to 1.1.4, though there could be, I’m not sure. It’s probably
not even a mongrel problem.

Maybe if somebody else who ‘suspects’ a memory leak could try my code
and see if it leaks theirs, that would be well appreciated.

The good news is that “maybe” updating rubygems and recompiling mongrel
helps with memory leaks in rails apps.

Thoughts?
-R
OS X PPC patchlevel 111

Maybe the upgrade installed fastthread?

Most likely though it’s related to the Rails app itself, not Mongrel,
and isn’t immediately reproducible. Cache freshness could have a large
effect on which codepaths get traversed.

What did you add that Thread loop for? I don’t understand.

Evan

On Mon, Apr 28, 2008 at 5:14 PM, Roger P. [email protected]
wrote:

Previously posted on ruby talk. Response…silence :slight_smile:

Didn’t saw that post, sorry.

first hit its memory consumption would sky-rocket. [i.e. it would
No leak! Yea!
Perhaps rubygems was updated and so it “compiled it right this time”?
Thoughts?
-R
OS X PPC patchlevel 111

Did you upgraded rubygems at the time you updated mongrel from 1.1.3 to
1.1.4?

Also, did you hit the same request/url during testing and recreating
this bug? (just to be sure).

I’ve seen lot of complains about “my mongrel is eating my memory”, but
as Erza commented in his Rack Rails [1], it seems lot of stuff at
Rails needs attention and lot of fixes, since they eat memory like a
hungry baby… and is not directly fault of Mongrel.

Could be Ruby, rubygems, some plugin you’re using, even Rails :stuck_out_tongue:

If you cannot revert back to the version of rubygems you was using to
test the exact same environment, there is nothing more we can do
except: thank god you got that solved! :stuck_out_tongue:

Maybe you can see bleakhouse [2] and give it a whirl?

[1] Ruby on Rails Blog / What is Ruby on Rails for?
[2] bleakhouse 4 « evan weaver

Luis L.
Multimedia systems

Human beings, who are almost unique in having the ability to learn from
the experience of others, are also remarkable for their apparent
disinclination to do so.
Douglas Adams

On Mon, Apr 28, 2008 at 2:23 PM, Evan W. [email protected] wrote:

Maybe the upgrade installed fastthread?
Could have. Prolly wouldn’t have done much since I’m running patch

Most likely though it’s related to the Rails app itself, not Mongrel,
and isn’t immediately reproducible. Cache freshness could have a large
effect on which codepaths get traversed.
It happened even with a ‘blank, vanilla’ rails app. All I had to do
was add an extra thread and run it in mongrel. rails 1.2.3
I believe the problem ‘kicked off’ no matter what page I
requested–meaning that basically rails had to load.
I wonder if the problem is that I started a thread BEFORE rails
loaded, and so some of that ‘non thread safeness’ of Ruby came back to
haunt me or something. It confused scopes or something. [1]

What did you add that Thread loop for? I don’t understand.
Originally it was a ‘helper thread’ that restarted my app if in
production and any files were changed. Then I saw that leaks were
happening, and replaced it with that one, to exaggerate the problem.
and was successful :slight_smile:

I almost wish I could recreate it so I could kill these leaking bugs
once and for all. I was this close from diving into eval.c and
tearing into ruby’s gc until I squashed it. So much for recreation I
guess.

As a side-note, from [1] we see that Ruby, at least the MRI of that
post, had some sockets that would be cross-threaded. I still can’t
figure out why this doesn’t occur with mongrel. Hmm.
-R

[1]
http://64.233.167.104/search?q=cache:Ilkzvy47LgcJ:permalink.gmane.org/gmane.comp.lang.ruby.mongrel.general/245+zed+shaw+rails+thread+safety&hl=en&gl=us&ct=clnk&cd=1