Mongrel and memory usage

On 11/6/07, Pete DeLaurentis [email protected] wrote:

Does Mongrel need to be multi-threaded at all if you’re working with
Rails applications?

No.

If all 8 mongrels are busy, I believe Lighty retries the cycle a few
times. So, who needs threads? I’m guessing this is a naive
question, but I’d appreciate it if you’d set me straight.

evented_mongrel still queues the request, but it does so without
creating any threads, so there isn’t the thread related RAM growth.

Once I get a breather in my release schedule, I plan to look at a
switch to evented mongrel. Performance benchmarks + community
feedback looks very good. But I still need to get a better grasp on
how it works + the differences from standard mongrel.

My intention is that switching to evented_mongrel or
swiftiplied_mongrel is transparent from the perspective of the
application (or whatever is running inside a mongrel handler).

Kirk H.

At 09:17 AM 11/5/2007, you wrote:

Otherwise, can you start a test instance of your application, and then
test it to see if there are certain actions which cause the memory
growth. That would help you pinpoint where the likely problems are.
Just use ab or httperf to send a large number of requests to specific
urls in your app, and see how ram usage changes as you do that.

Kirk H.

Thanks Kirk - I guess I’m totally OT at this point, but I hadn’t heard
about this bug before. From your description this is a specific problem
to the underlying C code implementing shift, which is not found in
related functions? So “array.slice!(0)” would be identical in function
to shift but not contain this leak?

Thanks again,

Steve

On 11/6/07, Bob H. [email protected] wrote:

It isn’t fixed in the ruby that ships with Leopard:
1.8.6 (2007-06-07 patchlevel 36) [universal-darwin9.0]

Ugh. IIRC I checked it with the patch release after 36, and it was
fixed there.

end
Note that this just mostly fixes things. You still end up with
array elements in memory carrying around Qnils, but most of the time
that’s good enough.

Kirk H.

Why not build from source:

ruby 1.8.6 (2007-09-23 patchlevel 110) [i686-darwin8.10.1]

Hi,

On 5-Nov-07, at 1:38 PM, Kirk H. wrote:

Yeah. It looked to me like whoever wrote the original array.c code
just forgot something when writing the code, because it’s just #shift
that has the problem.

This bug was fixed, but not until 1.8.6. I know it is fixed as of at
least the last couple of patch releases. I am unsure if it was fixed
in the original 1.8.6 release, however.

It isn’t fixed in the ruby that ships with Leopard:
1.8.6 (2007-06-07 patchlevel 36) [universal-darwin9.0]

This hack will fix things.

class Array
alias :naughty_shift :shift
def shift
result = self.first
self[0] = nil # This is the ‘magic’
self.naughty_shift
result
end
end

Kirk H.


Mongrel-users mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/mongrel-users


Bob H. – tumblelog at
http://www.recursive.ca/so/
Recursive Design Inc. – weblog at
http://www.recursive.ca/hutch
http://www.recursive.ca/ – works on
http://www.raconteur.info/cms-for-static-content/home/

On Mon, 5 Nov 2007 09:55:34 -0600
“Alexey V.” [email protected] wrote:

But Ruby processes never release memory back to the operating system.
So, the fact that its RSS never goes down is normal.

In normal circumstances, Mongrel should grow up to some point around
60-120 Mb and stay there. 300 Mb and growing is a sure sign you have a
memory leak somewhere.

Ehem, s/Mongrel/Ruby/g on the above.


Zed A. Shaw

On Tue, 6 Nov 2007 14:34:25 +1100
Dave C. [email protected] wrote:

Peak Obsession

Is this a bug in send_file?

You souldn’t use send_file at all really, because this streams the full
file into a StringIO so that mongrel can then send the StringIO outside
the rails lock, and because rails is inconsistent in how it sends
headers and the body.

You should be using either x-sendfile or simply redirect to the real
image. If you need to auth the images then check out some of the
auth-before-redirect modules available for various web servers.


Zed A. Shaw

If you need to auth the images then check out some of the auth-before-redirect modules available for various web servers.

I think Danga’s Perlbal was made for just this purpose.

Evan

On Mon, 5 Nov 2007 17:06:01 +0100
“Thomas B.” [email protected] wrote:

Hello Kirk,

Thanks for your answer.
I’m using ruby 1.8.5 (2006-08-25) [i486-linux].
The Rails app uses those plugins :

  • acts_as_taggable_on_steroids
  • attachment_fu
  • exception_notification
  • localization

Hmm, I seem to see this problem quite a lot with attachment_fu
installations. Just a hunch.


Zed A. Shaw

Hello,

I just wanted to tell the list that I’ve spent some time to optimize
my code a little, I’ve reworked some SQL queries, removed some part of
Rails I wasn’t using …
Now, both mongrel processes are stable at 150Mb each.

T.