Memory footprint

Server NGINX
1

Hi all,

I’m used to see nginx memory size quite low, in the few megabytes range.
However, I have an instance here where worker process is taking about
150MB. It’s true that it serves lots of connections (openads, about 500
req/s) and that it went through some resource shortage recently (lives
inside a VPS), but still …

Is there a method for me to check what are those 150MB used for?

–

Jure Pečar
http://jure.pecar.org

2

On Nov 5, 2007 8:38 AM, Jure Pečar [email protected] wrote:

Hi all,

I’m used to see nginx memory size quite low, in the few megabytes range. However, I have an instance here where worker process is taking about 150MB. It’s true that it serves lots of connections (openads, about 500 req/s) and that it went through some resource shortage recently (lives inside a VPS), but still …

Is there a method for me to check what are those 150MB used for?

Try compiling Nginx with debug and running in debug mode, then check
the logs. Beyond that you might have to get more hardcore and take it
apart while it’s running.

~Wayne

3

On Thu, 8 Nov 2007 18:31:33 +0300
Igor S. [email protected] wrote:

Do you use proxy or fastcgi ?

FastCGI.

I’m not blaming nginx for memory leaks yet, I just want to understand
the reason for such memory footprint.

–

Jure Pečar
http://jure.pecar.org

4

On Fri, Nov 09, 2007 at 09:48:05AM +0100, Jure Pe??ar wrote:

On Thu, 8 Nov 2007 18:31:33 +0300
Igor S. [email protected] wrote:

Do you use proxy or fastcgi ?

FastCGI.

I’m not blaming nginx for memory leaks yet, I just want to understand the reason for such memory footprint.

What size of

fastcgi_buffer_size
fastcgi_buffers

?

5

On Mon, Nov 05, 2007 at 02:38:23PM +0100, Jure Pe??ar wrote:

I’m used to see nginx memory size quite low, in the few megabytes range. However, I have an instance here where worker process is taking about 150MB. It’s true that it serves lots of connections (openads, about 500 req/s) and that it went through some resource shortage recently (lives inside a VPS), but still …

Is there a method for me to check what are those 150MB used for?

nginx should not have memory leaks. At least I run it on several sites
for months without reconfiguration (so workers may run too long).

This memory may be not used by nginx and free()ed, but libc does not
return
the memory to kernel, because free memory may be sparse.

Do you use proxy or fastcgi ?

6

On Fri, Nov 09, 2007 at 12:29:23PM +0200, Riku R?is?nen wrote:

Yeah,

and if the system has plenty of free memory the kernel might not free
it up - it’s called caching and it’s a good thing - like Igor mentioned.

Just thought that it was not clear enough in Igors post, maybe.

No, the memory that program free()s is useless in kernel, because it
is not mapped to a file. So it goes directly to Free memory, that you
see
in top.

The caching may at libc level. For example, in FreeBSD 6 and early
libc by default caches up to 16 free pages, that is 64K.

7

Yeah,

and if the system has plenty of free memory the kernel might not free
it up - it’s called caching and it’s a good thing - like Igor mentioned.

Just thought that it was not clear enough in Igors post, maybe.

Riku Räisänen
HelloIT
Profian Oy
[email protected]
+358 400 882030

Igor S. kirjoitti 9.11.2007 kello 10:52:

8

On Fri, 9 Nov 2007 11:52:17 +0300
Igor S. [email protected] wrote:

What size of

fastcgi_buffer_size
fastcgi_buffers

Defaults (none of these are set in config).

Right now I see that nginx process that got me worried shrunk to 70MB.
So the size definitely has to do something with buffers and number of
connections. Is this relationship documented somewhere?

–

Jure Pečar
http://jure.pecar.org

9

On Fri, Nov 09, 2007 at 01:25:04PM +0100, Jure Pe??ar wrote:

Right now I see that nginx process that got me worried shrunk to 70MB. So the size definitely has to do something with buffers and number of connections. Is this relationship documented somewhere?
Now nginx preallocated some memory for each conneciton in
worker_connections,
however, I’m going to change it. Other memory is allocated only on
request
depening on

client_header_buffer_size
large_client_header_buffers
client_body_buffer_size
fastcgi_buffer_size
fastcgi_buffers
gzip_buffers
zlib memory: 200-400K

Have you use SSL ?

As to sparse memory allocation this case is possible:

 allocated          unmapped

!!!-------------

Then when some pages will be free(.), number of allocated pages is the
same:

 allocated          unmapped

!!..!..!..!-------------

because in general libc can not return these pages to kernel if last
page
is still used. However, there are tricks such as FreeBSD
malloc_options=“H”
that allows to return this memory to kernel or so.