Forum: NGINX Memory consumption of ningx

2974d09ac2541e892966b762aad84943?d=identicon&s=25 peschuster (Guest)
on 2012-10-29 18:13
(Received via mailing list)
Hi,

I cross-compiled nginx for microblaze processors
(http://github.com/peschuster/nginx) and am currently doing some
performance
benchmarks with nginx running on a microblaze processor with a custom
designed SoC on a FPGA.

However, I am having problems with the memory consumption of nginx:

When I perform 10,000 requests with 20 conn/s and 2 requests/conn (using
httperf - 1), memory used by nginx grows to about 40 MB.
When I repeat this benchmark, the used memory grows from 40 to 80 MB.

The problem with this behavior is that my SoC only has 256 MB of RAM in
total (the file system also runs completely from RAM using a ramdisk).
Therefore nginx crashes the complete system by consuming all memory for
longer/extended benchmark scenarios.

Is this the intended behavior of nginx? Why isn't it "re-using" the
already
allocated memory?
Any hints on how I can circumvent or track down this problem?

Thanks.

Peter

1: httperf --timeout=5 --client=0/1 --server=192.168.2.125 --port=80
--uri=/index.html --rate=20 --send-buffer=4096 --recv-buffer=16384
--num-conns=5000 --num-calls=2

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,232328,232328#msg-232328
96321bb7fd6f712aa7785ce2d58388f2?d=identicon&s=25 Anton Yuzhaninov (Guest)
on 2012-10-30 11:15
(Received via mailing list)
On 10/29/12 21:13, peschuster wrote:
> However, I am having problems with the memory consumption of nginx:
>
> When I perform 10,000 requests with 20 conn/s and 2 requests/conn (using
> httperf - 1), memory used by nginx grows to about 40 MB.
> When I repeat this benchmark, the used memory grows from 40 to 80 MB.

1. Do you use 3rd party modules?
2. This request served by nginx (e. g. static files) or proxied to some
backend?
3. Memory usage depends on used features: SSL, SSI, gzip, limt rate, geo
module,
e. t. c.

If gzip is used for static files, better to pre-compress them, and use
ngx_http_gzip_static_module

Also yo save memory use 1 worker and set reasonable small limit on
connections:

   worker_processes  1;

   events {
     worker_connections 512;
   }

--
  Anton Yuzhaninov
2974d09ac2541e892966b762aad84943?d=identicon&s=25 peschuster (Guest)
on 2012-10-30 12:14
(Received via mailing list)
I don't use any third-party modules, neigther SSL or gzip. nginx was
compiled using the following parameters:

> --with-debug \
> --without-http_rewrite_module \
> --without-http_gzip_module

nginx only serves one static file. Changing the size of the file (10 KB
to
200 B)  has no effect on the memory comsumption of nginx.

Here is my nginx.conf:

>
> http {
>     include       mime.types;
>     default_type  application/octet-stream;
>
>     log_format  main  '$remote_addr - $remote_user [$time_local]
"$request" '
>     server {
>         listen       80;
>         server_name  localhost;
>
>         location / {
>             root   html;
>             index  index.html index.htm;
>         }
>     }
> }

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,232328,232368#msg-232368
C9264ddbc4a782fff755b0125364cbdf?d=identicon&s=25 Valentin V. Bartenev (Guest)
on 2012-10-30 12:42
(Received via mailing list)
On Monday 29 October 2012 21:13:01 peschuster wrote:
> httperf - 1), memory used by nginx grows to about 40 MB.
> When I repeat this benchmark, the used memory grows from 40 to 80 MB.
>
> The problem with this behavior is that my SoC only has 256 MB of RAM in
> total (the file system also runs completely from RAM using a ramdisk).
> Therefore nginx crashes the complete system by consuming all memory for
> longer/extended benchmark scenarios.
>
> Is this the intended behavior of nginx? Why isn't it "re-using" the already
> allocated memory?

Nginx releases allocated memory after it completes each request.

> Any hints on how I can circumvent or track down this problem?
>

It most likely that your system memory allocator do not return freed
memory to
the OS.

 wbr, Valentin V. Bartenev

--
http://nginx.com/support.html
http://nginx.org/en/donation.html
2974d09ac2541e892966b762aad84943?d=identicon&s=25 peschuster (Guest)
on 2012-10-31 12:29
(Received via mailing list)
VBart Wrote:

> It most likely that your system memory allocator do not return freed
> memory to the OS.

How can I check this? I suspect this should be part of the OS (Linux)?
Could
you give me any keyword to read more about this?

I looked at /proc/meminfo after the first and second batch of requests.
The newly allocated memory is categorized as "Active(anon)":

> Active:            36976 kB
> Inactive:          59280 kB
> Active(anon):      34376 kB
> ..
> AnonPages:         34400 kB

vs.

> Active:            71092 kB
> Inactive:          60088 kB
> Active(anon):      68428 kB
> ..
> AnonPages:         68452 kB

Thanks.
Peter

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,232328,232426#msg-232426
C9264ddbc4a782fff755b0125364cbdf?d=identicon&s=25 Valentin V. Bartenev (Guest)
on 2012-10-31 15:43
(Received via mailing list)
On Wednesday 31 October 2012 15:28:58 peschuster wrote:
> VBart Wrote:
> > It most likely that your system memory allocator do not return freed
> > memory to the OS.
>
> How can I check this? I suspect this should be part of the OS (Linux)?

Usually it's a part of glibc in Linux.

> Could you give me any keyword to read more about this?

Try man mallopt.

 wbr, Valentin V. Bartenev

--
http://nginx.com/support.html
http://nginx.org/en/donation.html
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.