# Normal memory usage for SSL terminating, reverse proxy nginx? **URL:** \ **Category:** NGINX\ **Created:** [May 27, 2016, 9:16am UTC](https://www.ruby-forum.com/t/normal-memory-usage-for-ssl-terminating-reverse-proxy-nginx/244691 "2016-05-27T09:16:06Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![Sadaf\_N](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/858c86/32.png) [@Sadaf\_N](https://www.ruby-forum.com/u/Sadaf_N)\ **Post date:** [May 27, 2016, 9:16am UTC](https://www.ruby-forum.com/t/normal-memory-usage-for-ssl-terminating-reverse-proxy-nginx/244691/1 "2016-05-27T09:16:06Z")
It has been a difficult topic to research. The nginx instance is doing nothing more than what the subject stated. It reverse proxies to a backend, load balanced set of web app instances and terminates SSL for a large number of unique domains each with their own SSL cert. Here’s a `ps aux` of nginx running after a clean start and zero (out of rotation) traffic. root 20 0 676052 598224 1848 S 0.0 16.5 0:00.06 nginx nginx 20 0 675552 597204 1228 S 0.0 16.5 0:00.44 nginx nginx 20 0 675552 596612 636 S 0.0 16.5 0:00.36 nginx Looking at that process list, nginx is using about 676mb of RAM for ~400 vhosts each with their own unique SSL cert for a unique domain. Here’s an example of a vhost server config. They’re all generated based on the same base template: server { listen 443 ssl proxy\_protocol; server\_name www.; access\_log /var/log/nginx/access\_vhost\_443.log accesslog; error\_log /var/log/nginx/error\_vhost\_443.log warn; real\_ip\_header proxy\_protocol; ssl on; ssl\_certificate /etc/nginx/ssl//.crt; ssl\_certificate\_key /etc/nginx/ssl//.key; ssl\_stapling on; ssl\_stapling\_verify on; resolver internal-dns.vpc valid=60s; set $internal “upstream-load-balancer.vpc”; location / { if ($denied) { return 444; } proxy\_pass http://$internal; } } Now, this wouldn’t be all that bad. 1.69mb of memory per vhost isn’t horrible, high, but not unsustainable. However, if I do `nginx -s reload` or restart via systemd service… root 20 0 1370188 1.176g 3240 S 0.0 33.4 0:14.98 nginx nginx 20 0 1370192 1.175g 2584 S 0.3 33.4 2:39.95 nginx nginx 20 0 1370192 1.175g 2584 S 1.7 33.4 2:28.42 nginx It doubles the memory consumption! It never goes up or down drastically again. It’s as if it duplicates and never frees or releases unless you do a restart. This was tested on a handful of AWS EC2 instance types using vanilla Centos7 and both nginx 1.6.3 (stable in centos repos) and nginx 1.10.0 ([nginx.org](http://nginx.org) repo). In summary, my questions are thus: - Is it normal for nginx to use ~1.7mb per SSL vhost? - Is there a way to reduce that memory usage? - Am I the only one that experiences the doubling of nginx memory usage after a nginx reload? - Is that a bug? Thanks! Posted at Nginx Forum: > **[Normal memory usage for SSL terminating, reverse proxy nginx?](https://forum.nginx.org/read.php?2,267189,267189#msg-267189)** > > It has been a difficult topic to research. The nginx instance is doing nothing more than what the subject stated. It reverse proxies to a backend, load balanced set of web app instances and terminates SSL for a large number of unique domains each... ---
**Author:** ![onecrazymonkey](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/o/b5a626/32.png) [@onecrazymonkey](https://www.ruby-forum.com/u/onecrazymonkey)\ **Post date:** [May 27, 2016, 3:58pm UTC](https://www.ruby-forum.com/t/normal-memory-usage-for-ssl-terminating-reverse-proxy-nginx/244691/2 "2016-05-27T15:58:37Z")
Hello! On Fri, May 27, 2016 at 03:15:25AM -0400, onecrazymonkey wrote: > nginx 20 0 675552 596612 636 S 0.0 16.5 0:00.36 nginx > error\_log /var/log/nginx/error\_vhost\_443.log warn; > real\_ip\_header proxy\_protocol; > > ssl on; > ssl\_certificate /etc/nginx/ssl//.crt; > ssl\_certificate\_key /etc/nginx/ssl//.key; > > ssl\_stapling on; > ssl\_stapling\_verify on; > > resolver internal-dns.vpc valid=60s; Note: by using using own resolver in each server{} you add more flexibility, but waste memory and other resources. Consider using single resolver defined at http{} level instead. > horrible, high, but not unsustainable. However, if I do `nginx -s reload` or > restart. > As nginx allocates a new configuration before releasing the old > one, and this doubles memory consumption of the master process for a > while. The old configuration is then freed, but in many cases > this memory isn’t really returned to kernel by the allocator, and > hence seen in various stats. > This was tested on a handful of AWS EC2 instance types using vanilla Centos7 > and both nginx 1.6.3 (stable in centos repos) and nginx 1.10.0 ([nginx.org](http://nginx.org) > repo). > > In summary, my questions are thus: > > - Is it normal for nginx to use ~1.7mb per SSL vhost? Not really. This is likely due to a particular configuration you are using: own resolver in each vhost, own logs, etc. Consider moving common settings to the http{} level, it should somewhat reduce memory consumption. Just for the reference: a simple test with 10k SSL vhosts like: server { listen 8443; ssl on; server\_name 1.foo; ssl\_certificate test.crt; ssl\_certificate\_key test.key; } server { listen 8443; ssl on; server\_name 2.foo; ssl\_certificate test.crt; ssl\_certificate\_key test.key; } … server { listen 8443; ssl on; server\_name 10000.foo; ssl\_certificate test.crt; ssl\_certificate\_key test.key; } takes about 200mb of memory (on a 32-bit host). That is, about 20k of memory per vhost. With more options it will take more memory, but 1.7mb is a bit too many - you are doing something wrong. Just a guess: if you use ssl\_trusted\_certificate at http{} level, make sure you are loading only needed certificates, not a full OS-provided bundle. > - Is there a way to reduce that memory usage? Yes, see various hints above. In summary: configure things at http{} level where possible, avoid memory-hungry things you don’t need. > - Am I the only one that experiences the doubling of nginx memory usage > after a nginx reload? > - Is that a bug? No and no, see above. – Maxim D. > **[nginx](http://nginx.org/)**