# Memory consumption in case of huge number of configs **URL:** \ **Category:** NGINX\ **Created:** [June 18, 2012, 9:47pm UTC](https://www.ruby-forum.com/t/memory-consumption-in-case-of-huge-number-of-configs/220729 "2012-06-18T21:47:45Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![aris](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/258eb7/32.png) [@aris](https://www.ruby-forum.com/u/aris)\ **Post date:** [June 18, 2012, 9:47pm UTC](https://www.ruby-forum.com/t/memory-consumption-in-case-of-huge-number-of-configs/220729/1 "2012-06-18T21:47:45Z")
Hello, I noticed following behavior of nginx recently, everything I describe here is related to the case when we have huge number of configuration files(\> 1000). Once nginx is started it occupies more than 100MB of memory, memory is not freed on fork. That is expected behavior but the weird thing happens on configuration reload, once HUP signal is sent, master process doubles occupied memory size, if reload is repeated memory consumption stays the same, so it looks like memory is not reused in process of reload, instead new pool is created, which leads to the waste of memory. Posted at Nginx Forum: > **[Memory consumption in case of huge number of configs](https://forum.nginx.org/read.php?2,227668,227668#msg-227668)** > > Hello, I noticed following behavior of nginx recently, everything I describe here is related to the case when we have huge number of configuration files(\> 1000). Once nginx is started it occupies more than 100MB of memory, memory is not freed on... ---
**Author:** ![dparshin](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/4491bb/32.png) [@dparshin](https://www.ruby-forum.com/u/dparshin)\ **Post date:** [June 19, 2012, 1:14am UTC](https://www.ruby-forum.com/t/memory-consumption-in-case-of-huge-number-of-configs/220729/2 "2012-06-19T01:14:44Z")
On Monday 18 June 2012 23:47:21 dparshin wrote: > I noticed following behavior of nginx recently, everything I describe > here is related to the case when we have huge number of configuration > files(\> 1000). Once nginx is started it occupies more than 100MB of > memory, memory is not freed on fork. That is expected behavior but the > weird thing happens on configuration reload, once HUP signal is sent, > master process doubles occupied memory size, if reload is repeated > memory consumption stays the same, so it looks like memory is not reused > in process of reload, instead new pool is created, which leads to the > waste of memory. Of course it creates a new pool. Nginx must continue to work and handle requests, even if it fails to load the new configuration. wbr, Valentin V. Bartenev ---
**Author:** ![dparshin](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/4491bb/32.png) [@dparshin](https://www.ruby-forum.com/u/dparshin)\ **Post date:** [June 19, 2012, 5:43pm UTC](https://www.ruby-forum.com/t/memory-consumption-in-case-of-huge-number-of-configs/220729/3 "2012-06-19T17:43:18Z")
## Valentin V. Bartenev Wrote: > once HUP signal is sent, > continue to work and handle > requests, even if it fails to load the new > configuration. Sounds reasonable, but unused pool(after reload process successfully finished) is not destroyed, and after fork the amount of unused memory is multiplied by the number of workers. I mean that we have two pools in every worker and in the master process - pool with active configuration and unused pool used for reload purposes. Posted at Nginx Forum: [http://forum.nginx.org/read.php?2,227668,227689#msg-227689](http://forum.nginx.org/read.php?2,227668,227689#msg-227689) ---
**Author:** ![dparshin](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/4491bb/32.png) [@dparshin](https://www.ruby-forum.com/u/dparshin)\ **Post date:** [June 19, 2012, 8:48pm UTC](https://www.ruby-forum.com/t/memory-consumption-in-case-of-huge-number-of-configs/220729/4 "2012-06-19T20:48:01Z")
On Tuesday 19 June 2012 19:42:14 dparshin wrote: […] > > Of course it creates a new pool. Nginx must > > continue to work and handle > > requests, even if it fails to load the new > > configuration. > > Sounds reasonable, but unused pool(after reload process successfully > finished) is not destroyed, Actually it is destroyed. But only after all old workers have finished servicing their clients. > and after fork the amount of unused memory > is multiplied by the number of workers. The memory used for configuration pool isn’t multiplied by the number of workers because of COW. wbr, Valentin V. Bartenev ---
**Author:** ![dparshin](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/4491bb/32.png) [@dparshin](https://www.ruby-forum.com/u/dparshin)\ **Post date:** [June 19, 2012, 8:22pm UTC](https://www.ruby-forum.com/t/memory-consumption-in-case-of-huge-number-of-configs/220729/5 "2012-06-19T20:22:53Z")
Hello! On Tue, Jun 19, 2012 at 11:42:14AM -0400, dparshin wrote: > > expected behavior but the > > > > Of course it creates a new pool. Nginx must > > continue to work and handle > > requests, even if it fails to load the new > > configuration. > > Sounds reasonable, but unused pool(after reload process successfully > finished) is not destroyed, It is destroyed, but you don’t see it as your system allocator doesn’t return the freed memory to the system. > and after fork the amount of unused memory > is multiplied by the number of workers. And this isn’t true even for really leaked memory, as fork() uses copy on write. See [Copy-on-write - Wikipedia](http://en.wikipedia.org/wiki/Copy_on_write). > I mean that we have two pools in > every worker and in the master process - pool with active configuration > and unused pool used for reload purposes. This is not true, see above. Maxim D. ---
**Author:** ![dparshin](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/4491bb/32.png) [@dparshin](https://www.ruby-forum.com/u/dparshin)\ **Post date:** [June 20, 2012, 7:43am UTC](https://www.ruby-forum.com/t/memory-consumption-in-case-of-huge-number-of-configs/220729/6 "2012-06-20T07:43:15Z")
Hello! On Tue, Jun 19, 2012 at 10:46:52PM +0400, Valentin V. Bartenev wrote: > Actually it is destroyed. But only after all old workers have finished > servicing their clients. No, old cycle’s pool is destroyed right after new configuration reading, at ngx\_init\_cycle() end. (Note that this applies to normal mode of operation with master process. Without master process there are some shims to delay old cycle’s pool destruction as it might be needed. This is for debugging only though.) Maxim D.