Forum: NGINX ngx_slab_alloc() failed: no memory (not push, not key space)

Hi Folks,

As ever, thanks for the mindblowingly great webserver.

Unfortunately, we've run into an odd little issue.

We've recently started auto-scaling at AWS, and we add and remove 
upstream hosts from the config via a dinky shell script which checks for 
hosts in the autoscaling group via the AWS API, and just adds them 
to/removes them from the config appropriately.

After modifying our upstream conf, we do

service nginx configtest

which if successful is followed by

service nginx reload

so far, so good.

Our config is pretty straightforward, and looks a bit like

upstream apache_v5 {
  server 10.0.1.81:80;
  server 10.0.1.78:80;
  server 10.0.1.218:80;
  server 10.0.1.96:80;
  server 10.0.1.71:80;
  server 10.0.1.97:80;
  server 10.0.1.237:80;
  server 10.0.1.224:80;
  server 10.0.1.66:80;
  server 10.0.1.51:80;
  server 10.0.1.21:80;

  fair;
}

This all works beautifully for a while, but after a number of reloads 
and config modifications (no particular number we've been able to 
establish), each reload results in

ngx_slab_alloc() failed: no memory

and no reload of the config, preventing us from adding or removing 
further upstream hosts without completely restarting nginx. There are no 
errors preceding this, and on each subsequent reload, it reports the 
same error.

We thought this may have been related to proxy buffers, so we decreased 
both their size and number, but this has had no impact.

Additionally, while there's no particular pattern in terms of an 
absolute number of reloads that we've been able to determine, the two 
nginx entry servers we're currently using both exhibit the same 
behaviour at the same time. We're running 1.0.14. We haven't tried with 
a more up-to-date version just yet, in the hope that this may be an 
obscure known issue.

Any help/insight/suggestions would be much appreciated!

Thanks,

Max