Hi Folks,
As ever, thanks for the mindblowingly great webserver.
Unfortunately, we’ve run into an odd little issue.
We’ve recently started auto-scaling at AWS, and we add and remove
upstream hosts from the config via a dinky shell script which checks for
hosts in the autoscaling group via the AWS API, and just adds them
to/removes them from the config appropriately.
After modifying our upstream conf, we do
service nginx configtest
which if successful is followed by
service nginx reload
so far, so good.
Our config is pretty straightforward, and looks a bit like
upstream apache_v5 {
server 10.0.1.81:80;
server 10.0.1.78:80;
server 10.0.1.218:80;
server 10.0.1.96:80;
server 10.0.1.71:80;
server 10.0.1.97:80;
server 10.0.1.237:80;
server 10.0.1.224:80;
server 10.0.1.66:80;
server 10.0.1.51:80;
server 10.0.1.21:80;
fair;
}
This all works beautifully for a while, but after a number of reloads
and config modifications (no particular number we’ve been able to
establish), each reload results in
ngx_slab_alloc() failed: no memory
and no reload of the config, preventing us from adding or removing
further upstream hosts without completely restarting nginx. There are no
errors preceding this, and on each subsequent reload, it reports the
same error.
We thought this may have been related to proxy buffers, so we decreased
both their size and number, but this has had no impact.
Additionally, while there’s no particular pattern in terms of an
absolute number of reloads that we’ve been able to determine, the two
nginx entry servers we’re currently using both exhibit the same
behaviour at the same time. We’re running 1.0.14. We haven’t tried with
a more up-to-date version just yet, in the hope that this may be an
obscure known issue.
Any help/insight/suggestions would be much appreciated!
Thanks,
Max