Reload / HUP signal not working anymore

luislavena · June 5, 2011, 2:05pm

I’m running the current stable version of Nginx (nginx: nginx version:
nginx/1.0.4), but reloading my configs doesn’t work anymore. Instead
I’ve to restart nginx on all webservers to get it reloading my configs,
but that takes some time I can’t lose.

I’ve a main config (nginx.conf) that loads up all configs from the
/domains/ folder. When altering a config inside the /domains/ folder,
reloading nginx with the -HUP signal or with the nginx -s reload
command, it doesn’t use the change I just made.

I run into this problem recently, but cannot really say if it’s since
1.0.4 or since 1.0.3. as we’ve updated directly from 1.0.2.

If there is more information required from my side, just say so. I’ve
spent hours debugging this thing now

Posted at Nginx Forum:

Renedx · June 5, 2011, 2:06pm

On Sun, Jun 05, 2011 at 08:04:25AM -0400, Renedx wrote:

I run into this problem recently, but cannot really say if it’s since
1.0.4 or since 1.0.3. as we’ve updated directly from 1.0.2.

If there is more information required from my side, just say so. I’ve
spent hours debugging this thing now

What does “nginx -t” show ?

–
Igor S.

Renedx · June 5, 2011, 2:25pm

Igor S. Wrote:

[email protected]
nginx Info Page

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Restart works as expected, its only reload that looks like it’s doing
nothing at all! Error log has nothing either.

Posted at Nginx Forum:

Renedx · June 5, 2011, 2:30pm

On Sun, Jun 5, 2011 at 7:04 PM, Renedx [email protected] wrote:

I’ve a main config (nginx.conf) that loads up all configs from the
/domains/ folder. When altering a config inside the /domains/ folder,
reloading nginx with the -HUP signal or with the nginx -s reload
command, it doesn’t use the change I just made.

I just tested on ubuntu 10.04 and freebsd 8.2 and both seemed ok with
config reloading.

Renedx · June 5, 2011, 2:53pm

Hi Igor,

I work at the same company as ReneDX, so I can deliver you with some
data.

nginx -v shows:
nginx: nginx version: nginx/1.0.4

Kind regards

Posted at Nginx Forum:

Renedx · June 5, 2011, 2:53pm

OS is CentOS 5.6

Posted at Nginx Forum:

Renedx · June 5, 2011, 2:54pm

Ah, and -V shows:
nginx -V
nginx: nginx version: nginx/1.0.4
nginx: built by gcc 4.1.2 20080704 (Red Hat 4.1.2-50)
nginx: TLS SNI support disabled
nginx: configure arguments: --prefix=/usr --sbin-path=/usr/sbin/nginx
–conf-path=/etc/nginx/nginx.conf
–error-log-path=/var/log/nginx/error.log
–pid-path=/var/run/nginx/nginx.pid --lock-path=/var/lock/nginx.lock
–with-http_ssl_module --with-http_realip_module --with-http_flv_module
–with-http_gzip_static_module --http-log-path=/var/log/nginx/access.log
–http-client-body-temp-path=/var/tmp/nginx/client/
–http-proxy-temp-path=/var/tmp/nginx/proxy/
–http-fastcgi-temp-path=/var/tmp/nginx/fcgi/ --with-ipv6

Posted at Nginx Forum:

Renedx · June 5, 2011, 2:37pm

On Sun, Jun 05, 2011 at 08:24:52AM -0400, Renedx wrote:

Igor S. Wrote:

On Sun, Jun 05, 2011 at 08:04:25AM -0400, Renedx
wrote:
What does “nginx -t” show ?

nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful

Restart works as expected, its only reload that looks like it’s doing
nothing at all! Error log has nothing either.

What does “nginx -V” show ?
What OS ?

–
Igor S.

Renedx · June 5, 2011, 10:30pm

Hello!

On Sun, Jun 05, 2011 at 09:30:55PM +0200, Srebrenko Šehić wrote:

–http-client-body-temp-path=/var/nginx/tmp/client_body_temp
This worked just fine with 0.8.50 which I recently upgraded from.
Please show global error log at notice level (at least) during
(and after) sending SIGHUP.

Maxim D.

Renedx · June 5, 2011, 9:31pm

I can confirm this bug too. What I noticed is that eg. changes to
directive “proxy_intercept_errors on;” have no effect until you issue
a full restart.

ssehic@amd64:/home/ssehic$ /usr/local/sbin/nginx -V
nginx: nginx version: nginx/1.0.3
nginx: TLS SNI support enabled
nginx: configure arguments: --prefix=/etc/nginx
–conf-path=/etc/nginx/nginx.conf --sbin-path=/usr/local/sbin/nginx
–pid-path=/var/run/nginx.pid --lock-path=/var/nginx/tmp/nginx.lock
–http-client-body-temp-path=/var/nginx/tmp/client_body_temp
–http-proxy-temp-path=/var/nginx/tmp/proxy_temp
–http-fastcgi-temp-path=/var/nginx/tmp/fastcgi_temp
–http-log-path=/var/log/nginx/access.log
–error-log-path=/var/log/nginx/error.log --user=_nginx --group=_nginx
–with-http_ssl_module --with-http_stub_status_module
–with-http_perl_module --with-debug

ssehic@amd64:/home/ssehic$ uname -a
OpenBSD amd64.lab 4.8 GENERIC.MP#7 amd64

This worked just fine with 0.8.50 which I recently upgraded from.

Cheers,
Srebrenko

Renedx · June 5, 2011, 11:05pm

On Sun, Jun 5, 2011 at 10:40 PM, Srebrenko Šehić [email protected]
wrote:

Please show global error log at notice level (at least) during
(and after) sending SIGHUP.

For my test case, this was caused by caching of 404 pages which was on
while I was doing my testing on proxy_intercept_errors on/off. HUP
works as designed.

Sorry for ths fuss and false alarm.

Cheers,
Srebrenko

Renedx · June 5, 2011, 10:41pm

On Sun, Jun 5, 2011 at 10:29 PM, Maxim D. [email protected]
wrote:

Please show global error log at notice level (at least) during
(and after) sending SIGHUP.

Here you go:

2011/06/05 22:37:42 [notice] 108#0: signal 1 (SIGHUP) received,
reconfiguring
2011/06/05 22:37:42 [notice] 108#0: reconfiguring
2011/06/05 22:37:42 [notice] 108#0: using the “kqueue” event method
2011/06/05 22:37:42 [notice] 108#0: start worker processes
2011/06/05 22:37:42 [notice] 108#0: start worker process 24157
2011/06/05 22:37:42 [notice] 108#0: start worker process 9796
2011/06/05 22:37:42 [notice] 108#0: start cache manager process 4289
2011/06/05 22:37:42 [notice] 108#0: start cache loader process 18265
2011/06/05 22:37:42 [notice] 12509#0: gracefully shutting down
2011/06/05 22:37:42 [notice] 26295#0: gracefully shutting down
2011/06/05 22:37:42 [notice] 26295#0: exiting
2011/06/05 22:37:42 [notice] 12509#0: exiting
2011/06/05 22:37:42 [notice] 26295#0: exit
2011/06/05 22:37:42 [notice] 12509#0: exit
2011/06/05 22:37:42 [notice] 22338#0: exiting
2011/06/05 22:37:42 [notice] 26133#0: exiting
2011/06/05 22:37:42 [notice] 108#0: signal 23 (SIGIO) received
2011/06/05 22:37:42 [notice] 108#0: signal 20 (SIGCHLD) received
2011/06/05 22:37:42 [notice] 108#0: cache loader process 22338 exited
with code 0
2011/06/05 22:37:42 [notice] 108#0: cache manager process 26133 exited
with code 0
2011/06/05 22:37:42 [notice] 108#0: worker process 12509 exited with
code 0
2011/06/05 22:37:42 [notice] 108#0: worker process 26295 exited with
code 0
2011/06/05 22:37:42 [notice] 108#0: signal 20 (SIGCHLD) received
2011/06/05 22:37:43 [notice] 108#0: signal 23 (SIGIO) received
2011/06/05 22:37:43 [notice] 108#0: signal 23 (SIGIO) received
2011/06/05 22:37:43 [notice] 108#0: signal 23 (SIGIO) received
2011/06/05 22:37:43 [notice] 108#0: signal 23 (SIGIO) received
2011/06/05 22:37:43 [notice] 108#0: signal 23 (SIGIO) received
2011/06/05 22:37:43 [notice] 108#0: signal 23 (SIGIO) received
2011/06/05 22:37:43 [notice] 108#0: signal 23 (SIGIO) received
2011/06/05 22:37:43 [notice] 108#0: signal 23 (SIGIO) received
2011/06/05 22:37:43 [notice] 108#0: signal 23 (SIGIO) received
2011/06/05 22:37:43 [debug] 4289#0: http file cache expire
2011/06/05 22:37:43 [debug] 4289#0: http file cache size: 0
2011/06/05 22:37:43 [debug] 4289#0: http file cache expire
2011/06/05 22:37:43 [debug] 4289#0: http file cache size: 0
2011/06/05 22:37:43 [debug] 4289#0: http file cache expire
2011/06/05 22:37:43 [debug] 4289#0: http file cache size: 0
2011/06/05 22:37:43 [debug] 4289#0: http file cache expire
2011/06/05 22:37:43 [debug] 4289#0: http file cache size: 0
2011/06/05 22:37:43 [debug] 4289#0: http file cache expire
2011/06/05 22:37:43 [debug] 4289#0: http file cache size: 0
2011/06/05 22:37:43 [debug] 4289#0: http file cache expire
2011/06/05 22:37:43 [debug] 4289#0: http file cache size: 0
2011/06/05 22:37:43 [debug] 4289#0: http file cache expire
2011/06/05 22:37:43 [debug] 4289#0: http file cache size: 0
2011/06/05 22:37:43 [debug] 4289#0: http file cache expire
2011/06/05 22:37:43 [debug] 4289#0: http file cache size: 0
2011/06/05 22:37:43 [debug] 4289#0: http file cache expire
2011/06/05 22:37:43 [debug] 4289#0: http file cache size: 0
2011/06/05 22:37:43 [notice] 108#0: signal 23 (SIGIO) received
2011/06/05 22:37:43 [notice] 108#0: signal 23 (SIGIO) received
2011/06/05 22:37:43 [notice] 108#0: signal 23 (SIGIO) received
2011/06/05 22:37:43 [notice] 108#0: signal 23 (SIGIO) received
2011/06/05 22:37:43 [notice] 108#0: signal 23 (SIGIO) received

Renedx · June 6, 2011, 10:25am

Hi Maxim,

We run into the open file limit on the nginx user, should check my
config better next time, the error log does give me errors after
rechecking error_log in the global config:
open() “/var/log/nginx/domains/[domain].log” failed (24: Too many open
files)

So this is indeed an error on our side. Thanks for helping out!

Posted at Nginx Forum:

Renedx · June 6, 2011, 12:20am

Hello!

On Sun, Jun 05, 2011 at 08:54:11AM -0400, DeviaVir wrote:

–with-http_gzip_static_module --http-log-path=/var/log/nginx/access.log
–http-client-body-temp-path=/var/tmp/nginx/client/
–http-proxy-temp-path=/var/tmp/nginx/proxy/
–http-fastcgi-temp-path=/var/tmp/nginx/fcgi/ --with-ipv6

The same as to Srebrenko Šehić:

Please show global error log at notice level (at least) during
(and after) sending SIGHUP.

Maxim D.