# Nginx does not re-open log files on SIGUSR1 **URL:** \ **Category:** NGINX\ **Created:** [January 3, 2011, 11:48am UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228 "2011-01-03T11:48:35Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![luislavena](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/0ea827/32.png) [@luislavena](https://www.ruby-forum.com/u/luislavena)\ **Post date:** [January 3, 2011, 11:48am UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/1 "2011-01-03T11:48:35Z")
Hi there. I think this is bug, I saw that after logrotate (which on the end running SIGUSR1 on master process) my access log is empty. However if I do SIGHUP it starts working. Looks like Nginx does not re-open logfiles after USR1. I was able reproduce it multiple times, nginx version 0.8.53, I am not sure if it was working in older versions. – Piotr. ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 12:40pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/2 "2011-01-03T12:40:03Z")
Hi Piotr, > I think this is bug, I saw that after logrotate (which on the end > running SIGUSR1 on master process) my access log is empty. However if I > do SIGHUP it starts working. Looks like Nginx does not re-open logfiles > after USR1. > > I was able reproduce it multiple times, nginx version 0.8.53, I am not > sure if it was working in older versions. Are you sure that logrotate sends SIGUSR1? Did you try sending this signal yourself? How did you verify that logs did not re-open? Everything works fine with nginx/0.8.53 on my system (OpenBSD): $ nginx -V 2\>&1 | head -1 nginx version: nginx/0.8.53 $ sudo rm -rf /var/log/nginx/\* $ sudo nginx $ curl localhost \>/dev/null 2\>&1 $ ls -l /var/log/nginx/ total 8 -rw-r–r-- 1 root wheel 167 Jan 3 11:30 access.log -rw-r–r-- 1 root wheel 775 Jan 3 11:30 error.log $ sudo pkill -USR1 nginx $ curl localhost \>/dev/null 2\>&1 $ ls -l /var/log/nginx/ total 8 -rw-r–r-- 1 \_nginx wheel 334 Jan 3 11:31 access.log -rw-r–r-- 1 \_nginx wheel 1593 Jan 3 11:31 error.log Best regards, Piotr S. \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 1:02pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/3 "2011-01-03T13:02:39Z")
On 03.01.2011 13:38, Piotr S. wrote: > > I think this is bug, I saw that after logrotate (which on the end > > running SIGUSR1 on master process) my access log is empty. However if > > I do SIGHUP it starts working. Looks like Nginx does not re-open > > logfiles after USR1. […] > $ sudo pkill -USR1 nginx pkill send USR1 signal to all nginx processes, not only master, but also all worker processes in random order - it may be first workers, and last master. if worker processes does not have write permissions to /var/log/nginx directory - worker processes can`t create new log files, and can’t write to log files. USR1 signal must be sent only to nginx master process and only as postrotate script: # cat /etc/logrotate.d/nginx /var/log/nginx/\*log { daily rotate 9 missingok notifempty compress sharedscripts postrotate [! -f /var/run/nginx.pid] || kill -USR1 `cat /var/run/nginx.pid` endscript } # documentation: [Управление nginx](http://sysoev.ru/nginx/docs/control.html#logs) – Best regards, Gena ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 1:50pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/4 "2011-01-03T13:50:21Z")
Hi Piotr, On 01/03/2011 12:38 PM, Piotr S. wrote: > Are you sure that logrotate sends SIGUSR1? > Did you try sending this signal yourself? > How did you verify that logs did not re-open? Yes, I checked twice logrotate’s nginx rules and I tried send USR1 by myself. To check, after rotate I did wc -l on my\_vhost.access\_log and it was empty, I was hitting it with wget but until I did restart or reload, log was empty. I was able to ‘fix’ it, which is more like workaround than a real fix, by adding permissions for nginx user to /var/log/nginx. Before I had 700 root:root on /var/log/nginx because I am a little paranoid and I saw no real reason to add workers there since master process, running as root, is writting there. After changing owner to nginx, nginx is able re-open logs after SIGUSR1. Looks like rotated empty logs have root:root 600 perms, maybe it is the problem? But again, I think master write there, not workers. – Piotr. ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 1:19pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/5 "2011-01-03T13:19:36Z")
Hi, > pkill send USR1 signal to all nginx processes, > not only master, but also all worker processes > in random order - it may be first workers, and last master. $ curl localhost \>/dev/null 2\>&1 $ ls -l /var/log/nginx/ total 8 -rw-r–r-- 1 root wheel 167 Jan 3 12:09 access.log -rw-r–r-- 1 root wheel 1202 Jan 3 12:12 error.log $ sudo pkill -USR1 -f ^nginx:.\*master.\*process $ curl localhost \>/dev/null 2\>&1 $ ls -l /var/log/nginx/ total 12 -rw-r–r-- 1 \_nginx wheel 334 Jan 3 12:13 access.log -rw-r–r-- 1 \_nginx wheel 2738 Jan 3 12:13 error.log Happy? Result is exactly the same… > if worker processes does not have write permissions > to /var/log/nginx directory - worker processes can`t > create new log files, and can’t write to log files. Worker process doesn’t create log files, master process does. Other than complaining about non-existing issue, you didn’t answer any of my questions… Best regards, Piotr S. \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 1:54pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/6 "2011-01-03T13:54:46Z")
Hi, > Before I had 700 root:root on /var/log/nginx because I am a little > paranoid and I saw no real reason to add workers there since master > process, running as root, is writting there. You need at least 711, otherwise workers won’t be able to open files in that directory. Best regards, Piotr S. \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 2:11pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/7 "2011-01-03T14:11:44Z")
Hi, > you didn’t ask any questions. I did… But I’ve realized just now that you’re not the OP 😉 Best regards, Piotr S. \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 2:00pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/8 "2011-01-03T14:00:25Z")
On 03.01.2011 14:18, Piotr S. wrote: > > pkill send USR1 signal to all nginx processes, > > not only master, but also all worker processes > > in random order - it may be first workers, and last master. > > if worker processes does not have write permissions > > to /var/log/nginx directory - worker processes can`t > > create new log files, and can’t write to log files. > Worker process doesn’t create log files, master process does. if worker processes have write permissions to /var/log/nginx - it can create log files, see nginx sources for reference: ngx\_process\_cycle.c file, ngx\_worker\_process\_cycle function. > you didn’t answer any of my questions… you didn’t ask any questions. P.S. nginx 0.8.53 log files rotation work fine on my machines with logrotate config mentioned in my previous message. – Best regards, Gena ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 2:57pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/9 "2011-01-03T14:57:38Z")
Is there a reason why every script or suggestion I see floating around the net still does it “the hard way” (and I wonder, even on _this_ mailing list): $ kill -USR1 `cat /var/run/nginx.pid` $ pkill -USR1 -f ^nginx:.\*master.\*process instead of simply using nginx’s built-in way to do this: $ nginx -s reopen After all, this is supported since 0.7.53. It should also work fine if using multiple nginx instances, just pass the path to the configuration file of the specific instance with -c. See also [CommandLine | NGINX](http://wiki.nginx.org/CommandLine) or `nginx -h`: -s signal : send signal to a master process: stop, quit, reopen, reload ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 2:17pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/10 "2011-01-03T14:17:23Z")
On 03.01.2011 14:49, Piotr K. wrote: > I was able to ‘fix’ it, which is more like workaround than a real fix, > by adding permissions for nginx user to /var/log/nginx. > > Before I had 700 root:root on /var/log/nginx because I am a little > paranoid and I saw no real reason to add workers there since master > process, running as root, is writting there. > > After changing owner to nginx, nginx is able re-open logs after SIGUSR1. master process running as root open/write files in /var/log/nginx - if nginx user have write permissions to this directory, 700 nginx:nginx - such setup is vulnerable by symlink attack better approach set permissions 750 root:nginx /var/log/nginx or 750 root:www-logs /var/log/nginx and add user nginx to group www-logs > Looks like rotated empty logs have root:root 600 perms, maybe it is the > problem? show your logrotate config for nginx log files. > But again, I think master write there, not workers. nginx workers also write to log files. – Best regards, Gena ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 3:10pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/11 "2011-01-03T15:10:50Z")
Hi, > So nginx’ workers need exec permission on logdir? Yes. > Exec on dir will allow only chdir there, why worker have to chdir there? They don’t, man chmod. Best regards, Piotr S. \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 3:06pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/12 "2011-01-03T15:06:26Z")
On 01/03/2011 02:16 PM, Gena M. wrote: > master process running as root open/write files in /var/log/nginx > > - if nginx user have write permissions to this directory, > 700 nginx:nginx - such setup is vulnerable by symlink attack > better approach set permissions 750 root:nginx /var/log/nginx > > or 750 root:www-logs /var/log/nginx and add user nginx to group www-logs Now when you mention it, if nginx worker have read perms there (as you suggested above), then if user symlink to any log, he will be able fetch it via nginx which is security hole. > nginx workers also write to log files. In what cases? And direct or somehow ‘via master proicess’? On 01/03/2011 01:54 PM, Piotr S. wrote: > You need at least 711, otherwise workers won’t be able to open > files in that directory. So nginx’ workers need exec permission on logdir? Exec on dir will allow only chdir there, why worker have to chdir there? The only problem is that after SIGUSR1 nginx worker _need_ access to logs (shouldn’t), where on restart/reload nginx can handle it without access to logs by workers, which as I said above, is [in my opinion] security hole. – Piotr ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 3:15pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/13 "2011-01-03T15:15:51Z")
Hi John, > Is there a reason why every script or suggestion I see floating around > the net still does it “the hard way” (and I wonder, even on _this_ > mailing list): > > $ kill -USR1 `cat /var/run/nginx.pid` > $ pkill -USR1 -f ^nginx:.\*master.\*process Yes, there is: consistency. Pretty much every daemon supports “the standard way” for control (graceful restarts, termination, reloads, etc), so people that use other daemons prefer to use the method that is consistent among all of them instead of using daemon-specific one. Best regards, Piotr S. \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 3:17pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/14 "2011-01-03T15:17:44Z")
On 01/03/2011 03:09 PM, Piotr S. wrote: > Hi, > > > So nginx’ workers need exec permission on logdir? > > Yes. Any reason to? Nginx works for me flawless on each box with 700 root:root on /var/log/nginx, the only problem I found is SIGUSR1, Whatever you agree with me or not, nginx shoudn’t need perms on its logs dir, because it will allow users to use symlink to fetch logs. – Piotr. ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 3:27pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/15 "2011-01-03T15:27:10Z")
Hi, > Any reason to? Yes, user requires “+x” permission to the directory in order to be able to open any file(s) inside it. Google/Bing/whatever for “unix permissions”, this is as simple as it gets. > Nginx works for me flawless on each box with 700 root:root on > /var/log/nginx, the only problem I found is SIGUSR1, Whatever you agree > with me or not, nginx shoudn’t need perms on its logs dir, because it will > allow users to use symlink to fetch logs. This is because: - on start and reload - master process opens log files before fork() and worker processes only inherit them, - on reopen - all processes need to open logs, so workers also need permission to open log files. Best regards, Piotr S. \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 3:44pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/16 "2011-01-03T15:44:09Z")
On 01/03/2011 03:25 PM, Piotr S. wrote: > Hi, > > > Any reason to? > > Yes, user requires “+x” permission to the directory in order to be able > to open any file(s) inside it. Google/Bing/whatever for “unix > permissions”, this is as simple as it gets. This is what I mean by ‘exec will allow only chdir there’. With X you can access dir content and depends on files rights, you can read them etc. Mental shortcut. > > Nginx works for me flawless on each box with 700 root:root on > > /var/log/nginx, the only problem I found is SIGUSR1, Whatever you > > agree with me or not, nginx shoudn’t need perms on its logs dir, > > because it will allow users to use symlink to fetch logs. > > This is because: > > - on start and reload - master process opens log files before fork() and > worker processes only inherit them, > - on reopen - all processes need to open logs, so workers also need > permission to open log files. Well ok, I understand [now] why it is needed (perms that is). However security issue still remains which in my opinion should be addressed as bug and fixed, can you agree with me? – Piotr. ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 4:21pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/17 "2011-01-03T16:21:11Z")
Hi Piotr! > Pretty much every daemon supports “the standard way” for control > (graceful restarts, termination, reloads, etc), so people that use other > daemons prefer to use the method that is consistent among all of them > instead of using daemon-specific one. I don’t argue that the most common set of signals and their similar handlers in various daemons (HUP=reload, TERM=clean shutdown, QUIT=graceful shutdown…) make life a lot easier for most administrators. But I’m not convinced that this is the case here. As soon as USR\* and whatever other custom signals are handled in a way that is special to the specific daemon, there’s not really a point to “consistency” any more. It _is_ special already. And IMHO this is exactly the time where a control CLI can shine, especially since the available signals are limited and the possible control knobs are not. Apache has apachectl[1] or httpd’s “-k” option for quite some time now. It is used by many distributions in init scripts and log rotation scripts instead of hardcoding signals and paths to pid files everywhere. Nginx has the “-s” option. For me, this appears to be cleaner and simpler than to remember or always look up what custom signal X does to this daemon. For the most basic set of control signals I agree with you, but it still has the downside of having to find the pid first (and hardcoding parameters of the daemon configuration into external scripts). [1] [apachectl - Apache HTTP Server Control Interface - Apache HTTP Server Version 2.2](http://httpd.apache.org/docs/2.2/programs/apachectl.html) ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 4:35pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/18 "2011-01-03T16:35:01Z")
Hi, > I don’t argue that the most common set of signals and their similar > handlers in various daemons (HUP=reload, TERM=clean shutdown, > QUIT=graceful shutdown…) make life a lot easier for most administrators. > > But I’m not convinced that this is the case here. As soon as USR\* and > whatever other custom signals are handled in a way that is special to > the specific daemon, there’s not really a point to “consistency” any > more. It _is_ special already. USR1 is pretty much “standard” for log files re-open. > For the most basic set of control signals I agree with you, > but it still has the downside of having to find the pid first (and > hardcoding parameters of the daemon configuration into external scripts). With “pkill” you don’t have to 😉 Anyway, I’m not saying that sending signals is better than “nginx -s”, I’m just saying it’s more consistent and that’s why I’m using it. Best regards, Piotr S. \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 4:48pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/19 "2011-01-03T16:48:12Z")
Hi, > This is what I mean by ‘exec will allow only chdir there’. With X you can > access dir content and depends on files rights, you can read them etc. > Mental shortcut. OK, but it has nothing to do with chdir 😉 > Well ok, I understand [now] why it is needed (perms that is). However > security issue still remains which in my opinion should be addressed as > bug and fixed, can you agree with me? I would imagine that there are far more important files than nginx logs that your workers have access to 😉 Best regards, Piotr S. \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \> ---
**Author:** ![hibernaculus](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/6de8d8/32.png) [@hibernaculus](https://www.ruby-forum.com/u/hibernaculus)\ **Post date:** [January 3, 2011, 5:03pm UTC](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228/20 "2011-01-03T17:03:17Z")
On 03.01.2011 15:56, John Feuerstein wrote: > kill -USR1 `cat /var/run/nginx.pid` works faster then nginx -s reopen and also kill -USR1 `cat /var/run/nginx.pid` has explicitly defined syntax and behavior. > It should also work fine if using multiple nginx instances, just pass > the path to the configuration file of the specific instance with -c. in many cases nginx config can be very big, with thousands of includes. nginx -s reopen feature was added for windows version of nginx, AFAIK. – Best regards, Gena [Next page](https://www.ruby-forum.com/t/nginx-does-not-re-open-log-files-on-sigusr1/201228.md?page=2)