Forum: NGINX Quick question about using kill -USR1 to recreate access.log

2974d09ac2541e892966b762aad84943?d=identicon&s=25 samingrassia (Guest)
on 2014-05-19 21:06
(Received via mailing list)
Thanks to everyone in advance!

I have a cron that runs the following:

mv $NGINX_ACCESS_LOG $ACCESS_LOG_DROPBOX/$LOG_FILENAME
kill -USR1 `cat $NGINX_PID`

My questions is during time between the mv and the kill, is there any
log
writes that are being discarded or are they being stacked in memory and
dumped into the new access.log after it is recreated?

Thanks!

Sam

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,250214,250214#msg-250214
36a8284995fa0fb82e6aa2bede32adac?d=identicon&s=25 Francis Daly (Guest)
on 2014-05-19 21:37
(Received via mailing list)
On Mon, May 19, 2014 at 03:06:06PM -0400, samingrassia wrote:

Hi there,

> mv $NGINX_ACCESS_LOG $ACCESS_LOG_DROPBOX/$LOG_FILENAME
> kill -USR1 `cat $NGINX_PID`
>
> My questions is during time between the mv and the kill, is there any log
> writes that are being discarded or are they being stacked in memory and
> dumped into the new access.log after it is recreated?

What happens when you do

  mv $NGINX_ACCESS_LOG $ACCESS_LOG_DROPBOX/$LOG_FILENAME

and then issue a http request of your nginx server, before the kill?

Do you see the log line go into $NGINX_ACCESS_LOG; onto the end of
$ACCESS_LOG_DROPBOX/$LOG_FILENAME; or disappear without being written
anywhere?

I'd expect the first option not to happen; the second option to happen
if the "mv" is a "rename"; and the third option to happen if the "mv"
is a "copy and delete". So make sure that your "mv" is a "rename",
and you'll be fine.

Actually, I'd expect the first option to happen if you are using
variables
in your log file name, according to its documentation.

  f
--
Francis Daly        francis@daoine.org
Ba5061f1803f9ab0cfc3d53366d49546?d=identicon&s=25 Lord Nynex (Guest)
on 2014-05-19 21:54
(Received via mailing list)
The name of the file is really sort of irrelevant. The file descriptor
will
still point at $ACCESS_LOG_DROPBOX/$LOG_FILENAME. Any log lines between
MV
and KILL *should* still be written there.

Why not use logrotate? Why not use nginx reload? Why not use HUP?
1266aa99d1601b47bbd3ec22affbb81c?d=identicon&s=25 B.R. (Guest)
on 2014-05-20 09:03
(Received via mailing list)
On Mon, May 19, 2014 at 9:53 PM, Lord Nynex <lordnynex@gmail.com> wrote:

>
> The name of the file is really sort of irrelevant. The file descriptor
> will still point at $ACCESS_LOG_DROPBOX/$LOG_FILENAME. Any log lines
> between MV and KILL *should* still be written there.
>
> Why not use logrotate?
>

​Th​ere has been no precision on *why* the user wanted to mv the log
file
somewhere else. Stating that is for log rotation/storage purpose is
speculation.

Why not use nginx reload? Why not use HUP?
>

​You​

​are repeating yourself, as the first usually equals the second (the
service 'reload' command is usually a wrap-up of the -HUP signal)​

As Francis suggested, there should be special care about what the moving
command will do:
The docs abut controlling nginx
<http://nginx.org/en/docs/control.html#logs>say that the old log file
will remain opened until the -USR1 signal closes
file descriptors pointing to it.
- A local 'mv' will keep the same inode, thus only renaming the file and
keeping opened file descriptors on it intact. You want that since all
incoming log messages will continue to be sent to this file until the
-USR1
signal switches to a new one.
- On the contrary, a remote 'mv' (ie moving file to network storage)
will
copy and *delete* the file, making the file descriptor opened by nginx
invalid.

The docs do not say what happens to log messages trying to be sent to
invalid fd, although as Francis suggest, it might be simply discarded.
Maybe are there some buffers FIFO-like n front of the fd that would
retain
some of the messages? Pure speculation.
I have not had a look to the sources, if someone is around to provide us
with that information (or better: put a wuick word on it in the docs), I
would be glad. :o)
​
---
*B. R.*
40b4c848b8fcd63b0cb60b9d170c3a77?d=identicon&s=25 Valentin V. Bartenev (Guest)
on 2014-05-20 09:38
(Received via mailing list)
On Tuesday 20 May 2014 09:01:45 B.R. wrote:
>
> The docs abut controlling nginx
> <http://nginx.org/en/docs/control.html#logs>say that the old log file
> will remain opened until the -USR1 signal closes
> file descriptors pointing to it.
> - A local 'mv' will keep the same inode, thus only renaming the file and
> keeping opened file descriptors on it intact. You want that since all
> incoming log messages will continue to be sent to this file until the -USR1
> signal switches to a new one.
> - On the contrary, a remote 'mv' (ie moving file to network storage) will
> copy and *delete* the file, making the file descriptor opened by nginx
> invalid.
[..]

Deleting a file doesn't make file descriptor "invalid".  It's valid and
the
file actually exists on file system till there is at least one
descriptor
pointing to that file.

  wbr, Valentin V. Bartenev
B8df2f301be5a8f1c6475d98a7562592?d=identicon&s=25 Lasse Laursen (Guest)
on 2014-05-20 09:52
(Received via mailing list)
Alternatively have a look here:
http://www.cambus.net/log-rotation-directly-within...

Kind regards
Lasse Laursen
40b4c848b8fcd63b0cb60b9d170c3a77?d=identicon&s=25 Valentin V. Bartenev (Guest)
on 2014-05-20 10:05
(Received via mailing list)
On Tuesday 20 May 2014 09:52:09 Lasse Laursen wrote:
> Alternatively have a look here:
> http://www.cambus.net/log-rotation-directly-within...
> /
>
[..]

This is a bad way to solve the problem.

 1. It introduces two additional open()/close() system calls per
access_log
    per request (not to mention all regexp and variables processing);

 2. It unnecessarily complicates nginx configuration, which is intended
to
    stay clear and simple.

What is the problem with log rotation tools?  Why even is someone even
trying
to avoid using them?

  wbr, Valentin V. Bartenev
1266aa99d1601b47bbd3ec22affbb81c?d=identicon&s=25 B.R. (Guest)
on 2014-05-20 12:50
(Received via mailing list)
On Tue, May 20, 2014 at 9:37 AM, Valentin V. Bartenev
<vbart@nginx.com>wrote:

> Deleting a file doesn't make file descriptor "invalid".  It's valid and the
> file actually exists on file system till there is at least one descriptor
> pointing to that file.
>

​Thanks for that Valentin, I learned something today.

I read
http://stackoverflow.com/questions/2028874/what-ha...
As I understand it, all log messages going to a moved file​

​will still be printed into it.
So if I move /var/log/nginx/access.log to /foo/bar.log, all error log
message will continue to be printed in /foo/bar.log, until I send the
USR1
signal to nginx (either master or workers), which will then close the
file
descriptor (thus effectively delete /var/log/nginx/access.log which was
marked for deletion until then)​ and will create a new file in
/var/log/nginx/access.log
Am I right?
I suspect the file does not really move out of /var/log/nginx if at
least
one fd is open on it. I suspect then that there is a symlink created in
/foo/bar.log pointing to the old file until it is effectively moved.

Does the same happens if the file is moved on a remote disk (ie does the
fact that a file is moved through copy+delete rather than rename have
any
impact?)
​
---
*B. R.*
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2014-05-20 13:33
(Received via mailing list)
Hello!

On Mon, May 19, 2014 at 03:06:06PM -0400, samingrassia wrote:

> Thanks to everyone in advance!
>
> I have a cron that runs the following:
>
> mv $NGINX_ACCESS_LOG $ACCESS_LOG_DROPBOX/$LOG_FILENAME
> kill -USR1 `cat $NGINX_PID`
>
> My questions is during time between the mv and the kill, is there any log
> writes that are being discarded or are they being stacked in memory and
> dumped into the new access.log after it is recreated?

Unless you are trying to move logs to a different filesystem,
logging will continue to the old file till USR1 is processed.

From nginx point of view, the "mv" command does nothing - as nginx
has open file descriptor, it will continue to write to it, and log
lines will appear in the (old) file - the file which is now have a
new name.  After USR1 nginx will reopen the log, and will continue
further logging to a new file.

--
Maxim Dounin
http://nginx.org/
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.