Problem with split log

Server NGINX
1

how to split nginx log with cronolog , other method?

just as:
TransferLog “|/www/sbin/cronolog
/www/logs/%Y/%m/%d/access.log”
ErrorLog “|/www/sbin/cronolog
/www/logs/%Y/%m/%d/errors.log”

2

Yo,

We do something like this at Engine Y…

Daily nginx logrotation during non-peak hours:

Daily log migration to a drive with more space:

3

Meh - i’ve had logrotate screw up on my too many times and delete a
log file that things are logging into.

4

Try something like this

#!/bin/bash

YEAR=date "+%Y"
MONTH=date "+%m"
DAY=date "+%d"

HOSTNAME=hostname -s

LOG_FILES=“access.log error.log images.log redirect.log ssl.log
click.log uploads.log”

DATE=$YEAR/$MONTH/$DAY

REDBUBBLE_LOG_ROOT=/var/log/redbubble
NGINX_LOG_ROOT=$REDBUBBLE_LOG_ROOT/nginx

make path

mkdir -p $NGINX_LOG_ROOT/$DATE

touch and symlink in new log files

for FILE in $LOG_FILES; do
LOG_FILE=$NGINX_LOG_ROOT/$DATE/$HOSTNAME.$FILE
touch $LOG_FILE
ln -fs $LOG_FILE $NGINX_LOG_ROOT/$FILE
done

tell nginx to re-open its log files

kill -USR1 cat /var/run/nginx.pid

5

We use that logrotate config on like 1500+ linux instances. It
works… really. I too have been burned by various things wrt log
rotation, but nginx shows nothing but love. :wink:

6

Hi there,

Unfortunately nginx doesn’t support piping log method inside it’s config
script, so you have to set it separately.
try this:

~ point your access/error log to a file, such as /logs/access.log/ and
/logs/error.log/
~ if /logs/access.log/ and /logs/error.log/ has been exist, delete them,
and recreate as fifo’s
~ /mkfifo logs/access.log logs/error.log/
~ make a .sh files to put a cronolog script and piping, like:

#!/bin/bash

|(cat /usr/local/nginx/logs/access.log |/usr/local/sbin/cronolog -l /var/log/nginx/access.log
/var/log/nginx/%Y/%m/%d/%H/access.log) &
(cat /usr/local/nginx/logs/error.log |/usr/local/sbin/cronolog -l
/var/log/nginx/error.log
/var/log/nginx/%Y/%m/%d/%H/error.log) &|

~ start/execute thus file whenever you start your nginx

Source and references:
http://pjkh.com/articles/nginx-chronolog.html

rgrds,
Andika
INDONESIAN

7

Is there any tool to provide anti-multi-thread-spider function based on
logs?
Some mad spiders drive my host to heavy load. :frowning:

2008/3/13, dika [email protected]:

8

On Thu, Mar 13, 2008 at 02:03:47PM +0700, dika wrote:

Source and references:
http://pjkh.com/articles/nginx-chronolog.html

This is total overkill for such trivial operation as hourly log
rotation:

nginx writes to fifo
cat reads from fifo
cat writes to pipe
cronolog read from pipe

3 useless process context switches and 3 useless copy operations for
each request or buffered bunch of requests.

The cronolog is usefull with Apache because Apache can not rotate logs
immidiately. But nginx rotates logs in a moment.