Since I need my nginx to be integrated with syslog, I looked at the
unofficial
patches (udpremote and syslog patch).
I saw I can have syslog logging with these, but I can not set a custom
program
identifier per vhost, so that I can easily distinguish in the syslog
daemon and
make it save the right log in the right file, making syslog manage
location(remote/local) and rotation of the logs.
Since the patches do not permit that, I think I’m going to write yet
another
syslog module/patch.
So I have a couple of questions:
why weren’t the patches/modules merged into the main ngix project?
Any
criticism on those patches so that I don’t make the same mistakes?
Do you think it’s better to support syslog in the
access_log/error_log
directives, just like the patch, or to have a different error_syslog /
access_syslog directives?
Do you have any preference in the format of the directive? (ex:
“access_syslog host:local6:notice” instead of “syslog_access host local6
notice”
etc…)
wbr, Valentin V. Bartenev
I saw that patch, and I understand that syslog(3) does block, but since
I need
to set one program identifier per vhost, I wouldn’t be using syslog(3),
but
rather reimplementing it (already done, taken a lot from glib, bsd
license).
Using udp connections would make the call non-blocking, and should be
safe
enough on localhost.
Then again, other solutions can be found, such as having a dedicated
thread send
the message to syslog, so that logging calls are non-blocking.
Or I could write on a fifo buffer/file the messages and have the
dedicated
thread send the messages to syslog (date is set on the sent message, not
at the
time the message arrives to syslog, so no problem with that).
Hello!
The main problem with syslog in general is that it’s interface
blocks, and if syslogd daemon is dead/can’t cope with load nginx
won’t be able to work. Additionally, it’s slow/resource
consuming. Writing normal files is much faster and more reliable.
Given the above, it’s highly unlikely that syslog patches of any
quality will be accepted.
Maxim D.
Thank you for your reply, but as I said, I already have reimplemented
syslog(3),
and with the aid of a udp or fifo with a thread things can be made
completely
asynchronous.
Or I could use an aio_write() instead of a write() to write to the local
syslog
socket, although a fifo is more secure for syslog crashes/hangups (which
I’ve
never seen, anyway).
Anyway, let’s assume for a second that I can get file-like performance.
The questions that remains are 2:
1- What format do you like more? I’m inclined towards something like
“syslog_access ident:facility:priority:host:port:connectiontype”
2- How should I benchmark it all?
The performance of syslog depends heavily on what implementation you are
using. For example, rsyslog has been measured at 1M logs/sec under ideal
conditions (and I’ve used it at over 100K logs/sec). rsyslog is now the
default syslog on just about every linux distro. This isn’t like the old
sysklogd that was could only process one message at a time and blocked
everything until it finished processing the message (limiting it to a
few hundred logs/sec, even on really good disks)
I can understand why you would not want to make logging to syslog be the
default, but why are you so opposed to having it as an option?
yes, if the syslog daemon dies, things writing to it will block, but if
your partition that you are writing logfiles to fills up you will block
as well. Deciding how to log, and if blocking on another daemon is
acceptable should be up to the system administrator. Sometimes getting a
copy of the logs elsewhere in near real-time is worth reducing the peak
requests/sec that a box can handle
The other approach you could allow someone to submit would be to sent
the logs out in real-time via UDP, this will mean that logs are lost if
the syslog daemon goes down, but nginx won’t block
On Thu, Apr 26, 2012 at 03:14:06PM +0200, Luke wrote:
Since the patches do not permit that, I think I’m going to write yet another
syslog module/patch.
So I have a couple of questions:
why weren’t the patches/modules merged into the main ngix project? Any
criticism on those patches so that I don’t make the same mistakes?
The main problem with syslog in general is that it’s interface
blocks, and if syslogd daemon is dead/can’t cope with load nginx
won’t be able to work. Additionally, it’s slow/resource
consuming. Writing normal files is much faster and more reliable.
Given the above, it’s highly unlikely that syslog patches of any
quality will be accepted.
Maybe you can try the patch from Tengine
(ngx_http_log_module - The Tengine Web Server). We implement the
syslog protocol which indeed use the UDP socket. It will not block
Nginx any more.
Just thought i would add that support for error logging over syslog
would
be great. Udp would be even better as we could send it straight out to
the
remote storage and skip the local disk (io). If you need a test for your
module once you write it just let me know.
Although i plan on taking a look at tengines solution since im already
using some of his patches.
This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.