# Status of Nginx centralised logging **URL:** \ **Category:** NGINX\ **Created:** [June 7, 2012, 12:39am UTC](https://www.ruby-forum.com/t/status-of-nginx-centralised-logging/220362 "2012-06-07T00:39:56Z")\ **Posts on this page:** 6\ **Page:** 1
**Author:** ![aris](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/258eb7/32.png) [@aris](https://www.ruby-forum.com/u/aris)\ **Post date:** [June 7, 2012, 12:39am UTC](https://www.ruby-forum.com/t/status-of-nginx-centralised-logging/220362/1 "2012-06-07T00:39:56Z")
Hi all - As many of us will have discovered over the course of our careers, centralised/non-local logging is an important part of any non-trivial infrastructure. I’m aware of different patches that add syslogging capabilities to different Nginx versions, but I’ve yet to see an official description of how we should achieve non-local logging. Preferably syslog, personally speaking, but anything scalable, supportable, debug-able and sane would, I feel, be acceptable to the wider community. I’m aware of at least the following options, but I feel they’re all lacking to some degree: - log to local disk and syslog/logstash/rsync them off: undesirable due to the management overhead of the additional logging process/logic - the wasted disk I/O when creating the per-request logs - use post\_action to hit a logging endpoint after each request: would add overhead to both configuration and network traffic; post\_action has been described as “a bit of a hack”, IIRC - use syslog patches: not “official”, hence troublesome to debug on-list; rightly or wrongly, the usual response of “please replicate problem without 3rd party patches” would cause problems when debugging production systems - use another 3rd-party logging protocol, e.g. statsd, redis: as similarly unsupportable as syslog patches I wonder if someone could officially comment on the potential for resolving this gap in Nginx’s feature set? Or perhaps I’m missing something … 🙂 ## Many thanks, all Jonathan Jonathan M. Oxford, London, UK [http://www.jpluscplusm.com/contact.html](http://www.jpluscplusm.com/contact.html) ---
**Author:** ![jonathan2000](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/fbc32d/32.png) [@jonathan2000](https://www.ruby-forum.com/u/jonathan2000)\ **Post date:** [June 7, 2012, 1:03am UTC](https://www.ruby-forum.com/t/status-of-nginx-centralised-logging/220362/2 "2012-06-07T01:03:57Z")
On Wed, Jun 6, 2012 at 3:39 PM, Jonathan M. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > and sane would, I feel, be acceptable to the wider community. > has been described as “a bit of a hack”, IIRC > resolving this gap in Nginx’s feature set? Or perhaps I’m missing > something … 🙂 Hi! I think this thread from a few days ago might answer some of your questions : [http://forum.nginx.org/read.php?2,227234](http://forum.nginx.org/read.php?2,227234) We use log\_by\_lua to aggregate data, and then fetch the aggregated data from another location. In turns this feeds into a bigger centralized system (OpenTSDB). This way we don’t have to output one log line, or hit one logging endpoint for each request that goes through. One interesting thing we’ve noticed while developing this is that even an error in our logging code really doesn’t affect serving pages, so this might be the safest use case for 3rd party modules you could imagine. Currently your only other official alternative to get data, besides the access\_log is the stub\_status module : > **[GitHub - nginxinc/nginx-wiki: ARCHIVED -- Source for the now archived NGINX Wiki...](https://github.com/nginxinc/nginx-wiki)** > > ARCHIVED -- Source for the now archived NGINX Wiki section of https://www.nginx.com Hope this helps, Matthieu ---
**Author:** ![jonathan2000](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/fbc32d/32.png) [@jonathan2000](https://www.ruby-forum.com/u/jonathan2000)\ **Post date:** [June 7, 2012, 1:40am UTC](https://www.ruby-forum.com/t/status-of-nginx-centralised-logging/220362/3 "2012-06-07T01:40:52Z")
On 7 June 2012 00:03, Matthieu T. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > I think this thread from a few days ago might answer some of your questions : > [http://forum.nginx.org/read.php?2,227234](http://forum.nginx.org/read.php?2,227234) Yes, it was glancing over this thread that made me think I needed to raise this issue kinda sorta formally, and ask for some official recognition of this relatively important issue. > We use log\_by\_lua to aggregate data, and then fetch the aggregated > data from another location. > In turns this feeds into a bigger centralized system (OpenTSDB). > This way we don’t have to output one log line, or hit one logging > endpoint for each request that goes through. > > One interesting thing we’ve noticed while developing this is that even > an error in our logging code really doesn’t affect serving pages, so > this might be the safest use case for 3rd party modules you could > imagine. Interesting, thank you. [ I’ve so far avoided enabling lua in my production Nginx instances as it’s almost /too/ powerful. If our systems team (or devs!) get the idea we can do /anything/ we like in the reverse proxy layer in front of our app servers, I’m concerned we’ll start to add too much logic in there that should actually live in the apps 🙂 ] I’ll definitely take a look at log\_by\_lua, however I still think we should get the centralised logging issue recognised officially as a area where Nginx is deficient, along with suggestions for bringing it up to spec. Nginx is a vital part of many people’s infrastructures now: it’s a shame it’s behind the (devops’y!) times in this area. > Currently your only other official alternative to get data, besides > the access\_log is the stub\_status module : > [GitHub - nginxinc/nginx-wiki: ARCHIVED -- Source for the now archived NGINX Wiki section of https://www.nginx.com · GitHub](http://wiki.nginx.org/HttpStubStatusModule) Appreciated but, not being per-request, it isn’t really in the same space as the other things we’re discussing. ## Cheers, Jonathan Jonathan M. Oxford, London, UK > **[Jonathan Matthews :: Words Typed, Great Rates For Cash](https://www.jpluscplusm.com)** ---
**Author:** ![jonathan2000](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/fbc32d/32.png) [@jonathan2000](https://www.ruby-forum.com/u/jonathan2000)\ **Post date:** [June 7, 2012, 12:10pm UTC](https://www.ruby-forum.com/t/status-of-nginx-centralised-logging/220362/4 "2012-06-07T12:10:51Z")
Hello! On Wed, Jun 06, 2012 at 11:39:30PM +0100, Jonathan M. wrote: > and sane would, I feel, be acceptable to the wider community. > > I’m aware of at least the following options, but I feel they’re all > lacking to some degree: > > - log to local disk and syslog/logstash/rsync them off: undesirable > due to the management overhead of the additional logging process/logic > > - the wasted disk I/O when creating the per-request logs This is, actually, recommended solution. It’s main advantage is that nginx will continue to serve requests even if something really bad will happen with your “non-trivial infrastructure”. Basically it will work till the server is alive. […] > - use another 3rd-party logging protocol, e.g. statsd, redis: as > similarly unsupportable as syslog patches You may also take a look at Valery K.'s udplog module, [http://grid.net.ru/nginx/udplog.en.html](http://grid.net.ru/nginx/udplog.en.html) I haven’t actually used it (as I prefer local logging solution, see above), but I tend to like the basic concept behind the module. At least it resolves the main problem with usual syslog interfaces (i.e. blocking). Maxim D. ---
**Author:** ![jonathan2000](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/fbc32d/32.png) [@jonathan2000](https://www.ruby-forum.com/u/jonathan2000)\ **Post date:** [June 7, 2012, 2:02am UTC](https://www.ruby-forum.com/t/status-of-nginx-centralised-logging/220362/5 "2012-06-07T02:02:30Z")
\> \* use another 3rd-party logging protocol, e.g. statsd, redis: as \> similarly unsupportable as syslog patches There’s also the nginx sflow module, [http://code.google.com/p/nginx-sflow-module/](http://code.google.com/p/nginx-sflow-module/) I realize that this is another 3rd party module, so falls into the same ‘not officially supported’ category, but I figured I’d mention it as something to add to the centralized logging bag of tricks. I’ve used it and it’s pretty nice, though obviously you need an sflow server somewhere to send to. ---
**Author:** ![jonathan2000](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/j/fbc32d/32.png) [@jonathan2000](https://www.ruby-forum.com/u/jonathan2000)\ **Post date:** [June 7, 2012, 7:33pm UTC](https://www.ruby-forum.com/t/status-of-nginx-centralised-logging/220362/6 "2012-06-07T19:33:12Z")
I just wanted to add a couple of comments on using sFlow as the logging protocol - there are also sFlow plugins for Apache and Tomcat, allowing you to get the same data from all the HTTP server pools. On the analyzer side, the HTTP sFlow export also includes performance counters that can be trended with Graphite or Ganglia. You can also use sflowtool to convert the binary sFlow datagrams to ASCII combined logfile format that can be fed into any log analyzer. > **[NGINX](https://blog.sflow.com/2011/04/nginx.html)** > > The nginx-sflow-module project is an open source implementation of sFlow monitoring for the Nginx ( pronounced engine x) web server. T... Mark M. wrote in post #1063450: > \> \* use another 3rd-party logging protocol, e.g. statsd, redis: as \> similarly unsupportable as syslog patches > > There’s also the nginx sflow module, > [Google Code Archive - Long-term storage for Google Code Project Hosting.](http://code.google.com/p/nginx-sflow-module/) > > I realize that this is another 3rd party module, so falls into the > same ‘not officially supported’ category, but I figured I’d mention it > as something to add to the centralized logging bag of tricks. I’ve > used it and it’s pretty nice, though obviously you need an sflow > server somewhere to send to.