# Nginx Log Parser **URL:** \ **Category:** NGINX\ **Created:** [October 8, 2010, 12:53pm UTC](https://www.ruby-forum.com/t/nginx-log-parser/196203 "2010-10-08T12:53:43Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![valor](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/v/ee59a6/32.png) [@valor](https://www.ruby-forum.com/u/valor)\ **Post date:** [October 8, 2010, 12:53pm UTC](https://www.ruby-forum.com/t/nginx-log-parser/196203/1 "2010-10-08T12:53:43Z")
Hello Everyone, First pardon me if this have already been discussed in the russian mailinglist. Second, thank you igor for this wonderful software, it’s amazing speed and stability. After googling and searching around I’ve not found a single page with information related to take advantage of nginx cache logging and tools to parse this data. So decided to create this topic to first check what’s everyone else using and then if there’s nothing ready for production, do it myself and share it here :-). 1. Common basic web stats By default nginx combined format is identical to apache, so out of the box anyone can generate graphs of nginx traffic by using any software that supports apache: Awstats, Webalizer, etc. This is not a problem in nginx, setup of these softwares for nginx have not a single difference with setup for apache logs. 1. Web stats of cache usage It’s well documented on the wiki ( [Module ngx\_http\_upstream\_module](http://wiki.nginx.org/NginxHttpUpstreamModule) ) how to add to the log useful information regarding caching: upstream\_cache\_status (hit,miss,expired,stale,updating,etc) upstream\_status (status returned by apache) upstream\_response\_time (time taken by backend to process the request) However, I’ve been completely unable to find a single reference to anyone having this information processed nicely by log analyzers allowing administrators to see nginx cache stats. What are other people using for this? I could not find any hack around to show this info in awstats, webalizer or visitors(hping). The only thing i found is a script made by a guy named Eliot from saltycrane using python to export log files into mongodb and then use orbited+stomp+jquery to plot that data into graphs. (You can see a presentation about it here: [Live Log Analyzer](http://www.saltycrane.com/site_media/html/lla.html) ) 1. Live status of nginx for sever monitoring/alert software like Nagios/Zabbix/Cacti/etc. Another thing I’m looking for it’s a way to get live data from nginx to pipe it into my monitoring software (zabbix). I’m right now using 0.7.67 so I can’t use the new status module, however AFAIK the status module do not provide information per virtualhost but nginx-wide stats so even by parsing that status I won’t be able to find the culprit domain for a traffic peak until I parse that domain log offline with another software. One way to do this would be to parse log files each minute parsing only last 60 seconds of activity, however I’m afraid this may result in the solution being worst than the problem as it will generate load in the server each minute. Have anyone done something like this already for nginx cache logs? Well… Hope this would be a very long and productive topic 🙂 Best Regards from Uruguay Guzmán Brasó Posted at Nginx Forum: > **[Nginx Log Parser](https://forum.nginx.org/read.php?2,138438,138438#msg-138438)** > > Hello Everyone, First pardon me if this have already been discussed in the russian mailinglist. Second, thank you igor for this wonderful software, it's amazing speed and stability. After googling and searching around I've not found a single page... ---
**Author:** ![valor](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/v/ee59a6/32.png) [@valor](https://www.ruby-forum.com/u/valor)\ **Post date:** [June 14, 2012, 10:59pm UTC](https://www.ruby-forum.com/t/nginx-log-parser/196203/2 "2012-06-14T22:59:09Z")
I implemented cache stats in Zabbix by parsing the access logs using logtail with an offset file every minute, and submitting value counts for $upstream\_cache\_status to the zabbix server trapper via zabbix\_sender. our rps is 50 or less, so processing this log every minute isn’t really a big deal yet. Posted at Nginx Forum: > **[Re: Nginx Log Parser](https://forum.nginx.org/read.php?2,138438,227541#msg-227541)** > > Hello Everyone, First pardon me if this have already been discussed in the russian mailinglist. Second, thank you igor for this wonderful software, it's amazing speed and stability. After googling and searching around I've not found a single page...