addis_a
September 30, 2014, 3:55pm
1
Hi,
Im trying to collect access log by passing them with tcp to flume.
http://www.binpress.com/issue/possible-race-condition-while-nginx-is-running-on-more-workerprocesses/6955
opened 01:31PM - 30 Sep 14 UTC
closed 07:21PM - 09 Feb 15 UTC
I have setup a simple configuration which just send the logs to localhost on por… t 4789:
```
worker_processes 1;
error_log logs/debug.log debug;
events {
worker_connections 1024;
}
http {
lua_package_path "/path/to/lua-resty-logger-socket/lib/?.lua;;";
include mime.types;
default_type application/octet-stream;
sendfile on;
keepalive_timeout 65;
server {
listen 80;
server_name localhost;
location / {
log_by_lua '
local logger = require "resty.logger.socket"
if not logger.initted() then
local ok, err = logger.init{
host = "127.0.0.1",
port = 4789,
flush_limit = 0, -- disable buffer, else flume cannot seperate the events
drop_limit = 8388608,
}
if not ok then
ngx.log(ngx.ERR, "failed to initialize the logger: ",
err)
return
end
end
-- construct the custom access log message in
-- the Lua variable "msg"
local bytes, err = logger.log("t")
if err then
ngx.log(ngx.ERR, "failed to log message: ", err)
return
end
';
root html;
index index.html index.htm;
}
}
}
```
A netcat server will pipe all logs to a file:
```
nc -l -q -1 -k -p 4789 > file.log
```
Then a kind of 'stress' test to assemble the logs:
```
for i in `seq 7000`; do sleep 0.1; curl 'http://127.0.0.1/?test=test' -H 'User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:32.0) Gecko/20100101 Firefox/32.0' -H 'Accept: text/javascript, application/javascript, application/ecmascript, application/x-ecmascript, */*; q=0.01' -H 'Accept-Language: en-US,en;q=0.5' -H 'Accept-Encoding: gzip, deflate' -H 'X-Requested-With: XMLHttpRequest' -H 'Referer: https://www.openindex.io/dev/' -H 'Connection: keep-alive' ; done
```
When worker_processes is set to one, after a while there are 7000 logs in my file and everything is fine.
But when I change the worker_processes to 2, it will stop collection on a random time. Could it be that there is a race condition? Or is this script simply not compatible with nginx 1.6.2?
Thanks in advice,
Ron
Anyone else have seen this problem, or maybe knows what is causing it?
Thanks in advice,
Ron
Nevermind, I found the problem. It seems that netcat is not reliable
On di, sep 30, 2014 at 3:54 , Ron van der Vegt