I've got a problem with Nginx running on Ubuntu 12.10. I'm running it mainly as a reverse proxy and there is no high load on the machine. It randomly crashes without any helpful log info (or at least I think so). Here's the error log *************************************************************************************************** 2013/01/31 09:19:03 [debug] 15238#0: *10555 event timer del: 68: 1359620363778 2013/01/31 09:19:03 [debug] 15238#0: *10555 generic phase: 0 2013/01/31 09:19:03 [debug] 15238#0: *10555 rewrite phase: 1 2013/01/31 09:19:03 [debug] 15238#0: *10555 http script regex: "^(.*)" 2013/01/31 09:19:03 [notice] 15238#0: *10555 "^(.*)" matches "/", client: 192.168.2.42, server: abc.def.com, request: "HEAD / HTTP/1.1", host: " abc.def.com" 2013/01/31 09:19:03 [debug] 15238#0: *10555 http script copy: "https://abc.def.com" 2013/01/31 09:19:03 [debug] 15238#0: *10555 http script regex end 2013/01/31 09:19:03 [notice] 15238#0: *10555 rewritten redirect: "https://abc.def.com", client: 192.168.2.42, server: abc.def.com, reques t: "HEAD / HTTP/1.1", host: "abc.def.com" 2013/01/31 09:19:03 [debug] 15238#0: *10555 http finalize request: 301, "/?" a:1, c:1 2013/01/31 09:19:03 [debug] 15238#0: *10555 http special response: 301, "/?" 2013/01/31 09:19:03 [debug] 15238#0: *10555 http set discard body 2013/01/31 09:19:03 [debug] 15238#0: *10555 xslt filter header 2013/01/31 09:19:03 [debug] 15238#0: *10555 HTTP/1.1 301 Moved Permanently Server: nginx/1.2.6 Date: Thu, 31 Jan 2013 08:19:03 GMT Content-Type: text/html Content-Length: 184 Connection: keep-alive Location: https://abc.def.com 2013/01/31 09:19:03 [debug] 15238#0: *10555 write new buf t:1 f:0 00000000027A0868, pos 00000000027A0868, size: 200 file: 0, size: 0 2013/01/31 09:19:03 [debug] 15238#0: *10555 http write filter: l:1 f:0 s:200 2013/01/31 09:19:03 [debug] 15238#0: *10555 http write filter limit 0 2013/01/31 09:19:03 [debug] 15238#0: *10555 writev: 200 2013/01/31 09:19:03 [debug] 15238#0: *10555 http write filter 0000000000000000 2013/01/31 09:19:03 [debug] 15238#0: *10555 http finalize request: 0, "/?" a:1, c:1 2013/01/31 09:19:03 [debug] 15238#0: *10555 set http keepalive handler 2013/01/31 09:19:03 [debug] 15238#0: *10555 http close request 2013/01/31 09:19:03 [debug] 15238#0: *10555 http log handler 2013/01/31 09:19:03 [debug] 15238#0: *10555 free: 000000000279FBA0, unused: 360 2013/01/31 09:19:03 [debug] 15238#0: *10555 event timer add: 68: 30000:1359620373779 2013/01/31 09:19:03 [debug] 15238#0: *10555 free: 00000000027DE460 2013/01/31 09:19:03 [debug] 15238#0: *10555 free: 0000000002652060 2013/01/31 09:19:03 [debug] 15238#0: *10555 hc free: 0000000000000000 0 2013/01/31 09:19:03 [debug] 15238#0: *10555 hc busy: 0000000000000000 0 2013/01/31 09:19:03 [debug] 15238#0: *10555 tcp_nodelay 2013/01/31 09:19:03 [debug] 15238#0: *10555 reusable connection: 1 2013/01/31 09:19:03 [debug] 15238#0: *10555 post event 00000000025B5210 2013/01/31 09:19:03 [debug] 15238#0: *10555 delete posted event 00000000025B5210 2013/01/31 09:19:03 [debug] 15238#0: *10555 http keepalive handler 2013/01/31 09:19:03 [debug] 15238#0: *10555 malloc: 0000000002652060:8192 2013/01/31 09:19:03 [debug] 15238#0: *10555 recv: fd:68 -1 of 8192 2013/01/31 09:19:03 [debug] 15238#0: *10555 recv() not ready (11: Resource temporarily unavailable) 2013/01/31 09:19:03 [debug] 15238#0: *10555 free: 0000000002652060 2013/01/31 09:19:03 [debug] 15238#0: *10555 http keepalive handler 2013/01/31 09:19:03 [debug] 15238#0: *10555 malloc: 0000000002652060:8192 2013/01/31 09:19:03 [debug] 15238#0: *10555 recv: fd:68 0 of 8192 2013/01/31 09:19:03 [info] 15238#0: *10555 client 192.168.2.42 closed keepalive connection 2013/01/31 09:19:03 [debug] 15238#0: *10555 close http connection: 68 2013/01/31 09:19:03 [debug] 15238#0: *10555 event timer del: 68: 1359620373779 2013/01/31 09:19:03 [debug] 15238#0: *10555 reusable connection: 0 2013/01/31 09:19:03 [debug] 15238#0: *10555 free: 0000000002652060 2013/01/31 09:19:03 [debug] 15238#0: *10555 free: 0000000000000000 2013/01/31 09:19:03 [debug] 15238#0: *10555 free: 000000000283EBF0, unused: 8 2013/01/31 09:19:03 [debug] 15238#0: *10555 free: 00000000027F9CF0, unused: 128 2013/01/31 09:24:49 [alert] 15237#0: worker process 15238 exited on signal 9 ************************************************************************************************************************* At 09:24 I restarted the service because it was hanging. This is some system info about tcp connections: Thu Jan 31 09:18:01 CET 2013 14 ESTABLISHED 3 FIN_WAIT2 6 LISTEN 21 TIME_WAIT Thu Jan 31 09:19:01 CET 2013 28 ESTABLISHED 6 LISTEN 12 TIME_WAIT Thu Jan 31 09:20:01 CET 2013 13 CLOSE_WAIT 28 ESTABLISHED 6 LISTEN Thu Jan 31 09:21:01 CET 2013 35 CLOSE_WAIT 36 ESTABLISHED 8 LISTEN Thu Jan 31 09:22:01 CET 2013 46 CLOSE_WAIT 35 ESTABLISHED 7 LISTEN Thu Jan 31 09:23:01 CET 2013 80 CLOSE_WAIT 33 ESTABLISHED 7 LISTEN Thu Jan 31 09:24:01 CET 2013 128 CLOSE_WAIT 26 ESTABLISHED 7 LISTEN 8 SYN_RECV Thu Jan 31 09:25:02 CET 2013 26 ESTABLISHED 1 FIN_WAIT1 7 LISTEN 4 TIME_WAIT During that time I tried to connect with curl and this is the output: roberto@t500:~> curl -I -L -k -m 30 http://abc.def.com curl: (28) Operation timed out after 30001 milliseconds with 0 bytes received I upgraded Ubuntu package from 1.2.1 to 1.2.6 but no result. In /var/log/syslog I have no info about this. What can I do? Posted at Nginx Forum: http://forum.nginx.org/read.php?2,235763,235763#msg-235763
on 2013-01-31 09:49
on 2013-01-31 12:28
Hello! On Thu, Jan 31, 2013 at 03:48:31AM -0500, rg00 wrote: > I've got a problem with Nginx running on Ubuntu 12.10. > I'm running it mainly as a reverse proxy and there is no high load on the > machine. > It randomly crashes without any helpful log info (or at least I think so). (Note: this looks like a hang, not a crash. There is a difference.) [...] > > At 09:24 I restarted the service because it was hanging. > This is some system info about tcp connections: What nginx -V shows? As a first step I would recommend trying to reproduce the problem without any 3rd party modules/patches compiled in. You may also find some helpful tips about debugging here: http://wiki.nginx.org/Debugging -- Maxim Dounin http://nginx.com/support.html
on 2013-01-31 12:38
This is the output of nginx -V: nginx version: nginx/1.2.6 TLS SNI support enabled configure arguments: --prefix=/usr/share/nginx --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-client-body-temp-path=/var/lib/nginx/body --http-fastcgi-temp-path=/var/lib/nginx/fastcgi --http-log-path=/var/log/nginx/access.log --http-proxy-temp-path=/var/lib/nginx/proxy --http-scgi-temp-path=/var/lib/nginx/scgi --http-uwsgi-temp-path=/var/lib/nginx/uwsgi --lock-path=/var/lock/nginx.lock --pid-path=/run/nginx.pid --with-pcre-jit --with-debug --with-http_addition_module --with-http_dav_module --with-http_geoip_module --with-http_gzip_static_module --with-http_image_filter_module --with-http_realip_module --with-http_stub_status_module --with-http_ssl_module --with-http_sub_module --with-http_xslt_module --with-ipv6 --with-sha1=/usr/include/openssl --with-md5=/usr/include/openssl --with-mail --with-mail_ssl_module --add-module=/build/buildd/nginx-1.2.6/debian/modules/nginx-auth-pam --add-module=/build/buildd/nginx-1.2.6/debian/modules/nginx-echo --add-module=/build/buildd/nginx-1.2.6/debian/modules/nginx-upstream-fair --add-module=/build/buildd/nginx-1.2.6/debian/modules/nginx-dav-ext-module Posted at Nginx Forum: http://forum.nginx.org/read.php?2,235763,235771#msg-235771
on 2013-01-31 13:18
Hello! On Thu, Jan 31, 2013 at 06:38:27AM -0500, rg00 wrote: > --http-scgi-temp-path=/var/lib/nginx/scgi > --add-module=/build/buildd/nginx-1.2.6/debian/modules/nginx-upstream-fair > --add-module=/build/buildd/nginx-1.2.6/debian/modules/nginx-dav-ext-module Well, basic suggestion is the same: try to reproduce the problem without any 3rd party modules/patches compiled in. If you use geoip module, please also make sure it's database isn't corrupted (or just try without it), as MaxMind's geoip library is known to do bad things if database is corrupted. -- Maxim Dounin http://nginx.com/support.html
on 2013-02-01 10:45
I compiled 1.2.6 version and switched to it, here's my new nginx -V: nginx version: nginx/1.2.6 built by gcc 4.7.2 (Ubuntu/Linaro 4.7.2-2ubuntu1) TLS SNI support enabled configure arguments: --prefix=/opt/nginx-1.2.6 --with-http_flv_module --with-http_ssl_module --with-http_gzip_static_module --add-module=/root/src/ngx_http_auth_pam_module-1.2 --with-debug but *same* problem....................................... I'm really really confused.................................. Posted at Nginx Forum: http://forum.nginx.org/read.php?2,235763,235798#msg-235798
on 2013-02-01 11:50
On Friday 01 February 2013 13:45:05 rg00 wrote: > but *same* problem....................................... I'm really really > confused.................................. > It is very likely that the cause of your problem is the http_auth_pam 3rd-party module. This module is known as being able to block worker processes when it used. wbr, Valentin V. Bartenev -- http://nginx.com/support.html http://nginx.org/en/donation.html
on 2013-02-01 12:07
I really need ldap authentication, is there an alternative to that module? Posted at Nginx Forum: http://forum.nginx.org/read.php?2,235763,235800#msg-235800
on 2013-02-01 12:12
On 02/01/2013 12:07 PM, rg00 wrote: > I really need ldap authentication, is there an alternative to that module? A quick google gave: https://github.com/kvspb/nginx-auth-ldap http://forum.nginx.org/read.php?2,18552 Regards, Patrick
on 2013-02-01 12:31
On Friday 01 February 2013 15:11:58 Patrick Lists wrote: > On 02/01/2013 12:07 PM, rg00 wrote: > > I really need ldap authentication, is there an alternative to that > > module? > > A quick google gave: > > https://github.com/kvspb/nginx-auth-ldap > http://forum.nginx.org/read.php?2,18552 > Unfortunately, this module also uses blocking I/O operations. So, you should be ready for poor performance and arbitrary hangs. wbr, Valentin V. Bartenev -- http://nginx.com/support.html http://nginx.org/en/donation.html
on 2013-02-01 12:35
Hello! On Fri, Feb 01, 2013 at 12:11:58PM +0100, Patrick Lists wrote: > On 02/01/2013 12:07 PM, rg00 wrote: > >I really need ldap authentication, is there an alternative to that module? > > A quick google gave: > > https://github.com/kvspb/nginx-auth-ldap This one blocks as well. There is no non-blocking API in PAM, hence correct nginx module to use PAM for authentication is something impossible to write. And LDAP client libraries out there are blocking too, so writing LDAP authentication module isn't something simple. > http://forum.nginx.org/read.php?2,18552 Yep, using X-Accel-Redirect is a good way to handle any needed authentication/authorization. Alternatively, one may use auth request module: http://mdounin.ru/hg/ngx_http_auth_request_module I wrote it once tired reading questions about PAM/LDAP/whatever authentication modules and various blocking solutions people write and try to use. Compared to X-Accel-Redirect it is simplier for general use. -- Maxim Dounin http://nginx.com/support.html
on 2013-02-01 15:10
Compiled nginx without auth_pam and using it as a reverse proxy. Now I'm monitoring and hoping there are no more service crashes. I'm using Apache to manage ldap authentication on certain locations. Posted at Nginx Forum: http://forum.nginx.org/read.php?2,235763,235809#msg-235809
Enable email notification
Enable multi-page viewPlease log in before posting. Registration is free and takes only a minute.
Existing account
(Switch to SSL-encrypted connection)
NEW: Do you have a Google/GoogleMail or Yahoo account? No registration required!
Log in with Google account | Log in with Yahoo account
Log in with Google account | Log in with Yahoo account
No account? Register here.