Forum: NGINX socket CLOSE_WAIT problem and 404.html

Hi All,

I have a website that hits by automatic scheduled job every hour from
thousands sources.

last time, my website is down because of thousands CLOSE_WAIT when I 
check
using "netstat -na"
I already try to debug refer to http://www.ruby-forum.com/topic/204589
but I don't know what to do with the debug log.

when I check the "lsof -p <nginx>":
i found a lots of sockets (CLOSE_WAIT) and file open of my 404.html.
i think there's a problem with my 404 configuration.
-----
nginx   10500 nobody  357r   REG        8,3        141  52396049
/home/j/html/404.html
nginx   10500 nobody  359r   REG        8,3        141  52396049
/home/j/html/404.html
nginx   10500 nobody  360r   REG        8,3        141  52396049
/home/j/html/404.html
nginx   10500 nobody  362u  IPv4  174783995                  TCP
70.36.100.46:http->y.net:49333 (CLOSE_WAIT)
nginx   10500 nobody  363r   REG        8,3        141  52396049
/home/j/html/404.html
nginx   10500 nobody  365u  IPv4  174497664                  TCP
70.36.100.46:http->y.net:52096 (CLOSE_WAIT)
nginx   10500 nobody  366r   REG        8,3        141  52396049
/home/j/html/404.html
nginx   10500 nobody  369u  IPv4  174905532                  TCP
70.36.100.46:http->y.net:50057 (CLOSE_WAIT)
nginx   10500 nobody  370r   REG        8,3        141  52396049
/home/j/html/404.html
nginx   10500 nobody  371u  IPv4  174729756                  TCP
70.36.100.46:http->y.net:33731 (CLOSE_WAIT)
-----


here is my "nginx -V"
-----
nginx version: nginx/0.7.65
built by gcc 4.1.2 20080704 (Red Hat 4.1.2-46)
TLS SNI support disabled
configure arguments: --with-debug --with-http_ssl_module
--pid-path=/var/run/nginx.pid --without-mail_smtp_module
--without-mail_imap_module --without-mail_pop3_module
--add-module=../nginx_upload_module-2.0.10/
-----

here is my config:
-----
worker_processes  4;
error_log  logs/error.log  debug;
pid  /var/run/nginx.pid;
events {
    worker_connections  40960;
}
http {
    include       mime.types;
    default_type  application/octet-stream;
    sendfile        on;
    keepalive_timeout  65;
    server {
        listen       80;
        server_name  localhost;
        location / {
            root   html;
            index  index.html index.htm;
        }
        error_page   500 502 503 504  /50x.html;
        location = /50x.html {
            root   html;
        }
    }
    include /usr/local/nginx/sites-enabled/*;
}
-----
server {
    listen       80;
    server_name  jj.x.y.z jj.x.y;

    access_log /usr/local/nginx/logs/jj-access.log;
    error_log  /usr/local/nginx/logs/jj-error.log;

    location / {
        root   /home/j/html/;
        index  index.php index.html;
    }

    location ~ \.php$ {
        if (!-f /home/j/html/$fastcgi_script_name) { return 404; break; 
}
        fastcgi_pass 127.0.0.1:9000;
        fastcgi_index index.php;
        fastcgi_read_timeout 270;
        include /usr/local/nginx/conf/fastcgi_params;
        fastcgi_param SCRIPT_FILENAME /home/j/html/$fastcgi_script_name;
    }

    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   html;
    }
    error_page   404  /404.html;
}
-----


does anyone have idea what's the problem?

thank you.

Hello!

On Mon, Mar 08, 2010 at 11:16:25PM +0700, Oon Arfiandwi wrote:

> i think there's a problem with my 404 configuration.
> /home/j/html/404.html
> -----
> --add-module=../nginx_upload_module-2.0.10/
> -----

Some questions in mostly random order:

Is nginx still answers requests or it's stuck completely?

If it's stuck - where?  What nginx workers do?  You should be able
to find it out by attaching to them with gdb and asking for
backtrace.

Which event method do you use?  It should be logged at 'notice'
level on nginx startup.

Are you able to reproduce problem?  If yes, please try if you
are able to reproduce it without third party modules/patches.

> here is my config:
> -----
> worker_processes  4;
> error_log  logs/error.log  debug;

Ok, you swithed on debug log here...

[...]

> server {
>     listen       80;
>     server_name  jj.x.y.z jj.x.y;
> 
>     access_log /usr/local/nginx/logs/jj-access.log;
>     error_log  /usr/local/nginx/logs/jj-error.log;

... oops, you swithed it off for the server in question (default
logging level is 'error').

If you want to obtain usable debug log you should configure it at
global level and avoid overriding at other levels.  Though in this
case I believe gdb will help better.

Maxim Dounin