Lots of connections on TIME_WAIT state

Deepan_C · April 7, 2009, 5:01pm

Hi, I am using nginx with fast-cgi . When I run

$netstat -np | grep 127.0.0.1:9000

I find lot of connections in TIME_WAIT state. Is this because of high
keepalive_timeout value ? When lot of people use (5 requests per
second)
nginx takes more time to respond. System load goes more than 10 during
peak hours.

nginx.conf has the following settings.
worker_processes 5;
worker_connections 1024;
keepalive_timeout 3;

$ps aux | grep php shows 21 php-cgi process running.
$netstat -np | grep 127.0.0.1:9000 shows 281 connections.

Below are the exact outputs for both.
Thanks
Deepan

debian:~# netstat -np | grep 127.0.0.1:9000
tcp 0 0 127.0.0.1:9000 127.0.0.1:45603
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45601
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45089
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45600
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45605
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45093
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45604
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45348
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45092
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45099
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45611
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45354
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45609
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45097
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45103
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45359
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45613
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45356
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45107
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45363
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45618
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45105
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45616
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45623
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45109
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45365
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45620
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45108
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45115
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45371
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45627
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45370
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45624
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45112
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45119
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45374
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45630
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45629
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45117
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45628
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45372
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45571
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45570
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45569
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45063
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45575
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45318
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45062
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45323
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45579
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45322
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45578
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45320
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45327
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45583
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45581
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45069
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45325
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45587
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45074
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45074
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45585
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45328
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45334
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45078
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45590
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45589
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45082
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45594
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45593
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45343
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45342
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45086
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45597
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45084
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45410
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45154
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45409
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45664
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45408
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45152
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45415
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45671
ESTABLISHED29504/php-cgi
tcp 0 0 127.0.0.1:9000 127.0.0.1:45414
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45158
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45670
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45668
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45419
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45673
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45417
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45422
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45420
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45164
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45171
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45427
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45170
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45425
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45680
ESTABLISHED29452/php-cgi
tcp 0 0 127.0.0.1:9000 127.0.0.1:45430
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45685
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45173
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45429
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45684
ESTABLISHED29465/php-cgi
tcp 0 0 127.0.0.1:9000 127.0.0.1:45435
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45691
ESTABLISHED29456/php-cgi
tcp 0 0 127.0.0.1:9000 127.0.0.1:45434
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45178
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45688
ESTABLISHED29463/php-cgi
tcp 0 0 127.0.0.1:9000 127.0.0.1:45432
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45176
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45439
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45438
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45693
ESTABLISHED29438/php-cgi
tcp 0 0 127.0.0.1:9000 127.0.0.1:45123
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45376
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45120
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45127
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45383
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45637
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45131
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45643
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45642
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45640
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45391
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45389
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45132
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45139
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45394
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45650
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45393
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45399
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45142
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45653
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45141
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45397
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45402
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45146
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45658
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45656
ESTABLISHED29489/php-cgi
tcp 0 0 127.0.0.1:9000 127.0.0.1:45400
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45406
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45150
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45405
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45660
ESTABLISHED29468/php-cgi
tcp 0 0 127.0.0.1:9000 127.0.0.1:45219
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45475
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45472
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45478
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45222
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45733
ESTABLISHED-
tcp 0 0 127.0.0.1:9000 127.0.0.1:45477
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45483
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45225
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45481
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45486
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45229
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45484
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45228
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45490
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45234
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45488
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45239
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45493
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45492
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45236
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45499
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45242
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45497
TIME_WAIT -
tcp 0 0 127.0.0.1:45733 127.0.0.1:9000
ESTABLISHED-
tcp 0 0 127.0.0.1:45726 127.0.0.1:9000
ESTABLISHED-
tcp 0 0 127.0.0.1:45713 127.0.0.1:9000
ESTABLISHED710/nginx: worker p
tcp 0 0 127.0.0.1:45714 127.0.0.1:9000
ESTABLISHED710/nginx: worker p
tcp 0 0 127.0.0.1:45716 127.0.0.1:9000
ESTABLISHED710/nginx: worker p
tcp 0 0 127.0.0.1:45719 127.0.0.1:9000
ESTABLISHED710/nginx: worker p
tcp 0 0 127.0.0.1:45705 127.0.0.1:9000
ESTABLISHED710/nginx: worker p
tcp 0 0 127.0.0.1:45707 127.0.0.1:9000
ESTABLISHED710/nginx: worker p
tcp 0 0 127.0.0.1:45710 127.0.0.1:9000
ESTABLISHED710/nginx: worker p
tcp 0 0 127.0.0.1:45698 127.0.0.1:9000
ESTABLISHED711/nginx: worker p
tcp 0 0 127.0.0.1:45701 127.0.0.1:9000
ESTABLISHED711/nginx: worker p
tcp 0 0 127.0.0.1:45688 127.0.0.1:9000
ESTABLISHED711/nginx: worker p
tcp 0 0 127.0.0.1:45691 127.0.0.1:9000
ESTABLISHED711/nginx: worker p
tcp 0 0 127.0.0.1:45693 127.0.0.1:9000
ESTABLISHED711/nginx: worker p
tcp 0 0 127.0.0.1:45680 127.0.0.1:9000
ESTABLISHED711/nginx: worker p
tcp 0 0 127.0.0.1:45684 127.0.0.1:9000
ESTABLISHED711/nginx: worker p
tcp 0 0 127.0.0.1:45671 127.0.0.1:9000
ESTABLISHED710/nginx: worker p
tcp 0 0 127.0.0.1:45656 127.0.0.1:9000
ESTABLISHED710/nginx: worker p
tcp 0 0 127.0.0.1:45660 127.0.0.1:9000
ESTABLISHED710/nginx: worker p
tcp 0 0 127.0.0.1:9000 127.0.0.1:45496
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45503
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45502
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45246
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45244
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45187
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45186
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45698
ESTABLISHED29455/php-cgi
tcp 0 0 127.0.0.1:9000 127.0.0.1:45696
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45440
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45184
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45446
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45701
ESTABLISHED29484/php-cgi
tcp 0 0 127.0.0.1:9000 127.0.0.1:45444
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45707
ESTABLISHED29499/php-cgi
tcp 0 0 127.0.0.1:9000 127.0.0.1:45450
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45705
ESTABLISHED29486/php-cgi
tcp 0 0 127.0.0.1:9000 127.0.0.1:45193
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45448
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45199
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45710
ESTABLISHED29464/php-cgi
tcp 0 0 127.0.0.1:9000 127.0.0.1:45453
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45452
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45458
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45202
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45714
ESTABLISHED29500/php-cgi
tcp 8 0 127.0.0.1:9000 127.0.0.1:45713
ESTABLISHED29458/php-cgi
tcp 0 0 127.0.0.1:9000 127.0.0.1:45457
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45456
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45719
ESTABLISHED29457/php-cgi
tcp 0 0 127.0.0.1:9000 127.0.0.1:45462
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45206
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45205
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45716
ESTABLISHED29485/php-cgi
tcp 0 0 127.0.0.1:9000 127.0.0.1:45466
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45210
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45464
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45215
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45471
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45214
TIME_WAIT -
tcp 8 0 127.0.0.1:9000 127.0.0.1:45726
ESTABLISHED29469/php-cgi
tcp 0 0 127.0.0.1:9000 127.0.0.1:45468
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45538
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45282
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45536
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45543
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45285
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45541
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45547
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45289
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45032
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45544
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45288
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45551
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45549
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45298
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45297
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45553
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45552
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45296
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45558
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45301
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45556
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45563
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45049
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45560
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45567
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45567
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45055
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45309
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45053
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45564
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45308
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45251
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45506
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45248
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45511
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45510
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45254
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45508
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45259
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45515
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45257
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45513
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45519
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45518
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45262
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45516
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45523
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45266
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45265
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45520
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45527
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45526
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45269
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45268
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45275
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45531
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45274
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45272
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45534
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45278
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45533
TIME_WAIT -
tcp 0 0 127.0.0.1:9000 127.0.0.1:45276
TIME_WAIT -

$ ps aux | grep php
www-data 723 0.0 0.6 47792 6448 ? Ss Apr06 0:00
/usr/bin/php-cgi -q -b localhost:9000
www-data 29438 4.0 0.8 50852 9088 ? S 01:19 32:18
/usr/bin/php-cgi -q -b localhost:9000
www-data 29452 3.9 0.6 48776 7256 ? S 01:24 31:32
/usr/bin/php-cgi -q -b localhost:9000
www-data 29455 3.9 0.6 48772 7252 ? S 01:25 31:48
/usr/bin/php-cgi -q -b localhost:9000
www-data 29456 4.0 0.6 48776 7228 ? S 01:25 32:05
/usr/bin/php-cgi -q -b localhost:9000
www-data 29457 4.1 0.6 48772 7224 ? S 01:26 32:51
/usr/bin/php-cgi -q -b localhost:9000
www-data 29458 4.1 0.6 48780 7232 ? S 01:28 32:33
/usr/bin/php-cgi -q -b localhost:9000
www-data 29463 4.1 0.6 48772 7224 ? S 01:33 32:47
/usr/bin/php-cgi -q -b localhost:9000
www-data 29464 3.9 0.6 48772 7252 ? S 01:34 31:20
/usr/bin/php-cgi -q -b localhost:9000
www-data 29465 4.1 0.6 48772 7224 ? S 01:34 32:52
/usr/bin/php-cgi -q -b localhost:9000
www-data 29468 4.2 0.6 48772 7224 ? S 01:36 33:28
/usr/bin/php-cgi -q -b localhost:9000
www-data 29469 4.2 0.6 48776 7240 ? S 01:37 33:09
/usr/bin/php-cgi -q -b localhost:9000
www-data 29484 4.1 0.6 48776 7236 ? S 01:40 32:10
/usr/bin/php-cgi -q -b localhost:9000
www-data 29485 3.9 0.6 48772 7224 ? S 01:41 31:05
/usr/bin/php-cgi -q -b localhost:9000
www-data 29486 4.1 1.6 58632 16856 ? R 01:41 32:32
/usr/bin/php-cgi -q -b localhost:9000
www-data 29489 4.1 0.6 48772 7220 ? S 01:43 32:11
/usr/bin/php-cgi -q -b localhost:9000
www-data 29499 4.0 0.6 48764 7216 ? S 01:45 31:18
/usr/bin/php-cgi -q -b localhost:9000
www-data 29500 4.0 0.6 48772 7252 ? S 01:45 31:48
/usr/bin/php-cgi -q -b localhost:9000
www-data 29501 4.1 1.3 55760 13964 ? R 01:45 32:16
/usr/bin/php-cgi -q -b localhost:9000
www-data 29504 4.0 0.6 48780 7228 ? S 01:50 31:10
/usr/bin/php-cgi -q -b localhost:9000
www-data 29505 4.0 0.6 48776 7240 ? S 01:51 31:26
/usr/bin/php-cgi -q -b localhost:9000

Deepan_C · April 7, 2009, 5:07pm

Probably an issue with your database if you use one ;check if you have a
slow query or something

Deepan_C · April 7, 2009, 5:16pm

TIME_WAIT is a specific state for tcp connection. When a TCP
connection has been closed, the system keep it in the state TIME_WAIT
for a constant time (which depends on the OS). This is done to prevent
packet collision. This is the short version. (more explication here:
Winsock Programmer’s FAQ: Debugging TCP/IP or google)

So i you have a lot of connection per/s and thoses connections are
short (in time), this is normal to have a lot of TIME_WAIT.

You maybe can reduce this timeout, check this specific for your OS.

++ jerome

2009/4/7 Deepan C. [email protected]:

Deepan_C · April 7, 2009, 6:17pm

You may want to get a count on the TIME_WAITs i.e. netstat -n | grep
TIME_WAIT | grep wc -l

If you’re starting to hit the ceiling for your kernel then you will
need to tune it.

Cheers
Kon

Deepan_C · April 8, 2009, 10:26am

On Wed, Apr 08, 2009 at 10:47:16AM +0300, Artis Caune wrote:

sysctl net.inet.tcp.nolocaltimewait=1

Due to the incorrect implementation this remedy is worse than the
disease.
The net.inet.tcp.nolocaltimewait relys on unlimited RST delivery,
therefore
if there are too many RSTs, they will be limited by
net.inet.icmp.icmplim
and you will have a lot of sockets in the LAST_ACK state on server side
instead of lot of sockets in the TIME_WAIT on client side.

Deepan_C · April 8, 2009, 5:11pm

So, what is the solution? I have exactly the same problem, my nginx is
in a virtual server (openvz), working as a proxy server in front of
apache and oftenly (after 2k-3k requests) server becomes “out of
sockets” even I raise the allowed numbers of sockets to a very big
number.

Igor S. yazmÄ±ÅŸ:

Deepan_C · April 8, 2009, 9:53am

2009/4/7 Deepan C. [email protected]:

Hi,
Â Â I am using nginx with fast-cgi . Â When I run
$netstat -np | grep 127.0.0.1:9000
I find lot of connections in TIME_WAIT state. Is this because of high
keepalive_timeout value ? Â When lot of people use (5 requests per second)
Â nginx takes more Â time to respond. System load goes more than 10 during
peak hours.

This is because of how TCP works.

debian:~# netstat -np | grep 127.0.0.1:9000
tcp Â Â Â Â 0 Â Â Â 0 127.0.0.1:9000 Â Â Â Â Â 127.0.0.1:45603
TIME_WAIT Â -
tcp Â Â Â Â 0 Â Â Â 0 127.0.0.1:9000 Â Â Â Â Â 127.0.0.1:45601
TIME_WAIT Â -

If you were on FreeBSD, you could disable TIME_WAIT on loopback
completely by setting:

sysctl net.inet.tcp.nolocaltimewait=1

Maybe debian has similar /proc tuning or you can try to decrease msl
(maximum segment lifetime) or beter don’t use inet sockets on loopback

use unix sockets.

–
regards,
Artis Caune

<----. CCNA | BSDA
<----|====================
<----’ didii FreeBSD

Deepan_C · April 8, 2009, 5:50pm

Hello!

On Wed, Apr 08, 2009 at 05:56:51PM +0300, AnÄ±l Ã‡etin wrote:

So, what is the solution? I have exactly the same problem, my nginx is
in a virtual server (openvz), working as a proxy server in front of
apache and oftenly (after 2k-3k requests) server becomes “out of
sockets” even I raise the allowed numbers of sockets to a very big
number.

You probably “out of ports”, not out of sockets. Solution is to
configure TIME_WAIT reusing (tw_reuse, tw_recyle or something like
depending on your OS). You may also allow your system to use more
ports for outgoing connections.

Under FreeBSD reusing of TIME_WAIT sockets is the default, and
portrange for outgoing connections may be tuned via
net.inet.ip.portrange.hifirst and net.inet.ip.portrange.hilast
sysctls.

Not sure about Linux, but Google suggests reusing of TIME_WAIT
sockets may be turned on via /proc/sys/net/ipv4/tcp_tw_recycle.

Maxim D.

Deepan_C · April 8, 2009, 7:04pm

Thanks for answer Maxim, but it is exactly “out of sockets” openvz has
resource limit option for opened tcp sockets (numtcpsock) and when time
comes there becomes thousands (ten thousands approx.) of open
connections while there is only 2000-3000 clients. Apache keepalive is
off also. I read about TIME_WAIT state, I know the connection is
reusable but why there is so much sockets opened? As my knowledge while
nginx is a proxy it opens 2 connections for one client with apache, isnt
it right? So there must be 4000-6000 tcp sockets not 10000-15000.

Isn’t “/proc/sys/net/ipv4/tcp_tw_recycle” turned on by default in
linux? This can be my problem I will check and try it. May be it isnt
using sockets in TIME_WAIT states and timeout for connections in this
state is very big, so that it is opening new connections again and again
without dropping any of before opened sockets.

Maxim D. yazmÄ±ÅŸ:

Deepan_C · April 8, 2009, 7:18pm

just a shot in the dark, but do you have any dynamic javascript request
on
page load? Could be that these opens the additional connections. Most
browsers can do 2 simultaneous requests, given the concurrency rate,
that
could easily end up at 3xclients, which would still not explain the
15,000…

hmmm…

Deepan_C · April 9, 2009, 11:22am

I learned that, in linux, generally /proc/sys/net/ipv4/tcp_tw_recycle
is on by default but in redhat-linux it is 0 by default. I set it to 1
now, I will see what happens on next days.