# TCP Socket TIME\_WAIT problem

**URL:** <https://www.ruby-forum.com/t/tcp-socket-time-wait-problem/220859>\
**Category:** NGINX\
**Created:** [June 22, 2012, 11:52am UTC](https://www.ruby-forum.com/t/tcp-socket-time-wait-problem/220859 "2012-06-22T11:52:17Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![aris](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/258eb7/32.png) [@aris](https://www.ruby-forum.com/u/aris)\
**Post date:** [June 22, 2012, 11:52am UTC](https://www.ruby-forum.com/t/tcp-socket-time-wait-problem/220859/1 "2012-06-22T11:52:17Z")

</div>

Hi folks,  
I’m deeply testing NGINX to understand boundaries and limits because I’d  
like to create a webservice for an high performance application in our  
LAN. I’m currently running NGINX 1.2.1 on Linux and I’m just serving a  
static webpage (“Hello World”) to see  
performances. This website is accessed on localhost from a dummy bash  
script:  
#!/bin/bash  
for i in {0…50000}; do  
echo $i  
wget [http://127.0.0.1/test.html](http://127.0.0.1/test.html) -q -O - \>\> /dev/null  
done

In other terminals I’m monitoring sockets and their status ( netstat -an  
|grep WAIT|wc -l )  
When I run the bash script I can see no troubles until I reach 28233  
TIME\_WAIT connections, then it freezes for a while until some TIME\_WAIT  
connections are closed. Performances are ok until I reach this limit,  
then everything is frozen for a while until TCP time\_wait connections  
are disposed  
After few researches with google I’ve found these info:  
[http://developerweb.net/viewtopic.php?id=2941](http://developerweb.net/viewtopic.php?id=2941)  
[http://developerweb.net/viewtopic.php?id=2982](http://developerweb.net/viewtopic.php?id=2982)

I’m running tests on a pretty fast machine: Intel core i7 with 8Gb RAM  
and a Gentoo Linux OS with kernel 3.12, is there a way to get a rid of  
this limit ? I’ll post here my nginx.conf but please let me know if you  
need more info or config files

user nginx nginx;  
worker\_processes 1;  
error\_log /var/log/nginx/error\_log info;  
events {  
worker\_connections 1024;  
use epoll;  
}  
http {  
include /etc/nginx/mime.types;  
default\_type application/octet-stream;  
log\_format main  
'$remote\_addr - $remote\_user [$time\_local] ’  
'“$request” $status $bytes\_sent ’  
'“$http\_referer” “$http\_user\_agent” ’  
‘“$gzip\_ratio”’;  
client\_header\_timeout 10m;  
client\_body\_timeout 10m;  
send\_timeout 10m;  
connection\_pool\_size 256;  
client\_header\_buffer\_size 1k;  
large\_client\_header\_buffers 4 2k;  
request\_pool\_size 4k;  
gzip on;  
gzip\_min\_length 1100;  
gzip\_buffers 4 8k;  
gzip\_types text/plain;  
output\_buffers 1 32k;  
postpone\_output 1460;  
sendfile on;  
tcp\_nopush on;  
tcp\_nodelay on;  
keepalive\_timeout 75 20;  
ignore\_invalid\_headers on;  
index index.php index.html;  
server {  
listen 127.0.0.1;  
server\_name localhost;  
root /var/www/localhost/htdocs;  
location ~ .\*.php$ {  
include /etc/nginx/fastcgi.conf;  
fastcgi\_pass 127.0.0.1:1234;  
fastcgi\_index index.php;  
}  
}  
}

Thanks in advance for your reply  
Ben

Posted at Nginx Forum:

> **[TCP Socket TIME\_WAIT problem](https://forum.nginx.org/read.php?2,227804,227804#msg-227804)**
>
> Hi folks, I'm deeply testing NGINX to understand boundaries and limits because I'd like to create a webservice for an high performance application in our LAN. I'm currently running NGINX 1.2.1 on Linux and I'm just serving a static webpage...

---

<div class="post-metadata">

**Author:** ![andreabenini](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/8797f3/32.png) [@andreabenini](https://www.ruby-forum.com/u/andreabenini)\
**Post date:** [June 22, 2012, 1:15pm UTC](https://www.ruby-forum.com/t/tcp-socket-time-wait-problem/220859/2 "2012-06-22T13:15:52Z")

</div>

Hello!

On Fri, Jun 22, 2012 at 05:51:46AM -0400, andreabenini wrote:

> ```
> wget http://127.0.0.1/test.html -q -O - >> /dev/null
> 
> ```
> 
> [DeveloperWeb - Developerweb Agency dan Service, Blog Tips](http://developerweb.net/viewtopic.php?id=2941)  
> [DeveloperWeb - Developerweb Agency dan Service, Blog Tips](http://developerweb.net/viewtopic.php?id=2982)
> 
> I’m running tests on a pretty fast machine: Intel core i7 with 8Gb RAM  
> and a Gentoo Linux OS with kernel 3.12, is there a way to get a rid of  
> this limit ? I’ll post here my nginx.conf but please let me know if you  
> need more info or config files

The problem you’ve hit is local port exhaustion on client side due  
sockets in TIME\_WAIT state. Possible solutions are:

1. Add more client local ports. Trivial way to do so is to add  
more clients (or, rather, client IPs). You may also tune  
single client to use wider local port range  
(/proc/sys/net/ipv4/ip\_local\_port\_range on linux), though it’s  
limited to 64k anyway.

2. Configure your system to reuse/recycle timewait sockets  
(tcp\_tw\_reuse, tcp\_tw\_recycle in the same /proc/ directory on  
Linux).

3. Reduce MSL as used on your system (no idea if it’s tunable on  
Linux), this will cause TIME\_WAIT sockets to expire faster.

Maxim D.