# Client wrongly closed keepalive connection problem

**URL:** <https://www.ruby-forum.com/t/client-wrongly-closed-keepalive-connection-problem/214314>\
**Category:** NGINX\
**Created:** [November 28, 2011, 4:17pm UTC](https://www.ruby-forum.com/t/client-wrongly-closed-keepalive-connection-problem/214314 "2011-11-28T16:17:46Z")\
**Posts on this page:** 2\
**Page:** 1

<div class="post-metadata">

**Author:** ![luislavena](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/0ea827/32.png) [@luislavena](https://www.ruby-forum.com/u/luislavena)\
**Post date:** [November 28, 2011, 4:17pm UTC](https://www.ruby-forum.com/t/client-wrongly-closed-keepalive-connection-problem/214314/1 "2011-11-28T16:17:46Z")

</div>

Hi all, we are some college students maintaining a linux distribution  
mirror. We are not experienced but now meet with  
this problem.  
There is a pxe system in our college to help every computer boot a linux  
system or install one on Ethernet. Now most users  
cannot install, cause most of the packages will time out when being  
downloaded.  
We used lftp to mirror the directories, trying to find out the problem.  
On this directory (on our mirror with same content)  
[http://ftp.debian.org/debian/pool/main/a/abiword/](http://ftp.debian.org/debian/pool/main/a/abiword/)  
lftp hints:

\<— Accept-Ranges: bytes  
\<—  
—\> HEAD /debian/pool/main/a/abiword/abiword\_2.8.2.orig.tar.gz  
HTTP/1.1  
—\> Host: [mirrors.ustc.edu.cn](http://mirrors.ustc.edu.cn)  
—\> User-Agent: lftp/4.0.6  
—\> Accept: _/_  
—\> Connection: keep-alive  
—\>  
\<— HTTP/1.1 200 OK  
\*\*\*\* recv: : Connection reset by peer  
---- Closing HTTP connection

sometimes might be  
“send:: Connection reset by peer”

and the nginx error.log shows"[info] 8864#0: \*1661715 client xxxx closed  
keepalive connection"

After trying mirror with lftp many times, we found that after request  
around 100 files in this directory,  
the server starts to send data back. And around receiving each 15-17  
files, the reset error will show once,  
until the server start to send out bigger files in this directory.

At first we suspect that it might because there are too many small files  
at the start in this directory,  
but when we move this directory to another host and mirror it, it is  
just ok.

Following is our nginx conf without some not that import configuration:

user www-data;  
worker\_processes 16;  
worker\_cpu\_affinity xxxxx(just skip it);  
worker\_rlimit\_nofile 65535;  
events {  
use epoll;  
worker\_connections 10000;  
#multi\_accept off;  
}  
http {  
include /etc/nginx/mime.types;  
access\_log /var/log/nginx/access.log;  
error\_log /var/log/nginx/error.log info;

```
sendfile on;
tcp_nopush on;

keepalive_timeout 30;
tcp_nodelay off;(we also tried nopush off with nodelay on)
limit_zone conn_per_ip $binary_remote_addr 10m;

```

}

server {  
root /srv/www/;  
autoindex on;  
limit\_conn conn\_per\_ip 20;

```
## limit speed for MSIE UAs
if ($http_user_agent ~ "MSIE" ) {
    set $limit_rate 10k;
}

```

}

netstat -ant |wc -l : 4000-5000

sysctl.conf:  
net.ipv4.tcp\_max\_syn\_backlog = 65536  
net.core.netdev\_max\_backlog = 32768  
net.core.somaxconn = 32768  
net.core.wmem\_default = 8388608  
net.core.rmem\_default = 8388608  
net.core.rmem\_max = 16777216  
net.core.wmem\_max = 16777216  
net.ipv4.tcp\_timestamps = 0  
net.ipv4.tcp\_synack\_retries = 2  
net.ipv4.tcp\_syn\_retries = 2  
net.ipv4.tcp\_tw\_recycle = 1  
net.ipv4.tcp\_tw\_reuse = 1  
net.ipv4.tcp\_mem = 94500000 915000000 927000000  
net.ipv4.tcp\_max\_orphans = 3276800

The nginx version we have tried: 1.0.5, 1.0.10, 0.7.67  
Server system: debian squeeze.

Currently we found that limit\_req might solve the problem, but is this  
the right way? Because it  
will restrict the performance.

Thank you for your help.

Posted at Nginx Forum:

> **[client wrongly closed keepalive connection problem](https://forum.nginx.org/read.php?2,219220,219220#msg-219220)**
>
> Hi all, we are some college students maintaining a linux distribution mirror. We are not experienced but now meet with this problem. There is a pxe system in our college to help every computer boot a linux system or install one on Ethernet. Now most...

---

<div class="post-metadata">

**Author:** ![freshsun](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/f/8491ac/32.png) [@freshsun](https://www.ruby-forum.com/u/freshsun)\
**Post date:** [November 28, 2011, 7:14pm UTC](https://www.ruby-forum.com/t/client-wrongly-closed-keepalive-connection-problem/214314/2 "2011-11-28T19:14:46Z")

</div>

Hello!

On Mon, Nov 28, 2011 at 10:17:07AM -0500, freshsun wrote:

> lftp hints:  
> \<— HTTP/1.1 200 OK  
> around 100 files in this directory,  
> the server starts to send data back. And around receiving each 15-17  
> files, the reset error will show once,  
> until the server start to send out bigger files in this directory.
> 
> At first we suspect that it might because there are too many small files  
> at the start in this directory,  
> but when we move this directory to another host and mirror it, it is  
> just ok.

It looks like lftp uses pipelining excessively, you may try  
setting

```
lingering_close always;

```

to see if it helps (available in nginx 1.1.0+, 1.0.6+).

Maxim D.