Partial downloads

Server NGINX
#1

Hello, guys,

Recently, I’ve started seeing a strange problem - sometimes it happens
that nginx only transfers only a small part of the file and the download
gets interrupted. When I debug it, the issue narrows down to “client
timed out (110: Connection timed out) while sending response to client”,
but this is certainly not the case, as it happens very quickly (almost
immediately after starting the download, in a less than a second). I
don’t see anything unusual with tcpdump. I was wondering whether some of
you has an idea what can be the root cause for this strange behaviour?
Thanks in advance.

nginx version: nginx/1.1.15
built by gcc 4.1.2 20080704 (Red Hat 4.1.2-51)
TLS SNI support disabled
configure arguments: --user=nginx --group=nginx
–prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx
–conf-path=/etc/nginx/nginx.conf
–error-log-path=/var/log/nginx/nginx.log
–http-log-path=/var/log/nginx/access.log
–http-client-body-temp-path=/var/lib/nginx/tmp/client_body
–http-proxy-temp-path=/var/lib/nginx/tmp/proxy
–http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi
–http-scgi-temp-path=/var/lib/nginx/tmp/scgi
–pid-path=/var/run/nginx.pid --lock-path=/var/lock/subsys/nginx
–with-debug --with-http_ssl_module --with-http_geoip_module
–with-http_sub_module --with-http_realip_module
–with-http_gzip_static_module --with-http_random_index_module
–with-http_secure_link_module --with-http_degradation_module
–with-http_stub_status_module --with-http_dav_module
–without-mail_pop3_module --without-mail_imap_module
–without-mail_smtp_module --without-http_uwsgi_module --with-file-aio
–with-cc-opt=’-O2 -g -m32 -march=i686 -mtune=generic
-fasynchronous-unwind-tables’

2.6.18-274.7.1.el5PAE i686

aio on
sendfile on
directio on
tcp_nodelay off
tcp_nopush off
open_file_cache on
client_body_timeout 10
client_header_timeout 10
send_timeout 10
keepalive_timeout 30
max_ranges 5

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,222614,222614#msg-222614

#2

Hello!

On Sat, Feb 18, 2012 at 03:43:59PM -0500, Samael wrote:

Thanks in advance.
Could you please provide debug log? See
http://wiki.nginx.org/Debugging for details.

Maxim D.

#3

Of course - http://pastebin.com/raw.php?i=RnZv58Zq

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,222614,222619#msg-222619

#4

Hello!

On Sat, Feb 18, 2012 at 05:13:37PM -0500, Samael wrote:

Of course - http://pastebin.com/raw.php?i=RnZv58Zq

The log in question suggests there is just normal send_timeout:

…
2012/02/18 22:03:36 [debug] 4368#0: *1 sendfile: @2273599 684918465
2012/02/18 22:03:36 [debug] 4368#0: *1 sendfile: 96360, @2273599
96360:684918465
2012/02/18 22:03:36 [debug] 4368#0: *1 http write filter 0835AC9C
2012/02/18 22:03:36 [debug] 4368#0: *1 http copy filter: -2
“/test/bigfile?”
2012/02/18 22:03:36 [debug] 4368#0: *1 http writer output filter: -2,
“/test/bigfile?”
2012/02/18 22:03:36 [debug] 4368#0: *1 event timer: 23, old: 2450531909,
new: 2450532133
2012/02/18 22:03:49 [debug] 4368#0: *1 event timer del: 23: 2450531909
2012/02/18 22:03:49 [debug] 4368#0: *1 http run request:
“/test/bigfile?”
2012/02/18 22:03:49 [debug] 4368#0: *1 http writer handler:
“/test/bigfile?”
2012/02/18 22:03:49 [info] 4368#0: *1 client timed out (110: Connection
timed out) while sending response to client, client: 1.1.1.2, server:
1.1.1.1, request: “GET /test/bigfile HTTP/1.1”, host: “1.1.1.1”,
referrer: “http://1.1.1.1/test/”
2012/02/18 22:03:49 [debug] 4368#0: *1 http finalize request: 408,
“/test/bigfile?” a:1, c:1
…

Note that nothing happens with the connection in question between
“22:03:36” and “22:03:49”. Looks ok unless you have additional
info which suggests that data from socket buffer was sent to
the client and it’s nginx fault it doesn’t sent more data.

It’s slightly off from expected 10 seconds as per your config part
you’ve provided, though this is likely related to the fact that your
nginx processes are disk-bound and nginx wasn’t able to process
timer in time. You may want to take a look at sendfile_max_chunk
directive, see

http://nginx.org/en/docs/http/ngx_http_core_module.html#sendfile_max_chunk

Maxim D.

#5

Thank you, this was useful. I’ve tried it again and this time I
correlated the events in the log and the tcpdump capture.

It seems that the client is requesting a window update, which doesn’t
receive an answer and triggers the send timeout. I think that this is
the issue.

Thank you very much for your help.

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,222614,222621#msg-222621