Truncated output for large files through proxy

Server NGINX
1

Hello,

nginx 1.4.1 is truncating proxy output for large files.

*** Steps to reproduce ***

nginx.conf:

 location ^~ /download/ {
     proxy_pass        http://localhost:9000;
     proxy_set_header  Host $host;
 }

The back end is a very simple servlet that copies a static file to the
output stream. The test file is 7147652 bytes. Small files are
delivered correctly,

*** Test results ***

First to show that the back end is sending a full file access it
directly on port 9000:

$ curl http://domain.local:9000/download/file.pdf | wc
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
100 6980k 0 6980k 0 0 39.3M 0 --:–:-- --:–:-- --:–:–
39.6M
39699 166238 7147652

7147652 bytes delivered correctly.

Example of failure, request via nginx:

$ curl http://domain.local/download/file.pdf | wc
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
100 167k 0 167k 0 0 5779k 0 --:–:-- --:–:-- --:–:–
5992k
1959 3853 171828

Only 171828 bytes delivered.

By restricting the download speed more bytes are delivered, slower is
better, eg:

$ curl --limit-rate 200k http://domain.local/download/file.pdf | wc
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
100 6980k 0 6980k 0 0 200k 0 --:–:-- 0:00:34 --:–:–
200k
39699 166238 7147652

*** Workaround ***

Add limit-rate to nginx.conf:

 location ^~ /download/ {
     proxy_pass        http://localhost:9000;
     proxy_set_header  Host $host;
     limit_rate        1000k;
 }

$ curl http://domain.local/download/file.pdf | wc
% Total % Received % Xferd Average Speed Time Time Time
Current
Dload Upload Total Spent Left
Speed
100 6980k 0 6980k 0 0 1000k 0 --:–:-- 0:00:06 --:–:–
1001k
39699 166238 7147652

Full 7147652bytes delivered

*** Other information ***

This is similar to:
http://mailman.nginx.org/pipermail/nginx/2012-April/033272.html
The workaround for this was also set a limit rate in nginx.conf.

nginx 1.4.1
Solaris 10u11

James.

2

Hello!

On Fri, Jun 14, 2013 at 12:05:10PM +0100, James Lee wrote:

Hello,

nginx 1.4.1 is truncating proxy output for large files.

[…]

*** Other information ***

This is similar to:
truncated output with gzip, large files and slow rate
The workaround for this was also set a limit rate in nginx.conf.

nginx 1.4.1
Solaris 10u11

Most interesting line, as usual, is at the end of your message.

Could you please provide debug log, nginx -V output and a minimal
full config to reproduce the problem?

–
Maxim D.
http://nginx.org/en/donation.html

3

Hello!

On Mon, Jun 17, 2013 at 11:56:31AM +0100, James Lee wrote:

error_log /path/to/logs/error.log debug;

events {
worker_connections 1024;
use eventport;

The eventport event method is known to have problems. Use devpoll
(default) instead.

–
Maxim D.
http://nginx.org/en/donation.html

4

On 16/06/2013 02:37, Maxim D. wrote:

Hello,

Could you please provide debug log, nginx -V output and a minimal
full config to reproduce the problem?

*** cut down http.conf ***

error_log /path/to/logs/error.log debug;

events {
worker_connections 1024;
use eventport;
}

http {
server {
listen 80;
server_name domain.local;

     location ^~ / {
         proxy_pass        http://localhost:9000;
         proxy_set_header  Host $host;
     }
 }

}

*** workaround ***

remove “use eventport;”. This was added as per:
http://wiki.nginx.org/NginxOptimizations
It uses “devpoll” by default.

*** debug log ***

Attached as “nginx.debug.log.bz2”

*** nginx -V ***

nginx -V

nginx version: nginx/1.4.1
TLS SNI support enabled
configure arguments: --prefix=/opt/PREFIX
–http-client-body-temp-path=/var/opt/PREFIX/nginx/client
–http-fastcgi-temp-path=/var/opt/PREFIX/nginx/fastcgi
–http-uwsgi-temp-path=/var/opt/PREFIX/nginx/uwsgi_temp
–http-scgi-temp-path=/var/opt/PREFIX/nginx/scgi
–http-proxy-temp-path=/var/opt/PREFIX/nginx/proxy
–conf-path=/etc/opt/PREFIX/nginx/nginx.conf
–error-log-path=/var/opt/PREFIX/nginx/logs/error.log
–http-log-path=/var/opt/PREFIX/nginx/logs/access.log
–pid-path=/var/opt/PREFIX/nginx/nginx.pid
–lock-path=/var/opt/PREFIX/nginx/nginx.lock --user=webservd
–group=webservd --without-mail_pop3_module --without-mail_imap_module
–without-mail_smtp_module --with-http_gzip_static_module
–with-http_realip_module --with-http_ssl_module
–with-http_stub_status_module --with-debug

James.

5

On 17/06/2013 12:06, Maxim D. wrote:

Hello,

The eventport event method is known to have problems. Use devpoll
(default) instead.

Not by me [until now]. Please remove it and update the documentation!

James.