# Truncated output with gzip, large files and slow rate **URL:** \ **Category:** NGINX\ **Created:** [April 16, 2012, 10:38am UTC](https://www.ruby-forum.com/t/truncated-output-with-gzip-large-files-and-slow-rate/218928 "2012-04-16T10:38:16Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![Gurdipe\_D](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/b77776/32.png) [@Gurdipe\_D](https://www.ruby-forum.com/u/Gurdipe_D)\ **Post date:** [April 16, 2012, 10:38am UTC](https://www.ruby-forum.com/t/truncated-output-with-gzip-large-files-and-slow-rate/218928/1 "2012-04-16T10:38:16Z")
Hello, nginx is truncating output with the following: - gzip on - compressed size greater than 100k (approx) - client downloads at a slow rate \*\*\* Steps to reproduce \*\*\* nginx.conf: events { } http { types { text/plain txt; } gzip on; gzip\_types text/plain; server { listen 80 default\_server; server\_name \_; root /path/to/root; } } Create a test file that is larger than 100k compressed: $ head -1000 /dev/random \> /path/to/root/test.txt Request exhibiting failure: $ curl -H “Accept-Encoding: gzip” --limit-rate 50k localhost/test.txt | gzcat | cksum % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 180k 0 180k 0 0 2966 0 --:–:-- 0:01:02 –:–:-- 0 curl: (18) transfer closed with outstanding read data remaining gzcat: stdin: unexpected end of file 649550918 184260 and with wget: $ wget --header “Accept-Encoding: gzip” --limit-rate=50k -O /dev/stdout localhost/test.txt | gzcat | cksum –2012-04-15 11:27:04-- [http://localhost/test.txt](http://localhost/test.txt) Resolving localhost (localhost)… 127.0.0.1, ::1 Connecting to localhost (localhost)|127.0.0.1|:80… connected. HTTP request sent, awaiting response… 200 OK Length: unspecified [text/plain] Saving to: `/dev/stdout’ ``` [<=>] 184,330 --.-K/s in ``` 61s 2012-04-15 11:28:06 (2.96 KB/s) - `/dev/stdout’ saved [184330] gzcat: stdin: unexpected end of file 649550918 184260 Some variants showing success (correct size and cksum) by avoiding one of compression or limit-rate: $ curl -H “Accept-Encoding: gzip” localhost/test.txt | gzcat | cksum % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 235k 0 235k 0 0 5970k 0 --:–:-- --:–:-- --:–:– 6370k 2871747365 241269 $ curl --limit-rate 50k localhost/test.txt | cksum % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 100 235k 100 235k 0 0 47700 0 0:00:05 0:00:05 --:–:– 55650 2871747365 241269 There are also failures with major web browsers including Firefox 11.0, MSIE 9.0 and Safari. No error is reported by nginx. The http return code and logged code is 200. The only clue in the server is the sent size reported is smaller than the expected compressed size. \*\*\* Other information \*\*\* nginx 1.1.19 Solaris 10 u10 James. ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [April 16, 2012, 12:18pm UTC](https://www.ruby-forum.com/t/truncated-output-with-gzip-large-files-and-slow-rate/218928/2 "2012-04-16T12:18:15Z")
Hello! On Mon, Apr 16, 2012 at 09:37:27AM +0100, James Lee wrote: > nginx.conf: > listen 80 default\_server; > > curl: (18) transfer closed with outstanding read data remaining > –2012-04-15 11:27:04-- [http://localhost/test.txt](http://localhost/test.txt) > > % Total % Received % Xferd Average Speed Time Time > Time Current > > No error is reported by nginx. The http return code and logged code > is 200. The only clue in the server is the sent size reported is > smaller than the expected compressed size. > > \*\*\* Other information \*\*\* > > nginx 1.1.19 > Solaris 10 u10 Could you please provide debug log and nginx -V output? See [GitHub - nginxinc/nginx-wiki: ARCHIVED -- Source for the now archived NGINX Wiki section of https://www.nginx.com · GitHub](http://wiki.nginx.org/Debugging) for details. Maxim D. ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [April 16, 2012, 7:56pm UTC](https://www.ruby-forum.com/t/truncated-output-with-gzip-large-files-and-slow-rate/218928/3 "2012-04-16T19:56:44Z")
Hello! On Mon, Apr 16, 2012 at 02:27:08PM +0100, James Lee wrote: > On Mon, 16 Apr 2012 11:17:41 +0100, Maxim D. > [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > > > Could you please provide debug log and nginx -V output? > > debug.log.bz2 as attachment. Thanks, the problem looks clear. Could you please test if the following patch fixes things for you? # HG changeset patch # User Maxim D. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) # Date 1334597965 -14400 # Node ID f58e907f57e1136223c178ea81a18e52a136ec60 # Parent 56f926e54f3430f1f9ecc9ef154262de10c33005 Fixed loop in ngx\_writev\_chain() and ngx\_solaris\_sendfilev\_chain(). diff --git a/src/os/unix/ngx\_solaris\_sendfilev\_chain.c b/src/os/unix/ngx\_solaris\_sendfilev\_chain.c — a/src/os/unix/ngx\_solaris\_sendfilev\_chain.c +++ b/src/os/unix/ngx\_solaris\_sendfilev\_chain.c @@ -74,7 +74,6 @@ ngx\_solaris\_sendfilev\_chain(ngx\_connecti ``` send = 0; ``` - complete = 0; - ``` complete = 0; sent = 0; prev_send = send; ``` diff --git a/src/os/unix/ngx\_writev\_chain.c b/src/os/unix/ngx\_writev\_chain.c — a/src/os/unix/ngx\_writev\_chain.c +++ b/src/os/unix/ngx\_writev\_chain.c @@ -54,7 +54,6 @@ ngx\_writev\_chain(ngx\_connection\_t \*c, ng } ``` send = 0; ``` - complete = 0; - ``` complete = 0; prev_send = send; vec.nelts = 0; ``` Maxim D. ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [April 16, 2012, 3:27pm UTC](https://www.ruby-forum.com/t/truncated-output-with-gzip-large-files-and-slow-rate/218928/4 "2012-04-16T15:27:52Z")
On Mon, 16 Apr 2012 11:17:41 +0100, Maxim D. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Could you please provide debug log and nginx -V output? debug.log.bz2 as attachment. $ nginx -V nginx version: nginx/1.1.19 TLS SNI support enabled configure arguments: --prefix=/opt/prefix --prefix=/var/opt/prefix/nginx –http-client-body-temp-path=/var/opt/prefix/nginx/client –http-fastcgi-temp-path=/var/opt/prefix/nginx/fastcgi –http-scgi-temp-path=/var/opt/prefix/nginx/scgi –http-proxy-temp-path=/var/opt/prefix/nginx/proxy –conf-path=/etc/opt/prefix/nginx/nginx.conf –error-log-path=/var/opt/prefix/nginx/logs/error.log –http-log-path=/var/opt/prefix/nginx/logs/access.log –pid-path=/var/opt/prefix/nginx/nginx.pid –lock-path=/var/opt/prefix/nginx/nginx.lock --user=webservd –group=webservd --without-mail\_pop3\_module --without-mail\_imap\_module –without-mail\_smtp\_module --with-http\_gzip\_static\_module –with-http\_realip\_module --with-http\_ssl\_module –with-http\_stub\_status\_module --with-debug James. ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [April 16, 2012, 8:22pm UTC](https://www.ruby-forum.com/t/truncated-output-with-gzip-large-files-and-slow-rate/218928/5 "2012-04-16T20:22:51Z")
On Mon, 16 Apr 2012 18:56:03 +0100, Maxim D. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: Hello, > Thanks, the problem looks clear. Could you please test if the > following patch fixes things for you? Glad it looked clear to you! Quick test shows this fixes the problem. Thank you for the fix, I’ll check tomorrow on a working installation and hopefully users will get full files at busy times. James.