Keep-alive connection cause download hangs in Nginx after 0.7.67

Hello,

I am running into this download hanging issue. We are using nginx as a
download server with SSL and secure download module enabled. When using
nginx-0.7.67, it works well. But when we try to upgrade to nginx-0.8.54,
there is a download hanging issue. We figured out it is related to
keep-alive header or reusing TCP connection. When we download a small
file and it finished, then in a few seconds, in same browser, if trying
to download again, the download hangs. Nginx doesn’t print out any log
message. If we turn off keep-alive in either browser or nginx server in
0.8.54 by setting keepalive_timeout 0;, then it works well.

Our environment is: Solaris 10, nginx 0.7.67/0.8.54, secure download
module, SSL.

The other difference between 0.7.67 and 0.8.54 is in the error.log
message. In 0.7.67, when download finished, it always print out message
as below:
2011/01/27 15:27:31 [info] 12616#0: *1 client xxx.xxx.xxx.xxx
closed keepalive connection
But in 0.8.54, it never prints out this message. We have same config
file for both 0.7.67 and 0.8.54.

I think setting keepalive_timeout to 0 is just a workaround and it may
have performance impact. Trying to find out if there is better solution
for this.

Thanks for any suggestions.

Posted at Nginx Forum:

Hello!

On Thu, Jan 27, 2011 at 10:48:40AM -0500, WilliamOMS wrote:

Our environment is: Solaris 10, nginx 0.7.67/0.8.54, secure download
I think setting keepalive_timeout to 0 is just a workaround and it may
have performance impact. Trying to find out if there is better solution
for this.

Thanks for any suggestions.

Are you able to reproduce the problem without 3rd party modules?

If yes - please provide more information (nginx -V output, config,
debug log), see Debugging | NGINX for details.

Maxim D.

Hi Maxim,

I can not reproduce this problem without secure download module. I will
send email to author for help.

Thanks

Posted at Nginx Forum:

Greetings,

I had the exact same issues you are describing. I use nginx to reverse
proxy to an application server backend running Windows+Apache+PHP. If
keepalive_timeout was non-zero, PDF downloads would hang on our website.
If we set it to zero, everything worked perfectly. However, we run all
our sites on SSL exclusively; The overhead of doing SSL Negotiation for
every single HTTP request was hurting performance significantly.

I noticed that the length of “hang” of the download was directly
proportional to the value of keepalive_timeout. If the keepalive was set
to 60, it would take 60 seconds before the download appeared to
“finish”. If it was 75, it appeared to the client that the download took
75 seconds.

When inspecting the headers for the PDF response, I realized the issue.
“Content-Encoding: gzip”. Why is this an issue? Because when the
“Content-Length” header is computed on the backend in PHP, it is
computing the length BEFORE it was gzipped. So, there is a difference
between the “Content-Length” and the size of what is actually sent to
the client. The actual amount of data sent to the client was gzipped,
and SMALLER than the “Content-Length” header.

In my case, PHP had zlib.output_compression set to “On”, which
compressed and altered the size of the PDF, making it smaller.
Therefore, the client was waiting for data that wasn’t there. The client
received all the data, but didn’t know it. It was waiting for the
additional bytes. When the connection timed out (keepalive_timeout), it
finally “finished” successfully (although the client already had 100% of
the data).

When keepalive_timeout was 0, nginx would close the connection
immediately because nginx knew the request was complete- regardless of
what Content-Length says- thus signalling the client to stop waiting for
additional data.

I would suggest making sure the data sent to the client is actually
equal to what the “Content-Length” header says. Your application code
could be inadvertently compressing it (like PHP was doing for me) or
your backend webserver could be doing it (like Apache). I turned zlib
compression Off in php.ini on my application server, let nginx do gzip
for HTML data (nothing is modifying the PDFs anymore), and it completely
resolved the issue. My keepalive_timeout is set to 60 and performance
has improved dramatically since we eliminated countless SSL Negotiation
steps.

Hope this helps!

Posted at Nginx Forum:

Am 27.01.2011 um 20:25 schrieb WilliamOMS:

Hi Maxim,

I can not reproduce this problem without secure download module. I
will
send email to author for help.

We have a similar problem in a configuration not involving NGINX at
all (it’s a complex configuration with Load-Balancers, commercial
firewalls and a “WAF” appliances…)

I would be highly interested in explanations of this phenomenon.

So, if you get any leads, would you mind sharing?

Thanks in advance.