We use nginx as both a load-balancer and webserver. This issue is with
nginx functioning as a load-balancer.
We reverse proxy to 6 nginx webservers running a number of Unicorn
application servers, these webserver nginx instances also run Evan
mod_zip to assemble archives on the fly. We have discovered under
circumstances the load-balancing nginx will “hang-up” on the webserver
the load-balancer is configured with proxy_buffering off, however
proxy_buffering on seems to succeed. We would prefer to run without
proxy_buffering to prevent the load-balancer’s local storage from being
Our default setup uses nginx 0.7.65 for both the load-balancer and the
webserver, however switching to using 1.0.12 as the load-balancer has
same problem. We have experimented with different software doing the
load-balancing and it does not exhibit this issue.
I’ve have linked the nginx configuration file we’re using on the load
balancer, and debug logs for both 0.7.65 and 1.0.12.
The buffering on log is very long, but it does show success of a 4.8GB
response, the other responses always fail at the same point (826 MB).
The client sees the following (in access.log):
$ curl -b cookie.txt -o US.zip
% Total % Received % Xferd Average Speed Time Time Time
Dload Upload Total Spent Left
16 4922M 16 826M 0 0 1906k 0 0:44:03 0:07:23 0:36:40
curl: (18) transfer closed with 4294967296 bytes remaining to read
The nginx instance serving as the webserver logs:
Feb 07 15:03:39 ip-10-2-185-35 error.log: 2012/02/07 23:03:39 [info]
21425#0: *16656299 client closed prematurely connection, so upstream
connection is closed too (104: Connection reset by peer) while reading
upstream, client: 10.254.174.80, server: mydomain.com, request: “GET
/folders/7816672/archive HTTP/1.0”, subrequest: “/s3/asset-27235062”,