Problem with connection rate limiting and backend redirects

I love nginx! We’re using as a caching front-end proxy in front of our
wordpress mu at http://blogs.law.harvard.edu/, and it’s enabled us to
handle MUCH more traffic on less hardware. Roxors!

Anyway - I discovered an issue with connection rate limiting and backend
redirects.

When the backend issues a redirect for a page containing a large amount
of html / xml content, the content of the subsequent 200 status request
may get truncated by nginx. I had fairly aggressive rules to the effect
of:

limit_req_zone $binary_remote_addr zone=pagereqzone:10m rate=3r/s;
limit_req zone=pagereqzone burst=8;

I seemed that increasing the burst to 16 fixed the issue, but this was
the last place I looked because nothing was getting thrown into the
error logs. I also didn’t expect backend responses to get impacted by
rate limiting.

We’re using 0.7.62 in front of apache, running wordpress 2.8.4a. Apache
and wordpress do no caching any more - nginx handles it all. At some
point I plan to document how this works, because the caching works
(mostly) perfectly for logged in / non-logged in users, rss feeds,
files, etc. It’s great.

Thoughts? Pointers? Places to look?

–DJCP

Hello!

On Fri, Oct 23, 2009 at 08:39:46PM +0200, Daniel Collis-puro wrote:

of:

limit_req_zone $binary_remote_addr zone=pagereqzone:10m rate=3r/s;
limit_req zone=pagereqzone burst=8;

Either use “limit_req … nodelay” or upgrade/patch nginx. This
problem was fixed 0.8.5, but the fix isn’t yet merged into stable.

Maxim D.