Confused about proxy_buffer_size and proxy_buffers

Server NGINX
1

Hi,

I am trying to tune our nginx frontends (proxying a pack of mongrels)
to avoid this type of message

2008/04/30 11:06:50 [warn] 35585#0: *40967931 an upstream response is
buffered to a temporary file /opt/local/var/run/nginx/proxy_temp/
0/58/0000370580 while reading upstream, client: 202.129.81.166,
server: www.redbubble.com, request: “GET /people/basiamc/art/732916-4-
fire-fairy? HTTP/1.0”, upstream:
"http://172.16.0.76:8006/people/basiamc/art/732916-4-fire-fairy
", host: “www.redbubble.com”, referrer:
"http://www.redbubble.com/people/basiamc/art/732916-4-fire-fairy
"

A little bit of curling reveals that our minimum page size is between
11k and 14k, but can be up to 35k for the common pages. So I have set

proxy_buffer_size 32k

In the expectation that this buffer will be allocated for all proxy
requests before the overflow into proxy_buffers.

For pages that have a large number of comments the sizes can range up
to 150k but even with a setting like this

proxy_buffers 128 4k;

which should yield a maximum buffer of 546k, this request

[dave@crimson ~]$ curl -I
http://www.redbubble.com/people/basiamc/art/732916-4-fire-fairy
HTTP/1.1 200 OK
Server: nginx
Date: Wed, 30 Apr 2008 01:13:45 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Set-Cookie: _session_id=1263f18e47b978d84cab76592cf240c4; path=/
Cache-Control: private, max-age=0, must-revalidate
Content-Length: 282541

Generates a warning.

Should I continue to increase the buffer size? Should I allocate the
buffer in larger sizes? Is there an interaction with proxy_buffers and
gzip or the output buffer settings ?

Cheers

Dave

2

On Wed, Apr 30, 2008 at 11:15:14AM +1000, Dave C. wrote:

"http://172.16.0.76:8006/people/basiamc/art/732916-4-fire-fairy ", host:
requests before the overflow into proxy_buffers.
HTTP/1.1 200 OK
Should I continue to increase the buffer size? Should I allocate the
buffer in larger sizes? Is there an interaction with proxy_buffers and
gzip or the output buffer settings ?

No, these buffers should be enough.
Are you sure that nginx was successfully reconfigured ?
Is nginx -t OK ?

Could you create debug log of the sinlge connection ?

BTW I think that
proxy_buffer_size 16k;
proxy_buffers 32 16k;

3

A little of topic but you definitely should implement some pagination
for the comments.

Kiril

4

Thanks Igor I’ll apply those settings now and let you know how it goes

[root@xserve1 nginx]# grep buffer proxy.conf
proxy_buffering on;
proxy_buffer_size 16k;
proxy_buffers 32 16k;
client_body_buffer_size 128k;
proxy_busy_buffers_size 64k;
[root@xserve1 nginx]# nginx -t
2008/04/30 15:41:34 [info] 66631#0: the configuration file /opt/local/
etc/nginx/nginx.conf syntax is ok
2008/04/30 15:41:34 [info] 66631#0: the configuration file /opt/local/
etc/nginx/nginx.conf was tested successfully

Cheers

Dave