Unsure about proxy_temp_file_write_size to prevent upstream response buffering

Server NGINX
1

For large dynamic responses generated by our application occasionally
they are being buffered to disk

2008/01/29 14:21:13 [warn] 19297#0: *12289266 an upstream response is
buffered to a temporary file /opt/local/var/run/nginx/proxy_temp/
3/50/0000123503 while reading upstream, client: 4.224.132.81, server:
www.redbubble.com
, request: “GET /people/redbubble/journal/206322-redbubble-post-card-
everyones-an-artist HTTP/1.1”, upstream:
"http://172.16.0.72:8005/people/redbubble/journal/206322-redbubble-post-card-everyones-an-artist
", host: “www.redbubble.com”

In this case the resource is

[dave@crimson ~]$ curl -I
http://www.redbubble.com/people/redbubble/journal/206322-redbubble-post-card-everyones-an-artist
HTTP/1.1 200 OK
Server: nginx
Date: Tue, 29 Jan 2008 03:22:08 GMT
Content-Type: text/html; charset=utf-8
Connection: keep-alive
Set-Cookie: _session_id=223812df7b7e519d8ca3b2a6540b7016; path=/
Cache-Control: private, max-age=0, must-revalidate
Content-Length: 73827

73827 bytes long (before compression)

I have

proxy_temp_file_write_size 128k

set to 128k, which I thought would buffer up to 128k of data in memory
before writing it to disk.

Cheers

Dave

2

Hello!

On Tue, Jan 29, 2008 at 02:27:47PM +1100, Dave C. wrote:

host: “www.redbubble.com”
Cache-Control: private, max-age=0, must-revalidate
Content-Length: 73827

73827 bytes long (before compression)

I have

proxy_temp_file_write_size 128k

set to 128k, which I thought would buffer up to 128k of data in memory
before writing it to disk.

You should configure proxy_buffer_size / proxy_buffers large enough
instead.

Directive proxy_temp_file_write_size basically tells nginx how
many data it can flush to disk at once when writing temporary
file. It may be used to prevent nginx worker process from blocking
for too long in io path.

Maxim D.

3

Hi Maxim,

Thanks for your response, I’ve set my proxy directives to

proxy_buffering on;
proxy_buffer_size 4k;
-proxy_buffers 4 32k;
+proxy_buffers 8 32k;

Which I’m guessing has 256k of buffering capacity, is that correct?

Cheers

Dave

4

Hi,

I have /usr/local/nginx/proxy_temp/

In it there are 9 folders: 1…9, and in each of them there are plenty
files 01…75

Is there anything with my setup, are these files going to grow in the
future? Will there be more files added?

5

Hello!

On Tue, Jan 29, 2008 at 11:11:05PM +1100, Dave C. wrote:

Hi Maxim,

Thanks for your response, I’ve set my proxy directives to

proxy_buffering on;
proxy_buffer_size 4k;
-proxy_buffers 4 32k;
+proxy_buffers 8 32k;

Which I’m guessing has 256k of buffering capacity, is that correct?

With this settings nginx should be able to buffer up to 260k (4k +
8*32k) of upstream response in memory. This number includes length
of headers returned by upstream.

Maxim D.