Question about proxy_buffers


#1

Hi all,

I’m not sure how proxy_buffers are used.

1- requests to the backend can use only one buffer. If the reponse is
bigger than one buffer, some part of the response is buffered to disk.
2- requests to the backen can use multiple buffers. If the response is
bigger than on buffer, nginx will use more buffers (if available of
course).

So depending on the answer, I would use :
1- big buffers so that 80% of the requests can feet in on buffer. And
enough buffers so that 80% of the requests can be buffered
2- small buffers and a lots of buffers

don’t know what to use :slight_smile: any clue ?

thx
++ jerome


#2

I’m not Igor so I cannot be totally sure, but my understanding of it
has always been 1- requests to the backend can use only one buffer and
the response gets buffered to disk. As such, I generally start out
sites with log level info (might be too low) and try to tune down
requests getting buffered if at all possible. I also try to stick
with magic numbers good for the architecture (so 8 and 16 based,
generally).

– Merlin

2009/6/9 Jérôme Loyet removed_email_address@domain.invalid:


#3

2009/6/10 merlin corey removed_email_address@domain.invalid:

I’m not Igor so I cannot be totally sure, but my understanding of it
has always been 1- requests to the backend can use only one buffer and
the response gets buffered to disk. As such, I generally start out
sites with log level info (might be too low) and try to tune down
requests getting buffered if at all possible. I also try to stick
with magic numbers good for the architecture (so 8 and 16 based,
generally).

Thx for the answer merlin,

I did some test. The request I make to nginx is proxied to the
backend. There is only one request made at the same time.
I request a 5Mo file.

First: with minimum buffers:
proxy_buffers 2 4k;

I have this in the log: 2009/06/11 19:40:05 [warn] 24607#0: *104 an
upstream response is buffered to a temporary file
/LIBRE/nginx/proxy_temp/0000000000 while reading upstream, …

Then I set 10 buffers of 1Mo:
proxy_buffers 10 1m;
and nothing appears in logs.

This tells me that a request can use more than one buffer.

I’ll set up small buffers but a large number:
proxy_buffers 2048 8k;

Is it too much ?


#4

2009/6/11 Jérôme Loyet removed_email_address@domain.invalid:

course).

Nice work! I guess for figuring out if it is too much you need to
look at your system and determine if it will be using too much memory,
but I would bet you can easily handle the 16MB + whatever structure
overhead for those 2048 buffers. Possibly other factors like paging
and caching (on CPU level) might come into play, too, but probably not
at our (human) granuality.


#5

Hello!

On Thu, Jun 11, 2009 at 08:00:04PM +0200, Jérôme Loyet wrote:

Then I set 10 buffers of 1Mo:
proxy_buffers 10 1m;
and nothing appears in logs.

This tells me that a request can use more than one buffer.

Yes.

I’ll set up small buffers but a large number:
proxy_buffers 2048 8k;

Is it too much ?

Each connection that uses proxy can use up to 16M of memory with
this setting (even after actual connection to backend has been
already closed). With 1024 client connections this means up to
16G of memory, probably a bit too many.

And keep in mind that:

  • Doing reconfiguration will spawn new worker processes and memory
    usage by nginx nearly doubles for a while.

  • You also need some memory for OS’s disk cache.

Some hints about seveal big vs. many small buffers:

  • Using bigger buffers may help a bit with CPU usage, but usually
    it’s not a big deal.

  • Using bigger buffers means less disk seeks if reply won’t fit
    into memory and will be buffered to disk.

  • On the other hand, small buffers allow better granularity (and
    less memory wasted for last partially filled buffer).

Since there are different load patterns (and different hardware)
it’s hard to recommend some particular setting, but the about
hints should allow you to tune it for your system appropriately.

Maxim D.


#6

2009/6/11 Jérôme Loyet removed_email_address@domain.invalid:

Do you mean that proxy_buffers are dedicated to each connection ?
I thought it was shared by all connections.

Positive, I read it as each connection gets that many buffers, in
which case, I agree with Maxim you MIGHT have too much :).


#7

2009/6/11 Maxim D. removed_email_address@domain.invalid:

with magic numbers good for the architecture (so 8 and 16 based,

Yes.

I’ll set up small buffers but a large number:
proxy_buffers 2048 8k;

Is it too much ?

Each connection that uses proxy can use up to 16M of memory with
this setting (even after actual connection to backend has been
already closed). With 1024 client connections this means up to
16G of memory, probably a bit too many.

Do you mean that proxy_buffers are dedicated to each connection ?
I thought it was shared by all connections.


#8

Hello!

On Fri, Jun 12, 2009 at 12:30:58AM +0200, Jérôme Loyet wrote:

[…]

Do you mean that proxy_buffers are dedicated to each connection ?
I thought it was shared by all connections.

Directive proxy_buffers specify maximum number and size of buffers
that can be used by one (each) connection. Not every connection
uses so many buffers as they are allocated on demand, but every
connection can do so.

Original documentation clearly states “… number and size of
buffers for one connection …” (in russian):

http://sysoev.ru/nginx/docs/http/ngx_http_proxy_module.html#proxy_buffers

Probably this was lost in translation (or was clarified later).
Feel free to fix wiki.

Maxim D.


#9

2009/6/12 Maxim D. removed_email_address@domain.invalid:

uses so many buffers as they are allocated on demand, but every
connection can do so.

Original documentation clearly states “… number and size of
buffers for one connection …” (in russian):

http://sysoev.ru/nginx/docs/http/ngx_http_proxy_module.html#proxy_buffers

Probably this was lost in translation (or was clarified later).
Feel free to fix wiki.

OK Thanks you very much. In this case 16Mo (2048 * 8k is way way too
much)

I’ll analyse my logs to know what is the distribution of my requests
size so that I can do the better compromise. Thx you guys


#10

Le 12 juin 2009 07:45, Jérôme Loyetremoved_email_address@domain.invalid a écrit :

Is it too much ?
that can be used by one (each) connection. Not every connection

OK Thanks you very much. In this case 16Mo (2048 * 8k is way way too much)

I’ll analyse my logs to know what is the distribution of my requests
size so that I can do the better compromise. Thx you guys

After analysing my logs. I have found out:

1- on the dynamic part of the site : more than 80% of requests are
less or equal than 32ko
2- on the static part of the site : more than 80% of requests are less
or equal than 12ko

So I’ll set this configuration.
proxy_buffers 4 8k; #32Ko to handle about 88% of static requests
proxy_buffers 4 4k; #16Ko to handle about 90% of dynamic requests

it seems more secure :slight_smile: what do you think ?