First let me show my understanding of ngnix working till now…
In my ngnix.conf - inside php-fastcgi handler block I have settings…
fastcgi_buffers 64 4k;
i.e. 256K buffer size (I am on 32-bit platform). This means that if php
scripts generate more than 256K size responses nginx will read first
256K sends it to a client and empties the buffer’s contents. If there’s
more responce data - it reads & sends again.
At this point when ngnix performs multiple reads does it writes to the
error_log a line like an “upstream response is buffered to a temporary
file /var/lib/nginx/fastcgi/”
Is this something to worry about?
As ngnix uses a function like below to allocate memory while sending PHP
script response to client
min(buffer_size,response_size)
If site often provides small script-generated response, large
buffer_size will waste a lot of memory.
But using large fastcgi_buffers removed warning “upstream response is
buffered to a temporary file /var/lib/nginx/fastcgi/” from error_logs.
I really lots of free RAM but I wanted to allocated buffers in most
optimized way and use rest of my free RAM to start more php-cgi child
processes (I think it will help me handle more traffic bursts)
Any inputs on this greatly appreciated.
Thanks,
-Rahul
Posted at Nginx Forum:
Thanks Igor for personally responding to my query and of-course for
ngnix 
Now I have few more questions…
In my ngnix.conf - inside php-fastcgi handler block I have settings…
fastcgi_buffers 64 4k;
i.e. 256K buffer size (I am on 32-bit platform). This means that if php scripts generate more than 256K size responses nginx will read first 256K sends it to a client and empties the buffer’s contents. If there’s more responce data - it reads & sends again.
At this point when ngnix performs multiple reads does it writes to the error_log a line like an “upstream response is buffered to a temporary file /var/lib/nginx/fastcgi/”
This depends on typical repsonse size.
So there is no way to avoid this warning - “upstream response is
buffered to a temporary file /var/lib/nginx/fastcgi/” ??
Since no matter how large buffer we set, this will exceed at some point
as in my case I am serving images and other “files” through PHP using
readfile() function.
If a response is 12k then “fastcgi_buffers 64 4k” will allocated 3 buffers of 4k.
This is why fastcgi_buffers has two parameters, while fastcgi_buffer_size (used for response header and first part) has just one paramter.
BTW, after fastcgi_buffer_size has been emptied, it used together with fastcgi_buffers.
Based on above info - if my average response size is 128K and max
response size is 32M and I’m on 32-bit platform,
then following must be most optimized configuration - It will use memory
efficiently as well as will not generate any warning?
fastcgi_buffer_size 128K
fastcgi_buffers 8192 4k
Am I right?
Posted at Nginx Forum:
Hi Igor,
All doubts are clear now.
Thanks.
Posted at Nginx Forum:
Since no matter how large buffer we set, this will exceed at some point as
in my case I am serving images and other “files” through PHP using
readfile() function.
In those cases, you should use X-Accel-Redirect.
–
Robot Parade
http://www.robotparade.com.au/
m. +61 423 989 818 b. http://bethesignal.org/ p. +61 2 9043
2940
Rob S. Wrote:
If you have control over the scripts instead of
using the php readfile() function wouldn’t it be
better to use nginx’s
http://wiki.nginx.org/NginxXSendfile to allow
nginx to serve the files directly without having
to pass them over fastcgi?
I did not know about http://wiki.nginx.org/NginxXSendfile.
It is exactly the same thing I wanted. I tried it after you gave hint to
me.
For some reason it didn’t work and resulted in 404 error.
Is it in nginx core or do I need to enable it from somewhere?
Posted at Nginx Forum:
By the way if possible can somebody please answer my original question
as well…
if my average response size is 128K and max response size is 32M and I’m on 32-bit platform,
then following must be most optimized configuration - It will use memory efficiently as well as will not generate any warning?
fastcgi_buffer_size 128K
fastcgi_buffers 8192 4k
Am I right?
I need to know because in some cases PHP scripts like phpMyAdmin
exporting a large database generates much bigger response.
Also in some other service I will be generating large XML output from
PHP files.
Posted at Nginx Forum:
On Thu, Nov 26, 2009 at 03:14:48AM -0500, rahul286 wrote:
First let me show my understanding of ngnix working till now…
In my ngnix.conf - inside php-fastcgi handler block I have settings…
fastcgi_buffers 64 4k;
i.e. 256K buffer size (I am on 32-bit platform). This means that if php scripts generate more than 256K size responses nginx will read first 256K sends it to a client and empties the buffer’s contents. If there’s more responce data - it reads & sends again.
At this point when ngnix performs multiple reads does it writes to the error_log a line like an “upstream response is buffered to a temporary file /var/lib/nginx/fastcgi/”
Is this something to worry about?
This depends on typical repsonse size.
As ngnix uses a function like below to allocate memory while sending PHP script response to client
min(buffer_size,response_size)
If site often provides small script-generated response, large buffer_size will waste a lot of memory.
But using large fastcgi_buffers removed warning “upstream response is buffered to a temporary file /var/lib/nginx/fastcgi/” from error_logs.
I really lots of free RAM but I wanted to allocated buffers in most optimized way and use rest of my free RAM to start more php-cgi child processes (I think it will help me handle more traffic bursts)
Any inputs on this greatly appreciated.
If a response is 12k then “fastcgi_buffers 64 4k” will allocated 3
buffers
of 4k. This is why fastcgi_buffers has two parameters, while
fastcgi_buffer_size (used for response header and first part) has just
one paramter. BTW, after fastcgi_buffer_size has been emptied, it used
together with fastcgi_buffers.
–
Igor S.
http://sysoev.ru/en/
So there is no way to avoid this warning - “upstream response is buffered to a temporary file /var/lib/nginx/fastcgi/” ??
Since no matter how large buffer we set, this will exceed at some point as in my case I am serving images and other “files” through PHP using readfile() function.
If you have control over the scripts instead of using the php readfile()
function wouldn’t it be better to use nginx’s
http://wiki.nginx.org/NginxXSendfile to allow nginx to serve the files
directly without having to pass them over fastcgi?
Rob
On Fri, Nov 27, 2009 at 01:33:35AM -0500, rahul286 wrote:
By the way if possible can somebody please answer my original question as well…
if my average response size is 128K and max response size is 32M and I’m on 32-bit platform,
then following must be most optimized configuration - It will use memory efficiently as well as will not generate any warning?
fastcgi_buffer_size 128K
fastcgi_buffers 8192 4k
Am I right?
Yes, a whole 32M response will be in memory, however, I do not think
that
is right thing. You should handle the most responses in memory and allow
to use disk for biggest ones:
fastcgi_buffer_size 4K;
fastcgi_buffers 64 4k;
This settings store responses up to 4K+64*4K=260K in memory.
I need to know because in some cases PHP scripts like phpMyAdmin exporting a large database generates much bigger response.
Also in some other service I will be generating large XML output from PHP files.
Posted at Nginx Forum: Re: Tweaking FastCGI buffers settings
nginx mailing list
[email protected]
nginx Info Page
–
Igor S.
http://sysoev.ru/en/
On Fri, Nov 27, 2009 at 02:32:58AM -0500, rahul286 wrote:
Yes, a whole 32M response will be in memory, however, I do not think that
is right thing. You should handle the most responses in memory and allow
to use disk for biggest ones:
I guess as of now nginx is doing this only.
I guess its safe to ignore warnings in error_log.
In optimal case, I will find less number of warnings i.e. " upstream response is buffered to a temporary file /var/lib/nginx/fastcgi/" messages.
Yes.
–
Igor S.
http://sysoev.ru/en/
I did not know about http://wiki.nginx.org/NginxXSendfile.
It is exactly the same thing I wanted. I tried it after you gave hint to me.
For some reason it didn’t work and resulted in 404 error.
Is it in nginx core or do I need to enable it from somewhere?
It’s in the nginx core. And when you are sending your url back make sure
its an absolute url for the server to serve the file. Not a path to the
file on the filesystem.
Rob
On Fri, Nov 27, 2009 at 01:28:08AM -0500, rahul286 wrote:
It is exactly the same thing I wanted. I tried it after you gave hint to me.
For some reason it didn’t work and resulted in 404 error.
Is it in nginx core or do I need to enable it from somewhere?
Yes, this is a core feature, you should look in error_log for 404
reason.
–
Igor S.
http://sysoev.ru/en/