Hello everybody:
When working with Fast CGI Module, I find that Nginx buffers the whole
request content before send it to the FastCGI Server.
For example, the client uploads a file of 10MB to Nginx, I notice that
the
backend Fast CGI server didn’t receive any request until
all the 10MB data has been upload to the Nginx.
I want to upload a very very large file like a stream and process it in
the
backend FastCGIServer as the request begins, how can I do
this with Nginx + FastCGIModule?
Thanks!!
Hello!
On Wed, Mar 17, 2010 at 11:54:33PM +0800, li zJay wrote:
this with Nginx + FastCGIModule?
No way.
Maxim D.
Why the Nginx has to buffer the whole request body before send it to
FastCGI
Server?
I tried to change the following configurations:
fastcgi_buffer_size
fastcgi_buffers
fastcgi_busy_buffers_size
fastcgi_temp_file_write_size
But it didn’t help.
Thanks!
On Thu, Mar 18, 2010 at 12:07 AM, Maxim D. [email protected]
wrote:
backend Fast CGI server didn’t receive any request until
nginx mailing list
[email protected]
nginx Info Page
zJay
Hello:
Why the Nginx has to buffer the whole request body before send it to
FastCGI Server?
I tried to change the following configurations:
fastcgi_buffer_size
fastcgi_buffers
fastcgi_busy_buffers_size
fastcgi_temp_file_write_size
But it didn’t help.
Thanks!
On Thu, Mar 18, 2010 at 12:07 AM, Maxim D. [email protected]
wrote:
backend Fast CGI server didn’t receive any request until
nginx mailing list
[email protected]
nginx Info Page
zJay
Hello!
On Thu, Mar 18, 2010 at 09:12:39AM +0800, li zJay wrote:
Why the Nginx has to buffer the whole request body before send it to FastCGI
Server?
By design.
I tried to change the following configurations:
fastcgi_buffer_size
fastcgi_buffers
fastcgi_busy_buffers_size
fastcgi_temp_file_write_size
But it didn’t help.
These are for response buffering.
Maxim D.
On Mar 19, 2010, at 2:25 AM, li zJay [email protected] wrote:
This is really a problem for those who want to use Nginx as a
file-upload server.
Why do you suggest that? This prevents your application worker
processes from being tied up waiting for uploads. If you need
something like upload progress there is a module for that
v/r,
Rob
Thanks, Maxim D.!
In fact I work with python, and I find that no matter “Nginx +
FastCGI” or “Nginx + Mod_Wsgi”, the nginx force to buffer the whole
client request body in either memory or file system.
This is really a problem for those who want to use Nginx as a
file-upload server.
On Thu, Mar 18, 2010 at 5:25 PM, Maxim D. [email protected]
wrote:
fastcgi_buffer_size
nginx mailing list
[email protected]
nginx Info Page
zJay
Thanks, Rob.
If the the raw upload file is to be saved in the local server, Nginx
has good performance.
But I want to compress the upload file and split it and then transfer
to another central storage system,
the file is not to be stored locally.
In fact, the problem is that, if there’s 1000 concurrent requests,
each request has 500MB post body,
It’s to comsume 500G temporary disk space, and it will take a lot of
time to process these data too.
So, I think that if there’s stream processing support, It will be
better under this application.
On Fri, Mar 19, 2010 at 10:56 PM, Rob S. [email protected]
wrote:
v/r,
Rob
–
zJay
On Mar 19, 2010, at 9:19 AM, li zJay [email protected] wrote:
So, I think that if there’s stream processing support, It will be
better under this application.
There is a mogilefs module
also you can turn proxy buffering off if you can implement http on
the backend instead of wsgi or fastcgi.
V/r,
Rob S.