# The request buffer in Nginx Http FcgiModule **URL:** \ **Category:** NGINX\ **Created:** [March 17, 2010, 4:54pm UTC](https://www.ruby-forum.com/t/the-request-buffer-in-nginx-http-fcgimodule/185296 "2010-03-17T16:54:54Z")\ **Posts on this page:** 9\ **Page:** 1
**Author:** ![li\_zJay](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/8baadc/32.png) [@li\_zJay](https://www.ruby-forum.com/u/li_zJay)\ **Post date:** [March 17, 2010, 4:54pm UTC](https://www.ruby-forum.com/t/the-request-buffer-in-nginx-http-fcgimodule/185296/1 "2010-03-17T16:54:54Z")
Hello everybody: When working with Fast CGI Module, I find that Nginx buffers the whole request content before send it to the FastCGI Server. For example, the client uploads a file of 10MB to Nginx, I notice that the backend Fast CGI server didn’t receive any request until all the 10MB data has been upload to the Nginx. I want to upload a very very large file like a stream and process it in the backend FastCGIServer as the request begins, how can I do this with Nginx + FastCGIModule? Thanks!! ---
**Author:** ![li\_zJay](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/8baadc/32.png) [@li\_zJay](https://www.ruby-forum.com/u/li_zJay)\ **Post date:** [March 17, 2010, 5:07pm UTC](https://www.ruby-forum.com/t/the-request-buffer-in-nginx-http-fcgimodule/185296/2 "2010-03-17T17:07:47Z")
Hello! On Wed, Mar 17, 2010 at 11:54:33PM +0800, li zJay wrote: > this with Nginx + FastCGIModule? > No way. Maxim D. ---
**Author:** ![li\_zJay](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/8baadc/32.png) [@li\_zJay](https://www.ruby-forum.com/u/li_zJay)\ **Post date:** [March 18, 2010, 2:13am UTC](https://www.ruby-forum.com/t/the-request-buffer-in-nginx-http-fcgimodule/185296/3 "2010-03-18T02:13:17Z")
Why the Nginx has to buffer the whole request body before send it to FastCGI Server? I tried to change the following configurations: fastcgi\_buffer\_size fastcgi\_buffers fastcgi\_busy\_buffers\_size fastcgi\_temp\_file\_write\_size But it didn’t help. Thanks! On Thu, Mar 18, 2010 at 12:07 AM, Maxim D. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > > backend Fast CGI server didn’t receive any request until > > * * * > > nginx mailing list > [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > [mailman.nginx.org archive](http://nginx.org/mailman/listinfo/nginx) zJay ---
**Author:** ![li\_zJay](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/8baadc/32.png) [@li\_zJay](https://www.ruby-forum.com/u/li_zJay)\ **Post date:** [March 18, 2010, 2:14am UTC](https://www.ruby-forum.com/t/the-request-buffer-in-nginx-http-fcgimodule/185296/4 "2010-03-18T02:14:27Z")
Hello: Why the Nginx has to buffer the whole request body before send it to FastCGI Server? I tried to change the following configurations: fastcgi\_buffer\_size fastcgi\_buffers fastcgi\_busy\_buffers\_size fastcgi\_temp\_file\_write\_size But it didn’t help. Thanks! On Thu, Mar 18, 2010 at 12:07 AM, Maxim D. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > > backend Fast CGI server didn’t receive any request until > > * * * > > nginx mailing list > [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > [mailman.nginx.org archive](http://nginx.org/mailman/listinfo/nginx) zJay ---
**Author:** ![li\_zJay](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/8baadc/32.png) [@li\_zJay](https://www.ruby-forum.com/u/li_zJay)\ **Post date:** [March 18, 2010, 10:26am UTC](https://www.ruby-forum.com/t/the-request-buffer-in-nginx-http-fcgimodule/185296/5 "2010-03-18T10:26:03Z")
Hello! On Thu, Mar 18, 2010 at 09:12:39AM +0800, li zJay wrote: > Why the Nginx has to buffer the whole request body before send it to FastCGI > Server? By design. > I tried to change the following configurations: > fastcgi\_buffer\_size > fastcgi\_buffers > fastcgi\_busy\_buffers\_size > fastcgi\_temp\_file\_write\_size > > But it didn’t help. These are for response buffering. Maxim D. ---
**Author:** ![li\_zJay](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/8baadc/32.png) [@li\_zJay](https://www.ruby-forum.com/u/li_zJay)\ **Post date:** [March 19, 2010, 3:58pm UTC](https://www.ruby-forum.com/t/the-request-buffer-in-nginx-http-fcgimodule/185296/6 "2010-03-19T15:58:03Z")
On Mar 19, 2010, at 2:25 AM, li zJay [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > This is really a problem for those who want to use Nginx as a > file-upload server. Why do you suggest that? This prevents your application worker processes from being tied up waiting for uploads. If you need something like upload progress there is a module for that > **[GitHub - nginxinc/nginx-wiki: ARCHIVED -- Source for the now archived NGINX Wiki...](https://github.com/nginxinc/nginx-wiki)** > > ARCHIVED -- Source for the now archived NGINX Wiki section of https://www.nginx.com v/r, Rob ---
**Author:** ![li\_zJay](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/8baadc/32.png) [@li\_zJay](https://www.ruby-forum.com/u/li_zJay)\ **Post date:** [March 19, 2010, 9:26am UTC](https://www.ruby-forum.com/t/the-request-buffer-in-nginx-http-fcgimodule/185296/7 "2010-03-19T09:26:25Z")
Thanks, Maxim D.! In fact I work with python, and I find that no matter “Nginx + FastCGI” or “Nginx + Mod\_Wsgi”, the nginx force to buffer the whole client request body in either memory or file system. This is really a problem for those who want to use Nginx as a file-upload server. On Thu, Mar 18, 2010 at 5:25 PM, Maxim D. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > > ``` > > fastcgi_buffer_size > > > > ``` > > * * * > > nginx mailing list > [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > [mailman.nginx.org archive](http://nginx.org/mailman/listinfo/nginx) zJay ---
**Author:** ![li\_zJay](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/8baadc/32.png) [@li\_zJay](https://www.ruby-forum.com/u/li_zJay)\ **Post date:** [March 19, 2010, 5:20pm UTC](https://www.ruby-forum.com/t/the-request-buffer-in-nginx-http-fcgimodule/185296/8 "2010-03-19T17:20:02Z")
Thanks, Rob. If the the raw upload file is to be saved in the local server, Nginx has good performance. But I want to compress the upload file and split it and then transfer to another central storage system, the file is not to be stored locally. In fact, the problem is that, if there’s 1000 concurrent requests, each request has 500MB post body, It’s to comsume 500G temporary disk space, and it will take a lot of time to process these data too. So, I think that if there’s stream processing support, It will be better under this application. On Fri, Mar 19, 2010 at 10:56 PM, Rob S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > v/r, > Rob – zJay ---
**Author:** ![li\_zJay](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/8baadc/32.png) [@li\_zJay](https://www.ruby-forum.com/u/li_zJay)\ **Post date:** [March 19, 2010, 6:39pm UTC](https://www.ruby-forum.com/t/the-request-buffer-in-nginx-http-fcgimodule/185296/9 "2010-03-19T18:39:04Z")
On Mar 19, 2010, at 9:19 AM, li zJay [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > So, I think that if there’s stream processing support, It will be > better under this application. There is a mogilefs module > **[GitHub - nginxinc/nginx-wiki: ARCHIVED -- Source for the now archived NGINX Wiki...](https://github.com/nginxinc/nginx-wiki)** > > ARCHIVED -- Source for the now archived NGINX Wiki section of https://www.nginx.com also you can turn proxy buffering off if you can implement http on the backend instead of wsgi or fastcgi. V/r, Rob S.