Forum: NGINX Proxy without buffering

Hello,

Large POST-request are buffered to disk, before passed to the backend.
The backend has troubles to parse the POST-data, if the requests are 
huge
(some GB).

We use "haproxy" in front of "nginx", to workaround this issue.
But this causes extra load on the server.
Is there a chance to disable the request buffering?

Thanks a lot
Markus

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,236568,236568#msg-236568

On Mon, Feb 25, 2013 at 04:39:53AM -0500, double wrote:
> Hello,
>
> Large POST-request are buffered to disk, before passed to the backend.
> The backend has troubles to parse the POST-data, if the requests are huge
> (some GB).
>
> We use "haproxy" in front of "nginx", to workaround this issue.
> But this causes extra load on the server.
> Is there a chance to disable the request buffering?
>

Yep, it's possible, see the other active threads for a patch from 
tengine
to implement non-buffered uploads.

Also it'd be very nice to get this feature to standard nginx,
many people need it!

-- Pasi

Thanks a lot!

> Also it'd be very nice to get this feature to standard nginx,
> many people need it!

100% true.
I hate this workaround via "haproxy"

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,236568,236576#msg-236576

Hello,

I have two questions.

1. why does backend has troubles to parse request? because parsing huge
request make heavy load?
2. why haproxy in front nginx can workaround the issue? I think nginx 
will
still buffer the request to temp file

Thanks a lot

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,236568,236649#msg-236649

Example:
Size of POST-request: 10 GB (e.g. HD video)
Upload-time: 4h
The front-server proxies the request to the upload-server.
After the upload has finished, the uploading client wants a quick "all
fine"-message (maximum 30 seconds).

If the upload-server gets the stream directly, the machine has has 4h to
parse the request (e.g. "upload-module" or "apache-fastcgi-module"). And
this does not overload the system.

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,236568,236656#msg-236656

Hello,

      thanks for your reply. but i am still confused with the workground
about haproxy.
      i know haproxy dose not buffer request,  but if use haproxy in 
front
of nginx,
      when haproxy pass request to nginx, nginx will still buffer the
request to disk

Thanks a lot

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,236568,236665#msg-236665

haproxy does not buffer the request - it proxies immediately.

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,236568,236668#msg-236668

Hello,

      as you said, you use haproxy in front of nginx, that means the 
request
will be proxied by haproxy first, and then be proxied by nginx,
although haproxy can proxy request immediately, but nginx can not pass 
the
request on-the-fly,  it will buffer request anagin. Have i made
some mistakes?

thanks a lot

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,236568,236695#msg-236695

Machine ""front-end"":
haproxy (port 80) -> proxies to "port 8080" and "machine upload"
nginx (port 8080)

Machine "upload":
nginx (port 80) via "front-end"

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,236568,236699#msg-236699