Forum: NGINX forwarding POST requests to upstream

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Michał Jaszczyk (Guest)
on 2009-04-06 14:08
(Received via mailing list)
Hey,

I work at a company where we use Nginx a lot and so far we are very
happy with it :). Now we want to add another feature to our website
and long story short we need to forward files that users send by POST
requests to backends. Now I know that there exists upload module for
Nginx, but it is a little bit not suitable for us because we would
like not to store request body on disk, but instead immediately
forward it to backend, even without storing the request body in
memory, i.e. initiate request to backend as soon as the request to
Nginx is started. Another gotcha is that the backend has to be chosen
in a way that might require sending one more request to determine the
backend.

In other words, this is what we would like to achieve:
1. User sends file to Nginx using POST.
2. Nginx sends request to an application server.
3. The application server returns address of one of backend servers
(something like X-Accel-Redirect, but for POST).
4. Nginx forwards user's request to the backend, but doesn't store
request body on disk or in memory.

There were some posts on the group regarding similar topics and it
looks like it can't be done in Nginx yet. So I will probably have to
write a module that combines functionality from upstream and upload
modules. Or maybe it can be done in some other way? I remember a
discussion about input filter chain a week or two ago. This may ease
the task, but the chain is not ready yet and I'm in a bit of hurry and
I guess I can't wait until it's ready.

What are your thoughts about this problem? Any comment or hint will be
appreciated :).

Cheers,

Michał Jaszczyk
Valery K. (Guest)
on 2009-04-06 19:15
(Received via mailing list)
----- Michał Jaszczyk <removed_email_address@domain.invalid> wrote:
> Nginx is started. Another gotcha is that the backend has to be chosen
> in a way that might require sending one more request to determine the
> backend.

[...]

> What are your thoughts about this problem? Any comment or hint will be
> appreciated :).

As far as I know nginx cannot pipe a request into upstream, since
probably there was no intention initially to do like this. Moreover it
is not trivial to implement from my point of view.
Michał Jaszczyk (Guest)
on 2009-04-06 19:55
(Received via mailing list)
> As far as I know nginx cannot pipe a request into upstream, since probably there was no 
intention initially to do like this. Moreover it is not trivial to implement from my point 
of view.

Does it mean that every POST request is saved to disk before it's
forwarded to a backend? If this is the case then, well, we will just
have to live with that, but this will degrade our performance
dramatically.

And how about the input filter chain you are working on? Will it allow
us to make a GET request in order to determine the backend to which we
will forward the original POST request?

Maybe you can think of another way of choosing the backend based on
what is inside POST body?

Regards,

Mike
Igor S. (Guest)
on 2009-04-06 20:56
(Received via mailing list)
On Mon, Apr 06, 2009 at 05:45:09PM +0200, Micha?? Jaszczyk wrote:

> > As far as I know nginx cannot pipe a request into upstream, since probably there was 
no intention initially to do like this. Moreover it is not trivial to implement from my 
point of view.
>
> Does it mean that every POST request is saved to disk before it's
> forwarded to a backend? If this is the case then, well, we will just
> have to live with that, but this will degrade our performance
> dramatically.

No, a request body is saved to a disk only if it is larger than
client_body_buffer_size:

http://wiki.nginx.org/NginxHttpCoreModule#client_b...
Michał Jaszczyk (Guest)
on 2009-04-06 21:55
(Received via mailing list)
> No, a request body is saved to a disk only if it is larger than
> client_body_buffer_size:
>
> http://wiki.nginx.org/NginxHttpCoreModule#client_b...

Wow, good to know :). How bad would it be to set it to, say, 50MB?
Let's assume that memory consumption is not an issue.

Regards,

Mike
Igor S. (Guest)
on 2009-04-07 11:16
(Received via mailing list)
On Mon, Apr 06, 2009 at 07:45:14PM +0200, Micha?? Jaszczyk wrote:

> > No, a request body is saved to a disk only if it is larger than
> > client_body_buffer_size:
> >
> > http://wiki.nginx.org/NginxHttpCoreModule#client_b...
>
> Wow, good to know :). How bad would it be to set it to, say, 50MB?
> Let's assume that memory consumption is not an issue.

Yes, you may set to 50m.
This topic is locked and can not be replied to.