Hey,
I work at a company where we use Nginx a lot and so far we are very
happy with it :). Now we want to add another feature to our website
and long story short we need to forward files that users send by POST
requests to backends. Now I know that there exists upload module for
Nginx, but it is a little bit not suitable for us because we would
like not to store request body on disk, but instead immediately
forward it to backend, even without storing the request body in
memory, i.e. initiate request to backend as soon as the request to
Nginx is started. Another gotcha is that the backend has to be chosen
in a way that might require sending one more request to determine the
backend.
In other words, this is what we would like to achieve:
- User sends file to Nginx using POST.
- Nginx sends request to an application server.
- The application server returns address of one of backend servers
(something like X-Accel-Redirect, but for POST).
- Nginx forwards user’s request to the backend, but doesn’t store
request body on disk or in memory.
There were some posts on the group regarding similar topics and it
looks like it can’t be done in Nginx yet. So I will probably have to
write a module that combines functionality from upstream and upload
modules. Or maybe it can be done in some other way? I remember a
discussion about input filter chain a week or two ago. This may ease
the task, but the chain is not ready yet and I’m in a bit of hurry and
I guess I can’t wait until it’s ready.
What are your thoughts about this problem? Any comment or hint will be
appreciated :).
Cheers,
Michał Jaszczyk
----- Michał Jaszczyk [email protected] wrote:
Nginx is started. Another gotcha is that the backend has to be chosen
in a way that might require sending one more request to determine the
backend.
[…]
What are your thoughts about this problem? Any comment or hint will be
appreciated :).
As far as I know nginx cannot pipe a request into upstream, since
probably there was no intention initially to do like this. Moreover it
is not trivial to implement from my point of view.
As far as I know nginx cannot pipe a request into upstream, since probably there was no intention initially to do like this. Moreover it is not trivial to implement from my point of view.
Does it mean that every POST request is saved to disk before it’s
forwarded to a backend? If this is the case then, well, we will just
have to live with that, but this will degrade our performance
dramatically.
And how about the input filter chain you are working on? Will it allow
us to make a GET request in order to determine the backend to which we
will forward the original POST request?
Maybe you can think of another way of choosing the backend based on
what is inside POST body?
Regards,
Mike
On Mon, Apr 06, 2009 at 05:45:09PM +0200, Micha?? Jaszczyk wrote:
As far as I know nginx cannot pipe a request into upstream, since probably there was no intention initially to do like this. Moreover it is not trivial to implement from my point of view.
Does it mean that every POST request is saved to disk before it’s
forwarded to a backend? If this is the case then, well, we will just
have to live with that, but this will degrade our performance
dramatically.
No, a request body is saved to a disk only if it is larger than
client_body_buffer_size:
http://wiki.nginx.org/NginxHttpCoreModule#client_body_buffer_size
No, a request body is saved to a disk only if it is larger than
client_body_buffer_size:
Module ngx_http_core_module
Wow, good to know :). How bad would it be to set it to, say, 50MB?
Let’s assume that memory consumption is not an issue.
Regards,
Mike
On Mon, Apr 06, 2009 at 07:45:14PM +0200, Micha?? Jaszczyk wrote:
No, a request body is saved to a disk only if it is larger than
client_body_buffer_size:
Module ngx_http_core_module
Wow, good to know :). How bad would it be to set it to, say, 50MB?
Let’s assume that memory consumption is not an issue.
Yes, you may set to 50m.