Forum: NGINX Questions about proxy_pass and internal directives

Hello,

My question is two-part:

1. given the following setup:

root    /home/www/mysite/static;

location /foo/bar/ {
    proxy_pass        http://127.0.0.1:8080;
    proxy_redirect    off;
}

Why a request for "mysite.com/foo/bar/sth.html" does not get proxied to 
the
server at "127.0.0.1:8080"? Instead, the file
"/home/www/mysite/static/foo/bar/sth.html" is served.

2. is there a way to use 'internal' directive together with 'proxy_pass' 
for
the same location? e.g.

root    /home/www/mysite/static;

location /foo/bar/ {
    proxy_pass        http://127.0.0.1:8080;
    proxy_redirect    off;
    internal;
}

I've tried with the above, but it seems 'internal' directive takes
precedence, and just return 404 immediately. What I want to achieve is 
to
block direct access to files under "/home/www/mysite/static/foo/bar", 
and
proxy all requests to an application server which will decide the 
access.

Any suggestion? Thank you.

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,231998,231998#msg-231998

A correction to my earlier post: the request in my first question above
should be for "mysite.com/foo/bar/sth/sth.html", and there is a
corresponding file on the filesystem:
"/home/www/mysite/static/foo/bar/sth/sth.html".

Just tried something different, with this config:

root /home/www/mysite/static;

location /foo/bar/ {
                internal;
                proxy_pass              http://127.0.0.1:8080;
                proxy_redirect          off;
}

request to "mysite.com/foo/bar/sth" will return 404, and I can see that 
the
request does not even get proxied to the application server. In 
addition,
this access is not logged in error log but in the access log of nginx.
However, request to "mysite.com/foo/bar/sth/sth.html" is served fine,
despite of using "internal" directive.

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,231998,231999#msg-231999

On Thursday 18 October 2012 23:54:43 mrtn wrote:
>                 internal;
>                 proxy_pass              http://127.0.0.1:8080;
>                 proxy_redirect          off;
> }
>
> request to "mysite.com/foo/bar/sth" will return 404, and I can see that the
> request does not even get proxied to the application server. In addition,
> this access is not logged in error log but in the access log of nginx.
> However, request to "mysite.com/foo/bar/sth/sth.html" is served fine,
> despite of using "internal" directive.
>

Is this the only one "location" that you have in your "server" block?

 wbr, Valentin V. Bartenev

--
http://nginx.com/support.html
http://nginx.org/en/donation.html

Hello Valentin,

No, it is not.

I have a few other location blocks, but none of them should be matched 
to
"/foo/bar". For examples, i have:

location /foo/a/
location /foo/b/
location /foo/c/

also,

location ~* (\.jpg|\.png|\.css|\.js|\.html)$ {
                valid_referers none blocked www.mysite.com 
static.mysite.com
mysite.com;
                if ($invalid_referer) { return 405; }
}

Is there anything particular that will interfere with the "/foo/bar"
location block I should look for? Thank you.

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,231998,232001#msg-232001

> also,
> location ~* (\.jpg|\.png|\.css|\.js|\.html)$ {
>                 valid_referers none blocked www.mysite.com
> static.mysite.com

Nginx uses only one location and since your request is
"mysite.com/foo/bar/sth.html" it is matched by this regexp it is the one
used by nginx / so not really proxy_passed.

http://nginx.org/en/docs/http/ngx_http_core_module... - see 
the
order.


rr

Hello Reinis,

Thanks for pointing that out. Just to make sure I understand the doc
correctly for my case.

Given that: location /foo/bar/ (directive with conventional strings) and
location ~* (\.jpg|\.png|\.css|\.js|\.html)$ (regular expressions). The
/foo/bar/ location is actually checked and matched first, however, since 
I
don't have a '^~' prefix in front of /foo/bar/, the search continues 
with
regex match and matches with location ~* 
(\.jpg|\.png|\.css|\.js|\.html)$.
In the end, it is the regex location directive that is used by nginx.

So to correct the problem, I simply need to add a prefix '^~' in front 
of
/foo/bar/ to stop the search once matched.

Please correct me if I get anything wrong. Thanks.

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,231998,232005#msg-232005

hmm, so adding '^~' to the front of location /foo/bar/ makes "internal"
directive work correctly. All direct access to "/foo/bar/sth/sth.html" 
are
blocked with 404 now.

However, the proxy_pass inside '/foo/bar/' location still doesn't work. 
I
even put debugging echo inside all the location blocks, and only the
/foo/bar one appears in the log file. But there is still no request 
proxied
to the other server. In the debugging log, I see:

2012/10/18 18:27:57 [debug] 6983#0: *3 http script if
2012/10/18 18:27:57 [debug] 6983#0: *3 http script if: false
2012/10/18 18:27:57 [debug] 6983#0: *3 test location: "/game/play/"
2012/10/18 18:27:57 [debug] 6983#0: *3 http finalize request: 404,
"/foo/bar/test-sth?" a:1, c:1
2012/10/18 18:27:57 [debug] 6983#0: *3 http special response: 404,
"/foo/bar/test-sth?"
2012/10/18 18:27:57 [debug] 6983#0: *3 internal redirect: "/404.html?"
2012/10/18 18:27:57 [debug] 6983#0: *3 rewrite phase: 1

Then I get 404 in the requesting browser obviously. Any other guess?

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,231998,232006#msg-232006

> hmm, so adding '^~' to the front of location /foo/bar/ makes "internal"
> directive work correctly. All direct access to "/foo/bar/sth/sth.html" are
> blocked with 404 now.
> However, the proxy_pass inside '/foo/bar/' location still doesn't work.

You are using 'internal' in a wrong way (at least judging from your
configuration excerpts).

If you read the documentation
http://nginx.org/en/docs/http/ngx_http_core_module... you 
should
see that internal locations can't be accessed directly from 
client/browser
but need some sort of _internal_ redirect.



If there is a need for a backend application to check for permissions 
but
serve the file from nginx (while the same time denying direct access) 
one
way to do it is making the backend application to send 
'X-Accel-Redirect'
header ( some examples: http://wiki.nginx.org/XSendfile ).

You can also try auth_request_module by Maxim Dounin (
http://mdounin.ru/hg/ngx_http_auth_request_module/...
/  http://forum.nginx.org/read.php?2,58047,58047#msg-58047 )

.. or access_by_lua from nginx_lua module by agentzh (
https://github.com/chaoslawful/lua-nginx-module )


rr

Reinis Rozitis Wrote:
-------------------------------------------------------
>
>
> If there is a need for a backend application to check for permissions
> but
> serve the file from nginx (while the same time denying direct access)
> one
> way to do it is making the backend application to send
> 'X-Accel-Redirect'
> header ( some examples: http://wiki.nginx.org/XSendfile ).

That's exactly what I am doing.

location ^~ /foo/bar/ {
                internal;
                proxy_pass              http://127.0.0.1:8080;
                proxy_redirect         off;
}

I use "internal" directive to block direct access to anything
"/foo/bar/,,,", which seems to be what nginx is doing. At the same time, 
I
proxy_pass the request to the backend application server to check for
permissions. If success, the backend server sends a 'X-Accel-Redirect'
header back to nginx to serve the file.

Now the problem is proxy_pass does not seem to work, as no request is 
being
proxied to the backend server. I wondered if this is because I put it
together with "internal" directive inside the location block, but from 
here:
http://wiki.nginx.org/X-accel, it seems that this combination should be
fine, as it is given as an example.

I'm scratching my head now...

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,231998,232008#msg-232008

On a second thought, I think I get what you mean by "internal" 
directive. It
will blocks ALL external requests from browsers, thus my browser request 
to
"/foo/bar/sth" is block immediately and 404 is returned, without doing 
any
proxy_pass.

I may need to rethink my design here. Ideally, I want users who request
"/foo/bar/sth" in their browsers get served by nginx with the file
"/foo/bar/sth/sth.html", while letting the backend application server
control the access to the file. I may need to rethink the design here.
Thanks for the pointers.

Posted at Nginx Forum: 
http://forum.nginx.org/read.php?2,231998,232009#msg-232009

On Friday 19 October 2012 04:08:21 mrtn wrote:
> > client/browser
> > header ( some examples: http://wiki.nginx.org/XSendfile ).
> "/foo/bar/,,,", which seems to be what nginx is doing. At the same time, I
> proxy_pass the request to the backend application server to check for
> permissions. If success, the backend server sends a 'X-Accel-Redirect'
> header back to nginx to serve the file.
> [...]

It seems you misunderstand the "location" directive.
http://nginx.org/en/docs/http/ngx_http_core_module...

"Sets a configuration based on a request URI.", that's all. You protect 
this
location (i.e. configuration, including proxy_pass and so on) from 
external
requests.

 wbr, Valentin V. Bartenev

--
http://nginx.com/support.html
http://nginx.org/en/donation.html

> I use "internal" directive to block direct access to anything
> "/foo/bar/,,,", which seems to be what nginx is doing. At the same time, I
> proxy_pass the request to the backend application server to check for
> permissions. If success, the backend server sends a 'X-Accel-Redirect'
> header back to nginx to serve the file.
> I may need to rethink my design here. Ideally, I want users who request
> "/foo/bar/sth" in their browsers get served by nginx with the file
> "/foo/bar/sth/sth.html", while letting the backend application server
> control the access to the file.

Well then you are doing it generally right, the only tricky part to
innitially understand is using different location blocks - one for the
proxy_pass and one for the protected files.
The example is shown also in the XSendfile wiki page.

- To really protect the files while not necessary you should keep them 
out
of the default webroot.
- First you define the location you will be using as URLs on your 
website
(there is no need for such directories or files to actually exist as all 
the
requests will be sent to the backend for it to decide what to do next).

location /foo/bar {
                proxy_pass              http://127.0.0.1:8080;
                proxy_redirect         off;
}


- Second you define the location what will be used in the 
X-Accel-Redirect
header sent from the backend server.

location /protected/ {
    internal;
    root /data/files;
    #or alias  /data/files/; - in case you want to leave the 
'/protected'
out of your physical data path.
}


1. Now if you request mysite.com/foo/bar/sth.html the request is sent 
the to
backend (   http://127.0.0.1:8080/foo/bar/sth.html )
2. If the download is allowed (whatever logic the application 
implements)
backend should respond with X-Accel-Redirect: 
/protected/foo/bar/sth.html
( you can change the directory tree or even the resulting file names as 
you
wish / the only requirement is to leave the defined internal path (in 
this
case '/protected').
3. Depending on what you used ('root' or 'alias') in the protected 
location
block a file from /data/files/protected/foo/bar/sth.html  or
/data/files/foo/bar/sth.html  will be served by nginx.
4 .Even if people discover the backend url or the  X-Accel-Redirect 
header
there is no way for them to acess the files directly since
mysite.com/protected/foo/bar/sth.html wont work for them.


rr