Questions about proxy_pass and internal directives

[aris]

Hello,

My question is two-part:

1. given the following setup:

root /home/www/mysite/static;

location /foo/bar/ {
proxy_pass http://127.0.0.1:8080;
proxy_redirect off;
}

Why a request for “mysite.com/foo/bar/sth.html” does not get proxied to
the
server at “127.0.0.1:8080”? Instead, the file
“/home/www/mysite/static/foo/bar/sth.html” is served.

2. is there a way to use ‘internal’ directive together with ‘proxy_pass’
   for
   the same location? e.g.

root /home/www/mysite/static;

location /foo/bar/ {
proxy_pass http://127.0.0.1:8080;
proxy_redirect off;
internal;
}

I’ve tried with the above, but it seems ‘internal’ directive takes
precedence, and just return 404 immediately. What I want to achieve is
to
block direct access to files under “/home/www/mysite/static/foo/bar”,
and
proxy all requests to an application server which will decide the
access.

Any suggestion? Thank you.

Posted at Nginx Forum:

Questions about proxy_pass and internal directives

[mrtn]

A correction to my earlier post: the request in my first question above
should be for “mysite.com/foo/bar/sth/sth.html”, and there is a
corresponding file on the filesystem:
“/home/www/mysite/static/foo/bar/sth/sth.html”.

Just tried something different, with this config:

root /home/www/mysite/static;

location /foo/bar/ {
internal;
proxy_pass http://127.0.0.1:8080;
proxy_redirect off;
}

request to “mysite.com/foo/bar/sth” will return 404, and I can see that
the
request does not even get proxied to the application server. In
addition,
this access is not logged in error log but in the access log of nginx.
However, request to “mysite.com/foo/bar/sth/sth.html” is served fine,
despite of using “internal” directive.

Posted at Nginx Forum:

Re: Questions about proxy_pass and internal directives

[mrtn]

On Thursday 18 October 2012 23:54:43 mrtn wrote:

            internal;
            proxy_pass              http://127.0.0.1:8080;
            proxy_redirect          off;

}

request to “mysite.com/foo/bar/sth” will return 404, and I can see that the
request does not even get proxied to the application server. In addition,
this access is not logged in error log but in the access log of nginx.
However, request to “mysite.com/foo/bar/sth/sth.html” is served fine,
despite of using “internal” directive.

Is this the only one “location” that you have in your “server” block?

wbr, Valentin V. Bartenev

–

Technical Support for NGINX and NGINX Plus Software

http://nginx.org/en/donation.html

[mrtn]

Hello Valentin,

No, it is not.

I have a few other location blocks, but none of them should be matched
to
“/foo/bar”. For examples, i have:

location /foo/a/
location /foo/b/
location /foo/c/

also,

location ~* (.jpg|.png|.css|.js|.html)$ {
valid_referers none blocked www.mysite.com
static.mysite.com
mysite.com;
if ($invalid_referer) { return 405; }
}

Is there anything particular that will interfere with the “/foo/bar”
location block I should look for? Thank you.

Posted at Nginx Forum:

Re: Questions about proxy_pass and internal directives

[mrtn]

Hello Reinis,

Thanks for pointing that out. Just to make sure I understand the doc
correctly for my case.

Given that: location /foo/bar/ (directive with conventional strings) and
location ~* (.jpg|.png|.css|.js|.html)$ (regular expressions). The
/foo/bar/ location is actually checked and matched first, however, since
I
don’t have a ‘^~’ prefix in front of /foo/bar/, the search continues
with
regex match and matches with location ~*
(.jpg|.png|.css|.js|.html)$.
In the end, it is the regex location directive that is used by nginx.

So to correct the problem, I simply need to add a prefix ‘^~’ in front
of
/foo/bar/ to stop the search once matched.

Please correct me if I get anything wrong. Thanks.

Posted at Nginx Forum:

Re: Questions about proxy_pass and internal directives

[mrtn]

hmm, so adding ‘^~’ to the front of location /foo/bar/ makes “internal”
directive work correctly. All direct access to “/foo/bar/sth/sth.html”
are
blocked with 404 now.

However, the proxy_pass inside ‘/foo/bar/’ location still doesn’t work.
I
even put debugging echo inside all the location blocks, and only the
/foo/bar one appears in the log file. But there is still no request
proxied
to the other server. In the debugging log, I see:

2012/10/18 18:27:57 [debug] 6983#0: *3 http script if
2012/10/18 18:27:57 [debug] 6983#0: *3 http script if: false
2012/10/18 18:27:57 [debug] 6983#0: *3 test location: “/game/play/”
2012/10/18 18:27:57 [debug] 6983#0: *3 http finalize request: 404,
“/foo/bar/test-sth?” a:1, c:1
2012/10/18 18:27:57 [debug] 6983#0: *3 http special response: 404,
“/foo/bar/test-sth?”
2012/10/18 18:27:57 [debug] 6983#0: *3 internal redirect: “/404.html?”
2012/10/18 18:27:57 [debug] 6983#0: *3 rewrite phase: 1

Then I get 404 in the requesting browser obviously. Any other guess?

Posted at Nginx Forum:

Re: Questions about proxy_pass and internal directives

[mrtn]

hmm, so adding ‘^~’ to the front of location /foo/bar/ makes “internal”
directive work correctly. All direct access to “/foo/bar/sth/sth.html” are
blocked with 404 now.
However, the proxy_pass inside ‘/foo/bar/’ location still doesn’t work.

You are using ‘internal’ in a wrong way (at least judging from your
configuration excerpts).

If you read the documentation
Module ngx_http_core_module you
should
see that internal locations can’t be accessed directly from
client/browser
but need some sort of internal redirect.

If there is a need for a backend application to check for permissions
but
serve the file from nginx (while the same time denying direct access)
one
way to do it is making the backend application to send
‘X-Accel-Redirect’
header ( some examples: XSendfile | NGINX ).

You can also try auth_request_module by Maxim D. (
http://mdounin.ru/hg/ngx_http_auth_request_module/file/a29d74804ff1/README
/ [ANNOUNCE] auth request module )

… or access_by_lua from nginx_lua module by agentzh (
GitHub - openresty/lua-nginx-module: Embed the Power of Lua into NGINX HTTP servers )

rr

[mrtn]

Reinis R. Wrote:

If there is a need for a backend application to check for permissions
but
serve the file from nginx (while the same time denying direct access)
one
way to do it is making the backend application to send
‘X-Accel-Redirect’
header ( some examples: XSendfile | NGINX ).

That’s exactly what I am doing.

location ^~ /foo/bar/ {
internal;
proxy_pass http://127.0.0.1:8080;
proxy_redirect off;
}

I use “internal” directive to block direct access to anything
“/foo/bar/,”, which seems to be what nginx is doing. At the same time,
I
proxy_pass the request to the backend application server to check for
permissions. If success, the backend server sends a ‘X-Accel-Redirect’
header back to nginx to serve the file.

Now the problem is proxy_pass does not seem to work, as no request is
being
proxied to the backend server. I wondered if this is because I put it
together with “internal” directive inside the location block, but from
here:
X-Accel | NGINX, it seems that this combination should be
fine, as it is given as an example.

I’m scratching my head now…

Posted at Nginx Forum:

Re: Questions about proxy_pass and internal directives

[mrtn]

also,
location ~* (.jpg|.png|.css|.js|.html)$ {
valid_referers none blocked www.mysite.com
static.mysite.com

Nginx uses only one location and since your request is
“mysite.com/foo/bar/sth.html” it is matched by this regexp it is the one
used by nginx / so not really proxy_passed.

Module ngx_http_core_module - see
the
order.

rr

[mrtn]

On Friday 19 October 2012 04:08:21 mrtn wrote:

client/browser
header ( some examples: XSendfile | NGINX ).
“/foo/bar/,”, which seems to be what nginx is doing. At the same time, I
proxy_pass the request to the backend application server to check for
permissions. If success, the backend server sends a ‘X-Accel-Redirect’
header back to nginx to serve the file.
[…]

It seems you misunderstand the “location” directive.
http://nginx.org/en/docs/http/ngx_http_core_module.html#location

“Sets a configuration based on a request URI.”, that’s all. You protect
this
location (i.e. configuration, including proxy_pass and so on) from
external
requests.

wbr, Valentin V. Bartenev

–

Technical Support for NGINX and NGINX Plus Software

http://nginx.org/en/donation.html

[mrtn]

On a second thought, I think I get what you mean by “internal”
directive. It
will blocks ALL external requests from browsers, thus my browser request
to
“/foo/bar/sth” is block immediately and 404 is returned, without doing
any
proxy_pass.

I may need to rethink my design here. Ideally, I want users who request
“/foo/bar/sth” in their browsers get served by nginx with the file
“/foo/bar/sth/sth.html”, while letting the backend application server
control the access to the file. I may need to rethink the design here.
Thanks for the pointers.

Posted at Nginx Forum:

Re: Questions about proxy_pass and internal directives

[mrtn]

I use “internal” directive to block direct access to anything
“/foo/bar/,”, which seems to be what nginx is doing. At the same time, I
proxy_pass the request to the backend application server to check for
permissions. If success, the backend server sends a ‘X-Accel-Redirect’
header back to nginx to serve the file.
I may need to rethink my design here. Ideally, I want users who request
“/foo/bar/sth” in their browsers get served by nginx with the file
“/foo/bar/sth/sth.html”, while letting the backend application server
control the access to the file.

Well then you are doing it generally right, the only tricky part to
innitially understand is using different location blocks - one for the
proxy_pass and one for the protected files.
The example is shown also in the XSendfile wiki page.

• To really protect the files while not necessary you should keep them
  out
  of the default webroot.
• First you define the location you will be using as URLs on your
  website
  (there is no need for such directories or files to actually exist as all
  the
  requests will be sent to the backend for it to decide what to do next).

location /foo/bar {
proxy_pass http://127.0.0.1:8080;
proxy_redirect off;
}

• Second you define the location what will be used in the
  X-Accel-Redirect
  header sent from the backend server.

location /protected/ {
internal;
root /data/files;
#or alias /data/files/; - in case you want to leave the
‘/protected’
out of your physical data path.
}

1. Now if you request mysite.com/foo/bar/sth.html the request is sent
   the to
   backend ( http://127.0.0.1:8080/foo/bar/sth.html )
2. If the download is allowed (whatever logic the application
   implements)
   backend should respond with X-Accel-Redirect:
   /protected/foo/bar/sth.html
   ( you can change the directory tree or even the resulting file names as
   you
   wish / the only requirement is to leave the defined internal path (in
   this
   case ‘/protected’).
3. Depending on what you used (‘root’ or ‘alias’) in the protected
   location
   block a file from /data/files/protected/foo/bar/sth.html or
   /data/files/foo/bar/sth.html will be served by nginx.
   4 .Even if people discover the backend url or the X-Accel-Redirect
   header
   there is no way for them to acess the files directly since
   mysite.com/protected/foo/bar/sth.html wont work for them.

rr