Having issues with nginx / root captures (0.7.53)


#1

i have this, which seemed like it would work according to my attempt
just a week ago…

server {
listen 80;
index index.php index.html;
server_name ~^foo(.*?).bar.ssgisp.com$;
root /home/mike/web/foo$1;
include /etc/nginx/defaults.conf;
include /etc/nginx/development.conf;
include /etc/nginx/expires.conf;
location ~ .php$ {
fastcgi_pass 127.0.0.1:11001;
}
}

this is what the log shows in debug mode:

2009/04/30 14:43:25 [debug] 8702#0: *16 http script copy:
“/home/mike/web/foo”
2009/04/30 14:43:25 [debug] 8702#0: *16 http script capture: “html”
2009/04/30 14:43:25 [debug] 8702#0: *16 http filename:
“/home/mike/web/foohtml/index.html”
2009/04/30 14:43:25 [debug] 8702#0: *16 add cleanup: 0000000015092358
2009/04/30 14:43:25 [error] 8702#0: 16 open()
“/home/mike/web/foohtml/index.html” failed (13: Permission denied),
client: 134.134.139.71, server: ~^foo(.
?).mike.bar.com$, request:
“GET /index.html HTTP/1.1”, host: “foo2.mike.bar.com

non-debug:

2009/04/30 14:46:29 [crit] 8726#0: 17 stat()
“/home/mike/web/foo/index.html” failed (13: Permission denied),
client: 13.13.13.71, server: ~^foo(.
?).mike.bar.com$, request:
“GET /index.html HTTP/1.1”, host: “foo2.mike.bar.com
2009/04/30 14:46:29 [error] 8726#0: 17 open()
“/home/mike/web/foohtml/index.html” failed (13: Permission denied),
client: 13.13.13.71, server: ~^foo(.
?).mike.bar.com$, request:
“GET /index.html HTTP/1.1”, host: “foo2.mike.bar.com

All I want is

foo.mike.bar.com
foo2.mike.bar.com
foo-anything.mike.bar.com

to map to /home/mike/web/foo, /home/mike/web/foo2,
/home/mike/web/foo-anything, etc…

Am I missing something here? and why is it capturing ‘html’ in the
capture? I’m only creating a $1 from the server_name, is “html” a
default somewhere?


#2

On Thu, Apr 30, 2009 at 03:51:04PM -0700, Michael S. wrote:

    include /etc/nginx/expires.conf;

“/home/mike/web/foohtml/index.html”
“/home/mike/web/foo/index.html” failed (13: Permission denied),
foo2.mike.bar.com
foo-anything.mike.bar.com

to map to /home/mike/web/foo, /home/mike/web/foo2,
/home/mike/web/foo-anything, etc…

Am I missing something here? and why is it capturing ‘html’ in the
capture? I’m only creating a $1 from the server_name, is “html” a
default somewhere?

First, “~^foo(.?).bar.ssgisp.com$" will never match
foo2.mike.bar.com”.
Second, "~^foo(.
?).bar.com$” will capture “2.mike” with
foo2.mike.bar.com”.

As to
2009/04/30 14:43:25 [debug] 8702#0: *16 http script capture: “html”

Probably somewhere in

     include /etc/nginx/defaults.conf;
     include /etc/nginx/development.conf;
     include /etc/nginx/expires.conf;

this “html” is captured.

nginx configuration is declarative except ugly if/rewrite.
And these if/rewrite will run before the root variable substitution even
if root is described before if/rewrite:

     root /home/mike/web/foo$1;
     if (...)

#3

sorry, that was supposed to be bar.com - i just messed up substituting

2009/4/30 Igor S. removed_email_address@domain.invalid:

First, “~^foo(.?).bar.ssgisp.com$" will never match “foo2.mike.bar.com”.
Second, "~^foo(.
?).bar.com$” will capture “2.mike” with “foo2.mike.bar.com”.

You’re right though, something in the files is messing with my matching.

What is it in this file that is setting up some sort of capture?

For some reason this turns a

foo123.mike.bar.com into /home/mike/web/foo, not foo123

does -any- regular expression mess with the regexps

location ^~ /robots.txt {
auth_basic off;
root /etc/nginx/robots;
break;
}

if ($http_user_agent ~* googlebot) {
return 404;
break;
}

if ($http_user_agent ~* looksmart) {
return 404;
break;
}

if ($http_user_agent ~* crawl) {
return 404;
break;
}

if ($http_user_agent ~* robot) {
return 404;
break;
}

if ($http_user_agent ~* findlinks) {
return 404;
break;
}

if ($http_user_agent ~* infoseek) {
return 404;
break;
}

if ($http_user_agent ~* search) {
return 404;
break;
}


#4

On Thu, Apr 30, 2009 at 11:26:05PM -0700, Michael S. wrote:

    expires max;
    access_log off;

}

so … how can i use root captures but still have these other options?

make a location / {} block and then put the sub-includes in with it?

       server_name  ~...

       set  $name  $1;

       root  /....$name;

The “set” runs early as well as “if/rewrite”.


#5

2009/4/30 Igor S. removed_email_address@domain.invalid:

    include /etc/nginx/expires.conf;

this “html” is captured.

yep

cat expires.conf

location ~* .(jpg|jpeg|gif|css|png|js|ico|html)$ {
expires max;
access_log off;
}

so … how can i use root captures but still have these other options?

make a location / {} block and then put the sub-includes in with it?


#6

On Thu, Apr 30, 2009 at 11:16:42PM -0700, Michael S. wrote:

For some reason this turns a

foo123.mike.bar.com into /home/mike/web/foo, not foo123

does -any- regular expression mess with the regexps

location ^~ /robots.txt {
auth_basic off;
root /etc/nginx/robots;
break;

You do not “break” here. This is waste of CPU cycles.

}

if ($http_user_agent ~* search) {
return 404;
break;
}

The “break” after “return” costs nothing, but useless.
Also, it’ better to combine all check in single regex - it will be run
much faster:

if ($http_user_agent ~* “googlebot|looksmart|…”) {
return 404;
}


#7

2009/4/30 Igor S. removed_email_address@domain.invalid:

The “break” after “return” costs nothing, but useless.
Also, it’ better to combine all check in single regex - it will be run
much faster:

if ($http_user_agent ~* “googlebot|looksmart|…”) {
   return 404;
}

Thanks for the pointers. Originally I was doing it one at a time just
to keep it simple. I can clean that up.


#8

On Thu, Apr 30, 2009 at 11:53:16PM -0700, Michael S. wrote:

    listen 80;

include /etc/nginx/redfort.conf;

    expires max;

location = /robots.txt {
any ideas on how to get development.conf and defaults.conf to work
with the root regex? even with the “set” it still seems to not capture
it at the right time.

It should work, could you create debug log of request ?
The only “if/rewrite/set” order is important. If “set” is first, then
it should save the capture in $name.


#9

On Fri, May 01, 2009 at 10:54:49AM +0400, Igor S. wrote:

            fastcgi_pass 127.0.0.1:11001;

}

any ideas on how to get development.conf and defaults.conf to work
with the root regex? even with the “set” it still seems to not capture
it at the right time.

It should work, could you create debug log of request ?
The only “if/rewrite/set” order is important. If “set” is first, then
it should save the capture in $name.

As it was discovered, the configuration has no “location /”.

nginx runs if/set/rewrite scripts twice: once in server context, and
once in locaiton context. Any server has special empty location context.
If you have no “location /” nginx runs “set $name $1” again in location
context. This why the simple (even empty)

location / { }

is usefull.


#10

2009/4/30 Igor S. removed_email_address@domain.invalid:

     server_name  ~…

     set  $name  $1;

     root  /…$name;

server {
listen 80;
index index.php index.html;
server_name ~^foo(.+).mike.bar.com$;
set $name $1;
root /home/mike/web/foo$name;
include /etc/nginx/defaults.conf;

include /etc/nginx/development.conf;

include /etc/nginx/expires.conf;

    location ~ \.php$ {
            fastcgi_pass 127.0.0.1:11001;
    }

include /etc/nginx/redfort.conf;

}

like that?

you saw development.conf in the last email…

expires.conf (this causes an issue)

location ~* .(jpg|jpeg|gif|css|png|js|ico|html)$ {
expires max;
access_log off;
}

defaults.conf (this is okay):

location ~ /.ht {
deny all;
}

location = /robots.txt {
log_not_found off;
}

location = /favicon.ico {
log_not_found off;
}

log_not_found off;

any ideas on how to get development.conf and defaults.conf to work
with the root regex? even with the “set” it still seems to not capture
it at the right time.