Forum: NGINX Having issues with nginx / root captures (0.7.53)

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
Michael S. (Guest)
on 2009-05-01 02:57
(Received via mailing list)
i have this, which seemed like it would work according to my attempt
just a week ago...

server {
        listen 80;
        index index.php index.html;
        server_name ~^foo(.*?)\.bar\.ssgisp\.com$;
        root /home/mike/web/foo$1;
        include /etc/nginx/defaults.conf;
        include /etc/nginx/development.conf;
        include /etc/nginx/expires.conf;
        location ~ \.php$ {
                fastcgi_pass 127.0.0.1:11001;
        }
}

this is what the log shows in debug mode:

2009/04/30 14:43:25 [debug] 8702#0: *16 http script copy:
"/home/mike/web/foo"
2009/04/30 14:43:25 [debug] 8702#0: *16 http script capture: "html"
2009/04/30 14:43:25 [debug] 8702#0: *16 http filename:
"/home/mike/web/foohtml/index.html"
2009/04/30 14:43:25 [debug] 8702#0: *16 add cleanup: 0000000015092358
2009/04/30 14:43:25 [error] 8702#0: *16 open()
"/home/mike/web/foohtml/index.html" failed (13: Permission denied),
client: 134.134.139.71, server: ~^foo(.*?)\.mike\.bar\.com$, request:
"GET /index.html HTTP/1.1", host: "foo2.mike.bar.com"

non-debug:


2009/04/30 14:46:29 [crit] 8726#0: *17 stat()
"/home/mike/web/foo/index.html" failed (13: Permission denied),
client: 13.13.13.71, server: ~^foo(.*?)\.mike\.bar\.com$, request:
"GET /index.html HTTP/1.1", host: "foo2.mike.bar.com"
2009/04/30 14:46:29 [error] 8726#0: *17 open()
"/home/mike/web/foohtml/index.html" failed (13: Permission denied),
client: 13.13.13.71, server: ~^foo(.*?)\.mike\.bar\.com$, request:
"GET /index.html HTTP/1.1", host: "foo2.mike.bar.com"

All I want is

foo.mike.bar.com
foo2.mike.bar.com
foo-anything.mike.bar.com

to map to /home/mike/web/foo, /home/mike/web/foo2,
/home/mike/web/foo-anything, etc...

Am I missing something here? and why is it capturing 'html' in the
capture? I'm only creating a $1 from the server_name, is "html" a
default somewhere?
Igor S. (Guest)
on 2009-05-01 10:13
(Received via mailing list)
On Thu, Apr 30, 2009 at 03:51:04PM -0700, Michael S. wrote:

>         include /etc/nginx/expires.conf;
> "/home/mike/web/foohtml/index.html"
> "/home/mike/web/foo/index.html" failed (13: Permission denied),
> foo2.mike.bar.com
> foo-anything.mike.bar.com
>
> to map to /home/mike/web/foo, /home/mike/web/foo2,
> /home/mike/web/foo-anything, etc...
>
> Am I missing something here? and why is it capturing 'html' in the
> capture? I'm only creating a $1 from the server_name, is "html" a
> default somewhere?

First, "~^foo(.*?)\.bar\.ssgisp\.com$" will never match
"foo2.mike.bar.com".
Second, "~^foo(.*?)\.bar\.com$" will capture "2.mike" with
"foo2.mike.bar.com".

As to
 2009/04/30 14:43:25 [debug] 8702#0: *16 http script capture: "html"

Probably somewhere in

         include /etc/nginx/defaults.conf;
         include /etc/nginx/development.conf;
         include /etc/nginx/expires.conf;

this "html" is captured.

nginx configuration is declarative except ugly if/rewrite.
And these if/rewrite will run before the root variable substitution even
if root is described before if/rewrite:

         root /home/mike/web/foo$1;
         if (...)
Michael S. (Guest)
on 2009-05-01 10:25
(Received via mailing list)
sorry, that was supposed to be bar.com - i just messed up substituting

2009/4/30 Igor S. <removed_email_address@domain.invalid>:

> First, "~^foo(.*?)\.bar\.ssgisp\.com$" will never match "foo2.mike.bar.com".
> Second, "~^foo(.*?)\.bar\.com$" will capture "2.mike" with "foo2.mike.bar.com".

You're right though, something in the files is messing with my matching.

What is it in this file that is setting up some sort of capture?

For some reason this turns a

foo123.mike.bar.com into /home/mike/web/foo, not foo123

does -any- regular expression mess with the regexps

location ^~ /robots.txt {
        auth_basic off;
        root /etc/nginx/robots;
        break;
}

if ($http_user_agent ~* googlebot) {
        return 404;
        break;
}

if ($http_user_agent ~* looksmart) {
        return 404;
        break;
}

if ($http_user_agent ~* crawl) {
        return 404;
        break;
}

if ($http_user_agent ~* robot) {
        return 404;
        break;
}

if ($http_user_agent ~* findlinks) {
        return 404;
        break;
}

if ($http_user_agent ~* infoseek) {
        return 404;
        break;
}

if ($http_user_agent ~* search) {
        return 404;
        break;
}
Michael S. (Guest)
on 2009-05-01 10:33
(Received via mailing list)
2009/4/30 Igor S. <removed_email_address@domain.invalid>:

>         include /etc/nginx/expires.conf;
>
> this "html" is captured.

yep

# cat expires.conf
location ~* \.(jpg|jpeg|gif|css|png|js|ico|html)$ {
        expires max;
        access_log off;
}

so ... how can i use root captures but still have these other options?

make a location / {} block and then put the sub-includes in with it?
Igor S. (Guest)
on 2009-05-01 10:48
(Received via mailing list)
On Thu, Apr 30, 2009 at 11:26:05PM -0700, Michael S. wrote:

>         expires max;
>         access_log off;
> }
>
> so ... how can i use root captures but still have these other options?
>
> make a location / {} block and then put the sub-includes in with it?

           server_name  ~...

           set  $name  $1;

           root  /....$name;

The "set" runs early as well as "if/rewrite".
Igor S. (Guest)
on 2009-05-01 10:52
(Received via mailing list)
On Thu, Apr 30, 2009 at 11:16:42PM -0700, Michael S. wrote:

>
> For some reason this turns a
>
> foo123.mike.bar.com into /home/mike/web/foo, not foo123
>
> does -any- regular expression mess with the regexps
>
> location ^~ /robots.txt {
>         auth_basic off;
>         root /etc/nginx/robots;
>         break;

You do not "break" here. This is waste of CPU cycles.

> }
>
> if ($http_user_agent ~* search) {
>         return 404;
>         break;
> }

The "break" after "return" costs nothing, but useless.
Also, it' better to combine all check in single regex - it will be run
much faster:

if ($http_user_agent ~* "googlebot|looksmart|...") {
       return 404;
}
Michael S. (Guest)
on 2009-05-01 10:55
(Received via mailing list)
2009/4/30 Igor S. <removed_email_address@domain.invalid>:

> The "break" after "return" costs nothing, but useless.
> Also, it' better to combine all check in single regex - it will be run
> much faster:
>
> if ($http_user_agent ~* "googlebot|looksmart|...") {
>       return 404;
> }

Thanks for the pointers. Originally I was doing it one at a time just
to keep it simple. I can clean that up.
Michael S. (Guest)
on 2009-05-01 10:59
(Received via mailing list)
2009/4/30 Igor S. <removed_email_address@domain.invalid>:

>           server_name  ~...
>
>           set  $name  $1;
>
>           root  /....$name;


server {
        listen 80;
        index index.php index.html;
        server_name ~^foo(.+)\.mike\.bar\.com$;
        set $name $1;
        root /home/mike/web/foo$name;
        include /etc/nginx/defaults.conf;
#        include /etc/nginx/development.conf;
#       include /etc/nginx/expires.conf;
        location ~ \.php$ {
                fastcgi_pass 127.0.0.1:11001;
        }
#       include /etc/nginx/redfort.conf;
}

like that?

you saw development.conf in the last email...


expires.conf (this causes an issue)

location ~* \.(jpg|jpeg|gif|css|png|js|ico|html)$ {
        expires max;
        access_log off;
}


defaults.conf (this is okay):

location ~ /\.ht {
        deny all;
}

location = /robots.txt {
        log_not_found off;
}

location = /favicon.ico {
        log_not_found off;
}

log_not_found off;


any ideas on how to get development.conf and defaults.conf to work
with the root regex? even with the "set" it still seems to not capture
it at the right time.
Igor S. (Guest)
on 2009-05-01 11:08
(Received via mailing list)
On Thu, Apr 30, 2009 at 11:53:16PM -0700, Michael S. wrote:

>         listen 80;
> #       include /etc/nginx/redfort.conf;
>         expires max;
> location = /robots.txt {
> any ideas on how to get development.conf and defaults.conf to work
> with the root regex? even with the "set" it still seems to not capture
> it at the right time.

It should work, could you create debug log of request ?
The only "if/rewrite/set" order is important. If "set" is first, then
it should save the capture in $name.
Igor S. (Guest)
on 2009-05-01 11:25
(Received via mailing list)
On Fri, May 01, 2009 at 10:54:49AM +0400, Igor S. wrote:

> >
> >                 fastcgi_pass 127.0.0.1:11001;
> >
> > }
> >
> >
> > any ideas on how to get development.conf and defaults.conf to work
> > with the root regex? even with the "set" it still seems to not capture
> > it at the right time.
>
> It should work, could you create debug log of request ?
> The only "if/rewrite/set" order is important. If "set" is first, then
> it should save the capture in $name.

As it was discovered, the configuration has no "location /".

nginx runs if/set/rewrite scripts twice: once in server context, and
once in locaiton context. Any server has special empty location context.
If you have no "location /" nginx runs "set $name $1" again in location
context. This why the simple (even empty)

   location / { }

is usefull.
This topic is locked and can not be replied to.