# Re: loading a different robots.txt file for a different sub domain? (Ray) **URL:** \ **Category:** NGINX\ **Created:** [May 22, 2010, 4:13am UTC](https://www.ruby-forum.com/t/re-loading-a-different-robots-txt-file-for-a-different-sub-domain-ray/189027 "2010-05-22T04:13:55Z")\ **Posts on this page:** 2\ **Page:** 1
**Author:** ![zhys99](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/z/c77e96/32.png) [@zhys99](https://www.ruby-forum.com/u/zhys99)\ **Post date:** [May 22, 2010, 4:13am UTC](https://www.ruby-forum.com/t/re-loading-a-different-robots-txt-file-for-a-different-sub-domain-ray/189027/1 "2010-05-22T04:13:55Z")
load different robots in the same one “server” block location ~ /robots.txt { if ($host = ‘first.domain’) { rewrite ^/robots.txt /path/to/another/robots.txt last; } } zhys9 ------------------ Original ------------------ From: "nginx-request"[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid); Date: Sat, May 22, 2010 01:33 AM To: "nginx"[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid); Subject: nginx Digest, Vol 7, Issue 47 Send nginx mailing list submissions to removed\_email\_address@domain.invalid To subscribe or unsubscribe via the World Wide Web, visit [nginx Info Page](http://nginx.org/mailman/listinfo/nginx) or, via email, send a message with subject or body ‘help’ to removed\_email\_address@domain.invalid You can reach the person managing the list at removed\_email\_address@domain.invalid When replying, please edit your Subject line so it is more specific than “Re: Contents of nginx digest…” Today’s Topics: 1. Re: loading a different robots.txt file for a different sub domain? (Ray) 2. Re: loading a different robots.txt file for a different sub domain? (Ray) 3. Re: loading a different robots.txt file for a different sub domain? (Ilan B.) 4. Re: loading a different robots.txt file for a different sub domain? (Igor S.) 5. nginx 0day exploit for nginx + fastcgi PHP (Avleen Vig) 6. Re: nginx 0day exploit for nginx + fastcgi PHP (Avleen Vig) 7. Re: nginx 0day exploit for nginx + fastcgi PHP (Michael S.) 8. Re: nginx 0day exploit for nginx + fastcgi PHP (Michael S.) 9. Re: nginx 0day exploit for nginx + fastcgi PHP (Igor S.) * * * Message: 1 Date: Fri, 21 May 2010 21:53:45 +0800 From: Ray [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) To: removed\_email\_address@domain.invalid Cc: removed\_email\_address@domain.invalid Subject: Re: loading a different robots.txt file for a different sub domain? Message-ID: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Content-Type: text/plain; charset=UTF-8 Yes. server { listen 80 default; server\_name www; ``` location /robots.txt { alias /path/to/the/file1; } ``` } server { listen 80; server\_name server2; ``` location /robots.txt { alias /path/to/the/file2; } ``` } Ray. On Fri, May 21, 2010 at 9:43 PM, Ilan B. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > * * * > > nginx mailing list > removed\_email\_address@domain.invalid > [nginx Info Page](http://nginx.org/mailman/listinfo/nginx) * * * Message: 2 Date: Fri, 21 May 2010 21:53:45 +0800 From: Ray [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) To: removed\_email\_address@domain.invalid Cc: removed\_email\_address@domain.invalid Subject: Re: loading a different robots.txt file for a different sub domain? Message-ID: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Content-Type: text/plain; charset=UTF-8 Yes. server { listen 80 default; server\_name www; ``` location /robots.txt { alias /path/to/the/file1; } ``` } server { listen 80; server\_name server2; ``` location /robots.txt { alias /path/to/the/file2; } ``` } Ray. On Fri, May 21, 2010 at 9:43 PM, Ilan B. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > * * * > > nginx mailing list > removed\_email\_address@domain.invalid > [nginx Info Page](http://nginx.org/mailman/listinfo/nginx) * * * Message: 3 Date: Fri, 21 May 2010 10:03:10 -0400 From: Ilan B. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) To: removed\_email\_address@domain.invalid Subject: Re: loading a different robots.txt file for a different sub domain? Message-ID: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Content-Type: text/plain; charset=“iso-8859-1” Worked like a charm, thanks! On Fri, May 21, 2010 at 9:53 AM, Ray [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > Ray. > > > file when requested for one group vs. the other? > > * * * > > nginx mailing list > removed\_email\_address@domain.invalid > [nginx Info Page](http://nginx.org/mailman/listinfo/nginx) -------------- next part -------------- An HTML attachment was scrubbed… URL: [http://nginx.org/pipermail/nginx/attachments/20100521/dc7ff991/attachment-0001.html](http://nginx.org/pipermail/nginx/attachments/20100521/dc7ff991/attachment-0001.html) * * * Message: 4 Date: Fri, 21 May 2010 18:24:01 +0400 From: Igor S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) To: removed\_email\_address@domain.invalid Subject: Re: loading a different robots.txt file for a different sub domain? Message-ID: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Content-Type: text/plain; charset=koi8-r On Fri, May 21, 2010 at 10:03:10AM -0400, Ilan B. wrote: > Worked like a charm, thanks! You may also use: location = /robots.txt { > > } > > > > > (html, images, swf, etc.) but use different PHP backends. > > > > * * * > > > > nginx mailing list > > removed\_email\_address@domain.invalid > > [nginx Info Page](http://nginx.org/mailman/listinfo/nginx) > * * * > > nginx mailing list > removed\_email\_address@domain.invalid > [nginx Info Page](http://nginx.org/mailman/listinfo/nginx) – Igor S. [http://sysoev.ru/en/](http://sysoev.ru/en/) * * * Message: 5 Date: Fri, 21 May 2010 10:07:00 -0700 From: Avleen Vig [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) To: removed\_email\_address@domain.invalid Subject: nginx 0day exploit for nginx + fastcgi PHP Message-ID: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Content-Type: text/plain; charset=ISO-8859-1 This is currently doing the rounds, so I thought it pertinent to post it here too. [http://www.webhostingtalk.com/showthread.php?p=6807475#post6807475](http://www.webhostingtalk.com/showthread.php?p=6807475#post6807475) I don’t know what nginx should do to fix this, but there are two workarounds given. If you allow file uploads (especially things like images) and use PHP FastCGI in the back end, you should take a loot at this now. The exploit allows for any arbitrary file which is uploaded, to be executed as PHP. * * * Message: 6 Date: Fri, 21 May 2010 10:27:14 -0700 From: Avleen Vig [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) To: removed\_email\_address@domain.invalid Subject: Re: nginx 0day exploit for nginx + fastcgi PHP Message-ID: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Content-Type: text/plain; charset=ISO-8859-1 On Fri, May 21, 2010 at 10:07 AM, Avleen Vig [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > executed as PHP. > I should add that this isn’t a bug in the traditional broken-code sense. > More that this is a gaping configuration hole which is now widely > published, and could lead to many people being exploited. * * * Message: 7 Date: Fri, 21 May 2010 10:28:16 -0700 From: Michael S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) To: removed\_email\_address@domain.invalid Cc: removed\_email\_address@domain.invalid Subject: Re: nginx 0day exploit for nginx + fastcgi PHP Message-ID: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Content-Type: text/plain; charset=UTF-8 Question is, what functionality is lost by changing cgi.fix\_pathinfo = 0 Looks like the other workaround is something like this: if ( $fastcgi\_script\_name ~ ..\*/.\*php ) { return 403; } Which i basically saying what exactly? If there is a period and slash somewhere prior to the last “filename” to return a 403? Ideally while this is being thought out it would be cool to fix the common “no input file specified” issue that a lot of people have - have it return a 404 instead. Not sure if it’s a simple php.ini change (perhaps the path info?) or change fastcgi\_param REDIRECT\_STATUS 200? On Fri, May 21, 2010 at 10:07 AM, Avleen Vig [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > executed as PHP. > > * * * > > nginx mailing list > removed\_email\_address@domain.invalid > [nginx Info Page](http://nginx.org/mailman/listinfo/nginx) * * * Message: 8 Date: Fri, 21 May 2010 10:28:16 -0700 From: Michael S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) To: removed\_email\_address@domain.invalid Cc: removed\_email\_address@domain.invalid Subject: Re: nginx 0day exploit for nginx + fastcgi PHP Message-ID: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Content-Type: text/plain; charset=UTF-8 Question is, what functionality is lost by changing cgi.fix\_pathinfo = 0 Looks like the other workaround is something like this: if ( $fastcgi\_script\_name ~ ..\*/.\*php ) { return 403; } Which i basically saying what exactly? If there is a period and slash somewhere prior to the last “filename” to return a 403? Ideally while this is being thought out it would be cool to fix the common “no input file specified” issue that a lot of people have - have it return a 404 instead. Not sure if it’s a simple php.ini change (perhaps the path info?) or change fastcgi\_param REDIRECT\_STATUS 200? On Fri, May 21, 2010 at 10:07 AM, Avleen Vig [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > executed as PHP. > > * * * > > nginx mailing list > removed\_email\_address@domain.invalid > [nginx Info Page](http://nginx.org/mailman/listinfo/nginx) * * * Message: 9 Date: Fri, 21 May 2010 21:33:02 +0400 From: Igor S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) To: removed\_email\_address@domain.invalid Subject: Re: nginx 0day exploit for nginx + fastcgi PHP Message-ID: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) Content-Type: text/plain; charset=koi8-r On Fri, May 21, 2010 at 10:07:00AM -0700, Avleen Vig wrote: > executed as PHP. > I do not see why this is treated as nginx bug ? > Why is anyone able at all to upload images to /scripts directory ? > Why does PHP have cgi.fix\_pathinfo option ? > BTW, I’m just curious how does lighttpd resolve this issue ? Also instead of if ( $fastcgi\_script\_name ~ ..\*/.\*php ) { return 403; } it should be worked around as location ~ .._/._.php$ { return 403; } location ~ .php$ { return 403; } – Igor S. [http://sysoev.ru/en/](http://sysoev.ru/en/) * * * * * * nginx mailing list removed\_email\_address@domain.invalid [http://nginx.org/mailman/listinfo/nginx](http://nginx.org/mailman/listinfo/nginx) End of nginx Digest, Vol 7, Issue 47 * * * ---
**Author:** ![zhys99](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/z/c77e96/32.png) [@zhys99](https://www.ruby-forum.com/u/zhys99)\ **Post date:** [May 22, 2010, 8:04am UTC](https://www.ruby-forum.com/t/re-loading-a-different-robots-txt-file-for-a-different-sub-domain-ray/189027/2 "2010-05-22T08:04:32Z")
On Sat, May 22, 2010 at 10:12:45AM +0800, zhys99 wrote: > load different robots in the same one “server” block > > location ~ /robots.txt > { > if ($host = ‘first.domain’) { > rewrite ^/robots.txt /path/to/another/robots.txt last; > } > } This is highly unrecommneded way to handle different servers. I use it only once on a site with more than hundred locaitons that has some unofficial but public names, to prevent crawling: ``` location = /robots.txt { if ($http_host ~* ^.......$) { root /data/w3; } } ``` Otherwise these sites should be the same. If your sites are lesser or have more differences, you should use the right way: ``` server { ... } server { ... } ``` > ``` > domain? (Igor S.) > > ``` > > Date: Fri, 21 May 2010 21:53:45 +0800 > > ``` > listen 80; > > ``` > > > > > * * * > > Content-Type: text/plain; charset=UTF-8 > } > > > Is there a way, using nginx configuration to load a different robots.txt > > Message-ID: > > > server { > > server\_name server2; > > On Fri, May 21, 2010 at 9:43 PM, Ilan B. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: > > > > > > > > > [mailman.nginx.org archive](http://nginx.org/mailman/listinfo/nginx) > > To: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > > location = /robots.txt { > > > > > ``` > > > alias /path/to/the/file1; > > > > > > ``` > > > > > > } > > > > > > > The 2 sub-domains share the same directory for delivery of static files > > > > [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > > it here too. > > Content-Type: text/plain; charset=ISO-8859-1 > > > FastCGI in the back end, you should take a loot at this now. > > Question is, what functionality is lost by changing > somewhere prior to the last “filename” to return a 403? > > > nginx mailing list > > From: Michael S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > > common “no input file specified” issue that a lot of people have - > > > I don’t know what nginx should do to fix this, but there are two > > Message-ID: [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > > > workarounds given. > > Also instead of > > * * * > > * * * > > nginx mailing list > [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > [mailman.nginx.org archive](http://nginx.org/mailman/listinfo/nginx) > > End of nginx Digest, Vol 7, Issue 47 > > * * * > * * * > > nginx mailing list > [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) > [mailman.nginx.org archive](http://nginx.org/mailman/listinfo/nginx) – Igor S. > **[Igor Sysoev](http://sysoev.ru/en/)**