# Problem with rewrite last giving a HTTP 301? **URL:** \ **Category:** NGINX\ **Created:** [September 22, 2010, 12:10am UTC](https://www.ruby-forum.com/t/problem-with-rewrite-last-giving-a-http-301/195346 "2010-09-22T00:10:42Z")\ **Posts on this page:** 4\ **Page:** 1
**Author:** ![toto2008](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/8797f3/32.png) [@toto2008](https://www.ruby-forum.com/u/toto2008)\ **Post date:** [September 22, 2010, 12:10am UTC](https://www.ruby-forum.com/t/problem-with-rewrite-last-giving-a-http-301/195346/1 "2010-09-22T00:10:42Z")
Hello, I’m having a weird problem with my website. In my nginx conf, I have this rule: rewrite ^/robots.txt$ /cms/robotstxt.php last; and a location to handle PHP files: location ~ .php$ { fastcgi\_pass 127.0.0.1:9000; fastcgi\_index index.php; fastcgi\_param SCRIPT\_FILENAME $document\_root$fastcgi\_script\_name; include fastcgi\_params; } When I visit robots.txt, I get the expected result (a robots.txt dynamically generated). If I check the nginx log file, as expected I get a 200 HTTP answer : IPADDRESS - - [21/Sep/2010:16:22:57 +0200] “GET /robots.txt HTTP/1.0” 200 231 “-” “Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 1.1.4322; .NET CLR 2.0.50727; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET4.0C; .NET4.0E)” If I check with fiddler (a HTTP debugger), everything is also OK : I have my robots.txt file with the 200 HTTP code. By checking the logs most of the time the 200 code is here. Now, here is the problem : sometimes, for some reasons that I don’t know, instead of the 200 code, nginx send a 301 code. Here’s an example of the googlebot visiting my website : 66.249.65.168 - - [21/Sep/2010:12:02:09 +0200] “GET /robots.txt HTTP/1.0” 301 178 “-” “Googlebot-Image/1.0” 66.249.65.166 - - [21/Sep/2010:12:02:09 +0200] “GET /cms/robotstxt.php HTTP/1.0” 200 231 “-” “Googlebot-Image/1.0” Of course, I don’t want bots or people to visit this /cms/robotstxt.php page directly… Tonight, for the first time I’ve also found this problem with another rewrite rule. Again, I have a rule like : rewrite “^/([0-9]+)-([a-z0-9-]\*)-([a-z]{2})$” “/cms/pages.php?id=$1;title=$2;language=$3” last; and normally I get a 200 HTTP code when I visit one of my page. But I discovered some weird logs from the Yandex bot : 95.108.151.244 - - [21/Sep/2010:21:20:56 +0200] “GET /123-mypage-fr HTTP/1.0” 301 178 “-” “Mozilla/5.0 (compatible; YandexBot/3.0; MirrorDetector; +http://yandex.com/bots)” I’ve also tried to fetch the robots.txt file from the google webmaster tools, but I got a correct 200 HTTP answer. Is anybody know what the problem could be ? I have no idea how I could reproduce this strange results with my browser or wget or whatever tool. I’m using nginx 0.7.67, the version packaged with Debian Squeeze 32bits. Thanks. Posted at Nginx Forum: > **[Problem with rewrite last giving a HTTP 301 ?](https://forum.nginx.org/read.php?2,132723,132723#msg-132723)** > > Hello, I'm having a weird problem with my website. In my nginx conf, I have this rule: rewrite ^/robots\\.txt$ /cms/robotstxt.php last; and a location to handle PHP files: location ~ \\.php$ { fastcgi\_pass 127.0.0.1:9000; fastc ---
**Author:** ![toto2008](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/8797f3/32.png) [@toto2008](https://www.ruby-forum.com/u/toto2008)\ **Post date:** [September 22, 2010, 4:14am UTC](https://www.ruby-forum.com/t/problem-with-rewrite-last-giving-a-http-301/195346/2 "2010-09-22T04:14:18Z")
Hello! On Tue, Sep 21, 2010 at 06:10:05PM -0400, toto2008 wrote: > ``` > fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; > > ``` > > If I check with fiddler (a HTTP debugger), everything is also OK : I > HTTP/1.0" 200 231 “-” “Googlebot-Image/1.0” > 95.108.151.244 - - [21/Sep/2010:21:20:56 +0200] “GET /123-mypage-fr > HTTP/1.0” 301 178 “-” “Mozilla/5.0 (compatible; YandexBot/3.0; > MirrorDetector; +http://yandex.com/bots)” > > I’ve also tried to fetch the robots.txt file from the google webmaster > tools, but I got a correct 200 HTTP answer. > > Is anybody know what the problem could be ? I have no idea how I could > reproduce this strange results with my browser or wget or whatever > tool. Most likely it’s your cms code which detects something in bot’s requests (e.g. wrong Host header) and issues redirect. Either look into it’s code or try logging something like $upstream\_http\_location, it should give you some better idea what’s going on. Maxim D. ---
**Author:** ![toto2008](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/8797f3/32.png) [@toto2008](https://www.ruby-forum.com/u/toto2008)\ **Post date:** [September 22, 2010, 9:25am UTC](https://www.ruby-forum.com/t/problem-with-rewrite-last-giving-a-http-301/195346/3 "2010-09-22T09:25:06Z")
Hello Maxim, Thanks a lot for your answer ! I wrote the CMS, and I had already changed the few 301 redirects into 302 to be sure they weren’t the problem. So I’ll log $upstream\_http\_location and I’ll report back when I’ll get this problem again. In my PHP code, I’ll also log the headers so maybe I can find what could trigger that strange behavior. Olivier Posted at Nginx Forum: > **[Re: Problem with rewrite last giving a HTTP 301 ?](https://forum.nginx.org/read.php?2,132723,132862#msg-132862)** > > Hello, I'm having a weird problem with my website. In my nginx conf, I have this rule: rewrite ^/robots\\.txt$ /cms/robotstxt.php last; and a location to handle PHP files: location ~ \\.php$ { fastcgi\_pass 127.0.0.1:9000; fastc ---
**Author:** ![toto2008](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/t/8797f3/32.png) [@toto2008](https://www.ruby-forum.com/u/toto2008)\ **Post date:** [September 26, 2010, 5:06pm UTC](https://www.ruby-forum.com/t/problem-with-rewrite-last-giving-a-http-301/195346/4 "2010-09-26T17:06:12Z")
Hello, Thanks again Maxim ! With your help I’ve discovered that the bots which were redirected with a 301 weren’t coming from my main domain ([something.fr](http://something.fr)) but they were coming from another domain I have ([something.com](http://something.com)). I had just setup the .com to redirect to the .fr but I weren’t using it. Anyway, somehow some bots were visiting it… I’ll fix this redirection problem in my configuration file so it won’t happen anymore. Olivier Posted at Nginx Forum: > **[Re: Problem with rewrite last giving a HTTP 301 ?](https://forum.nginx.org/read.php?2,132723,134250#msg-134250)** > > Hello, I'm having a weird problem with my website. In my nginx conf, I have this rule: rewrite ^/robots\\.txt$ /cms/robotstxt.php last; and a location to handle PHP files: location ~ \\.php$ { fastcgi\_pass 127.0.0.1:9000; fastc