Forum: NGINX NGINX redirection issue

2974d09ac2541e892966b762aad84943?d=identicon&s=25 manish-ezest (Guest)
on 2014-08-07 05:38
(Received via mailing list)
Hello All,

I am facing some issue regarding nginx redirection. I want to remove
.html
extension from all the html pages and all the index.html pages. For
example

1.
http://www.aaa.com/bbb/ccc.html should show the content of
http://www.aaa.com/bbb/ccc.html but the URL should show
http://www.aaa.com/bbb/ccc

2. http://www.aaa.com/index.html should show the content of
http://www.aaa.com/index.html but the URL should show
http://www.aaa.com.

There are few rewrites already there which are given below

1. Whenever you try to access the http://www.aaa.com site, it will
redirect
to http://www.aaa.com/aaa/bbb/ccc/ddd/index.html.
 http://www.aaa.com    ----->
http://www.aaa.com/aaa/bbb/ccc/ddd/index.html.

2. All directory lookups to 'index.html'
rewrite ^(.*)/$ $1/index.html permanent;
3. All open strings to index.html
rewrite ^(.*)/([^\.\?/]+)$  $1/$2/index.html permanent;

Now whenever I try to implement the change, the site went into a
redirect
loop and gives following error in the error log.

2014/08/05 14:48:03 [error] 10392#0: *750 rewrite or internal
redirection
cycle while internally redirecting to "/index.html", client: 1.1.1.1,
server: www.aaa.com, request: "GET /aaa/ccc HTTP/1.1", host:
"www.aaa.com"

Please suggest how to solve the issue

--Manish

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,252379,252379#msg-252379
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2014-08-07 10:04
(Received via mailing list)
Hello!

On Wed, Aug 06, 2014 at 11:37:42PM -0400, manish-ezest wrote:

> 2. http://www.aaa.com/index.html should show the content of
> http://www.aaa.com/index.html but the URL should show http://www.aaa.com.

This is something as simple as:

    location / {
        index index.html;
        try_files $uri.html $uri/ =404;
    }

with "index index.html" being the default - that is, you don't
actually need to configure it explicitly.  Or like this, using
only try_files instead of try_files + index:

    location / {
        try_files $uri.html $uri/index.html =404;
    }


The only potential difference from what you describe is that this
won't do a redirect to hide "/index.html" if explicitly requested
by a user, but it's not clear from your description if it's
something really needed.  This is not something usually done.

See here for details:

http://nginx.org/r/index
http://nginx.org/r/try_files
http://nginx.org/en/docs/http/request_processing.html

>
> Now whenever I try to implement the change, the site went into a redirect
> loop and gives following error in the error log.
>
> 2014/08/05 14:48:03 [error] 10392#0: *750 rewrite or internal redirection
> cycle while internally redirecting to "/index.html", client: 1.1.1.1,
> server: www.aaa.com, request: "GET /aaa/ccc HTTP/1.1", host: "www.aaa.com"
>
> Please suggest how to solve the issue

Both rewrites you've provided will return external 301 redirects
without doing internal redirects, so the loop is caused by
something else in your config.  You have to show your config for
others to be able to help.

--
Maxim Dounin
http://nginx.org/
2974d09ac2541e892966b762aad84943?d=identicon&s=25 manish-ezest (Guest)
on 2014-08-07 18:02
(Received via mailing list)
Hello Maxim,

Thank you for your response. Here is the my nginx settings

**************my NGINX.conf********************
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log notice;
pid /var/run/nginx.pid;
worker_rlimit_nofile 30000;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;

log_format main '$remote_addr - $remote_user [$time_local] $request '
'"$status" $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
log_format combined_time '$remote_addr - $remote_user [$time_local]'
'"$request" $status $body_bytes_sent '
'"$http_referer" "$http_user_agent" $request_time';

access_log /var/log/nginx/access.log combined_time;
include /etc/nginx/servers/*.conf;
}
**********************************************

******************************VHOST SETTING*****
server {
listen 80;
server_name www-aaa.com;
add_header Cache-Control off;
expires 1d;
root /var/empty;
error_log /var/log/nginx/www.aaa.com-error.log;
access_log /var/log/nginx/www.aaa.com-access.log combined_time;
location = /favicon.ico {
root /www;
}
proxy_intercept_errors on;
error_page 400 401 402 403 404 406 407 408 409 410 411 412 413 414 415
416
417 495 496 497 500 501 502 503 504 505 506 507 = /error_page.pl;
error_page 405 =200 $uri;
location /error_page.pl {
fastcgi_pass 127.0.0.1:8999;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_pass_header "Status";
}

location / {
## Rewrite root to index of bbb folder.
rewrite ^/$ http://www.aaa.com/aaaa/bbb/index.html permanent;
## Rewrite all directory lookups to 'index.html'
rewrite ^(.*)/$ $1/index.html permanent;
## Rewrite all open strings to index.html
rewrite ^(.*)/([^\.\?/]+)$  $1/$2/index.html permanent;
proxy_redirect off;
proxy_set_header Host www.aaa.com.s3.amazonaws.com;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Referer www.aaa.com;
proxy_pass http://www.aaa.com.s3.amazonaws.com/;
}
}

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,252379,252391#msg-252391
2974d09ac2541e892966b762aad84943?d=identicon&s=25 manish-ezest (Guest)
on 2014-08-08 12:02
(Received via mailing list)
Hello Maxim,

I am getting 499 error when I try using try files directive as you
suggested. It got stuck and no response from the server.

50.50.50.50 - - [08/Aug/2014:10:12:01 +0200]"GET /aaa/bbb/ccc/index.html
HTTP/1.1" 499 0 "-" "Wget/1.14 (linux-gnu)" 4.678

--Manish

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,252379,252410#msg-252410
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2014-08-08 16:52
(Received via mailing list)
Hello!

On Thu, Aug 07, 2014 at 12:02:43PM -0400, manish-ezest wrote:

> Hello Maxim,
>
> Thank you for your response. Here is the my nginx settings

[...]

> server {
> listen 80;
> server_name www-aaa.com;

Note that server_name doesn't match domain in logs you've provided
and in other parts of your config.

[...]

> proxy_intercept_errors on;
> error_page 400 401 402 403 404 406 407 408 409 410 411 412 413 414 415 416
> 417 495 496 497 500 501 502 503 504 505 506 507 = /error_page.pl;
> error_page 405 =200 $uri;

I don't see anything in the config provided which may trigger
internal redirect to "/index.html" except this "error_page 405
=200 $uri".  But this shouldn't trigger a loop unless you are also
using "recursive_error_pages", which is off by default.

Are you sure you are looking into the right config?

--
Maxim Dounin
http://nginx.org/
2974d09ac2541e892966b762aad84943?d=identicon&s=25 manish-ezest (Guest)
on 2014-08-09 04:49
(Received via mailing list)
Hello Maxim,

The server_name(website) is a dummy name as I don't want to disclose the
site name. I might have mistaken while editing the name. But I ensure
you
that the logs I have provided is correct. I am pasting my original
configuration which is working like a charm. If I change anything from
it
like adding try_files as you suggested, or some rewrite rule for
removing
.html extension in location / {} directive, causes either 499(mostly) or
"750 rewrite or internal redirection cycle" error. I am not able to
figure
out what actually causing the issue.

**************my NGINX.conf********************
 user nginx;
 worker_processes 1;
 error_log /var/log/nginx/error.log notice;
 pid /var/run/nginx.pid;
 worker_rlimit_nofile 30000;
 events {
 worker_connections 1024;
 }
 http {
 include /etc/nginx/mime.types;
 default_type application/octet-stream;

 log_format main '$remote_addr - $remote_user [$time_local] $request '
 '"$status" $body_bytes_sent "$http_referer" '
 '"$http_user_agent" "$http_x_forwarded_for"';
 log_format combined_time '$remote_addr - $remote_user [$time_local]'
 '"$request" $status $body_bytes_sent '
 '"$http_referer" "$http_user_agent" $request_time';

 access_log /var/log/nginx/access.log combined_time;
 include /etc/nginx/servers/*.conf;
 }
 **********************************************

 ******************************VHOST SETTING*****
 server {
 listen 80;
 server_name www.aaa.com;
 add_header Cache-Control off;
 expires 1d;
 root /var/empty;
 error_log /var/log/nginx/www.aaa.com-error.log;
 access_log /var/log/nginx/www.aaa.com-access.log
 combined_time;
 location = /favicon.ico {
 root /www;
 }
 proxy_intercept_errors on;
 error_page 400 401 402 403 404 406 407 408 409 410 411 412 413 414 415
416
 417 495 496 497 500 501 502 503 504 505 506 507 = /error_page.pl;
 error_page 405 =200 $uri;
 location /error_page.pl {
 fastcgi_pass 127.0.0.1:8999;
 fastcgi_param REQUEST_URI $request_uri;
 fastcgi_pass_header "Status";
 }

 location / {
## Rewrite root to index of bbb folder.
rewrite ^/$ http://www.aaa.com/aaaa/bbb/index.html permanent;
## Rewrite all directory lookups to 'index.html'
rewrite ^(.*)/$ $1/index.html permanent;
## Rewrite all open strings to index.html
rewrite ^(.*)/([^\.\?/]+)$ $1/$2/index.html permanent;
 proxy_redirect off;
 proxy_set_header Host www.aaa.com.s3.amazonaws.com;
 proxy_set_header X-Real-IP $remote_addr;
 proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
 proxy_set_header Referer www.aaa.com;
 proxy_pass http://www.aaa.com.s3.amazonaws.com/;
 add_header         Access-Control-Allow-Origin *;
 }
 }

--Manish

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,252379,252440#msg-252440
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2014-08-10 03:11
(Received via mailing list)
Hello!

On Fri, Aug 08, 2014 at 10:49:15PM -0400, manish-ezest wrote:

> Hello Maxim,
>
> The server_name(website) is a dummy name as I don't want to disclose the
> site name. I might have mistaken while editing the name. But I ensure you
> that the logs I have provided is correct. I am pasting my original

Editing configs and logs is usually very bad idea.  Instead, it's
much better to reproduce the problem in a sandbox which doesn't
contain information you don't want to share.

> configuration which is working like a charm. If I change anything from it
> like adding try_files as you suggested, or some rewrite rule for removing
> .html extension in location / {} directive, causes either 499(mostly) or
> "750 rewrite or internal redirection cycle" error. I am not able to figure
> out what actually causing the issue.

There is a number of ways to find it out, and most powerful one is
debugging log, see here:

http://nginx.org/en/docs/debugging_log.html

[...]

>  proxy_pass http://www.aaa.com.s3.amazonaws.com/;

Note well that try_files (and index) won't work if you don't
actually have files, but proxy everything to another server.

--
Maxim Dounin
http://nginx.org/
2974d09ac2541e892966b762aad84943?d=identicon&s=25 manish-ezest (Guest)
on 2014-08-29 16:09
(Received via mailing list)
Hello Maxim,

I tried enabling debug mode but still I am not able to figure out the
issue
and its possible solutions.

I disabled this rewrites which are rewriting to index.html
## Rewrite all directory lookups to 'index.html'
#rewrite ^(.*)/$ $1/index.html permanent;

## Rewrite all open strings to index.html
#rewrite ^(.*)/([^\.\?/]+)$  $1/$2/index.html permanent;

and added this one as you suggested but now it is going to
www.google.com(very strange)

location / {
        index index.html;
        try_files $uri.html $uri/ =404;
    }

Error Log Content

2014/08/29 09:40:28 [error] 935#0: *64 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: "GET /favicon.ico HTTP/1.1", upstream:
"fastcgi://127.0.0.1:8999", host: "www.aaa.com"
2014/08/29 09:40:38 [error] 935#0: *64 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: "GET /favicon.ico HTTP/1.1", upstream:
"fastcgi://127.0.0.1:8999", host: "www.aaa.com"
2014/08/29 09:40:48 [error] 935#0: *64 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: "GET /favicon.ico HTTP/1.1", upstream:
"fastcgi://127.0.0.1:8999", host: "www.aaa.com"
2014/08/29 09:40:58 [error] 935#0: *64 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: "GET /favicon.ico HTTP/1.1", upstream:
"fastcgi://127.0.0.1:8999", host: "www.aaa.com"
2014/08/29 09:41:08 [error] 935#0: *64 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: "GET /favicon.ico HTTP/1.1", upstream:
"fastcgi://127.0.0.1:8999", host: "www.aaa.com"
2014/08/29 09:44:15 [error] 957#0: *73 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: "GET /images/icons/product/chrome-48.png
HTTP/1.1", upstream: "fastcgi://127.0.0.1:8999", host: "www.aaa.com",
referrer: "http://www.aaa.com/bbb/ccc/index.html"
2014/08/29 09:44:15 [error] 957#0: *74 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: "GET /images/srpr/logo9w.png HTTP/1.1",
upstream: "fastcgi://127.0.0.1:8999", host: "www.aaa.com", referrer:
"http://www.aaa.com/bbb/ccc/index.html"
2014/08/29 09:44:15 [error] 957#0: *75 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: "GET
/xjs/_/js/k=xjs.hp.en_US.QgXXqNe9H8s.O/m=sb_he,pcc/rt=j/d=1/t=zcms/rs=AItRSTMZ4gjPthsSL3eRaZarMVJV0YA7aQ
HTTP/1.1", upstream: "fastcgi://127.0.0.1:8999", host: "www.aaa.com",
referrer: "http://www.aaa.com/bbb/ccc/index.html"
2014/08/29 09:44:15 [error] 957#0: *76 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: "GET /images/srpr/nav_logo80.png
HTTP/1.1",
upstream: "fastcgi://127.0.0.1:8999", host: "www.aaa.com", referrer:
"http://www.aaa.com/bbb/ccc/index.html"

Access Log Content

11.11.11.11 - - [29/Aug/2014:09:44:32 +0200]"GET
/images/icons/product/chrome-48.png HTTP/1.1" 499 0
"http://www.aaa.com/bbb/ccc/index.html" "Mozilla/5.0 (Windows NT 6.1;
WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143
Safari/537.36"
26.375
11.11.11.11 - - [29/Aug/2014:09:44:32 +0200]"GET /images/srpr/logo9w.png
HTTP/1.1" 499 0 "http://www.aaa.com/bbb/ccc/index.html" "Mozilla/5.0
(Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/36.0.1985.143 Safari/537.36" 26.221
11.11.11.11 - - [29/Aug/2014:09:44:32 +0200]"GET
/xjs/_/js/k=xjs.hp.en_US.QgXXqNe9H8s.O/m=sb_he,pcc/rt=j/d=1/t=zcms/rs=AItRSTMZ4gjPthsSL3eRaZarMVJV0YA7aQ
HTTP/1.1" 499 0 "http://www.aaa.com/bbb/ccc/index.html" "Mozilla/5.0
(Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/36.0.1985.143 Safari/537.36" 26.222
11.11.11.11 - - [29/Aug/2014:09:44:32 +0200]"GET
/images/srpr/nav_logo80.png
HTTP/1.1" 499 0 "http://www.aaa.com/bbb/ccc/index.html" "Mozilla/5.0
(Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/36.0.1985.143 Safari/537.36" 26.222

Regards,
Manish

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,252379,252939#msg-252939
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2014-08-31 04:46
(Received via mailing list)
Hello!

On Fri, Aug 29, 2014 at 10:08:57AM -0400, manish-ezest wrote:

> #rewrite ^(.*)/([^\.\?/]+)$  $1/$2/index.html permanent;
>
> server: www.aaa.com, request: "GET /favicon.ico HTTP/1.1", upstream:
> "fastcgi://127.0.0.1:8999", host: "www.aaa.com"
> 2014/08/29 09:40:58 [error] 935#0: *64 upstream timed out (110: Connection
> timed out) while reading response header from upstream, client: 11.11.11.11,
> server: www.aaa.com, request: "GET /favicon.ico HTTP/1.1", upstream:
> "fastcgi://127.0.0.1:8999", host: "www.aaa.com"
> 2014/08/29 09:41:08 [error] 935#0: *64 upstream timed out (110: Connection
> timed out) while reading response header from upstream, client: 11.11.11.11,
> server: www.aaa.com, request: "GET /favicon.ico HTTP/1.1", upstream:
> "fastcgi://127.0.0.1:8999", host: "www.aaa.com"

[...]

From the logs it looks like the loop is caused by error_page you
use, and recursive_error_pages set somewhere in your
configuration.  As long as your error handler script times out
(likely because it can't cope with load), nginx tries to call it
again, then again, then again.  This continues till nginx reaches
the redirections limit and logs the message about it.

Find where recursive_error_pages is set and switch it off - or,
rather, just remove it from the configuration.  As previously
said, the default is correct and should not be changed unless you
understand what you are doing.  This should resolve your problem.

--
Maxim Dounin
http://nginx.org/
2974d09ac2541e892966b762aad84943?d=identicon&s=25 manish-ezest (Guest)
on 2014-09-01 20:06
(Received via mailing list)
Hello Maxim,

Like you suggested I have set "recursive_error_pages" to off but still I
am
facing the problem. This time I am getting "504 Gateway Time-out" error.
I
have already shared my NGINX and vhost configuration. We have one
fastcgi
script running for serving error pages which checks the entry of
sample.xml(contains url) file and redirect the link to particular
location.
If it doesn't find any page then it returns a 404 page. I am pasting the
log
file of fastcgi script as well.

==> fastcgi-404.log <==
[2014-09-01T15:45:24] Got request for [/bbb/ccc/index.html] on host
[www.aaa.com]
[2014-09-01T15:45:24] Target not found in sample.xml, importing default
404
[/bbb/fff/error_404.html]
[2014-09-01T15:45:24] Retrieving target
[http://www.aaa.com/bbb/fff/error_404.html]

==> www.aaa.com-error.log <==
2014/09/01 15:45:34 [error] 15900#0: *175 upstream timed out (110:
Connection timed out) while reading response header from upstream,
client:
11.11.11.11, server: www.aaa.com, request: "GET /bbb/ccc/index.html
HTTP/1.1", upstream: "fastcgi://127.0.0.1:8999", host: "www.aaa.com"
2014/09/01 15:45:34 [error] 15900#0: *175 open() "/www/favicon.ico"
failed
(2: No such file or directory), client: 11.11.11.11, server:
www.aaa.com,
request: "GET /favicon.ico HTTP/1.1", host: "www.aaa.com"
2014/09/01 15:45:44 [error] 15900#0: *175 upstream timed out (110:
Connection timed out) while reading response header from upstream,
client:
11.11.11.11, server: www.aaa.com, request: "GET /favicon.ico HTTP/1.1",
upstream: "fastcgi://127.0.0.1:8999", host: "www.aaa.com"
2014/09/01 15:46:57 [error] 15900#0: *178 upstream timed out (110:
Connection timed out) while reading response header from upstream,
client:
11.11.11.11, server: www.aaa.com, request: "GET /bbb/ccc/index.html
HTTP/1.1", upstream: "fastcgi://127.0.0.1:8999", host: "www.aaa.com"
2014/09/01 15:46:57 [error] 15900#0: *178 open() "/www/favicon.ico"
failed
(2: No such file or directory), client: 11.11.11.11, server:
www.aaa.com,
request: "GET /favicon.ico HTTP/1.1", host: "www.aaa.com"
2014/09/01 15:47:07 [error] 15900#0: *178 upstream timed out (110:
Connection timed out) while reading response header from upstream,
client:
11.11.11.11, server: www.aaa.com, request: "GET /favicon.ico HTTP/1.1",
upstream: "fastcgi://127.0.0.1:8999", host: "www.aaa.com"

--Manish

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,252379,252998#msg-252998
A8108a0961c6087c43cda32c8616dcba?d=identicon&s=25 Maxim Dounin (Guest)
on 2014-09-02 15:59
(Received via mailing list)
Hello!

On Mon, Sep 01, 2014 at 02:05:46PM -0400, manish-ezest wrote:

> Hello Maxim,
>
> Like you suggested I have set "recursive_error_pages" to off but still I am
> facing the problem. This time I am getting "504 Gateway Time-out" error. I
> have already shared my NGINX and vhost configuration. We have one fastcgi
> script running for serving error pages which checks the entry of
> sample.xml(contains url) file and redirect the link to particular location.
> If it doesn't find any page then it returns a 404 page. I am pasting the log
> file of fastcgi script as well.

The error is as clear as it could be: your backend failed to
respond in time.  As previously suggested, it may be due to the
fact that it's overloaded.

There are many options on how to fix this:

- Improve your backend performance by optimizing the code.

- Add more resources to the backend cluster (more processes on a
  single server and/or more servers).

- Rethink your nginx configuration to avoid using the script
  (e.g., use nginx configuration to do redirects instead, this
  should be much more efficient).

In either case, everything in your configuration seems to work
correctly as configured.

--
Maxim Dounin
http://nginx.org/
2974d09ac2541e892966b762aad84943?d=identicon&s=25 manish-ezest (Guest)
on 2014-09-03 09:53
(Received via mailing list)
OK thanks Maxim. I will check your recommendation and will let you know.

--Manish

Posted at Nginx Forum:
http://forum.nginx.org/read.php?2,252379,253036#msg-253036
Please log in before posting. Registration is free and takes only a minute.
Existing account

NEW: Do you have a Google/GoogleMail, Yahoo or Facebook account? No registration required!
Log in with Google account | Log in with Yahoo account | Log in with Facebook account
No account? Register here.