NGINX redirection issue

Hello All,

I am facing some issue regarding nginx redirection. I want to remove
.html
extension from all the html pages and all the index.html pages. For
example

AAA Sorry Page should show the content of
AAA Sorry Page but the URL should show
AAA Sorry Page

  1. Welcome to AAA should show the content of
    Welcome to AAA but the URL should show
    http://www.aaa.com.

There are few rewrites already there which are given below

  1. Whenever you try to access the http://www.aaa.com site, it will
    redirect
    to AAA Sorry Page.
    http://www.aaa.com ----->
    AAA Sorry Page.

  2. All directory lookups to ‘index.html’
    rewrite ^(.*)/$ $1/index.html permanent;

  3. All open strings to index.html
    rewrite ^(.*)/([^.?/]+)$ $1/$2/index.html permanent;

Now whenever I try to implement the change, the site went into a
redirect
loop and gives following error in the error log.

2014/08/05 14:48:03 [error] 10392#0: *750 rewrite or internal
redirection
cycle while internally redirecting to “/index.html”, client: 1.1.1.1,
server: www.aaa.com, request: “GET /aaa/ccc HTTP/1.1”, host:
www.aaa.com

Please suggest how to solve the issue

–Manish

Posted at Nginx Forum:

Hello!

On Wed, Aug 06, 2014 at 11:37:42PM -0400, manish-ezest wrote:

  1. Welcome to AAA should show the content of
    Welcome to AAA but the URL should show http://www.aaa.com.

This is something as simple as:

location / {
    index index.html;
    try_files $uri.html $uri/ =404;
}

with “index index.html” being the default - that is, you don’t
actually need to configure it explicitly. Or like this, using
only try_files instead of try_files + index:

location / {
    try_files $uri.html $uri/index.html =404;
}

The only potential difference from what you describe is that this
won’t do a redirect to hide “/index.html” if explicitly requested
by a user, but it’s not clear from your description if it’s
something really needed. This is not something usually done.

See here for details:

http://nginx.org/r/index
http://nginx.org/r/try_files
http://nginx.org/en/docs/http/request_processing.html

Now whenever I try to implement the change, the site went into a redirect
loop and gives following error in the error log.

2014/08/05 14:48:03 [error] 10392#0: *750 rewrite or internal redirection
cycle while internally redirecting to “/index.html”, client: 1.1.1.1,
server: www.aaa.com, request: “GET /aaa/ccc HTTP/1.1”, host: “www.aaa.com

Please suggest how to solve the issue

Both rewrites you’ve provided will return external 301 redirects
without doing internal redirects, so the loop is caused by
something else in your config. You have to show your config for
others to be able to help.


Maxim D.
http://nginx.org/

Hello Maxim,

Thank you for your response. Here is the my nginx settings

my NGINX.conf******
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log notice;
pid /var/run/nginx.pid;
worker_rlimit_nofile 30000;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;

log_format main '$remote_addr - $remote_user [$time_local] $request ’
'“$status” $body_bytes_sent “$http_referer” ’
‘“$http_user_agent” “$http_x_forwarded_for”’;
log_format combined_time ‘$remote_addr - $remote_user [$time_local]’
'“$request” $status $body_bytes_sent ’
‘“$http_referer” “$http_user_agent” $request_time’;

access_log /var/log/nginx/access.log combined_time;
include /etc/nginx/servers/*.conf;
}


*************************VHOST SETTING
server {
listen 80;
server_name www-aaa.com;
add_header Cache-Control off;
expires 1d;
root /var/empty;
error_log /var/log/nginx/www.aaa.com-error.log;
access_log /var/log/nginx/www.aaa.com-access.log combined_time;
location = /favicon.ico {
root /www;
}
proxy_intercept_errors on;
error_page 400 401 402 403 404 406 407 408 409 410 411 412 413 414 415
416
417 495 496 497 500 501 502 503 504 505 506 507 = /error_page.pl;
error_page 405 =200 $uri;
location /error_page.pl {
fastcgi_pass 127.0.0.1:8999;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_pass_header “Status”;
}

location / {

Rewrite root to index of bbb folder.

rewrite ^/$ AAA Sorry Page permanent;

Rewrite all directory lookups to ‘index.html’

rewrite ^(.*)/$ $1/index.html permanent;

Rewrite all open strings to index.html

rewrite ^(.*)/([^.?/]+)$ $1/$2/index.html permanent;
proxy_redirect off;
proxy_set_header Host www.aaa.com.s3.amazonaws.com;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Referer www.aaa.com;
proxy_pass http://www.aaa.com.s3.amazonaws.com/;
}
}

Posted at Nginx Forum:

Hello Maxim,

I am getting 499 error when I try using try files directive as you
suggested. It got stuck and no response from the server.

50.50.50.50 - - [08/Aug/2014:10:12:01 +0200]“GET /aaa/bbb/ccc/index.html
HTTP/1.1” 499 0 “-” “Wget/1.14 (linux-gnu)” 4.678

–Manish

Posted at Nginx Forum:

Hello Maxim,

The server_name(website) is a dummy name as I don’t want to disclose the
site name. I might have mistaken while editing the name. But I ensure
you
that the logs I have provided is correct. I am pasting my original
configuration which is working like a charm. If I change anything from
it
like adding try_files as you suggested, or some rewrite rule for
removing
.html extension in location / {} directive, causes either 499(mostly) or
“750 rewrite or internal redirection cycle” error. I am not able to
figure
out what actually causing the issue.

my NGINX.conf******
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log notice;
pid /var/run/nginx.pid;
worker_rlimit_nofile 30000;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;

log_format main '$remote_addr - $remote_user [$time_local] $request ’
'“$status” $body_bytes_sent “$http_referer” ’
‘“$http_user_agent” “$http_x_forwarded_for”’;
log_format combined_time ‘$remote_addr - $remote_user [$time_local]’
'“$request” $status $body_bytes_sent ’
‘“$http_referer” “$http_user_agent” $request_time’;

access_log /var/log/nginx/access.log combined_time;
include /etc/nginx/servers/*.conf;
}


*************************VHOST SETTING
server {
listen 80;
server_name www.aaa.com;
add_header Cache-Control off;
expires 1d;
root /var/empty;
error_log /var/log/nginx/www.aaa.com-error.log;
access_log /var/log/nginx/www.aaa.com-access.log
combined_time;
location = /favicon.ico {
root /www;
}
proxy_intercept_errors on;
error_page 400 401 402 403 404 406 407 408 409 410 411 412 413 414 415
416
417 495 496 497 500 501 502 503 504 505 506 507 = /error_page.pl;
error_page 405 =200 $uri;
location /error_page.pl {
fastcgi_pass 127.0.0.1:8999;
fastcgi_param REQUEST_URI $request_uri;
fastcgi_pass_header “Status”;
}

location / {

Rewrite root to index of bbb folder.

rewrite ^/$ AAA Sorry Page permanent;

Rewrite all directory lookups to ‘index.html’

rewrite ^(.*)/$ $1/index.html permanent;

Rewrite all open strings to index.html

rewrite ^(.*)/([^.?/]+)$ $1/$2/index.html permanent;
proxy_redirect off;
proxy_set_header Host www.aaa.com.s3.amazonaws.com;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Referer www.aaa.com;
proxy_pass http://www.aaa.com.s3.amazonaws.com/;
add_header Access-Control-Allow-Origin *;
}
}

–Manish

Posted at Nginx Forum:

Hello!

On Fri, Aug 08, 2014 at 10:49:15PM -0400, manish-ezest wrote:

Hello Maxim,

The server_name(website) is a dummy name as I don’t want to disclose the
site name. I might have mistaken while editing the name. But I ensure you
that the logs I have provided is correct. I am pasting my original

Editing configs and logs is usually very bad idea. Instead, it’s
much better to reproduce the problem in a sandbox which doesn’t
contain information you don’t want to share.

configuration which is working like a charm. If I change anything from it
like adding try_files as you suggested, or some rewrite rule for removing
.html extension in location / {} directive, causes either 499(mostly) or
“750 rewrite or internal redirection cycle” error. I am not able to figure
out what actually causing the issue.

There is a number of ways to find it out, and most powerful one is
debugging log, see here:

http://nginx.org/en/docs/debugging_log.html

[…]

proxy_pass http://www.aaa.com.s3.amazonaws.com/;

Note well that try_files (and index) won’t work if you don’t
actually have files, but proxy everything to another server.


Maxim D.
http://nginx.org/

Hello!

On Thu, Aug 07, 2014 at 12:02:43PM -0400, manish-ezest wrote:

Hello Maxim,

Thank you for your response. Here is the my nginx settings

[…]

server {
listen 80;
server_name www-aaa.com;

Note that server_name doesn’t match domain in logs you’ve provided
and in other parts of your config.

[…]

proxy_intercept_errors on;
error_page 400 401 402 403 404 406 407 408 409 410 411 412 413 414 415 416
417 495 496 497 500 501 502 503 504 505 506 507 = /error_page.pl;
error_page 405 =200 $uri;

I don’t see anything in the config provided which may trigger
internal redirect to “/index.html” except this “error_page 405
=200 $uri”. But this shouldn’t trigger a loop unless you are also
using “recursive_error_pages”, which is off by default.

Are you sure you are looking into the right config?


Maxim D.
http://nginx.org/

Hello!

On Fri, Aug 29, 2014 at 10:08:57AM -0400, manish-ezest wrote:

#rewrite ^(.*)/([^.?/]+)$ $1/$2/index.html permanent;

server: www.aaa.com, request: “GET /favicon.ico HTTP/1.1”, upstream:
“fastcgi://127.0.0.1:8999”, host: “www.aaa.com
2014/08/29 09:40:58 [error] 935#0: *64 upstream timed out (110: Connection
timed out) while reading response header from upstream, client: 11.11.11.11,
server: www.aaa.com, request: “GET /favicon.ico HTTP/1.1”, upstream:
“fastcgi://127.0.0.1:8999”, host: “www.aaa.com
2014/08/29 09:41:08 [error] 935#0: *64 upstream timed out (110: Connection
timed out) while reading response header from upstream, client: 11.11.11.11,
server: www.aaa.com, request: “GET /favicon.ico HTTP/1.1”, upstream:
“fastcgi://127.0.0.1:8999”, host: “www.aaa.com

[…]

From the logs it looks like the loop is caused by error_page you
use, and recursive_error_pages set somewhere in your
configuration. As long as your error handler script times out
(likely because it can’t cope with load), nginx tries to call it
again, then again, then again. This continues till nginx reaches
the redirections limit and logs the message about it.

Find where recursive_error_pages is set and switch it off - or,
rather, just remove it from the configuration. As previously
said, the default is correct and should not be changed unless you
understand what you are doing. This should resolve your problem.


Maxim D.
http://nginx.org/

Hello Maxim,

Like you suggested I have set “recursive_error_pages” to off but still I
am
facing the problem. This time I am getting “504 Gateway Time-out” error.
I
have already shared my NGINX and vhost configuration. We have one
fastcgi
script running for serving error pages which checks the entry of
sample.xml(contains url) file and redirect the link to particular
location.
If it doesn’t find any page then it returns a 404 page. I am pasting the
log
file of fastcgi script as well.

==> fastcgi-404.log <==
[2014-09-01T15:45:24] Got request for [/bbb/ccc/index.html] on host
[www.aaa.com]
[2014-09-01T15:45:24] Target not found in sample.xml, importing default
404
[/bbb/fff/error_404.html]
[2014-09-01T15:45:24] Retrieving target
[AAA Sorry Page]

==> www.aaa.com-error.log <==
2014/09/01 15:45:34 [error] 15900#0: *175 upstream timed out (110:
Connection timed out) while reading response header from upstream,
client:
11.11.11.11, server: www.aaa.com, request: “GET /bbb/ccc/index.html
HTTP/1.1”, upstream: “fastcgi://127.0.0.1:8999”, host: “www.aaa.com
2014/09/01 15:45:34 [error] 15900#0: *175 open() “/www/favicon.ico”
failed
(2: No such file or directory), client: 11.11.11.11, server:
www.aaa.com,
request: “GET /favicon.ico HTTP/1.1”, host: “www.aaa.com
2014/09/01 15:45:44 [error] 15900#0: *175 upstream timed out (110:
Connection timed out) while reading response header from upstream,
client:
11.11.11.11, server: www.aaa.com, request: “GET /favicon.ico HTTP/1.1”,
upstream: “fastcgi://127.0.0.1:8999”, host: “www.aaa.com
2014/09/01 15:46:57 [error] 15900#0: *178 upstream timed out (110:
Connection timed out) while reading response header from upstream,
client:
11.11.11.11, server: www.aaa.com, request: “GET /bbb/ccc/index.html
HTTP/1.1”, upstream: “fastcgi://127.0.0.1:8999”, host: “www.aaa.com
2014/09/01 15:46:57 [error] 15900#0: *178 open() “/www/favicon.ico”
failed
(2: No such file or directory), client: 11.11.11.11, server:
www.aaa.com,
request: “GET /favicon.ico HTTP/1.1”, host: “www.aaa.com
2014/09/01 15:47:07 [error] 15900#0: *178 upstream timed out (110:
Connection timed out) while reading response header from upstream,
client:
11.11.11.11, server: www.aaa.com, request: “GET /favicon.ico HTTP/1.1”,
upstream: “fastcgi://127.0.0.1:8999”, host: “www.aaa.com

–Manish

Posted at Nginx Forum:

Hello Maxim,

I tried enabling debug mode but still I am not able to figure out the
issue
and its possible solutions.

I disabled this rewrites which are rewriting to index.html

Rewrite all directory lookups to ‘index.html’

#rewrite ^(.*)/$ $1/index.html permanent;

Rewrite all open strings to index.html

#rewrite ^(.*)/([^.?/]+)$ $1/$2/index.html permanent;

and added this one as you suggested but now it is going to
www.google.com(very strange)

location / {
index index.html;
try_files $uri.html $uri/ =404;
}

Error Log Content

2014/08/29 09:40:28 [error] 935#0: *64 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: “GET /favicon.ico HTTP/1.1”, upstream:
“fastcgi://127.0.0.1:8999”, host: “www.aaa.com
2014/08/29 09:40:38 [error] 935#0: *64 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: “GET /favicon.ico HTTP/1.1”, upstream:
“fastcgi://127.0.0.1:8999”, host: “www.aaa.com
2014/08/29 09:40:48 [error] 935#0: *64 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: “GET /favicon.ico HTTP/1.1”, upstream:
“fastcgi://127.0.0.1:8999”, host: “www.aaa.com
2014/08/29 09:40:58 [error] 935#0: *64 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: “GET /favicon.ico HTTP/1.1”, upstream:
“fastcgi://127.0.0.1:8999”, host: “www.aaa.com
2014/08/29 09:41:08 [error] 935#0: *64 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: “GET /favicon.ico HTTP/1.1”, upstream:
“fastcgi://127.0.0.1:8999”, host: “www.aaa.com
2014/08/29 09:44:15 [error] 957#0: *73 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: “GET /images/icons/product/chrome-48.png
HTTP/1.1”, upstream: “fastcgi://127.0.0.1:8999”, host: “www.aaa.com”,
referrer: “AAA Sorry Page
2014/08/29 09:44:15 [error] 957#0: *74 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: “GET /images/srpr/logo9w.png HTTP/1.1”,
upstream: “fastcgi://127.0.0.1:8999”, host: “www.aaa.com”, referrer:
AAA Sorry Page
2014/08/29 09:44:15 [error] 957#0: *75 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: “GET
/xjs/_/js/k=xjs.hp.en_US.QgXXqNe9H8s.O/m=sb_he,pcc/rt=j/d=1/t=zcms/rs=AItRSTMZ4gjPthsSL3eRaZarMVJV0YA7aQ
HTTP/1.1”, upstream: “fastcgi://127.0.0.1:8999”, host: “www.aaa.com”,
referrer: “AAA Sorry Page
2014/08/29 09:44:15 [error] 957#0: *76 upstream timed out (110:
Connection
timed out) while reading response header from upstream, client:
11.11.11.11,
server: www.aaa.com, request: “GET /images/srpr/nav_logo80.png
HTTP/1.1”,
upstream: “fastcgi://127.0.0.1:8999”, host: “www.aaa.com”, referrer:
AAA Sorry Page

Access Log Content

11.11.11.11 - - [29/Aug/2014:09:44:32 +0200]“GET
/images/icons/product/chrome-48.png HTTP/1.1” 499 0
AAA Sorry Page” “Mozilla/5.0 (Windows NT 6.1;
WOW64)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.143
Safari/537.36”
26.375
11.11.11.11 - - [29/Aug/2014:09:44:32 +0200]“GET /images/srpr/logo9w.png
HTTP/1.1” 499 0 “AAA Sorry Page” “Mozilla/5.0
(Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/36.0.1985.143 Safari/537.36” 26.221
11.11.11.11 - - [29/Aug/2014:09:44:32 +0200]“GET
/xjs/_/js/k=xjs.hp.en_US.QgXXqNe9H8s.O/m=sb_he,pcc/rt=j/d=1/t=zcms/rs=AItRSTMZ4gjPthsSL3eRaZarMVJV0YA7aQ
HTTP/1.1” 499 0 “AAA Sorry Page” “Mozilla/5.0
(Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/36.0.1985.143 Safari/537.36” 26.222
11.11.11.11 - - [29/Aug/2014:09:44:32 +0200]“GET
/images/srpr/nav_logo80.png
HTTP/1.1” 499 0 “AAA Sorry Page” “Mozilla/5.0
(Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML, like Gecko)
Chrome/36.0.1985.143 Safari/537.36” 26.222

Regards,
Manish

Posted at Nginx Forum:

OK thanks Maxim. I will check your recommendation and will let you know.

–Manish

Posted at Nginx Forum:

Hello!

On Mon, Sep 01, 2014 at 02:05:46PM -0400, manish-ezest wrote:

Hello Maxim,

Like you suggested I have set “recursive_error_pages” to off but still I am
facing the problem. This time I am getting “504 Gateway Time-out” error. I
have already shared my NGINX and vhost configuration. We have one fastcgi
script running for serving error pages which checks the entry of
sample.xml(contains url) file and redirect the link to particular location.
If it doesn’t find any page then it returns a 404 page. I am pasting the log
file of fastcgi script as well.

The error is as clear as it could be: your backend failed to
respond in time. As previously suggested, it may be due to the
fact that it’s overloaded.

There are many options on how to fix this:

  • Improve your backend performance by optimizing the code.

  • Add more resources to the backend cluster (more processes on a
    single server and/or more servers).

  • Rethink your nginx configuration to avoid using the script
    (e.g., use nginx configuration to do redirects instead, this
    should be much more efficient).

In either case, everything in your configuration seems to work
correctly as configured.


Maxim D.
http://nginx.org/