# 404 fallback only hitting 2/3 upstream sets

**URL:** <https://www.ruby-forum.com/t/404-fallback-only-hitting-2-3-upstream-sets/221969>\
**Category:** NGINX\
**Created:** [August 7, 2012, 6:55pm UTC](https://www.ruby-forum.com/t/404-fallback-only-hitting-2-3-upstream-sets/221969 "2012-08-07T18:55:52Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![aris](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/258eb7/32.png) [@aris](https://www.ruby-forum.com/u/aris)\
**Post date:** [August 7, 2012, 6:55pm UTC](https://www.ruby-forum.com/t/404-fallback-only-hitting-2-3-upstream-sets/221969/1 "2012-08-07T18:55:52Z")

</div>

I currently have a setup where I have 3 upstream sets (directives?), and  
it will fallback in a chain. So if it gets a 404 from setA, it’ll go to  
setB. If setB also has a 404, then setC. The config is:

upstream setA {  
server ipA:port  
}

upstream setB {  
server ipB:port  
}

upstream setC {  
server ipC:port  
}

```
server {
    listen 80;
    server_name localhost;

  #SetAHandler
  location /
  {
    access_log logs/access.log main;

    error_page 404 502 503 504 = @SetBHandler;
    proxy_intercept_errors on;
    proxy_pass http://setA;
    proxy_set_header Host $host;
    proxy_connect_timeout 60s;
    proxy_next_upstream error timeout http_500 http_502 http_503

```

http\_504 http\_404;

```
  # proxy_cache nginx_cache;
  # proxy_cache_valid 200 304 12h;
  # expires 1d;

  # break;
  }

  location @SetBHandler
  {
    access_log logs/access.log main;
    proxy_intercept_errors on;

    error_page 404 502 503 504 = @SetCHandler;
    proxy_pass http://setB;
    proxy_set_header Host $host;
    proxy_connect_timeout 60s;
    proxy_next_upstream error timeout http_500 http_502 http_503

```

http\_504 http\_404;

```
  # proxy_cache nginx_cache;
  # proxy_cache_valid 200 304 12h;
  # expires 1d;

  # break;
  }

  location @SetCHandler
  {
    access_log logs/access.log main;
    proxy_pass http://setC;
    proxy_set_header Host $host;
    proxy_connect_timeout 60s;
    proxy_next_upstream error timeout http_500 http_502 http_503

```

http\_504 http\_404;  
#Last proxy, return whatever it gives  
proxy\_intercept\_errors off;

```
  # proxy_cache nginx_cache;
  # proxy_cache_valid 200 304 12h;
  # expires 1d;

  # break;
  }

```

The problem is, it only seems to check setA and setB, and bypasses setC  
completely. I tested it by putting a file on the third IP in setC that  
wasn’t in setA or setB. Whenever I hit Nginx, it always returns 404,  
when the expected behavior is for it to return the file at ipC.

I have it logging th $upstream\_addr, and everytime it 404s, the logs  
always show:

:

Is there some directive from the HttpProxyModule that limits it to 2 by  
default?

Posted at Nginx Forum:

> **[404 fallback only hitting 2/3 upstream sets](https://forum.nginx.org/read.php?2,229402,229402#msg-229402)**
>
> I currently have a setup where I have 3 upstream sets (directives?), and it will fallback in a chain. So if it gets a 404 from setA, it'll go to setB. If setB also has a 404, then setC. The config is: upstream setA { server ipA:port } upstream setB {...

---

<div class="post-metadata">

**Author:** ![helliax](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/d2c977/32.png) [@helliax](https://www.ruby-forum.com/u/helliax)\
**Post date:** [August 7, 2012, 7:16pm UTC](https://www.ruby-forum.com/t/404-fallback-only-hitting-2-3-upstream-sets/221969/2 "2012-08-07T19:16:20Z")

</div>

I think i got it. I had to add the following directive under the server  
{ } heading:

recursive\_error\_pages on;

I’m checking the access logs, and I’m seeing the third one now (ipA :  
ipB : ipC), and it’s also returning the file correctly. I hope this is  
the right way.

Posted at Nginx Forum:

> **[Re: 404 fallback only hitting 2/3 upstream sets](https://forum.nginx.org/read.php?2,229402,229405#msg-229405)**
>
> I currently have a setup where I have 3 upstream sets (directives?), and it will fallback in a chain. So if it gets a 404 from setA, it'll go to setB. If setB also has a 404, then setC. The config is: upstream setA { server ipA:port } upstream setB {...

---

<div class="post-metadata">

**Author:** ![helliax](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/d2c977/32.png) [@helliax](https://www.ruby-forum.com/u/helliax)\
**Post date:** [August 8, 2012, 12:31pm UTC](https://www.ruby-forum.com/t/404-fallback-only-hitting-2-3-upstream-sets/221969/3 "2012-08-08T12:31:23Z")

</div>

Hello!

On Tue, Aug 07, 2012 at 01:15:45PM -0400, helliax wrote:

> I think i got it. I had to add the following directive under the server  
> { } heading:
> 
> recursive\_error\_pages on;
> 
> I’m checking the access logs, and I’m seeing the third one now (ipA :  
> ipB : ipC), and it’s also returning the file correctly. I hope this is  
> the right way.

It is, more or less. You should be careful though, as  
recursive\_error\_pages set to on might easily result in infinite  
loop (nginx will break it eventually, but nevertheless). I would  
recommend limiting recursive\_error\_pages scope to minimum  
possible.

In your case, it should be enough to set recursive\_error\_pages to  
on in “location /”.

Maxim D.

---

<div class="post-metadata">

**Author:** ![helliax](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/d2c977/32.png) [@helliax](https://www.ruby-forum.com/u/helliax)\
**Post date:** [August 8, 2012, 6:36pm UTC](https://www.ruby-forum.com/t/404-fallback-only-hitting-2-3-upstream-sets/221969/4 "2012-08-08T18:36:07Z")

</div>

Thanks for heads up, Maxim. In what cases would it go into an infinite  
loop if I left it outside? I had thought that “location /” would catch  
all requests, and setting

proxy\_intercept\_errors off

at the last handler would break the chain by returning whatever upstream  
setC returned?

Posted at Nginx Forum:

> **[Re: 404 fallback only hitting 2/3 upstream sets](https://forum.nginx.org/read.php?2,229402,229466#msg-229466)**
>
> I currently have a setup where I have 3 upstream sets (directives?), and it will fallback in a chain. So if it gets a 404 from setA, it'll go to setB. If setB also has a 404, then setC. The config is: upstream setA { server ipA:port } upstream setB {...

---

<div class="post-metadata">

**Author:** ![helliax](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/d2c977/32.png) [@helliax](https://www.ruby-forum.com/u/helliax)\
**Post date:** [August 8, 2012, 6:55pm UTC](https://www.ruby-forum.com/t/404-fallback-only-hitting-2-3-upstream-sets/221969/5 "2012-08-08T18:55:23Z")

</div>

That makes sense. Thanks so much for all your help.

Posted at Nginx Forum:

> **[Re: 404 fallback only hitting 2/3 upstream sets](https://forum.nginx.org/read.php?2,229402,229468#msg-229468)**
>
> I currently have a setup where I have 3 upstream sets (directives?), and it will fallback in a chain. So if it gets a 404 from setA, it'll go to setB. If setB also has a 404, then setC. The config is: upstream setA { server ipA:port } upstream setB {...

---

<div class="post-metadata">

**Author:** ![helliax](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/d2c977/32.png) [@helliax](https://www.ruby-forum.com/u/helliax)\
**Post date:** [August 8, 2012, 6:51pm UTC](https://www.ruby-forum.com/t/404-fallback-only-hitting-2-3-upstream-sets/221969/6 "2012-08-08T18:51:53Z")

</div>

Hello!

On Wed, Aug 08, 2012 at 12:35:41PM -0400, helliax wrote:

> Thanks for heads up, Maxim. In what cases would it go into an infinite  
> loop if I left it outside? I had thought that “location /” would catch  
> all requests, and setting
> 
> proxy\_intercept\_errors off
> 
> at the last handler would break the chain by returning whatever upstream  
> setC returned?

Even with proxy\_intercept\_errors set to off errors might happen -  
e.g. 502 will be generated if nginx won’t be able to connect to  
upstream for some reason.

In any case I don’t think that your config creates a loop (unless  
you have some error\_pages defined at http level). I’ve mostly  
wrote about recursive\_error\_pages in general.

Maxim D.