# Nginx doesn't switch upstream in some cases

**URL:** <https://www.ruby-forum.com/t/nginx-doesnt-switch-upstream-in-some-cases/183241>\
**Category:** NGINX\
**Created:** [February 10, 2010, 3:20pm UTC](https://www.ruby-forum.com/t/nginx-doesnt-switch-upstream-in-some-cases/183241 "2010-02-10T15:20:49Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![Guillaume\_F](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/5fc32e/32.png) [@Guillaume\_F](https://www.ruby-forum.com/u/Guillaume_F)\
**Post date:** [February 10, 2010, 3:20pm UTC](https://www.ruby-forum.com/t/nginx-doesnt-switch-upstream-in-some-cases/183241/1 "2010-02-10T15:20:49Z")

</div>

Hi all,

We’re running nginx as a load balancer in front of two reverse proxies  
(Apache2/mod\_security) for our web site.

It’s been working great except last night apache stopped on one of the  
reverse proxy and nginx would continue sending HTTP requests to it. The  
result was that about 50% of the requests to the web site failed.

If I unplug the reverse proxy, nginx detects that it’s down and will  
only send requests to the “good” reverse proxy.

Here’s my config, I guess that my “proxy\_next\_upstream” is wrong but I  
can’t figure out how… It would be great if someone could shed some  
light on this issue for me.

=====  
upstream SRACQ {  
server 192.168.1.57:80;  
server 192.168.1.67:80;  
}

server {  
listen 66.254.57.167:80;  
server\_name www.sracq.qc.ca;  
access\_log /var/log/nginx/www.sracq.qc.ca\_HTTP.access.log;  
error\_log /var/log/nginx/www.sracq.qc.ca\_HTTP.error.log;

location / {  
proxy\_next\_upstream error timeout invalid\_header http\_500;  
proxy\_set\_header X-Real-IP $remote\_addr;  
proxy\_set\_header X-Forwarded-For $proxy\_add\_x\_forwarded\_for;  
proxy\_set\_header Host $http\_host;

```
proxy_pass http://SRACQ;

```

# } }

Thanks a lot,  
GFK’s

---

<div class="post-metadata">

**Author:** ![Guillaume\_F](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/5fc32e/32.png) [@Guillaume\_F](https://www.ruby-forum.com/u/Guillaume_F)\
**Post date:** [February 10, 2010, 3:35pm UTC](https://www.ruby-forum.com/t/nginx-doesnt-switch-upstream-in-some-cases/183241/2 "2010-02-10T15:35:27Z")

</div>

Hello!

On Wed, Feb 10, 2010 at 09:19:07AM -0500, Guillaume F. wrote:

> only send requests to the “good” reverse proxy.
> 
> Here’s my config, I guess that my “proxy\_next\_upstream” is wrong but I  
> can’t figure out how… It would be great if someone could shed some  
> light on this issue for me.

[…]

> ```
> proxy_next_upstream error timeout invalid_header http_500;
> 
> ```

Which status code your reverse proxy returns once backend is down?  
Most likely it’s 502 or 504, so you have to add them to  
proxy\_next\_upstream.

Maxim D.

---

<div class="post-metadata">

**Author:** ![Guillaume\_F](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/5fc32e/32.png) [@Guillaume\_F](https://www.ruby-forum.com/u/Guillaume_F)\
**Post date:** [February 10, 2010, 4:35pm UTC](https://www.ruby-forum.com/t/nginx-doesnt-switch-upstream-in-some-cases/183241/3 "2010-02-10T16:35:10Z")

</div>

Maxim D. a écrit :

> Which status code your reverse proxy returns once backend is down?  
> Most likely it’s 502 or 504, so you have to add them to  
> proxy\_next\_upstream.

Thanks for the tip, effectively I should add http\_502, http\_503 and  
http\_504 to my proxy\_next\_upstream.

However, last night’s problem was not with a backend, but with a reverse  
proxy. The machine was still up (responding to pings) but apache was not  
responding.

The was no response to HTTP requests to the reverse proxy. I guess  
that’s the “timeout” directive, but somehow that didn’t work…

Thanks,  
GFK’s

---

<div class="post-metadata">

**Author:** ![Guillaume\_F](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/5fc32e/32.png) [@Guillaume\_F](https://www.ruby-forum.com/u/Guillaume_F)\
**Post date:** [February 10, 2010, 7:38pm UTC](https://www.ruby-forum.com/t/nginx-doesnt-switch-upstream-in-some-cases/183241/4 "2010-02-10T19:38:24Z")

</div>

Guillaume F. a écrit :

> However, last night’s problem was not with a backend, but with a reverse  
> proxy. The machine was still up (responding to pings) but apache was not  
> responding.
> 
> The was no response to HTTP requests to the reverse proxy. I guess  
> that’s the “timeout” directive, but somehow that didn’t work…

It turned out that I was missing this directive:  
proxy\_connect\_timeout 2;

I couldn’t find the default value, but I guess that it was too high for  
my application.

What this directive in place, when one reverse proxy is down, half the  
requests take 0.2sec to return and half take 2.2sec to return. This is  
good enough for me. 🙂

Thanks,  
GFK’s

---

<div class="post-metadata">

**Author:** ![Guillaume\_F](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/5fc32e/32.png) [@Guillaume\_F](https://www.ruby-forum.com/u/Guillaume_F)\
**Post date:** [February 12, 2010, 9:23am UTC](https://www.ruby-forum.com/t/nginx-doesnt-switch-upstream-in-some-cases/183241/5 "2010-02-12T09:23:46Z")

</div>

Piotr S. a écrit :

> > What this directive in place, when one reverse proxy is down, half the  
> > requests take 0.2sec to return and half take 2.2sec to return. This is  
> > good enough for me. 🙂
> 
> You should consider using “max\_fails” and “fail\_timeout”:  
> [GitHub - nginxinc/nginx-wiki: ARCHIVED -- Source for the now archived NGINX Wiki section of https://www.nginx.com · GitHub](http://wiki.nginx.org/NginxHttpUpstreamModule#server)

Well, that’s strange. According to the documentation, the default values  
for those are “max\_fails=1 fail\_timeout=10s”.

With those default settings (and with “timeout” in proxy\_next\_upstream),  
Guillaume shouldn’t hit the timeout on half the requests, but rather  
only once in a while, right ?

How can this happen ?

---

<div class="post-metadata">

**Author:** ![Guillaume\_F](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/5fc32e/32.png) [@Guillaume\_F](https://www.ruby-forum.com/u/Guillaume_F)\
**Post date:** [February 10, 2010, 7:46pm UTC](https://www.ruby-forum.com/t/nginx-doesnt-switch-upstream-in-some-cases/183241/6 "2010-02-10T19:46:01Z")

</div>

> What this directive in place, when one reverse proxy is down, half the  
> requests take 0.2sec to return and half take 2.2sec to return. This is  
> good enough for me. 🙂

You should consider using “max\_fails” and “fail\_timeout”:

> **[GitHub - nginxinc/nginx-wiki: ARCHIVED -- Source for the now archived NGINX Wiki...](https://github.com/nginxinc/nginx-wiki)**
>
> ARCHIVED -- Source for the now archived NGINX Wiki section of https://www.nginx.com

Best regards,  
Piotr S. \< [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) \>

---

<div class="post-metadata">

**Author:** ![Guillaume\_F](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/g/5fc32e/32.png) [@Guillaume\_F](https://www.ruby-forum.com/u/Guillaume_F)\
**Post date:** [February 12, 2010, 2:54pm UTC](https://www.ruby-forum.com/t/nginx-doesnt-switch-upstream-in-some-cases/183241/7 "2010-02-12T14:54:27Z")

</div>

Benjamin Pineau a écrit :

> With those default settings (and with “timeout” in proxy\_next\_upstream),  
> Guillaume shouldn’t hit the timeout on half the requests, but rather  
> only once in a while, right ?
> 
> How can this happen ?

Actually, my “half the time” was based on running “time wget -O  
/dev/null [http://www.sracq.qc.ca/](http://www.sracq.qc.ca/)” a few times and looking at the  
results. There’s nothing statistical in my approach. 🙂