# HttpHealthcheckModule server not marked down

**URL:** <https://www.ruby-forum.com/t/httphealthcheckmodule-server-not-marked-down/212134>\
**Category:** NGINX\
**Created:** [September 22, 2011, 10:35pm UTC](https://www.ruby-forum.com/t/httphealthcheckmodule-server-not-marked-down/212134 "2011-09-22T22:35:27Z")\
**Posts on this page:** 10\
**Page:** 1

<div class="post-metadata">

**Author:** ![luislavena](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/l/0ea827/32.png) [@luislavena](https://www.ruby-forum.com/u/luislavena)\
**Post date:** [September 22, 2011, 10:35pm UTC](https://www.ruby-forum.com/t/httphealthcheckmodule-server-not-marked-down/212134/1 "2011-09-22T22:35:27Z")

</div>

Hi there,

i’m trying to use the HttpHealthcheckModule for nginx, but i have some  
troubles with it.

i have two servers in my upstream, when sabotaging the health for one  
server i see in the status view of healthcheck that the server is  
down(1), but if i go to the website i’m checking i still come out on  
it and see a broken page.

how can i arrange that the server automatically is marked as down when  
the check fails?

sorry for my bad english and maybe noob questions.

config:  
upstream www-health{  
server x.x.x.1 ;  
server x.x.x.2 ;  
healthcheck\_enabled;  
healthcheck\_delay 10000 ;  
healthcheck\_timeout 1000;  
healthcheck\_failcount 2;  
#healthcheck\_expected ‘I\_AM\_ALIVE’;  
#Important: HTTP/1.0  
healthcheck\_send “GET / HTTP/1.0” ‘Host: [health.test.x.com](http://health.test.x.com)’  
‘Conection: close’ ;  
}

nginx: nginx version: nginx/1.0.6  
nginx: built by gcc 4.4.3 (Ubuntu 4.4.3-4ubuntu5)  
nginx: TLS SNI support enabled  
nginx: configure arguments: --with-http\_ssl\_module  
–add-module=/gnosek-nginx-upstream-fair-2131c73  
–with-http\_stub\_status\_module  
–add-module=/cep21-healthcheck\_nginx\_upstreams-b33a846  
–prefix=/usr/local/nginx-1.0.6 --with-debug

used:  
peckhardt@test-nginx:~/nginx-1.0.6$patch -p1 \<  
/cep21-healthcheck\_nginx\_upstreams-5fa4bff/nginx.patch

hope someone would help me.

greetings

---

<div class="post-metadata">

**Author:** ![Sjaak\_P](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/fbc32d/32.png) [@Sjaak\_P](https://www.ruby-forum.com/u/Sjaak_P)\
**Post date:** [September 30, 2011, 11:07am UTC](https://www.ruby-forum.com/t/httphealthcheckmodule-server-not-marked-down/212134/2 "2011-09-30T11:07:01Z")

</div>

Hi,

I believe that the healthcheck module is only used for checking if the  
backend(node) is down.

As I read it, it doesn’t do anything with bringing the node “Down” in  
the  
nginx config but maybe I misunderstand it.

Regards,  
Jaap

---

<div class="post-metadata">

**Author:** ![Sjaak\_P](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/fbc32d/32.png) [@Sjaak\_P](https://www.ruby-forum.com/u/Sjaak_P)\
**Post date:** [October 1, 2011, 12:06am UTC](https://www.ruby-forum.com/t/httphealthcheckmodule-server-not-marked-down/212134/3 "2011-10-01T00:06:55Z")

</div>

Hi,

It is a bug.

the ngx\_upstream\_get\_peer only check the index greater than i; forgot  
to  
check i itself.

I used my nginx patch for healthcheck, I have used it in production  
more  
than half a year. I will upload it to my github in some hours.

liseen

---

<div class="post-metadata">

**Author:** ![Sjaak\_P](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/fbc32d/32.png) [@Sjaak\_P](https://www.ruby-forum.com/u/Sjaak_P)\
**Post date:** [October 1, 2011, 10:35am UTC](https://www.ruby-forum.com/t/httphealthcheckmodule-server-not-marked-down/212134/4 "2011-10-01T10:35:31Z")

</div>

Hi,

On Sat, Oct 1, 2011 at 7:16 AM, liseen [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> if you use healthcheck with upstream hash, please compile with branch  
> support\_http\_healthchecks of cep21’s fork
> 
> [GitHub - cep21/nginx\_upstream\_hash at support\_http\_healthchecks · GitHub](https://github.com/cep21/nginx_upstream_hash/tree/support_http_healthchecks)

if all upstreams’ backends are down(healthcheck), cep’s upstream\_hash  
will  
ignore Healthcheck, if it is not you need, Please try:  
[GitHub - liseen/nginx\_upstream\_hash: An hashing load-balancer for nginx · GitHub](https://github.com/liseen/nginx_upstream_hash)

If you find something wrong, please open an issue on github. thanks.

liseen

> 

liseen

---

<div class="post-metadata">

**Author:** ![Sjaak\_P](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/fbc32d/32.png) [@Sjaak\_P](https://www.ruby-forum.com/u/Sjaak_P)\
**Post date:** [October 1, 2011, 1:16am UTC](https://www.ruby-forum.com/t/httphealthcheckmodule-server-not-marked-down/212134/5 "2011-10-01T01:16:38Z")

</div>

Hi,

Please try:

> <https://github.com/liseen/healthcheck_nginx_upstreams/blob/master/healthcheck.patch>

patch -p1 \< healthcheck.patch  
./configure …

if you use healthcheck with upstream hash, please compile with branch  
support\_http\_healthchecks of cep21’s fork

> **[GitHub - cep21/nginx\_upstream\_hash at support\_http\_healthchecks](https://github.com/cep21/nginx_upstream_hash/tree/support_http_healthchecks)**
>
> support\_http\_healthchecks

liseen

---

<div class="post-metadata">

**Author:** ![Sjaak\_P](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/fbc32d/32.png) [@Sjaak\_P](https://www.ruby-forum.com/u/Sjaak_P)\
**Post date:** [October 3, 2011, 7:54pm UTC](https://www.ruby-forum.com/t/httphealthcheckmodule-server-not-marked-down/212134/6 "2011-10-03T19:54:04Z")

</div>

Hi, sjaak

On Mon, Oct 3, 2011 at 10:39 PM, Sjaak P. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
wrote:

> Hey Liseen,
> 
> Thank you for teh fix, it’s working now for standard round robin, for  
> upstream\_hash it’s not working but that’s no problem for us we are not  
> using that in production.

The fail test also add one time to hash\_again. Can you set hash\_again  
10(greater than or equal servers’s number), try again and tell me the  
result? The upstream\_hash module has worked for some time.

what we often use is gnosek-nginx-upstream-fair, to make it work with

> that, can you tell how to handle?

Patch the upstream-fair module like round\_robin and upstream\_hash  
module.

Maybe these should be a module that contains all of the following  
features:  
RR, Hash, Fair, Health check. Hope some body will provide such module.  
I  
don’t like patching code.

---

<div class="post-metadata">

**Author:** ![Sjaak\_P](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/fbc32d/32.png) [@Sjaak\_P](https://www.ruby-forum.com/u/Sjaak_P)\
**Post date:** [October 4, 2011, 9:40am UTC](https://www.ruby-forum.com/t/httphealthcheckmodule-server-not-marked-down/212134/7 "2011-10-04T09:40:48Z")

</div>

Hi Liseen,

setting hash\_again greater than the number of servers works, equal not.

i think i will get in the code some day to patch the upstream\_fair,  
but than i need some free time.

thnx for the help.

grt

2011/10/3 liseen [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):

---

<div class="post-metadata">

**Author:** ![Sjaak\_P](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/fbc32d/32.png) [@Sjaak\_P](https://www.ruby-forum.com/u/Sjaak_P)\
**Post date:** [October 3, 2011, 4:40pm UTC](https://www.ruby-forum.com/t/httphealthcheckmodule-server-not-marked-down/212134/8 "2011-10-03T16:40:11Z")

</div>

Hey Liseen,

Thank you for teh fix, it’s working now for standard round robin, for  
upstream\_hash it’s not working but that’s no problem for us we are not  
using that in production.  
what we often use is gnosek-nginx-upstream-fair, to make it work with  
that, can you tell how to handle?

this is what i’ve done to make it work for now:

peckhardt@test-nginx:~/nginx-1.0.6$ patch -p1 \<  
…/liseen-healthcheck\_nginx\_upstreams-17298cf/healthcheck.patch  
patching file src/http/ngx\_http\_upstream.c  
Hunk #1 succeeded at 4270 (offset 11 lines).  
patching file src/http/ngx\_http\_upstream.h  
patching file src/http/ngx\_http\_upstream\_round\_robin.c  
Hunk #2 succeeded at 25 with fuzz 2 (offset 9 lines).  
Hunk #3 succeeded at 33 (offset 9 lines).  
Hunk #4 succeeded at 68 (offset 9 lines).  
Hunk #5 succeeded at 416 (offset 7 lines).  
Hunk #6 succeeded at 448 (offset 7 lines).  
Hunk #7 succeeded at 465 (offset 7 lines).  
Hunk #8 succeeded at 506 (offset 7 lines).  
Hunk #9 succeeded at 523 (offset 7 lines).  
Hunk #10 succeeded at 617 (offset 7 lines).  
patching file src/http/ngx\_http\_upstream\_round\_robin.h

peckhardt@test-nginx:~/nginx-1.0.6$sudo ./configure  
–with-http\_ssl\_module  
–add-module=/home/peckhardt/gnosek-nginx-upstream-fair-2131c73  
–with-http\_stub\_status\_module  
–add-module=/home/peckhardt/liseen-healthcheck\_nginx\_upstreams-17298cf  
–add-module=/home/peckhardt/liseen-nginx\_upstream\_hash-43fab03  
–prefix=/usr/local/nginx-1.0.6 --with-debug

peckhardt@test-nginx:~/nginx-1.0.6$sudo su  
peckhardt@test-nginx:~/nginx-1.0.6$make install clean

nginx config:  
########### test healthcheck ######  
upstream www-health{  
server 213.154.235.185 ;  
server 213.136.14.13 ;  
#hash $request\_uri;  
#hash\_again 1;  
healthcheck\_enabled;  
healthcheck\_delay 10000 ;  
healthcheck\_timeout 1000;  
healthcheck\_failcount 2;  
#healthcheck\_expected ‘I\_AM\_ALIVE’;  
#Important: HTTP/1.0  
healthcheck\_send “GET / HTTP/1.0” ‘Host: [health.test.x.com](http://health.test.x.com)’;  
}

2011/10/1 liseen [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):

---

<div class="post-metadata">

**Author:** ![Sjaak\_P](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/fbc32d/32.png) [@Sjaak\_P](https://www.ruby-forum.com/u/Sjaak_P)\
**Post date:** [April 21, 2012, 10:52pm UTC](https://www.ruby-forum.com/t/httphealthcheckmodule-server-not-marked-down/212134/9 "2012-04-21T22:52:01Z")

</div>

I confirmed that this is an issue only when ip\_hash is used - without it  
healthcheck fail causes upstream to stop routing requests to failing  
node. Could use a bit of help understanding:

- is this by design (i.e ip\_hash support never implemented) or a bug?

I am happy to create an enhancement/fix but, as a c-newbie, would  
appreciate a bit of guidance on where specifically to look in nginx or

> <https://github.com/liseen/healthcheck_nginx_upstreams/blob/master/ngx_http_healthcheck_module.c>

thanks!

Posted at Nginx Forum:

> **[Re: HttpHealthcheckModule server not marked down](https://forum.nginx.org/read.php?2,215656,225595#msg-225595)**
>
> Hi there, i'm trying to use the HttpHealthcheckModule for nginx, but i have some troubles with it. i have two servers in my upstream, when sabotaging the health for one server i see in the status view of healthcheck that the server is down(1), but if...

---

<div class="post-metadata">

**Author:** ![Sjaak\_P](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/s/fbc32d/32.png) [@Sjaak\_P](https://www.ruby-forum.com/u/Sjaak_P)\
**Post date:** [April 21, 2012, 4:23am UTC](https://www.ruby-forum.com/t/httphealthcheckmodule-server-not-marked-down/212134/10 "2012-04-21T04:23:13Z")

</div>

Hello,

I was seeing the same issue: when upstream node is down, healthcheck  
correctly flags it as down yet subsequent requests are still routed to  
that node (even though others are healthy).

Tried this patch with 1.0.13 and .15 - no relief:

> <https://github.com/liseen/healthcheck_nginx_upstreams/blob/master/healthcheck.patch>

Is the above supposed to work when ip\_hash directive is used?

Thank you,

-nikita

My config:

```
    upstream admin-cluster {
            ip_hash; # clientIP-based session affinity

```

healthcheck\_enabled;  
healthcheck\_send “GET /running HTTP/1.0” “Host: [www.mydomain.com](http://www.mydomain.com)”  
“User-Agent: FooBar/1.0 nginx” “Connection: close”;  
healthcheck\_delay 2000;  
healthcheck\_failcount 2;  
healthcheck\_timeout 2000;  
server [admin1.staging.mydomain.com:8080](http://admin1.staging.mydomain.com:8080) max\_fails=1 fail\_timeout=5; #max  
sec to connect to upstream host  
}

Posted at Nginx Forum:

> **[Re: HttpHealthcheckModule server not marked down](https://forum.nginx.org/read.php?2,215656,225577#msg-225577)**
>
> Hi there, i'm trying to use the HttpHealthcheckModule for nginx, but i have some troubles with it. i have two servers in my upstream, when sabotaging the health for one server i see in the status view of healthcheck that the server is down(1), but if...