Dual load balancer (one failsafe)?

Hi all,

Did anyone setup an nginx dual load balancer, one being set as failsafe?
If possible, please post your configuration example.

I plan to use nginx as load balancer for 3 servers, with php-fpm
installed
on each serving node.

Thanks for your help with this matter.

On Tue, Jun 10, 2008 at 7:16 AM, Floren M. [email protected] wrote:

Did anyone setup an nginx dual load balancer, one being set as failsafe?
If possible, please post your configuration example.

We use Nginx on six boxes in conjunction with Heartbeat2. The main web
server IP is automatically reassigned if the current web server box
goes down.

This setup generally works as intended, although Heartbeat2
occasionally misbehaves. It is apparently not as mature as the
original Heartbeat.

If you only have two boxes, consider using Heartbeart (not
Heartbeat2), which is also simpler to configure and maintain.

Alexander.

I don’t think that nginx dual load balancer is a configuration problem.
You can read the following article :

It’s a good way to explain you how to do that !

For summary you have two nodes (load balancer) with two Ip address. You
must create a new one for Ip aliasing. The others host must reach it by
the latter. (only one host is available, the second monitoring the state
of the first)

I hope that it is comprehensible despite my bad English :slight_smile:

On Tue, Jun 10, 2008 at 12:33 PM, Floren M. [email protected]
wrote:

We use Nginx on six boxes in conjunction with Heartbeat2. The main web
server IP is automatically reassigned if the current web server box
goes down.

I presume Heartbeat does that for you?

Yes. That’s the point of Heartbeat. You have two boxes that monitor
each other; when one goes down, the other node uses ARP to negotiate
to take over the IP. (ARP is the underlying protocol that LANs use to
assign IPs to physical machines.) One of the boxes always has two IPs.

Box A
10.0.0.1
10.0.1.100 # magic IP

Box B
10.0.0.2

In the event of a failure on box A:

Box A # now dead
10.0.0.1

Box B
10.0.0.2
10.0.1.100 # magic IP

You set your DNS (www.example.com) to point to 10.0.1.100. When the IP
is reassigned, the requests go to box B.

I was hoping we can do all the above simply by using the nginx configuration file.
I wonder if Igor can comment on that, when he is back from vacation. I would be curious to see if we can do the dual balancer setup, using only native nginx software.

Not possible if you you really mean “failsafe” in the sense that both
boxes are redundant in the event of a total failure – such as a
kernel panic, hardware crash or power disruption. For such a scenario,
the failure mechanism has to exist outside the failing box.

You could add another load-balancer to balance between box A and B,
but that merely moves the single point of failure (SPF) to another
box. It might lower the probability of total failure, but you can’t
reasonably keep adding load-balancers to resolve an SPF.

Heartbeat solves the SPF issue by relegating the routing to the
network hardware by using ARP – the non-failing box uses ARP to tell
the router that it should have the new IP. Thus your router becomes
the SPF, but that’s okay because if your router dies, there’s no
network connectivity.

You will probably want redundant routers, of course. If you have
redundant routers, your Internet connection becomes a SPF. So you will
want two connections. Then your hosting facility becomes the SPF. :slight_smile:

Alexander.

Hi Alex,

Thank you for the useful information, it is very clear now.

Regards,

Floren

Hi Alex,

Did anyone setup an nginx dual load balancer, one being set as
failsafe?
If possible, please post your configuration example.

We use Nginx on six boxes in conjunction with Heartbeat2. The main web
server IP is automatically reassigned if the current web server box
goes down.

I presume Heartbeat does that for you?

This setup generally works as intended, although Heartbeat2
occasionally misbehaves. It is apparently not as mature as the
original Heartbeat.

If you only have two boxes, consider using Heartbeart (not
Heartbeat2), which is also simpler to configure and maintain.

I was hoping we can do all the above simply by using the nginx
configuration file.
I wonder if Igor can comment on that, when he is back from vacation. I
would be curious to see if we can do the dual balancer setup, using only
native nginx software.

Regards,

Floren

Hi Vince,

http://www.howtoforge.com/high_availability_loadbalanced_apache_cluster

It’s a good way to explain you how to do that !

Yes, you are right, it is clear. However, I do not want to install Ultra
Monkey, just Heartbeat.

For summary you have two nodes (load balancer) with two Ip address. You
must create a new one for Ip aliasing. The others host must reach it by
the latter. (only one host is available, the second monitoring the
state
of the first)

Do you know a way to create the aliasing, directly on CentOS 5.1 x64?
Like editing the config files directly.

I hope that it is comprehensible despite my bad English :slight_smile:

Yes you are very comprehensible. :slight_smile:

Regards,

Floren

More thoughts on that, related to Heartbeat usage:

The problem is the proxy load balancing algorithm. Nginx only comes with
a round-robin balancer and a hash-based balancer. Only the former is of
interest to us since our object is to distribute the load evenly across
a pack of Mongrel back ends. The round-robin algorithm is often an
acceptable tool: if every request finishes within a few milliseconds,
there’s no problem.

But if a page takes a while to load, Nginx will start routing requests
to backends that are already processing requests — as a result, some
backends will be queuing up requests while some backends will remain
idle. You will get an uneven load distribution, and the unevenness will
increase with the amount of load subject to the load-balancer.

So I think a more suitable software would be HaProxy, instead of
Heartbeat.
I plan to serve PHP mixed with static files, on all nodes.

On Sun, Jun 15, 2008 at 5:24 AM, Floren M. [email protected] wrote:

More thoughts on that, related to Heartbeat usage:
Comparing Nginx and HAProxy for web applications « Affection Code
[…]

So I think a more suitable software would be HaProxy, instead of Heartbeat.
I plan to serve PHP mixed with static files, on all nodes.

HAProxy does not replace Heartbeat. They complement each other,
certainly, but if you have a single HAProxy then that’s a single point
of failure.

It would be useful if you could clarify what you are looking for,
since you seem to be missing the point, even after I gave you detailed
explanation of the nature of single points of failure.

I assumed – perhaps erroneously – you were looking for a way to
prevent your site from going down if one of your three boxes went
down. Also, since you mentioned load balancing, I assumed you wanted
an automatic prevention mechanism, as opposed to one that had to be
executed manually.

Just to reiterate, then: If you have two boxes and one of the boxes
dies, ipso facto the dead box cannot do anything about the situation.
It’s dead. If it’s your main web server (and/or load balancer),
you’re screwed. It can’t even tell anyone that it’s dead. The solution
to this problem is redundancy: delegating the task of failure
detection and failure management to several boxes that are each
capable of taking control. Heartbeat does that.

As for HAProxy vs. Nginx, they are equally good at most types of
loads. If you read my article closely you will notice that it
describes a particular scenario where Nginx’s load-balancing algorithm
is not well-suited. For most PHP apps it should be just dandy.

Alexander.

On Sun, Jun 15, 2008 at 6:52 AM, Floren M. [email protected] wrote:

What to do, use HaProxy or the built-in nginx balancer?

Stick with Nginx if you are already familiar with it and you are using
it for serving your PHP. There’s no need to complicate things with yet
another new component. I think HAProxy is a much better load balancer,
but there are pros and cons to both approaches.

You might want to check out layer 4 load-balancing as an option. The
LVS project, of which Heartbeat is a part, also offers the IPVS
system, which can load-balance network traffic to multiple nodes at
the layer 4, ie. by distributing network traffic at the packet level.

This is simpler and more limited than the kind of fancy HTTP magic
that Nginx and HAProxy can do, but it’s much faster since there no
proxying involved. Here’s a howto (based on Heartbeat 2, which you
might want to avoid):

http://www.novell.com/communities/node/4846/load-balancing-howto-lvs-ldirectord-heartbeat-2

All of this stuff requires Linux, by the way.

Alexander.

Hi Alex,

of failure.

It would be useful if you could clarify what you are looking for,
since you seem to be missing the point, even after I gave you detailed
explanation of the nature of single points of failure.

You were very useful with your explanation. I’m looking at the current
setup:

  1. Heartbeat to monitor 2 nginx load balancers (one failsafe).
  2. The load balancers (the active one) will proxy 6 web boxes, using the
    built-in round robin nginx balancer system.

So my question is more like a continuation to my current setup.
What to do, use HaProxy or the built-in nginx balancer?

Thank you for your patience.

Regards,

Floren

Yeah - I use LVS right now with ldirectord to do healthchecking.

I used to run it using heartbeat (v1 I believe) but right now I am not
even running it in active/passive mode.

I believe FreeBSD also has it’s equivalents (VRRP?) and the Wordpress
guys just mentioned another product which probably can replace the
need for heartbeat called Wackamole
(http://www.backhand.org/wackamole/)

As long as you keep your nginx configs synced up and make sure to
reload them after changes, the only thing you should need is something
to manage taking over the IPs for you; and I believe Wackamole might
be a good solution for that. LVS won’t help you with IP management
without adding in heartbeat, it is simply a layer 4 load balancing
solution.

I would look at doing 2 nginx installs, a small custom script to keep
the configs in sync, and Wackamole to manage the IP switching. In
theory Wackamole sounds like it can handle a ton of machines so you
could have multiple - not just a single active/passive pair…

Hi Alex,

What to do, use HaProxy or the built-in nginx balancer?

Alexander.

I will follow your advice and stick with nginx. Thank you for taking the
time to reply.

Regards,

Floren

On pon, cze 16, 2008 at 10:33:57 +0300, Andrei GUDIU wrote:

Salut, o rugaminte am
se poate sa trimiti si tu mailuirle catre lista NGINX ca toata lumea catre
[email protected] ?
Nu de alta dar mai are omu si niste filtre.

Hi Alex,

(snip)

Regards,

Floren

No idea what Andrei has just said but I think I agree. I (also?) have
filters
set by the recipient address and your mail ends up in my inbox instead
of the Nginx folder.

Thank you for your consideration :slight_smile:

Best regards,
Grzegorz N.

The answer is to not filter on recipient address. Use the mailing list
header that is provided for this purpose:

X-BeenThere: [email protected]

Regards,
Cliff

Salut, o rugaminte am
se poate sa trimiti si tu mailuirle catre lista NGINX ca toata lumea
catre
[email protected] ?
Nu de alta dar mai are omu si niste filtre.

Merci.

Thank you for the tipppp:)
I’ll check sieve, my squirrelmail does not have such an advanced
filtering
system :stuck_out_tongue:

Andrei GUDIU