What are the limitations of ip_hash?

Server NGINX
1

upstream tomcatcluster {
ip_hash;
server test1:8080 max_fails=0;
server test2:8080 max_fails=0;
}

It seems that the size of ip_hash is too small and it can’t hold enough
keys - or perhaps nginx clears out keys after a period of time?

I’m curious what the limitations of ip_hash are because it’s not working
100% of the time.

Thank you.

2

Hi Mark,

Can you explain ‘not working’. From my experience with ip_hash given a
large enough set of incoming IP addresses, requests would be distributed
equally across both machines. Its not so much a hash as a modulo of the
incoming ip address.

Cheers

Dave

On Tue, 30 Dec 2008 22:15:21 -0500, Mark Swanson
[email protected]

3

Dave C. wrote:

Hi Mark,

Can you explain ‘not working’. From my experience with ip_hash given a
large enough set of incoming IP addresses, requests would be distributed
equally across both machines. Its not so much a hash as a modulo of the
incoming ip address.

  1. IP 1.1.1.1 -> ip_hash -> server A
  2. IP 2.2.2.2 -> ip_hash -> server B
  3. IP 1.1.1.1 -> ip_hash -> server A

so far so good. 1.1.1.1 always goes to server A just like the docs
state. Until …

  1. IP 1.1.1.1 -> ip_hash -> server B

This should never happen if I understand ip_hash correctly, yet this it
seems to happen about 10% of the time (not useful to anyone needing
sticky sessions).

Btw, modulo? If you have the source handy would you mind posting the
code for this? I was really hoping the class C address would be used as
a key into a hash table and the value was the previously used server ID.

I was also really hoping this hash table would be preserved after
sending a HUP signal to nginx.

Cheers.

4

Indeed - what does your error log say ? I notice you have max fails=0
which means (AFAIK) that a temporarily failed backend will be re added
to the queue immediately.

Cheers

Dave

5

On śro, gru 31, 2008 at 02:58:30 -0500, Mark Swanson wrote:

seems to happen about 10% of the time (not useful to anyone needing
sticky sessions).

If I understand the source correctly, this happens when one of your
backends is down (as determined by Nginx). The IP address is then
rehashed and the connection retried until it succeeds or you run out of
backends.

Btw, modulo? If you have the source handy would you mind posting the
code for this? I was really hoping the class C address would be used as
a key into a hash table and the value was the previously used server ID.

The two functions starting from here:

I was also really hoping this hash table would be preserved after
sending a HUP signal to nginx.

There’s no hash table in ip_hash, there’s a hash function which maps
incoming IP to a server id. It isn’t remembered anywhere. But the
backend up/down status and failure count is indeed reset upon SIGHUP
(AFAIK).

Best regards,
Grzegorz N.

6

Grzegorz N. wrote:

This should never happen if I understand ip_hash correctly, yet this it
a key into a hash table and the value was the previously used server ID.
backend up/down status and failure count is indeed reset upon SIGHUP
(AFAIK).

Best regards,
Grzegorz N.

Ok, thank you for the info.

7

Dave C. wrote:

Indeed - what does your error log say ? I notice you have max fails=0

Error log: er… hold the phone…
I’ve just finished some analysis of the session conversations and found:

  1. Funambol client connects to SWorld to start a sync.
  2. Funambol client takes 80 minutes to figure out what to send.
  3. Funambol client sends, and fails because the session timed out and
    has been deleted.

When a session is terminated it looks like nginx is routing a request to
the wrong machine because the session isn’t there, but it’s not nginx’s
fault.

Summary:
I have no evidence that nginx ip_hash is at fault.
I’m sorry for the noise. I’m thankful for the responses.

which means (AFAIK) that a temporarily failed backend will be re added
to the queue immediately.

I did this because the docs don’t state what to look for in the
error.log file if a server is marked as inoperative. I was worried that
a server was being marked as inoperative and the requests are being sent
to the wrong machine. I should be able to force this condition and
derive the text from the logs. If I get around to doing this I’ll post
to the list.

Cheers.