Re: Fair Proxy Balancer

Thanks. Perhaps my needs are unique, but from my perspective what
matters is how fast my upstreams are completing requests. So for
instance, if nginx is able to make a connection to upstream x and get a
completed response back from it in 5ms, yet upstreams y and z take about
10ms, then I would want it to weight upstream x to get more requests.
That seems to be the bottom line for what should matter. Unfortunately
I am not grasping how your fair module’s weighting works if not based on
the overall time to serve a request. In my case I will never be running
at full capacity. If there is an internal network issue that affects
the connection/transmit time from nginx to a particular upstream, then
I’d like that upstream to be penalized and get let traffic until the
condition improves and it “proves itself” equally fast again. Hope this
makes sense…?

  ____________________________________________________________________________________

Be a better friend, newshound, and
know-it-all with Yahoo! Mobile. Try it now.
http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ

On Tue, May 06, 2008 at 08:12:24AM -0700, Rt Ibmer wrote:

Thanks. Perhaps my needs are unique, but from my perspective what matters is how fast my upstreams are completing requests. So for instance, if nginx is able to make a connection to upstream x and get a completed response back from it in 5ms, yet upstreams y and z take about 10ms, then I would want it to weight upstream x to get more requests. That seems to be the bottom line for what should matter. Unfortunately I am not grasping how your fair module’s weighting works if not based on the overall time to serve a request. In my case I will never be running at full capacity. If there is an internal network issue that affects the connection/transmit time from nginx to a particular upstream, then I’d like that upstream to be penalized and get let traffic until the condition improves and it “proves itself” equally fast again. Hope this makes sense…?

Hi,

The original use case for upstream_fair is slightly different. Consider
a site where some URLs take more time to serve (e.g. file uploads, big
reports etc.). The default balancer knows nothing about it so it may
schedule requests to a busy backend, even though there are idle ones
available. upstream_fair always chooses an idle backend if possible.

This behaviour extends to a case when all backends are servicing several
requests at once – upstream_fair schedules to a backend which has the
least outstanding request (basically a weighted-least-connections
round-robin).

Tracking latency is definitely interesting but I see several obstacles
to implementing it in upstream_fair.

  1. There’s no load balancer hook to call when a response starts. This
    means that large responses would get penalised because they (naturally)
    take longer.

  2. Not every request takes the same amount of time. If a backend gets a
    “hard” request, it would again get penalised, although any other backend
    would take as much time to service it. You could try to work around this
    by smoothing the latency estimate (exponential weighted moving average
    is particularly nice for this) but it’s never going to be really
    representative.

  3. Do you really feel that the network latency differences will be
    consistent enough to automatically tune the algorithm? I’d guess they’ll
    drown in e.g. database access times, though that obviously depends on
    the application.

  4. Tracking latency implies keeping peristent per-request state, instead
    of per-backend. And managing memory to keep this state will be
    non-trivial (high fragmentation risk etc.)

If you know that certain backends are faster, you may always set their
weights higher.

Overall, yes, I’d say your needs are unique but feel free to send a
patch :slight_smile:

Best regards,
Grzegorz N.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs