Ruby Forum NGINX > Fair Proxy Balancer

Hello,

In case people haven't seen it:
  http://www.brainspl.at/articles/2007/11/09/a-fair-proxy-balancer-for-nginx-and-mongrel

"So EngineYard.com put out a bounty on getting a fair proxy balancer
that would keep track of when a mongrel is busy and not send requests
to it until it has finished and is ready to take requests. This means
that now we can effectively queue inside nginx very efficiently and
only send requests to non busy mongrels. Thereby avoiding the waiting
in the wrong line for service problem described above."

I am guessing that the strategy is at the network level, so it should
work with non-mongrel back-ends.

Am 22.11.2007 um 22:38 schrieb Adam Zell:

> only send requests to non busy mongrels. Thereby avoiding the waiting
> in the wrong line for service problem described above."
>
> I am guessing that the strategy is at the network level, so it should
> work with non-mongrel back-ends.

Great news, this will be a huge step also for the case of Zope zeo-
deployments. One question, how is the busy state determined? In case
of zeo each backend client can take some defined number of requests
in parallel, how is such a case handled? (Not knowing mongrels, are
they taking only one request each?)

With regards,

__Janko Hauser

On 11/22/07, Janko Hauser <jh@zscout.de> wrote:
> Am 22.11.2007 um 22:38 schrieb Adam Zell:
> > In case people haven't seen it:
> >   http://www.brainspl.at/articles/2007/11/09/a-fair-proxy-balancer-
> > for-nginx-and-mongrel
>
> Great news, this will be a huge step also for the case of Zope zeo-
> deployments.

Great news indeed. We recently switched from Lighttpd to Nginx using
this patch on a live site (20 mongrels across two boxes, load average
~3.0), and it seems completely stable.

> One question, how is the busy state determined? In case
> of zeo each backend client can take some defined number of requests
> in parallel, how is such a case handled?

I have not studied the sources, but I expect it will pick the upstream
with the fewest number of current pending requests; among upstreams
with the same number of concurrent requests, the one picked is
probably arbitrary.

> (Not knowing mongrels, are
> they taking only one request each?)

Mongrel supports multiple concurrent requests, but uses Ruby's green
threads (which I like to think are closer to yellow and shaped like a
banana) to process them.

The Mongrel Rails dispatcher uses a single process-wide lock around
the dispatching logic, meaning that a single Mongrel-Rails process can
only process on request at a time.

Alexander.

On Nov 22, 2007, at 2:16 PM, Janko Hauser wrote:

>> to it until it has finished and is ready to take requests. This means
> in parallel, how is such a case handled? (Not knowing mongrels, are  
> they taking only one request each?)
>
> With regards,
>
> __Janko Hauser


  Mongrels are single threaded when running rails so this fair balancer
tries to only send requests to mongrels that are not currently serving
requests. But once you surpass all the backends being busy it uses
some nice weighting algorithms to try and still serve requests to the
least congested backends. there is some nice rbtree and scheduling/
weighting code  in there now as well. The module works with 0.5.x and
0.6.x and once it is completely stable we plan to offer it to Igor to
see if he wants to include it in the main distro.

  There was just a new push of code so if anyone wants to play you can
go here:

http://git.localdomain.pl/?p=nginx.git;a=tree;hb=upstream_fair-0.6

  And click on snapshot to grab the latest snapshot.  Please play  with
this and report any weird results or problems you experience so we can
improve the module.

  I am already using this in a number of production sites ans it works
great, a huge improvement for rails apps running on nginx + mongrel.
Of course this is still alpha stuff so don't put it into mission
critical production setups yet> But in general and I very happy with
the stability and performance of this new module.


  This new module will actually work for any http backends and is not
directly tied to mongrel at all so this is probably good for other non
rails backends as well. Please test your setups with this module.

  Grezegorz has done an awesome job on this and should be commended.

Cheers-
- Ezra
- EngineYard.com

On 22.11.2007, at 22:38, Adam Zell wrote:

> in the wrong line for service problem described above."
>
> I am guessing that the strategy is at the network level, so it should
> work with non-mongrel back-ends.

good! that means we can get rid of our LVS. i can really agree that
least-open-connections load balancing adds a lot of speed.

jodok


>
>
> -- 
> Adam
> zellster@gmail.com
>

--
Lovely Systems, Partner

phone: +43 5572 908060, fax: +43 5572 908060-77
Schmelzhütterstraße 26a, 6850 Dornbirn, Austria

Hi,

2007/11/23, Alexander Staubo <alex@purefiction.net>:
> > One question, how is the busy state determined? In case
> > of zeo each backend client can take some defined number of requests
> > in parallel, how is such a case handled?

Should work out of the box, distributing the load equally. You may
wish to specify weight for each backend but if all are equal, this
should have no effect.

>
> I have not studied the sources, but I expect it will pick the upstream
> with the fewest number of current pending requests; among upstreams
> with the same number of concurrent requests, the one picked is
> probably arbitrary.

The scheduling logic looks like this:
 - The backends are selected _mostly_ round-robin (i.e. if you get 1
req/hour, they'll be serviced by successive backends)
 - Idle (no requests currently serviced) backends have absolute
priority (an idle backend will be always chosen if available)
 - Otherwise, the scheduler walks around the list of backends
(remembering where it finished last time) until the scheduler score
stops increasing. The highest scored backend is chosen (note: not all
backends are probed, or at least not always).
 - The scheduler score is calculated roughly as follows (yes, it could
be cleaned up a little bit):

score = (1 - nreq) * 1000 + last_active_delta;
if (score < 0) {
  score /= current_weight;
} else {
  score *= current_weight;
}

nreq is the number of currently processed requests
last_active_delta is time since last request start _or_ stop (serviced
by this backend), in milliseconds
current_weight is a counter decreasing from the backend's weight to 1
with every serviced request

It has a few properties which (I hope) make it good:
 - penalizing busy backends, with something like a pessimistic
estimate of request time
 - rewarding backends which have been servicing a request for a long
time (statistically they should finish earlier)
 - rewarding backends with higher weight more or less proportionally.

Please give the module a try and report any issues you might find.

Best regards,
 Grzegorz Nosek

Thanks for the explanation. I will give it a try shortly.

With regards,

__Janko

Am 23.11.2007 um 13:38 schrieb Grzegorz Nosek:

Hi. It has been a while since the introduction of fair proxy balancer.
How stable is it for production use. I was looking at potentially using
haproxy or lvm but hoping the new balancer is stable enough since I
don't want to unnecessarily complicate things with even more layers of
software in the stack. Anyone using it for production that can comment.

Regards,
David

On Jan 31, 2008 4:27 AM, David Pratt <fairwinds@eastlink.ca> wrote:
> Hi. It has been a while since the introduction of fair proxy balancer.
> How stable is it for production use. I was looking at potentially using
> haproxy or lvm but hoping the new balancer is stable enough since I
> don't want to unnecessarily complicate things with even more layers of
> software in the stack. Anyone using it for production that can comment.

We have been running the fair proxy balancer patch in production since
November. We had a problem with an earlier version of the patch in
combination with long-running requests (if you're interested, search
the list archives for my post), but the current one solves this issue.
Other than that it's been smooth sailing.

Alexander.

<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN">
<html>
<head>
  <meta content="text/html;charset=UTF-8" http-equiv="Content-Type">
  <title></title>
</head>
<body bgcolor="#ffffff" text="#000000">
Alex the patch for the bug you mention is it part of the latest 
nginx<br>
and how to use the fair proxy balancer<br>
thanks a lot<br>
<br>
Alexander Staubo wrote:
<blockquote
 cite="mid:88daf38c0801310251u60a4ee79n2707719f6fcf5daf@mail.gmail.com"
 type="cite">
  <pre wrap="">On Jan 31, 2008 4:27 AM, David Pratt <a 
class="moz-txt-link-rfc2396E" 
href="mailto:fairwinds@eastlink.ca">&lt;fairwinds@eastlink.ca&gt;</a> 
wrote:
  </pre>
  <blockquote type="cite">
    <pre wrap="">Hi. It has been a while since the introduction of fair 
proxy balancer.
How stable is it for production use. I was looking at potentially using
haproxy or lvm but hoping the new balancer is stable enough since I
don't want to unnecessarily complicate things with even more layers of
software in the stack. Anyone using it for production that can comment.
    </pre>
  </blockquote>
  <pre wrap=""><!---->
We have been running the fair proxy balancer patch in production since
November. We had a problem with an earlier version of the patch in
combination with long-running requests (if you're interested, search
the list archives for my post), but the current one solves this issue.
Other than that it's been smooth sailing.

Alexander.

  </pre>
</blockquote>
</body>
</html>

On Jan 31, 2008 12:07 PM, Mark <mobiledreamers@gmail.com> wrote:
>  Alex the patch for the bug you mention is it part of the latest nginx
>  and how to use the fair proxy balancer

The module is not part of Nginx. See
http://wiki.codemongers.com/NginxHttpUpstreamFairModule.

Alexander.

Hi Alexander. This is great. BTW, what would you recommend for
monitoring connections to see that it is fairly balancing without idle
connections or directing to a potentially blocked connection due to long
running requests. Many thanks.

Regards,
David

Hi Igor. Since the fair proxy balancer appears to be doing the right
thing, is there a stopper in bringing the necessary changes into the
trunk (without patching) so it would only require the module? Many 
thanks.

Regards,
David

On Jan 31, 2008 2:23 PM, David Pratt <fairwinds@eastlink.ca> wrote:
> Hi Alexander. This is great. BTW, what would you recommend for
> monitoring connections to see that it is fairly balancing without idle
> connections or directing to a potentially blocked connection due to long
> running requests. Many thanks.

The number of "writing" connections (seen according to the stub status
module) will in part reflect the number of connections blocking.

Other than that, we're using Mongrel with my process title
monkeypatcher (http://purefiction.net/mongrel_proctitle/), which
changes the Unix process title to reflect the number of queued
requests.

Alexander.

On Thu, Jan 31, 2008 at 09:31:29AM -0400, David Pratt wrote:
> Hi Igor. Since the fair proxy balancer appears to be doing the right 
> thing, is there a stopper in bringing the necessary changes into the 
> trunk (without patching) so it would only require the module? Many thanks.

Hi,

The fair proxy balancer doesn't really need any patches in nginx core. I
have published it as a complete source repository for my own convenience
but One Day(tm) I'll publish it as a standalone module.

The code is self-contained in src/http/modules/ngx_http_upstream_fair.c
so if you wire it into the nginx build process any other way, it should
work fine.

There are a few functions which are generic enough to warrant inclusion
in nginx core (and converting nginx code to use it) but by no means is
it neccessary for the module to function.

One known issue (again, waiting for the One Day) is that the round-robin
part doesn't work too well. E.g. if your load is very low, all requests
will go to the first backend. Anyway, I'll keep the current behaviour as
an option as it may be useful in dimensioning your backend cluster (i.e.
if the Nth backend has serviced no requests, N-1 should be enough).

Best regards,
 Grzegorz Nosek

Hi Grzegorz. I appreciate your explanation. It would be more convenient
to compile as an option since I am using an automated build process. If
it is self contained, can you forsee any problems building with most
current 0.5.x branch or is this strictly 0.6.x? Also, what is the
request threshold that triggers the issue with round robin issue that I
am aware. Many thanks.

Regards,
David

On Thu, Jan 31, 2008 at 01:29:32PM -0400, David Pratt wrote:
> Hi Grzegorz. I appreciate your explanation. It would be more convenient 
> to compile as an option since I am using an automated build process. If 
> it is self contained, can you forsee any problems building with most 
> current 0.5.x branch or is this strictly 0.6.x? Also, what is the 
> request threshold that triggers the issue with round robin issue that I 
> am aware. Many thanks.

The module works with 0.5.x as well as 0.6.x (if it doesn't work for
you, please mail me with a bug report).

There's no threshold per se, it's just that the original load balancer
directs requests strictly round robin, i.e. 0-1-2-3-0-1-2-3 etc. This
ensures that every backend gets the same number of requests.

upstream_fair always starts from backend 0 and works its way up until it
finds an idle peer (more or less). If your load effectively uses a
single backend at one time, it'll always be backend 0. If it uses the
power of two backends, they'll be 0 and 1 etc. Thus the first backend
will always have the most requests served, the second one will have more
than the third etc.

Best regards,
 Grzegorz Nosek

Hi Grzegorz. This gives me a much better idea of what to expect. Thank
you for this. I am curious whether you have you done anything in the way
of comparing the effectiveness of the fair proxy balancer to other
balancing schemes like haproxy or lvm. Speed is a big factor for
deployments so hoping speed will be good with the simplicity that this
option presents. Many thanks.

Regards,
David

yes it l b great if fair proxy balancer is added to nginx trunk

Hey, sorry for the late response here, but I thought I should mention, 
we're
using the fair proxy balancer on a rails site that averages over 100 
million
hits/month.  Been using it for about a month now, and love it.

Also, to Alexander, our programmer especially wanted me to thank you for
coming up with that mongrel process title patch, that thing is awesome! 
:)

-Rob.

Hey Grzegorz,

First, actually thank YOU for coming up with the balancer.  It's made my
life much easier.  And please, keep the round-robin behaviour as-is!  I
mean, it's a great way to tell if you're running too many mongrels 
and/or
too many nginx connections.

-Rob.

Hi Rob. This is encouraging news and I am working on a setup to
incorporate this into my process. I would really like to hear if there
has been any attempt to evaluate the fair proxy balancer in relation to
other balancing schemes. From the standpoint of server resources, it is
attractive and much simpler than the haproxy or lvm for setup. I realize
speed is subject to all sorts of additional parameters but a comparison
of the balancer with others would be quite interesting.

Rob, can you elaborate a bit more on your mongrels situation. I do not
use ruby but have a similar situation other types of backend servers. In
the current scenario, the last server will always get less hits. Are you
setting some sort of threshold to determine how many mongrels to run (or
  just starting up mongrels until the last is getting no hits). Many 
thanks.

Regards,
David

Hi,

On Wed, Feb 06, 2008 at 10:07:08AM -0400, David Pratt wrote:
> Hi Rob. This is encouraging news and I am working on a setup to 
> incorporate this into my process. I would really like to hear if there 
> has been any attempt to evaluate the fair proxy balancer in relation to 
> other balancing schemes. From the standpoint of server resources, it is 
> attractive and much simpler than the haproxy or lvm for setup. I realize 
> speed is subject to all sorts of additional parameters but a comparison 
> of the balancer with others would be quite interesting.

(disclaimer: I wrote upstream_fair, I'm biased).

No, I haven't compared haproxy or lvs (I assume that was what you
meant). However, haproxy is a TCP forwarder which makes it uncomfortable
at times. For example, even if your backends are down, connections to
haproxy will succeed and the only thing haproxy can do is to reset your
new connection (even though nginx has already happily sent the request).
This is a bit different than a failed backend, which returns a system
error (connection refused) or times out. Besides, AFAIK haproxy does not
offer least-connection balancing.

LVS, I cannot comment (haven't used it) but it has a wider choice of
balancing algorithms, including weighted least-connection. If you have
the resources to set it up (looks a bit hairy to me), it should perform
very well.

> Rob, can you elaborate a bit more on your mongrels situation. I do not 
> use ruby but have a similar situation other types of backend servers. In 
> the current scenario, the last server will always get less hits. Are you 
> setting some sort of threshold to determine how many mongrels to run (or 
>  just starting up mongrels until the last is getting no hits). Many thanks.
> 

Hmm, let me use your message to reply to Rob too :)

> >First, actually thank YOU for coming up with the balancer.  It's made my
> >life much easier.  And please, keep the round-robin behaviour as-is!  I
> >mean, it's a great way to tell if you're running too many mongrels and/or
> >too many nginx connections.

Unfortunately, pure WLC behaviour causes problems for mongrel as it
apparently doesn't like to be slammed too hard (looks like it leaks
memory but that's just a guess).

In the newest snapshot I added (or rather fixed) the round-robin part.
I'll make it configurable, but the default will probably be round-robin
from now on. But yes, it is handy.

Best regards,
 Grzegorz Nosek

Hi. Both haproxy and lvs have setups that are more involved for sure.
haproxy 1.3 has more balancing algorithms than 1.2. I have seen patches
that provide least connection balancing for 1.2 also. lvs is what I
believe to be 'the' mainstream balancer but needs to be compiled into
the linux kernel - it as not as portable and simple as incorporating the
fair proxy balancer as a result. Interested in Rob's experience to
determine no of servers. Many thanks Grzegorz.

Regards,
David

On Feb 6, 2008, at 10:44 AM, David Pratt wrote:

> Hi. Both haproxy and lvs have setups that are more involved for  
> sure. haproxy 1.3 has more balancing algorithms than 1.2. I have  
> seen patches that provide least connection balancing for 1.2 also.  
> lvs is what I believe to be 'the' mainstream balancer but needs to  
> be compiled into the linux kernel - it as not as portable and simple  
> as incorporating the fair proxy balancer as a result. Interested in  
> Rob's experience to determine no of servers. Many thanks Grzegorz.
>
> Regards,
> David



Hey David-

  We're running the fair balancer on about 100 servers with good
success. We had some issues with the fair balancer in lower load
situations only sending requests to the first backend instead of doing
a round robin when under lower load, this was causing the single
backend to become overloaded. The latest version Grzegorz has just
pushed to his git repo works much better in all the situations we have
put it under.

  We run LVS at the edge of our clusters and have LVS balance to nginx
on each VM with nginx doing fair balancing directly to the mongrels
and it is working great. Much fewer moving parts then throwing haproxy
in the mix. In my benchmarks having haproxy between nginx and the
mongrels was s lower since there was one more level of indirection. So
having nginx serving static content and fair balancing to the backends
is ideal for us.


Cheers-
- Ezra Zygmuntowicz
-- Founder & Software Architect
-- ezra@engineyard.com
-- EngineYard.com

Hi,

I've run into a behavioral difference between different versions of
nginx, but I'm not sure which behavior is expected...

I need to proxy a request for a dynamically-assembled javascript file
to a set of app servers, but want to serve static js files directly
from nginx.  I want to compress both proxied and unproxied js files.

The config below works, but doesn't return a compressed file for the
proxied request in all environments.  Unproxied javascript requests
are compressed in all cases.

nginx-0.5.20 on FreeBSD-6.2:
  returns compressed files for all /js/ paths, including proxied
nginx-0.5.30 on Solaris 10:
  returns compressed files for local /js/ paths, but
  returns UNcompressed files for all upstream proxied /js/ paths
nginx-0.5.35 on Solaris 10:
  (same as nginx-0.5.30 on Solaris 10)

I couldn't find anything in the 0.5.20 - 0.5.30 release notes that
looked like a relevant bugfix or intentional behavior change.

I realize the environment diffs are substantial, but I'm also not
sure from the docs whether this _should_ work.  :)  gzip_proxied
seems to control a conditional on the headers of the UA request,
and not be relevant to the gzip-or-not decision for the results
of upstream requests before returning to the UA.  If it hadn't
worked in 0.5.20 either, I'd assume that was the whole story.

The best answer might be "Hmm, weird, but you should just cache
the first request to /js/all.js and let nginx treat it as local
thereafter anyway".  Fair enough, but it adds some complexity
elsewhere that I'd like to avoid for now.

Here are the relevant bits of the config file:

## handle static files directly
location ~ ^/(js|css|themes)/
  {
  root /path/to/docroot

  gzip_types text/javascript text/css text/plain text/js 
application/x-javascript application/javascript;
  gzip_proxied any;

  if ( $uri ~ ^/(js|css)/ ) { gzip on; }

  ## force proxy for all.js, as it is ephemeral
  if ( $uri = "/js/all.js" )
    { proxy_pass http://appservers; break; }
  }


Thank you for any insights!
Andrew

Hi Ezra. Cool. The setup I am looking at is quite similar so great to
hear it is doing the job well. Many thanks for sharing your experience.

Regards
David

On Feb 6, 2008 4:41 PM, Grzegorz Nosek <grzegorz.nosek@gmail.com> wrote:
> No, I haven't compared haproxy or lvs (I assume that was what you
> meant). However, haproxy is a TCP forwarder which makes it uncomfortable
> at times. For example, even if your backends are down, connections to
> haproxy will succeed and the only thing haproxy can do is to reset your
> new connection (even though nginx has already happily sent the request).

Could you explain what you mean by "if your backends are down,
connections to haproxy will succeed"? By backend, do you mean Nginx
or, say, a Mongrel server?

Alexander.

On Fri, Feb 08, 2008 at 04:29:27PM +0100, Alexander Staubo wrote:
> On Feb 6, 2008 4:41 PM, Grzegorz Nosek <grzegorz.nosek@gmail.com> wrote:
> > No, I haven't compared haproxy or lvs (I assume that was what you
> > meant). However, haproxy is a TCP forwarder which makes it uncomfortable
> > at times. For example, even if your backends are down, connections to
> > haproxy will succeed and the only thing haproxy can do is to reset your
> > new connection (even though nginx has already happily sent the request).
> 
> Could you explain what you mean by "if your backends are down,
> connections to haproxy will succeed"? By backend, do you mean Nginx
> or, say, a Mongrel server?

Say you have:

nginx --> haproxy --> (a bunch of backends)

When nginx connects to haproxy, the connection will succeed (as haproxy
is still alive) but when haproxy tries to connect to any backend (let's
say they're all dead or a switch failed etc.), the connection
(haproxy->backend) will eventually time out. However, from nginx's point
of view, the connection succeeded (haproxy replied), so it sends the
request which then times out or dies with a connection reset.

If you had several haproxy instances, each fronting its own set of
mongrels, you'd have just lost a request unneccessarily [sp?].

Note, I don't know whether haproxy is smart enough to shut down its
listening socket when all backends fail (it might be). I rather meant to
say that a TCP forwarder must explicitely support such situations to
handle them.

Best regards,
 Grzegorz Nosek

On Feb 8, 2008 4:47 PM, Grzegorz Nosek <grzegorz.nosek@gmail.com> wrote:
> When nginx connects to haproxy, the connection will succeed (as haproxy
> is still alive) but when haproxy tries to connect to any backend (let's
> say they're all dead or a switch failed etc.), the connection
> (haproxy->backend) will eventually time out. However, from nginx's point
> of view, the connection succeeded (haproxy replied), so it sends the
> request which then times out or dies with a connection reset.

I see. That's a valid point.

Actually, I never considered that one might use HAProxy behind Nginx
in the first place. Rather, I assumed HAProxy would be a more
naturally placed *before* Nginx, if one wanted, say, a more strictly
layered setup where a dedicated proxy did the proxying and a web
server such as Nginx did the static file serving.

Alexander.

On Sat, Feb 09, 2008 at 01:33:56AM +0100, Alexander Staubo wrote:
> Actually, I never considered that one might use HAProxy behind Nginx
> in the first place. Rather, I assumed HAProxy would be a more
> naturally placed *before* Nginx, if one wanted, say, a more strictly
> layered setup where a dedicated proxy did the proxying and a web
> server such as Nginx did the static file serving.

Yes, that would be more natural for me too. However, the question was to
compare upstream_fair and haproxy. In order to compare them in any way,
you'd have to put haproxy _behind_ nginx or set up haproxy in front of a
cluster of nginx instances, each fronting one backend.

In both of these setups, the nginx load balancer will be replaced by
the one from haproxy (the difference is the number of nginx instances),
so it would be possible to compare the two load balancers somehow.

Best regards,
 Grzegorz Nosek

Grzegorz Nosek wrote:
>
> Please give the module a try and report any issues you might find.

This module works great, thanks for releasing it.  While validating its 
behavior, I noticed a few gotchas that I wasn't expecting, but I'm not 
sure if there is anything upstream_fair can do about it:

1) If a client performs a request which is long running, and then gets 
fed up with waiting and closes the connection, upstream_fair marks the 
backend process as idle even though it is still servicing the request - 
I guess it is monitoring the front side of the connection rather than 
the back side...?

2) If I have a long running request active on a mongrel, and nginx times 
it out (proxy_read_timeout defaults to 60s), the request is still 
running on the mongrel, but upstream_fair marks it as idle.

These gotchas can both cause upstream_fair to schedule requests on 
mongrels that are not actually idle when there are other idle ones 
waiting

Matt

On Mon, May 05, 2008 at 06:21:49PM +0200, Matt Conway wrote:
> Grzegorz Nosek wrote:
> >
> > Please give the module a try and report any issues you might find.
> 
> This module works great, thanks for releasing it.  While validating its 
> behavior, I noticed a few gotchas that I wasn't expecting, but I'm not 
> sure if there is anything upstream_fair can do about it:

I'm glad you like it :)

> 1) If a client performs a request which is long running, and then gets 
> fed up with waiting and closes the connection, upstream_fair marks the 
> backend process as idle even though it is still servicing the request - 
> I guess it is monitoring the front side of the connection rather than 
> the back side...?
> 
> 2) If I have a long running request active on a mongrel, and nginx times 
> it out (proxy_read_timeout defaults to 60s), the request is still 
> running on the mongrel, but upstream_fair marks it as idle.

upstream_fair doesn't monitor anything by itself, it's told by the nginx
core when a request is started and when it ends. So unfortunately I 
cannot
do much about the issues you mention.

Igor, off the top of your head, do you know what status is passed to the
->free callback when a request times out? I could use it to penalise 
that
backend somehow.

Anyway, when nginx closes the backend connection, there's no way to know
what the backend is doing so it would be guessing at best (the backend
may finish half a second later or stay busy forever, we'll never know).

> 
> These gotchas can both cause upstream_fair to schedule requests on 
> mongrels that are not actually idle when there are other idle ones 
> waiting

Since commit 4329e708 (Feb 2008) the round-robin mechanism should work
so this effect should be reduced to perform at least no worse than the
default round-robin balancer.

Best regards,
 Grzegorz Nosek