Maxconn 1 in nginx

Server NGINX
1

Hey,

I’m creating a pretty big HTTP RESTful API that is organized in the
following way: There are many different functionalities in the API.
Different functionalities are served by different backend machines and
each functionality is another smaller RESTful API. In front of all
that mess I’d like to put Nginx. The idea is that Nginx would check
the requested path and dispatch the request to proper backend, like
this:

So far everything can be done in Nginx easily. But: backends are
written in Ruby on Rails and we all know how good HAProxy’s “maxconn
1” is for Rails. How can I achieve the same in Nginx?

Regards,

Mike

2

On Thu, Oct 09, 2008 at 11:28:54AM +0200, Michał Jaszczyk wrote:

backend-1.mydomain.com,

So far everything can be done in Nginx easily. But: backends are
written in Ruby on Rails and we all know how good HAProxy’s “maxconn
1” is for Rails. How can I achieve the same in Nginx?

If I’m right about what maxconn=1 does, grab a fresh copy of
upstream_fair
and set it up like this:

upstream foo {
server 10.0.0.1:80 weight=1;
server 10.0.0.2:80 weight=1;

etc.

fair weight-mode=peak;
}

Weight=1 is the default but it’s nice to make it explicit in this case.

…but give it a try with simple ‘fair’ (without parameters) too.

Best regards,
Grzegorz N.

3

It’s definitly not the same thing. Nginx hasn’t got yet the maxconn
parameter.

From haproxy documentation :
“It’s a per-server ‘maxconn’, associated with a per-server and a
per-proxy
queue. This transforms haproxy into a request buffer between the
thousands
of clients and the few servers. On many circumstances, lowering the
maxconn
value will increase the server’s performance and decrease the overall
response times because the servers will be less congested.”

4

On Thu, Oct 09, 2008 at 02:54:09PM +0200, Benjamin wrote:

It’s definitly not the same thing. Nginx hasn’t got yet the maxconn
parameter.

From haproxy documentation :
“It’s a per-server ‘maxconn’, associated with a per-server and a per-proxy
queue. This transforms haproxy into a request buffer between the thousands
of clients and the few servers. On many circumstances, lowering the maxconn
value will increase the server’s performance and decrease the overall
response times because the servers will be less congested.”

Thanks for the info. So, to make sure I understand: haproxy will keep
only ‘maxconn’ requests in flight to a server and sit on others until
the backends catch up, right?

upstream_fair (with weight-mode=peak) will only limit the number of
requests, causing Nginx to return 502 errors. It could be hacked to
never return a hard error (I think), so if that’s the only difference
and the OP is willing to do some testing, I can implement that.

Best regards,
Grzegorz N.

5

On Thu, Oct 09, 2008 at 04:28:37PM +0200, Benjamin wrote:

upstream_fair (with weight-mode=peak) will only limit the number of

requests, causing Nginx to return 502 errors. It could be hacked to
never return a hard error (I think), so if that’s the only difference
and the OP is willing to do some testing, I can implement that.

I do not know the weight-mode option. But it’s the same principle. Instead
of returning an error, haproxy created a queue of requests.

Thanks.

@Igor: Will keeping pc->tries above zero at all times achieve this
effect without eating lots of CPU? (pc is the ngx_peer_connection_t*
passed to ->peer.get())

@Michał: If Igor confirms, would you have a way to test the patched
upstream_fair?

Best regards,
Grzegorz N.

6

Thanks for the info. So, to make sure I understand: haproxy will keep
only ‘maxconn’ requests in flight to a server and sit on others until
the backends catch up, right?

True. It’s really useful with ruby on rails, because mongrel can just
treat
one request at time.

upstream_fair (with weight-mode=peak) will only limit the number of

requests, causing Nginx to return 502 errors. It could be hacked to
never return a hard error (I think), so if that’s the only difference
and the OP is willing to do some testing, I can implement that.

I do not know the weight-mode option. But it’s the same principle.
Instead
of returning an error, haproxy created a queue of requests.

7

On Oct 9, 2008, at 8:48 AM, Grzegorz N. wrote:

Grzegorz N.

Grzegorz-

Yeah the haproxy behavior is what we really want for proxying to
rails apps. I’d like nginx to only feed a request to mongrels that are
not currently processing a request and any other requests will queue
in nginx in the efficient event loop and only dole out the requests
once a backend comes available or a timeout is hit.

This is the ideal load balancer situation for rails apps. I;ve been
putting haproxy in between nginx and mongrels to do this but I would
much rather see the fair balancer support this use case so I have less
items in the chain .

Cheers-
-Ezra

8

On Thu, Oct 09, 2008 at 09:23:08AM -0700, Ezra Z. wrote:

Yeah the haproxy behavior is what we really want for proxying to
rails apps. I’d like nginx to only feed a request to mongrels that are
not currently processing a request and any other requests will queue
in nginx in the efficient event loop and only dole out the requests
once a backend comes available or a timeout is hit.

Thinking about it, it’s not a trivial problem actually. The load
balancer must answer now with either a backend address or an error.
AFAIK there’s no mechanism to defer the decision (and thus dispatching
the request).

Even if there was a way to do that (e.g. the balancer returns
NGX_AGAIN), it wouldn’t still solve the problem because Nginx core
wouldn’t know when to retry, so would probably have to busy loop. The
API might include passing back an interval but it would always be a
rough estimate. Tying the retry with more load balancer interaction
(e.g. when a request ends with a call to ->peer.free, retry all pending
requests) would mostly work (for this case) but wouldn’t be 100%
bulletproof, as the multiple workers don’t communicate (in the usual
case; upstream_fair only uses shared memory). Attaching retries to FD
actions presents a similar problem and changes the behaviour slightly
(more often polling, more accurate timing, more CPU usage).

Regarding multiple workers, I can’t see a clean solution currently,
but restricting ourselves to the one-worker case, the upstream core
could keep a queue (or rbtree or…) of pending requests per-upstream
and export a function like

ngx_http_upstream_retry(ngx_http_upstream_srv_conf_t *us);

Which would walk the queue calling ->peer.get again. The balancer
could then respond “this guy isn’t ready yet, try another one”
(NGX_EAGAIN) separately from “everything is still busy, no point in
asking again” (NGX_DONE?). A nice touch would be a timer interface to
set up a hard timeout (I’m sure it can be done now but don’t know the
whole API yet :wink:

The queue could be also maintained in the load balancer but still Nginx
needs to be modified to accept NGX_AGAIN as a balancer verdict.

Details would need some ironing out but I think the overall idea is
workable.

@Igor, do you think it could work?

This is the ideal load balancer situation for rails apps. I;ve been
putting haproxy in between nginx and mongrels to do this but I would
much rather see the fair balancer support this use case so I have less
items in the chain .

BTW, that’s just insane (IMVHO). If a Rails app can process requests
sequentially faster than concurrently, then why TF does it accept more
requests than it can handle? If the app had a short listen queue and ran
as a simple iterative server, overlimit requests would simply error out
with ECONNREFUSED, which can be rather gracefully handled inside Nginx.

Best regards,
Grzegorz N.

9

Ezra Z. wrote:

Grzegorz-

Yeah the haproxy behavior is what we really want for proxying to
rails apps. I’d like nginx to only feed a request to mongrels that are
not currently processing a request and any other requests will queue
in nginx in the efficient event loop and only dole out the requests
once a backend comes available or a timeout is hit.

This is the ideal load balancer situation for rails apps. I;ve been
putting haproxy in between nginx and mongrels to do this but I would
much rather see the fair balancer support this use case so I have less
items in the chain .

This depends on whether the backend is actually hitting Rails or not.
If you have a backend Mongrel cluster that doles out page-cached pages,
for example, then maxconn can (and should) be set much higher.

Now if you hit Rails…we have done measurements that show noticeable
increase in throughput with maxconn set to 2 instead. But you only want
to do this if the behavior of your controller(s) is very predictable
(and they’re pretty fast).

-Dave Pascoe
Ourstage.com
http://www.ourstage.com

10

On Thu, 2008-10-09 at 09:23 -0700, Ezra Z. wrote:

Yeah the haproxy behavior is what we really want for proxying to
rails apps. I’d like nginx to only feed a request to mongrels that are
not currently processing a request and any other requests will queue
in nginx in the efficient event loop and only dole out the requests
once a backend comes available or a timeout is hit.

This is the ideal load balancer situation for rails apps. I;ve been
putting haproxy in between nginx and mongrels to do this but I would
much rather see the fair balancer support this use case so I have less
items in the chain .

This isn’t really the right forum for this, but this request comes up so
often I may as well ask. I can’t help but think this is something that
should be fixed in Rails. If people want to throw money at this
problem, wouldn’t the Rails devs (or some smart Ruby guy) be the most
productive place to throw it?

I’m not a Rails user (it’s lack of support for concurrency being one of
the primary reasons I’ve never seriously considered it), but it seems
that fixing a Ruby framework (Rails) should be easier than adding
features to a C framework (Nginx). Is this not the case?

Regards,
Cliff