Possible limitation of ngx_http_limit_req_module

Server NGINX
1

Hi,

I’m observing an inconsistent behavior of ngx_http_limit_req_module in
nginx
1.7.12. The relevant excerpts from my config:

http {
…
# A fixed string used as a key, to make all requests fall into the
same
zone
limit_req_zone test_zone zone=test_zone:1m rate=5r/s;
…
server {
…
location /limit {
root /test
limit_req zone=test_zone nodelay;
}
…
}
}

I use wrk to hammer the server for 5 secs:

$ ./wrk -t 100 -c 100 -d 5 http://127.0.0.1/limit/test
Running 5s test @ http://127.0.0.1/limit/test
100 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 2.82ms 2.96ms 15.12ms 88.92%
Req/Sec 469.03 190.97 0.89k 62.05%
221531 requests in 5.00s, 81.96MB read
Non-2xx or 3xx responses: 221506
Requests/sec: 44344.69
Transfer/sec: 16.41MB

So, out of 221531 sent requests, 221506 came back with error. This gives
(221531 - 221506) = 25 successful requests in 5 secs, so 5r/s, just as
expected. So far so good.

Now, what happens if I set rate=5000r/s:

$ ./wrk -t 100 -c 100 -d 5 http://127.0.0.1/limit/test
Running 5s test @ http://127.0.0.1/limit/test
100 threads and 100 connections
Thread Stats Avg Stdev Max +/- Stdev
Latency 3.64ms 5.70ms 36.58ms 87.43%
Req/Sec 443.50 191.55 0.89k 65.04%
210117 requests in 4.99s, 77.38MB read
Non-2xx or 3xx responses: 207671
Requests/sec: 42070.61
Transfer/sec: 15.49MB

This time it is (210117 - 207671) = 2446 successful requests in 5 secs,
which means 490r/s. Ten times lower then expected.

I gathered some more figures, showing the number of 200 responses for
growing value of “rate” parameter.

rate=***r/s in zone cfg – number of 200 responses
100 – 87
200 – 149
500 – 344
1000 – 452
10000 – 468
100000 – 466

As you can see, the server keeps returning pretty much constant number
of
200 responses once the “rate” parameter has surpassed 1000.

I had a glimpse into the module’s code, and this part caught my eye:
https://github.com/nginx/nginx/blob/nginx-1.7/src/http/modules/ngx_http_limit_req_module.c#L402-L414.
Basically, if consecutive requests hit the server in the same
millisecond,
the “ms = (ngx_msec_int_t) (now - lr->last)” part evaluates to 0, which
sets
“excess” to 1000, which is very likely to be greater than a “burst”
value,
which results in rejecting the request. This could also mean, that only
the
very first request hitting the server in given millisecond would be
handled,
which seems to be in line with the wrk test results, I’ve presented
above.

Please let me know if this makes sense to you!

Best regards,
Jakub Wroblewski

Posted at Nginx Forum:

2

On Monday 11 May 2015 10:59:46 jwroblewski wrote:

…
I use wrk to hammer the server for 5 secs:
Transfer/sec: 16.41MB
Thread Stats Avg Stdev Max +/- Stdev
I gathered some more figures, showing the number of 200 responses for
As you can see, the server keeps returning pretty much constant number of

Please let me know if this makes sense to you!

[…]

Yes, the module is limited by millisecond timer resolution, but using so
high
rate values without the burst parameter is meaningless.

So, just don’t do that.

wbr, Valentin V. Bartenev

3

My use case is that upstreams are supposed to return within ~100ms,
therefore using burst is not an option. I wanted to use limit_req to
filter
out traffic which is exceeds my backend’s processing capacity, but
apparently it is not the right tool to use, if it only operates with
millisecond-precision… Could you please document this limitation?

Posted at Nginx Forum:

4

On Tuesday 12 May 2015 09:25:05 jwroblewski wrote:

My use case is that upstreams are supposed to return within ~100ms,
therefore using burst is not an option. I wanted to use limit_req to filter
out traffic which is exceeds my backend’s processing capacity, but
apparently it is not the right tool to use, if it only operates with
millisecond-precision…

What’s problem with using burst? Could you explain why it’s not an
option
for your case?

Could you please document this limitation?

Patches are welcome.

But you’re the only person I can remember who cares about it. For most
of
the users it will be just superfluous details.

wbr, Valentin V. Bartenev

5

I do not necessarily have a say on what is discussed here, but:

  1. I believe putting known limitations in the docs makes sense. Who
    defined the docs as sticking to the most common use cases? Technical
    docs
    are technical docs.
  2. Using burst answers a specific need which has not been expressed
    here
    (on the contrary I think it is mandatory such a behavior does not
    happen in
    this use case).
  3. Awaiting nginx to serve more than 1000 r/s is reasonable for a Web
    server claiming to scale and having been designed for HA. Silently
    ignoring
    extra requests per millisecond is awful, especially if no-one knows
    about
    it.
    The users are not supposed to guess by themselves that using requests
    limiting will make their availability silently drop…
  4. This user is maybe the first, but it is over-optimistic to
    consider
    he might be alone.

It could be interesting to know if you discarded documenting some other
known limitations in your product(s), deciding on the behalf of the
users
what they should/need to know or not.

B. R.

On Tue, May 12, 2015 at 3:43 PM, Valentin V. Bartenev [email protected]

6

On Tuesday 12 May 2015 16:46:29 B.R. wrote:

I do not necessarily have a say on what is discussed here, but:

  1. I believe putting known limitations in the docs makes sense. Who
    defined the docs as sticking to the most common use cases? Technical docs
    are technical docs.

I’m agree with you. But there are thousands “limitations” that imposed
by
the algorithm used, or by the nginx internal optimizations, or by the
how
hardware works.

If we will try to document every detail, then the documentation will be
over-bloated and not human readable at all.

The most detailed technical documentation is the source code itself. So
we
are trying to balance and don’t turn our documentation into source code.

  1. Using burst answers a specific need which has not been expressed here
    (on the contrary I think it is mandatory such a behavior does not happen in
    this use case).

Using the limit req module based on the “leaky bucket” algorithm with
the bucket size actually set to zero (i.e. burst isn’t set) in most
cases is just a misconfiguration.

I’m sure in the current case it’s a misconfiguration too.

  1. Awaiting nginx to serve more than 1000 r/s is reasonable for a Web
    server claiming to scale and having been designed for HA. Silently ignoring
    extra requests per millisecond is awful, especially if no-one knows about
    it.
    The users are not supposed to guess by themselves that using requests
    limiting will make their availability silently drop…

Nobody said that it cannot serve more then 1000 r/s, but you must
configure
burst in this case (and actually you should in most cases).

Missing burst option usually is just a result of misunderstanding how
leaky
bucket works.

  1. This user is maybe the first, but it is over-optimistic to consider
    he might be alone.

Probably it’s worth to be documented, but first of all let’s find out if
it’s
really a limitation, or just misconfiguration.

It could be interesting to know if you discarded documenting some other
known limitations in your product(s), deciding on the behalf of the users
what they should/need to know or not.

nginx-devel@ archive is here:
http://mailman.nginx.org/pipermail/nginx-devel/

Could you find a patch for documentation that has been discarded?

wbr, Valentin V. Bartenev

7

On Tuesday 12 May 2015 12:33:11 jwroblewski wrote:

What’s problem with using burst? Could you explain why it’s not an
from Y+1, are to be immediately 503’ed.
The problem that there’s no given second in the leaky bucket algorithm
and
the immediately doesn’t a technical term, since there’s nothing
happens
immediately. The “immediately” has some measurable time.

For example setting rate=10r/s literally means every request that comes
earlier
than 100ms after the previous will be discarded. Even if a client have
made
only 100 requests but within 100 ms interval then 99 will be discarded.

To allow bursts in request rate (bursts are natural and they always
exist
in normal http traffic) the burst parameter must be configured.

The reason why I can not use burst, it that burst introduces queuing, which
means by the time the request leaves nginx, it is already late by some
milliseconds, while I want the whole solution to be as real time as
possible.

No, it doesn’t introduce any delays or queuing if you have the “nodelay”
option
set (btw, “nodelay” actually does nothing without the burst set).

You should try something like:

limit_req zone=test_zone burst=100 nodelay;

Even burst=5 will let limit req with rate=5000r/s to behave in your
tests as you
expected.

wbr, Valentin V. Bartenev

8

Valentin V. Bartenev Wrote:

for your case?
My nginx receives X r/s (lets assume X equals ~50000), and is supposed
to
respond within 100ms to every single of them.
Requests are dispatched to upstreams, which can only handle a total of Y
r/s
in a timely manner (Y being less than X, say 20000).
Knowing the capacity of my upstreams, I want nginx to immediately drop
all
excessive requests. This means, only first Y requests which came in
during
given second are to be pushed to upstreams, the remaining ones, starting
from Y+1, are to be immediately 503’ed.

The reason why I can not use burst, it that burst introduces queuing,
which
means by the time the request leaves nginx, it is already late by some
milliseconds, while I want the whole solution to be as real time as
possible.

Having read the docs, I got the impressions that with “burst=0 nodelay”
will
let me achieve the goal outlined above. Burst enables “recovery” of
excessive requests, while I want these dropped. Still, I might have
gotten
the docs wrong…

Could you please document this limitation?

Patches are welcome.

But you’re the only person I can remember who cares about it. For
most of
the users it will be just superfluous details.

I will be happy to submit a doc patch. It is possible that some users
are
simply not aware of this limitation, and loose requests because of it.

wbr, Valentin V. Bartenev

nginx mailing list
[email protected]
nginx Info Page

Posted at Nginx Forum:

9

Hello!

On Tue, May 12, 2015 at 12:33:11PM -0400, jwroblewski wrote:

What’s problem with using burst? Could you explain why it’s not an
from Y+1, are to be immediately 503’ed.

The reason why I can not use burst, it that burst introduces queuing, which
means by the time the request leaves nginx, it is already late by some
milliseconds, while I want the whole solution to be as real time as
possible.

Having read the docs, I got the impressions that with “burst=0 nodelay” will
let me achieve the goal outlined above. Burst enables “recovery” of
excessive requests, while I want these dropped. Still, I might have gotten
the docs wrong…

The “nodelay” alone will let you achieve the goal.

The “burst” should be set to a non-zero value to allow the
algorithm to tolerate peaks - that is, to tolerate cases when
several requests are processed at once.

As timekeeping in nginx uses millisecond resolution, it certainly
doesn’t make sense to use burst less than expected traffic in 1ms,
20 requests in your case.

In practice, 1ms is rather optimistic - e.g., a disk seek can
easily take 10ms alone, and you’ll see a 10ms burst if a request
will trigger a disk seek. As long as you want to respond within
100ms, you’ll probably should tolerate bursts equal to at least
about 10ms of traffic, that is, up to 200 requests.

Note well that all the words you write about “first Y requests
during given second” imply counting requests over a second.
That is, it’s about burst equal to expected traffic in 1 second,
20000 requests in your case.

–
Maxim D.
http://nginx.org/