Any way to deal with clients breaking the limit zone

Server NGINX
1

Just in case it is relevant I’ll explain the setup. It is vbulletin site
with apache serving php and nginx static content.
I have this in nginx.conf
limit_zone one $binary_remote_addr 10m;
and this in vhost config of the site in question:
limit_conn one 35;

I hope that is reasonable setting.
Haven’t noticed any issues with regular traffic. But occasionally we get
some ips breaking the zone limit, I am pretty sure it is not a regular
visitor. And it was generating hundreds of lines in vhost-error_log:
limiting connections by zone “one”, client: xx.xx…

So my question is if there is perhaps a way to stop that client after he
repeats the limit breaking constantly for like 5 minutes, or X times or
something like that.
Or at least to stop it generating the logs.
I ended up blocking that ip with firewall today, but I won’t be there
all the time to see it and it is gonna create a lot of junk lines in the
log. It is not that big of an issue but I just thought it is worth a
shot to see if there is a way around it.

Posted at Nginx Forum:

2

Hello!

On Wed, Jun 01, 2011 at 10:52:03AM -0400, karabaja wrote:

visitor. And it was generating hundreds of lines in vhost-error_log:
limiting connections by zone “one”, client: xx.xx…

So my question is if there is perhaps a way to stop that client after he
repeats the limit breaking constantly for like 5 minutes, or X times or
something like that.
Or at least to stop it generating the logs.
I ended up blocking that ip with firewall today, but I won’t be there
all the time to see it and it is gonna create a lot of junk lines in the
log. It is not that big of an issue but I just thought it is worth a
shot to see if there is a way around it.

Usual aproach is to read logs with some script (fail2ban or
something) and block offenders on firewall by the script.

Maxim D.

3

Hello Maxim,

Sorry about highjacking this thread, but I think my question is
relevant. In the nginx configuration I have:

    # limit simultaneous connections
    limit_conn_zone $binary_remote_addr zone=addr:1m;
    limit_conn addr 16;

But the error_log file says:

2012/01/11 21:41:39 [warn] 10140#0: *2959 limiting connections by zone
“addr”, client: 127.0.0.1, server: , request: “GET / HTTP/1.0”, host:
“mysite.com”, referrer: “GitHub - shekyan/slowhttptest: Application Layer DoS attack simulator”
2012/01/11 21:41:40 [warn] 10140#0: *2962 limiting connections by zone
“addr”, client: 127.0.0.1, server: , request: “GET / HTTP/1.0”, host:
“mysite.com”, referrer: “GitHub - shekyan/slowhttptest: Application Layer DoS attack simulator”

Question is, why 127.0.0.1 for the remote client address? It should
have nothing to do with loopback host address. It must be external
IP.

Any workaround? Also, I noticed that “server:” is empty. Any other
reason for that? I have a dozen of virtual hosts configured with
fastcgi_pass to php-fpm.

Many thanks for your assistance.

nginx/1.1.12

Andrejs

Posted at Nginx Forum:

4

Hello!

On Wed, Jan 11, 2012 at 04:31:45PM -0500, locojohn wrote:

2012/01/11 21:41:39 [warn] 10140#0: *2959 limiting connections by zone
“addr”, client: 127.0.0.1, server: , request: “GET / HTTP/1.0”, host:
“mysite.com”, referrer: “GitHub - shekyan/slowhttptest: Application Layer DoS attack simulator”
2012/01/11 21:41:40 [warn] 10140#0: *2962 limiting connections by zone
“addr”, client: 127.0.0.1, server: , request: “GET / HTTP/1.0”, host:
“mysite.com”, referrer: “GitHub - shekyan/slowhttptest: Application Layer DoS attack simulator”

Question is, why 127.0.0.1 for the remote client address? It should
have nothing to do with loopback host address. It must be external
IP.

Obvious reason is: because connection was got from 127.0.0.1.

Any workaround? Also, I noticed that “server:” is empty. Any other
reason for that? I have a dozen of virtual hosts configured with
fastcgi_pass to php-fpm.

Obvious reason is: the request was matched by a server{} block
without server_name set. See here:

http://nginx.org/en/docs/http/request_processing.html

for more details on how nginx matches requests to servers.

Maxim D.

5

Note: Unfortunately, I cannot edit my messages here. In both error log
entries, Host: is the same: “testsite”.

Andrejs

Posted at Nginx Forum:

6

Hello!

On Fri, Jan 13, 2012 at 01:55:38AM -0500, locojohn wrote:

    server {

the slowhttptest program is clearly “testsite” with its own error_log,
HTTP/1.1", host: “devel.ahlerstoday.com”, referrer:
“GitHub - shekyan/slowhttptest: Application Layer DoS attack simulator”
2012/01/12 18:10:35 [error] 10508#0: *15989 limiting connections by zone
“addr”, client: 217.24.78.177, server: testsite, request: “GET /
HTTP/1.1”, host: “devel.ahlerstoday.com”, referrer:
“GitHub - shekyan/slowhttptest: Application Layer DoS attack simulator”

Note HTTP/1.1. It’s likely to be original request.

/var/log/nginx/error_log:

2012/01/12 18:10:34 [error] 10509#0: *12134 limiting connections by zone
“addr”, client: 127.0.0.1, server: , request: “GET / HTTP/1.0”, host:
“testsite”, referrer: “GitHub - shekyan/slowhttptest: Application Layer DoS attack simulator”
2012/01/12 18:10:35 [error] 10509#0: *12136 limiting connections by zone
“addr”, client: 127.0.0.1, server: , request: “GET / HTTP/1.0”, host:
“testsite”, referrer: “GitHub - shekyan/slowhttptest: Application Layer DoS attack simulator”

Note HTTP/1.0 here. It looks like requests here appear after
proxy_pass to 127.0.0.1 in your original “testsite” server.

The fact the request isn’t matched to the same testsite server
suggests that things happen on different socket, i.e. you use
listen directive in the server block in question (the one without
server_name) either on different port or explicitly on 127.0.0.1.

Could it have anything to do with the fact that limit_zone and
limit_conn are defined on http level? Yet, I think the behaviour is
not fully correct.

Behaviour seems correct for me.

Maxim D.

7

Hello Maxim,

I am surely aware of how server requests are matched in nginx. Thanks
for the info though.

The thing is, I have multiple virtual hosts configured all with
server_name defined, of course. Then there’s default fallback server
configured as this:

default fallback server

    server {
            listen 80 default_server;
            server_name  _;
            return       444;
    }

After reading your reply I went to check my error logs and the strange
thing is that error messages go both into one of my virtual host
server’s error log with correct data (remote address, server name,
etc.), but also go to nginx default (?) error_log with entries that I
reported? How come, if the virtual host server being accessed using
the slowhttptest program is clearly “testsite” with its own error_log,
yet the connection limit error message is reported in two error logs?

slowhttptest -c 1000 -r 1000 -X -u http://testsite

results:

/var/log/nginx/testsite/error_log:

2012/01/12 18:10:34 [error] 10508#0: *15985 limiting connections by zone
“addr”, client: 217.24.78.177, server: testsite, request: “GET /
HTTP/1.1”, host: “devel.ahlerstoday.com”, referrer:
“GitHub - shekyan/slowhttptest: Application Layer DoS attack simulator”
2012/01/12 18:10:35 [error] 10508#0: *15989 limiting connections by zone
“addr”, client: 217.24.78.177, server: testsite, request: “GET /
HTTP/1.1”, host: “devel.ahlerstoday.com”, referrer:
“GitHub - shekyan/slowhttptest: Application Layer DoS attack simulator”

/var/log/nginx/error_log:

2012/01/12 18:10:34 [error] 10509#0: *12134 limiting connections by zone
“addr”, client: 127.0.0.1, server: , request: “GET / HTTP/1.0”, host:
“testsite”, referrer: “GitHub - shekyan/slowhttptest: Application Layer DoS attack simulator”
2012/01/12 18:10:35 [error] 10509#0: *12136 limiting connections by zone
“addr”, client: 127.0.0.1, server: , request: “GET / HTTP/1.0”, host:
“testsite”, referrer: “GitHub - shekyan/slowhttptest: Application Layer DoS attack simulator”

Could it have anything to do with the fact that limit_zone and
limit_conn are defined on http level? Yet, I think the behaviour is
not fully correct.

I would appreciate your feedback,

Andrejs

Posted at Nginx Forum: