Protecting nginx from syn flood and DOS vs legit heavy traffic

We are using nginx as a public web server and need to do good common
sense things to try and limit or prevent syn floods and related types of
DOS attacks.

I’ve researched iptables extensively and have found a lot of info on how
to use it to limit syn floods and so forth.

However these articles do not explain how to apply these iptable
restrictions to public web servers that get very large amounts of
traffic. So I am hoping others here can share how they are using
iptables, because I am concerned that I will inadvertently block good
traffic!

For instance, consider a case whereby a huge company with thousands of
employees that all share one public IP when accessing the internet.
Further, consider that everyone in the company gets an email that says
to go to our site and review some web pages.

In this scenario it is possible we could have a few thousand requests
coming in all at the same time from the same IP, but be legitimate
requests. So I have to be very careful with the rules that can try (if
possible?) to tell the difference between heavy traffic from the same IP
(as in this scenario) vs. some bot hammering on the server.

As another example, from the syn flood iptable rules I’ve seen I can’t
tell whether it is possible to detect the difference between syn packets
that are purposeful vs a large number of syn packets for new connections
that are rushing in but legitimate.

Also as a side question - if a request comes in to nginx and nginx then
uses proxy_pass to talk to an external server that handles the request,
am I right to assume that as far as iptables is concerned this is an
INPUT and not a FORWARD? In the case where we only want the public to
access the nginx server is there ever a case where we may legitimately
want to take FORWARD requests or should these all be blocked?

I would GREATLY appreciate you sharing your thoughts on how to address
this and approaches you have taken that may apply in this case too.

For reference I am using the latest nginx 6 on Fedora 8 core.

Thank you!!

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Rt Ibmer pisze:
| We are using nginx as a public web server and need to do good common
sense things to try and limit or prevent syn floods and related types of
DOS attacks.
|
| I’ve researched iptables extensively and have found a lot of info on
how to use it to limit syn floods and so forth.
|
| However these articles do not explain how to apply these iptable
restrictions to public web servers that get very large amounts of
traffic. So I am hoping others here can share how they are using
iptables, because I am concerned that I will inadvertently block good
traffic!
|
| For instance, consider a case whereby a huge company with thousands of
employees that all share one public IP when accessing the internet.
Further, consider that everyone in the company gets an email that says
to go to our site and review some web pages.
|
| In this scenario it is possible we could have a few thousand requests
coming in all at the same time from the same IP, but be legitimate
requests. So I have to be very careful with the rules that can try (if
possible?) to tell the difference between heavy traffic from the same IP
(as in this scenario) vs. some bot hammering on the server.
|
| As another example, from the syn flood iptable rules I’ve seen I can’t
tell whether it is possible to detect the difference between syn packets
that are purposeful vs a large number of syn packets for new connections
that are rushing in but legitimate.
|
| Also as a side question - if a request comes in to nginx and nginx
then uses proxy_pass to talk to an external server that handles the
request, am I right to assume that as far as iptables is concerned this
is an INPUT and not a FORWARD? In the case where we only want the public
to access the nginx server is there ever a case where we may
legitimately want to take FORWARD requests or should these all be
blocked?
|
| I would GREATLY appreciate you sharing your thoughts on how to address
this and approaches you have taken that may apply in this case too.
|
| For reference I am using the latest nginx 6 on Fedora 8 core.
How do you limit the SYN packets (show your iptables rules)? Are you
tried TCP SYN Proxy? pf from OpenBSD is good tool for that
(OpenBSD PF: Packet Filtering).


Bart³omiej Syryjczyk
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkhp4jgACgkQt25UBlJ9LC4Z7gCeLfEXqW3cWYg1TY3fqHgzyR2M
jpQAn0tGqvqa4DS9aaVHwUVW8j4RPNZ9
=on62
-----END PGP SIGNATURE-----

Well, synflooding is more of a OS problem than it is a nginx problem,
since the connection is never actually established only initiated.
A severe TCP connect attack would impact nginx more, but the affect of
it
can be
minimized with a small client_header_timeout and/or client_body timeout.

The synflooding can be somewhat managed with varios OS variables
(sysctl),
I only use FreeBSD, but it might help you anyway, just look at
net.inet.tcp.drop_synfin (which is always good to be set to 1)
and other values of net.inet.tcp.syncache.* and net.inet.tcp.syncookies.

You can also limit the number of syn packets on your network to a
reasonable number per time unit.
The bottom line is, that the attacks are not avoidable but only
managable.

On Tue, 01 Jul 2008 09:52:24 +0200, Bartlomiej Syryjczyk
[email protected] wrote:

|
|
|
| Also as a side question - if a request comes in to nginx and nginx
then uses proxy_pass to talk to an external server that handles the
request, am I right to assume that as far as iptables is concerned this
is an INPUT and not a FORWARD? In the case where we only want the public
to access the nginx server is there ever a case where we may
legitimately want to take FORWARD requests or should these all be
blocked?
Bartłomiej Syryjczyk
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAkhp4jgACgkQt25UBlJ9LC4Z7gCeLfEXqW3cWYg1TY3fqHgzyR2M
jpQAn0tGqvqa4DS9aaVHwUVW8j4RPNZ9
=on62
-----END PGP SIGNATURE-----

iDEV, podjetje za spletne storitve, d.o.o.
Tomaž Marhat, CEO
Brda 15A
SI-2383 � martno pri Slovenj Gradcu
SLOVENIA - EU

Web: http://idev.si/
VAT #: SI43028195
Company ID #: 2346265
Mobile #: +386 70 819 789

A severe TCP connect attack would impact nginx more, but
the affect of it
can be
minimized with a small client_header_timeout and/or
client_body timeout.

Thanks for the tips. Can you explain what LEGITIMATE conditions could
cause a client_header_timeout and/or client_body_timeout condition?

The default for those is 60 seconds which seems much higher than I think
we need, so I would like to more aggressively shrink those down.

In case it matters, our nginx is the front end server for a web service.
Basically we get a request for a small htm or xml or js or gif file and
serve it, and then that is the end of the content.

What setting do you think I can use for those timeouts to be more
aggressive but without running the chance of returning timing legitimate
requests out? Thanks!!