Re: Large number of invalid packets detected

Hi,

On Sun, May 18, 2008 22:11, Rt Ibmer wrote:

You didn’t show us what was important: the log that was produced.

I included various samples from the different types of invalid state
errors at the end of this message.

If the rejected packet was a ACK/FIN, then it is harmless.

About 25% of the invalid packets are of this type. Should I drop them
using DROP or using RESET?

I think you can drop them.

How about for the other types? I take it
there is no harm in dropping or rejecting these - as they are obviously
bad packets anyway?

I think you really might better ask those questions on the netfilter
list:
http://www.netfilter.org/mailinglists.html
There is certainly lot of people way more knowledgeable than on the
nginx
list about iptables/netfilter :slight_smile:

I should note that in EVERY case it appears that nginx served the content
successfully and that browser got it just fine (no obvious errors), and
these bad packet log messages are showing up anywhere from 2-5 minutes
after the user hits the page and not part of a new request. So it seems
there is some packet activity that occurs minutes after the user has left
the site.

Maybe you could tcpdump the whole traffic to analyze exactly what’s
happening?

On busy servers conntrack removes the connection from its table as soon
as
it gets the FIN packet. When the ACK/FIN then comes, the connection is
not
known and this is not a SYN. It logs it as invalid.

My server is not very busy (yet). It is handling about 500-1500 requests
per hour and no more than a 2 or 3 concurrently at times.

Yes, definitely not busy. Anyway, I think what I wrote still applies,
although I’m really not a netfilter expert.

This could also have been a scan (nmap can trigger invalid packets too).

I really don’t think this could be a scan. I took the site live for the
first time and immediately started seeing these invalid packets.

You bet? I get so much automatic scans per seconds, that I don’t even
look
to the logs anymore :frowning:

but will give you an idea of what % are for what type of bad packet):
MAC=12:…:ff:08:00 SRC=x.y.z.a DST=10.x.x.x LEN=40 TOS=0x00 PREC=0x00
TTL=240 ID=17970 DF PROTO=TCP SPT=49671 DPT=80 WINDOW=0 RES=0x00 ACK RST
URGP=0

It is indeed the same issue than ACK/FIN
http://lists.netfilter.org/pipermail/netfilter/2004-June/053782.html

-------- (INPUT)
May 18 15:24:08 domU kernel: PCK-INP-BADSTATE-DROP:IN=eth0 OUT=
MAC=12:…:ff:08:00 SRC==x.y.z.a DST=10.x.x.x LEN=40 TOS=0x00 PREC=0x00
TTL=49 ID=25196 DF PROTO=TCP SPT=57214 DPT=80 WINDOW=65535 RES=0x00 ACK
FIN URGP=0

May 18 12:58:10 domU kernel: PCK-INP-BADSTATE-DROP:IN=eth0 OUT=
MAC=12:…:ff:08:00 SRC=x.y.z.a DST=10.x.x.x LEN=52 TOS=0x00 PREC=0x00
TTL=47 ID=52221 DF PROTO=TCP SPT=55817 DPT=80 WINDOW=65535 RES=0x00 ACK
FIN URGP=0

We already talked about this one above. This could also be a Maimon scan
which uses ACK+FIN.

-------- (INPUT)
May 18 13:54:58 domU kernel: PCK-INP-BADSTATE-DROP:IN=eth0 OUT=
MAC=12:…:ff:08:00 SRC==x.y.z.a DST=10.x.x.x LEN=40 TOS=0x00 PREC=0x00
TTL=238 ID=9943 PROTO=TCP SPT=4361 DPT=80 WINDOW=0 RES=0x00 RST URGP=0

May 18 12:31:37 domU kernel: PCK-INP-BADSTATE-DROP:IN=eth0 OUT=
MAC=12:…:ff:08:00 SRC=x.y.z.a DST=10.x.x.x LEN=40 TOS=0x00 PREC=0x00
TTL=237 ID=65259 PROTO=TCP SPT=12568 DPT=80 WINDOW=6432 RES=0x00 RST
URGP=0

Hum, that one seems strange. I don’t see any easy explanation.
Maybe with a full capture of the traffic you will be able to understand
what’s that.

9 are for type “ACK PSH FIN”. Example:

May 18 02:00:03 domU kernel: PCK-OUT-BADSTATE-DROP:IN= OUT=eth0
SRC=10.x.x.x DST=x.y.z.a LEN=40 TOS=0x00 PREC=0x00 TTL=64 ID=29585 DF
PROTO=TCP SPT=80 DPT=5178 WINDOW=54 RES=0x00 ACK PSH FIN URGP=0

http://lists.netfilter.org/pipermail/netfilter/2005-August/062059.html

As a side note, I’m also running conntrack on my servers, and I do see
some of the ACK+FIN and ACK+RST invalid packets. I drop them and doesn’t
really care: nobody ever complained :slight_smile:

Good luck :slight_smile: