Client IP address

Server NGINX
1

We have a cluster of 4 nginx proxies behind a piranha load balancer
setup.
This morning we suffered a DOS attack, however the “client” address
appears
to have only gotten logged correctly the first time, the rest have the
virtual ip address targeted as the “client”, and it’s unclear how or why
that would happen. The setup is in “direct return” mode… Thanks for
any
insights!

Sep 16 05:45:25 mailproxy-lb-01 nginx: 2014/09/16 05:45:25 [error]
16529#0:
*111301 upstream timed out (110: Connection timed out) while reading
response header from upstream, client: 213.5.67.223, server:
mail.wvi.com,
request: “POST
/cgi-bin/php?%2D%64+%61%6C%6C%6F%77%5F%75%72%6C%5F%69%6E%63%6C%75%64%65%3D%6F%6E+%2D%64+%73%61%66%65%5F%6D%6F%64%65%3D%6F%66%66+%2D%64+%73%75%68%6F%73%69%6E%2E%73%69%6D%75%6C%61%74%69%6F%6E%3D%6F%6E+%2D%64+%64%69%73%61%62%6C%65%5F%66%75%6E%63%74%69%6F%6E%73%3D%22%22+%2D%64+%6F%70%65%6E%5F%62%61%73%65%64%69%72%3D%6E%6F%6E%65+%2D%64+%61%75%74%6F%5F%70%72%65%70%65%6E%64%5F%66%69%6C%65%3D%70%68%70%3A%2F%2F%69%6E%70%75%74+%2D%64+%63%67%69%2E%66%6F%72%63%65%5F%72%65%64%69%72%65%63%74%3D%30+%2D%64+%63%67%69%2E%72%65%64%69%72%65%63%74%5F%73%74%61%74%75%73%5F%65%6E%76%3D%30+%2D%6E
HTTP/1.1”, upstream:
"http://207.55.17.73:80/cgi-bin/php?-d+allow_url_include%3Don+-d+safe_mode%3Doff+-d+suhosin.simul%6
Sep 16 05:45:25 mailproxy-lb-01 nginx: 2014/09/16 05:45:25 [error]
16529#0:
*111303 upstream timed out (110: Connection timed out) while reading
response header from upstream, client: 207.55.17.73, server:
mail.wvi.com,
request: “POST
/cgi-bin/php?%2D%64+%61%6C%6C%6F%77%5F%75%72%6C%5F%69%6E%63%6C%75%64%65%3D%6F%6E+%2D%64+%73%61%66%65%5F%6D%6F%64%65%3D%6F%66%66+%2D%64+%73%75%68%6F%73%69%6E%2E%73%69%6D%75%6C%61%74%69%6F%6E%3D%6F%6E+%2D%64+%64%69%73%61%62%6C%65%5F%66%75%6E%63%74%69%6F%6E%73%3D%22%22+%2D%64+%6F%70%65%6E%5F%62%61%73%65%64%69%72%3D%6E%6F%6E%65+%2D%64+%61%75%74%6F%5F%70%72%65%70%65%6E%64%5F%66%69%6C%65%3D%70%68%70%3A%2F%2F%69%6E%70%75%74+%2D%64+%63%67%69%2E%66%6F%72%63%65%5F%72%65%64%69%72%65%63%74%3D%30+%2D%64+%63%67%69%2E%72%65%64%69%72%65%63%74%5F%73%74%61%74%75%73%5F%65%6E%76%3D%30+%2D%6E
HTTP/1.0”, upstream:
"http://207.55.17.73:80/cgi-bin/php?-d+allow_url_include%3Don+-d+safe_mode%3Doff+-d+suhosin.simul%6
Sep 16 05:45:25 mailproxy-lb-01 nginx: 2014/09/16 05:45:25 [info]
16529#0:
*111303 shutdown() failed (107: Transport endpoint is not connected)
while
sending to client, client: 207.55.17.73, server: mail.wvi.com, request:
“POST
/cgi-bin/php?%2D%64+%61%6C%6C%6F%77%5F%75%72%6C%5F%69%6E%63%6C%75%64%65%3D%6F%6E+%2D%64+%73%61%66%65%5F%6D%6F%64%65%3D%6F%66%66+%2D%64+%73%75%68%6F%73%69%6E%2E%73%69%6D%75%6C%61%74%69%6F%6E%3D%6F%6E+%2D%64+%64%69%73%61%62%6C%65%5F%66%75%6E%63%74%69%6F%6E%73%3D%22%22+%2D%64+%6F%70%65%6E%5F%62%61%73%65%64%69%72%3D%6E%6F%6E%65+%2D%64+%61%75%74%6F%5F%70%72%65%70%65%6E%64%5F%66%69%6C%65%3D%70%68%70%3A%2F%2F%69%6E%70%75%74+%2D%64+%63%67%69%2E%66%6F%72%63%65%5F%72%65%64%69%72%65%63%74%3D%30+%2D%64+%63%67%69%2E%72%65%64%69%72%65%63%74%5F%73%74%61%74%75%73%5F%65%6E%76%3D%30+%2D%6E
HTTP/1.0”, upstream:
"http://207.55.17.73:80/cgi-bin/php?-d+allow_url_include%3Don+-d+safe_mode%3Doff+-d+suhosin.simulati

Posted at Nginx Forum:

2

Additional information: I caught it in the act, and something about this
trigger and the setup is causing nginx to loop - the client ip address
is
actually right and nginx is proxying the request to itself as fast as it
can. Restarting nginx stops the loop. This is version 0.7.65. I tried
upgrading to 1.7.4 recently but the syslog support doesn’t seem to work
for
mail and I haven’t gotten a chance to try the old patches yet.

Posted at Nginx Forum:

3

Never mind - a new proxy target was misconfigured. Doh!

Posted at Nginx Forum: