Hi, I installed nginx on an EC2 instance. After few hours, I started getting repeated requests from a set of servers. I tried using limit_req with the following options: limit_req_zone $binary_remote_addr zone=ratezone:10m rate=3r/s; limit_req zone=ratezone burst=5 nodelay; But I found that it is not effective. If you take a look at the following access_log content, you would notice that the IP addresses are different. I don't see more than 3 requests in a sec. Another weird thing is GET requests are starting with *"http://". *I never saw it before. Is there any way I can filter requests or possibly throw 503? Any help is really appreciated. 108.62.157.221 - - [16/Mar/2013:06:48:32 +0000] "GET http://ad.tagjunction.com/st?ad_type=iframe&ad_siz... 404 570 " http://www.oslims.com/green-coffee/pure-coffee/why... "Mozilla/4.0 (compatible; MSIE 6.01; Windows 95; Alexa Toolbar)" "-" 108.62.192.236 - - [16/Mar/2013:06:48:32 +0000] "GET http://ads1.ministerial5.com/creative/2-002134604-... HTTP/1.0" 404 570 " http://femalefashionroad.com/index.php?option=com_... "Mozilla/4.0 (compatible; MSIE 4.01; Windows 95)" "-" 173.208.16.212 - - [16/Mar/2013:06:48:32 +0000] "GET http://ib.adnxs.com/ttj?id=1184170 HTTP/1.0" 404 570 " http://ffwoman.com/index.php?option=com_content&vi... "Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.20 Safari/535.1" "-" 173.234.116.220 - - [16/Mar/2013:06:48:32 +0000] "GET http://ad.globe7.com/st?ad_type=pop&ad_size=0x0&se... 404 570 " http://www.economysea.com/index.php?option=com_con... "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.11 (KHTML, like Gecko) Ubuntu/11.04 Chromium/17.0.963.65 Chrome/17.0.963.65 Safari/535.11" "-" 72.52.75.73 - - [16/Mar/2013:06:48:32 +0000] "GET http://ib.adnxs.com/tt?id=1121510&cb=${CACHEBUSTER... 404 570 " http://www.tvzhou.com/?tag=lisa&paged=2" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/18.6.872.0 Safari/535.2 UNTRUSTED/1.0 3gpp-gba UNTRUSTED/1.0" "-" 23.19.67.56 - - [16/Mar/2013:06:48:32 +0000] "GET http://ad.tagjunction.com/st?ad_type=iframe&ad_siz... 404 168 " http://economicface.com/index.php?option=com_mailt... "Mozilla/5.0 (Windows; U; WinNT3.51; en-US; rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7" "-" 173.234.145.205 - - [16/Mar/2013:06:48:32 +0000] "GET http://ad.globe7.com/st?ad_type=iframe&ad_size=728... 404 570 " http://classidressing.com/index.php?view=article&c... "Mozilla/4.0 (compatible; MSIE 5.01; Windows 95; MSIECrawler)" "-" 142.4.126.137 - - [16/Mar/2013:06:48:32 +0000] "GET http://ads.clovenetwork.com/ttj?id=801591&pubclick... 404 570 " http://www.today-car.com/?cat=601" "Mozilla/4.0 (compatible; MSIE 6.0; Update a; Win32)" "-" 23.19.130.109 - - [16/Mar/2013:06:48:32 +0000] "GET http://ads1.ministerial5.com/creative/2-002134516-... HTTP/1.0" 500 594 " http://likecatpink.com/index.php?option=com_conten... "Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 4.0; Alexa Toolbar)" "-" 108.62.17.245 - - [16/Mar/2013:06:48:32 +0000] "GET http://ib.adnxs.com/ttj?id=1200348&cb=${CACHEBUSTE... 404 168 " http://styleear.com/index.php?option=com_mailto&tm... "Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.7) Gecko/20040626 Firefox/0.8" "-" -Praveen
on 2013-03-16 10:35
on 2013-03-16 11:06
On 16 March 2013 09:34, Praveen Yarlagadda <praveen.yarlagadda@gmail.com> wrote: > access_log content, you would notice that the IP addresses are different. I > don't see more than 3 requests in a sec. Another weird thing is GET requests > are starting with "http://". I never saw it before. Is there any way I can > filter requests or possibly throw 503? How about location http:// { access_log off; return 444; } Jonathan -- Jonathan Matthews // Oxford, London, UK http://www.jpluscplusm.com/contact.html
on 2013-03-16 11:39
On Sat, Mar 16, 2013 at 02:34:32AM -0700, Praveen Yarlagadda wrote: Hi there, > I installed nginx on an EC2 instance. > Another weird thing is GET > requests are starting with *"http://". *I never saw it before. Is there any > way I can filter requests or possibly throw 503? These might be innocent requests from browsers configured to use your IP address as a proxy server. (Maybe there was a proxy server on a previous instance that used your current address?) I suggest making your current server{} blocks list all of the server_name:s that you want to handle, and then let the default server{} block handle these other requests, with "return 503" or any other configuration you like. See http://nginx.org/r/listen and http://nginx.org/r/server_name for how to configure server names and the default server for a given address:port. f -- Francis Daly francis@daoine.org
on 2013-03-16 20:38
Thanks a lot, Jonathan and Francis!
It works great. I am able to significantly reduce the load. Here is my
final configuration:
* limit_req_zone $binary_remote_addr zone=ratezone:10m rate=3r/s;*
* server {*
* listen 80;*
* server_name www.example.com;*
*
*
* location / {*
* limit_req zone=ratezone burst=5 nodelay;*
* proxy_pass http://appservers;*
* }*
* }*
*
*
* server {*
* listen 80;*
* server_name ~.*;*
* location / {*
* access_log off;*
* return 503;*
* }*
* }*
-Praveen
Please log in before posting. Registration is free and takes only a minute.
Existing account
(Switch to SSL-encrypted connection)
NEW: Do you have a Google/GoogleMail or Yahoo account? No registration required!
Log in with Google account | Log in with Yahoo account
Log in with Google account | Log in with Yahoo account
No account? Register here.