# Bot-taming: which rules should apply, and in which order?

**URL:** <https://www.ruby-forum.com/t/bot-taming-which-rules-should-apply-and-in-which-order/221763>\
**Category:** NGINX\
**Created:** [July 29, 2012, 6:20pm UTC](https://www.ruby-forum.com/t/bot-taming-which-rules-should-apply-and-in-which-order/221763 "2012-07-29T18:20:26Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![aris](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/258eb7/32.png) [@aris](https://www.ruby-forum.com/u/aris)\
**Post date:** [July 29, 2012, 6:20pm UTC](https://www.ruby-forum.com/t/bot-taming-which-rules-should-apply-and-in-which-order/221763/1 "2012-07-29T18:20:26Z")

</div>

I’m attempting to tame (minimize or eliminate) Yandex bot access.

I’d like to understand the application/precedence of the rules I apply.

To my site config I’ve added

map $http\_user\_agent $bad\_bot {  
default 0;  
~(Yandex|YandexBot) 1;  
}

map $http\_referrer $bad\_referrer {  
default 0;  
~\*(yandex) 1;  
}

valid\_referers [mydomain.com](http://mydomain.com) \*.mydomain.com localhost 127.0.0.1  
[::1];

location / {  
if ($bad\_bot) {return 403;}  
if ($bad\_referrer) {return 403;}  
if ($invalid\_referer) {return 444;}  
…  
}

and

cat /robots.txt  
User-agent: \*  
Disallow: /

cat /robot\_ssl.txt  
User-agent: \*  
Disallow: /

In my logs I see repeating ‘444’ rejections:

100.43.83.148 - - [28/Jul/2012:06:02:14 -0500] GET /robots.txt  
HTTP/1.1 “444” 0 “-” “Mozilla/5.0 (compatible; YandexBot/3.0;  
+http://yandex.com/bots)” “-”  
100.43.83.148 - - [28/Jul/2012:06:06:23 -0500] GET /robots.txt  
HTTP/1.1 “444” 0 “-” “Mozilla/5.0 (compatible; YandexBot/3.0;  
+http://yandex.com/bots)” “-”

With my rules above, I’d expect that to be a ‘403’ rejection, as  
specified for the “$bad\_bot” check.

Why am I seeing the ‘444’ instead of the ‘403’?