# Preventing abusive behavior

**URL:** <https://www.ruby-forum.com/t/preventing-abusive-behavior/204073>\
**Category:** NGINX\
**Created:** [March 4, 2011, 11:09am UTC](https://www.ruby-forum.com/t/preventing-abusive-behavior/204073 "2011-03-04T11:09:53Z")\
**Posts on this page:** 1\
**Page:** 1

<div class="post-metadata">

**Author:** ![dubstep](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/d/cdc98d/32.png) [@dubstep](https://www.ruby-forum.com/u/dubstep)\
**Post date:** [March 4, 2011, 11:09am UTC](https://www.ruby-forum.com/t/preventing-abusive-behavior/204073/1 "2011-03-04T11:09:53Z")

</div>

Hi -

What do the users of this ML use to block abusive behavior and spiders  
that don’t respect robots.txt?

I observe 2 kinds of abusive behavior on my site:

- Vulnerability scanners/ abusive crawlers
- Targeted PHP app failed logins

For 1, I block a few bots by manually adding them to a deny list in  
nginx and a few others through spider traps, which are essentially  
locations that log to a separate log file which is then scanned by  
fail2ban.

For 2, I do the same as above (the trap is an image on the login page)  
but also use the Limit Requests module that logs to error.log and is  
also scanned by fail2ban.

The problem is that I regularly have to go through the fail2ban logs  
to see what it has caught and possibly add to the static list in the  
nginx conf. It would be nice to have an auto-updating list of bad bots  
from [user-agents.org](http://user-agents.org) or similar sites instead of the hassle of having  
to create my own list.

So, what do you use?

bb