I’ve been faced the the following symptoms for some time.
I have links coded as :post or :put, so I can make sure that bots
aren’t hitting particular links.
But it something is either hitting them as :get through an error I’ve
made (like link_to not working well in some browsers?), or there’s 1
or more plugins that pre load urls; or I have scrapers.
Each day I’ll get 50-100 error messages - where routes aren’t found -
of this nature.
When I get many of these hits from the same ip address, I usually
assume a scraper, then block that ip address…but I don’t want to do
this in very case if it’s possible that a legimate (pre-loader browser
plugin) is causing this to happen.
Does anybody else this kind of behaviour? How do you handle it?