I have a very strange situation that I’ve been tracking for some
months now.
This may or may not be a rails issue. The symptoms don’t feel like it,
so I guess I’m appealing to my rails colleagues to lend me their
thoughts.
Throughout the day, HomeStars.com will get a number of invalid
requests(< 50) such as :
/companies/1199238-action-assembly/forums - is not valid
/companies/1199238-action-assembly - is valid
/forums - is valid
/forums is linked to in the menu on each page (so is /sessions/new
which also exhibits the same behaviour)
All I can surmise is that something is hitting the forums relative url
thinking it’s relative, but it’s an absolute reference.
So this feels like either a bug in a scraper, or a page pre-loader
(plugin that visits links on a page to pre-load them).
I see enough of them, with differing user agents, and vastly different
IPs, and not enough requests to not feel like a scraper.
So I’m left with thinking it’s a browser plugin - pre-loading.
any thoughts or pointers are appreciated.
thank you.
Jodi