Forum: Ruby on Rails re: relative urls - browser plugin? scraper? other?

Announcement (2017-05-07): www.ruby-forum.com is now read-only since I unfortunately do not have the time to support and maintain the forum any more. Please see rubyonrails.org/community and ruby-lang.org/en/community for other Rails- und Ruby-related community platforms.
5aecb4f86cca29ff0419d764ff37281c?d=identicon&s=25 Jodi Showers (Guest)
on 2008-12-17 17:42
(Received via mailing list)
A short update on this - never saw this behaviour before - but will
see more now

Googlebot just hit this url - /companies/1199238-action-assembly/forum

All I can imagine is that the relative url must exist - but for the
life of me I can't see it.

Jodi

------


I have a very strange situation that I've been tracking for some
months now.

This may or may not be a rails issue. The symptoms don't feel like it,
so I guess I'm appealing to my rails colleagues to lend me their
thoughts.



Throughout the day, HomeStars.com will get a number of invalid
requests(< 50) such as :

/companies/1199238-action-assembly/forums - is not valid

/companies/1199238-action-assembly - is valid
/forums - is valid

/forums is linked to in the menu on each page (so is /sessions/new
which also exhibits the same behaviour)

All I can surmise is that something is hitting the forums relative url
thinking it's relative, but it's an absolute reference.

<a href="/forums">Forum</a>

So this feels like either a bug in a scraper, or a page pre-loader
(plugin that visits links on a page to pre-load them).

I see enough of them, with differing user agents, and vastly different
IPs, and not enough requests to not feel like a scraper.

So I'm left with thinking it's a browser plugin - pre-loading.

any thoughts or pointers are appreciated.

thank you.
Jodi
This topic is locked and can not be replied to.