# Preventing crawlers on link\_to's **URL:** \ **Category:** Rails\ **Created:** [April 16, 2006, 8:26pm UTC](https://www.ruby-forum.com/t/preventing-crawlers-on-link-tos/93965 "2006-04-16T20:26:03Z")\ **Posts on this page:** 5\ **Page:** 1
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [April 16, 2006, 8:26pm UTC](https://www.ruby-forum.com/t/preventing-crawlers-on-link-tos/93965/1 "2006-04-16T20:26:03Z")
My understanding was that using the :post=\>true on a link\_to() was supposed to prevent search engine crawlers from triggering the link. However, this does not seem to be working for me. Is there something else that I should be/can be doing to accomplish this? Thanks. -Matt ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [April 16, 2006, 8:56pm UTC](https://www.ruby-forum.com/t/preventing-crawlers-on-link-tos/93965/2 "2006-04-16T20:56:27Z")
On Sunday 16 Apr 2006 19:23, Belorion wrote: > My understanding was that using the :post=\>true on a link\_to() was supposed > to prevent search engine crawlers from triggering the link. However, this > does not seem to be working for me. Is there something else that I should > be/can be doing to accomplish this? Thanks. Adding :post doesn’t do anything other than insert dynamically generated (Javascript) tags, so it won’t do anything for non-Javascript clients such as web crawlers. From the Rails API: “And a third for making the link do a POST request (instead of the regular GET) through a dynamically added form element that is instantly submitted. Note that if the user has turned off Javascript, the request will fall back on the GET. So its your responsibility to determine what the action should be once it arrives at the controller. The POST form is turned on by passing :post as true.” So basically, you need to check if the request is a GET, and if so most likely fall back to a second action which displays an actual form in order to call the first action again. In your controller… def destructive\_action if request.post? # do some destructive action else redirect\_to :action =\> “confirm\_destruct” end end … where confirm\_destruct will be another action that displays an actual POST form which then goes on to call destructive\_action again using the same parameters but requiring an extra click of a form submission button. HTH. ~Dave – Dave S. Rent-A-Monkey Website Development > **[Tree Removal Service For Home & Businesses in Salt Lake City, UT](https://www.rentamonkey.com/)** > > Rent A Monkey Tree Service provides the safest, highest-quality tree removal for Salt Lake City, UT homes and businesses. Request a service today! PGP Key: [http://www.rentamonkey.com/pgpkey.asc](http://www.rentamonkey.com/pgpkey.asc) ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [April 16, 2006, 9:54pm UTC](https://www.ruby-forum.com/t/preventing-crawlers-on-link-tos/93965/3 "2006-04-16T21:54:52Z")
You can always use robots.txt to prevent search engines from indexing certain areas of your site > **[robots.txt](https://en.wikipedia.org/wiki/Robots.txt)** > > robots.txt is the filename used for implementing the Robots Exclusion Protocol, a standard used by websites to indicate to visiting web crawlers and other web robots which portions of the website they are allowed to visit. > This relies on voluntary compliance. Not all robots comply with the standard; email harvesters, spambots, malware and robots that scan for security vulnerabilities may even start with the portions of the website where they have been told to stay out. > The "robots.txt" file c... ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [April 16, 2006, 11:24pm UTC](https://www.ruby-forum.com/t/preventing-crawlers-on-link-tos/93965/4 "2006-04-16T23:24:15Z")
Belorion \ writes: > My understanding was that using the :post=\>true on a link\_to() was supposed to > prevent search engine crawlers from triggering the link. However, this > does not > seem to be working for me. Is there something else that I should be/can > be > doing to accomplish this? Thanks. > -Matt Google, Yahoo! and MSFT honor the rel=“nofollow” attribute on a link. Here’s some info: > **[Preventing comment spam](https://googleblog.blogspot.com/2005/01/preventing-comment-spam.html)** > > Insights from Googlers into our products, technology, and the Google culture Also note that some crawlers are starting to parse an execute javascript. We noticed that the Google crawler started executing code sitting behind Ajax.Updaters earlier this year. ---
**Author:** ![casper\_the\_ghost](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/58956e/32.png) [@casper\_the\_ghost](https://www.ruby-forum.com/u/casper_the_ghost)\ **Post date:** [April 16, 2006, 11:27pm UTC](https://www.ruby-forum.com/t/preventing-crawlers-on-link-tos/93965/5 "2006-04-16T23:27:24Z")
i was wondering the opposite. if you have a ‘single page’ site model, are the crawlers smart enough to sift thru the javascript? what about contextual ads, is there a way to trigger updates to these when substantial parts of the page content change?