Scrapping Content from a website

Rails
#1

Hey all,
I want to scrap some content from a website to rails application and
wants to authenticate the url of the website which I want to scarp so
please help me out…

Thank You!!!

#2

I don’t have a lot of experience but did some scraping recently. I
used the nokogiri gem. found it to be pretty self explanatory and
much easier to implement than i had anticipated. google it and i’m
sure you can figure it out…

Max

#3

I use Mechanize as user agent and Nokogiri as html/xml parser.
http://mechanize.rubyforge.org/
http://nokogiri.org/
Mechanize supports both http- and application- layer auth, so that
could be what You are looking for.

#4

Hey all,

Hi,

I want to scrap some content from a website to rails application and
wants to authenticate the url of the website which I want to scarp so
please help me out…

Mechanize, Nokogiri , Hpricot.
Some examples in ruby done by me
http://sathia27.wordpress.com/2011/09/04/version1-easy-dictionary/
http://sathia27.wordpress.com/2011/12/05/english-to-tamil-translator-script/
http://sathia27.wordpress.com/2011/09/06/tbus-version-1-search-bus-routes-from-terminal/
http://sathia27.wordpress.com/2011/05/30/update-foss-job-script/

–

#5

Thanks Dear for the information.