Scrapping Content from a website

Hey all,
I want to scrap some content from a website to rails application and
wants to authenticate the url of the website which I want to scarp so
please help me out…

Thank You!!!

I don’t have a lot of experience but did some scraping recently. I
used the nokogiri gem. found it to be pretty self explanatory and
much easier to implement than i had anticipated. google it and i’m
sure you can figure it out…

Max

I use Mechanize as user agent and Nokogiri as html/xml parser.
http://mechanize.rubyforge.org/
http://nokogiri.org/
Mechanize supports both http- and application- layer auth, so that
could be what You are looking for.

Hey all,

Hi,

I want to scrap some content from a website to rails application and
wants to authenticate the url of the website which I want to scarp so
please help me out…

Mechanize, Nokogiri , Hpricot.
Some examples in ruby done by me
http://sathia27.wordpress.com/2011/09/04/version1-easy-dictionary/
http://sathia27.wordpress.com/2011/12/05/english-to-tamil-translator-script/
http://sathia27.wordpress.com/2011/09/06/tbus-version-1-search-bus-routes-from-terminal/
http://sathia27.wordpress.com/2011/05/30/update-foss-job-script/

Thanks Dear for the information.

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs