Hi I am currently doing a project that requires me to extract news stories from a website and add them to a mysql database. I downloaded and installed ruby and was thinking on using firebug and hpricot to help me along the way. I am using a windows machine and haven't got a clue how to start though :( Where do I write the ruby files in windows? Is there anything else I will need?
on 2009-04-15 14:00
on 2009-04-15 15:32
Ren Egade wrote: > I am currently doing a project that requires me to extract news stories > from a website and add them to a mysql database. This can be quite easy if your database is already in existence and if it has a good front end for adding, updating, deleting and saving stories. Depending on the frequency of story updates, you can simply copy and paste the required portions of the stories and reference them appropriately.(should be taken care of by the database developer/ data manager) > I downloaded and installed ruby and was thinking on using firebug and > hpricot to help me along the way. So copy and paste is mundane and tiresome and we want the machine to work for us while sipping coffee and drawing a healthy salary. hpricot is a one of the many libraries you can use for screen scraping, while firebug can help you to examine the DOM tree and know what tags to go for > > I am using a windows machine and haven't got a clue how to start though Am also lost as to why you have been using windows sorry DOS for this long :( > :( Where do I write the ruby files in windows? There are pretty of places you can write ruby files on DOS. create a folder under your drive or "your Documents", then use notepad to create your ruby files. make sure they end with a .rb extension. for example test.rb :) Assuming that ruby is installed, you can run your ruby file on the dos prompt e.g. C:/>ruby myfile.rb Is there anything else I > will need? Yes there are a couple of things you may need to know. Instead of using Notepad, edit your files using Notepad++ (it supports syntax highlighting, code folding etc and even macros i think. if you type irb on your dos command prompt, the interactive ruby interpreter wakes up. C:/>irb irb(main):001:0> use it to you advantage. For more information use this website http://www.google.com to find more about irb or shoot a specific question to this helpful mailing list if need be To really make work easier at the expense of a little more learning. Download the Netbeans Ruby editor from this website http://www.netbeans.com It has a full integrated development enviroment for ruby, all for free! you can write and run all the code within the IDE. Lastly i assume you know how to write simple computer programs. ruby supports different frameworks for developing database backed websites, for example merb, nitro, camping, rails, sinatra etc. rails is the most well known and has a lot of documentation(www.rubyonrails.com) use one of these existing frameworks depending on your needs and specification and level of expertise and support in your company to build a database front end for your project. The respective screen scrapping libraries have good documentation on how to use them and how to integrate them with rails or merb or whatever you choose to use. if things are too elephant with ruby switch to python and django or to php and mysql. they too can solve your problem. :) All the best! George
on 2009-04-15 17:01
Thanks George G. :) That info was very helpful :D Was just doing some ruby tutorials there and i'm finding my way about ok :) Still not sure how to use hpricot though :/