Q re nokogiri - searching Google and counting web pages with specific text

People,

I think using nokogiri with Ruby should be able to do what I want but
I’m not sure how to do it - I want to search Google and list or count
the number of web pages from a particular domain:

http://www.unitednations.org

that contain specific text eg “george harrison” or “oxfam” etc.

Could someone get me started?

Thanks,

Phil.

Philip R.

GPO Box 3411
Sydney NSW 2001
Australia
E-mail: [email protected]

On Aug 18, 2009, at 10:26, Philip R. wrote:

I think using nokogiri with Ruby should be able to do what I want
but I’m not sure how to do it - I want to search Google and list or
count the number of web pages from a particular domain:

http://www.unitednations.org

that contain specific text eg “george harrison” or “oxfam” etc.

Could someone get me started?

Why don’t you use the google search API?

Eric,

On 2009-08-19 04:22, Eric H. wrote:

Why don’t you use the google search API?

Seems like a good idea but I have hassles installing the dependency
ruby-SOAP4R rpm . .

Phil.

Philip R.

GPO Box 3411
Sydney NSW 2001
Australia
E-mail: [email protected]

On Tue, Aug 18, 2009 at 3:31 PM, Philip R.[email protected]
wrote:

ruby-SOAP4R rpm . .

The google search API does not use SOAP anymore, that’s been
deprecated for a while. You can use this:

Or just use httparty or open-uri and parse the json that’s returned
from the new search API.

Cheers,
Jason

On Aug 18, 2009, at 12:31, Philip R. wrote:

that contain specific text eg “george harrison” or “oxfam” etc.

Could someone get me started?

Why don’t you use the google search API?

Seems like a good idea but I have hassles installing the dependency
ruby-SOAP4R rpm . .

Looks like the SOAP API isn’t being maintained anymore, use the modern
one: