# Q re nokogiri - searching Google and counting web pages with specific text

**URL:** <https://www.ruby-forum.com/t/q-re-nokogiri-searching-google-and-counting-web-pages-with-specific-text/173964>\
**Category:** Ruby\
**Created:** [August 18, 2009, 7:26pm UTC](https://www.ruby-forum.com/t/q-re-nokogiri-searching-google-and-counting-web-pages-with-specific-text/173964 "2009-08-18T19:26:44Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![unebaguettesvp](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/47e85d/32.png) [@unebaguettesvp](https://www.ruby-forum.com/u/unebaguettesvp)\
**Post date:** [August 18, 2009, 7:26pm UTC](https://www.ruby-forum.com/t/q-re-nokogiri-searching-google-and-counting-web-pages-with-specific-text/173964/1 "2009-08-18T19:26:44Z")

</div>

People,

I think using nokogiri with Ruby should be able to do what I want but  
I’m not sure how to do it - I want to search Google and list or count  
the number of web pages from a particular domain:

[http://www.unitednations.org](http://www.unitednations.org)

that contain specific text eg “george harrison” or “oxfam” etc.

Could someone get me started?

Thanks,

## Phil.

Philip R.

GPO Box 3411  
Sydney NSW 2001  
Australia  
E-mail: removed\_email\_address@domain.invalid

---

<div class="post-metadata">

**Author:** ![unebaguettesvp](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/47e85d/32.png) [@unebaguettesvp](https://www.ruby-forum.com/u/unebaguettesvp)\
**Post date:** [August 18, 2009, 8:24pm UTC](https://www.ruby-forum.com/t/q-re-nokogiri-searching-google-and-counting-web-pages-with-specific-text/173964/2 "2009-08-18T20:24:06Z")

</div>

On Aug 18, 2009, at 10:26, Philip R. wrote:

> I think using nokogiri with Ruby should be able to do what I want  
> but I’m not sure how to do it - I want to search Google and list or  
> count the number of web pages from a particular domain:
> 
> [http://www.unitednations.org](http://www.unitednations.org)
> 
> that contain specific text eg “george harrison” or “oxfam” etc.
> 
> Could someone get me started?

Why don’t you use the google search API?

---

<div class="post-metadata">

**Author:** ![unebaguettesvp](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/47e85d/32.png) [@unebaguettesvp](https://www.ruby-forum.com/u/unebaguettesvp)\
**Post date:** [August 18, 2009, 9:31pm UTC](https://www.ruby-forum.com/t/q-re-nokogiri-searching-google-and-counting-web-pages-with-specific-text/173964/3 "2009-08-18T21:31:26Z")

</div>

Eric,

On 2009-08-19 04:22, Eric H. wrote:

> Why don’t you use the google search API?

Seems like a good idea but I have hassles installing the dependency  
ruby-SOAP4R rpm . .

## Phil.

Philip R.

GPO Box 3411  
Sydney NSW 2001  
Australia  
E-mail: removed\_email\_address@domain.invalid

---

<div class="post-metadata">

**Author:** ![unebaguettesvp](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/47e85d/32.png) [@unebaguettesvp](https://www.ruby-forum.com/u/unebaguettesvp)\
**Post date:** [August 18, 2009, 11:02pm UTC](https://www.ruby-forum.com/t/q-re-nokogiri-searching-google-and-counting-web-pages-with-specific-text/173964/4 "2009-08-18T23:02:10Z")

</div>

On Tue, Aug 18, 2009 at 3:31 PM, Philip R.[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
wrote:

> > >
> 
> ruby-SOAP4R rpm . .

The google search API does not use SOAP anymore, that’s been  
deprecated for a while. You can use this:

> **[GitHub - visionmedia/google-search: Ruby - Google Search API](https://github.com/visionmedia/google-search/tree/master)**
>
> master

Or just use httparty or open-uri and parse the json that’s returned  
from the new search API.

Cheers,  
Jason

---

<div class="post-metadata">

**Author:** ![unebaguettesvp](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/u/47e85d/32.png) [@unebaguettesvp](https://www.ruby-forum.com/u/unebaguettesvp)\
**Post date:** [August 18, 2009, 11:02pm UTC](https://www.ruby-forum.com/t/q-re-nokogiri-searching-google-and-counting-web-pages-with-specific-text/173964/5 "2009-08-18T23:02:46Z")

</div>

On Aug 18, 2009, at 12:31, Philip R. wrote:

> > > that contain specific text eg “george harrison” or “oxfam” etc.
> > > 
> > > Could someone get me started?
> > 
> > Why don’t you use the google search API?
> 
> Seems like a good idea but I have hassles installing the dependency  
> ruby-SOAP4R rpm . .

Looks like the SOAP API isn’t being maintained anymore, use the modern  
one:

> **[Developer Products and Tools - Google for Developers](https://developers.google.com/products/?csw=1)**
>
> Try the Google developer products and tools to build softwares, apps, and more. Explore our solutions for development, cloud, AI, and business growth.