# Hpricot and regexp?

**URL:** <https://www.ruby-forum.com/t/hpricot-and-regexp/137567>\
**Category:** Ruby\
**Created:** [May 14, 2008, 7:43am UTC](https://www.ruby-forum.com/t/hpricot-and-regexp/137567 "2008-05-14T07:43:25Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![pood](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/3bc359/32.png) [@pood](https://www.ruby-forum.com/u/pood)\
**Post date:** [May 14, 2008, 7:43am UTC](https://www.ruby-forum.com/t/hpricot-and-regexp/137567/1 "2008-05-14T07:43:25Z")

</div>

I’m trying to grab the “cache date” off of the google search.

using Mechanize (and built in hpricot)

agent = WWW::Mechanize.new  
agent.user\_agent\_alias = ‘Mac Safari’  
page = agent.get(“[http://www.google.com/](http://www.google.com/)”)  
search\_form = page.forms.with.name(“f”).first  
search\_form.q = “Hello”  
search\_results = agent.submit(search\_form)  
cache\_date = agent.click search\_results.links.text(‘Cached’)

date = cache\_date.search(‘table table \> td’).inner\_html

How do i grab the date like on this page:  
[http://209.85.173.104/search?q=cache%3Ashacknews.com&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a](http://209.85.173.104/search?q=cache%3Ashacknews.com&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a)

the part that’s right after “as retrieved on” (the date)  
Is there a built in hpricot method that can search by rexep?  
or will I have to use something like gsub?

---

<div class="post-metadata">

**Author:** ![pood](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/3bc359/32.png) [@pood](https://www.ruby-forum.com/u/pood)\
**Post date:** [May 14, 2008, 7:46am UTC](https://www.ruby-forum.com/t/hpricot-and-regexp/137567/2 "2008-05-14T07:46:57Z")

</div>

Feng T. wrote:

> I’m trying to grab the “cache date” off of the google search.
> 
> using Mechanize (and built in hpricot)
> 
> agent = WWW::Mechanize.new  
> agent.user\_agent\_alias = ‘Mac Safari’  
> page = agent.get(“[http://www.google.com/](http://www.google.com/)”)  
> search\_form = page.forms.with.name(“f”).first  
> search\_form.q = “Hello”  
> search\_results = agent.submit(search\_form)  
> cache\_date = agent.click search\_results.links.text(‘Cached’)
> 
> date = cache\_date.search(‘table table \> td’).inner\_html
> 
> How do i grab the date like on this page:  
> [http://209.85.173.104/search?q=cache%3Ashacknews.com&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a](http://209.85.173.104/search?q=cache%3Ashacknews.com&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a)
> 
> the part that’s right after “as retrieved on” (the date)  
> Is there a built in hpricot method that can search by rexep?  
> or will I have to use something like gsub?

oops, I mean, grep.

oh, i got it down to this:

date = cache\_date.search(‘table table \> td’).inner\_text.grep(/retrieved  
on (.+)./)

which outputs:[“This is G o o g l e’s cache of [http://www.hello.com/](http://www.hello.com/) as  
retrieved on May 11, 2008 01:09:29 GMT.\n”]

How do I get rid of everything before the date?

---

<div class="post-metadata">

**Author:** ![pood](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/3bc359/32.png) [@pood](https://www.ruby-forum.com/u/pood)\
**Post date:** [May 14, 2008, 8:01am UTC](https://www.ruby-forum.com/t/hpricot-and-regexp/137567/3 "2008-05-14T08:01:54Z")

</div>

Feng T. wrote:

> Feng T. wrote:
> 
> > I’m trying to grab the “cache date” off of the google search.
> > 
> > using Mechanize (and built in hpricot)
> > 
> > agent = WWW::Mechanize.new  
> > agent.user\_agent\_alias = ‘Mac Safari’  
> > page = agent.get(“[http://www.google.com/](http://www.google.com/)”)  
> > search\_form = page.forms.with.name(“f”).first  
> > search\_form.q = “Hello”  
> > search\_results = agent.submit(search\_form)  
> > cache\_date = agent.click search\_results.links.text(‘Cached’)
> > 
> > date = cache\_date.search(‘table table \> td’).inner\_html
> > 
> > How do i grab the date like on this page:  
> > [http://209.85.173.104/search?q=cache%3Ashacknews.com&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a](http://209.85.173.104/search?q=cache%3Ashacknews.com&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a)
> > 
> > the part that’s right after “as retrieved on” (the date)  
> > Is there a built in hpricot method that can search by rexep?  
> > or will I have to use something like gsub?
> 
> oops, I mean, grep.
> 
> oh, i got it down to this:
> 
> date = cache\_date.search(‘table table \> td’).inner\_text.grep(/retrieved  
> on (.+)./)
> 
> which outputs:[“This is G o o g l e’s cache of [http://www.hello.com/](http://www.hello.com/) as  
> retrieved on May 11, 2008 01:09:29 GMT.\n”]
> 
> How do I get rid of everything before the date?

Now I have this:

date = cache\_date.search(‘table table \> td’).inner\_text.grep(/retrieved  
on (.+)./).to\_s.gsub(/.+as retrieved on /,“”).gsub(/.\n/,“”)

which gives me exactly what i need. is there a better way to doing this?

---

<div class="post-metadata">

**Author:** ![pood](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/3bc359/32.png) [@pood](https://www.ruby-forum.com/u/pood)\
**Post date:** [May 14, 2008, 8:54am UTC](https://www.ruby-forum.com/t/hpricot-and-regexp/137567/4 "2008-05-14T08:54:30Z")

</div>

Google cached pages have this structure:

* * *

where the first

 contains boilerplate cache text and a copy of  
the page is in the 
.

This is what I would use to clip out the date:

url=“[http://64.233.167.104/search?q=cache:hydO8fs-rmQJ:en.wikipedia.org/wiki/Court-martial+court+martial&hl=en&ct=clnk&cd=1&gl=us&client=firefox-a](http://64.233.167.104/search?q=cache:hydO8fs-rmQJ:en.wikipedia.org/wiki/Court-martial+court+martial&hl=en&ct=clnk&cd=1&gl=us&client=firefox-a)”  
doc = Hpricot(open(url))  
a=doc.search(“/table”).inner\_text  
a[/retrieved on (.\*?) GMT/,1]  
=\>May 13, 2008 11:37:34

Feng T. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote: Feng T. wrote:

> > search\_form.q = “Hello”  
> > Is there a built in hpricot method that can search by rexep?
> 
> which outputs:[“This is G o o g l e’s cache of [http://www.hello.com/](http://www.hello.com/) as  
> retrieved on May 11, 2008 01:09:29 GMT.\n”]
> 
> How do I get rid of everything before the date?

Now I have this:

date = cache\_date.search(‘table table \> td’).inner\_text.grep(/retrieved  
on (.+)./).to\_s.gsub(/.+as retrieved on /,“”).gsub(/.\n/,“”)

which gives me exactly what i need. is there a better way to doing this?

---

<div class="post-metadata">

**Author:** ![pood](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/p/3bc359/32.png) [@pood](https://www.ruby-forum.com/u/pood)\
**Post date:** [May 14, 2008, 6:08pm UTC](https://www.ruby-forum.com/t/hpricot-and-regexp/137567/5 "2008-05-14T18:08:07Z")

</div>

Dan D.,

wow, much shorter, thanks!