# Yet another Hpricot question

**URL:** <https://www.ruby-forum.com/t/yet-another-hpricot-question/71826>\
**Category:** Ruby\
**Created:** [October 11, 2006, 7:13pm UTC](https://www.ruby-forum.com/t/yet-another-hpricot-question/71826 "2006-10-11T19:13:28Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![rykkers](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e5b9ba/32.png) [@rykkers](https://www.ruby-forum.com/u/rykkers)\
**Post date:** [October 11, 2006, 7:13pm UTC](https://www.ruby-forum.com/t/yet-another-hpricot-question/71826/1 "2006-10-11T19:13:28Z")

</div>

I’m trying to scan an html file using Hpricot to produce a table of  
links within the file.

Right now I’ve got something like this.

doc = Hpricot(open(url)).  
doc.search(‘a’).each do | element |  
puts "#{element.inner\_html}  
puts " #{element.attributes[‘href’]  
end

This works, but in this document some of the a tags use markup on  
their contents. Something like  
[**blah blah** blah](http://blah.org/blah.htm)

I’d like to strip out the markup tags so that I’d get

blah blah blah  
[http://blah.org/blah.htm](http://blah.org/blah.htm)

Is there some way to search for or iterate over the leaf elements of  
the tree rooted by an element in Hpricot?

–  
Rick DeNatale

My blog on Ruby  
[http://talklikeaduck.denhaven2.com/](http://talklikeaduck.denhaven2.com/)

---

<div class="post-metadata">

**Author:** ![rykkers](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e5b9ba/32.png) [@rykkers](https://www.ruby-forum.com/u/rykkers)\
**Post date:** [October 11, 2006, 7:30pm UTC](https://www.ruby-forum.com/t/yet-another-hpricot-question/71826/2 "2006-10-11T19:30:01Z")

</div>

On Thu, Oct 12, 2006 at 02:13:07AM +0900, Rick DeNatale wrote:

> the tree rooted by an element in Hpricot?  
> I had to do something similar in Mechanize, and this is what I came up  
> with:

class Hpricot::Elem  
def all\_text  
text = ‘’  
children.each do |child|  
if child.respond\_to? :content  
text \<\< child.content  
end  
if child.respond\_to? :all\_text  
text \<\< child.all\_text  
end  
end  
text  
end  
end

doc = Hpricot("\<a href=“http://blah.org/blah.htm”\> **blah blah**  
blah")  
doc.search(‘a’).each do |e|  
puts “#{e.all\_text}”  
puts " #{e.attributes[‘href’]}"  
end

Hope that helps!

–Aaron

---

<div class="post-metadata">

**Author:** ![rykkers](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e5b9ba/32.png) [@rykkers](https://www.ruby-forum.com/u/rykkers)\
**Post date:** [October 11, 2006, 7:50pm UTC](https://www.ruby-forum.com/t/yet-another-hpricot-question/71826/3 "2006-10-11T19:50:09Z")

</div>

On Thu, Oct 12, 2006 at 02:28:13AM +0900, Aaron P. wrote:  
} On Thu, Oct 12, 2006 at 02:13:07AM +0900, Rick DeNatale wrote:  
} \> I’m trying to scan an html file using Hpricot to produce a table of  
} \> links within the file.  
} \>  
} \> Right now I’ve got something like this.  
} \>  
} \> doc = Hpricot(open(url)).  
} \> doc.search(‘a’).each do | element |  
} \> puts “#{element.inner\_html}  
} \> puts " #{element.attributes[‘href’]  
} \> end  
} \>  
} \> This works, but in this document some of the a tags use markup on  
} \> their contents. Something like  
} \> [**blah blah** blah](http://blah.org/blah.htm)  
} \>  
} \> I’d like to strip out the markup tags so that I’d get  
} \>  
} \> blah blah blah  
} \> [http://blah.org/blah.htm](http://blah.org/blah.htm)  
} \>  
} \> Is there some way to search for or iterate over the leaf elements of  
} \> the tree rooted by an element in Hpricot?  
}  
} I had to do something similar in Mechanize, and this is what I came up  
} with:  
}  
} class Hpricot::Elem  
} def all\_text  
} text = ‘’  
} children.each do |child|  
} if child.respond\_to? :content  
} text \<\< child.content  
} end  
} if child.respond\_to? :all\_text  
} text \<\< child.all\_text  
} end  
} end  
} text  
} end  
} end  
}  
} doc = Hpricot(”\<a href="[http://blah.org/blah.htm\](http://blah.org/blah.htm%5C)“\> **blah blah**  
blah”)  
} doc.search(‘a’).each do |e|  
} puts “#{e.all\_text}”  
} puts " #{e.attributes[‘href’]}"  
} end

There is a simpler implementation of all\_text:

class Hpricot::Elem  
def all\_text  
text = ‘’  
traverse\_text {|t| text \<\< t.content }  
text  
end  
end

} Hope that helps!  
} --Aaron  
–Greg

---

<div class="post-metadata">

**Author:** ![rykkers](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e5b9ba/32.png) [@rykkers](https://www.ruby-forum.com/u/rykkers)\
**Post date:** [October 18, 2006, 10:38am UTC](https://www.ruby-forum.com/t/yet-another-hpricot-question/71826/4 "2006-10-18T10:38:55Z")

</div>

On Thu, Oct 12, 2006 at 10:29:54PM +0900, Rick DeNatale wrote:

> Thanks Aaron and Greg, works a treat!  
> If three of you have independantly used this, let’s check it in.  
> Elements#text  
> which parallels Elements#html (I think jQuery also has this) and  
> Elem#inner\_text  
> as well.

\_why

---

<div class="post-metadata">

**Author:** ![rykkers](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e5b9ba/32.png) [@rykkers](https://www.ruby-forum.com/u/rykkers)\
**Post date:** [October 12, 2006, 3:30pm UTC](https://www.ruby-forum.com/t/yet-another-hpricot-question/71826/5 "2006-10-12T15:30:06Z")

</div>

On 10/11/06, Gregory S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> } \> puts " #{element.attributes[‘href’]  
> } \>  
> } if child.respond\_to? :content  
> } doc = Hpricot(“\<a href="[http://blah.org/blah.htm\](http://blah.org/blah.htm%5C)”\> **blah blah** blah")  
> traverse\_text {|t| text \<\< t.content }  
> text  
> end  
> end
> 
> } Hope that helps!  
> } --Aaron  
> –Greg

Thanks Aaron and Greg, works a treat!

–  
Rick DeNatale

My blog on Ruby  
[http://talklikeaduck.denhaven2.com/](http://talklikeaduck.denhaven2.com/)

---

<div class="post-metadata">

**Author:** ![rykkers](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/r/e5b9ba/32.png) [@rykkers](https://www.ruby-forum.com/u/rykkers)\
**Post date:** [October 18, 2006, 10:39am UTC](https://www.ruby-forum.com/t/yet-another-hpricot-question/71826/6 "2006-10-18T10:39:34Z")

</div>

On Fri, Oct 13, 2006 at 02:38:10AM +0900, why the lucky stiff wrote:  
} On Thu, Oct 12, 2006 at 10:29:54PM +0900, Rick DeNatale wrote:  
} \> \>There is a simpler implementation of all\_text:  
} \> \>  
} \> \>class Hpricot::Elem  
} \> \> def all\_text  
} \> \> text = ‘’  
} \> \> traverse\_text {|t| text \<\< t.content }  
} \> \> text  
} \> \> end  
} \> \>end  
} \>  
} \> Thanks Aaron and Greg, works a treat!  
}  
} If three of you have independantly used this, let’s check it in.  
} Elements#text which parallels Elements#html (I think jQuery also has  
} this) and Elem#inner\_text as well.

Actually, I haven’t used it. I just knew how to go about it. What I  
would  
find much more useful is an #inject\_text (and a corresponding  
#inject\_elements, though that isn’t as important since the / notation  
retrieves an Enumerable).

} \_why  
–Greg