Yet another Hpricot question

I’m trying to scan an html file using Hpricot to produce a table of
links within the file.

Right now I’ve got something like this.

doc = Hpricot(open(url)).
doc.search(‘a’).each do | element |
puts "#{element.inner_html}
puts " #{element.attributes[‘href’]
end

This works, but in this document some of the a tags use markup on
their contents. Something like
blah blah blah

I’d like to strip out the markup tags so that I’d get

blah blah blah
http://blah.org/blah.htm

Is there some way to search for or iterate over the leaf elements of
the tree rooted by an element in Hpricot?


Rick DeNatale

My blog on Ruby
http://talklikeaduck.denhaven2.com/

On Thu, Oct 12, 2006 at 02:13:07AM +0900, Rick DeNatale wrote:

the tree rooted by an element in Hpricot?
I had to do something similar in Mechanize, and this is what I came up
with:

class Hpricot::Elem
def all_text
text = ‘’
children.each do |child|
if child.respond_to? :content
text << child.content
end
if child.respond_to? :all_text
text << child.all_text
end
end
text
end
end

doc = Hpricot("<a href=“http://blah.org/blah.htm”>blah blah
blah")
doc.search(‘a’).each do |e|
puts “#{e.all_text}”
puts " #{e.attributes[‘href’]}"
end

Hope that helps!

–Aaron

On Thu, Oct 12, 2006 at 02:28:13AM +0900, Aaron P. wrote:
} On Thu, Oct 12, 2006 at 02:13:07AM +0900, Rick DeNatale wrote:
} > I’m trying to scan an html file using Hpricot to produce a table of
} > links within the file.
} >
} > Right now I’ve got something like this.
} >
} > doc = Hpricot(open(url)).
} > doc.search(‘a’).each do | element |
} > puts “#{element.inner_html}
} > puts " #{element.attributes[‘href’]
} > end
} >
} > This works, but in this document some of the a tags use markup on
} > their contents. Something like
} > blah blah blah
} >
} > I’d like to strip out the markup tags so that I’d get
} >
} > blah blah blah
} > http://blah.org/blah.htm
} >
} > Is there some way to search for or iterate over the leaf elements of
} > the tree rooted by an element in Hpricot?
}
} I had to do something similar in Mechanize, and this is what I came up
} with:
}
} class Hpricot::Elem
} def all_text
} text = ‘’
} children.each do |child|
} if child.respond_to? :content
} text << child.content
} end
} if child.respond_to? :all_text
} text << child.all_text
} end
} end
} text
} end
} end
}
} doc = Hpricot(”<a href=“http://blah.org/blah.htm”>blah blah
blah")
} doc.search(‘a’).each do |e|
} puts “#{e.all_text}”
} puts " #{e.attributes[‘href’]}"
} end

There is a simpler implementation of all_text:

class Hpricot::Elem
def all_text
text = ‘’
traverse_text {|t| text << t.content }
text
end
end

} Hope that helps!
} --Aaron
–Greg

On Thu, Oct 12, 2006 at 10:29:54PM +0900, Rick DeNatale wrote:

Thanks Aaron and Greg, works a treat!
If three of you have independantly used this, let’s check it in.
Elements#text
which parallels Elements#html (I think jQuery also has this) and
Elem#inner_text
as well.

_why

On 10/11/06, Gregory S. [email protected] wrote:

} > puts " #{element.attributes[‘href’]
} >
} if child.respond_to? :content
} doc = Hpricot("<a href=“http://blah.org/blah.htm”>blah blah blah")
traverse_text {|t| text << t.content }
text
end
end

} Hope that helps!
} --Aaron
–Greg

Thanks Aaron and Greg, works a treat!


Rick DeNatale

My blog on Ruby
http://talklikeaduck.denhaven2.com/

On Fri, Oct 13, 2006 at 02:38:10AM +0900, why the lucky stiff wrote:
} On Thu, Oct 12, 2006 at 10:29:54PM +0900, Rick DeNatale wrote:
} > >There is a simpler implementation of all_text:
} > >
} > >class Hpricot::Elem
} > > def all_text
} > > text = ‘’
} > > traverse_text {|t| text << t.content }
} > > text
} > > end
} > >end
} >
} > Thanks Aaron and Greg, works a treat!
}
} If three of you have independantly used this, let’s check it in.
} Elements#text which parallels Elements#html (I think jQuery also has
} this) and Elem#inner_text as well.

Actually, I haven’t used it. I just knew how to go about it. What I
would
find much more useful is an #inject_text (and a corresponding
#inject_elements, though that isn’t as important since the / notation
retrieves an Enumerable).

} _why
–Greg

This forum is not affiliated to the Ruby language, Ruby on Rails framework, nor any Ruby applications discussed here.

| Privacy Policy | Terms of Service | Remote Ruby Jobs