# Hpricot parsing

**URL:** <https://www.ruby-forum.com/t/hpricot-parsing/165911>\
**Category:** Ruby\
**Created:** [April 19, 2009, 6:12pm UTC](https://www.ruby-forum.com/t/hpricot-parsing/165911 "2009-04-19T18:12:20Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![mrcfab3](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/9fc29f/32.png) [@mrcfab3](https://www.ruby-forum.com/u/mrcfab3)\
**Post date:** [April 19, 2009, 6:12pm UTC](https://www.ruby-forum.com/t/hpricot-parsing/165911/1 "2009-04-19T18:12:20Z")

</div>

Ruby newbie here

Have successfully used hpricot to scrape correct

 from desired page  
[http://www.montgomeryadvertiser.com/section/obits](http://www.montgomeryadvertiser.com/section/obits) using

doc = Hpricot(uri above)  
…  
@grab1 = doc.search(“//div[@class=‘article-bodytext’]”)

target data is in following logical form

### name of funeral home

deceased1

advertising crap

### funeral home 2

deceased 2

deceased 3

I’m struggling to iterate thru this div, plucking a array or hash where  
I can feed a database with each record being a funeral home and person.  
I was thinking I could go thru each of the @grab1 elements and process  
according to tag type and establish the “record” logic thru simple  
knowing that a new record starts with each new h3 tag.

Any help for a newbie with first Ruby script?

Thx

---

<div class="post-metadata">

**Author:** ![mrcfab3](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/9fc29f/32.png) [@mrcfab3](https://www.ruby-forum.com/u/mrcfab3)\
**Post date:** [April 20, 2009, 1:20am UTC](https://www.ruby-forum.com/t/hpricot-parsing/165911/2 "2009-04-20T01:20:41Z")

</div>

Marc F. wrote:

> Ruby newbie here
> 
> Have successfully used hpricot to scrape correct
> 
> from desired page  
> [http://www.montgomeryadvertiser.com/section/obits](http://www.montgomeryadvertiser.com/section/obits) using
> 
> doc = Hpricot(uri above)  
> …  
> @grab1 = doc.search(“//div[@class=‘article-bodytext’]”)
> 
> target data is in following logical form
> 
> ### name of funeral home
> 
> deceased1
> 
> advertising crap
> 
> ### funeral home 2
> 
> deceased 2
> 
> deceased 3
> 
> I’m struggling to iterate thru this div…  
> I [want to insert a record into a table with each] record being a funeral home and person.  
> I was thinking I could go thru each of the @grab1 elements and process  
> according to tag type:

These methods seem like the ones you need:

elm.next\_sibling (skips the newlines in the html)  
elm.name

How about this:

require “rubygems”  
require ‘hpricot’

str =\<\<ENDOFSTRING

### name of funeral home

deceased1

advertising crap

### funeral home 2

deceased 2

deceased 3

ENDOFSTRING

doc = Hpricot(str)  
h3\_tags = doc.search(“h3”)

h3\_tags.each do |h3|  
elm = h3

while elm = elm.next\_sibling  
break if elm.name != ‘p’

```
puts h3.inner_text
puts "\t #{elm.inner_text}"

```

end

end

–output:–  
name of funeral home  
deceased1  
funeral home 2  
deceased 2  
funeral home 2  
deceased 3

---

<div class="post-metadata">

**Author:** ![mrcfab3](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/9fc29f/32.png) [@mrcfab3](https://www.ruby-forum.com/u/mrcfab3)\
**Post date:** [April 20, 2009, 1:40am UTC](https://www.ruby-forum.com/t/hpricot-parsing/165911/3 "2009-04-20T01:40:33Z")

</div>

7stud – wrote:

> h3\_tags.each do |h3|  
> elm = h3
> 
> while elm = elm.next\_sibling  
> break if elm.name != ‘p’
> 
> ```
> puts h3.inner_text
> puts "\t #{elm.inner_text}"
> 
> ```
> 
> end
> 
> end

To avoid having to lookup the inner\_text of the funeral home for each  
deceased person at that funeral home, this would be more efficient:

h3\_tags.each do |elm|  
funeral\_home = elm.inner\_text

while elm = elm.next\_sibling  
break if elm.name != ‘p’

```
puts funeral_home
puts "\t #{elm.inner_text}"

```

end  
end

---

<div class="post-metadata">

**Author:** ![mrcfab3](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/9fc29f/32.png) [@mrcfab3](https://www.ruby-forum.com/u/mrcfab3)\
**Post date:** [April 20, 2009, 1:49am UTC](https://www.ruby-forum.com/t/hpricot-parsing/165911/4 "2009-04-20T01:49:45Z")

</div>

Thanks so much 7-stud

I had been fixated on next\_child thinking that next\_sibling would skip  
over the “p” tags. I really appreciate your thoughtfulness to provide a  
working code snippet.

Marc

---

<div class="post-metadata">

**Author:** ![mrcfab3](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/9fc29f/32.png) [@mrcfab3](https://www.ruby-forum.com/u/mrcfab3)\
**Post date:** [April 20, 2009, 4:05am UTC](https://www.ruby-forum.com/t/hpricot-parsing/165911/5 "2009-04-20T04:05:24Z")

</div>

Makes me wonder if ReXML, Hpricot or Nokogiri has a to\_hash method…not  
yet  
found.  
I’d also be glad to know.  
2009/4/20 Marc F. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)

---

<div class="post-metadata">

**Author:** ![mrcfab3](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/9fc29f/32.png) [@mrcfab3](https://www.ruby-forum.com/u/mrcfab3)\
**Post date:** [April 20, 2009, 4:25am UTC](https://www.ruby-forum.com/t/hpricot-parsing/165911/6 "2009-04-20T04:25:51Z")

</div>

Wang J. wrote:

> Makes me wonder if ReXML, Hpricot or Nokogiri has a to\_hash method…not yet  
> found.

Try to write it. I hope I’m wrong, but I suspect that starting will be  
easy, and  
hitting your own target XML will be easy…

…but making it generic enough to publish will be another story!