# Bypassing XML inconsistencies with REXML::StreamListener

**URL:** <https://www.ruby-forum.com/t/bypassing-xml-inconsistencies-with-rexml-streamlistener/114759>\
**Category:** Ruby\
**Created:** [September 17, 2007, 1:25pm UTC](https://www.ruby-forum.com/t/bypassing-xml-inconsistencies-with-rexml-streamlistener/114759 "2007-09-17T13:25:14Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![nutsmuggler](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/57b2e6/32.png) [@nutsmuggler](https://www.ruby-forum.com/u/nutsmuggler)\
**Post date:** [September 17, 2007, 1:25pm UTC](https://www.ruby-forum.com/t/bypassing-xml-inconsistencies-with-rexml-streamlistener/114759/1 "2007-09-17T13:25:14Z")

</div>

Hello folks.  
I am trying to build a simple XML parser to extract data from IBM  
translation manager memories. Here is a sample os such memory files:

0000000001 00012200000001178876638English(U.S.)ITALIANIBMIDDOCBB1CTMST. 000BB1CTmst.idd 0000000002 00000300000001178876638English(U.S.)ITALIANIBMIDDOCCONFIGUR. 000Configuration\_PDSG.IDE Configuration information and guidelines Informazioni e istruzioni per la configurazione etc...

These memory files are quite similar to XML files, but I suspect they  
actually conform to another standard. In fact, they often include  
“opened” tags; these because they store segments of translation; thus,  
when the translation is referred to a website or a SGML document, the  
original HTML or SGML might be split in two or more parts. So I often  
encounter faulty segments; open tags generate a REXML fault.  
My code is quite simple :

require ‘rexml/document’  
require ‘rexml/streamlistener’  
include REXML

class Listener  
include StreamListener  
$segment = “”  
$result = “”  
$is\_there = false  
def tag\_start(name, attributes)  
if name == “Source”  
$segment \<\< “EN:”  
end  
if name == “Target”  
$segment \<\< “IT:”  
end  
end  
def tag\_end(name)  
if name == “Target”  
if $is\_there  
$result \<\< $segment  
end  
$segment = “”  
$is\_there = false  
end  
if name == “NTMMemoryDb”  
puts $result  
end  
end  
def text(text)  
$segment \<\< text  
if text =~ /blade/  
$is\_there = true  
end  
end  
end

listener = Listener.new  
parser =  
Parsers::StreamParser.new(File.new(“bch01aad006\_MEMORIA.EXP”),  
listener)  
parser.parse

I need to bypass mistakes, and tell StreamListener: “when you  
encounter a faulty segment, don’t bother!”  
How do I achieve this?  
Thanks in advance,  
Davide

---

<div class="post-metadata">

**Author:** ![nutsmuggler](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/57b2e6/32.png) [@nutsmuggler](https://www.ruby-forum.com/u/nutsmuggler)\
**Post date:** [September 25, 2007, 11:03pm UTC](https://www.ruby-forum.com/t/bypassing-xml-inconsistencies-with-rexml-streamlistener/114759/2 "2007-09-25T23:03:10Z")

</div>

nutsmuggler wrote:

> 00012200000001178876638English(U.S.)ITALIANIBMIDDOCBB1CTMST.
> 
> These memory files are quite similar to XML files, but I suspect they  
> actually conform to another standard. In fact, they often include  
> “opened” tags; these because they store segments of translation; thus,  
> when the translation is referred to a website or a SGML document, the  
> original HTML or SGML might be split in two or more parts. So I often  
> encounter faulty segments; open tags generate a REXML fault.  
>   
> It might be worth trying HTML Tidy in XML mode. I can’t remember off  
> the top of my head how it’ll react to missing close tags, but it’s worth  
> a shot…

---

<div class="post-metadata">

**Author:** ![nutsmuggler](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/57b2e6/32.png) [@nutsmuggler](https://www.ruby-forum.com/u/nutsmuggler)\
**Post date:** [September 25, 2007, 11:09pm UTC](https://www.ruby-forum.com/t/bypassing-xml-inconsistencies-with-rexml-streamlistener/114759/3 "2007-09-25T23:09:12Z")

</div>

nutsmuggler wrote:

> Hello folks.  
> I am trying to build a simple XML parser to extract data from IBM  
> translation manager memories. Here is a sample os such memory files:  
> …
> 
> I need to bypass mistakes, and tell StreamListener: “when you  
> encounter a faulty segment, don’t bother!”  
> How do I achieve this?

Don’t use an XML parser to handle non-XML?

Alternatively, have you tried the REXML pull parser? A bit more work in  
that you have to explicitly pop items off the tag stack, but it may have  
better options for recovering from bad markup.

However, the underlying parser may still barf in trying to segment the  
source into tags and such.

Also, I don’t know if Hpricot is happy with non-HTML, but it’s worth a  
shot to see if it can read and “fix” the source before you pass it to  
another parser. You’ll want to check that any modification made to the  
input do not change the essential semantics.

(Or perhaps you could just use Hpricot and extract data with XPath)

–  
James B.

“In Ruby, no one cares who your parents were, all they care  
about is if you know what you are talking about.”

- Logan C.

---

<div class="post-metadata">

**Author:** ![nutsmuggler](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/n/57b2e6/32.png) [@nutsmuggler](https://www.ruby-forum.com/u/nutsmuggler)\
**Post date:** [September 25, 2007, 11:09pm UTC](https://www.ruby-forum.com/t/bypassing-xml-inconsistencies-with-rexml-streamlistener/114759/4 "2007-09-25T23:09:52Z")

</div>

hpricot is my man 🙂  
Being an HTML parser, it’s much less hard to please.  
Here is the basic code I am using:

require ‘rubygems’  
require ‘hpricot’

doc = Hpricot.XML(open(“bch01aad006\_MEMORIA.EXP”))  
doc.search(“Source”).each do |item|  
if item.innerHTML =~ /firmware/  
puts “EN: #{item}”  
puts “IT: #{item.next\_sibling}”  
end  
end

The principle it’s quite easy, and much more coincise than the rexml  
solution.  
Thanks a million for the tip.  
Davide