# REXML UTF-16 trouble

**URL:** <https://www.ruby-forum.com/t/rexml-utf-16-trouble/145912>\
**Category:** Rails\
**Created:** [August 14, 2008, 11:31pm UTC](https://www.ruby-forum.com/t/rexml-utf-16-trouble/145912 "2008-08-14T23:31:54Z")\
**Posts on this page:** 7\
**Page:** 1

<div class="post-metadata">

**Author:** ![iga](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/e19b73/32.png) [@iga](https://www.ruby-forum.com/u/iga)\
**Post date:** [August 14, 2008, 11:31pm UTC](https://www.ruby-forum.com/t/rexml-utf-16-trouble/145912/1 "2008-08-14T23:31:54Z")

</div>

I’ve run into a problem parsing XML with REXML, and it looks like the  
problem has to do with UTF-16 encoding and a bug with REXML.

I’m still running OS X 10.4.11, Ruby 1.8.6 (using Locomotive). I even  
tried upgrading to the latest version of REXML 3.1.7.3 - no luck. It  
still gives errors, only now it’s saying the following.

Iconv::InvalidCharacter: “\>”

Is this old news to everyone? If so, is there known solution for  
this?

I can’t be the only person who needs to parse UTF-16 xml in Rails.

Thanks in advance.

---

<div class="post-metadata">

**Author:** ![iga](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/e19b73/32.png) [@iga](https://www.ruby-forum.com/u/iga)\
**Post date:** [August 14, 2008, 11:59pm UTC](https://www.ruby-forum.com/t/rexml-utf-16-trouble/145912/2 "2008-08-14T23:59:25Z")

</div>

Random thought - the XML file claims to be utf 16, but is it really?

Sent from my iPhone

---

<div class="post-metadata">

**Author:** ![iga](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/e19b73/32.png) [@iga](https://www.ruby-forum.com/u/iga)\
**Post date:** [August 15, 2008, 12:10am UTC](https://www.ruby-forum.com/t/rexml-utf-16-trouble/145912/3 "2008-08-15T00:10:32Z")

</div>

Good question. How would I know?

If I stash the result straight into a variable and do a “puts” I get  
the following…

\<?xml version="1.0" encoding="utf-16"?\>

`InvalidPassword`  
false  
Invalid password

Again, REXML chokes on this, but if I change utf-16 to utf-8, no  
problem.

On Aug 14, 4:58 pm, Frederick C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)

---

<div class="post-metadata">

**Author:** ![iga](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/e19b73/32.png) [@iga](https://www.ruby-forum.com/u/iga)\
**Post date:** [August 15, 2008, 5:30am UTC](https://www.ruby-forum.com/t/rexml-utf-16-trouble/145912/4 "2008-08-15T05:30:29Z")

</div>

I have an interim Kludge for now. I’m chopping the BOM piece off of  
the XML and then sending it through as XML via REXML. It’s working  
like a charm now. It’s not pretty, but it’ll do until I figure out if  
this is part of a bigger problem.

# Since this part of the string is always going to be same, slice the

first 39 characters

# Off of it.

bom\_string\_to\_remove = login\_result.slice(0, 39)  
login\_results = login\_result.gsub(bom\_string\_to\_remove,‘’)

So this…

\<?xml version="1.0" encoding="utf-16"?\>

`InvalidPassword`  
false  
Invalid password

Becomes this…  
`InvalidPassword`  
false  
Invalid password

And, now I treat it as XML.

On Aug 14, 4:58 pm, Frederick C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)

---

<div class="post-metadata">

**Author:** ![iga](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/e19b73/32.png) [@iga](https://www.ruby-forum.com/u/iga)\
**Post date:** [August 15, 2008, 3:55pm UTC](https://www.ruby-forum.com/t/rexml-utf-16-trouble/145912/5 "2008-08-15T15:55:42Z")

</div>

Fred,

Thanks, I will give that a try – eventually, I need this to work  
without the workaround.

+bm

On Aug 15, 4:35 am, Frederick C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)

---

<div class="post-metadata">

**Author:** ![iga](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/e19b73/32.png) [@iga](https://www.ruby-forum.com/u/iga)\
**Post date:** [November 8, 2010, 4:01pm UTC](https://www.ruby-forum.com/t/rexml-utf-16-trouble/145912/6 "2010-11-08T16:01:03Z")

</div>

Was this ever solved? How do I load a UTF-16 xml file into REXML?

Thanks!  
Naren

---

<div class="post-metadata">

**Author:** ![iga](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/e19b73/32.png) [@iga](https://www.ruby-forum.com/u/iga)\
**Post date:** [August 15, 2008, 11:36am UTC](https://www.ruby-forum.com/t/rexml-utf-16-trouble/145912/7 "2008-08-15T11:36:27Z")

</div>

On 14 Aug 2008, at 23:09, iga wrote:

> Good question. How would I know?

I’d ogle the bytes via unpack or something like that.

Fred