# REXML Input File Question **URL:** \ **Category:** Ruby\ **Created:** [July 19, 2010, 9:13pm UTC](https://www.ruby-forum.com/t/rexml-input-file-question/191854 "2010-07-19T21:13:38Z")\ **Posts on this page:** 8\ **Page:** 1
**Author:** ![mperham](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e47c2d/32.png) [@mperham](https://www.ruby-forum.com/u/mperham)\ **Post date:** [July 19, 2010, 9:13pm UTC](https://www.ruby-forum.com/t/rexml-input-file-question/191854/1 "2010-07-19T21:13:38Z")
Hi, So I am having issues parsing in a document using the Ruby XML parser REXML. The issue seems to be with the first line of my file that identifies the XML file. Here are two xml files, the first is not parsed with REXML and the second is parsed properly: error = \<\ EOF noerror = \<\ nil doc = Document.new noerror puts doc.root.attributes[“test”] --\> yes I considered the fact that REXML only takes in UTF-8 unicoded files, but when I convert these files from UTF-16 to UTF-8, it still does not parse properly. Does anyone know what I am doing wrong? Thank you very Much. Mike ---
**Author:** ![mperham](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e47c2d/32.png) [@mperham](https://www.ruby-forum.com/u/mperham)\ **Post date:** [July 19, 2010, 9:57pm UTC](https://www.ruby-forum.com/t/rexml-input-file-question/191854/2 "2010-07-19T21:57:45Z")
On 19.07.2010 21:13, Mike Pe wrote: > ``` > > > ``` > > EOF The string is most likely not UTF-16 encoded so REXML cannot parse it properly. Which Ruby version? If it is 1.9ish you’ll find information about i18n here: [http://blog.grayproductions.net/articles/understanding\_m17n](http://blog.grayproductions.net/articles/understanding_m17n) > puts doc.root.attributes[“test”] → nil > > doc = Document.new noerror > puts doc.root.attributes[“test”] → yes > > I considered the fact that REXML only takes in UTF-8 unicoded files, but > when I convert these files from UTF-16 to UTF-8, it still does not parse > properly. > > Does anyone know what I am doing wrong? Thank you very Much. Can you show what exactly you did? Kind regards robert ---
**Author:** ![mperham](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e47c2d/32.png) [@mperham](https://www.ruby-forum.com/u/mperham)\ **Post date:** [July 24, 2010, 12:09am UTC](https://www.ruby-forum.com/t/rexml-input-file-question/191854/3 "2010-07-24T00:09:38Z")
Robert K. wrote: > On 19.07.2010 21:13, Mike Pe wrote: > > > ``` > > > > > > ``` > > > > EOF > > The string is most likely not UTF-16 encoded so REXML cannot parse it > properly. Which Ruby version? If it is 1.9ish you’ll find information > about i18n here: > [http://blog.grayproductions.net/articles/understanding\_m17n](http://blog.grayproductions.net/articles/understanding_m17n) > > > puts doc.root.attributes[“test”] → nil > > > > doc = Document.new noerror > > puts doc.root.attributes[“test”] → yes > > > > I considered the fact that REXML only takes in UTF-8 unicoded files, but > > when I convert these files from UTF-16 to UTF-8, it still does not parse > > properly. > > > > Does anyone know what I am doing wrong? Thank you very Much. > > Can you show what exactly you did? > > Kind regards > > robert Hi Robert, The issue is that the first line of my input file: \ Causes the file to be read as an “xml application”. Basically, I just want to be able to use REXML to parse out this xml file, but it does not parse properly with this line in the beginning of my input file. (otherwise it works fine). I tried converting the files using iconv commands from your link, but it UTF-16 and UTF-8, the same error occurs, without regard for format. Why is this line interfering with the parser and how would I fix it? Thank you for your help. Best, Mike ---
**Author:** ![mperham](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e47c2d/32.png) [@mperham](https://www.ruby-forum.com/u/mperham)\ **Post date:** [July 27, 2010, 6:47pm UTC](https://www.ruby-forum.com/t/rexml-input-file-question/191854/4 "2010-07-27T18:47:19Z")
Robert K. wrote: > 2010/7/24 Mike Pe [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid): > > > > > puts doc.root.attributes[“test”] → �nil > > > > Can you show what exactly you did? > > > > The issue is that the first line of my input file: > > > > \ > > > > Causes the file to be read as an “xml application”. Basically, I just > > want to be able to use REXML to parse out this xml file, but it does not > > parse properly with this line in the beginning of my input file. > > (otherwise it works fine). > > Please provide the code you are using so others can try this out > themselves. I asked for this already (see above). > > > I tried converting the files using iconv commands from your link, but it > > UTF-16 and UTF-8, the same error occurs, without regard for format. > > > > Why is this line interfering with the parser and how would I fix it? > > Thank you for your help. > > It seems there is no UTF-16 support: > > irb(main):009:0\> f=File.open “x”, “r:UTF-16” > (irb):9: warning: Unsupported encoding UTF-16 ignored > =\> #\ > > So there is no point in trying to import a UTF-16 encoded file in Ruby. > > Kind regards > > robert Hi Robert, As for the code that I am using, I simplified the code in my original post. The first line: doc = REXML::Document.new error Should parse in the XML document and recognize all of the roots, elements, attributes, etc. from the input document. i.e.: puts doc.root.attributes[“test”] Should return “yes” because the attribute in the error xml file (see above) is "yes. With the extra line, it puts “nil”. (because the parser did not do its job). I tried converting all of the files to UTF-8 and they still did not work. (If you remove the extra line, it does work) I do not think the problem with is in the unicode. Thanks, Mike ---
**Author:** ![mperham](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e47c2d/32.png) [@mperham](https://www.ruby-forum.com/u/mperham)\ **Post date:** [July 27, 2010, 11:02am UTC](https://www.ruby-forum.com/t/rexml-input-file-question/191854/5 "2010-07-27T11:02:06Z")
2010/7/24 Mike Pe [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid): > > > puts doc.root.attributes[“test”] → nil > > > Can you show what exactly you did? > > > The issue is that the first line of my input file: > > \ > > Causes the file to be read as an “xml application”. Basically, I just > want to be able to use REXML to parse out this xml file, but it does not > parse properly with this line in the beginning of my input file. > (otherwise it works fine). Please provide the code you are using so others can try this out themselves. I asked for this already (see above). > I tried converting the files using iconv commands from your link, but it > UTF-16 and UTF-8, the same error occurs, without regard for format. > > Why is this line interfering with the parser and how would I fix it? > Thank you for your help. It seems there is no UTF-16 support: irb(main):009:0\> f=File.open “x”, “r:UTF-16” (irb):9: warning: Unsupported encoding UTF-16 ignored =\> #\ So there is no point in trying to import a UTF-16 encoded file in Ruby. Kind regards robert ---
**Author:** ![mperham](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e47c2d/32.png) [@mperham](https://www.ruby-forum.com/u/mperham)\ **Post date:** [July 28, 2010, 8:23am UTC](https://www.ruby-forum.com/t/rexml-input-file-question/191854/6 "2010-07-28T08:23:10Z")
Robert K. wrote: > On 27.07.2010 18:47, Mike Pe wrote: > > > > > parse properly with this line in the beginning of my input file. > > > > > > It seems there is no UTF-16 support: > > > > > > irb(main):009:0\> f=File.open “x”, “r:UTF-16” > > > (irb):9: warning: Unsupported encoding UTF-16 ignored > > > =\> #\ > > > > > > So there is no point in trying to import a UTF-16 encoded file in Ruby. > > > As for the code that I am using, I simplified the code in my original > > post. The first line: > > > > doc = REXML::Document.new error > > What is “error”? How do you obtain it? By “error”, I meant my file called error from my first post: error = \<\ EOF > > I tried converting all of the files to UTF-8 and they still did not > > work. (If you remove the extra line, it does work) I do not think the > > problem with is in the unicode. > > Hmm… > > robert ---
**Author:** ![mperham](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e47c2d/32.png) [@mperham](https://www.ruby-forum.com/u/mperham)\ **Post date:** [July 27, 2010, 11:20pm UTC](https://www.ruby-forum.com/t/rexml-input-file-question/191854/7 "2010-07-27T23:20:56Z")
On 27.07.2010 18:47, Mike Pe wrote: > > > parse properly with this line in the beginning of my input file. > > > > It seems there is no UTF-16 support: > > > > irb(main):009:0\> f=File.open “x”, “r:UTF-16” > > (irb):9: warning: Unsupported encoding UTF-16 ignored > > =\> #\ > > > > So there is no point in trying to import a UTF-16 encoded file in Ruby. > As for the code that I am using, I simplified the code in my original > post. The first line: > > doc = REXML::Document.new error What is “error”? How do you obtain it? > I tried converting all of the files to UTF-8 and they still did not > work. (If you remove the extra line, it does work) I do not think the > problem with is in the unicode. Hmm… robert ---
**Author:** ![mperham](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/m/e47c2d/32.png) [@mperham](https://www.ruby-forum.com/u/mperham)\ **Post date:** [July 28, 2010, 3:50pm UTC](https://www.ruby-forum.com/t/rexml-input-file-question/191854/8 "2010-07-28T15:50:16Z")
> > > > > Can you show what exactly you did? > > > > > > Please provide the code you are using so others can try this out > > > themselves. I asked for this already (see above). Could you provide a link to a zip file that contains an original input that fails, a re-encoded input file that fails, and an input file that does not fail and a script that loads them? Or, provide a more detailed step-by-step of what you did, e.g.: # poke at the original file to see what it looks like ls -l orig-utf16.xml file orig-utf16.xml wc -c orig-utf16.xml enca orig-utf16.xml head orig-utf16.xml # convert the file iconv -t UTF8 -f UTF16 \< orig-utf16.xml \> new-utf8.xml # poke at the new file to see what it looks like ls -l new-utf8.xml file new-utf8.xml wc -c new-utf8.xml enca new-utf8.xml head new-utf8.xml # load the files in the script cat rexmltest.rb ruby rexmltest.rb old-utf16.xml ruby rexmltest.rb new-utf8.xml Thanks.