# Reading Files: how to I specify the encoding?

**URL:** <https://www.ruby-forum.com/t/reading-files-how-to-i-specify-the-encoding/104127>\
**Category:** Ruby\
**Created:** [May 14, 2007, 4:39pm UTC](https://www.ruby-forum.com/t/reading-files-how-to-i-specify-the-encoding/104127 "2007-05-14T16:39:04Z")\
**Posts on this page:** 3\
**Page:** 1

<div class="post-metadata">

**Author:** ![Claus\_Hausberger](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/5daacb/32.png) [@Claus\_Hausberger](https://www.ruby-forum.com/u/Claus_Hausberger)\
**Post date:** [May 14, 2007, 4:39pm UTC](https://www.ruby-forum.com/t/reading-files-how-to-i-specify-the-encoding/104127/1 "2007-05-14T16:39:04Z")

</div>

Hello

I have a lot of xml and java files witch have German Umlauts and other  
non ASCII files in them.

I want to read the files and convert them to UTF-8 using a Ruby script.

I convert the strings with this code:

def to\_utf8(str)  
str.unpack(‘U\*’).map do |c|  
if c \< 0x80  
c.chr  
else  
‘( u%04X )’ % c  
end  
end.join  
end

(taken from “The Ruby Way” by Hal F.).

sometimes it works, sometimes I get this error:  
“malformed UTF-8 character”

I tought this might happen because the File is encoded in ISO-8859-1  
(was written with Eclipse set to ISO-8859-1 for text encoding).

how can I read a file with Ruby and specify that it is read with  
ISO-8859-1 encoding (similar to Java’s BufferedReader where I can  
specify the encoding).

any help welcome. best wishes

Claus

---

<div class="post-metadata">

**Author:** ![Claus\_Hausberger](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/5daacb/32.png) [@Claus\_Hausberger](https://www.ruby-forum.com/u/Claus_Hausberger)\
**Post date:** [May 14, 2007, 4:49pm UTC](https://www.ruby-forum.com/t/reading-files-how-to-i-specify-the-encoding/104127/2 "2007-05-14T16:49:32Z")

</div>

Claus Hausberger wrote:

> str.unpack(‘U\*’).map do |c|  
> I’d be surprised if this was right - you’re telling it that you’re  
> expecting the string to be UTF-8 already with that unpack format.

\> how can I read a file with Ruby and specify that it is read with \> ISO-8859-1 encoding (similar to Java's BufferedReader where I can \> specify the encoding).

Investigate Iconv in the standard library. It does what you need.

---

<div class="post-metadata">

**Author:** ![Claus\_Hausberger](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/c/5daacb/32.png) [@Claus\_Hausberger](https://www.ruby-forum.com/u/Claus_Hausberger)\
**Post date:** [May 14, 2007, 4:50pm UTC](https://www.ruby-forum.com/t/reading-files-how-to-i-specify-the-encoding/104127/3 "2007-05-14T16:50:32Z")

</div>

On 14 May 2007, at 16:39, Claus Hausberger wrote:

> def to\_utf8(str)
> 
> any help welcome. best wishes
> 
> Claus
> 
> –  
> Posted via [http://www.ruby-forum.com/](http://www.ruby-forum.com/).

Hallo Claus,

you could use jcode…

$KCODE = ‘UTF8’  
require ‘jcode’

Cheers,

Enrique Comba R.