# Utf8 encoding problem

**URL:** <https://www.ruby-forum.com/t/utf8-encoding-problem/171065>\
**Category:** Ruby\
**Created:** [June 25, 2009, 11:26pm UTC](https://www.ruby-forum.com/t/utf8-encoding-problem/171065 "2009-06-25T23:26:55Z")\
**Posts on this page:** 4\
**Page:** 1

<div class="post-metadata">

**Author:** ![adad](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/9e8a1a/32.png) [@adad](https://www.ruby-forum.com/u/adad)\
**Post date:** [June 25, 2009, 11:26pm UTC](https://www.ruby-forum.com/t/utf8-encoding-problem/171065/1 "2009-06-25T23:26:55Z")

</div>

Hi,  
I am retrieving a string from a txt file.  
The file contains some utf8 characters.

I am comparing these characters against a default string.

The problem is that some of the characters are not stored in a default  
format.

For example:  
A is stored as Ａ

Naturally when I compare the character it fails.  
Strangely when I unpacked the character it appears as 65313 which is the  
correct utf8 number for A.

Any way around this?

thanks.

---

<div class="post-metadata">

**Author:** ![adad](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/9e8a1a/32.png) [@adad](https://www.ruby-forum.com/u/adad)\
**Post date:** [June 26, 2009, 12:11am UTC](https://www.ruby-forum.com/t/utf8-encoding-problem/171065/2 "2009-06-26T00:11:48Z")

</div>

On Jun 25, 2009, at 14:29, Ad Ad wrote:

> A is stored as Ａ
> 
> Naturally when I compare the character it fails.  
> Strangely when I unpacked the character it appears as 65313 which is  
> the  
> correct utf8 number for A.
> 
> Any way around this?

Well, Ａ is “Fullwidth Latin Capital Letter A” from the “Hiragana and  
Katakana” category (Unicode FF21) whereas A is “Latin Capital Letter  
A” from the “Latin” category (Unicode 0041).

I don’t know of a way to translate between the two categories, but  
maybe that will help.

---

<div class="post-metadata">

**Author:** ![adad](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/9e8a1a/32.png) [@adad](https://www.ruby-forum.com/u/adad)\
**Post date:** [June 26, 2009, 5:22am UTC](https://www.ruby-forum.com/t/utf8-encoding-problem/171065/3 "2009-06-26T05:22:24Z")

</div>

Although I haven’t tried it myself, I did a search for  
e$BA43QH\>3QJQ49e(B and  
found this page.  
It appears people use jcode and tr to solve this problem.

[http://www.eml.ele.cst.nihon-u.ac.jp/~momma/wiki/wiki.cgi/Ruby/全角半角変換.html](http://www.eml.ele.cst.nihon-u.ac.jp/~momma/wiki/wiki.cgi/Ruby/%E5%85%A8%E8%A7%92%E5%8D%8A%E8%A7%92%E5%A4%89%E6%8F%9B.html)  
[http://blog.grayproductions.net/articles/the\_kcode\_variable\_and\_jcode\_library](http://blog.grayproductions.net/articles/the_kcode_variable_and_jcode_library)

2009/6/25 Eric H. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):

---

<div class="post-metadata">

**Author:** ![adad](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/a/9e8a1a/32.png) [@adad](https://www.ruby-forum.com/u/adad)\
**Post date:** [June 26, 2009, 4:38pm UTC](https://www.ruby-forum.com/t/utf8-encoding-problem/171065/4 "2009-06-26T16:38:16Z")

</div>

James Rubingh wrote:

> Although I haven’t tried it myself, I did a search for  
> e$BA43QH\>3QJQ49e(B and  
> found this page.  
> It appears people use jcode and tr to solve this problem.
> 
> [http://www.eml.ele.cst.nihon-u.ac.jp/~momma/wiki/wiki.cgi/Ruby/全角半角変換.html](http://www.eml.ele.cst.nihon-u.ac.jp/~momma/wiki/wiki.cgi/Ruby/%E5%85%A8%E8%A7%92%E5%8D%8A%E8%A7%92%E5%A4%89%E6%8F%9B.html)  
> [http://blog.grayproductions.net/articles/the\_kcode\_variable\_and\_jcode\_library](http://blog.grayproductions.net/articles/the_kcode_variable_and_jcode_library)
> 
> 2009/6/25 Eric H. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid):

brilliant!  
str.tr!(‘ａ-ｚＡ-Ｚ’,‘a-zA-z’) worked like a charm. 🙂