# Multibyte regexps

**URL:** <https://www.ruby-forum.com/t/multibyte-regexps/50807>\
**Category:** Ruby\
**Created:** [December 21, 2005, 11:02am UTC](https://www.ruby-forum.com/t/multibyte-regexps/50807 "2005-12-21T11:02:57Z")\
**Posts on this page:** 5\
**Page:** 1

<div class="post-metadata">

**Author:** ![Horacio\_S](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/97f17d/32.png) [@Horacio\_S](https://www.ruby-forum.com/u/Horacio_S)\
**Post date:** [December 21, 2005, 11:02am UTC](https://www.ruby-forum.com/t/multibyte-regexps/50807/1 "2005-12-21T11:02:57Z")

</div>

I am having some issues with regular expressions when working with  
japanese  
strings.

Using ruby-1.8.3 on Windows XP home (Japanese version) I have this test:

irb(main):271:0\> s = “é??”  
=\> “\212\223”  
irb(main):272:0\> l = “è¡?”  
=\> “\215s”  
irb(main):273:0\> l =~ /s/  
=\> 1  
irb(main):274:0\> puts “#{$`}<<#{$&}>>#{$'}" E<s>> => nil irb(main):275:0> "#{$`}\<\<#{$&}\>\>#{$’}”  
=\> “\215\<~~\>”  
irb(main):276:0\> s =~ /l/  
=\> nil~~

As you can see comparing two totally different characters (kanji) gives  
me a  
match. Reversing the match gives nil.

How can I get ruby to match things correctly??

regards,  
Horacio

---

<div class="post-metadata">

**Author:** ![Horacio\_S](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/97f17d/32.png) [@Horacio\_S](https://www.ruby-forum.com/u/Horacio_S)\
**Post date:** [December 21, 2005, 1:51pm UTC](https://www.ruby-forum.com/t/multibyte-regexps/50807/2 "2005-12-21T13:51:29Z")

</div>

Hi,

In message “Re: Multibyte regexps…”  
on Wed, 21 Dec 2005 18:59:59 +0900, Horacio S.  
[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) writes:

|I am having some issues with regular expressions when working with japanese  
|strings.  
|  
|Using ruby-1.8.3 on Windows XP home (Japanese version) I have this test:  
|  
|irb(main):271:0\> s = “e$B3se(B”  
|=\> “\212\223”  
|irb(main):272:0\> l = “e$B9Te(B”  
|=\> “\215s”  
|irb(main):273:0\> l =~ /s/  
|=\> 1  
|irb(main):274:0\> puts “#{$`}<<#{$&}>>#{$'}" |E<s>> |=> nil |irb(main):275:0> "#{$`}\<\<#{$&}\>\>#{$'}”  
|=\> “\215\<~~\>”  
|irb(main):276:0\> s =~ /l/  
|=\> nil~~

 

The encoding seems to be Shift\_JIS. You have to specify encoding  
before you make regular expression matching. Put s after every  
regular expression.

$KCODE=“sjis” # to make p work right  
p s = “e$B3se(B”  
p l = “e$B9Te(B”  
p l =~ /s/s  
puts “#{$`}<<#{$&}>>#{$'}" p "#{$`}\<\<#{$&}\>\>#{$'}”  
p s =~ /l/s

```
						matz.

```

---

<div class="post-metadata">

**Author:** ![Horacio\_S](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/97f17d/32.png) [@Horacio\_S](https://www.ruby-forum.com/u/Horacio_S)\
**Post date:** [December 26, 2005, 2:32am UTC](https://www.ruby-forum.com/t/multibyte-regexps/50807/3 "2005-12-26T02:32:05Z")

</div>

Thanks a lot… this seems to work ok.

Where can I find documentation about this $KCODE global var and the “s”  
thing  
after each regexp? What does the s exactly mean?

Do I have to put it only in regexps with japanese characters or any  
regexp? I  
tried both and saw no difference.

When using Regexp.new to construct the regular expression how can I set  
the s  
to the end of it??

sorry for so many questions but I don’t seem to find any docs about  
these  
options.

Horacio

Wednesday 21 December 2005 21:48e$B!"e(BYukihiro M.  
e$B$5$s$O=q$-$^$7$?e(B:

> Hi,
> 
> In message “Re: Multibyte regexps…”
> 
> ```
> on Wed, 21 Dec 2005 18:59:59 +0900, Horacio S. 
> 
> ```

[removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) writes:

---

<div class="post-metadata">

**Author:** ![Horacio\_S](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/97f17d/32.png) [@Horacio\_S](https://www.ruby-forum.com/u/Horacio_S)\
**Post date:** [December 26, 2005, 3:53am UTC](https://www.ruby-forum.com/t/multibyte-regexps/50807/4 "2005-12-26T03:53:10Z")

</div>

I found some documentation about this. Thanks.

Just one question, it seems to me that I can make two different things  
to  
allow Regexp’s to handle multibyte Shift\_JIS strings. One is to set the  
$KCODE global variable to “sjis” and the other one is to use the “s”  
modifier  
when constructing the regular expresion.

The question is do I use only one of the two methods or shall I use the  
“s”  
modifier even if I set $KCODE to “sjis”??

My testing tells me that only setting the $KCODE global var is enough to  
get  
Shift\_JIS strings and Regexp’s to work correctly but I just want to make  
sure.

thanks,  
Horacio

Monday 26 December 2005 10:29e$B!"e(BHoracio S.  
e$B$5$s$O=q$-$^$7$?e(B:

---

<div class="post-metadata">

**Author:** ![Horacio\_S](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/h/97f17d/32.png) [@Horacio\_S](https://www.ruby-forum.com/u/Horacio_S)\
**Post date:** [December 21, 2005, 12:33pm UTC](https://www.ruby-forum.com/t/multibyte-regexps/50807/5 "2005-12-21T12:33:11Z")

</div>

l =~ /s/ ??

It will try to find a char “s” in string l and not the value remained  
in variable s.

Horacio S. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

I am having some issues with regular expressions when working with  
japanese  
strings.

Using ruby-1.8.3 on Windows XP home (Japanese version) I have this test:

irb(main):271:0\> s = “éÂ?Â?”  
=\> “\212\223”  
irb(main):272:0\> l = “è¡Â?”  
=\> “\215s”  
irb(main):273:0\> l =~ /s/  
=\> 1  
irb(main):274:0\> puts “#{$`}<<#{$&}>>#{$'}" E> => nil irb(main):275:0> "#{$`}\<\<#{$&}\>\>#{$'}”  
=\> “\215\<\>”  
irb(main):276:0\> s =~ /l/  
=\> nil

As you can see comparing two totally different characters (kanji) gives  
me a  
match. Reversing the match gives nil.

How can I get ruby to match things correctly??

regards,  
Horacio