# Problem with String encoding when modifying it in C method

**URL:** <https://www.ruby-forum.com/t/problem-with-string-encoding-when-modifying-it-in-c-method/164661>\
**Category:** Ruby\
**Created:** [April 3, 2009, 6:41pm UTC](https://www.ruby-forum.com/t/problem-with-string-encoding-when-modifying-it-in-c-method/164661 "2009-04-03T18:41:06Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![ISSSSaki\_Baz\_C](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/51bf81/32.png) [@ISSSSaki\_Baz\_C](https://www.ruby-forum.com/u/ISSSSaki_Baz_C)\
**Post date:** [April 3, 2009, 6:41pm UTC](https://www.ruby-forum.com/t/problem-with-string-encoding-when-modifying-it-in-c-method/164661/1 "2009-04-03T18:41:06Z")

</div>

Hi, I’ve added a method “multi\_capitalize” to String class. This  
method is done in C and basically modifies the string:

“record-roUTE”.multi\_capitalize =\> “Record-Route”

The problem is that after the method execution, the new String has  
ASCII-8BIT encoding, while the original string had UTF-8 (using Ruby  
1.9.1).

* * *

irb\> hname = “record-rouTE-€”  
“record-rouTE-€”

irb\> hname.encoding  
#Encoding:UTF-8

irb\> hname2 = hname.multi\_capitalize  
“Record-Route-\xE2\x82\xAC” \<------- !!!

irb\> hname2.encoding  
#Encoding:ASCII-8BIT \<------- !!!

irb\> hname2.force\_encoding(“utf-8”)  
“Record-Route-€”

## irb\> hname2.encoding #Encoding:UTF-8

What should I add to my C method to mantain the UTF-8 codification  
after the changes in the string?  
Could I invoke the C “force\_encoding()” function from the C code  
before returning the modified string? How to invoke it?

Thanks a lot.

---

<div class="post-metadata">

**Author:** ![ISSSSaki\_Baz\_C](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/51bf81/32.png) [@ISSSSaki\_Baz\_C](https://www.ruby-forum.com/u/ISSSSaki_Baz_C)\
**Post date:** [April 3, 2009, 8:18pm UTC](https://www.ruby-forum.com/t/problem-with-string-encoding-when-modifying-it-in-c-method/164661/2 "2009-04-03T20:18:59Z")

</div>

On Sat, 2009-04-04 at 01:39 +0900, Iñaki Baz C. wrote:

> Could I invoke the C “force\_encoding()” function from the C code  
> before returning the modified string? How to invoke it?

You can call it as (untested):

rb\_funcall(str, rb\_intern(“force\_encoding”), 1, rb\_str\_new2(“utf-8”));

I’m not sure how to make your multi-capitalize method do the right  
thing, but maybe reading the source of rb\_str\_capitalize\_bang in  
string.c helps.

Best,  
Andre

---

<div class="post-metadata">

**Author:** ![ISSSSaki\_Baz\_C](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/51bf81/32.png) [@ISSSSaki\_Baz\_C](https://www.ruby-forum.com/u/ISSSSaki_Baz_C)\
**Post date:** [April 3, 2009, 8:34pm UTC](https://www.ruby-forum.com/t/problem-with-string-encoding-when-modifying-it-in-c-method/164661/3 "2009-04-03T20:34:46Z")

</div>

El Viernes 03 Abril 2009, Andre N. escribió:

> string.c helps.  
> Thanks a lot, I will check it.

---

<div class="post-metadata">

**Author:** ![ISSSSaki\_Baz\_C](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/51bf81/32.png) [@ISSSSaki\_Baz\_C](https://www.ruby-forum.com/u/ISSSSaki_Baz_C)\
**Post date:** [April 3, 2009, 9:01pm UTC](https://www.ruby-forum.com/t/problem-with-string-encoding-when-modifying-it-in-c-method/164661/4 "2009-04-03T21:01:45Z")

</div>

El Viernes 03 Abril 2009, Iñaki Baz C. escribió:

> > thing, but maybe reading the source of rb\_str\_capitalize\_bang in  
> > string.c helps.
> 
> Thanks a lot, I will check it.

Yes, rb\_str\_capitralize\_bang handles a lot of stuf realted to encoding:

```
c = rb_enc_codepoint(s, send, enc);
if (rb_enc_islower(c, enc)) {

```

rb\_enc\_mbcput(rb\_enc\_toupper(c, enc), s, enc);  
modify = 1;  
}  
s += rb\_enc\_codelen(c, enc);

so this is the way 🙂

Thanks a lot.

---

<div class="post-metadata">

**Author:** ![ISSSSaki\_Baz\_C](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/51bf81/32.png) [@ISSSSaki\_Baz\_C](https://www.ruby-forum.com/u/ISSSSaki_Baz_C)\
**Post date:** [April 4, 2009, 12:39pm UTC](https://www.ruby-forum.com/t/problem-with-string-encoding-when-modifying-it-in-c-method/164661/5 "2009-04-04T12:39:56Z")

</div>

El Sábado 04 Abril 2009, KUBO Takehiro  
escribió:\> \> 1.9.1).

> ```
> rb_encoding *enc = rb_enc_get(original_string)
> 
> /* create a new string with the encoding same with the original string
> 
> ```
> 
> \*/ return rb\_enc\_str\_new(char\_pointer, length, enc);
> 
> rb\_str\_new() makes a ASCII-8BIT string.

Thanks.

---

<div class="post-metadata">

**Author:** ![ISSSSaki\_Baz\_C](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/i/51bf81/32.png) [@ISSSSaki\_Baz\_C](https://www.ruby-forum.com/u/ISSSSaki_Baz_C)\
**Post date:** [April 4, 2009, 12:34pm UTC](https://www.ruby-forum.com/t/problem-with-string-encoding-when-modifying-it-in-c-method/164661/6 "2009-04-04T12:34:05Z")

</div>

Hi,

On Sat, Apr 4, 2009 at 1:39 AM, Iñaki Baz C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> Hi, I’ve added a method “multi\_capitalize” to String class. This  
> method is done in C and basically modifies the string:
> 
> “record-roUTE”.multi\_capitalize =\> “Record-Route”
> 
> The problem is that after the method execution, the new String has  
> ASCII-8BIT encoding, while the original string had UTF-8 (using Ruby  
> 1.9.1).

```
rb_encoding *enc = rb_enc_get(original_string)

/* create a new string with the encoding same with the original 

```

string \*/  
return rb\_enc\_str\_new(char\_pointer, length, enc);

rb\_str\_new() makes a ASCII-8BIT string.