# String encoding problem with ruby 1.9.2p0

**URL:** <https://www.ruby-forum.com/t/string-encoding-problem-with-ruby-1-9-2p0/195470>\
**Category:** Ruby-Gnome 2\
**Created:** [September 23, 2010, 3:49pm UTC](https://www.ruby-forum.com/t/string-encoding-problem-with-ruby-1-9-2p0/195470 "2010-09-23T15:49:53Z")\
**Posts on this page:** 11\
**Page:** 1

<div class="post-metadata">

**Author:** ![blueprint](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/cc9497/32.png) [@blueprint](https://www.ruby-forum.com/u/blueprint)\
**Post date:** [September 23, 2010, 3:49pm UTC](https://www.ruby-forum.com/t/string-encoding-problem-with-ruby-1-9-2p0/195470/1 "2010-09-23T15:49:53Z")

</div>

I am trying to make my [http://booh.org/](http://booh.org/) app work with ruby 1.9.2p0.

There’s a problem retrieving String data from a Gtk::TreeIter. Ruby  
says its encoding is ASCII-8BIT which I believe happens when the  
encoding was not marked at String creation, and then if it contains  
non ASCII characters it fails (on me, it fails in REXML).

-=-=—=-=—=-=—=-=—=-=–  
#! /usr/bin/ruby

# encoding: UTF-8

require ‘gtk2’  
require ‘rexml/document’  
include REXML

def check\_with\_rexml(string)  
xmldoc = Document.new(“”)  
xmldoc \<\< XMLDecl.new(XMLDecl::DEFAULT\_VERSION, ‘UTF-8’)  
xmldoc.root.attributes[‘key’] = string  
end

Gtk.init  
treestore = Gtk::TreeStore.new(String)  
iter = treestore.append(nil)

puts “\n\*\*\* PURE ASCII TEST \*\*\*”  
pureascii = “pure ascii”  
iter[0] = pureascii  
puts "my string has encoding " + pureascii.encoding.to\_s + " and  
retrieved from iter has encoding " + iter[0].encoding.to\_s  
check\_with\_rexml(iter[0])

puts “\n\*\*\* WITH ACCENT TEST \*\*\*”  
withaccent =  
"finalisé"iter[0] = withaccent  
puts "my string has encoding " + withaccent.encoding.to\_s + " and  
retrieved from iter has encoding " + iter[0].encoding.to\_s  
check\_with\_rexml(iter[0])  
-=-=—=-=—=-=—=-=—=-=–

=\>

\*\*\* PURE ASCII TEST \*\*\*  
my string has encoding UTF-8 and retrieved from iter has encoding  
ASCII-8BIT

\*\*\* WITH ACCENT TEST \*\*\*  
my string has encoding UTF-8 and retrieved from iter has encoding  
ASCII-8BIT  
/usr/lib/ruby/1.9.1/rexml/text.rb:131:in `=~': incompatible encoding regexp match (UTF-8 regexp with ASCII-8BIT string) (Encoding::CompatibilityError) from /usr/lib/ruby/1.9.1/rexml/text.rb:131:in `!~’  
from /usr/lib/ruby/1.9.1/rexml/text.rb:131:in `check' from /usr/lib/ruby/1.9.1/rexml/attribute.rb:153:in `element=’  
from /usr/lib/ruby/1.9.1/rexml/element.rb:1104:in `[]=' from /tmp/rb19encodingpb.rb:11:in `check\_with\_rexml’  
from /tmp/rb19encodingpb.rb:28:in `’

This is ruby 1.9.2p0 (2010-08-18 revision 29036), with rg2 0.19.2 and  
0.90.0.  
Thanks.

–  
Guillaume C. - [http://zarb.org/~gc/](http://zarb.org/~gc/)

---

<div class="post-metadata">

**Author:** ![blueprint](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/cc9497/32.png) [@blueprint](https://www.ruby-forum.com/u/blueprint)\
**Post date:** [September 23, 2010, 3:53pm UTC](https://www.ruby-forum.com/t/string-encoding-problem-with-ruby-1-9-2p0/195470/2 "2010-09-23T15:53:58Z")

</div>

Hi,

In [AANLkTimCKm1X-bwsNb7dVd=removed\_email\_address@domain.invalid](mailto:AANLkTimCKm1X-bwsNb7dVd=removed_email_address@domain.invalid)  
“[ruby-gnome2-devel-en] String encoding problem with ruby 1.9.2p0” on  
Thu, 23 Sep 2010 15:49:10 +0200,  
Guillaume C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> I am trying to make my [http://booh.org/](http://booh.org/) app work with ruby 1.9.2p0.

Thanks.

> There’s a problem retrieving String data from a Gtk::TreeIter. Ruby  
> says its encoding is ASCII-8BIT which I believe happens when the  
> encoding was not marked at String creation, and then if it contains  
> non ASCII characters it fails (on me, it fails in REXML).

String encoding support is one of the important TODO items.  
We need help for the work.

## Thanks,

kou

---

<div class="post-metadata">

**Author:** ![blueprint](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/cc9497/32.png) [@blueprint](https://www.ruby-forum.com/u/blueprint)\
**Post date:** [September 23, 2010, 4:02pm UTC](https://www.ruby-forum.com/t/string-encoding-problem-with-ruby-1-9-2p0/195470/3 "2010-09-23T16:02:45Z")

</div>

> > I am trying to make my [http://booh.org/](http://booh.org/) app work with ruby 1.9.2p0.
> 
> Thanks.

It’s the fault of one of my users 😢

> > There’s a problem retrieving String data from a Gtk::TreeIter. Ruby  
> > says its encoding is ASCII-8BIT which I believe happens when the  
> > encoding was not marked at String creation, and then if it contains  
> > non ASCII characters it fails (on me, it fails in REXML).
> 
> String encoding support is one of the important TODO items.  
> We need help for the work.

I see. I’ll may throw it an eye but time is sparse, as for you I  
guess. RG2 definitely lacks manpower, but it’s a recurrent problem  
with little solutions (one workaround might be to concentrate more on  
“core” bindings without sending too much time on subbindings with less  
users?).

–  
Guillaume C. - [NVC Communication Obstacles Game](http://zarb.org/~gc/)

---

<div class="post-metadata">

**Author:** ![blueprint](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/cc9497/32.png) [@blueprint](https://www.ruby-forum.com/u/blueprint)\
**Post date:** [September 26, 2010, 12:44am UTC](https://www.ruby-forum.com/t/string-encoding-problem-with-ruby-1-9-2p0/195470/4 "2010-09-26T00:44:45Z")

</div>

> > There’s a problem retrieving String data from a Gtk::TreeIter. Ruby  
> > says its encoding is ASCII-8BIT which I believe happens when the  
> > encoding was not marked at String creation, and then if it contains  
> > non ASCII characters it fails (on me, it fails in REXML).
> 
> String encoding support is one of the important TODO items.  
> We need help for the work.

I have looked at the source code. I feared that all Ruby String  
creations from GTK+ would miss setting the encoding, which is the  
case, they seem to be (nearly) all ASCII-8BIT 😕

So this simple GtkLabel test shows ASCII-8BIT encoding 😕

f = “bla”  
l = Gtk::Label.new  
l.text = f  
puts f.encoding  
puts l.text.encoding

Not good…

So my proposal is the following: all Ruby String created by rg2 must  
have an encoding set, otherwise here and there, there will be similar  
problems to mine (operations failing on an ASCII-8BIT String gotten  
from rg2).

But, as GTK+ returns gchar\*, we don’t know what encoding to use. They  
may fall into these categories:

- Ruby String passed to GTK+ then retrieved from GTK+ (my case in  
GtkTreeStore, cases when reading GtkLabel, etc)

- default stuff from GTK+ (not much I guess, and much probably in  
US-ASCII - some people have actual ideas about it?)

so if we use always a unique encoding when passing Ruby String to  
GTK+, and this encoding is US-ASCII compatible, then we can use this  
encoding when retrieving from GTK+; I propose to use UTF-8 which is  
“standard” within GTK+ since 2.0, and quite standard elsewhere too  
(sadly, not much standard in Japan, I heard?). Anyways, it would  
always be better than current situation, no?

So my patch converts Ruby String to UTF-8 if it’s not already in UTF-8  
before passing to GTK+; and properly creates an UTF-8 Ruby String when  
retrieving from GTK+.

Notice: RVAL2CSTR would be now an alias to RVAL2CSTR\_ACCEPT\_NIL, and I  
think it is more safe because it’s always better to properly handle  
NIL case.

Notice: I also patched README because the partial modules building  
documentation was broken.

What do you think of my patch?

Now, this patch properly works on the simple GtkLabel test, and my  
GtkTreeStore based test posted in this thread earlier. If the  
principle is fine with you, I think we must convert all rb\_str\_new2 of  
sourcecode to CSTR2RVAL, and StringValuePtr to RVAL2CSTR (are there  
other macros/ruby internals used to convert between Ruby String and  
C?), to fix other similar problems.

Thanks,

---

<div class="post-metadata">

**Author:** ![blueprint](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/cc9497/32.png) [@blueprint](https://www.ruby-forum.com/u/blueprint)\
**Post date:** [September 26, 2010, 2:09am UTC](https://www.ruby-forum.com/t/string-encoding-problem-with-ruby-1-9-2p0/195470/5 "2010-09-26T02:09:01Z")

</div>

On 26 September 2010 07:43, Guillaume C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
wrote:

> I propose to use UTF-8 which is  
> “standard” within GTK+ since 2.0, and quite standard elsewhere too  
> (sadly, not much standard in Japan, I heard?). Anyways, it would  
> always be better than current situation, no?

Perhaps I’m wrong, but I thought all strings in GTK+ were UTF-8.  
I don’t know where I read it, but I’m pretty sure that other encodings  
don’t work properly.

As for Japan, I think there are 3 main standards. On older Unix  
machines they tend to use EUC. On older Windows boxes they  
use Shift-JIS. But on modern Unix-like machines I think most people  
use UTF-8 while on modern Windows boxes they use UCS2.

Having said that, when I use ruby gnome with ruby version 1.8.x  
on Windows, the strings I get back from the IME are definitely  
UTF-8 (tested on both Windows XP and Windows 7). So I’m  
relatively sure that GTK+ is storing the string as UTF-8 somehow.

If that’s true then it should be OK to simply mark UTF-8 as the  
encoding for all GTK+ strings.

```
    MikeC

```

---

<div class="post-metadata">

**Author:** ![blueprint](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/cc9497/32.png) [@blueprint](https://www.ruby-forum.com/u/blueprint)\
**Post date:** [September 26, 2010, 4:55am UTC](https://www.ruby-forum.com/t/string-encoding-problem-with-ruby-1-9-2p0/195470/6 "2010-09-26T04:55:15Z")

</div>

Hi,

In [AANLkTi=removed\_email\_address@domain.invalid](mailto:AANLkTi=removed_email_address@domain.invalid)  
“Re: [ruby-gnome2-devel-en] String encoding problem with ruby 1.9.2p0”  
on Sun, 26 Sep 2010 00:43:56 +0200,  
Guillaume C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:

> I have looked at the source code. I feared that all Ruby String  
> creations from GTK+ would miss setting the encoding, which is the  
> case, they seem to be (nearly) all ASCII-8BIT 😕

Yes. All those String doesn’t have encoding information.

> GTK+, and this encoding is US-ASCII compatible, then we can use this  
> encoding when retrieving from GTK+; I propose to use UTF-8 which is  
> “standard” within GTK+ since 2.0, and quite standard elsewhere too  
> (sadly, not much standard in Japan, I heard?). Anyways, it would  
> always be better than current situation, no?
> 
> So my patch converts Ruby String to UTF-8 if it’s not already in UTF-8  
> before passing to GTK+; and properly creates an UTF-8 Ruby String when  
> retrieving from GTK+.

I almost agree with your patch but I have some comments.

- It’s true that GTK+ uses UTF-8 as it’s internal/external  
encoding as Mike said. It’s OK to convert Ruby String to  
UTF-8 for passing to GTK+.

- But there are some cases that GTK+ doesn’t return  
UTF-8. e.g. g\_convert()(\*1), g\_utf8\_to\_utf16()(\*2) and  
so on. So, we need to provide an API for specify  
encoding like CSTR2RVAL\_ENCODING().

- UTF-8 is the standard encoding in Japan, now. 🙂

- Please wait to change this after 0.90.2 is  
released. I’ll release 0.90.2 soon. 🙂

> Notice: RVAL2CSTR would be now an alias to RVAL2CSTR\_ACCEPT\_NIL, and I  
> think it is more safe because it’s always better to properly handle  
> NIL case.

We need to RVAL2CSTR with nil check because there are many  
API that doesn’t accept NULL as gchar \* argument. We will  
need a function that do StringValuePtr() with encoding handling.

> Notice: I also patched README because the partial modules building  
> documentation was broken.
> 
> What do you think of my patch?

Please commit README patch for now. Please wait other patches.

> Now, this patch properly works on the simple GtkLabel test, and my  
> GtkTreeStore based test posted in this thread earlier. If the  
> principle is fine with you, I think we must convert all rb\_str\_new2 of  
> sourcecode to CSTR2RVAL, and StringValuePtr to RVAL2CSTR (are there  
> other macros/ruby internals used to convert between Ruby String and  
> C?), to fix other similar problems.

It seems OK for me. But we don’t forget that there are some  
exceptions like g\_convert(). We need to handle those  
exceptions by hands.

## Thanks,

kou

---

<div class="post-metadata">

**Author:** ![blueprint](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/cc9497/32.png) [@blueprint](https://www.ruby-forum.com/u/blueprint)\
**Post date:** [September 26, 2010, 10:12am UTC](https://www.ruby-forum.com/t/string-encoding-problem-with-ruby-1-9-2p0/195470/7 "2010-09-26T10:12:25Z")

</div>

On Sun, Sep 26, 2010 at 2:08 AM, Mike C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
wrote:

> On 26 September 2010 07:43, Guillaume C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid) wrote:
> 
> > I propose to use UTF-8 which is  
> > “standard” within GTK+ since 2.0, and quite standard elsewhere too  
> > (sadly, not much standard in Japan, I heard?). Anyways, it would  
> > always be better than current situation, no?
> 
> Perhaps I’m wrong, but I thought all strings in GTK+ were UTF-8.  
> I don’t know where I read it, but I’m pretty sure that other encodings  
> don’t work properly.

IIRC, it’s related to Pango mainly. For example, if your system is in  
ISO-8859-1, then files gotten from, say, GtkFileChooser will be in  
that encoding, not UTF-8 - that’s a point I forgot to handle in my  
yesterday’s patch, actually 😕

–  
Guillaume C. - [NVC Communication Obstacles Game](http://zarb.org/~gc/)

---

<div class="post-metadata">

**Author:** ![blueprint](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/cc9497/32.png) [@blueprint](https://www.ruby-forum.com/u/blueprint)\
**Post date:** [September 26, 2010, 2:57am UTC](https://www.ruby-forum.com/t/string-encoding-problem-with-ruby-1-9-2p0/195470/8 "2010-09-26T02:57:01Z")

</div>

Mike C. wrote:

> On 26 September 2010 07:43, Guillaume C. [removed\_email\_address@domain.invalid](mailto:removed_email_address@domain.invalid)  
> wrote:
> 
> > I propose to use UTF-8 which is  
> > “standard” within GTK+ since 2.0, and quite standard elsewhere too  
> > (sadly, not much standard in Japan, I heard?). Anyways, it would  
> > always be better than current situation, no?
> 
> Perhaps I’m wrong, but I thought all strings in GTK+ were UTF-8.  
> I don’t know where I read it, but I’m pretty sure that other encodings  
> don’t work properly.
> 
> As for Japan, I think there are 3 main standards. On older Unix  
> machines they tend to use EUC. On older Windows boxes they  
> use Shift-JIS. But on modern Unix-like machines I think most people  
> use UTF-8 while on modern Windows boxes they use UCS2.
> 
> Having said that, when I use ruby gnome with ruby version 1.8.x  
> on Windows, the strings I get back from the IME are definitely  
> UTF-8 (tested on both Windows XP and Windows 7). So I’m  
> relatively sure that GTK+ is storing the string as UTF-8 somehow.
> 
> If that’s true then it should be OK to simply mark UTF-8 as the  
> encoding for all GTK+ strings.
> 
> ```
> MikeC
> 
> ```

I’m not sure about the encoding. But with ruby-gnome 0.19.4 and ruby  
1.9.2p0, the textview example failed, testgtk also fail on encoding:

C:\Users\swong24\Development\ruby-gtk2-0.19.4\gtk\sample\testgtk\>testgtk.rb  
internal:lib/rubygems/custom\_require:29:in `require': C:/Users/swong24/Develop ment/ruby-gtk2-0.19.4/gtk/sample/testgtk/labels.rb:54: invalid multibyte char (U S-ASCII) (SyntaxError) C:/Users/swong24/Development/ruby-gtk2-0.19.4/gtk/sample/testgtk/labels.rb:54: i nvalid multibyte char (US-ASCII) C:/Users/swong24/Development/ruby-gtk2-0.19.4/gtk/sample/testgtk/labels.rb:54: s yntax error, unexpected $end, expecting ')' ...!\nThis one is underlined in ÂµÃ¹Ã‘ÂµÂ£Â¼Î¦Â¬â‚§Ï€Ã¼Â«ÏƒÃ Ã‘Ï„Ã¶Â¿quite a funky... ... ^ from <internal:lib/rubygems/custom_require>:29:in `require’  
from  
C:/Users/swong24/Development/ruby-gtk2-0.19.4/gtk/sample/testgtk/te  
stgtk.rb:48:in `’

---

<div class="post-metadata">

**Author:** ![blueprint](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/cc9497/32.png) [@blueprint](https://www.ruby-forum.com/u/blueprint)\
**Post date:** [September 26, 2010, 10:18am UTC](https://www.ruby-forum.com/t/string-encoding-problem-with-ruby-1-9-2p0/195470/9 "2010-09-26T10:18:16Z")

</div>

> > - Ruby String passed to GTK+ then retrieved from GTK+ (my case in  
> > GtkTreeStore, cases when reading GtkLabel, etc)
> > 
> > - default stuff from GTK+ (not much I guess, and much probably in  
> > US-ASCII - some people have actual ideas about it?)

I forgot to handle cases of Strings got from GTK+ to represent file  
paths (from GtkFileChooser for example). There, the system encoding is  
used and we must handle that specifically.

> > So my patch converts Ruby String to UTF-8 if it’s not already in UTF-8  
> > before passing to GTK+; and properly creates an UTF-8 Ruby String when  
> > retrieving from GTK+.
> 
> I almost agree with your patch but I have some comments.

Thanks for the fast review 🙂

> (\*2) [Learn GNOME](http://library.gnome.org/devel/glib/stable/glib-Unicode-Manipulation.html#g-utf8-to-utf16)  
> True enough! I am not too fluent in glib stuff 😕

> - UTF-8 is the standard encoding in Japan, now. 🙂

Great 🙂 But it represents an overhead for you guys, no? E.g. even  
katakanas seem to need 3 bytes in UTF-8 😕

> **[KATAKANA LETTER A (U+30A2)](https://www.fileformat.info/info/unicode/char/30a2/index.htm)**
>
> Get the complete details on Unicode character U+30A2 on FileFormat.Info

> - Please wait to change this after 0.90.2 is  
> released. I’ll release 0.90.2 soon. 🙂

Yes, of course. It has been too much time, and this is too  
“sensitive”, for me to commit without your approval!

> > Notice: RVAL2CSTR would be now an alias to RVAL2CSTR\_ACCEPT\_NIL, and I  
> > think it is more safe because it’s always better to properly handle  
> > NIL case.
> 
> We need to RVAL2CSTR with nil check because there are many  
> API that doesn’t accept NULL as gchar \* argument. We will  
> need a function that do StringValuePtr() with encoding handling.

Sorry, I didn’t properly understand that. I see now.

> > Now, this patch properly works on the simple GtkLabel test, and my  
> > GtkTreeStore based test posted in this thread earlier. If the  
> > principle is fine with you, I think we must convert all rb\_str\_new2 of  
> > sourcecode to CSTR2RVAL, and StringValuePtr to RVAL2CSTR (are there  
> > other macros/ruby internals used to convert between Ruby String and  
> > C?), to fix other similar problems.
> 
> It seems OK for me. But we don’t forget that there are some  
> exceptions like g\_convert(). We need to handle those  
> exceptions by hands.

Ok, I’ll try to improve my patch soon.

–  
Guillaume C. - [NVC Communication Obstacles Game](http://zarb.org/~gc/)

---

<div class="post-metadata">

**Author:** ![blueprint](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/cc9497/32.png) [@blueprint](https://www.ruby-forum.com/u/blueprint)\
**Post date:** [September 26, 2010, 10:59pm UTC](https://www.ruby-forum.com/t/string-encoding-problem-with-ruby-1-9-2p0/195470/10 "2010-09-26T22:59:09Z")

</div>

> > > Â - default stuff from GTK+ (not much I guess, and much probably in  
> > > US-ASCII - some people have actual ideas about it?)
> 
> I forgot to handle cases of Strings got from GTK+ to represent file  
> paths (from GtkFileChooser for example). There, the system encoding is  
> used and we must handle that specifically.

After reading GtkFileChooser documentation, it confirms my memories.  
“filenames are always returned in the character set specified by the  
G\_FILENAME\_ENCODING environment variable”. Also, “while you can pass  
the result of gtk\_file\_chooser\_get\_filename() to open(2) or fopen(3),  
you may not be able to directly set it as the text of a GtkLabel  
widget unless you convert it first to UTF-8, which all GTK+ widgets  
expect. You should use g\_filename\_to\_utf8() to convert filenames into  
strings that can be passed to GTK+ widgets.”

Also, I wasn’t sure what is the behaviour of File methods in Ruby 1.9.  
My experimentation shows that the bytes from the encoding of the  
String passed are used, e.g. without converting. For example in a  
UTF-8 system, File.new(“mÃ©mÃ©”) isn’t the same as  
File.new(“mÃ©mÃ©”.encode(“ISO-8859-1”)) (the last one gives ENOENT).

So, I think Strings representing files got from GTK+ in rg2 should:

1- have an associated encoding, so that they can be displayed in GTK+  
without the programmer needing to do conversions (because my new  
RVAL2CSTR will send UTF-8 char\* to GtkLabel etc)  
2- use the encoding expected by the operating system so that File.new  
will work

First step is to use g\_filename\_to\_utf8(), so that we are 100%  
confident we can create a correct String (by using these bytes in Ruby  
UTF-8 encoding).

Second step is to change the encoding of the String for realizing the  
(2) item above. After looking at Glib documentation, it seems that  
using the first item from g\_get\_filename\_charsets() should be fine.

I have made a patch for doing this. I have tested in a UTF-8 system  
with a file with an accent, with default settings and also by forcing  
filename encoding to ISO-8859-1 (by launching the program with  
LC\_ALL=en\_US in parameter), all works great! The patch is  
“string-encoding-for-files.diff” and my little test program was:

-=-=—=-=—=-=—=-=–  
#! /usr/bin/ruby

require ‘gtk2’

Gtk.init

fc = Gtk::FileChooserDialog.new(“Foo”,  
nil,  
Gtk::FileChooser::ACTION\_OPEN,  
nil,  
[Gtk::Stock::OK,  
Gtk::Dialog::RESPONSE\_ACCEPT], [Gtk::Stock::CANCEL,  
Gtk::Dialog::RESPONSE\_CANCEL])  
if fc.run == Gtk::Dialog::RESPONSE\_ACCEPT  
puts fc.filename  
puts fc.filename.encoding  
puts File.exists?(fc.filename)  
end  
-=-=—=-=—=-=—=-=–

Of course, the patch is not commitable right now: if the principle of  
the patch is fine, it should probably be placed into glib2 and shared  
accross other APIs of RG2 returning filenames, and an opposite process  
be used for APIs receiving filenames.

What do you think of that analysis and patch?

> > Â \* But there are some cases that GTK+ doesn’t return  
> > Â Â UTF-8. e.g. g\_convert()(\*1), g\_utf8\_to\_utf16()(\*2) and  
> > Â Â so on. So, we need to provide an API for specify  
> > Â Â encoding like CSTR2RVAL\_ENCODING().
> > 
> > Â Â (\*1) [Learn GNOME](http://library.gnome.org/devel/glib/stable/glib-Character-Set-Conversion.html#g-convert)  
> > Â Â (\*2) [Learn GNOME](http://library.gnome.org/devel/glib/stable/glib-Unicode-Manipulation.html#g-utf8-to-utf16)

I am not sure.. Normally, in Ruby I would say the charset/encoding  
conversions should be realized in Ruby. Also, specifying a charset to  
convert “from” doesn’t make sense in Ruby 1.9 because Ruby Strings  
have an encoding already. What is the use of these functions in Ruby  
1.9 where Strings already carry an encoding and allow encoding  
conversions? In my opinion, there should be no binding for these  
functions anymore. Or maybe I missing something, what do you think we  
should do for these?

> > > Notice: RVAL2CSTR would be now an alias to RVAL2CSTR\_ACCEPT\_NIL, and I  
> > > think it is more safe because it’s always better to properly handle  
> > > NIL case.
> > 
> > We need to RVAL2CSTR with nil check because there are many  
> > API that doesn’t accept NULL as gchar \* argument. We will  
> > need a function that do StringValuePtr() with encoding handling.
> 
> Sorry, I didn’t properly understand that. I see now.

Please see the patch “string-encoding-glib2.diff”, RVAL2CSTR has same  
behaviour with nil passed as before.

Thanks,

---

<div class="post-metadata">

**Author:** ![blueprint](https://www.ruby-forum.com/letter_avatar_proxy/v4/letter/b/cc9497/32.png) [@blueprint](https://www.ruby-forum.com/u/blueprint)\
**Post date:** [September 27, 2010, 2:53pm UTC](https://www.ruby-forum.com/t/string-encoding-problem-with-ruby-1-9-2p0/195470/11 "2010-09-27T14:53:20Z")

</div>

Hi,

Thanks for your proposals!  
But can we separate those proposals with each thread?

1. FileChooser encoding
2. g\_covert() related things
3. RVAL2CSTR encoding support

I want to resolve those proposals with 3., 1., 2. order.

First, could you please send a mail to this ML about only  
3.? I’ll reply the mail with patch review result.

I’m sorry for your inconvenient but please help me. I can’t  
process a long English mail at a time with my English skill  
and hobby time. :-\<

## Thanks,

kou

In [AANLkTim=removed\_email\_address@domain.invalid](mailto:AANLkTim=removed_email_address@domain.invalid)  
“Re: [ruby-gnome2-devel-en] String encoding problem with ruby 1.9.2p0”  
on Sun, 26 Sep 2010 22:58:43 +0200,
